google-cloud-rust

repository·master·Indexed 41 days ago

https://github.com/mozilla-services/google-cloud-rust

A collection of Rust client libraries for interacting with Google Cloud Platform services. It provides production-ready support for Cloud Spanner and experimental support for BigTable, Pub/Sub, and Cloud Storage. The project includes the `googleapis-raw` package for raw gRPC bindings based on `grpcio` version 0.13.0 and `protobuf` version 2.28.0.

Tokens
17.5K
Snippets
76
Records
91
Agent score
51%

What's inside google-cloud-rust

  1. Choose between raw and high-level Google API bindings

    master

    The repository provides two tiers of bindings for interacting with Google APIs:

    1. Raw Bindings (googleapis-raw): These are based on grpcio and provide direct access to the underlying service definitions. You can find usage examples in the googleapis-raw/examples directory.
    2. High-level Bindings (googleapis): These are intended to provide a more ergonomic interface, but they are not ready for use yet.

    Currently, only the Cloud Spanner client is considered in production. Clients for Cloud BigTable, Cloud Pub/Sub, and Cloud Storage are auto-generated and should be considered experimental.

  2. Match dependency versions for Google Cloud Rust Client

    master

    To avoid compatibility issues, your calling code must use the exact same versions of protobuf and grpcio as this library. This requirement is due to an internal dependency constraint within grpcio.

    Required versions:

    • protobuf = 2.28.0
    • grpcio = 0.13.0 (Note: The README mentions a lock to 0.12.0 in the disclaimer section, but specifies 0.13.0 in the dependency list. Always verify against the Cargo.toml of the specific version you are using.)
    # Ensure your Cargo.toml matches these versions
    protobuf = "=2.28.0"
    grpcio = "0.13.0"
  3. Generate Rust bindings from .proto files

    master

    If you need to regenerate the bindings from .proto files, follow these steps. This process requires the protobuf library and the protoc-gen-rust plugin.

    Prerequisites:

    1. Install the protobuf library.
    2. Install the protobuf-codegen plugin via cargo:
      cargo install protobuf-codegen
    3. Ensure protoc-gen-rust is in your $PATH.
    4. Ensure the installed protobuf version matches the protobuf crate version (e.g., 2.7.0).

    Generation Process:

    1. Navigate to the ./src directory and remove all *.rs files except for mod.rs files. You can use the prepare.sh script to assist with this.
    2. Run the generation script:
      ./generate.sh
    3. Verify the build by running cargo build.

    Note on mod.rs files: The generate.sh script does not generate mod.rs files. You may need to manually add module declarations to mod.rs files to satisfy the generated code's requirements for super modules. For example:

    pub(crate) use crate::{
        empty,
        iam::v1::{iam_policy, policy},
        longrunning::operations,
    };
    cargo install protobuf-codegen
    ./generate.sh
  4. Update mod.rs files using mod_updater.py

    master

    The mod_updater.py script is a utility used to synchronize local generated files with existing mod.rs files after a grpcio submodule update. This is useful when third-party APIs change radically, causing mismatches between generated code and the module declarations.

    By default, the script presumes it is being run from within its own directory. To target a specific directory (such as googleapis-raw), use the MOD_UP_PATH environment variable.

    MOD_UP_PATH=path/to/googleapis-raw python3 mod_updater.py
  5. Configure Google Authentication in Docker

    master

    To run examples inside the Docker container, the environment variable GOOGLE_APPLICATION_CREDENTIALS must point to your Google credentials file.

    The Docker image is configured to look for credentials at files/credentials.json. To make your credentials available, copy your JSON credentials file to that specific location within the project directory.

    Warning: The files/credentials.json file is not under version control; ensure you do not commit it to your repository.

    # Example: Copying your credentials to the expected location
    cp /path/to/your/actual-credentials.json files/credentials.json
  6. Update submodule dependencies

    master

    To pull the latest versions of the required libraries (grpc, protobuf, and googleapis), use the following commands to perform a targeted update of the submodules:

    git submodule update --remote grpc
    pushd grpc/third_party
    git submodule update --remote protobuf
    git submodule update --remote googleapis
    popd

    Important: It is recommended to update the grpc/third_party/googleapis submodule at least once per quarter, as it contains Protobuf declarations that may change without notice. Updating these may require manual corrections to mod.rs files to account for new or removed modules.

  7. Set up Google Cloud SDK for development

    master

    Before running examples or using the bindings, you must have the Google Cloud SDK configured in your environment.

    Verification Steps:

    1. Check SDK configuration: gcloud info
    2. Verify credentials: echo $GOOGLE_APPLICATION_CREDENTIALS
    3. Ensure you are logged in: gcloud auth login

    For a pre-configured environment, refer to the Docker setup documentation in the ./docker folder of the repository.

  8. Install and run the project using Docker

    master

    If you are on an operating system like macOS where local setup is difficult, or if you want to avoid installing various programming languages and libraries on your host system, you can use the provided Docker setup. The Docker image includes a recent Rust toolchain and mounts the project repository into the container.

    Prerequisites

    • Install Docker (e.g., Docker Desktop on macOS).
    • Install docker-compose.
    # Build the Docker image
    docker-compose build rust
    
    # Start the container and open a bash shell
    docker-compose run rust /bin/bash
  9. Understand FieldBehavior annotations

    master

    The FieldBehavior enum is used to denote the behavior of specific fields within Google API protobuf messages. These annotations are used by API tooling to understand how to handle fields (e.g., whether a field is required in a request or only provided in a response).

    Note: These annotations do not change the underlying protocol buffer behavior itself; they serve as metadata for API tooling and documentation.

  10. Configure bucket options for a Distribution

    master

    The Distribution type uses a oneof field called bucket_option to allow different ways of defining buckets. You can choose one of the following three types:

    1. linear_buckets: Uses Distribution_LinearBuckets for uniform bucket widths.
    2. exponential_buckets: Uses Distribution_ExponentialBuckets for buckets that grow exponentially.
    3. explicit_buckets: Uses Distribution_ExplicitBuckets for manually defined bucket boundaries.

    Use the set_*_buckets methods to assign a specific bucket configuration to the Distribution instance.

    // Example: Setting linear buckets
    let mut dist = Distribution::new();
    let mut linear = Distribution_LinearBuckets::new();
    linear.set_num_finite_buckets(10);
    linear.set_width(1.0);
    linear.set_offset(0.0);
    
    dist.set_linear_buckets(linear);
  11. Understand Spanner KeyRange and KeySet

    master

    In Google Cloud Spanner, KeyRange and KeySet are used to define sets of rows to be operated on (e.g., for reads or mutations).

    KeyRange

    A KeyRange represents a range of rows in a table or index. It is defined by a start_key and an end_key.

    • Open vs. Closed Keys: Keys can be 'open' or 'closed'.
      • A closed key includes rows that exactly match the provided key components.
      • An open key excludes rows that exactly match the provided key components.
    • Lexicographical Order: For multi-column primary keys, ranges are interpreted lexicographically based on the table's declared sort order.
    • Trailing Components: Start and end keys can omit trailing components of a primary key, which affects how the range boundaries are interpreted.

    KeySet

    A KeySet is a collection of Cloud Spanner keys and/or key ranges.

    • Composition: A KeySet can contain a list of specific keys, a list of ranges, or both.
    • Uniqueness: If the same key is specified multiple times (e.g., via overlapping ranges or a key appearing in both keys and ranges), Spanner treats it as if it were only specified once.
    • All Match: Setting the all field to true indicates that the KeySet matches all keys in the table or index.