progenitor

repository·main·Indexed 21 days ago

https://github.com/oxidecomputer/progenitor

A Rust tool and cargo command (cargo-progenitor) for generating strongly-typed, async API clients from OpenAPI 3.0.x specifications. It supports multiple integration patterns, including the `generate_api!` macro, build scripts via `progenitor::Generator`, and the generation of standalone Rust crates. It offers two interface styles: Positional (default) and Builder, with the latter supporting method chaining and automatic pagination streaming via `.stream()` for Dropshot mechanisms.

Tokens
14.7K
Snippets
40
Records
62
Agent score
75%

What's inside progenitor

  1. Understand the structure of generated client methods

    main

    Generated client methods in Progenitor wrap their return values and errors in specific wrapper types: ResponseValue<T> for successful responses and Error<E> for failures. This allows the client to provide metadata like HTTP status codes and headers alongside the actual data.

    A typical method signature looks like this:

    impl Client {
        pub async fn operation_name<'a>(
            &'a self,
            // parameters ...
        ) -> Result<
            ResponseValue<types::SuccessResponseType>,
            Error<types::ErrorResponseType>
        > {
            // ...
        }
    }
    impl Client {
        pub async fn operation_name<'a>(
            &'a self,
            // parameters ...
        ) -> Result<
            ResponseValue<types::SuccessResponseType>,
            Error<types::ErrorResponseType>>
        {
            // ...
        }
    }
  2. Use positional generation style for API clients

    main

    When using the 'positional' generation style, Progenitor generates async methods for each operation defined in the OpenAPI document. The arguments for these methods follow a strict order based on the operation's requirements:

    1. Path parameters: Always come first and are mandatory.
    2. Query parameters: Follow path parameters and may be optional (wrapped in Option<T>).
    3. Body parameters: If the operation specifies a request body, it is passed as the final argument.

    Because these methods are async, you must .await them to receive the Result containing the response.

    impl Client {
        pub async fn operation_name<'a>(
            &'a self,
            // Path parameters (if any) come first and are always mandatory
            path_parameter_1: String,
            path_parameter_2: u32,
            // Query parameters (if any) come next and may be optional
            query_parameter_1: String,
            query_parameter_2: Option<u32>,
            // A body parameter (if specified) comes last
            body: &types::ThisOperationBody,
        ) -> Result<
            ResponseValue<types::SuccessResponseType>,
            Error<types::ErrorResponseType>,
        > {
            // ...
        }
    }
  3. Compare Positional vs Builder generation styles

    main

    Progenitor supports two interface styles for the generated Client methods:

    1. Positional Style (Default)

    Methods accept parameters in a specific order. This is concise but requires exact type matching and strict parameter ordering.

    Example usage:

    let result = client.instance_create(org, proj, body).await?;

    2. Builder Style

    Methods return a builder struct. You apply parameters via method calls and then call .send().await. This style is more legible for complex APIs, allows for implicit type conversions (e.g., String to a custom type via TryInto), and allows omitting optional parameters.

    Example usage:

    let result = client
        .instance_create()
        .organization_name("org")
        .project_name("proj")
        .body(body)
        .send()
        .await?;

    To enable the builder style in build.rs, use GenerationSettings with InterfaceStyle::Builder.

    // Enabling Builder style in build.rs
    let mut binding = GenerationSettings::default();
    let settings = binding.with_interface(InterfaceStyle::Builder);
    let mut generator = progenitor::Generator::new(&settings);
  4. Use the Builder generation style

    main

    When the builder generation style is specified, Progenitor generates a builder struct for each operation in the OpenAPI document. Instead of calling an operation directly with arguments, you call a method on the Client that returns a builder instance. You then chain methods to set path parameters, query parameters, and the request body before calling .send() to execute the request.

    Key characteristics:

    • Method Chaining: Parameters are set via methods named after the parameter (e.g., .path_parameter_1(value)).
    • Type Conversion: Parameter methods use TryInto, allowing you to pass values that can be converted into the required type (e.g., String, u32). Errors in conversion are captured within the builder.
    • Async Execution: The .send() method is async and must be .awaited to retrieve the Result<ResponseValue<T>, Error<E>>.
    // Example of the builder pattern flow
    let response = client
        .operation_name()
        .path_parameter_1("my_value")
        .query_parameter_1("search_term")
        .body(my_body_struct)
        .send()
        .await?;
  5. Consume paginated operations using Streams

    main

    If an OpenAPI operation uses the Dropshot pagination mechanism, Progenitor generates a companion method suffixed with _stream. This method returns a futures::Stream that allows you to iterate over all items in a collection across multiple pages automatically.

    To use the stream, you can use try_next().await within a loop to process items until the stream returns Ok(None), indicating the end of the collection.

        pub fn operation_name_stream<'a>(
            &'a self,
            // Specific parameters...
            limit: Option<std::num::NonZeroU32>,
        ) -> impl futures::Stream<
    	    Item = Result<types::SuccessResponseType, Error<types::ErrorResponseType>>
        > + Unpin + '_ {
            // ...
        }
    
    // Example usage:
    let mut stream = client.operation_name_stream(None);
    loop {
        match stream.try_next().await {
            Ok(Some(item)) => println!("item {:?}", item),
            Ok(None) => {
                println!("done.");
                break;
            }
            Err(_) => {
                println!("error!");
                break;
            }
        }
    }
  6. Use `build.rs` for advanced code generation

    main

    Using a build.rs script allows you to make the generated code visible and enables the generation of a CLI and httpmock helpers. This method uses the progenitor::Generator API.

    Implementation Steps

    1. In build.rs, use progenitor::Generator to generate tokens from your OpenAPI spec.
    2. Parse the tokens into an AST using syn and unparse them using prettyplease to create a readable .rs file in OUT_DIR.
    3. In your main source code, include the generated file using include!(concat!(env!("OUT_DIR"), "/codegen.rs"));.

    Required Dependencies

    Cargo.toml [dependencies]:

    • futures = "0.3"
    • progenitor-client (Note: the generated code requires this, while build.rs uses progenitor)
    • reqwest = { version = "0.13", features = ["json", "query", "stream"] }
    • serde = { version = "1.0", features = ["derive"] }
    • serde_json = "1.0"

    Cargo.toml [build-dependencies]:

    • prettyplease = "0.2.22"
    • progenitor = { git = "https://github.com/oxidecomputer/progenitor" }
    • serde_json = "1.0"
    • syn = "2.0"
    // build.rs
    fn main() {
        let src = "../sample_openapi/keeper.json";
        println!("cargo:rerun-if-changed={}", src);
        let file = std::fs::File::open(src).unwrap();
        let spec = serde_json::from_reader(file).unwrap();
        let mut generator = progenitor::Generator::default();
    
        let tokens = generator.generate_tokens(&spec).unwrap();
        let ast = syn::parse2(tokens).unwrap();
        let content = prettyplease::unparse(&ast);
    
        let mut out_file = std::path::Path::new(&std::env::var("OUT_DIR").unwrap()).to_path_buf();
        out_file.push("codegen.rs");
    
        std::fs::write(out_file, content).unwrap();
    }
    
    // main.rs
    include!(concat!(env!("OUT_DIR"), "/codegen.rs"));
  7. Stream paginated operations with `.stream()`

    main

    If an OpenAPI operation uses the Dropshot pagination mechanism, Progenitor generates a .stream() method on the operation's builder struct. This method returns a futures::Stream that allows you to iterate over all items in a paginated collection automatically, handling page transitions internally.

    To use it, call .stream() instead of .send() on the builder. The stream yields Result<SuccessResponseType, Error<ErrorResponseType>> items. You can iterate using try_next().await from the futures crate.

    let mut stream = client.operation_name().stream();
    loop {
        match stream.try_next().await {
            Ok(Some(item)) => println!("item {:?}", item),
            Ok(None) => {
                println!("done.");
                break;
            }
            Err(_) => {
                println!("error!");
                break;
            }
        }
    }
  8. Add default headers to the generated Client

    main

    The generated Progenitor code does not automatically handle request headers. To include default headers (such as Authorization) in every request made by the client, you must construct a reqwest::Client using reqwest::ClientBuilder with the default_headers method, and then initialize your Progenitor Client using Client::new_with_client.

        let baseurl = std::env::var("API_URL").expect("$API_URL not set");
        
        let access_token = std::env::var("API_ACCESS_TOKEN").expect("$API_ACCESS_TOKEN not set");
        let authorization_header = format!("Bearer {}", access_token);
    
        let mut headers = reqwest::header::HeaderMap::new();
        headers.insert(
            reqwest::header::AUTHORIZATION,
            authorization_header.parse().unwrap(),
        );
    
        let client_with_custom_defaults = reqwest::ClientBuilder::new()
            .connect_timeout(Duration::from_secs(15))
            .timeout(Duration::from_secs(15))
            .default_headers(headers)
            .build()
            .unwrap();
    
        let client = Client::new_with_client(baseurl, client_with_custom_defaults);
  9. Initialize a customized Client with Client::new_with_client()

    main

    If you need to configure custom headers, timeouts, or other settings via reqwest::ClientBuilder, use Client::new_with_client(). This method accepts the base URL and a pre-configured reqwest::Client instance.

    let mut val = reqwest::header::HeaderValue::from_static("super-secret");
    val.set_sensitive(true);
    let mut headers = reqwest::header::HeaderMap::new();
    headers.insert(reqwest::header::AUTHORIZATION, val);
    
    let client_builder = reqwest::ClientBuilder::new()
        .connect_timeout(Duration::new(60, 0))
        .default_headers(headers)
        .build()
        .unwrap();
    
    let client = Client::new_with_client("https://foo/bar", client_builder);
  10. Use the `generate_api!` macro for quick setup

    main

    The simplest way to use Progenitor is via the generate_api! macro. This macro automatically generates a Client type with methods corresponding to your OpenAPI operations. The macro is re-evaluated whenever the specified OpenAPI document's modification time changes.

    Dependencies

    You must include futures, progenitor, reqwest (with json, query, and stream features), and serde in your Cargo.toml. Depending on your OpenAPI spec, you may also need:

    • chrono (for date or date-time formats)
    • uuid (for uuid formats)
    • base64 and rand (for websocket endpoints)
    • regress (for regular expression validation)

    If you use derives = [ schemars::JsonSchema ], you must also add schemars with appropriate features (e.g., features = ["chrono", "uuid1"]) to your dependencies.

    // In main.rs, lib.rs, or mod.rs
    generate_api!("path/to/openapi_document.json");