maxminddb Rust Library

repository·main·Indexed 42 days ago

https://github.com/oschwald/maxminddb-rust

A high-performance Rust implementation of a MaxMind DB reader for reading MaxMind DB format files used by GeoIP2 and GeoLite2. Version 0.32.0 provides features for IP lookups, CIDR range iteration via within(), and flexible data extraction using Serde decoding or path-based decoding. It supports memory-mapped file access via the mmap feature and offers optimized string decoding through the unsafe-str-decode feature.

Tokens
8.6K
Snippets
29
Records
35
Agent score
50%

What's inside maxminddb-rust

  1. Migrate GeoIP2 Name Fields (v0.26 to v0.27)

    main

    The names fields in GeoIP2 structures have transitioned from a BTreeMap to a dedicated Names struct. This allows for direct access to common languages.

    Before (0.26):

    let name = city.city
        .as_ref()
        .and_then(|c| c.names.as_ref())
        .and_then(|n| n.get("en"));

    After (0.27):

    let name = city.city.names.english;

    Supported language fields:

    • german
    • english
    • spanish
    • french
    • japanese
    • brazilian_portuguese
    • russian
    • simplified_chinese
  2. Migrate Lookup API to use lazy decoding (v0.26 to v0.27)

    main

    Starting in version 0.27, the lookup() method returns a LookupResult instead of an Option<T>. This enables lazy decoding, allowing you to inspect the result without fully decoding the data.

    Key features of LookupResult:

    • result.has_data(): Check if data exists without decoding.
    • result.network()?: Get the network for the IP (returns IpNetwork).
    • result.decode::<T>()?: Decode the result into a specific type.
    • result.decode_path(&[...])?: Decode only specific fields.

    Before (0.26):

    let city: Option<geoip2::City> = reader.lookup(ip)?;
    if let Some(city) = city {
        println!("{:?}", city.city);
    }

    After (0.27):

    let result = reader.lookup(ip)?;
    if let Some(city) = result.decode::<geoip2::City>()? {
        println!("{:?}", city.city);
    }
  3. Migrate Reader metadata access (v0.28 to v0.29)

    main

    In version 0.29, the metadata field on the Reader struct was made private. You must now use the metadata() method to access validated metadata via a shared reference.

    Before (0.28):

    let database_type = &reader.metadata.database_type;

    After (0.29):

    let database_type = &reader.metadata().database_type;
  4. Migrate Metadata build_time access (v0.28 to v0.29)

    main

    In version 0.29, Metadata::build_time() now returns a Result<SystemTime, MaxMindDbError>. You must handle the potential error using the ? operator or similar error handling.

    Before (0.28):

    let build_time = reader.metadata.build_time();

    After (0.29):

    let build_time = reader.metadata().build_time()?;
  5. Migrate MaxMindDbError matching (v0.26 to v0.27)

    main

    Error variants in MaxMindDbError now use structured fields instead of simple strings. When matching errors, you must use struct patterns.

    Before (0.26):

    match error {
        MaxMindDbError::InvalidDatabase(msg) => {
            println!("Invalid database: {}", msg);
        }
        // ...
    }

    After (0.27):

    match error {
        MaxMindDbError::InvalidDatabase { message, offset } => {
            println!("Invalid database: {} at {:?}", message, offset);
        }
        MaxMindDbError::InvalidInput { message } => {
            println!("Invalid input: {}", message);
        }
        // ...
    }

    Note: The InvalidInput variant is used for user errors, such as attempting to look up an IPv6 address in an IPv4-only database.

  6. Migrate within iterator and use WithinOptions (v0.26 to v0.27)

    main

    The within() method now requires a WithinOptions parameter. Additionally, the iterator now yields Result items that should be handled.

    Before (0.26):

    for item in reader.within::<geoip2::City>(cidr)? {
        let item = item?;
        println!("{}: {:?}", item.ip_net, item.info);
    }

    After (0.27):

    use maxminddb::WithinOptions;
    
    for result in reader.within(cidr, Default::default())? {
        let result = result?;
        let network = result.network()?;
        if let Some(city) = result.decode::<geoip2::City>()? {
            println!("{}: {:?}", network, city);
        }
    }

    Customizing iteration with WithinOptions:

    let options = WithinOptions::default()
        .include_aliased_networks()      // Include IPv4 via IPv6 aliases
        .include_networks_without_data() // Include networks without data
        .skip_empty_values();            // Skip empty maps/arrays
    
    for result in reader.within(cidr, options)? {
        // ...
    }
  7. Replace lookup_prefix with lookup and network (v0.26 to v0.27)

    main

    The lookup_prefix() method has been removed. To achieve the same result, use lookup() followed by network() on the returned LookupResult.

    Before (0.26):

    let (city, prefix_len) = reader.lookup_prefix(ip)?;

    After (0.27):

    let result = reader.lookup(ip)?;
    let city = result.decode::<geoip2::City>()?;
    let network = result.network()?;  // Returns IpNetwork with prefix
    let result = reader.lookup(ip)?;
    let city = result.decode::<geoip2::City>()?;
    let network = result.network()?;
  8. Migrate GeoIP2 Nested Structs (v0.26 to v0.27)

    main

    In version 0.27, nested struct fields are now non-optional and use Default instead of requiring .as_ref() chains. Note that leaf values (strings, numbers, bools) remain Option<T>.

    Before (0.26):

    let iso_code = city.country
        .as_ref()
        .and_then(|c| c.iso_code.as_ref());
    
    let subdivisions = city.subdivisions
        .as_ref()
        .map(|v| v.iter())
        .into_iter()
        .flatten();

    After (0.27):

    let iso_code = city.country.iso_code;
    
    for subdivision in &city.subdivisions {
        // ...
    }
  9. Set up and run fuzzing for maxminddb-rust

    main

    The fuzzing suite uses cargo-fuzz and requires a nightly Rust toolchain.

    To run the fuzzers, install the runner and execute the desired target from the repository root.

    Fuzzing Targets

    • decode: Sends arbitrary data-section values through Serde decoding and the data-value verification decoder.
    • reader: Exercises database opening, verification, lookups, full decoding, path decoding, and network iteration. It tests inputs both as arbitrary databases and as mutations to a small valid database.

    Running a Bounded Smoke Test

    To run a quick local test that terminates after a set duration, pass libFuzzer options after a -- separator.

    # Install the runner
    cargo install cargo-fuzz --locked
    
    # Run the decode target
    cargo +nightly fuzz run decode
    
    # Run the reader target
    cargo +nightly fuzz run reader
    
    # Run a 30-second smoke test for decode
    cargo +nightly fuzz run decode -- -max_total_time=30
    
    # Run a 30-second smoke test for reader
    cargo +nightly fuzz run reader -- -max_total_time=30
  10. Use LookupResult for deferred decoding

    main

    When you perform a lookup using Reader::lookup, you receive a LookupResult. This is a lightweight handle (~40 bytes) that does not immediately deserialize the data. This allows you to efficiently check for data existence, retrieve the network information, or selectively decode only the specific fields you need, rather than parsing the entire record.

    use maxminddb::{geoip2, path, Reader};
    use std::net::IpAddr;
    
    let reader = Reader::open_readfile("test-data/test-data/GeoIP2-City-Test.mmdb").unwrap();
    let ip: IpAddr = "89.160.20.128".parse().unwrap();
    
    let result = reader.lookup(ip).unwrap();
    
    if result.has_data() {
        // Full decode
        let city: geoip2::City = result.decode().unwrap().unwrap();
    
        // Or selective decode via path
        let country_code: Option<String> = result
            .decode_path(&path!["country", "iso_code"])
            .unwrap();
        println!("Country: {:?}", country_code);
    }
  11. Enable optional features: mmap and unsafe-str-decode

    main

    You can enable specific features in your Cargo.toml to optimize performance:

    • mmap: Enables memory-mapped file access, which is recommended for long-running applications.
    • unsafe-str-decode: Skips UTF-8 validation when deserializing trusted database strings into Rust str or String values. Use this with caution; cross-runtime format adapters should prefer deserialize_any_with_raw_strings() and validate strings during construction instead.
    [dependencies]
    maxminddb = { version = "0.32", features = ["mmap"] }