go-ipam IP Address Management

repository·master·Indexed 41 days ago

https://github.com/metal-stack/go-ipam

An IP address management module and microservice for handling networks, prefixes, and IP addresses. It can be used as a Go library or a standalone gRPC service supporting multiple database backends including Postgres, Redis, MongoDB, Etcd, CockroachDB, and in-memory storage. Features include namespace isolation, a CLI tool for manual management, Prometheus metrics, and pprof profiling.

Tokens
14.6K
Snippets
40
Records
58
Agent score
99%

What's inside go-ipam

  1. Use go-ipam as a gRPC microservice

    master

    The project can be run as a standalone microservice providing a gRPC API.

    Running the service

    Start the container with your chosen database backend. For example, using Postgres:

    docker run -it --rm ghcr.io/metal-stack/go-ipam postgres

    Client implementation

    Use the generated gRPC client to interact with the service. You can specify a namespace using goipam.NewContextWithNamespace to ensure requests are scoped correctly.

    package main
    
    import (
        "context"
        "http"
    
        v1 "github.com/metal-stack/go-ipam/api/v1"
        "github.com/metal-stack/go-ipam/api/v1/apiv1connect"
    )
    
    func main() {
        c := apiv1connect.NewIpamServiceClient(
                http.DefaultClient,
                "http://localhost:9090",
                connect.WithGRPC(),
        )
    
        bgCtx := context.Background()
        // Optional with Namespace
        ctx := goipam.NewContextWithNamespace(bgCtx, "tenant-a")
    
        result, err := c.CreatePrefix(ctx, &v1.CreatePrefixRequest{Cidr: "192.168.0.0/16"})
        if err != nil {
            panic(err)
        }
        fmt.Println("Prefix:%q created", result.GetPrefix().GetCidr())
    }
  2. Use go-ipam as a Go library

    master

    You can use go-ipam as a Go module to manage networks, prefixes, and IP addresses. The library supports namespaces to isolate IP management for different tenants.

    Key operations include:

    • New(ctx): Initializes an ipamer (e.g., with in-memory storage).
    • CreateNamespace(ctx, name): Creates a new namespace.
    • NewContextWithNamespace(ctx, name): Returns a new context associated with a specific namespace.
    • NewPrefix(ctx, cidr): Creates a new prefix (e.g., 192.168.0.0/24).
    • AcquireIP(ctx, cidr): Acquires an IP from the specified prefix.
    • ReleaseIP(ctx, ip): Releases an acquired IP back to its prefix.
    • AcquireChildPrefix(ctx, cidr, mask): Acquires a sub-prefix from an existing prefix (useful for IPv6 management).
    package main
    
    import (
        "context"
        "fmt"
        "time"
    
        goipam "github.com/metal-stack/go-ipam"
    )
    
    func main() {
        bgCtx := context.Background()
        ipam := goipam.New(bgCtx)
    
        namespace := "tenant-a"
        err := ipam.CreateNamespace(bgCtx, namespace)
        if err != nil {
            panic(err)
        }
        ctx := goipam.NewContextWithNamespace(bgCtx, namespace)
        ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
        defer cancel()
    
        // Create a prefix
        prefix, err := ipam.NewPrefix(ctx, "192.168.0.0/24")
        if err != nil {
            panic(err)
        }
    
        // Acquire and release an IP
        ip, err := ipam.AcquireIP(ctx, prefix.Cidr)
        if err != nil {
            panic(err)
        }
        fmt.Printf("got IP: %s\n", ip.IP)
    
        prefix, err = ipam.ReleaseIP(ctx, ip)
        if err != nil {
            panic(err)
        }
        fmt.Printf("IP: %s released.\n", ip.IP)
    }
  3. Handle optimistic locking errors during prefix updates

    master

    The UpdatePrefix method implements optimistic concurrency control using a Version field stored within the prefix's JSONB data.

    When an update is attempted, the provider executes a SELECT ... FOR UPDATE and an UPDATE statement that includes a check: prefix->>'Version'=$3. If the version in the database does not match the version of the Prefix object being passed in, the update will affect zero rows, and the provider will return ErrOptimisticLockError.

    When to handle this: You should catch ErrOptimisticLockError when performing write operations on prefixes that might be modified by other clients or processes simultaneously. The standard pattern is to re-read the latest version of the prefix, apply changes, and retry the update.

  4. Optimistic Locking with UpdatePrefix

    master

    The Etcd storage implementation uses optimistic locking to prevent concurrent update conflicts. When calling UpdatePrefix, the Prefix object must have a version that matches the version currently stored in etcd.

    1. The UpdatePrefix method increments the version of the provided object.
    2. It fetches the existing record from etcd.
    3. It compares the version of the existing record with the version of the object you provided.
    4. If they do not match, it returns ErrOptimisticLockError.

    To avoid this error, always use the Prefix object returned by a previous ReadPrefix or CreatePrefix call, as it contains the correct current version.

  5. Handle Optimistic Locking Errors

    master

    The ipamer uses optimistic locking to ensure data consistency during concurrent updates. If a conflict occurs, the system returns an ErrOptimisticLockError.

    Internal methods like AcquireChildPrefix and ReleaseIP automatically use retryOnOptimisticLock, which retries the operation up to 10 times with exponential backoff and jitter. When calling lower-level storage or manual update methods, you should implement your own retry logic if you encounter ErrOptimisticLockError.

  6. Use the file-based Storage interface

    master

    The file implementation satisfies the Storage interface. When using NewLocalFile, you can perform standard IPAM operations which are automatically persisted to the JSON file.

    Note on Synchronization: The provider uses a reload() mechanism. Before any read or write operation, it checks if the file's modification time has changed. If it has, it reloads the file content into memory to ensure consistency. Write operations (Create, Update, Delete) are followed by a persist() call to save the updated state back to disk.

    // Example operations on the Storage interface
    // Create a new prefix in a namespace
    newPrefix, err := storage.CreatePrefix(ctx, prefix, "my-namespace")
    
    // Read a prefix
    prefix, err := storage.ReadPrefix(ctx, prefix, "my-namespace")
    
    // List all namespaces
    namespaces, err := storage.ListNamespaces(ctx)
    
    // Delete a namespace
    err := storage.DeleteNamespace(ctx, "my-namespace")
  7. Debug the server using pprof

    master

    The go-ipam server exposes a pprof endpoint on port :2113. You can inspect the server's runtime state (such as heap usage or goroutines) using the go tool pprof command.

    To inspect the heap via a web interface:

    go tool pprof -http :8080 localhost:2113/debug/pprof/heap

    To inspect goroutines via a web interface:

    go tool pprof -http :8080 localhost:2113/debug/pprof/goroutine
    # Example command to inspect heap
    go tool pprof -http :8080 localhost:2113/debug/pprof/heap
  8. Configure the go-ipam CLI server endpoint

    master

    The cli tool uses a gRPC server endpoint to communicate with the IPAM service. By default, it connects to http://localhost:9090. You can override this using the --grpc-server-endpoint flag or by setting the GOIPAM_CLI_GRPC_SERVER_ENDPOINT environment variable.

    # Using a flag
    ./cli --grpc-server-endpoint http://remote-ipam:9090 prefix list
    
    # Using an environment variable
    export GOIPAM_CLI_GRPC_SERVER_ENDPOINT=http://remote-ipam:9090
    ./cli prefix list
  9. Manage IPv4 prefixes and IPs

    master

    You can manage IPv4 networks by creating a prefix, acquiring an IP from that prefix, and later releasing it. If no IPs remain in a prefix, the prefix itself can be deleted.

    ipamer := New()
    prefix, err := ipamer.NewPrefix("192.168.0.0/24")
    if err != nil {
        panic(err)
    }
    
    ip1, err := ipamer.AcquireIP(prefix.Cidr)
    if err != nil {
        panic(err)
    }
    fmt.Printf("Super Prefix  : %s\n", prefix)
    fmt.Printf("Super Prefix IP1 : %s\n", ip1.IP.String())
    fmt.Printf("Super Prefix IP1 Parent : %s\n", ip1.ParentPrefix)
    // Output:
    // Super Prefix  : 192.168.0.0/24
    // Super Prefix IP1 : 192.168.0.1
    // Super Prefix IP1 Parent : 192.168.0.0/24
    
    _, err = ipamer.ReleaseIP(ip1)
    if err != nil {
        panic(err)
    }
    _, err = ipamer.DeletePrefix(prefix.Cidr)
    if err != nil {
        panic(err)
    }
  10. Manage IPv6 prefixes and child subnets

    master

    The library supports IPv6 and allows creating child prefixes (subnets) from a parent prefix with a specific bitlength (mask).

    ipamer := New()
    prefix, err := ipamer.NewPrefix("2001:aabb::/48")
    if err != nil {
        panic(err)
    }
    
    cp1, err := ipamer.AcquireChildPrefix(prefix.Cidr, 64)
    if err != nil {
        panic(err)
    }
    cp2, err := ipamer.AcquireChildPrefix(prefix.Cidr, 72)
    if err != nil {
        panic(err)
    }
    
    ip21, err := ipamer.AcquireIP(cp2.Cidr)
    if err != nil {
        panic(err)
    }
    
    prefix = ipamer.PrefixFrom(prefix.Cidr)
    fmt.Printf("Super Prefix  : %s\n", prefix)
    fmt.Printf("Child Prefix 1: %s\n", cp1)
    fmt.Printf("Child Prefix 2: %s\n", cp2)
    fmt.Printf("Child Prefix 2 IP1: %s\n", ip21.IP)
    // Output:
    // Super Prefix  : 2001:aabb::/48
    // Child Prefix 1: 2001:aabb::/64
    // Child Prefix 2: 2001:aabb:0:1::/72
    // Child Prefix 2 IP1: 2001:aabb:0:1::1
  11. Use the Ipamer interface for IP management

    master

    The Ipamer interface provides the core functionality for managing IP addresses and prefixes. It supports creating new prefixes, acquiring/releasing IPs, and managing child prefixes (subnets) with specific bitlengths. The library is IPv6 compatible and uses inet.af/netaddr for high-performance network manipulation.

    type Ipamer interface {
        // NewPrefix create a new Prefix from a string notation.
        NewPrefix(cidr string) (*Prefix, error)
        // DeletePrefix delete a Prefix from a string notation.
        // If the Prefix is not found an NotFoundError is returned.
        DeletePrefix(cidr string) (*Prefix, error)
        // AcquireChildPrefix will return a Prefix with a smaller length from the given Prefix.
        AcquireChildPrefix(parentCidr string, length uint8) (*Prefix, error)
        // ReleaseChildPrefix will mark this child Prefix as available again.
        ReleaseChildPrefix(child *Prefix) error
        // PrefixFrom will return a known Prefix.
        PrefixFrom(cidr string) *Prefix
        // AcquireSpecificIP will acquire given IP and mark this IP as used, if already in use, return nil.
        // If specificIP is empty, the next free IP is returned.
        // If there is no free IP an NoIPAvailableError is returned.
        AcquireSpecificIP(prefixCidr, specificIP string) (*IP, error)
        // AcquireIP will return the next unused IP from this Prefix.
        AcquireIP(prefixCidr string) (*IP, error)
        // ReleaseIP will release the given IP for later usage and returns the updated Prefix.
        // If the IP is not found an NotFoundError is returned.
        ReleaseIP(ip *IP) (*Prefix, error)
        // ReleaseIPFromPrefix will release the given IP for later usage.
        ReleaseIPFromPrefix(prefixCidr, ip string) error
    }
  12. Configure the go-ipam server

    master

    To instantiate a server, you must provide a config struct. The server requires a Storage implementation (from the goipam package) to persist IPAM data.

    Key configuration fields:

    • GrpcServerEndpoint: The address where the gRPC/Connect service will listen.
    • MetricsEndpoint: The address where the Prometheus /metrics endpoint will be served.
    • Log: An *slog.Logger for structured logging.
    • Storage: An implementation of goipam.Storage.
    type config struct {
    	GrpcServerEndpoint string
    	MetricsEndpoint    string
    	Log                *slog.Logger
    	Storage            goipam.Storage
    }