golang/mod

repository·master·Indexed 42 days ago

https://github.com/golang/mod

Low-level packages for direct manipulation of Go module mechanics, including parsing and modifying go.mod files, version selection logic, and module graph structures. Includes the sumdb package for implementing and interacting with Go checksum database servers and clients, as well as the gosumcheck CLI for verifying go.sum files.

Tokens
2.5K
Snippets
11
Records
12
Agent score
52%

What's inside golang/mod

  1. Understand the purpose of the mod package

    master

    The golang.org/x/mod repository provides packages designed for writing tools that perform direct manipulation of Go module mechanics.

    When to use this repository: Use these packages if your tool's primary purpose is to interact with, parse, or modify Go module files and structures (e.g., go.mod files, version selection logic, or module graph structures).

    When NOT to use this repository: Do not use this repository if you are building general development tools that simply need to load packages in module mode. For tools where modules are incidental to the task (like static analysis or refactoring), use golang.org/x/tools/go/packages instead. For the most accurate package loading, you should invoke the go command directly, as it is the single source of truth for loading algorithms.

  2. Implement a Go checksum database server with sumdb.Server

    master

    The sumdb.Server type implements the http.Handler interface and serves as the HTTP entry point for a module checksum database. To use it, you must provide an implementation of the ServerOps interface, which handles the actual database access (e.g., looking up records, reading tiles, and retrieving signed hashes).

    To set up the server, use sumdb.NewServer(ops) and register it to handle the paths defined in sumdb.ServerPaths.

    // Define your implementation of ServerOps
    type myOps struct { /* ... */ }
    // ... implement Signed, ReadRecords, Lookup, and ReadTileData ...
    
    // Initialize the server
    ops := &myOps{}
    srv := sumdb.NewServer(ops)
    
    // Register the server for the required paths
    for _, path := range sumdb.ServerPaths {
    	http.Handle(path, srv)
    }
    
    // Start your HTTP server
    http.ListenAndServe(":8080", nil)
  3. Lookup go.sum lines with Lookup

    master

    The Lookup(path, vers string) method returns the go.sum lines for a specific module path and version.

    • If vers ends in /go.mod, it returns the hashes for the module's go.mod file.
    • If the path matches a pattern set via SetGONOSUMDB, it returns ErrGONOSUMDB.
    • The returned []string contains lines in the format: path version hash.
    lines, err := client.Lookup("example.com/mod", "v1.0.0")
    if err != nil {
    	// handle error
    }
    for _, line := range lines {
    	fmt.Println(line)
    }
  4. Configure tile height with SetTileHeight

    master

    Use SetTileHeight(height int) to define the tile height for the client.

    Constraints:

    • Must be called before the first call to Lookup.
    • Can be called at most once.
    • If not called, the client defaults to a tile height of 8.
    • Passing a non-positive integer will cause a panic.
    client.SetTileHeight(8)
  5. Initialize a new sumdb.Client

    master

    Create a new sumdb.Client by passing an implementation of ClientOps to NewClient. The client is safe for concurrent use by multiple goroutines.

    Note that certain configuration methods like SetTileHeight and SetGONOSUMDB must be called before the first call to Lookup.

    client := sumdb.NewClient(myOpsImplementation)
  6. Implement the ClientOps interface

    master

    To use the sumdb.Client, you must provide an implementation of the ClientOps interface. This interface abstracts the external operations required by the client, such as network communication and local storage.

    Required methods:

    • ReadRemote(path string) ([]byte, error): Fetches content from the remote database server (e.g., paths starting with /lookup or /tile/). Must return an error for non-200 HTTP responses.
    • ReadConfig(file string) ([]byte, error): Reads configuration files. Special keys include "key" for the verifier key and serverName + "/latest" for the latest signed tree.
    • WriteConfig(file string, old, new []byte) error: Atomically updates configuration files. Must return ErrWriteConflict if old does not match the current stored content. The "key" file is read-only.
    • ReadCache(file string) ([]byte, error): Reads from a local cache. Errors are treated as a cache miss.
    • WriteCache(file string, data []byte): Writes data to the local cache.
    • Log(msg string): Standard logging.
    • SecurityError(msg string): Handles security violations. In production, this should typically log and terminate the process.
    type ClientOps interface {
    	ReadRemote(path string) ([]byte, error)
    	ReadConfig(file string) ([]byte, error)
    	WriteConfig(file string, old, new []byte) error
    	ReadCache(file string) ([]byte, error)
    	WriteCache(file string, data []byte)
    	Log(msg string)
    	SecurityError(msg string)
    }
  7. Configure GONOSUMDB patterns with SetGONOSUMDB

    master

    Use SetGONOSUMDB(list string) to provide a comma-separated list of module path patterns. If a module path matches one of these patterns, Lookup will return ErrGONOSUMDB instead of querying the database.

    Constraints:

    • Must be called before the first call to Lookup.
    • Can be called at most once.
    client.SetGONOSUMDB("example.com/bad-module,github.com/private/*")
  8. Implement the ServerOps interface

    master

    To use sumdb.Server, you must implement the ServerOps interface to bridge the HTTP server with your underlying data storage. The interface requires the following methods:

    • Signed(ctx context.Context) ([]byte, error): Returns the signed hash of the latest tree.
    • ReadRecords(ctx context.Context, id, n int64) ([][]byte, error): Returns the content for n records starting from record ID id through id+n-1.
    • Lookup(ctx context.Context, m module.Version) (int64, error): Looks up a record for the given module version and returns its record ID.
    • ReadTileData(ctx context.Context, t tlog.Tile) ([]byte, error): Reads the content of a specific tile t. This is only invoked for hash tiles (where t.L >= 0).
    type ServerOps interface {
    	Signed(ctx context.Context) ([]byte, error)
    	ReadRecords(ctx context.Context, id, n int64) ([][]byte, error)
    	Lookup(ctx context.Context, m module.Version) (int64, error)
    	ReadTileData(ctx context.Context, t tlog.Tile) ([]byte, error)
    }
  9. Handle sumdb errors

    master

    The sumdb package defines several specific error variables:

    • ErrWriteConflict: Returned by WriteConfig if the old data provided does not match the current configuration.
    • ErrSecurity: Returned when the client detects a security violation (e.g., a misbehaving server or inconsistent tree).
    • ErrGONOSUMDB: Returned by Lookup when the requested module path matches a pattern in the GONOSUMDB list.
    var (
    	ErrWriteConflict = errors.New("write conflict")
    	ErrSecurity     = errors.New("security error: misbehaving server")
    	ErrGONOSUMDB    = errors.New("skipped (listed in GONOSUMDB)")
    )
  10. Reference the sumdb.ServerPaths

    master

    The sumdb.Server is designed to serve specific URL paths. When configuring your HTTP router, you should register the server for all paths listed in sumdb.ServerPaths to ensure full protocol compliance.

    var ServerPaths = []string{
    	"/lookup/",
    	"/latest",
    	"/tile/",
    }
  11. Configure gosumcheck CLI flags

    master

    The gosumcheck command supports the following flags to customize the verification process:

    FlagShortDefaultDescription
    --help-h8Changes the tile height (Note: This flag is documented but its functional impact on output is specific to the internal demo implementation).
    --key-ksum.golang.org+033de0ae+Ac4zctda0e5eza+HJyk9SxEdh+s3Ux18htTTAD8OuAn8Sets the Go checksum database server key.
    --url-u(empty)Overrides the URL of the server (usually derived from the key name).
    --verbose-vfalseEnables verbose output. When enabled, it reports the URL and elapsed time for each server request to stderr.
    # Example using a custom URL and verbose output
    gosumcheck -u https://my-custom-sumdb.com -v go.sum
    
    # Example specifying a specific key
    gosumcheck -k my-custom-key go.sum
  12. Use the gosumcheck CLI to verify go.sum files

    master

    The gosumcheck command verifies the checksums in a go.sum file against a Go checksum database server.

    WARNING: gosumcheck is a proof-of-concept demo. It is not intended for production scripts or continuous integration (CI) testing because it does not cache information, making it expensive and unable to detect certain server misbehaviors or HTTPS man-in-the-middle attacks. Additionally, it does not set an exit status to report problems found, which makes it unsuitable for automated error detection.

    To use it, provide one or more go.sum files as arguments.

    gosumcheck go.sum