katsubushi

katsubushi uses a Snowflake-like algorithm to generate 64-bit unsigned integer IDs.

Default Format (64-bit)

• timestamp (42 bits): Milliseconds since katsubushi epoch (2015-01-01 00:00:00 UTC). Lasts ~139 years.
• worker ID (10 bits): Supports up to 1024 workers.
• sequence (12 bits): Up to 4096 IDs per millisecond per worker.

JS-safe Format (53-bit)

Use the -js-safe-id flag to generate IDs that fit within JavaScript's Number.MAX_SAFE_INTEGER (2^53 - 1). This prevents precision loss in JS environments.

• timestamp (39 bits): 10ms units since epoch. Lasts ~174 years.
• worker ID (6 bits): Up to 64 workers.
• sequence (8 bits): Up to 256 IDs per 10ms per worker.

Warning: Do not mix default and JS-safe formats in the same service. If using -js-safe-id with -redis, use a different namespace (?ns=) to avoid worker ID collisions.

You can run katsubushi using Docker images from Docker Hub or GHCR. Use the -worker-id flag to specify a unique ID for the worker, or use -redis to manage worker IDs via Redis.

# Run with a specific worker ID
$ docker run -p 11212:11212 katsubushi/katsubushi -worker-id 1

# Run using a Redis server for worker ID management
$ docker run -p 11212:11212 katsubushi/katsubushi -redis redis://your.redis.host:6379/0

You can install katsubushi using go install or by downloading releases from GitHub.

$ go install github.com/kayac/go-katsubushi/v2/cmd/katsubushi@latest

katsubushi is configured using command-line flags. Most flags can also be set via environment variables.

Precedence: Command-line options > Environment variables.

Environment Variable Naming: For a flag like -worker-id, the following are checked in order:

1. KATSUBUSHI_WORKER_ID (Recommended)
2. WORKER_ID
3. worker_id

Note: -version cannot be set via environment variables. Port options (-port, -http-port, etc.) accept URLs (e.g., tcp://10.0.0.1:11212) or plain port numbers.

The katsubushi CLI supports configuration via environment variables. The CLI maps environment variables to flags by converting the flag name to uppercase and replacing hyphens with underscores, prefixed with KATSUBUSHI_.

For example, the flag -worker-id can be set via the KATSUBUSHI_WORKER_ID environment variable. The CLI also attempts to fall back to non-prefixed versions (e.g., WORKER_ID) for backward compatibility, though these may trigger warnings if they collide with platform-provided variables.

You can enable runtime profiling and statistics by using the -enable-pprof and -enable-stats flags. These services run on a separate debug port (defaulting to 8080 via -debug-port).

• pprof: Exposes standard Go profiling endpoints at /debug/pprof/.
• stats: Exposes statistics via the golang-stats-api-handler at /debug/stats.

# Enable profiling and stats on port 9090
katsubushi -enable-pprof -enable-stats -debug-port 9090 -port 11212 -redis redis://localhost:6379

The katsubushi binary serves as a distributed ID generator. You can run it as a standalone server by specifying at least one server type (Memcached-compatible, HTTP, or gRPC).

To ensure unique worker IDs in a distributed environment, you should either provide a specific -worker-id or use a Redis instance for automated allocation via -redis.

# Example: Run a Memcached-compatible server with automated worker ID allocation via Redis
katsubushi -port 11212 -redis redis://localhost:6379

# Example: Run an HTTP server with a specific worker ID and JS-safe IDs
katsubushi -http-port 8080 -worker-id 1 -js-safe

The Generator service provides methods to retrieve IDs. You can fetch a single ID or multiple IDs in a single request.

Methods

• Fetch: Retrieves a single ID based on a FetchRequest.
• FetchMulti: Retrieves multiple IDs based on a FetchMultiRequest.

### Generator

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------ |
| Fetch | [FetchRequest](#katsubushi-FetchRequest) | [FetchResponse](#katsubushi-FetchResponse) |  |
| FetchMulti | [FetchMultiRequest](#katsubushi-FetchMultiRequest) | [FetchMultiResponse](#katsubushi-FetchMultiResponse) |  |

If you enable the HTTP server using the -http-port flag, you can interact with katsubushi via HTTP.

GET /id

Returns a single ID.

• If Accept: application/json is set, returns {"id":"..."}.
• Otherwise, returns the ID as plain text.

GET /ids?n=(number_of_ids)

Returns multiple IDs. The parameter n must be between 1 and 1000.

• If Accept: application/json is set, returns {"ids":["..."]}.
• Otherwise, returns IDs as newline-delimited text.

GET /stats

Returns katsubushi statistics in JSON format.

{
  "pid": 1859630,
  "uptime": 50,
  "time": 1664761614,
  "version": "1.8.0",
  "curr_connections": 1,
  "total_connections": 5,
  "cmd_get": 15,
  "get_hits": 25,
  "get_misses": 0
}

katsubushi implements a protocol compatible with memcached. You can use standard memcached clients to request unique IDs using the GET command.

GET / GETS

Request one or more IDs by key. The returned VALUE is the unique ID.

GET id1 id2
VALUE id1 0 18
283890203179880448
VALUE id2 0 18
283890203179880449
END

GET new
VALUE new 0 20
8070450532247928832
END

The Stats service provides operational metrics and health information about the katsubushi instance.

Methods

• Get: Retrieves current statistics using a StatsRequest and returns a StatsResponse.

### Stats

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------ |
| Get | [StatsRequest](#katsubushi-StatsRequest) | [StatsResponse](#katsubushi-StatsResponse) |  |

The application supports controlling how long connections remain open while idle. Use the following constants to configure IdleTimeout in your Config:

• katsubushi.DefaultIdleTimeout: 600 seconds.
• katsubushi.InfiniteIdleTimeout: A duration of 0, which disables the idle timeout.
• katsubushi.ShutdownTimeout: 10 seconds (used for waiting for in-flight requests during graceful shutdown).