Sentinel Go

Sentinel Go is a reliability and resiliency library for microservices that focuses on managing traffic flow. It provides several key capabilities to protect distributed systems:

• Flow Control: Managing the rate of incoming requests.
• Traffic Shaping: Smoothing out burst traffic to match system capacity.
• Concurrency Limiting: Restricting the number of concurrent requests.
• Circuit Breaking: Automatically cutting off requests to unreliable downstream services to prevent cascading failures.
• System Adaptive Overload Protection: Automatically adjusting to system load to prevent crashes.

Additionally, Sentinel supports real-time monitoring of single-machine runtime information and can export metrics to external components like Prometheus.

To simulate business-level failures (e.g., 5XX errors), set the first argument of setup.sh to false (the default).

In this mode, the service nodes remain reachable, but specific nodes enter a 'fault period' based on their ID. For each node, the fault starts at startTime + 5s + (id * 5s) and lasts for 20 seconds. During this window, the Ping method in handler.go returns errors.New("internal server error") instead of a successful response.

This tests Sentinel's ability to remove nodes from the load balancer that are technically online but returning application-level errors, a capability that standard service registries often lack.

Sentinel Go provides specialized modules for easier integration into existing ecosystems:

• Framework Adapters: Out-of-the-box integrations for popular Go web frameworks and microservice components.
• Dynamic Data-source Modules: Support for dynamic configuration of rules via external data sources.

To simulate a node crash scenario, set the first argument of setup.sh to true.

When node_crash=true, the script performs the following sequence:

1. Start Processes: Launches node_count service processes on unique ports (e.g., 9001 to 9009) using go run . --server_address=:$port.
2. Stop Processes: After 5 seconds, it kills the processes sequentially using pgrep -f "hello_micro --server_address=:$port" | xargs kill.
3. Restart Processes: After another 5 seconds, it restarts the processes on the same ports.

This simulates network-level failures where RPC calls might block or fail due to the node being unreachable.

cd hello_micro && ./setup.sh true 4

To use the Sentinel Go Kubernetes CRD data-source, you must first install the required Custom Resource Definitions (CRDs) into your Kubernetes cluster using kubectl.

Run the following command to apply the CRD bases:

kubectl apply -f config/crd/bases

To enable outlier ejection (ejecting unhealthy nodes) in a Kitex client, you must:

1. Enable outlier detection in SentinelClientMiddleware via options.
2. Wrap your existing discovery.Resolver with OutlierClientResolver.

When EnableOutlier is true, the middleware uses a custom slot chain containing outlier.DefaultSlot and outlier.DefaultMetricStatSlot to track node health and identify nodes to filter.

To protect go-zero services with Sentinel, you can use the SentinelRouteMiddleware. This middleware automatically generates a resource name based on the HTTP method and URL path (e.g., GET:/api/user) and applies Sentinel flow control using the ResTypeWeb resource type and Inbound traffic type.

To use this in a go-zero project, define the middleware in your .api file using the @server directive:

@server(
    middleware: SentinelRoute
)
service your_service {
    // ... routes
}

Note: The middleware implementation provided is a template designed to be used with goctl. The Handle method intercepts requests, calls sentinel.Entry, and returns a 429 Too Many Requests error with the message Blocked by Sentinel if the request is throttled.

When using Sentinel with a Hertz server, you can customize how resources are identified and how blocked requests are handled using ServerOption.

By default:

• Resource Extraction: Uses the format METHOD:PATH (e.g., GET:/api/user).
• Block Fallback: Aborts the request with http.StatusTooManyRequests (429).

Use WithServerResourceExtractor to define a custom function for identifying resources and WithServerBlockFallback to define custom logic when a request is blocked.

Integrate Sentinel into a Gear-based application using SentinelMiddleware. This middleware automatically creates Sentinel entries for incoming web requests to provide flow control and protection.

Default Behavior:

• Resource Name: Uses the pattern {method}:{path} (e.g., GET:/api/users/:id).
• Block Fallback: If a request is blocked by Sentinel, the middleware returns an http.StatusTooManyRequests (429) status code with the body Blocked by Sentinel.

Customization: You can customize the middleware behavior by passing Option arguments to SentinelMiddleware to:

• Provide a custom resourceExtract function to define how the resource name is generated from the gear.Context.
• Provide a custom blockFallback function to define specific logic (e.g., custom error responses) when a request is blocked.

Use NewDataSource to create a datasource.DataSource that watches a specific key in Consul for configuration updates. You can provide an existing *api.Client from the HashiCorp Consul API or provide a *api.Config to let Sentinel initialize the client for you.

After creation, call Initialize() to perform the first read and start the background watching process. To stop watching and release resources, call Close().