Semian

Semian is a library that uses heuristics to implement 'failing fast'. It is designed to prevent slow or unresponsive resources from degrading the overall capacity and response time of an application.

When to use Semian

Semian is most beneficial for applications aiming to eliminate Single Points of Failure (SPOFs) that are experiencing issues with slow resources. It is particularly useful when a resource becomes unresponsive or consistently slow, causing requests to hit full timeouts.

Key Problems Solved

1. Response Time Spikes: Without Semian, if a resource (like a session store) is down or slow, every request attempting to access it must wait for the full timeout duration. Semian uses heuristics to detect these failures and fail the requests instantly, preventing the average response time from spiking.
2. Capacity Loss: When workers (e.g., single-threaded web workers) wait for a slow resource to time out, they cannot serve other requests. This leads to a significant degradation in the cluster's total capacity. By failing fast, Semian frees up workers to handle other requests immediately.

Important Considerations

• Complexity: Semian introduces complexity and should be introduced with care. It works by raising exceptions based on heuristics; you must understand its behavior before using it in production.
• Not a Magic Wand: It does not solve all latency problems; it specifically addresses the impact of slow/unresponsive resources.
• Threaded vs. Evented: If your application is multithreaded or evented (unlike Resque or Unicorn), these problems may be less pressing, though Semian can still provide value.

Semian addresses the limitations of standard timeouts. While timeouts prevent a request from waiting forever, they do not prevent the 'slow failure' problem where every request incurs the cost of the timeout duration.

In a typical scenario where a resource like Redis is slow:

• Without Semian: Every request to the resource waits for the full timeout (e.g., 200ms). This increases the average response time for all pages and consumes worker capacity, as workers are stuck waiting for timeouts rather than serving fast requests.
• With Semian: After a small amount of convergence time, Semian's heuristics identify the resource as failing and cause subsequent requests to fail instantly. This preserves the application's capacity and keeps response times low even during resource downtime.

Example of a pattern Semian protects:

# This code is resilient to failure but vulnerable to latency
def index
  @user = fetch_user
  @posts = Post.all
end

private
def fetch_user
  user = User.find(session[:user_id])
rescue Redis::CannotConnectError
  nil
end

In the example above, if Redis is unavailable, the code handles the error. However, if Redis is merely slow, the fetch_user method will hang for the duration of the timeout on every single request, causing the issues Semian is designed to solve.

Semian provides a multi-layered defense line for resource access:

1. Circuit Breaker: The first check. If the circuit is open, Semian raises an exception immediately to trigger a fallback (defaulting to a 500 response). This prevents calls from even attempting to reach the resource when it is known to be failing.
2. Bulkhead: If the circuit is closed, Semian checks the bulkhead. If too many workers are already querying the resource, it fails instantly. This limits the number of concurrent connections to a resource, preventing a single slow resource from consuming all application workers.
3. Driver/Data Store: If both checks pass, the driver attempts to query the data store. If the data store is slow or fails, the driver raises an exception (e.g., after a timeout), which in turn affects the circuit breaker and bulkhead state for future calls.

Key distinction:

• Circuit Breakers allow for regaining 100% capacity once the resource recovers.
• Bulkheads guarantee a minimum capacity by limiting concurrency, ensuring that even during an incident, a portion of your workers remain available for other tasks.

The ExperimentalResource class is an experimental adapter designed to simulate a distributed service with multiple endpoints. It is used to test how your application handles various failure scenarios and performance characteristics by simulating latencies and errors.

Key Simulation Capabilities

• Multiple Endpoints: Configure multiple endpoints, each with its own latency profile.
• Statistical Latencies: Latencies can be assigned using statistical distributions (currently supports Log-normal distribution).
• Latency Bounds: You can set minimum and maximum latency constraints.
• Request Timeouts: You can configure a maximum timeout. If a request would exceed this timeout, the resource sleeps for the timeout duration and then raises an exception, simulating real-world timeout behavior.
• Baseline Error Rate: You can configure a probability of request failure. Failed requests throw RequestError exceptions after partial processing.
• Service-Wide Degradation: You can simulate service degradation across all endpoints, including:
  • Latency degradation: Adding fixed latency to all requests.
  • Error rate changes: Modifying the error rate for the entire service.
  • Gradual ramp-up: Both latency and error rate degradations support gradual transitions over time.

Instead of manually calculating the number of tickets, you can use a quota. A quota is a proportion (0.0 to 1.0) of the active workers allowed to connect to the resource. This is ideal for environments with non-uniform worker distribution.

Rules:

• You must pass exactly one of tickets or quota.
• Tickets available will be the ceiling of the quota ratio to the number of workers.
• If using a forking web server (like Unicorn), call Semian.unregister_all_resources before/after forking.

Note on Thread Safety: In threaded environments (Puma, Sidekiq), you should disable bulkheads (bulkhead: false) because Semian's internal use of SEM_UNDO is tied to the process, not the thread, which can lead to ticket starvation or deadlocks if threads are killed.

client = Redis.new(semian: {
  name: "inventory",
  quota: 0.51, # Allow 51% of workers to connect
  success_threshold: 2,
  error_threshold: 4,
  error_timeout: 20
})