DOCSEARCHbyAlgolia
Submit Docs

EVCache Documentation

repository·master·Indexed 24 days ago

https://github.com/netflix/evcache

EVCache (Ephemeral Volatile memCache) is a high-performance, distributed, in-memory key-value caching solution built on Memcached and Spymemcached, optimized for AWS EC2 environments. It features multi-zone replication, Ketama consistent hashing for intra-zone sharding, and a client library supporting LRU caching and dynamic compression. The system includes a REST API (evcrest) for non-Java clients and integrates with Kafka for write-failure handling and cross-region replication.

Tokens
2.5K
Snippets
0
Records
25
Agent score
80%

What's inside EVCache

  1. What is EVCache?

    master

    EVCache (Ephemeral Volatile memCache) is a fast, distributed, resilient, and highly available in-memory key-value data store designed for cloud environments. It is intended for storing transient data such as strings and objects.

    Key characteristics:

    • Ephemeral: Designed for transient data that can be lost.
    • Volatile: An in-memory store where data is subject to change.
    • Cache: Primarily uses memcached and can, in some configurations, be backed by RocksDB.

  2. What is EVCache and how does it work?

    master

    EVCache is a distributed, in-memory Data Management service. It provides a client library used by various application types (Web services, microservices, standalone apps, SPARK, and FLINK).

    Core Architecture:

    • Internal Mechanism: It uses spymemcached to communicate with a cluster of memcached servers.
    • Multi-Zone Replication: EVCache clusters (ServerGroups) are provisioned in each zone. Every write is replicated across all zones (e.g., three writes for every one application-level write).
    • Read Strategy: To meet SLAs, reads should primarily occur from the local zone.
    • Consistency Model: EVCache prioritizes availability and speed. Write-failures or Write-After-Write (WAW) hazards may lead to temporary data inconsistencies across zones. This is mitigated by short TTLs, Kafka-based write-failure logging for consistency checkers, or application-level retries.
    • Fault Tolerance: Read failures are automatically retried on other copies to mask transient failures.

  3. Understand EVCache key canonicalization

    master

    To prevent namespace collisions when multiple caches share a single memcached instance, EVCache uses key canonicalization. A key is canonicalized by prepending a cache prefix and a : character.

    Example:

    • Prefix: cid
    • Original Key: foo
    • Canonicalized Key: cid:foo

  4. Data mirroring and sharding behavior

    master

    EVCache employs different strategies for data distribution depending on the scope:

    • Cross-Zone (Mirroring/Replication): Data is always mirrored between zones. Two cache servers in different zones supporting the same cache will contain the same data.
    • Intra-Zone (Sharding): Within a single zone, data is sharded across the set of instances using the Ketama consistent hashing algorithm.

  5. EVCache workload characteristics and constraints

    master

    When designing workloads for EVCache, consider the following performance characteristics and constraints:

    Key and Value Constraints:

    • Key Size: Should be < 255 bytes (including the cache prefix).
    • Key Format: Keys cannot contain spaces or newline characters.
    • Value Size: Values larger than 1MB are chunked on the server, which increases retrieval latency.

    Performance Expectations:

    • Throughput: Can reach over 125K ops/sec (for 1KB data size) on an m4.xlarge instance.
    • Latency:
      • Intra-zone (1KB value): < 1ms
      • Inter-zone: > 2ms
      • Target SLA: Average latency < 1ms; 99th percentile < 5ms (note: 99% is heavily influenced by JVM pauses).
    • Typical TTL: A few minutes to a few hours. Long TTLs (multiple days) are not recommended.
    • Hit Rate: Typically in the 90% range.

  6. Enable client-side LRU caching

    master
    The EVCache client supports an LRU-based in-memory cache. This can be enabled dynamically and is transparent to the application. It is most effective for immutable objects that experience duplicate reads within a short duration.

  7. How EVCache discovery and client management works

    master

    EVCache uses a Discovery mechanism to locate suitable cache instances:

    1. Server Advertisement: EVCache servers advertise themselves as an EVCache App and register with a discovery service.
    2. Client Lookup: Clients looking for instances of a specific app retrieve the list of available instances from the discovery service.
    3. Client Management: The EVCache client manages the retrieved list of instances.

    Bootstrapping without Discovery: If the client is in an environment without Discovery/Eureka, you can bootstrap the client by providing ServerGroup configuration via System properties or FastProperties. The client also uses the Spinnaker API as a backup to find instances.

  12. Configure Simple Node List

    master

    When using a simple node list instead of service discovery, configure the following system properties and EVCache properties:

    • NETFLIX_ENVIRONMENT: The environment (e.g., via @environment or eureka.environment).
    • EC2_REGION: The AWS region (e.g., via @region or eureka.region).
    • <CACHE>-NODES: The list of EVCache nodes.
    • <CACHE>.use.batch.port: Whether to use the batch port (overrides evcache.use.batch.port).