System Design 101

Resiliency patterns are cloud design patterns used to reduce the damage caused by minor errors that can escalate into system-wide failures (the 'snowball effect'). These patterns are typically used in combination rather than in isolation to build robust distributed systems.

The 8 core resiliency patterns are:

• Timeout: Prevents a system from waiting indefinitely for a response.
• Retry: Attempts to re-execute a failed operation.
• Circuit breaker: Stops requests to a failing service to allow it to recover.
• Rate limiting: Controls the rate of incoming requests to prevent exhaustion.
• Load shedding: Drops excess requests when the system is overloaded.
• Bulkhead: Isolates failures by partitioning resources.
• Back pressure: Signals upstream components to slow down production when downstream components are overwhelmed.
• Let it crash: Allows a process to fail and be restarted by a supervisor to return to a clean state.

This guide identifies the seven most common patterns used in distributed system design to solve challenges related to scalability, reliability, and data consistency. These patterns include:

• Ambassador: A pattern where a helper service (the ambassador) handles common tasks like logging, monitoring, or security on behalf of a primary service.
• Circuit Breaker: A pattern used to prevent a system from repeatedly trying to execute an operation that is likely to fail, allowing the system to recover and preventing cascading failures.
• CQRS (Command Query Responsibility Segregation): A pattern that separates read (query) operations from write (command) operations to optimize performance and scalability.
• Event Sourcing: A pattern where state changes are captured as a sequence of immutable events rather than just storing the current state.
• Leader Election: A pattern used to designate a single node as the coordinator or 'leader' among a cluster of nodes to manage shared resources or tasks.
• Publisher/Subscriber (Pub/Sub): A messaging pattern where senders (publishers) do not program the messages to be sent directly to specific receivers, but instead categorize messages into classes/topics that subscribers listen to.
• Sharding: A database partitioning pattern that breaks a large dataset into smaller, more manageable pieces (shards) distributed across multiple nodes.

This guide identifies the five most common strategies used for software releases to balance speed, risk, and availability:

1. Big Bang Deployment: The entire system is updated at once. This is high-risk as it requires downtime and makes rollbacks difficult if errors occur.
2. Rolling Deployment: Updates are applied to instances incrementally (one by one or in small batches). This maintains availability but can lead to version mismatch issues during the transition.
3. Blue-Green Deployment: Two identical production environments exist (Blue and Green). One is live, while the other receives the new version. Traffic is switched instantly once the new version is verified, allowing for near-instant rollbacks.
4. Canary Deployment: The new version is rolled out to a small subset of users first. If no issues are detected, it is gradually expanded to the rest of the infrastructure.
5. Feature Toggle: Code is deployed to production but remains inactive behind a conditional flag. Features are enabled for specific users or segments without requiring a new deployment.

Reddit's architecture is designed to serve over 1 billion monthly users by transitioning from a Python-based monolith to a microservices architecture built with Go. The system utilizes a multi-layered approach for request routing, data storage, and asynchronous processing.

Key Architectural Components:

• Edge & Routing: Uses a Fastly CDN as the front end, followed by a load balancer that routes requests to appropriate services.
• API Layer: Primarily uses GraphQL. The architecture has evolved from a GraphQL monolith to GraphQL Federation, which combines multiple Domain Graph Services (DGS). Core entities are served via Go subgraphs.
• Data Storage:
  • Postgres: Serves as the primary core data model.
  • Memcached: Used in front of Postgres to reduce database load.
  • Cassandra: Utilized for new features due to its high resiliency and availability.
  • Debezium: Implements Change Data Capture (CDC) to support data replication and maintain cache consistency.
• Asynchronous Processing & Messaging:
  • RabbitMQ: An async job queue used to defer expensive operations (e.g., user voting, submitting links) to job workers.
  • Kafka: Used for real-time data transfer to perform content safety checks and moderation rules.
• Infrastructure & Deployment:
  • Hosting: AWS and Kubernetes.
  • CI/CD & IaC: Spinnaker, Drone CI, and Terraform.

Netflix utilizes a polyglot persistence strategy, selecting specific database types based on the requirements of different services. Use the following mapping to understand which database categories are appropriate for specific workloads:

• Relational (ACID compliance): Use MySQL for billing, subscriptions, taxes, and revenue. Use CockroachDB for multi-region active-active architectures, global transactions, and data pipeline workflows.
• Columnar (Analytics): Use Redshift and Druid for structured data storage and processing with Spark. Use Tableau for visualization.
• Key-Value (Caching): Use EVCache (built on Memcached) for high-scale caching of service data like homepages and personal recommendations.
• Wide-Column (High Availability/Scale): Use Cassandra for high-volume data such as Video/Actor info, User Data, Device info, and Viewing History.
• Time-Series (Metrics): Use Atlas (an in-memory database) for metrics storage and aggregation.
• Unstructured (Object Storage): Use S3 for images, videos, metrics, and log files. Use Apache Iceberg alongside S3 for big data storage management.

When designing a system, choose a database model based on your data structure and access patterns. The six primary models are:

1. Flat Model: Data is organized into a single table (rows and columns) similar to a spreadsheet. Best for simple data with no complex relationships.
2. Hierarchical Model: Data is organized in a tree structure where each record has one parent and multiple children. Best for clear parent-child relationships, but poor for many-to-many relationships.
3. Relational Model: Data is represented in tables (relations) using rows (tuples) and columns (attributes). It uses keys and normalization to ensure integrity and is the standard for SQL-based systems. It excels at handling many-to-many relationships and complex queries.
4. Star Schema: A specialized model for OLAP (Online Analytical Processing) in data warehousing. It consists of a central fact table (quantitative data) surrounded by dimension tables (descriptive attributes). Optimized for fast analytical query performance by minimizing joins.
5. Snowflake Model: A normalized version of the Star Schema where dimension tables are split into multiple related tables. This improves storage efficiency and data integrity but increases query complexity due to more joins.
6. Network Model: A graph-based structure where records can have multiple parents and multiple children. This is used to represent complex relationships and many-to-many connections more efficiently than the hierarchical model.

C++ is a versatile language used in performance-critical and resource-constrained environments. Key industries and applications include:

• Embedded Systems: Leveraging efficiency and fine hardware control.
• Game Development: Utilizing high performance and efficiency.
• Operating Systems: Providing extensive control over system resources and memory for OS and low-level utilities.
• Databases: Implementing high-performance systems for efficient memory management and fast query execution.
• Financial Applications: Used in high-frequency trading and other performance-sensitive financial software.
• Web Browsers: Developing browsers and core components like rendering engines.
• Networking: Developing network devices and simulation tools.
• Scientific Computing: Powering engineering and scientific applications requiring high performance and precise computational control.