DOCSEARCHbyAlgolia
Submit Docs

Advanced Java Knowledge Repository

repository·main·Indexed 13 days ago

https://github.com/doocs/advanced-java

A comprehensive guide for Java engineers focusing on high-concurrency architecture, distributed systems, microservices, and big data processing. Key topics include Dubbo RPC, Spring Cloud, Redis, Elasticsearch, Kafka, and high-availability patterns using Hystrix and Sentinel. It covers distributed locks (Zookeeper/Redis), distributed transactions (TCC, XA), database sharding, and algorithmic approaches for massive data processing.

Tokens
79K
Snippets
75
Records
303
Agent score
98%

What's inside Advanced Java

  1. Overview of Advanced Java Knowledge for Internet Engineers

    main
    This project provides a systematic collection of knowledge for Java engineers aiming to advance their skills in high-concurrency, distributed systems, high-availability, microservices, and massive data processing. It is designed as a study and reference guide for technical interviews and real-world architectural design.

  2. Common Zookeeper usage scenarios

    main

    Zookeeper is a fundamental component in distributed systems used to solve coordination and consistency challenges. The primary application scenarios include:

    • Distributed Coordination: Facilitating communication and state synchronization between decoupled systems (e.g., System A notifying System B of a task completion via watchers).
    • Distributed Locking: Ensuring mutual exclusion across multiple machines to prevent race conditions during concurrent data modifications.
    • Metadata and Configuration Management: Serving as a centralized, highly available store for system configurations and service discovery (e.g., used by Kafka, Storm, and Dubbo registry).
    • High Availability (HA): Implementing active-standby (master-slave) mechanisms where a standby process takes over immediately if the primary process fails, detected via Zookeeper.

  3. Explore the advanced-java learning roadmap

    main

    The advanced-java repository provides a comprehensive knowledge base for internet Java engineers focusing on high-concurrency, distributed systems, and large-scale data processing. You can navigate the learning path through the following core modules:

    • High-Concurrency Architecture (高并发架构): Covers Message Queues (MQ), Search Engines (ES), Caching, Database Sharding (分库分表), and Read-Write Separation.
    • Distributed Systems (分布式系统): Covers Dubbo, Distributed Locks, Transactions, and Session management.
    • High-Availability Architecture (高可用架构): Covers Rate Limiting, Circuit Breaking, Degradation, and Hystrix implementation.
    • Microservices Architecture (微服务架构): Covers Microservices fundamentals, Spring Cloud, Service Governance, and Communication mechanisms.
    • Massive Data Processing (海量数据处理): Covers classic algorithms and design patterns for big data scenarios.

  4. Big Data Processing Problem Solving Guides

    main

    This section provides technical guides and algorithmic approaches for common big data processing challenges. Key topics include:

    • URL Deduplication: How to find duplicate URLs from a massive set.
    • Frequency Analysis: Finding high-frequency words, the most popular query strings, or the top 500 numbers.
    • IP & Traffic Analysis: Identifying the most frequent IP address visiting a specific site on a given day.
    • Set Operations: Finding non-repeating integers or checking for the existence of a specific number in massive datasets.
    • Statistical Calculations: Finding the median value from a set of 500 million numbers.
    • Counting & Sorting: Counting unique phone numbers or sorting query strings by their frequency.

  5. Strategies for Redis High Concurrency and High Availability

    main

    To ensure a Redis deployment can handle high traffic (concurrency) and remain operational during failures (availability), you must implement specific architectural patterns depending on your scale requirements:

    1. Achieving High Concurrency

    • Master-Slave Architecture (Read/Write Splitting): Use a single Master instance for write operations and multiple Slave instances for read operations. This allows you to scale read throughput (e.g., multiple slaves can provide >100k QPS).
    • Redis Cluster: If you need to handle both massive read/write concurrency and large datasets that exceed a single machine's capacity, use Redis Cluster. This provides hundreds of thousands of operations per second across multiple nodes.

    2. Achieving High Availability

    • Sentinel (Automatic Failover): When using a Master-Slave architecture, deploy Redis Sentinel. Sentinel monitors the instances and performs automatic failover (promoting a slave to master) if the current master crashes, ensuring the system remains available.

  6. Distributed Systems Knowledge Roadmap

    main
    This section provides a comprehensive guide to mastering distributed systems, covering system decomposition, service frameworks, locking mechanisms, transactions, and session management. It is structured to help developers prepare for high-level technical interviews and architectural design tasks.

  7. Compare microservice deployment strategies

    main

    Choosing a deployment strategy for microservices involves balancing resource efficiency, isolation, deployment speed, and operational complexity. The primary strategies are:

    1. Single Host, Multiple Service Instances: Multiple services run on one physical or virtual machine, often sharing a single process group (e.g., multiple web apps on one Apache Tomcat server).
    2. Single VM, Single Service Instance: Each service is packaged as a Virtual Machine image (e.g., Amazon EC2 AMI) and runs in its own isolated VM.
    3. Single Container, Single Service Instance: Each service runs in an isolated container (e.g., Docker) with its own namespace and filesystem, managed by a cluster orchestrator like Kubernetes (k8s).
    4. Serverless Deployment: Services are uploaded as code/packages (e.g., AWS Lambda ZIP files) and executed only in response to events, with billing based on execution time and memory usage.

  8. Explore High Availability Architecture topics

    main

    The High Availability (HA) architecture section provides deep dives into fault tolerance, resource isolation, and stability patterns. Key areas of study include:

    Hystrix Implementation Details

    • Resource Isolation: Learn about thread pool isolation vs semaphore isolation and how to control isolation strategies at a fine-grained level.
    • Internal Mechanics: Understand the internal execution process of Hystrix and how the Circuit Breaker functions.
    • Optimization & Protection: Explore Request Cache for optimizing batch queries, fallback mechanisms using local caches, and using timeout mechanisms to protect service calls.
    • Limiting: Study how thread pools are used for interface rate limiting.

    System Design & Stability Patterns

    • High Availability Systems: Principles for designing resilient systems.
    • Rate Limiting (限流): Implementation strategies and practical applications in production.
    • Circuit Breaking (熔断): Framework selection (e.g., choosing between Sentinel and Hystrix) and underlying implementation principles.
    • Degradation (降级): Strategies for gracefully degrading service functionality during failures.

  10. Key features of Sentinel

    main

    Sentinel provides several advanced capabilities for high-concurrency systems that distinguish it from Hystrix:

    • Lightweight and High Performance: Designed for minimal overhead.
    • Traffic Control: Supports rate limiting based on QPS and relationship-based limiting.
    • Traffic Shaping: Supports 'slow start' and 'uniform rate' (匀速器) modes to smooth out traffic spikes.
    • System Load Protection: Can protect the system based on overall load metrics.
    • Real-time Monitoring and Control Panel: Provides an out-of-the-box dashboard for configuring rules, viewing second-level monitoring, and discovering machines.

  11. Sentinel Core Features and Performance

    main

    Sentinel is designed to be a lightweight and high-performance component:

    • Lightweight: The sentinel-core module has no unnecessary dependencies and is less than 200KB.
    • High Performance: Performance overhead is minimal. Significant impact (5% - 10%) is typically only observed when single-machine QPS exceeds 250,000.
    • Extensibility: Sentinel provides multiple extension points for users to customize behavior.
    • Ecosystem Support: Native adapters are available for Servlet, Dubbo, Spring Boot, Spring Cloud, and gRPC.

  12. Explore High Concurrency Architecture topics

    main

    The High Concurrency Architecture section provides deep dives into several critical components of modern distributed systems. You can navigate to specific guides for:

    • Message Queues (MQ): Covers advantages/disadvantages of Kafka, ActiveMQ, RabbitMQ, and RocketMQ; high availability; idempotency; reliability; message ordering; handling delays/backlogs; and architectural design.
    • Search Engines (Elasticsearch): Covers distributed architecture; write/query principles; Lucene and inverted indices; optimizing performance for billions of records; and production cluster deployment.
    • Caching (Redis/Memcached): Covers usage scenarios; Redis thread models; data types; expiration policies (LRU); high availability (Master-Slave, Sentinel); persistence mechanisms; Cluster mode and consistent hashing; handling Avalanche, Penetration, and Breakdown; cache-database consistency; and concurrency/CAS solutions.
    • Database Sharding (分库分表): Covers vertical/horizontal splitting; middleware comparison; dynamic migration from single to sharded databases; dynamic scaling; and global ID generation.
    • Read-Write Separation: Covers MySQL implementation, master-slave replication principles, and handling replication lag.
    • High Concurrency System Design: General principles for designing high-concurrency systems.