DOCSEARCHbyAlgolia
Submit Docs

Muduo C++ Network Library

repository·master·Indexed 12 days ago

https://github.com/chenshuo/muduo

A high-performance, multithreaded C++ network library built on the reactor pattern for Linux environments. It includes support for asynchronous HTTP requests via a libcurl bridge, a broadcasting hub system, and various protocol examples including Memcached, Finger (RFC 1288), and a Google Protobuf RPC proof of concept.

Tokens
1.6K
Snippets
3
Records
11
Agent score
97%

What's inside Muduo

  3. Overview of the muduo/net/protorpc proof of concept

    master

    The muduo/net/protorpc directory contains a proof of concept implementation of Google Protobuf RPC built on top of the muduo network library.

    Warning: This implementation is a proof of concept only. The object lifetime management is not ideal and deviates from the standard muduo approach.

    For a more stable or production-ready version, users are encouraged to use the dedicated repository: http://github.com/chenshuo/muduo-protorpc.

  4. Understand the Hub broadcasting system

    master

    The hub system is a broadcasting architecture composed of three main components:

    1. hub: A central server designed for broadcasting content.
    2. pubsub: A client library used to interact with the hub server.
    3. pub: A command-line tool used to publish content to specific topics.
    4. sub: A demonstration tool used to subscribe to and view content from specific topics.

    Developers can use the pubsub library to build custom clients that communicate with the hub server for real-time message distribution.

  5. Understand the Finger protocol (RFC 1288)

    master

    The Finger protocol is a text-based protocol used to obtain information about users on a remote host. It operates over TCP on port 79.

    Protocol Workflow:

    1. Connection: The local host opens a TCP connection to the remote host on port 79.
    2. Request: The local host sends a one-line query following the Finger query specification to the RUIP (Remote User Information Protocol) process.
    3. Processing: The RUIP receives the query, processes it, and sends back the answer.
    4. Termination: The RUIP initiates the connection closure after sending the response. The local host receives the answer and the close signal, then closes its end of the connection.

  6. Run the distributed word counting example

    master

    The wordcount example demonstrates a distributed system where wordcount_hasher shards <word,count> pairs to multiple wordcount_receiver instances based on the hash of the word.

    Architecture

    1. Receivers: Collect <word,count> pairs from multiple hashers and write the aggregated results to disk.
    2. Hashers: Shard data to receivers using hash(word).

    Deployment Steps

    To run a cluster with 3 hashers and 4 receivers:

    1. Start the Receivers

    Run one receiver on each of the 4 target machines. The syntax is bin/wordcount_receiver <port> <num_hashers>.

    2. Start the Hashers

    Run the hashers on 3 machines. The syntax is bin/wordcount_hasher '<receiver_addresses>' <input_file1> [<input_file2> ...].

    3. Completion

    Wait for all hasher and receiver processes to exit to ensure all data is processed and written to disk.

    ### Example: 3 Hashers and 4 Receivers

# 1. Run 4 receivers on 4 machines
bin/wordcount_receiver port1 3
bin/wordcount_receiver port2 3
bin/wordcount_receiver port3 3
bin/wordcount_receiver port4 3

# 2. Run 3 hashers on 3 machines
bin/wordcount_hasher 'ip1:port1,ip2:port2,ip3:port3,ip4:port4' input1
bin/wordcount_hasher 'ip1:port1,ip2:port2,ip3:port3,ip4:port4' input2
bin/wordcount_hasher 'ip1:port1,ip2:port2,ip3:port3,ip4:port4' input3 input4

  8. Install dependencies for Muduo

    master

    Before building Muduo, ensure your system meets the following requirements:

    System Requirements

    • Linux kernel: version >= 2.6.28
    • Compiler: GCC >= 4.7 or Clang >= 3.5
    • Boost: Required for boost::any only.

    Package Installation

    Install the necessary build tools and libraries using your package manager:

    For Debian, Ubuntu, and derivatives:

    $ sudo apt install g++ cmake make libboost-dev

    For CentOS and derivatives:

    $ sudo yum install gcc-c++ cmake make boost-devel
    # Debian, Ubuntu, etc.
$ sudo apt install g++ cmake make libboost-dev

# CentOS
$ sudo yum install gcc-c++ cmake make boost-devel

  9. Use the muduo-curl bridge proof-of-concept

    master

    The muduo-curl bridge is a proof-of-concept implementation demonstrating how to integrate libcurl with the muduo network library. It provides the simplest use case for performing asynchronous HTTP requests within a muduo-based application.

    Critical Implementation Notes:

    1. DNS Blocking: DNS resolution in curl may be a blocking operation if your libcurl was not built with c-ares support. This can impact the performance of the muduo event loop.
    2. Object Lifecycle: The Request object must remain in scope and survive until the doneCallback is executed. Do not destroy the request object before the callback completes.

  10. Use the Memcached protocol example for server and client implementation

    master

    The examples/memcached directory provides a simple implementation of the Memcached protocol for both the server and client sides. This is intended as sample code to demonstrate network programming patterns using the muduo library rather than a production-ready Memcached replacement.

    Key Characteristics

    Server Implementation:

    • Simplicity over Performance: The server is designed to pass feature tests while maintaining a simple codebase.
    • Memory Management: Uses standard (tc)malloc without customized memory management or footprint control.
    • Networking: Supports TCP only; does not support Unix domain sockets and is limited to listening on a single TCP port.

    Current Limitations (Not implemented):

    • incr/decr operations
    • UDP protocol support
    • Binary protocol support
    • Key expiration
    • LRU (Least Recently Used) eviction policy

  11. Use LengthHeaderCodec for echo server and client examples

    master

    The ZeroMQ examples in this repository demonstrate how to implement an echo server and client using a LengthHeaderCodec. This codec is used to handle message framing by prefixing messages with their length, ensuring reliable message delimitation in the stream.

    • local_lat.cc: Implements an echo server using LengthHeaderCodec.
    • remote_lat.cc: Implements an echo client using LengthHeaderCodec.