bytedance/g3

The G3 Project is a collection of enterprise-grade general-purpose proxy solutions. It includes tools for forward proxying, reverse proxying (in development), load balancing, and NAT traversal (in development).

Note: The G3 project is currently in maintenance mode. Active development has moved to the VEY project.

g3proxy is an enterprise-level forward proxy that supports TCP/TLS streaming, transparent proxying, and reverse proxying.

Its architecture is composed of several key components:

• Servers: Handle incoming client connections (Forward, Socks, Transparent, Reverse, Streaming, and Alias Port servers).
• Escapers: Define how to connect to upstream targets (Direct, Proxy Chaining, and Router escapers).
• Resolvers: Handle DNS resolution (c-ares, hickory, and fail-over).
• Auth: Manages user authentication and access control (ACLs, rate limits, and user-specific site configs).
• Audit & Logging: Provides protocol inspection, interception, and structured logging to backends like journald, syslog, or fluentd.
• Metrics: Provides multi-level metrics (Server, Escaper, User, Resolver, etc.) exportable via StatsD.

The g3proxy project utilizes several specialized helper protocols to extend its functionality across different components. These protocols allow different parts of the system (like escapers, auditors, or interceptors) to communicate or delegate specific tasks.

Key helper protocols include:

• route_query: Used for making queries within the route_query escaper.
• cert_generator: Used by the auditor during TLS interception.
• ip_locate: Used by the route_geoip escaper to resolve IP locations.
• icap_h2, icap_imap, and icap_smtp: Protocols used to enable ICAP (Internet Content Adaptation Protocol) support for HTTP/2, IMAP, and SMTP respectively.
• stream_detour: Used by the auditor to redirect client or remote streams to an external interception server.

The http_rproxy server type provides an HTTP reverse proxy. It supports several common configuration keys inherited from standard server settings (such as escaper, auditor, user_group, and shared_logger).

Authentication Support

Authentication schemes are determined by the user_group type:

• Basic Auth: Supported via hashed_user type.
• Negotiate Auth: Not yet supported (via gss_api type).

Required Configuration

You must define hosts to specify which hostnames the proxy should handle and where to route them.

g3proxy supports the following SOCKS proxy features:

• SOCKS4: Supports socks4 and socks4a (without ident verification) with most escapers.
• SOCKS5 TCP: socks5 TcpConnect is supported with most escapers.
• SOCKS5 UDP:
  • socks5 UdpAssociate is supported with some escapers but is disabled by default on the server side.
  • The default enabled method is UdpConnect, which is simplified and requires the target address for each packet to remain the same.
  • If no explicit bind IP is set, the address family for the TCP and UDP connections at the client side must be the same.
• Authentication: Only socks5 User auth is supported.
• Limitations:
  • No TLS or DTLS support.
  • No SOCKS6 support.

For advanced configuration, refer to documentation on SOCKS5 custom reply fields.

The route_client escaper selects the next escaper in a chain based on the client's IP address. Note that this escaper does not support path selection.

It supports the common default_next key to define a fallback escaper if no rules match. You can define selection logic using either exact_match for specific IP addresses or subnet_match for network ranges.

# Conceptual structure of a route_client configuration
route_client:
  default_next: "fallback_escaper"
  exact_match:
    - next: "escaper_a"
      ips: ["192.168.1.1"]
  subnet_match:
    - next: "escaper_b"
      subnets: ["10.0.0.0/24"]

A server configuration in g3proxy is defined as a map and requires two mandatory keys:

1. name: A unique identifier for the server (metric node name).
2. type: A string specifying the server type (e.g., http_proxy, socks_proxy, tcp_stream), which determines how other keys in the configuration are parsed.

Common server types include dummy_close, tcp_stream, tcp_tproxy, tls_stream, http_proxy, socks_proxy, http_rproxy, sni_proxy, plain_tcp_port, plain_tls_port, native_tls_port, plain_quic_port, and intelli_proxy.

You can control how specific protocols are handled using inspection policies. Most policies default to intercept.

Available protocol policies include:

• h2_inspect_policy: Controls HTTP/2.0 traffic (default: intercept).
• websocket_inspect_policy: Controls WebSocket traffic (default: intercept).
• smtp_inspect_policy: Controls SMTP traffic (default: intercept).
• imap_inspect_policy: Controls IMAP traffic (default: intercept).

To use a detour policy, you must also configure the stream_detour_service. If no detour service is configured, protocols set to detour will be bypassed.

h2_inspect_policy: <protocol inspect policy>
websocket_inspect_policy: <protocol inspect policy>
smtp_inspect_policy: <protocol inspect policy>
imap_inspect_policy: <protocol inspect policy>
stream_detour_service: <stream detour service config>

The direct_float escaper allows g3proxy to access target upstreams directly from the local machine. It supports the following interfaces:

• tcp connect
• udp relay
• udp connect
• http(s) forward
• ftp over http

To configure the local bind IP, you must use the Cap'n Proto RPC publish command. The published data must be a map containing ipv4 or ipv6 keys, which can hold a single bind IP or an array of bind IPs.

// Example structure for Cap'n Proto RPC publish command
{
  "ipv4": ["192.168.1.10", "192.168.1.11"],
  "ipv6": "2001:db8::1"
}