DOCSEARCHbyAlgolia
Submit Docs

Smokescreen

repository·master·Indexed 23 days ago

https://github.com/stripe/smokescreen

An HTTP CONNECT proxy used to secure and centralize egress traffic. Smokescreen provides security layers including hostname allow-listing (ACLs), IP routing validation to prevent internal network scanning, and mTLS-based client authentication. It allows for centralized egress IP addresses and includes features for rate limiting, concurrency control, and Prometheus/statsd observability.

Tokens
7.2K
Snippets
12
Records
26
Agent score
79%

What's inside smokescreen

  1. What is Smokescreen?

    master

    Smokescreen is an HTTP CONNECT proxy designed to manage and secure egress traffic from Stripe to the external world (e.g., webhooks).

    It provides several security layers:

    • Hostname ACLs: Restricts connections to a pre-configured allow-list of hostnames to prevent requests to unexpected services.
    • IP Filtering: Resolves domain names to ensure they are publicly routable IP addresses, preventing internal network scanning attacks. It can also be configured to allow or deny specific IP addresses or ranges.
    • Client Identification & Authentication: Typically uses mTLS to authenticate clients against configurable CAs and CRLs. It extracts client identity and validates the requested CONNECT destination against per-client ACLs.
    • Centralized Egress: Provides stable egress IP addresses for financial partners, abstracting the underlying service details.

  2. How Hostname ACLs work

    master

    Hostname Access Control Lists (ACLs) are defined in YAML files and control which remote hosts a specific role can access.

    Supported Policies

    • Open: Allows all traffic for this service.
    • Report: Allows all traffic and warns if the client accesses a remote host not in the list.
    • Enforce: Only allows traffic to remote hosts in the list. Warns and denies if the host is not listed.

    Global Lists

    • global_allow_list: Overrides the policy to allow specific domains for all roles.
    • global_deny_list: Overrides the policy to deny specific domains for all roles. If a domain is in both, global_deny_list takes priority.

    Critical Limitations

    • Hostname Only: The ACL is applied to hostnames as they appear in the request. It does not account for IP addresses after DNS resolution. To block by IP, use IP Filtering configuration.
    • Punycode Required: Hostnames without globbing prefixes must be in Punycode to prevent ambiguity.
    • Denylists vs Allowlists: Because global_deny_list only blocks hostnames, a client could bypass it by using the destination IP directly. It is recommended to use allowlists instead of denylists.

  3. Run Smokescreen as an HTTP CONNECT Proxy over TLS

    master

    To use Smokescreen with mTLS, you must generate server and client certificates.

    1. Generate Certificates: Use openssl to create a Server CA, a Client CA, and then issue certificates for the server and the client (where the client CN is typically localhost).
    2. Configure TLS: In config.yaml, provide the tls.cert_file, tls.key_file, and tls.client_ca_files.
    3. Configure ACL: In acl.yaml, define services using their name (matching the client certificate CN) to apply specific rules.
    4. Client Usage: When using curl, you must provide the proxy CA certificate (--proxy-cacert), the client certificate (--proxy-cert), and the client key (--proxy-key).

  4. Override Client Identification in Go

    master

    To customize how Smokescreen identifies clients (e.g., extracting a role from a TLS certificate), you must create a custom Go project that imports Smokescreen and uses smokescreen.StartWithConfig.

    Steps:

    1. Create a new Go project.
    2. Import Smokescreen.
    3. Use cmd.NewConfiguration to create a configuration.
    4. Replace conf.RoleFromRequest with a custom function of type func(request *http.Request) (string, error).
    5. Call smokescreen.StartWithConfig(conf, nil).
    6. Build and run your executable.
    package main

import (...)

func main() {
	// Here is an opportunity to pass your logger
	conf, err := cmd.NewConfiguration(nil, nil)
	if err != nil {
		log.Fatal(err)
	}
	if conf == nil {
		os.Exit(1)
	}

	conf.RoleFromRequest = func(request *http.Request) (string, error) {
		fail := func(err error) (string, error) { return "", err }

		subject := request.TLS.PeerCertificates[0].Subject
		if len(subject.OrganizationalUnit) == 0 {
			fail(fmt.Errorf("warn: Provided cert has no 'OrganizationalUnit'. Can't extract service role."))
		}
		return strings.SplitN(subject.OrganizationalUnit[0], ".", 2)[0], nil
	}

	smokescreen.StartWithConfig(conf, nil)
}

  5. Run Smokescreen as an HTTP CONNECT Proxy

    master

    For HTTPS traffic, run Smokescreen as an HTTP CONNECT proxy. Ensure your acl.yaml includes the required domains in the allowed_domains list for the relevant service or default policy.

    When using curl, use the --proxytunnel flag to enable the CONNECT method, or set the HTTPS_PROXY environment variable.

  6. Run Smokescreen as a MITM (Man-in-the-middle) Proxy

    master

    Smokescreen can perform MITM inspection on specific domains.

    1. Configure MITM CA: In config.yaml, specify mitm_ca_cert_file and mitm_ca_key_file (e.g., using certificates from the goproxy library).
    2. Configure MITM Domains: In acl.yaml, add domains to the mitm_domains list within a service or default block. You can use add_headers to inject headers into the intercepted request and detailed_http_logs to enable logging.
    3. Client Usage: When using curl, you must trust the MITM CA using the --cacert flag.
    # config.yaml
mitm_ca_cert_file: "vendor/github.com/stripe/goproxy/ca.pem"
mitm_ca_key_file: "vendor/github.com/stripe/goproxy/key.pem"

# acl.yaml
default:
  name: default
  project: security
  action: enforce
  allowed_domains:
    - wttr.in
  mitm_domains:
  - domain: wttr.in
    add_headers:
      Accept-Language: el
    detailed_http_logs: true
    detailed_http_logs_full_headers:
      - User-Agent
    # Curl with MITM
curl --proxytunnel -x localhost:4750 --cacert vendor/github.com/stripe/goproxy/ca.pem https://wttr.in

  7. Run Smokescreen as a MITM Proxy over TLS

    master

    This combines mTLS for the connection to Smokescreen with MITM inspection for the intercepted traffic.

    1. Setup TLS: Configure tls settings in config.yaml as described in the 'HTTP CONNECT Proxy over TLS' guide.
    2. Setup MITM: Configure mitm_ca_cert_file and mitm_ca_key_file in config.yaml, and define mitm_domains in acl.yaml.
    3. Client Usage: curl requires both the proxy's TLS certificates (for the connection to Smokescreen) and the MITM CA certificate (to trust the intercepted traffic).

  8. Manage dependencies in Smokescreen

    master

    Smokescreen uses Go modules for dependency management. You can use the following commands to manage your environment:

    • Adding a dependency: Run go build, go test, or go mod tidy to automatically fetch the latest versions of new dependencies. Use go mod vendor to vendor them.
    • Updating a dependency: Use go get dep@v1.1.1 or go get dep@commit-hash to pull specific versions, then run go mod vendor to update the vendored dependencies.
    go build
go test
go mod tidy
go mod vendor
go get dep@v1.1.1

  9. Monitor Smokescreen metrics via StatsD

    master

    Smokescreen can emit metrics to a StatsD collector. In config.yaml, specify the statsd_address (e.g., 127.0.0.1:8200). You can verify metrics are being emitted by listening on that port using a tool like nc (netcat).

    # config.yaml
statsd_address: 127.0.0.1:8200
    # Listen for metrics
nc -uklv 127.0.0.1 8200

  10. Run Smokescreen as an HTTP Proxy

    master

    To run Smokescreen as a standard HTTP proxy, configure config.yaml and acl.yaml. In config.yaml, you can set allow_missing_role: true to skip mTLS client validation. In acl.yaml, define your default service with allowed_domains to permit specific egress traffic.

    To start the proxy, use go run . with the --config-file and --egress-acl-file flags. You can then use curl with the -x flag or set the ALL_PROXY environment variable to route traffic through the proxy.

    # Configuration examples
# config.yaml
allow_missing_role: true

# acl.yaml
version: v1
services: []
default:
  name: default
  project: security
  action: enforce
  allowed_domains: 
    - example.com

# Run Smokescreen
go run . --config-file config.yaml --egress-acl-file acl.yaml

# Use with Curl
curl -x localhost:4750 http://example.com
ALL_PROXY=localhost:4750 curl -v http://example.com

  11. Configure IP Filtering

    master

    To control the routing of requests to specific IP addresses or IP blocks, use the following configuration options:

    • deny-address: Add IP[:PORT] to a list of blocked IPs.
    • allow-address: Add IP[:PORT] to a list of allowed IPs.
    • deny-range: Add a range in CIDR notation to a list of blocked IP ranges.
    • allow-range: Add a range in CIDR notation to a list of allowed IP ranges.

  12. Configure Rate Limiting

    master

    Smokescreen protects against overload using rate and concurrency limiting.

    OptionDescription
    max-concurrent-requestsLimits simultaneous in-flight requests. Excess requests receive 503 Service Unavailable.
    max-request-rateLimits requests per second using a token bucket algorithm. Excess requests receive 429 Too Many Requests.
    max-request-burstSets token bucket capacity. Must be greater than max-request-rate. Defaults to 2x the rate if omitted.
    max-concurrent-connect-tunnelsLimits the number of active CONNECT tunnels (long-lived connections). Excess requests receive 429 Too Many Requests.

    Important Distinction:

    • max-concurrent-requests limits requests being *processed`.
    • max-concurrent-connect-tunnels limits active tunnel connections. For CONNECT requests, use both to prevent resource exhaustion.
    # CLI Example
smokescreen --max-concurrent-requests=100 --max-request-rate=50 --max-concurrent-connect-tunnels=50
    # YAML Example
max_concurrent_requests: 100
max_request_rate: 50
max_request_burst: 150  # optional, defaults to 2x rate
max_concurrent_connect_tunnels: 50  # limits active CONNECT tunnel connections