ProxyBroker2 Documentation

ProxyBroker2 is an asynchronous, open-source tool designed for finding and managing public proxies. It can find working proxies from over 50 sources (finding up to 7,000 working proxies), validate them against judge servers, and operate as a rotating proxy server that distributes incoming requests to external proxies.

Key Capabilities:

• Protocol Support: HTTP(S), SOCKS4/5, and the CONNECT method to ports 80 and 23 (SMTP).
• Filtering: Proxies can be filtered by type, anonymity level, response time, country, and DNSBL status.
• Validation: Checks for support of Cookies, Referer, and POST requests.
• Management: Automatically removes duplicate proxies and operates asynchronously.
• Rotation: Acts as a proxy server with automatic proxy rotation.

ProxyBroker2 is an asynchronous tool for proxy management with the following capabilities:

• Discovery: Finds proxies from ~50 sources (7000+ working proxies).
• Protocols: Supports HTTP(S), SOCKS4/5, and CONNECT method to ports 80 and 23 (SMTP).
• Filtering: Filter by type, anonymity level, response time, country, and DNSBL status.
• Rotation: Operates as a proxy server that automatically rotates through an external proxy pool.
• Validation: Checks for Cookie and Referer support (and POST requests if required).
• Deduplication: Automatically removes duplicate proxies.

When looking for custom provider configurations, the CLI searches in the following order (the first match wins):

1. Paths provided via the --provider-dir PATH CLI flag (this flag is repeatable).
2. The $PROXYBROKER_PROVIDER_DIR environment variable.
3. The /configs directory (standard for Docker users).

If none of these are configured, ProxyBroker2 defaults to using only its bundled provider list.

If you are upgrading from the original (and now abandoned) proxybroker v0.3.2, note the following changes:

Breaking Changes:

• Python Version: v0.3.2 supported Python 3.5.3+. ProxyBroker2 requires Python 3.10+.
• Installation: Do not use pip install proxybroker. Instead, uninstall the old version and install from GitHub:

  pip uninstall proxybroker
  pip install git+https://github.com/bluet/proxybroker2.git

• CLI Usage: The command prefix has changed. Use python -m proxybroker instead of the bare proxybroker command.

Compatible Changes:

• Python API: The API remains 100% compatible. Existing code using from proxybroker import Broker should work with minimal changes (ensure you use modern asyncio.run() patterns).

By default, custom configurations are added to the existing bundled providers. To use only your own custom providers and skip all bundled sources, you must use the Python API. There is currently no CLI flag to achieve this.

To implement a 'no-defaults' workflow, initialize the Broker with an empty providers list.

from proxybroker import Broker

# Only providers from /configs, no bundled defaults.
broker = Broker(providers=[], provider_dirs=['/configs'])

You can use the pre-built Docker image from Docker Hub.

Pull the image:

$ docker pull bluet/proxybroker2

Run with custom proxy sources (no coding required): Drop YAML/JSON configuration files into a folder and bind-mount it to /configs inside the container.

Example configuration (~/proxy-sources/my_list.yaml):

name: My Internal Source
type: simple
url: https://my-server.example.com/proxies.txt
format: text
protocols: [HTTP, HTTPS]

Run command:

docker run --rm \
  -v ~/proxy-sources:/configs \
  bluet/proxybroker2 \
  find --types HTTP --limit 10

You can install ProxyBroker2 directly from GitHub to get the latest features, or clone the repository for local development.

Install from GitHub (Recommended):

$ pip install git+https://github.com/bluet/proxybroker2.git

Local Installation:

$ git clone https://github.com/bluet/proxybroker2.git
$ cd proxybroker2
$ pip install -e .

To install the production-ready v2.0.0b1 release, use the following command. Note that the original proxybroker package on PyPI is outdated and should not be used; always install from the GitHub repository.

Install stable v2.0.0b1 release:

$ pip install -U git+https://github.com/bluet/proxybroker2.git@v2.0.0b1

Install latest development version:

$ pip install -U git+https://github.com/bluet/proxybroker2.git

# Install stable v2.0.0b1 release
$ pip install -U git+https://github.com/bluet/proxybroker2.git@v2.0.0b1

You can add custom proxy sources to the ProxyBroker2 Docker image without writing code by using YAML or JSON configuration files.

1. Create a configuration file (e.g., my_secret_list.yaml) with the desired provider details.
2. Bind-mount the directory containing your config files to /configs in the container.

ProxyBroker2 will automatically load all *.yaml, *.yml, and *.json files in that directory. Files starting with an underscore (_) are ignored.

# ~/proxy-sources/my_secret_list.yaml
name: My Internal Source
type: simple
url: https://my-private-server.example.com/proxies.txt
format: text          # auto, text, json, csv, html
protocols:
  - HTTP
  - HTTPS
max_connections: 4
timeout: 20

docker run --rm \
  -v ~/proxy-sources:/configs \
  bluet/proxybroker2 \
  find --types HTTP --limit 10

Ensure your environment meets the following requirements:

• Python: 3.10 or higher
• aiohttp: >= 3.8.0
• aiodns: >= 3.0.0
• maxminddb: >= 2.0.0

The Server exposes a control API via specific HTTP headers. You can manage the proxy pool by sending requests to the server with the following header configuration:

Remove a proxy:

• Header: Host: proxycontrol
• Path: /api/remove/{host}:{port}
• Result: Returns 204 No Content if successful.

Get proxy history:

• Header: Host: proxycontrol
• Path: /api/history/{query_type}:{url}
• Query Types: url (e.g., /api/history/url:https://example.com)
• Result: Returns 200 OK with a JSON body containing the previous proxy used for that specific client/URL combination, or 204 No Content if no history exists.

The proxybroker executable provides three primary subcommands for managing proxies:

• find: Finds and checks proxies based on specific parameters (protocols, anonymity, countries, etc.).
• grab: Finds proxies without performing a check.
• serve: Runs a local proxy server that distributes requests to external proxies found using specified parameters.

Note: The update-geo command is currently broken and will raise an error.

Run proxybroker <command> --help to see specific options for each command.

proxybroker find --help
proxybroker grab --help
proxybroker serve --help