mitmproxy2swagger

To reverse-engineer a REST API from mitmproxy traffic, follow these steps:

1. Capture Traffic: Use mitmweb or mitmproxy to capture the desired HTTP traffic. Ensure your client is configured to use the proxy.
2. Save Flow: Save the captured traffic to a flow file (e.g., via the "File" > "Save" menu in mitmweb).
3. First Pass: Run the tool to generate an initial schema with path templates.
4. Edit Schema: Open the generated schema file. You will see paths prefixed with ignore:. Remove the ignore: prefix from the paths you want to include in the final specification and adjust parameters as needed.
5. Second Pass: Run the tool again on the same schema file to generate the actual endpoint descriptions. Use the --examples or --headers flags to include extra data (use caution with sensitive information).

You can process HAR (HTTP Archive) files exported from browser DevTools. mitmproxy2swagger automatically detects HAR files.

1. Open browser DevTools and go to the Network tab.
2. Click the Export HAR button.
3. Run mitmproxy2swagger using the HAR file as the input -i argument.

You can install mitmproxy2swagger using pip, pip3, or by building the Docker image from the source repository.

$ pip install mitmproxy2swagger
# or
$ pip3 install mitmproxy2swagger
# or via Docker
$ git clone git@github.com:alufers/mitmproxy2swagger.git
$ cd mitmproxy2swagger
$ docker build -t mitmproxy2swagger .

When running the tool, it identifies new paths and adds them to the x-path-templates array in the output YAML file. Some suggested paths are prefixed with ignore: to prevent them from being automatically converted into full endpoints immediately.

To turn a suggested template into a real endpoint in your Swagger file, remove the ignore: prefix from the path string in the x-path-templates section of the YAML file.

The CLI tool accepts the following arguments to process traffic captures:

• -i, --input: Path to the mitmproxy flow file or HAR file.
• -o, --output: Path to the output OpenAPI schema file. If the file exists, it will be extended/merged.
• -p, --prefix: The base URL of the API you wish to reverse-engineer (e.g., https://api.example.com/v1).
• --examples: (Optional) Adds example data to requests and responses. Warning: May include sensitive data.
• --headers: (Optional) Adds header data to requests and responses. Warning: May include sensitive data.

Recursively scans an example object (list or dict) and truncates it to prevent excessively large Swagger documentation. It enforces the following limits:

• MAX_EXAMPLE_ARRAY_ELEMENTS: 10
• MAX_EXAMPLE_OBJECT_PROPERTIES: 150

The HarCaptureReader class is used to parse and iterate through requests captured in a HAR file.

Initialization: HarCaptureReader(file_path: str, progress_callback=None)

• file_path: Path to the .har file.
• progress_callback: An optional function called with a float (0.0 to 1.0) representing the current progress through the file.

Methods:

• captured_requests() -> Iterator[HarFlowWrapper]: Yields HarFlowWrapper objects for each entry in the HAR log.

The mitmproxy_dump_file_huristic function provides a heuristic score to determine if a file is a valid mitmproxy dump file. It checks for keywords in the filename and inspects the file content for specific patterns (like status_code or regular) and non-printable characters.

A higher score indicates a higher probability that the file is a mitmproxy flow file.

These functions map raw header dictionaries to OpenAPI-compliant structures.

• request_to_headers(headers, add_example=False): Converts a dictionary of headers into a list of parameter objects with in: "header". If add_example is True, it includes the first value of the header as an example field.
• response_to_headers(headers): Converts a dictionary of headers into an object where each key maps to a description and a schema (detecting number or string).

Use the har_archive_heuristic(file_path: str) -> int function to determine if a file is a valid HAR (HTTP Archive) file. It returns an integer score based on several checks:

• File extension .har (+25)
• File content is printable ASCII (+25)
• File starts with { (+23)
• Presence of Chrome or Firefox export markers (+15)
• Presence of the "entries" key (+15)
• Presence of the "version" key (+15)

The MitmproxyCaptureReader class is used to read and stream http.HTTPFlow objects from a mitmproxy log file. You can provide an optional progress_callback function that receives a float (0.0 to 1.0) representing the current progress of the file reading.

Use the captured_requests() method to get an iterator of MitmproxyFlowWrapper objects. Note that flows without a response are skipped and a warning is printed to stdout.