mitmproxy-reference

name	mitmproxy-reference
description	mitmproxy and mitmdump documentation — proxy modes, addon development, event hooks, traffic capture, filtering, and replay. Use when working with HTTP interception, building capture addons, or debugging proxy behavior.
allowed-tools	Read, Grep, Glob

mitmproxy Onboarding Guide: Research & Best Practices

This document serves as the primary onboarding guide for any developer joining the LLMitM v2 project. It provides a high-level map of the mitmproxy documentation, followed by a curated set of deep-dive research reports that are essential for understanding our architecture, design patterns, and implementation choices. Each report is summarized to explain its relevance and provide context for why it is required reading.

---

mitmproxy Documentation Map

This section provides a comprehensive, hierarchically structured map of the mitmproxy documentation, based on the cloned source files. It is designed to serve as a top-level exploration guide for understanding how mitmproxy powers LLMitM v2's traffic interception layer.

• Concepts (Entry Point)

  • Core Architecture
    • How mitmproxy Works: MITM Mechanism
    • Proxy Modes: Regular, Reverse, Transparent, WireGuard, Local
    • Protocol Support: HTTP/1, HTTP/2, HTTP/3, WebSocket, TCP, UDP
    • Certificates: CA Generation, Pinning, mTLS
  • Configuration & Filtering
    • Options System: YAML Config, Runtime Control
    • Filter Expressions: URL, Header, Method, Status Code Matching
    • Commands: Interactive Console, Key Bindings

• Addon Development (Entry Point)

  • Building Addons
    • Addon Architecture: Event Hooks, Options, Commands
    • Event Hooks: request, response, Connection Lifecycle
    • Custom Options: Typed Config, Validation
    • Custom Commands: @command.command, Flow Arguments
  • Reference
    • Content Views: Pretty-Printing, Syntax Highlighting
    • API Changelog: Breaking Changes Across Versions

• Overview (Entry Point)

  • Built-in Features: Replay, Anticache, Map Local/Remote, Streaming
  • Installation: macOS, Linux, Windows, Docker, PyPI
  • Getting Started: First Launch, Browser Config

• How-To Guides (Entry Point)

  • Transparent Proxying: iptables, pf, Network-Layer Setup
  • Ignoring Domains: ignore_hosts, Certificate Pinning Workarounds

• Tutorials (Entry Point)

  • Client Replay: Capture Once, Replay Forever

• API Reference (Entry Point)

  • Core Traffic: mitmproxy.http (Headers, Request, Response, HTTPFlow), mitmproxy.flow (Flow base, serialization), mitmproxy.connection (Client, Server, TLS metadata)
  • Data Structures: mitmproxy.coretypes.multidict (MultiDict used by headers, cookies, query, forms)
  • TLS/Certs: mitmproxy.tls (ClientHello, TlsData), mitmproxy.certs (Cert parsing, CA generation)
  • Protocols: dns, tcp, udp, websocket
  • Proxy: mode_specs, context, server_hooks
  • Addon System: addonmanager, contentviews

---

Curated Research Reports

1. Core Architecture

How mitmproxy Works

Core MITM mechanism: explicit HTTP/HTTPS proxying, transparent proxying, SNI handling, upstream certificate sniffing. Foundational for troubleshooting fingerprinting and traffic capture phases.

Proxy Modes

All proxy modes: regular, transparent, reverse, WireGuard, local capture. LLMitM v2 uses regular proxy (explicit) and reverse proxy (in front of target). See API Research — Proxy Modes for the full mode reference table.

Protocol Support

HTTP/1, HTTP/2, HTTP/3, WebSocket, DNS, TCP/TLS, UDP/DTLS. LLMitM v2 focuses on HTTP/HTTPS. WebSocket hooks exist for future extensions.

Certificates

CA certificate system for HTTPS interception. Certificate pinning bypass via ignore_hosts or Android unpinning tools.

2. Python API (Critical for LLMitM v2)

Full API reference: api/_index.md — clean markdown docs for all 16 modules Architecture insights: api_research.md — FlowReader patterns, bounded tool design, codebase integration notes

FlowReader — The Key Insight

The .mitm file IS the structured format. FlowReader (mitmproxy.io) is a deserializer that yields fully hydrated Python objects — no subprocess, no text parsing, no truncation:

from mitmproxy.io import FlowReader
with open("capture.mitm", "rb") as f:
    for flow in FlowReader(f).stream():
        flow.request.method       # "POST"
        flow.request.pretty_url   # "http://localhost:3000/rest/user/login"
        flow.request.json()       # {"email": "[e-mail]", "password": "admin123"}
        flow.request.cookies      # MultiDict of cookies
        flow.response.status_code # 200
        flow.response.json()      # {"authentication": {"token": "eyJ..."}}
        flow.response.cookies     # Set-Cookie values parsed
        flow.response.headers     # case-insensitive multidict

The CLI command mitmdump -nr capture.mitm --flow-detail 3 is literally FlowReader -> format as text -> print to stdout. Shelling out to mitmdump gives a lossy text representation of data that's already structured.

HTTPFlow Object — What's Available

Every flow captured by mitmproxy gives you (full signatures in mitmproxy.http):

Category	Attributes
Request basics	method, url, pretty_url, scheme, host, port, path, http_version
Request data	headers (Headers — case-insensitive MultiDict), content (decompressed bytes), text, json(), cookies, query, urlencoded_form, multipart_form
Response basics	status_code, reason, http_version
Response data	headers, content, text, json(), cookies
Connection/TLS	Client/Server: sni, tls_version, alpn, cipher, certificate_list (Cert objects with subject, issuer, SANs)
Flow lifecycle	Flow base: id (UUID), timestamp_created, is_replay, error, metadata (arbitrary dict), get_state()/set_state(), copy(), kill()

Programmatic Flow Filtering

from mitmproxy import flowfilter
flt = flowfilter.parse("~d example.com & ~m POST")
if flowfilter.match(flt, flow):
    # Flow matches

Same filter syntax as CLI (~u, ~m, ~c, ~h, ~b, ~t, ~d, &, |, !) but compiled and evaluated in Python. See api_research.md §7 for the full filter table.

Useful Utilities

Utility	What It Does
flow.response.refresh()	Update date/expires/cookie timestamps for replay freshness
Response.make(status_code, content, headers)	Factory for mock responses (mitmproxy.http)
flow.get_state() / set_state()	Serialize flow to/from dict (mitmproxy.flow)
flow.copy()	Deep copy with live=False
FlowWriter(fo).add(flow)	Write flows to .mitm binary format
FilteredFlowWriter(fo, flt)	Write only matching flows
read_flows_from_paths(paths)	Bulk read from multiple files

3. Addon Development

Addon Architecture Overview

Class-based addons respond to event hooks, define options, expose commands. For LLMitM v2, addons are the natural way to implement live traffic capture and real-time fingerprinting without subprocess-based mitmdump invocations.

Event Hooks — What Fires When

Hook	When	Use Case
request(flow)	Full request received	Capture for fingerprinting
response(flow)	Full response received	Tech stack detection, token extraction
requestheaders(flow)	Headers only, before body	Set flow.request.stream = True for large files
responseheaders(flow)	Headers only, before body	Streaming decisions
tls_clienthello(data)	TLS ClientHello	SNI, cipher suite analysis
websocket_message(flow)	WebSocket message	Future: non-HTTP protocol testing

See API Research — Event Hooks for the complete hook list.

Custom Options & Custom Commands

Addons can define typed options (str, int, bool, sequences) and expose commands that accept flows, paths, and other typed arguments. Relevant for future: "compile ActionGraph from current flows" or "execute stored graph for domain X".

4. Configuration & Filtering

Options System

Global options via ~/.mitmproxy/config.yaml and --set. Key options: ignore_hosts, tcp_hosts, mode, anticache, stickycookie, stickyauth.

Filter Expressions

Flow matching language: ~u /api & ~m POST & ~c 200. Works both in CLI and programmatically via flowfilter.parse().

Built-in Features Worth Knowing

Feature	Flag	Why It Matters
Anticache	--anticache	Forces full responses during fingerprinting
Sticky cookies	--stickycookie "~d target"	Auto-replay session cookies (our ExecutionContext.cookies does this manually)
Sticky auth	--stickyauth "~d target"	Auto-replay auth headers
Client replay	-C replay.mitm	Replay captured requests against live server
Streaming	--stream_large_bodies=10m	Forward large bodies without buffering

5. Operations & Deployment

Transparent Proxying

Network-layer setup via iptables (Linux), pf (macOS). Captures traffic from proxy-oblivious applications.

Ignoring Domains

ignore_hosts option exempts traffic from interception. Filter out CDNs, analytics, etc.

6. API Compatibility

API Changelog

Key breaking changes: mitmproxy 9+ uses Python logging (not custom); mitmproxy 7+ revised connection events (.client_conn -> .peername).

7. Tutorials

Client Replay Tutorial

Capture and replay HTTP login sequences: mitmdump -w (record) -> mitmdump -C (replay). Validates the core LLMitM v2 thesis: capture once, replay deterministically forever.