Capturing Traffic

This page explains how Ghost actually captures and records the network traffic flowing between your apps and the internet. Whether you’re a QA engineer testing a mobile app or a security researcher analyzing API calls, understanding the capture process helps you get the most out of Ghost.

How Traffic Capture Works

Ghost runs a proxy server on your computer at localhost:4545 (meaning it only listens for traffic from your own machine — it’s not accessible from the network). When your browser, desktop app, or mobile device is configured to send traffic through this proxy, Ghost sits in the middle and records everything.

The process differs slightly depending on whether the traffic is encrypted (HTTPS) or not (HTTP):

• HTTP traffic (unencrypted) — Ghost acts as a standard forwarding proxy. It receives the request from your browser, reads its contents (URL, headers, body), records it, and forwards it to the destination server. The response follows the reverse path. No special certificates needed.

• HTTPS traffic (encrypted) — This is more complex because the whole point of HTTPS is to prevent anyone in the middle from reading the traffic. Ghost uses a technique called MITM (Man-in-the-Middle) interception to decrypt, record, and re-encrypt the traffic. Here’s exactly what happens:

What this diagram shows — the full lifecycle of an HTTPS request through Ghost:

1. CONNECT tunnel request: Your browser wants to reach api.example.com over HTTPS. Since it’s configured to use Ghost as its proxy, it first sends a CONNECT request asking Ghost to set up a tunnel to that domain on port 443 (the standard HTTPS port).

2. Tunnel established: Ghost responds with “200 Connection Established,” telling the browser the tunnel is ready. At this point, the browser thinks it’s talking directly to the real server.

3. TLS handshake with the browser: The browser starts a TLS (encryption) handshake. Normally, the real server would present its certificate here. Instead, Ghost dynamically generates a certificate for api.example.com signed by Ghost’s CA certificate (the one you installed during first-run setup). Because you trusted Ghost’s CA, your browser accepts this certificate without warnings.

4. TLS handshake with the real server: Simultaneously, Ghost establishes its own separate encrypted connection to the real api.example.com server. Ghost uses uTLS with a Chrome browser fingerprint (explained below) so the server can’t tell it’s talking to a proxy instead of a real browser.

5. Request forwarded: The browser sends its actual HTTP request (encrypted with Ghost’s certificate). Ghost decrypts it, records the full request (method, URL, headers, body), runs it through the interceptor pipeline (breakpoints, map rules, addons, security scanning), and then re-encrypts and forwards it to the real server.

6. Response returned: The real server responds. Ghost decrypts the response, records it (status code, headers, body), runs it through the pipeline, and re-encrypts it back to the browser using Ghost’s certificate.

The entire process adds negligible latency — typically imperceptible to the user.

uTLS Chrome Fingerprinting

When Ghost connects to upstream servers, it uses a library called uTLS (from the Refraction Networking project) to make its TLS handshake look exactly like Google Chrome’s.

Why does this matter? Some servers and CDNs (Content Delivery Networks) inspect the TLS handshake — the specific cipher suites, extensions, and ordering — to fingerprint the client. If the fingerprint doesn’t match a known browser, they might block the connection or return different content. By mimicking Chrome’s exact TLS fingerprint, Ghost avoids being detected as a proxy by these servers.

This is completely transparent to you — it happens automatically for every HTTPS connection.

Certificate Generation

Every unique hostname your browser visits gets its own dynamically generated TLS certificate. Here’s how the certificate system works:

• Algorithm: ECDSA P-256 — the same modern cryptographic algorithm used by real websites. This is fast (microseconds per certificate) and widely supported.
• Cache: Ghost keeps up to 10,000 certificates in an LRU (Least Recently Used) memory cache. If you visit the same site again, Ghost reuses the cached certificate instead of generating a new one. When the cache is full, the oldest unused certificates are evicted.
• Validity: Each generated certificate is valid for 24 hours. After that, Ghost generates a fresh one. This keeps the security surface small.
• Deduplication: If your browser opens 50 simultaneous connections to api.example.com (which happens with modern web apps), Ghost uses a “single-flight” pattern — only the first request triggers certificate generation. All other requests wait for and share the same certificate. This prevents wasting CPU on duplicate work.

Connection Methods

There are several ways to route traffic through Ghost, depending on what you’re testing:

System Proxy (Recommended for Desktop)

The simplest option. Ghost configures your operating system’s network settings to route all HTTP and HTTPS traffic through Ghost’s proxy. Every browser and most desktop applications automatically use the system proxy settings.

How to enable/disable:

• Click the proxy toggle button in the status bar (bottom of the Ghost window)
• Right-click the Ghost icon in the system tray → “Proxy”
• Use the keyboard shortcut (configurable in Settings)

What Ghost configures on macOS: Ghost uses networksetup to set the HTTP proxy (-setwebproxy), HTTPS proxy (-setsecurewebproxy), and a PAC (Proxy Auto-Configuration) URL (-setautoproxyurl) on all active non-tunnel network services (Wi-Fi, Ethernet, etc.). VPN and security agent services (GlobalProtect, Zscaler, AnyConnect, WireGuard, etc.) are automatically skipped because they manage their own proxy settings and would fight with Ghost. Ghost also disables WPAD (Web Proxy Auto-Discovery) to prevent corporate auto-proxy from overriding Ghost’s settings. All of this is reversed when you disable the system proxy — including re-enabling WPAD.

If a corporate VPN is overriding Ghost’s proxy at the macOS SCDynamicStore level (which takes precedence over networksetup settings), Ghost detects this and can override the VPN’s State key with admin privileges — showing the native macOS password dialog. This is automatically restored on shutdown.

What Ghost configures on Windows: Ghost writes to the current user’s Windows Registry at HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings. It sets ProxyEnable=1, ProxyServer to Ghost’s address, and ProxyOverride to bypass localhost. After writing, Ghost calls the WinINet InternetSetOptionW API to notify running browsers (Chrome, Edge, etc.) of the change — so they pick up the new settings without a restart. Disabling sets ProxyEnable=0 (the address is left in place but ignored by Windows).

Manual Proxy

If you only want to capture traffic from a specific application (leaving everything else unaffected), you can manually configure that application’s proxy settings:

• HTTP Proxy: 127.0.0.1:4545
• HTTPS Proxy: 127.0.0.1:4545

For example, in Firefox: Settings → Network Settings → Manual proxy configuration → enter the address above for both HTTP and HTTPS.

This is also useful if your system proxy doesn’t work with certain apps, or if you want to test with the system proxy disabled.

Browser Extension

The Ghost browser extension (available for Chrome and Firefox) adds capabilities on top of the proxy connection. Install the extension and it automatically connects to Ghost via WebSocket.

Note

The extension does NOT replace the proxy. Traffic still flows through the proxy for interception. The extension adds additional capabilities that the proxy alone can’t provide:

• DOM observation — captures user interactions (clicks, form inputs, page navigations) in the browser and correlates them with the network traffic those interactions triggered
• Browser actions — Ghost or the AI agent can remotely click elements, fill forms, take screenshots, and interact with the page
• JavaScript injection — inject custom JavaScript into web pages for testing

Mobile Devices

For testing mobile apps, you configure the mobile device (iOS Simulator, Android emulator, or physical phone) to route its traffic through Ghost. This is covered in detail in the Mobile Device Setup guide.

Filtering Traffic

Out of the box, Ghost captures everything — which can be overwhelming. A single web page might generate 50-100 requests (API calls, images, scripts, fonts, analytics). Ghost provides several filtering mechanisms to help you focus on what matters:

Host Filters (Capture-Level)

These filters control what Ghost captures at the proxy level — filtered traffic is never recorded:

Configure in Settings → Proxy:

• Include hosts (allowlist) — Only capture traffic to these hostnames. Everything else passes through the proxy unrecorded. Example: set *.hepsiburada.com to only capture Hepsiburada API traffic while letting everything else through silently. Supports wildcards: *.example.com matches any subdomain.
• Exclude hosts (blocklist) — Capture everything EXCEPT traffic to these hostnames. Ghost ships with a comprehensive default exclude list of 140+ entries covering: Windows telemetry (*.events.data.microsoft.com, etc.), macOS noise (*.apple.com, *.cdn-apple.com), browser telemetry (*.safebrowsing.googleapis.com), certificate/OCSP checks (ocsp.pki.goog, ocsp.digicert.com), social media (*.facebook.com, *.twitter.com, etc.), streaming services (*.youtube.com, *.netflix.com, *.spotify.com), analytics and ads (*.google-analytics.com, *.hotjar.com, *.sentry.io), CDN infrastructure (*.cloudfront.net, *.akamaized.net), and Microsoft/Google productivity services. You can clear or customize this list in Settings → Proxy or in ~/.ghost/config.toml under [proxy] host_exclude.

SSL Bypass

Some servers use a security feature called certificate pinning — they only accept connections using their specific certificate, not Ghost’s generated certificate. When this happens, the connection fails entirely (you’ll see errors in the app).

The solution: add those hostnames to the SSL Bypass list in Settings. Ghost will tunnel traffic to those hosts through without decrypting it. You won’t be able to see the content of those requests, but the connections won’t break. Common candidates include Apple services, Google Play services, and certain banking APIs.

Content Type Filtering (UI-Level)

In the Ghost interface, a content type filter bar above the flow list lets you show or hide different types of traffic:

• API — JSON, XML, and GraphQL requests (usually what you care about most)
• Pages — HTML documents
• JS / CSS — JavaScript and stylesheet files
• Images / Fonts / Media — Binary assets like PNG, WOFF2, MP4, etc.

These filters don’t affect what’s captured — they only control what’s visible in the flow list. Filtered-out flows are still recorded and searchable.

Noise Detection (Automatic)

Modern apps generate a lot of repetitive background traffic: polling endpoints (checking for updates every 5 seconds), heartbeat pings, health checks, analytics beacons. This “noise” drowns out the interesting traffic you’re trying to inspect.

Ghost has built-in noise detection that automatically identifies these repetitive patterns and suppresses them. Here’s exactly how the algorithm works:

1. Fingerprinting — Every request is fingerprinted as METHOD:host:path (query parameters are stripped, because polling endpoints hit the same path with different timestamps/tokens). For example, GET:api.example.com:/health is one fingerprint.
2. Counting — Ghost tracks how many requests match each fingerprint within a sliding window (default: 30 seconds). All requests below the threshold are captured normally — not just the first one.
3. Threshold reached — When 20 requests (the default threshold) to the same fingerprint occur within the 30-second window, Ghost classifies that endpoint as “noisy” and switches to sampling mode.
4. Sampling — Once noisy, Ghost keeps 1 out of every 10 requests (the sample rate) and silently suppresses the rest. Suppressed flows are never stored in the database or shown in the UI — but the client still receives the response normally. Each kept sample is tagged with _noise_suppressed:N showing how many flows were suppressed since the last sample.
5. Hysteresis exit — When traffic drops below half the threshold (10 requests in the window), the endpoint exits noisy state and returns to normal capture. A final sample is emitted with the remaining suppressed count.

Important: Flows that have been tagged by addons (business logic applied) are never suppressed, regardless of noise classification.

The result: instead of seeing 500 identical health-check requests flooding your flow list, you see a few representative samples with counts indicating the rest were suppressed.

Configuration — You can tune these values in Settings or directly in ~/.ghost/config.toml:

[proxy]
noise_enabled = true       # Master switch (default: true)
noise_threshold = 20       # Requests in window to trigger (default: 20)
noise_window_secs = 30     # Sliding window size in seconds (default: 30)
noise_sample_rate = 10     # Keep 1 in N while noisy (default: 10)

Real-Time Updates

One of Ghost’s key features is that captured traffic appears in the UI instantly — there’s no refresh button, no polling delay. Here’s how that works:

1. The proxy captures a flow (request + response)
2. The storage interceptor saves it to the SQLite database
3. A flow.created event is broadcast through the WebSocket hub
4. The frontend receives the event and adds the new flow to the flow list
5. The flow appears in the UI with a brief highlight animation so you can see what just arrived

The entire path from “server responds” to “you see it in the list” takes milliseconds. This real-time feedback is essential for QA testing — you can tap a button in your mobile app and immediately see the resulting API call in Ghost.

If the WebSocket connection drops for any reason (rare, but possible), the frontend automatically falls back to polling the API every 3 seconds until the WebSocket reconnects.

Session Management

All captured traffic is organized into sessions — named containers that keep your flows organized. Think of sessions like folders for your captured traffic.

• Default session — Created automatically on first launch. All captured traffic goes here until you create additional sessions.
• Creating sessions — Click the ”+” button in the session sidebar, or use the API. Give each session a descriptive name like “Login Flow Testing,” “Payment Regression v2.4,” or “Security Audit 2024-03.”
• Switching sessions — Click a session name to make it active. Only flows from the active session appear in the flow list. New captured traffic goes to whichever session is active.
• Exporting sessions — Right-click a session to export it as HAR (HTTP Archive — the industry standard for recorded traffic), JSON, CSV, or Postman Collection format. Useful for sharing captured traffic with teammates or importing into other tools.
• Deleting sessions — Right-click → Delete. This permanently removes the session and all its flows from the database, freeing disk space.

The active session ID is shared across all Ghost subsystems: the proxy tags new flows with the session ID, the AI agent searches within the active session, the browser extension correlates interactions with the active session’s flows, and device attribution links mobile traffic to the correct session.