Noise Detection

Modern web and mobile applications generate a lot of repetitive background traffic that has nothing to do with what you’re testing. Health check endpoints that ping every 5 seconds, analytics beacons firing on every page view, WebSocket heartbeats, CDN chunk downloads, auto-save timers, polling APIs checking for notifications — this “noise” can overwhelm the useful traffic you’re trying to inspect.

Ghost solves this with automatic noise detection. It identifies repetitive traffic patterns in real-time and suppresses them, keeping your flow list focused on the requests that actually matter. You see 50 meaningful API calls instead of 50 meaningful calls buried under 2,000 identical health checks.

How It Works (The Big Picture)

Ghost watches every request that passes through the proxy and identifies patterns based on a “fingerprint” — a combination of the HTTP method, hostname, and URL path. When the same fingerprint appears too many times within a short time window, Ghost marks that pattern as “noisy” and starts suppressing it.

Here’s the key: suppression only affects what you see in Ghost’s UI. Your applications still get their responses normally — Ghost never blocks or delays the actual traffic. It only skips recording certain repetitive flows to the database and broadcasting them to the frontend.

What this diagram shows — the decision tree for every incoming flow:

1. Fingerprint generation — When a new flow arrives, Ghost creates a fingerprint from its HTTP method, hostname, and URL path. For example, GET:api.example.com:/healthcheck. This means two GET requests to the same path are considered the same pattern, regardless of different query parameters, headers, or timestamps.

2. Sliding window count — Ghost maintains a counter for each fingerprint within a 30-second sliding window. Every time a flow with this fingerprint arrives, the counter increments. Old counts outside the window decay automatically.

3. Threshold check — If the counter reaches the threshold (default: 20 requests within 30 seconds), the pattern is classified as “noisy.” That’s roughly one request every 1.5 seconds or faster — clearly automated/repetitive, not human-driven.

4. Sampling decision — Once a pattern is noisy, most occurrences are suppressed. But Ghost keeps 1 in every 10 as a representative sample, so you still see some evidence of the pattern and can inspect what the request/response looks like.

5. Verdicts — Each flow gets one of three verdicts:

   • Keep — Normal traffic. Stored in the database, pushed to the UI, counted in stats.
   • Sample — Part of a noisy pattern, but kept as a representative. Stored and shown, tagged with _noise_suppressed:N where N is the number of suppressed siblings.
   • Suppress — Noisy and not sampled. The client still receives the response (Ghost never interferes with traffic), but the flow is NOT stored in the database and NOT broadcast to the frontend. It’s as if Ghost never saw it.

Configuration

You can adjust noise detection behavior in Settings → Proxy → Noise Detection or directly in ~/.ghost/config.toml:

Setting	TOML field	Default Value	What It Controls
Enabled	noise_enabled	true	Master switch. When false, Evaluate() always returns Keep — no suppression occurs.
Window	noise_window_secs	30 seconds	The time period for counting repetitions. A longer window catches slower patterns (like one request every 3 seconds) but takes longer to classify them as noisy.
Threshold	noise_threshold	20 requests	How many requests with the same fingerprint must occur within the window before the pattern is classified as noisy. Lower values are more aggressive (suppress sooner), higher values are more lenient.
Exit threshold	(computed: threshold/2)	10	When a noisy pattern drops below this count within the window, it’s reclassified as normal. Not directly configurable — always half of the threshold.
Sample rate	noise_sample_rate	1 in 10	How many noisy flows to keep as representative samples. 1 in 10 means 10% of noisy flows are kept. Set to 1 to keep every noisy flow (effectively disabling suppression while still detecting patterns).
Cleanup interval	(hardcoded)	5 minutes	How often Ghost removes stale fingerprints. Patterns not seen for StaleAfter (also 5 minutes) are garbage-collected, freeing memory.

Verdicts in Detail

Verdict	What Happens to the Flow	Stored in Database?	Shows in UI?	Client Gets Response?
Keep	Normal processing — flow passes through the full interceptor pipeline, is saved, and appears in your traffic list	Yes	Yes	Yes
Sample	Kept as a representative of a noisy pattern. Tagged with _noise_suppressed:N showing how many repetitions were suppressed since the last sample.	Yes	Yes (with noise badge)	Yes
Suppress	Flow is silently skipped by Ghost’s recording system. The proxy still forwards the request to the upstream server and returns the response to the client — only Ghost’s recording is affected.	No	No	Yes — always

The most important thing to understand: suppression never breaks your application. Your browser, mobile app, or any other client continues to receive responses normally. Ghost just chooses not to record certain repetitive flows to keep your analysis clean.

Pattern Fingerprinting

Ghost identifies patterns by combining three elements into a fingerprint: METHOD:host:path (host is lowercased for case-insensitive matching). This means:

• GET:api.example.com:/healthcheck and GET:api.example.com:/healthcheck?ts=12345 are the same pattern — query parameters are ignored
• GET:api.example.com:/healthcheck and POST:api.example.com:/healthcheck are different patterns — different HTTP methods
• GET:api.example.com:/healthcheck and GET:staging.example.com:/healthcheck are different patterns — different hosts

Query parameters and headers are intentionally excluded from the fingerprint. This is because repetitive requests (like polling endpoints) often include changing timestamps, nonces, or request IDs in their query parameters. If Ghost considered these, each request would look unique and noise would never be detected.

Hysteresis — Preventing Oscillation

The noise detector uses hysteresis to avoid a flickering problem. Here’s what would happen without it:

Imagine a pattern that averages exactly 20 requests per 30 seconds (right at the threshold). Due to natural timing variation, the count might be 21 for one window (noisy!) then 19 for the next (not noisy!) then 22 (noisy again!) — the pattern would flip between noisy and normal every few seconds, providing an inconsistent experience.

Ghost solves this with different entry and exit thresholds:

• Entering noisy state: The count must reach the threshold (default: 20)
• Exiting noisy state: The count must drop below threshold/2 (default: 10)

This gap means that once a pattern is classified as noisy, it stays noisy until traffic genuinely slows down to half the detection rate — not just briefly dipping below the threshold. The result is stable, predictable behavior.

Tagged Flow Exception

If an addon script tags a flow with any user-defined tag (via flow.tag("something")), that flow is never suppressed, regardless of how noisy its pattern is.

Why? Because if you wrote an addon that specifically tags certain flows, those flows have business significance to you. Ghost respects that decision and always shows them, even if they’re part of a repetitive pattern.

For example, if you have an addon that tags flows to your payment API with payment, those flows will always appear in the traffic list even if the payment endpoint happens to receive 50 requests per second during a load test.

Real-World Impact

Without noise detection, a typical 30-minute QA testing session on a Hepsiburada mobile app might capture:

• ~500 meaningful flows — API calls to product listing, search, cart, checkout, etc.
• ~15,000 noise flows — CDN asset checks, analytics pings, health endpoints, telemetry, certificate transparency logs, etc.

With noise detection enabled, you see ~600 flows (500 meaningful + ~100 representative noise samples). The signal-to-noise ratio goes from 1:30 to about 5:1 — a massive improvement in usability.

Troubleshooting

“I can’t find a request I expected to see” — If a request was part of a noisy pattern, it might have been suppressed. Try searching for it by URL in the search bar — if noise detection suppressed it, you won’t find it. Options:

1. Temporarily disable noise detection in Settings
2. Lower the threshold to be less aggressive
3. Add the host to the exclude-from-noise-detection list (if available)
4. Use an addon to tag the specific endpoint so it’s exempt from suppression

“Noise detection is too aggressive / not aggressive enough” — Adjust the threshold in Settings. A threshold of 10 starts suppressing after 10 identical requests in 30 seconds (aggressive). A threshold of 50 waits until 50 repetitions (lenient). The default of 20 works well for most testing scenarios.