Journey Recording
Journey recording captures the relationship between what a user does in the browser (clicking buttons, filling forms, navigating pages) and what happens on the network as a result (API calls, resource loads, background requests). Each journey is a named, ordered sequence of “steps” — some steps are browser interactions, others are the HTTP flows those interactions triggered, and some are correlated pairs of both. Together, they tell the complete story of a user flow.
Think of a journey like a detailed screenplay: “The user clicked the login button (interaction), which sent a POST to /api/auth (correlated action), then the page navigated to /dashboard (interaction), which loaded 12 API calls (flows).” This combination of UI actions and network activity in a single timeline is what makes journeys powerful for understanding, documenting, reproducing, and testing user behavior.
Journey recording requires the browser extension to be connected. The extension injects correlation headers into outgoing requests so that Ghost can match each browser interaction to the exact HTTP flows it triggered.
Architecture
Section titled “Architecture”What this diagram shows:
- When recording starts, Ghost tells the extension to begin injecting
X-Ghost-Interactionheaders on all outgoing requests via Chrome’sdeclarativeNetRequestAPI - When the user interacts with the page (click, form change, submit), the extension generates a unique interaction ID and immediately updates the header rule with that ID
- Any HTTP requests sent within the next 600ms carry that unique ID in their headers — Ghost’s proxy reads and strips the header, storing the interaction ID on the flow
- The backend recorder sees the flow has an
InteractionIDand creates a step of type"action"(correlated interaction + flow) - After the 600ms correlation window expires, the extension resets the header to an “ambient” ID — subsequent requests become standalone
"flow"steps (no correlation) - When recording stops, the extension removes the header injection rule entirely
Data Model
Section titled “Data Model”Journey
Section titled “Journey”A journey represents one complete user flow — like “Add item to cart and checkout” or “Login and navigate to settings.”
| Field | Type | Description |
|---|---|---|
| id | string | Unique identifier (ULID, time-ordered) for this journey |
| session_id | string | Which session this journey belongs to. Journeys are scoped to a session — when the session is deleted, all its journeys are automatically deleted too (CASCADE). |
| name | string | A human-readable label. When starting from the UI, Ghost auto-generates names like "Journey Mar 09 14:32". When starting via the AI agent, you provide a custom name. |
| status | string | Current lifecycle state: recording, completed, failed, or replaying |
| flow_count | integer | Number of HTTP flows in this journey — automatically calculated when recording completes by counting steps with a non-NULL flow_id |
| interaction_count | integer | Number of browser interactions — automatically calculated when recording completes by counting steps with a non-NULL interaction_id |
| started_at | datetime | When the recording began (RFC3339Nano for nanosecond precision) |
| completed_at | datetime | When the recording was completed (null while still recording) |
| created_at | datetime | Database creation timestamp |
Journey Steps
Section titled “Journey Steps”Each step represents one discrete event in the journey — either something the user did in the browser, an HTTP request captured by the proxy, or a correlated pair of both.
| Field | Type | Description |
|---|---|---|
| id | string | Unique identifier (ULID) for this step |
| journey_id | string | Which journey this step belongs to. Deleted with the journey (CASCADE). |
| step_order | integer | Sequence position within the journey (1-indexed). Steps are always returned sorted by this field. Assigned by a monotonic counter protected by a mutex in the recorder. |
| type | string | One of three values: "action" (correlated interaction+flow), "flow" (standalone HTTP request), or "interaction" (user action without network activity) |
| interaction_id | string | Reference to the browser interaction record. SET NULL on delete — if the interaction is deleted, the step remains. |
| flow_id | string | Reference to the HTTP flow record. SET NULL on delete — deleting a flow doesn’t break the journey sequence. |
| data | string | JSON metadata about the step. For interaction steps: StepInteractionData (interaction type, CSS selector, element text, page URL, element attributes). For flow/action steps: StepFlowData (HTTP method, URL, host, path, status code, content type, content length, duration). See Interaction Data Model for the full StepInteractionData schema. |
| timestamp | datetime | When this step occurred (RFC3339Nano precision) |
Step Types
Section titled “Step Types”Three distinct step types capture different aspects of user behavior:
| Type | Color in UI | When Created | What It Means |
|---|---|---|---|
| action | Purple | A proxy flow arrives with a non-empty InteractionID | The user did something (click, submit, change) that directly triggered this HTTP request. The step has both a flow_id and an interaction_id. This is the most valuable step type — it maps a UI action to its exact API call. |
| flow | Cyan | A proxy flow arrives during recording without an InteractionID, or with an expired ambient ID | An HTTP request happened during the recording but wasn’t directly triggered by a captured interaction. Examples: background polling, analytics pings, resource loads. The step has a flow_id but no interaction_id. |
| interaction | Yellow/Warning | A browser interaction is captured but no flow arrives within the correlation window | The user did something in the browser that didn’t trigger any network request. Examples: clicking a toggle that only changes local state, hovering over elements, focusing a field without submitting. The step has an interaction_id but no flow_id. |
Reference Lifecycle
Section titled “Reference Lifecycle”The foreign key behavior is designed to preserve journey integrity even when referenced data is deleted:
What this diagram shows:
- Solid arrows (CASCADE): Deleting a session deletes all its journeys. Deleting a journey deletes all its steps. This is hierarchical ownership — the parent owns the children.
- Dashed arrows (SET NULL): Deleting an interaction or flow does NOT delete the step that references it. Instead, the reference (
interaction_idorflow_id) is set to NULL. The step itself remains, keeping its position in the journey sequence. This means you can clean up old flows without breaking journey recordings — the journey still shows “step 3 was an action, step 4 was a flow” even if the original records are gone.
Journey Lifecycle
Section titled “Journey Lifecycle”What this diagram shows — the four states a journey passes through:
| Status | What’s Happening | How It Transitions |
|---|---|---|
| recording | Actively capturing steps. Browser interactions and correlated proxy flows are being added as new steps. The extension is injecting X-Ghost-Interaction headers. | Set when the journey is first created via POST /journeys/start. Only one recording per session is allowed — attempting to start a second returns 409 Conflict. |
| completed | Recording is finished. flow_count and interaction_count are populated, completed_at is set. The journey is now a complete record that can be reviewed, exported, or replayed. | Set when POST /journeys/stop is called. Counts are computed via SQL subqueries: SELECT COUNT(*) FROM journey_steps WHERE journey_id = ? AND flow_id IS NOT NULL for flows, same pattern for interactions. |
| failed | The recording was interrupted — the server crashed or restarted while a journey was in recording status. | Set by the orphan recovery process at startup. The recorder scans all sessions for journeys stuck in recording status and transitions them to failed. |
| replaying | The journey’s HTTP flows are being re-sent via the replay engine (SSE streaming). | Set at the start of replay, reverts to completed when replay finishes. |
Completion Logic
Section titled “Completion Logic”When a journey is completed, Ghost runs a single SQL UPDATE that does three things simultaneously:
- Sets
statusto"completed" - Sets
completed_atto the current time - Counts flows and interactions via correlated subqueries:
flow_count = (SELECT COUNT(*) FROM journey_steps WHERE journey_id = ? AND flow_id IS NOT NULL)interaction_count = (SELECT COUNT(*) FROM journey_steps WHERE journey_id = ? AND interaction_id IS NOT NULL)
This means the counts reflect exactly how many steps have valid references at the time of completion — steps where the referenced flow or interaction was already deleted (SET NULL) are not counted.
Orphan Recovery
Section titled “Orphan Recovery”When Ghost starts, the recorder runs RecoverOrphaned(), which:
- Lists all sessions in the database
- For each session, lists all journeys
- Any journey with
status = "recording"is transitioned tostatus = "failed" - This prevents ghost recordings that can never be completed from blocking new recordings on that session
Failed journeys can be manually completed via POST /journeys/{id}/complete if you want to recover their step data.
How Recording Works
Section titled “How Recording Works”Starting a Recording
Section titled “Starting a Recording”Recording can be triggered from two places:
-
Ghost UI — Click the record button in the command bar. Ghost auto-generates a name like
"Journey Mar 09 14:32". The button is only enabled when the browser extension is connected and a session is active. -
AI Agent — Call
record_journeywithaction: "start"and a customname. The agent tool uses the active session automatically.
Both paths call the same API endpoint (POST /api/v1/journeys/start), which:
- Verifies the extension is connected (returns 400 if not)
- Checks no other recording is active for this session (returns 409 if one exists)
- Creates the journey in the database with
status: "recording" - Sends a
journey.recording_startWebSocket message to the extension with the journey ID - Broadcasts a
journey:startedevent to the frontend via WebSocket
Interaction-Flow Correlation
Section titled “Interaction-Flow Correlation”The correlation mechanism has three components working together:
Extension (declarativeNetRequest):
- When recording starts, the extension creates a dynamic
declarativeNetRequestrule (ID9999) that injects anX-Ghost-Interactionheader on all outgoing requests - The rule targets three resource types:
XMLHTTPREQUEST(covers both XHR and fetch),MAIN_FRAME, andSUB_FRAME - On each meaningful interaction (click, change, submit — NOT hover or focus), the extension increments a counter and updates the header value to
{journeyId}-{counter}-{timestamp} - After 600ms, the header resets to an “ambient” value (
{journeyId}-ambient-{timestamp}) so unrelated requests don’t get incorrectly correlated - The 600ms window exceeds the backend’s 500ms correlation window to ensure coverage
Proxy (header reading):
- The proxy reads the
X-Ghost-Interactionheader from incoming requests - If present, it stores the value on the flow’s
InteractionIDfield - The header is stripped before forwarding — the destination server never sees it
Recorder (step creation):
- When a completed flow arrives via the
OnFlowhook, the recorder checks if the flow’s session has an active recording - If the flow has a non-empty
InteractionIDthat doesn’t contain"ambient", the step is created as type"action"with bothflow_idandinteraction_idset - If the flow has no
InteractionIDor an ambient one, the step is created as type"flow"with onlyflow_id
Stopping a Recording
Section titled “Stopping a Recording”When recording stops (via UI button or record_journey action:stop):
- The backend removes the recording from the active map
- Calls
CompleteJourney()to update counts and status - Sends
journey.recording_stopto the extension, which removes thedeclarativeNetRequestrule - Broadcasts
journey:completedto the frontend
Noise Filtering
Section titled “Noise Filtering”Not every HTTP flow during a recording belongs in the journey. isJourneyNoise() filters out requests that are browser infrastructure, analytics, or static assets — keeping only meaningful API calls and page navigations. This filter applies to all flows, including correlated ones (a favicon triggered by a click is still noise, not the user’s intended action).
| Category | What’s Filtered | Examples |
|---|---|---|
| CORS preflight | OPTIONS requests | Browser-generated, not user actions |
| Analytics/telemetry domains | Known SDK domains (via inspector.IsNoiseDomain()) | Google Analytics, Segment, Sentry, Datadog, Hotjar, etc. |
| Browser infrastructure paths | /favicon*, /.well-known/, /sw.js, /service-worker*, /manifest.json, /robots.txt, /sitemap*, /apple-touch-icon*, /browserconfig.xml | Browser-initiated, not user-triggered |
| Static asset extensions | .js, .mjs, .cjs, .css, .woff, .woff2, .ttf, .eot, .otf, .png, .jpg, .jpeg, .gif, .svg, .ico, .webp, .avif, .map, .mp3, .mp4, .webm, .ogg | Resource loads, not API calls |
| Content-type based | Responses with image/*, font/*, audio/*, video/*, text/css, application/font* | Binary resources |
| Dev server HMR | Paths containing /__vite, /__webpack, /hot-update, or starting with /@ | Hot module replacement during development |
Noise filtering is deliberately aggressive — a journey should capture “user clicked login, which called POST /api/auth” not “user clicked login, which loaded 3 fonts, 12 images, a service worker, and POST /api/auth.”
Stale Rule Cleanup
Section titled “Stale Rule Cleanup”On extension startup (page load, service worker restart), disableJourneyHeaderInjection() is called unconditionally. This removes any lingering journey rule from a previous session or crash, preventing phantom X-Ghost-Interaction headers on regular browsing.
Frontend UI
Section titled “Frontend UI”Command Bar Controls
Section titled “Command Bar Controls”The recording controls live in the command bar between the proxy toggle and the search bar:
Idle state (not recording):
- Record button (red circle icon) — disabled when extension is disconnected or no active session
- Tooltip explains why:
"Connect browser extension to record journeys"or"Start a session first"
Recording state:
- Pulsing red recording pill with the journey name (truncated to 80px), elapsed time (m:ss format, updated every second via
setInterval), live flow count, and a stop button - Elapsed timer starts from
journey.started_atand runs until stopped - Step count updates in real-time via WebSocket
journey:step_addedevents
Journey Panel (Slide-Over)
Section titled “Journey Panel (Slide-Over)”The Journeys slide-over panel (accessed via the Path icon in the command bar actions) has three views:
List view:
- Shows all journeys for the active session, sorted by newest first
- Each row shows: status icon (colored by state), journey name, duration, flow count, and interaction count
- Hover reveals export dropdown and delete button
- Click a journey to open detail view
Detail view:
- Header: back button, journey name, status badge, duration, counts
- Action buttons (hidden during recording): Replay button and Export dropdown. For failed journeys, a “Mark Complete” button appears — clicking it calls
POST /journeys/{id}/completeto recover the journey and transition it tocompletedstatus, re-enabling export and replay. - Step timeline: vertical timeline with colored dots per step type (purple=action, cyan=flow, yellow=interaction)
- Each step shows: type badge, step order number, timestamp (HH:MM:SS), truncated flow/interaction IDs
- Clicking a step with a
flow_idnavigates to that flow in the main traffic view
Replay view:
- Appears within the detail view when replay is running or complete
- Shows replay results inline above the step timeline
- Each result row: step number, HTTP method, status comparison (original → replayed) with color coding (green=match, red=changed), and duration delta
State Management
Section titled “State Management”The useJourneyStore (Zustand) manages:
| State | Description |
|---|---|
journeys | Array of all journeys for the active session |
activeRecording | Current recording state (journey_id, session_id, is_recording) or null |
selectedJourney | The journey being viewed in detail view |
selectedJourneySteps | Steps for the selected journey |
loading | Whether the journey list is being fetched |
recordingLoading | Whether a start/stop operation is in progress |
WebSocket event handlers keep the UI in sync without polling:
| WebSocket Event | Handler | Effect |
|---|---|---|
journey:started | onJourneyStarted | Adds the new journey to the list, sets activeRecording |
journey:step_added | onJourneyStepAdded | Increments flow_count/interaction_count on the journey, appends step to detail view if open |
journey:completed | onJourneyCompleted | Updates status to completed, sets final counts, clears activeRecording |
journey:deleted | onJourneyDeleted | Removes journey from list, clears detail view if it was selected |
Export Engine
Section titled “Export Engine”Ghost can export a completed journey as deterministic test code in 8 formats. No AI is involved — the output is pure codegen derived directly from the captured HTTP flows and browser interactions.
Export Flow
Section titled “Export Flow”What this diagram shows:
- The exporter loads the journey and all its steps
- API-based formats (curl, postman, cypress-api, playwright-api, k6, har) follow the flow path: load HTTP flows for each step, filter deleted/incomplete, then pass to the format-specific generator. Steps without flows (pure interactions) are skipped. If no exportable flows remain, the export returns an error:
"journey has no exportable flows"(HTTP 422). - UI-based formats (cypress-ui, playwright-ui) follow the POM path: the
analyzePOM()pipeline processes interaction steps through deduplication, page grouping, business method detection, credential extraction, and API wait mapping — producing multi-file Page Object Model output. Flow steps are not loaded individually; instead, flow/action step data is used for API wait mapping. If no interaction steps exist, the export returns an error:"journey has no interactions"(HTTP 422).
Supported Formats
Section titled “Supported Formats”| Format | Content-Type | File Extension | What It Generates |
|---|---|---|---|
| curl | text/x-shellscript | .sh | Bash script with one curl command per flow. Includes method, URL, headers, and body. Uses shellQuote() for safe shell escaping (single quotes with '\'' for embedded quotes). Body is truncated at 4,096 bytes with a (truncated) marker. Adds -L for 3xx redirect responses. |
| postman | application/json | .postman_collection.json | Postman Collection v2.1 JSON. Each flow becomes an item with method, URL (split into protocol/host/path/query), sorted headers, and raw body. Named "Step N — METHOD /path". Deduplicates by method+URL. |
| playwright-api | text/typescript | .api.spec.ts | TypeScript test using @playwright/test’s request API. Each flow becomes a request.method() call with headers object and data. Includes expect(response.status()).toBe(N) assertion. Deduplicates by method+URL. |
| playwright-ui | text/typescript | .ui.spec.ts | TypeScript E2E test with Page Object Model — multi-file output (constants.ts, pages/*.ts, *.spec.ts) using // ===== File: <path> ===== separators. Page objects use readonly Locator fields with typed constructor, framework-native locators (getByTestId, getByRole, getByLabel, getByText), async business methods, and credential externalization via process.env. Spec uses test.describe/test.beforeEach with page.waitForURL() for navigation after login redirects. See POM Architecture for full details. |
| cypress-api | application/javascript | .api.cy.js | JavaScript test with describe/it block. Each flow becomes a cy.request() call with headers and body. Includes expect(response.status).to.eq(N) assertion. Adds failOnStatusCode: false for 4xx/5xx responses. |
| cypress-ui | application/javascript | .ui.cy.js | JavaScript E2E test with Page Object Model — multi-file output (constants.js, pages/*.js, *.cy.js) using // ===== File: <path> ===== separators. Page objects use method-based getters (getUsernameField()), { force: true } on all interactions, .should('be.visible') visibility assertions, .should('have.value', ...) value assertions, cy.intercept()/cy.wait() for API response verification, and cy.wait(1000) for React render timing. Credential externalization via Cypress.env(). Only first page uses visit(); subsequent pages use cy.url().should('include', path) for natural navigation (login redirects, link clicks). See POM Architecture for full details. |
| k6 | application/javascript | .k6.js | k6 load test script. Imports http and check. Each flow becomes an http.method() call with check() assertions on status code. sleep(1) between steps. Default: 1 VU, 1 iteration. Falls back to http.request() for GET requests with a body. |
| har | application/json | .har | HTTP Archive 1.2 with full request/response data. Includes request headers, query parameters, cookies, response headers, response body (text for UTF-8, base64 for binary), timing data. If the flow has detailed timings (TCP, TLS, TTFB, transfer), those populate the HAR timings object. Otherwise, total duration is placed in the wait field. Creator: Ghost 1.0. |
Page Object Model (POM) Architecture
Section titled “Page Object Model (POM) Architecture”The cypress-ui and playwright-ui formats generate multi-file Page Object Model output — not flat single-file tests. This produces maintainable, reusable test code that mirrors how professional QA teams structure their automation suites.
Analysis Pipeline
Section titled “Analysis Pipeline”What this diagram shows:
- Journey steps are parsed into
StepInteractionDataobjects — only interaction steps (not flow-only steps) are used for UI export - Two deduplication passes clean up redundant interactions before grouping
- API waits are extracted from flow/action steps and mapped to the interaction that triggered them
- Interactions are grouped by page URL — each distinct URL becomes a page object
- Business methods are detected from consecutive fill+click patterns (e.g., login form)
- Credentials are detected and externalized to environment variables
- The structured data is emitted as either Cypress JavaScript or Playwright TypeScript
Multi-File Output Structure
Section titled “Multi-File Output Structure”POM exports produce multiple files concatenated with separator lines. The separator format is:
// ===== File: <relative-path> =====Consumers (API handler, agent tool, frontend) use SplitMultiFileExport() to split the output into individual FileEntry objects, each with a Path (e.g., "pages/LoginPage.js") and Content (the file body). IsMultiFileExport() checks for the separator prefix to detect multi-file output.
Cypress POM structure:
constants.js — BASE_URL + credential env vars (Cypress.env())pages/LoginPage.js — Page object class with method getters + visit() + business methodspages/DashboardPage.js — One file per distinct page URLlogin-and-add-to-cart.cy.js — Test spec with imports, page instantiation, and test bodyPlaywright POM structure:
constants.ts — BASE_URL + credential env vars (process.env)pages/LoginPage.ts — Page object class with readonly Locator fields + constructor + goto() + async methodspages/DashboardPage.ts — One file per distinct page URLlogin-and-add-to-cart.spec.ts — Test spec with test.describe, test.beforeEach, and test bodyInteraction Data Model
Section titled “Interaction Data Model”Each interaction step’s JSON data field contains a StepInteractionData object with these fields:
| Field | Type | Description |
|---|---|---|
interaction_type | string | click, input, change, submit, navigation |
selector | string | CSS selector path for the element (auto-generated by the extension) |
element_type | string | HTML tag name: input, button, a, div, textarea, select, etc. |
element_text | string | For inputs: the typed value. For clicks: the element’s visible text content. |
page_url | string | Full URL of the page where the interaction occurred |
page_title | string | The <title> of the page |
attributes | object | Semantic element attributes for robust selector generation (see below) |
The attributes object captures stable, semantic data that survives DOM changes:
| Attribute | Example | How It’s Used |
|---|---|---|
id | "login-form" | #login-form CSS selector |
name | "username" | input[name="username"] selector, usernameField property name |
role | "button" | cy.get('[role="button"]').contains(text) or page.getByRole('button', { name: text }) |
aria_label | "Submit order" | [aria-label="Submit order"] or page.getByLabel('Submit order') |
placeholder | "Enter email" | Last-resort selector for form fields with fragile CSS paths |
type | "password" | Credential detection (password fields become PASSWORD env var) |
href | "/dashboard" | Not used for selectors, but available for link analysis |
title | "Click to expand" | Not used for selectors |
test_id | "login-button" | Highest-priority selector: button[data-test-id="login-button"] |
test_id_attr | "data-test-id" | Which attribute name holds the test ID (varies by project: data-testid, data-test, data-id, data-cy) |
visible_text | "Sign In" | cy.contains('button', 'Sign In') or page.getByRole('button', { name: 'Sign In' }) |
tag | "button" | Used to scope selectors by tag (e.g., button[name="x"] instead of bare [name="x"]) |
Interaction Deduplication
Section titled “Interaction Deduplication”Before grouping by page, the raw interaction stream goes through two deduplication passes in parseUIInteractions():
Pass 1 — Click-before-type: When a user clicks an input field and then types in it, the browser fires both a click event and an input/change event on the same selector. The click is redundant — the type action implies focus. If a click is immediately followed by an input or change on the same selector, the click is marked as skipped.
Pass 2 — Consecutive duplicate clicks: Dropdown triggers, accordion headers, and overlay toggles often fire multiple click events on the same element. If consecutive clicks target the same selector AND same text, only the first is kept. Clicks on the same selector with different text are preserved — these represent distinct user actions (e.g., selecting “Mordor”, “Navigation”, “Payment” from the same dropdown container).
Raw interactions: [click email] [input email "test@ex.com"] [click dropdown] [click dropdown] [click "Mordor"] [click "Navigation"]After Pass 1: [input email "test@ex.com"] [click dropdown] [click dropdown] [click "Mordor"] [click "Navigation"]After Pass 2: [input email "test@ex.com"] [click dropdown] [click "Mordor"] [click "Navigation"]The text comparison in Pass 2 is critical — without it, selecting multiple options from the same dropdown container (which share a parent selector like div[data-test-id="tag_type"]) would collapse to a single click.
Page Grouping
Section titled “Page Grouping”After deduplication, interactions are grouped into pages by their page_url. URL normalization strips query parameters and fragments — so https://app.com/dashboard?tab=overview and https://app.com/dashboard?tab=settings group into the same page.
Within each page, elements are deduplicated by a selector key:
- Click actions: key =
selector + "|" + elementText. Different text on the same selector creates separate elements (dropdown options). - Fill/change actions: key =
selectoralone. The same input field with different typed values is still one element.
Each unique element gets a PropertyName derived from its attributes (see Element Name Derivation).
Class Name Derivation
Section titled “Class Name Derivation”Page URLs are converted to PascalCase class names by deriveClassName():
| URL Path | Class Name | Logic |
|---|---|---|
/ | HomePage | Empty/root path defaults to Home |
/login | LoginPage | Single segment, capitalized |
/auth/login | AuthLoginPage | Multiple segments joined |
/products/123/edit | ProductsEditPage | Numeric segments (IDs) stripped |
/api/v1/users | UsersPage | API prefixes (api, v1, v2, v3) stripped |
/product-details | ProductDetailsPage | Hyphens/underscores split into words |
/search?q=shoes | SearchPage | Query params already stripped by normalization |
/settings.html | SettingsPage | File extensions (≤5 chars) stripped |
If multiple pages resolve to the same class name (e.g., two different query param variants), numeric suffixes are appended: SearchPage, Search2Page.
Page is always appended as a suffix unless the name already ends with Page.
Element Name Derivation
Section titled “Element Name Derivation”Each element gets a camelCase property name via deriveElementName(), following this priority chain:
| Priority | Source | Example Input | Example Output |
|---|---|---|---|
| 1 | test_id attribute | "login-button" | loginButton |
| 2 | name attribute | "username" | usernameField |
| 3 | aria_label attribute | "Go to settings" | goToSettingsLink |
| 4 | visible_text (buttons/links only) | "Sign In" | signInButton |
| 5 | CSS selector data attributes | button[data-id="home_btn_create"] | homeBtnCreateButton |
| 6 | visible_text (any non-form element) | "Fiyat Filtresi" | fiyatFiltresi |
| 7 | Fallback: tag + index | (no meaningful data) | div1, button3, element2 |
Contextual suffixes are appended based on the element’s tag: Button for <button> and <input type="submit">, Link for <a>, Field for <input> and <textarea>, Select for <select>. Suffixes like Btn, Button, Link, Field, Input, Select, Checkbox, Radio that already exist in the name are not duplicated.
Form field exclusion: For <input>, <textarea>, and <select> elements, element_text contains the typed value (not a label), so it’s never used for naming — a field where the user typed "KTIMAS43002" should not be named ktimas43002Field.
Uniqueness: If a derived name collides with an existing element on the same page, a numeric suffix is appended: loginButton, loginButton2, loginButton3.
Character Transliteration
Section titled “Character Transliteration”Element names and identifiers must be valid JavaScript identifiers. Ghost transliterates non-ASCII characters to their ASCII equivalents before building names, covering Turkish, German, French, Spanish, Portuguese, Polish, and Nordic character sets:
| Character | Transliteration | Character | Transliteration |
|---|---|---|---|
ş / Ş | s / S | ö / Ö | o / O |
ı / İ | i / I | ü / Ü | u / U |
ğ / Ğ | g / G | ç / Ç | c / C |
é / è / ê | e | ñ / Ñ | n / N |
ß | ss | æ / Æ | ae / AE |
ø / Ø | o / O | å / Å | a / A |
ą / ę / ł | a / e / l | ś / ź / ż | s / z / z |
This ensures that a button labeled "Giriş Yap" (Turkish for “Sign In”) produces the identifier girisYap instead of containing characters that would break JavaScript syntax.
After transliteration, non-alphanumeric characters are split into words and joined as camelCase: "login-button" → loginButton, "Add to Cart" → addToCart.
Dynamic Text Filtering
Section titled “Dynamic Text Filtering”Text content that changes between recording and replay (dates, times, numbers, prices) is never used for selectors or element names. isDynamicText() checks against four regex patterns:
| Pattern | Examples Matched |
|---|---|
Date formats (dd-mm-yyyy, dd/mm/yyyy, yyyy-mm-dd, mm.dd.yyyy) | 17-03-2026, 2026/03/10, 10.03.2026 |
Time patterns (HH:MM, HH:MM:SS) | 14:30, 02:45:00 |
ISO timestamps (yyyy-mm-ddT...) | 2026-03-10T14:30:00 |
| Pure numbers, prices, percentages | 1234, $99.99, ₺1.250,50, 100%, €500 |
When dynamic text is detected, the selector falls through to the next priority level (CSS attributes, raw CSS selector, etc.) instead of using cy.contains() or page.getByText() with a value that will be different on replay.
Container Text Detection
Section titled “Container Text Detection”Elements whose element_text contains newline characters (\n) are detected as container elements — their text is concatenated from multiple child elements (e.g., a modal backdrop <div> that contains a heading, subtitle, and form). Using cy.contains("heading text") on such an element would match the child heading element, not the container itself. Clicking the heading doesn’t trigger the container’s click handler (e.g., modal dismiss on backdrop click).
isContainerText() checks for newlines in element_text. When detected:
cy.contains()is NOT used (would select a child, not the container)page.getByText()is NOT used (same problem)- The selector falls through to the test-ID anchor or raw CSS selector
This prevents a common failure mode: recording a modal dismiss by clicking outside the modal produces a selector for the backdrop <div>, but if text-based selectors are used, Cypress/Playwright clicks the heading inside the modal instead — and the modal never closes.
Selector Generation
Section titled “Selector Generation”Ghost generates selectors using a priority chain that prefers stable, semantic attributes over fragile positional CSS paths. The chains differ between Cypress and Playwright to use each framework’s idiomatic APIs.
Cypress Selector Priority (10 steps):
| Step | Condition | Selector Generated | Example |
|---|---|---|---|
| 1 | test_id present | cy.get('tag[attr="value"]') | cy.get('button[data-test-id="login-btn"]') |
| 2 | id present | cy.get('#id') | cy.get('#login-form') |
| 3 | name present | cy.get('tag[name="value"]') | cy.get('input[name="email"]') |
| 4 | aria_label present | cy.get('tag[aria-label="value"]') | cy.get('button[aria-label="Submit"]') |
| 5 | role + visible_text (non-dynamic) | cy.get('[role="x"]').contains('text') | cy.get('[role="button"]').contains('Save') |
| 6 | visible_text on <button>/<a> (non-dynamic) | cy.contains('tag', 'text') | cy.contains('button', 'Sign In') |
| 7 | Test-ID anchor + visible text (non-container, non-dynamic) | cy.get('parent[data-test-id="x"]').contains('text') | cy.get('div[data-test-id="tag_type"]').contains('Mordor') |
| 8 | Visible text + fragile selector (non-container, non-dynamic) | cy.contains('tag', 'text') or cy.contains('text') | cy.contains('Fiyat Filtresi') |
| 9 | placeholder + form field + fragile selector | cy.get('tag[placeholder="value"]') | cy.get('input[placeholder="Enter email"]') |
| 10 | Raw CSS selector (tag-scoped) | cy.get('selector') or cy.get('selector').first() | cy.get('button.submit-btn') |
Playwright Selector Priority (11 steps):
| Step | Condition | Selector Generated | Example |
|---|---|---|---|
| 1 | test_id with data-testid attr | page.getByTestId('value') | page.getByTestId('login-btn') |
| 1b | test_id with other attr | page.locator('tag[attr="value"]') | page.locator('button[data-id="login-btn"]') |
| 2 | role + name/text | page.getByRole('role', { name: 'text' }) | page.getByRole('button', { name: 'Sign In' }) |
| 3 | visible_text on button/link (non-dynamic) | page.getByRole('button/link', { name: 'text' }) | page.getByRole('link', { name: 'Dashboard' }) |
| 4 | aria_label present | page.getByLabel('label') | page.getByLabel('Email address') |
| 5 | id present | page.locator('#id') | page.locator('#login-form') |
| 6 | name present | page.locator('tag[name="value"]') | page.locator('input[name="email"]') |
| 7 | visible_text (non-dynamic, non-container) | page.getByText('text') | page.getByText('Welcome back') |
| 8 | Test-ID anchor + visible text (non-container) | page.locator('parent').getByText('text') | page.locator('div[data-test-id="options"]').getByText('Mordor') |
| 9 | Visible text + fragile selector (non-container) | page.getByText('text') | page.getByText('Fiyat Filtresi') |
| 10 | placeholder + form field + fragile selector | page.getByPlaceholder('text') | page.getByPlaceholder('Enter email') |
| 11 | Raw CSS selector (tag-scoped) | page.locator('selector') or .first() | page.locator('button.submit-btn') |
Key concepts used across both chains:
- Tag scoping: Bare attribute selectors like
[name="email"]are prefixed with the element’s tag:input[name="email"]. This prevents matching unrelated elements with the same attribute. Compound selectors with combinators (spaces,>) are NOT prefixed — the[attr]belongs to an ancestor, not the target element. - Test-ID anchor extraction: From compound selectors like
div[data-test-id="tag_type"] > div:nth-of-type(3), Ghost extracts the stable ancestordiv[data-test-id="tag_type"]and uses.contains(text)/.getByText(text)to find the target within it. This is the Cypress/Playwright-idiomatic way to select dropdown options, list items, and other children of stable containers. - Fragile selector detection: Selectors consisting entirely of
nth-of-typechains with no attributes, IDs, or classes are flagged as fragile. These break when DOM structure changes. Ghost falls through to text-based selectors when possible, or adds.first()as a safety measure. - CSS attribute quoting: Values in CSS attribute selectors use CSS escaping rules (backslash before
\and") viacssQuoteAttr(), not Go’s%qwhich uses Go string escaping. containsText()sanitization: Text used incy.contains()/page.getByText()is limited to the first line only (multi-line text spans multiple DOM elements) and capped at 80 runes (long text is brittle). Truncation happens at the last space before 80 runes to avoid breaking words, using rune count (not byte count) to handle multi-byte UTF-8 characters (Turkishş,ğ, etc.).
Business Method Detection
Section titled “Business Method Detection”When Ghost detects 2+ consecutive fill/clear actions followed by a click/submit action, it groups them into a business method. This transforms verbose step-by-step interactions into a single semantic method call.
Interactions: [fill email] [fill password] [click loginBtn]Detected as: login(email, password) method → Params: email (from emailField), password (from passwordField) → Actions: fill emailField, fill passwordField, click loginButton → Submit: loginButtonThe method name is derived from the submit button’s property name by stripping suffixes: loginButton → login, submitBtn → submit, signInButton → signIn. Generic names (submit, button, ok, go) fall back to submitForm.
Parameter names are derived from element property names by stripping type suffixes: usernameField → username, emailInput → email, passwordField → password.
Actions consumed by a business method are marked Consumed = true and won’t appear as standalone actions in the spec file.
Cypress business method output:
login(username, password) { this.getUsernameField().should('be.visible').clear({ force: true }).type(username, { force: true }); this.getUsernameField().should('have.value', username); this.getPasswordField().should('be.visible').clear({ force: true }).type(password, { force: true }); this.getPasswordField().should('have.value', password); this.getLoginButton().should('be.visible').click({ force: true }); return this;}Playwright business method output:
async login(username: string, password: string) { await this.usernameField.fill(username); await this.passwordField.fill(password); await this.loginButton.click();}Note the differences: Cypress methods use fluent chaining with return this, method-based getters (this.getUsernameField()), visibility assertions (.should('be.visible')), value assertions (.should('have.value', ...)), and { force: true }. Playwright methods use async/await, direct property access (this.usernameField), and Playwright’s built-in auto-waiting.
Credential Detection and Externalization
Section titled “Credential Detection and Externalization”Fields that look like credentials are automatically externalized to environment variables instead of hardcoding recorded values:
| Detection Rule | Env Var | How Detected |
|---|---|---|
type="password" attribute | PASSWORD | Element’s HTML type attribute |
Property name contains password/passwd/secret/pin | PASSWORD | Case-insensitive substring match |
Property name contains email | EMAIL | Case-insensitive substring match |
Property name contains username/user | USERNAME | Case-insensitive substring match |
The recorded values are preserved as defaults in the constants file:
// Cypress constants.jsexport const BASE_URL = Cypress.env('BASE_URL') || "https://app.example.com";export const USERNAME = Cypress.env('USERNAME') || "standard_user";export const PASSWORD = Cypress.env('PASSWORD') || "secret_sauce";// Playwright constants.tsexport const BASE_URL = process.env.BASE_URL || "https://app.example.com";export const USERNAME = process.env.USERNAME || "standard_user";export const PASSWORD = process.env.PASSWORD || "secret_sauce";The spec file imports these constants and passes them to business methods or standalone fill actions, so tests work out of the box with recorded values but can be overridden via environment variables for different environments.
API Wait Mapping
Section titled “API Wait Mapping”Ghost maps HTTP flows to the interactions that triggered them, enabling the test to verify that API calls complete successfully before proceeding.
What this diagram shows:
extractAPIWaits()walks all journey steps chronologically, tracking thestep_orderof the last interaction step seen- When a flow/action step appears, it’s associated with the preceding interaction’s
step_order - Only non-GET API calls are included — GET requests are typically static resource fetches, not user-triggered mutations
- Only paths containing
/api/or/tokenare included — static asset paths are excluded - Only flows with a non-zero status code are included — failed/incomplete requests are skipped
Each mapped API call gets an alias derived from its URL path via buildAPIAlias():
| API Path | Alias |
|---|---|
/api/v1/Tag/preview?onlyOpenedToSale=false | tagPreviewReq |
/api/v1/User/me@example.com/Login | userLoginReq |
/token | tokenReq |
Alias generation strips query strings, skips common prefixes (api, v1, v2), skips ID-like segments (containing @ or long UUID-like strings with -), takes the last 2 meaningful segments, joins them as camelCase, and appends Req. Duplicate aliases get numeric suffixes: tagPreviewReq, tagPreviewReq2.
The path is converted to a glob pattern for cy.intercept(): /api/v1/Tag/preview?onlyOpenedToSale=false → **/api/v1/Tag/preview*.
Cypress spec output (API waits):
// Set up intercepts BEFORE the actioncy.intercept('POST', '**/api/v1/auth/token*').as('authTokenReq');cy.intercept('POST', '**/api/v1/User/Login*').as('userLoginReq');
// Execute the actionloginPage.login(USERNAME, PASSWORD);
// Wait for API calls AFTER the actioncy.wait('@authTokenReq').its('response.statusCode').should('eq', 200);cy.wait('@userLoginReq').its('response.statusCode').should('eq', 200);
// Let React process API responses and re-render// eslint-disable-next-line cypress/no-unnecessary-waitingcy.wait(1000);The intercepts are registered before the action (so Cypress catches the request) and waited on after (so the test verifies the response). The cy.wait(1000) pause after API waits gives React time to process the responses and re-render the UI — without this, Cypress proceeds to the next step before the UI reflects the data (e.g., cards still show skeleton loading).
Playwright POM does not currently generate API intercepts — Playwright’s page.waitForURL() and auto-waiting handle most timing issues natively.
Page Navigation Strategy
Section titled “Page Navigation Strategy”Multi-page journeys (e.g., login page → dashboard) require careful navigation:
- First page: navigated explicitly via
visit()(Cypress) orgoto()(Playwright) - Subsequent pages: the test waits for natural navigation — login redirects, link clicks, form submissions — rather than forcing a
visit()/goto()which would race with async auth/token flows
// Cypress specloginPage.visit(); // First page: explicit navigation// ... login actions ...cy.url().should('include', '/dashboard'); // Second page: wait for redirect// ... dashboard actions ...// Playwright specawait loginPage.goto(); // First page: explicit navigation// ... login actions ...await page.waitForURL('**/dashboard*'); // Second page: wait for redirect// ... dashboard actions ...This prevents a common test failure: the login form submits, POST /api/token is still pending, and the test immediately navigates to /dashboard — but the app hasn’t completed authentication yet and redirects back to /login.
Assertion Patterns (Cypress)
Section titled “Assertion Patterns (Cypress)”Cypress POM tests include three types of assertions to verify each step succeeds:
1. Visibility assertions — before every action:
this.getEmailField().should('be.visible').clear({ force: true }).type(email, { force: true });loginPage.getSubmitButton().should('be.visible').click({ force: true });2. Value assertions — after every fill/clear:
this.getEmailField().should('have.value', email); // After fillthis.getEmailField().should('have.value', ''); // After clear3. API status assertions — after actions that trigger API calls:
cy.wait('@authTokenReq').its('response.statusCode').should('eq', 200);The { force: true } option is added to all interactions to bypass Cypress’s actionability checks (element must be visible, not covered, not disabled). This is necessary because recorded interactions may target elements behind overlays, within scrollable containers, or with CSS transforms that Cypress’s hit-testing doesn’t handle correctly.
Header Filtering
Section titled “Header Filtering”All export formats skip headers that shouldn’t appear in exported code. The full list is organized by category:
| Category | Skipped Headers | Why |
|---|---|---|
| Transport / Hop-by-hop | host, content-length, transfer-encoding, connection, proxy-connection, keep-alive, proxy-authenticate, proxy-authorization, te, trailer, upgrade | Connection-level headers that HTTP clients generate automatically |
| Ghost internal | x-ghost-interaction | Ghost’s journey correlation header — stripped by the proxy before forwarding, should never appear in test code |
| Browser identity | user-agent, sec-ch-ua, sec-ch-ua-mobile, sec-ch-ua-platform, sec-ch-ua-full-version-list | Tests shouldn’t depend on a specific browser’s identity string |
| Fetch metadata | sec-fetch-dest, sec-fetch-mode, sec-fetch-site, sec-fetch-user | Browser security hints, not relevant to API contracts |
| Cache negotiation | if-none-match, if-modified-since, if-match, if-unmodified-since, if-range | Tests should get fresh responses, not cached ones |
| Encoding / Locale | accept-encoding, accept-language | Transport and locale details |
| Navigation context | referer, referrer, origin | Page navigation context that varies between recording and replay |
| Session cookies | cookie | Stale after recording — tests should use auth headers instead |
| Telemetry / Observability | sentry-trace, baggage, traceparent, tracestate, x-request-id, x-correlation-id | Observability headers that are not part of the API contract |
| Privacy | dnt | Do Not Track — not relevant to test code |
Headers are sorted alphabetically in the output for deterministic, diffable results.
Content-Disposition
Section titled “Content-Disposition”Export responses include a Content-Disposition header for browser download:
Content-Disposition: attachment; filename="journey-01HWXYZ.spec.ts"The filename uses the first 8 characters of the journey ID plus the format-specific extension.
Replay Engine
Section titled “Replay Engine”The replay engine re-sends every HTTP flow in a journey and compares the results with the original responses. This is useful for regression testing — “does the same sequence of API calls still return the same results?”
How Replay Works
Section titled “How Replay Works”What this diagram shows:
- The replay endpoint is an SSE (Server-Sent Events) stream — the client opens a long-lived connection and receives events as each step completes
- For each flow in the journey, the engine reconstructs the original HTTP request (method, URL, headers, body) and sends it to the target server
- After receiving the response, the engine compares it with the original response and emits a
step_completeevent with the comparison data - When all steps are done, a final
doneevent is emitted and the stream closes
SSE Event Types
Section titled “SSE Event Types”| Event Type | When Emitted | Payload |
|---|---|---|
step_start | Before each request is sent | step, total, flow_id, method, url |
step_complete | After a response is received | step, total, flow_id, method, url, original, replayed, diff, duration_ms |
step_error | When a step fails (build error, network error) | step, total, error — if a response was received before error, original is included |
done | When all steps finish | total |
Comparison Data
Section titled “Comparison Data”Each step_complete event includes three comparison objects:
Original (from the recorded flow):
| Field | Description |
|---|---|
status_code | Original HTTP status code |
content_type | Original response content type |
body_size | Original response body size in bytes |
duration_ms | Original request duration |
Replayed (from the new request):
| Field | Description |
|---|---|
status_code | New HTTP status code |
content_type | New response content type |
body_size | New response body size in bytes |
duration_ms | New request duration |
Diff (computed comparison):
| Field | Description |
|---|---|
status_changed | true if the status codes differ |
body_size_delta | Replayed body size minus original body size (bytes) |
duration_delta_ms | Replayed duration minus original duration (positive = slower) |
Replay Constraints
Section titled “Replay Constraints”| Constraint | Value | Description |
|---|---|---|
| Max response body | 10 MB | io.LimitReader caps response body reading to prevent memory exhaustion |
| SSRF protection | Uses replayTransport | The HTTP transport is SSRF-safe — prevents replaying requests to internal/private hosts |
| Delay between steps | 0–10,000 ms | Configurable via delay_ms query parameter. Default: 0 (no delay). Values outside 0–10,000 are clamped to the range. |
| Context cancellation | Supported | If the client disconnects or the frontend’s AbortController fires, the replay goroutine exits cleanly |
| Write deadline | 30 seconds | Each SSE event write has a 30-second deadline via http.ResponseController |
| Header filtering | Same as export | Hop-by-hop and internal headers (including X-Ghost-Interaction) are stripped from replayed requests. Headers must be lowercased before checking — the isSkippedHeader() map uses lowercase keys. |
| Steps without flows | Skipped | Only steps with a flow_id and a valid Request object are replayed. Pure interaction steps are silently skipped. |
Frontend Replay UI
Section titled “Frontend Replay UI”The replay is initiated from the journey detail view’s “Replay” button. The frontend:
- Creates an
AbortControllerfor cancellation - Calls
api.streamJourneyReplay()which opens afetchrequest and reads the response as aReadableStream - Parses the SSE text stream line by line, extracting
event:anddata:fields - Each parsed event updates the replay state (
replayEventsarray andreplayProgress) - Results appear inline in the detail view as colored rows:
- Green status comparison: original and replayed status codes match
- Red status comparison: status codes differ
- Yellow/Warning duration delta: replayed was >500ms slower
- Green duration delta: replayed was >500ms faster
- Gray duration delta: within ±500ms
- Clicking “Stop Replay” during a replay aborts the
AbortController, which closes the stream and cancels the server-side goroutine - The
AbortControlleris also cleaned up on component unmount to prevent memory leaks
Agent Integration
Section titled “Agent Integration”Five AI agent tools interact with journeys:
list_journeys
Section titled “list_journeys”| Property | Value |
|---|---|
| Description | Lists all recorded journeys for a session |
| Input | session_id (optional — defaults to active session) |
| Output | JSON array of journey summaries (id, name, status, flow_count, interaction_count, started_at, completed_at) |
| Limit | Returns at most 100 journeys. If more exist, the response includes "truncated": true. |
| Classification | Read-only. Routed to the "recon" phase in the tool router. Safe for parallel execution. |
get_journey_steps
Section titled “get_journey_steps”| Property | Value |
|---|---|
| Description | Get the recorded steps of a journey — interactions (clicks, inputs, submits) and HTTP flows |
| Input | journey_id (required) |
| Output | JSON array of steps with type, selector, element details, page URL, and attributes |
| Limit | Returns at most 500 steps. |
| Classification | Read-only. Routed to the "recon" phase in the tool router. Safe for parallel execution. |
Each step includes the step type (interaction/flow/action), selector, element details, page URL, and element attributes. Use this to understand what the user did before generating tests.
journey_export
Section titled “journey_export”| Property | Value |
|---|---|
| Description | Export a recorded journey as deterministic test code using the same exporter as the Ghost UI |
| Input | journey_id (required), format (required: cypress-ui, cypress-api, playwright-ui, playwright-api, k6, postman, curl, har) |
| Output | Writes the exported file to the agent workspace and returns the full absolute path, file size, and line count |
| Classification | Always available (base tool, not phase-dependent). Uses stable selectors from the recording — no AI hallucination. |
This is the preferred tool for journey-based test generation. It produces reliable, deterministic output using the exact selectors captured during recording. The agent should prefer journey_export over generate_test for journey-based tests, and only use generate_test when the user wants AI-enhanced custom assertions.
For cypress-ui and playwright-ui formats, the output is a multi-file Page Object Model — the agent tool detects the // ===== File: <path> ===== separators and writes each file to the correct path within the workspace directory, creating a proper folder structure (constants.js, pages/*.js, *.cy.js for Cypress; constants.ts, pages/*.ts, *.spec.ts for Playwright).
Files are saved to format-appropriate subdirectories in the workspace:
- Cypress POM tests →
cypress/e2e/<journey-name>/constants.js,pages/*.js,*.cy.js - Playwright POM tests →
playwright/<journey-name>/constants.ts,pages/*.ts,*.spec.ts - Cypress API tests →
cypress/e2e/journey-<id>.api.cy.js - Playwright API tests →
playwright/journey-<id>.api.spec.ts - Other formats →
exports/journey-<id>.<ext>
record_journey
Section titled “record_journey”| Property | Value |
|---|---|
| Description | Start or stop recording a user journey with correlated interactions and flows |
| Input | action (required: "start" or "stop"), name (required when starting) |
| Prerequisite | Browser extension must be connected. Returns error if not. |
| Constraint | Only one recording per session. Starting when one exists returns error. |
| Classification | Mutating. Routed to the tool router’s mutation phase. Cannot run in parallel. |
When the agent starts a recording, it receives: journey ID, name, and status. The response tells the agent to call record_journey action:stop when done.
When the agent stops a recording, it receives: journey ID, name, final flow count, interaction count, and duration.
replay_journey
Section titled “replay_journey”| Property | Value |
|---|---|
| Description | Replay a recorded journey in the browser and optionally compare with the original |
| Input | journey_id (required), compare (optional boolean) |
| Prerequisite | Browser extension must be connected. Returns error if not. |
| Classification | Execution phase. Routed to the tool router’s execution phase (requires approval). |
The replay tool updates the journey status to replaying, executes each interaction step through the browser extension, and returns a summary of what happened.
Persistence
Section titled “Persistence”Journeys are stored in SQLite across two tables, added in migration 5:
journeys table
Section titled “journeys table”| Column | Type | Constraint |
|---|---|---|
id | TEXT | PRIMARY KEY |
session_id | TEXT | NOT NULL, FOREIGN KEY → sessions(id) ON DELETE CASCADE |
name | TEXT | NOT NULL |
status | TEXT | NOT NULL, DEFAULT ‘recording’ |
flow_count | INTEGER | DEFAULT 0 |
interaction_count | INTEGER | DEFAULT 0 |
started_at | DATETIME | NOT NULL |
completed_at | DATETIME | |
created_at | DATETIME | NOT NULL |
Index: idx_journeys_session_id on session_id for fast session lookup.
journey_steps table
Section titled “journey_steps table”| Column | Type | Constraint |
|---|---|---|
id | TEXT | PRIMARY KEY |
journey_id | TEXT | NOT NULL, FOREIGN KEY → journeys(id) ON DELETE CASCADE |
step_order | INTEGER | NOT NULL |
type | TEXT | NOT NULL |
interaction_id | TEXT | FOREIGN KEY → interactions(id) ON DELETE SET NULL |
flow_id | TEXT | FOREIGN KEY → flows(id) ON DELETE SET NULL |
data | TEXT | |
timestamp | DATETIME | NOT NULL |
Index: idx_journey_steps_journey_order on (journey_id, step_order) for fast ordered step retrieval.
A related screenshot_baselines table (migration 6) has an optional journey_id foreign key (SET NULL on delete) for associating visual regression baselines with specific journeys.
Extension Protocol
Section titled “Extension Protocol”Journey recording adds two new message types to the Ghost → Extension WebSocket protocol:
| Message Type | Direction | Payload | Effect |
|---|---|---|---|
journey.recording_start | Ghost → Extension | { journey_id, session_id?, name? } | Extension enables declarativeNetRequest header injection rule (ID 9999), initializes interaction counter |
journey.recording_stop | Ghost → Extension | { journey_id } | Extension removes the declarativeNetRequest rule, resets counter and journey state |
declarativeNetRequest Rule
Section titled “declarativeNetRequest Rule”The extension uses a single dynamic declarativeNetRequest rule that modifies request headers:
{ "id": 9999, "priority": 1, "action": { "type": "modifyHeaders", "requestHeaders": [{ "header": "X-Ghost-Interaction", "operation": "set", "value": "<interaction-id>" }] }, "condition": { "urlFilter": "*", "resourceTypes": ["xmlhttprequest", "main_frame", "sub_frame"] }}Key implementation details:
- Rule ID 9999 — a fixed ID so the rule can be updated in place (remove old + add new in a single
updateDynamicRulescall) xmlhttprequestcovers both XHR andfetch()in Chrome’s API — there is no separatefetchresource typemain_frameandsub_frameare included to catch page navigations and iframe loads- Interaction filtering —
hoverandfocusevents do NOT update the correlation header (they’re too noisy). Onlyclick,change, andsubmittrigger correlation.
Correlation Window Timing
Section titled “Correlation Window Timing”Time ──────────────────────────────────────────────────────► │ │ │ User clicks button │ │ ├──── Extension updates header with unique ID │ │ │ (journeyId-counter-timestamp) │ │ │ │ │ │ ← 600ms correlation window → │ │ │ All requests in this window get the unique ID │ │ │ and become "action" steps │ │ │ │ │ └──── Extension resets to ambient ID │ │ (journeyId-ambient-timestamp) │ │ Requests become standalone "flow" steps │The 600ms extension window is deliberately longer than the 500ms backend correlation window to ensure coverage. The extension doesn’t know exactly when the backend will process the flow — the extra 100ms provides margin.
API Reference
Section titled “API Reference”Journey CRUD
Section titled “Journey CRUD”| Method | Path | Description |
|---|---|---|
GET | /api/v1/journeys?session_id=X | List all journeys for a session (newest first). Returns 400 if session_id is empty. Always returns an array (empty if none). |
POST | /api/v1/journeys | Create a journey. Body: {"session_id": "...", "name": "..."}. Both fields required. Returns 201. Request body capped at 1 MB. |
GET | /api/v1/journeys/{id} | Get a single journey. Returns 404 if not found. |
PUT | /api/v1/journeys/{id} | Update journey name. Returns 501 (not yet implemented). |
DELETE | /api/v1/journeys/{id} | Delete journey and all steps (CASCADE). Returns 204. Broadcasts journey:deleted via WebSocket. |
GET | /api/v1/journeys/{id}/steps | List all steps for a journey, ordered by step_order ascending. Always returns an array. |
Recording Control
Section titled “Recording Control”| Method | Path | Description |
|---|---|---|
POST | /api/v1/journeys/start | Start recording. Body: {"session_id": "...", "name": "..."}. Requires extension connected (400 if not). Returns 409 if session already has an active recording. Sends journey.recording_start to extension. |
POST | /api/v1/journeys/stop | Stop recording. Body: {"session_id": "..."}. Returns 404 if no active recording. Sends journey.recording_stop to extension. |
GET | /api/v1/journeys/active?session_id=X | Check if session has an active recording. Returns {"is_recording": true/false, "journey_id": "...", "session_id": "..."}. |
POST | /api/v1/journeys/{id}/complete | Manually complete a journey (e.g., recovering a failed recording). Updates counts and status. |
Export
Section titled “Export”GET /api/v1/journeys/{id}/export?format=playwright| Parameter | Type | Required | Values |
|---|---|---|---|
format | string | Yes | curl, postman, playwright-api, playwright-ui, cypress-api, cypress-ui, k6, har |
Responses:
- 200 — Export file with
Content-Disposition: attachmentheader - 400 — Missing or invalid format
- 404 — Journey not found
- 422 — Journey has no exportable flows
Replay (SSE Stream)
Section titled “Replay (SSE Stream)”GET /api/v1/journeys/{id}/replay?delay_ms=500| Parameter | Type | Required | Range | Default |
|---|---|---|---|---|
delay_ms | integer | No | 0–10,000 | 0 |
Response: SSE stream with Content-Type: text/event-stream. Events: step_start, step_complete, step_error, done.
Headers set:
Content-Type: text/event-streamCache-Control: no-cacheConnection: keep-aliveX-Accel-Buffering: no(prevents Nginx buffering)
WebSocket Events
Section titled “WebSocket Events”| Event | Payload | When |
|---|---|---|
journey:started | { journey_id, session_id, name } | Recording begins |
journey:step_added | { journey_id, step_order, type, flow_id?, interaction_id? } | New step added during recording |
journey:completed | { journey_id, flow_count, interaction_count, duration_ms } | Recording finishes |
journey:deleted | { journey_id } | Journey is deleted |
Use Cases
Section titled “Use Cases”Document regression test paths — Record a journey through your application’s critical flows (login, checkout, profile update). The journey captures exactly which buttons were clicked, which forms were filled, and which API calls were made. Export as Playwright or Cypress test code for your CI pipeline.
Understand UI-to-API mapping — A single button click might trigger 5 API calls. Journey recording’s correlation makes the mapping explicit: “When I click ‘Submit Order’, which API call is the one that’s failing?” The action step type connects the click directly to its POST request.
Detect API regressions — Record a journey in staging, then replay it after a deployment. The replay engine compares status codes and response sizes — if /api/checkout returned 200 last week but returns 500 now, you see it immediately as a red status comparison row.
Generate load test scripts from real traffic — Record a user flow, export as k6. You get a load test that replays actual API calls with real headers and bodies — not hand-crafted synthetic requests that might miss parameters.
Capture exploratory testing sessions — QA engineers record their exploration of new features as named journeys. The AI agent queries these journeys (list_journeys) to understand what was tested, identify gaps in coverage, or generate test scenarios based on actual user behavior patterns.
Share reproducible steps — Export a journey as cURL commands or a Postman collection. Share the file with a developer — they can reproduce the exact sequence of API calls that triggered the bug, complete with headers and request bodies.
Compare environments — Record the same journey in staging vs production. Compare the captured flows to see if the same user actions produce different API calls, different response times, or different error rates.