> For the complete documentation index, see [llms.txt](https://docs.promptshuttle.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.promptshuttle.com/platform/analytics.md).

# Invocation Log & Analytics

PromptShuttle provides detailed logging of every request and three levels of analytics aggregation for understanding cost, performance, and usage patterns.

## Invocation log

Every request is logged with timing, cost, and metadata. Full prompt and answer text, original input sequences, and template parameter values, are kept by default. A tenant can reduce that period or keep text for only a sample of runs. Set this on Tenant Profile under Request / response text. Cost, model and status stay in all cases. Debug runs always keep full text. Already-stored runs are stripped on the next cleanup pass. The run row itself stays for the platform request-retention window.

### Browse the log

```
GET /api/v1/llm-logs
```

### Filters

| Parameter         | Type         | Description                                         |
| ----------------- | ------------ | --------------------------------------------------- |
| `cursor`          | string       | Pagination cursor (from previous response)          |
| `pageSize`        | integer      | Results per page (default 50)                       |
| `from`            | datetime     | Start time filter                                   |
| `to`              | datetime     | End time filter                                     |
| `environments`    | string array | Filter by environment name                          |
| `flows`           | string array | Filter by flow name                                 |
| `statuses`        | string array | Filter by status (`Pending`, `Succeeded`, `Failed`) |
| `tags`            | string array | Filter by tags (any match)                          |
| `customerId`      | string       | Filter by end-customer external ID                  |
| `rootOnly`        | boolean      | Show only root-level requests (exclude sub-agents)  |
| `parentRequestId` | string       | Show children of a specific parent request          |

### Log entry fields

Each log entry includes:

* **Request details** — Messages, model, parameters, environment, tags
* **Response details** — Text output, tool calls, citations, finish reason
* **Tool calls** — For each call, the endpoint it was actually placed against and where that origin came from: the tool definition, an environment binding, or the caller's [`X-Shuttle-Callback-Url`](/tools/caller-hosted-tools.md)
* **Usage** — Tokens in/out, reasoning tokens, cost in credits and USD
* **Timing** — Duration in milliseconds
* **Hierarchy** — Parent/root request IDs, agent depth, agent role
* **Customer** — End-customer ID and external ID
* **Feedback** — Feedback score if submitted
* **Detail dropped** — `detailDropped` is true when the tenant's sample rate skipped persisting this run's prompt/response text
* **Cancellation** — `cancelRequestedAt` / `cancelledAt` / `cancelRequestedBy`, if the run was stopped
* **Progress** — `progressScore` / `progressCertainty` / `minProgressScore`, if the run opted into [verification](/platform/run-verification.md). Absent means unscored, which is not the same as a score of zero

### Stopping a run

Runs still executing show as **running** in the log, and the list refreshes on its own while any of them are on screen. Each has a stop control, as does the run detail — stopping a run stops every sub-agent beneath it. When anything is running, a **Stop everything running** button appears above the list; it stops every run in the workspace, including ones started by other people and by your applications.

Stopping is not instant: a run notices at its next step, so a model call already in flight finishes first. What it had produced by then is kept and still billed — stopping saves the work that had not happened yet. A stopped run shows a grey **cancelled** badge and carries a `Cancelled` warning explaining how far it got.

To find stopped runs, filter on the `Cancelled` flag; the status filter cannot express it, because a run that was stopped after producing something still records `Succeeded`. The dashboard's **Running now** counter links straight to the runs it counts.

The same controls are available over the API ([`POST /api/v1/runs/{runId}/cancel`](/api-reference/flow-execution.md#stop-a-running-run)) and over MCP (`cancel_run`, `cancel_all_runs`).

### Request hierarchy

For multi-agent requests, view the full hierarchy:

```
GET /api/v1/llm-logs/{id}/hierarchy
```

Returns:

* **Breadcrumb** — Ancestor path from root to current request
* **Parent** — Direct parent summary
* **Children** — Direct child agent summaries

### Streaming events

View the SSE events emitted during a request:

```
GET /api/v1/llm-logs/{id}/events
```

Filter by event type: `?types=agentStarted,agentCompleted`

Get event type counts: `GET /api/v1/llm-logs/{id}/events/stats`

### Log entries (debug)

For requests with elevated log levels, view detailed internal log entries:

```
GET /api/v1/llm-logs/{id}/log-entries
```

## Analytics

PromptShuttle provides analytics at three levels of granularity, each answering different questions.

### Level 1: Per-inference stats

**What it answers:** How are individual LLM calls performing?

```
GET /api/v1/statistics/flow/{flowName}
```

Metrics per model:

* Average tokens in/out
* Average cost and latency
* Percentiles (P50, P90, P95, P99) for tokens, latency, and cost
* Token and latency histograms

### Level 2: Per-request stats

**What it answers:** How are complete requests performing (including tool-calling loops)?

```
GET /api/v1/statistics/flow/{flowName}/request
```

Metrics:

* Total cost per request (across all LLM calls and tool invocations)
* Total function calls per request
* Request latency distribution
* Broken down by primary model

### Level 3: Per-tree stats (multi-agent)

**What it answers:** How are entire agent trees performing?

```
GET /api/v1/statistics/flow/{flowName}/tree
```

Metrics:

* Total tree cost (root + all sub-agents)
* Agent count per tree
* Maximum depth reached
* Depth distribution
* Agent count distribution

### All three levels at once

```
GET /api/v1/statistics/flow/{flowName}/extended
```

Returns all three levels plus time series data for invocations and cost.

### Tenant-wide statistics

```
GET /api/v1/statistics
```

Query parameters:

| Parameter    | Type     | Description                                                      |
| ------------ | -------- | ---------------------------------------------------------------- |
| `period`     | TimeSpan | Time window (e.g. `7.00:00:00` for 7 days)                       |
| `resolution` | string   | Time bucket size: `Minutes5`, `Minutes15`, `Hour`, `Day`, `Week` |

Returns:

* Invocation count (current vs. previous period)
* Per-flow invocation time series
* Cost time series
* Token usage time series
* Top flows by invocation count
* Credit usage comparison

### Account overview

```
GET /api/v1/statistics/overview
```

Returns cost and request counts broken down by model and by flow, with percentage distribution.

### Cost by model over time

```
GET /api/v1/statistics/model-costs?period=90.00:00:00
GET /api/v1/statistics/model-costs?period=90.00:00:00&flowName=report_agent
```

Answers "did switching models actually cut unit cost". One cell per time bucket × served model: call count, billed cost, raw provider cost, and average billed cost per call. Buckets are hour / day / week from the period length (the same rule as the dashboard). Reads the hourly usage rollup, so a 180-day window stays cheap.

The Usage page and each flow's Analysis tab render this as a stacked bar plus a heatmap table (total cost, avg $/call, or calls). Long Analysis windows (90d / 180d) show only this map — the percentile histograms still scan every request and do not survive that range.

Hours from before the live rollup writer started are filled in once from stored inference rows. Embedding traffic has no per-request history and is not reconstructed.

### Cost per tag

```
GET /api/v1/statistics/tag-costs?tag=TenderId:12313
```

Answers "what did this unit of work cost". If your application stamps its own correlation tag on every request — an order id, a document id, a tender id — this returns that tag's total spend as a **flow × model matrix**: how many LLM calls each flow made against each model, and what they cost.

Because a tagged job can run for weeks and the model behind a flow may change during it, the breakdown keys on the model that *actually served* each call. A model swapped mid-job simply shows up as a second column rather than silently replacing the first.

Accepts API keys as well as session tokens, so your own application can display the figure.

| Parameter  | Type     | Description                                                             |
| ---------- | -------- | ----------------------------------------------------------------------- |
| `tag`      | string   | **Required.** The exact tag, e.g. `TenderId:12313`. Not a prefix match. |
| `grouping` | string   | `split` (default) or `rollup` — see below                               |
| `from`     | DateTime | Optional start. Unbounded by default                                    |
| `to`       | DateTime | Optional end                                                            |

Sub-agent requests inherit their parent's tags, so one tag covers an entire agent tree. `grouping` decides how that tree is presented — the totals are identical either way:

* `split` — one row per flow that actually ran, so a sub-agent's own flow is visible with its own model and cost. Use this to find which agent is expensive.
* `rollup` — sub-agent spend folds into the entry-point flow you invoked. Use this to answer "what did each feature cost me".

Each cell reports `costUsd` (billed), `rawCostUsd` (provider cost before markup), `calls`, `requests`, token counts, and `pricingUncertain` — the last flags a cell whose served model is not in the model catalogue, so its price is an estimate.

```
GET /api/v1/statistics/tag-costs/by-key?prefix=TenderId:
```

One row per distinct tag under a key prefix, ranked by spend — "which tender cost the most". Only tags starting with the prefix are reported, which also keeps PromptShuttle's own internal marker tags (`Agent:`, `Panel`, `DebugUrl:`) out of the result.

| Parameter | Type     | Description                                                          |
| --------- | -------- | -------------------------------------------------------------------- |
| `prefix`  | string   | **Required.** Anchored prefix, e.g. `TenderId:`                      |
| `from`    | DateTime | Optional start. Defaults to 90 days ago; the window is capped at 365 |
| `to`      | DateTime | Optional end                                                         |
| `limit`   | int      | Rows to return, 1–200 (default 50). `truncated` says if more exist   |

Both are also available in the UI under **Tag Costs**.

**Caveats.** Panel deliberations run as their own top-level requests with fixed tags, so they do not inherit a caller tag and will not appear in a tag report.

## Data export

Export your flows and tools for backup or migration:

```
GET /api/v1/export
```

Returns all flows (with all versions and templates) and tools in a single JSON document.

Import into another tenant:

```
POST /api/v1/export/import
```

Duplicate flows and tools are skipped (matched by name).