> For the complete documentation index, see [llms.txt](https://docs.promptshuttle.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.promptshuttle.com/api-reference/streaming.md).

# Streaming (SSE)

PromptShuttle supports Server-Sent Events for real-time visibility into multi-agent execution. When you enable streaming, you get a structured event stream covering the full lifecycle of your request — from agent starts through tool calls to final completion.

A stream can also carry the answer text while the model writes it. See [Answer text](#answer-text).

Two endpoints can stream:

| Endpoint                        | Answer text in the stream                           |
| ------------------------------- | --------------------------------------------------- |
| `POST /api/v1/inference`        | On. Set `content_deltas` to `false` to turn it off. |
| `POST /api/v1/chat/completions` | Off. Set `content_deltas` to `true` to turn it on.  |

Both endpoints send the same event format.

## Enabling streaming

Set `stream` to `true` on the inference endpoint:

```bash
curl -N -X POST https://app.promptshuttle.com/api/v1/inference \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [{"role": "client", "text": "Explain trust in two sentences."}],
    "stream": true
  }'
```

Or set `stream: true` on the OpenAI-compatible endpoint:

```bash
curl -N -X POST https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": [{"type": "text", "text": "Research quantum computing"}]}],
    "stream": true,
    "stream_options": {
      "include_usage": true,
      "usage_interval_ms": 3000
    }
  }'
```

## Response format

The response uses standard SSE format with `Content-Type: text/event-stream`:

```
id: 67a1b2c3d4e5f6a7b8c9d0e1
data: {"id":"...","timestamp":"...","requestId":"...","type":"requestStarted","data":{...}}

id: 67a1b2c3d4e5f6a7b8c9d0e2
data: {"id":"...","timestamp":"...","requestId":"...","type":"agentInferenceStarted","data":{...}}

...

data: [DONE]
```

Each event is a JSON object:

| Field       | Type     | Description                                                                        |
| ----------- | -------- | ---------------------------------------------------------------------------------- |
| `id`        | string   | Unique event ID                                                                    |
| `timestamp` | datetime | UTC ISO 8601 with milliseconds                                                     |
| `requestId` | string   | Root request ID                                                                    |
| `type`      | string   | Event type (see below)                                                             |
| `agentPath` | array    | Breadcrumb of agent roles from root to current (e.g. `["main", "research_agent"]`) |
| `depth`     | integer  | Nesting depth in the agent tree                                                    |
| `data`      | object   | Event-specific payload                                                             |

The stream ends with `data: [DONE]`.

The response includes the header `X-Request-Id` with the root request ID.

If the client disconnects, the run does not stop. It finishes, it is stored, and it is billed. Use the request ID to read the result later. To stop a run, use [`POST /api/v1/runs/{runId}/cancel`](https://docs.promptshuttle.com/api-reference/flow-execution.md#stop-a-running-run).

## Answer text

The stream sends the answer text in `contentDelta` events while the model writes it.

Follow these two rules:

1. When you receive `contentDelta`, append `data.text` to the text that you show.
2. When you receive `contentReset`, clear the text that you show.

If you follow the rules, the text that you show at the end is the same as `textResponse` in the final `requestCompleted` event. The final event is always correct. Use it as the stored result.

### `contentDelta`

```json
{
  "text": "Trust grows ",
  "inferenceRequestId": "67a1b2c3d4e5f6a7b8c9d0e5"
}
```

One event can contain more than one token. The server sends a maximum of approximately 30 of these events each second.

`inferenceRequestId` identifies the model call that wrote the text. You do not need it to show the correct text.

### `contentReset`

```json
{
  "reason": "nextInference",
  "inferenceRequestId": "67a1b2c3d4e5f6a7b8c9d0e6"
}
```

The text sent before this event is not the answer. The values of `reason` are:

| `reason`        | Cause                                                                                                |
| --------------- | ---------------------------------------------------------------------------------------------------- |
| `attemptFailed` | The model call failed after it sent text. PromptShuttle tries again, possibly with a fallback model. |
| `nextInference` | The run makes another model call. For example, the model wrote some words and then called a tool.    |
| `corrected`     | The complete response is different from the streamed text. The correct text follows in one event.    |

Do the same thing for each value: clear the text. New values can be added.

### Tool calls that your application runs

If you send `injectedTools` to `/api/v1/inference`, PromptShuttle returns the tool calls to you. Words that the model wrote before the tool call arrive as `contentDelta` events. They are also in `textResponse` of the final event, together with `toolCalls`. PromptShuttle does not send `contentReset` for them. Your application decides whether to show these words.

When you send the next request of your tool loop, send the turn of the model back as one message: put the words in `text` and the tool calls in `content`. Then add the message with the tool results. PromptShuttle sends both parts of the turn to the model. This applies with and without a stream.

#### Lower the cost of a tool loop on Claude models

Each request of your tool loop sends the complete history again, with the tool results of the rounds before. On Claude models, you can make the next round read this history from the prompt cache of the provider. Set `cache` to `true` on the last message of the request. This is usually the message with the tool results.

* The provider writes the history up to that message to the cache. The next request that starts with the same messages reads it at the cache price.
* Move the flag to the new last message in each round.
* The written part costs 1.25 times the input price. Do not set the flag on a request that no later request continues in the next 5 minutes.
* The response shows the result in `usage`: `cacheCreationTokens` (written) and `cachedTokensIn` (read).
* A request can have 4 cache points. PromptShuttle uses up to 2 for the system prompt and the tools. If more messages have the flag than fit, the last messages get the cache points.
* The other providers cache without this flag and ignore it.

#### `claude-sonnet-5-5` after a tool result

When the history ends with a tool result, `claude-sonnet-5-5` does not send the first part of a long answer while it writes it. The first text arrives after 7 to 9 seconds, in one large event. After that, the text streams normally. Thus the first words come later than from `claude-sonnet-5` or `claude-haiku-5-5` (about 1 second), but the complete answer is not later.

This is how the model works at the provider. It is the same with each thinking setting, and PromptShuttle cannot change it.

Until the provider releases the text, the tokens count as thinking. If `maxTokens` stops the answer before that point, the response has no text: `textResponse` is empty, `finishReason` is `stopped`, and `usage.reasoningTokens` shows the tokens that were used. In our measurement, 700 was too low and 2,500 was sufficient. Use a `maxTokens` of 2,000 or more for an answer after a tool result on this model.

### When the text arrives in one event

In these cases the stream sends the complete text in one `contentDelta` event, when the model call is complete:

* The request has a response schema (structured output).
* The response comes from the response cache.
* The provider of the model does not have token streaming in PromptShuttle. At this time, these models stream while they write: Anthropic, OpenAI, xAI, DeepSeek, and Google Gemini 3 and later. Gemini sends a sentence or two in each event, not single words.

Your client does not need a special case for this. The two rules are sufficient.

### Storage

`contentDelta` and `contentReset` are not stored. [`GET /api/v1/llm-logs/{id}/events`](https://docs.promptshuttle.com/platform/analytics.md) does not return them. The answer text is stored one time, with the response.

The stored events are a timeline. They do not contain the text of the run:

* A stored `requestCompleted` event does not have `result`. The stream has it.
* In all stored events, text longer than 500 characters is cut to 500 characters. This applies to tool arguments, to the input of a sub-agent and to messages. The stream sends the complete text.

## Event types

### Lifecycle events

#### `requestStarted`

Emitted when the request begins processing.

```json
{
  "flowName": "my_flow",
  "contextId": "flow_object_id",
  "model": "gpt-4o",
  "hasTools": true,
  "toolCount": 3
}
```

#### `requestCompleted`

Emitted when the entire request (including all agents) finishes.

```json
{
  "durationMs": 5420,
  "totalCreditsUsed": 8200,
  "totalCostUsd": 0.0082,
  "totalTokensIn": 1250,
  "totalTokensOut": 890,
  "totalInferenceCount": 4,
  "totalToolCalls": 2,
  "totalAgentSpawns": 1,
  "maxDepthReached": 1,
  "result": { "textResponse": "..." }
}
```

The shape of `result` depends on the endpoint:

* On `/api/v1/chat/completions`, `result` is a `chat.completion` object.
* On `/api/v1/inference`, `result` is the same object as the response body without a stream: `id`, `creditsUsed`, `creditsLeft`, `milliseconds` and `result` (with `textResponse` and `toolCalls`).

#### `requestFailed`

Emitted if the request fails fatally.

#### Cancellation

A run stopped with [`POST /api/v1/runs/{runId}/cancel`](https://docs.promptshuttle.com/api-reference/flow-execution.md#stop-a-running-run) emits an `error` event with `code: "RUN_CANCELLED"`, then finishes the stream normally — `requestCompleted` if it had produced anything, `requestFailed` if it had not. There is no separate terminal event, so existing clients need no changes: they see the run end where they already expect it to.

### Agent events

#### `agentStarted`

An agent (sub-template) begins execution.

```json
{
  "agentRole": "research_agent",
  "templateId": "template_object_id",
  "templateName": "research",
  "parentRequestId": "parent_id",
  "childRequestId": "child_id",
  "parameters": { "topic": "quantum computing" }
}
```

#### `agentInferenceStarted`

An LLM call begins within an agent.

```json
{
  "inferenceRequestId": "inference_id",
  "model": "gpt-4o",
  "provider": "openai",
  "messageCount": 5,
  "hasTools": true,
  "toolCount": 2
}
```

#### `agentInferenceCompleted`

An LLM call finishes.

```json
{
  "inferenceRequestId": "inference_id",
  "model": "gpt-4o",
  "provider": "openai",
  "durationMs": 1200,
  "usage": {
    "tokensIn": 450,
    "tokensOut": 200,
    "reasoningTokens": 0,
    "costCredits": 650,
    "costUsd": 0.00065
  },
  "finishReason": "stop",
  "toolCallCount": 0,
  "wasCached": false,
  "wasFallback": false
}
```

#### `agentCompleted`

An agent finishes all its work.

```json
{
  "childRequestId": "child_id",
  "agentRole": "research_agent",
  "durationMs": 3200,
  "totalCreditsUsed": 4700,
  "directCreditsUsed": 4700,
  "inferenceCount": 2,
  "toolCallCount": 1,
  "childAgentCount": 0,
  "status": "completed",
  "resultPreview": "Based on my research..."
}
```

#### `agentFailed`

An agent encounters an error.

### Tool events

#### `toolStarted`

A tool is about to be invoked.

```json
{
  "toolName": "search_api",
  "toolType": "external",
  "callId": "call_abc123",
  "arguments": { "query": "quantum computing breakthroughs 2025" },
  "targetUrl": "https://api.example.com/search",
  "originSource": "tool"
}
```

For external tools, `targetUrl` is the endpoint the call is actually placed against — after environment bindings and any caller-supplied callback origin, not the URL stored on the definition. `originSource` says which of those won:

| `originSource` | The endpoint came from                                                                                      |
| -------------- | ----------------------------------------------------------------------------------------------------------- |
| `tool`         | The tool's base `webUrl`                                                                                    |
| `environment`  | An environment override for this run's environment                                                          |
| `callback`     | The caller's [`X-Shuttle-Callback-Url`](https://docs.promptshuttle.com/tools/caller-hosted-tools.md) header |

#### `toolCompleted`

A tool invocation finishes.

```json
{
  "toolName": "search_api",
  "toolType": "external",
  "callId": "call_abc123",
  "durationMs": 450,
  "status": "succeeded",
  "resultPreview": "Found 15 results for...",
  "resultSize": 4200,
  "targetUrl": "https://api.example.com/search",
  "originSource": "tool"
}
```

For agent-type tools, also includes:

```json
{
  "childRequestId": "spawned_request_id",
  "childCreditsUsed": 2300
}
```

#### `toolFailed`

A tool invocation errors. Same shape as `toolCompleted` with `"status": "failed"` and the error in `resultPreview`. `targetUrl` and `originSource` are repeated here so a failure is self-describing — a call that could not be bound to a host at all has no successful `toolStarted` to read them from.

### Usage events

#### `usageUpdate`

Periodic cost and progress updates (interval controlled by `usage_interval_ms`).

```json
{
  "cumulativeCreditsUsed": 3400,
  "cumulativeCostUsd": 0.0034,
  "cumulativeTokensIn": 800,
  "cumulativeTokensOut": 350,
  "activeAgents": 1,
  "completedAgents": 2,
  "elapsedMs": 4500
}
```

### System events

#### `heartbeat`

Keep-alive sent at `heartbeat_interval_ms` intervals.

```json
{
  "elapsedMs": 30000,
  "eventCount": 12
}
```

#### `error`

A recoverable or non-recoverable error occurred.

```json
{
  "code": "COST_LIMIT_EXCEEDED",
  "message": "Request exceeded the maximum cost of 50000 credits",
  "agentRole": "research_agent",
  "recoverable": false
}
```

## Filtering events

> Not available at this time. The server accepts `event_types` and sends all event types.

Use `event_types` in stream options to receive only the events you need:

```json
{
  "stream": true,
  "stream_options": {
    "event_types": ["requestCompleted", "error", "usageUpdate"]
  }
}
```

## Client example: answer text

```python
import json
import httpx

text = ""

with httpx.stream(
    "POST",
    "https://app.promptshuttle.com/api/v1/inference",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "claude-sonnet-5",
        "messages": [{"role": "client", "text": "Explain trust in two sentences."}],
        "stream": True,
    },
    timeout=None,
) as response:
    for line in response.iter_lines():
        if not line.startswith("data: "):
            continue
        payload = line[6:]
        if payload == "[DONE]":
            break
        event = json.loads(payload)
        if event["type"] == "contentDelta":
            text += event["data"]["text"]          # rule 1: append
        elif event["type"] == "contentReset":
            text = ""                              # rule 2: clear
        elif event["type"] == "requestCompleted":
            result = event["data"]["result"]
            text = result["result"]["textResponse"]  # the stored result
            run_id = result["id"]
        elif event["type"] in ("requestFailed", "error"):
            raise RuntimeError(event["data"])
```

## Client example: all events

```python
import json
import httpx

with httpx.stream(
    "POST",
    "https://app.promptshuttle.com/api/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "openai/gpt-4o",
        "messages": [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}],
        "stream": True,
    },
) as response:
    for line in response.iter_lines():
        if line.startswith("data: "):
            payload = line[6:]
            if payload == "[DONE]":
                break
            event = json.loads(payload)
            print(f"[{event['type']}] {json.dumps(event['data'], indent=2)}")
```
