# Welcome

PromptShuttle is an LLM orchestration platform that sits between your application and the major LLM providers. It gives you a single API to manage prompts, route across models, run multi-step agentic workflows, and track costs — all with multi-tenant isolation out of the box.

## Why PromptShuttle?

**One API, every provider.** Write your integration once and route to OpenAI, Anthropic, Google Gemini, Groq, DeepSeek, xAI, or Perplexity. Switch models without changing client code.

**Prompt versioning and environments.** Manage prompt templates with version control. Promote versions across dev, staging, and production environments without redeploying your app.

**Multi-agent orchestration.** Build DAG-based workflows where templates can invoke other templates as sub-agents, with full execution tracing, cost tracking, and depth limits.

**Drop-in OpenAI replacement.** Use the OpenAI-compatible endpoint with any existing OpenAI SDK client — just change the base URL and API key.

**Built-in cost control.** Per-request cost ceilings, tenant-level budgets, detailed usage analytics at three levels of granularity (per-inference, per-request, per-agent-tree).

**Function calling and tools.** Define external HTTP tools, MCP server tools, or agent tools. PromptShuttle handles the tool-calling loop automatically.

## Quick links

| I want to...                          | Go to                                                          |
| ------------------------------------- | -------------------------------------------------------------- |
| Make my first API call                | [Quickstart](/getting-started/quickstart)                      |
| Use my existing OpenAI SDK            | [OpenAI-Compatible Endpoint](/api-reference/openai-compatible) |
| Build a prompt template with versions | [Flows](/flows/flows)                                          |
| Execute a flow via API                | [Flow Execution API](/api-reference/flow-execution)            |
| Add function calling / tools          | [Tools & Function Calling](/tools/tools)                       |
| Serve tools from my own app           | [Caller-Hosted Tools](/tools/caller-hosted-tools)              |
| Connect an MCP server                 | [MCP Server Integration](/tools/mcp-servers)                   |
| Set up model aliases and fallbacks    | [Model Routing](/platform/model-routing)                       |
| Stream agent execution events         | [Streaming (SSE)](/api-reference/streaming)                    |
| Track per-customer usage              | [Customers API](/api-reference/customers)                      |
| Understand costs and credits          | [Billing & Credits](/platform/billing)                         |
| Browse request history                | [Invocation Log & Analytics](/platform/analytics)              |
| Understand the core model             | [Key Concepts](/getting-started/key-concepts)                  |

## Supported providers

| Provider   | Example models                                   |
| ---------- | ------------------------------------------------ |
| OpenAI     | GPT-4o, GPT-4o mini, o1, o3                      |
| Anthropic  | Claude 4 Opus, Claude 4 Sonnet, Claude 3.5 Haiku |
| Google     | Gemini 2.5 Pro, Gemini 2.5 Flash                 |
| Groq       | Llama, Mixtral (fast inference)                  |
| DeepSeek   | DeepSeek Chat, DeepSeek Reasoner                 |
| xAI        | Grok                                             |
| Perplexity | Sonar (web-search integrated)                    |

List all models and pricing programmatically via [`GET /api/v1/models/descriptors`](/api-reference/openai-compatible).

## Base URL

```
https://app.promptshuttle.com/api/v1
```


# Quickstart

Get a response from an LLM in under two minutes.

## Prerequisites

* A PromptShuttle account at [app.promptshuttle.com](https://app.promptshuttle.com)
* An API key (see [Authentication](/getting-started/authentication))

## Option 1: OpenAI-compatible endpoint

If you already use the OpenAI SDK, just swap the base URL and key:

{% tabs %}
{% tab title="curl" %}

```bash
curl https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "Say hello!"}]}
    ]
  }'
```

{% endtab %}

{% tab title="Python (OpenAI SDK)" %}

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://app.promptshuttle.com/api/v1",
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Say hello!"}],
)

print(response.choices[0].message.content)
```

{% endtab %}

{% tab title="TypeScript (OpenAI SDK)" %}

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://app.promptshuttle.com/api/v1",
  apiKey: "YOUR_API_KEY",
});

const response = await client.chat.completions.create({
  model: "openai/gpt-4o",
  messages: [{ role: "user", content: "Say hello!" }],
});

console.log(response.choices[0].message.content);
```

{% endtab %}
{% endtabs %}

The response follows the standard OpenAI chat completion format. See the [full endpoint reference](/api-reference/openai-compatible) for all options.

## Option 2: Create and run a flow

Flows let you version your prompts, add parameters, and route across environments.

### 1. Create a flow

In the [PromptShuttle UI](https://app.promptshuttle.com), click **Flows > Create Flow**. Give it a title (e.g. "Product Description Generator") — a slug name is auto-generated.

### 2. Edit the template

In the flow editor, write your prompt template:

```
You are a product copywriter. Write a compelling product description
for [[product_name]]. The tone should be [[tone]].
Keep it under 100 words.
```

Parameters use double square brackets: `[[parameter_name]]`. PromptShuttle auto-detects them.

Select a model (e.g. `openai/gpt-4o`) and save.

### 3. Activate for an environment

Go to the flow's environment settings and activate your version for an environment (e.g. `production`).

### 4. Execute via API

```bash
curl -X POST https://app.promptshuttle.com/api/v1/flows/product_description_generator/runs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "parameters": {
      "product_name": "Wireless Noise-Cancelling Headphones",
      "tone": "professional yet approachable"
    },
    "environment": "production"
  }'
```

The response includes the LLM output, token usage, cost, and any warnings about unresolved parameters. See [Flow Execution API](/api-reference/flow-execution) for the full reference.

## Next steps

* [Authentication](/getting-started/authentication) — API keys and bearer tokens
* [Key Concepts](/getting-started/key-concepts) — Flows, templates, tools, environments
* [OpenAI-Compatible Endpoint](/api-reference/openai-compatible) — Full reference for the drop-in endpoint
* [Streaming](/api-reference/streaming) — Real-time SSE events for agent execution


# Authentication

All API requests require authentication via the `Authorization` header.

## API keys

API keys are the recommended authentication method for server-to-server integrations.

### Creating an API key

1. Log in to [app.promptshuttle.com](https://app.promptshuttle.com)
2. Navigate to your **Tenant Settings**
3. Under **API Keys**, click **Create API Key**
4. Copy the key — it is only shown once

### Using an API key

Pass your API key as a Bearer token:

```bash
curl https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer ps_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{ ... }'
```

API keys are scoped to your tenant. All requests made with a key are billed to that tenant and inherit its model routing rules, cost limits, and environment configurations.

## Bearer tokens (JWT)

The web UI uses JWT bearer tokens obtained via the login endpoint. These are primarily for interactive sessions and are not recommended for programmatic access — use API keys instead.

## Authentication errors

| Status code | Meaning                              |
| ----------- | ------------------------------------ |
| `401`       | Missing or invalid API key / token   |
| `429`       | Rate limit exceeded (login endpoint) |

## End-customer tracking

When making requests on behalf of your own users, pass a customer identifier to attribute usage:

```bash
# Via header (works on all endpoints)
curl ... -H "X-Shuttle-Customer-Id: customer_123"

# Via request body (flow execution only)
curl ... -d '{ "customerId": "customer_123", ... }'
```

PromptShuttle auto-creates customer records on first use. You can manage customers via the [Customers API](/getting-started/authentication) and view per-customer usage in the dashboard.


# Key Concepts

## Flows

A **flow** is the central unit of work in PromptShuttle. It groups versioned prompt templates, model configurations, and tool assignments into a single deployable unit.

Each flow has:

* A **slug name** (e.g. `product_description`) used in API URLs
* A **title** for display in the UI
* One or more **versions**, each containing prompt templates
* **Active versions** pinned to environments (e.g. version 3 is active in `production`)

## Templates

A **prompt template** is a prompt with parameter placeholders. Each flow version contains one or more templates.

```
You are a [[role]]. Summarize the following text in [[language]]:

[[input_text]]
```

Parameters use `[[double_brackets]]` and are substituted at execution time. Templates can reference other templates by name for composition.

Each template also configures:

* **Model** — which LLM to use (e.g. `openai/gpt-4o`, `anthropic/claude-sonnet-4-20250514`)
* **Temperature** — creativity control
* **Tools** — function-calling tools available to this template
* **Response schema** — optional JSON Schema for structured outputs
* **Fallbacks** — ordered list of fallback models if the primary fails

See [Templates & Parameters](/flows/templates-and-parameters) for the full reference.

## Versions

Prompt templates are grouped into **versions**. A version is an immutable snapshot of all templates in a flow (once activated). The lifecycle is:

1. **Create** a new editable version (or **fork** an existing one)
2. **Edit** templates, models, and tools
3. **Activate** the version for an environment

Once activated, a version becomes read-only. To make changes, fork it into a new version.

## Environments

**Environments** (e.g. `development`, `staging`, `production`) control which version of a flow runs when. Each flow has an `ActiveVersions` map:

```json
{
  "development": "version_5",
  "production": "version_3"
}
```

When executing a flow, pass `"environment": "production"` to use the version pinned to that environment. You can also promote versions from one environment to another in bulk.

See [Environments](/flows/environments) for details.

## Execution modes

Flows support two execution modes:

* **Direct** (default) — Runs the entrypoint template directly.
* **Route** — An LLM classifier reads the user input and selects the best template based on each template's description. Useful for flows that handle multiple intents (e.g. a customer support flow with templates for billing, technical help, and general inquiries).

## Tools

**Tools** extend what an LLM can do during execution. PromptShuttle supports five tool types:

| Type             | Description                                                                                    |
| ---------------- | ---------------------------------------------------------------------------------------------- |
| **External**     | HTTP endpoint called with query parameters. Supports environment-specific URL overrides.       |
| **Virtual**      | Provider-native capabilities (e.g. `web_search`, `code_interpreter`).                          |
| **Agent**        | Invokes another prompt template as a sub-agent, enabling multi-agent orchestration.            |
| **CritiqueLoop** | Runs a producer/critic loop: one flow generates, another critiques, iterating to refine.       |
| **MCP**          | Calls tools from a connected [Model Context Protocol](https://modelcontextprotocol.io) server. |

When a template has tools assigned, PromptShuttle automatically manages the tool-calling loop: the LLM returns tool calls, PromptShuttle executes them, appends results, and re-invokes the LLM until the model is done.

## Model routing

**Model routing** lets you configure tenant-level rules for model selection:

* **Aliases** — Map friendly names to specific models (e.g. `default` -> `openai/gpt-4o`)
* **Fallback chains** — If model A fails, try model B, then C
* **Cost optimization** — Prefer cheaper models when quality is equivalent

Routing rules are evaluated in order: request override > template config > tenant routing rules > defaults.

## Credits and cost tracking

PromptShuttle tracks costs in **credits** (1,000,000 credits = $1 USD). Every request logs:

* Token usage (input, output, reasoning, cached)
* Cost per inference
* Cumulative cost across agent trees
* Per-customer attribution

You can set per-request cost ceilings and tenant-level budget alerts with webhook notifications.

## Multi-model format

Models are referenced as `provider/model-name`:

```
openai/gpt-4o
anthropic/claude-sonnet-4-20250514
google/gemini-2.5-flash
groq/llama-3.3-70b-versatile
deepseek/deepseek-chat
xai/grok-3
perplexity/sonar-pro
```

List all available models and their capabilities via `GET /api/v1/models/descriptors`.


# Overview

Flows are the core building block of PromptShuttle. A flow packages your prompt templates, model configuration, and tool assignments into a versioned, deployable unit.

## Why flows?

Without flows, prompt management quickly becomes painful:

* Prompts are hardcoded in your application, requiring redeployment to change them
* No audit trail of what changed, when, and by whom
* No way to A/B test or gradually roll out prompt changes
* No separation between environments (dev/staging/production)

Flows solve all of this. You edit prompts in PromptShuttle, version them, and promote across environments — your application code stays the same.

## Structure

```
Flow (e.g. "customer_support")
├── Version 1 (activated in production)
│   ├── Template: "main" (entrypoint)
│   ├── Template: "billing_handler"
│   └── Template: "technical_handler"
├── Version 2 (activated in development)
│   ├── Template: "main" (entrypoint, updated prompt)
│   ├── Template: "billing_handler"
│   └── Template: "technical_handler" (new model)
└── Version 3 (draft, editable)
    └── ...
```

* **Flow** — A named container. Has a slug (e.g. `customer_support`) used in API URLs.
* **Version** — An immutable snapshot of templates (once activated). Fork to create a new editable version.
* **Template** — A single prompt with model config, tools, and parameters. One template is the entrypoint.

## Creating a flow

### Via the UI

1. Go to **Flows** and click **Create Flow**
2. Enter a title — the slug name is auto-generated
3. A first version with a "main" template is created automatically
4. Edit the template prompt, select a model, and save

### Via the API

```bash
curl -X POST https://app.promptshuttle.com/api/v1/flows \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Customer Support"
  }'
```

## Execution modes

### Direct mode (default)

The entrypoint template runs directly. Best for single-purpose flows.

### Route mode

An LLM classifier analyzes the user's input and selects the best template based on each template's `description` field. This is useful for flows that handle multiple intents.

For example, a customer support flow might have:

| Template            | Description                                      |
| ------------------- | ------------------------------------------------ |
| `billing_handler`   | Handles billing questions, refunds, and payments |
| `technical_handler` | Handles technical issues and bug reports         |
| `general_handler`   | Handles general inquiries and feedback           |

The classifier model defaults to `openai/gpt-4o-mini` but can be overridden per flow via the `classifierModel` field.

{% hint style="info" %}
Route mode requires at least 2 templates with descriptions. With only 1 template, the flow executes it directly regardless of mode.
{% endhint %}

## Version lifecycle

```
Create/Fork  →  Edit  →  Activate  →  (Read-only)
                             │
                             └→  Fork into new version
```

1. **Create** a new empty version, or **fork** an existing version to start from its templates
2. **Edit** templates freely while the version is in draft state
3. **Activate** the version for an environment (e.g. `production`)
4. Activated versions are read-only. Fork again to make changes.

## Multi-template flows

A version can contain multiple templates. This enables:

* **Composition** — Templates can reference each other as sub-templates via `[[other_template_name]]` parameter syntax
* **Agent orchestration** — Templates can invoke other templates as sub-agents via Agent tools
* **Route mode** — The classifier picks the best template per request

## Running a flow

See [Flow Execution API](/api-reference/flow-execution) for the full API reference. Quick example:

```bash
curl -X POST https://app.promptshuttle.com/api/v1/flows/customer_support/runs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "parameters": { "customer_name": "Alice" },
    "environment": "production"
  }'
```

## Managing flows via API

| Method   | Endpoint                                                   | Description                  |
| -------- | ---------------------------------------------------------- | ---------------------------- |
| `GET`    | `/api/v1/flows`                                            | List all flows               |
| `GET`    | `/api/v1/flows/{flowId}`                                   | Get a flow by ID             |
| `POST`   | `/api/v1/flows`                                            | Create a new flow            |
| `PUT`    | `/api/v1/flows/{flowId}`                                   | Update flow metadata         |
| `DELETE` | `/api/v1/flows/{flowId}`                                   | Soft-delete a flow           |
| `POST`   | `/api/v1/flows/{flowId}/versions`                          | Create a new empty version   |
| `POST`   | `/api/v1/flows/{flowId}/versions/{versionId}/fork`         | Fork a version               |
| `POST`   | `/api/v1/flows/{flowId}/versions/{versionId}/environments` | Activate for an environment  |
| `GET`    | `/api/v1/flows/{flowId}/parameters`                        | Discover expected parameters |


# Templates & Parameters

## Template anatomy

Each template in a flow version has:

| Field            | Type         | Description                                                         |
| ---------------- | ------------ | ------------------------------------------------------------------- |
| `name`           | string       | Technical name (e.g. `main`, `billing_handler`). Used in API calls. |
| `description`    | string       | Human-readable description. Used by the classifier in Route mode.   |
| `template`       | string       | The system prompt with `[[parameter]]` placeholders.                |
| `userTemplate`   | string       | Optional user message template. Same parameter syntax.              |
| `llm`            | string       | Model to use (e.g. `openai/gpt-4o`).                                |
| `temperature`    | float        | Sampling temperature (0-2).                                         |
| `maxToolCalls`   | integer      | Maximum tool-calling iterations before stopping.                    |
| `toolIds`        | string array | IDs of tools available to this template.                            |
| `fallbacks`      | string array | Ordered fallback models if the primary model fails.                 |
| `responseSchema` | object       | JSON Schema for structured outputs.                                 |

## Parameter syntax

Parameters use double square brackets:

```
You are a [[role]]. Write a [[document_type]] about [[topic]].
```

### Rules

* Parameter names must be **lowercase** with **letters, digits, and underscores** only
* `[[my_param]]` — valid
* `[[MyParam]]` — invalid (no uppercase)
* `[[my-param]]` — invalid (no hyphens)
* `[[my param]]` — invalid (no spaces)

### How parameters are resolved

When a flow is executed, parameters are resolved in this order:

1. **Caller-supplied values** — Parameters passed in the API request body
2. **Sub-template references** — If a parameter name matches another template in the same version, that template's content is substituted
3. **Unresolved** — If neither applies, a warning is returned in the response

### Comments in templates

Templates support two comment styles (comments are stripped before sending to the LLM):

```
/* This is a block comment */
// This is a line comment
```

## System prompt vs. user message

Each template has two prompt fields:

* **`template`** (system prompt) — Rendered as a `system` message. This is where you define the LLM's behavior, persona, and instructions.
* **`userTemplate`** (user message) — Rendered as a `user` message after the system prompt. Use this when you want to separate instructions from the user's input.

Both fields support the same `[[parameter]]` syntax.

### Example

**System prompt (`template`):**

```
You are a professional translator specializing in [[source_language]] to [[target_language]] translation.
Maintain the original tone and style.
```

**User message (`userTemplate`):**

```
Translate the following text:

[[input_text]]
```

## Discovering parameters

Before executing a flow, you can discover what parameters it expects:

```bash
GET /api/v1/flows/{flowId}/parameters?environment=production
```

**Response:**

```json
[
  { "token": "source_language", "source": "template" },
  { "token": "target_language", "source": "template" },
  { "token": "input_text", "source": "userTemplate" }
]
```

The `source` field indicates where the parameter appears:

| Source         | Meaning                           |
| -------------- | --------------------------------- |
| `template`     | In the system prompt              |
| `userTemplate` | In the user message template      |
| `toolUrl`      | In an external tool's URL pattern |

Parameters that match a sub-template name also include a `promptTemplate` object.

## Structured outputs

You can constrain the LLM to return JSON matching a schema. Set `responseSchema` on the template:

```json
{
  "type": "object",
  "properties": {
    "sentiment": { "type": "string", "enum": ["positive", "negative", "neutral"] },
    "confidence": { "type": "number" },
    "summary": { "type": "string" }
  },
  "required": ["sentiment", "confidence", "summary"]
}
```

The caller can also override this at execution time by passing `responseSchema` in the flow run request.

{% hint style="warning" %}
Not all models support structured outputs. Check the model's `supportsStructuredOutput` field via `GET /api/v1/models/descriptors`.
{% endhint %}

## Tool assignments

Templates can reference tools by their IDs in the `toolIds` array. When tools are assigned:

1. PromptShuttle includes the tool definitions in the LLM request
2. If the LLM returns tool calls, PromptShuttle executes them automatically
3. Tool results are appended to the conversation and the LLM is re-invoked
4. This loop continues until the LLM responds without tool calls (or `maxToolCalls` is reached)

See [Key Concepts: Tools](/getting-started/key-concepts#tools) for the five tool types.

## Fallback models

Set `fallbacks` on a template to define backup models:

```json
{
  "llm": "anthropic/claude-sonnet-4-20250514",
  "fallbacks": ["openai/gpt-4o", "google/gemini-2.5-flash"]
}
```

If the primary model fails (rate limit, timeout, outage), PromptShuttle automatically tries the next model in the list. The response indicates if a fallback was used.


# Environments

Environments let you run different versions of a flow in different contexts — so you can iterate on prompts in development without affecting production traffic.

## How it works

Each flow maintains an `ActiveVersions` map:

```json
{
  "development": "67a1b2c3d4e5f6a7b8c9d0e1",
  "staging": "67a1b2c3d4e5f6a7b8c9d0e2",
  "production": "67a1b2c3d4e5f6a7b8c9d0e3"
}
```

When you execute a flow with `"environment": "production"`, PromptShuttle resolves to the version ID pinned to that environment.

## Creating environments

Environments are tenant-level resources. Create them once and use them across all flows.

```bash
POST /api/v1/environments
```

```json
{ "name": "production" }
```

Common setups:

* Simple: `development`, `production`
* Standard: `development`, `staging`, `production`
* Custom: any names you choose (e.g. `canary`, `eu-prod`)

## Activating a version

To pin a version to an environment:

```bash
POST /api/v1/flows/{flowId}/versions/{versionId}/environments
```

```json
{
  "environment": "production",
  "comment": "Improved billing handler prompt"
}
```

Once activated, the version becomes read-only. Fork it to make further changes.

## Promoting versions

You can promote all active versions from one environment to another in bulk — for example, promoting everything in `staging` to `production`.

### Diff first

Before promoting, check what will change:

```bash
GET /api/v1/environments/diff?sourceEnvironment=staging&targetEnvironment=production
```

Returns a list of flows with their current version in each environment, so you can review before promoting.

### Promote

```bash
POST /api/v1/environments/promote
```

```json
{
  "sourceEnvironment": "staging",
  "targetEnvironment": "production",
  "flowIds": ["flow_id_1", "flow_id_2"],
  "comment": "March release"
}
```

If `flowIds` is omitted, all flows with an active version in the source environment are promoted.

## Environment in API requests

### Flow execution

Pass the environment name when running a flow:

```bash
POST /api/v1/flows/my_flow/runs
```

```json
{
  "environment": "production",
  "parameters": { ... }
}
```

If no environment is specified, the first environment in the flow's `ActiveVersions` is used.

### OpenAI endpoint

Requests through the OpenAI-compatible endpoint are logged with the environment `"OpenAI"`. The OpenAI endpoint does not use flow versions — it routes directly to the specified model.

### Direct inference

The inference endpoint accepts an optional `environment` field for logging and analytics segmentation, but it doesn't affect model selection (since there's no flow/template involved).

## Filtering by environment

The invocation log and statistics endpoints support filtering by environment, so you can:

* Track production usage separately from development testing
* Monitor costs per environment
* Compare performance across environments


# Tools & Function Calling

Tools extend what an LLM can do during execution. When a template has tools assigned, PromptShuttle automatically manages the tool-calling loop: the LLM decides which tools to call, PromptShuttle executes them, feeds results back, and re-invokes the LLM until it produces a final answer.

## How the tool loop works

```
1. Send prompt + tool definitions to LLM
2. LLM responds with tool_calls (or final text)
3. If tool_calls:
   a. Execute all tool calls in parallel
   b. Append results to conversation
   c. Go to step 1 (re-invoke LLM with results)
4. If final text: return response
```

The loop continues until the LLM stops calling tools or the `maxToolCalls` limit is reached.

## Tool types

PromptShuttle supports five tool types, each suited to different use cases:

| Type             | What it does                                                            |
| ---------------- | ----------------------------------------------------------------------- |
| **External**     | Calls an HTTP endpoint with parameters from the LLM's tool call         |
| **Virtual**      | Enables provider-native capabilities (web search, code interpreter)     |
| **Agent**        | Invokes another flow as a sub-agent, enabling multi-agent orchestration |
| **CritiqueLoop** | Runs a producer/critic refinement loop to iteratively improve output    |
| **MCP**          | Calls a tool on an external Model Context Protocol server               |

## External tools

External tools call HTTP endpoints. The LLM decides the arguments; PromptShuttle builds the request and returns the response.

### Configuration

| Field                 | Description                                                                                                                                                       |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`                | Tool name (shown to the LLM)                                                                                                                                      |
| `description`         | What the tool does (helps the LLM decide when to use it)                                                                                                          |
| `parameters`          | Parameter schema (name, type, description, required, enum values)                                                                                                 |
| `webUrl`              | Base HTTP endpoint URL. An absolute `https://…` names a fixed host; a relative path (`/api/v1/search`) makes the tool [caller-hosted](/tools/caller-hosted-tools) |
| `webUrls`             | Environment-specific URL overrides (`{ "production": "https://...", "staging": "https://..." }`)                                                                  |
| `headers`             | Custom HTTP headers (e.g. API keys)                                                                                                                               |
| `callbackTokenHeader` | Header carrying the caller's per-run token on caller-hosted calls (default `X-Tool-Token`)                                                                        |

### How it works

1. The LLM returns a tool call with arguments (e.g. `{ "query": "weather in Berlin" }`)
2. PromptShuttle sends an HTTP GET to the configured URL with arguments as query parameters
3. The response body (up to 1MB) is returned to the LLM as the tool result

### Parameter placeholders in URLs

External tool URLs support `[[param]]` placeholders that are substituted from the flow's request parameters:

```
https://api.example.com/v1/[[region]]/search
```

If the flow was called with `"parameters": { "region": "eu" }`, the URL becomes `https://api.example.com/v1/eu/search`.

### Environment-specific URLs

Use `webUrls` to point to different endpoints per environment:

```json
{
  "webUrl": "https://staging-api.example.com/search",
  "webUrls": {
    "production": "https://api.example.com/search",
    "staging": "https://staging-api.example.com/search"
  }
}
```

### Tools your own app serves

If the endpoint lives in the same application that calls PromptShuttle, its address differs on every machine that application runs on — so it does not belong on the tool at all. Give the tool a relative path instead and let each request supply the host:

```json
{ "webUrl": "/api/v1/tools/search?schema_id=[[schema_id]]" }
```

The origin then comes from the `X-Shuttle-Callback-Url` header per request. If a request omits it, the tool call fails rather than falling back to a stored URL.

See [Caller-Hosted Tools](/tools/caller-hosted-tools).

## Virtual tools (provider-native)

Virtual tools enable capabilities built into the LLM provider, such as web search and code execution. They are handled natively by the provider — PromptShuttle passes them through.

### Available virtual tools

| Tool ID               | Provider    | Description                    |
| --------------------- | ----------- | ------------------------------ |
| `web_search`          | OpenAI, xAI | Search the web for information |
| `code_interpreter`    | OpenAI      | Execute code in a sandbox      |
| `web_search_20250305` | Anthropic   | Claude web search              |

### Using virtual tools

Virtual tools can be enabled in two ways:

1. **On the template** — Create a tool with `type: Virtual` and `virtualToolId: "web_search"`, then assign it to the template
2. **At request time** — Pass `vendorTools: ["web_search"]` in the flow run request

## Agent tools

Agent tools invoke another flow as a sub-agent. This is the foundation of multi-agent orchestration in PromptShuttle.

### Configuration

| Field             | Description                                                              |
| ----------------- | ------------------------------------------------------------------------ |
| `agentTemplateId` | The ID of the flow to invoke as a sub-agent                              |
| `maxAgentDepth`   | Maximum nesting depth for this specific agent (overrides tenant default) |

### How it works

1. The LLM calls the agent tool with arguments
2. PromptShuttle creates a child request linked to the parent
3. The child flow executes with its own tool-calling loop
4. The child's text response is returned to the parent LLM as the tool result
5. Cost is tracked both per-agent (direct) and across the tree (cumulative)

### Safety mechanisms

* **Depth limits** — Default max depth is 10. Configurable per tool, per tenant, or per request.
* **Cycle detection** — PromptShuttle tracks the ancestor chain and prevents an agent from calling itself (directly or indirectly).
* **Cost limits** — Per-request cost ceilings apply across the entire agent tree.

### Context tools

When a template is running as a sub-agent, PromptShuttle automatically injects context tools:

| Tool                 | Description                                                                |
| -------------------- | -------------------------------------------------------------------------- |
| `get_context`        | Returns execution metadata: depth, agent path, cost used, budget remaining |
| `get_original_input` | Returns the root request's original user messages                          |
| `get_state`          | Read from shared state (lexically scoped — child values shadow parent)     |
| `set_state`          | Write to shared state (visible to child agents)                            |

These let agents be context-aware without hardcoding parent/child relationships.

## CritiqueLoop tools

CritiqueLoop tools implement an iterative refinement pattern: a "producer" flow generates output, then a "critic" flow evaluates it. If the critic rejects the output, the producer tries again with the critic's feedback.

### Configuration

| Field               | Description                               |
| ------------------- | ----------------------------------------- |
| `producerFlowId`    | Flow ID of the producer                   |
| `criticFlowId`      | Flow ID of the critic                     |
| `maxLoopIterations` | Maximum refinement iterations (default 3) |

### How it works

```
for each iteration (up to maxLoopIterations):
    1. Run producer with original input + last critique feedback
    2. Run critic with (original input + producer output)
    3. Critic returns: { "approved": true/false, "feedback": "..." }
    4. If approved → return producer output
    5. If rejected → feed critique back to producer, continue
```

The critic always runs with `temperature: 0` for consistent evaluation.

### Critic response format

The critic flow must return JSON matching this schema:

```json
{
  "approved": true,
  "feedback": "The response addresses all key points."
}
```

If `approved` is `false`, the `feedback` string is passed back to the producer for the next iteration.

## MCP tools

MCP tools call tools on external [Model Context Protocol](https://modelcontextprotocol.io) servers. This lets you integrate with any MCP-compatible tool server.

See [MCP Server Integration](/tools/mcp-servers) for setup details.

## Managing tools via API

| Method   | Endpoint                   | Description                         |
| -------- | -------------------------- | ----------------------------------- |
| `GET`    | `/api/v1/tools`            | List all tools                      |
| `GET`    | `/api/v1/tools/{id}`       | Get a tool by ID                    |
| `POST`   | `/api/v1/tools`            | Create a new tool                   |
| `PUT`    | `/api/v1/tools/{id}`       | Update a tool                       |
| `DELETE` | `/api/v1/tools/{id}`       | Delete a tool                       |
| `GET`    | `/api/v1/tools/{id}/usage` | List flows that reference this tool |

### Tool parameter schema

Each tool defines its parameters with a list of properties:

```json
{
  "name": "search_api",
  "description": "Search for products by keyword",
  "toolType": "External",
  "webUrl": "https://api.example.com/search",
  "parameters": {
    "type": "Object",
    "properties": [
      {
        "name": "query",
        "description": "Search query string",
        "type": "String",
        "required": true
      },
      {
        "name": "category",
        "description": "Product category to filter by",
        "type": "String",
        "required": false,
        "enum": ["electronics", "clothing", "books"]
      },
      {
        "name": "limit",
        "description": "Maximum number of results",
        "type": "Number",
        "required": false
      }
    ]
  }
}
```

Property types: `String`, `Number`. Use `isList: true` for array parameters. Use `enum` to constrain values.


# Caller-Hosted Tools

Most external tools live at a fixed address — a public API, a service you run at a known hostname. You put the URL on the tool and PromptShuttle calls it.

Caller-hosted tools are the other case: the tool is served by **your own application**, the same one making the request. That means its address is different on every machine your app runs on — your laptop, a preview environment, production — and none of those addresses belong on a shared tool definition.

So don't store one. Give the tool a **relative path** and let each request say where its tools live.

```json
{
  "name": "search_categories",
  "toolType": "external",
  "webUrl": "/api/v1/tools/categorization/search?schema_id=[[schema_id]]"
}
```

```bash
curl -X POST https://app.promptshuttle.com/api/v1/flows/categorize/runs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Shuttle-Callback-Url: https://app.example.com" \
  -H "Content-Type: application/json" \
  -d '{ "parameters": { "schema_id": "abc123" } }'
```

One tool definition. Your developer machine answers its own tool calls, production answers its own, and neither is written down anywhere.

## Absolute or relative

The shape of `webUrl` is the whole switch. There is no separate flag.

| `webUrl`                         | Meaning                                                                    |
| -------------------------------- | -------------------------------------------------------------------------- |
| `https://api.example.com/search` | Fixed host. Resolves as it always has; the callback header is not needed.  |
| `/api/v1/search`                 | Caller-hosted. Origin comes from `X-Shuttle-Callback-Url` on each request. |

Environment overrides compose with this: a tool can be caller-hosted by default and pinned to a real host in one environment.

```json
{
  "webUrl": "/api/v1/search",
  "environmentOverrides": {
    "production": { "webUrl": "https://api.example.com/v1/search" }
  }
}
```

`[[parameter]]` placeholders work in relative paths exactly as in absolute URLs.

## A missing origin fails the call

If a caller-hosted tool is invoked on a request that carries no `X-Shuttle-Callback-Url`, the tool call **fails**. The model receives an explicit error naming the cause, and the run records it. No stored URL is substituted.

This is deliberate, and it is worth understanding why, because the alternative looks harmless:

> A flow ran on a developer machine with no callback origin configured. Its tools fell back to the URL on the definition — the deployed app. Production had never heard of that developer's data, so it answered honestly: `{"results": [], "total": 0}`. Three times. The model concluded no categories existed, invented a plausible taxonomy, and returned an answer with 0.85 confidence. Downstream, a batch import ran against nothing at roughly 100× the normal cost and reported success.
>
> Every tool call was a 200. The run is recorded as `Succeeded`.

A wrong answer that looks exactly like a right one is worse than an error. So: send the header from every environment, production included. Treat it as part of your PromptShuttle configuration, not as a debugging aid you switch on when something breaks.

## Registering allowed origins

The origin arrives in a request header, so it needs a boundary. Otherwise an API key would buy the ability to point PromptShuttle's outbound calls at any host on the internet and read back whatever it returns.

Register the origins your apps serve tools from under **Settings → Callback origins**, one per line:

```
https://app.example.com
*.ngrok-free.dev
localhost:5173
```

* Scheme and port are optional. When present, they must match.
* A leading `*.` matches sub-domains — `*.ngrok-free.dev` matches `lexi-wanning.ngrok-free.dev` but not `ngrok-free.dev` itself, and not a look-alike like `evil-ngrok-free.dev`.
* **With no origins registered, caller-hosted tools do not run.** Deny-by-default: an unregistered origin fails loudly rather than resolving to something plausible.

Private and reserved IP ranges are blocked regardless of the allowlist, so a registered origin still cannot reach internal infrastructure. That check is relaxed only in local development builds — which is why `localhost` in the example above works against a local PromptShuttle but not against the hosted one. To reach a laptop from hosted PromptShuttle, use a tunnel.

## Per-run tokens

Your tool endpoints should authenticate their callers. A single secret stored on the tool definition can't do that once the host varies per request — a developer's tunnel would present production's token.

Send the secret with the request instead:

```
X-Shuttle-Callback-Token: <this instance's secret>
```

PromptShuttle holds it for the duration of the run, never persists it, and presents it back on caller-origin calls under `X-Tool-Token` — or whatever header name the tool's **Callback token header** field specifies. It takes precedence over any same-named header stored on the tool.

Each instance can then hold its own secret, and a leaked tunnel token is worth nothing anywhere else.

## Configuring the client SDK

The .NET client sends both headers on every request when configured:

```json
{
  "PromptShuttle": {
    "ApiKey": "...",
    "CallbackUrl": "https://app.example.com",
    "CallbackToken": "..."
  }
}
```

Set `CallbackUrl` per deployment — that is the point of it. Leave it unset only if none of your tools are caller-hosted.

## Seeing which host answered

Every tool call records the endpoint it was actually placed against, plus where that origin came from. A run served by a tunnel is distinguishable from one served by the deployed app at a glance:

* **Invocation log** — each tool-call step shows the endpoint and a badge reading *tool definition*, *environment binding*, or *caller-supplied origin*.
* **`get_run` (MCP)** — a `toolCalls` array with `targetUrl` and `originSource` per call, and `callbackOrigin` on the run itself.
* **Streaming events** — `tool.started`, `tool.completed` and `tool.failed` carry `targetUrl` and `originSource`.

## Header reference

| Header                     | Required                | Description                                                                                            |
| -------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------ |
| `X-Shuttle-Callback-Url`   | For caller-hosted tools | Origin serving this caller's tools, e.g. `https://app.example.com`. Any path is ignored.               |
| `X-Shuttle-Callback-Token` | No                      | Secret presented back on caller-origin calls. Never stored.                                            |
| `X-Shuttle-Debug-Url`      | No — deprecated         | Legacy alias of `X-Shuttle-Callback-Url`. Also attaches a `DebugUrl:` tag. Prefer the callback header. |

## Migrating an existing tool

1. Register your origins under **Settings → Callback origins** — including production.
2. Set `CallbackUrl` (or send `X-Shuttle-Callback-Url`) everywhere the app runs. Deploy that first.
3. Change the tool's `webUrl` from `https://app.example.com/api/v1/search` to `/api/v1/search`.

Steps 1 and 2 change nothing on their own: while the tool's URL is still absolute, a callback origin only redirects the host, which is the behaviour the old `X-Shuttle-Debug-Url` header already had. Step 3 is the switch, and it is reversible.


# MCP Server Integration

PromptShuttle can connect to external [Model Context Protocol](https://modelcontextprotocol.io) (MCP) servers, discover their tools, and make them available to your flows. This lets you integrate with any MCP-compatible tool server without writing custom HTTP integrations.

## Workflow

```
1. Register an MCP server (URL + auth headers)
2. Discover available tools from the server
3. Import selected tools into your PromptShuttle tool library
4. Assign imported tools to flow templates
```

## Register an MCP server

```bash
curl -X POST https://app.promptshuttle.com/api/v1/mcp-servers \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Tool Server",
    "url": "https://mcp.example.com/sse",
    "headers": {
      "Authorization": "Bearer server_token_here"
    }
  }'
```

| Field     | Type   | Required | Description                                       |
| --------- | ------ | -------- | ------------------------------------------------- |
| `name`    | string | yes      | Display name for the server                       |
| `url`     | string | yes      | MCP server HTTP endpoint                          |
| `headers` | object | no       | Custom HTTP headers (auth tokens, API keys, etc.) |

## Discover tools

After registering a server, discover what tools it offers:

```bash
POST /api/v1/mcp-servers/{serverId}/discover
```

Returns a list of discovered tools:

```json
[
  {
    "name": "get_weather",
    "description": "Get current weather for a location",
    "inputSchema": {
      "type": "object",
      "properties": {
        "location": { "type": "string", "description": "City name" }
      },
      "required": ["location"]
    },
    "alreadyImported": false
  },
  {
    "name": "search_docs",
    "description": "Search documentation by keyword",
    "inputSchema": { "..." },
    "alreadyImported": true
  }
]
```

The `alreadyImported` flag indicates whether you've already imported this tool.

## Import tools

Select which discovered tools to import into your tool library:

```bash
curl -X POST https://app.promptshuttle.com/api/v1/mcp-servers/{serverId}/import \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "toolNames": ["get_weather", "search_docs"]
  }'
```

Imported tools appear in your tool library with:

* `type: Mcp`
* `mcpServerId` linking back to the registered server
* `rawInputSchema` preserving the original MCP schema
* Parameters converted to PromptShuttle's format for display

## Using imported MCP tools

Once imported, MCP tools work like any other tool:

1. Assign the tool to a template via `toolIds`
2. When the LLM calls the tool, PromptShuttle routes the call to the MCP server
3. The server's response is returned to the LLM as the tool result

## Managing MCP servers

| Method   | Endpoint                            | Description                 |
| -------- | ----------------------------------- | --------------------------- |
| `GET`    | `/api/v1/mcp-servers`               | List all registered servers |
| `GET`    | `/api/v1/mcp-servers/{id}`          | Get server details          |
| `POST`   | `/api/v1/mcp-servers`               | Register a new server       |
| `PUT`    | `/api/v1/mcp-servers/{id}`          | Update server config        |
| `DELETE` | `/api/v1/mcp-servers/{id}`          | Remove a server             |
| `POST`   | `/api/v1/mcp-servers/{id}/discover` | Discover tools from server  |
| `POST`   | `/api/v1/mcp-servers/{id}/import`   | Import selected tools       |

## Server status

Each registered server tracks:

| Field                 | Description                            |
| --------------------- | -------------------------------------- |
| `discoveredToolCount` | Number of tools found on last connect  |
| `lastConnectedAt`     | Timestamp of last successful discovery |


# Model Routing

Model routing lets you configure tenant-level rules for how models are selected. You can create aliases, define fallback chains, and use load-balancing strategies — all without changing your application code.

## How model resolution works

When a request is executed, PromptShuttle resolves the model in this order:

1. **Request override** — If `overrideModel` is set, use it (but check if it's an alias first)
2. **Routing rules** — Check tenant routing rules for the template's model name
3. **Template config** — Use the template's `llm` field and `fallbacks`
4. **Defaults** — Use provider defaults

## Routing rules

A routing rule maps an **alias** to one or more **actual models** with a **strategy** for selection.

### Example rules

```json
[
  {
    "alias": "fast",
    "models": ["groq/llama-3.3-70b-versatile", "openai/gpt-4o-mini"],
    "strategy": "Sequential",
    "description": "Fast, cheap model with OpenAI fallback"
  },
  {
    "alias": "smart",
    "models": ["anthropic/claude-sonnet-4-20250514", "openai/gpt-4o"],
    "strategy": "Sequential",
    "description": "High-quality model with fallback"
  },
  {
    "alias": "balanced",
    "models": ["openai/gpt-4o", "anthropic/claude-sonnet-4-20250514", "google/gemini-2.5-flash"],
    "strategy": "RoundRobin",
    "description": "Distribute load across providers"
  }
]
```

### Using aliases

Once defined, use an alias anywhere you'd use a model name:

* In a template's `llm` field: set it to `"fast"` instead of `"groq/llama-3.3-70b-versatile"`
* In a request's `overrideModel`: `"overrideModel": "smart"`
* In the OpenAI endpoint's `model` field: `"model": "balanced"`

### Strategies

| Strategy           | Behavior                                                                   |
| ------------------ | -------------------------------------------------------------------------- |
| **Sequential**     | Uses the first model. If it fails, tries the next. Classic fallback chain. |
| **Random**         | Randomly selects one model per request.                                    |
| **WeightedRandom** | Randomly selects with weights (e.g. 70% model A, 30% model B).             |
| **RoundRobin**     | Rotates through models evenly across requests.                             |

### Weighted random

For `WeightedRandom`, provide a `weights` array matching the `models` array:

```json
{
  "alias": "cost_optimized",
  "models": ["groq/llama-3.3-70b-versatile", "openai/gpt-4o-mini", "openai/gpt-4o"],
  "strategy": "WeightedRandom",
  "weights": [0.5, 0.3, 0.2],
  "description": "50% Groq, 30% GPT-4o-mini, 20% GPT-4o"
}
```

Weights are normalized automatically — they don't need to sum to 1.

### Environment-scoped rules

Rules can be restricted to specific environments:

```json
{
  "alias": "default",
  "models": ["openai/gpt-4o"],
  "strategy": "Sequential",
  "environments": ["production"],
  "description": "Use GPT-4o in production only"
}
```

Rules without `environments` apply to all environments.

## Managing routing rules

### Get current rules

```bash
GET /api/v1/model-routing
```

### Update rules

```bash
PUT /api/v1/model-routing
```

```json
[
  {
    "alias": "fast",
    "models": ["groq/llama-3.3-70b-versatile"],
    "strategy": "Sequential"
  },
  {
    "alias": "default",
    "models": ["openai/gpt-4o", "anthropic/claude-sonnet-4-20250514"],
    "strategy": "Sequential"
  }
]
```

The entire rule set is replaced on update.

### Validation

Rules are validated before saving:

* Aliases must be non-empty
* Each rule must have at least one model
* All model names must be recognized (use `GET /api/v1/models/descriptors` to check)
* `WeightedRandom` rules must have a `weights` array matching the `models` count
* Weights must be non-negative
* No duplicate aliases

## Fallback behavior

When using `Sequential` strategy, the first model is the primary. If it fails (rate limit, timeout, outage), PromptShuttle automatically tries the next model in the list.

The response indicates whether a fallback was used via the `wasFallback` and `fallbackReason` fields in streaming events.

## Use cases

| Scenario                  | Configuration                                                   |
| ------------------------- | --------------------------------------------------------------- |
| **Cost optimization**     | Alias `"default"` → cheap model first, expensive fallback       |
| **Provider redundancy**   | Sequential across 2-3 providers for high availability           |
| **A/B testing models**    | WeightedRandom to split traffic between models                  |
| **Load distribution**     | RoundRobin across equivalent models from different providers    |
| **Environment isolation** | Different model in staging vs production                        |
| **Easy model upgrades**   | Change the alias target — all flows using it switch immediately |


# Billing & Credits

PromptShuttle tracks all LLM costs using a credit system and provides controls to prevent runaway spending.

## Credit system

All costs are tracked in **credits**:

```
1,000,000 credits = $1.00 USD
```

Every LLM call has a cost based on token usage and the model's pricing:

```
cost = (input_tokens * input_rate) + (output_tokens * output_rate)
```

Some models have additional pricing components:

* **Reasoning tokens** — Models like o1 charge for thinking tokens
* **Cached tokens** — Anthropic offers discounted rates for cache-hit tokens
* **Cache creation tokens** — Anthropic charges a premium for writing to cache
* **Tool costs** — Some provider-native tools (e.g. web search) have per-use charges

## Purchasing credits

Credits are purchased through Stripe:

1. Navigate to **Billing** in the PromptShuttle UI
2. Select a credit package
3. Complete checkout via Stripe
4. Credits are added to your tenant balance immediately

Manage your subscription and payment methods via the Stripe Customer Portal, accessible from the billing page.

## Cost tracking

Every request tracks cost at multiple levels:

### Per-inference costs

Each individual LLM call records:

* Input/output/reasoning tokens
* Cached token counts
* Cost in USD and credits
* Model and provider used
* Tool invocation costs

### Per-request costs (agentic)

When an agent makes multiple LLM calls (tool-calling loops), the request aggregates:

* Total credits used across all iterations
* Direct LLM costs vs. child agent costs
* Total duration
* Total tool calls

### Per-tree costs (multi-agent)

For multi-agent workflows, the root request tracks:

* Cumulative credits across the entire agent tree
* Number of agents spawned
* Maximum depth reached

## Cost controls

### Per-request limit

Set a maximum cost per request to prevent expensive runaway agent loops:

**Tenant-level** (applies to all requests):

Configure `maxRequestCostCredits` in your tenant settings.

**Per-request override:**

```json
{
  "parameters": { "..." },
  "maxRequestCostCredits": 50000
}
```

When the limit is exceeded, the request stops and returns an error with code `COST_LIMIT_EXCEEDED`.

### Agent depth limit

Limit how deep agent nesting can go:

* **System default:** 10 levels
* **Tenant override:** Set `maxAgentDepth` in tenant settings
* **Per-tool override:** Set `maxAgentDepth` on individual agent tools
* **Per-request override:** Pass in the request body

When exceeded, returns `DEPTH_LIMIT_EXCEEDED`.

### Cost alert webhooks

Configure a webhook to be notified when cumulative costs exceed a threshold:

* Set `costAlertThresholdCredits` on your tenant
* Set `alertWebhookUrl` to receive POST notifications

PromptShuttle sends a POST to your webhook URL with cost details when the threshold is crossed during a request.

## Credit balance

Check your current balance in the API response — every flow run and inference response includes:

* `creditsUsed` — how many credits this request consumed
* `creditsLeft` — remaining tenant balance

The balance is also visible on the dashboard.

## Cost in responses

### Flow execution response

```json
{
  "creditsUsed": 1250,
  "usage": {
    "creditsUsed": 1250,
    "creditsLeft": 998750,
    "tokensIn": 85,
    "tokensOut": 142,
    "reasoningTokens": 0,
    "toolCost": 0,
    "costUsd": 0.00125
  }
}
```

### Streaming usage updates

When streaming with `include_usage: true`, periodic `usageUpdate` events show real-time cost:

```json
{
  "type": "usageUpdate",
  "data": {
    "cumulativeCreditsUsed": 3400,
    "cumulativeCostUsd": 0.0034,
    "activeAgents": 1,
    "completedAgents": 2,
    "elapsedMs": 4500
  }
}
```

## Pricing

Model pricing varies by provider and model. View current pricing via:

```bash
GET /api/v1/models/descriptors
```

Each model includes `pricingGraduations` — volume-based tiers:

```json
{
  "name": "gpt-4o",
  "pricingGraduations": [
    { "fromPromptLength": 0, "input": 2.50, "output": 10.00 },
    { "fromPromptLength": 128000, "input": 1.25, "output": 5.00 }
  ]
}
```

Prices are in **$/million tokens**.

For Anthropic models, pricing also includes:

* `cachedInput` — Rate for cache-hit tokens
* `cacheCreationInput` — Rate for cache-write tokens


# Invocation Log & Analytics

PromptShuttle provides detailed logging of every request and three levels of analytics aggregation for understanding cost, performance, and usage patterns.

## Invocation log

Every request is logged with full request/response pairs, timing, cost, and metadata.

### Browse the log

```
GET /api/v1/llm-logs
```

### Filters

| Parameter         | Type         | Description                                         |
| ----------------- | ------------ | --------------------------------------------------- |
| `cursor`          | string       | Pagination cursor (from previous response)          |
| `pageSize`        | integer      | Results per page (default 50)                       |
| `from`            | datetime     | Start time filter                                   |
| `to`              | datetime     | End time filter                                     |
| `environments`    | string array | Filter by environment name                          |
| `flows`           | string array | Filter by flow name                                 |
| `statuses`        | string array | Filter by status (`Pending`, `Succeeded`, `Failed`) |
| `tags`            | string array | Filter by tags (any match)                          |
| `customerId`      | string       | Filter by end-customer external ID                  |
| `rootOnly`        | boolean      | Show only root-level requests (exclude sub-agents)  |
| `parentRequestId` | string       | Show children of a specific parent request          |

### Log entry fields

Each log entry includes:

* **Request details** — Messages, model, parameters, environment, tags
* **Response details** — Text output, tool calls, citations, finish reason
* **Tool calls** — For each call, the endpoint it was actually placed against and where that origin came from: the tool definition, an environment binding, or the caller's [`X-Shuttle-Callback-Url`](/tools/caller-hosted-tools)
* **Usage** — Tokens in/out, reasoning tokens, cost in credits and USD
* **Timing** — Duration in milliseconds
* **Hierarchy** — Parent/root request IDs, agent depth, agent role
* **Customer** — End-customer ID and external ID
* **Feedback** — Feedback score if submitted
* **Cancellation** — `cancelRequestedAt` / `cancelledAt` / `cancelRequestedBy`, if the run was stopped

### Stopping a run

Runs still executing show as **running** in the log, and the list refreshes on its own while any of them are on screen. Each has a stop control, as does the run detail — stopping a run stops every sub-agent beneath it. When anything is running, a **Stop everything running** button appears above the list; it stops every run in the workspace, including ones started by other people and by your applications.

Stopping is not instant: a run notices at its next step, so a model call already in flight finishes first. What it had produced by then is kept and still billed — stopping saves the work that had not happened yet. A stopped run shows a grey **cancelled** badge and carries a `Cancelled` warning explaining how far it got.

To find stopped runs, filter on the `Cancelled` flag; the status filter cannot express it, because a run that was stopped after producing something still records `Succeeded`. The dashboard's **Running now** counter links straight to the runs it counts.

The same controls are available over the API ([`POST /api/v1/runs/{runId}/cancel`](/api-reference/flow-execution#stop-a-running-run)) and over MCP (`cancel_run`, `cancel_all_runs`).

### Request hierarchy

For multi-agent requests, view the full hierarchy:

```
GET /api/v1/llm-logs/{id}/hierarchy
```

Returns:

* **Breadcrumb** — Ancestor path from root to current request
* **Parent** — Direct parent summary
* **Children** — Direct child agent summaries

### Streaming events

View the SSE events emitted during a request:

```
GET /api/v1/llm-logs/{id}/events
```

Filter by event type: `?types=agentStarted,agentCompleted`

Get event type counts: `GET /api/v1/llm-logs/{id}/events/stats`

### Log entries (debug)

For requests with elevated log levels, view detailed internal log entries:

```
GET /api/v1/llm-logs/{id}/log-entries
```

## Analytics

PromptShuttle provides analytics at three levels of granularity, each answering different questions.

### Level 1: Per-inference stats

**What it answers:** How are individual LLM calls performing?

```
GET /api/v1/statistics/flow/{flowName}
```

Metrics per model:

* Average tokens in/out
* Average cost and latency
* Percentiles (P50, P90, P95, P99) for tokens, latency, and cost
* Token and latency histograms

### Level 2: Per-request stats

**What it answers:** How are complete requests performing (including tool-calling loops)?

```
GET /api/v1/statistics/flow/{flowName}/request
```

Metrics:

* Total cost per request (across all LLM calls and tool invocations)
* Total function calls per request
* Request latency distribution
* Broken down by primary model

### Level 3: Per-tree stats (multi-agent)

**What it answers:** How are entire agent trees performing?

```
GET /api/v1/statistics/flow/{flowName}/tree
```

Metrics:

* Total tree cost (root + all sub-agents)
* Agent count per tree
* Maximum depth reached
* Depth distribution
* Agent count distribution

### All three levels at once

```
GET /api/v1/statistics/flow/{flowName}/extended
```

Returns all three levels plus time series data for invocations and cost.

### Tenant-wide statistics

```
GET /api/v1/statistics
```

Query parameters:

| Parameter    | Type     | Description                                                      |
| ------------ | -------- | ---------------------------------------------------------------- |
| `period`     | TimeSpan | Time window (e.g. `7.00:00:00` for 7 days)                       |
| `resolution` | string   | Time bucket size: `Minutes5`, `Minutes15`, `Hour`, `Day`, `Week` |

Returns:

* Invocation count (current vs. previous period)
* Per-flow invocation time series
* Cost time series
* Token usage time series
* Top flows by invocation count
* Credit usage comparison

### Account overview

```
GET /api/v1/statistics/overview
```

Returns cost and request counts broken down by model and by flow, with percentage distribution.

### Cost per tag

```
GET /api/v1/statistics/tag-costs?tag=TenderId:12313
```

Answers "what did this unit of work cost". If your application stamps its own correlation tag on every request — an order id, a document id, a tender id — this returns that tag's total spend as a **flow × model matrix**: how many LLM calls each flow made against each model, and what they cost.

Because a tagged job can run for weeks and the model behind a flow may change during it, the breakdown keys on the model that *actually served* each call. A model swapped mid-job simply shows up as a second column rather than silently replacing the first.

Accepts API keys as well as session tokens, so your own application can display the figure.

| Parameter  | Type     | Description                                                             |
| ---------- | -------- | ----------------------------------------------------------------------- |
| `tag`      | string   | **Required.** The exact tag, e.g. `TenderId:12313`. Not a prefix match. |
| `grouping` | string   | `split` (default) or `rollup` — see below                               |
| `from`     | DateTime | Optional start. Unbounded by default                                    |
| `to`       | DateTime | Optional end                                                            |

Sub-agent requests inherit their parent's tags, so one tag covers an entire agent tree. `grouping` decides how that tree is presented — the totals are identical either way:

* `split` — one row per flow that actually ran, so a sub-agent's own flow is visible with its own model and cost. Use this to find which agent is expensive.
* `rollup` — sub-agent spend folds into the entry-point flow you invoked. Use this to answer "what did each feature cost me".

Each cell reports `costUsd` (billed), `rawCostUsd` (provider cost before markup), `calls`, `requests`, token counts, and `pricingUncertain` — the last flags a cell whose served model is not in the model catalogue, so its price is an estimate.

```
GET /api/v1/statistics/tag-costs/by-key?prefix=TenderId:
```

One row per distinct tag under a key prefix, ranked by spend — "which tender cost the most". Only tags starting with the prefix are reported, which also keeps PromptShuttle's own internal marker tags (`Agent:`, `Panel`, `DebugUrl:`) out of the result.

| Parameter | Type     | Description                                                          |
| --------- | -------- | -------------------------------------------------------------------- |
| `prefix`  | string   | **Required.** Anchored prefix, e.g. `TenderId:`                      |
| `from`    | DateTime | Optional start. Defaults to 90 days ago; the window is capped at 365 |
| `to`      | DateTime | Optional end                                                         |
| `limit`   | int      | Rows to return, 1–200 (default 50). `truncated` says if more exist   |

Both are also available in the UI under **Tag Costs**.

**Caveats.** Panel deliberations run as their own top-level requests with fixed tags, so they do not inherit a caller tag and will not appear in a tag report.

## Data export

Export your flows and tools for backup or migration:

```
GET /api/v1/export
```

Returns all flows (with all versions and templates) and tools in a single JSON document.

Import into another tenant:

```
POST /api/v1/export/import
```

Duplicate flows and tools are skipped (matched by name).


# OpenAI-Compatible Endpoint

PromptShuttle exposes an OpenAI-compatible chat completion endpoint. If you already use the OpenAI SDK, you can switch to PromptShuttle by changing the base URL and API key — no other code changes needed.

## Endpoint

```
POST /api/v1/chat/completions
```

## Request

The request body follows the [OpenAI Chat Completion](https://platform.openai.com/docs/api-reference/chat/create) format with PromptShuttle extensions.

### Required fields

| Field      | Type   | Description                                                        |
| ---------- | ------ | ------------------------------------------------------------------ |
| `model`    | string | Model identifier in `provider/model` format (e.g. `openai/gpt-4o`) |
| `messages` | array  | Array of message objects (see [Messages](#messages) below)         |

### Optional fields

| Field             | Type    | Description                                                                                       |
| ----------------- | ------- | ------------------------------------------------------------------------------------------------- |
| `temperature`     | float   | Sampling temperature (0-2). Lower = more deterministic.                                           |
| `top_p`           | float   | Nucleus sampling threshold.                                                                       |
| `max_tokens`      | integer | Maximum tokens to generate.                                                                       |
| `seed`            | integer | Seed for deterministic sampling (provider support varies).                                        |
| `stream`          | boolean | Enable SSE streaming. Default `false`. See [Streaming](/api-reference/streaming).                 |
| `stream_options`  | object  | Streaming configuration (see below).                                                              |
| `tools`           | array   | Tool definitions in OpenAI format.                                                                |
| `tool_choice`     | string  | Tool selection mode: `"auto"` (default), `"none"`, `"required"`, or a specific tool.              |
| `response_format` | object  | Structured output format. Use `{ "type": "json_schema", "json_schema": { ... } }`.                |
| `user`            | string  | End-user identifier. Used for per-customer tracking if `X-Shuttle-Customer-Id` header is not set. |

### PromptShuttle extensions

These fields are non-standard and specific to PromptShuttle:

| Field         | Type   | Description                                                                                         |
| ------------- | ------ | --------------------------------------------------------------------------------------------------- |
| `x_log_level` | string | Override logging level: `"Trace"`, `"Debug"`, `"Information"`, `"Warning"`, `"Error"`, `"Critical"` |
| `x_nonce`     | string | Opaque cache-bust string. Included in the response-cache hash but never sent to providers.          |

### Custom headers

| Header                     | Description                                                                                  |
| -------------------------- | -------------------------------------------------------------------------------------------- |
| `X-Shuttle-Customer-Id`    | End-customer identifier for per-customer usage attribution.                                  |
| `X-Shuttle-Callback-Url`   | Origin where your app serves its own tools. Required for caller-hosted (relative-URL) tools. |
| `X-Shuttle-Callback-Token` | Secret presented back on caller-hosted tool calls. Held for the run, never stored.           |
| `X-Shuttle-Debug-Url`      | Deprecated alias of `X-Shuttle-Callback-Url`; also attaches a `DebugUrl:` tag.               |

See [Caller-Hosted Tools](/tools/caller-hosted-tools) for how relative tool URLs resolve, and why a missing callback origin fails the call instead of falling back.

## Messages

Each message in the `messages` array has:

| Field     | Type   | Values                              |
| --------- | ------ | ----------------------------------- |
| `role`    | string | `"user"`, `"assistant"`, `"system"` |
| `content` | array  | Array of content parts              |

### Content parts

Each content part has a `type` field:

**Text content:**

```json
{ "type": "text", "text": "Your prompt here" }
```

**Image content (URL):**

```json
{ "type": "image_url", "image_url": { "url": "https://example.com/image.png" } }
```

**Image content (base64 data URL):**

```json
{ "type": "image_url", "image_url": { "url": "data:image/png;base64,iVBOR..." } }
```

Data URLs are automatically parsed into base64 + media type for providers that require it (e.g. Gemini).

## Response

### Synchronous response

```json
{
  "id": "request_id",
  "object": "chat.completion",
  "created": 1700000000,
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 9,
    "total_tokens": 21
  }
}
```

### Streaming response

When `stream: true`, the endpoint returns Server-Sent Events. See [Streaming (SSE)](/api-reference/streaming) for the full event reference.

## Stream options

When streaming, you can configure the event stream:

| Field                   | Type    | Default | Description                                          |
| ----------------------- | ------- | ------- | ---------------------------------------------------- |
| `include_usage`         | boolean | `true`  | Emit periodic `usage.update` events.                 |
| `usage_interval_ms`     | integer | `5000`  | Interval between usage updates (milliseconds).       |
| `include_tool_results`  | boolean | `false` | Include full tool results in events (verbose).       |
| `include_agent_results` | boolean | `false` | Include full agent results in events (verbose).      |
| `heartbeat_interval_ms` | integer | `30000` | Keep-alive heartbeat interval (milliseconds).        |
| `event_types`           | array   | all     | Filter to specific event types (supports wildcards). |

## Examples

### Basic completion

```bash
curl -X POST https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "What is the capital of France?"}]}
    ],
    "temperature": 0.3,
    "max_tokens": 100
  }'
```

### With customer tracking

```bash
curl -X POST https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Shuttle-Customer-Id: user_456" \
  -d '{
    "model": "anthropic/claude-sonnet-4-20250514",
    "messages": [
      {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},
      {"role": "user", "content": [{"type": "text", "text": "Explain quantum computing simply."}]}
    ]
  }'
```

### Using OpenAI Python SDK

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://app.promptshuttle.com/api/v1",
    api_key="YOUR_API_KEY",
)

# Streaming
stream = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Write a haiku about APIs"}],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

### With structured output

```bash
curl -X POST https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [
      {"role": "user", "content": [{"type": "text", "text": "List the planets in our solar system"}]}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "planets",
        "schema": {
          "type": "object",
          "properties": {
            "planets": {
              "type": "array",
              "items": { "type": "string" }
            }
          },
          "required": ["planets"]
        }
      }
    }
  }'
```

## Other endpoints

### List models

```
GET /api/v1/models/descriptors
```

Returns all supported models with capabilities, pricing, and token limits.

### List providers

```
GET /api/v1/providers
```

Returns all configured LLM providers and their supported models.

### Direct inference (PromptShuttle native)

```
POST /api/v1/inference
```

A simpler endpoint for direct LLM inference without OpenAI response formatting. Supports the same models and features.

| Field                 | Type    | Description                                  |
| --------------------- | ------- | -------------------------------------------- |
| `messages`            | array   | ChatMessage array (PromptShuttle format)     |
| `model`               | string  | Model identifier                             |
| `environment`         | string  | Environment name for logging                 |
| `temperature`         | float   | Sampling temperature                         |
| `top_p`               | float   | Nucleus sampling                             |
| `top_k`               | integer | Top-k sampling (supported by some providers) |
| `max_tokens`          | integer | Max output tokens                            |
| `seed`                | integer | Deterministic seed                           |
| `max_thinking_tokens` | integer | Extended thinking budget (reasoning models)  |
| `response_schema`     | object  | JSON Schema for structured outputs           |
| `vendor_tools`        | array   | Provider-native tools (e.g. `web_search`)    |
| `tags`                | array   | Tags for filtering in invocation log         |
| `is_debug`            | boolean | Enable verbose logging                       |

### Tagging requests

The OpenAI request schema has no field for tags, so on `/api/v1/chat/completions` use the header instead:

```
X-Shuttle-Tags: TenderId:12313,Stage:analysis
```

Comma-separated, up to 20 tags. The header works on `/api/v1/inference` and `/api/v1/flows/{flowId}/runs` too, where it is merged with any `tags` in the body — handy for stamping a correlation tag centrally in an HTTP client rather than at every call site.

Tags are inherited by sub-agents, and drive both the invocation log filter and the [cost-per-tag report](/platform/analytics#cost-per-tag).


# Flow Execution

Execute flows programmatically and retrieve results.

## Run a flow

```
POST /api/v1/flows/{flowId}/runs
```

The `flowId` can be either the flow's **slug name** (e.g. `product_description`) or its **ObjectId**.

### Request body

| Field               | Type         | Required | Description                                                                                                                                                       |
| ------------------- | ------------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `parameters`        | object       | no       | Key-value map of template parameters. Keys must match `[[param]]` tokens.                                                                                         |
| `environment`       | string       | no       | Environment name (e.g. `"production"`). Determines which version runs.                                                                                            |
| `messages`          | array        | no       | Additional chat messages appended after the template's system/user messages.                                                                                      |
| `entrypoint`        | string       | no       | Override which template to use as entrypoint (by name).                                                                                                           |
| `version`           | string       | no       | Override which version to use (by ID). Takes precedence over environment.                                                                                         |
| `overrideModel`     | string       | no       | Force a specific model, bypassing template and routing config.                                                                                                    |
| `temperature`       | float        | no       | Override temperature.                                                                                                                                             |
| `maxTokens`         | integer      | no       | Override max output tokens.                                                                                                                                       |
| `maxThinkingTokens` | integer      | no       | Budget for extended thinking (reasoning models).                                                                                                                  |
| `maxToolCalls`      | integer      | no       | Max tool-calling loop iterations.                                                                                                                                 |
| `responseSchema`    | object       | no       | JSON Schema for structured output. Overrides template's schema.                                                                                                   |
| `vendorTools`       | string array | no       | Provider-native tools to enable (e.g. `["web_search"]`).                                                                                                          |
| `tags`              | string array | no       | Tags for filtering the invocation log, and for the [cost-per-tag report](/platform/analytics#cost-per-tag). Can also be supplied via the `X-Shuttle-Tags` header. |
| `nonce`             | string       | no       | Cache-bust string. Affects response cache hash, never sent to LLM.                                                                                                |
| `logLevel`          | string       | no       | Override logging level: `Trace`, `Debug`, `Information`, `Warning`, `Error`.                                                                                      |
| `customerId`        | string       | no       | End-customer ID for per-customer usage tracking.                                                                                                                  |

### Headers

| Header                     | Description                                                                    |
| -------------------------- | ------------------------------------------------------------------------------ |
| `Authorization`            | `Bearer YOUR_API_KEY` (required)                                               |
| `X-Shuttle-Customer-Id`    | End-customer ID (takes precedence over `customerId` in body)                   |
| `X-Shuttle-Callback-Url`   | Origin where **your** app serves its own tools, e.g. `https://app.example.com` |
| `X-Shuttle-Callback-Token` | Secret presented back on caller-hosted tool calls. Never stored.               |
| `X-Shuttle-Debug-Url`      | Deprecated alias of `X-Shuttle-Callback-Url`; also adds a `DebugUrl:` tag      |

#### Caller-hosted tools

If your flow uses tools served by **your own application**, give those tools a relative `webUrl` (`/api/v1/tools/search`) and send `X-Shuttle-Callback-Url` on every request — from production as much as from a laptop. Omit it and the tool call fails with an explicit error rather than falling back to a stored URL, because a run answered by the wrong instance looks exactly like a successful one.

Tools with an absolute `https://…` URL are unaffected and behave as they always have.

See [Caller-Hosted Tools](/tools/caller-hosted-tools) for the allowlist, per-run tokens, and migration steps.

### Example

```bash
curl -X POST https://app.promptshuttle.com/api/v1/flows/product_description/runs \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "parameters": {
      "product_name": "Smart Water Bottle",
      "tone": "playful"
    },
    "environment": "production",
    "tags": ["campaign-spring-2025"]
  }'
```

### Response

```json
{
  "id": "67a1b2c3d4e5f6a7b8c9d0e1",
  "timestamp": "2025-03-15T10:30:00Z",
  "flowId": "67a1b2c3d4e5f6a7b8c9d0e2",
  "tenantId": "67a1b2c3d4e5f6a7b8c9d0e3",
  "creditsUsed": 1250,
  "milliseconds": 1832,
  "usage": {
    "creditsUsed": 1250,
    "creditsLeft": 998750,
    "tokensIn": 85,
    "tokensOut": 142,
    "reasoningTokens": 0,
    "toolCost": 0,
    "costUsd": 0.00125
  },
  "flowRequest": {
    "parameters": {
      "product_name": "Smart Water Bottle",
      "tone": "playful"
    },
    "environment": "production",
    "tags": ["campaign-spring-2025"]
  },
  "responses": [
    {
      "model": "gpt-4o",
      "provider": "openai",
      "textResponse": "Meet the Smart Water Bottle — your hydration sidekick that...",
      "status": "completed",
      "finishReason": "stop",
      "usage": {
        "tokensIn": 85,
        "tokensOut": 142,
        "costUsd": 0.00125
      }
    }
  ],
  "warnings": []
}
```

### Response fields

| Field                      | Type         | Description                                              |
| -------------------------- | ------------ | -------------------------------------------------------- |
| `id`                       | string       | Unique run ID. Use this for feedback, agent trees, etc.  |
| `timestamp`                | datetime     | When the run was created.                                |
| `creditsUsed`              | long         | Total credits consumed (1M credits = $1).                |
| `milliseconds`             | integer      | Total execution time.                                    |
| `usage`                    | object       | Detailed usage breakdown.                                |
| `usage.creditsLeft`        | long         | Remaining tenant credit balance.                         |
| `responses`                | array        | Inference responses (one per LLM call in the chain).     |
| `responses[].textResponse` | string       | The LLM's text output.                                   |
| `responses[].toolCalls`    | array        | Any tool calls returned (if tool loop hasn't completed). |
| `responses[].citations`    | array        | Citation URLs (Perplexity only).                         |
| `warnings`                 | string array | Warnings about unused/unresolved parameters.             |

## Stop a running run

```
POST /api/v1/runs/{runId}/cancel
POST /api/v1/runs/cancel-all
```

`{runId}/cancel` stops one run **and every sub-agent beneath it**. `cancel-all` is the emergency stop: it stops every run currently executing for the tenant, including runs started by other users and applications.

Both return:

```json
{ "signalled": 3, "message": "Stopping 3 request(s). Runs finish their current model call first." }
```

### What stopping does and does not do

* **It is not instant.** Each executing request checks for the signal between tool-calling iterations, so a model call already in flight completes first. Expect a run to settle within roughly one model turn.
* **Single-shot calls cannot be stopped.** A request with no tools reaches the provider on its first and only turn, so there is no later checkpoint at which to notice. Stopping bites on agentic and tool-calling runs, which is where the money is.
* **Partial results are returned, not discarded.** The caller waiting on the run receives everything it produced before stopping, with a final response saying it was cancelled. That output was already paid for; stopping saves the work that had not happened yet.
* **The run is still billed** for what it spent.
* **`signalled` counts runs told to stop**, not runs that stopped. `0` means nothing was still executing — usually because the run finished first. That is not an error.
* **Panels and bulk calls** execute as independent runs with no shared parent, so cancelling one of them stops only that one. `cancel-all` reaches them.

A stopped run records `status: "succeeded"` if it produced anything (`"failed"` if it did not) plus a `cancelled` warning and a `cancelledAt` timestamp — status stays binary so cost and usage reporting keep working. Read `cancelledAt`, or the MCP `Run.Outcome` field, to tell a stopped run from a clean one. If it produced nothing at all, the waiting caller gets `409 Conflict`.

The OpenAI-compatible endpoint sets `X-PromptShuttle-Cancelled: true` on a partial answer.

## Discover parameters

Before running a flow, you can check what parameters it expects:

```
GET /api/v1/flows/{flowId}/parameters
```

**Query parameters:**

| Param         | Description                                   |
| ------------- | --------------------------------------------- |
| `environment` | Environment to resolve the active version for |
| `version`     | Specific version ID                           |
| `entrypoint`  | Specific template name                        |

**Response:**

```json
[
  { "token": "product_name", "source": "template" },
  { "token": "tone", "source": "template" }
]
```

## Submit feedback

Collect feedback on run quality for monitoring and fine-tuning:

```
POST /api/v1/feedback
```

```json
{
  "shuttleRequestId": "67a1b2c3d4e5f6a7b8c9d0e1",
  "score": 1,
  "text": "Great response, very accurate",
  "endUserId": "user_123"
}
```

| Field              | Type    | Required | Description                                |
| ------------------ | ------- | -------- | ------------------------------------------ |
| `shuttleRequestId` | string  | yes      | The run ID to attach feedback to           |
| `score`            | integer | yes      | Must be `+1` (positive) or `-1` (negative) |
| `text`             | string  | no       | Free-text feedback                         |
| `endUserId`        | string  | no       | Your end-user's identifier                 |

Feedback is upserted: submitting again for the same request replaces the previous feedback.

## Get agent execution tree

For flows that use agent tools (multi-agent orchestration), retrieve the full execution hierarchy:

```
GET /api/v1/flows/{flowId}/runs/{runId}/agent-tree
```

Returns a tree structure showing:

* Each agent invocation with its role, duration, and cost
* Tool calls made by each agent
* Child agents spawned
* Cumulative vs. direct credit usage
* Status and error messages

```json
{
  "requestId": "root_request_id",
  "agentRole": "main",
  "depth": 0,
  "durationMs": 5420,
  "creditsUsed": 3500,
  "totalTreeCreditsUsed": 8200,
  "toolCalls": [
    {
      "toolName": "research_agent",
      "toolType": "Agent",
      "durationMs": 2100,
      "spawnedRequestId": "child_request_id"
    }
  ],
  "children": [
    {
      "requestId": "child_request_id",
      "agentRole": "research_agent",
      "depth": 1,
      "creditsUsed": 4700,
      "children": []
    }
  ]
}
```


# Streaming (SSE)

PromptShuttle supports Server-Sent Events for real-time visibility into multi-agent execution. When you enable streaming on the OpenAI-compatible endpoint, you get a structured event stream covering the full lifecycle of your request — from agent starts through tool calls to final completion.

## Enabling streaming

Set `stream: true` on the OpenAI-compatible endpoint:

```bash
curl -N -X POST https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": [{"type": "text", "text": "Research quantum computing"}]}],
    "stream": true,
    "stream_options": {
      "include_usage": true,
      "usage_interval_ms": 3000
    }
  }'
```

## Response format

The response uses standard SSE format with `Content-Type: text/event-stream`:

```
id: 67a1b2c3d4e5f6a7b8c9d0e1
data: {"id":"...","timestamp":"...","requestId":"...","type":"requestStarted","data":{...}}

id: 67a1b2c3d4e5f6a7b8c9d0e2
data: {"id":"...","timestamp":"...","requestId":"...","type":"agentInferenceStarted","data":{...}}

...

data: [DONE]
```

Each event is a JSON object:

| Field       | Type     | Description                                                                        |
| ----------- | -------- | ---------------------------------------------------------------------------------- |
| `id`        | string   | Unique event ID                                                                    |
| `timestamp` | datetime | UTC ISO 8601 with milliseconds                                                     |
| `requestId` | string   | Root request ID                                                                    |
| `type`      | string   | Event type (see below)                                                             |
| `agentPath` | array    | Breadcrumb of agent roles from root to current (e.g. `["main", "research_agent"]`) |
| `depth`     | integer  | Nesting depth in the agent tree                                                    |
| `data`      | object   | Event-specific payload                                                             |

The stream ends with `data: [DONE]`.

The response includes the header `X-Request-Id` with the root request ID.

## Event types

### Lifecycle events

#### `requestStarted`

Emitted when the request begins processing.

```json
{
  "flowName": "my_flow",
  "contextId": "flow_object_id",
  "model": "gpt-4o",
  "hasTools": true,
  "toolCount": 3
}
```

#### `requestCompleted`

Emitted when the entire request (including all agents) finishes.

```json
{
  "durationMs": 5420,
  "totalCreditsUsed": 8200,
  "totalCostUsd": 0.0082,
  "totalTokensIn": 1250,
  "totalTokensOut": 890,
  "totalInferenceCount": 4,
  "totalToolCalls": 2,
  "totalAgentSpawns": 1,
  "maxDepthReached": 1,
  "result": { "textResponse": "..." }
}
```

#### `requestFailed`

Emitted if the request fails fatally.

#### Cancellation

A run stopped with [`POST /api/v1/runs/{runId}/cancel`](/api-reference/flow-execution#stop-a-running-run) emits an `error` event with `code: "RUN_CANCELLED"`, then finishes the stream normally — `requestCompleted` if it had produced anything, `requestFailed` if it had not. There is no separate terminal event, so existing clients need no changes: they see the run end where they already expect it to.

### Agent events

#### `agentStarted`

An agent (sub-template) begins execution.

```json
{
  "agentRole": "research_agent",
  "templateId": "template_object_id",
  "templateName": "research",
  "parentRequestId": "parent_id",
  "childRequestId": "child_id",
  "parameters": { "topic": "quantum computing" }
}
```

#### `agentInferenceStarted`

An LLM call begins within an agent.

```json
{
  "inferenceRequestId": "inference_id",
  "model": "gpt-4o",
  "provider": "openai",
  "messageCount": 5,
  "hasTools": true,
  "toolCount": 2
}
```

#### `agentInferenceCompleted`

An LLM call finishes.

```json
{
  "inferenceRequestId": "inference_id",
  "model": "gpt-4o",
  "provider": "openai",
  "durationMs": 1200,
  "usage": {
    "tokensIn": 450,
    "tokensOut": 200,
    "reasoningTokens": 0,
    "costCredits": 650,
    "costUsd": 0.00065
  },
  "finishReason": "stop",
  "toolCallCount": 0,
  "wasCached": false,
  "wasFallback": false
}
```

#### `agentCompleted`

An agent finishes all its work.

```json
{
  "childRequestId": "child_id",
  "agentRole": "research_agent",
  "durationMs": 3200,
  "totalCreditsUsed": 4700,
  "directCreditsUsed": 4700,
  "inferenceCount": 2,
  "toolCallCount": 1,
  "childAgentCount": 0,
  "status": "completed",
  "resultPreview": "Based on my research..."
}
```

#### `agentFailed`

An agent encounters an error.

### Tool events

#### `toolStarted`

A tool is about to be invoked.

```json
{
  "toolName": "search_api",
  "toolType": "external",
  "callId": "call_abc123",
  "arguments": { "query": "quantum computing breakthroughs 2025" },
  "targetUrl": "https://api.example.com/search",
  "originSource": "tool"
}
```

For external tools, `targetUrl` is the endpoint the call is actually placed against — after environment bindings and any caller-supplied callback origin, not the URL stored on the definition. `originSource` says which of those won:

| `originSource` | The endpoint came from                                                     |
| -------------- | -------------------------------------------------------------------------- |
| `tool`         | The tool's base `webUrl`                                                   |
| `environment`  | An environment override for this run's environment                         |
| `callback`     | The caller's [`X-Shuttle-Callback-Url`](/tools/caller-hosted-tools) header |

#### `toolCompleted`

A tool invocation finishes.

```json
{
  "toolName": "search_api",
  "toolType": "external",
  "callId": "call_abc123",
  "durationMs": 450,
  "status": "succeeded",
  "resultPreview": "Found 15 results for...",
  "resultSize": 4200,
  "targetUrl": "https://api.example.com/search",
  "originSource": "tool"
}
```

For agent-type tools, also includes:

```json
{
  "childRequestId": "spawned_request_id",
  "childCreditsUsed": 2300
}
```

#### `toolFailed`

A tool invocation errors. Same shape as `toolCompleted` with `"status": "failed"` and the error in `resultPreview`. `targetUrl` and `originSource` are repeated here so a failure is self-describing — a call that could not be bound to a host at all has no successful `toolStarted` to read them from.

### Usage events

#### `usageUpdate`

Periodic cost and progress updates (interval controlled by `usage_interval_ms`).

```json
{
  "cumulativeCreditsUsed": 3400,
  "cumulativeCostUsd": 0.0034,
  "cumulativeTokensIn": 800,
  "cumulativeTokensOut": 350,
  "activeAgents": 1,
  "completedAgents": 2,
  "elapsedMs": 4500
}
```

### System events

#### `heartbeat`

Keep-alive sent at `heartbeat_interval_ms` intervals.

```json
{
  "elapsedMs": 30000,
  "eventCount": 12
}
```

#### `error`

A recoverable or non-recoverable error occurred.

```json
{
  "code": "COST_LIMIT_EXCEEDED",
  "message": "Request exceeded the maximum cost of 50000 credits",
  "agentRole": "research_agent",
  "recoverable": false
}
```

## Filtering events

Use `event_types` in stream options to receive only the events you need:

```json
{
  "stream": true,
  "stream_options": {
    "event_types": ["requestCompleted", "error", "usageUpdate"]
  }
}
```

## Client example

```python
import json
import httpx

with httpx.stream(
    "POST",
    "https://app.promptshuttle.com/api/v1/chat/completions",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "model": "openai/gpt-4o",
        "messages": [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}],
        "stream": True,
    },
) as response:
    for line in response.iter_lines():
        if line.startswith("data: "):
            payload = line[6:]
            if payload == "[DONE]":
                break
            event = json.loads(payload)
            print(f"[{event['type']}] {json.dumps(event['data'], indent=2)}")
```


# Customers

Track per-customer usage and costs by associating requests with your end-customers. This is useful when you're building on top of PromptShuttle and want to attribute costs to your own users.

## How it works

1. Pass a customer identifier with each request
2. PromptShuttle auto-creates a customer record on first use
3. View per-customer usage and cost breakdowns

## Identifying customers on requests

Pass the customer ID on any request via header or body field:

### Via header (all endpoints)

```bash
curl -X POST https://app.promptshuttle.com/api/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "X-Shuttle-Customer-Id: customer_123" \
  -H "Content-Type: application/json" \
  -d '{ ... }'
```

### Via request body (flow execution)

```json
{
  "parameters": { "..." },
  "customerId": "customer_123"
}
```

The header takes precedence over the body field. The `user` field on the OpenAI endpoint also works as a fallback.

## Managing customers

### Create a customer

```bash
POST /api/v1/customers
```

```json
{
  "externalId": "customer_123",
  "name": "Acme Corp",
  "email": "contact@acme.com",
  "metadata": {
    "plan": "enterprise",
    "region": "eu"
  }
}
```

`externalId` must be unique within your tenant. This is the ID you pass in `X-Shuttle-Customer-Id`.

{% hint style="info" %}
Customers are auto-created when you pass an unknown `X-Shuttle-Customer-Id` or `customerId`. You only need to explicitly create customers if you want to set name, email, or metadata upfront.
{% endhint %}

### List customers

```bash
GET /api/v1/customers
```

### Get a customer

```bash
GET /api/v1/customers/{id}
```

### Update a customer

```bash
PUT /api/v1/customers/{id}
```

```json
{
  "name": "Acme Corporation",
  "metadata": { "plan": "enterprise", "region": "us" }
}
```

Only provided (non-null) fields are updated.

### Delete a customer

```bash
DELETE /api/v1/customers/{id}
```

### Deactivate a customer

Set `isActive: false` to exclude a customer from usage reports without deleting them:

```bash
PUT /api/v1/customers/{id}
```

```json
{ "isActive": false }
```

## Usage tracking

### Per-customer usage

```bash
GET /api/v1/customers/{id}/usage?period=30.00:00:00
```

Returns credits used, request count, and cost for the specified period (default 30 days).

### All customers usage

```bash
GET /api/v1/usage/by-customer?period=7.00:00:00
```

Returns usage summary grouped by all customers — useful for billing dashboards.

## Filtering logs by customer

The invocation log supports filtering by customer:

```bash
GET /api/v1/llm-logs?customerId=customer_123
```

This shows all requests attributed to that customer.