# Primate Vision API — full documentation corpus > Generated from https://primateintelligence.ai/docs. Spec: https://api.primateintelligence.ai/v1/openapi.json --- # Quickstart # Quickstart Primate Vision answers questions about video. Upload a video (or point us at a URL), ask a question in plain English, and get a deterministic answer with confidence and clip timestamps — no hallucinations. This page gets you from zero to a verified result in about a minute, **without creating an account**. ## 1. Get a key One command. No email, no card: ```bash doc-test id=quickstart-sandbox curl -s -X POST https://api.primateintelligence.ai/v1/sandbox ``` The response contains your `pv_test_` key plus a ready-to-analyze fixture video: ```json { "object": "sandbox_key", "api_key": "pv_test_…", "mode": "test", "expires_at": "2026-07-16T00:00:00Z", "fixture_video_id": "video_…", "fixture_prompt": "Is there a person in this video?", "expected_answer": "yes" } ``` Export it: ```bash export PRIMATE_API_KEY="pv_test_…" # from the response above ``` Test keys return **deterministic canned results** for the fixture corpus — perfect for CI. They can't spend credits or touch the GPU, and expire in 7 days. Graduating to a `pv_live_` key requires [signup](https://primateintelligence.ai/signup) (a billing gate, not an integration gate — your code doesn't change). ## 2. Create an analysis Your sandbox account comes pre-seeded with a fixture video. Ask a question about it — `Prefer: wait` holds the connection until the result is ready (up to 120s), collapsing the poll loop: ```bash doc-test id=quickstart-analyze env=VIDEO_ID curl -s -X POST https://api.primateintelligence.ai/v1/analyses \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -H "Prefer: wait=60" \ -d '{"video_id": "'"$VIDEO_ID"'", "prompt": "Is there a person in this video?"}' ``` ## 3. Read the result The response is the full analysis resource: ```json { "id": "an_…", "object": "analysis", "status": "completed", "result": { "answer": "yes", "confidence": 0.93, "clips": [{ "start_s": 1, "end_s": 4, "confidence": 0.93 }], "query_type": "object" } } ``` That's the whole loop: **video → question → answer**. `result.answer` is always `yes`, `no`, or `indeterminate`; `result.clips` tells you *where* in the video. ## The same thing in TypeScript ```typescript doc-test id=quickstart-ts sdk=ts import Primate from '@primate-intelligence/sdk'; const client = new Primate(); // reads PRIMATE_API_KEY const videos = await client.videos.list(); // fixture video is pre-seeded const analysis = await client.analyses.createAndWait({ video_id: videos.data[0].id, prompt: 'Is there a person in this video?', }); console.log(analysis.result?.answer, analysis.result?.confidence); ``` ## The same thing in Python ```python doc-test id=quickstart-py sdk=py from primate_intelligence import Primate client = Primate() # reads PRIMATE_API_KEY videos = client.videos.list() # fixture video is pre-seeded analysis = client.analyses.create_and_wait( video_id=videos["data"][0]["id"], prompt="Is there a person in this video?", ) print(analysis["result"]["answer"], analysis["result"]["confidence"]) ``` ## Next steps - **[Upload your own video](/docs/guides/uploading)** — presigned upload, URL ingest, or the SDK helper - **[Write better prompts](/docs/guides/prompts)** — what's answerable, structured queries, `unassessable_components` - **[Stop polling](/docs/guides/waiting)** — `Prefer: wait` vs webhooks - **[Building with an AI agent?](/docs/agents)** — the condensed machine-readable path - **[API reference](/docs/reference)** — every endpoint, generated from the OpenAPI spec --- # Primate Intelligence Public API — Agent Guide Machine-consumable reference for AI agents integrating with the Primate Intelligence Public API (`/v1`). --- ## Quick Start ### 1. Get a sandbox key (zero-human, instant) ```http POST /v1/sandbox ``` Returns a `pv_test_` key, a `fixture_video_id`, and a `fixture_prompt`. No auth required. Provisioning is IP-capped per client (rolling 24h) — hitting the cap returns `429 sandbox_limit_exceeded` with `retry_after`. **Response includes:** - `livemode: false` — all sandbox responses are always test-mode - `upgrade` — instructions for upgrading to a live key (see below) ### 2. Upgrade to a free live key (GitHub-verified) Call `POST /v1/keys/upgrade` with your `pv_test_` key. Receives a real `pv_live_` key with a **free credit grant** (currently ledgered as 6,000 seconds; moving to a **108,000-frame budget** — up to 30 hours of monitoring at 1 fps — see `signup_grant_frames` in `GET /v1/credit-pricing`). No email, no card — but a **verified GitHub account is required** (one free grant per GitHub account, ever). **Step A — first call returns the GitHub device-flow bootstrap:** ```http POST /v1/keys/upgrade Authorization: Bearer pv_test_ ``` Returns `403 github_verification_required`. The error `details` carry everything needed: `client_id`, `device_code_url` (`https://github.com/login/device/code`), and `access_token_url` (`https://github.com/login/oauth/access_token`). **Step B — GitHub device flow (standard):** 1. `POST https://github.com/login/device/code` with `client_id` (and `Accept: application/json`) → `{device_code, user_code, verification_uri, interval}` 2. Show the user `verification_uri` + `user_code` (or open it in a browser); the user signs in to GitHub and enters the code 3. Poll `POST https://github.com/login/oauth/access_token` with `client_id`, `device_code`, `grant_type=urn:ietf:params:oauth:grant-type:device_code` every `interval` seconds until it returns `access_token` No scopes are requested — the token only proves account identity. **Step C — retry the upgrade with the token:** ```http POST /v1/keys/upgrade Authorization: Bearer pv_test_ Content-Type: application/json {"github_token": "gho_..."} ``` The token is verified with GitHub and **discarded — never stored** (only the numeric GitHub account id is retained to enforce one-grant-per-account). A GitHub account that already claimed its grant gets `409 github_account_already_used` — use the billed claim flow (step 3) instead. Response includes a `pv_live_` key and `tier: "free_grant"`. Free-grant keys: - Run real GPU inference (`livemode: true`) - Have no overage — when the grant is exhausted, analyses return `402 grant_exhausted` with a link to the claim flow for billed keys ### 3. Get a billed live key (device-code flow) ```http POST /v1/keys/request ``` Returns `{device_code, claim_url, poll_interval}`. Direct the user to `claim_url`. Poll: ```http GET /v1/keys/request/{device_code} ``` Returns `{status: "pending"}` until the user approves in the browser. On approval returns the billed `pv_live_` key ONCE (subsequent polls return `410 Gone`). --- ## The `livemode` Field **Every public API resource includes a top-level `"livemode": boolean` field** (Stripe-style). | Value | Meaning | |-------|---------| | `true` | Request used a `pv_live_` key — results are real GPU inference | | `false` | Request used a `pv_test_` key or sandbox — results are deterministic canned fixtures | **Agents SHOULD verify `livemode` matches their expectation before relaying results to users or downstream systems.** A result with `livemode: false` is a fixture/test result and must never be treated as evidence about real video content. Example analysis response: ```json { "object": "analysis", "livemode": true, "status": "completed", "result": { "answer": "yes", "confidence": 0.97, ... } } ``` --- ## Test Key Restrictions Test (`pv_test_`) keys **only analyze the fixture video** seeded by `/v1/sandbox`. Attempting to analyze any other video returns: ```json { "error": { "code": "test_key_fixture_only", "message": "Test keys can only analyze the provided fixture video. Create a live key to analyze your own uploads.", "status": 403 } } ``` This prevents fabricated results on real video content. --- ## Video Metadata After `POST /v1/videos/{id}/complete`, the API probes the uploaded S3 object and populates: | Field | Type | Description | |-------|------|-------------| | `duration_s` | number \| null | Duration in seconds | | `width` | integer \| null | Frame width in pixels | | `height` | integer \| null | Frame height in pixels | | `fps` | number \| null | Frames per second | Agents can use these fields to verify the right file arrived before submitting an analysis. Probe failures are non-fatal — fields remain `null` if probing fails. --- ## Free Re-runs (platform incidents) When an analysis **fails because of a platform incident** (GPU outage, degraded inference), the failed analysis carries **`rerun_eligible: true`**. Call `POST /v1/analyses/{id}/rerun` to create a **fresh analysis at no charge** — same video, prompt/query, model, and options; new `an_` id; `usage` stays `null` on the re-run (never billed). One free re-run per failed analysis. Analyses that fail for non-platform reasons return `409 rerun_not_eligible`. If the re-run dispatch itself fails (503), your free re-run is NOT consumed — retry later. --- ## Demo Videos (public onboarding samples) `GET /v1/demo-videos` (no auth) returns curated sample videos with **precomputed results** for example prompts — `{videos: [{id, display_name, duration_seconds, thumbnail_url, results_by_prompt}]}` where `results_by_prompt` maps prompt text → `{answer, confidence, query_type, video_url, computed_at}`. Use these to inspect real result shapes before uploading anything. Documented subset only — extra fields exist but may change. --- ## Video Playback & Result History (PRI-496) **Source playback** — `GET /v1/videos/{id}` (and the list) populates `media: {url, expires_at}` on `ready` videos: a signed playback URL for the original source video, **fresh on every read with a 1-hour TTL**. When it expires, re-fetch the video for a new one — old videos always stay playable. `media` is `null` while the video is not ready or has no stored source object. **Per-video result history** — `GET /v1/videos/{id}/analyses` lists every analysis run against a video (newest first), with the same list envelope and filters as `GET /v1/analyses` (`status`, `limit`, `starting_after`). It is the resource-nested spelling of `GET /v1/analyses?video_id={id}` — use whichever fits your client. Together with `Analysis.artifacts` (annotated result video, also sign-on-read) this gives full library semantics: list videos, replay sources, and rebuild each video's analysis history. --- ## Error Codes | Code | HTTP | Retryable | Description | |------|------|-----------|-------------| | `test_key_fixture_only` | 403 | no | Test key tried to analyze a non-fixture video. Use a live key. | | `grant_exhausted` | 402 | no | Free-grant credits depleted. Use the claim flow to get a billed key. | | `insufficient_credits` | 402 | no | Live key ran out of credits. Add credits or upgrade. | | `sandbox_limit_exceeded` | 429 | yes | Too many sandbox provisions from this IP. Retry after the window. | | `upgrade_limit_exceeded` | 429 | yes | Too many free-grant upgrades from this IP today. | | `invalid_api_key` | 401 | no | Key not found or malformed. | | `key_expired` | 401 | no | Key has passed its `expires_at`. | Full registry: `GET /v1/errors` --- ## Key Lifecycle ``` POST /v1/sandbox → pv_test_ key (livemode: false, fixture-only, 7-day TTL) │ ▼ POST /v1/keys/upgrade (auth: pv_test_ key + github_token from GitHub device flow) → pv_live_ key (livemode: true, tier: free_grant, 6,000s grant, 30-day idle expiry) (one free grant per GitHub account — duplicate → 409 github_account_already_used) │ (grant exhausted → 402 grant_exhausted) ▼ POST /v1/keys/request → GET /v1/keys/request/{code} → billed pv_live_ key (device-code claim flow — user approves in browser) ``` --- ## What the Model Supports The current model is **`Darwin-preview-1.3B`** (status: **preview**, default — see `GET /v1/models`). Preview means early access: detection quality and the set of reliably detected actions are still evolving, and results may improve or change between checkpoint updates. The API contract (request/response shapes, billing) is stable. Full capability list: [Supported actions](https://primateintelligence.ai/docs/supported-actions). Design prompts as targeted visual questions. The model natively handles: | Type | Example | `answer` | `detected_count` | `clips` | Accuracy | |------|---------|----------|-----------------|--------|---------| | **Presence** | "Is there a person?" | yes/no/indeterminate | — | ✅ when yes | Stable | | **Absence** | "Are there no dogs?" | yes/no | — | — | Stable | | **Action** | "Is someone walking?" | yes/no/indeterminate | — | ✅ when yes | Beta | | **Count — bare** | "How many people walking?" | — | ✅ integer | — | Stable | | **Count — threshold** | "More than 2 people?" | yes/no | ✅ integer | — | Stable | | **Compound** | "Person AND dog?" | yes/no | — | ✅ when yes | Stable | | **Attribute** | "Is anyone in a red jacket?" | description | — | — | Stable | | **Location** | "Where is the dog?" | description | — | — | Stable | | **State** | "Is the door open?" | description | — | — | Stable | | **Segment** | "Show me where the person is" | — | — | ✅ with masks | Stable | > **Accuracy note:** Action queries (motion/activity detection) are **beta accuracy** — results may be less reliable than presence/counting queries on short or fast-moving clips. **Not supported:** audio/sound, identity (who, not what), OCR/text in frame, subjective judgment, exhaustive count over long multi-hour footage. Open-ended prompts ("Tell me what you see") return a description but have lower accuracy — prefer a targeted question. Full guide: [Prompts & queries](https://primateintelligence.ai/docs/guides/prompts) --- ## Remote MCP Endpoint Primate Intelligence exposes a hosted [Model Context Protocol](https://modelcontextprotocol.io) server at: ``` https://api.primateintelligence.ai/mcp ``` Transport: **Streamable HTTP** (MCP spec 2025-03-26+, stateless mode). No npx or local install required — any remote agent can connect directly. **Dual-era: 2025-11-25 and 2026-07-28 both served on the same endpoint.** A request carrying per-request `_meta` (`io.modelcontextprotocol/protocolVersion` etc.) is served under the 2026-07-28 revision; an `initialize` handshake selects the 2025-era legacy path. `server/discover` (2026-07-28, [spec](https://modelcontextprotocol.io/specification/2026-07-28/server/discover)) answers **without auth** — pure protocol metadata, no tools, no user data — since connector-directory scanners (OpenAI's Scan Tools) probe it before OAuth completes (PRI-514). Response shape, matching the spec exactly: ```json { "jsonrpc": "2.0", "id": 1, "result": { "supportedVersions": ["2026-07-28", "2025-11-25"], "capabilities": { "tools": {} }, "resultType": "complete", "_meta": { "io.modelcontextprotocol/serverInfo": { "name": "primate-intelligence", "version": "0.3.0" } } } } ``` Note the field is `supportedVersions` (not `protocolVersions`), and server identity lives only in `_meta['io.modelcontextprotocol/serverInfo']` — there is no top-level `serverInfo` field. `POST /mcp` also tolerates a missing/nonstandard `Content-Type` header rather than 415ing (same PRI-514 fix — the scanner posts without `application/json`). All other methods (`tools/list`, `tools/call`, `prompts/list`, `resources/list`) remain auth-gated on both eras. ### Authentication Pass your Primate Vision API key as a Bearer token in every request: ``` Authorization: Bearer pv_live_ ``` Test keys (`pv_test_…`) also work and return deterministic fixture results. Get a free key: ```bash # Instant sandbox key (no auth required) curl -X POST https://api.primateintelligence.ai/v1/sandbox # Upgrade to a live key with a 6,000-second free credit grant # (requires a GitHub device-flow token — calling without one returns 403 # github_verification_required with the client_id + URLs to complete it) curl -X POST https://api.primateintelligence.ai/v1/keys/upgrade \ -H "Authorization: Bearer pv_test_" \ -H "Content-Type: application/json" \ -d '{"github_token": "gho_..."}' ``` Requests without a valid key receive a 401 JSON-RPC error with provisioning guidance. ### Available Tools The remote endpoint exposes the same 10 tools as the npx package: `create_video_from_url`, `create_analysis`, `validate_analysis`, `create_analysis_batch`, `get_analysis`, `wait_for_analysis`, `list_models`, `get_usage`, `get_credits`, `get_test_fixture`. - `validate_analysis` — free dry-run (`validate_only: true`): compiled query, assessability, and cost estimate before you spend credits. - `create_analysis_batch` — 2–10 prompts on one video; the first is full price, each additional is billed at 50%. - `get_credits` — balance plus the per-analysis transaction ledger (`GET /v1/credits`); prefer it over `get_usage` for auditing what each analysis cost. Every tool declares an `outputSchema` (visible in `tools/list` and on the static server card at `/.well-known/mcp/server-card.json`), generated from the same schemas that validate the public /v1 responses — and every result carries `structuredContent` conforming to it, alongside the human-readable JSON text content. **`wait_for_analysis` response envelope:** the tool returns `{ analysis, retry }` — the [Analysis resource](https://primateintelligence.ai/docs/reference#analyses) unmodified under `analysis`, plus `retry: null` when the analysis reached a terminal state, or `retry: { reason: "timeout", note }` when the wait expired (call `wait_for_analysis` or `get_analysis` again). Earlier versions merged an `_mcp_note` field into the analysis object on timeout; that field is gone. ### Client Configuration Examples **Claude Desktop / MCP config JSON:** ```json { "mcpServers": { "primate-intelligence": { "type": "streamable-http", "url": "https://api.primateintelligence.ai/mcp", "headers": { "Authorization": "Bearer pv_live_" } } } } ``` **ChatGPT app directory / custom GPT:** ``` URL: https://api.primateintelligence.ai/mcp Auth: Bearer token → your pv_live_ or pv_test_ key ``` **OpenAI Agents SDK (Python):** ```python from agents.mcp import MCPServerStreamableHttp server = MCPServerStreamableHttp( url="https://api.primateintelligence.ai/mcp", headers={"Authorization": "Bearer pv_live_"}, ) ``` ### MCP Registry The server is also listed in the MCP registry at `ai.primateintelligence/mcp` with both the npx stdio package and this remote entry. See `mcp/server.json` in the repository. --- ## Result Contract ```json { "object": "analysis", "livemode": true, "status": "completed", "result": { "answer": "yes", "confidence": 0.97, "detected_count": 3, "clips": [{"start_s": 1.2, "end_s": 3.4, "confidence": 0.94}] }, "usage": { "billed_seconds": 6, "credit_balance_after": 5994 } } ``` - `result.answer` ∈ `yes | no | indeterminate`. `indeterminate` = model couldn't commit; don't ship a decision. - `result.detected_count` — populated for count-intent queries. Integer ≥ 0. - `result.confidence` ∈ [0, 1]. Zero confidence with no positive detections returns `indeterminate`, not `no`. For count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected. See the [Count queries](#count-queries--the-contract) section for full semantics. - `result.clips[]` — present for presence/action/compound/segment when `answer: "yes"`. Null otherwise. - `query.unassessable_components[]` — lists what couldn't be evaluated (e.g. audio). API answers what it can. - `usage.billed_seconds` — seconds charged for this analysis. - `usage.credit_balance_after` — the balance immediately after **this** analysis settled. Immutable point-in-time snapshot — it never changes as later analyses run. A `null` value (`snapshot_unavailable`) means the analysis settled before the snapshot feature shipped (migration 075) and the original balance is unknowable; treat it as missing data rather than zero. - `livemode: true` = real GPU inference. Never relay `livemode: false` results as evidence about real content. - `origin` ∈ `api | console | system` — how the analysis was created (public API, dashboard upload, or internal). **System-initiated analyses are never billed** — only `api`/`console` analyses reserve and settle credits. - `narrative` (when created with `options: {narrative: true}`): `{status: "generating"|"ready"|"failed", entries: [{t_s, text}]}` — timestamped event sentences for the video. Generation runs asynchronously **after** the analysis completes: the first completed read may show `status: "generating"` with empty entries — poll the GET until `ready`. Included in the analysis price (no surcharge). Without the opt-in: `narrative: null`, always. - `artifacts` (completed analyses with an annotated result video): `{annotated_video_url, expires_at}` — a **fresh 1-hour signed URL on every GET**; when it expires, re-fetch the analysis for a new one. `null` when no annotated video exists. --- ## Count queries — the contract A count query ("how many X…", "more than N X?") succeeds like this: ```json doc-test id=count-contract-success { "answer": "yes", "confidence": 0.94, "detected_count": 3, "clips": [{ "start_s": 1.2, "end_s": 3.4, "confidence": 0.94, "terms": { "person": 0.94 } }], "term_confidences": { "person": 0.94 }, "query_type": "object", "video_duration_s": 6.0, "indeterminate_reason": null } ``` And fails like this: ```json doc-test id=count-contract-failure { "answer": "indeterminate", "confidence": 0, "detected_count": 0, "clips": [], "term_confidences": {}, "query_type": "object", "video_duration_s": 6.0, "indeterminate_reason": "nothing_detected" } ``` **Rules:** - `detected_count` is only meaningful when `answer` is determinate (`yes` or `no`). - `detected_count: 0` with `answer: "indeterminate"` means the pipeline found **nothing assessable** — NOT "zero occurrences". - A true "zero occurrences" result is `answer: "no"`, `detected_count: 0`. - **Never branch on `detected_count` without checking `answer` first.** **Confidence semantics for count queries:** for count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected. `{"confidence": 0.94, "detected_count": 3}` asserts "there are 3" at 94% confidence, not "there is at least one". Numerically it is derived from the per-term detection confidences underlying the count (max across detected clips, clamped to [0, 1]). When `answer` is `indeterminate`, `confidence` is always 0 (enforced by the API regardless of what inference returned); `indeterminate_reason: "nothing_detected"` additionally forces `detected_count` to 0 because no objects were seen at any confidence level. Treat `confidence ≥ 0.7` as reliable for production decisions, `0.4–0.69` as marginal (verify with a second query or tighter prompt), and `< 0.4` as unreliable. Example: `{"answer": "yes", "confidence": 0.94, "detected_count": 3}` means the model is 94% confident the count is exactly 3 — act on it; `{"answer": "indeterminate", "confidence": 0, "detected_count": 0, "indeterminate_reason": "nothing_detected"}` means no objects were detected at any confidence — do not infer "zero occurrences". --- ## Query-Type Maturity | Query type | Maturity | Notes | |---|---|---| | **presence** (`object`) | GA | "Is there X in this video?" — most reliable query form. | | **counting** (`object` + count intent) | GA | "How many X?" — returns `result.detected_count`. | | **action** / **temporal** | Beta | "Does X happen during the first N seconds?" — duration-sensitive; result reliability depends on accurate video duration metadata. High `duration_mismatch` rate if source duration is missing. | | **open-ended** | Rejected | Freeform prompts that don't resolve to a closed yes/no or count question are rejected immediately with `answer: "indeterminate"`, `indeterminate_reason: "unsupported_query_form"`, at **zero cost** — no credits billed. Rephrase as a presence or count query. | **Agent guidance:** - Prefer presence and counting queries for production pipelines. - Action/temporal queries require the source video to have accurate `duration_s` metadata (populated after `POST /v1/videos/:id/complete`). - If you receive `indeterminate_reason: "unsupported_query_form"`, rewrite the prompt before retrying — retrying the same open-ended form will always fail. The result also carries `unsupported_action: {reason, docs_url}` (PRI-516) — `docs_url` points to the [supported-actions capability list](https://primateintelligence.ai/docs/supported-actions), which documents exactly what the model can and cannot detect (including the full 532-class action vocabulary). Consult it before rewriting. --- ## Action pipeline verification A ground-truth walking fixture is available for integration testing. Use it to verify the full pipeline (upload → analyze → result) against a known answer before deploying to production. ### Walking fixture | Field | Value | |-------|-------| | **URL** | *(URL pending — clip `IMG_3281.MOV` not yet uploaded to CDN; see Slack thread for ETA)* | | **CDN key** | `fixtures/walking-ground-truth.mov` on `d3silto12vjvss.cloudfront.net` (pending upload) | | **Scene** | Outdoor walkway; several people walking. Clip is ~6–10 seconds. | | **Ground truth** | At least 2 people visibly walking during the clip. | | **Expected ideal API answer** | `POST /v1/analyses` with prompt `"Is someone walking?"` → `answer: "yes"`, `confidence ≥ 0.8`; with prompt `"How many people are walking?"` → `detected_count ≥ 2`, `confidence ≥ 0.7`. | > **Note:** this fixture is for action-query validation. For basic presence testing, use the sandbox fixture (`POST /v1/sandbox` → `fixture_video_id`). --- ## Batch analyses & discounts Run 2–10 prompts against the same video in a single request: ```http POST /v1/analyses/batch Authorization: Bearer pv_live_ { "video_id": "video_01J...", "prompts": ["Is there a person?", "How many people are walking?"] } ``` **Pricing rule:** the first prompt bills at full price; each additional prompt is discounted by `batch_discount_pct` (served by `GET /v1/credit-pricing`; currently **50%**, i.e. additional prompts bill at half price). The discount is config-driven — read it from the endpoint instead of hardcoding. Credits are reserved at the discounted rate, so the discount is observable directly in the ledger (`GET /v1/credits` will show a smaller `seconds_delta` for analysis 2+). **Worked example:** 6-second video at 1¢/s: - Prompt 1 (full price): 6s × 1¢ = **6¢** - Prompt 2 (50% off): 3s × 1¢ = **3¢** - Prompt 3 (50% off): 3s × 1¢ = **3¢** - Total for 3 prompts: **12¢** (vs 18¢ if billed separately) **Response shape:** ```json { "object": "analysis_batch", "id": "batch_...", "video_id": "video_01J...", "analyses": [ { "id": "analysis_01J...", "status": "queued", ... }, { "id": "analysis_01J...", "status": "queued", ... } ], "pricing": { "full_price_prompts": 1, "discounted_prompts": 1, "discount_pct": 50 } } ``` Each analysis in `analyses[]` can be polled individually via `GET /v1/analyses/{id}`. **Listing / filtering analyses:** `GET /v1/analyses` supports `status`, `video_id`, `model`, `created_after`, `created_before` plus cursor pagination (`limit`, `starting_after`). The `status` filter takes the public vocabulary: `queued | preparing | analyzing | rendering | completed | failed | canceled` (use `status=analyzing` for currently-running analyses; unknown values → 400 `validation_failed`). **Dry-run (`validate_only`):** pass `"validate_only": true` to parse all prompts and get per-prompt cost estimates without reserving credits or creating jobs (HTTP 200): ```json { "object": "analysis_batch_preview", "video_id": "video_01J...", "prompts": [ { "index": 0, "query": {...}, "parse_mode": "heuristic", "assessable": true, "estimated_seconds": 10, "estimated_cost_usd": 0.10, "discount_pct": 0 }, { "index": 1, "query": {...}, "parse_mode": "heuristic", "assessable": true, "estimated_seconds": 5, "estimated_cost_usd": 0.05, "discount_pct": 50 } ], "pricing": { "full_price_prompts": 1, "discounted_prompts": 1, "discount_pct": 50, "estimated_total_seconds": 15, "estimated_total_cost_usd": 0.15 } } ``` Estimates are `null` when the video has no known duration (still processing or URL-sourced before probe completes). > **Note:** sending a `prompts` array to `POST /v1/analyses` returns a validation error with a pointer to this endpoint. --- ## Streaming (real-time video over WebRTC) `GET /v1/models` advertises streaming support per model — `Darwin-preview-1.3B` carries `capabilities: {prompt, structured_query, narrative, streaming: true}`. (The former id `darwin-1.3` remains accepted on requests as a deprecated alias — responses always emit the canonical id.) Streams analyze live video in real time — same prompt semantics and same result contract as file analyses, delivered per-frame over a signaling WebSocket. ### Lifecycle ``` POST /v1/streams → queued|ready → (WS join → offer/answer/ICE) → live → ended ``` 1. `POST /v1/streams {prompt}` (secret key) → returns `signaling.url`, `ice_servers` (STUN + TURN with credentials), `limits`. 2. `POST /v1/client_tokens {scopes: ["streams:signal"], stream_id, ttl_s}` → `pvct_` token for the device. The signaling WS **never** accepts secret keys. 3. Connect `signaling.url?token=pvct_…`, send `join`, receive `ready` (or `queued {position}`), then standard WebRTC offer/answer + **bidirectional trickle ICE** (`ice {candidate}` messages flow both ways — the server trickles late-gathered srflx/relay candidates after its answer; keep consuming them). 4. `live` → `result {frame_num, detections}` per analyzed frame, `metering {elapsed_s, billed_s, session_remaining_s}` every 5s, `warning {remaining_s}` before credit exhaustion, `end {reason}`. 5. Mid-stream, send `{"type": "update_prompt", "prompt": "…"}` (client→server, ≤2000 chars) to change the question without reconnecting — the server recompiles it, applies it to the live engine, and confirms with `{"type": "prompt_updated"}`. Subsequent result frames echo the new prompt. Invalid prompts return `{"type": "error", "code": "validation_failed" | "parse_failed"}` and the old prompt stays active. On narrative-opted-in streams, expect `status` events (e.g. `recalculating`) as the engine rebuilds context after the update. ### Metering tick fields Every 5s while live: `{type: "metering", elapsed_s, billed_s, session_remaining_s, balance_s}`. - `session_remaining_s` — seconds remaining in **this session's** credit reservation (session cap), NOT your account credit balance. Account balance lives at `GET /v1/billing/credits` (`balance_seconds`). - `balance_s` — **deprecated** alias of `session_remaining_s` (identical value). Removed **~2026-08-28**; migrate reads to `session_remaining_s`. - `elapsed_s` / `billed_s` — live-clock seconds elapsed = billed (identical by construction; join/negotiation free). ### Result sampling (results are sampled, not per-frame) Results are **sampled**, not emitted for every source frame — inference cadence is adaptive (roughly every 8th source frame under load). `frame_num` is the **source-frame index** the result was computed on (so gaps between consecutive `frame_num` values are normal), and `results_summary.result_frames` on the terminal resource counts the number of **result events emitted** — the two are different axes. Expect results at roughly 8–15/s depending on load; do not assume a fixed cadence. ### No-media warning (the server tells you when YOUR media is the problem) If the WebRTC transport connects but **no decodable video frame arrives within 5 seconds**, the server pushes a `warning` event on the signaling WS (re-warned once at 15s, then quiet): ```json {"type": "warning", "code": "no_media_frames", "transport_connected": true, "elapsed_s": 5, "packets_received": 312, "frames_decoded": 0, "hint": "…"} ``` Read `packets_received` to self-diagnose: - `packets_received: 0` — your client is **not sending media** (track not attached, muted, or the capture source is dead). - `packets_received > 0` with `frames_decoded: 0` — media is arriving but is **not decodable** (wrong codec — must be VP8 or H.264 — or the source produces no real frames, e.g. a canvas capture without an active draw loop). If the session then ends without ever going live, the terminal resource records `end_reason: "media_timeout"` with the same counters in `failure_diagnostic`, and bills 0. ### Terminal results_summary The ended stream resource carries `results_summary: {result_frames, frames, last_detections}`: - `result_frames` — number of result events emitted over the session (sampled — see “Result sampling” above; this is NOT a source-frame count). - `frames` — **deprecated** alias of `result_frames` (identical value). Removed **~2026-08-28**; migrate reads to `result_frames`. - `last_detections` — the final `result.detections[]` rows, same contract as live results. ### Streaming result contract (identical to file analyses — transport never changes enums) Each `result.detections[]` row: - `answer`: `"yes" | "no" | "indeterminate"` — lowercase, same enum as file-API results - `confidence`: 0..1; `term_confidences`: per-term map - `prompt`: echoed **exactly as you submitted it** (byte-identical) - `query_type`, `search_terms`, `prompt_intent`: same vocabulary as `POST /v1/parse` - `frame_num`, `elapsed_s`, `session_id`, `session_fps` - `timing`: server-side latency telemetry (per-stage breakdowns: inference, encode, per-hop p50s). Rich, production-grade, safe to log — field names may grow, existing names are stable. - `narrative_update` (only when the stream was created with `options: {narrative: true}`): `{t_s, text}` — a new narrative sentence, emitted event-driven when the engine detects an appearance/disappearance/action, NOT per frame. Absent (not `null`) on frames without one. Terminal `results_summary.last_detections` never carries it. ### Status events (narrative-opted-in streams only, PRI-496) Streams created with `options: {narrative: true}` also receive a `status` server→client event on the signaling WS: ```json {"type": "status", "status": "prompt_context" | "combined_prompt" | "recalculating", "message": "…"} ``` Use these to order/annotate narrative entries (e.g. mark when the engine is recalculating context after a prompt update). The vocabulary is a **closed set** — exactly these three values; internal engine statuses never leak. Streams without the narrative opt-in never receive `status` events (their event vocabulary is unchanged). ### Session recordings (PRI-496) Create the stream with **`recording: true`** (top-level boolean) to retrieve the server-side session recording afterwards: 1. The stream resource carries `recording: {status}` — `recording` while live, `available` once ended with a stored recording, `failed` if capture failed, `none` if nothing was stored. Streams without the opt-in have `recording: null`. 2. Once ended with `recording.status: "available"`, call **`GET /v1/streams/{id}/recording`** → `{url, expires_at, content_type: "video/h264", container: "h264-annex-b"}`. The URL is **signed fresh on every GET with a 1-hour TTL** — re-fetch for a new one; old recordings always stay retrievable. 3. The recording is a raw H.264 Annex-B elementary stream (the exact annotated frames the client saw). Lossless remux to MP4: `ffmpeg -i recording.h264 -c copy recording.mp4`. ### end_reason vocabulary (honest by construction) | Reason | Meaning | Billed? | |---|---|---| | `completed` | Normal end after the session was live | live seconds | | `canceled` | Ended before ever going live (client action) | 0 | | `ice_failed` | WebRTC ICE never connected — see `failure_diagnostic` | 0 | | `media_timeout` | Transport connected but no media/results flowed | 0 | | `insufficient_credits` | Balance exhausted mid-stream | live seconds | | `timeout` | `limits.max_session_s` cutoff reached | live seconds | | `error` | Server-side failure | live seconds (0 if never live) | **A stream that never went live is never `completed`** — the API enforces this. On `ice_failed`, the terminal resource carries `failure_diagnostic` (`{local_candidates, hint}`) — the server-side ICE candidate summary. If `local_candidates` shows only private addresses (`172.x`, `10.x`), the fault is server-side config; if it shows srflx/relay candidates, check your client's network path to the TURN servers in `ice_servers`. ### Client profiles that MUST work (and are CI-tested) - Browser on home/office NAT (webcam) — the easy case - **Datacenter/CI/edge-fleet clients: UDP-blocked, TURN-over-TCP:443 relay-only** — the demanding case. The server advertises a public host candidate and srflx/relay candidates (trickled when gathering outlasts the answer). Regression-tested every deploy with a relay-only aiortc client. ### Stream a file (regression harness recipe) Re-running a known file through the live path is the natural streaming regression test. The examples repo ships `stream_file.py` (aiortc): loops a video file as the WebRTC source, applies your prompt, and audits the full session (candidates, states, results, billing) to JSON. See `python/streaming/` in the examples repo: **https://github.com/Primate-Intelligence/primate-examples** (public — no auth needed) ### Billing Billed per second of **live clock time** (join/queue/negotiation free), from the same credit ledger as uploads. Sessions that never go live bill exactly 0. Metering ticks arrive every 5s; `usage.billed_seconds` on the terminal resource is the reconciled charge. --- ## Pricing **Pricing status: preview.** `GET /v1/credit-pricing` returns `pricing_tier: "preview"` plus a plain-language `pricing_tier_note` while preview pricing is in effect. Policy: the rate is early-access and may change — up or down — as the model matures, customer demand develops, and processing costs change. Rate changes apply to new processing only — purchased credits keep their value — and are announced in the [changelog](./changelog.md) before taking effect. Agents should (1) fetch the live rate per session rather than caching or hardcoding it, (2) relay to users that the rate is preview-stage and what can move it, and (3) treat `pricing_tier` transitioning away from `"preview"` as the signal that rates have stabilized. Billing is metered in **source-clock video-seconds** — the duration of video analyzed, independent of resolution or fps. Current rates are discoverable (public, no auth): ```http GET /v1/credit-pricing ``` Key fields: | Field | Meaning | |-------|---------| | `price_per_second_cents` | Price per billed video-second, in cents | | `price_per_frame_cents` | Price per processed frame, in cents (PRI-552). Per-second remains the billed ledger unit until the announced metering cutover — always read the live rate from this endpoint | | `signup_grant_frames` | Free signup grant as a frame budget — 108,000 frames = 1 hour @ 30fps (6 h @ 5fps, 30 h @ 1fps). Ledger enforces `signup_grant_seconds` until the metering cutover | | `signup_grant_seconds` | Free credit-seconds on signup / free-grant upgrade | | `allowed_purchase_cents` | Preset top-up amounts | | `batch_discount_pct` | Percent discount for each prompt after the first in `POST /v1/analyses/batch` | | `batch_min_prompts` / `batch_max_prompts` | Allowed prompts-per-batch range | | `pricing_tier` | Pricing-policy status (PRI-554): `"preview"` while preview pricing is in effect; `"standard"` once rates stabilize | | `pricing_tier_note` | Plain-language pricing policy — why the rate is preview-stage and what can move it | **Cost of an analysis** = `usage.billed_seconds × price_per_second_cents`. Worked example: a 6-second analysis at 1¢/second → `6 × 1 = 6` cents = **$0.06**. Always read rates from the endpoint rather than hardcoding — pricing is config-driven and can change without an API version bump. --- ## Public vs Console-Internal Boundary (PRI-496 close-out) The public contract is exactly what `GET /v1/openapi.json` documents. A small set of endpoints that the Primate Intelligence web console uses are **intentionally console-internal** — they are not part of the public contract, carry no compatibility promise, and agents should never call them: | Endpoint | Status | Public alternative | |----------|--------|--------------------| | `GET /v1/stream/config` | Console-internal | Stream limits arrive in the `POST /v1/streams` create response (`limits`, ICE servers) — no separate config fetch needed | | `POST /v1/feedback` | Console-internal | None (console narrative-feedback votes only) | | `GET /v1/prompts` | Console-internal | `GET /v1/demo-videos` — public onboarding samples with precomputed results per prompt | | `GET /v1/model-info` | Superseded (kept for legacy console compat) | `GET /v1/models` — see parity note below | | `GET /v1/demo-videos` | **Public** | — (documented in openapi.json; no auth) | | `POST/GET /v1/admin/design-partner-agreements` (+`/:id`), `PATCH /v1/admin/design-partner-agreements/:id`, `DELETE /v1/admin/design-partner-agreements/:id`, `GET /v1/admin/design-partner-agreements/:id/pdf` | Console-internal (admin-only) | None — Admin UI creates click-to-sign Design Partner Pilot Agreement links. Create accepts `companyLegalName`, `useCase`, `targetMetrics` (required) and optional `valueOfSolving`, `scopeOfWork`, `estimatedCostToDeliver`, `priceUsd` (2026-08-13), `publicReferenceOptOut` (2026-08-14 — default token fee is $500; partner may decline PUBLIC use of name/logo/quote, private investor-facing reference is always retained regardless). `PATCH .../:id` (2026-08-14) sets `status` to one of `sent`\|`paid`\|`terminated`\|`completed` (the other states — `draft`, `signed`, `expired` — remain automated-only, set by the click-to-sign/pay flow). `DELETE .../:id` (2026-08-14) removes the row and its signed PDF from Storage. `GET .../:id/pdf` (2026-08-14) mints a 5-minute signed download URL for the signed PDF (404 if the agreement was never signed). | | `/v1/design-partner-agreements/by-token/:token` (get / `/sign` / `/create-checkout-session`) | Intentionally unauthenticated (token-gated) | None — external design partners follow an emailed share link; no Primate account exists. Not part of the public API contract. | **`/v1/model-info` vs `/v1/models` field parity (verified 2026-08-04):** the public `/v1/model-info` whitelist returns only `{model_name, status}`, where `status` is a *live inference-server readiness* signal (`ready` / `inference_unavailable` on 503) and `model_name` is the raw inference-server brand string. `GET /v1/models` is the canonical catalog: stable public id, lifecycle `status` (e.g. `preview`), `default`, `aliases`, `capabilities` (prompt / structured_query / narrative / streaming), `sunset_at`. Everything an integration needs is on `/v1/models`; checkpoint provenance (filenames, step counts) is intentionally absent from BOTH public endpoints (admin-only via the gated console proxy — 2026-06-26 security decision). Use `/v1/models` for model discovery; treat `/v1/model-info` as internal. --- ## API versioning Every response — success, error, even 404 — carries an `X-Api-Version` header: ``` X-Api-Version: + ``` Example: `X-Api-Version: 0.1.0+a7c1993`. The value is computed once at boot and is stable for the lifetime of a deploy. **Agent guidance:** if the value changes between two requests in the same session, a deploy happened mid-session. Re-check the [changelog](https://api.primateintelligence.ai/docs/changelog.md) before attributing new behaviour to a bug in your integration. --- ## Rate Limits All public endpoints respond with `X-RateLimit-*` headers. Retry after `Retry-After` on `429`/`503`. Sandbox/upgrade provisioning is additionally IP-rate-limited (3 sandbox provisions / IP / 24h; configurable global daily cap on free-grant upgrades). --- ## Changelog All API behaviour changes are recorded in [`/docs/changelog.md`](https://api.primateintelligence.ai/docs/changelog.md), sorted newest-first. Subscribe via RSS: [`/docs/changelog.xml`](https://api.primateintelligence.ai/docs/changelog.xml). This file and the changelog ship inside the API deploy artifact itself — `https://api.primateintelligence.ai/docs/agents.md` is the canonical copy, and `https://primateintelligence.ai/docs/agents.md` serves the same bytes (the website proxies the API). Spec, quickstart, and changelog therefore update atomically with the code they describe. **Agent guidance:** check the changelog when `X-Api-Version` changes between requests in the same session — a deploy happened mid-session and a new feature or fix may affect your integration. > Canonical, always-current copy: https://api.primateintelligence.ai/docs/agents.md (also served at https://primateintelligence.ai/docs/agents.md — same bytes, proxied). --- # Uploading video # Uploading video There are three paths to get video into Primate Vision. All of them end with a `video` resource in status `ready`, at which point you can create analyses against it. **Formats:** `video/mp4` (H.264) and `video/quicktime`. **Max size:** 2 GiB. **Retention:** source videos are deleted 30 days after upload (see [security model](/docs/guides/security-model)). ## Choosing a path | Path | Use when | |---|---| | **URL ingest** | The video is already at a public https URL (CDN, S3, YouTube-dl output) | | **Presigned upload** | You have the bytes (server file, browser file input) — bytes go straight to S3, never through our API servers | | **SDK `upload()` helper** | You use an SDK and want the presigned dance done for you | ## URL ingest One call. The API fetches the video asynchronously: ```bash doc-test id=uploading-url skip-live curl -s -X POST https://api.primateintelligence.ai/v1/videos \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com/clip.mp4"}' ``` The video starts `processing` and transitions to `ready` (or `failed` with `error.code` = `url_fetch_failed` / `url_forbidden` / `video_unreadable`). Poll `GET /v1/videos/{id}` or subscribe to the `video.ready` webhook. URL rules (SSRF policy): `https` only, port 443, public hosts only, max 3 redirects, 2 GiB cap. Private/internal addresses are rejected with `url_forbidden`. ## Presigned upload Three steps — create, PUT, complete: ```bash # 1. Create — declares filename/type/size, returns a presigned S3 PUT URL curl -s -X POST https://api.primateintelligence.ai/v1/videos \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"filename": "clip.mp4", "content_type": "video/mp4", "size_bytes": 12345678}' # 2. PUT the bytes directly to S3 (the upload.url + upload.headers from step 1) curl -s -X PUT "$UPLOAD_URL" -H "Content-Type: video/mp4" --data-binary @clip.mp4 # 3. Tell the API the PUT finished curl -s -X POST https://api.primateintelligence.ai/v1/videos/$VIDEO_ID/complete \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` The presigned URL expires (see `upload.expires_at`, typically 1h). If the declared `size_bytes` doesn't match what landed in S3, `complete` fails with `upload_incomplete` — re-create the video. ## SDK helper ```typescript import Primate from '@primate-intelligence/sdk'; import { readFile } from 'fs/promises'; const client = new Primate(); const video = await client.videos.upload(await readFile('clip.mp4'), { filename: 'clip.mp4', content_type: 'video/mp4', }); ``` ```python from primate_intelligence import Primate client = Primate() with open("clip.mp4", "rb") as f: video = client.videos.upload(f.read(), filename="clip.mp4") ``` Both run create → PUT → complete and return the video resource. ## Browser uploads Never put a secret key in a browser. Mint an ephemeral [client token](/docs/guides/security-model#client-tokens) server-side with the `videos:write` scope, then run the presigned flow from the browser with the `pvct_` token. Working example: the [browser SPA sample](https://github.com/Primate-Intelligence/primate-examples). ## Lifecycle ``` awaiting_upload → uploading → processing → ready ↘ failed ``` `DELETE /v1/videos/{id}` removes the video (409 `resource_conflict` while analyses are still running against it). Deletion propagates from S3 + CDN within 72h; signed URLs expire within 1h regardless. ## Your video library (playback + history) Uploaded videos stay usable as a library — three fields/endpoints together give full list-replay-history semantics: - **`Video.media`** — ready videos carry `media: {url, expires_at}`, a signed playback URL for the original source video. Sign-on-read: generated **fresh on every GET with a 1-hour TTL** — re-fetch the video for a new URL after expiry; old videos always stay playable. `media` is `null` for non-ready videos and sandbox fixture videos. - **`GET /v1/videos/{id}/analyses`** — every analysis ever run against a video, newest first, with the same list envelope + filters (`status`, `limit`, `starting_after`) as `GET /v1/analyses`. (Equivalent spelling: `GET /v1/analyses?video_id={id}`.) - **`Analysis.artifacts`** — completed analyses with an annotated result video carry `annotated_video_url` (same sign-on-read semantics). --- # Prompts & queries # Prompts & queries Every analysis is a **question about a video**. You can ask it two ways: free text (`prompt`) or a pre-compiled structured query (`query`). Provide exactly one. ## What Primate can answer The model supports ten distinct prompt types. The table below shows what to expect from each. | Type | Example prompt | `answer` | `detected_count` | `clips` | |------|---------------|----------|-----------------|---------| | **Presence** | "Is there a person?" | yes / no / indeterminate | — | ✅ when yes | | **Absence** | "Are there no dogs?" | yes / no | — | — | | **Action** | "Is someone walking?" | yes / no | — | ✅ when yes | | **Count — bare** | "How many people are in this video?" | — | ✅ integer | — | | **Count — threshold** | "Are there more than 2 people?" | yes / no | ✅ integer | — | | **Compound** | "Is there a person AND a dog?" | yes / no | — | ✅ when yes | | **Attribute** | "Is anyone wearing a red jacket?" | description | — | — | | **Location** | "Where is the dog?" | description | — | — | | **State** | "Is the door open?" | description | — | — | | **Segment** | "Show me where the person is" | — | — | ✅ with masks | **Open-ended prompts** ("Tell me what you see") return a description but have lower accuracy than targeted queries. Prefer a specific question. **Not supported:** - Audio / sound ("Is there loud noise?") - Identity ("Is this John?") - OCR / text in frame ("What does the sign say?") - Subjective judgment ("Does the room look nice?") - Exhaustive counting over long footage ("Count every bird in this 2-hour archive") — count queries are accurate on a single clip or short segment, not long multi-hour recordings --- ## Free-text prompts ```json { "video_id": "video_…", "prompt": "Is there a person near the door?" } ``` The server compiles your prompt into a structured query (the `query` field on the analysis shows exactly what it understood, `parse_mode` shows how: `llm` or `heuristic`). Max 2000 characters. ### Prompt design tips **Do this:** - "Is there a person near the door?" — presence - "Are there more than two people in frame?" — count + threshold, returns `yes`/`no` + `detected_count` - "How many people are walking?" — bare count, returns `detected_count` + natural-language narrative - "Does anyone fall down?" — action - "Is there a person AND a dog?" — compound - "Is anyone wearing a red jacket?" — attribute - "Is the door open or closed?" — state - "Where is the bicycle relative to the car?" — location **Avoid this:** - "Tell me everything that happens in the video" — open-ended; use a targeted question instead - "Count every pedestrian across this 30-minute file" — count works best on clips, not long recordings - "Is this suspicious activity?" — subjective; rephrase as a specific visual condition **One question per analysis beats a paragraph.** If you need to answer "Is there a person?" AND "Are there more than 2 people?", submit two analyses — the answers are independent and the second one gives you `detected_count`. --- ## The answer contract ### `result.answer` `result.answer` is one of `yes`, `no`, `indeterminate`: | Value | Meaning | |-------|---------| | `yes` | The condition is present with meaningful confidence | | `no` | The condition was not detected | | `indeterminate` | The model could not commit either way — treat as "don't ship a decision on this" | Count queries (bare "how many?" form) do not populate `result.answer`. Use `result.detected_count` instead. ### `result.detected_count` Populated for `count`-intent analyses (when the prompt asks "how many?" or uses a threshold): ```json { "result": { "answer": "yes", "confidence": 0.94, "detected_count": 3, "clips": [...] } } ``` For count + threshold queries, `answer` tells you whether the threshold was met; `detected_count` tells you the actual number observed. ### `result.confidence` `result.confidence` ∈ [0, 1]. Meaningful confidence is ≥ 0.5. A result with `confidence: 0` and no positive term detections returns `answer: "indeterminate"`, not `"no"`. ### `result.clips[]` Clip objects carry `start_s`, `end_s`, `confidence`. Present for presence, action, compound, and segment queries when the answer is `yes`. Null or absent for description/count responses. ### `result.query_type` How the question was classified: `object`, `action`, `compound`, `attribute`, or `open_ended`. ### `narrative` (opt-in) Create the analysis with `options: {narrative: true}` and the completed analysis carries a top-level `narrative` object — timestamped event sentences describing what happened in the video: ```json {"narrative": {"status": "ready", "entries": [{"t_s": 3.2, "text": "A person enters from the left."}]}} ``` Generation runs **asynchronously after the analysis completes**: the first completed read may show `status: "generating"` with empty entries — keep polling the GET until `ready` (or `failed`). Included in the analysis price, no surcharge. Without the opt-in, `narrative` is always `null`. (Streams have a live equivalent — see [Streaming § Live narrative](/docs/guides/streaming#live-narrative-opt-in).) --- ## Unassessable components If part of your question can't be assessed, the API **answers what it can and tells you what it couldn't** — it never silently drops half your question: ```json { "prompt": "Is there a person talking loudly?", "query": { "search_terms": ["person"], "unassessable_components": ["talking loudly (audio)"] }, "result": { "answer": "yes", "confidence": 0.91 } } ``` Check `query.unassessable_components` when your prompts mix visual and non-visual language. --- ## Structured queries Skip the compiler and send the compiled form directly (`parse_mode: "client"`). Useful when you generate queries programmatically or need exact control: ```json { "video_id": "video_…", "query": { "search_terms": ["person", "dog"], "conditions": [ { "type": "presence", "target": "person", "negated": false }, { "type": "presence", "target": "dog", "negated": false } ], "connective": "and", "query_type": "compound", "prompt_intent": "presence", "response_framing": "yes_no" } } ``` Key structured query fields: | Field | Values | Description | |-------|--------|-------------| | `query_type` | `object` `action` `compound` `open_ended` | Top-level classification | | `prompt_intent` | `presence` `absence` `count` `action` `segment` | What the model optimizes for | | `response_framing` | `yes_no` `description` `count` | Shape of the result | | `count_terms` | string[] | What to count (for count intent) | | `count_threshold` | integer \| null | Threshold for threshold comparison | | `count_comparison` | `gt` `gte` `lt` `lte` `eq` \| null | How to compare against threshold | | `quantifier` | `{operator, value}` \| null | Alternative to count_threshold | If the body fails the CompiledQuery schema you get `400 query_invalid` with `param` naming the field. If the compiler can't parse a free-text prompt you get `422 parse_failed` — rephrase, or fall back to a structured query. --- ## Prompt errors | Code | Meaning | |------|---------| | `prompt_empty` | Provide `prompt` or `query` | | `prompt_too_long` | > 2000 chars | | `query_invalid` | Structured query fails the schema (`param` names the field) | | `parse_failed` | Compiler couldn't produce a query — retryable after rephrasing | --- # Waiting for results # Waiting for results Analyses run asynchronously. There are three ways to learn when one finishes; pick by context. | Strategy | Use when | |---|---| | **`Prefer: wait`** | Interactive flows, scripts, first success experience — simplest | | **Polling** | You want progress bars / queue position, or waits longer than 120s | | **Webhooks** | Production pipelines, high volume, serverless — no held connections | ## `Prefer: wait` (sync sugar) Add one header to the create and the API holds the connection until the analysis reaches a terminal state (or the wait cap): ```bash doc-test id=waiting-prefer env=VIDEO_ID curl -s -X POST https://api.primateintelligence.ai/v1/analyses \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -H "Prefer: wait=60" \ -d '{"video_id": "'"$VIDEO_ID"'", "prompt": "Is there a person in this video?"}' ``` - Cap: **120 seconds**. If the analysis isn't terminal by then you get the current (running) resource back — fall through to polling. - Test-mode analyses complete in seconds, so `Prefer: wait` virtually always returns the finished result. - Under capacity brownouts `Prefer: wait` may be temporarily disabled (you get an immediate 202-style response with the running resource) — code must handle a non-terminal response either way. The SDK helpers `createAndWait()` (TS) / `create_and_wait()` (Python) use this header, then fall back to polling automatically. ## Polling ```bash curl -s https://api.primateintelligence.ai/v1/analyses/$ANALYSIS_ID \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` While running, the resource carries honest progress signals: ```json { "status": "queued", "queue_position": 3, "progress": null } ``` `queue_position` and `estimated_start_s` are accurate to within ±50% — we publish transparency rather than an analysis-latency SLO. **Backoff:** poll at 1–2s intervals for interactive flows; add jitter for fleets. Terminal states are `completed`, `failed`, `canceled` — anything else keeps polling. ## Webhooks Register once, get an HTTPS POST on every terminal transition: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://yourapp.com/webhooks/primate", "events": ["analysis.completed", "analysis.failed"]}' ``` See the [webhooks guide](/docs/guides/webhooks) for verification, retries, and the #1 integration rule (the webhook is a notification, not the source of truth). ## Cancellation ```bash curl -s -X POST https://api.primateintelligence.ai/v1/analyses/$ANALYSIS_ID/cancel \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` Best-effort once the analysis is `analyzing`; a 409 `analysis_not_cancelable` means it already reached a terminal state. You are not billed for canceled work that never ran. --- # Webhooks # Webhooks Webhook endpoints receive an HTTPS POST whenever a subscribed event fires. Event vocabulary (closed, v1): `video.ready`, `video.failed`, `analysis.completed`, `analysis.failed`, `analysis.canceled`, `stream.ended`. ## Rule #1: the webhook is a notification, not the source of truth Payloads over 256 KiB are truncated (`data.truncated: true`). Delivery is **at-least-once with no ordering guarantee**. The reliable pattern is always: 1. Receive the event → verify the signature → dedupe on the `webhook-id` header 2. `GET` the resource by id for the authoritative state 3. Act on that Never assume `analysis.completed` arrives before (or after) your poll sees `completed`. ## Register an endpoint ```bash curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://yourapp.com/webhooks/primate", "events": ["analysis.completed", "analysis.failed"]}' ``` The response contains the signing secret (`whsec_…`) **exactly once**. Store it. Max 10 endpoints per account. ## Delivery contract - Body: `{"id": "evt_…", "type": "analysis.completed", "created_at": "…", "data": {…full resource DTO…}}` - Timeout **10s**; only 2xx counts as delivered (3xx = failure) - Retry schedule: `1m, 5m, 30m, 2h, 8h, 24h` with jitter; then the event dead-letters (visible in the deliveries API for 30 days) - 72h of 100% failure → the endpoint auto-disables + you get an email. Re-enable with `POST /v1/webhook_endpoints/{id}/enable` ## Verify signatures Deliveries are signed per [Standard Webhooks](https://www.standardwebhooks.com). Three headers: `webhook-id`, `webhook-timestamp`, `webhook-signature` (`v1,`; multiple space-separated during rotation). Signed content is `` `${id}.${timestamp}.${rawBody}` ``; the HMAC key is the base64-decoded part of your `whsec_` secret. Reject skew > 5 minutes; compare constant-time. Both SDKs ship verifiers: ```typescript import { verifyWebhookRequest } from '@primate-intelligence/sdk'; app.post('/webhooks/primate', (req, reply) => { verifyWebhookRequest({ secret: process.env.PRIMATE_WEBHOOK_SECRET!, headers: req.headers, rawBody: req.rawBody, // MUST be the raw bytes, not re-serialized JSON }); // throws on failure const event = JSON.parse(req.rawBody); if (alreadyProcessed(event.id)) return reply.code(204).send(); // dedupe on evt_ id // … fetch the resource, act … reply.code(204).send(); }); ``` ```python from primate_intelligence import verify_webhook_request @app.post("/webhooks/primate") async def receive(request: Request): raw = await request.body() verify_webhook_request( secret=os.environ["PRIMATE_WEBHOOK_SECRET"], headers=dict(request.headers), raw_body=raw, # raw bytes, not re-serialized JSON ) # raises ValueError on failure event = json.loads(raw) ... return Response(status_code=204) ``` > Per-request `webhook` overrides on `POST /v1/analyses` / `POST /v1/streams` have no secret exchange and are delivered **unsigned** — register an endpoint when you need verification. ## Rotate secrets ```bash curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/rotate_secret \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` Both old and new secrets sign deliveries for **24h** (the signature header carries two entries); verification passes when any signature matches *your* secret. Full key-compromise procedure: [security model](/docs/guides/security-model). ## Debug deliveries ```bash # last 100 deliveries with response codes curl -s https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/deliveries \ -H "Authorization: Bearer $PRIMATE_API_KEY" # redeliver a failed/dead one now curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/deliveries/$DELIVERY_ID/redeliver \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` ## Local development Use a tunnel (ngrok, cloudflared) so the API can reach your dev machine, or skip webhooks locally and use `Prefer: wait` — the two strategies are drop-in replacements for the completion signal. --- # Errors & retries # Errors & retries Every non-2xx response uses one envelope: ```json { "error": { "code": "insufficient_credits", "message": "Insufficient credits. Add credits or enable auto-refill.", "status": 402, "param": null, "docs_url": "https://primateintelligence.ai/docs/errors#insufficient_credits", "request_id": "req_01H…" } } ``` - **`code`** — stable, machine-readable, from a closed registry. Codes are append-only; meanings never change. - **`docs_url`** — deep link to the [error registry page](/docs/errors) anchor for that code. - **`request_id`** — include it in support requests; it indexes our logs. - **`error.errors[]`** — on `validation_failed`, every violation is listed (param + message), not just the first. - **`details`** — structured extras (e.g. `details.meter` on `quota_exceeded`). ## The registry is machine-readable ```bash doc-test id=errors-registry curl -s https://api.primateintelligence.ai/v1/errors ``` Each entry carries two retry flags: - **`retryable`** — safe to retry the same request unchanged (with backoff) - **`idem`** — idempotent to retry with the same `Idempotency-Key` The SDKs generate their retry policy from this table — if you build your own client, do the same and your retry logic can never drift from the API. ## What to retry | Situation | Codes | Strategy | |---|---|---| | Rate limited | `rate_limit_exceeded`, `concurrency_limit_exceeded` | Wait `Retry-After` seconds, retry | | Platform transient | `inference_unavailable`, `db_unavailable`, `internal_error`, `capacity_exhausted`, `idempotency_unavailable` | Exponential backoff + jitter, same `Idempotency-Key` | | URL fetch flake | `url_fetch_failed` | Retry the create; check the source URL is reachable | | Prompt didn't compile | `parse_failed` (422) | Rephrase the prompt or send a structured `query` | | Everything else | `validation_failed`, `invalid_api_key`, `insufficient_credits`, … | **Do not retry unchanged** — fix the cause | ## Idempotency (never double-create) Send `Idempotency-Key` (any string, 1–255 chars; UUIDv4 recommended) on `POST /v1/videos`, `/v1/analyses`, `/v1/webhook_endpoints`: - Same key + same body within 24h → the stored response replays with `Idempotency-Replayed: true` - Same key + **different** body → `409 idempotency_key_reused` - Body comparison is canonical (JCS) — key order / whitespace / number formatting never false-positive The SDKs auto-generate keys on create calls, which makes network-level retries safe by default. ## Resource-level errors Two codes never appear as HTTP errors — only inside a failed resource's `error` field: - **`inference_error`** — the model failed on this input; safe to create a new analysis - **`stuck_timeout`** — the platform lost the job; not billed; resubmit Check `analysis.error` / `video.error` when `status` is `failed`. ### Free re-runs after platform incidents Failed analyses carry **`rerun_eligible: boolean`** (present only on `failed` status) — `true` when the failure occurred during a declared platform incident or was flagged by ops. When eligible, **`POST /v1/analyses/{id}/rerun`** creates a **fresh analysis at no charge** — same video, prompt, model, and options; new id; `usage` stays `null` (never billed). One free re-run per failed analysis; calling on an ineligible analysis returns `409 rerun_not_eligible`. Check `rerun_eligible` before writing your own retry — a free re-run beats paying for a resubmit. ## A worked retry loop (Python, no SDK) ```python import time, requests def create_analysis(body, key, attempts=4): idem = str(uuid.uuid4()) for attempt in range(attempts): r = requests.post( "https://api.primateintelligence.ai/v1/analyses", json=body, headers={"Authorization": f"Bearer {key}", "Idempotency-Key": idem}, ) if r.ok: return r.json() err = r.json()["error"] registry = requests.get("https://api.primateintelligence.ai/v1/errors").json() flags = next(e for e in registry["data"] if e["code"] == err["code"]) if not flags["retryable"] or attempt == attempts - 1: raise RuntimeError(f"{err['code']}: {err['message']} ({err['request_id']})") time.sleep(int(r.headers.get("Retry-After", 2 ** attempt))) ``` (Or just use the SDK — this is exactly what it does, with the registry vendored at build time.) ## See also - [Error registry](/docs/errors) — every code, one anchor each (the `docs_url` targets) - [Rate limits & quotas](/docs/guides/rate-limits) - [Billing & credits](/docs/guides/billing) — handling `insufficient_credits` gracefully --- # Rate limits & quotas # Rate limits & quotas ## Rate limits Applied per API key, sliding window: | Class | Limit | Covers | |---|---|---| | `reads` | 600/min | All GETs | | `writes` | 300/min | POST/DELETE except analysis creation | | `analyses.create` | 60/min | `POST /v1/analyses` | Plan multipliers can raise these. Every response — including 429s — carries: ``` X-RateLimit-Limit: 600 X-RateLimit-Remaining: 597 X-RateLimit-Reset: 1751980860 ``` A 429 adds `Retry-After` (integer seconds, ≥ 1). **Honor it** — the SDKs do automatically. ```json { "error": { "code": "rate_limit_exceeded", "status": 429, "message": "Rate limit exceeded. Back off per Retry-After." } } ``` ## Concurrency limits Separate from request rates: your plan caps **simultaneous running analyses**. Exceeding it returns `429 concurrency_limit_exceeded` — wait for an active analysis to finish rather than backing off blindly. ## Quotas (metered limits) Quotas are about *how much* you process, not how fast you call. Exceeding a metered limit returns `429 quota_exceeded` with `details.meter` naming the meter. Check consumption any time: ```bash doc-test id=rate-limits-usage curl -s https://api.primateintelligence.ai/v1/usage \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` ```json { "meters": [ { "meter": "credit_seconds", "unit": "seconds", "balance": 5990 }, { "meter": "seconds_processed.period", "unit": "seconds", "used": 10, "resets_at": "2026-08-01T00:00:00Z" } ] } ``` Running out of credits is `402 insufficient_credits` (not 429) — see [billing & credits](/docs/guides/billing). ## Capacity Under platform-wide pressure new creates may get `503 capacity_exhausted` with `Retry-After` while in-flight work drains. Test-mode traffic is never shed (it doesn't touch the GPU). See [service expectations](/docs/guides/versioning#service-expectations). ## Client checklist 1. Honor `Retry-After` on every 429/503 — never hammer 2. Watch `X-RateLimit-Remaining` and pre-throttle when it gets low 3. Treat `concurrency_limit_exceeded` as "wait for completion", not "back off" 4. Treat `quota_exceeded`/`insufficient_credits` as account states — alert a human, don't retry 5. Single region (us-west-2) in v1 — no cross-region limit ambiguity --- # Test mode # Test mode Every account has two key modes. They hit the same API surface with the same code — only the backend differs. | | `pv_test_` | `pv_live_` | |---|---|---| | Getting one | `POST /v1/sandbox` — instant, anonymous | Signup + card | | Results | Deterministic, canned, fixture corpus | Real GPU inference | | Credits | Can never hold or spend credits | Metered | | Speed | Analyses complete in seconds | Workload-shaped | | Expiry | 7 days (sandbox) | Until revoked | ## Instant sandbox ```bash doc-test id=test-mode-sandbox curl -s -X POST https://api.primateintelligence.ai/v1/sandbox ``` No email, no card, no human. The response includes the key, a pre-seeded fixture video id, the fixture prompt, and the expected answer. IP-limited (default 3 provisions per 24h → `429 sandbox_limit_exceeded`). This is the zero-touch path for agents and CI: build and verify the *entire* integration before any human signs up. Graduating to live is a **billing gate, not an integration gate** — your code doesn't change. ## Deterministic fixtures On test keys, analyses against the fixture corpus return canned results — same input, same output, every time. That makes them assertable in CI: ```bash doc-test id=test-mode-fixture curl -s https://api.primateintelligence.ai/v1/test-fixture ``` ```json { "name": "presence", "test_video_url": "https://app.primateintelligence.ai/empty-state/forklift-demo.mp4", "test_prompt": "Is there a forklift in this video?", "expected_answer": "yes", "expected_confidence_min": 0.8 } ``` ## CI verification pattern ```python # conftest-friendly integration check (runs in seconds, costs nothing) from primate_intelligence import Primate def test_primate_integration(): client = Primate() # PRIMATE_API_KEY = a pv_test_ key in CI secrets videos = client.videos.list() analysis = client.analyses.create_and_wait( video_id=videos["data"][0]["id"], prompt="Is there a person in this video?", timeout_s=60, ) assert analysis["result"]["answer"] == "yes" assert analysis["result"]["confidence"] >= 0.8 ``` Every code sample in these docs runs in CI against this exact mechanism — docs that don't run don't ship. ## Limits of test mode - Fixture corpus only — you can't upload arbitrary videos for real analysis on a test key (uploads work mechanically, but analysis results are canned) - No live streaming GPU sessions - `403 test_mode_only` marks operations unavailable in your key's mode When you need real inference on your own footage: [sign up](https://primateintelligence.ai/signup), add a card, create a `pv_live_` key, change the env var. Done. --- # Versioning & deprecation # Versioning & deprecation ## Compatibility contract The v1 API is **additive-only**. Within v1 we may: - Add new endpoints, optional request fields, response fields, enum values on *new* fields, and error codes (the registry is append-only) We will never (without a major version): - Remove or rename fields/endpoints, change field types or meanings, change error-code semantics, or tighten validation on existing fields Write clients that **ignore unknown fields** — that's the only client-side requirement for staying compatible. The gate is mechanical: every spec change runs `oasdiff breaking` in CI; a breaking diff fails the build unless it's an explicitly approved major-version migration. ## Deprecation process When something must go: 1. It's marked `deprecated` in the OpenAPI spec + the [changelog](/docs/changelog) (typed entry: `breaking | additive | fix | deprecation`) 2. A **migration guide** ships with before/after code in curl/TS/Python, the deadline, and an automated check command 3. Sunset headers appear on responses where applicable 4. Breaking removals only ever land in a new major version ## Legacy `pk_` key sunset API keys created before v1 used the `pk_` prefix. They remain valid during the launch migration so existing integrations do not break. New keys and rotations now create `pv_live_` / `pv_test_` keys only. The `pk_` compatibility window is **12 months from GA announcement**. The exact sunset date starts when Matt approves the production GA cutover and will be published here, in the changelog, and in dashboard key-management copy. Before that date, rotate each legacy key from the dashboard or `POST /v1/api-keys/{id}/rotate`; the replacement key is drop-in for `Authorization: Bearer`. After the sunset, `pk_` keys return `401 invalid_api_key`. No paid usage is charged by the act of rotating a key. ## Model versioning Models are versioned resources (`GET /v1/models`) with lifecycle: `preview → stable → deprecated`, and a `sunset_at` date once deprecated. - Omitting `model` on create uses the current default — fine for exploration - **Production integrations should pin a model id** (e.g. `"model": "Darwin-preview-1.3B"`) so behavior changes only when you choose - `400 model_deprecated` past sunset tells you to re-pin; the changelog carries the migration entry ## SDK versioning SDKs follow semver. An SDK major bump happens only with an API major. Minor/patch releases are additive (new endpoints, fixes) and safe to auto-update within `^`. ## Service expectations Published targets (not a contractual SLA in v1): - **Availability:** 99.9%/mo control plane; ≥ 99% completion success for admitted analyses - **Latency:** p95 reads < 300ms; creates < 1.5s (excluding upload bytes); webhook dispatch p95 < 60s after terminal transition; test-mode analyses < 10s - **Analysis latency is workload-shaped** — we publish honest transparency instead: `queue_position` + `estimated_start_s` accurate to ±50% - Single region: us-west-2 (US). All processing and storage. EU is roadmap, not promise. ## Changelog & RSS Every change lands in the [changelog](/docs/changelog) with a date and type tag. Subscribe: [RSS feed](/docs/changelog.xml). Agents: the changelog is also in `llms-full.txt`. --- # Security model # Security model ## API keys - Two prefixes: `pv_live_` (real processing, metered) and `pv_test_` ([test mode](/docs/guides/test-mode)). 256-bit CSPRNG entropy. - Shown **exactly once** at creation/rotation. We store only a SHA-256 digest. - Send via `Authorization: Bearer …` header only. Keys never belong in URLs, logs, or browser code. - Env-var contract: `PRIMATE_API_KEY` — both SDKs and the MCP server read it. The MCP server never accepts keys as tool arguments (keys must not land in agent transcripts). - Revocation propagates in ≤ 5 seconds. ## Client tokens (browser auth) Secret keys must never reach a browser. For client-side flows, your server mints an **ephemeral client token**: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/client_tokens \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"scopes": ["videos:write", "analyses:read"], "ttl_s": 300}' ``` - Prefix `pvct_`, TTL 60–900s (default 900), **never refreshable** — mint a new one when it expires (`401 token_expired`) - Scopes (v1): `videos:write`, `analyses:read`, `analyses:write`, `streams:create`, `streams:signal` — a token's scopes must be a subset of the minting key's - Optionally bind a token to one `video_id` or `stream_id` — the token then works only for that resource - The streaming signaling WebSocket accepts **only** client tokens (or first-party session JWTs) — never secret keys Pattern: a ≤ 20-line "token mint" route on your server, browser does the rest. Working example: the token-mint server sample in the [examples](https://github.com/Primate-Intelligence/primate-examples). ## Data handling **Prompts and result metadata are retained to operate and improve the service; customer video content is never used to train models.** The full contract: | Data | Retention | |---|---| | Source videos | Deleted 30 days after upload; `DELETE /v1/videos/{id}` propagates from S3 + CDN within 72h (signed URLs expire ≤ 1h) | | Result artifacts (annotated videos) | 30 days, same deletion path | | Result JSON + analysis metadata | 13 months (billing/audit), then aggregated | | Prompts | Retained per the sentence above; org-level opt-out available on request | | Account deletion | Cascades videos/analyses/keys within 30 days; webhook endpoints immediately | - Encryption: TLS ≥ 1.2 in transit; AES-256 at rest - No customer video bytes in logs or analytics — enforced by a redaction test in CI - Region: all processing/storage in us-west-2 (US) - Audit log: key lifecycle + webhook changes retained 13 months ## URL ingest (SSRF policy) `POST /v1/videos {url}` fetches are locked down: https only, port 443, public addresses only (RFC-1918/link-local/metadata ranges rejected), resolved IPs pinned (no TOCTOU), max 3 redirects each re-validated, 2 GiB streamed cap, content validated by magic bytes. Policy violations return `url_forbidden`; network failures `url_fetch_failed`. ## If a key is compromised 1. **Rotate immediately** — create a new key, revoke the old one in the [dashboard](https://primateintelligence.ai/dashboard/api-keys) (propagation ≤ 5s) 2. **Audit** — `GET /v1/analyses?created_after=…` for activity you don't recognize 3. **Contact support** with request ids for usage-forgiveness on abusive spend We monitor per-key spend anomalies (> 5× 7-day baseline), page on them, and may auto-suspend a key pending confirmation — with an email + dashboard banner, never silently. ## Webhook signing Deliveries are signed (Standard Webhooks, HMAC-SHA256, 5-minute replay window, 24h dual-secret rotation). Details: [webhooks guide](/docs/guides/webhooks). --- # Billing & credits # Billing & credits Primate Vision bills by **frames processed**, prepaid as credits. Each processed frame costs `price_per_frame_cents`, and the total settles to your credit ledger as equivalent credit-seconds. Current pricing is always at `GET /v1/credit-pricing` (public, no auth — see `price_per_frame_cents`) and on the [pricing page](/pricing). ## Preview pricing policy The current rate (`price_per_frame_cents = 0.00015¢/frame`) is **preview pricing**: early-access pricing, set before we have full demand data, for a model that is itself in preview. You're charged because the product delivers value today — metered per-frame cost, streaming ingestion, low latency. The rate may change — **up or down** — as the model matures, customer demand develops, and processing costs change. - **What is stable:** the API contract — request/response shapes, the metering unit (frames processed), and the billing mechanics on this page. - **What may change, and why:** the per-frame rate — driven by model capability (better checkpoints may cost more to run), demand data, and per-frame GPU economics. Any change applies to *new* processing only — credits you already bought keep their frame value at purchase-equivalent terms, and we announce rate changes in the [changelog](/docs/changelog) before they take effect. - **How to stay current:** read the live rate from `GET /v1/credit-pricing` (public, no auth) — it also returns `pricing_tier: "preview"` so integrations can detect preview pricing programmatically. Never hardcode the rate. ## How credits flow - **Signup grant** — new accounts get a free frame budget to evaluate with (currently 108,000 frames — up to 30 hours of monitoring at 1 fps, 1 hour at 30 fps; expiring) - **Card grant** — adding a card grants another tranche - **Purchases** — buy credit blocks in the [dashboard](https://primateintelligence.ai/dashboard/billing); custom amounts supported - **Auto-refill** — opt in to top up automatically when the balance crosses the threshold (currently 600s) Test-mode keys never hold or spend credits — build and CI on `pv_test_` for free, forever. ## Check your balance ```bash doc-test id=billing-usage curl -s https://api.primateintelligence.ai/v1/usage \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` ```json { "meters": [ { "meter": "credit_seconds", "unit": "seconds", "balance": 5990 }, { "meter": "seconds_processed.period", "unit": "seconds", "used": 10, "resets_at": "2026-08-01T00:00:00Z" } ] } ``` ## What you're charged for - **Analyses** — the frames actually processed, debited when the analysis completes. Failed platform-side work (`inference_error`, `stuck_timeout`) is **not billed**; resubmit freely. - **Streams** — live sessions reserve credits up front and reconcile to the processed-frame count at end (`GET /v1/streams/{id}` shows the final usage). Mid-stream you get `warning` messages as the reservation runs low, then the session ends with `end_reason: "insufficient_credits"`. ## Settlement method (`metering_rule`) Every stream and analysis is pinned to a settlement rule **at creation time**, immutable for its lifetime, and reported back on the terminal object: - `per_frame` — the current default. Cost = frames processed × `price_per_frame_cents`, settled to the ledger as equivalent credit-seconds: `round(frames × price_per_frame_cents ÷ price_per_second_cents)`. The slower your capture rate, the less you pay for the same wall-clock coverage. - `per_second` — the legacy rule: source clock seconds, fps-independent. Reported for resources created before the frame-metering cutover; in-flight sessions always finish under the rule they started with. Read it from the terminal response — both the top-level field and the `usage` block carry it: ```json { "metering_rule": "per_frame", "usage": { "billed_seconds": 16, "credit_balance_after": 5978, "metering_rule": "per_frame", "frames": 108000 } } ``` Worked example at current pricing (`price_per_frame_cents: 0.00015`, `price_per_second_cents: 1`): 108,000 frames × 0.00015¢ = 16.2¢ → `round(16.2 ÷ 1)` = **16 credit-seconds** settled. `usage.frames` is the processed-frame count and is reported for **both** rules (frame counting predates the billing cutover) — for `per_second` rows it's informational, not the billing basis. `usage` is terminal-only: null until the analysis completes or the stream ends. ## Handling `insufficient_credits` in code The one billing error every integration must handle (`402`, **not retryable**): ```typescript doc-test id=billing-handling sdk=ts offline import Primate, { PrimateError } from '@primate-intelligence/sdk'; const client = new Primate(); try { await client.analyses.create({ video_id: videoId, prompt: 'Is the loading dock clear?' }); } catch (err) { if (err instanceof PrimateError && err.code === 'insufficient_credits') { const { meters } = await client.usage.retrieve(); const balance = meters.find((m) => m.meter === 'credit_seconds')?.balance ?? 0; notifyOwner( `Primate Vision balance is ${balance}s — top up or enable auto-refill: ` + `https://primateintelligence.ai/dashboard/billing`, ); // Do NOT retry unchanged — queue the job for after refill instead. } else { throw err; } } ``` The robust pattern: 1. Catch `insufficient_credits` → **stop submitting** (it will keep failing) 2. Read `GET /v1/usage` for the real balance 3. Surface a human-actionable message with the billing URL 4. If the account has auto-refill, retry after a short delay **once** — refill is triggered by the threshold crossing 5. Alert *before* you hit zero: poll `credit_seconds` and warn below your own threshold ## Free-credit expiry Granted (free) credits expire; purchased credits don't. Expiry dates show in the dashboard. Expired grants simply vanish from `balance` — no negative surprises. ## Invoices & history Purchase history and receipts live in the [dashboard billing page](https://primateintelligence.ai/dashboard/billing). Usage aggregates by day/analysis are on the [usage page](https://primateintelligence.ai/dashboard/usage). --- # Streaming # Streaming Streams analyze live video in real time: point a camera (or any MediaStream) at the API and get per-frame detection results back over a WebRTC data channel, with the same prompt semantics as file analyses. ## Lifecycle ``` POST /v1/streams → queued|ready → (WS join → offer/answer/ICE) → live → ended ``` 1. **Create** the stream server-side with your secret key: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/streams \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"prompt": "Is there a person in the frame?", "options": {"narrative": true}, "recording": true}' ``` The response carries everything the client needs: `signaling.url` (a WebSocket), `ice_servers` (STUN/TURN with credentials), `limits` (`max_session_s`, `warn_at_remaining_s`), and `queue_position`/`estimated_start_s` when capacity is busy. Two optional opt-ins at create time: - **`options: {narrative: true}`** — [live narrative](#live-narrative-opt-in): result detections carry `narrative_update` sentences, and the WS delivers `status` ordering events. No surcharge. - **`recording: true`** — [session recording](#session-recordings-opt-in): retrieve the annotated session video after the stream ends. 2. **Mint a client token** for the browser/device (the signaling WS **never** accepts secret keys): ```bash curl -s -X POST https://api.primateintelligence.ai/v1/client_tokens \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"scopes": ["streams:signal"], "stream_id": "'"$STREAM_ID"'", "ttl_s": 300}' ``` 3. **Connect** from the client. With the TS SDK this is one call: ```typescript import { connectStream } from '@primate-intelligence/sdk/browser'; const session = await connectStream({ stream, // the Stream resource (relay it from your server) clientToken, // pvct_… minted in step 2 mediaStream: await navigator.mediaDevices.getUserMedia({ video: true }), onResult: (r) => console.log(r.frame_num, r.detections), onNarrative: (n) => log(`${n.t_s}s: ${n.text}`), // narrative streams only onStatus: (s) => log(`engine: ${s.status}`), // narrative streams only onWarning: (w) => (w.code ? diagnose(w) : showCountdown(w.remaining_s)), onEnd: (reason) => cleanup(reason), }); // change the question mid-stream (server confirms with prompt_updated): session.updatePrompt('Is there a forklift moving?'); // done: session.end(); ``` Under the hood: WS `join` → `ready` (or `queued {position}`) → WebRTC `offer`/`answer` + ICE trickle → media flows → `result` messages per analyzed frame. ## Signaling protocol (build your own client) Messages you send: `join`, `offer {sdp}`, `ice {candidate}`, `update_prompt {prompt}`, `end`, `ping`. Messages you receive: `queued {position, estimated_start_s}`, `ready`, `answer {sdp}`, `ice {candidate}`, `live`, `result {frame_num, detections}`, `prompt_updated`, `status {…}` (narrative streams only), `metering {…}`, `warning {…}`, `end {reason}`, `error {code, message}`, `pong`. There are two kinds of `warning`: the credit-exhaustion countdown (`warning {remaining_s}`, no `code` field) and diagnostic warnings that carry a `code` field — see [No-media warning](#no-media-warning-the-server-tells-you-when-your-media-is-the-problem) below. Distinguish them by the presence of `code`. A Python/aiortc backend example (robotics-style, no browser — including a "stream a file" recipe and a relay-only datacenter/CI profile) ships in the public [examples repo](https://github.com/Primate-Intelligence/primate-examples), alongside a self-contained browser webcam example. **Trickle ICE is bidirectional**: after the `answer`, the server may keep sending `ice {candidate}` messages as late srflx/relay candidates finish gathering — keep consuming and adding them. Clients on UDP-blocked networks (datacenters, CI, strict NATs) complete via the TURN servers in `ice_servers`; TURN-over-TCP:443 is supported. ### Result contract (identical to file analyses) Each `result.detections[]` row uses the same enums and casing as the file API: `answer` is lowercase `yes|no|indeterminate`, `confidence` is 0..1, and `prompt` is echoed exactly as you submitted it. Latency telemetry (per-stage breakdowns) lives under a `timing` object. ### Update the prompt mid-stream Send `{"type": "update_prompt", "prompt": "…"}` (≤2000 chars) at any point while live — no reconnect, no new stream. The server recompiles the prompt, applies it to the live engine, and confirms with `{"type": "prompt_updated"}`; subsequent result frames echo the new prompt. An invalid prompt returns `{"type": "error", "code": "validation_failed" | "parse_failed"}` and the **old prompt stays active**. On narrative streams, expect a `recalculating` status event while the engine rebuilds context. ### Live narrative (opt-in) Create the stream with `options: {narrative: true}` and two things change (nothing changes without the opt-in): 1. Result detections carry a **`narrative_update`** object — `{t_s, text}`, a new narrative sentence — whenever the engine detects an appearance/disappearance/action. Event-driven, NOT per frame: absent (not `null`) on frames without one. 2. The WS delivers additive **`status`** events for ordering/annotating narrative entries: ```json {"type": "status", "status": "prompt_context" | "combined_prompt" | "recalculating", "message": "…"} ``` The vocabulary is a **closed set** — exactly these three values; internal engine statuses never leak. Typical use: mark when the engine is recalculating context right after an `update_prompt`. Included in the per-second price — no surcharge. ### Session recordings (opt-in) Create the stream with **`recording: true`** (top-level boolean) to retrieve the server-side session recording afterwards: 1. The stream resource carries `recording: {status}` — `recording` while live, `available` once ended with a stored recording, `failed`, or `none`. Streams without the opt-in have `recording: null`. 2. Once `available`, call `GET /v1/streams/{id}/recording` → `{url, expires_at, content_type: "video/h264", container: "h264-annex-b"}`. The URL is **signed fresh on every GET, 1-hour TTL** — re-fetch for a new one; old recordings always stay retrievable. 3. The recording is a raw H.264 Annex-B elementary stream (the exact annotated frames the client saw). Lossless remux to MP4: `ffmpeg -i recording.h264 -c copy recording.mp4`. ### Result sampling — results are sampled, not per-frame Results are **sampled**, not emitted for every source frame — inference cadence is adaptive (roughly every 8th source frame under load). Two different axes to keep straight: - `frame_num` on each live `result` is the **source-frame index** the result was computed on. Gaps between consecutive `frame_num` values are normal and expected — don't treat them as dropped results. - `results_summary.result_frames` on the ended stream resource counts the number of **result events emitted** over the session — NOT a source-frame count. Expect roughly **8–15 results per second** depending on load; never assume a fixed cadence. > **Alias note:** `results_summary.frames` is a supported alias of `results_summary.result_frames` (identical value), superseded by `result_frames`. Prefer `result_frames` in new code; existing reads of `frames` keep working. ### Metering ticks While live, a `metering` message arrives every 5 seconds: ```json {"type": "metering", "elapsed_s": 45, "billed_s": 45, "session_remaining_s": 555, "balance_s": 555} ``` - `session_remaining_s` — seconds remaining in **this session's** credit reservation (the session cap). This is NOT your account credit balance — that lives at `GET /v1/billing/credits` (`balance_seconds`). - `balance_s` — supported alias of `session_remaining_s` (identical value), superseded by `session_remaining_s`. Prefer `session_remaining_s` in new code; existing reads of `balance_s` keep working. - `elapsed_s` / `billed_s` — live-clock seconds elapsed = billed (identical by construction; join, queueing, and negotiation are free). ### No-media warning (the server tells you when your media is the problem) If the WebRTC transport connects but **no decodable video frame arrives within 5 seconds**, the server pushes a diagnostic `warning` on the signaling WS (re-warned once at 15s, then quiet; canceled by the first decoded frame): ```json {"type": "warning", "code": "no_media_frames", "transport_connected": true, "elapsed_s": 5, "packets_received": 312, "frames_decoded": 0, "hint": "…"} ``` Read `packets_received` to self-diagnose: - `packets_received: 0` — your client is **not sending media** (track not attached, muted, or the capture source is dead). - `packets_received > 0` with `frames_decoded: 0` — media is arriving but is **not decodable** (wrong codec — must be VP8 or H.264 — or the source produces no real frames, e.g. a canvas capture without an active draw loop). If the session then ends without ever going live, the terminal resource records `end_reason: "media_timeout"` with the same media counters in `failure_diagnostic`, and bills 0. ### Honest end reasons `end {reason}` and the terminal resource's `end_reason` distinguish real failures — a session that never went live is never `completed`: | Reason | Meaning | Billed | |---|---|---| | `completed` | Normal end after live | live seconds | | `canceled` | Ended before going live (client action) | 0 | | `ice_failed` | ICE never connected — resource carries `failure_diagnostic` (server candidate summary) | 0 | | `media_timeout` | Transport connected but no media flowed — `failure_diagnostic` carries the media counters | 0 | | `insufficient_credits` | Balance exhausted mid-stream | live seconds | | `timeout` | `limits.max_session_s` reached | live seconds | | `error` | Server-side failure | live seconds (0 if never live) | ### failure_diagnostic — debugging a failed session from the API alone When a session ends without ever going live on a transport failure, the terminal Stream resource carries a nullable `failure_diagnostic` object (and the session bills **0**): - On `ice_failed` it holds the server-side ICE candidate summary: ```json {"end_reason": "ice_failed", "failure_diagnostic": { "local_candidates": ["host 203.0.113.7", "srflx 203.0.113.7", "relay 198.51.100.20"], "hint": "Server advertised public host + srflx/relay candidates; check the client's path to the TURN servers in ice_servers." }} ``` If `local_candidates` shows only private addresses (`172.x`, `10.x`), the fault is on our side — contact support. If it shows srflx/relay candidates, check your client's network path to the TURN servers in `ice_servers` (TURN-over-TCP:443 and `turns:` TLS relays are available for locked-down egress networks). - On `media_timeout` it holds the media counters from the no-media warning (`packets_received`, `frames_decoded`) so you can tell "never sent media" apart from "sent undecodable media" after the fact. `failure_diagnostic` is `null` on successful sessions and on ends unrelated to transport. ## Credits & limits - One active (queued/ready/live) stream per account by default — a second create returns `409 stream_already_active` - Sessions reserve credits up front; metering ticks every 5s (see [Metering ticks](#metering-ticks) for the field contract); you get `warning {remaining_s}` before funds run out, then `end {reason: "insufficient_credits"}` - `limits.max_session_s` is the hard per-session cap from your plan - Your account-wide balance is at `GET /v1/billing/credits` — metering ticks only report the session reservation - After `ended`, `GET /v1/streams/{id}` shows `duration_s`, `usage` (reconciled to the second), and `results_summary` (`result_frames` = count of result events — see [Result sampling](#result-sampling--results-are-sampled-not-per-frame)) ## End a stream From the client: send `end` (or just `session.end()`). Server-side / cleanup: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/streams/$STREAM_ID/end \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` Idempotent — safe to call on an already-ended stream. `stream.ended` fires as a [webhook](/docs/guides/webhooks) if subscribed. --- # Tutorial: build from scratch # Tutorial: build from scratch We'll build `askvideo` — a Node CLI that takes a video URL and a question, and prints the answer. Zero to working in about 15 minutes, entirely on a free test key. ## Step 0: get a key (30 seconds) ```bash doc-test id=tutorial-sandbox curl -s -X POST https://api.primateintelligence.ai/v1/sandbox ``` ```bash export PRIMATE_API_KEY="pv_test_…" # from the response ``` ## Step 1: project setup ```bash mkdir askvideo && cd askvideo npm init -y ``` ## Step 2: the whole program `askvideo.mjs`: ```javascript const [url, ...promptParts] = process.argv.slice(2); const prompt = promptParts.join(' '); if (!url || !prompt) { console.error('usage: node askvideo.mjs '); process.exit(2); } const apiKey = process.env.PRIMATE_API_KEY; const baseUrl = process.env.PRIMATE_BASE_URL ?? 'https://api.primateintelligence.ai'; if (!apiKey) { console.error('PRIMATE_API_KEY is required'); process.exit(2); } async function api(path, options = {}) { const res = await fetch(`${baseUrl}${path}`, { ...options, headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', ...(options.headers ?? {}), }, }); const body = await res.json(); if (!res.ok) { const err = body.error ?? { code: 'request_failed', message: res.statusText }; console.error(`${err.code}: ${err.message}`); if (err.docs_url) console.error(`docs: ${err.docs_url}`); process.exit(1); } return body; } async function createAndWait(videoId, question) { let analysis = await api('/v1/analyses', { method: 'POST', headers: { Prefer: 'wait=60' }, body: JSON.stringify({ video_id: videoId, prompt: question }), }); for (let i = 0; analysis.status !== 'completed' && i < 30; i += 1) { if (analysis.status === 'failed' || analysis.status === 'canceled') { throw new Error(`analysis_${analysis.status}: ${JSON.stringify(analysis.error)}`); } await new Promise(resolve => setTimeout(resolve, 1000)); analysis = await api(`/v1/analyses/${analysis.id}`); } return analysis; } const video = await api('/v1/videos', { method: 'POST', body: JSON.stringify({ url }), }); console.error(`video ${video.id} (${video.status})`); const analysis = await createAndWait(video.id, prompt); console.log(JSON.stringify(analysis.result, null, 2)); ``` ## Step 3: run it Test keys analyze the fixture corpus deterministically, so use the official fixture: ```bash FIXTURE=$(curl -s https://api.primateintelligence.ai/v1/test-fixture) node askvideo.mjs \ "$(echo $FIXTURE | python3 -c 'import json,sys; print(json.load(sys.stdin)["test_video_url"])')" \ "Is there a person in this video?" ``` ```json { "answer": "yes", "confidence": 0.93, "clips": [{ "start_s": 1, "end_s": 4, "confidence": 0.93 }], "query_type": "object", "video_duration_s": 10 } ``` That's a working integration. Everything below is production hardening. ## Step 4: make it CI-verifiable `test.mjs`: ```javascript const baseUrl = process.env.PRIMATE_BASE_URL ?? 'https://api.primateintelligence.ai'; const headers = { Authorization: `Bearer ${process.env.PRIMATE_API_KEY}`, 'Content-Type': 'application/json', }; const fixture = await fetch(`${baseUrl}/v1/test-fixture`).then(r => r.json()); const video = await fetch(`${baseUrl}/v1/videos`, { method: 'POST', headers, body: JSON.stringify({ url: fixture.test_video_url }), }).then(r => r.json()); const analysis = await fetch(`${baseUrl}/v1/analyses`, { method: 'POST', headers: { ...headers, Prefer: 'wait=60' }, body: JSON.stringify({ video_id: video.id, prompt: fixture.test_prompt }), }).then(r => r.json()); const ok = analysis.result.answer.toLowerCase() === fixture.expected_answer.toLowerCase() && analysis.result.confidence >= fixture.expected_confidence_min; console.log(ok ? 'INTEGRATION OK' : 'INTEGRATION BROKEN'); process.exit(ok ? 0 : 1); ``` Run that in CI with a `pv_test_` key in your secrets — it completes in seconds and costs nothing. ## Step 5: production hardening checklist - **Error handling** — retry only codes marked retryable in the [error registry](/docs/guides/errors-retries); never retry [`insufficient_credits`](/docs/guides/billing) - **Stop polling** — swap `createAndWait` for [webhooks](/docs/guides/webhooks) at volume - **Pin a model** — pass `model: 'Darwin-preview-1.3B'` so behavior changes only when you choose ([versioning](/docs/guides/versioning)) - **Your own videos** — swap URL ingest for [presigned upload](/docs/guides/uploading) when the bytes live with you - **Go live** — [sign up](https://primateintelligence.ai/signup), create a `pv_live_` key, change the env var. No code changes. ## Where to next - [Sample matrix](https://github.com/Primate-Intelligence/primate-examples) — Python, webhook receivers, browser SPA, streaming, token-mint server - [Agent quickstart](/docs/agents) — the condensed machine-readable version of everything you just did - [API reference](/docs/reference) --- # Supported actions # Supported actions This page is the honest capability list for **Darwin-preview-1.3B** (status: preview). It documents what the model can reliably detect, what it detects with reduced accuracy, and what it cannot detect at all — so you can design prompts that work instead of discovering the limits by trial and error. **Preview means early access**: detection quality and the set of reliably detected actions are still evolving, and results may improve or change between checkpoint updates. The API contract (request/response shapes, billing) is stable. If an analysis returns `answer: "indeterminate"` with `indeterminate_reason: "unsupported_query_form"`, or an action prompt produces a nonsensical match, this page explains why — and what to ask instead. ## Summary: what works, by prompt type | Prompt type | Example | Reliability | |---|---|---| | **Presence** | "Is there a person?" | Stable — most reliable query form | | **Absence** | "Are there no dogs?" | Stable | | **Count** (bare + threshold) | "How many people?" / "More than 2 people?" | Stable on clips; not long multi-hour footage | | **Compound** | "Is there a person AND a dog?" | Stable | | **Attribute** | "Is anyone wearing a red jacket?" | Stable | | **Location** | "Where is the dog?" | Stable | | **State** | "Is the door open?" | Stable | | **Segment** | "Show me where the person is" | Stable (returns masks) | | **Action** | "Is someone walking?" | **Beta** — see the vocabulary below | | **Open-ended** | "Tell me what you see" | Rejected at zero cost (`unsupported_query_form`) | Presence, counting, attribute, location, state, and segmentation queries recognize the visual world broadly — common objects, people, animals, vehicles, scenes. They are not limited to a fixed list. **Action queries are different**: they run against a fixed action vocabulary, and that is where the honest limits live. ## How action detection works — and why the vocabulary matters When you ask an action question ("Is someone running?", "Does anyone fall down?"), the model matches what it sees against a **fixed vocabulary of 908 action classes** (Kinetics-derived, plus industrial/vehicle/warehouse domain classes). Two consequences: 1. **In-vocabulary actions** can be detected, at beta accuracy. Short clips, fast motion, and unusual camera angles reduce reliability. 2. **Out-of-vocabulary actions cannot be detected.** The model has no embedding for the action, so it falls back to the nearest in-vocabulary neighbour — which can produce confidently wrong or nonsensical matches. If the action you care about is not in the list below, do not ship a decision on the result. When the system can tell at submit time that a requested action is outside the vocabulary, the API flags it (see [Unsupported action signal](#unsupported-action-signal) below). When it cannot tell, you may still get a low-quality nearest-neighbour result — treat unexpected action answers with suspicion and check this list. ## Known gaps (confirmed, tracked) These are actions users commonly ask about that the current vocabulary does **not** cover: - **Hand gestures** — thumbs up, thumbs down, OK sign, peace sign. (Exception: `waving hand`, `finger snapping`, `clapping`-family classes exist.) Gesture prompts currently produce unreliable nearest-neighbour matches. - **Some vehicle actions** — the vocabulary now covers driving, parking, reversing, turning, towing, refueling/charging, entering/exiting, and loading/unloading for cars, trucks, forklifts, and other work vehicles (see the vocabulary below). Still missing: lane changes, braking, overtaking, collisions/crashes, honking. Treat prompts for those specific maneuvers as **not detected**. - **Everyday ambient actions phrased generically** — "walking" and "running" as bare actions are matched via the model's broader action understanding but the vocabulary's walking/running entries are context-specific (`walking the dog`, `running on treadmill`, `jogging`). Generic phrasings usually work but are beta-accuracy; be specific where you can. Vocabulary expansion (a ~5,000-action taxonomy with synonym grouping) is on the roadmap; this page tracks the current shipped reality. ## What is never supported (any prompt type) - **Audio / sound** — "Is there loud noise?", "What did they say?" - **Identity** — "Is this John?" (who, not what) - **OCR / reading text in frame** — "What does the sign say?" - **Subjective judgment** — "Does this look dangerous?", "Is the room clean?" (subjective terms are surfaced in `query.unassessable_components`) - **Exhaustive counting over long footage** — counts are accurate on a clip or short segment, not a multi-hour archive ## Unsupported action signal When prompt compilation determines the request cannot be scored, the API responds honestly instead of burning credits: - **Open-ended prompts** are rejected at submit time, at zero cost: the analysis completes immediately with `answer: "indeterminate"`, `indeterminate_reason: "unsupported_query_form"`, and `confidence: 0`. - Analyses whose result cannot be trusted carry an `indeterminate_reason` (`low_confidence`, `nothing_detected`, `unsupported_query_form`, `duration_mismatch`) — [full semantics in the prompts guide](/docs/guides/prompts). - Responses in the unsupported path link back to this page so agents and users can see the capability list without guessing. ## The full action vocabulary (908 classes) The complete, current list. If an action question maps to one of these, it is detectable (beta accuracy). If it does not, rephrase — often a **presence** question ("Is there a person on a ladder?") is more reliable than an action question ("Is someone climbing?"). **A** — `abseiling` · `abusing someone` · `acting in play` · `adjusting glasses` · `air drumming` · `answering questions` · `applauding` · `applying cream` · `arc welding` · `archery` · `arm wrestling` · `arranging flowers` · `assaulting someone` · `assembling bicycle` · `assembling computer` · `assembling parts` · `assembly line work` · `attaching safety harness` · `attending conference` · `auctioning` **B** — `baby waking up` · `baking cookies` · `balloon blowing` · `bandaging` · `barbequing` · `bartending` · `base jumping` · `bathing dog` · `beatboxing` · `bee keeping` · `being arrested` · `being excited` · `being in zero gravity` · `belly dancing` · `bench pressing` · `bending back` · `bending metal` · `biking through mud` · `biking through snow` · `blasting sand` · `blending fruit` · `blocking emergency exit` · `blowdrying hair` · `blowing bubble gum` · `blowing glass` · `blowing leaves` · `blowing nose` · `blowing out candles` · `bobsledding` · `bolting parts` · `bookbinding` · `bouncing ball (not juggling)` · `bouncing on bouncy castle` · `bouncing on trampoline` · `bowling` · `braiding hair` · `breading or breadcrumbing` · `breakdancing` · `breaking boards` · `breaking glass` · `breathing fire` · `brush painting` · `brushing floor` · `brushing hair` · `brushing teeth` · `building cabinet` · `building lego` · `building sandcastle` · `building shed` · `bungee jumping` · `busking` **C** — `calibrating equipment` · `canoeing or kayaking` · `capoeira` · `carrying baby` · `carrying box` · `carrying heavy load` · `carrying weight` · `cartwheeling` · `carving ice` · `carving marble` · `carving pumpkin` · `carving wood with a knife` · `casting fishing line` · `catching fish` · `catching or throwing baseball` · `catching or throwing frisbee` · `catching or throwing softball` · `celebrating` · `changing gear in car` · `changing oil` · `changing tire` · `changing wheel` · `changing wheel (not on bike)` · `charging electric vehicle` · `checking tires` · `checking watch` · `cheerleading` · `chewing gum` · `chiseling stone` · `chiseling wood` · `chopping meat` · `chopping wood` · `clapping` · `clay pottery making` · `clean and jerk` · `cleaning floor` · `cleaning gutters` · `cleaning pool` · `cleaning shoes` · `cleaning toilet` · `cleaning up spill` · `cleaning windows` · `climbing a rope` · `climbing ladder` · `climbing scaffolding` · `climbing tree` · `closing door` · `closing something` · `coloring in` · `combing hair` · `committing a burglary` · `committing a robbery` · `committing arson` · `contact juggling` · `cooking chicken` · `cooking egg` · `cooking on campfire` · `cooking sausages` · `cooking sausages (not on barbeque)` · `cooking scallops` · `counting money` · `country line dancing` · `covering something` · `cracking back` · `cracking knuckles` · `cracking neck` · `crawling baby` · `crossing eyes` · `crossing river` · `crossing safety barrier` · `crying` · `curling (sport)` · `curling eyelashes` · `curling hair` · `cutting apple` · `cutting cake` · `cutting lumber` · `cutting nails` · `cutting orange` · `cutting pineapple` · `cutting watermelon` **D** — `dancing ballet` · `dancing charleston` · `dancing gangnam style` · `dancing macarena` · `deadlifting` · `dealing cards` · `decorating the christmas tree` · `delivering mail` · `descending ladder` · `digging` · `digging trench` · `dining` · `directing traffic` · `disc golfing` · `dismounting forklift` · `diving cliff` · `docking boat` · `dodgeball` · `doing aerobics` · `doing jigsaw puzzle` · `doing laundry` · `doing nails` · `doing sudoku` · `drawing` · `dribbling basketball` · `drilling concrete` · `drinking` · `drinking beer` · `drinking shots` · `driving bus` · `driving car` · `driving delivery truck` · `driving forklift` · `driving golf cart` · `driving order picker` · `driving pallet jack` · `driving reach truck` · `driving tractor` · `driving truck` · `driving tugger` · `driving utility vehicle` · `driving van` · `drop kicking` · `drumming fingers` · `dump truck unloading` · `dunking basketball` · `dyeing eyebrows` · `dyeing hair` · `dying hair` **E** — `eating burger` · `eating cake` · `eating carrots` · `eating chips` · `eating doughnuts` · `eating hotdog` · `eating ice cream` · `eating nachos` · `eating spaghetti` · `eating watermelon` · `egg hunting` · `entering church` · `entering restricted area` · `entering vehicle` · `excavator digging` · `exercising arm` · `exercising with an exercise ball` · `exiting vehicle` · `extinguishing fire` **F** — `faceplanting` · `falling from height` · `falling off bike` · `falling off chair` · `falling off ladder` · `feeding birds` · `feeding fish` · `feeding goats` · `felling tree` · `fencing (sport)` · `fighting` · `filling cake` · `filling eyebrows` · `finger snapping` · `fixing bicycle` · `fixing hair` · `flagging traffic` · `flipping bottle` · `flipping pancake` · `fly tying` · `flying kite` · `folding clothes` · `folding napkins` · `folding paper` · `folding something` · `forklift carrying pallet` · `forklift lifting load` · `forklift lowering forks` · `forklift moving` · `forklift raising forks` · `forklift reversing` · `forklift turning` · `front raises` · `frying vegetables` **G** — `gargling` · `getting a haircut` · `getting a piercing` · `getting a tattoo` · `giving or receiving award` · `golf chipping` · `golf driving` · `golf putting` · `grinding meat` · `grinding metal` · `grooming cat` · `grooming dog` · `grooming horse` · `gymnastics tumbling` **H** — `hammer throw` · `hand washing clothes` · `head stand` · `headbanging` · `headbutting` · `herding cattle` · `high jump` · `high kick` · `hitching trailer` · `hitting baseball` · `hockey stop` · `holding snake` · `holding something` · `holding something in front of something` · `holding something next to something` · `holding something over something` · `hoverboarding` · `hugging` · `hugging (not baby)` · `hugging baby` · `hula hooping` · `hurdling` · `hurling (sport)` · `hurling sport` **I** — `ice climbing` · `ice fishing` · `ice skating` · `in a car accident` · `in a riot` · `in a road accident` · `in an explosion` · `inflating balloons` · `inspecting parts` · `inspecting vehicle` · `installing carpet` · `installing drywall` · `ironing` · `ironing clothes` · `ironing hair` **J** — `javelin throw` · `jetskiing` · `jogging` · `juggling balls` · `juggling fire` · `juggling soccer ball` · `jumping bicycle` · `jumping into pool` · `jumping jacks` · `jumping sofa` · `jumpstyle dancing` **K** — `kicking field goal` · `kicking soccer ball` · `kissing` · `kitesurfing` · `knitting` · `krumping` **L** — `labeling package` · `laughing` · `laying bricks` · `laying concrete` · `laying decking` · `laying stone` · `laying tiles` · `letting go of balloon` · `letting something roll down a surface` · `licking` · `lifting a surface with something on it` · `lifting box` · `lifting hat` · `lifting something` · `lifting something up completely` · `lifting with bent back` · `lighting candle` · `lighting fire` · `listening with headphones` · `loading cargo into vehicle` · `loading dock work` · `loading pallet` · `loading shipping container` · `loading truck` · `lock picking` · `long jump` · `looking at phone` · `looking in mirror` · `lowering box` · `lubricating machine` · `lunge` **M** — `machine wash` · `making a cake` · `making a sandwich` · `making balloon shapes` · `making bed` · `making bubbles` · `making cheese` · `making horseshoes` · `making jewelry` · `making latte art` · `making pizza` · `making slime` · `making snowman` · `making sushi` · `making tea` · `making the bed` · `marching` · `marking measurements` · `massaging back` · `massaging feet` · `massaging legs` · `massaging neck` · `massaging person's head` · `massaging persons head` · `measuring with tape` · `milking cow` · `milking goat` · `mixing colours` · `monitoring gauges` · `mopping floor` · `motorcycling` · `mounting forklift` · `moving baby` · `moving child` · `moving furniture` · `moving pallet` · `moving something across a surface` · `moving something away from something` · `moving something closer to something` · `moving something down` · `moving something up` · `mowing lawn` · `mushroom foraging` **N** — `news anchoring` **O** — `opening bottle` · `opening bottle (not wine)` · `opening coconuts` · `opening door` · `opening present` · `opening refrigerator` · `opening something` · `opening wine bottle` · `operating backhoe` · `operating bulldozer` · `operating cement mixer` · `operating chainsaw` · `operating cherry picker` · `operating CNC machine` · `operating control panel` · `operating conveyor belt` · `operating crane` · `operating drill press` · `operating dump truck` · `operating excavator` · `operating forklift` · `operating forklift charger` · `operating injection molding machine` · `operating lathe` · `operating machinery` · `operating pallet jack` · `operating press brake` · `operating road roller` · `operating scissor lift` · `operating skid steer` **P** — `packing` · `packing box` · `pan roast` · `paragliding` · `parasailing` · `parking forklift` · `parking truck` · `parkour` · `passing American football (in game)` · `passing American football (not in game)` · `passing American football in game` · `passing American football not in game` · `passing soccer ball` · `paving road` · `peeling apple` · `peeling apples` · `peeling banana` · `peeling potatoes` · `performing maintenance` · `petting animal (not cat)` · `petting animal not cat` · `petting cat` · `petting horse` · `picking apples` · `picking blueberries` · `picking fruit` · `picking order` · `picking something up` · `pillow fight` · `pinching` · `placing item on conveyor` · `plastering` · `playing accordion` · `playing american football` · `playing badminton` · `playing bagpipes` · `playing basketball` · `playing bass guitar` · `playing beer pong` · `playing billiards` · `playing blackjack` · `playing cards` · `playing cello` · `playing checkers` · `playing chess` · `playing clarinet` · `playing controller` · `playing cricket` · `playing cymbals` · `playing darts` · `playing didgeridoo` · `playing dominoes` · `playing drums` · `playing field hockey` · `playing flute` · `playing gong` · `playing guitar` · `playing hand clapping games` · `playing handball` · `playing harmonica` · `playing harp` · `playing ice hockey` · `playing keyboard` · `playing kickball` · `playing laser tag` · `playing lute` · `playing mahjong` · `playing maracas` · `playing marbles` · `playing monopoly` · `playing netball` · `playing nose flute` · `playing oboe` · `playing ocarina` · `playing organ` · `playing paintball` · `playing pan pipes` · `playing piano` · `playing piccolo` · `playing pinball` · `playing ping pong` · `playing poker` · `playing polo` · `playing recorder` · `playing road hockey` · `playing rounders` · `playing rubiks cube` · `playing saxophone` · `playing scrabble` · `playing shuffleboard` · `playing slot machine` · `playing squash or racquetball` · `playing tennis` · `playing trombone` · `playing trumpet` · `playing ukulele` · `playing violin` · `playing volleyball` · `playing with trains` · `playing xylophone` · `plumbing` · `poaching eggs` · `poking bellybutton` · `poking something so it falls over` · `poking something so it slightly moves` · `pole vault` · `polishing furniture` · `polishing metal` · `popping balloons` · `pouring beer` · `pouring concrete` · `pouring milk` · `pouring something into something` · `pouring something onto something` · `pouring wine` · `preparing salad` · `presenting weather forecast` · `pressing emergency stop` · `pretending to be a statue` · `pretending to pick something up` · `pretending to put something next to something` · `pretending to take something from somewhere` · `pretending to throw something` · `pull ups` · `pulling cart` · `pulling espresso shot` · `pulling rope (game)` · `pulling something from behind of something` · `pulling something from left to right` · `pulling something from right to left` · `pulling something onto something` · `pulling something out of something` · `pulling two ends of something so that it gets stretched` · `pumping fist` · `pumping gas` · `punching bag` · `punching person (boxing)` · `punching person boxing` · `push up` · `pushing car` · `pushing cart` · `pushing hand truck` · `pushing something from left to right` · `pushing something from right to left` · `pushing something off of something` · `pushing something onto something` · `pushing something so it spins` · `pushing something so that it almost falls off but doesnt` · `pushing something so that it falls off the table` · `pushing something with something` · `pushing wheelbarrow` · `pushing wheelchair` · `putting in contact lenses` · `putting makeup on someone` · `putting on eyeliner` · `putting on foundation` · `putting on gloves` · `putting on hard hat` · `putting on lipstick` · `putting on mascara` · `putting on safety glasses` · `putting on safety vest` · `putting on sari` · `putting on shoes` · `putting something and something on the table` · `putting something behind something` · `putting something in front of something` · `putting something into something` · `putting something next to something` · `putting something on a flat surface without letting it roll` · `putting something on a surface` · `putting something on the edge of something so it is not supported and falls down` · `putting something onto a slanted surface but it doesnt glide down` · `putting something onto something` · `putting something similar to other things that are already on the table` · `putting something that cant roll onto a slanted surface so it slides down` · `putting something underneath something` · `putting something upright on the table` · `putting wallpaper on wall` **R** — `raising eyebrows` · `reading book` · `reading instruments` · `reading newspaper` · `recording music` · `refueling vehicle` · `removing item from conveyor` · `removing something spreading it apart from something` · `repairing machine` · `repairing puncture` · `restocking shelves` · `reversing truck` · `riding a bike` · `riding camel` · `riding elephant` · `riding mechanical bull` · `riding mountain bike` · `riding mule` · `riding or walking with horse` · `riding scooter` · `riding snow blower` · `riding unicycle` · `ripping paper` · `roasting marshmallows` · `roasting pig` · `robot dancing` · `rock climbing` · `rock scissors paper` · `roller skating` · `rolling eyes` · `rolling pastry` · `rolling something on a flat surface` · `roofing work` · `rope pushdown` · `rowing boat` · `running on treadmill` **S** — `sailing` · `salsa dancing` · `sanding floor` · `sanding metal` · `sanding wood` · `sawing wood` · `scanning barcode` · `scrambling eggs` · `scrubbing face` · `scuba diving` · `sealing box` · `seasoning food` · `separating eggs` · `setting table` · `sewing` · `shaking hands` · `shaking head` · `shaking something` · `shaping bread dough` · `sharpening knives` · `sharpening pencil` · `shaving head` · `shaving legs` · `shearing sheep` · `shining flashlight` · `shining shoes` · `shooting` · `shooting basketball` · `shooting goal (soccer)` · `shooting goal soccer` · `shooting off fireworks` · `shoplifting` · `shopping` · `shot put` · `shoveling snow` · `showing something behind something` · `showing something next to something` · `showing something on top of something` · `showing something to the camera` · `shredding paper` · `shucking oysters` · `shuffling cards` · `shuffling feet` · `side kick` · `sign language interpreting` · `singing` · `sipping cup` · `situp` · `skateboarding` · `ski ballet` · `ski jumping` · `skiing crosscountry` · `skiing mono` · `skiing slalom` · `skipping rope` · `skipping stone` · `skydiving` · `slacklining` · `slapping` · `sled dog racing` · `slicing onion` · `slipping on floor` · `smelling feet` · `smoking` · `smoking hookah` · `smoking pipe` · `snatch weight lifting` · `sneezing` · `sniffing` · `snorkeling` · `snowboarding` · `snowkiting` · `snowmobiling` · `somersaulting` · `sorting packages` · `spinning plates` · `spinning poi` · `spinning something so it continues spinning` · `spinning something that quickly stops spinning` · `splashing water` · `spraying` · `springboard diving` · `squat` · `squeezing orange` · `squeezing something` · `stacking boxes` · `stacking cups` · `stacking dice` · `stacking pallets` · `stacking something` · `stamping something` · `standing on hands` · `starting machine` · `stealing something` · `steering car` · `sticking something on something` · `sticking tongue out` · `stir fry` · `stirring` · `stocking shelves` · `stomping grapes` · `stopping machine` · `stretching arm` · `stretching leg` · `strumming guitar` · `stuffing something into something` · `sucking lolly` · `surfing crowd` · `surfing water` · `surveying site` · `sweeping floor` · `sweeping warehouse floor` · `swimming backstroke` · `swimming breast stroke` · `swimming butterfly stroke` · `swimming front crawl` · `swimming with dolphins` · `swimming with sharks` · `swing dancing` · `swinging baseball bat` · `swinging legs` · `swinging on something` · `sword fighting` · `sword swallowing` **T** — `tackling football` · `tagging graffiti` · `tai chi` · `taking a shower` · `taking inventory` · `taking photo` · `taking something from somewhere` · `taking something out of something` · `talking on cell phone` · `tango dancing` · `tap dancing` · `tapping guitar` · `tapping pen` · `tasting beer` · `tasting food` · `tasting wine` · `texting` · `throwing axe` · `throwing ball` · `throwing ball (not baseball or American football)` · `throwing discus` · `throwing knife` · `throwing snowballs` · `throwing something` · `throwing something against something` · `throwing something in the air and catching it` · `throwing something in the air and letting it fall` · `throwing something onto a surface` · `throwing tantrum` · `throwing water balloon` · `tickling` · `tipping something over` · `tipping something with something in it over so everything falls out` · `tobogganing` · `torquing bolts` · `tossing coin` · `tossing salad` · `towing trailer` · `towing vehicle` · `training dog` · `trapezing` · `treating wood` · `trimming or shaving beard` · `trimming shrubs` · `trimming trees` · `triple jump` · `tripping over object` · `truck backing to dock` · `tumble dry` · `tumbling something downwards` · `turning something upside down` · `turning the camera left while filming something` · `turning the camera right while filming something` · `twiddling fingers` · `tying bow tie` · `tying knot (not on a tie)` · `tying knot not on a+rope` · `tying necktie` · `tying shoe laces` **U** — `uncorking champagne` · `uncovering something` · `unfolding something` · `unloading cargo from vehicle` · `unloading pallet` · `unloading shipping container` · `unloading truck` · `unpacking box` · `unstacking boxes` · `using a microscope` · `using a paint roller` · `using a power drill` · `using a sledge hammer` · `using a wrench` · `using atm` · `using bagging machine` · `using circular saw` · `using computer` · `using inhaler` · `using megaphone` · `using puppets` · `using remote controller (not gaming)` · `using remote controller not gaming` · `using segway` **V** — `vacuuming car` · `vacuuming floor` · `vandalizing something` · `vault` · `vehicle accelerating` · `vehicle driving` · `vehicle idling` · `vehicle moving` · `vehicle parking` · `vehicle reversing` · `vehicle stopping` · `vehicle turning` · `visiting museum` · `visiting the zoo` **W** — `wading through mud` · `wading through water` · `waiting in line` · `waking up` · `walking on stilts` · `walking the dog` · `walking through snow` · `walking with crutches` · `washing dishes` · `washing feet` · `washing hair` · `washing hands` · `washing vehicle` · `watching conveyor belt` · `watching tv` · `water skiing` · `water sliding` · `watering plants` · `waving hand` · `waxing armpits` · `waxing back` · `waxing chest` · `waxing eyebrows` · `waxing legs` · `wearing hard hat` · `wearing respirator` · `wearing safety harness` · `wearing safety vest` · `weaving basket` · `weaving fabric` · `welding` · `welding metal` · `whistling` · `windsurfing` · `wiping something off of something` · `working on scaffolding` · `wrapping pallet` · `wrapping present` · `wrestling` · `writing` **Y** — `yawning` · `yoga` **Z** — `zumba` --- *This list reflects the deployed vocabulary and updates when the model checkpoint updates. Last verified: 2026-08-07.* --- # Error registry Every error code the API can return, with retry semantics. This registry is closed and append-only — codes are never removed or repurposed within v1. Machine-readable at `GET https://api.primateintelligence.ai/v1/errors`. **retryable** = safe to retry the same request unchanged (with backoff). **idem** = idempotent to retry with the same `Idempotency-Key`. ## HTTP errors ### `invalid_api_key` **HTTP 401** · retryable: **no** · idem: **no** API key is invalid. Check the key, env var, and whitespace. ### `key_revoked` **HTTP 401** · retryable: **no** · idem: **no** This API key has been revoked. Rotate or create a new key. ### `key_expired` **HTTP 401** · retryable: **no** · idem: **no** This API key has expired. Create a new key. ### `token_expired` **HTTP 401** · retryable: **no** · idem: **no** Client token has expired. Mint a new token from your server (client tokens are never refreshable). ### `insufficient_scope` **HTTP 403** · retryable: **no** · idem: **no** The key does not carry the scope required for this operation. ### `account_restricted` **HTTP 403** · retryable: **no** · idem: **no** Account is restricted. Complete signup/verification. ### `test_mode_only` **HTTP 403** · retryable: **no** · idem: **no** Operation not available in this key mode. ### `test_key_fixture_only` **HTTP 403** · retryable: **no** · idem: **no** Test keys can only analyze the provided fixture video. Create a live key to analyze your own uploads. ### `validation_failed` **HTTP 400** · retryable: **no** · idem: **no** Request validation failed. All violations are listed in error.errors[]. ### `unsupported_media_type` **HTTP 415** · retryable: **no** · idem: **no** Send video/mp4 or video/quicktime with a correct Content-Type. ### `payload_too_large` **HTTP 413** · retryable: **no** · idem: **no** Payload exceeds the 100MB multipart cap. Use the presigned upload path. ### `resource_not_found` **HTTP 404** · retryable: **no** · idem: **no** No such resource. IDs are account-scoped; check the ID and prefix. ### `resource_conflict` **HTTP 409** · retryable: **no** · idem: **no** Invalid state transition. GET the resource for its current state. ### `idempotency_key_reused` **HTTP 409** · retryable: **no** · idem: **no** Idempotency-Key was reused with a different request body. ### `idempotency_unavailable` **HTTP 409** · retryable: **yes** · idem: **yes** Idempotency store unavailable. Retry with the same key after Retry-After. ### `stream_already_active` **HTTP 409** · retryable: **no** · idem: **no** An active stream already exists on this account. End it (POST /v1/streams/{id}/end) or wait for it to finish. ### `upload_incomplete` **HTTP 400** · retryable: **no** · idem: **no** Upload incomplete or size mismatch. Re-create the video and complete within 24h. ### `video_unreadable` **HTTP 400** · retryable: **no** · idem: **no** Container/codec unsupported. Re-encode to H.264 MP4. ### `video_too_large` **HTTP 400** · retryable: **no** · idem: **no** Video exceeds the 2 GiB limit. ### `video_too_long` **HTTP 400** · retryable: **no** · idem: **no** Video exceeds the plan duration limit. Upgrade or trim. ### `url_fetch_failed` **HTTP 400** · retryable: **yes** · idem: **no** Source URL unreachable or timed out. ### `url_forbidden` **HTTP 400** · retryable: **no** · idem: **no** URL scheme or host blocked by the ingest policy. ### `prompt_empty` **HTTP 400** · retryable: **no** · idem: **no** Provide prompt or query. ### `prompt_too_long` **HTTP 400** · retryable: **no** · idem: **no** Prompt exceeds 2000 characters. ### `query_invalid` **HTTP 400** · retryable: **no** · idem: **no** Body fails the CompiledQuery schema; param names the field. ### `parse_failed` **HTTP 422** · retryable: **yes** · idem: **no** The compiler could not produce a query. Rephrase or send a structured query. ### `model_not_found` **HTTP 400** · retryable: **no** · idem: **no** Unknown model. See GET /v1/models. ### `model_deprecated` **HTTP 400** · retryable: **no** · idem: **no** Model is deprecated. Pin a supported version; see the changelog. ### `analysis_not_cancelable` **HTTP 409** · retryable: **no** · idem: **no** Analysis is already in a terminal state. ### `rate_limit_exceeded` **HTTP 429** · retryable: **yes** · idem: **yes** Rate limit exceeded. Back off per Retry-After. ### `concurrency_limit_exceeded` **HTTP 429** · retryable: **yes** · idem: **yes** Concurrent analysis limit reached. Wait for an active analysis to finish. ### `quota_exceeded` **HTTP 429** · retryable: **no** · idem: **no** Quota exceeded for a metered limit. See details.meter. ### `insufficient_credits` **HTTP 402** · retryable: **no** · idem: **no** Insufficient credits. Add credits or enable auto-refill. ### `grant_exhausted` **HTTP 402** · retryable: **no** · idem: **no** Free grant exhausted. Claim a full account via POST /v1/keys/request to continue. ### `capacity_exhausted` **HTTP 503** · retryable: **yes** · idem: **yes** Platform queue is full. Honor Retry-After. ### `sandbox_limit_exceeded` **HTTP 429** · retryable: **yes** · idem: **yes** Sandbox provisioning limit reached for this IP. Retry after the window resets, or sign up for a live key. ### `upgrade_limit_exceeded` **HTTP 429** · retryable: **no** · idem: **no** Too many upgrade attempts from this IP. Retry after the window resets. ### `device_code_expired` **HTTP 410** · retryable: **no** · idem: **no** Device code has expired or been consumed. Start a new request. ### `inference_unavailable` **HTTP 503** · retryable: **yes** · idem: **yes** Transient inference backend outage. Retry with backoff. ### `upstream_error` **HTTP 502** · retryable: **no** · idem: **no** Inference upstream returned an error. Message is sanitized — no internal paths or stack traces. ### `db_unavailable` **HTTP 503** · retryable: **yes** · idem: **yes** Transient database outage. Retry with backoff. ### `internal_error` **HTTP 500** · retryable: **yes** · idem: **yes** Internal error. Includes request_id; contact support if persistent. ### `webhook_endpoint_limit` **HTTP 400** · retryable: **no** · idem: **no** At most 10 webhook endpoints per account. ## Resource-level errors These never appear as HTTP responses — only in `analysis.error.code` / `video.error.code` on failed resources. ### `inference_error` Model failed on this input. Safe to create a new analysis; persistent failures on one video → support with request_id. ### `stuck_timeout` The platform lost the job past the sweep window. Not billed; resubmit. --- # Primate Intelligence API — Changelog Sorted newest-first. Agent-readable summary of shipped changes, broken by date. Full field contracts: [openapi.json](./openapi.json) · [agents.md](./agents.md) --- ## 2026-08-14 — Design Partner Agreements admin: status dropdown, delete, signed-PDF link ### Added - `PATCH /v1/admin/design-partner-agreements/:id` — set `status` to `sent`, `paid`, `terminated`, or `completed` from the admin table (manual states not produced by the automated click-to-sign/pay flow). - `DELETE /v1/admin/design-partner-agreements/:id` — deletes the row and removes its signed PDF from the private `design-partner-agreements` Storage bucket (best-effort on the Storage side; the DB delete is authoritative). - `GET /v1/admin/design-partner-agreements/:id/pdf` — mints a 5-minute signed Storage URL for the signed PDF; 404 if the agreement was never signed (no `pdf_url`). - Migration: widened the `design_partner_agreements.status` CHECK constraint to include `terminated` and `completed`. - Admin UI: status pull-down (styled as the existing status badge), delete icon with a confirm dialog, and a document icon linking to the signed PDF for each agreement row. ## 2026-08-14 — Design Partner Agreement: $500 token default, public-reference opt-out, legal fixes ### Changed - Default token payment corrected to $500 (was ~$10K) when `priceUsd` is omitted on create; admin can still override per-agreement. - New `public_reference_opt_out` column + end-to-end plumbing: partner can decline PUBLIC use of name/logo/quote; PRIVATE investor-facing reference is always retained regardless (non-negotiable), asserted unconditionally in Section 10. - Section 10 states the default scope of work explicitly (API access at no additional charge beyond the token fee, founder feedback line, case study conditioned on not opting out of public reference). ### Fixed - PDF design-review bugs: mid-sentence page-break split, orphaned section heading across pages (keep-with-next), Section 6 survival list missing 9/10, Section 8 liability floor lower than the $500 fee (now greater-of), Section 1 scope-card figures marked non-binding. ## 2026-08-14 — PRI-586: exempt /llms.txt, /llms-full.txt, /docs from dev password gate ### Fixed - `/llms.txt`, `/llms-full.txt`, and `/docs/*` are now exempt from the dev Basic-auth password gate. These are meant to be publicly fetchable everywhere per the llmstxt.org convention (PRI-104/PRI-487) — CI (Docs code samples / Sample matrix workflows) and external agents/LLMs following the documented discovery flow hit them unauthenticated. Gating them behind Basic-auth 401'd the served corpus, failing both checks on every dev push and blocking Railway's GitHub-integration auto-deploy for dev (forcing manual Railway-API deploys as a workaround). ## 2026-08-13 — Design Partner Agreement: scope fields + branded PDF + admin dev-gate exemption ### New - Three new optional agreement fields wired end-to-end: `valueOfSolving`, `scopeOfWork`, `estimatedCostToDeliver` — accepted on admin create, returned on admin list/detail and the public by-token view, interpolated into the canonical terms text (and therefore `doc_content_hash`). - Agreement PDF redesigned as a branded marketing-style document: Darwin logo + wordmark header, signal-blue accent band, "Scope of this Pilot" summary card, styled signature block. Terms text remains the verbatim hashed string (tamper evidence unchanged). ### Fixed - Share links now use `APP_BASE_URL` (the React app host) instead of `FRONTEND_URL` (the marketing site, which has no /design-partner route — 404 on prod host, edge password gate on dev). This was the remaining half of the blocked click-through flow on dev. - `/v1/admin/design-partner-agreements` added to `DEV_GATE_EXEMPT_PREFIXES` — admin create/list calls from the Admin UI 401'd behind the dev Basic-auth gate (they're already Bearer-JWT + ADMIN_EMAILS gated). ## 2026-08-13 — Fix dev password gate 500ing every non-Basic-auth request - The dev preview gate's `WWW-Authenticate` header value contained an em-dash (non-Latin1 byte), which crashes Node's header serialization (`ERR_INVALID_CHAR`) on every request that doesn't already carry Basic auth — including Bearer-JWT admin API calls. Found testing the Design Partner Agreement admin routes; replaced with a plain hyphen. ## 2026-08-13 — Design Partner Agreement public routes exempted from dev password gate - `/v1/design-partner-agreements` added to `DEV_GATE_EXEMPT_PREFIXES` (dev only) — the public by-token get/sign/create-checkout-session routes have no auth by design and were 401ing behind the dev Basic-auth gate. ## 2026-08-13 — Design Partner Agreement click-to-sign / click-to-pay - Replaces the DocuSign-based Design Partner Agreement flow. Admin-only `POST/GET /v1/admin/design-partner-agreements` (+`/:id` detail) create and list shareable agreement links. Public, token-gated (no auth): `GET /v1/design-partner-agreements/by-token/:token`, `POST .../sign` (captures signer name/title/IP/UA, hashes the canonical terms text, generates + emails a signed PDF), and `POST .../create-checkout-session` (Stripe Checkout for priced pilots). The Stripe webhook's `checkout.session.completed` handler now marks a matching agreement `paid`. New `design_partner_agreements` table + private `design-partner-agreements` Storage bucket. ## 2026-08-10 — Changelog reset ahead of launch - This changelog now tracks **public product releases only** — API, SDK, and web-app changes that affect the published contract. Release notes will appear here as they ship. For the current API contract, see [openapi.json](./openapi.json) and [agents.md](./agents.md). > Canonical, always-current copy: https://api.primateintelligence.ai/docs/changelog.md (also served at https://primateintelligence.ai/docs/changelog.md — same bytes, proxied).