# Primate Vision API — full documentation corpus > Generated from https://primateintelligence.ai/docs. Spec: https://api.primateintelligence.ai/v1/openapi.json --- # Quickstart # Quickstart Primate Vision answers questions about video. Upload a video (or point us at a URL), ask a question in plain English, and get a deterministic answer with confidence and clip timestamps — no hallucinations. This page gets you from zero to a verified result in about a minute, **without creating an account**. ## 1. Get a key One command. No email, no card: ```bash doc-test id=quickstart-sandbox curl -s -X POST https://api.primateintelligence.ai/v1/sandbox ``` The response contains your `pv_test_` key plus a ready-to-analyze fixture video: ```json { "object": "sandbox_key", "api_key": "pv_test_…", "mode": "test", "expires_at": "2026-07-16T00:00:00Z", "fixture_video_id": "video_…", "fixture_prompt": "Is there a person in this video?", "expected_answer": "yes" } ``` Export it: ```bash export PRIMATE_API_KEY="pv_test_…" # from the response above ``` Test keys return **deterministic canned results** for the fixture corpus — perfect for CI. They can't spend credits or touch the GPU, and expire in 7 days. Graduating to a `pv_live_` key requires [signup](https://primateintelligence.ai/signup) (a billing gate, not an integration gate — your code doesn't change). ## 2. Create an analysis Your sandbox account comes pre-seeded with a fixture video. Ask a question about it — `Prefer: wait` holds the connection until the result is ready (up to 120s), collapsing the poll loop: ```bash doc-test id=quickstart-analyze env=VIDEO_ID curl -s -X POST https://api.primateintelligence.ai/v1/analyses \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -H "Prefer: wait=60" \ -d '{"video_id": "'"$VIDEO_ID"'", "prompt": "Is there a person in this video?"}' ``` ## 3. Read the result The response is the full analysis resource: ```json { "id": "an_…", "object": "analysis", "status": "completed", "result": { "answer": "yes", "confidence": 0.93, "clips": [{ "start_s": 1, "end_s": 4, "confidence": 0.93 }], "query_type": "object" } } ``` That's the whole loop: **video → question → answer**. `result.answer` is always `yes`, `no`, or `indeterminate`; `result.clips` tells you *where* in the video. ## The same thing in TypeScript ```typescript doc-test id=quickstart-ts sdk=ts import Primate from '@primate-intelligence/sdk'; const client = new Primate(); // reads PRIMATE_API_KEY const videos = await client.videos.list(); // fixture video is pre-seeded const analysis = await client.analyses.createAndWait({ video_id: videos.data[0].id, prompt: 'Is there a person in this video?', }); console.log(analysis.result?.answer, analysis.result?.confidence); ``` ## The same thing in Python ```python doc-test id=quickstart-py sdk=py from primate_intelligence import Primate client = Primate() # reads PRIMATE_API_KEY videos = client.videos.list() # fixture video is pre-seeded analysis = client.analyses.create_and_wait( video_id=videos["data"][0]["id"], prompt="Is there a person in this video?", ) print(analysis["result"]["answer"], analysis["result"]["confidence"]) ``` ## Next steps - **[Upload your own video](/docs/guides/uploading)** — presigned upload, URL ingest, or the SDK helper - **[Write better prompts](/docs/guides/prompts)** — what's answerable, structured queries, `unassessable_components` - **[Stop polling](/docs/guides/waiting)** — `Prefer: wait` vs webhooks - **[Building with an AI agent?](/docs/agents)** — the condensed machine-readable path - **[API reference](/docs/reference)** — every endpoint, generated from the OpenAPI spec --- # Primate Intelligence Public API — Agent Guide Machine-consumable reference for AI agents integrating with the Primate Intelligence Public API (`/v1`). --- ## Quick Start ### 1. Get a sandbox key (zero-human, instant) ```http POST /v1/sandbox ``` Returns a `pv_test_` key, a `fixture_video_id`, and a `fixture_prompt`. No auth required. **Response includes:** - `livemode: false` — all sandbox responses are always test-mode - `upgrade` — instructions for upgrading to a live key (see below) ### 2. Upgrade to a free live key (no account required) Call `POST /v1/keys/upgrade` with your `pv_test_` key. Receives a real `pv_live_` key with a **6,000-second free credit grant**. No email, no card, no human-in-the-loop. ```http POST /v1/keys/upgrade Authorization: Bearer pv_test_ ``` Response includes a `pv_live_` key and `tier: "free_grant"`. Free-grant keys: - Run real GPU inference (`livemode: true`) - Have no overage — when the grant is exhausted, analyses return `402 grant_exhausted` with a link to the claim flow for billed keys ### 3. Get a billed live key (device-code flow) ```http POST /v1/keys/request ``` Returns `{device_code, claim_url, poll_interval}`. Direct the user to `claim_url`. Poll: ```http GET /v1/keys/request/{device_code} ``` Returns `{status: "pending"}` until the user approves in the browser. On approval returns the billed `pv_live_` key ONCE (subsequent polls return `410 Gone`). --- ## The `livemode` Field **Every public API resource includes a top-level `"livemode": boolean` field** (Stripe-style). | Value | Meaning | |-------|---------| | `true` | Request used a `pv_live_` key — results are real GPU inference | | `false` | Request used a `pv_test_` key or sandbox — results are deterministic canned fixtures | **Agents SHOULD verify `livemode` matches their expectation before relaying results to users or downstream systems.** A result with `livemode: false` is a fixture/test result and must never be treated as evidence about real video content. Example analysis response: ```json { "object": "analysis", "livemode": true, "status": "completed", "result": { "answer": "yes", "confidence": 0.97, ... } } ``` --- ## Test Key Restrictions Test (`pv_test_`) keys **only analyze the fixture video** seeded by `/v1/sandbox`. Attempting to analyze any other video returns: ```json { "error": { "code": "test_key_fixture_only", "message": "Test keys can only analyze the provided fixture video. Create a live key to analyze your own uploads.", "status": 403 } } ``` This prevents fabricated results on real video content. --- ## Video Metadata After `POST /v1/videos/{id}/complete`, the API probes the uploaded S3 object and populates: | Field | Type | Description | |-------|------|-------------| | `duration_s` | number \| null | Duration in seconds | | `width` | integer \| null | Frame width in pixels | | `height` | integer \| null | Frame height in pixels | | `fps` | number \| null | Frames per second | Agents can use these fields to verify the right file arrived before submitting an analysis. Probe failures are non-fatal — fields remain `null` if probing fails. --- ## Error Codes | Code | HTTP | Retryable | Description | |------|------|-----------|-------------| | `test_key_fixture_only` | 403 | no | Test key tried to analyze a non-fixture video. Use a live key. | | `grant_exhausted` | 402 | no | Free-grant credits depleted. Use the claim flow to get a billed key. | | `insufficient_credits` | 402 | no | Live key ran out of credits. Add credits or upgrade. | | `sandbox_limit_exceeded` | 429 | yes | Too many sandbox provisions from this IP. Retry after the window. | | `upgrade_limit_exceeded` | 429 | yes | Too many free-grant upgrades from this IP today. | | `invalid_api_key` | 401 | no | Key not found or malformed. | | `key_expired` | 401 | no | Key has passed its `expires_at`. | Full registry: `GET /v1/errors` --- ## Key Lifecycle ``` POST /v1/sandbox → pv_test_ key (livemode: false, fixture-only, 7-day TTL) │ ▼ POST /v1/keys/upgrade (auth: pv_test_ key) → pv_live_ key (livemode: true, tier: free_grant, 6,000s grant, 30-day idle expiry) │ (grant exhausted → 402 grant_exhausted) ▼ POST /v1/keys/request → GET /v1/keys/request/{code} → billed pv_live_ key (device-code claim flow — user approves in browser) ``` --- ## What the Model Supports Design prompts as targeted visual questions. The model natively handles: | Type | Example | `answer` | `detected_count` | `clips` | Accuracy | |------|---------|----------|-----------------|--------|---------| | **Presence** | "Is there a person?" | yes/no/indeterminate | — | ✅ when yes | Stable | | **Absence** | "Are there no dogs?" | yes/no | — | — | Stable | | **Action** | "Is someone walking?" | yes/no/indeterminate | — | ✅ when yes | Beta | | **Count — bare** | "How many people walking?" | — | ✅ integer | — | Stable | | **Count — threshold** | "More than 2 people?" | yes/no | ✅ integer | — | Stable | | **Compound** | "Person AND dog?" | yes/no | — | ✅ when yes | Stable | | **Attribute** | "Is anyone in a red jacket?" | description | — | — | Stable | | **Location** | "Where is the dog?" | description | — | — | Stable | | **State** | "Is the door open?" | description | — | — | Stable | | **Segment** | "Show me where the person is" | — | — | ✅ with masks | Stable | > **Accuracy note:** Action queries (motion/activity detection) are **beta accuracy** — results may be less reliable than presence/counting queries on short or fast-moving clips. **Not supported:** audio/sound, identity (who, not what), OCR/text in frame, subjective judgment, exhaustive count over long multi-hour footage. Open-ended prompts ("Tell me what you see") return a description but have lower accuracy — prefer a targeted question. Full guide: [Prompts & queries](https://primateintelligence.ai/docs/guides/prompts) --- ## Remote MCP Endpoint Primate Intelligence exposes a hosted [Model Context Protocol](https://modelcontextprotocol.io) server at: ``` https://api.primateintelligence.ai/mcp ``` Transport: **Streamable HTTP** (MCP spec 2025-03-26+, stateless mode). No npx or local install required — any remote agent can connect directly. ### Authentication Pass your Primate Vision API key as a Bearer token in every request: ``` Authorization: Bearer pv_live_ ``` Test keys (`pv_test_…`) also work and return deterministic fixture results. Get a free key: ```bash # Instant sandbox key (no auth required) curl -X POST https://api.primateintelligence.ai/v1/sandbox # Upgrade to a live key with a 6,000-second free credit grant curl -X POST https://api.primateintelligence.ai/v1/keys/upgrade \ -H "Authorization: Bearer pv_test_" ``` Requests without a valid key receive a 401 JSON-RPC error with provisioning guidance. ### Available Tools The remote endpoint exposes the same 10 tools as the npx package: `create_video_from_url`, `create_analysis`, `validate_analysis`, `create_analysis_batch`, `get_analysis`, `wait_for_analysis`, `list_models`, `get_usage`, `get_credits`, `get_test_fixture`. - `validate_analysis` — free dry-run (`validate_only: true`): compiled query, assessability, and cost estimate before you spend credits. - `create_analysis_batch` — 2–10 prompts on one video; the first is full price, each additional is billed at 50%. - `get_credits` — balance plus the per-analysis transaction ledger (`GET /v1/credits`); prefer it over `get_usage` for auditing what each analysis cost. Every tool declares an `outputSchema` (visible in `tools/list` and on the static server card at `/.well-known/mcp/server-card.json`), generated from the same schemas that validate the public /v1 responses — and every result carries `structuredContent` conforming to it, alongside the human-readable JSON text content. **`wait_for_analysis` response envelope:** the tool returns `{ analysis, retry }` — the [Analysis resource](https://primateintelligence.ai/docs/api#analyses) unmodified under `analysis`, plus `retry: null` when the analysis reached a terminal state, or `retry: { reason: "timeout", note }` when the wait expired (call `wait_for_analysis` or `get_analysis` again). Earlier versions merged an `_mcp_note` field into the analysis object on timeout; that field is gone. ### Client Configuration Examples **Claude Desktop / MCP config JSON:** ```json { "mcpServers": { "primate-intelligence": { "type": "streamable-http", "url": "https://api.primateintelligence.ai/mcp", "headers": { "Authorization": "Bearer pv_live_" } } } } ``` **ChatGPT app directory / custom GPT:** ``` URL: https://api.primateintelligence.ai/mcp Auth: Bearer token → your pv_live_ or pv_test_ key ``` **OpenAI Agents SDK (Python):** ```python from agents.mcp import MCPServerStreamableHttp server = MCPServerStreamableHttp( url="https://api.primateintelligence.ai/mcp", headers={"Authorization": "Bearer pv_live_"}, ) ``` ### MCP Registry The server is also listed in the MCP registry at `ai.primateintelligence/mcp` with both the npx stdio package and this remote entry. See `mcp/server.json` in the repository. --- ## Result Contract ```json { "object": "analysis", "livemode": true, "status": "completed", "result": { "answer": "yes", "confidence": 0.97, "detected_count": 3, "clips": [{"start_s": 1.2, "end_s": 3.4, "confidence": 0.94}] }, "usage": { "billed_seconds": 6, "credit_balance_after": 5994 } } ``` - `result.answer` ∈ `yes | no | indeterminate`. `indeterminate` = model couldn't commit; don't ship a decision. - `result.detected_count` — populated for count-intent queries. Integer ≥ 0. - `result.confidence` ∈ [0, 1]. Zero confidence with no positive detections returns `indeterminate`, not `no`. For count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected. See the [Count queries](#count-queries--the-contract) section for full semantics. - `result.clips[]` — present for presence/action/compound/segment when `answer: "yes"`. Null otherwise. - `query.unassessable_components[]` — lists what couldn't be evaluated (e.g. audio). API answers what it can. - `usage.billed_seconds` — seconds charged for this analysis. - `usage.credit_balance_after` — the balance immediately after **this** analysis settled. Immutable point-in-time snapshot — it never changes as later analyses run. A `null` value (`snapshot_unavailable`) means the analysis settled before the snapshot feature shipped (migration 075) and the original balance is unknowable; treat it as missing data rather than zero. - `livemode: true` = real GPU inference. Never relay `livemode: false` results as evidence about real content. - `origin` ∈ `api | console | system` — how the analysis was created (public API, dashboard upload, or internal). **System-initiated analyses are never billed** — only `api`/`console` analyses reserve and settle credits. --- ## Count queries — the contract A count query ("how many X…", "more than N X?") succeeds like this: ```json doc-test id=count-contract-success { "answer": "yes", "confidence": 0.94, "detected_count": 3, "clips": [{ "start_s": 1.2, "end_s": 3.4, "confidence": 0.94, "terms": { "person": 0.94 } }], "term_confidences": { "person": 0.94 }, "query_type": "object", "video_duration_s": 6.0, "indeterminate_reason": null } ``` And fails like this: ```json doc-test id=count-contract-failure { "answer": "indeterminate", "confidence": 0, "detected_count": 0, "clips": [], "term_confidences": {}, "query_type": "object", "video_duration_s": 6.0, "indeterminate_reason": "nothing_detected" } ``` **Rules:** - `detected_count` is only meaningful when `answer` is determinate (`yes` or `no`). - `detected_count: 0` with `answer: "indeterminate"` means the pipeline found **nothing assessable** — NOT "zero occurrences". - A true "zero occurrences" result is `answer: "no"`, `detected_count: 0`. - **Never branch on `detected_count` without checking `answer` first.** **Confidence semantics for count queries:** for count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected. `{"confidence": 0.94, "detected_count": 3}` asserts "there are 3" at 94% confidence, not "there is at least one". Numerically it is derived from the per-term detection confidences underlying the count (max across detected clips, clamped to [0, 1]). When `answer` is `indeterminate`, `confidence` is always 0 (enforced by the API regardless of what inference returned); `indeterminate_reason: "nothing_detected"` additionally forces `detected_count` to 0 because no objects were seen at any confidence level. Treat `confidence ≥ 0.7` as reliable for production decisions, `0.4–0.69` as marginal (verify with a second query or tighter prompt), and `< 0.4` as unreliable. Example: `{"answer": "yes", "confidence": 0.94, "detected_count": 3}` means the model is 94% confident the count is exactly 3 — act on it; `{"answer": "indeterminate", "confidence": 0, "detected_count": 0, "indeterminate_reason": "nothing_detected"}` means no objects were detected at any confidence — do not infer "zero occurrences". --- ## Query-Type Maturity | Query type | Maturity | Notes | |---|---|---| | **presence** (`object`) | GA | "Is there X in this video?" — most reliable query form. | | **counting** (`object` + count intent) | GA | "How many X?" — returns `result.detected_count`. | | **action** / **temporal** | Beta | "Does X happen during the first N seconds?" — duration-sensitive; result reliability depends on accurate video duration metadata. High `duration_mismatch` rate if source duration is missing. | | **open-ended** | Rejected | Freeform prompts that don't resolve to a closed yes/no or count question are rejected immediately with `answer: "indeterminate"`, `indeterminate_reason: "unsupported_query_form"`, at **zero cost** — no credits billed. Rephrase as a presence or count query. | **Agent guidance:** - Prefer presence and counting queries for production pipelines. - Action/temporal queries require the source video to have accurate `duration_s` metadata (populated after `POST /v1/videos/:id/complete`). - If you receive `indeterminate_reason: "unsupported_query_form"`, rewrite the prompt before retrying — retrying the same open-ended form will always fail. --- ## Action pipeline verification A ground-truth walking fixture is available for integration testing. Use it to verify the full pipeline (upload → analyze → result) against a known answer before deploying to production. ### Walking fixture | Field | Value | |-------|-------| | **URL** | *(URL pending — clip `IMG_3281.MOV` not yet uploaded to CDN; see Slack thread for ETA)* | | **CDN key** | `fixtures/walking-ground-truth.mov` on `d3silto12vjvss.cloudfront.net` (pending upload) | | **Scene** | Outdoor walkway; several people walking. Clip is ~6–10 seconds. | | **Ground truth** | At least 2 people visibly walking during the clip. | | **Expected ideal API answer** | `POST /v1/analyses` with prompt `"Is someone walking?"` → `answer: "yes"`, `confidence ≥ 0.8`; with prompt `"How many people are walking?"` → `detected_count ≥ 2`, `confidence ≥ 0.7`. | > **Note:** this fixture is for action-query validation. For basic presence testing, use the sandbox fixture (`POST /v1/sandbox` → `fixture_video_id`). --- ## Batch analyses & discounts Run 2–10 prompts against the same video in a single request: ```http POST /v1/analyses/batch Authorization: Bearer pv_live_ { "video_id": "video_01J...", "prompts": ["Is there a person?", "How many people are walking?"] } ``` **Pricing rule:** the first prompt bills at full price; each additional prompt bills at **50%**. Credits are reserved at the discounted rate, so the discount is observable directly in the ledger (`GET /v1/credits` will show a smaller `seconds_delta` for analysis 2+). **Worked example:** 6-second video at 1¢/s: - Prompt 1 (full price): 6s × 1¢ = **6¢** - Prompt 2 (50% off): 3s × 1¢ = **3¢** - Prompt 3 (50% off): 3s × 1¢ = **3¢** - Total for 3 prompts: **12¢** (vs 18¢ if billed separately) **Response shape:** ```json { "object": "analysis_batch", "id": "batch_...", "video_id": "video_01J...", "analyses": [ { "id": "analysis_01J...", "status": "queued", ... }, { "id": "analysis_01J...", "status": "queued", ... } ], "pricing": { "full_price_prompts": 1, "discounted_prompts": 1, "discount_pct": 50 } } ``` Each analysis in `analyses[]` can be polled individually via `GET /v1/analyses/{id}`. **Listing / filtering analyses:** `GET /v1/analyses` supports `status`, `video_id`, `model`, `created_after`, `created_before` plus cursor pagination (`limit`, `starting_after`). The `status` filter takes the public vocabulary: `queued | preparing | analyzing | rendering | completed | failed | canceled` (use `status=analyzing` for currently-running analyses; unknown values → 400 `validation_failed`). **Dry-run (`validate_only`):** pass `"validate_only": true` to parse all prompts and get per-prompt cost estimates without reserving credits or creating jobs (HTTP 200): ```json { "object": "analysis_batch_preview", "video_id": "video_01J...", "prompts": [ { "index": 0, "query": {...}, "parse_mode": "heuristic", "assessable": true, "estimated_seconds": 10, "estimated_cost_usd": 0.10, "discount_pct": 0 }, { "index": 1, "query": {...}, "parse_mode": "heuristic", "assessable": true, "estimated_seconds": 5, "estimated_cost_usd": 0.05, "discount_pct": 50 } ], "pricing": { "full_price_prompts": 1, "discounted_prompts": 1, "discount_pct": 50, "estimated_total_seconds": 15, "estimated_total_cost_usd": 0.15 } } ``` Estimates are `null` when the video has no known duration (still processing or URL-sourced before probe completes). > **Note:** sending a `prompts` array to `POST /v1/analyses` returns a validation error with a pointer to this endpoint. --- ## Streaming (real-time video over WebRTC) Streams analyze live video in real time — same prompt semantics and same result contract as file analyses, delivered per-frame over a signaling WebSocket. ### Lifecycle ``` POST /v1/streams → queued|ready → (WS join → offer/answer/ICE) → live → ended ``` 1. `POST /v1/streams {prompt}` (secret key) → returns `signaling.url`, `ice_servers` (STUN + TURN with credentials), `limits`. 2. `POST /v1/client_tokens {scopes: ["streams:signal"], stream_id, ttl_s}` → `pvct_` token for the device. The signaling WS **never** accepts secret keys. 3. Connect `signaling.url?token=pvct_…`, send `join`, receive `ready` (or `queued {position}`), then standard WebRTC offer/answer + **bidirectional trickle ICE** (`ice {candidate}` messages flow both ways — the server trickles late-gathered srflx/relay candidates after its answer; keep consuming them). 4. `live` → `result {frame_num, detections}` per analyzed frame, `metering {elapsed_s, billed_s, session_remaining_s}` every 5s, `warning {remaining_s}` before credit exhaustion, `end {reason}`. ### Metering tick fields Every 5s while live: `{type: "metering", elapsed_s, billed_s, session_remaining_s, balance_s}`. - `session_remaining_s` — seconds remaining in **this session's** credit reservation (session cap), NOT your account credit balance. Account balance lives at `GET /v1/billing/credits` (`balance_seconds`). - `balance_s` — **deprecated** alias of `session_remaining_s` (identical value). Removed **~2026-08-28**; migrate reads to `session_remaining_s`. - `elapsed_s` / `billed_s` — live-clock seconds elapsed = billed (identical by construction; join/negotiation free). ### Result sampling (results are sampled, not per-frame) Results are **sampled**, not emitted for every source frame — inference cadence is adaptive (roughly every 8th source frame under load). `frame_num` is the **source-frame index** the result was computed on (so gaps between consecutive `frame_num` values are normal), and `results_summary.result_frames` on the terminal resource counts the number of **result events emitted** — the two are different axes. Expect results at roughly 8–15/s depending on load; do not assume a fixed cadence. ### No-media warning (the server tells you when YOUR media is the problem) If the WebRTC transport connects but **no decodable video frame arrives within 5 seconds**, the server pushes a `warning` event on the signaling WS (re-warned once at 15s, then quiet): ```json {"type": "warning", "code": "no_media_frames", "transport_connected": true, "elapsed_s": 5, "packets_received": 312, "frames_decoded": 0, "hint": "…"} ``` Read `packets_received` to self-diagnose: - `packets_received: 0` — your client is **not sending media** (track not attached, muted, or the capture source is dead). - `packets_received > 0` with `frames_decoded: 0` — media is arriving but is **not decodable** (wrong codec — must be VP8 or H.264 — or the source produces no real frames, e.g. a canvas capture without an active draw loop). If the session then ends without ever going live, the terminal resource records `end_reason: "media_timeout"` with the same counters in `failure_diagnostic`, and bills 0. ### Terminal results_summary The ended stream resource carries `results_summary: {result_frames, frames, last_detections}`: - `result_frames` — number of result events emitted over the session (sampled — see “Result sampling” above; this is NOT a source-frame count). - `frames` — **deprecated** alias of `result_frames` (identical value). Removed **~2026-08-28**; migrate reads to `result_frames`. - `last_detections` — the final `result.detections[]` rows, same contract as live results. ### Streaming result contract (identical to file analyses — transport never changes enums) Each `result.detections[]` row: - `answer`: `"yes" | "no" | "indeterminate"` — lowercase, same enum as file-API results - `confidence`: 0..1; `term_confidences`: per-term map - `prompt`: echoed **exactly as you submitted it** (byte-identical) - `query_type`, `search_terms`, `prompt_intent`: same vocabulary as `POST /v1/parse` - `frame_num`, `elapsed_s`, `session_id`, `session_fps` - `timing`: server-side latency telemetry (per-stage breakdowns: inference, encode, per-hop p50s). Rich, production-grade, safe to log — field names may grow, existing names are stable. ### end_reason vocabulary (honest by construction) | Reason | Meaning | Billed? | |---|---|---| | `completed` | Normal end after the session was live | live seconds | | `canceled` | Ended before ever going live (client action) | 0 | | `ice_failed` | WebRTC ICE never connected — see `failure_diagnostic` | 0 | | `media_timeout` | Transport connected but no media/results flowed | 0 | | `insufficient_credits` | Balance exhausted mid-stream | live seconds | | `timeout` | `limits.max_session_s` cutoff reached | live seconds | | `error` | Server-side failure | live seconds (0 if never live) | **A stream that never went live is never `completed`** — the API enforces this. On `ice_failed`, the terminal resource carries `failure_diagnostic` (`{local_candidates, hint}`) — the server-side ICE candidate summary. If `local_candidates` shows only private addresses (`172.x`, `10.x`), the fault is server-side config; if it shows srflx/relay candidates, check your client's network path to the TURN servers in `ice_servers`. ### Client profiles that MUST work (and are CI-tested) - Browser on home/office NAT (webcam) — the easy case - **Datacenter/CI/edge-fleet clients: UDP-blocked, TURN-over-TCP:443 relay-only** — the demanding case. The server advertises a public host candidate and srflx/relay candidates (trickled when gathering outlasts the answer). Regression-tested every deploy with a relay-only aiortc client. ### Stream a file (regression harness recipe) Re-running a known file through the live path is the natural streaming regression test. The examples repo ships `stream_file.py` (aiortc): loops a video file as the WebRTC source, applies your prompt, and audits the full session (candidates, states, results, billing) to JSON. See `python/streaming/` in the examples repo: **https://github.com/Primate-Intelligence/primate-examples** (public — no auth needed) ### Billing Billed per second of **live clock time** (join/queue/negotiation free), from the same credit ledger as uploads. Sessions that never go live bill exactly 0. Metering ticks arrive every 5s; `usage.billed_seconds` on the terminal resource is the reconciled charge. --- ## Pricing Billing is metered in **source-clock video-seconds** — the duration of video analyzed, independent of resolution or fps. Current rates are discoverable (public, no auth): ```http GET /v1/credit-pricing ``` Key fields: | Field | Meaning | |-------|---------| | `price_per_second_cents` | Price per billed video-second, in cents | | `signup_grant_seconds` | Free credit-seconds on signup / free-grant upgrade | | `allowed_purchase_cents` | Preset top-up amounts | **Cost of an analysis** = `usage.billed_seconds × price_per_second_cents`. Worked example: a 6-second analysis at 1¢/second → `6 × 1 = 6` cents = **$0.06**. Always read rates from the endpoint rather than hardcoding — pricing is config-driven and can change without an API version bump. --- ## API versioning Every response — success, error, even 404 — carries an `X-Api-Version` header: ``` X-Api-Version: + ``` Example: `X-Api-Version: 0.1.0+a7c1993`. The value is computed once at boot and is stable for the lifetime of a deploy. **Agent guidance:** if the value changes between two requests in the same session, a deploy happened mid-session. Re-check the [changelog](https://api.primateintelligence.ai/docs/changelog.md) before attributing new behaviour to a bug in your integration. --- ## Rate Limits All public endpoints respond with `X-RateLimit-*` headers. Retry after `Retry-After` on `429`/`503`. Sandbox/upgrade provisioning is additionally IP-rate-limited (3 sandbox provisions / IP / 24h; configurable global daily cap on free-grant upgrades). --- ## Changelog All API behaviour changes are recorded in [`/docs/changelog.md`](https://api.primateintelligence.ai/docs/changelog.md), sorted newest-first. Subscribe via RSS: [`/docs/changelog.xml`](https://api.primateintelligence.ai/docs/changelog.xml). This file and the changelog ship inside the API deploy artifact itself — `https://api.primateintelligence.ai/docs/agents.md` is the canonical copy, and `https://primateintelligence.ai/docs/agents.md` serves the same bytes (the website proxies the API). Spec, quickstart, and changelog therefore update atomically with the code they describe. **Agent guidance:** check the changelog when `X-Api-Version` changes between requests in the same session — a deploy happened mid-session and a new feature or fix may affect your integration. > Canonical, always-current copy: https://api.primateintelligence.ai/docs/agents.md (also served at https://primateintelligence.ai/docs/agents.md — same bytes, proxied). --- # Uploading video # Uploading video There are three paths to get video into Primate Vision. All of them end with a `video` resource in status `ready`, at which point you can create analyses against it. **Formats:** `video/mp4` (H.264) and `video/quicktime`. **Max size:** 2 GiB. **Retention:** source videos are deleted 30 days after upload (see [security model](/docs/guides/security-model)). ## Choosing a path | Path | Use when | |---|---| | **URL ingest** | The video is already at a public https URL (CDN, S3, YouTube-dl output) | | **Presigned upload** | You have the bytes (server file, browser file input) — bytes go straight to S3, never through our API servers | | **SDK `upload()` helper** | You use an SDK and want the presigned dance done for you | ## URL ingest One call. The API fetches the video asynchronously: ```bash doc-test id=uploading-url skip-live curl -s -X POST https://api.primateintelligence.ai/v1/videos \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com/clip.mp4"}' ``` The video starts `processing` and transitions to `ready` (or `failed` with `error.code` = `url_fetch_failed` / `url_forbidden` / `video_unreadable`). Poll `GET /v1/videos/{id}` or subscribe to the `video.ready` webhook. URL rules (SSRF policy): `https` only, port 443, public hosts only, max 3 redirects, 2 GiB cap. Private/internal addresses are rejected with `url_forbidden`. ## Presigned upload Three steps — create, PUT, complete: ```bash # 1. Create — declares filename/type/size, returns a presigned S3 PUT URL curl -s -X POST https://api.primateintelligence.ai/v1/videos \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"filename": "clip.mp4", "content_type": "video/mp4", "size_bytes": 12345678}' # 2. PUT the bytes directly to S3 (the upload.url + upload.headers from step 1) curl -s -X PUT "$UPLOAD_URL" -H "Content-Type: video/mp4" --data-binary @clip.mp4 # 3. Tell the API the PUT finished curl -s -X POST https://api.primateintelligence.ai/v1/videos/$VIDEO_ID/complete \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` The presigned URL expires (see `upload.expires_at`, typically 1h). If the declared `size_bytes` doesn't match what landed in S3, `complete` fails with `upload_incomplete` — re-create the video. ## SDK helper ```typescript import Primate from '@primate-intelligence/sdk'; import { readFile } from 'fs/promises'; const client = new Primate(); const video = await client.videos.upload(await readFile('clip.mp4'), { filename: 'clip.mp4', content_type: 'video/mp4', }); ``` ```python from primate_intelligence import Primate client = Primate() with open("clip.mp4", "rb") as f: video = client.videos.upload(f.read(), filename="clip.mp4") ``` Both run create → PUT → complete and return the video resource. ## Browser uploads Never put a secret key in a browser. Mint an ephemeral [client token](/docs/guides/security-model#client-tokens) server-side with the `videos:write` scope, then run the presigned flow from the browser with the `pvct_` token. Working example: the [browser SPA sample](https://github.com/Primate-Intelligence/primate-examples). ## Lifecycle ``` awaiting_upload → uploading → processing → ready ↘ failed ``` `DELETE /v1/videos/{id}` removes the video (409 `resource_conflict` while analyses are still running against it). Deletion propagates from S3 + CDN within 72h; signed URLs expire within 1h regardless. --- # Prompts & queries # Prompts & queries Every analysis is a **question about a video**. You can ask it two ways: free text (`prompt`) or a pre-compiled structured query (`query`). Provide exactly one. ## What Primate can answer The model supports ten distinct prompt types. The table below shows what to expect from each. | Type | Example prompt | `answer` | `detected_count` | `clips` | |------|---------------|----------|-----------------|---------| | **Presence** | "Is there a person?" | yes / no / indeterminate | — | ✅ when yes | | **Absence** | "Are there no dogs?" | yes / no | — | — | | **Action** | "Is someone walking?" | yes / no | — | ✅ when yes | | **Count — bare** | "How many people are in this video?" | — | ✅ integer | — | | **Count — threshold** | "Are there more than 2 people?" | yes / no | ✅ integer | — | | **Compound** | "Is there a person AND a dog?" | yes / no | — | ✅ when yes | | **Attribute** | "Is anyone wearing a red jacket?" | description | — | — | | **Location** | "Where is the dog?" | description | — | — | | **State** | "Is the door open?" | description | — | — | | **Segment** | "Show me where the person is" | — | — | ✅ with masks | **Open-ended prompts** ("Tell me what you see") return a description but have lower accuracy than targeted queries. Prefer a specific question. **Not supported:** - Audio / sound ("Is there loud noise?") - Identity ("Is this John?") - OCR / text in frame ("What does the sign say?") - Subjective judgment ("Does the room look nice?") - Exhaustive counting over long footage ("Count every bird in this 2-hour archive") — count queries are accurate on a single clip or short segment, not long multi-hour recordings --- ## Free-text prompts ```json { "video_id": "video_…", "prompt": "Is there a person near the door?" } ``` The server compiles your prompt into a structured query (the `query` field on the analysis shows exactly what it understood, `parse_mode` shows how: `llm` or `heuristic`). Max 2000 characters. ### Prompt design tips **Do this:** - "Is there a person near the door?" — presence - "Are there more than two people in frame?" — count + threshold, returns `yes`/`no` + `detected_count` - "How many people are walking?" — bare count, returns `detected_count` + natural-language narrative - "Does anyone fall down?" — action - "Is there a person AND a dog?" — compound - "Is anyone wearing a red jacket?" — attribute - "Is the door open or closed?" — state - "Where is the bicycle relative to the car?" — location **Avoid this:** - "Tell me everything that happens in the video" — open-ended; use a targeted question instead - "Count every pedestrian across this 30-minute file" — count works best on clips, not long recordings - "Is this suspicious activity?" — subjective; rephrase as a specific visual condition **One question per analysis beats a paragraph.** If you need to answer "Is there a person?" AND "Are there more than 2 people?", submit two analyses — the answers are independent and the second one gives you `detected_count`. --- ## The answer contract ### `result.answer` `result.answer` is one of `yes`, `no`, `indeterminate`: | Value | Meaning | |-------|---------| | `yes` | The condition is present with meaningful confidence | | `no` | The condition was not detected | | `indeterminate` | The model could not commit either way — treat as "don't ship a decision on this" | Count queries (bare "how many?" form) do not populate `result.answer`. Use `result.detected_count` instead. ### `result.detected_count` Populated for `count`-intent analyses (when the prompt asks "how many?" or uses a threshold): ```json { "result": { "answer": "yes", "confidence": 0.94, "detected_count": 3, "clips": [...] } } ``` For count + threshold queries, `answer` tells you whether the threshold was met; `detected_count` tells you the actual number observed. ### `result.confidence` `result.confidence` ∈ [0, 1]. Meaningful confidence is ≥ 0.5. A result with `confidence: 0` and no positive term detections returns `answer: "indeterminate"`, not `"no"`. ### `result.clips[]` Clip objects carry `start_s`, `end_s`, `confidence`. Present for presence, action, compound, and segment queries when the answer is `yes`. Null or absent for description/count responses. ### `result.query_type` How the question was classified: `object`, `action`, `compound`, `attribute`, or `open_ended`. --- ## Unassessable components If part of your question can't be assessed, the API **answers what it can and tells you what it couldn't** — it never silently drops half your question: ```json { "prompt": "Is there a person talking loudly?", "query": { "search_terms": ["person"], "unassessable_components": ["talking loudly (audio)"] }, "result": { "answer": "yes", "confidence": 0.91 } } ``` Check `query.unassessable_components` when your prompts mix visual and non-visual language. --- ## Structured queries Skip the compiler and send the compiled form directly (`parse_mode: "client"`). Useful when you generate queries programmatically or need exact control: ```json { "video_id": "video_…", "query": { "search_terms": ["person", "dog"], "conditions": [ { "type": "presence", "target": "person", "negated": false }, { "type": "presence", "target": "dog", "negated": false } ], "connective": "and", "query_type": "compound", "prompt_intent": "presence", "response_framing": "yes_no" } } ``` Key structured query fields: | Field | Values | Description | |-------|--------|-------------| | `query_type` | `object` `action` `compound` `open_ended` | Top-level classification | | `prompt_intent` | `presence` `absence` `count` `action` `segment` | What the model optimizes for | | `response_framing` | `yes_no` `description` `count` | Shape of the result | | `count_terms` | string[] | What to count (for count intent) | | `count_threshold` | integer \| null | Threshold for threshold comparison | | `count_comparison` | `gt` `gte` `lt` `lte` `eq` \| null | How to compare against threshold | | `quantifier` | `{operator, value}` \| null | Alternative to count_threshold | If the body fails the CompiledQuery schema you get `400 query_invalid` with `param` naming the field. If the compiler can't parse a free-text prompt you get `422 parse_failed` — rephrase, or fall back to a structured query. --- ## Prompt errors | Code | Meaning | |------|---------| | `prompt_empty` | Provide `prompt` or `query` | | `prompt_too_long` | > 2000 chars | | `query_invalid` | Structured query fails the schema (`param` names the field) | | `parse_failed` | Compiler couldn't produce a query — retryable after rephrasing | --- # Waiting for results # Waiting for results Analyses run asynchronously. There are three ways to learn when one finishes; pick by context. | Strategy | Use when | |---|---| | **`Prefer: wait`** | Interactive flows, scripts, first success experience — simplest | | **Polling** | You want progress bars / queue position, or waits longer than 120s | | **Webhooks** | Production pipelines, high volume, serverless — no held connections | ## `Prefer: wait` (sync sugar) Add one header to the create and the API holds the connection until the analysis reaches a terminal state (or the wait cap): ```bash doc-test id=waiting-prefer env=VIDEO_ID curl -s -X POST https://api.primateintelligence.ai/v1/analyses \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -H "Prefer: wait=60" \ -d '{"video_id": "'"$VIDEO_ID"'", "prompt": "Is there a person in this video?"}' ``` - Cap: **120 seconds**. If the analysis isn't terminal by then you get the current (running) resource back — fall through to polling. - Test-mode analyses complete in seconds, so `Prefer: wait` virtually always returns the finished result. - Under capacity brownouts `Prefer: wait` may be temporarily disabled (you get an immediate 202-style response with the running resource) — code must handle a non-terminal response either way. The SDK helpers `createAndWait()` (TS) / `create_and_wait()` (Python) use this header, then fall back to polling automatically. ## Polling ```bash curl -s https://api.primateintelligence.ai/v1/analyses/$ANALYSIS_ID \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` While running, the resource carries honest progress signals: ```json { "status": "queued", "queue_position": 3, "progress": null } ``` `queue_position` and `estimated_start_s` are accurate to within ±50% — we publish transparency rather than an analysis-latency SLO. **Backoff:** poll at 1–2s intervals for interactive flows; add jitter for fleets. Terminal states are `completed`, `failed`, `canceled` — anything else keeps polling. ## Webhooks Register once, get an HTTPS POST on every terminal transition: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://yourapp.com/webhooks/primate", "events": ["analysis.completed", "analysis.failed"]}' ``` See the [webhooks guide](/docs/guides/webhooks) for verification, retries, and the #1 integration rule (the webhook is a notification, not the source of truth). ## Cancellation ```bash curl -s -X POST https://api.primateintelligence.ai/v1/analyses/$ANALYSIS_ID/cancel \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` Best-effort once the analysis is `analyzing`; a 409 `analysis_not_cancelable` means it already reached a terminal state. You are not billed for canceled work that never ran. --- # Webhooks # Webhooks Webhook endpoints receive an HTTPS POST whenever a subscribed event fires. Event vocabulary (closed, v1): `video.ready`, `video.failed`, `analysis.completed`, `analysis.failed`, `analysis.canceled`, `stream.ended`. ## Rule #1: the webhook is a notification, not the source of truth Payloads over 256 KiB are truncated (`data.truncated: true`). Delivery is **at-least-once with no ordering guarantee**. The reliable pattern is always: 1. Receive the event → verify the signature → dedupe on the `webhook-id` header 2. `GET` the resource by id for the authoritative state 3. Act on that Never assume `analysis.completed` arrives before (or after) your poll sees `completed`. ## Register an endpoint ```bash curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://yourapp.com/webhooks/primate", "events": ["analysis.completed", "analysis.failed"]}' ``` The response contains the signing secret (`whsec_…`) **exactly once**. Store it. Max 10 endpoints per account. ## Delivery contract - Body: `{"id": "evt_…", "type": "analysis.completed", "created_at": "…", "data": {…full resource DTO…}}` - Timeout **10s**; only 2xx counts as delivered (3xx = failure) - Retry schedule: `1m, 5m, 30m, 2h, 8h, 24h` with jitter; then the event dead-letters (visible in the deliveries API for 30 days) - 72h of 100% failure → the endpoint auto-disables + you get an email. Re-enable with `POST /v1/webhook_endpoints/{id}/enable` ## Verify signatures Deliveries are signed per [Standard Webhooks](https://www.standardwebhooks.com). Three headers: `webhook-id`, `webhook-timestamp`, `webhook-signature` (`v1,`; multiple space-separated during rotation). Signed content is `` `${id}.${timestamp}.${rawBody}` ``; the HMAC key is the base64-decoded part of your `whsec_` secret. Reject skew > 5 minutes; compare constant-time. Both SDKs ship verifiers: ```typescript import { verifyWebhookRequest } from '@primate-intelligence/sdk'; app.post('/webhooks/primate', (req, reply) => { verifyWebhookRequest({ secret: process.env.PRIMATE_WEBHOOK_SECRET!, headers: req.headers, rawBody: req.rawBody, // MUST be the raw bytes, not re-serialized JSON }); // throws on failure const event = JSON.parse(req.rawBody); if (alreadyProcessed(event.id)) return reply.code(204).send(); // dedupe on evt_ id // … fetch the resource, act … reply.code(204).send(); }); ``` ```python from primate_intelligence import verify_webhook_request @app.post("/webhooks/primate") async def receive(request: Request): raw = await request.body() verify_webhook_request( secret=os.environ["PRIMATE_WEBHOOK_SECRET"], headers=dict(request.headers), raw_body=raw, # raw bytes, not re-serialized JSON ) # raises ValueError on failure event = json.loads(raw) ... return Response(status_code=204) ``` > Per-request `webhook` overrides on `POST /v1/analyses` / `POST /v1/streams` have no secret exchange and are delivered **unsigned** — register an endpoint when you need verification. ## Rotate secrets ```bash curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/rotate_secret \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` Both old and new secrets sign deliveries for **24h** (the signature header carries two entries); verification passes when any signature matches *your* secret. Full key-compromise procedure: [security model](/docs/guides/security-model). ## Debug deliveries ```bash # last 100 deliveries with response codes curl -s https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/deliveries \ -H "Authorization: Bearer $PRIMATE_API_KEY" # redeliver a failed/dead one now curl -s -X POST https://api.primateintelligence.ai/v1/webhook_endpoints/$ID/deliveries/$DELIVERY_ID/redeliver \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` ## Local development Use a tunnel (ngrok, cloudflared) so the API can reach your dev machine, or skip webhooks locally and use `Prefer: wait` — the two strategies are drop-in replacements for the completion signal. --- # Errors & retries # Errors & retries Every non-2xx response uses one envelope: ```json { "error": { "code": "insufficient_credits", "message": "Insufficient credits. Add credits or enable auto-refill.", "status": 402, "param": null, "docs_url": "https://primateintelligence.ai/docs/errors#insufficient_credits", "request_id": "req_01H…" } } ``` - **`code`** — stable, machine-readable, from a closed registry. Codes are append-only; meanings never change. - **`docs_url`** — deep link to the [error registry page](/docs/errors) anchor for that code. - **`request_id`** — include it in support requests; it indexes our logs. - **`error.errors[]`** — on `validation_failed`, every violation is listed (param + message), not just the first. - **`details`** — structured extras (e.g. `details.meter` on `quota_exceeded`). ## The registry is machine-readable ```bash doc-test id=errors-registry curl -s https://api.primateintelligence.ai/v1/errors ``` Each entry carries two retry flags: - **`retryable`** — safe to retry the same request unchanged (with backoff) - **`idem`** — idempotent to retry with the same `Idempotency-Key` The SDKs generate their retry policy from this table — if you build your own client, do the same and your retry logic can never drift from the API. ## What to retry | Situation | Codes | Strategy | |---|---|---| | Rate limited | `rate_limit_exceeded`, `concurrency_limit_exceeded` | Wait `Retry-After` seconds, retry | | Platform transient | `inference_unavailable`, `db_unavailable`, `internal_error`, `capacity_exhausted`, `idempotency_unavailable` | Exponential backoff + jitter, same `Idempotency-Key` | | URL fetch flake | `url_fetch_failed` | Retry the create; check the source URL is reachable | | Prompt didn't compile | `parse_failed` (422) | Rephrase the prompt or send a structured `query` | | Everything else | `validation_failed`, `invalid_api_key`, `insufficient_credits`, … | **Do not retry unchanged** — fix the cause | ## Idempotency (never double-create) Send `Idempotency-Key` (any string, 1–255 chars; UUIDv4 recommended) on `POST /v1/videos`, `/v1/analyses`, `/v1/webhook_endpoints`: - Same key + same body within 24h → the stored response replays with `Idempotency-Replayed: true` - Same key + **different** body → `409 idempotency_key_reused` - Body comparison is canonical (JCS) — key order / whitespace / number formatting never false-positive The SDKs auto-generate keys on create calls, which makes network-level retries safe by default. ## Resource-level errors Two codes never appear as HTTP errors — only inside a failed resource's `error` field: - **`inference_error`** — the model failed on this input; safe to create a new analysis - **`stuck_timeout`** — the platform lost the job; not billed; resubmit Check `analysis.error` / `video.error` when `status` is `failed`. ## A worked retry loop (Python, no SDK) ```python import time, requests def create_analysis(body, key, attempts=4): idem = str(uuid.uuid4()) for attempt in range(attempts): r = requests.post( "https://api.primateintelligence.ai/v1/analyses", json=body, headers={"Authorization": f"Bearer {key}", "Idempotency-Key": idem}, ) if r.ok: return r.json() err = r.json()["error"] registry = requests.get("https://api.primateintelligence.ai/v1/errors").json() flags = next(e for e in registry["data"] if e["code"] == err["code"]) if not flags["retryable"] or attempt == attempts - 1: raise RuntimeError(f"{err['code']}: {err['message']} ({err['request_id']})") time.sleep(int(r.headers.get("Retry-After", 2 ** attempt))) ``` (Or just use the SDK — this is exactly what it does, with the registry vendored at build time.) ## See also - [Error registry](/docs/errors) — every code, one anchor each (the `docs_url` targets) - [Rate limits & quotas](/docs/guides/rate-limits) - [Billing & credits](/docs/guides/billing) — handling `insufficient_credits` gracefully --- # Rate limits & quotas # Rate limits & quotas ## Rate limits Applied per API key, sliding window: | Class | Limit | Covers | |---|---|---| | `reads` | 600/min | All GETs | | `writes` | 300/min | POST/DELETE except analysis creation | | `analyses.create` | 60/min | `POST /v1/analyses` | Plan multipliers can raise these. Every response — including 429s — carries: ``` X-RateLimit-Limit: 600 X-RateLimit-Remaining: 597 X-RateLimit-Reset: 1751980860 ``` A 429 adds `Retry-After` (integer seconds, ≥ 1). **Honor it** — the SDKs do automatically. ```json { "error": { "code": "rate_limit_exceeded", "status": 429, "message": "Rate limit exceeded. Back off per Retry-After." } } ``` ## Concurrency limits Separate from request rates: your plan caps **simultaneous running analyses**. Exceeding it returns `429 concurrency_limit_exceeded` — wait for an active analysis to finish rather than backing off blindly. ## Quotas (metered limits) Quotas are about *how much* you process, not how fast you call. Exceeding a metered limit returns `429 quota_exceeded` with `details.meter` naming the meter. Check consumption any time: ```bash doc-test id=rate-limits-usage curl -s https://api.primateintelligence.ai/v1/usage \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` ```json { "meters": [ { "meter": "credit_seconds", "unit": "seconds", "balance": 5990 }, { "meter": "seconds_processed.period", "unit": "seconds", "used": 10, "resets_at": "2026-08-01T00:00:00Z" } ] } ``` Running out of credits is `402 insufficient_credits` (not 429) — see [billing & credits](/docs/guides/billing). ## Capacity Under platform-wide pressure new creates may get `503 capacity_exhausted` with `Retry-After` while in-flight work drains. Test-mode traffic is never shed (it doesn't touch the GPU). See [service expectations](/docs/guides/versioning#service-expectations). ## Client checklist 1. Honor `Retry-After` on every 429/503 — never hammer 2. Watch `X-RateLimit-Remaining` and pre-throttle when it gets low 3. Treat `concurrency_limit_exceeded` as "wait for completion", not "back off" 4. Treat `quota_exceeded`/`insufficient_credits` as account states — alert a human, don't retry 5. Single region (us-west-2) in v1 — no cross-region limit ambiguity --- # Test mode # Test mode Every account has two key modes. They hit the same API surface with the same code — only the backend differs. | | `pv_test_` | `pv_live_` | |---|---|---| | Getting one | `POST /v1/sandbox` — instant, anonymous | Signup + card | | Results | Deterministic, canned, fixture corpus | Real GPU inference | | Credits | Can never hold or spend credits | Metered | | Speed | Analyses complete in seconds | Workload-shaped | | Expiry | 7 days (sandbox) | Until revoked | ## Instant sandbox ```bash doc-test id=test-mode-sandbox curl -s -X POST https://api.primateintelligence.ai/v1/sandbox ``` No email, no card, no human. The response includes the key, a pre-seeded fixture video id, the fixture prompt, and the expected answer. IP-limited (default 3 provisions per 24h → `429 sandbox_limit_exceeded`). This is the zero-touch path for agents and CI: build and verify the *entire* integration before any human signs up. Graduating to live is a **billing gate, not an integration gate** — your code doesn't change. ## Deterministic fixtures On test keys, analyses against the fixture corpus return canned results — same input, same output, every time. That makes them assertable in CI: ```bash doc-test id=test-mode-fixture curl -s https://api.primateintelligence.ai/v1/test-fixture ``` ```json { "name": "presence", "test_video_url": "https://primateintelligence.ai/empty-state/forklift-demo.mp4", "test_prompt": "Is there a forklift in this video?", "expected_answer": "yes", "expected_confidence_min": 0.8 } ``` ## CI verification pattern ```python # conftest-friendly integration check (runs in seconds, costs nothing) from primate_intelligence import Primate def test_primate_integration(): client = Primate() # PRIMATE_API_KEY = a pv_test_ key in CI secrets videos = client.videos.list() analysis = client.analyses.create_and_wait( video_id=videos["data"][0]["id"], prompt="Is there a person in this video?", timeout_s=60, ) assert analysis["result"]["answer"] == "yes" assert analysis["result"]["confidence"] >= 0.8 ``` Every code sample in these docs runs in CI against this exact mechanism — docs that don't run don't ship. ## Limits of test mode - Fixture corpus only — you can't upload arbitrary videos for real analysis on a test key (uploads work mechanically, but analysis results are canned) - No live streaming GPU sessions - `403 test_mode_only` marks operations unavailable in your key's mode When you need real inference on your own footage: [sign up](https://primateintelligence.ai/signup), add a card, create a `pv_live_` key, change the env var. Done. --- # Versioning & deprecation # Versioning & deprecation ## Compatibility contract The v1 API is **additive-only**. Within v1 we may: - Add new endpoints, optional request fields, response fields, enum values on *new* fields, and error codes (the registry is append-only) We will never (without a major version): - Remove or rename fields/endpoints, change field types or meanings, change error-code semantics, or tighten validation on existing fields Write clients that **ignore unknown fields** — that's the only client-side requirement for staying compatible. The gate is mechanical: every spec change runs `oasdiff breaking` in CI; a breaking diff fails the build unless it's an explicitly approved major-version migration. ## Deprecation process When something must go: 1. It's marked `deprecated` in the OpenAPI spec + the [changelog](/docs/changelog) (typed entry: `breaking | additive | fix | deprecation`) 2. A **migration guide** ships with before/after code in curl/TS/Python, the deadline, and an automated check command 3. Sunset headers appear on responses where applicable 4. Breaking removals only ever land in a new major version ## Legacy `pk_` key sunset API keys created before v1 used the `pk_` prefix. They remain valid during the launch migration so existing integrations do not break. New keys and rotations now create `pv_live_` / `pv_test_` keys only. The `pk_` compatibility window is **12 months from GA announcement**. The exact sunset date starts when Matt approves the production GA cutover and will be published here, in the changelog, and in dashboard key-management copy. Before that date, rotate each legacy key from the dashboard or `POST /v1/api-keys/{id}/rotate`; the replacement key is drop-in for `Authorization: Bearer`. After the sunset, `pk_` keys return `401 invalid_api_key`. No paid usage is charged by the act of rotating a key. ## Model versioning Models are versioned resources (`GET /v1/models`) with lifecycle: `preview → stable → deprecated`, and a `sunset_at` date once deprecated. - Omitting `model` on create uses the current default — fine for exploration - **Production integrations should pin a model id** (e.g. `"model": "darwin-1.3"`) so behavior changes only when you choose - `400 model_deprecated` past sunset tells you to re-pin; the changelog carries the migration entry ## SDK versioning SDKs follow semver. An SDK major bump happens only with an API major. Minor/patch releases are additive (new endpoints, fixes) and safe to auto-update within `^`. ## Service expectations Published targets (not a contractual SLA in v1): - **Availability:** 99.9%/mo control plane; ≥ 99% completion success for admitted analyses - **Latency:** p95 reads < 300ms; creates < 1.5s (excluding upload bytes); webhook dispatch p95 < 60s after terminal transition; test-mode analyses < 10s - **Analysis latency is workload-shaped** — we publish honest transparency instead: `queue_position` + `estimated_start_s` accurate to ±50% - Single region: us-west-2 (US). All processing and storage. EU is roadmap, not promise. ## Changelog & RSS Every change lands in the [changelog](/docs/changelog) with a date and type tag. Subscribe: [RSS feed](/docs/changelog.xml). Agents: the changelog is also in `llms-full.txt`. --- # Security model # Security model ## API keys - Two prefixes: `pv_live_` (real processing, metered) and `pv_test_` ([test mode](/docs/guides/test-mode)). 256-bit CSPRNG entropy. - Shown **exactly once** at creation/rotation. We store only a SHA-256 digest. - Send via `Authorization: Bearer …` header only. Keys never belong in URLs, logs, or browser code. - Env-var contract: `PRIMATE_API_KEY` — both SDKs and the MCP server read it. The MCP server never accepts keys as tool arguments (keys must not land in agent transcripts). - Revocation propagates in ≤ 5 seconds. ## Client tokens (browser auth) Secret keys must never reach a browser. For client-side flows, your server mints an **ephemeral client token**: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/client_tokens \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"scopes": ["videos:write", "analyses:read"], "ttl_s": 300}' ``` - Prefix `pvct_`, TTL 60–900s (default 900), **never refreshable** — mint a new one when it expires (`401 token_expired`) - Scopes (v1): `videos:write`, `analyses:read`, `analyses:write`, `streams:create`, `streams:signal` — a token's scopes must be a subset of the minting key's - Optionally bind a token to one `video_id` or `stream_id` — the token then works only for that resource - The streaming signaling WebSocket accepts **only** client tokens (or first-party session JWTs) — never secret keys Pattern: a ≤ 20-line "token mint" route on your server, browser does the rest. Working example: the token-mint server sample in the [examples](https://github.com/Primate-Intelligence/primate-examples). ## Data handling **Prompts and result metadata are retained to operate and improve the service; customer video content is never used to train models.** The full contract: | Data | Retention | |---|---| | Source videos | Deleted 30 days after upload; `DELETE /v1/videos/{id}` propagates from S3 + CDN within 72h (signed URLs expire ≤ 1h) | | Result artifacts (annotated videos) | 30 days, same deletion path | | Result JSON + analysis metadata | 13 months (billing/audit), then aggregated | | Prompts | Retained per the sentence above; org-level opt-out available on request | | Account deletion | Cascades videos/analyses/keys within 30 days; webhook endpoints immediately | - Encryption: TLS ≥ 1.2 in transit; AES-256 at rest - No customer video bytes in logs or analytics — enforced by a redaction test in CI - Region: all processing/storage in us-west-2 (US) - Audit log: key lifecycle + webhook changes retained 13 months ## URL ingest (SSRF policy) `POST /v1/videos {url}` fetches are locked down: https only, port 443, public addresses only (RFC-1918/link-local/metadata ranges rejected), resolved IPs pinned (no TOCTOU), max 3 redirects each re-validated, 2 GiB streamed cap, content validated by magic bytes. Policy violations return `url_forbidden`; network failures `url_fetch_failed`. ## If a key is compromised 1. **Rotate immediately** — create a new key, revoke the old one in the [dashboard](https://primateintelligence.ai/dashboard/api-keys) (propagation ≤ 5s) 2. **Audit** — `GET /v1/analyses?created_after=…` for activity you don't recognize 3. **Contact support** with request ids for usage-forgiveness on abusive spend We monitor per-key spend anomalies (> 5× 7-day baseline), page on them, and may auto-suspend a key pending confirmation — with an email + dashboard banner, never silently. ## Webhook signing Deliveries are signed (Standard Webhooks, HMAC-SHA256, 5-minute replay window, 24h dual-secret rotation). Details: [webhooks guide](/docs/guides/webhooks). --- # Billing & credits # Billing & credits Primate Vision bills by **video-seconds processed**, prepaid as credits. 1 credit-second = 1 second of video analyzed. Current pricing is always at `GET /v1/credit-pricing` (public, no auth) and on the [pricing page](/pricing). ## How credits flow - **Signup grant** — new accounts get free credit-seconds to evaluate with (currently 6,000s ≈ 100 minutes; expiring) - **Card grant** — adding a card grants another tranche - **Purchases** — buy credit blocks in the [dashboard](https://primateintelligence.ai/dashboard/billing); custom amounts supported - **Auto-refill** — opt in to top up automatically when the balance crosses the threshold (currently 600s) Test-mode keys never hold or spend credits — build and CI on `pv_test_` for free, forever. ## Check your balance ```bash doc-test id=billing-usage curl -s https://api.primateintelligence.ai/v1/usage \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` ```json { "meters": [ { "meter": "credit_seconds", "unit": "seconds", "balance": 5990 }, { "meter": "seconds_processed.period", "unit": "seconds", "used": 10, "resets_at": "2026-08-01T00:00:00Z" } ] } ``` ## What you're charged for - **Analyses** — the duration of video actually analyzed, debited when the analysis completes. Failed platform-side work (`inference_error`, `stuck_timeout`) is **not billed**; resubmit freely. - **Streams** — live sessions reserve credits up front and reconcile to actual connected seconds at end (`GET /v1/streams/{id}` shows the final usage). Mid-stream you get `warning` messages as the reservation runs low, then the session ends with `end_reason: "insufficient_credits"`. ## Handling `insufficient_credits` in code The one billing error every integration must handle (`402`, **not retryable**): ```typescript doc-test id=billing-handling sdk=ts offline import Primate, { PrimateError } from '@primate-intelligence/sdk'; const client = new Primate(); try { await client.analyses.create({ video_id: videoId, prompt: 'Is the loading dock clear?' }); } catch (err) { if (err instanceof PrimateError && err.code === 'insufficient_credits') { const { meters } = await client.usage.retrieve(); const balance = meters.find((m) => m.meter === 'credit_seconds')?.balance ?? 0; notifyOwner( `Primate Vision balance is ${balance}s — top up or enable auto-refill: ` + `https://primateintelligence.ai/dashboard/billing`, ); // Do NOT retry unchanged — queue the job for after refill instead. } else { throw err; } } ``` The robust pattern: 1. Catch `insufficient_credits` → **stop submitting** (it will keep failing) 2. Read `GET /v1/usage` for the real balance 3. Surface a human-actionable message with the billing URL 4. If the account has auto-refill, retry after a short delay **once** — refill is triggered by the threshold crossing 5. Alert *before* you hit zero: poll `credit_seconds` and warn below your own threshold ## Free-credit expiry Granted (free) credits expire; purchased credits don't. Expiry dates show in the dashboard. Expired grants simply vanish from `balance` — no negative surprises. ## Invoices & history Purchase history and receipts live in the [dashboard billing page](https://primateintelligence.ai/dashboard/billing). Usage aggregates by day/analysis are on the [usage page](https://primateintelligence.ai/dashboard/usage). --- # Streaming # Streaming Streams analyze live video in real time: point a camera (or any MediaStream) at the API and get per-frame detection results back over a WebRTC data channel, with the same prompt semantics as file analyses. ## Lifecycle ``` POST /v1/streams → queued|ready → (WS join → offer/answer/ICE) → live → ended ``` 1. **Create** the stream server-side with your secret key: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/streams \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"prompt": "Is there a person in the frame?"}' ``` The response carries everything the client needs: `signaling.url` (a WebSocket), `ice_servers` (STUN/TURN with credentials), `limits` (`max_session_s`, `warn_at_remaining_s`), and `queue_position`/`estimated_start_s` when capacity is busy. 2. **Mint a client token** for the browser/device (the signaling WS **never** accepts secret keys): ```bash curl -s -X POST https://api.primateintelligence.ai/v1/client_tokens \ -H "Authorization: Bearer $PRIMATE_API_KEY" \ -H "Content-Type: application/json" \ -d '{"scopes": ["streams:signal"], "stream_id": "'"$STREAM_ID"'", "ttl_s": 300}' ``` 3. **Connect** from the client. With the TS SDK this is one call: ```typescript import { connectStream } from '@primate-intelligence/sdk/browser'; const session = await connectStream({ stream, // the Stream resource (relay it from your server) clientToken, // pvct_… minted in step 2 mediaStream: await navigator.mediaDevices.getUserMedia({ video: true }), onResult: (r) => console.log(r.frame_num, r.detections), onWarning: (remainingS) => showCountdown(remainingS), onEnd: (reason) => cleanup(reason), }); // change the question mid-stream: session.updatePrompt('Is there a forklift moving?'); // done: session.end(); ``` Under the hood: WS `join` → `ready` (or `queued {position}`) → WebRTC `offer`/`answer` + ICE trickle → media flows → `result` messages per analyzed frame. ## Signaling protocol (build your own client) Messages you send: `join`, `offer {sdp}`, `ice {candidate}`, `update_prompt {prompt}`, `end`, `ping`. Messages you receive: `queued {position, estimated_start_s}`, `ready`, `answer {sdp}`, `ice {candidate}`, `live`, `result {frame_num, detections}`, `metering {…}`, `warning {…}`, `end {reason}`, `error {code, message}`, `pong`. There are two kinds of `warning`: the credit-exhaustion countdown (`warning {remaining_s}`, no `code` field) and diagnostic warnings that carry a `code` field — see [No-media warning](#no-media-warning-the-server-tells-you-when-your-media-is-the-problem) below. Distinguish them by the presence of `code`. A Python/aiortc backend example (robotics-style, no browser — including a "stream a file" recipe and a relay-only datacenter/CI profile) ships in the public [examples repo](https://github.com/Primate-Intelligence/primate-examples), alongside a self-contained browser webcam example. **Trickle ICE is bidirectional**: after the `answer`, the server may keep sending `ice {candidate}` messages as late srflx/relay candidates finish gathering — keep consuming and adding them. Clients on UDP-blocked networks (datacenters, CI, strict NATs) complete via the TURN servers in `ice_servers`; TURN-over-TCP:443 is supported. ### Result contract (identical to file analyses) Each `result.detections[]` row uses the same enums and casing as the file API: `answer` is lowercase `yes|no|indeterminate`, `confidence` is 0..1, and `prompt` is echoed exactly as you submitted it. Latency telemetry (per-stage breakdowns) lives under a `timing` object. ### Result sampling — results are sampled, not per-frame Results are **sampled**, not emitted for every source frame — inference cadence is adaptive (roughly every 8th source frame under load). Two different axes to keep straight: - `frame_num` on each live `result` is the **source-frame index** the result was computed on. Gaps between consecutive `frame_num` values are normal and expected — don't treat them as dropped results. - `results_summary.result_frames` on the ended stream resource counts the number of **result events emitted** over the session — NOT a source-frame count. Expect roughly **8–15 results per second** depending on load; never assume a fixed cadence. > **Deprecation:** `results_summary.frames` is a deprecated alias of `results_summary.result_frames` (identical value). It will be **removed ~2026-08-28** — migrate reads to `result_frames`. ### Metering ticks While live, a `metering` message arrives every 5 seconds: ```json {"type": "metering", "elapsed_s": 45, "billed_s": 45, "session_remaining_s": 555, "balance_s": 555} ``` - `session_remaining_s` — seconds remaining in **this session's** credit reservation (the session cap). This is NOT your account credit balance — that lives at `GET /v1/billing/credits` (`balance_seconds`). - `balance_s` — **deprecated** alias of `session_remaining_s` (identical value). It will be **removed ~2026-08-28** — migrate reads to `session_remaining_s`. - `elapsed_s` / `billed_s` — live-clock seconds elapsed = billed (identical by construction; join, queueing, and negotiation are free). ### No-media warning (the server tells you when your media is the problem) If the WebRTC transport connects but **no decodable video frame arrives within 5 seconds**, the server pushes a diagnostic `warning` on the signaling WS (re-warned once at 15s, then quiet; canceled by the first decoded frame): ```json {"type": "warning", "code": "no_media_frames", "transport_connected": true, "elapsed_s": 5, "packets_received": 312, "frames_decoded": 0, "hint": "…"} ``` Read `packets_received` to self-diagnose: - `packets_received: 0` — your client is **not sending media** (track not attached, muted, or the capture source is dead). - `packets_received > 0` with `frames_decoded: 0` — media is arriving but is **not decodable** (wrong codec — must be VP8 or H.264 — or the source produces no real frames, e.g. a canvas capture without an active draw loop). If the session then ends without ever going live, the terminal resource records `end_reason: "media_timeout"` with the same media counters in `failure_diagnostic`, and bills 0. ### Honest end reasons `end {reason}` and the terminal resource's `end_reason` distinguish real failures — a session that never went live is never `completed`: | Reason | Meaning | Billed | |---|---|---| | `completed` | Normal end after live | live seconds | | `canceled` | Ended before going live (client action) | 0 | | `ice_failed` | ICE never connected — resource carries `failure_diagnostic` (server candidate summary) | 0 | | `media_timeout` | Transport connected but no media flowed — `failure_diagnostic` carries the media counters | 0 | | `insufficient_credits` | Balance exhausted mid-stream | live seconds | | `timeout` | `limits.max_session_s` reached | live seconds | | `error` | Server-side failure | live seconds (0 if never live) | ### failure_diagnostic — debugging a failed session from the API alone When a session ends without ever going live on a transport failure, the terminal Stream resource carries a nullable `failure_diagnostic` object (and the session bills **0**): - On `ice_failed` it holds the server-side ICE candidate summary: ```json {"end_reason": "ice_failed", "failure_diagnostic": { "local_candidates": ["host 203.0.113.7", "srflx 203.0.113.7", "relay 198.51.100.20"], "hint": "Server advertised public host + srflx/relay candidates; check the client's path to the TURN servers in ice_servers." }} ``` If `local_candidates` shows only private addresses (`172.x`, `10.x`), the fault is on our side — contact support. If it shows srflx/relay candidates, check your client's network path to the TURN servers in `ice_servers` (TURN-over-TCP:443 and `turns:` TLS relays are available for locked-down egress networks). - On `media_timeout` it holds the media counters from the no-media warning (`packets_received`, `frames_decoded`) so you can tell "never sent media" apart from "sent undecodable media" after the fact. `failure_diagnostic` is `null` on successful sessions and on ends unrelated to transport. ## Credits & limits - One active (queued/ready/live) stream per account by default — a second create returns `409 stream_already_active` - Sessions reserve credits up front; metering ticks every 5s (see [Metering ticks](#metering-ticks) for the field contract); you get `warning {remaining_s}` before funds run out, then `end {reason: "insufficient_credits"}` - `limits.max_session_s` is the hard per-session cap from your plan - Your account-wide balance is at `GET /v1/billing/credits` — metering ticks only report the session reservation - After `ended`, `GET /v1/streams/{id}` shows `duration_s`, `usage` (reconciled to the second), and `results_summary` (`result_frames` = count of result events — see [Result sampling](#result-sampling--results-are-sampled-not-per-frame)) ## End a stream From the client: send `end` (or just `session.end()`). Server-side / cleanup: ```bash curl -s -X POST https://api.primateintelligence.ai/v1/streams/$STREAM_ID/end \ -H "Authorization: Bearer $PRIMATE_API_KEY" ``` Idempotent — safe to call on an already-ended stream. `stream.ended` fires as a [webhook](/docs/guides/webhooks) if subscribed. --- # Tutorial: build from scratch # Tutorial: build from scratch We'll build `askvideo` — a Node CLI that takes a video URL and a question, and prints the answer. Zero to working in about 15 minutes, entirely on a free test key. ## Step 0: get a key (30 seconds) ```bash doc-test id=tutorial-sandbox curl -s -X POST https://api.primateintelligence.ai/v1/sandbox ``` ```bash export PRIMATE_API_KEY="pv_test_…" # from the response ``` ## Step 1: project setup ```bash mkdir askvideo && cd askvideo npm init -y ``` ## Step 2: the whole program `askvideo.mjs`: ```javascript const [url, ...promptParts] = process.argv.slice(2); const prompt = promptParts.join(' '); if (!url || !prompt) { console.error('usage: node askvideo.mjs '); process.exit(2); } const apiKey = process.env.PRIMATE_API_KEY; const baseUrl = process.env.PRIMATE_BASE_URL ?? 'https://api.primateintelligence.ai'; if (!apiKey) { console.error('PRIMATE_API_KEY is required'); process.exit(2); } async function api(path, options = {}) { const res = await fetch(`${baseUrl}${path}`, { ...options, headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', ...(options.headers ?? {}), }, }); const body = await res.json(); if (!res.ok) { const err = body.error ?? { code: 'request_failed', message: res.statusText }; console.error(`${err.code}: ${err.message}`); if (err.docs_url) console.error(`docs: ${err.docs_url}`); process.exit(1); } return body; } async function createAndWait(videoId, question) { let analysis = await api('/v1/analyses', { method: 'POST', headers: { Prefer: 'wait=60' }, body: JSON.stringify({ video_id: videoId, prompt: question }), }); for (let i = 0; analysis.status !== 'completed' && i < 30; i += 1) { if (analysis.status === 'failed' || analysis.status === 'canceled') { throw new Error(`analysis_${analysis.status}: ${JSON.stringify(analysis.error)}`); } await new Promise(resolve => setTimeout(resolve, 1000)); analysis = await api(`/v1/analyses/${analysis.id}`); } return analysis; } const video = await api('/v1/videos', { method: 'POST', body: JSON.stringify({ url }), }); console.error(`video ${video.id} (${video.status})`); const analysis = await createAndWait(video.id, prompt); console.log(JSON.stringify(analysis.result, null, 2)); ``` ## Step 3: run it Test keys analyze the fixture corpus deterministically, so use the official fixture: ```bash FIXTURE=$(curl -s https://api.primateintelligence.ai/v1/test-fixture) node askvideo.mjs \ "$(echo $FIXTURE | python3 -c 'import json,sys; print(json.load(sys.stdin)["test_video_url"])')" \ "Is there a person in this video?" ``` ```json { "answer": "yes", "confidence": 0.93, "clips": [{ "start_s": 1, "end_s": 4, "confidence": 0.93 }], "query_type": "object", "video_duration_s": 10 } ``` That's a working integration. Everything below is production hardening. ## Step 4: make it CI-verifiable `test.mjs`: ```javascript const baseUrl = process.env.PRIMATE_BASE_URL ?? 'https://api.primateintelligence.ai'; const headers = { Authorization: `Bearer ${process.env.PRIMATE_API_KEY}`, 'Content-Type': 'application/json', }; const fixture = await fetch(`${baseUrl}/v1/test-fixture`).then(r => r.json()); const video = await fetch(`${baseUrl}/v1/videos`, { method: 'POST', headers, body: JSON.stringify({ url: fixture.test_video_url }), }).then(r => r.json()); const analysis = await fetch(`${baseUrl}/v1/analyses`, { method: 'POST', headers: { ...headers, Prefer: 'wait=60' }, body: JSON.stringify({ video_id: video.id, prompt: fixture.test_prompt }), }).then(r => r.json()); const ok = analysis.result.answer.toLowerCase() === fixture.expected_answer.toLowerCase() && analysis.result.confidence >= fixture.expected_confidence_min; console.log(ok ? 'INTEGRATION OK' : 'INTEGRATION BROKEN'); process.exit(ok ? 0 : 1); ``` Run that in CI with a `pv_test_` key in your secrets — it completes in seconds and costs nothing. ## Step 5: production hardening checklist - **Error handling** — retry only codes marked retryable in the [error registry](/docs/guides/errors-retries); never retry [`insufficient_credits`](/docs/guides/billing) - **Stop polling** — swap `createAndWait` for [webhooks](/docs/guides/webhooks) at volume - **Pin a model** — pass `model: 'darwin-1.3'` so behavior changes only when you choose ([versioning](/docs/guides/versioning)) - **Your own videos** — swap URL ingest for [presigned upload](/docs/guides/uploading) when the bytes live with you - **Go live** — [sign up](https://primateintelligence.ai/signup), create a `pv_live_` key, change the env var. No code changes. ## Where to next - [Sample matrix](https://github.com/Primate-Intelligence/primate-examples) — Python, webhook receivers, browser SPA, streaming, token-mint server - [Agent quickstart](/docs/agents) — the condensed machine-readable version of everything you just did - [API reference](/docs/reference) --- # Error registry Every error code the API can return, with retry semantics. This registry is closed and append-only — codes are never removed or repurposed within v1. Machine-readable at `GET https://api.primateintelligence.ai/v1/errors`. **retryable** = safe to retry the same request unchanged (with backoff). **idem** = idempotent to retry with the same `Idempotency-Key`. ## HTTP errors ### `invalid_api_key` **HTTP 401** · retryable: **no** · idem: **no** API key is invalid. Check the key, env var, and whitespace. ### `key_revoked` **HTTP 401** · retryable: **no** · idem: **no** This API key has been revoked. Rotate or create a new key. ### `key_expired` **HTTP 401** · retryable: **no** · idem: **no** This API key has expired. Create a new key. ### `token_expired` **HTTP 401** · retryable: **no** · idem: **no** Client token has expired. Mint a new token from your server (client tokens are never refreshable). ### `insufficient_scope` **HTTP 403** · retryable: **no** · idem: **no** The key does not carry the scope required for this operation. ### `account_restricted` **HTTP 403** · retryable: **no** · idem: **no** Account is restricted. Complete signup/verification. ### `test_mode_only` **HTTP 403** · retryable: **no** · idem: **no** Operation not available in this key mode. ### `test_key_fixture_only` **HTTP 403** · retryable: **no** · idem: **no** Test keys can only analyze the provided fixture video. Create a live key to analyze your own uploads. ### `validation_failed` **HTTP 400** · retryable: **no** · idem: **no** Request validation failed. All violations are listed in error.errors[]. ### `unsupported_media_type` **HTTP 415** · retryable: **no** · idem: **no** Send video/mp4 or video/quicktime with a correct Content-Type. ### `payload_too_large` **HTTP 413** · retryable: **no** · idem: **no** Payload exceeds the 100MB multipart cap. Use the presigned upload path. ### `resource_not_found` **HTTP 404** · retryable: **no** · idem: **no** No such resource. IDs are account-scoped; check the ID and prefix. ### `resource_conflict` **HTTP 409** · retryable: **no** · idem: **no** Invalid state transition. GET the resource for its current state. ### `idempotency_key_reused` **HTTP 409** · retryable: **no** · idem: **no** Idempotency-Key was reused with a different request body. ### `idempotency_unavailable` **HTTP 409** · retryable: **yes** · idem: **yes** Idempotency store unavailable. Retry with the same key after Retry-After. ### `stream_already_active` **HTTP 409** · retryable: **no** · idem: **no** An active stream already exists on this account. End it (POST /v1/streams/{id}/end) or wait for it to finish. ### `upload_incomplete` **HTTP 400** · retryable: **no** · idem: **no** Upload incomplete or size mismatch. Re-create the video and complete within 24h. ### `video_unreadable` **HTTP 400** · retryable: **no** · idem: **no** Container/codec unsupported. Re-encode to H.264 MP4. ### `video_too_large` **HTTP 400** · retryable: **no** · idem: **no** Video exceeds the 2 GiB limit. ### `video_too_long` **HTTP 400** · retryable: **no** · idem: **no** Video exceeds the plan duration limit. Upgrade or trim. ### `url_fetch_failed` **HTTP 400** · retryable: **yes** · idem: **no** Source URL unreachable or timed out. ### `url_forbidden` **HTTP 400** · retryable: **no** · idem: **no** URL scheme or host blocked by the ingest policy. ### `prompt_empty` **HTTP 400** · retryable: **no** · idem: **no** Provide prompt or query. ### `prompt_too_long` **HTTP 400** · retryable: **no** · idem: **no** Prompt exceeds 2000 characters. ### `query_invalid` **HTTP 400** · retryable: **no** · idem: **no** Body fails the CompiledQuery schema; param names the field. ### `parse_failed` **HTTP 422** · retryable: **yes** · idem: **no** The compiler could not produce a query. Rephrase or send a structured query. ### `model_not_found` **HTTP 400** · retryable: **no** · idem: **no** Unknown model. See GET /v1/models. ### `model_deprecated` **HTTP 400** · retryable: **no** · idem: **no** Model is deprecated. Pin a supported version; see the changelog. ### `analysis_not_cancelable` **HTTP 409** · retryable: **no** · idem: **no** Analysis is already in a terminal state. ### `rate_limit_exceeded` **HTTP 429** · retryable: **yes** · idem: **yes** Rate limit exceeded. Back off per Retry-After. ### `concurrency_limit_exceeded` **HTTP 429** · retryable: **yes** · idem: **yes** Concurrent analysis limit reached. Wait for an active analysis to finish. ### `quota_exceeded` **HTTP 429** · retryable: **no** · idem: **no** Quota exceeded for a metered limit. See details.meter. ### `insufficient_credits` **HTTP 402** · retryable: **no** · idem: **no** Insufficient credits. Add credits or enable auto-refill. ### `grant_exhausted` **HTTP 402** · retryable: **no** · idem: **no** Free grant exhausted. Claim a full account via POST /v1/keys/request to continue. ### `capacity_exhausted` **HTTP 503** · retryable: **yes** · idem: **yes** Platform queue is full. Honor Retry-After. ### `sandbox_limit_exceeded` **HTTP 429** · retryable: **yes** · idem: **yes** Sandbox provisioning limit reached for this IP. Retry after the window resets, or sign up for a live key. ### `upgrade_limit_exceeded` **HTTP 429** · retryable: **no** · idem: **no** Too many upgrade attempts from this IP. Retry after the window resets. ### `device_code_expired` **HTTP 410** · retryable: **no** · idem: **no** Device code has expired or been consumed. Start a new request. ### `inference_unavailable` **HTTP 503** · retryable: **yes** · idem: **yes** Transient inference backend outage. Retry with backoff. ### `upstream_error` **HTTP 502** · retryable: **no** · idem: **no** Inference upstream returned an error. Message is sanitized — no internal paths or stack traces. ### `db_unavailable` **HTTP 503** · retryable: **yes** · idem: **yes** Transient database outage. Retry with backoff. ### `internal_error` **HTTP 500** · retryable: **yes** · idem: **yes** Internal error. Includes request_id; contact support if persistent. ### `webhook_endpoint_limit` **HTTP 400** · retryable: **no** · idem: **no** At most 10 webhook endpoints per account. ## Resource-level errors These never appear as HTTP responses — only in `analysis.error.code` / `video.error.code` on failed resources. ### `inference_error` Model failed on this input. Safe to create a new analysis; persistent failures on one video → support with request_id. ### `stuck_timeout` The platform lost the job past the sweep window. Not billed; resubmit. --- # Primate Intelligence API — Changelog Sorted newest-first. Agent-readable summary of shipped changes, broken by date. Full field contracts: [openapi.json](./openapi.json) · [agents.md](./agents.md) --- ## 2026-07-29 — docs-only — POST /v1/parse documented in openapi.json; GET /v1/analyses status-filter vocabulary documented - **`POST /v1/parse` is now in the OpenAPI document** (PRI-496 P0). The endpoint has been live for months (stateless prompt compiler: `{prompt}` → `{compiled_query, parse_mode: "haiku"|"heuristic"}`; auth optional; 30 req/min per IP; errors `empty_prompt` 400 / `parse_failed` 422 / `parse_unavailable` 503) but was undocumented — for agents, undocumented is nonexistent. No behavior change. Also added to the `/llms.txt` key-endpoints list. - **`GET /v1/analyses` `status` filter vocabulary documented**: `queued | preparing | analyzing | rendering | completed | failed | canceled` (§13.2). The filter has always worked (unknown values → 400 `validation_failed`); its accepted values were just not written down. Documented in the parameter description — the machine schema stays `type: string`, so nothing tightens contractually. Use `status=analyzing` for currently-running analyses. - Zero contract changes — the public surface is frozen; this is documentation of already-shipped behavior. --- ## 2026-07-28 — additive — SDK 0.2.0 (TS + Python) and MCP 0.2.0: full parity with the July API rounds - **TypeScript SDK `@primate-intelligence/sdk@0.2.0`** (breaking — type corrections): `AnalysisUsage` fixed to the real contract `{billed_seconds, credit_balance_after}`; `StreamEndReason` fixed (adds `canceled`/`ice_failed`/`media_timeout`, drops never-returned `client_disconnect`); `video_duration_s` nullable; `Analysis` gains `livemode` + `origin`. New: `client.credits.retrieve()` (`GET /v1/credits`), `client.creditPricing.retrieve()`, `analyses.validateOnly()`, `analyses.createBatch()`/`validateBatch()`, `detected_count` + `indeterminate_reason` on results, typed streaming contract in `/browser` (`session_remaining_s` metering, `no_media_frames` warning union, typed detections, `failure_diagnostic`, typed `results_summary`). Migrate off the `balance_s`/`results_summary.frames` aliases before their ~2026-08-28 removal. - **Python SDK `primate-intelligence==0.2.0`** (additive): `client.credits` + `client.credit_pricing` resources, `analyses.validate_only()`, `analyses.create_batch()`/`validate_batch()`, docstrings updated to the real result/streaming contracts. - **MCP 0.2.0 (hosted `/mcp` + npm `@primate-intelligence/mcp`)**: three new tools — `validate_analysis` (free dry-run with cost estimate), `create_analysis_batch` (batch discount), `get_credits` (transaction ledger). `get_analysis`/`wait_for_analysis` descriptions now document `detected_count`, `indeterminate_reason`, nullable duration, and the usage snapshot. The npm package catches up with the hosted server: every tool declares `outputSchema` and returns `structuredContent`; `wait_for_analysis` returns the `{analysis, retry}` envelope (the legacy `_mcp_note` field merged into the analysis resource is gone from the npm package too). --- ## 2026-07-28 — failure_diagnostic hint wording: customer-appropriate, no internal references - **`failure_diagnostic.hint` (on `end_reason: "ice_failed"`) reworded.** The hint previously referenced an internal server environment variable — meaningless (and confusing) to API consumers. It now states plainly which side the fault is likely on and what to do: when the server did not advertise a publicly reachable network candidate → *“server-side configuration issue — nothing to change on your end; retry, and contact support if it persists”*; when the server did advertise public candidates → *“check your network allows UDP or TCP/443 outbound and that your client uses the TURN servers in `ice_servers`”*. Field shape unchanged (`{local_candidates, hint}`); only the hint text changed. Same sweep reworded the transient status messages sent on the signaling WS (`inference_unavailable`, capacity/restart rejections) to remove internal jargon. --- ## 2026-07-28 — Metering tick + results_summary field renames (deprecation aliases until ~2026-08-28) - **Metering tick: `balance_s` renamed to `session_remaining_s`** (Streaming DX Finding 2). The value was always the seconds remaining in **this session's** credit reservation (the session cap) — the old name read like account credit balance, which lives at `GET /v1/billing/credits` (`balance_seconds`). Every tick now carries BOTH `session_remaining_s` (canonical) and `balance_s` (deprecated alias, identical value). **`balance_s` will be removed ~2026-08-28** — migrate reads to `session_remaining_s`. - **Terminal `results_summary`: `frames` renamed to `result_frames`** (Streaming DX Finding 3). Results are **sampled**, not per-source-frame (adaptive inference cadence, roughly every 8th source frame under load), so the count is the number of result **events emitted** — `result_frames` says that; `frames` implied a source-frame count. Both fields present (identical value); **`frames` will be removed ~2026-08-28** — migrate reads to `result_frames`. - **Docs: result-sampling semantics now documented** in agents.md — `frame_num` on live results is the *source-frame index* (gaps between consecutive values are normal); `results_summary.result_frames` counts result events; expect roughly 8–15 results/s depending on load. - No behavior changes — both renames are additive aliases for 30 days, then the deprecated names drop. --- ## 2026-07-28 — No-media warning: the server now tells you when your media is the problem - **New signaling-WS event: `warning {code: "no_media_frames"}`** — when the WebRTC transport connects but no decodable video frame arrives within 5 seconds, the server pushes `{type:"warning", code:"no_media_frames", transport_connected:true, elapsed_s, packets_received, frames_decoded, hint}` (re-warned once at 15s, then quiet; canceled by the first decoded frame). Previously the server stayed silent for the full media-timeout window (~50s) with zero client-visible signal. `packets_received` distinguishes the two failure classes: `0` = your client is not sending media (track dead/muted); `>0` with `frames_decoded: 0` = media arrives but is not decodable (wrong codec, or a source producing no real frames — e.g. canvas capture without a draw loop). - **`end_reason: "media_timeout"` now carries the media counters in `failure_diagnostic`** when a connected-but-no-media session ends without ever going live. Billed 0, as before. - Existing `warning {remaining_s}` credit-exhaustion event is unchanged (distinguish by the `code` field — credit warnings have no `code`). --- ## 2026-07-28 — TURN-over-TLS (`turns:`) relay advertised for TLS-only egress networks - **ICE server list now always includes a `turns::443?transport=tcp` relay** in session credentials. Twilio's NTS token response carries `turn:` relays (UDP/TCP 3478, TCP 443) but no `turns:` URL, so clients behind TLS-only firewalls (datacenter/corporate egress allowing only TLS on 443) could never reach a relay and sessions failed with `ice_failed`. The API now synthesizes the `turns:` entry from the existing relay host, reusing its ephemeral credentials. No-op when the provider already returns a `turns:` URL or in STUN-only fallback. No request/response field changes — the `ice_servers` array simply gains one entry. --- ## 2026-07-28 — MCP tools declare outputSchema + structuredContent; wait_for_analysis timeout shape cleanup - **All 7 hosted MCP tools now declare an `outputSchema`** (in `tools/list` on `POST /mcp` and on the static server card at `/.well-known/mcp/server-card.json`). Schemas are generated from the same Zod DTOs that validate the public /v1 responses (`src/dto/schemas.ts`), so they cannot drift from the real contract. Per the MCP spec, every tool result now also carries `structuredContent` conforming to its declared schema (the SDK validates each result at runtime); the human-readable JSON text content is unchanged. - **`wait_for_analysis` response shape changed** (privacy-audit finding F-1): the tool now returns `{ analysis, retry }` — the analysis resource unmodified under `analysis`, and `retry: null` on terminal states or `retry: { reason: "timeout", note }` when the wait expired. The old behavior merged an undocumented `_mcp_note` field into the analysis object on timeout, mutating a documented resource shape. Agents reading the analysis should use `result.analysis` (or `structuredContent.analysis`). - **`query` field description** on analysis outputs now states explicitly that it is the compiled, deterministic interpretation of the caller's own prompt — a transparency feature (privacy-audit finding F-2, documentation only; no data change). - REST API surface unchanged — this release only affects the MCP endpoint metadata and the MCP-layer `wait_for_analysis` envelope. --- ## 2026-07-28 — OpenAI apps domain verification + explicit destructiveHint on read-only MCP tools - **New public endpoint: `GET /.well-known/openai-apps-challenge`** — returns the OpenAI apps domain-verification challenge token (`OPENAI_APPS_CHALLENGE_TOKEN` env var) as raw `text/plain`; 404 when unset. Unauthenticated by design — OpenAI's plugin/apps portal fetches it to verify domain ownership (PRI-475). Not part of the product API surface. - **MCP tool annotations: explicit `destructiveHint: false`** added to all five read-only tools (`get_analysis`, `wait_for_analysis`, `list_models`, `get_usage`, `get_test_fixture`) in both the live `tools/list` response and the static server card at `/.well-known/mcp/server-card.json`. Previously the hint was omitted (spec default is `true`), which OpenAI's tool scanner flagged. No changes to tool names, descriptions, or input schemas. --- ## 2026-07-28 — Streaming DX: server ICE fix + trickle, honest end reasons, unified result contract, public examples repo ### Transport (the critical fix) - **Server-side ICE candidates are now reachable from anywhere** — the streaming server advertises its public address as a host candidate and **trickles late-gathered srflx/relay candidates** to the client as `ice` messages after the answer SDP (previously candidates that outlasted the bounded gathering window were silently dropped, so relay-only clients — datacenters, CI, UDP-blocked networks, strict NATs — stalled in `ice: checking` until timeout while browser clients survived via peer-reflexive discovery). Keep consuming `ice {candidate}` messages after `answer`. - **Relay-only regression client in CI** — every deploy is tested with an aiortc client that offers only TURN relay candidates (TURN-over-TCP:443), the datacenter/edge-fleet profile. ### Honest `end_reason` (breaking-adjacent: two new enum values) - **New values: `ice_failed`, `media_timeout`** on `stream.end_reason` (WS `end {reason}` and the terminal Stream resource). A session that never went live is **never** `completed` — enforced server-side. - **New nullable field: `failure_diagnostic`** on the Stream resource — on `ice_failed`, carries `{local_candidates, hint}` (the server-side candidate summary) so transport failures are debuggable from the API alone. - Billing unchanged: never-live sessions bill exactly 0 (join/negotiation/queue free). ### Streaming result contract unified with file analyses (breaking for undocumented internals) - `result.detections[].answer` is now **lowercase `yes|no|indeterminate`** — the same enum as file-API results (was `"Yes"/"No"`, an internal casing leak; enums never vary by transport). - `result.detections[].prompt` now echoes **exactly what you submitted** — the internal `|||COMPILED_QUERY:` serialization no longer leaks into the public field. - Latency telemetry (`inference_ms`, `server_elapsed_ms`, per-hop p50s, per-stage `latency` breakdown) is grouped under a documented **`timing`** object. Internal scheduling fields (`queue_fill`, `queue_length`, `is_warm`, `video_enabled`) are no longer emitted. - Same normalization applies to `results_summary.last_detections` on the terminal resource. ### Docs & examples - **Public examples repo**: https://github.com/Primate-Intelligence/primate-examples — Python streaming client (aiortc, the "stream a file" recipe), browser webcam example, relay-only CI harness. The old private-repo examples link in the streaming guide is gone. - **agents.md gains a full Streaming section** — lifecycle, signaling protocol incl. server trickle, the streaming result contract, the honest end_reason vocabulary, the datacenter client profile, and billing semantics. --- ## 2026-07-27 — PRI-482 round 8: one-clock docs — canonical docs served by the API, website proxies them, hardened doc-sync CI gate, count-confidence semantics clarified ### Documentation architecture (the "separate clocks" fix) - **Single source of truth** — `docs/agents.md` and `docs/changelog.md` ship inside the API deploy artifact and are served at `https://api.primateintelligence.ai/docs/agents.md` / `/docs/changelog.md`. The website (`https://primateintelligence.ai/docs/agents.md` and `/docs/changelog.md`) now **proxies these API routes** instead of serving a hand-copied duplicate. Spec, quickstart, and changelog update atomically with the code they describe — there is no copy step left to go stale. - **RSS moves to the API** — `GET /docs/changelog.xml` is generated from `docs/changelog.md` at request time, so the feed can never drift from the changelog it is derived from. - **Hardened doc-sync CI gate** — a diff touching `src/routes/**`, `src/dto/paths.ts`, or billing/pricing services (`credit-ledger-service`, `billing-service`, `usage-meter-service`, `quota-service`) now fails CI unless **both** a docs surface (`docs/agents.md` or `docs/openapi.json`) **and** `docs/changelog.md` move in the same diff. Previously route changes required only one doc surface and billing changes required none. - **Nightly drift monitor** — an external probe diffs the website-served docs against the API canonical and alerts on any byte mismatch (belt-and-suspenders against pipeline-level failures CI can't see). ### Contract clarification - **Count-query confidence** — agents.md (Result Contract + Count queries sections) and `llms.txt` now state explicitly: for count queries, `confidence` applies to the detected count itself — it is the model's confidence that `detected_count` is the correct number, not merely that something was detected. --- ## 2026-07-27 — PRI-482 round 7: batch validate_only, enriched batch pricing, llms.txt batch listing, docs-atomicity CI gate, result-contract confidence cross-reference ### New capabilities - **`POST /v1/analyses/batch` with `validate_only: true`** — dry-run for batch: parses all prompts, returns `analysis_batch_preview` (HTTP 200) with per-prompt `query`, `parse_mode`, `assessable`, `estimated_seconds`, `estimated_cost_usd`, and `discount_pct`; plus batch-level totals `estimated_total_seconds` and `estimated_total_cost_usd`. No credits reserved, no jobs created. Estimates are `null` when the video has no known duration. - **Enriched batch pricing on real create** — `POST /v1/analyses/batch` response `pricing` block now includes `estimated_total_seconds` and `estimated_total_cost_usd` (null when duration unknown), so callers can see the expected cost without a separate `validate_only` round-trip. ### Docs & CI - **`llms.txt` batch endpoint** — `POST /v1/analyses/batch` added to the `## Key endpoints` section. - **Docs-atomicity CI gate** — `scripts/changelog-gate.ts` now enforces a second rule: any diff touching `src/routes/**` or `src/dto/paths.ts` must also update `docs/agents.md` or `docs/openapi.json`. Bypass with `[skip-docs-gate]` in a commit message. - **Result Contract confidence cross-reference** — `result.confidence` bullet in agents.md "Result Contract" section now notes that for count queries confidence reflects detection certainty (not count certainty), with a link to the Count queries section. - **agents.md batch section** — added `validate_only` documentation with worked JSON example. - **OpenAPI** — `validate_only` added to `CreateAnalysisBatchRequest`; `AnalysisBatchPreview` response schema added; `AnalysisBatch.pricing` updated with estimated fields. --- ## 2026-07-27 — PRI-482 round 6: batch discounts real, confidence semantics, key lifecycle in OpenAPI, 404 envelope, ledger source_type, historical balance freeze ### New endpoints - **`POST /v1/analyses/batch`** — submit 2–10 prompts against the same video in one request. First prompt billed at full price; each additional prompt billed at **50%**. Credits reserved per-prompt at the discounted rate (observable directly in `GET /v1/credits` ledger). Response includes `analyses[]` array + `pricing: {full_price_prompts, discounted_prompts, discount_pct: 50}`. Registered in OpenAPI spec and documented in agents.md "Batch analyses & discounts" section. - **`POST /v1/keys/upgrade`** and **`POST /v1/keys/request` / `GET /v1/keys/request/{code}`** now registered in `docs/openapi.json`. These routes existed but were absent from the spec — they are the first two calls any agent makes. ### Error handling - **404 catch-all** — unknown routes previously returned the framework-default `{"message":"Route ... not found"}`. They now return the standard typed error envelope (`code: "resource_not_found"`, same shape as all other errors). - **`POST /v1/analyses` with `prompts` array** now returns a helpful validation error pointing to `POST /v1/analyses/batch` instead of a generic "Unknown field". ### Billing fixes - **`source_type: "analysis"` for analysis debits** — credit ledger rows for public-API analyses were labelled `source_type: "upload"`. They are now labelled `source_type: "analysis"` (new DB constraint value). Migration 077 backfills existing reservation and transaction rows in dev. Agents reading `GET /v1/credits` will see `source_type: "analysis"` on analysis debits going forward. - **Historical `credit_balance_after` frozen** — analysis rows settled before migration 075 had a null `balance_after_seconds` snapshot, causing `credit_balance_after` to drift as later analyses ran. Migration 078 stamps those rows with the current account balance, freezing the value. `credit_balance_after: null` (`snapshot_unavailable`) now only appears when the account balance itself is unavailable. ### Docs & contract - **Confidence semantics for count queries** — added a paragraph to agents.md "Count queries — the contract" section: `confidence` is the aggregate detection confidence (max of per-term values, [0,1]); forced to 0 when `answer: "indeterminate"`; `≥0.7` reliable, `0.4–0.69` marginal, `<0.4` unreliable. - **Changelog link in agents.md** — added a dedicated "Changelog" section linking `/docs/changelog.md`. - **`snapshot_unavailable` semantics** — `usage.credit_balance_after: null` documented in agents.md Result Contract. - **Walking fixture section** added to agents.md "Action pipeline verification" (clip URL pending upload). ### Migrations applied to DEV - `077_analysis_source_type.sql` — adds `'analysis'` to `credit_reservations.source_type` constraint; backfills existing rows. - `078_freeze_historical_balance_after.sql` — stamps null `balance_after_seconds` on settled reservations with current account balance. --- ## 2026-07-27 — PRI-480 round 5: agent DX — versioned responses, immutable billing snapshot, truthful origin, pricing discovery ### Breaking changes - **`analysis.origin` enum changed: `user` → `api`, new value `console`.** The round-2 entry below introduced `origin: "user" | "system"` derived at read time from `source_type` — that heuristic misclassified console (dashboard) uploads as `system` and recorded nothing. `origin` is now stamped truthfully at job creation: `api` (public API), `console` (dashboard upload), `system` (internal — auto-probe, fixture seeding, run_prompt). Legacy rows without the column fall back to the heuristic (`public_api*` → `api`, `upload` → `console`, else `system`). The field shipped yesterday with near-zero consumers. ### New fields & headers - **`X-Api-Version` response header** — every response (including errors) carries `+`. If the value changes mid-session, a deploy happened — re-check this changelog before assuming a bug. - **`usage.credit_balance_after` is now immutable.** Previously it re-read the live account balance at GET time, so an old analysis's value mutated as later analyses ran. It is now a point-in-time snapshot stamped at settlement (`credit_reservations.balance_after_seconds`). Analyses settled before this deploy fall back to the live balance (previous behaviour). ### Billing invariant - **System-initiated analyses are never billed.** `origin: "system"` jobs (e.g. internal run_prompt) never reserve or settle credits — enforced by a CI test auditing every `reserveCredits` call site. ### Docs & discoverability - **`GET /v1/credit-pricing`** (public, no auth) is now in the OpenAPI spec, the [agents.md](./agents.md) Pricing section, and `/llms.txt`. Returns `price_per_second_cents`, `signup_grant_seconds`, `card_grant_seconds` — cost of an analysis = `usage.billed_seconds × price_per_second_cents`. - **Count-query contract** section added to [agents.md](./agents.md): `detected_count` is only meaningful when `answer` is determinate; success/failure examples are schema-validated in CI. - **Changelog is now CI-enforced** — any push/PR diff touching `src/**` must also update this file with a well-formed topmost `## YYYY-MM-DD — ` entry (`scripts/changelog-gate.ts` + `changelog-gate.yml`). - **Late note (shipped earlier today in commit df7f0ce without an entry):** `GET /docs/agents.md` and `GET /docs/changelog.md` are now served publicly — `/llms.txt` referenced both but they 404'd. - Changelog re-sorted newest-first (the 2026-07-24/22/18 entries were filed below 2026-07-09). --- ## 2026-07-27 — PRI-480 round 2: result invariants, synchronous metadata, billed_seconds, origin ### New fields - **`analysis.origin`** — `"user"` (submitted via the public API) or `"system"` (generated internally, e.g. auto-probe or fixture seeding). Agents should normally only act on `origin: "user"` analyses. ### Behaviour fixes - **Result consistency invariants.** A completed analysis returning `detected_count: 5` + `answer: "indeterminate"` + `indeterminate_reason: "nothing_detected"` is now repaired at the API boundary: (a) `nothing_detected` forces `detected_count` to 0, (b) `indeterminate` forces `confidence` to 0, (c) determinate `yes`/`no` answers never carry an `indeterminate_reason`. - **Video metadata populated synchronously on `/complete`.** `POST /v1/videos/:id/complete` now awaits the ffprobe enrichment (15 s timeout) before building the response, so `duration_s`, `width`, `height`, and `fps` are present in the response body and in immediate `GET /v1/videos/:id` reads. On timeout the endpoint returns the row as-is and enrichment continues in the background. - **`billed_seconds` on list analyses now correct.** The `GET /v1/analyses` list was querying `credit_reservations.job_id` (a column that does not exist). Fixed to join via `jobs.reservation_id → credit_reservations.id`, matching the single-GET path. `usage.billed_seconds` is now populated on all settled list items. ### New endpoints - **`GET /llms.txt`** — plain-text LLM index following the llms.txt convention. No auth required. Lists the API base URL, `/docs`, `/openapi.json`, and changelog. Lets AI coding assistants discover the API surface without a browser. ### Docs - **Query-type maturity note** added to [agents.md](./agents.md): presence and counting queries are mature/GA; action and temporal queries are beta (duration-sensitive); open-ended prompts are rejected with `unsupported_query_form` rather than billed. --- ## 2026-07-27 — PRI-480: API hardening, credits, and fixture parity ### Breaking changes None. ### New fields - **`result.video_duration_s`** — now `number | null` (was always `number`). Returns `null` when the source video duration could not be determined (short uploads, metadata-less files). Agents must null-check this field. - **`result.indeterminate_reason`** — why the answer is `indeterminate`. Enum: - `low_confidence` — model saw something but confidence was below threshold; try a more specific prompt. - `nothing_detected` — all confidence scores were 0; no subject detected in any frame. - `unsupported_query_form` — open-ended or freeform query; rephrase as a closed yes/no or count question. - `duration_mismatch` — inference result duration disagrees with source video duration (>20%); result is untrusted. - **`result.detected_count`** — integer count for count-intent queries ("how many X…"). `null` on yes/no queries. ### New endpoints - **`GET /v1/credits`** — returns `{ balance_seconds, grant_seconds, used_seconds, transactions: [...] }`. Per-analysis transaction history. Lets agents audit every charge independently. No parameters needed; key-scoped. ### New parameters - **`POST /v1/analyses` → `validate_only: boolean`** — parse + estimate cost, no GPU, no credits deducted. Returns `AnalysisPreview` with `estimated_cost_seconds`, `query_type`, and `answerability` signal. Use before committing to an analysis. ### Behaviour fixes - **Action pipeline duration now deterministic.** The action query path (`how many X…`) was returning GPU wall-clock time as `video_duration_s` instead of source video duration. For a 6s video this produced 40–75s billing. Root cause: `duration_seconds` was absent from the action result dict in the inference server; the API fell back to GPU wall-clock. Fixed in inference I-1 (inference commit `4e0825a`). - **`detected_count` now populates for action queries.** The count was computed by inference but not forwarded in the callback payload. Fixed in inference I-2. - **`indeterminate_reason` forwarded from inference.** When inference explicitly sets a reason, the API now respects it rather than re-classifying. - **Duration invariant cross-check.** If inference returns a duration >20% different from the admitted source duration, the API flags the result `indeterminate` with `indeterminate_reason: duration_mismatch` rather than silently returning wrong timing. - **Open-ended query gate.** Submitting a query the model cannot answer (open-ended form) now returns `answer: indeterminate`, `indeterminate_reason: unsupported_query_form` immediately, at zero cost, instead of burning GPU credits and returning an unreliable result. - **Video metadata probed at upload-complete.** `duration_seconds`, `width`, `height`, `fps` are populated on the video object immediately after `POST /v1/videos/:id/complete`. Fire-and-forget ffprobe enrichment runs after response is sent for non-faststart files. ### Test fixture update `GET /v1/test-fixture` now returns a `fixtures[]` array with two entries: - `presence` — forklift yes/no (existing fixture, unchanged) - `action` — forklift count query; assert `detected_count >= 1` Legacy flat fields (`test_video_url`, `test_prompt`, `expected_answer`, `expected_confidence_min`) preserved for backwards compatibility. --- ## 2026-07-24 — PRI-477: billing cap, detected_count, error envelope ### New fields - **`result.detected_count`** — integer count on count-intent queries (object path). - **`analyses.list` response** — includes `billed_seconds` per analysis. - **Error envelope** — all errors now return `{ error: { code, message, request_id } }` (was bare `{ error, message }`). ### Behaviour fixes - **Billing cap** — per-analysis billing is capped at the source video duration. A slow GPU run (warmup, recompile) cannot overbill beyond the video length. - **`answer: indeterminate`** — when confidence is 0 and all term confidences are 0 the answer is now `indeterminate` instead of `no`. A `no` that can't be substantiated is indistinguishable from a model that saw nothing. - **`usage` deprecated** — the `GET /v1/usage` response is kept but `usage` inline on analysis objects has moved to `GET /v1/credits`. --- ## 2026-07-22 — PRI-476: inline usage, url-ingest probe, liveProgress ### New fields - **`analysis.usage`** — inline `{ billed_seconds, credit_balance_after }` on completed analyses. Agents no longer need a second GET /v1/usage round-trip. - **URL-ingest video probe** — `POST /v1/videos` with `source_url` now runs the same metadata probe that upload-complete uses. --- ## 2026-07-18 — Public API v1 GA Initial public release of the REST API with: - `POST /v1/sandbox` — instant `pv_test_` key, no email, no card - `POST /v1/keys/upgrade` — upgrade to `pv_live_` + 6,000s free grant - `POST /v1/videos` — presigned upload or URL ingest - `POST /v1/analyses` — submit prompt + video → analysis - `GET /v1/analyses/:id` — poll or use `Prefer: wait=60` - `GET /v1/analyses` — paginated list - `GET /v1/credits` — balance + transactions - `GET /v1/test-fixture` — stable CI fixture --- ## 2026-07-12 — Credit notifications, ToS consent, CORS ### New features - **Expiring-credits reminder email** — sent once per credit lot, 7 days before expiry. Requires opt-in notification prefs. - **Low-balance email** — fires once per episode (was: every 24h). ### Behaviour fixes - **Admin API-key mint/rotate** now stamps implied Terms of Service acceptance. - **Admins exempt from ToS gate** on stream and upload endpoints. - **CORS preflight** now allows `X-Primate-Client` header (dashboard regression fix). --- ## 2026-07-09 — PRI-440/439: P11 hardening, pricing analytics, SDK 0.1.1 ### New features - **Brownout ladder + status page** — spend anomaly auto-suspend with configurable ladder (`BROWNOUT_STATE`). Public status reflected at `/v1/status`. - **SSRF IP-pinning** — all outbound URL fetches (e.g., source_url ingest) are blocked from internal RFC-1918 and link-local ranges. - **Key-hygiene redaction** — raw key values are redacted from server logs after mint. - **C1 runtime measurement** — per-analysis GPU wall-clock recorded for pricing analytics (Phase 5). ### SDK - **`PrimateError.rawMessage`** — verbatim server message, for UI display. TS SDK + MCP bumped to 0.1.1 via npm OIDC Trusted Publishing. - **Python SDK** — PyPI Trusted Publishing workflow added (tag `sdk-py-v*`). --- <!-- Backfilled 2026-07-27 (PRI-482 r8) from the website changelog when docs collapsed to one clock. --> ## 2026-07-09 — `additive` — SDKs, MCP server, agent artifacts - Official SDKs: TypeScript (`@primate-intelligence/sdk`) and Python (`primate-intelligence`) — typed resources, registry-driven retries, auto idempotency keys, `createAndWait()`, webhook verification, pagination iterators, browser `connectStream()`. - MCP server (`@primate-intelligence/mcp`) with seven tools mirroring the spec. - `llms.txt` / `llms-full.txt` + `/.well-known/llms.txt`; all docs pages served as raw markdown at `/docs/*.md`. - New documentation site: quickstart, agent quickstart, ten guides, Scalar API reference, error-registry anchors. ## 2026-07-09 — `deprecation` — Legacy `pk_` API key prefix sunset prepared - Existing `pk_` keys remain valid through the launch migration. - New keys and rotations create `pv_live_` / `pv_test_` keys only. - The 12-month `pk_` sunset clock starts at production GA announcement after Matt approves the cutover; the exact date will be published in the versioning guide and dashboard before enforcement. ## 2026-07-08 — `additive` — Billing: credit purchases + auto-refill - `GET /v1/credit-pricing` (public): price per second, grant sizes, purchase options. - Dashboard credit purchases, saved payment methods, and threshold-triggered auto-refill. - `GET /v1/billing/usage-summary` for dashboard usage aggregation. ## 2026-07-07 — `additive` — Webhooks + usage meters - `webhook_endpoints` resource: CRUD, Standard-Webhooks signing, delivery worker with `1m…24h` retry schedule, deliveries API, redelivery, secret rotation with 24h dual-secret overlap, auto-disable after 72h of failures. - Event vocabulary: `video.ready`, `video.failed`, `analysis.completed`, `analysis.failed`, `analysis.canceled`, `stream.ended`. - Per-request `webhook` overrides on `POST /v1/analyses` / `POST /v1/streams` (unsigned). - `GET /v1/usage` generic meters shape (`credit_seconds` balance + period consumption). ## 2026-07-06 — `additive` — Streams public API - `streams` resource (§4.8): create → signal (WS) → WebRTC → live per-frame results → end. Queueing with honest `queue_position`/`estimated_start_s`, per-plan session caps, 5s metering ticks, credit reservation reconciled to the second. - `stream_already_active` (409) error code registered. ## 2026-07-05 — `additive` — Client tokens - `POST /v1/client_tokens`: ephemeral browser-safe `pvct_` tokens (60–900s TTL, scope subsets, optional video/stream binding). Signaling WS accepts client tokens only — never secret keys. - `token_expired` (401) error code registered. ## 2026-07-04 — `additive` — Instant sandbox + auth cutoff - Anonymous `POST /v1/sandbox` issues instant `pv_test_` keys (IP-limited, 7-day expiry, fixture corpus, deterministic results) with a pre-seeded fixture video. - Metered credit enforcement on the live plane; open signup. - `sandbox_limit_exceeded` (429) error code registered. ## 2026-07-03 — `additive` — Public core: videos + analyses - `videos` resource: presigned upload (create → PUT → complete), SSRF-guarded URL ingest, cursor-paginated lists, delete with 409 protection. - `analyses` resource: free-text prompts or structured queries, `Prefer: wait` sync sugar (cap 120s), live progress + queue position, cancel, deterministic `result` contract (`answer`/`confidence`/`clips`). - `GET /v1/models`, `GET /v1/errors` (machine-readable registry), idempotency keys (JCS canonical body comparison, 24h replay window). ## 2026-07-02 — `additive` — OpenAPI spec - `GET /v1/openapi.json` — OpenAPI 3.1, generated from the DTO registry; CI drift + `oasdiff breaking` gates. The spec is the source of truth for docs, SDKs, and this changelog's compatibility promises. > Canonical, always-current copy: https://api.primateintelligence.ai/docs/changelog.md (also served at https://primateintelligence.ai/docs/changelog.md — same bytes, proxied).