Batch

Endpoints

All batch routes use the same bearer authentication as the rest of the public API and require the Developer tier with the inference scope. A batch belonging to another account returns 404 BATCH_NOT_FOUND: indistinguishable from one that does not exist.

Creating a batch

The body is a top-level requests array; each item uses exactly the same shape as the Responses API, minus stream, which batch ignores.

curl -X POST https://evaesi.dev/api/v1/batch \
    -H "Authorization: Bearer esi_dev_YOUR_KEY" \
    -H "Content-Type: application/json" \
    -H "Idempotency-Key: nightly-summaries-2026-07-27" \
  -d '{
    "requests": [
      { "prompt": "Summarise this contract.", "model": "instant", "max_output_rungs": 512 },
      { "prompt": "Extract the payment terms.", "model": "reasoning" }
    ]
  }'

A successful create returns 202 Accepted with the batch object. Every item is validated before the batch is stored, EMPTY_PROMPT, PROMPT_TOO_LARGE, INVALID_MODEL, MODEL_UNAVAILABLE, and INVALID_REQUEST (output cap) all fire at creation, with an x-evaesi-batch-index header naming the offending item. A batch is accepted whole or not at all.

What a batch looks like

{
  "id": "b7c1e0f2-...",
  "status": "in_progress",
  "total_requests": 2,
  "completed_requests": 1,
  "failed_requests": 0,
  "usage": { "input_rungs": 52, "output_rungs": 847, "total_rungs": 899 },
  "created_at": "2026-07-27T10:00:00Z",
  "started_at": "2026-07-27T10:00:15Z",
  "expires_at": "2026-07-28T10:00:00Z"
}

started_at, completed_at, and error are omitted while absent rather than returned as null. usage accumulates as items finish, so it is a live view of what the batch has cost so far.

Lifecycle

completed, failed, cancelled, and expired are terminal: the batch will not change again. Poll GET /api/v1/batch/{batch_id} until you see one of them. Progress is committed per item, so a platform restart resumes a batch exactly where it stopped, items are never run, or billed, twice.

Results

Results are available as soon as items start finishing; you do not have to wait for the batch to be terminal. Order matches submission order via index.

{
  "batch_id": "b7c1e0f2-...",
  "status": "completed",
  "results": [
    {
      "index": 0,
      "status": "completed",
      "request_id": "8f3a...",
      "model": "instant",
      "content": "The contract obliges...",
      "usage": { "input_rungs": 24, "output_rungs": 402, "total_rungs": 426 }
    },
    {
      "index": 1,
      "status": "failed",
      "error": { "code": "INFERENCE_BUSY", "message": "The inference request failed." },
      "usage": { "input_rungs": 0, "output_rungs": 0, "total_rungs": 0 }
    }
  ]
}

Per-item status is one of queued, in_progress, completed, failed, or cancelled. A failed item carries an error using the same codes as the interactive API. An item interrupted by a platform restart is reported as failed with code INTERRUPTED and is deliberately not retried, the inference may already have run, and charging twice for one request would be worse than reporting it honestly.

Use request_id with the Sources API to fetch evidence for an individual item while the gateway still holds it.

Cancellation

POST /api/v1/batch/{batch_id}/cancel stops a batch that is queued or in_progress and returns the updated batch object. Anything else returns 409 BATCH_NOT_CANCELLABLE.

Cancellation is cooperative and honest: items that have not started are cancelled immediately, but an item already dispatched to the model finishes and is billed, because the work really was performed. A batch cancelled while draining passes through cancelling before settling as cancelled.

Pricing

Batch usage is metered per item exactly like interactive requests, the same rungs, the same per-model multipliers, and each item is recorded against your project budget as it completes. Batch requests carry the platform's batch discount at pricing time; see Billing for the current rate card.

Two admission controls run at creation, not mid-drain: the whole batch's estimated cost is charged against your per-day rung ceiling, and each item reserves project budget before it runs. A batch that would exhaust your daily ceiling is rejected up front; if the project hard cap is reached mid-batch, the remaining items fail with 402 BUDGET_EXCEEDED and the batch still reports every item's outcome.

Errors

• 400 EMPTY_BATCH: the requests array was empty
• 400 BATCH_TOO_LARGE: more than 100 requests (exactly 100 is accepted)
• 400 per-item validation failures, with x-evaesi-batch-index naming the item
• 404 BATCH_NOT_FOUND: unknown batch, or one belonging to another account
• 409 BATCH_NOT_CANCELLABLE: the batch already reached a state that cannot be cancelled
• 429 RATE_LIMITED: the batch estimate exceeds your remaining daily rung ceiling
• 403 INSUFFICIENT_SCOPE: the key lacks the inference scope