Batch Processing

The Message Batches API processes large volumes of requests asynchronously at roughly 50% of the standard cost. In exchange you accept an up to 24-hour processing window with no latency guarantee, and no support for multi-turn tool calling within a single batch request. It is a throughput-and-cost tool, not a speed tool.

Synchronous versus batch

The decision rule is simple: if a human or a blocking workflow is waiting on the result, use the synchronous API. If the work is latency-tolerant, use batch.

# CI code check submitted to batch:
batch.create(requests=[...])  # PR blocked
# for up to 24h waiting on results

# Real-time review -> synchronous:
client.messages.create(...)
# Overnight report -> batch:
client.messages.batches.create(...)

Correlate results with custom_id

Each request in a batch carries a unique custom_id. Results come back unordered, so the custom_id is how you match a response to its request — and how you identify exactly which items failed for targeted resubmission.

import anthropic
from anthropic.types.messages.batch_create_params import Request

client = anthropic.Anthropic()

batch = client.messages.batches.create(
    requests=[
        Request(
            custom_id=f"doc-{doc_id}",
            params={
                "model": "claude-sonnet-4-5",
                "max_tokens": 1024,
                "messages": [{"role": "user", "content": text}],
            },
        )
        for doc_id, text in documents.items()
    ]
)

# Later, once status == "ended":
for result in client.messages.batches.results(batch.id):
    if result.result.type == "succeeded":
        store(result.custom_id, result.result.message)
    else:
        mark_for_resubmit(result.custom_id, result.result.type)

Handling failures

1. 1Identify failures by inspecting the custom_id of each errored result.
2. 2Resubmit only the failures, with modifications — chunk oversized documents, simplify prompts, or add few-shot examples.
3. 3Refine prompts up front on a 5-10 document sample covering edge cases and format variety before submitting the full batch.

Planning around the SLA

Work backward from your deadline. With a 30-hour SLA and a 24-hour maximum window, you have a 6-hour buffer. Submitting a fresh batch every 4-6 hours keeps one in flight at all times, so a slow batch never blows the deadline.