AWS documents six Amazon Bedrock prompt-caching patterns for lower inference costs

How Bedrock prompt caching works

A developer inserts a cachePoint after a stable request prefix. On the first request, Bedrock processes that prefix and writes its partially processed state to a cache. If a later request contains a matching prefix, the model can read the saved state instead of performing the same work again; a changed prefix or expired entry produces another cache write.

Entries are scoped to an AWS account and Region. They also have model-specific activation thresholds: AWS lists at least 1,024 tokens per checkpoint for Claude Sonnet 4.5 and 4.6, compared with 4,096 tokens for Opus models. Content below the applicable threshold is processed normally rather than cached.

The default time to live is five minutes, while select models support up to one hour. The guide notes that its cross-Region inference profile can occasionally increase cache-write frequency because requests automatically route across Regions. Developers using the mixed-TTL example also need Python 3.10 or later and Boto3 1.43.0 or newer for the ttl parameter.

The cost case depends on repeated hits

A cache read costs 90% less than standard input according to AWS, but populating the cache carries a premium. A standard cache write costs 25% more than ordinary input, and a write with a one-hour TTL costs twice the standard input rate.

AWS illustrates the trade-off with a 10,000-token document and 10 different questions. The first request pays the write cost; if the next nine requests hit the cache before it expires, AWS estimates approximately 75% net savings on input-token costs for the document context. Requests after expiration trigger another write, so the headline reduction for cached tokens should not be treated as a guaranteed reduction in the application’s total bill.

Documents, system prompts and tool schemas

The first pattern targets repeated analysis of a document, such as a RAG application asking different questions about the same reference material. The static document appears before the cache marker and each changing question follows it. In AWS’s Claude Sonnet 4.5 test, the initial request wrote 1,898 tokens to cache. A later request with the same document prefix read all 1,898 tokens from cache, while only 28 question tokens were processed as standard input.

The same marker works with converse_stream. The operational difference is that streaming usage figures arrive in the final metadata event rather than in the immediate response. Applications can inspect cacheWriteInputTokens and cacheReadInputTokens to confirm writes and hits.

Claude models also support simplified cache management. AWS says a single marker can check eligible prefixes across approximately 20 preceding content blocks. Multiple markers remain useful when an application needs partial hits: unchanged early sections can be reused even if later sections differ.

For large, stable system instructions, the second pattern places the cache point in the Converse API’s system array after the prompt text. The user message stays separate and can change from request to request. AWS identifies persona-based assistants, policy-heavy customer-service bots, domain assistants and agentic workflows with durable instructions as likely uses.

The third pattern addresses agents whose JSON tool schemas collectively consume thousands of tokens. Developers append the marker after the definitions in the tools array inside toolConfig. Subsequent turns can then reuse the unchanged tool prefix rather than making the model process the same schemas again.

Different lifetimes within one request

Mixed-TTL caching separates content by how often it changes. AWS’s implementation assigns a one-hour lifetime to core reference material and a five-minute lifetime to session context. Dynamic information such as real-time data and individual queries is left uncached.

There is a strict ordering rule: longer-lived checkpoints must come before shorter-lived checkpoints, including when checkpoints span messages, system prompts and tool definitions. A request ordered as one hour followed by five minutes is valid; reversing the order causes an API error.

The Converse response’s cacheDetails field provides a per-TTL token breakdown. AWS’s example shows 1,547 tokens for the one-hour tier and 1,823 for the five-minute tier. Each tier must independently meet the model’s minimum token requirement; otherwise the response may show only one cache entry.

Separating caches for multiple tenants

Account-and-Region scoping does not by itself separate customers served from the same account and Region. AWS therefore documents a content-prefix pattern for multi-tenant applications: prepend a SHA-256 hash of the tenant ID to the cached instructions. Different hashes change the prefix and cause Bedrock to create independent cache entries for different tenants.

In the guide’s expected request sequence, the first request from each tenant writes a distinct entry and the next request from that tenant reads its own entry. AWS estimates that the 64-character hash adds approximately 16 prompt tokens and says the pattern requires no server-side configuration or separate AWS accounts.

LangChain and API differences

LangChain users can create the marker through ChatBedrockConverse.create_cache_point(), either in directly constructed message arrays or in reusable ChatPromptTemplate chains. LangChain exposes writes and reads under usage_metadata.input_token_details as cache_creation and cache_read.

The underlying InvokeModel API uses different, model-family-specific conventions. For Anthropic requests, the marker is an embedded cache_control object rather than a standalone Converse content block; it also reports cache_creation_input_tokens and does not return Converse’s cacheDetails field. AWS recommends Converse for new applications because its marker format remains consistent across supported model families.

What production teams need to measure

AWS recommends profiling prompts to separate static components from questions, session state and other changing data. Teams should verify that every checkpoint clears its model threshold, select TTLs that reflect update frequency, and log cache-write and cache-read counts. A low hit rate can indicate that prefixes change too often or expire too quickly. System prompts, message content and tool definitions can be cached together in one request, provided their placement and TTL ordering are valid.

Latency gains are workload-dependent. AWS says tests with prefixes of roughly 2,000–5,000 tokens may not show statistically significant improvement over a small number of iterations. The benefit becomes more pronounced for cached prefixes above 10,000 tokens, but exact results vary with prefix size, model and current load. Developers therefore need to measure both cost and time to first token on their own traffic rather than infer application-wide performance from the pricing maximum.

The examples use on-demand inference and create no persistent AWS resources, so AWS says cleanup is limited to stopping any running notebook kernels. The publisher also recommends applying Bedrock Guardrails for content filtering and grounding validation alongside caching in production.

Source: AWS Machine Learning Blog

Definition. Amazon Bedrock prompt caching saves the processed state of a stable request prefix so matching later requests can reuse it instead of processing the same input again.

Key takeaways

FAQ

Sources