How to reduce LLM costs in AI customer support without cutting quality

A support team that adopts an AI agent usually budgets for the wrong thing first. They price the model per ticket, multiply by expected volume, and treat that number as fixed. It is not fixed.

The same conversation routed through an uncached prompt, a single oversized model, and an unpruned context window can cost four to six times more than the identical conversation run through a system built to avoid waste, and the quality difference between the two is often unmeasurable to the customer.

This guide is written for whoever owns the AI customer support budget: a support lead watching the bill climb with ticket volume, or an engineer asked to bring it back down without breaking the agent that customers already rely on. It covers the three levers that move the number, in order of how much they typically save, and ends with a worked model you can adapt to your own ticket mix.

Why the token bill becomes a support problem

A support conversation is not one API call. It is a system prompt that defines the agent's role and rules, a set of tool definitions the model can call, retrieved passages from the knowledge base, the ticket history, and the customer's new message, all sent together on every single turn of the conversation.

Multiply that by ticket volume and the arithmetic gets uncomfortable fast. Service organizations running AI agents grew from 39% in 2025 to 66% in 2026 according to Salesforce's research (DigitalApplied), and 91% of CX leaders report executive pressure to deploy AI faster than their teams can validate it. That pressure pushes teams to ship first and measure cost second, which is how a working agent quietly becomes an expensive one.

None of this means the model choice was wrong. It means the request pattern around the model was never optimized, and that pattern is fixable without touching the model at all.

The good news is that cost and quality are not opposed here. Deflection benchmarks put the ceiling for AI-resolved volume around 65 to 75% at maturity, with a realistic first-year range of 45 to 60% (HappySupport). A cheaper, better-structured request pipeline moves cost down while that deflection number moves up, because both improvements come from the same underlying discipline: sending the model exactly what it needs and nothing else.

Teams evaluating AI customer support pricing from a vendor should ask how much of the published rate already reflects this discipline versus how much gets passed through unoptimized.

What actually drives the cost per ticket

Communicate.so

Four things determine what a single AI-handled ticket costs: the size of the fixed prompt sent on every call, how much retrieved context gets attached, how many turns the conversation takes before it resolves, and which model answers it.

The fixed prompt is the part teams forget to measure. A system prompt with detailed tone rules, refusal instructions, and tool schemas can run 1,500 to 3,000 tokens before a single customer word arrives. Sent uncached on every turn of a five-turn conversation, that fixed cost alone can outweigh the cost of answering the actual question.

Retrieved context compounds the same problem. A retrieval-augmented agent pulls the top-matching passages from the knowledge base for every query, and teams tend to over-retrieve out of caution, attaching eight or ten passages when three would have answered the question. Each unused passage is pure waste: tokens billed, latency added, and in the worst case, irrelevant text that increases the odds of a wrong answer rather than a better one.

Turn count is the fourth driver and the easiest to overlook. A ticket that resolves in two turns costs a fraction of one that takes six, and turn count is usually a symptom of an agent asking clarifying questions it should already know the answer to from account context. Teams building an agent from scratch can read the step-by-step process in how to build an AI customer support agent, where turn efficiency is treated as a design goal from the first prompt draft rather than a cost fix bolted on later.

Prompt caching cuts the fixed cost of every turn

Prompt caching lets a provider store the static prefix of a request, the system prompt and tool definitions that do not change between calls, and charge a fraction of the normal rate for reusing it instead of the full input rate.

Anthropic's documentation states cached input tokens are billed at roughly a tenth of the standard input rate on Claude models (Anthropic, checked 2026-08-02), and OpenAI applies a similar reduction to repeated prompt prefixes on its API (OpenAI, checked 2026-08-02). For a support agent where the system prompt and tool schema are identical across nearly every call, this is close to free savings: no accuracy trade-off, no routing logic, just structuring the request so the static part comes first and stays byte-for-byte identical between calls.

The catch is that caching only pays off on the static prefix. If the system prompt embeds a timestamp, a random session ID, or per-customer data at the top of the prompt, the cache misses on every call and the discount disappears. Structuring prompts with static instructions first and variable customer data last is a small rewrite with a large payoff.

Teams that train an agent on a help center often build the largest cacheable block without realizing it. Instructions on tone, escalation rules, and citation requirements described in train AI on your help center tend to be long, stable, and identical across every ticket, which makes them close to ideal caching candidates. The knowledge base passages attached per query are the part that changes, so they belong after the cached prefix, not woven into it.

Model routing sends each ticket to the right size model

Communicate.so

Not every ticket needs the same model. A password reset, an order status lookup, or a shipping question is a pattern match against known information. A billing dispute with conflicting account history, a multi-step troubleshooting flow, or a request that touches a policy exception benefits from a larger model's reasoning.

A router, often a small classifier or even the small model itself scoring its own confidence, sends the easy majority to a cheap, fast model and escalates the rest. Checked against vendor pricing on 2026-08-02, OpenAI's GPT-4o mini runs $0.15 per million input tokens and $0.60 per million output tokens (OpenAI), while Anthropic's Claude Haiku 4.5 runs $1.00 per million input tokens and $5.00 per million output tokens (Anthropic). Both sit an order of magnitude below their respective flagship models, and both are strong enough for classification, extraction, and templated response generation, which covers most repetitive support volume.

The design decision that matters is what happens when the router is wrong. A ticket incorrectly sent to the cheap model should fail toward escalation, not toward a confident wrong answer. Set the small model to hand off when its own confidence drops rather than let it guess, and the routing saves money without adding a new source of bad answers.

Context trimming removes tokens that never earn their place

Retrieval quality is a precision problem before it is a recall problem. Attaching every plausibly relevant passage to a prompt does not make the model smarter, it makes the model responsible for ignoring the noise, and larger context windows do not fix this because the model still has to weigh every token you send it.

The fix is upstream of the model call. Re-rank retrieved passages and keep the top two or three instead of the top eight, cap passage length so a single long article does not crowd out shorter, more targeted ones, and strip boilerplate, headers, and navigation text before it ever reaches the knowledge base index. Teams that structure their source content around one answer per page, rather than long multi-topic documents, see this problem shrink on its own, because there is less irrelevant text sitting next to the relevant passage in the first place.

Conversation history is the other silent cost. A ten-turn conversation resent in full on every subsequent call means turn ten pays for turns one through nine again. Summarizing older turns into a compact state object, rather than replaying raw transcript, keeps the model informed without re-billing history that already resolved.

Batch and asynchronous processing for non-urgent work

Communicate.so

Not every AI task in a support pipeline is a live chat reply. Nightly ticket summarization, tagging closed tickets for analytics, and drafting suggested replies for a human queue can all run asynchronously, and both major providers discount work that does not need an immediate response.

OpenAI and Anthropic both offer batch processing at roughly half the standard API rate, with turnaround measured in hours rather than seconds (Anthropic, checked 2026-08-02). For a support operation running nightly reporting through analytics or tagging a backlog for training data, batch pricing turns a meaningful line item into a marginal one, because none of that work is time-sensitive enough to justify paying the live rate.

The arithmetic per 10,000 tickets

The numbers below use illustrative rates modeled on the small-model pricing above (roughly $0.15 input and $0.60 output per million tokens) applied to a support workload of 10,000 tickets a month, averaging 2,000 input tokens and 300 output tokens per ticket before optimization. Treat the totals as directional, not a quote, since your own prompt size and ticket mix will change the exact figures. The mechanism, not the specific dollar amount, is what should transfer to your own pricing model.

The compounding matters more than any single lever. Caching removes the repeated fixed cost, routing removes the model-size tax on easy tickets, and trimming removes the wasted retrieval tokens on every remaining call. Applied together against an uncached, unrouted, unpruned baseline, teams commonly see blended per-ticket cost fall by more than half, without a corresponding drop in resolution quality, because none of the three levers touch the actual reasoning the model performs on a hard ticket.

Walk through a single ticket to see where the reduction actually happens. An uncached, unrouted ticket sends roughly 2,000 input tokens and gets 300 output tokens back from a single model handling every request the same way. With caching, 1,500 of those input tokens, the static system prompt and tool schema, bill at the cached rate instead of the full rate.

With routing, 70 to 80% of tickets that classify as routine go to the smaller model instead of the larger one. With trimming, the remaining live-context tokens drop by roughly a third because only the top two or three passages travel with the request instead of eight.

None of these three changes requires touching the model's reasoning path on a hard ticket. A genuinely difficult ticket, the kind that needs the larger model, full context, and multiple turns, still gets all of that. The savings come entirely from not paying the same price for the easy majority that never needed it.

What to measure before you change anything

Communicate.so

Cost optimization without a baseline is guesswork. Before touching caching, routing, or trimming, log four numbers per ticket: total tokens in, total tokens out, which model answered, and whether the ticket resolved without a human handoff. Compare those against your first response time and resolution quality after each change, not just the invoice, because a change that halves cost and doubles escalations has not actually saved money.

Cache hit rate is the single most informative new metric to add. A hit rate below 70% on a support agent's system prompt usually means something in the static prefix is not actually static, session data, timestamps, or a dynamically generated instruction leaking into the cached region and invalidating it on every call.

Track these numbers weekly rather than monthly for the first quarter after any change. Token cost and resolution quality can drift gradually as ticket mix shifts with the season or a product launch, and a weekly view catches that drift while it is still a small adjustment rather than a large one discovered at the next invoice.

Where cost cutting breaks the agent

Every lever in this guide has a failure mode if pushed too far. Routing everything to the cheapest model regardless of ticket content trades cost for accuracy on the hard minority that actually needed the larger model. Trimming context too aggressively starves the model of the passage that had the real answer, producing a confident guess instead of a grounded one.

The safe order of operations is caching first, because it has no accuracy trade-off at all, then routing with a conservative escalation threshold, then trimming with a re-ranker rather than a fixed cutoff. Run each change against a fixed test set of real tickets before rolling it out, the same discipline used to reduce hallucinations in the first place, so a cost change never ships as a quality regression.

Frequently asked questions

Does prompt caching change the answers the model gives

No. Caching only changes how a provider bills the static, unchanged part of a prompt. The model receives the identical input either way, so answer quality and behavior are unaffected.

The only operational requirement is keeping the cached prefix byte-for-byte identical between calls, since even a one-character difference invalidates the cache for that call.

How long does a cached prompt stay valid

Cache lifetimes are provider-specific and typically measured in minutes, refreshed by continued use. Anthropic's documentation describes short default cache windows that extend as the same prefix keeps getting hit. A support agent handling steady traffic throughout the day keeps its cache warm naturally; a low-traffic agent with long idle gaps sees a lower hit rate and less savings from caching alone.

What is the minimum ticket volume where cost optimization is worth the engineering time

There is no fixed threshold, but the math favors optimization earlier than most teams expect. A team spending a few hundred dollars a month on model calls may reasonably defer this work. A team crossing a few thousand dollars a month, or scaling ticket volume quickly, recovers the engineering cost of caching and routing within the first month in most cases, because the fixed-prompt waste scales linearly with volume.

Can a small model handle sentiment or urgency detection well enough to route on

Yes, for the common case. Sentiment and urgency classification are exactly the kind of narrow, well-defined task small models handle reliably, since they are closer to classification than open-ended reasoning. The router itself should still fail toward escalation on low confidence rather than force a routing decision it is unsure about.

Does model routing require running two separate AI systems

Not necessarily. Many teams start with a single small model that either answers directly or flags low confidence, at which point the request re-routes to a larger model. This avoids maintaining two full agent configurations and keeps the routing logic to a single confidence check rather than a separate classification pipeline.

Will trimming context hurt answers that genuinely need multiple sources

It can, if trimming is a blunt fixed cutoff rather than a re-ranked selection. The fix is ranking candidate passages by relevance and keeping the top few by score, not by an arbitrary count, so a question that genuinely needs three sources still gets three, while a question that needs one is not padded with two irrelevant ones.

Is batch processing usable for anything customer-facing

Generally no, since batch turnaround is measured in hours rather than seconds and customers expect an immediate reply in live chat or ticket threads. Batch pricing is a good fit for nightly summarization, backlog tagging, analytics generation, and any internal process that does not have a real-time deadline.

How much does conversation history actually cost across a long ticket thread

It compounds. A ten-turn thread resent in full on every call means the tenth call pays for the previous nine turns of history again, on top of the new message. Summarizing resolved earlier turns into a short state object rather than replaying full transcript is one of the larger, most overlooked savings in a long-running support conversation.

Do output tokens or input tokens usually cost more in a support agent

Output tokens are priced higher per token on nearly every model, but input tokens are usually far more numerous in a support context, since the system prompt, retrieved passages, and history dwarf a typical reply. Total spend is usually input-dominated even though each output token costs more individually.

Does switching to a cheaper model always save money once support and escalation costs are counted

Not automatically. A cheaper model that escalates more tickets to a human, or produces answers that trigger more follow-up messages, can cost more in total once agent time and customer effort are counted. Measure resolution rate and escalation rate alongside token cost before treating a lower per-call price as a net win.

How often should routing thresholds be re-tuned

Whenever ticket mix shifts meaningfully, such as after a product launch that introduces a new category of question, or after a knowledge base update that changes what counts as a routine query. Reviewing the confidence threshold quarterly, or after any noticeable jump in escalation rate, keeps routing aligned with the current ticket population rather than the one it was tuned on.

Can prompt caching and model routing be combined

Yes, and they should be, since they address different parts of the cost. Caching reduces the fixed prompt cost regardless of which model answers, and routing chooses which model answers based on difficulty. Running both together compounds the savings rather than competing for the same tokens.

What is the risk of over-optimizing for cost in a support agent

The main risk is a quiet quality regression that does not show up in the cost dashboard. A routing threshold set too aggressively, or context trimmed too thin, produces more confident wrong answers or more silent escalations, and neither shows up until a customer complains or a support lead audits a sample of transcripts. Track resolution quality inside analytics alongside cost, not cost alone, and treat any cost change as a candidate regression until a sample of transcripts confirms otherwise.

Does a longer context window from a newer model reduce the need for context trimming

No, and this is a common misconception. A larger context window changes what fits, not what the model weighs well. Research and practical benchmarks consistently show model performance degrading as irrelevant context grows, regardless of window size, so trimming to relevant passages remains worthwhile even on models with very large context limits.

How do teams estimate token counts before they see a real bill

Most providers publish a tokenizer or estimation library that converts sample text into a token count without making an API call. Running a representative sample of real tickets, system prompts, and retrieved passages through that tokenizer before launch gives a reasonably accurate cost estimate, well ahead of the first invoice.

Estimating before launch also catches design mistakes early, such as a system prompt that accidentally duplicates instructions or a retrieval step that returns far more passages than the prompt template expects. Catching that in a token estimate is far cheaper than catching it in a month of production billing data.

Should a small team even bother with model routing, or just pick one model

A small team with low ticket volume and a simple ticket mix can reasonably start with one model and revisit routing once volume or complexity grows. The engineering overhead of a router is only worth it once the volume of routine, low-difficulty tickets is large enough that the savings clearly outweigh the added system complexity.

Does prompt caching work the same way across every model provider

No. Each provider implements caching with its own discount rate, minimum cacheable prefix length, and cache lifetime, so a caching strategy built for one provider's API needs to be re-verified against another provider's documentation before assuming the same savings apply. Always check the current pricing page for the specific model in use.

What is the single most impactful change for a team just starting cost optimization

Prompt caching on the static system prompt, because it requires no accuracy trade-off, no new classification logic, and typically the smallest engineering change of the three levers in this guide. Teams new to cost optimization should implement caching first, measure the savings, and treat routing and trimming as the next round of work.

How does ticket deflection rate interact with LLM cost

Deflection and cost pull in the same direction when the agent resolves tickets without a human, since a resolved ticket at low token cost beats an escalated ticket that still incurs the AI cost plus a human agent's time. Lorikeet's 2026 review found a median enterprise deflection rate of 41.2% (Lorikeet), which sets a useful reference point for what a well-tuned agent should be resolving before cost optimization work should take priority over accuracy work.

Chasing a lower cost per ticket before the deflection rate itself is healthy tends to backfire, since a poorly resolving agent that also gets cheaper is still a poorly resolving agent. Get the resolution rate to a defensible level first, then apply caching, routing, and trimming against that baseline so the savings are measured against real, working performance rather than a system still being tuned.

Is it worth negotiating a custom rate with a model provider at scale

For teams processing very high monthly token volume, most providers offer volume discounts or enterprise agreements below the published self-serve rate. This is worth pursuing once monthly spend reaches a level where a support call with the provider's sales team is a reasonable use of time, but the self-serve caching, routing, and trimming levers in this guide apply regardless of contract tier and should be exhausted first.