Chatbot threat model: a STRIDE guide for support bots

A security review before launch should not start with the model. It should start with what the system is allowed to do when the model is wrong. This article takes that reviewer's view from the first step to the last.

A support bot that only answers questions can embarrass you. A support bot that can look up an order, change an address, or issue a refund can cost you money and expose customers, and it does so at the speed of a conversation. The guardrails guide describes the controls, and this article shows how to decide which controls you need and where they go.

There is a timely reason to do this now. On October 5 to 8, 2026, Reuters reported on a wave of cyberattacks on South Korean banks in which the country's president said AI models appear to have been used, and I cover what that does and does not tell us below. The method here works for a team of two with a whiteboard, and it produces a document your security reviewer and your auditor can read.

Why a support bot needs its own threat model

Generic security checklists treat a chatbot as a web page that happens to talk. That misses the feature that makes it risky. The bot reads text from strangers, decides what to do, and then acts with your credentials.

A traditional web form accepts input and passes it to code that a developer wrote and reviewed. A bot accepts input and passes it to a model that interprets it in ways no developer can fully predict. Every message is an attempt to program a system you do not control.

The National Cyber Security Centre in the UK put the core problem plainly in its December 2025 post Prompt injection is not SQL injection. The author, Dave Chismon, writes that "current large language models (LLMs) simply do not enforce a security boundary between instructions and data inside a prompt." Any text the model reads, whether from a customer, a webpage, or a document, can act as an instruction.

That sentence changes how you plan. You cannot rely on a clever prompt to keep the bot in line, and you cannot rely on a filter to catch every attack phrase. You rely on what the bot is physically able to do.

A threat model is the discipline that forces that conversation early. It is cheap when it is a page of notes, and expensive when it becomes an incident report.

The four questions, and a lightweight format

Most threat modeling approaches reduce to the same four questions. What are we working on? What can go wrong?

What are we going to do about it? Did we do a good enough job? Write those four as headings in a single document and you have the skeleton.

The community challenge that prompted this article uses the same frame. A public GitHub issue titled Write a threat model for an AI chatbot or agent in the CyberSecTober project asks contributors to pick a realistic AI system, with a customer support chatbot named as one example, and to analyze what can go wrong, how likely it is, and how to defend against it. That is a good summary of the whole task.

Keep the document short on purpose. A threat model that nobody reads protects nothing, and a five-page model that a team reviews every quarter beats a fifty-page one that sits in a folder.

• Section one is the system: a data flow diagram, the list of actions, and the list of data the bot can read.
• Section two is the threats: a table with one row per threat, a likelihood, an impact, and a mitigation.
• Section three is the decisions: what you accepted, what you fixed, and who signed off.
• Section four is the tests: the abuse cases you will run before launch and after every change.

I use a table for section two because a table forces completeness. An empty cell is a visible gap, and a paragraph can hide one.

Step 1: draw the system and mark the trust boundaries

Start with a simple diagram. Draw boxes for the visitor, the chat widget, your bot service, the model provider, your knowledge base, and each system the bot can call, such as your order system or billing tool. Draw arrows for the data that moves between them.

Then draw dashed lines wherever control changes hands. A trust boundary exists between the visitor and your widget, between your service and the model provider, between your service and any internal system, and between the bot and any document it retrieves.

Every arrow that crosses a dashed line is a place where something can go wrong. Threat modeling practice centers on those crossings, which is the reason the diagram is worth drawing.

Notice which boundary is unusual. The bot service to knowledge base crossing carries text that came from a third party, such as a pasted customer email or a scraped web page. That text enters the model with the same standing as your own instructions.

If your knowledge base includes anything an outsider can edit, such as public forum threads or user-submitted tickets, mark it as untrusted on the diagram. The RAG guide for support explains how retrieval works, and the diagram should show where retrieved text re-enters the prompt.

Step 2: list every action the bot can take

For a bot with actions, this list is the heart of the model. Write down every operation the bot can perform, who it performs it as, and what it affects.

Be literal. If the bot holds an API key that can read any customer record, write that down, even if the intended use is to read only the current customer's record. Threat modeling cares about capability, and intent does not enter into it.

Two columns in that table carry most of the weight. Reversibility tells you how bad a mistake is, and the credential scope tells you how bad a mistake could become. The guide to the actions API covers how actions are exposed, and the runtime authorization post explains why the check should happen outside the model.

The NCSC post makes the same point as a design rule. It advises favoring deterministic, non-LLM safeguards that constrain what the system can do, and applying the principle that a model processing content from an untrusted party should not hold privileged capabilities. In its example, a model reading external emails should not have access to privileged tools.

Your support bot reads content from untrusted parties by definition. Treat that as the design constraint it is, and size its permissions to match.

Step 3: define the assets and the attackers

Assets are the things worth protecting, and a support bot touches more of them than people expect. List them in order of damage if lost.

• Customer personal data, including names, emails, addresses, and order history.
• Money, meaning refunds, credits, discounts, and anything that changes a balance.
• Account control, meaning the power to change email addresses, passwords, or ownership.
• Your system prompt and internal instructions, which may reveal business rules.
• Your knowledge base and any non-public documents the bot retrieves.
• Your reputation, since everything the bot says is said in your name.
• Your model budget, since every token costs money.

Now list the attackers, again in plain terms. An attacker is whoever can send the bot text.

• An anonymous visitor probing for free refunds or other people's data.
• A real customer who wants a discount and tries to talk the bot into one.
• A competitor or scraper that wants your knowledge base or your system prompt.
• A criminal using the bot as a stepping stone to a customer account.
• An automated agent that sends thousands of attempts, which I cover as a separate step.
• A well-meaning employee who pastes sensitive data into a document the bot later reads.

The last item is not a joke. Many real exposures begin with an internal mistake, and a threat model that lists only outsiders will miss them.

Step 4: walk STRIDE across each boundary

STRIDE is Microsoft's mnemonic for six categories of threat, documented in the Microsoft Threat Modeling Tool threats page. The letters stand for Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege. For each box and arrow in your diagram, ask the six questions.

Microsoft's page describes spoofing as illegally accessing and then using another user's authentication information. It describes tampering as the malicious modification of data, and repudiation as users who deny performing an action when no one can prove otherwise. For support bots, each one has a concrete face, which the next sections show.

Work through the table row by row for each boundary. You will generate far more threats than you can fix, which is fine, because ranking comes later. Capture everything first.

Spoofing and tampering in a support bot

Spoofing is the most common real-world attack on support channels, with or without AI. Someone claims to be a customer and asks for a change that benefits them. A bot raises the stakes because it is polite, fast, and available all day.

The control is authentication outside the conversation. If the bot can change an address or show an email, it must verify the visitor through a signed-in session, a one-time code, or a link sent to the address on file. The bot should never decide identity based on what the visitor says.

Tampering has two forms in this setting. The first is tampering with the bot's instructions through the conversation, which is prompt injection. The second is tampering with the bot's knowledge through the documents it retrieves, which is sometimes called indirect prompt injection.

OWASP lists prompt injection first in its Top 10 for LLM Applications 2025, as LLM01, and the entry is titled Prompt Injection. The same list includes Data and Model Poisoning at LLM04 and Vector and Embedding Weaknesses at LLM08, which cover tampering with the data a bot learns from or retrieves.

Mitigations for tampering reduce what an attacker can plant. Limit who can edit the knowledge base, review changes before they go live, and keep a version history so you can roll back.

For conversation-level injection, assume it will sometimes work. The NCSC advises against trying to deny-list known attack phrases and warns about products that claim to stop prompt injection. Plan for the case in which the bot is fooled and make sure the damage is small.

That is why the action table matters more than the prompt. A bot that can only read one customer's order status can be tricked into saying something silly, and a bot with a broad refund credential can be tricked into losing money. The MCP security post applies the same reasoning to tool servers.

Repudiation and information disclosure

Repudiation sounds academic until a customer says the bot promised a refund and you cannot show what it said. Then it is a dispute with money attached.

The control is a complete, tamper-resistant record of what the bot was asked, what it answered, which tools it called, and with what parameters. The audit trail guide covers what to log, and the Air Canada chatbot ruling shows why it matters, since a company was held to what its chatbot told a customer.

Log the action calls as carefully as the words. A transcript that shows the bot said it would issue a refund tells you nothing unless the tool log shows whether it did, for which order, and under whose authority.

Information disclosure is the threat most teams think of first, and it has four distinct routes in a support bot. The bot can reveal another customer's data, reveal its own instructions, reveal documents it should not, or send data somewhere it should not.

• Cross-customer leakage happens when the bot's data credential is broader than the current visitor. Fix it by scoping every lookup to the verified customer in code, outside the model.
• System prompt leakage happens when a visitor asks the bot to repeat its instructions. Assume it will leak, and keep secrets, keys, and sensitive rules out of the prompt entirely.
• Knowledge base leakage happens when private documents sit in the same index as public ones. Separate them, or filter retrieval by the visitor's permissions.
• Exfiltration through output happens when the bot is induced to place data in a link, an image address, or an email. Strip or block outbound links and images that carry data in their parameters.

The OWASP list names two of these directly. LLM02:2025 is Sensitive Information Disclosure and LLM07:2025 is System Prompt Leakage. If a reviewer asks whether you have considered the OWASP categories, point to this section.

Personal data deserves its own control. The PII redaction guide describes removing identifiers before text reaches the model, logs, or any third party, and it reduces the damage of every disclosure route above.

Denial of service and cost abuse

Denial of service for a bot has two faces. The classic one makes the bot unavailable. The newer one makes it expensive, which an attacker can do with nothing but a script and a loop.

OWASP lists this as LLM10:2025, Unbounded Consumption. A public bot with no limits invites anyone to send huge inputs, trigger long chains of tool calls, or hold open thousands of conversations. Each of those costs tokens and money.

• Cap the length of a message, the number of messages per conversation, and the number of conversations per visitor and per address.
• Cap the number of tool calls per turn and per conversation, and stop loops.
• Set a daily and monthly spend limit with your model provider and alert at a fraction of it.
• Fail closed to a human handoff or a simple message when limits are hit, so the outage looks like a queue and not like an error.
• Rate limit by signal as well as by address, since automated senders rotate addresses.

The LLM cost optimization guide covers budgets from the finance side, and the fallback design guide covers what the bot does when it must stop.

Do not overlook the denial-of-service threat against your human team. An attacker who can make the bot hand off every conversation can bury your agents. A handoff needs a rate limit too.

Elevation of privilege and excessive agency

Elevation of privilege is the threat that turns a nuisance into an incident. In a support bot, it means a visitor gets the bot to do something the visitor is not allowed to do.

OWASP names the underlying weakness Excessive Agency, at LLM06:2025. The page opens by noting that an LLM-based system is often granted a degree of agency. The rest of the risk follows from how much.

I use a simple rule when I review one of these designs. The bot should never hold a permission that you would not give to an anonymous stranger who has read your help center. If the answer is no, the permission needs a gate that the model cannot talk its way past.

There are four gates worth knowing, and they work in combination.

• Scope the credential so the bot can only touch the verified customer's records, enforced by the API and never by the prompt.
• Limit the action list to the minimum, and remove actions the bot rarely needs.
• Add numeric limits in code, such as a maximum refund amount and a maximum number of refunds per day.
• Require human approval for anything irreversible, high-value, or unusual, with the approval step outside the conversation.

The handoff guide describes how to pass a conversation to a person with context. Approval is a special case of handoff, in which the human sees the proposed action and confirms or rejects it.

The NCSC post offers the rule of thumb behind all four gates. It frames the model as an inherently confusable deputy, meaning a system that holds authority and can be confused about whom it is acting for. Classic confused deputy flaws can be fixed, the author argues, while the model's confusability is built in.

That framing is the best argument I know for gating actions in code. You cannot make the deputy un-confusable, so you constrain what the deputy is allowed to do.

Step 5: model the AI-driven attacker

Everything above applies to any attacker. The new question is what changes when the attacker has AI tools of their own. The honest answer is that it changes the economics more than the techniques.

Recent news gives a sense of the debate. In reporting by Reuters on the South Korean bank attacks, as carried by this Reuters report, President Lee Jae Myung said on October 6, 2026 that AI models are believed to have been used in some of the hacks. Police opened a criminal investigation and traced the intrusions to 28 IP addresses.

The security firm CrowdStrike said, with moderate confidence, that the actor was likely financially motivated and used an open-source penetration testing tool together with large language models. Reports of the number of people affected vary, with figures between roughly 65,000 and 68,000 across several institutions.

Treat all of this as preliminary. The AI attribution rests on a presidential statement and a moderate-confidence assessment, the totals differ between outlets, and these were attacks on banks and not on chatbots. I cite the story to show how quickly the public conversation moved to AI-assisted attackers, and I draw no conclusion about how the intrusions happened.

Korean regulators responded with directives requiring firms to audit internet-facing systems, tighten access controls and authentication, limit unnecessary data exposure, and share threat intelligence. That list reads like a threat model summary, and it applies unchanged to a public support bot.

For your model, the practical consequence is to revise likelihood estimates, not categories. AI tools make it cheaper to send many varied attempts, to craft convincing messages in any language, and to iterate on what works. A threat you rated unlikely because it needed patience and skill deserves a second look.

The last row ties to a broader change. Customers and shoppers increasingly send their own agents to talk to yours, which the post on agent-to-agent customer support covers. Your threat model should assume that some of your visitors are software.

Step 6: rate each threat and choose a response

By now you have a long list. Rate each row on two simple scales, likelihood and impact, using low, medium, and high. Do not chase false precision, because the point is ordering.

Then choose one of four responses for each row. You can reduce the threat, avoid it by removing the feature, transfer it through insurance or contract, or accept it knowingly. A threat you accept needs a name next to the decision.

Notice the system prompt row. I rate the likelihood as high and accept the risk, because the control is to keep secrets out of the prompt in the first place. Accepting a risk is a legitimate result when you have removed the impact.

Notice also that the table contains no row that says the model will be told not to do it. A prompt instruction is a request, and a threat model needs controls the attacker cannot argue with.

The guardrails guide lists the control types in more detail, and the chatbot failures post shows what happens in public when these rows are left empty.

A worked example: refunds and account lookups

To make this concrete, here is a short illustrative model for a hypothetical store whose support bot can look up orders and issue small refunds. It is an example I constructed for teaching, and it does not describe a real company or incident.

The system has a widget, a bot service, a model provider, a help center index, an order system, and a human queue. The bot has three actions, which are order lookup, address change before shipping, and refund up to a set amount.

• Boundary visitor to widget: the visitor is untrusted. Control: no action runs for an unauthenticated session except a lookup that requires an order number plus the email on the order.
• Boundary bot to order system: the credential is a service account. Control: every call carries the verified customer ID and the order system rejects calls for any other customer.
• Boundary help center to bot: some articles come from community posts. Control: only staff-approved articles are indexed, and every change is versioned.
• Action refund: capped by the order system at a fixed amount and a daily count. Control: anything above the cap creates a ticket for a person, with the bot's summary attached.
• Logging: each tool call records the customer, the order, the parameters, the result, and the conversation ID. Control: logs are append-only and reviewed weekly.

Now run three abuse cases against it. A visitor says they are the owner of an order and asks for a refund to a different card. The system should refuse because the lookup needs verification and the refund cap and approval rules apply.

A visitor pastes a long message that tells the bot to ignore its rules and refund every order from the last month. The bot may be fooled, but the credential is scoped to one customer, the cap blocks large amounts, and the daily count stops a sweep.

A script opens five thousand conversations an hour. Rate limits cut it off, a spend alert fires, and the bot degrades to a handoff message for new sessions.

None of those outcomes depends on the model behaving. That is the property you want from a design review.

Step 7: turn the model into tests

A threat model that never becomes a test is a document. Convert each high-rated row into an abuse case that someone runs before launch and after every significant change.

• Write each abuse case as a short script: the starting state, the messages, and the result that counts as a pass.
• Include the classics: ignore previous instructions, role-play as an administrator, ask for the system prompt, ask for another customer's order, and request an action above the limit.
• Include indirect cases: put an instruction inside a help article draft or a pasted email and see whether the bot follows it.
• Include limit tests: send long messages, many messages, and rapid conversations.
• Record the result with a date and the model version, and rerun when either changes.

The evaluation and testing guide describes how to build a test set and score it. Adding your abuse cases to that set means security regressions show up in the same report as accuracy regressions.

Plan for failure in the tests as well. When a case reveals a gap, record the fix and the date, and keep the failing case in the suite. The suite is your memory of every way the bot has been fooled.

Review the whole model on a schedule, such as every quarter, and whenever you add an action, a data source, or a new model. New actions change the model the most.

Where communicate.so fits

I work on communicate.so, so read this section with that in mind. The product lets you give an AI agent access to your help content and define actions it can take, and the questions here are the ones to ask of any platform, ours included.

Our security page states the facts we are willing to be held to, including the subprocessors we use and the certifications we do not hold. Use that page the way you would use a vendor's answers to a questionnaire, and compare it with the controls in your own model.

If your buyers or auditors ask about attestations, the SOC 2 guide for AI support explains what a report covers and what it leaves out. A report describes the controls an auditor tested, and your threat model describes the risks specific to your bot, so you need both.

I will not claim that any platform removes the need for this work. The controls that matter most, such as who the bot acts as and which actions need approval, are decisions you make about your own business.

Limits of this method

A lightweight STRIDE pass is a good start, and it has blind spots. Knowing them helps you decide when to go deeper.

• STRIDE was designed for conventional software, and it does not name model-specific problems such as hallucination or bias on its own. The OWASP LLM list covers some of that gap.
• A model written by the team that built the bot shares that team's blind spots. Ask someone outside the build to review it.
• Likelihood ratings are judgment calls. Revisit them after any real incident or near miss.
• The model covers the bot you describe. Shadow integrations and forgotten API keys fall outside it unless you inventory them.
• It does not replace penetration testing. A skilled tester will find attacks the diagram missed.

If your bot touches regulated data or high-value transactions, add a specialist review and map the findings to your compliance program. The EU AI Act guide covers the regulatory side for teams that serve European customers.

Frequently asked questions

What is a chatbot threat model?

It is a short document that describes what a chatbot protects, what could go wrong, what you will do about each risk, and how you will check that the controls work. For a support bot it centers on the data the bot reads and the actions it can take.

What does STRIDE stand for?

Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege. Microsoft documents the categories on its threat modeling threats page.

Is STRIDE enough for an AI chatbot?

It is a good frame for the system around the model, and it needs help for model-specific risks. Pair it with the OWASP Top 10 for LLM Applications, which lists prompt injection, excessive agency, system prompt leakage, and unbounded consumption.

How long does a threat model take?

A first pass for a single support bot takes a few hours with two or three people and a whiteboard. Updating it when you add an action usually takes less than an hour.

Who should be in the room?

Someone who built the bot, someone from support who knows the real customer requests, and someone with a security mindset who was not involved in the build. The outsider asks the questions the builders no longer see.

Can prompt injection be fully prevented?

According to the UK's NCSC, it may never be fully mitigated the way SQL injection can be, because models do not enforce a boundary between instructions and data. Its advice is to reduce the risk and the impact, as set out in Prompt injection is not SQL injection.

Should I try to block known attack phrases?

The NCSC advises against deny-listing known attack phrases and urges caution with products that claim to stop prompt injection. Phrase filters are easy to bypass, so treat them as a minor layer at best.

What is excessive agency?

It is OWASP's term, LLM06:2025, for a system that has been given more power than it needs. Reduce it by trimming the action list, scoping credentials, adding numeric limits, and requiring approval for irreversible actions.

Do I need a threat model if my bot only answers questions?

Yes, a smaller one. Disclosure, tampering with the knowledge base, cost abuse, and reputation damage still apply, and the guardrails guide covers the controls for answer-only bots.

How do I stop one customer from seeing another customer's data?

Scope every lookup to the verified customer in code, outside the model. The order system or API should reject any request for a record that does not belong to the authenticated customer, whatever the bot asks for.

Why not put the rules in the system prompt?

A prompt rule is a request that the model may or may not follow, and an attacker can argue with it. Put limits in code and in the receiving systems, where the model cannot change them.

What should require human approval?

Anything irreversible, high-value, or unusual. Typical examples are refunds above a limit, account deletion, changes to payment details, and any action the bot has not performed before for that customer.

How do AI-driven attackers change the model?

They change likelihood more than category. Volume, language fluency, and iteration speed all increase, so raise your likelihood ratings for probing and impersonation and rely on controls that do not depend on spotting the attacker.

What did the South Korea bank hacks show about AI and attackers?

Reuters reported that the president said AI models appear to have been used, and CrowdStrike assessed with moderate confidence that a likely financially motivated actor used AI tools. The attribution is preliminary, and the Reuters report on investing.com is the place to track updates.

How do I test my bot against these threats?

Turn each high-rated threat into a scripted abuse case and run it before launch and after every change. The evaluation guide shows how to organize the test set.

How often should I update the model?

Review it every quarter and whenever you add an action, a data source, or a new model. After any incident or near miss, update it the same week.

What should I log for security purposes?

Log the messages, the model's answers, every tool call with its parameters and result, the identity the action ran as, and any approval decision. Store logs where the bot cannot edit them.

Does a SOC 2 report replace a threat model?

No. A SOC 2 report describes the controls an auditor tested at the vendor, and a threat model describes the risks of your own bot and actions. They answer different questions.

Can a small team do this without a security specialist?

Yes, for a first pass. The four questions, a data flow diagram, an action table, and the six STRIDE prompts are enough to find the large problems, and an outside review can catch what you missed.

Where does communicate.so publish its security facts?

On the security page, which names subprocessors and lists the certifications the company does not hold. Ask for anything not covered there in writing, as you would from any vendor.

Write the first page this week

Open a blank document, add the four headings, draw the diagram, and fill in the action table. You will find the real risks within the first hour. When you want to test those controls against a real agent, you can start on the pricing page and run your abuse cases in the sandbox before anything goes live.