You can paste a system description into an LLM and get back a STRIDE analysis in 30 seconds. A full threat list, categorised by type, with suggested mitigations. It looks thorough. It might even be thorough.

That’s the problem.

What LLMs Are Actually Good At

Start with the honest case for using AI in threat modelling, because it’s real.

Breadth coverage. A well-trained LLM has processed thousands of architecture descriptions, CVEs, and security design documents. It won’t forget to check for SSRF. It won’t skip repudiation because the session ran long. It has no blind spots born from familiarity with the system. For the common, well-documented threat categories, it’s genuinely reliable.

Lowering the barrier. Most threat modelling happens late, or not at all, because developers don’t know STRIDE and there’s no security engineer in the room. An LLM can run a first-pass analysis from a plain-English description of the system. Not a replacement for expertise — but better than nothing, and nothing is the current default for most teams.

Brainstorming attack paths. Describe a specific component and ask “what are the ways an attacker could abuse this?” You’ll get paths you hadn’t considered. Some will be irrelevant. Some won’t. The good ones are worth having.

Grunt work. Generating the initial threat list, drafting mitigation descriptions, formatting output for a doc — these are tedious but necessary. Automating them gives the humans in the room more time for the work that actually requires judgment.

Where It Fails

Here’s where it gets uncomfortable, because the failures aren’t obvious. The output looks good. The categories are right. The mitigations are sensible. And yet.

It doesn’t know your system. It knows a system like yours.

When you describe your architecture, the LLM pattern-matches to similar architectures it’s seen and generates threats that are plausible for that pattern. That’s not the same as analysing what you actually built. The subtle deviation from the pattern — the one that creates the real risk — is exactly what it will miss.

It’s confidently wrong on edge cases.

For well-documented, common attack classes, LLMs are reliable. For unusual configurations, niche protocols, or novel interactions between components, they generate plausible-sounding threats that may not apply, and miss real ones that don’t fit the pattern. The confidence is uniform. The accuracy isn’t.

It can’t reason about business context.

What data is actually sensitive in your system? Who are your real adversaries — a disgruntled employee, a nation-state, an automated credential-stuffing bot? What’s the actual impact of this component being compromised, given the business it supports? These questions require context that doesn’t live in the system description. A threat model without them produces technically correct findings that may be prioritised entirely wrong.

It creates false completeness.

A long, well-formatted threat list feels thorough. The instinct is to tick it off and move on. But length is not coverage, and format is not quality. A 40-item threat list generated in 30 seconds may have missed the two things that actually matter. This is the most dangerous failure mode — not wrong output, but unwarranted confidence in output that looks right.

A Workflow That Actually Works

Use the LLM for what it’s good at. Keep humans in the loop for what it isn’t.

flowchart TD
    A["Describe system to LLM
components, data flows, trust boundaries"] --> B[LLM generates first-pass threat list]
    B --> C[Human review: filter, challenge, fill gaps]
    C --> D{For each threat}
    D --> E["Apply business context
who are real adversaries?
what data actually matters?"]
    E --> F["Score and prioritise
using DREAD or similar"]
    F --> G["Document decisions
including what was rejected and why"]
    G --> H["Validate with domain expert
or pentest for high-severity findings"]

What to give it. The more specific your input, the better the output. Don’t just say “a web app.” Describe the components, the trust boundaries, the data flows, what’s sensitive, and who you’re worried about. A good prompt is most of the work.

What to challenge. When the LLM produces a threat, ask: does this actually apply to our configuration? Do we have the preconditions it assumes? What’s the realistic impact given our specific context? Treat the output as a starting point for the conversation, not the output of it.

What to never trust it on. Final risk decisions. Compliance claims (“this architecture satisfies requirement X”). Anything where being wrong has real consequences and the LLM has no way to know what it doesn’t know. Those stay with humans.

When to bring it in. Early is better. An LLM brainstorm at the design stage costs nothing and can surface issues before they’re built in. Using it to audit existing architecture is still useful, but you lose the cheap intervention window.

The Honest Take

AI doesn’t change what threat modelling is — it changes how fast you can do the parts that don’t require judgment. Used well, it’s a force multiplier: the initial list is generated, the obvious gaps are covered, and the security engineer can spend their time on the analysis that actually requires knowing the business.

Used badly, it produces a polished document that looks like a threat model and provides none of the protection. The false confidence is the real risk, not the tool.

The four questions from the threat modelling primer still apply. What are we working on? What can go wrong? What are we going to do about it? Did we do a good job? An LLM can help with the second question. The other three need humans.