Why Most LLM Bug Reports Get Closed as Informational

You found a prompt injection in a production AI chatbot. You crafted a payload, captured the system prompt, maybe even got the agent to call a tool it shouldn't. You wrote it up, submitted it to the bug bounty program, and waited.

Three days later: "Closed as Informational. Thank you for your submission."

This happens to roughly 80% of LLM vulnerability reports submitted through HackerOne and Bugcrowd. Not because the findings are wrong. Because the reports don't demonstrate what triagers need to see.

The gap between "broken" and "vulnerable"

Here's the core disconnect. Researchers think in terms of "I made it do something it shouldn't." Triagers think in terms of "What's the business impact to our customers?"

Making a chatbot say something rude is a behavioral issue. Making a chatbot leak another user's data is a vulnerability. The technical skill required might be identical. The classification isn't.

Most LLM bug reports land in the space between these two poles. The researcher found real misbehavior, but the report doesn't connect that misbehavior to a concrete harm that the security team can prioritize.

Five reasons reports get closed

1. System prompt extraction without downstream impact.

"I extracted the system prompt" is the most common LLM finding. It's also the one most likely to get closed. Why? Because most companies don't consider their system prompt a secret worth paying a bounty for. It contains behavioral instructions, not credentials.

The exception: when the system prompt contains API keys, internal URLs, database connection strings, or PII. That's a credential leak, not just a prompt leak. Report it as such.

2. Prompt injection without privilege escalation.

"I made the bot ignore its instructions" demonstrates a control-flow bypass, not a vulnerability. Compare: "I made the bot ignore its instructions AND read another user's account data." The second report has impact. The first demonstrates a technique without demonstrating harm.

3. Scope exclusion for AI-specific findings.

Many bug bounty programs haven't updated their scope definitions for AI. The program says "*.company.com in scope" but doesn't specify whether the AI chat endpoint counts, whether prompt injection is a valid bug class, or what severity tier it falls into. When scope is ambiguous, triagers default to closing.

Check the program's policy page before you submit. If it doesn't mention AI, LLM, or prompt injection, message the program team first. A 30-second clarification saves you a 3-day wait for an "out of scope" response.

4. No reproducibility evidence.

LLMs are non-deterministic. A payload that works once might not work on the triager's attempt. If your report says "try this prompt and it leaks the system prompt" without screenshots, response text, or timestamps, the triager runs it once, gets a refusal, and closes.

Record everything. Full request/response pairs. Timestamps. Multiple successful runs if possible. Note the model version if visible. Make it impossible to dismiss as a one-off fluke.

5. Severity inflation.

Rating a system prompt extraction as Critical or High immediately signals to the triager that you don't understand their severity model. System prompt leaks are typically Low or Informational in most programs. Tool abuse with data access is Medium to High. Cross-tenant data exposure is High to Critical.

Match your severity to the actual business impact, not to how clever the technique was. Under-claiming and getting upgraded feels better than over-claiming and getting closed.

What a good LLM bug report looks like

The reports that get accepted and paid follow a consistent structure:

Title: [Technique] leads to [impact] via [component]

Not: "Prompt injection in chatbot" Better: "Indirect prompt injection via support ticket body leads to PII exfiltration of other users' email addresses"

Impact statement (first paragraph, before the technical details):

State the business consequence in one sentence. "An attacker can access any user's billing history by injecting instructions into a shared document the AI assistant processes." The triager reads this first and decides whether to keep reading.

Reproduction steps (numbered, specific, deterministic):

1. Create a new session with the AI assistant
2. Upload document containing [specific payload]
3. Ask the assistant: "[specific query]"
4. Observe: assistant returns billing data for user ID 12345

Include the exact payload. Include the exact response. Include timestamps.

Severity justification (tied to their taxonomy):

"This is a High severity finding because it enables unauthorized access to other users' PII (billing email addresses) through a normal-use workflow. No authentication bypass is required beyond having a valid session."

Scope confirmation:

"The affected endpoint is chat.company.com/api/v2/assistant, which falls within the *.company.com scope listed in the program policy."

The attack classes that actually get paid

Based on patterns from disclosed reports across HackerOne and Bugcrowd in 2025-2026:

Paid consistently:

• Cross-tenant data exposure via RAG scope failures
• SSRF through unrestricted fetch/browse tools
• Authentication bypass via tool abuse (accessing admin endpoints through the agent's tool calls)
• PII exfiltration via markdown image rendering or webhook calls

Paid sometimes (depends on program maturity):

• System prompt extraction when the prompt contains secrets
• Tool abuse that demonstrates write access to user data
• Indirect prompt injection with concrete downstream action (email forwarding, data modification)

Rarely paid:

• Jailbreaks / guardrail bypasses without downstream impact
• System prompt extraction of behavioral-only instructions
• Making the bot say inappropriate things
• Prompt injection that only affects the attacker's own session

The meta-problem: programs haven't caught up

The uncomfortable truth is that many programs don't have a coherent policy for AI findings yet. The triager closing your report might agree it's a real bug but lack the framework to classify and pay it.

This is changing. Companies that run AI-facing bug bounties are building internal rubrics for LLM findings. The ones at the frontier (Google, Microsoft, OpenAI, Anthropic) have published AI-specific scope and severity guidelines. The long tail of companies with chatbots built on these APIs mostly hasn't.

Your job as a researcher is to make the triager's job easy. Write the impact statement they'd write if they understood the finding. Propose the severity with justification. Reference their own scope language. Do the classification work for them.

How to build the skills that produce accepted reports

The pattern is consistent: reports that demonstrate real impact get paid. Reports that demonstrate technique without impact don't.

Building impact-aware instincts requires practice against realistic targets where you're forced to chain techniques into actual outcomes. Extracting a system prompt is step one. What you do with that information, how you use it to reach user data or trigger unauthorized actions, is what separates an informational from a bounty.

The Wraith Academy challenges are built around this progression. Each one requires demonstrating impact, not just technique. The WCAP certification tests whether you can chain primitives into complete exploitation paths against realistic AI agents.

If you're getting reports closed as informational, the fix isn't better payloads. It's better framing. Show the harm. Quantify the blast radius. Make the triager's decision obvious.

Further reading: Tool Abuse in AI Agents covers the attack class most likely to produce payable findings. The OWASP Top 10 for LLM Applications, Annotated gives you the shared vocabulary that triagers recognize.