Researchers Reconstructed Prompts From Model Output

Picture the person who wrote your company chatbot's instructions. They spent a week tuning the tone, the escalation rules, the discount logic. All of it sits in a system prompt nobody outside the team was ever meant to read.

Research announced on August 12 suggests that assumption no longer holds. Teams at IIT Bombay and Adobe Research have shown that the prompt behind a language model's output can be reconstructed from the output text alone.

How the method works

They call it Previous-Token Prediction. Language models are normally trained to predict the next token in a sequence; the idea here is to train one that runs backwards, predicting what came before. The inverse model learns from synthetic data generated by the target model, then reads an output and rebuilds the input that produced it.

The results are uncomfortably good. In one case a prompt was reconstructed word for word, and the inverse model also produced six variants carrying the same meaning.

The scale involved is the part that should worry people. The inverse model was trained on Qwen-3-0.6B, a small 600 million parameter model, and still recovered prompts from GPT-4o output. The technique needs no access to the target model's weights and transfers across different architectures.

It has limits worth stating. Training the inverse model requires output generated by the target, and reconstruction needs a reasonable volume of text to work from. A single short sentence will not surrender a long system prompt. But a customer facing assistant producing thousands of responses a day supplies that volume by design.

Why this is a security problem, not a curiosity

Because a great deal of commercial logic gets written directly into prompts. Typical contents include brand voice rules, moderation policies, pricing and discount logic, routing instructions for specific question types, and sometimes a summary of internal process.

Some of that leaking would bother nobody. An instruction to answer politely and briefly is not a trade secret. But if the prompt contains a line granting up to fifteen percent discretionary discount for a particular customer segment, a competitor no longer needs to breach your systems to learn your pricing strategy. They need your chatbot's answers, which you publish for free.

A prompt is a behavior specification. Used as a safe, it turns out the door was never locked.

What to change this week

The habit this research punishes is common, because it is the fastest way to ship: pile every business rule into one long system prompt and move on.

• Move pricing logic, discount authority and business rules out of the prompt and into a service behind it. Let the service decide and the model report the decision.
• Keep contract text, customer lists and internal price tables out of the prompt entirely. Retrieve them through access-controlled calls instead.
• Answer one question honestly today: if this prompt leaked in full, what would we lose? If the answer is substantial, the architecture needs work, not the wording.
• Treat personal data written into a prompt as a processing activity with a real disclosure risk, and review it accordingly.

This was published as academic work, and the intent is to strengthen defenses. Once a method is public, though, who picks it up is no longer your decision.

Sources: The Decoder, arXiv