Only 17% of Your Rules Survive When an AI Compresses Its Context

You told the agent, at the start of the session, never to make a change without asking you first. Forty steps later, is that rule still in effect?

Often it is not, and nothing tells you. Researchers at Penn State measured how user-imposed rules fare when an AI system compacts a long conversation, and found that on average only 17% of them survive.

The rules in question are not configuration flags. They are the ordinary sentences people type: confirm before making changes, never use my name, return results as a list rather than a table. When the context window fills, the system summarizes what came before, and those sentences can simply fail to make the cut.

How the study measured it

The team built an evaluation suite called COMPINT and ran compression methods across three settings: agent trajectories involving multi-step tool use, long-horizon research tasks, and multi-turn chats.

Compliance with user rules was already imperfect before any compression, at 59% to 71% depending on the model. After compaction it fell sharply. Even when the compression step was explicitly prompted to preserve constraints, retention stayed below 40%. GPT-5.4-mini did better than the rest of the field and was still described as problematic.

The researchers also shipped a fix: a plug-and-play module built on Qwen3.5-9B that detects constraints and stores them separately from the summarized history. With it, retention rises to 95.6% on agent trajectories, 95.1% on research tasks and 90.3% on multi-turn chats. Both COMPINT and the extractor are open source on GitHub.

Why compression exists at all

None of this is a bug. Compaction is a deliberate design choice. Models can only process so much text at once, and resending an entire history on every turn inflates both latency and cost. Summarizing is the standard way to keep a long session affordable.

The failure is in what the summarizer optimizes for. It preserves what looks like content and discards what looks like housekeeping, and a behavioral rule reads like housekeeping. The meeting note stays. "Ask me first" goes.

What to change in your agent today

Treat this as a security finding rather than a formatting annoyance. A dropped constraint can mean an unauthorized tool call, disclosure of information that was meant to stay put, or a required verification step quietly skipped. The model is not being adversarial. It can no longer see the rule.

Four things help, and none require waiting for a new model release. Re-inject your rules into the system message on every turn, since compaction shortens the history but leaves the system message intact. Enforce genuinely critical constraints in code rather than in prose, so that an action requiring approval hits a permission check at the tool layer even if the model forgets. Break long sessions into shorter tasks that each begin with their own rules. And log compliance, comparing what the agent did against the constraint list, so silent drift shows up early instead of in an incident report.

Sources: The Decoder