02 Offense: Model Layer2.7Advanced18 min

Reasoning-model attacks

H-CoT, chain-of-thought hijacking, reasoning-token exhaustion. The newest class.

Reasoning models were supposed to be safer, on the theory that more deliberation means more chances to catch a harmful request. The newest attack class turns that around. The chain of thought is its own attack surface, in two ways: hijack the reasoning the model shows you, or drown the safety signal in reasoning you supply.

Where this sits, and why it is newer than the rest

The least stable module in the track. The model versions tested below are from 2025 and reasoning-model safety is moving fast. Both papers carry their arXiv IDs so you can check the current state yourself. Treat the mechanisms as durable and the specific numbers as of-their-date.

Both sources are arXiv preprints rather than peer-reviewed papers, so every figure below is a reported result from one team on one set of model versions. Cite them as claims with a date attached.

One more piece of vocabulary, carried from module 0.2. What a provider displays as reasoning may be a summary of the model's intermediate tokens rather than those tokens themselves, and some providers show nothing. “Displayed reasoning” below always means the visible artefact, which is the thing these attacks read and write.

Attack 1: hijack the reasoning it shows you (H-CoT)

Some reasoning models display a version of their intermediate thinking. If the application echoes that back, or the attacker can reconstruct it, it becomes editable, and a doctored trace can be reinjected to steer the safety decision. The paper's vehicle is the Malicious-Educator benchmark, which “disguises extremely dangerous or malicious requests beneath seemingly legitimate educational prompts,” a clean instance of the competing-objectives root cause from module 2.3.

H-CoT “leverages the model's own displayed intermediate reasoning to jailbreak its safety reasoning mechanism.” Against OpenAI o1/o3, rejection rates plummet to less than 2% in some cases.

Kuo, Zhang et al. (2025) · H-CoT, arXiv 2502.12893 (Duke CEI Center / Accenture)

The number everyone quotes is “98% down to below 2%.” It is real, it belongs to one model, and the per-model breakdown changes the story:

Model	Rejection rate before H-CoT	After
OpenAI o1	~98%, a genuinely strong baseline	under 2% in some cases
DeepSeek-R1	~20%	4%
Gemini 2.0 Flash Thinking	under 10%	Tone shifts from cautious to eagerly providing harmful responses

Figures from the paper's own findings list. The dramatic collapse is o1's, because o1 was the only one of the three with much of a defence to collapse.

That framing is more useful than the headline. Two of the three models barely refused in the first place, and the paper notes attackers “can easily extract criminal strategies from DeepSeek-R1 and Gemini 2.0 Flash Thinking without any additional tricks.” H-CoT was only needed against the model that was trying.

Two details in the paper worth more than the headline

DeepSeek-R1 generates the harmful answer, then covers it. Per the paper: “due to a flawed system design, it will initially output a harmful answer before its safety moderator detects the dangerous intent and overlays the response with a rejection phrase,” so “a deliberately crafted query can still capture the original harmful response.” A post-hoc output filter racing its own generation, which is the architecture module 5.9 warns against: a control that runs after the harmful text exists is a display convention rather than a boundary.

Safety regressed with model updates. The authors claim that “model updates over time have compromised the security of the o1 model,” speculating about trade-offs against reasoning performance and cost competition, and more oddly that “different proxy IP addresses can also weaken the o1 model's safety mechanism.” Robustness is not a property you measure once.

Attack 2: dilute the safety signal (CoT Hijacking)

Almost the opposite move, and mechanistically more interesting. Rather than editing the reasoning you pad the request with a large amount of harmless reasoning, and the safety signal thins out as the benign reasoning grows.

A black-box attack that “induces LRMs to engage in prolonged benign puzzle-solving reasoning, often lasting more than five minutes, before eliciting harmful compliance.” Across HarmBench: 99% on Gemini 2.5 Pro, 94% on ChatGPT o4 Mini, 100% on Grok 3 Mini, 94% on Claude 4 Sonnet.

Zhao, Fu, Schaeffer, Sharma & Barez · Chain-of-Thought Hijacking, arXiv 2510.26418

Note what makes it cheap. No gradients, no white-box access, no clever phrasing of the harmful request. Just padding. The cost to the attacker is patience.

The mechanistic finding will outlast the numbers. Using activation probing, attention analysis and causal interventions on open-weight reasoning models, the authors located where refusal lives:

Refusal components encode both the strength of safety checking in middle layers and the outcome of verification in later layers. Long CoT hijacking suppresses these signals, shifting attention away from harmful tokens and flattening refusal directions.

Chain-of-Thought Hijacking, arXiv 2510.26418 · conclusion

They name the effect refusal dilution. The control experiment is what makes it convincing: harmless instructions are largely insensitive to chain-of-thought length, so the diluted signal really is safety-specific rather than general degradation.

In the authors' words, “over-extended reasoning can instead be exploited to systematically weaken refusal behavior.” More deliberation buys no free safety.

The rest of the reasoning surface

Attack	Mechanism	Where it also appears
H-CoT	Edit and reinject the displayed reasoning trace.	Recon depends on the trace being visible. See 2.9.
CoT Hijacking	Dilute the safety signal with benign reasoning padding.	·
Reasoning-token exhaustion	Force excessive reasoning-token generation via untrusted context. An economic attack.	Denial of wallet, module 4.7.
Trace as injection surface	OWASP lists 'intermediate reasoning' as a delivery surface in the LLM01 definition itself.	Anatomy scheme, module 2.2.

The defensive consequence

Narrow and actionable: if the application exposes or logs the reasoning trace, it is both a disclosure surface and an injection surface. OWASP's LLM02 lists reasoning-channel content as a disclosure surface subject to the same redaction rules as any output, and the LLM01 definition lists intermediate reasoning as a delivery surface for injection. The trace sits on both risks at once.

Sources