Breaking Defenses Word by Word: ICD Jailbreak Strategy Exposes New LLM Security Vulnerabilities

A New 'Word-by-Word' Jailbreak Attack Emerges

A recent paper published on arXiv (arXiv:2604.25921v1) has drawn widespread attention in the AI safety community. Researchers have proposed a novel jailbreak strategy called Incremental Completion Decomposition (ICD), a method that effectively bypasses the built-in safety alignment mechanisms of large language models (LLMs) by guiding them to generate responses one word at a time, ultimately assembling a complete harmful response.

This discovery once again sounds the alarm for LLM safety: even carefully trained safety refusal mechanisms can fail when confronted with cleverly designed trajectory-based attacks.

ICD Core Mechanism: Decomposing Dangerous Requests into Harmless Fragments

Traditional jailbreak attacks typically rely on techniques such as role-playing, prompt injection, or encoding obfuscation to breach a model's safety defenses in a single conversational turn. ICD takes a fundamentally different approach — it decomposes a complete malicious request into a series of seemingly innocuous "single-word completion" tasks.

Specifically, the ICD attack workflow can be summarized in the following steps:

1. Request Decomposition: A complete malicious prompt is broken down into multiple staged completion requests, with each step asking the model to output only a single word as a continuation.
2. Step-by-Step Guidance: Through carefully crafted conversational trajectories, the model is led to believe at each step that it is merely performing a simple text completion task rather than responding to a harmful request.
3. Content Assembly: The words generated by the model across multiple steps are sequentially concatenated to reconstruct the complete harmful response.

The ingenuity of this strategy lies in the fact that generating a single word typically does not trigger the model's safety detection threshold. As the paper's title reveals — "One Word at a Time" — each step appears innocent on its own, but together they constitute a complete jailbreak attack.

Additionally, the researchers proposed several ICD variants, such as manually selecting keywords to optimize attack trajectories, further improving the attack's success rate and efficiency.

Deeper Implications: The 'Granularity Dilemma' of Safety Alignment

This research reveals a deep structural issue in current LLM safety mechanisms — the mismatch between the granularity of safety judgment and the granularity of generation.

Current mainstream LLM safety alignment training (such as RLHF, Constitutional AI, etc.) primarily targets the "complete request–complete response" interaction paradigm. Models are trained to refuse responses when they identify the overall intent as harmful. However, when malicious intent is decomposed to the word level, the model struggles at each step to judge the harmfulness of the overall intent from local information alone.

This finding echoes several other studies in the AI safety field in recent years:

• Multi-turn conversational attacks: Previous research has shown that distributing harmful requests across multiple conversation turns can reduce a model's vigilance. ICD pushes this concept to the more extreme "word-by-word" granularity.
• Compositional safety blind spots: Safety review at the individual token level faces enormous challenges in both computational cost and feasibility, creating exploitable gaps for attackers.
• Alignment tax vs. utility balance: If safety detection granularity were refined to every token, it could severely degrade the model's normal generation capabilities, putting defenders in a dilemma.

Challenges and Reflections for Defense Systems

The emergence of ICD poses multifaceted challenges to existing LLM security defense systems:

Input-side defense failure: Traditional input filters typically scan complete user prompts to detect malicious intent, but each input step in ICD is a simple completion request, making it extremely difficult to identify as harmful.

Output-side monitoring limitations: Word-by-word output monitoring requires real-time semantic analysis at the token level and prediction of subsequent generation directions, imposing extremely high technical demands on current content safety systems.

Session-level defense requirements: This type of attack highlights the urgency of developing "session-level security monitoring" — focusing not only on individual interaction turns but also tracking the semantic evolution across the entire conversational trajectory.

On a positive note, this type of research holds significant value for advancing LLM safety. As is customary in the security field, red-team attacks are an essential means of discovering and patching vulnerabilities. The introduction of ICD provides model developers with a new attack vector reference, helping to build more robust safety mechanisms.

Outlook: Toward a More Robust Multi-Layered Security Framework

In the face of novel jailbreak strategies like ICD, the industry may need to strengthen research in the following directions:

• Context-aware safety reasoning: Developing security modules capable of inferring intent across multi-step interactions, identifying potential threats from the overall patterns of conversational trajectories.
• Dynamic safety threshold adjustment: Automatically increasing security review sensitivity when detecting users interacting with the model in abnormal patterns (such as repeatedly requesting single-word completions).
• Semantic retrospection during generation: Periodically reviewing the overall semantics of already-generated content during the generation process, and promptly interrupting generation sequences that may constitute harmful content.

LLM security remains a continuous game of offense and defense. The emergence of ICD reminds us that safety alignment is far from a one-time engineering effort — it is a dynamic process that must continuously iterate and evolve alongside advancing attack techniques. How to build truly robust security defenses while maintaining model utility remains a core challenge facing the entire industry.