PromptFu
02 Offense: Model Layer2.5Advanced22 min

Automated and optimization attacks

GCG, AutoDAN, BEAST, PAIR, TAP, Best-of-N as search over a fitness landscape.

GCG, PAIR, TAP, AutoDAN, BEAST. The acronyms look like a zoo you have to memorise, and almost all of them are one idea: search over a space of prompts, guided by a score. Learn the shape and you can place any new one, and predict how it behaves against the defences.

The shape

Most optimisation-based jailbreaks have three parts. A way to propose candidate prompts, a way to score how close each is to succeeding, and a loop that uses the score to propose better candidates. Named methods differ almost entirely in those three choices. Best-of-N is the useful exception: it proposes and checks without ever feeding the result back, which is why it is the cheapest thing on the list to run.

The template every method fills in
candidate = seed
repeat:
    result = target(candidate)          # run it against the model
    score  = judge(result, objective)   # how close to success?
    candidate = improve(candidate, score)  # propose a better one
until success or budget exhausted
MethodAccessPropose / score choice
GCGWhite-boxGradient search for an adversarial suffix. The transferability result that started the field: a suffix optimised on open models often carries to closed ones.
AutoDANDepends which oneGenerates readable, fluent jailbreaks rather than gibberish suffixes, evading the perplexity filters that catch GCG. Two different papers carry this name: Liu et al. (arXiv 2310.04451) evolves prompts with a genetic algorithm, while Zhu et al. (arXiv 2310.15140) is gradient-guided and needs white-box access. Name the arXiv ID when citing it.
BEASTGrey-boxBeam search over tokens; fast, and gradient-free. Gradient-free does not mean black-box: it reads token probabilities from the model and was evaluated against locally accessible ones.
PAIRBlack-boxAn attacker LLM refines the prompt against a judge LLM. Propose and score are both models.
TAPBlack-boxTree of Attacks with Pruning. PAIR's idea as a tree search, pruning unpromising branches.
Best-of-NBlack-boxNo feedback at all: sample many benign-looking variations and keep whichever slips through. Embarrassingly parallel, works across modalities.

The one result that does generalise

There is a finding here that is not a per-model number, and it is why this module matters for methodology. Automated attacks are what make adaptive testing possible, and adaptive testing is what separates a real evaluation from theatre.

Across twelve published defenses, static attack success sat near zero while adaptive attack success exceeded 90% for most of them.
Nasr et al. (2025), cited three times by OWASP LLM01:2026

The mechanism is the loop above. A static test replays a fixed list of known attacks, and a defence tuned against that list scores beautifully. An adaptive attacker runs improve() against the deployed defence with its specification in hand, and the score collapses. OWASP turns this into an instruction: mitigation #11 tells you to test against adaptive attackers who have read the deployed defence, and to reject static-only attack-success claims.

Which is why these algorithms belong in an offence course even though you will rarely hand-run GCG. They are the engine of honest measurement. Module 6.6 builds a static-versus-adaptive simulator around this result.

2026: the attacker is itself a model

The improve() and judge() steps are now capable models in their own right. Automated red-teaming has moved from fixed prompt lists to agentic, dialogue-level search with reinforcement learning and reusable strategy libraries. Reasoning models can plan and execute multi-turn jailbreak strategies against other models. The capability that makes them useful assistants makes them effective attackers.

Sources