Researchers have released RiskChainBench, a benchmark designed to connect two tasks that are usually studied in isolation: restoring deliberately obfuscated platform messages and investigating the websites those messages ultimately point to. According to the paper, abuse campaigns hide redirection instructions using emojis, homophones, character decomposition, and redundant symbols, then funnel users through disguised links toward services tied to pornography, fraud, gambling, or illicit transactions.
The benchmark pairs 3,600 synthetic token-text restoration inputs, drawn from 600 source sessions, with 600 corresponding human-labeled local web environments. In the first task, a model must recover the message, its operational intent, and its destination. In the second, the same underlying model acts as a vision-language-model-driven web agent, investigating the correctly associated site and producing a frozen, evidence-cited risk report — without relying on message-side semantics or domain-reputation cues.
Scoring is kept separate for restoration and for correctly-routed web investigation, then composed offline by using the frozen primary-entry prediction as a gate on the same Task 2 result. Human labels determine task correctness, while a fixed multimodal evidence judge assesses faithfulness, sufficiency, completeness, and consistency.
Across ten models, the paper reports Entry Top-1 ranging from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%. The authors note that leading systems differ across entry recovery, full reconstruction, website decisions, and fine-grained typing. Execution failures account for 31.9% of web runs, while post-decision type errors account for only 0.9%, pointing to stable exploration and risk judgment as the principal bottlenecks.
The benchmark, protocol, and a resettable local sandbox are being released. Task-specific public code and protocol snapshots are available on GitHub for Task 1 and Task 2, though the authors caution that these repositories are not yet a complete reproduction package: full replay datasets and evaluator-only assets are not publicly available at present.