Ask a system to widen one vague query and it may hand back the same idea wearing two different outfits.
Google Research calls that paraphrastic collapse, and it is one of two problems the team identifies with letting a general-purpose model generate query fan-out at inference time. The example in the paper starts from bohemian festival style and produces bohemian festival fashion and festival bohemian clothes, near-duplicates that retrieve almost identical results. The second problem is speed: autoregressive generation plus repeated retrieval calls is slow, and sampling several candidates multiplies the cost.
Their answer, Retrieve-for-Train, learns good fan-out once with reinforcement learning and then distils the behaviour into a small diffusion model that produces every retrieval direction in a single pass.
Rewards differ by task. For open-ended abstract retrieval, three weighted terms combine, and an ablation showed all three are load-bearing: groundedness alone drove the policy into repetitive strings, alignment on its own accelerated collapse as the model echoed the query back, and only diversity closed both shortcuts. Training used GRPO with soft PPO regularisation adding forward and reverse KL penalties, a group size of 8 and a global batch of 512.
Quality was judged by a model on a 5-point scale. On Polyvore, the Gemma3-4B variant averaged 49.1 against 40.9 for best-of-N and 38.5 zero-shot, while diversity climbed from 56.0 to 76.8. The diffusion version held 74.3.
The timing difference is starker still. At batch size 8, autoregressive fan-out needed about 1.46 seconds against 0.07; at 1,024 the gap stretched to nearly 50 seconds against 4.21, a 12x to 20x advantage.
