Whether It Answers Correctly Depends on Token Positioning
DeepSeek’s erratic “genius-or-blunder” behavior has been caught red-handed by ByteDance’s Seed team.
For the exact same question, with zero changes to the prompt itself, simply prepending a few irrelevant characters causes the model to suddenly fail?
And this isn’t just a random fluke.
Researchers from the Seed team discovered that DeepSeek-V4’s performance actually exhibits periodic fluctuations every 4 tokens depending on the input position of the information.