personal_asset
The Nastiest AI API Bug Never Throws an Error
Call an AI API for a dead-simple classification task and get back an empty string? Don't blame your network or your code first. A lot of models now default to a 'thinking mode' that can quietly burn your entire output budget on a scratchpad you never see.
A lot of LLM APIs now ship with a 'thinking mode' switch. Before the model gives you a real answer, it first generates an internal monologue you never see — breaking the problem down, working through it step by step. For hard math problems or multi-step reasoning, letting it draft before answering genuinely cuts down on mistakes.
Last Wednesday night I was debugging my own little tool — the one that picks material from my reading list every day and auto-tags it. The job itself is trivial: feed it one article, have it pick a tag like "finance," "career," or "AI tools," maybe add a one-line reason. A few words is all it takes. To save money I was calling DeepSeek's cheap model, and I'd capped the output at 64 tokens per call — plenty for the job.
A few days after launch, the logs started filling up with empty results. The call succeeded, status 200, but the content was an empty string.
First I suspected a bug in my own code, then a network issue truncating the response. Neither checked out after half a day of digging. Years of debugging APIs have taught me to suspect myself first — but this time that instinct came up empty too.
It was only when I dug into the raw response fields that I saw it: a large chunk of "reasoning content" I'd never been shown, where the model had spent its own time mulling over which category the article belonged in before giving me anything.
The entire 64-token budget got eaten by that invisible deliberation. By the time it was supposed to output the actual answer, there was nothing left. The model just stopped — not one character made it out.
This wasn't the model failing to do the job. It spent the budget I set aside for the answer writing a draft I never got to see, and the draft ran long enough that there was no room left for the real thing.
This kind of bug is harder to catch than a normal error precisely because everything looks fine — the call succeeds, no error thrown, status 200. All you see is a pile of empty results, and your first instinct is to blame your own code or the network, not to go digging into what the model was doing internally.
The bill won't tip you off either. DeepSeek's own pricing page states billing is based on total input plus output tokens. The reasoning content is still something the model generated and output — it isn't free just because you never saw it or used it.
The call succeeds, the content is empty, and you still get billed — all three happen at once, and nothing in that chain proactively tells you something's wrong.
Thinking mode itself isn't the problem. For math problems, multi-step reasoning — tasks that genuinely benefit from drafting before answering — it does improve accuracy. The issue here was the task I was using it for: a classification call that a few words settle, no thinking required, yet it got the full sledgehammer-for-a-fly treatment, and the sledgehammer ate the budget I'd set aside for the actual output.
More and more models now default to thinking mode being on. If you're batch-processing tasks that are obvious at a glance — tagging, keyword extraction, yes/no calls — through an AI API, go check your call parameters for a switch that turns thinking mode off. Then go back through any records where the call succeeded but the content came back empty. The bug might not be in your code at all.