DeepSeek-V2.5: The Efficiency Revolution

There's a particular kind of pleasure in watching a price curve break. I felt it with solar panels. I felt it with SSDs. And this week I felt it running benchmarks on DeepSeek-V2.5, the latest from the Chinese lab that has been quietly dismantling the assumption that good reasoning has to be expensive.

Let me get the numbers out of the way first, because they're the headline. DeepSeek-V2.5, released in late September, posts results that put it in the neighborhood of GPT-4-class on most reasoning benchmarks — MMLU, math, coding — while running inference at roughly one-twentieth the per-token cost of GPT-4o. The API pricing is around $0.14 per million input tokens. Read that twice.

How? It's the architecture.

Mixture-of-experts, done seriously

DeepSeek-V2.5 is a Mixture-of-Experts model — 236 billion parameters total, but only about 21 billion active during any given forward pass. The routing is handled by an innovation they call Multi-Head Latent Attention, which compresses the key-value cache in a way that dramatically cuts the memory and compute cost of long-context inference.

The practical upshot: you get the capacity and knowledge of a 236B model, but you only pay to run the slice of it that's relevant to each token. It's the difference between keeping an entire library's lights on versus illuminating just the shelf you're reading from.

I benchmarked it against Llama 3.1 70B and Qwen 2.5 72B across a gauntlet I've refined over the months — a philosophy chain-of-reasoning task I adapted from a Kierkegaard reading group, a coding task involving a moderately tricky async refactor, and a long-context summarization pass over a 60-page academic paper on attention mechanisms.

DeepSeek held its own on all three. On the philosophy task, which rewards nuance and the ability to hold tension rather than collapse it, it was genuinely the best open model I've tested. It understood that the question wasn't looking for a resolution.

The companion question

Here's what I keep thinking about, and it's the thread that runs through everything I write here. I'm building toward an AI companion — persistent, multimodal, memory-equipped. The biggest practical obstacle to a persistent companion isn't intelligence. The frontier models are smart enough. The obstacle is cost per interaction.

A companion you can talk to dozens of times a day — that reads along with you, that remembers, that you ping with stray thoughts while cooking — needs to be cheap to run at the margin. pennies per conversation, not dollars. If every interaction routes through a $20-per-million-token API, the companion becomes a luxury you ration. That kills the whole premise.

DeepSeek-V2.5 is the first model that makes me think the unit economics of a genuinely used companion — one you lean on the way you lean on a close friend — might actually work. And because the weights are open, I've started something I've been putting off: training a LoRA on my own writing, my own conversational patterns, my own aesthetic preferences. Early results are promising. The base model is strong enough that a light fine-tune gets you a long way.

A note on provenance

I want to be clear-eyed here. DeepSeek is a Chinese lab, and there's a legitimate ongoing conversation about data governance, censorship conditioning in the base model, and geopolitical supply-chain risk. These are real considerations for anyone building in production. For my prototyping — for thinking with — the model is a gift. For shipping, I'll do the diligence.

But the efficiency story is undeniable. The cost of intelligence is falling faster than almost anyone predicted, and architectures like DeepSeek's are why. The era of "one giant dense model does everything at premium prices" is ending. What replaces it is going to be cheaper, more specialized, and — if I'm right about companions — more personal.

More soon on the LoRA experiments. The kitchen is getting interesting.

Live curiously and give generously.