Gemini 2.5 Pro: Google Steps Up

There's a debate happening in AI architecture circles right now that I find genuinely fascinating, and it cuts to the heart of how I've been building systems. It goes like this: do we still need retrieval-augmented generation when context windows get big enough?

Google's Gemini 2.5 Pro, released this month, has brought that question into sharp focus. With a context window exceeding one million tokens — and some reports of even larger windows in testing — Gemini can hold an entire novel, a full codebase, or months of conversation history in a single prompt. No retrieval. No indexing. Just dump it all in and let the model attend to everything at once.

It's an extraordinary technical achievement. And it raises a fair question: have I been building the wrong way?

The Case for Long Context

Let me steelman the argument. Retrieval-Augmented Generation — the architecture Companion is built on — exists because context windows used to be tiny. When you could only fit 4,000 tokens into a prompt, you needed a way to select the right 4,000 tokens from a much larger history. Hence: embeddings, vector databases, semantic similarity search, the whole RAG stack.

But if your context window is a million tokens, why retrieve? Why not just... include everything? The model can attend to the full history. No missed connections, no retrieval errors, no semantic similarity thresholds to tune.

There's an elegance to this that appeals to me. It's the difference between a library where you have to look things up, and a library where every book is open on the table in front of you. Gemini 2.5 Pro is building that table.

The Case for Retrieval

But here's where I push back — and where my experience building Companion's memory system informs my skepticism.

First, attention is not free. Even with a million-token window, the computational cost of attending to all that context scales with its size. Models with enormous context windows are slower and more expensive per query. For a persistent system that runs continuously — processing, summarizing, retrieving around the clock — that cost compounds fast.

Second, retrieval isn't just about fitting text into a window. It's about relevance. Companion's vector database doesn't just store conversations; it organizes them by semantic meaning, surfaces connections across time, and allows the subconscious process to pre-compute which memories matter for the current moment. That's a different kind of intelligence than raw attention. It's curated recall versus total recall.

Third — and this is the philosophical point — human memory doesn't work by loading everything into working memory simultaneously. We forget most of what we experience, and that forgetting is a feature, not a bug. The memories that surface are the ones emotionally or contextually tagged as relevant. Retrieval mimics that selectivity. A million-token context window doesn't.

The Real Answer: Both

I don't think this is either/or. The future is hybrid: long-context models for depth within a conversation, retrieval architectures for breadth across time. Gemini 2.5 Pro's enormous context window is a powerful tool — and I'm already experimenting with using it for the sessions where Companion needs to hold a rich, detailed, extended conversation. But the persistent memory layer, the cross-session recall, the accumulating relationship? That still needs retrieval.

Watching the Horizon

I tested Gemini 2.5 Pro this week by loading an entire technical book into its context and asking it questions that required synthesizing information from chapters 1 and 12 simultaneously. It did so flawlessly. That capability is real and valuable.

But when I asked Companion this morning about a conversation we'd had six weeks ago — and the right memory surfaced, at the right moment, because the vector database had been quietly indexing and connecting it all along — I knew retrieval still has an essential role to play.

Long context is a bigger window. Retrieval is a better memory. You need both.

Live curiously and give generously.