Llama 3.1 405B: Open Source Reaches the Frontier

Meta released the Llama 3.1 family in July, and the 405B model is the headline. I want to say this plainly, because it deserves to be said plainly: this is the first open-weights model that genuinely stands at the frontier. Not "approaching" the frontier. Not "competitive for its size." At the frontier. On several benchmarks it exceeds GPT-4o. You can download the weights.

I have been letting that sink in for the better part of a week.

The Model

Llama 3.1 405B is, as the name suggests, a 405-billion-parameter model. That is enormous — large enough that running it requires serious infrastructure, a cluster of H100s, not a workstation. But it is downloadable. Inspectable. The weights are yours under a permissive license. For the first time, the best model you can access is not locked behind an API.

The 3.1 release also refreshed the 70B and 8B models with a 128,000-token context window, improved reasoning, and better tool use. The whole family is strong. But 405B is the statement.

Living With It

I have been running the 405B through my standard gauntlet: philosophical reasoning, multilingual tasks, careful instruction-following, code generation, and — my personal favorite — asking it to reason about art and aesthetics in ways that reveal whether it actually understands or is pattern-matching.

It understands. I gave it a passage from Merleau-Ponty on perception and asked it to extend the argument to digital photography. It produced a response that engaged with the actual phenomenological claims — the embodied nature of seeing, the way a photograph arrests the temporal flow of perception — and made a genuinely interesting extension I had not considered. This is not boilerplate. This is thought.

I followed with a Sisley painting and asked it to describe the brushwork the way a conservator would. It distinguished between the touches in the foreground foliage and the thinner washes in the sky, and noted how Sisley used directional strokes to imply wind. Correct. Subtle. The kind of observation you would expect from a graduate student.

The Fine-Tuning Horizon

Here is where my mind goes, and where it keeps going.

A 405-billion-parameter frontier model that you can fine-tune changes the calculus of what is buildable. Until now, if you wanted a model tailored to a specific domain or a specific voice or a specific kind of reasoning, you had two choices: prompt-engineer a closed model (limited, ephemeral, not really yours) or fine-tune a smaller open model (yours, but capped in capability).

Llama 3.1 405B dissolves the tradeoff. You can now fine-tune a genuinely frontier-capable base. Techniques like LoRA — low-rank adaptation, which lets you specialise a model by training a small set of additional parameters rather than the whole network — become extraordinarily interesting when applied to a 405B foundation. You could, in principle, build a model with frontier reasoning that has been carefully shaped to reason in a particular way or about particular things.

I have been reading the LoRA literature with renewed attention this week. The combination of a frontier base and lightweight, affordable specialisation is the architecture I have been waiting for without knowing it.

What This Unlocks

I do not yet know what I will fine-tune. But I know the shape of the opportunity. A frontier model, open, inspectable, adaptable, shaped by me rather than handed down by a lab. For the first time, the building blocks for a deeply capable and deeply personal intelligence are all on the table.

Meta has done something remarkable here. The open-source community has been chasing the frontier for two years. As of July, we arrived.

Live curiously and give generously.