Building the Proxmox Cluster: Virtualization for AI Workloads

It's 8:15 PM. I've been in the closet with the servers since 5. The rack smells like warm metal and the kind of determination that only comes from flashing your fourth BIOS update of the evening. The good news: the Proxmox cluster is up. The bad news: I still need to write about it.

Let me back up.

The Hardware

We acquired three Dell PowerEdge servers. They're not new. They're not fast by 2024 datacenter standards. But they have dual Xeons, a pile of RAM, and enough PCIe slots to matter when we get to GPU workloads. Two of them have RAID controllers; the third has a dead battery on its RAID cache, which I discovered the hard way.

The plan: turn these three boxes into a virtualization cluster that runs everything Sorren.ai needs — databases, dev environments, eventually model inference servers. One management interface. VMs that can migrate between nodes. Snapshots before I do something stupid.

Proxmox VE is the hypervisor. It's Debian-based, it uses KVM, and its web UI doesn't make me angry. That last part is more important than people realize.

Booting the ISO

I downloaded the Proxmox VE ISO and flashed it to a USB drive with Rufus. Each server boots from USB — the installer is a text-mode wizard, not pretty but fast. A few choices matter:

• Target disk: I'm installing to a small SSD, keeping the large RAID arrays for VM storage. The OS doesn't need much.
• Network config: Static IP on the management interface. I reserved .10, .11, and .12 in the 192.168.1 range on our router.
• Hostname: pve1.sorren.local, pve2.sorren.local, pve3.sorren.local. Use the FQDN — Proxmox cares about it more than you'd expect.

The install takes about ten minutes per node. The part that took me hours was the BIOS updates, because Dell's iDRAC wanted firmware that was itself behind, and apparently you can't skip versions. Servers are fun.

The Network Bridge

By default, Proxmox creates a bridge called vmbr0 bridged to your management NIC. VMs attached to this bridge get their own IP on your LAN. That's what I want — no NAT, no extra layer. Each VM is a real addressable host.

But I added a second bridge for storage traffic, on a separate NIC, on a separate VLAN:

# /etc/network/interfaces (on each node)
auto vmbr1
iface vmbr1 inet static
  address 10.10.10.x/24
  bridge-ports eno2
  bridge-stp off
  bridge-fd 0

VM storage traffic (Ceph, later) stays off the management network. This is the kind of separation that doesn't matter until it suddenly matters a lot.

Creating the Cluster

Once all three nodes were individually installed and reachable, creating the cluster was anticlimactic — in a good way:

# On pve1
pvecm create sorren-cluster

# On pve2 and pve3
pvecm add 192.168.1.10

Done. The web UI now shows all three nodes. VMs created on one are visible from all. Migration works. This is the part where virtualization starts paying for itself.

VMs: What's Running

Right now the cluster runs:

• dev-box — the Linux development server I SSH into from VS Code (the setup from my first post).
• postgres-db — PostgreSQL for application data. Backed up nightly via pg_dump to a NAS. Simple, boring, reliable.
• build-runner — a CI executor for when we formalize the pipeline.
• gpu-host (planned) — a VM with PCIe passthrough for the GPU card. This one requires kernel command-line arguments (intel_iommu=on iommu=pt) and a matching grub config. I haven't finished it yet, and it deserves its own post.

Each VM gets its own VLAN tag on vmbr0 where appropriate, and snapshots are taken before any configuration change. Snapshots are free. Rebuilding from scratch isn't.

The Honest Assessment

Is a three-node Proxmox cluster overkill for a two-person company that hasn't launched a product yet? Probably. But the alternative — running services directly on bare metal with no isolation, no snapshots, no migration — is a maintenance nightmare I refuse to walk into. Virtualization is cheap insurance. The hardware was essentially free, Proxmox is open source, and the flexibility is worth the upfront effort.

The BIOS updates are not worth the upfront effort. But you don't get to choose.

What's Next

The cluster is the substrate. Next I need identity — a way to manage users and access across all these VMs and the Windows dev machines without maintaining separate password files on each box. That means a domain controller, which means Windows Server in a VM, which means I'm about to mix Active Directory with Linux authentication.

That's the next post. For now, the servers are humming, the closet is warm, and I'm going to go eat something that isn't a granola bar.

It's past 8 PM on a Thursday. Good enough.