Bare metal is having a moment. Teams running AI training, high-throughput storage, latency-sensitive services, or simply large steady workloads keep rediscovering that owning hardware — in your own racks or in colocation — can be dramatically cheaper and faster than renting it by the hour. The catch is that "buy servers" is the easy decision. Making hundreds of machines across several locations behave like one programmable system is the actual work.
Here's what operating bare metal at scale really involves, and where it goes wrong.
The core shift: treat servers like cattle, not pets
The mistake that sinks bare-metal efforts is operating each machine by hand — SSHing in, installing packages, tweaking configs until it works, and never quite being able to reproduce it. That's fine for three servers and fatal for three hundred. At scale, a server has to be disposable and reproducible: any machine can be wiped and rebuilt to a known state automatically, and a replacement is indistinguishable from the original. If rebuilding a node is a project, you don't have a fleet — you have a pile of snowflakes.
Provisioning: from power-on to ready, untouched
The foundation is automated provisioning — a bare machine powering on and configuring itself into the fleet with no one at the keyboard:
- Network boot (PXE / iPXE) so a server with empty disks boots an installer or image straight from the network. This is the backbone of hands-off provisioning, and getting DHCP, TFTP/HTTP, and boot images right across hardware generations is most of the battle.
- Hardware discovery and inventory. New machines should announce themselves — MAC addresses, hardware profile, location — and land in an inventory automatically, not a spreadsheet someone updates by hand.
- Declarative configuration. Once the OS is on, configuration is described as code and applied consistently. The desired state lives in a repository, not in someone's memory of what they typed.
- Immutable where possible. Image-based deployments — build a known-good image, ship it, replace rather than mutate — make a fleet predictable and rollbacks trivial.
The payoff is simple to state and hard to earn: a failed server is replaced by racking a new one and letting it provision itself, with zero manual setup.
The multi-site problem
One location is a solved problem with good tooling. The difficulty multiplies when the fleet spans several sites — multiple datacenters, colos, or edge locations:
- Networking and identity across sites. Machines in different locations need consistent addressing, secure connectivity between sites, and one identity and access model — not a separate snowflake setup per location.
- Local autonomy, central control. Each site should keep running if the link to the others drops (local provisioning, local services), while still being managed from one place. Fully centralized breaks when the network does; fully independent becomes unmanageable.
- One source of truth. Inventory, configuration, and state need a single authoritative view across every site, so "what's running where" has one answer.
- Data gravity and placement. Where workloads and their data live across sites is a deliberate design decision — for latency, redundancy, and cost — not an accident of where there was free space.
Operating it: you can't manage what you can't see
A fleet you can't observe is a fleet that surprises you. Across sites you need unified visibility — hardware health, capacity, and service status in one place — plus the boring disciplines that keep a fleet healthy: rolling OS and firmware updates without taking everything down at once, hardware lifecycle from burn-in to decommission, and capacity planning so you provision ahead of need instead of scrambling. Done well, a small team runs a large multi-site fleet. Done by hand, the fleet runs the team.
Bare metal and the cloud aren't enemies
The strongest setups aren't purist. They put steady, capacity-heavy workloads on owned bare metal where it's cheapest, keep data where it needs to be, and use the cloud for elasticity and reach. The same provisioning and configuration discipline ideally spans both, so a workload can move based on cost and need rather than being trapped by where it was first deployed.
Where to start
You don't begin by buying a hundred servers. You begin by making one server fully reproducible — network-booted, configured from code, rebuildable from scratch without a human — and then proving that pattern holds across a second site. Once provisioning and configuration are truly automated, scale becomes a capacity question instead of a labor one.
That automation foundation — network boot, declarative config, multi-site inventory and operations — is exactly the kind of system worth designing deliberately before the fleet grows past the point where you can wrangle it by hand. If you're standing up bare metal across more than one location, that's the conversation worth having early.