Datacenter Cooling: From Air to Immersion Liquid

The fundamental physics of computing haven't changed: every watt of electricity flowing through a processor becomes heat, and that heat must go somewhere. What has changed is the density. An AI training cluster running NVIDIA H100s or GB200s can pack 10–20 kilowatts into a single server rack, compared to 3–5 kW for a conventional server rack a decade ago. At that density, pushing cold air past hot components simply stops working.

## PUE: The Efficiency Metric That Defined an Era

**Power Usage Effectiveness (PUE)** measures total datacenter power divided by IT equipment power. A PUE of 1.0 is perfect — every watt goes to computing. A PUE of 2.0 means equal energy is spent on cooling, lighting, and overhead as on computation.

Hyperscale operators spent the 2010s engineering their way from industry-average PUEs of 1.8–2.0 down to 1.1–1.2 through increasingly sophisticated air cooling: hot aisle/cold aisle containment, free cooling (using outside air when temperatures allow), adiabatic cooling, and precise computational fluid dynamics modelling of airflow.

Google's most efficient datacenters report PUEs of 1.10. Meta's Prineville facility achieved 1.07 using outside air cooling extensively. These are remarkable numbers — but they were achieved at conventional computing densities. AI workloads are shattering those assumptions.

## Why Air Cooling Hits a Wall

A standard 19-inch rack has a defined footprint — roughly 0.6 m × 1.0 m. Cooling that rack with air requires moving a large volume of air past hot components, keeping the air temperature low enough that the temperature gradient drives heat transfer. As rack power density rises above ~20 kW, the airflow required becomes physically impractical: the fans themselves consume significant power, the air velocity creates noise problems, and the sheer volume flow required through raised floors or overhead systems becomes difficult to engineer.

The thermal resistance between a CPU/GPU junction and the surrounding air is also relatively high. Water has roughly 3,500× higher volumetric heat capacity than air and much better thermal conductivity. Liquid cooling eliminates the fundamental limitation of air as a heat transfer medium.

## Direct Liquid Cooling (DLC): The First Transition

**Direct liquid cooling** routes chilled water through cold plates attached directly to processors, memory, and power delivery components. The liquid absorbs heat conductively through the cold plate surface, is pumped back to a heat exchanger, and the heat is rejected to the building's cooling infrastructure.

DLC is the dominant technology for high-density AI servers today. NVIDIA's DGX H100 and GB200 NVL72 systems are designed to be DLC-capable. The technology has several advantages: it integrates with existing datacenter water infrastructure, it can be retrofitted into existing racks, and it is an evolution rather than a revolution in datacenter design. Facility operators can deploy DLC pods alongside existing air-cooled systems.

The limitation of DLC is that it only cools the components that have cold plates attached. Power delivery components, PCBs, drives, and other peripheral heat-generating hardware still require supplemental air cooling, meaning DLC systems are typically "hybrid" — liquid for the high-power compute elements, air for everything else.

## Immersion Cooling: Single-Phase and Two-Phase

**Single-phase immersion cooling** submerges entire server hardware directly into tanks of non-conductive, thermally stable dielectric fluid — typically engineered fluorocarbon or synthetic hydrocarbon fluids. The fluid circulates naturally (or with pumping) past the hardware, absorbs heat, and is cooled by a heat exchanger at the top of the tank. Hardware operates normally while fully submerged; the fluid is chemically inert and non-conductive.

The thermal performance is dramatically better than air: the fluid's direct contact with every component surface (no cold plate required) achieves very low thermal resistance. Rack densities of 50–100+ kW are achievable. The systems are also notably quiet — no fans needed inside the immersion tanks.

**Two-phase immersion cooling** uses a dielectric fluid with a lower boiling point, typically around 50°C. The fluid boils directly on hot component surfaces, and the latent heat of vaporisation provides cooling. Vapor rises, condenses on water-cooled coils at the top of the tank, and returns as liquid. The two-phase process is extremely efficient at handling heat spikes and achieves very uniform component temperatures.

The challenges with immersion cooling are significant: the upfront infrastructure cost is higher, maintenance requires draining tanks or using specialized tools, and not all components are rated for immersion (some electrolytic capacitors can leak). Seal integrity around the tank and fluid management require careful engineering.

## Meta's AI Datacenter Thermal Strategy

Meta's approach to its AI training infrastructure illustrates the current frontier of thermal engineering. The company's **Grand Teton** AI training clusters (used for Llama model training) use direct liquid cooling for GPU hot spots while maintaining air cooling for other components. Meta has designed purpose-built rack form factors for AI training that separate the thermal management system from the compute hardware, allowing higher densities.

For its next-generation AI datacenters, Meta has committed to a path toward facility-level liquid cooling infrastructure — treating chilled water as the primary cooling medium distributed throughout the facility rather than as an optional supplement. This represents a fundamental shift in datacenter infrastructure philosophy.

## Water Usage Effectiveness (WUE) and the Sustainability Tension

The shift from air to liquid cooling trades one environmental concern for another. **Water Usage Effectiveness (WUE)** — litres of water consumed per kWh of IT load — becomes critical when large volumes of cooling water are involved.

Evaporative cooling towers, which are efficient at rejecting heat, consume substantial water. A large AI training datacenter can consume millions of litres of water annually. In water-stressed regions, this creates genuine tension with local water resources.

Closed-loop cooling systems that reject heat to dry coolers or geothermal sinks can dramatically reduce WUE, but at higher capital cost. The engineering choices made today about cooling infrastructure will have meaningful environmental consequences at the scale that AI computing is reaching.

Datacenter Cooling: From Air to Immersion Liquid

// COMMENTS

ON THIS PAGE