Dark Energy — The Stranger Problem

By the 1990s, cosmologists thought they had a reasonable picture of the universe's future. It was expanding — Edwin Hubble had established that in 1929 — and gravity would eventually slow that expansion down. The question was just whether the universe had enough mass to eventually stop and collapse (a "closed" universe) or whether it would expand forever at a decreasing rate.

In 1998, two independent teams trying to measure that deceleration got the same result: the expansion isn't slowing down. It's speeding up.

This wasn't expected by anyone.

The teams — the High-Z Supernova Search Team and the Supernova Cosmology Project — were using Type Ia supernovae as "standard candles." A Type Ia supernova occurs when a white dwarf accumulates enough mass to trigger runaway carbon fusion, producing an explosion with a consistent peak luminosity. By comparing how bright the supernova appears to how bright it should be, you get its distance. Combined with redshift measurements, you map the expansion history of the universe.

Both groups found that distant supernovae appeared dimmer than expected — they were farther away than they should have been if the universe were decelerating. The universe had been accelerating for at least the past 5 billion years.

Something was driving that acceleration. Whatever it was, it had to be distributed uniformly through space (otherwise its effects would be different in different directions), it couldn't cluster like matter (otherwise we'd see it affecting galaxy clusters), and it had to have a large negative pressure — repulsive, rather than attractive. Physicists called it dark energy, which is as much a placeholder name as it is a description.

**The cosmological constant**

The most mathematically natural explanation is Einstein's cosmological constant, Λ — a term he originally added to his field equations in 1917 to create a static universe and then discarded when Hubble showed the universe was expanding. As it turns out, the cosmological constant corresponds to a fixed energy density of empty space. Quantum field theory predicts that the vacuum should have energy — virtual particle pairs briefly flickering into and out of existence — but the predicted vacuum energy density from quantum mechanics is off from the observed cosmological constant by roughly 120 orders of magnitude.

That discrepancy is sometimes called "the worst prediction in physics." It isn't that the prediction is wrong in direction, but that it's wrong in magnitude by a factor incomprehensible to reason about. Nobody knows how to reconcile it.

**Alternatives**

Quintessence is a proposed dynamic field — rather than a constant energy density, the dark energy density could change over time, evolving through some potential. Different quintessence models predict different acceleration histories that future experiments could potentially distinguish from a pure cosmological constant.

In 2023, data from the Dark Energy Spectroscopic Instrument (DESI) provided the most precise map yet of the large-scale structure of the universe, tracking galaxy positions back 11 billion years. Early results from DESI, presented in 2024, show a hint that dark energy may not be constant — it may have changed over time. The statistical significance is not yet high enough to make a firm claim, but it's the kind of signal that people are watching.

**What we're left with**

Dark energy makes up 68% of the universe. It's causing everything in the universe to fly apart at an accelerating rate. We don't know what it is, we can't detect it directly, and our best theory for what it might be is off by 120 orders of magnitude.

At least with dark matter, we have particle candidates and detectors we can point at the sky. Dark energy doesn't even have clear candidates yet.

These two problems together mean that 95% of the universe is described by the phrase "we have no idea." That's either depressing or exciting depending on your disposition.

Dark Energy — The Stranger Problem

// COMMENTS

ON THIS PAGE