Forecasting the distributed grid: machine learning where the meter runs both ways
Rooftop solar, batteries, and electrified demand have broken the load curves utilities planned around for a century. Forecasting is now a core grid asset — and a data engineering problem before it is a modeling one.
For a hundred years, a utility could forecast demand with a calendar, a weather report, and institutional memory: load followed people, and people followed routines. Distributed energy resources ended that settlement. Rooftop solar turns feeders into generators at noon. Home batteries and electric vehicles shift megawatt-hours on price signals and habit. Electrified heating makes cold snaps into grid events. The net-load curve — what the utility must actually supply — has become spikier, weather-coupled, and locally idiosyncratic in ways aggregate models cannot see.
This is not an inconvenience at the planning department. Forecast error now prices directly into procurement, curtailment, battery dispatch, and the decision to reinforce a feeder or defer it. Forecasting has quietly become one of the highest-leverage assets a grid operator owns.
The problem is granularity
The forecasting task has shifted from one curve to thousands: per-substation, per-feeder, sometimes per-large-customer, at horizons from minutes to seasons. At that granularity, classical time-series methods — still excellent baselines at system level — start to miss the local structure: the orientation of a neighborhood’s rooftops, the cloud band that hits one feeder and not the next, the school holiday that empties a district.
Machine learning earns its place here, not as a slogan but as a toolset for exactly this shape of problem: many related series, rich covariates (weather, irradiance, calendar, tariff events), and value concentrated in the tails — the days the grid is actually stressed. Gradient-boosted models and increasingly transformer-based forecasters that share learning across thousands of series are consistently strong; the research frontier matters less than most vendors imply.
The model is the easy part
In our applied work, the modeling is rarely where forecasting programs fail. They fail upstream: meter and SCADA data with silent gaps and timestamp drift; behind-the-meter solar that is invisible because it was never registered; weather feeds at the wrong spatial resolution; no versioned record of which forecast the operator actually acted on. A disciplined data layer — quality monitoring, provenance, backtesting infrastructure that can replay any historical day — routinely improves outcomes more than any architecture change.
Equally decisive is evaluation honesty. Average error is a vanity metric in grid forecasting; the money and the risk live in peak days and ramps. A model should be judged on the hundred worst hours of the year, and on whether its uncertainty bands are calibrated well enough for an operator to trust them when committing reserves.
What to do about it
Pick the decisions forecast error actually costs you — procurement, dispatch, curtailment — and price the error before buying any model. Build the replayable data foundation first. Then set the benchmark discipline: every candidate model, however fashionable, competes against a tuned classical baseline on tail performance, not annual averages. The grid’s new intelligence will be earned in the data layer; the models are the visible tip of it.