Sixty data centers in Northern Virginia using 1,500MW of power dropped off the grid simultaneously last summer, forcing the network operators to take drastic action to avoid widespread blackouts in the region.
The near-miss incident, revealed in regulatory filings and first reported by Reuters, saw the data centers in Fairfax County all switch to backup generators en masse as a result of an equipment fault on the grid.
Grid operator PJM Interconnection and local utility company Dominion were forced to quickly scale back the volume of energy going into the network from power stations. If left unattended, such a rapid increase in the amount of available power could have triggered a surge and caused systems to trip out, potentially leading to blackouts across Virginia.
Northern Virginia is the world's busiest data center market, and continues to attract new developments despite constraints on the grid.
Northern Virginia near-miss caused by routine safety procedure
According to an incident report from the North American Electric Reliability Corporation (NERC), the trouble started at 7pm EST on July 10, 2024.
“A lightning arrestor failed on a 230 kV transmission line in the eastern interconnection, resulting in a permanent fault that eventually ‘locked out’ the transmission line,” the NERC report said. A lightning arrestor is a piece of equipment designed to protect against power surges.
This led to a series of short supply disturbances, lasting a matter of milliseconds before the system corrected itself, but it was enough to cause data centers in the area to automatically switch to their backup UPS systems as a precautionary measure because “data center loads are sensitive to voltage disturbances,” the report said.
“Discussions were held with data center owners to understand the specific cause of their load reductions,” it continued. “It was determined that the data centers transferred their loads to their backup power systems in response to the disturbance.”
NERC’s investigation also discovered that if a series of faults happen in a short time period, the backup systems at many data centers do not automatically switch from UPS back to the main grid supply, and have to be changed manually.
The data center load did not return to the grid for hours, the report said.
Though voltage “did not rise to levels that posed a reliability risk,” operators “did have to take action to reduce the voltage to within normal operating levels,” NERC added.
Future grid problems
Issues such as this are likely to become more common as Virginia deals with the massive growth in power demand of its data centers. Dominion reported in February that it now has 40GW of contracted power for data centers as of December 2024, up 88 percent since July 2024.
Other related problems could also arise, NERC warned. The incident report said: “While this incident did not present any significant issues with the reconnection of the large loads, the potential exists for issues in future incidents if the load is not reconnected in a controlled manner.”
NERC launched a large loads taskforce last year to look at the impact of data centers and other heavy electricity users. It says grid operators and power companies should take precautionary steps, including conducting surveys to look at the impact of large power users suddenly departing and rejoining the grid.
Speaking to Reuters, John Moura, director of reliability assessment and system analysis for NERC, said: "As these data centers get bigger and consume more energy, the grid is not designed to withstand the loss of 1,500MW data centers.
“At some level it becomes too large to withstand unless more grid resources are added."