The concurrency console
Where this idea comes from
The silent alarm
On 14 August 2003 a blackout cut power to about 50 million people in the north-eastern United States and Ontario. In the control room of one power company, FirstEnergy, the alarm software had stopped working that afternoon without telling anyone, so operators did not see the warnings as lines failed.
One thing to hold: The joint US–Canada task force report (April 2004). It describes the alarm system freezing; later investigation traced the freeze to a race condition, two parts of the program updating the same data at the same moment.
Why it matters: A race condition does not crash on every run. It waits for the one moment when two things happen in the wrong order, which is why it can pass every test and still fail on the worst day.
What we don’t know: The race condition was one cause among many: trees touching power lines, missing tools and training also played a part. The report does not blame the software alone.
Dates:
- 2003 The north-east blackout; an alarm system freezes silently.
- 2004 The task force publishes its final report.