Wednesday, May 3, 2017

The Next Great Northeastern Blackout


It has already been more than thirteen years since the last great northeastern blackout.  The mean time between such blackouts is roughly twenty years.  

Blackouts are caused by a number of simultaneous component failures that overwhelm the ability of the system to cope.  While the system copes with most component failures so well that the consumer never even notices, there is an upper bound to the number of concurrent failures that it can tolerates. In the face of these failures the system automatically does an orderly protective shutdown that assures its ability to restart within tens of hours to days.

However, such surprising shutdowns are  experienced by the community as a "failure."  One result is finger pointing, blaming, and shaming.  Rather than being seen as a normal and predictable occurrence with a proper and timely response, the Blackout is seen as a "failure" that should have been "prevented."  

These outages are less disruptive than an ice storm.  However, even though they are as natural and as inevitable as weather related outages, they are not perceived that way.  The public and their political representatives see weather related outages as unavoidable but inevitable technology failures as one hundred percent preventable.

Security people understand that perfect prevention has infinite cost.  That as we increase the meantime between outages, we stop, at about twenty years, way short of infinity.  This is in part because the cost of the next increment exceeds the value and in part because we reach a natural limit. We increase the resistance to failure by adding redundancy and automation. However, we do this at the cost of ever increasing complexity.  There is a point, at about twenty years MTBF, at which increased complexity causes more failures than it prevents.

Much of the inconvenience to the public is a function of surprise.  Since they have come to expect prevention, they are not prepared for outages.  The message should be that we are closer to the next Blackout than to the last.  If you are not surprised and are prepared, you will minimize your own cost and inconvenience.