Posted on 04/26/2009 8:35:06 AM PDT by 2banana
A Cyber-Attack on an American City
Bruce Perens
Just after midnight on Thursday, April 9, unidentified attackers climbed down four manholes serving the Northern California city of Morgan Hill and cut eight fiber cables in what appears to have been an organized attack on the electronic infrastructure of an American city. Its implications, though startling, have gone almost un-reported.
That attack demonstrated a severe fault in American infrastructure: its centralization. The city of Morgan Hill and parts of three counties lost 911 service, cellular mobile telephone communications, land-line telephone, DSL internet and private networks, central station fire and burglar alarms, ATMs, credit card terminals, and monitoring of critical utilities. In addition, resources that should not have failed, like the local hospital's internal computer network, proved to be dependent on external resources, leaving the hospital with a "paper system" for the day.
Commerce was disrupted in a 100-mile swath around the community, from San Jose to Gilroy and Monterey. Cash was king for the day as ATMs and credit card systems were down, and many found they didn't have sufficient cash on hand. Services employees dependent on communication were sent home. The many businesses providing just-in-time operations to agriculture could not communicate.
In technical terms, the area was partitioned from the surrounding internet. What was the attackers goal? Nothing has been revealed. Robbery? With wires cut, silent alarms were useless. Manipulation of the stock market? Companies, brokerages, and investors in the very wealthy community were cut off. Mayhem, murder, terrorism? But nothing like that seems to have happened. Some theorize unhappy communications workers, given the apparent knowledge of the community's infrastructure necessary for this attack. Or did the attackers simply want to teach us a lesson?
Although they are silent on the topic, I hope those responsible for emergency services, be they in business or government, are learning the lessons of Morgan Hill. The first lesson is what stayed up: stand-alone radio systems and not much else. Cell phones failed. Cellular towers can not, in general, connect phone calls on their own, even if both phones are near the same tower. They communicate with a central switching computer to operate, and when that system doesn't respond, they're useless. But police and fire authorities still had internal communications via two-way radio.
Realizing that they'd need more two-way radio, authorities dispatched police to wake up the emergency coordinator of the regional ham radio club, and escort him to the community hospital with his equipment. Area hams dispatched ambulances and doctors, arranged for essential supplies, and relayed emergency communications out of the area to those with working telephones.
That the hospital's local network failed is evidence of over-dependence on centralized services. The development of the internet's communications protocols was sponsored by the U.S. Department of Defense, and they were designed to survive large failures. But it still takes local engineering skill to implement robust networking services. Most companies stop when something works, not considering whether or how it will work in an emergency.
Institutional networks, even those of emergency services providers, are rarely tested for operation while disconnected from the outside world. Many such networks depend on outside services to match host names to network addresses, and thus stop operating the moment they are disconnected from the internet. Even when the internal network stays up, email is often hosted on some outside service, and thus becomes unavailable. Programs that depend on an internet connection for license verification will fail, and this feature is often found in server software. Commercial VoIP telephone systems will stay up for internal use if properly engineered to be independent of outside resources, but consumer VoIP equipment will fail.
This should lead managers of critical services to reconsider their dependence on software-as-a-service rather than local servers. Having your email live at Google means you don't have to manage it, but you can count on it being unavailable if your facility loses its internet connection. The same is true for any web service. And that's not acceptable if you work at a hospital or other emergency services provider, and really shouldn't be accepted at any company that expects to provide services during an infrastructure failure. Email from others in your office should continue to operate.
What to do? Local infrastructure is the key. The services that you depend on, all critical web applications and email, should be based at your site. They need to be able to operate without access to databases elsewhere, and to resynchronize with the rest of your operation when the network comes back up. This takes professional IT engineering to implement, and will cost more to manage, but won't leave you sitting on your hands in an emergency.
Communications will be a problem during any emergency. Two-way radios have, to a great extent, been replaced by cellular "walkie-talkie" services that can not be relied upon to work during an infrastructure failure. Real two-way radios, stand-alone pager systems, and radio repeaters that enable regional communications are still available to the governments and businesses that endure the expense of planning, acquiring, maintaining, and testing them. Corporate disaster planners should look into such facilities. Municipalities, regardless of their size, should not consider abandoning such resources in favor of the less-robust cellular services.
Satellite telephones can be expected to keep operating, although they too depend on a land infrastructure. They are expensive, and they frequently fail in emergency situations simply because their users, administrative officials rather than technical staff, fail to keep them charged and have no back-up power resource once they are discharged.
A big plus for Morgan Hill was that emergency services had an well-practiced partnership with the local hams. Since you can never budget for all of the communications technicians you'll need in an emergency, using these volunteers is a must for any civil authority. They come with their own equipment, they run their own emergency drills and thus are ready to serve, and they are tinkerers able to improvise the communications system needed to meet a particular emergency.
Which brings us to the issue of testing. No disaster system can be expected to work without regular testing, not only of the physical infrastructure provided for an emergency but of the people who are expected to use it, in its disaster mode. But such testing takes much time and work, and tends to trigger any lurking infrastructure problems, creating outages of its own. It's much better to work such things out as a result of testing than to meet them during a real disaster.
We should also consider whether it might be necessary to harden some of the local infrastructure of our communities. The old Bell System used to arrange cables in a ring around a city, so that a cut in any one location could be routed around. It's not clear how much modern telephone companies have continued that practice. It might not have helped in Morgan Hill, as the attackers apparently even disabled an unused cable that could have been used to recover from the broken connections.
Surprisingly, manholes don't usually have locks. They rely on the weight of the cover and general revulsion to keep people out. They are more likely to provide alarms for flooding than intrusion. Utility poles are similarly accessible. Much of our infrastructure isn't protected by anything so tough as a manhole cover. Underground cables are easily accessible in surface posts and "tombstones", boxes often located in residential neighborhoods. These can be wrecked with a screwdriver.
Most buried cable cuts are caused by operating a back-hoe without first using one of the "call before digging" services to mark out the location of all of the buried utilities. What's done accidentally can also be done deliberately, and the same services that help diggers avoid utilities might point them out to an attacker.
The most surprising news from Morgan Hill is that they survived reasonably unscathed. That they did so is a result of emergency planning in place for California's four seasons: fire, floods, earthquakes, and riots. Most communities don't practice disaster plans as intensively.
Will there be another Morgan Hill? Definitely. And the next time it might happen to a denser community that won't be so astonishingly able to sustain the trouble using its two-way radios and hams. The next time, it might be connected with some other event, be it crime or terrorism. Company and government officers take notice: the only way you'll fare well is if you start planning now.
>>>Isn’t calling them “Manholes” sexist? s/
Need to change to “maintenance access portals”
My son who lives in Troy, NY called me earlier this morning to tell me that his Time Warner phone and Road Runner are not functioning. The TV cable is fine. When trying to call, he gets a message to please call back because they are getting too many calls. I live 90 miles west of him in Rome, NY. I have Time Warner, and am having the same problem, although my digital phone and cable are working. I can even access my email through my mail program. However, I cannot get Google, Matt Drudge or several other sites to load. I get a message that they cannot local the server. FreeRepublic however is working fine. When I try calling Time Warner, I get an “all circuits are busy” message. Is anyone else experiencing any problems? I did a search on FR and couldn’t find threads on it.
Of interest: CWA and SEIU work together! SEIU is ACORN!
“...The Communications Workers of America represents 700,000 workers in the United States, Canada and Puerto Rico. It’s one of America’s fastest growing unions. CWA members work in communications, media, airlines, manufacturing and public service. CWA has been a member of the Blue Green Alliance, a strategic partnership between labor unions and environmental organizations to expand the job-creating potential of the green economy, since 2007. Other Blue Green Alliance members include the Sierra Club, the United Steelworkers (USW), the Natural Resource Defense Council (NRDC), the Service Employees International Union (SEIU) and the Laborers’ International Union of North America (LIUNA)”
“...The Blue Green Alliance is a national partnership of labor unions and environmental organizations dedicated to expanding the number and quality of jobs in the green economy. Formed initially by the United Steelworkers and the Sierra Club in 2006, the organization has recently grown to include SEIU, NRDC, CWA, and LIUNA.”
http://www.seiu.org/2009/03/seiu-the-blue-green-alliance-urge-climate-change-legislation-in-2009.php
There are hundreds of thousands of miles of fiber running around the US. There is no way to secure it all. The only way to protect against things like this are to build in as much redundancy as possible.
The people who made this cut had enough knowledge of the physical topology of AT&T’s fiber network to see that they could isolate an area some distance away by cutting the fiber in San Carlos and South San Jose - Two locations about 30 miles apart. There aren’t many large areas in urban, suburban or even exurban regions where you can isolate a territory as large as they did by making just two cable cuts. This took some research into things that are not a part of the public database. It was an inside job. Nobody will ever prove it but the union did it.
Not in parts of northern California...
So I’ll take your answer to my question as an unequivocal YES.
I can’t say I disagree with you.
Or ‘A’ holes either...
I assumed this was the local telephone workers union giving everyone a wake-up call.
Just turned my TV to the Time Warner news channel, and there is an alert message that customers in the Northeastern part of the state are experiencing a highspeed data and digital phone service interruption. I live in Central NY and it isn’t really considered the Northeastern part of the state. My sister lives in Rochester, which is even farther West than me, and her digital phone is obviously not working either.
Exactly right. It's an unfortunate fact and inherent problem with free-market capitalism, which isn't perfect. With competition, the push comes for efficiency...those who don't move that direction die off (and consultants make lots of money pushing for and writing about Just-In-Time techniques, etc.) That is, capitalist competition forces that directional move.
Then, when a problem occurs, there's less robustness in the existing (i.e., remaining) producers. Perhaps that weeds out the less robust, who have sacrificed so much for efficiency (...and the consultants all come out with the new supply-chain/procurement models that emphasize flexibility, redundancy, robustness... :-). The cycle goes back and forth, but even many conservatives forget that the great efficiency that capitalism provides does not come without some costs and risks.
[...]Our entire military, with its immense cost, can be considered a standby system [...]
Correct. ...and which is a scary notion to get into the minds of some, who might use that idea as a justification for inappropriate actions. Do the words "John Warner Defense Authorization" ring any bells?
The Republic is a great entity, but it does require eternal vigilance to maintain the balancing act.
gross
The military, for all its necessity, is economically not only unproductive, it’s antiproductive. Its primary mission, although not implemented all that often, is to kill people and break things. This destroys wealth, both in the targets and the ammo. It also diverts lots of highly competent people from the production of wealth.
OTOH, you can make a darn good case that without the military and LEO, the internal equivalent, free market capitalism wouldn’t be able to implement its efficiencies at all. Entrepeneurs and inventors can’t accomplish much when armies or mobs trash the neighborhood every couple years. Peace is the most basic component of economic progress.
Communications Workers of America (CWA)
http://www.unionfacts.com/unions/unionProfile.cfm?id=188
Communications Workers of America (CWA) is the largest communications and media labor union in the United States (the union also has locals in Canada), representing over 700,000 workers in both the private and public sectors. CWA is headquarted in Washington, DC and affiliated with the AFL-CIO, the Canadian Labour Congress, and Union Network International. The current president is Larry Cohen, a member of the AFL-CIO Executive Council.
Leadership
Top 10 International CWA Leaders & Staff (by Salary)
Name Title Total Compensation
Barbara Easterling Secretary-treasurer $ 204,165
Anthony Bixler Vice President $ 176,783
Lawrence Cohen President $ 176,298
Jeffrey Rechenbach Vice President $ 171,099
William Boarman President-sector $ 168,838
Christopher Shelton Vice President $ 165,405
David Borer Afa-cwa General Counsel $ 161,971
James Clark Iue-cwa Division Presiden $ 161,080
Lauren Asplen Asst Dir Of Communication $ 159,379
Clarence Milburn Vice President $ 156,052
[show all officers & salaries]
Well, if columns are phallic symbols, what would "manholes" be sumbolic of?
As an example, when a low-income area has people breaking into transformers and causing tens of thousands of dollars of damage and environmental cleanup requirements all for a few cents of copper wiring, it should be a wakeup call to what vulnerabilities there are in many of our systems. Instead, costs just gets budgeted in and passed on to the consumer in their electric bills, and everyone goes on their merry way. Reporting vulnerabilities has not done much actual good even under the Bush Administration, which seemed more interested in registering citizens and collecting their biometric data than protecting our country.
This article hints at far more serious threats than most (nearly all) people realize.
Ødumba should cap union leaders salaries at $25k so they don’t take so much from the workers paychecks for their gravy train lifestyle
Bookmark.
“over dependence on centralized services.”
That pretty well sums everything up. That’s why, as Hayek showed, socialism never works. It shows why transportation is the Achilles heel of globalization. It shows why a truly Federal system is the best form of government. Subsidiarity works; centralization breaks down.
____________________________________
They were originally called Access Significant Systems Holes.
Rank and file, American
Leadership, internationalists
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.