Posted on 09/15/2022 7:45:02 AM PDT by ProtectOurFreedom
Earlier this month extreme heat downed a Twitter datacenter in California over the Labor Day weekend, leaving the website and app working on bare-bones infrastructure.
According to a memo obtained by CNN, Twitter lost access to its Sacramento (SMF) datacenter "due to extreme weather" on September 5, 2022. The memo, attributed to VP of engineering Carrie Fernandez, says, "The unprecedented event resulted in the total shutdown of physical equipment in SMF."
The temperature in downtown Sacramento, California reached a record high of 113°F and local temperature records were broken again within days.
While Twitter has backup datacenters in Atlanta and Portland that can handle the load of traffic that the social network handles each second, the loss of the Sacramento hub put it in a potentially perilous position.
"If we lose one of those remaining datacenters, we may not be able to serve traffic to all Twitter's users," Fernandez warned.
A 2020 study from The Uptime Institute, "The gathering storm: Climate change and datacenter resiliency," warns that extreme temperatures represent a growing threat to datacenters. Beyond hindering the ability to operate, high temperatures can make datacenters cost more than anticipated. "An outdoor air temperature increase of as little as four degrees Fahrenheit could make free air and evaporative systems ineffective on their own, and uneconomical in some situations," About 40% of the energy consumed by datacenters goes toward cooling IT equipment...
(Excerpt) Read more at theregister.com ...
I have an older house with wall A/C units for the summer—and wood stoves on both floors for the winter—so the energy fascists cannot touch me—yet.
Will you get hit if you supplement your central air with a window unit in your bedroom?
Don’t worry, they will sent out roving teams and fleets of drones equipped with sensors to look for guys like you.
They’ve actually done that here in the San Francisco Bay Area looking for miscreants burning wood on “Spare the Air” Days when air pollution is bad. Their roving teams will spot your chimney wood smoke and write you a ticket. An expensive ticket.
Fake News.
AZ has heat like this every summer and doesn’t have problems.
Actually the last two years in Phoenix have been cooler than normal.
No, it is not fake. Data centers in AZ are designed for it.
You have to design any power system for the environment in which it operates.
I built power plants for a living. Every subsystem was designed for a maximum output at given environmental conditions. If environmental conditions exceeded those design parameters, output and efficiency went down.
For any Rankine cycle (a steam plant cycle), you need a heat sink to dump heat. If the heat sink is too hot, you cannot dump enough heat. In a Rankine cycle, 2/3 of the heat cannot be converted to work and is dumped either to a river, lake, ocean or the atmosphere. If you cannot discharge that heat, your system output and efficiency decline. It’s basic engineering.
That severely limited the number of bots twitter could have running at any one time.
In fact Microsoft has several data farms in Phoenix
Earlier in my IT career I supported mainframes and our largest customer was Home Shopping Network when they first went live back in the mid-80s. I was told by my boss to stand in for the on-site engineer who went on vacation for two weeks.
Middle of the 2nd week I heard the massive diesel generators kick in which I knew was a real problem, when I got to the computer room floor I was told a car wreck had shut down power in the area, which was fine except the installation was so rushed the A/C system was not wired into the UPS and things were heating up rapidly.
We had already shut down everything but the production mainframe and were 10-15 minutes away from shutting down HSN completely before the power came back on, SHTF after that scare
Sounds about right for Twitter.
“the installation was so rushed the A/C system was not wired into the UPS”
Yikes, who made THAT decision? I guess it was a gamble somebody made. Modern DCs have multiple power feeds from different substations or their own drops and substations from high voltage transmission lines.
You can always spot the best locations for DCs — high voltage transmission nearby, railroad line nearby (fiber optic cables follow RR tracks), river nearby for cooling, good roads nearby (for diesel fuel delivery and equipment delivery).
You see lots of data centers along the Columbia River in Oregon for those reasons. They are always the huge nondescript buildings with no names or signage on them (very conspicuous because of that), huge backup generator systems and large fuel oil tanks.
Imagine how much more the power requirements for Twitter would be if they let President Mean Tweets back on. LOL
113 is above average for Sacramento, but not still nothing that will shatter records (maybe tie a few).
My guess is that they had some needle-nosed clown with an MBA from the Northeast running their Maintenance Department and he didn’t realize that hottest weather in California is two months later than the rest of the country, with the worst of it being from Aug-Oct. So he then wanted a big star next to his name, and so tried to service/replace some of the big units in September, rather than waiting until late October.
Hopefully he gets fired.
You have to remember the years, 1984-85, Datacenters were a relativity new thing.
HSN was a company that grew from broadcasting over AM radio until the founders decided with satellite TV aka (Directv) they could reach the entire nation and decided to go national, their growth was exponential and people in our company and HSN became filthy rich when the company went public.
The WHQ and Datacenter were all one building and it went up in record time because the company had venture capital and rich owners.
They went from one very small mainframe to 2 of the biggest mainframes we could sell in like 6 months.
The power for the system was never fully tested to see what would happen if a power failure happened. They had 6 very large CAT diesel generators and bragged the only thing that could stop their system was loss of diesel for more than 7 days aka a CAT 5 hurricane since they were located in St. Petersburg, Fl.
Are these huge data centers/farms one of the reasons why we are having power shortages and not evil homeowners/renters?
So, two down of Atlanta, Portland, and Sacramento makes Twitter go belly-up? Good to know. I mean, gosh, it would be awful if something like that were to happen.
You need a heat sink in which to dump the heat. Data centers dump their heat to the air. If you designed the AC for a maximum ambient air temperature of 112F, that’s your limit. I’m sure datacenters in Phoenix are designed for a maximum air temperature of 120F or 125F.
Regardless of the temperatures, the Coefficient of Performance (essentially, the efficiency of a refrigeration or AC system) is...
Thot is the outdoor temperature. Tcold is the cold end temperature. As Thot goes up, the COP goes down. That means lots more energy required to produce Tcold. You reach a point where it cannot keep up and Tcold starts going up.
15 posted on 9/15/2022, 8:03:25 AM by ProtectOurFreedom
Thanks for posting the above. That is what happened to many homes and businesses in N. California a week ago when our outside heat hit 112. Our AC/Hvacs couldn’t work to handle the excess heat.
They consume 2% of all US electric power! Two percent may not seem big, but considering we are shutting down all the coal, natural gas, and nuclear plants and there’s a huge push across the country to electrify everything (including EVs), even 2% is a huge contributor.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.
