If you can read this, either the style sheet didn't load or you have an older browser that doesn't support style sheets. Try clearing your browser cache and refreshing the page.

(Some Guy)   How Rackspace prepared its Virginia data centers for Hurricane Sandy. Maker's Mark curiously not mentioned   (rackspace.com) divider line 51
    More: Interesting, Rackspace, data centers, MREs, preventive action, customer service, storm surges, National Weather Service  
•       •       •

8942 clicks; posted to Main » on 30 Oct 2012 at 11:47 AM (1 year ago)   |  Favorite    |   share:  Share on Twitter share via Email Share on Facebook   more»



51 Comments   (+0 »)
   
View Voting Results: Smartest and Funniest

Archived thread

First | « | 1 | 2 | » | Last | Show all
 
2012-10-30 08:32:11 AM
i.imgur.com
 
2012-10-30 10:00:15 AM
Stuff a sock in it, Bob.
 
2012-10-30 10:29:35 AM
beerspill.fark.com
 
2012-10-30 10:29:40 AM
I feel bad for all the people that work there. "yeah we are sorry your house is probably getting washed away, and you have no way to even communicate with your wife and kids to see if they are safe, but someone has to answer these phones and be available in case a server needs manually reset or something."
 
2012-10-30 11:23:30 AM

NickelP: I feel bad for all the people that work there. "yeah we are sorry your house is probably getting washed away, and you have no way to even communicate with your wife and kids to see if they are safe, but someone has to answer these phones and be available in case a server needs manually reset or something."


True, but the rest of the world is still turning, in spite of the proportioned news coverage.
 
2012-10-30 11:50:13 AM
https://www.youtube.com/watch?v=piEUJfCJW_g
 
2012-10-30 11:50:57 AM

Radak: [i.imgur.com image 500x268]


Came for this, still leaving angry.

/WTF is wrong with you people?
 
2012-10-30 11:51:21 AM

How Rackspace prepared its Virginia data centers for Hurricane Sandy.


If they had a decent BC/DR plan tested and ready to go, they shouldn't have needed much prep work.
 
2012-10-30 11:52:08 AM
I expected much better pics on something called Rackspace.com.
 
2012-10-30 11:56:45 AM

NickelP: I feel bad for all the people that work there. "yeah we are sorry your house is probably getting washed away, and you have no way to even communicate with your wife and kids to see if they are safe, but someone has to answer these phones and be available in case a server needs manually reset or something."


If you do system admin it's part of the job. They had supplies and in someway having something to do is better than sitting around waiting. I would also bet the team figured out ways for those with young kids to not draw that shift. I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees
 
2012-10-30 12:02:00 PM
How Rackspace prepared it is Virginia data centers for Hurricane Sandy?

What does that even mean?
 
2012-10-30 12:03:38 PM
It is data centers.

Everywhere.
 
2012-10-30 12:05:39 PM
*Twitch*
 
2012-10-30 12:05:51 PM

Englebert Slaptyback: How Rackspace prepared its Virginia data centers for Hurricane Sandy.


If they had a decent BC/DR plan tested and ready to go, they shouldn't have needed much prep work.


yeah I'm sure Rackspace, only one of the most respected and advanced shops on the planet, is smacking themselves on the forehead saying "if only we had a disaster recovery plan!"

:eyeroll:
 
2012-10-30 12:09:25 PM

stuhayes2010: I expected much better pics on something called Rackspace.com.


To be honest, I had to google that. But rackspace does sound like an excellent website.
 
2012-10-30 12:12:05 PM
This was very boring
 
2012-10-30 12:12:06 PM
Nobody NOBODY farks with rackspace. Rackspace is a bad mutha
 
2012-10-30 12:13:22 PM
Rackspace: 14 years 231 days without an onsite death that you can pin on them.
 
2012-10-30 12:14:50 PM

mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.


How does a server admin get killed at work?
 
2012-10-30 12:15:56 PM

NickelP: mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.

How does a server admin get killed at work?


I believe it's caused by brain seizures when a user actually finds a way to contact them directly.
 
2012-10-30 12:18:25 PM
Anyone remember the name of the hosting provider/isp that lived thru katrina and had blog updates? Never lost full connectivity...
 
2012-10-30 12:22:30 PM

NickelP: mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.

How does a server admin get killed at work?


At Rackspace? Probably an accident on the giant slide they have. Their San Antonio site is an abandoned mall. They've got space, so they put in fun stuff, and one of their most well-known toys is a slide from the top floor of the mall to the ground.

I need to check their openings again soon.
 
2012-10-30 12:24:52 PM

zedster: I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees


I work in the data center industry so I talk to a lot of colo folks. I've heard numerous stories from former rackspace customers about behavior that would otherwise get a company sued. Specifically, their support department is run by a despot.
 
2012-10-30 12:26:02 PM
I wonder how bad the conditions have to be before they announce "abandon ship!!!"
 
2012-10-30 12:29:57 PM

stuhayes2010: I expected much better pics on something called Rackspace.com.


you and me both. it's like there's some rule on not having pics for a geek article. it must be read! what a bunch of maroons. why do people dealing with technology have a b&w print press mentality?

/ titty sprinkles
 
2012-10-30 12:35:38 PM

mikaloyd: Nobody NOBODY farks with rackspace. Rackspace is a bad mutha


Shut yo mouth!
 
2012-10-30 12:38:40 PM
so a couple of generators and people on standby? are there any colos not operating like this during a storm?
 
2012-10-30 12:39:35 PM

chasd00


Englebert Slaptyback: How Rackspace prepared its Virginia data centers for Hurricane Sandy.


If they had a decent BC/DR plan tested and ready to go, they shouldn't have needed much prep work.

yeah I'm sure Rackspace, only one of the most respected and advanced shops on the planet, is smacking themselves on the forehead saying "if only we had a disaster recovery plan!"

:eyeroll:


How the hell does your response relate in any way to what I said? I did not say they said anything of the sort.

The idea behind BC/DR is so you DON'T have to have hectic fire drills in an actual emergency. If you manage to RTFA you'll see that Rackspace basically said "we're all set for electricity, staffing, and ticketing, so we're good", which sounds like the opposite of a hectic fire drill - and exactly what should be expected from them.

BTW, Mr. Expert, we're talking about BC here - not DR. Maybe you should go learn the difference.
 
2012-10-30 12:39:47 PM

zedster: NickelP: I feel bad for all the people that work there. "yeah we are sorry your house is probably getting washed away, and you have no way to even communicate with your wife and kids to see if they are safe, but someone has to answer these phones and be available in case a server needs manually reset or something."

If you do system admin it's part of the job. They had supplies and in someway having something to do is better than sitting around waiting. I would also bet the team figured out ways for those with young kids to not draw that shift. I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees


Another reason kids are bullshiat, parents don't have to work shiatty shifts, they get maternity/paternity leave, and they up the insurance premiums of everyone else.
 
2012-10-30 01:13:01 PM
I'm amazed that ANYONE prepared for this storm. Based on all the news stories, it comes across like Sandy came out of nowhere in a matter of a couple of hours and that everyone was just going about their everyday business when it hit.
 
2012-10-30 01:15:50 PM

NickelP: mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.

How does a server admin get killed at work?


they fall victim to the deadly network cable snare jungle. eventually their dessicated corpse is found be later adventurers who don't wander in alone
 
2012-10-30 01:22:40 PM

Honest Bender: zedster: I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees

I work in the data center industry so I talk to a lot of colo folks. I've heard numerous stories from former rackspace customers about behavior that would otherwise get a company sued. Specifically, their support department is run by a despot.



I work in the data center industry and I talk to lot of colo folks. Hosting, too. I've heard numerous stories from competitors that wish their growth rates, ARPU and margins looked anything like Rackspace's. Successful company is successful, sour grapes is sour grapes.

If you're talking about Boogie, I have nothing to say about the man.
 
2012-10-30 01:29:07 PM

willfullyobscure: Successful company is successful, sour grapes is sour grapes.


Meh, I'm not financially invested in my employer beyond my next paycheck. To be honest, we're growing as fast as we can get new cabinets wired up. Success and growth are not an issue around here.

Take the anecdotal hearsay of an anonymous internet poster for what it's worth. Just relaying my experiences. I've had very little contact with the company directly.
 
2012-10-30 01:31:47 PM

netcentric: This was very boring


Fascinating! No, boring. You're right.
 
2012-10-30 01:46:45 PM

Honest Bender: willfullyobscure: Successful company is successful, sour grapes is sour grapes.

Meh, I'm not financially invested in my employer beyond my next paycheck. To be honest, we're growing as fast as we can get new cabinets wired up. Success and growth are not an issue around here.

Take the anecdotal hearsay of an anonymous internet poster for what it's worth. Just relaying my experiences. I've had very little contact with the company directly.


They overcharge by about 1000% but churn is very low. The whole cloud thing is their biggest headache, ARPU is 1/100th for IaaS as it is for physical hosts.
 
2012-10-30 02:00:36 PM

Girion47: zedster: NickelP: I feel bad for all the people that work there. "yeah we are sorry your house is probably getting washed away, and you have no way to even communicate with your wife and kids to see if they are safe, but someone has to answer these phones and be available in case a server needs manually reset or something."

If you do system admin it's part of the job. They had supplies and in someway having something to do is better than sitting around waiting. I would also bet the team figured out ways for those with young kids to not draw that shift. I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees

Another reason kids are bullshiat, parents don't have to work shiatty shifts, they get maternity/paternity leave, and they up the insurance premiums of everyone else.


True, but the other side of the argument is that without kids, it means THE END OF THE HUMAN RACE.
 
2012-10-30 02:30:48 PM
Cat 1 Hurricane in florida "hey guys.. it's raining"

Cat 1 Hurricane a wee bit farther up in to "perfect storm: the movie" territory... "oh no! it's all over!"
 
2012-10-30 02:34:52 PM

NickelP: mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.

How does a server admin get killed at work?


By trying to be an electrician and installing a breaker live in a pdu without the the correct safety precautions and wrong PPE?
 
2012-10-30 02:49:10 PM

i.r.id10t: Anyone remember the name of the hosting provider/isp that lived thru katrina and had blog updates? Never lost full connectivity...


That was DirectNIC. I still use them,.
 
2012-10-30 02:49:12 PM

xtonyx: Cat 1 Hurricane in florida "hey guys.. it's raining"

Cat 1 Hurricane a wee bit farther up in to "perfect storm: the movie" territory... "oh no! it's all over!"


To understand it better, think about how your neighbors would panic if the hurricane were followed by two inches of snow.
 
2012-10-30 02:51:43 PM

Petey4335: NickelP: mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.

How does a server admin get killed at work?

By trying to be an electrician and installing a breaker live in a pdu without the the correct safety precautions and wrong PPE?


Something like this. Or like the first place I worked, where they were rearranging cables under the raised floor, and found one that wasn't connected at one end. They started dragging it out, the loose end hit one of the flooring supports, and there was a big flash-bang. Turned out the other end was connected to a live 440 3-phase feed.
 
2012-10-30 03:13:20 PM

NickelP: mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.

How does a server admin get killed at work?


We know it's not by Snu-snu.
 
2012-10-30 03:18:33 PM

toraque: Girion47: zedster: NickelP: I feel bad for all the people that work there. "yeah we are sorry your house is probably getting washed away, and you have no way to even communicate with your wife and kids to see if they are safe, but someone has to answer these phones and be available in case a server needs manually reset or something."

If you do system admin it's part of the job. They had supplies and in someway having something to do is better than sitting around waiting. I would also bet the team figured out ways for those with young kids to not draw that shift. I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees

Another reason kids are bullshiat, parents don't have to work shiatty shifts, they get maternity/paternity leave, and they up the insurance premiums of everyone else.

True, but the other side of the argument is that without kids, it means THE END OF THE HUMAN RACE.


where's the downside?
 
2012-10-30 03:36:46 PM
I understand that server monkeys are also big fans of creating and then ignoring and occasionally falling through large holes in floors.
 
2012-10-30 03:47:37 PM
i50.tinypic.com
 
2012-10-30 03:47:38 PM

willfullyobscure: Honest Bender: zedster: I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees

I work in the data center industry so I talk to a lot of colo folks. I've heard numerous stories from former rackspace customers about behavior that would otherwise get a company sued. Specifically, their support department is run by a despot.


I work in the data center industry and I talk to lot of colo folks. Hosting, too. I've heard numerous stories from competitors that wish their growth rates, ARPU and margins looked anything like Rackspace's. Successful company is successful, sour grapes is sour grapes.

If you're talking about Boogie, I have nothing to say about the man.


interesting. I'm currently looking for Us based Co lo , full rack, min 20 megs sustained, with a company that is willing to receive and install the hardware into the site, plus offers hands on service a la cart. East to start. maybe west as well. It doesn't seem that Rackspace offers that, at least from their website. any recommendations?
 
2012-10-30 03:48:51 PM
Meanwhile: Gawker prepared its sites by mending all the screen doors and emptying the buckets under their leaky A/C unit.
 
2012-10-30 04:07:42 PM

NickelP: mikaloyd: Rackspace: 14 years 231 days without an onsite death that you can pin on them.

How does a server admin get killed at work?


Someone fat fingers kill -9 ?
 
2012-10-30 04:44:03 PM

willfullyobscure: Honest Bender: zedster: I have to say in my interactions with Rackspace (always their Texas teams) they seem like a good company and nothing in this statement puts up any red flags. It sounds like they were doing right by both their customers and employees

I work in the data center industry so I talk to a lot of colo folks. I've heard numerous stories from former rackspace customers about behavior that would otherwise get a company sued. Specifically, their support department is run by a despot.


I work in the data center industry and I talk to lot of colo folks. Hosting, too. I've heard numerous stories from competitors that wish their growth rates, ARPU and margins looked anything like Rackspace's. Successful company is successful, sour grapes is sour grapes.

If you're talking about Boogie, I have nothing to say about the man.


I have been a customer with Rackspace for a few years now. I have had zero problems, issues, or complaints. I have 24 hour access to three rotating teams of experienced admins and all my interactions with them have been quick, polite, and effective. I deal with the British team the most since I work odd hours.

My anecdote:
In 2009 I was removing an old temporary system and prepping the server for a new system. Since the old app was the only thing on the server at the time, I started changing the Apache and PHP configs.

Three minutes after I changed the Apache config to remove some settings for the old system (had it pointing nowhere for a few minutes), I received an urgent email from Rackspace stating there was a problem with my Apache server and that a ticket had been opened. They also called and left me a voice mail.

I then received a second email one minute later stating they had checked on it and saw that I was in the middle of making changes and saw my new settings were in effect.

Needless to say, I was very impressed.
 
2012-10-30 06:30:24 PM
After reading some of the outage information for some of the big datacenters in New York (see this announcement from Internap as an example) it seems interesting to note the failures they're running into: sure, they have big chunky UPSs and backup generators but the generators run from relatively small "day tanks" that only can supply a few hours of power. The big storage tanks are at the ground/basement level and have pumps to pump fuel up to the day tanks which, like the severs and generators, are on above-ground levels.

Unfortunately, the flooding destroyed the fuel pumps (and possibly contaminated the fuel in the bulk tanks) meaning that they're limited to what's in the day tanks only. Not really something you plan for in the normal "utility power out, switch to generator" scenarios. Rain, winds, etc. are fine, but massive flooding? Not so much.

While it does suck to have a facility go offline, it's not the end of the world. (Though it's definitely a concern when it's something like a hospital where lives are actually on the line; fortunately the incident with the NYU hospital seems to have not resulted in any casualties, thank goodness.)

I find it interesting (in a somewhat morbid way) to see how these sorts of things fail and otherwise well-prepared organizations end up failing in some way. I like being able to learn from other people's mistakes and try to avoid such things in the future.

/my sites, tiny as they are, are hosted in a Level3 facility in Baltimore. They lost one of their redundant fiber connections but that caused no disruption.
 
Displayed 50 of 51 comments

First | « | 1 | 2 | » | Last | Show all

View Voting Results: Smartest and Funniest


This thread is closed to new comments.

Continue Farking
Submit a Link »






Report