Skip to content
Do you have adblock enabled?
 
If you can read this, either the style sheet didn't load or you have an older browser that doesn't support style sheets. Try clearing your browser cache and refreshing the page.
Oops
Posted by Mike at 2007-12-04 10:17:37 PM (7 comments) | Permalink
More: FarkBlog

•       •       •

3 clicks; posted to Main » on 04 Dec 2007 at 10:17 PM (15 years ago)   |   Favorite    |   share:  Share on Twitter share via Email Share on Facebook



Well, I did say "expected" outages and "forseeable" future...

InnoDB decided to take a dump on itself earlier tonight, so I had to recover the entire database from a backup. Fortunately we have a replica set up for exactly this situation, so we didn't lose anything. Sucks being down for 3 hours but it could have been much, much worse.

Archives are still restoring; that should be done by midnight. Til then, any links older than 60 days probably won't work.
· · ·

(view entire blog)
7 Comments     (+0 »)
 
skaya
2007-12-05 12:46:51 AM  
I like that the original announcement said

"unplanned database crash"

the obvious question has to do with the planned database crashes .....

[smile]

of course, you just might want to hold on to those older systems you are thinking of selling off.
 
Mike [TotalFark] [OhFark]
2007-12-05 1:51:45 AM  
Nah. It wasn't a hardware problem.
 
omega male
2007-12-05 1:55:28 AM  
have you blamed the gnomes yet?
 
ZAZ [TotalFark]
2007-12-05 4:06:50 AM  
Nah. It wasn't a hardware problem.

Funny that it happened after a hardware upgrade. Fark had a massive failure with data loss a few years back due to hardware weirdness (motherboard not getting along with SCSI controller?).
 
realwx
2007-12-05 2:35:34 PM  
So... does this mean we can blame Drew for spilling beer on the database again?
 
Mike [TotalFark] [OhFark]
2007-12-05 8:28:49 PM  
Yeah, after a hardware upgrade, but this particular one was me fat-fingering something when I was in a hurry.

There are some weird random glitches with tables saying they're corrupt when they shouldn't be -- causes the db to die for about 30 seconds and come right back. That might be hardware and I'll look at that overnight. But last night's problem was all me. :p

The problem a few years back ended up being a motherboard chipset bug. Specifically, some PCI-X cards don't work reliably at 66 mhz on the AMD760 chipset. 33 mhz is fine. It took forever to find because a) hardware was 600 miles away at the time (it's now 80 miles away) and b) wtf, chipset bug?
 
No_47
2007-12-06 1:52:57 AM  
It went for a minute there, again
 
Displayed 7 of 7 comments


Redisplay/refresh comments
This thread is closed to new comments.

Continue Farking

Want more stories with less ads? Try

TotalFark

It's what the cool kids are doing.
It also helps us pay the bills.

learn more | sign up
On Twitter

  1. Links are submitted by members of the Fark community.

  2. When community members submit a link, they also write a custom headline for the story.

  3. Other Farkers comment on the links. This is the number of comments. Click here to read them.

  4. Click here to submit a link.