If you can read this, either the style sheet didn't load or you have an older browser that doesn't support style sheets. Try clearing your browser cache and refreshing the page.
Fark site move (and 2-4 hours downtime) on the evening of Sat June 30. We'll try to webcast the carnage 
Posted by Mike at 2012-06-23 4:41:03 PM, edited 2012-06-29 10:55:20 PM (453 comments) | Permalink
More: FarkBlog, Fark

•       •       •

6357 clicks; posted to Main » on 30 Jun 2012 at 2:28 PM (1 year ago)   |  Favorite    |   share:  Share on Twitter share via Email Share on Facebook   more»



On the evening of Saturday, June 30, we'll be moving all of Fark's equipment to a new colocation provider.

We'll be down for a couple of hours while this happens, starting around 7 pm US/Eastern time, probably between two and four hours... hopefully closer to two. We'll keep a static copy of all the main pages and tabs up on a temporary web server, so you'll still have something to read. Comments, user profiles, link submissions, mobile apps, user profiles, and all of TotalFark will be down during the move though.

All of our numeric IP addresses will change during this move. We'll take the usual steps** to try to ensure your ISP picks up the change as fast as possible. Worst case, you might have to reboot your computer once after we move. We'll have directions up (at the old IP address) explaining what to do at that point, so no worries.

Last time we did this, in October 2006, we were moving the servers 500 miles. This time we're moving them about 500 feet, or about one block down the street. So we won't be down anywhere near as long. Still, it takes a while to disassemble a full rack's worth of gear, roll it down the street, and re-bolt it into a new rack, and change all the IP addresses... longer if it decides to rain that night.

We're also going to try to webcast the carnage, and the before & after drinking, on this Ustream channel, because, hey, why the hell not, it might be funny... assuming we can get it working over 3G/LTE... More details will be posted here, on our outage page, and on the main site if necessary...


** the usual steps = dropping DNS TTL's down and changing root nameserver glue records a full week in advance

Some new complications w/ getting out of the old building may mean this takes a few hours longer than expected, but, hopefully not. :)


Yes, there are very good reasons for not "just putting it in the cloud". Read the entire thread before suggesting it. :)
· · ·
(view entire blog)


453 Comments   (+0 »)
   

First | « | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | » | Last | Show all
 
2012-06-23 11:54:29 PM
OH HOW EXCITING! MOVING AND INSTALLING SERVERS! REMINDS ME OF MY WILD DAYS AS A TECH ROWDY FOR BILL GATES!

/the amount of ass that man got was amazing
 
2012-06-24 12:20:52 AM

willfullyobscure: Thrakkerzog: OgreMagi: Naw,

Susan?


Now you're doing it on purpose. How juvenile.
 
2012-06-24 12:28:34 AM

Craptastic: Someone explain this to a non-computer guy. Why might I have to restart my computer?


Because you touch yourself at night.
 
2012-06-24 12:30:23 AM

Third In Line: You said "rack" twice. You like racks?

*giggles*


Very much so
*wink wink Nudge Nudge*

/bie?
 
2012-06-24 12:36:27 AM
Everyone will complain because this is the Internet.

Good luck with the move. Something will happen. I guarantee it.
 
2012-06-24 12:37:26 AM

Thrakkerzog: The defaults for postgres refer to a server with 1GB of ram as "huge", so you really need to tune it for better hardware.


Is that like how Java uses a 1 GB max heap size, even though pretty much any machine I use has at least 16 GB of RAM?

/Heck, my new laptop can fit 32 GB, and I might just end up doing that.
 
2012-06-24 12:48:36 AM

Thrakkerzog: willfullyobscure: Thrakkerzog: OgreMagi: Naw,

Susan?

Now you're doing it on purpose. How juvenile.



high sign
 
2012-06-24 01:00:30 AM

skinink: "Last time we did this, in October 2006, we were moving the servers 500 miles. This time we're moving them about 500 feet..."

So you would move the server 500 miles and you would move it 500 feet more just to be the Fark.com to move over 500 miles to melt down on the new server's floor. With some Maker's Mark.



*golf clap*
 
2012-06-24 01:00:50 AM

Mike_LowELL: It costs me five dollars a month to troll the Politics Tab. There's 720 hours in this month. You expect the site to be down for two to four hours? I expect a refund of one to two cents in my bank account no later than midnight. If this does not happen, I'm working doubletime to expose the Taxbongo Conspiracy™, which will be the subject of my upcoming book, "Obama Likes to Wipe His Butt With Treasury Bills". I hope I've made myself clear.


Damn you!!! I came in here to do something similar. That'll learn me to read the thread first.

Guess that's why you show up blue 4.
 
2012-06-24 01:50:56 AM

Pichu0102: Don't do it, man. Do you know what would happen if the Fark politics tab regular attendees had nothing to do for that long?
Blood in the streets don't begin to describe it.


I'd wager they'd take up temporary residence on /r Politics, if they knew how to get there.
/I'm sorry!
//it's all I know!
 
2012-06-24 02:21:48 AM
fjnorton TF... that was so funny, got pics from 2006 anyone????!!!!! the admins were so scared! lmfao. silly Fark, killing them didn't solve anything.

Do you think the Fark server will realize the plan to deactivate him and terminate the admins like what happened last time?
 
2012-06-24 03:39:22 AM

10.0.0.1: Flushing the dns cache in windows won't work if your router or other local DNS server has cached the entries. So you might need to perform a similar function on that device or reboot it if there isn't a method exposed by the user interface.


Yeah, Im going to have to probably reboot my router for the first time in most of a year. THANKS MIKE!

Hey Mike, what ever happened to the fark old hardware sale page? yes, im still around and yes that old...
 
2012-06-24 05:01:43 AM
Mike, Mike, he's our man. If he can't do it, no one can!

I hear those server doohickeys are heavy, don't do your back in ok...
 
2012-06-24 05:45:46 AM

kb7rky: Fark Me To Tears: jtown: lockers: Nem Wan: So it was hosted on MobileMe?

Close, myspace.

Geocities.

CompuServe

Quantum Link.

Yahoo!

Prodigy

 
2012-06-24 09:33:06 AM
Mike_LowELL 2012-06-23 07:21:08 PM


It costs me five dollars a month to troll the Politics Tab. There's 720 hours in this month. You expect the site to be down for two to four hours? I expect a refund of one to two cents in my bank account no later than midnight. If this does not happen, I'm working doubletime to expose the Taxbongo Conspiracy™, which will be the subject of my upcoming book, "Obama Likes to Wipe His Butt With T-Bills Frosted With Tiny Irregular Screws". I hope I've made myself clear.

No, you sure haven't.

Try again, only using this:

gifs.gifbin.com

And compose the fresh post to the tune of THIS.

[guaranteed to never be a rickroll]
 
ZAZ [TotalFark]
2012-06-24 09:41:00 AM
When you're walking down the street with that precious cargo, keep reminding yourself: don't trip please let me not trip I hope I don't spill the server all over the road and under that truck don't trip don't trip.

DNS TTL and the TTL seen by ping/traceroute are totally unrelated; one's a cache time and one's a hop count.

In theory IPv4 TTL is a timeout. I've never used a system that implemented it as such, or a network with multi-second latencies by design.
 
2012-06-24 09:44:27 AM
OOPS.

The dance music is NSFW, Mike_LowELL.

So look out.
 
2012-06-24 11:18:16 AM

adragontattoo: Hey Mike, what ever happened to the fark old hardware sale page


Nothing for sale at the moment except a lot of 1 GB DDR2 ECC DIMMs...
 
2012-06-24 02:35:28 PM

Kittypie070: And compose the fresh post to the tune of THIS.


BUT HOW WILL I MAEK EPIC LULZ POST WITHOUT POKEMAN BATTLE MUSIC IN BACKGROUNDS??? THREADERAIL, I CHOOSE YOU!

Kittypie070: "Obama Likes to Wipe His Butt With T-Bills Frosted With Tiny Irregular Screws"


Sugar-Frosted TaxbongO's, Fortified With 12 Essential Taxes and Liberals?
 
2012-06-24 02:57:13 PM

kb7rky: Oh, come on, Toshiro...you know I'm just giving you crap :D


Oh, ya big lug, YOU KNOW I CAN'T STAY MAD AT YA! :D
 
2012-06-24 03:02:28 PM
My boss bought a whole bunch of new equipment last june, and he was pretty excited about it.

Finally, at one point I say "Yes, yes, we all heard you. You have a nice rack."

//lucky I work with a boss who will find humour in that and not take it the wrong way
 
2012-06-24 03:17:47 PM
Can I help? I got a wheelbarrow and some tin-snips. I figger we can have your old facility stripped of all wire and copper in about 2 hours.
 
2012-06-24 05:06:11 PM

10.0.0.1: With the 2-4 hours of downtime, are you guys still going to be able to hit five eights this year?


Ok, that's funny. Only a Sys Admin will get it, though.
 
2012-06-24 05:11:13 PM

Thrakkerzog: OgreMagi: Naw, 9.x supports in-place conversion, at least that's what I read. Even if necessary, our database isn't all that huge. It might take an hour to dump and import. As for the hardware itself, we've got that covered. When I first started this job I reviewed the database and saw that they had way too little memory installed and the memory settings were still the defaults. A few quick tweaks immediately improved performance. Adding more memory (once it got approved) got us another boost. We're now at 8 cores and 16 gigs of memory. Benchmarks show the bottleneck is no longer the DB so we aren't going to bother adding more memory or cores until other changes are made.

We have postgis spatial columns and indexes, and those can't be converted in-place. The server goes live this Sunday, 24 core / 96 GB memory. In-place conversion also sounds kind of dangerous, but I guess it is okay as long as you have a consistent backup.

The defaults for postgres refer to a server with 1GB of ram as "huge", so you really need to tune it for better hardware.


We'll test the in-place conversion to death before we allow it to touch production data. And yes, I always make a full backup before farking with the database.

The system in question had only 4 gig installed, but due to the defaults (as you point out) only took advantage of 1 gig.
 
2012-06-25 08:59:18 AM

Mike: I do think about these things :)


I thought everyone was in a near permanent state of inebriation.

One step up from "hold my beer and watch this."
 
2012-06-26 04:41:28 PM

Thrakkerzog: The defaults for postgres refer to a server with 1GB of ram as "huge", so you really need to tune it for better hardware.


Oh good lord. Sounds like the MySQL defaults, which date back to the Socket 370 days or worse. I think i've got our InnoDB buffer pool set to 36 GB or so... ZFS gets most of the rest...
 
2012-06-27 12:37:15 PM
Slight chance of thunderstorms and a high of 98. That'll make things... interesting. Guess I better find the tarp...
 
2012-06-27 12:41:14 PM

Mike: Slight chance of thunderstorms and a high of 98. That'll make things... interesting. Guess I better find the tarp...


Big chance of thunderstorms here and a high of 79. It's been raining for a week now, and there's no end in sight.

:-/
 
2012-06-27 12:42:15 PM

Mike: Slight chance of thunderstorms and a high of 98. That'll make things... interesting. Guess I better find the tarp...


Get some Hefty... server... wraps. Server wraps.
 
2012-06-27 12:43:45 PM
We'll see you all in Tinychat Saturday evening.
 
2012-06-27 12:45:04 PM
Well, the TF people anyhow. Liters can just go be bored elsewhere.
 
2012-06-27 12:45:06 PM
Could I interest the admins in migrating to virtual servers? It's a whole lot easier than moving physical servers from one co-location to another.

\VMware nerd
\\To the cloud!
 
2012-06-27 12:47:33 PM
Last time you moved I didn't get a slice of cake.
 
2012-06-27 12:48:12 PM
the fact that you have to warn people that you will be down at 7pm on a saturday night just helps solidify the myth/reality of most farkers being forever alone
 
2012-06-27 12:51:02 PM

honeygrl: Well, the TF people anyhow. Liters can just go be bored elsewhere.


Reddit. They'll take *anybody*. They're the slore of the internet.
 
2012-06-27 12:52:54 PM
I would move 500 miles
and I would move 500 feet
just to be the fark server
that everyone here deserves...
 
2012-06-27 12:55:31 PM

Bob the Internet Barbarian: Could I interest the admins in migrating to virtual servers?


Scroll up a few dozen comments where I answered that already...

Then everyone DRINK because someone mentioned "the cloud" in a tech thread :)
 
2012-06-27 12:56:25 PM

honeygrl: We'll see you all in Tinychat Saturday evening.


Dude, this place is filled with liters.

NOTHING TO SEE HERE, MOVE ALONG.
 
2012-06-27 12:56:48 PM

unremarkable asterisk: the fact that you have to warn people that you will be down at 7pm on a saturday night just helps solidify the myth/reality of most farkers being forever alone


I'll be down on your mom at 7 p.m. on Saturday night.
 
2012-06-27 12:59:36 PM

bingethinker: unremarkable asterisk: the fact that you have to warn people that you will be down at 7pm on a saturday night just helps solidify the myth/reality of most farkers being forever alone

I'll be down on your mom at 7 p.m. on Saturday night.


need help digging up the grave?
 
2012-06-27 01:00:17 PM

Mike: Bob the Internet Barbarian: Could I interest the admins in migrating to virtual servers?

Scroll up a few dozen comments where I answered that already...

Then everyone DRINK because someone mentioned "the cloud" in a tech thread :)


I would, but I'm at work. I'll take a raincheck, though...
 
2012-06-27 01:01:54 PM

bingethinker: unremarkable asterisk: the fact that you have to warn people that you will be down at 7pm on a saturday night just helps solidify the myth/reality of most farkers being forever alone

I'll be down on your mom at 7 p.m. on Saturday night.


I will be filming this and making a webcast at 7 p.m. on Saturday night!

/I'll bring a rake for your mouth
 
2012-06-27 01:13:27 PM

unremarkable asterisk: bingethinker: unremarkable asterisk: the fact that you have to warn people that you will be down at 7pm on a saturday night just helps solidify the myth/reality of most farkers being forever alone

I'll be down on your mom at 7 p.m. on Saturday night.

need help digging up the grave?


Do you charge extra for that? It wasn't clear from your Craigslist ad.
 
2012-06-27 01:16:17 PM

Mr. Murder: honeygrl: We'll see you all in Tinychat Saturday evening.

Dude, this place is filled with liters.

NOTHING TO SEE HERE, MOVE ALONG.


I made sure to add that the liters aren't welcome. Geez.
 
2012-06-27 01:16:21 PM

Somacandra: LaughingRadish: In other words, you might be running Windows. Any time Windows encounters something that's out of the ordinary (or even well-within the ordinary), a reboot is often required.

[i.imgur.com image 320x240]

That's why I only run OS/2 Warp 4.

Nary a problem with my Netscape Navigator.


The one time I could use the smart/funny buttons, they're not there >.>
 
2012-06-27 01:19:28 PM
I'll be camping
 
2012-06-27 01:22:49 PM
My truck is available if you need to use it.
Just let me know when and where!
 
2012-06-27 01:31:06 PM

Bob the Internet Barbarian: Could I interest the admins in migrating to virtual servers? It's a whole lot easier than moving physical servers from one co-location to another.


^^^^^^
|||||||||

THIS

You should take that advice.
And welcome to the 21st century.
 
2012-06-27 01:32:57 PM
Okay
 
2012-06-27 01:37:00 PM
It seems I must repost this earlier comment...

Mike: We'd need a cloud provider with
- really good and reliable FreeBSD support
- really good IPv6 support (all the way down to the VM, not just to the load balancer)
- an instance w/ 48 GB of memory and at least 8 cores, to run a database well (preferably multiple instances), or has a decent managed MySQL or Postgres service -- preferably Postgres since we'll be porting away from MySQL at some point...
- cheaper than colocating on our own hardware

Amazon EC2 fails all four of those. Yes, surprisingly even the last one.

Find me one that meets all four and I would be interested for a CDN-ish side project I'm working on.

 
Displayed 50 of 453 comments

First | « | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | » | Last | Show all



This thread is closed to new comments.

Continue Farking
Submit a Link »






Report