Server Migration Complete
*sigh*. Another weekend wasted with a server migration (with 2 weeks of preparation). I had some overheating and defective hardware issues on the old machine and was forced to migrate though at least I'm in the same datacenter. If you're curious, you can see my server status here: http://72.232.184.154/~stats/.
I'll be posting a lot more now that this work has been done.
Internet Explorer, How I Loathe Thee
Folks, if you see that my web page looks wacky in Internet Explorer, please tell me. I only use Firefox (IE users should be, well, that's just not nice to say on a family blog...) and IE doesn't adhere to quite a few W3C CSS standards. This means that it may look fine for me in Firefox but the sidebar or title is missing in Internet Explorer.
(what it shouldn't look like)
The sidebar disappeared recently because I had an image that was over 450 pixels in width which caused internet explorer to move the sidebar down to the bottom. The page looked fine for me however in Firefox and Opera. I'll work on preventing this but if you see anything wacky, please drop me a mail or leave a comment. Thanks!
Painful Server Woes
You can see my full system performance graphs at http://www.capnstats.com/stats/
Ok, as if there is another kind of server woe...
Anyway, I apologize to all of my friends and customers for the recent outages and pledge that I'm doing everything within my power to fix the problems. Saturday night's incident was completely unacceptable and one of those "perfect storms" I have frequent nightmares about. Around 3:15 pm on Saturday 1/7, my server went offline. It went down so fast that syslogd and a maintenance script I have running couldn't send the "call for help" text message to my cell phone.
To make matters worse, my cell phone had died sometime early Saturday morning so I also didn't receive the pages from my two 3rd party monitoring services (Alertra and websitepulse). And to compound problems further, I was out shopping and away from the computer from about 9 am Saturday morning to 11:30 pm Saturday night (which is damn near a world record for me).
I wasn't aware of any problem until I checked my e-mail and had several hundred messages from my monitoring services and a message from Michael Hanscom of Eclecticism.
Also, my datacenter lost it's internet connectivity making it impossible for me to open a trouble ticket or reboot my server remotely (via a Cyclades unit). When connectivity was restored around 11:30, I rebooted my server and attempted the autopsy. While copying the log files to a remote server I manage in Ashburn, VA, the server went down again without logging anything of course. This incident required console access by the datacenter.
Both the datacenter and I think it's a hardware issue but can't find the culprit. No memory errors, I/O errors or load errors (that would cause the CPU to overheat) so I still have absolutely no idea.
Please take comfort in the fact that your data and website structure are very safe. In addition to having full, off-site backups, I rsync your data to two locations every hour and have been since fall of 2004.
Please bear with me during these trying times and take a peek at my server stats at http://www.capnstats.com/stats/.
Thanks for your patience everyone,
Michael
Still Alive
Sorry folks for the lack of updates but I've been spending a majority of my time supporting customers and staying afloat at work. I also want to upgrade to WordPress 2.0 and make a significant design change (yes, that again).
Thanks for your patience.
--Michael
Never Again!
at least until the next time. I just completed the hairiest server migration of my life and I hope I never have to do that again. I had a drive failure, two switch failures (one on the old DC and one at the new), 1 NIC replacement and a slew of OS errors (I run CentOS 4.1).
In all, I moved all of the accounts on this server back and forth 5 times in 30 hours. 21.97 GB, five times, lol. On a good note though, most of my customers never knew any of this was going on. With some DNS trickery and some slick rsync usage, only one hostee, Michael Hanscom of Eclecticism
experienced any problems and we quickly cleared that up over AIM this morning.
If you do notice any oddities, please e-mail me.
