/dev/blog

Turns out that 320GB “cheap investment” saved me from losing a lot of important data, almost a year later I have lost my 250GB Seagate hard drive. Some data is still recoverable, but I can’t rely on faulty hardware and went ahead with badblock’s destructive write-mode test (-w).

Here’s a sample of dmesg, smartd, and badblocks’s output. They are all a pretty good warning signs of the impending hardware failure to come.

$ dmesg
[…]
hdb: task_in_intr: status=0×59 { DriveReady SeekComplete DataRequest Error }
hdb: task_in_intr: error=0×40 { UncorrectableError }, LBAsect=170135808, high=10, low=2363648, sector=170135808
ide: failed opcode was: unknown
end_request: I/O error, dev hdb, sector 170135808
Buffer I/O error on device hdb, logical block 21266976

$ badblocks -wsv /dev/hdb
Testing with pattern 0xaa: done
Reading and comparing:
178003968
178003969
178003970
178003971
178003972
178003973
178003974
178003975
[…]

No more tarballs for me, following papmech this blog is now running Wordpress 2.3-alpha revision 5773 via Subversion (SVN). I have set up a cron job to run every 8 hours to automate the process, so I will always be running the latest development code.

The trunk contains the latest development code, which is currently tagged as 2.3-alpha. The major versions of WordPress (1.5, 2.0, 2.1, and 2.2) that are still maintained by developers and are split into branches.

Trying to run beta (let alone pre-beta) software in a stable environment is never suggested, especially when it changes every 8 hours. From personal experience I can say that living on the bleeding edge has its benefits but they are easily outweighed. This results in a small handful of errors from time to time, or the entire database disappearing altogether.

If absolutely necessary; my fall-back plan is to drop back from /trunk to the stable /branches/2.2 revisions.

My old PGP key, 0×8D1F7423, expired last April. Once again I generated a key that expires in 365 days, exported my public key to key servers, and generated a revocation certificate for it. Saving one revocation certificate to a file and printing the other in the event that my hardware becomes misplaced, destroyed, or otherwise ’superseded’.

I navigated to the directory and double clicked the revocation certificate expecting gedit to appear, but nothing happened. After launching GnuPG I was able to confirm what had just occurred.

Seahorse (a GnuPG front end for Gnome) imported the signature into my keyring. Ah Shit.

Normally I’d restore the backup, but the key was so new I didn’t have time to make one. Luckily I had yet to publish the newly revoked key to any key servers, so I figured there had to be a way to strip out the revocation signature as it only existed on my PC. After a little research I stumbled upon a message board giving me the solution I needed.

Computer forensics and cryptography have always been a hobby of mine.

Yuzu is a dedicated box in my basement with a LAMP installation that I can just ssh or rsync into form any computer within my home. As of April 9th this sever is now running Debian 4.0r0, or simply “Etch”. 21 months in development and worth every minute, Debian Etch is by far one of the most stable Linux distributions I’ve worked with to this day.

I’ve been experimenting with Debian’s ‘testing’ repository on my PC for a few months now, now that they officially support the AMD64 architecture. I’ve grown tired of Fedora, although it is professionally polished, Fedora has always lacked the vibrant community of Ubuntu. Not to mention yum (think: apt-get but for RPM’s) has always been a big pain (think: RPM hell… but automated).

And for those who were wondering; The last Debian Sarge install topped out at 149 Days, 0 Hours, 55 Minutes, and 32 Seconds of uptime.

100 days, 4 hours, 7 minutes, and 28 seconds.

Ok, lets see how long this server can stay up without a battery backup supply connected to it.

Seriously, how long does it take a tech-savvy person such as myself to copy a 4.2GB file from one partition to another? 5 hours. But considering that I learned something, it was well worth it.

Here’s what I discovered won’t work:

• I can’t write directly to NTFS from Fedora, which is understandable as NTFS is by far the largest, yet undocumented, project ever reversed-engendered.
• FAT32 has a size limit of 4GB, so using my iPod wasn’t an option. The file was already compressed, otherwise I might have been able to get it down enough with bzip2.
• Just like FAT32, it is not possible to burn a file over 4GB to a DVD-RW.
• Uploading it to my server and redownloading it wasn’t possible as Apache will commit seppuku if you try to serve files larger then 2GB. It seems to be a 32bit limitation resulting in a 403 Forbidden error message.

Not wanting to go out of my way and install Samba or a FTP server I figured I’ll split the files into 1GB chucks and recombine them with the Windows Command Line.

$ split -b 1024m bigfile.img

I copied the pieces to Windows using one of the above methods, then started MSDOS cmd.exe and combined them.

> copy /b xaa + xab + xac + xad + xae bigfile.img

It started off with random ticking, then the head of the drive started scraping the platters which closely resembled the sound of nails scraping on a chalkboard. I rebooted and tossed in Knoppix, booting off of a LiveCD would allow me to read the hard drives in read only and hopefully avoid any further data corruption while fsck’ing it. I quickly discovered that my efforts were futile.

I’ve had several hard drives fail on me in the past, but this is the first time I’ve had one fail on me while I was using it. My 80GB Western Digital hard drive was manufactured November 2001 and luckily only contained my root partitions. Although I have a pretty decent backup strategy, it was only then that I discovered that if my second hard drive, which contained my home directory, had failed I would have lost about five days of work.

I decided that a Fantom Titanium 320GB USB/Firewire external hard drive was a cheap investment. And with 320GB I wouldn’t have to update it anytime soon. My new backup strategy quietly sits under my desk, out of site, out of mind. This way it wont get stepped on or, worst case scenario, unlikely to be discovered in the event someone tries to rob my house.

After connecting the dive, Fedora automatically mounted it as FAT32. Why it was preformatted as FAT32 and not NTFS? I’m not sure but I don’t care as I planed on wiping it out and formatting it with ext3 anyways.

# umount /dev/sda1
# dd if=/dev/zero of=/dev/sda bs=64K
4883926+0 records in
4883925+0 records out
320072933376 bytes (320 GB) copied, 11148.3 seconds, 28.7 MB/s
# cfdisk /dev/sda
# mkfs.ext3 /dev/sda1
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
# df -Th
/dev/sda1 ext3 294G 191M 279G 1% /media/disk

Running Ubuntu 6.06.1 on kernel 2.6.15-23-amd64 with 3GB of RAM, 0.5TB of hard drive space, a Nvidia GeForce 6800 GT (overclocked) GPU, and 20.1″ wide screen LCD monitor. Enjoy the eye candy, you can download the wallpaper from here.

1680×1050, ~1.3MB

My server, which I call Yuzu, is a basic LAMP install with Debian and backports enabled. It hosts the website you are viewing right now, and also runs mprime. Today I setup an IRC bot called Trix using Supybot and Pisg to generate channel stisitics. Here’s a quick rundown of what I did, but you can see the final result here.

First I had to install Supybot and Pisg, then generate Supybot’s configuration files using the supplied supybot-wizard (which is way better then EggDrop’s one billion lined config file). Once that was completed, I ran Supybot with disown so it wouldn’t quit when I closed my SSH session.

$ supybot-wizard
$ sudo apt-get install supybot pisg
$ supybot Trix.conf & disown %

I logged into the irc network and messaged the fallowing commands to my bot; Starting with identifying myself as the owner using the password I supplied in supybot-wizard:

identify Krhis [password]

Load the Services plugin so the bot will tell Nickserv its password, then tell the bot who Nickserv is, and finally the bots password that it will give to Nickserv:

load Services
config supybot.plugins.Services.NickServ NickServ
password Trix [password]

Load the ChannelLogger plugin so Pisg can generate statistics. Normally Supybot will only flush logs once an hour to reduce hard drive usage but Pisg will be running twice as often so I want it to have up-to-the-second logs it can retrieve data from. It seems that the version of Pisg that comes with apt is a little outdated and dosn’t like Supybot’s newer time stamp, I should have manually updated Pisg but because I’m lazy so I just reset the bots time stamp format:

load ChannelLogger
config supybot.plugins.ChannelLogger.flushImmediately True
config supybot.log.timestampFormat “[%d-%b-%Y %H:%M:%S]”

Load other less important plugins like ChannelStats, Dict, Seen, and Unix. I had to do a little configuring to the Unix plugin so it knew where to find fortune. Everyone enjoys a random fortune, right?

load ChannelStats
load Dict
load Seen
load Unix
config supybot.plugins.Unix.fortune.command fortune

Getting Pisg up and running was very simple after hacking away at the configuration file, then I just had to copy the images form /usr/share/pisg/gfx/ into the site directory. Pisg now updates every hour, on the hour by adding this to my servers crontab:

0 * * * * pisg –configfile=/home/chris/pisg.cfg –network=irc.irctoo.net –silent

Programming for FOSS (Free Open Source Software) is a hobby for some kernel developers, donating there time doing what they like to do best. One thing I’ve noticed in this community is that language can be very relaxed. I downloaded the latest Linux kernel (2.6.17) and egrep‘ed some swear words. It returned about 290 humorous results, you can egrep your own copy of the source code or read the entire output here. Some of my personal favorites:

Documentation/ManagementStyle:guy who lost his whole 36GB porn-collection because of your incompetence

arch/sparc/kernel/process.c: /* fuck me plenty */

arch/sparc64/kernel/binfmt_aout32.c: /* Fuck me plenty… */

drivers/net/sunhme.c: /* This card is _fucking_ hot… */

arch/mips/pci/pci-ip27.c: * IOC3 is fucked fucked beyond believe … Don’t try to access

net/ipv4/netfilter/ip_nat_snmp_basic.c: * (And this is the fucking ‘basic’ method).

drivers/parisc/superio.c: DBG_INIT(”superio_probe: WTF? Fire Extinguisher?\n”);

lib/vsprintf.c: * Wirzenius wrote this portably, Torvalds fucked it up :-)

sound/oss/maestro3.c: * port to sexy 2.4 interfaces

arch/sparc64/solaris/socksys.c: * Dave, _please_ give me specifications on this fscking mess so that I