I fixed the problem by following Eric Hammond’s article on how to Move a Running EBS Boot Instance to New Hardware on Amazon EC2. His original article was for scenarios where the instances become inaccessible due to power failures, but the methodology worked out in my case just fine.

Anyway, the online upgrade wasn’t as slow as I thought – took about 3 hours to download about 1GB data and set up everything. Did run into a couple of minor surprises:

The computer booted up fine… without any functioning dns! NetworkManager apparently decided that it was OK to silently overwrite all the changes I made in /etc/resolv.conf. I have had some history with the Network Manager before, and so was not hesitant to disable it with sudo update-rc.d -f NetworkManager remove.

The only real annoyance was that kde4 stopped honoring ~/.kde4/Autostart, and when restoring a previously saved session wouldn’t restore anything on the system tray either. So none of the utility programs I’m used to – klipper, kbiff, etc. – were automatically started any more. It was annoying because I had to restart X over and over to troubleshoot. Finally I spotted the Autostart item in System Settings, which seems to be using ~/.config/autostart (I guess that’s part of the freedesktop.org standards?), and works fine.

1. The sound system in kde 4 appeared to have stopped working. It simply refuses to recognize the onboard sound card. Sound works fine in kde 3, or in kde 4 but through other packages – for instance, amarok and the flash video player can play sound without any problems.
2. The Logitech Marble Mouse would arbitrarily stop responding out of blue, and won’t come back without rebooting. Good thing I have two mouses plugged in (the other Logitech with higher precision for games .
3. Firefox 3 Beta 5 crashed the X server once. I’ll have to wait and see if this is recurring. So far Firefox has been quite stable otherwise.

Let me list the hardware upgrade first so you know what the “players” had to deal with:

1. Replaced P4 3.0 GHz with Core 2 Quad Q6600,
2. Bumped RAM to 4G,
3. Replaced an Asus P4 motherboard with Abit IP35Pro.
4. Replaced nvidia 6800GT with 8600GTS.
5. Added a new SATA hard drive. Kept the IDE hard drive around.

I actually expected to spend a lot of time on installing and configuring Gutsy, after all, it’s still linux, right? Well, not really. The only hiccup I had was when trying to boot up from the live dvd for the first time, the kernel (2.6.22) got confused by my new SATA drive, and would take very long to boot. It took 1 minute to find the answer by google: add ‘irqpoll’ to the boot options.

And it just cruised from there:

1. Boot into the live dvd desktop, run installer, copy files, reboot into the new installation.
2. The “restricted driver manager” automatically prompts if I would like to enable the proprietary nvidia driver. Yes, I would. Reboot.
3. Run nvidia-settings to set up the dual monitors. Copy the config fragments for the logitech marble mouse and natural keyboard into /etc/X11/xorg.conf. Restart X server.

And that’s it. I was able to pick up whatever it was that I was working on before the upgrade as soon as I had the home dir copied over from the backup machine.

It wasn’t like that on the WinXP side. Not at all.

I booted into the XP installation CD. The installer apparently got confused by the combination of the IDE and SATA drives, but it didn’t say anything, instead went straight to the screen where it showed a list of two partitions, each from… wait… the same drive number with exactly the same id and bus numbers? And none of the actual, existing partitions on either drives were shown?

I then pressed Enter on one of the drives and got to the next screen where it warned that it was about to format the partition and “all data will be lost”. So naturally I decided to abort right then and there, and boot back into linux to make sure I’ve got everything backed up. Well, what do you know, all data were lost already by then, because, apparently, the XP installer had already wiped out the partition table.

In hindsight, and to be fair, I probably did something real stupid there by pressing that Enter key, after having seen the confused partition/drive list. But, hey, you are supposed to tell people before you wipe out their partition table, right?

So I had to go through the whole exercise again, except this time I installed XP first, and then Kubuntu. Even it worked this time, I still had to spent way more time in the XP part than I did in Kubuntu. To begin with, it took 40 minutes or so to see the initial XP desktop, and then another 6-7 reboots to install all the service packs, security patches, etc.

Now, before somebody screams “fanboy!” I’ll admit that it’s not exactly fair to directly compare Kubuntu 7.10 with the original Windows XP. After all, the latter was first released in 2001. But then, I will also argue that, as a paying customer of a retail Windows XP license, I have never received so much as an offer from Microsoft to replace the original installation CD with one that can handle my rather not so very unique SATA+IDE configuration.

So I fired up gparted and broke a fat32 partition into a few ext3 ones. Reboot… enter single user mode… move /opt, /tmp, and /var into their respective parition… edit /etc/fstab… reboot. Voila! df confirms exactly what I had planned for, except…

Except the networking is completely down – “completely” as in “even the loopback 127.0.0.1 interface isn’t up”.

Rebooting with “nosplash” showed some peculiar messages:

Mount point “/var/run” does not exist. Skip mounting.
Mount point “/var/lock” does not exist. Skip mounting.
(and later)
/var/run/network does not exist…

After a bit googling, I ran into Chris Siebenmann’s wiki page on how Ubuntu requires /var/run to stay on the root file system even though /var is relocated to somewhere else. Among his comments, I found this particular sentence amusingly depressing (well at the moment it was just “depressing”):

If you started out without a separate /var filesystem and now want to move to one, apparently your life just sucks.

Eventually I fixed the problem by entering the single user mode, umounting /var, and manually creating /var/run and /var/lock in the root file system.

My network card has its own certain eccentricity that requires a little special setting before running. Somehow that doesn’t get along with the Network Manager, which insists on starting off the whole networking business by turning off the card first. In the beginning I thought this was because of the beta, and I just had to wait until the release and there’d be some sort of “patch”. Alas, three days past the golden date, I suddenly realized there weren’t any cavalries coming over the top of the hill . In the end, I had to completely uninstall the Network Manager, and go back to the good old ifup/ifdown rc scripts.

I wouldn’t have written this to whine about the NM if that were the only incident. As I’m getting myself quite comfortable with Ubuntu, I decided to completely switch another older computer from XP to Ubuntu. I tried Debian Etch first, but the repositories proved to be too far behind – even the testing. So I went ahead and installed Feisty, only to find that the Network Manager seemed to have determined to play some game with me around every corner.

The problem with this second computer is that it’s got a dlink wireless card that’s no longer used (ever since I moved it to close to the router enough for a wired connection). The NM insists on hooking up both wired and wireless networks. Even after I completely commented out the wireless interface in /etc/network/interfaces, it still somehow managed to figure out there is a wireless interface to work with. Not that I didn’t appreciate it trying to do its job, but it really got to the point of being over-zealous, if you ask me. So again I had to uninstall the NM, and stick to the good old networking scripts.

For the APR and APR-UTIL packages required by subversion, there are libapr0, libapr1.0, and libapr1 and their corresponding libaprutil* in Synaptic. Despite that the libapr0 description says that it is "currently used by Apache2, Subversion...", or that 1.4.2 was happily built with libapr1.0, you want to get libapr1/libapr1-dev/libaprutil1/libaprutil1-dev, and use this configure line: (I don't care much about accessing through webdav)

Offlineimap works surprisingly fast. Now I can use mutt to view all of my IMAP accounts without having to switch back and forth.

This here talks about how to set up postfix to relay to my ISP's SMTP server.

One thing bothered me for a couple of days was all the emails sent through mutt, replayed by postfix to my ISP, had 'jingxue@my-internal-host-name' in the Return-Path header, which makes no sense on the Internet. I finally figured out setting 'use_envelope_from' would fix it.

So I backed up everything on drive E:, the big NTFS partition, fired up the disk manager in Windows XP (if I were recording a chess game, this would be where I'd put "?? very bad move"), deleted the logical drive and the extended partition holding it, created a new extended partition holding a FAT32 drive of 22GB, which I intended to be used for sharing files between the OS's. Everything looked fine, the new FAT32 drive automatically got assigned the same E: drive letter, so everything should keep working in Windows once I restored all the files. All cool. Rebooting into Ubuntu to create the ext3 partition for the new /home...

Grub stage 1.5
Grub
Error 17

OK... how did I manage to break Grub after having merely deleted a data partition in Windows? Reboot from the Ubuntu Live CD... start gparted... Well, apparently since I used the Windows disk manager to manipulate the partitions, the device nodes for the Linux partitions had not been preserved. The / used to be on /dev/hdb4, and the swap on /dev/hdb5. Now since /dev/hdb2 and /dev/hdb3 were deleted in Windows and the new partition and logic drive were named /dev/hdb2 and (surprise) /dev/hdb5, the new / and swap had become /dev/hdb3 and /dev/hdb4, respectively.

So, looked like grub time. First mount /dev/hdb3 onto the in-memory file system (remember I booted from Live CD), open up menu.lst and adjusted what used to be (hd0,3) to (hd0.2), and /dev/hdb4 to /devhdb3. Then enter grub command line:

Reboot. Yep, back in the game. This time redid the partitioning with gparted, and then moved /home over successfully.

In Windows, I had to go to the driver configuration, disable the "auto sensing", and explicitly set the NIC to 100baseT. In Ubuntu, this is done by adding the following script to /etc/network/if-pre-up.d:

Everytime when the network interfaces are initialized, this script (along with everything else in that if-pre-up.d directory) is executed before dhcpclient is involved.

Update:
Use sudo ifdown -a to bring down all interfaces, and sudo ifup -a to bring them back up.

The highlight of this would-have-been-mundane process was Web Start, according to the JDK installation notes, it's supposed to get set up automatically by the installer, which is kind of true, as I could see all the relevant entries being added properly into /etc/mailcap and /etc/mime.types. Yet, when I tried to Web Start an application, Firefox always downloads it and offers to handle it by downloading it again and offering to handle it by downloading it again and offering to handle it... And the Preferences->Content->File Types->Manage thingy drove me crazy because there didn't seem to be any way to add a new type.

The trick, I finally figured out, is to open up a File Browser, right click on a JNLP file, choose "Properties", then the "Open With" tab, and add the path to javaws in there, and make it the default open application.

Update:
To make the Sun JDK an alternative in the Debian alternatives system, simply do (assuming JDK 1.6 installation path /opt/jdk1.6/bin/java):

This makes /usr/bin/java link to the JDK 1.6 version. This works system-wide, compared to simply adding /opt/jdk1.6/bin to $PATH. Also you can have other versions JDK 1.5 added, and then use update-alternatives to easily switch back and forth between the two.

Installing Ubuntu 6.1 ("Edgy Eft") was smooth. I have 3 NTFS partitions on the harddrive. One of them was deleted to make room for Ubuntu. It's about 24GB. Made 2 partitions - 2GB for swap, and the rest for /. I know it'd be better to have things like /var and /home in separate partitions, but there can only be up to 4 patitions on one harddrive, so that's that.

As for the other two NTFS partitions, the installer discovered them and offered to mount them to the linux file system. The default root mount point for them was a little strange to be called "/media". So I changed them to, well, /win/c, and /win/e. Later on I'm planning on converting E: to FAT32 or FAT32+ext3, so the duo can safely share files. For now, though, the first thing after booting into my new Ubuntu installation was changing /etc/fstab, and replacing the "defaults" in options for those two mount points to "ro". That makes them read-only.

Well, about a couple months ago, the "Genuine Windows" update notification popup finally got on my nerve, to the point of "ok, it's that time again." After playing around a Ubuntu installed in VMWare, I got confident enough to take it one step further into the typical dual-boot setup. A couple of nights later, I am happy to report that I'm working in an environment capable of supporting about 90% of my daily tasks. And I'm pretty confident that, with more googling and tweaking, and the great wine as the last resort, I'll be pushing that number to very close to 100%.

That's not to say, everything went as smooth as I would have liked it. I'm starting this series to keep some notes on the pitfalls and (usually unpleasant) surprises.

Next: Installation