[ILUG] md weirdness

[Niall O Broin]

I'm suffering all sorts of weirdness with md on a box running Debian 
unstable. It  had an md RAID-5 of 4x120 disks which I want to replace 
with 4x120 disks. So, I made that array on another box, and rsynced the 
300GB or so of data.

Then it was time to move disks into new box, which is where the fun 
started. I had created the fs on the new array on a box running 2.6 and 
the unstable box was running 2.4. The filesystem I'm using is jfs, and 
a 2.6 jfs can't be mounted by 2.4. The annoying thing is that I knew 
that, but it didn't click as I was doing this work.

So, I installed 2.6.8 on the unstable box - the joys of apt-get, it got 
the kernel, updated the initrd tools and whatever else and I was off 
and running - except that I can't get the array to come up during a 
reboot. I have raid5 in /etc/modules and when the box comes up I see 
this in the console:

raid5: using function: p5_mmx (2680.000 MB/sec)
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: raid5 personality registered as nr 4

yet the boot fails, because the fsck of /dev/md0 fails. I have this as 
mdadm.conf

DEVICE /dev/hdh1 /dev/hdg1 /dev/hdf1 /dev/hde1
ARRAY /dev/md0 level=raid5 num-devices=4 
UUID=e545d338:d4fd8381:cf3c520f:6109024f 
devices=/dev/hdh1,/dev/hdg1,/dev/hdf1,/dev/hde1

yet somehow, the device isn't correctly initialised. In order to get it 
up and running, I have to do

mdadm --assemble /dev/md0 /dev/hde1 /dev/hdf1 /dev/hdg1 /dev/hdh1

Before the upgrade to 2.6 and the different array, this all just worked 
nicely, and indeed I didn't even have an mdadm.conf file. My raidtab 
has

raiddev /dev/md0
    raid-level       5
    nr-raid-disks    4
    nr-spare-disks   0
    persistent-superblock 1
    parity-algorithm left-symmetric
    chunk-size       128
    device   /dev/hde1
    raid-disk 0
    device   /dev/hdf1
    raid-disk 1
    device   /dev/hdg1
    raid-disk 2
    device   /dev/hdh1
    raid-disk 3

but actually, the contents of raidtab and mdadm.conf appear to be 
academic as they don't appear to be read (looking at ls -lu) during a 
boot. So, the boot hangs, and when I tell it to continue it comes up 
without md0.  As previously mentioned, --assemble gets it up and 
running. Before doing the assemble, if I do   mdadm --monitor /dev/md0  
  I get

Oct 20 16:06:06: DeviceDisappeared on /dev/md0 unknown device

So, what do I have to do to get this to boot properly?


Niall