[linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[Peter Valdemar Mørch]

I wanted to upgrade to a 2.6 kernel, but I seem to be running into
problems getting the new kernel to recognize my LVM root '/' partition
during boot. I also think I have a mixed LVM1 and LVM2 toolset as part
of the upgrade and that this is why I'm failing. Now, I'm even more
concerned with the stability of my system than with the original 2.6
upgrade task.

Sorry for the length of the post, but I think my main problem is that I
don't know what it is that I don't know... Makes it damn hard to Google
or RTFM otherwise. I'll start with the most obvious and move into the
more obscure. Stop if you get bored or see my problem! :-D

I'm running on RH9 with what used to be a pretty standard install except
for a EXT3 LVM root partition and a ReiserFS Software Raid1 /home
partition. (OK so a  LVM '/' partition is a bad idea. I didn't know that
then. I do now!)

I can't boot with my newly compiled 2.6.5 kernel, lvmcreate_initrd
doesn't work for either 2.4 or 2.6 kernels with my current setup and
tools, and "lvm version" gives an error (see below)

In one of the many guides about 2.6 upgrading I was suggested to upgrade
to the *.i386.rpms in http://people.redhat.com/arjanv/2.6/RPMS.kernel/
(The URL seemed to indicate a reputable source for RH9 rpms)
Among others, I installed
lvm-1.0.3-17.i386.rpm
lvm2-2.00.08-2.i386.rpm
device-mapper-1.00.07-3.1.i386.rpm

Now I'm not even sure what LVM version I'm running:
[root@valde root]# lvm version
   LVM version:     2.00.08 (2003-11-14)
   Library version: 1.00.07-ioctl (2003-11-21)
   /dev/mapper/control: open failed: No such file or directory
   Is device-mapper driver missing from kernel?

[root@valde root]# lvm lvs
   /proc/lvm/VGs/Volume00 exists: Is the original LVM driver using this
volume group?
   Can't lock Volume00: skipping

[root@valde new]# lvm pvscan
   PV /dev/hda3   VG Volume00   lvm1 [56.22 GB / 0    free]
   PV /dev/hdc3   VG Volume00   lvm1 [55.83 GB / 4.00 MB free]
   Total: 2 [4.00 MB] / in use: 2 [4.00 MB] / in no VG: 0 [0   ]

`man lvm` mentions /etc/lvm/lvm.conf. I have a /etc/lvm directory but no 
/etc/lvm/lvm.conf file.

In the output, I'm asked:
"Is device-mapper driver missing from kernel?".
I don't know. Is it? I'm running a stock 2.4.20-20.9 kernel from redhat
with the non-stock bridge-nf patch. If device-mapper driver is missing
from kernel what do I do about that?

FAQ entry: "I get errors about /dev/mapper/control when I try to use the
LVM 2 tools" from http://www.tldp.org/HOWTO/LVM-HOWTO/lvm2faq.html#AEN231
says  "The primary cause of this is not having run the devmap_mknod.sh
script after rebooting into a dm capable kernel."
True. I haven't run devmap_mknod.sh. In fact I don't have it on my
system. It isn't part of any of my lvm* rpms. But am I running "a dm
capable kernel" yet with my RedHat 2.4.20-20.9 kernel?

It seems I'm now in a chicken-and-egg situation. I can't get
lvmcreate_initrd to work until I update the kernel to a dm kernel. I
can't boot into one until I have lvmcreate_initrd (or mkinitrd) working.
   Hmm...

So: Should I
*) Get, compile, install and run "a dm capable kernel." Which one?
*) Accept that installing the lvm2 rpm was a bad idea and install lvm2
from tarball to get and run devmap_mknod.sh? Anything to watch out for here?
*) Upgrade my /dev/Volume00/LogVol00 to LVM2 somehow?
*) Turn my '/' into a non-LVM volume? How?
*) Backup all my 90 GB somewhere, reinstall with a fresh 2.6 kernel
distro and get all my apps & setup working again from scratch? (YIKES!)
*) Some other option I'm missing?

***************************
    Boot disk creation?
***************************
I also don't have a good boot disk around with the LVM access. How do I
create one, now that my tools are messed up but before I loose my system
forever? Once I have it, how do I test that its working and not
accessing the lvm tools from the / partition in some sneaky fashion?
Are all the LVM access tools already in /boot/initrd-2.4.20-20.9.img (It
does contain e.g. /bin/vgchange)? Am I fine as long as I leave an entry
for this 2.4.20-20.9 kernel in grub.conf?

***************************
    lvmcreate_initrd
    seems to not work for me
***************************
OK, so then I thought I'd just put MD, RAID1, EXT3 and REISERFS into the
kernel instead of as modules and follow the
http://www.tldp.org/HOWTO/LVM-HOWTO/
on how to run "lvmcreate_initrd 2.6.5" to create the initrd-lvm file. I
have not yet succeeded in running lvmcreate_initrd successfully. I
always get approx 2000 "cpio: No space left on device" errors that
finish with
lvmcreate_initrd -- ERROR cpio to ram disk
(Because of messed up lvm tools?)

Even with a fresh 2.4.20 kernel build with the above modules directly in
the kernel (that works fine with all the above as modules using
mkinitrd) I can't get lvmcreate_initrd to work. When running
lvmcreate_initrd with a parameter indicating a 2.4 kernel (other than
the one currently running), I get the same many may "No space left on
device" errors in addition to "depmod: *** Unresolved symbols in
/lib/modules/2.4.<snip>" errors. It also fails with "ERROR cpio to ram
disk".

Going to another machine, running Fedora Core 1 and 2.4.22 with a
gazillion patches (for "myth"), I get the same "depmod: *** Unresolved
symbols in /lib/modules/2.4.<snip>" errors, but it does finish by
creating /boot/initrd-lvm-2.4.22-1.2115.nptl.gz. Are the depmod errors
normal and to be expected as part of normal behavior?

***************************
    mkinitrd woes with
    device-mapper-1.00.07-3.1.i386.rpm
***************************
Among others, I installed device-mapper-1.00.07-3.1.i386.rpm from the
above location. With it, mkinitrd (also "new" & from above) doesn't
work. Removing it and reinstating the original /lib/libdevmapper.so.1.00
makes mkinitrd work, but "lvm version" and "lvm lvs" doesn't work with
either of them. (Even after rebooting after changing it).

***************************
   Running my 2.6.5 kernel
   anyway
***************************
If I ignore all of this, and create a 2.6.5 kernel with RAID1, EXT3, LVM
and REISERFS as modules, and mkinitrd, then when I try to boot it I get:

RAMDISK: Compressed image found at block 0
RAMDISK: incomplete write (-28 != 32768) 4194304
VFS: Cannot open root device "Volume00/LogVol00" or unknown-block(0,0)
Please append a correct "root=" boot option
Kernel panic: VFS: Unable to mount root fs on unknown-block(0,0

Help! I don't know what to do now!

Peter

-- 
Peter Valdemar M�rch
http://www.morch.com

Re: [linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[AJ Lewis]

[-- Attachment #1: Type: text/plain, Size: 6908 bytes --]

On Sun, Apr 25, 2004 at 11:07:15PM +0200, Peter Valdemar M??rch wrote:
> I'm running on RH9 with what used to be a pretty standard install except
> for a EXT3 LVM root partition and a ReiserFS Software Raid1 /home
> partition. (OK so a  LVM '/' partition is a bad idea. I didn't know that
> then. I do now!)

It's not necessarily a bad idea, it just makes it a lot more tricky in cases
like this.
 
> I can't boot with my newly compiled 2.6.5 kernel, lvmcreate_initrd
> doesn't work for either 2.4 or 2.6 kernels with my current setup and
> tools, and "lvm version" gives an error (see below)

Someone made a new initrd creation script and posted it here:
http://www.poochiereds.net/svn/lvm2/
I can't guarentee it'll work, but i've heard good things about it.  Use it
against the 2.6.5 kernel you compiled.
 
> In one of the many guides about 2.6 upgrading I was suggested to upgrade
> to the *.i386.rpms in http://people.redhat.com/arjanv/2.6/RPMS.kernel/
> (The URL seemed to indicate a reputable source for RH9 rpms)
> Among others, I installed
> lvm-1.0.3-17.i386.rpm
> lvm2-2.00.08-2.i386.rpm
> device-mapper-1.00.07-3.1.i386.rpm
> 
> Now I'm not even sure what LVM version I'm running:
> [root@valde root]# lvm version
>   LVM version:     2.00.08 (2003-11-14)
>   Library version: 1.00.07-ioctl (2003-11-21)
>   /dev/mapper/control: open failed: No such file or directory
>   Is device-mapper driver missing from kernel?

You're using lvm version 2 - however, can you run 'pvscan -V' without the
'lvm' prefix?  If so, you have both tools available to you, and running
'vgchange -ay' again, without the 'lvm' prefix should work just fine for you
in your 2.4 kernel.  If you're booted into your 2.4 kernel, it sounds like
you still have the lvm1 tools and are just fine (lvm1 did *not* have the
'lvm' command)
 
> In the output, I'm asked:
> "Is device-mapper driver missing from kernel?".
> I don't know. Is it? I'm running a stock 2.4.20-20.9 kernel from redhat
> with the non-stock bridge-nf patch. If device-mapper driver is missing
> from kernel what do I do about that?

Nope, device-mapper is *not* in that kernel.  You either need to upgrade to
a 2.6 kernel, or use the patches that are found in the lvm2 tarball to patch
a 2.4 kernel to contain device-mapper.
 
> FAQ entry: "I get errors about /dev/mapper/control when I try to use the
> LVM 2 tools" from http://www.tldp.org/HOWTO/LVM-HOWTO/lvm2faq.html#AEN231
> says  "The primary cause of this is not having run the devmap_mknod.sh
> script after rebooting into a dm capable kernel."
> True. I haven't run devmap_mknod.sh. In fact I don't have it on my
> system. It isn't part of any of my lvm* rpms. But am I running "a dm
> capable kernel" yet with my RedHat 2.4.20-20.9 kernel?

If that initrd creation script works for you, you may not need to run the
devmap_mknod.sh script.  However, what the script does is looks in
/proc/misc for a '$some-number device-mapper' entry, and creates
/dev/mapper/control with 'mknod /dev/mapper/control c 10 $some-number', so
you can do that by hand if you like. 
 
> It seems I'm now in a chicken-and-egg situation. I can't get
> lvmcreate_initrd to work until I update the kernel to a dm kernel. I
> can't boot into one until I have lvmcreate_initrd (or mkinitrd) working.
>   Hmm...
> 
> So: Should I
> *) Get, compile, install and run "a dm capable kernel." Which one?

2.6.5 will be fine as long as you don't need snapshots or pvmove.  If you do
need those capabilities right now, grab the lvm2 tarball and a 2.4.26
kernel, and apply the device-mapper kernel patches from the tarball.  You
don't need to use the tarball to install the tools

> *) Accept that installing the lvm2 rpm was a bad idea and install lvm2
> from tarball to get and run devmap_mknod.sh? Anything to watch out for here?

The lvm2 rpm should be fine, you just need to make sure /dev/mapper/control
is created.

> *) Upgrade my /dev/Volume00/LogVol00 to LVM2 somehow?

Don't need to do this

> *) Turn my '/' into a non-LVM volume? How?

Scary.  It's possible, but wait until the above options don't work before
trying it.

> *) Backup all my 90 GB somewhere, reinstall with a fresh 2.6 kernel
> distro and get all my apps & setup working again from scratch? (YIKES!)

Shouldn't need to

> ***************************
>    Boot disk creation?
> ***************************
> I also don't have a good boot disk around with the LVM access. How do I
> create one, now that my tools are messed up but before I loose my system
> forever? Once I have it, how do I test that its working and not
> accessing the lvm tools from the / partition in some sneaky fashion?
> Are all the LVM access tools already in /boot/initrd-2.4.20-20.9.img (It
> does contain e.g. /bin/vgchange)? Am I fine as long as I leave an entry
> for this 2.4.20-20.9 kernel in grub.conf?

Forget this for the moment, let's get the system stabilized first.
 
> ***************************
>    lvmcreate_initrd
>    seems to not work for me
> ***************************
> OK, so then I thought I'd just put MD, RAID1, EXT3 and REISERFS into the
> kernel instead of as modules and follow the
> http://www.tldp.org/HOWTO/LVM-HOWTO/
> on how to run "lvmcreate_initrd 2.6.5" to create the initrd-lvm file. I
> have not yet succeeded in running lvmcreate_initrd successfully. I
> always get approx 2000 "cpio: No space left on device" errors that
> finish with
> lvmcreate_initrd -- ERROR cpio to ram disk
> (Because of messed up lvm tools?)

This does not work for lvm2.
 
> If I ignore all of this, and create a 2.6.5 kernel with RAID1, EXT3, LVM
> and REISERFS as modules, and mkinitrd, then when I try to boot it I get:
> 
> RAMDISK: Compressed image found at block 0
> RAMDISK: incomplete write (-28 != 32768) 4194304
> VFS: Cannot open root device "Volume00/LogVol00" or unknown-block(0,0)
> Please append a correct "root=" boot option
> Kernel panic: VFS: Unable to mount root fs on unknown-block(0,0

Yep, need that lvm2-capable initrd.
 
Well, I've heard good things about the initrd creation script i mentioned at
the beginning, so maybe you could give that a shot first with the 2.6.5
kernel?

Regards,
-- 
AJ Lewis                                   Voice:  612-638-0500
Red Hat Inc.                               E-Mail: alewis@redhat.com
720 Washington Ave. SE, Suite 200
Minneapolis, MN 55414

Current GPG fingerprint = FE77 4B43 6A9B F982 A731  02FA 2BF5 7574 294A AA5A
Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the
many keyservers out there...
-----Begin Obligatory Humorous Quote----------------------------------------
Mountain Dew... because breakfast is the most important meal of the day.
-----End Obligatory Humorous Quote------------------------------------------

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[Peter Valdemar Morch]

AJ Lewis alewis-at-redhat.com |Lists| wrote:
> Someone made a new initrd creation script and posted it here:
> http://www.poochiereds.net/svn/lvm2/
> I can't guarentee it'll work, but i've heard good things about it.  Use it
> against the 2.6.5 kernel you compiled.

OK, so now I've tried to use that. After dealing with a missing 
/lib/lvm-200/lvm (mkdir /lib/lvm-200; ln -s /sbin/lvm /lib/lvm-200/lvm) 
  and a missing /bin/busybox (cp /sbin/busybox /bin) I now have a 
/boot/initrd-lvm2-2.6.5.gz!!!! This is progress!!!!

During boot of my freshly compiled 2.6.5 kernel+initrd, I get this 
message on the console:

RAMDISK: Compressed image found at block 0
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 236k freed
Warning unable to open an initial console.
Kernel panic: Attempted to kill init!

At this point all I can do is hit the reset button and reboot in good 
ol' 2.4.20. What do I need to do now?

It prints out a lot more than this, but I don't know how to capture it. 
(do you?) Also, it prints more than one screen, but I don't know how to 
scroll up, and the previous screens scroll by very fast... (E.g. it 
detects my RAID1 volumes apparently - that fills most of the screen I 
can see)

[root@valde linux-2.6.5]# grep -i console .config
# CONFIG_NETCONSOLE is not set
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_LP_CONSOLE=y
# Console display driver support
CONFIG_VGA_CONSOLE=y
CONFIG_MDA_CONSOLE=m
CONFIG_DUMMY_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE is not set

Uhm, what is the console that it can't open, and how do I fix that? Is 
this still LVM related? Or is it more of a standard 2.6.5 thing?

***********************************************************************
grub.conf entry
***********************************************************************
lvm2create_initrd.sh suggests that images should be relative to /boot/ 
but I could only get this far with images relative to / as in:

title 2.6.5
         kernel /vmlinuz-lvm2-2.6.5 root=/dev/ram0 
lvm2root=/dev/Volume00/LogVol00
         initrd /initrd-lvm2-2.6.5.gz

Does the missing '/boot/' prefix surprise anybody?

***********************************************************************
Maybe the following is relevant (I think not), but it seems to have to 
do with consoles...
***********************************************************************

During running lvm2create_initrd.sh, it prints out this:

lvm2create_initrd.sh -- adding required /dev files
don't know how to make device "consoleonly"

Well, neither do I! A little snippet from my console:

[root@valde root]# grep BASICDEV /sbin/lvm2create_initrd.sh
BASICDEVICES="std consoleonly fd"
verbose "BASICDEVICES: `echo $BASICDEVICES`"
(cd $TMPMNT/dev; /dev/MAKEDEV $OPT_Q $BASICDEVICES $BLOCKDEVICES)

So basically /sbin/lvm2create_initrd.sh has hardcoded the fact that it 
should:
/dev/MAKEDEV std consoleonly
among other params. But *my* MAKEDEV doesn't know how to make a consoleonly:

[root@valde root]# /dev/MAKEDEV std
[root@valde root]# /dev/MAKEDEV consoleonly
don't know how to make device "consoleonly"
[root@valde root]# rpm -qf /dev/MAKEDEV
MAKEDEV-3.3.2-5

-- 
Peter Valdemar M�rch
http://www.XXXXXXXXX

Re: [linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[Peter Valdemar Mørch]

AJ Lewis alewis-at-redhat.com |Lists| wrote:
> Someone made a new initrd creation script and posted it here:
> http://www.poochiereds.net/svn/lvm2/
> I can't guarentee it'll work, but i've heard good things about it.  Use it
> against the 2.6.5 kernel you compiled.

Well, I did finally did get it to work. Quite a strenuous road, though. 
For others that may google later, here are some of the things I ran 
into, and their solutions:

1) There was no console. The script created /dev/consoleonly, but I had 
& needed /dev/console.

2) "No space left on device" The initrd ran out of inodes during 
creation. Presumably because MKDEV on my system creates very, very many 
items for /dev (2601 /dev/tty* and 1728 /dev/sd* for instance)


3) MKDEV takes a -d option on my RH9 system telling it where to put the 
files. Instead, the script cd-ed to the directory and expected them to 
be put in ./

4) fsck.ext3 failed during boot. I discovered that adding " fastboot" as 
a parameter to the kernel allowed me to get past that point and still 
end up in a 2.6.5 kernel, so I could run "lvm vgmknodes" that apparently 
created what I needed, so that I could get past the fsck hurdle, even 
without " fastboot". (Took me a while to figure *that* one out! :-D)

I've created a new version (incl. svn diff) of the script here:
http://wwww.morch.com/postings/lvm2/

Sincerely,

Peter

-- 
Peter Valdemar M�rch
http://www.morch.com

Re: [linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[AJ Lewis]

[-- Attachment #1: Type: text/plain, Size: 1711 bytes --]

On Mon, May 03, 2004 at 12:08:28AM +0200, Peter Valdemar M??rch wrote:
> AJ Lewis alewis-at-redhat.com |Lists| wrote:
> >Someone made a new initrd creation script and posted it here:
> >http://www.poochiereds.net/svn/lvm2/
> >I can't guarentee it'll work, but i've heard good things about it.  Use 
> >it
> >against the 2.6.5 kernel you compiled.
> 
> Well, I did finally did get it to work. Quite a strenuous road, though. 
> For others that may google later, here are some of the things I ran 
> into, and their solutions:

Bah - I guess I should have pointed you at the fedora mkinitrd - it might
have saved you the hassle.  :(
http://distro.ibiblio.org/pub/linux/distributions/fedora/linux/core/development/i386/Fedora/RPMS/mkinitrd-3.5.21-1.i386.rpm
is the current version AFAIK.  I think this will work better with RH9
systems than the other one (which was originally built around debian)
 
> I've created a new version (incl. svn diff) of the script here:
> http://wwww.morch.com/postings/lvm2/

K - so if the fedora mkinitrd doesn't work for people using RH9, they can
try that one. :)  Thanks!

-- 
AJ Lewis                                   Voice:  612-638-0500
Red Hat Inc.                               E-Mail: alewis@redhat.com
720 Washington Ave. SE, Suite 200
Minneapolis, MN 55414

Current GPG fingerprint = FE77 4B43 6A9B F982 A731  02FA 2BF5 7574 294A AA5A
Grab the key at: http://people.redhat.com/alewis/gpg.html or one of the
many keyservers out there...
-----Begin Obligatory Humorous Quote----------------------------------------
APATHY ERROR: Don't bother striking any key.
-----End Obligatory Humorous Quote------------------------------------------

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[Peter Valdemar Morch]

AJ Lewis alewis-at-redhat.com |Lists| wrote:
> Bah - I guess I should have pointed you at the fedora mkinitrd - it might
> have saved you the hassle.  :(

> http://distro.ibiblio.org/pub/linux/distributions/fedora/linux/core/development/i386/Fedora/RPMS/mkinitrd-3.5.21-1.i386.rpm
> is the current version AFAIK.  I think this will work better with RH9
> systems than the other one (which was originally built around debian)

Possibly. I can't get it to work with either 2.6.5 or my 2.4 image, though:

valde@peter:~> rpm -q mkinitrd
mkinitrd-3.5.21-1
valde@peter:~> sudo mkinitrd ost.img 2.6.5
No module raid1 found for kernel 2.6.5, aborting.
valde@peter:~> sudo mkinitrd ost.img 2.4.20-20.9custom
pvscan -- LVM driver/module not loaded?

(While I'm running in a 2.6 kernel. module raid1 is not not found for 
2.6.5 because I've built it into the kernel now. I can't be bothered to 
test it as a module at this point)

>>I've created a new version (incl. svn diff) of the script here:
>>http://wwww.XXXXXXXXX/postings/lvm2/

Wups, what happened with the extra 'w' (4 in all)? It does work, though...

> K - so if the fedora mkinitrd doesn't work for people using RH9, they can
> try that one. :)  Thanks!

To make as few changes as possible, RH9 users will have to edit it and 
uncomment these lines

INITRDSIZE=4096
INITRDINODES=5000

and run it with the -d option, such as in
# lvm2create_initrd.sh -d 2.6.5

Also, I forgot to mention something weird. If I attempt to boot my old 
2.4 kernel, fsck complains. Booting with kernel option " fastboot" gets 
me past the fsck point, all the way to the final login screen. However, 
in the 2.4 kernel, I no longer have keyboard or mouse support. But I can 
ssh into it from outside. Rebooting again into 2.6 also stops at fsck, 
so the " fastboot" is required *again*. When running in 2.6, I then run 
"lvm vgmknodes" and then it can boot in 2.6 fsck and all - no problemo. 
But it seems I've passed the point of no-return-to-2.4.

Later,

Peter

-- 
Peter Valdemar M�rch
http://www.XXXXXXXXX

Re: [linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[Jeffrey Layton]

On Sun, 2004-05-02 at 18:08, Peter Valdemar M�rch wrote:
> Well, I did finally did get it to work. Quite a strenuous road, though. 
> For others that may google later, here are some of the things I ran 
> into, and their solutions:
> 
> 1) There was no console. The script created /dev/consoleonly, but I had 
> & needed /dev/console.

Sorry for the late post. I've hit this too when setting up a new box.
The problem here is that the kernel or init (?) requires a /dev/console
device very early in the boot process, before udev gets mounted.

This means that even if you're using udev (and maybe devfs) then you
still need some static entries in /dev that will be overlayed by the
udev ramdisk. Obviously, this is a difficult problem to correct in the
mkinitrd script, since you'd have to unmount /dev to make sure those
entries were present.

I'll prob. add something about this to the README sometime, when I get a
chance to work on this script...

-- Jeff

[linux-lvm] 2.6 upgrade - LVM version ? & root partition access problems...

[Peter Valdemar Mørch]

I wanted to upgrade to a 2.6 kernel, but I seem to be running into
problems getting the new kernel to recognize my LVM root '/' partition
during boot. I also think I have a mixed LVM1 and LVM2 toolset as part
of the upgrade and that this is why I'm failing. Now, I'm even more
concerned with the stability of my system than with the original 2.6
upgrade task.

Sorry for the length of the post, but I think my main problem is that I
don't know what it is that I don't know... Makes it damn hard to Google
or RTFM otherwise. I'll start with the most obvious and move into the
more obscure. Stop if you get bored or see my problem! :-D

I'm running on RH9 with what used to be a pretty standard install except
for a EXT3 LVM root partition and a ReiserFS Software Raid1 /home
partition. (OK so a  LVM '/' partition is a bad idea. I didn't know that
then. I do now!)

I can't boot with my newly compiled 2.6.5 kernel, lvmcreate_initrd
doesn't work for either 2.4 or 2.6 kernels with my current setup and
tools, and "lvm version" gives an error (see below)

In one of the many guides about 2.6 upgrading I was suggested to upgrade
to the *.i386.rpms in http://people.redhat.com/arjanv/2.6/RPMS.kernel/
(The URL seemed to indicate a reputable source for RH9 rpms)
Among others, I installed
lvm-1.0.3-17.i386.rpm
lvm2-2.00.08-2.i386.rpm
device-mapper-1.00.07-3.1.i386.rpm

Now I'm not even sure what LVM version I'm running:
[root@valde root]# lvm version
   LVM version:     2.00.08 (2003-11-14)
   Library version: 1.00.07-ioctl (2003-11-21)
   /dev/mapper/control: open failed: No such file or directory
   Is device-mapper driver missing from kernel?

[root@valde root]# lvm lvs
   /proc/lvm/VGs/Volume00 exists: Is the original LVM driver using this
volume group?
   Can't lock Volume00: skipping

[root@valde new]# lvm pvscan
   PV /dev/hda3   VG Volume00   lvm1 [56.22 GB / 0    free]
   PV /dev/hdc3   VG Volume00   lvm1 [55.83 GB / 4.00 MB free]
   Total: 2 [4.00 MB] / in use: 2 [4.00 MB] / in no VG: 0 [0   ]

`man lvm` mentions /etc/lvm/lvm.conf. I have a /etc/lvm directory but no
/etc/lvm/lvm.conf file.

In the output, I'm asked:
"Is device-mapper driver missing from kernel?".
I don't know. Is it? I'm running a stock 2.4.20-20.9 kernel from redhat
with the non-stock bridge-nf patch. If device-mapper driver is missing
from kernel what do I do about that?

FAQ entry: "I get errors about /dev/mapper/control when I try to use the
LVM 2 tools" from http://www.tldp.org/HOWTO/LVM-HOWTO/lvm2faq.html#AEN231
says  "The primary cause of this is not having run the devmap_mknod.sh
script after rebooting into a dm capable kernel."
True. I haven't run devmap_mknod.sh. In fact I don't have it on my
system. It isn't part of any of my lvm* rpms. But am I running "a dm
capable kernel" yet with my RedHat 2.4.20-20.9 kernel?

It seems I'm now in a chicken-and-egg situation. I can't get
lvmcreate_initrd to work until I update the kernel to a dm kernel. I
can't boot into one until I have lvmcreate_initrd (or mkinitrd) working.
   Hmm...

So: Should I
*) Get, compile, install and run "a dm capable kernel." Which one?
*) Accept that installing the lvm2 rpm was a bad idea and install lvm2
from tarball to get and run devmap_mknod.sh? Anything to watch out for here?
*) Upgrade my /dev/Volume00/LogVol00 to LVM2 somehow?
*) Turn my '/' into a non-LVM volume? How?
*) Backup all my 90 GB somewhere, reinstall with a fresh 2.6 kernel
distro and get all my apps & setup working again from scratch? (YIKES!)
*) Some other option I'm missing?

***************************
    Boot disk creation?
***************************
I also don't have a good boot disk around with the LVM access. How do I
create one, now that my tools are messed up but before I loose my system
forever? Once I have it, how do I test that its working and not
accessing the lvm tools from the / partition in some sneaky fashion?
Are all the LVM access tools already in /boot/initrd-2.4.20-20.9.img (It
does contain e.g. /bin/vgchange)? Am I fine as long as I leave an entry
for this 2.4.20-20.9 kernel in grub.conf?

***************************
    lvmcreate_initrd
    seems to not work for me
***************************
OK, so then I thought I'd just put MD, RAID1, EXT3 and REISERFS into the
kernel instead of as modules and follow the
http://www.tldp.org/HOWTO/LVM-HOWTO/
on how to run "lvmcreate_initrd 2.6.5" to create the initrd-lvm file. I
have not yet succeeded in running lvmcreate_initrd successfully. I
always get approx 2000 "cpio: No space left on device" errors that
finish with
lvmcreate_initrd -- ERROR cpio to ram disk
(Because of messed up lvm tools?)

Even with a fresh 2.4.20 kernel build with the above modules directly in
the kernel (that works fine with all the above as modules using
mkinitrd) I can't get lvmcreate_initrd to work. When running
lvmcreate_initrd with a parameter indicating a 2.4 kernel (other than
the one currently running), I get the same many may "No space left on
device" errors in addition to "depmod: *** Unresolved symbols in
/lib/modules/2.4.<snip>" errors. It also fails with "ERROR cpio to ram
disk".

Going to another machine, running Fedora Core 1 and 2.4.22 with a
gazillion patches (for "myth"), I get the same "depmod: *** Unresolved
symbols in /lib/modules/2.4.<snip>" errors, but it does finish by
creating /boot/initrd-lvm-2.4.22-1.2115.nptl.gz. Are the depmod errors
normal and to be expected as part of normal behavior?

***************************
    mkinitrd woes with
    device-mapper-1.00.07-3.1.i386.rpm
***************************
Among others, I installed device-mapper-1.00.07-3.1.i386.rpm from the
above location. With it, mkinitrd (also "new" & from above) doesn't
work. Removing it and reinstating the original /lib/libdevmapper.so.1.00
makes mkinitrd work, but "lvm version" and "lvm lvs" doesn't work with
either of them. (Even after rebooting after changing it).

***************************
   Running my 2.6.5 kernel
   anyway
***************************
If I ignore all of this, and create a 2.6.5 kernel with RAID1, EXT3, LVM
and REISERFS as modules, and mkinitrd, then when I try to boot it I get:

RAMDISK: Compressed image found at block 0
RAMDISK: incomplete write (-28 != 32768) 4194304
VFS: Cannot open root device "Volume00/LogVol00" or unknown-block(0,0)
Please append a correct "root=" boot option
Kernel panic: VFS: Unable to mount root fs on unknown-block(0,0

Help! I don't know what to do now!

Peter

-- 
Peter Valdemar M�rch
http://www.morch.com