From: Wojciech Pyczak <nissarin@gmail.com>
To: The development of GRUB 2 <grub-devel@gnu.org>
Subject: Re: LVM & RAID10
Date: Fri, 05 Dec 2008 22:11:50 +0100 [thread overview]
Message-ID: <49399916.5020307@gmail.com> (raw)
In-Reply-To: <ca0f59980812032038i2ed78cferfea6829ca8278094@mail.gmail.com>
>
> Hi,
>
> Perhaps you can describe the steps used to create the LVM volume,
> along with version number of tools. Let me see if the error can be
> recreated in qemu.
>
OK. I've made some less chaotic tests:
Hardware (probably insignificant):
- AMD Phenom X3
- Gigabyte mobo (780G/SB700 AMD chipset)
- 2x WD SE-16 6400AAKS (before I started testing I zeroed with dd a
few GB of each drive - just in case..)
Software:
- System RescueCD (used to manage partitions, RAID/LVM volumes)
- Linux sysresccd 2.6.26.07-std111-amd64 #1 SMP Sun Oct 26 11:28:49
UTC 2008 x86_64 AMD Phenom(tm) 8450 Triple-Core Processor AuthenticAMD
GNU/Linux
- LVM version: 2.02.39 (2008-06-27), Library version: 1.02.27
(2008-06-25), Driver version: 4.13.0
- mdadm - v2.6.4 - 19th October 2007
- Gentoo custom ("stable amd64") stage3 -
http://www.funtoo.org/linux/amd64/funtoo-amd64-2008.12.05/ (used for
chrooting)
- Grub2 rev 1933 (latest), build on my current PC:
- ./configure --prefix=/usr --enable-grub-emu --enable-grub-fstest
LDFLAGS=-static
- gcc version 4.1.2 (Gentoo 4.1.2 p1.1)
- GNU Make 3.81
- bison (GNU Bison) 2.3
- sys-devel/binutils-2.18-r3
* IMO also insignificant, since previously I build grub on my new
system which used many "experimental" packages, such as GCC 4.3.2, etc. *
* Small note - I renamed grub binaries (in bin/sbin) to *.org and
made a wrapper script, so I would know arguments used by
grub-install/setup: *
#!/bin/sh
echo \#${0} ${@} 1>&2
exec ${0}.org ${@}
I used cfdisk to partition disks. Commands used to create RAID arrays
and LVM volumes:
mdadm --create /dev/md0 (md1) --raid-devices=3 --level=10
--layout=f2 --chunk=256 /dev/sd{a,b}1 (sd{a,b}2) missing
pvcreate /dev/md0
vgcreate test /dev/md0
lvcreate test -ntest -l1788
mdadm --assemble /dev/md0 (md1) /dev/sd{a,b}1 (sd{a,b}2)
pvscan
vgchange -ay
Nothing fancy...
Test procedure looked something like that:
- booting system with RescueCD
- partitioning, creating RAID/LVM, formatting, reboot
- after reboot:
- assembling RAID/LVM volumes
- mounting corresponding device
- unpacking stage3
- unpacking package containing grub
- creating missing directories (/boot/grub) and files (device.map,
simple grub.cfg)
- mouting proc, sys, dev
- grub-install/setup, whatever
===== Single partition, RAID10, reiserfs
Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sda1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sda2 0 - 0 0 0 Empty
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
Disk /dev/sdb: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sdb1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sdb2 0 - 0 0 0 Empty
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty
--------------------------------------------------------------------------------------------------------------------------
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid10 sda1[0] sdb1[1]
7325184 blocks 256K chunks 2 far-copies [3/2] [UU_]
unused devices: <none>
--------------------------------------------------------------------------------------------------------------------------
sysresccd grub # grub-install "(md0)"
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/md0
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map
--target=abstraction --device /dev/md0
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive
--device /dev/md0
#/usr/bin/grub-mkimage --output=/boot/grub/core.img
--prefix=(md0)/boot/grub biosdisk reiserfs pc pc raid mdraid
#/usr/sbin/grub-setup --directory=/boot/grub
--device-map=/boot/grub/device.map
(md0)
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.
(hd0) /dev/sda
(hd1) /dev/sdb
(hd2) /dev/sdc <= USB stick
--------------------------------------------------------------------------------------------------------------------------
Everything works (I copied memtest86 from RescueCD to /boot and managed
to boot it).
===== Single partion, LVM @ RAID10, reiserfs
Partitions/RAID like above
--------------------------------------------------------------------------------------------------------------------------
root@sysresccd /root % pvdisplay
--- Physical volume ---
PV Name /dev/md0
VG Name test
PV Size 6.99 GB / not usable 1.50 MB
Allocatable yes (but full)
PE Size (KByte) 4096
Total PE 1788
Free PE 0
Allocated PE 1788
PV UUID 6i6MZH-7Lv7-ZcXb-yr2W-ShFz-2c13-srXd07
root@sysresccd /root % vgdisplay
--- Volume group ---
VG Name test
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 6.98 GB
PE Size 4.00 MB
Total PE 1788
Alloc PE / Size 1788 / 6.98 GB
Free PE / Size 0 / 0
VG UUID aCTAWb-xbpL-3oSp-1V2X-Oy3T-E2D7-gJoCZG
root@sysresccd /root % lvdisplay
--- Logical volume ---
LV Name /dev/test/test
VG Name test
LV UUID IzYAb2-9y4u-HqB2-27bD-ihvH-uCmn-2a1xlH
LV Write Access read/write
LV Status available
# open 2
LV Size 6.98 GB
Current LE 1788
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install '(md0)'
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: We don't support multiple metadata areas
grub-probe: error: no mapping exists for `test-test'
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install.org '(md0)' --modules='biosdisk pc raid
mdraid lvm reiserfs'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: We don't support multiple metadata areas
grub-probe: error: no mapping exists for `test-test'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map
--target=abstraction --device /dev/mapper/test-test
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive
--device /dev/mapper/test-test
error: We don't support multiple metadata areas
#/usr/bin/grub-mkimage --output=/boot/grub/core.img
--prefix=(test-test)/boot/grub biosdisk pc raid mdraid lvm reiserfs
biosdisk lvm
#/usr/sbin/grub-setup --directory=/boot/grub
--device-map=/boot/grub/device.map
(md0)
error: We don't support multiple metadata areas
grub-setup: error: no mapping exists for `test-test'
--------------------------------------------------------------------------------------------------------------------------
Grub works (after reboot) but I can't access filesystem, LVM module is
not present
====== Added second partition
Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sda1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sda2 608 1215 608 4883760 da Non-FS data
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
Disk /dev/sdb: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sdb1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sdb2 608 1215 608 4883760 da Non-FS data
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty
--------------------------------------------------------------------------------------------------------------------------
RAID/LVM like above
--------------------------------------------------------------------------------------------------------------------------
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: no such partition
grub-probe: error: no mapping exists for `test-test'
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install.org '(md0)' --modules="biosdisk pc raid
mdraid lvm reiserfs"
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: no such partition
grub-probe: error: no mapping exists for `test-test'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map
--target=abstraction --device /dev/mapper/test-test
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive
--device /dev/mapper/test-test
error: no such partition
#/usr/bin/grub-mkimage --output=/boot/grub/core.img
--prefix=(test-test)/boot/grub biosdisk pc raid mdraid lvm reiserfs
biosdisk lvm
#/usr/sbin/grub-setup --directory=/boot/grub
--device-map=/boot/grub/device.map
(md0)
grub-setup: error: no mapping exists for `test-test'
--------------------------------------------------------------------------------------------------------------------------
Second partition is unsed, however grub stoped complaining about
"multiple metadata". Desipte that I still can't boot system.
===== Added RAID10 using second partition pair, I'll try to boot system
from it
sysresccd / # grub-mkimage -o /boot/grub/core.img biosdisk pc raid
mdraid lvm reiserfs
#/usr/bin/grub-mkimage -o /boot/grub/core.img biosdisk pc raid mdraid
lvm reiserfs
sysresccd / # grub-setup '(md1)'
#/usr/sbin/grub-setup (md1)
grub-setup: error: Core image is too big for embedding, but this is
required when
the root device is on a RAID array or LVM volume.
--------------------------------------------------------------------------------------------------------------------------
After seeing that error message I created core.img without lvm module
and rebooted computer (I was able to boot system),
then I loaded lvm module but I couldn't see any lvm volumes (despite
fact that I saw md0 containing it). I tried to rmmod/inmod
again but it didn't help.
OK so here are my thoughts - most ppl use ext2 for /boot, I used
reiserfs which has 4k larger module so LVM just don't fit info core.img.
Using grub-install/setup on LVM@RAID always ends up with some errors I
just couldn't see it before (those errors were discarted
somewhere along the way ?) but it doesn't explain why I couldn't see lvm
volumes desipe the fact that loaded lvm module from disk.
Perhaps I should try to create LVM on md1 and boot from md0 to see what
will happen ?
Anyway there is something wrong with LVM+RAID on single partition ;)
>
> It's caused by size limitation of mbr, which is normally 63 sectors.
> When you use the following modules:
>
> biosdisk pc reiserfs lvm mdraid raid6 raid6rec
>
> The size of core.img would exceed 63 sectors, so that it can't be
> installed to mbr.
>
>
Is it possible to use more sectors by leaving some unused space before
fisrt partion ?
next prev parent reply other threads:[~2008-12-05 21:12 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-11-25 1:08 LVM & RAID10 Wojciech Pyczak
2008-11-25 2:52 ` Bean
2008-12-03 22:32 ` Wojciech Pyczak
2008-12-04 4:38 ` Bean
2008-12-05 21:11 ` Wojciech Pyczak [this message]
2009-05-01 18:29 ` Wojciech Pyczak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=49399916.5020307@gmail.com \
--to=nissarin@gmail.com \
--cc=grub-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.