From: Wojciech Pyczak <nissarin@gmail.com>
To: The development of GRUB 2 <grub-devel@gnu.org>
Subject: Re: LVM & RAID10
Date: Fri, 05 Dec 2008 22:11:50 +0100 [thread overview]
Message-ID: <49399916.5020307@gmail.com> (raw)
In-Reply-To: <ca0f59980812032038i2ed78cferfea6829ca8278094@mail.gmail.com>
>
> Hi,
>
> Perhaps you can describe the steps used to create the LVM volume,
> along with version number of tools. Let me see if the error can be
> recreated in qemu.
>
OK. I've made some less chaotic tests:
Hardware (probably insignificant):
- AMD Phenom X3
- Gigabyte mobo (780G/SB700 AMD chipset)
- 2x WD SE-16 6400AAKS (before I started testing I zeroed with dd a
few GB of each drive - just in case..)
Software:
- System RescueCD (used to manage partitions, RAID/LVM volumes)
- Linux sysresccd 2.6.26.07-std111-amd64 #1 SMP Sun Oct 26 11:28:49
UTC 2008 x86_64 AMD Phenom(tm) 8450 Triple-Core Processor AuthenticAMD
GNU/Linux
- LVM version: 2.02.39 (2008-06-27), Library version: 1.02.27
(2008-06-25), Driver version: 4.13.0
- mdadm - v2.6.4 - 19th October 2007
- Gentoo custom ("stable amd64") stage3 -
http://www.funtoo.org/linux/amd64/funtoo-amd64-2008.12.05/ (used for
chrooting)
- Grub2 rev 1933 (latest), build on my current PC:
- ./configure --prefix=/usr --enable-grub-emu --enable-grub-fstest
LDFLAGS=-static
- gcc version 4.1.2 (Gentoo 4.1.2 p1.1)
- GNU Make 3.81
- bison (GNU Bison) 2.3
- sys-devel/binutils-2.18-r3
* IMO also insignificant, since previously I build grub on my new
system which used many "experimental" packages, such as GCC 4.3.2, etc. *
* Small note - I renamed grub binaries (in bin/sbin) to *.org and
made a wrapper script, so I would know arguments used by
grub-install/setup: *
#!/bin/sh
echo \#${0} ${@} 1>&2
exec ${0}.org ${@}
I used cfdisk to partition disks. Commands used to create RAID arrays
and LVM volumes:
mdadm --create /dev/md0 (md1) --raid-devices=3 --level=10
--layout=f2 --chunk=256 /dev/sd{a,b}1 (sd{a,b}2) missing
pvcreate /dev/md0
vgcreate test /dev/md0
lvcreate test -ntest -l1788
mdadm --assemble /dev/md0 (md1) /dev/sd{a,b}1 (sd{a,b}2)
pvscan
vgchange -ay
Nothing fancy...
Test procedure looked something like that:
- booting system with RescueCD
- partitioning, creating RAID/LVM, formatting, reboot
- after reboot:
- assembling RAID/LVM volumes
- mounting corresponding device
- unpacking stage3
- unpacking package containing grub
- creating missing directories (/boot/grub) and files (device.map,
simple grub.cfg)
- mouting proc, sys, dev
- grub-install/setup, whatever
===== Single partition, RAID10, reiserfs
Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sda1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sda2 0 - 0 0 0 Empty
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
Disk /dev/sdb: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sdb1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sdb2 0 - 0 0 0 Empty
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty
--------------------------------------------------------------------------------------------------------------------------
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md0 : active raid10 sda1[0] sdb1[1]
7325184 blocks 256K chunks 2 far-copies [3/2] [UU_]
unused devices: <none>
--------------------------------------------------------------------------------------------------------------------------
sysresccd grub # grub-install "(md0)"
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/md0
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map
--target=abstraction --device /dev/md0
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive
--device /dev/md0
#/usr/bin/grub-mkimage --output=/boot/grub/core.img
--prefix=(md0)/boot/grub biosdisk reiserfs pc pc raid mdraid
#/usr/sbin/grub-setup --directory=/boot/grub
--device-map=/boot/grub/device.map
(md0)
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.
(hd0) /dev/sda
(hd1) /dev/sdb
(hd2) /dev/sdc <= USB stick
--------------------------------------------------------------------------------------------------------------------------
Everything works (I copied memtest86 from RescueCD to /boot and managed
to boot it).
===== Single partion, LVM @ RAID10, reiserfs
Partitions/RAID like above
--------------------------------------------------------------------------------------------------------------------------
root@sysresccd /root % pvdisplay
--- Physical volume ---
PV Name /dev/md0
VG Name test
PV Size 6.99 GB / not usable 1.50 MB
Allocatable yes (but full)
PE Size (KByte) 4096
Total PE 1788
Free PE 0
Allocated PE 1788
PV UUID 6i6MZH-7Lv7-ZcXb-yr2W-ShFz-2c13-srXd07
root@sysresccd /root % vgdisplay
--- Volume group ---
VG Name test
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 2
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 1
Act PV 1
VG Size 6.98 GB
PE Size 4.00 MB
Total PE 1788
Alloc PE / Size 1788 / 6.98 GB
Free PE / Size 0 / 0
VG UUID aCTAWb-xbpL-3oSp-1V2X-Oy3T-E2D7-gJoCZG
root@sysresccd /root % lvdisplay
--- Logical volume ---
LV Name /dev/test/test
VG Name test
LV UUID IzYAb2-9y4u-HqB2-27bD-ihvH-uCmn-2a1xlH
LV Write Access read/write
LV Status available
# open 2
LV Size 6.98 GB
Current LE 1788
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install '(md0)'
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: We don't support multiple metadata areas
grub-probe: error: no mapping exists for `test-test'
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install.org '(md0)' --modules='biosdisk pc raid
mdraid lvm reiserfs'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: We don't support multiple metadata areas
grub-probe: error: no mapping exists for `test-test'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map
--target=abstraction --device /dev/mapper/test-test
error: We don't support multiple metadata areas
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive
--device /dev/mapper/test-test
error: We don't support multiple metadata areas
#/usr/bin/grub-mkimage --output=/boot/grub/core.img
--prefix=(test-test)/boot/grub biosdisk pc raid mdraid lvm reiserfs
biosdisk lvm
#/usr/sbin/grub-setup --directory=/boot/grub
--device-map=/boot/grub/device.map
(md0)
error: We don't support multiple metadata areas
grub-setup: error: no mapping exists for `test-test'
--------------------------------------------------------------------------------------------------------------------------
Grub works (after reboot) but I can't access filesystem, LVM module is
not present
====== Added second partition
Disk /dev/sda: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sda1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sda2 608 1215 608 4883760 da Non-FS data
/dev/sda3 0 - 0 0 0 Empty
/dev/sda4 0 - 0 0 0 Empty
Disk /dev/sdb: 77825 cylinders, 255 heads, 63 sectors/track
Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
Device Boot Start End #cyls #blocks Id System
/dev/sdb1 * 0+ 607 608- 4883728+ da Non-FS data
/dev/sdb2 608 1215 608 4883760 da Non-FS data
/dev/sdb3 0 - 0 0 0 Empty
/dev/sdb4 0 - 0 0 0 Empty
--------------------------------------------------------------------------------------------------------------------------
RAID/LVM like above
--------------------------------------------------------------------------------------------------------------------------
#/usr/sbin/grub-install (md0)
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: no such partition
grub-probe: error: no mapping exists for `test-test'
Auto-detection of a filesystem module failed.
Please specify the module with the option `--modules' explicitly.
--------------------------------------------------------------------------------------------------------------------------
sysresccd / # grub-install.org '(md0)' --modules="biosdisk pc raid
mdraid lvm reiserfs"
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=device
/boot/grub
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=fs
--device /dev/mapper/test-test
error: no such partition
grub-probe: error: no mapping exists for `test-test'
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map
--target=abstraction --device /dev/mapper/test-test
error: no such partition
#/usr/sbin/grub-probe --device-map=/boot/grub/device.map --target=drive
--device /dev/mapper/test-test
error: no such partition
#/usr/bin/grub-mkimage --output=/boot/grub/core.img
--prefix=(test-test)/boot/grub biosdisk pc raid mdraid lvm reiserfs
biosdisk lvm
#/usr/sbin/grub-setup --directory=/boot/grub
--device-map=/boot/grub/device.map
(md0)
grub-setup: error: no mapping exists for `test-test'
--------------------------------------------------------------------------------------------------------------------------
Second partition is unsed, however grub stoped complaining about
"multiple metadata". Desipte that I still can't boot system.
===== Added RAID10 using second partition pair, I'll try to boot system
from it
sysresccd / # grub-mkimage -o /boot/grub/core.img biosdisk pc raid
mdraid lvm reiserfs
#/usr/bin/grub-mkimage -o /boot/grub/core.img biosdisk pc raid mdraid
lvm reiserfs
sysresccd / # grub-setup '(md1)'
#/usr/sbin/grub-setup (md1)
grub-setup: error: Core image is too big for embedding, but this is
required when
the root device is on a RAID array or LVM volume.
--------------------------------------------------------------------------------------------------------------------------
After seeing that error message I created core.img without lvm module
and rebooted computer (I was able to boot system),
then I loaded lvm module but I couldn't see any lvm volumes (despite
fact that I saw md0 containing it). I tried to rmmod/inmod
again but it didn't help.
OK so here are my thoughts - most ppl use ext2 for /boot, I used
reiserfs which has 4k larger module so LVM just don't fit info core.img.
Using grub-install/setup on LVM@RAID always ends up with some errors I
just couldn't see it before (those errors were discarted
somewhere along the way ?) but it doesn't explain why I couldn't see lvm
volumes desipe the fact that loaded lvm module from disk.
Perhaps I should try to create LVM on md1 and boot from md0 to see what
will happen ?
Anyway there is something wrong with LVM+RAID on single partition ;)
>
> It's caused by size limitation of mbr, which is normally 63 sectors.
> When you use the following modules:
>
> biosdisk pc reiserfs lvm mdraid raid6 raid6rec
>
> The size of core.img would exceed 63 sectors, so that it can't be
> installed to mbr.
>
>
Is it possible to use more sectors by leaving some unused space before
fisrt partion ?
next prev parent reply other threads:[~2008-12-05 21:12 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-11-25 1:08 LVM & RAID10 Wojciech Pyczak
2008-11-25 2:52 ` Bean
2008-12-03 22:32 ` Wojciech Pyczak
2008-12-04 4:38 ` Bean
2008-12-05 21:11 ` Wojciech Pyczak [this message]
2009-05-01 18:29 ` Wojciech Pyczak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=49399916.5020307@gmail.com \
--to=nissarin@gmail.com \
--cc=grub-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox