[linux-lvm] LVM and sw RAID1

[linux-lvm] LVM and sw RAID1

[Peter Green]

I am having a really difficult time being able to get LVM to recognize my
/dev/md[012] arrays. The system:

  (pcg@dmna) ~> uname -a
  Linux dmna 2.4.0-0.21 #2 Wed Aug 23 11:46:12 EDT 2000 i686 unknown
  
  (pcg@dmna) ~> cat /proc/lvm
  LVM driver version 0.8final  (15/02/2000)

  Total:  0 VGs  0 PVs  0 LVs (0 LVs open)
  Global: 143 bytes malloced   IOP version: 6   4:01:56 active
  
  (pcg@dmna) ~> cat /proc/mdstat 
  Personalities : [linear] [raid0] [raid1] [raid5] 
  read_ahead 1024 sectors
  md2 : active raid1 sdg1[1] sdd1[0]
        17920384 blocks [2/2] [UU]
	      
  md1 : active raid1 sdf1[1] sdc1[0]
        35840896 blocks [2/2] [UU]

  md0 : active raid1 sde1[1] sdb1[0]
        17936448 blocks [2/2] [UU]

(User-space tools are 0.8final+0.8final-2 patch. /dev/sda is on a BusLogic
BT-958, while sd[b-g] are on two separate channels of an Adaptec 39160.)
Here is the symptom:

  (pcg@dmna) ~> sudo pvcreate /dev/md0
  pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

  (pcg@dmna) ~> sudo pvcreate /dev/md1
  pvcreate -- invalid partition type 0x83 for "/dev/md1" (must be 0x8e)

  (pcg@dmna) ~> sudo pvcreate /dev/md2
  pvcreate -- can't use DOS extended partition "/dev/md2"

Just for completeness:

  (pcg@dmna) ~> sudo pvcreate -f /dev/md0
  pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

  (pcg@dmna) ~> sudo pvcreate -ff /dev/md0
  pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

  (pcg@dmna) ~> sudo pvcreate -f /dev/md1
  pvcreate -- invalid partition type 0x83 for "/dev/md1" (must be 0x8e)

  (pcg@dmna) ~> sudo pvcreate -ff /dev/md1
  pvcreate -- invalid partition type 0x83 for "/dev/md1" (must be 0x8e)

  (pcg@dmna) ~> sudo pvcreate -f /dev/md2
  pvcreate -- can't use DOS extended partition "/dev/md2"

  (pcg@dmna) ~> sudo pvcreate -ff /dev/md2
  pvcreate -- can't use DOS extended partition "/dev/md2"

What I don't understand is how those partition types got associated with my
/dev/md? devices. Some more output:

  (pcg@dmna) ~> sudo lvmdiskscan 
  lvmdiskscan -- reading all disks / partitions (this may take a while...)
  lvmdiskscan -- /dev/sda1 [      15.66 MB] Primary  LINUX native partition [0x83]
  lvmdiskscan -- /dev/sda2 [       8.53 GB] DOS extended partition [0x5]
  lvmdiskscan -- /dev/sda5 [     258.83 MB] Extended LINUX swap partition [0x82]
  lvmdiskscan -- /dev/sda6 [       8.28 GB] Extended LINUX native partition [0x83]
  lvmdiskscan -- /dev/sdb1 [      17.11 GB] Primary   [0xFD]
  lvmdiskscan -- /dev/sdc1 [      34.18 GB] Primary   [0xFD]
  lvmdiskscan -- /dev/sdd1 [      17.09 GB] Primary   [0xFD]
  lvmdiskscan -- /dev/sde1 [      17.11 GB] Primary   [0xFD]
  lvmdiskscan -- /dev/sdf1 [      34.18 GB] Primary   [0xFD]
  lvmdiskscan -- /dev/sdg1 [      17.09 GB] Primary   [0xFD]
  lvmdiskscan -- /dev/md1  [      15.66 MB] Primary  LINUX native partition [0x83]
  lvmdiskscan -- /dev/md2  [       8.53 GB] DOS extended partition [0x5]
  lvmdiskscan -- 7 disks
  lvmdiskscan -- 0 whole disks
  lvmdiskscan -- 0 loop devices
  lvmdiskscan -- 0 multiple devices
  lvmdiskscan -- 0 network block devices
  lvmdiskscan -- 10 partitions
  lvmdiskscan -- 0 LVM physical volume partitions

As you can see, there are some pretty weird entries for /dev/md[12] and
*nothing* for /dev/md0. lvmdiskscan also doesn't detect any multiple
devices, which doesn't seem right. (/dev/md0 and /dev/md2 are mirrors of two
18GB disks, while /dev/md1 is a mirror of two 36GB disks.) Interestingly
(and I just noticed this) the output for /dev/md1 matches that of /dev/sda1,
while the output for /dev/md2 matches /dev/sda2.

Any idea of what's going on and how to fix it? Is there any other
information I can provide to help?

Thanks!

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
> > Other than the fact Linux has a cool name, could someone explain why I
> > should use Linux over BSD?
>
> No.  That's it.  The cool name, that is.  We worked very hard on
> creating a name that would appeal to the majority of people, and it
> certainly paid off: thousands of people are using linux just to be able
> to say "OS/2? Hah.  I've got Linux.  What a cool name".  386BSD made the
> mistake of putting a lot of numbers and weird abbreviations into the
> name, and is scaring away a lot of people just because it sounds too
> technical.
(Linus Torvalds' follow-up to a question about Linux)

Re: [linux-lvm] LVM and sw RAID1

[Andreas Dilger]

Peter Green writes:
> I am having a really difficult time being able to get LVM to recognize my
> /dev/md[012] arrays.

What does /proc/partitions show you?  What about cfdisk -P s /dev/sdX for
each drive?

>   (pcg@dmna) ~> sudo pvcreate /dev/md0
>   pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

Have you tried doing this directly as root?  I'm not sure if it makes a
difference, but it is worth trying.

Have you applied the fixes from "BUGS" to the LVM tools?

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LVM and sw RAID1

[Peter Green]

Thanks for the response,

also sprach adilger:
> Peter Green writes:
> > I am having a really difficult time being able to get LVM to recognize my
> > /dev/md[012] arrays.
> 
> What does /proc/partitions show you?  What about cfdisk -P s /dev/sdX for
> each drive?

/proc/partitions:
  major minor  #blocks  name

     8     0    8965347 sda
     8     1      16033 sda1
     8     2          1 sda2
     8     5     265041 sda5
     8     6    8683101 sda6
     8    16   17942584 sdb
     8    17   17936541 sdb1
     8    32   35843684 sdc
     8    33   35840983 sdc1
     8    48   17921835 sdd
     8    49   17920476 sdd1
     8    64   17942584 sde
     8    65   17936541 sde1
     8    80   35843684 sdf
     8    81   35840983 sdf1
     8    96   17921835 sdg
     8    97   17920476 sdg1
     9     0   17936448 md0
     9     1   35840896 md1
     9     2   17920384 md2

cfdisk:
  [root@dmna /root]# cfdisk -P s /dev/sdb
  Partition Table for /dev/sdb
  
              First    Last
   # Type     Sector   Sector   Offset  Length   Filesystem Type (ID)   Flags
  -- ------- -------- --------- ------ --------- ---------------------- ---------
   1 Primary        0 35873144      63 35873145  Linux raid autode (FD) None (00)

It's the same for all six of the drives, except for last sector and length.
The first drive is obviously different, having the root fs on it:

  [root@dmna /root]# cfdisk -P s /dev/sda
  Partition Table for /dev/sda
  
              First    Last
   # Type     Sector   Sector   Offset  Length   Filesystem Type (ID)   Flags
  -- ------- -------- --------- ------ --------- ---------------------- ---------
   1 Primary        0    32129      63    32130  Linux (83)             Boot (80)
   2 Primary    32130 17928539       0 17896410  Extended (05)          None (00)
   5 Logical    32130   562274      63   530145  Linux swap (82)        None (00)
   6 Logical   562275 17928539      63 17366265  Linux (83)             None (00)

> >   (pcg@dmna) ~> sudo pvcreate /dev/md0
> >   pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)
> 
> Have you tried doing this directly as root?  I'm not sure if it makes a
> difference, but it is worth trying.

Same result:

[root@dmna /root]# pvcreate /dev/md0
pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

[root@dmna /root]# pvcreate -f /dev/md0
pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

[root@dmna /root]# pvcreate -ff /dev/md0
pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

> Have you applied the fixes from "BUGS" to the LVM tools?

In the BUGS list:

* ``patch-lvm_0.8final-2    - patch against LVM 0.8final user mode tools'':
  I applied this.
* ``If you compile devfs into the kernel...'': Not applicable, no devfs.
* ``Arjan Filius noted the following problem with kernel version
  2.3.99-pre[23]'': Not applicable, not my kernel *plus* this was a compile
  problem, with which I had no difficulty.
* ``Jorg de Jong[...]detected a bug...'': I can't even get this far.
* ``in LVM 0.8e (4/1/2000) (shipped with SuSE 6.4)...'': Not my version, not
  my OS, not the problem I'm seeing. :)
* ``Harald Milz[...]discovered a bug with pvdisplay'': Ditto (not the
  problem I'm seeing).

I was pretty careful about checking everything out before-hand. That's why
I'm so baffled. It seems like pvcreate shouldn't even be looking for a
partition type for MD, right?

Anyway, thanks for the help thus far!

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
Microsoft is not the answer.
Microsoft is the question.
NO (or Linux) is the answer.
(Taken from a .signature from someone from the UK, source unknown)

Re: [linux-lvm] LVM and sw RAID1

[Andreas Dilger]

Peter writes:
> [root@dmna /root]# pvcreate /dev/md0
> pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)
> 
> [root@dmna /root]# pvcreate -f /dev/md0
> pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)
> 
> [root@dmna /root]# pvcreate -ff /dev/md0
> pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

Have you tried running "pvcreate -v -d /dev/md0"?  It may be that you
get some more clues with this.  Maybe even strace of the above.  It
looks like pvcreate is not getting the correct "dev" or "rdev" from the
inode...

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LVM and sw RAID1

[Andreas Dilger]

I was just checking out pvcreate on my system, and it seems that LVM is doing
a bit more than it really should.  It appears that pv_get_size is actually
reading the partition table from the disk, but shouldn't this be left to
other code in the kernel (e.g. /proc/partitions or BLKGETSIZE ioctl)?

With the growing number of partition types that Linux supports, it makes
no sense for LVM to be doing things like this...

Cheers, Andreas

Re: [linux-lvm] LVM and sw RAID1

[Peter Green]

[-- Attachment #1: Type: text/plain, Size: 1738 bytes --]

also sprach adilger:
> Have you tried running "pvcreate -v -d /dev/md0"?  It may be that you
> get some more clues with this.  Maybe even strace of the above.  It
> looks like pvcreate is not getting the correct "dev" or "rdev" from the
> inode...

I ran ``strace -o pv.strace -ff -s 100 -v pvcreate -d -v /dev/md0 > pv.out
2>pv.debug''. I've attached the three output files (gzip'd for pv.strace).

The problem occurs in pv_get_size() starting around line 87. The device name
is looked up in the cache. If found, it is tested to be a partitioned device
or not. This call returns false for the /dev/md? devices; MD_MAJOR is not
listed as a valid major number for partitioned devices (rightly so, I
think).

Since lvm_check_partitioned_dev() returns FALSE, the for loop is never
executed, and ``i'' remains at the value to which it was initialized (0) on
line 50. However, i is then used at line 103 to get the device name:

  strncpy ( disk_dev_name, dir_cache[i].dev_name, sizeof ( disk_dev_name) - 1);

This obviously assumes we're dealing solely with partitioned devices, which
in my case I'm not. Since i=0, dir_cache[i].dev_name is the name of the
first device: /dev/sda. The strace provides evidence of this:

  write(2, "lvm_check_partitioned_dev -- LEAVING\n", 37) = 37
  open("/dev/sda", O_RDONLY)              = 5

Everywhere else in the strace references /dev/md0, which is what I used on
the cmdline.

Do these files help any?

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
"I don't know why, but first C programs tend to look a lot worse than
first programs in any other language (maybe except for fortran, but then
I suspect all fortran programs look like `firsts')"
(By Olaf Kirch)


[-- Attachment #2: pv.out --]
[-- Type: text/plain, Size: 173 bytes --]

pvcreate -- locking logical volume manager
pvcreate -- checking physical volume name "/dev/md0"
pvcreate -- getting physical volume size
pvcreate -- checking partition type

[-- Attachment #3: pv.debug --]
[-- Type: text/plain, Size: 7428 bytes --]

<1> lvm_get_iop_version -- CALLED
<22> lvm_check_special -- CALLED
<22> lvm_check_special -- LEAVING
<1> lvm_get_iop_version -- AFTER ioctl ret: 0
<1> lvm_get_iop_version -- LEAVING with ret: 6
<1> lvm_lock -- CALLED
<22> lvm_check_special -- CALLED
<22> lvm_check_special -- LEAVING
<1> lvm_lock -- LEAVING with ret: 0
<1> pv_check_name -- CALLED eith "/dev/md0"
<22> lvm_check_chars -- CALLED with name: "/dev/md0"
<22> lvm_check_chars -- LEAVING with ret: 0
<22> lvm_check_dev -- CALLED
<22> lvm_check_dev -- LEAVING with ret: 1
<1> pv_check_name -- LEAVING with ret: 0
<1> pv_get_size -- CALLED with /dev/md0 and BFFFF76C
<22> pv_check_name -- CALLED eith "/dev/md0"
<333> lvm_check_chars -- CALLED with name: "/dev/md0"
<333> lvm_check_chars -- LEAVING with ret: 0
<333> lvm_check_dev -- CALLED
<333> lvm_check_dev -- LEAVING with ret: 1
<22> pv_check_name -- LEAVING with ret: 0
<22> pv_check_part -- CALLED with /dev/md0
<333> pv_check_name -- CALLED eith "/dev/md0"
<4444> lvm_check_chars -- CALLED with name: "/dev/md0"
<4444> lvm_check_chars -- LEAVING with ret: 0
<4444> lvm_check_dev -- CALLED
<4444> lvm_check_dev -- LEAVING with ret: 1
<333> pv_check_name -- LEAVING with ret: 0
<333> lvm_dir_cache_find -- CALLED
<4444> pv_check_name -- CALLED eith "/dev/md0"
<55555> lvm_check_chars -- CALLED with name: "/dev/md0"
<55555> lvm_check_chars -- LEAVING with ret: 0
<55555> lvm_check_dev -- CALLED
<55555> lvm_check_dev -- LEAVING with ret: 1
<4444> pv_check_name -- LEAVING with ret: 0
<4444> lvm_dir_cache -- CALLED
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<55555> lvm_add_dir_cache -- CALLED
<666666> lvm_check_dev -- CALLED
<666666> lvm_check_dev -- LEAVING with ret: 1
<666666> lvm_dir_cache_hit -- CALLED
<666666> lvm_dir_cache_hit -- LEAVING with ret: 0
<55555> lvm_add_dir_cache -- LEAVING
<4444> lvm_dir_cache -- LEAVING with ret: 20
<333> lvm_dir_cache_find -- LEAVING
<333> lvm_partition_count -- CALLED
<333> lvm_partition_count -- LEAVING with ret: 16
<22> pv_check_part -- LEAVING with -218
<22> lvm_dir_cache -- CALLED
<22> lvm_dir_cache -- LEAVING with ret: 20
<22> lvm_dir_cache_find -- CALLED
<333> pv_check_name -- CALLED eith "/dev/md0"
<4444> lvm_check_chars -- CALLED with name: "/dev/md0"
<4444> lvm_check_chars -- LEAVING with ret: 0
<4444> lvm_check_dev -- CALLED
<4444> lvm_check_dev -- LEAVING with ret: 1
<333> pv_check_name -- LEAVING with ret: 0
<333> lvm_dir_cache -- CALLED
<333> lvm_dir_cache -- LEAVING with ret: 20
<22> lvm_dir_cache_find -- LEAVING
<22> lvm_check_partitioned_dev -- CALLED
<22> lvm_check_partitioned_dev -- LEAVING
<1> pv_get_size -- BEFORE llseek 0:0
<1> pv_get_size -- part[0].sys_ind: 83  part[0].nr_sects: 32067
<1> pv_get_size -- first == 1
<1> pv_get_size -- part_i == part_i_tmp
<1> pv_get_size -- LEAVING with ret: 32067
pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

<1> lvm_unlock -- CALLED
<1> lvm_unlock -- LEAVING with ret: 0

[-- Attachment #4: pv.strace.gz --]
[-- Type: application/x-gzip, Size: 3477 bytes --]

Re: [linux-lvm] LVM and sw RAID1

[Nils Juergens]

On Wed, 23.08.00, Peter Green <pcg@gospelcom.net> wrote:
>   (pcg@dmna) ~> uname -a
>   Linux dmna 2.4.0-0.21 #2 Wed Aug 23 11:46:12 EDT 2000 i686 unknown

I'm a bit confused. Shouldnt the kernel-version be something like
"2.4.0-testN" instead of "2.4.0-0.21"?

I for one am using 2.4.0-test5 with LVM 0.8final on sw-Raid Lv.5 wihout any
problems so far, my pvcreate behaved like it should.

'later,

Nils

-- 
 >   Nils Juergens  | nils@muon.de | icq 7090774   <
 >    Yes, Sir. Deathmatch _is_ a marketing term   <
 >        for "online brainstorming session"       <

Re: [linux-lvm] LVM and sw RAID1

[Peter.Wuestefeld]

[-- Attachment #1: Type: text/plain, Size: 1912 bytes --]

Hi,

I think you are trying to cure the wrong disease ;-)

> [root@dmna /root]# pvcreate -f /dev/md0
> pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

shows what is wrong:

try to set the partition id to 0x8e with fdisk/sfdisk/cfdisk - that should 
solve the problem.

HTH,

Peter

Mit freundlichem Gruss/Best Regards

Peter Wuestefeld
IBM Certified System Administrator AIX 4.3
IBM Certified Support Professional AIX 4.3
==================================
res nova Unternehmensberatung GmbH
                   ...aktiv in AIX
MAIL:  Ruppmannstrasse 41
       D-70565 Stuttgart, Germany
FON:   +49 711 78888 0
FAX:   +49 711 78888 99
WWW:   http://www.resnova.de
SMTP: mailto: Peter.Wuestefeld@resnova.de

Computers are useless. They can only give you answers.
        -- Pablo Picasso




Andreas Dilger <adilger@turbolinux.com>
Sent by: owner-linux-lvm@msede.com
24.08.00 02:02

 
        To:     Peter Green <pcg@gospelcom.net>
        cc:     Andreas Dilger <adilger@turbolinux.com>, linux-lvm@msede.com
        Subject:        Re: [linux-lvm] LVM and sw RAID1

Peter writes:
> [root@dmna /root]# pvcreate /dev/md0
> pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)
> 
> [root@dmna /root]# pvcreate -f /dev/md0
> pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)
> 
> [root@dmna /root]# pvcreate -ff /dev/md0
> pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be 0x8e)

Have you tried running "pvcreate -v -d /dev/md0"?  It may be that you
get some more clues with this.  Maybe even strace of the above.  It
looks like pvcreate is not getting the correct "dev" or "rdev" from the
inode...

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert



[-- Attachment #2: Type: text/html, Size: 3395 bytes --]

Re: [linux-lvm] LVM and sw RAID1

[Luca Berra]

On Wed, Aug 23, 2000 at 04:29:57PM -0400, Peter Green wrote:
> I am having a really difficult time being able to get LVM to recognize my
> /dev/md[012] arrays. The system:

hello you have stumbled upon a (nother) nice bug in lvm
i asked for help twice on this list, none helped even bothered to answer,
anyway i found the bug, well there is more than one preventing you to use
raid and LVM, fixed that and pvmoved everything onto my lvm+raid5 (fixing another bug
in the process).
Well, i discovered that reiserfs + LVM + raid DOES NOT WORK, it actually filled
95% of my files with zeroes :)

(*use the tape luke*)

unluckyli the patch to fix lvm+raid was not on tape, so i will post it as soon
i manage to rewrite it from memory,  hold your breath and mail me personally if
i forget :)

BTW i am back with ext2 and it works

Regards.
L.

-- 
Luca Berra -- bluca@comedia.it
    Communication Media & Services S.r.l.

Re: [linux-lvm] LVM and sw RAID1

[Peter Green]

Thanks for the response,

also sprach Peter.Wuestefeld:
> 
>    Hi,
>    I think you are trying to cure the wrong disease ;-)
>    > [root@dmna /root]# pvcreate -f /dev/md0
>    > pvcreate -- invalid partition type 0x83 for "/dev/md0" (must be
>    0x8e)
>    shows what is wrong:
>    try to set the partition id to 0x8e with fdisk/sfdisk/cfdisk - that
>    should solve the problem.

There *aren't* any partitions on a /dev/md? device. Look at the strace I
sent; it's trying to pvcreate /dev/sda, even though I specify /dev/md0 on
the command line.

Thanks for your help.

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
We come to bury DOS, not to praise it.
(Paul Vojta, vojta@math.berkeley.edu, paraphrasing a quote of Shakespeare)

Re: [linux-lvm] LVM and sw RAID1

[Peter Green]

also sprach nils:
> On Wed, 23.08.00, Peter Green <pcg@gospelcom.net> wrote:
> >   (pcg@dmna) ~> uname -a
> >   Linux dmna 2.4.0-0.21 #2 Wed Aug 23 11:46:12 EDT 2000 i686 unknown
> 
> I'm a bit confused. Shouldnt the kernel-version be something like
> "2.4.0-testN" instead of "2.4.0-0.21"?

It's a RedHat kernel from rawhide.

> I for one am using 2.4.0-test5 with LVM 0.8final on sw-Raid Lv.5 wihout any
> problems so far, my pvcreate behaved like it should.

Hrm, I might try the stock kernel instead. Usually, RedHat does fairly good
things with any patches they add to their kernel, but this might be a case
in which they've done something naughty.

Thanks for the pointers!

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
/*
 * Oops. The kernel tried to access some bad page. We'll have to
 * terminate things with extreme prejudice.
*/
die_if_kernel("Oops", regs, error_code);
(From linux/arch/i386/mm/fault.c)				   

Re: [linux-lvm] LVM and sw RAID1

[Peter Green]

also sprach nils:
> On Wed, 23.08.00, Peter Green <pcg@gospelcom.net> wrote:
> >   (pcg@dmna) ~> uname -a
> >   Linux dmna 2.4.0-0.21 #2 Wed Aug 23 11:46:12 EDT 2000 i686 unknown
> 
> I'm a bit confused. Shouldnt the kernel-version be something like
> "2.4.0-testN" instead of "2.4.0-0.21"?

Okay, I've installed 2.4.0-test6 with the exact same results. Stock lvm
tools give ``invalid partition type 0x83'' for /dev/md0, my altered tools
give no errors for pvcreate, but nothing shows up in pvscan.

> I for one am using 2.4.0-test5 with LVM 0.8final on sw-Raid Lv.5 wihout any
> problems so far, my pvcreate behaved like it should.

Could it be a LVM+RAIDn problem where n != 5?

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
Feel free to contact me (flames about my english and the useless of this
driver will be redirected to /dev/null, oh no, it's full...).
(Michael Beck, describing the PC-speaker sound device)

Re: [linux-lvm] LVM and sw RAID1

[Luca Berra]

good it seems you found the first bug in lvm_dir_cache

the second bug is in a function called something like 
pv_check_all_pv it tries to remove all block devices that are part
of an md device, finds none and removes the md device.

obviously this check is not needed unless you have a raid partition
of type 0x8e, or you use full disks.

good luck

L.

P.S. i noticed the result of this routines varies depending with
system configuration, at one particular point pv_check_all_pv did
segfault on me, i removed a disk an it did work :(

i suggest someone reviews that code.
On Thu, Aug 24, 2000 at 08:30:01AM -0400, Peter Green wrote:
> also sprach nils:
> > On Wed, 23.08.00, Peter Green <pcg@gospelcom.net> wrote:
> > >   (pcg@dmna) ~> uname -a
> > >   Linux dmna 2.4.0-0.21 #2 Wed Aug 23 11:46:12 EDT 2000 i686 unknown
> > 
> > I'm a bit confused. Shouldnt the kernel-version be something like
> > "2.4.0-testN" instead of "2.4.0-0.21"?
> 
> Okay, I've installed 2.4.0-test6 with the exact same results. Stock lvm
> tools give ``invalid partition type 0x83'' for /dev/md0, my altered tools
> give no errors for pvcreate, but nothing shows up in pvscan.
> 
> > I for one am using 2.4.0-test5 with LVM 0.8final on sw-Raid Lv.5 wihout any
> > problems so far, my pvcreate behaved like it should.
> 
> Could it be a LVM+RAIDn problem where n != 5?



-- 
Luca Berra -- bluca@comedia.it
    Communication Media & Services S.r.l.

Re: [linux-lvm] LVM and sw RAID1

[Andreas Dilger]

Luca Berra write:
> anyway i found the bug, well there is more than one preventing you to use
> raid and LVM,

Can you send your fixes to the list?  At least then everyone else doesn't
need to fix the same bugs.  It would be even better if Heinz made a new
bugfix release, but I watch the list for fixes anyways...

> fixed that and pvmoved everything onto my lvm+raid5 (fixing another bug
> in the process).  Well, i discovered that reiserfs + LVM + raid DOES NOT
> WORK, it actually filled 95% of my files with zeroes :)

No journalling filesystem will work properly with RAID 5 - not reiserfs
and not ext3.  You need to stick with a non-journalling filesystem, or
use RAID1.  It may be that this is supposed to be fixed in 2.4, but I'm
not 100% sure...

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LVM and sw RAID1

[Luca Berra]

On Thu, Aug 24, 2000 at 11:47:23AM -0600, Andreas Dilger wrote:
> Luca Berra write:
> > anyway i found the bug, well there is more than one preventing you to use
> > raid and LVM,
> 
> Can you send your fixes to the list?  At least then everyone else doesn't
> need to fix the same bugs.  It would be even better if Heinz made a new
> bugfix release, but I watch the list for fixes anyways...
Andreas _please_ read my emails
i said that my fix was trashed due to fs failure, i will post it to
the list as soon as i write it again.
i posted what i remembered, tomorrow i dun't have to work so i will look
into it.
i also watch for fixes, i even made a nice big patch encmpassing
everithing that went on the list, sadly it was on a reiserfs fs.

> > fixed that and pvmoved everything onto my lvm+raid5 (fixing another bug
> > in the process).  Well, i discovered that reiserfs + LVM + raid DOES NOT
> > WORK, it actually filled 95% of my files with zeroes :)
> 
> No journalling filesystem will work properly with RAID 5 - not reiserfs
> and not ext3.  You need to stick with a non-journalling filesystem, or
> use RAID1.  It may be that this is supposed to be fixed in 2.4, but I'm
> not 100% sure...
> 
I probably dinna specify but it i am running 2.4.0-test7pre5
so i can confirm reiserfs not working on raid+lvm with 2.4
AFAIR the problem with journaling and raid was due to raid's usage
of the buffer cache during recnstruction, there was no reconstrucion
happening on my system. i will also try reiserfs on raid without
lvm, i cannot try ext3 since there isn't any patch for 2.4

L.

-- 
Luca Berra -- bluca@comedia.it
    Communication Media & Services S.r.l.

Re: [linux-lvm] LVM and sw RAID1

[Andreas Dilger]

Peter writes:
> The problem occurs in pv_get_size() starting around line 87. The device name
> is looked up in the cache. If found, it is tested to be a partitioned device
> or not. This call returns false for the /dev/md? devices; MD_MAJOR is not
> listed as a valid major number for partitioned devices (rightly so, I
> think).

I _think_ it would be possible to replace all of this partition-reading crap
with a much simpler solution - read /proc/partitions, or do BLKGETSIZE ioctl,
or failing that do a binary search of the device trying to read data - code
exists in mke2fs and ext2resize.  The below is half of Andrea Arcangeli's
Alpha code, and half code from ext2resize.  Give it a try for pv_get_size.c.

I also don't see much use for lvm_check_dev(), which seems to be very high
maintenance (it needs to track all new devices added to the kernel), and
pv_check_name() also really relies on specific names for specific devices.
Can't lvm_check_dev() simply check if we are dealing with a rw block device,
and get rid of pv_check_name() entirely?  Granted that some people will try
to make PVs on /dev/ram0 or /dev/fd0, but maybe they should be allowed to
try, and maybe they have a good reason to do so (e.g. /dev/fd0 is really a
100MB zip disk or something).  I suppose it shouldn't be possible to make
a PV on an LV...

The only remaining difficulty lies in checking the partition type.  This could
be done by a shell command out to fdisk or cfdisk, or by simply trusting the
user to do the right thing.  None of the other tools like mke2fs check if the
partition type is correct - they simply go ahead and do what you ask.  I can
see checking if the partition is mounted but that's about it...

Cheers, Andreas
--- cut here ---
int valid_off(int fd, __loff_t offset)
{
	char c;

	/* should probably use lseek64 if available */
	return (llseek(fd, offset, SEEK_SET) < 0 || read(fd, &c, 1) < 1)? 0 : 1;  
}

unsigned long pv_get_size(char *dev_name, struct partition *dummy)
{
	loff_t low, high;
	int pv_fd;
	unsigned long size;

	pv_fd = open(dev_name, O_RDONLY);
	if (pv_fd < 0)
		return -LVM_EPV_GET_SIZE_OPEN;

	if (ioctl(pv_fd, BLKGETSIZE, &size) < 0) {
		for (low = 0, high = 1024; valid_off(pv_fd, high); high *= 2)
			low = high;

		while (low < high - 1) {
			loff_t mid = (low + high) / 2;

			if (valid_off(pv_fd, mid))
				low = mid;
			else
				high = mid;
		}

		size = high >> 9;
	}
	close(pv_fd);
	return size;
}
--- cut here ---


-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [linux-lvm] LVM and sw RAID1

[Nils Juergens]

On Thu, 24.08.00, Peter Green <pcg@gospelcom.net> wrote:
> 
> Could it be a LVM+RAIDn problem where n != 5?
> 

I'm not sure. I just tried raid1 on my setup and was able to use
pvcreate without (obvious) problems...

-----------------------------------------------------------------------
sisko:~# pvcreate -v /dev/md1
pvcreate -- locking logical volume manager
pvcreate -- checking physical volume name "/dev/md1"
pvcreate -- getting physical volume size
pvcreate -- checking partition type
pvcreate -- creating new physical volume
pvcreate -- setting up physical volume for /dev/md1 with 79746048 sectors
pvcreate -- writing physical volume data to disk "/dev/md1"
pvcreate -- physical volume "/dev/md1" successfully created
pvcreate -- unlocking logical volume manager
-----------------------------------------------------------------------

but then i find lvm reports the wrong size on that pv, it actually
reports the size to be the same as on the other pv (/dev/md0)

------------------------------------------------------------------------
sisko:~# pvscan
pvscan -- reading all physical volumes (this may take a while...)
pvscan -- ACTIVE   PV "/dev/md0" of VG "raidvolumes" [38.02 GB / 16.02 GB free]
pvscan -- inactive PV "/dev/md1"  is in no VG  [38.03 GB]
pvscan -- total: 2 [76.05 GB] / in use: 1 [38.03 GB] / in no VG: 1 [38.03 GB]
------------------------------------------------------------------------

/dev/md1 should be about 1.2 GB, not 38 GB.

I now remeber i also had troubles with vgchange which had to do with
vg_check_online_all_pv(). I applied a _very_dirty_ hack to get around the
problem (which i think only works because i just use raid-devices as
pv's so use at own risk :)) )

-----------------------------------------------------------------------
--- vgchange.c.orig     Thu Aug 24 20:24:38 2000
+++ vgchange.c  Thu Jul 27 20:48:09 2000
@@ -343,7 +343,7 @@
             if ( opt_v > 0) printf ( "%s -- checking if all physical volumes "
                                      "of volume group \"%s\" are available\n",
                                      cmd, vg_name);
-            if ( ( ret = vg_check_online_all_pv ( vg, &pv_offl,
+            if ( ! ( ret = vg_check_online_all_pv ( vg, &pv_offl,
                                                   &pv_incons)) < 0) {
                if ( ret == -LVM_EPV_READ_PV_CREATE_NAME_FROM_KDEV_T) {
                      fprintf ( stderr, "%s -- ERROR: can't get name(s) of "
-----------------------------------------------------------------------

have fun,

Nils
-- 
 >   Nils Juergens  | nils@muon.de | icq 7090774   <
 >    Yes, Sir. Deathmatch _is_ a marketing term   <
 >        for "online brainstorming session"       <

Re: [linux-lvm] LVM and sw RAID1

[Ragnar Kjørstad]

On Thu, Aug 24, 2000 at 08:04:38PM +0200, Luca Berra wrote:
> > > fixed that and pvmoved everything onto my lvm+raid5 (fixing another bug
> > > in the process).  Well, i discovered that reiserfs + LVM + raid DOES NOT
> > > WORK, it actually filled 95% of my files with zeroes :)
> > 
> > No journalling filesystem will work properly with RAID 5 - not reiserfs
> > and not ext3.  You need to stick with a non-journalling filesystem, or
> > use RAID1.  It may be that this is supposed to be fixed in 2.4, but I'm
> > not 100% sure...
> > 
> I probably dinna specify but it i am running 2.4.0-test7pre5
> so i can confirm reiserfs not working on raid+lvm with 2.4

If you have more information, please send a bugrapport to
reiserfs@devlinux.com.

> AFAIR the problem with journaling and raid was due to raid's usage
> of the buffer cache during recnstruction, there was no reconstrucion
> happening on my system. i will also try reiserfs on raid without
> lvm, i cannot try ext3 since there isn't any patch for 2.4

For linux 2.2 it will not work with raid5 at all, the reconstruction
issue is just for mirroring....I think.

From Chris Mason on reiserfs@devlinux.com:

> reiserfs and raid:
> The FAQ should state that hardware RAID is not a problem
> 2.2: I found a mail saying that concatening, striping and mirroring
>      are ok, but raid5 has issues, but I also seem to remember reading
>      there were issues with reconstructing mirroring?

Correct, any hardware raid is safe.  For software, striping is ok.
Mirroring is ok, but you *cannot* use the online mirror rebuild.
software raid5 is unsafe.  lvm is safe.

> 2.4: Is everything working here? both lvm and md?
>
Yes, software raid in 2.4 is working properly, and so is lvm.


-- 
Ragnar Kjorstad
Torque Systems / Big Storage

Re: [linux-lvm] LVM and sw RAID1

[Peter Green]

also sprach pcg:
> I am having a really difficult time being able to get LVM to recognize my
> /dev/md[012] arrays. The system:
> 
>   (pcg@dmna) ~> uname -a
>   Linux dmna 2.4.0-0.21 #2 Wed Aug 23 11:46:12 EDT 2000 i686 unknown
>   
>   (pcg@dmna) ~> cat /proc/lvm
>   LVM driver version 0.8final  (15/02/2000)
> 
>   Total:  0 VGs  0 PVs  0 LVs (0 LVs open)
>   Global: 143 bytes malloced   IOP version: 6   4:01:56 active
>   
>   (pcg@dmna) ~> cat /proc/mdstat 
>   Personalities : [linear] [raid0] [raid1] [raid5] 
>   read_ahead 1024 sectors
>   md2 : active raid1 sdg1[1] sdd1[0]
>         17920384 blocks [2/2] [UU]
> 	      
>   md1 : active raid1 sdf1[1] sdc1[0]
>         35840896 blocks [2/2] [UU]
> 
>   md0 : active raid1 sde1[1] sdb1[0]
>         17936448 blocks [2/2] [UU]
> 
> (User-space tools are 0.8final+0.8final-2 patch. /dev/sda is on a BusLogic
> BT-958, while sd[b-g] are on two separate channels of an Adaptec 39160.)

Just as a followup, with the exact same system BUT using the SuSE lvm tools
(lvm-0.8-70; see rpmfind.net), everything seems to work. (The Mandrake RPM
at rpmfind did *not* work.)

I would be interested in seeing what SuSE has done to the source for the
tools. I'm sure the developers here ought to be as well. :)

Anyway, thanks for all the help. Anyone who is running into similar problems
might consider fetching the RPM I've mentioned above; you'll be glad you
did!

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
I'd rather be rich than stupid.
 (Jack Handey)

Re: [linux-lvm] LVM and sw RAID1

[Jan Niehusmann]

On Fri, Aug 25, 2000 at 08:39:44PM -0400, Peter Green wrote:
> Just as a followup, with the exact same system BUT using the SuSE lvm tools
> (lvm-0.8-70; see rpmfind.net), everything seems to work. (The Mandrake RPM
> at rpmfind did *not* work.)

could you please try it again with lvm-0.8final-0.2mdk.i586.rpm? It's available
from http://www.hitnet.rwth-aachen.de/~jan/lvm/lvm-0.8final-0.2mdk.i586.rpm

I added some fixes to version -0.1mdk, and IIRC one of them had something
to do with md scanning. (As I don't use RAID myself, it am not so sure...)

Note that it's not an official mandrake rpm.

> I would be interested in seeing what SuSE has done to the source for the
> tools. I'm sure the developers here ought to be as well. :)

The source is here:
http://www.hitnet.rwth-aachen.de/~jan/lvm/lvm-0.8final-0.2mdk.src.rpm

Jan

Re: [linux-lvm] LVM and sw RAID1

[Peter Green]

also sprach jan:
> On Fri, Aug 25, 2000 at 08:39:44PM -0400, Peter Green wrote:
> > Just as a followup, with the exact same system BUT using the SuSE lvm tools
> > (lvm-0.8-70; see rpmfind.net), everything seems to work. (The Mandrake RPM
> > at rpmfind did *not* work.)
> 
> could you please try it again with lvm-0.8final-0.2mdk.i586.rpm? It's
> available from
> http://www.hitnet.rwth-aachen.de/~jan/lvm/lvm-0.8final-0.2mdk.i586.rpm

Not really; this system is too close to live to monkey around with it. If
I'm building another machine before too long, I'll try it. Otherwise, I'm
afraid I can't help.

/pg
-- 
Peter Green : Gospel Communications Network, SysAdmin : pcg@gospelcom.net
---
Linux is obsolete
(Andrew Tanenbaum)