pvcreate problem with 2 metadata areas.

pvcreate problem with 2 metadata areas.

[John Drescher]

I was attempting to demonstrate a bug in the latest git that I still
can not expand a pv that has 2 pvmetadata copies however I hit a
second bug in the process. Here are the steps I have taken (on a
virtual machine):

localhost ~ # uname -a
Linux localhost 2.6.37-gentoo #3 SMP Sun Feb 13 00:17:53 Local time
zone must be set--see zic  x86_64 Intel(R) Core(TM) i7 CPU 920 @
2.67GHz GenuineIntel GNU/Linux

localhost ~ # mdadm -C /dev/md0 -l 5 -n 3 /dev/sd[cde]1
mdadm: /dev/sdc1 appears to be part of a raid array:
    level=raid5 devices=4 ctime=Sat Feb 12 21:24:39 2011
mdadm: /dev/sdd1 appears to be part of a raid array:
    level=raid5 devices=4 ctime=Sat Feb 12 21:24:39 2011
mdadm: /dev/sde1 appears to be part of a raid array:
    level=raid5 devices=4 ctime=Sat Feb 12 21:24:39 2011
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
localhost ~ # cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid0] [raid1] [raid10]
md0 : active raid5 sde1[3] sdd1[1] sdc1[0]
      3140608 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_]
      [=>...................]  recovery =  6.7% (105472/1570304)
finish=1.1min speed=21094K/sec

unused devices: <none>

localhost ~ # lvm version
  LVM version:     2.02.85(2)-cvs (2011-02-09)
  Library version: 1.02.64-cvs (2011-02-09)
  Driver version:  4.18.0

localhost ~ # pvcreate /dev/md0 --pvmetadatacopies 2
  Physical volume "/dev/md0" successfully created
localhost ~ # pvck -v /dev/md0
    Scanning /dev/md0
  Found label on /dev/md0, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=1044480
  Found text metadata area: offset=3215852032, size=1048576
  /dev/md0: lseek 3216900096 failed: Invalid argument
  Read from /dev/md0 failed

It looks like the second metadata copy is 4096 bytes too large.

-- 
John M. Drescher

pvcreate problem with 2 metadata areas.

[Peter Rajnoha]

On 02/22/2011 03:57 PM +0100, John Drescher wrote:
> localhost ~ # pvcreate /dev/md0 --pvmetadatacopies 2
>   Physical volume "/dev/md0" successfully created
> localhost ~ # pvck -v /dev/md0
>     Scanning /dev/md0
>   Found label on /dev/md0, sector 1, type=LVM2 001
>   Found text metadata area: offset=4096, size=1044480
>   Found text metadata area: offset=3215852032, size=1048576
>   /dev/md0: lseek 3216900096 failed: Invalid argument
>   Read from /dev/md0 failed
> 
> It looks like the second metadata copy is 4096 bytes too large.
> 

The last patches sent upstream yesterday are quite risky and we
are still testing these changes. However, the best way we found
was to put these changes upstream directly for even more thorough
testing.

We're still working on this and we expect some time (hopefully
short) to catch and fix all remaining bugs.

Thank you very much for the feedback. You're welcome to report
any other issues you hit. This will help us a lot to resolve all
remaining problems.

Peter

pvcreate problem with 2 metadata areas.

[John Drescher]

On Tue, Feb 22, 2011 at 10:25 AM, Peter Rajnoha <prajnoha@redhat.com> wrote:
> On 02/22/2011 03:57 PM +0100, John Drescher wrote:
>> localhost ~ # pvcreate /dev/md0 --pvmetadatacopies 2
>> ? Physical volume "/dev/md0" successfully created
>> localhost ~ # pvck -v /dev/md0
>> ? ? Scanning /dev/md0
>> ? Found label on /dev/md0, sector 1, type=LVM2 001
>> ? Found text metadata area: offset=4096, size=1044480
>> ? Found text metadata area: offset=3215852032, size=1048576
>> ? /dev/md0: lseek 3216900096 failed: Invalid argument
>> ? Read from /dev/md0 failed
>>
>> It looks like the second metadata copy is 4096 bytes too large.
>>
>
> The last patches sent upstream yesterday are quite risky and we
> are still testing these changes. However, the best way we found
> was to put these changes upstream directly for even more thorough
> testing.
>
> We're still working on this and we expect some time (hopefully
> short) to catch and fix all remaining bugs.
>
> Thank you very much for the feedback. You're welcome to report
> any other issues you hit. This will help us a lot to resolve all
> remaining problems.
>

With todays update the creation of 2 pvmetadataareas works for me.

localhost ~ # pvcreate /dev/md0 --pvmetadatacopies 2
  Physical volume "/dev/md0" successfully created
localhost ~ # pvck -v /dev/md0
    Scanning /dev/md0
  Found label on /dev/md0, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=1044480
  Found text metadata area: offset=3214934016, size=1048576
localhost ~ #

I will now try expanding the array. This failed before this change
with a checksum error. I will test and get back to you in a few
minutes.

John

pvcreate problem with 2 metadata areas.

[John Drescher]

On Fri, Feb 25, 2011 at 9:30 AM, John Drescher <drescherjm@gmail.com> wrote:
> On Tue, Feb 22, 2011 at 10:25 AM, Peter Rajnoha <prajnoha@redhat.com> wrote:
>> On 02/22/2011 03:57 PM +0100, John Drescher wrote:
>>> localhost ~ # pvcreate /dev/md0 --pvmetadatacopies 2
>>> ? Physical volume "/dev/md0" successfully created
>>> localhost ~ # pvck -v /dev/md0
>>> ? ? Scanning /dev/md0
>>> ? Found label on /dev/md0, sector 1, type=LVM2 001
>>> ? Found text metadata area: offset=4096, size=1044480
>>> ? Found text metadata area: offset=3215852032, size=1048576
>>> ? /dev/md0: lseek 3216900096 failed: Invalid argument
>>> ? Read from /dev/md0 failed
>>>
>>> It looks like the second metadata copy is 4096 bytes too large.
>>>
>>
>> The last patches sent upstream yesterday are quite risky and we
>> are still testing these changes. However, the best way we found
>> was to put these changes upstream directly for even more thorough
>> testing.
>>
>> We're still working on this and we expect some time (hopefully
>> short) to catch and fix all remaining bugs.
>>
>> Thank you very much for the feedback. You're welcome to report
>> any other issues you hit. This will help us a lot to resolve all
>> remaining problems.
>>
>
> With todays update the creation of 2 pvmetadataareas works for me.
>
> localhost ~ # pvcreate /dev/md0 --pvmetadatacopies 2
> ?Physical volume "/dev/md0" successfully created
> localhost ~ # pvck -v /dev/md0
> ? ?Scanning /dev/md0
> ?Found label on /dev/md0, sector 1, type=LVM2 001
> ?Found text metadata area: offset=4096, size=1044480
> ?Found text metadata area: offset=3214934016, size=1048576
> localhost ~ #
>
> I will now try expanding the array. This failed before this change
> with a checksum error.

localhost ~ # mdadm -G /dev/md0 -l 5 -n 4
mdadm: Need to backup 3072K of critical section..
localhost ~ # cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid0] [raid1] [raid10]
md0 : active raid5 sdf1[4] sde1[3] sdd1[1] sdc1[0]
      3140608 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
      [>....................]  reshape =  2.7% (42524/1570304)
finish=2.3min speed=10631K/sec

unused devices: <none>
localhost ~ # pvresize /dev/md0
  Physical volume "/dev/md0" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized
localhost ~ # pvck -v /dev/md0
    Scanning /dev/md0
  Found label on /dev/md0, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=1044480
  Found text metadata area: offset=3214934016, size=1048576

Hmm. It looks like mdadm is not expanding the size until the grow
finishes.. Should lvm2 have tried to resize a pv by 0 bytes?

Okay now waiting a few minutes for the grow to almost finish:

localhost ~ # cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid0] [raid1] [raid10]
md0 : active raid5 sdf1[4] sde1[3] sdd1[1] sdc1[0]
      3140608 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
      [===================>.]  reshape = 98.6% (1550336/1570304)
finish=0.0min speed=17867K/sec

unused devices: <none>
localhost ~ # cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid0] [raid1] [raid10]
md0 : active raid5 sdf1[4] sde1[3] sdd1[1] sdc1[0]
      4710912 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]

unused devices: <none>

localhost ~ # pvresize /dev/md0 -v
    Using physical volume(s) on command line
    Wiping cache of LVM-capable devices
    Wiping cache of LVM-capable devices
    Resizing volume "/dev/md0" to 6281216 sectors.
    Updating physical volume "/dev/md0"
  Physical volume "/dev/md0" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized
localhost ~ # pvck -v /dev/md0
    Scanning /dev/md0
  Found label on /dev/md0, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=1044480
  Found text metadata area: offset=4822925312, size=1048576
localhost ~ #

Looks like that worked now.

Thank You.
-- 
John M. Drescher