[linux-lvm] lvm2 2.02.105 breaks snapshots

[linux-lvm] lvm2 2.02.105 breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 1799 bytes --]

Hello everybody,

looks like lvm2 2.02.105 breaks snapshots. This is my block device tree:

NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                   8:0    0   477G  0 disk
|-sda1                8:1    0   767M  0 part
| `-vg0-boot        254:0    0    64M  0 lvm   /boot
|-sda2                8:2    0 444,2G  0 part
| `-cvg             254:3    0 444,2G  0 crypt
|   |-cvg-root      254:4    0    40G  0 lvm   /
|   |-cvg-swap      254:5    0     4G  0 lvm   [SWAP]
|   |-cvg-log       254:6    0     1G  0 lvm   /var/log
|   `-cvg-home      254:8    0   320G  0 lvm   /home
|-sda3                8:3    0    32G  0 part
`-sda128            259:0    0     1M  0 part

Creating a snapshot succeeds, but it is broken and can not be mounted:

# lvcreate -s -pr -l50%free -n snap-home cvg/home
  Logical volume "snap-home" created
# mount /dev/cvg/snap-home /mnt/tmp 
mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
mount: wrong fs type, bad option, bad superblock on
/dev/mapper/cvg-snap--home, missing codepage or helper program, or other error

Syslog has a lot of these messages:

[ 4823.002220] EXT4-fs (dm-7): ext4_check_descriptors: Checksum for group 256
failed (43470!=57954)

Downgrading to lvm2 2.02.104 fixes the problem:

# lvcreate -s -pr -l50%free -n snap-home
cvg/home Logical volume "snap-home" created
# mount /dev/cvg/snap-home /mnt/tmp
mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only

This is an Arch Linux system with Linux 3.12.8.
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] lvm2 2.02.105 breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 2118 bytes --]

Christian Hesse <list@eworm.de> on Thu, 2014/01/23 14:27:
> Hello everybody,
> 
> looks like lvm2 2.02.105 breaks snapshots. This is my block device tree:
> 
> NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
> sda                   8:0    0   477G  0 disk
> |-sda1                8:1    0   767M  0 part
> | `-vg0-boot        254:0    0    64M  0 lvm   /boot
> |-sda2                8:2    0 444,2G  0 part
> | `-cvg             254:3    0 444,2G  0 crypt
> |   |-cvg-root      254:4    0    40G  0 lvm   /
> |   |-cvg-swap      254:5    0     4G  0 lvm   [SWAP]
> |   |-cvg-log       254:6    0     1G  0 lvm   /var/log
> |   `-cvg-home      254:8    0   320G  0 lvm   /home
> |-sda3                8:3    0    32G  0 part
> `-sda128            259:0    0     1M  0 part
> 
> Creating a snapshot succeeds, but it is broken and can not be mounted:
> 
> # lvcreate -s -pr -l50%free -n snap-home cvg/home
>   Logical volume "snap-home" created
> # mount /dev/cvg/snap-home /mnt/tmp 
> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
> mount: wrong fs type, bad option, bad superblock on
> /dev/mapper/cvg-snap--home, missing codepage or helper program, or other
> error
> 
> Syslog has a lot of these messages:
> 
> [ 4823.002220] EXT4-fs (dm-7): ext4_check_descriptors: Checksum for group
> 256 failed (43470!=57954)
> 
> Downgrading to lvm2 2.02.104 fixes the problem:
> 
> # lvcreate -s -pr -l50%free -n snap-home
> cvg/home Logical volume "snap-home" created
> # mount /dev/cvg/snap-home /mnt/tmp
> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
> 
> This is an Arch Linux system with Linux 3.12.8.

Hello everybody,

did anybody notice my mail? This is a real regression for me and I would like
to get this sorted. I will help with whatever is needed to find the problem.
-- 
main(a){char*c=/*    Best regards,                           */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] lvm2 2.02.105 breaks snapshots

[Zdenek Kabelac]

Dne 4.2.2014 09:55, Christian Hesse napsal(a):
> Christian Hesse <list@eworm.de> on Thu, 2014/01/23 14:27:
>> Hello everybody,
>>
>> looks like lvm2 2.02.105 breaks snapshots. This is my block device tree:
>>
>> NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
>> sda                   8:0    0   477G  0 disk
>> |-sda1                8:1    0   767M  0 part
>> | `-vg0-boot        254:0    0    64M  0 lvm   /boot
>> |-sda2                8:2    0 444,2G  0 part
>> | `-cvg             254:3    0 444,2G  0 crypt
>> |   |-cvg-root      254:4    0    40G  0 lvm   /
>> |   |-cvg-swap      254:5    0     4G  0 lvm   [SWAP]
>> |   |-cvg-log       254:6    0     1G  0 lvm   /var/log
>> |   `-cvg-home      254:8    0   320G  0 lvm   /home
>> |-sda3                8:3    0    32G  0 part
>> `-sda128            259:0    0     1M  0 part
>>
>> Creating a snapshot succeeds, but it is broken and can not be mounted:
>>
>> # lvcreate -s -pr -l50%free -n snap-home cvg/home
>>    Logical volume "snap-home" created
>> # mount /dev/cvg/snap-home /mnt/tmp
>> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
>> mount: wrong fs type, bad option, bad superblock on
>> /dev/mapper/cvg-snap--home, missing codepage or helper program, or other
>> error
>>
>> Syslog has a lot of these messages:
>>
>> [ 4823.002220] EXT4-fs (dm-7): ext4_check_descriptors: Checksum for group
>> 256 failed (43470!=57954)
>>
>> Downgrading to lvm2 2.02.104 fixes the problem:
>>
>> # lvcreate -s -pr -l50%free -n snap-home
>> cvg/home Logical volume "snap-home" created
>> # mount /dev/cvg/snap-home /mnt/tmp
>> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
>>
>> This is an Arch Linux system with Linux 3.12.8.
>
> Hello everybody,
>
> did anybody notice my mail? This is a real regression for me and I would like
> to get this sorted. I will help with whatever is needed to find the problem.
>


Ok -  could you test this:

Create an lv  -  ('lvcreate -Lsmallsize  vg'
mount this lv somewhere
modify this filesystem (via dd)

'fsfreeze --freeze  mountpoint'

now  - copy whole frozen device somewhere (via dd)

'fsfreeze --unfreeze mountpoint'

umount mountpoint

losetup -r  /dev/loop_free_number    frozen_copy_of_device

and now - try to mount your  read-only loop device copy.

Does it work for you ?

I've been testing this - and it seem fsfreeze API in kernel is not working 
properly and you need to reply journal.

Then try to repeat the same with 3.10 kernel and 3.4 kernel.

So far I'm not convinced lvm2 version has anything to do with this problem

For 3.10  ext4 seems to work, but xfs is broken.

Regards

Zdenek

Re: [linux-lvm] lvm2 2.02.105 breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 3702 bytes --]

Zdenek Kabelac <zkabelac@redhat.com> on Tue, 2014/02/04 17:47:
> Dne 4.2.2014 09:55, Christian Hesse napsal(a):
> > Christian Hesse <list@eworm.de> on Thu, 2014/01/23 14:27:
> >> Hello everybody,
> >>
> >> looks like lvm2 2.02.105 breaks snapshots. This is my block device tree:
> >>
> >> NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
> >> sda                   8:0    0   477G  0 disk
> >> |-sda1                8:1    0   767M  0 part
> >> | `-vg0-boot        254:0    0    64M  0 lvm   /boot
> >> |-sda2                8:2    0 444,2G  0 part
> >> | `-cvg             254:3    0 444,2G  0 crypt
> >> |   |-cvg-root      254:4    0    40G  0 lvm   /
> >> |   |-cvg-swap      254:5    0     4G  0 lvm   [SWAP]
> >> |   |-cvg-log       254:6    0     1G  0 lvm   /var/log
> >> |   `-cvg-home      254:8    0   320G  0 lvm   /home
> >> |-sda3                8:3    0    32G  0 part
> >> `-sda128            259:0    0     1M  0 part
> >>
> >> Creating a snapshot succeeds, but it is broken and can not be mounted:
> >>
> >> # lvcreate -s -pr -l50%free -n snap-home cvg/home
> >>    Logical volume "snap-home" created
> >> # mount /dev/cvg/snap-home /mnt/tmp
> >> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
> >> mount: wrong fs type, bad option, bad superblock on
> >> /dev/mapper/cvg-snap--home, missing codepage or helper program, or other
> >> error
> >>
> >> Syslog has a lot of these messages:
> >>
> >> [ 4823.002220] EXT4-fs (dm-7): ext4_check_descriptors: Checksum for group
> >> 256 failed (43470!=57954)
> >>
> >> Downgrading to lvm2 2.02.104 fixes the problem:
> >>
> >> # lvcreate -s -pr -l50%free -n snap-home
> >> cvg/home Logical volume "snap-home" created
> >> # mount /dev/cvg/snap-home /mnt/tmp
> >> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
> >>
> >> This is an Arch Linux system with Linux 3.12.8.
> >
> > Hello everybody,
> >
> > did anybody notice my mail? This is a real regression for me and I would
> > like to get this sorted. I will help with whatever is needed to find the
> > problem.
> >
> 
> 
> Ok -  could you test this:
> 
> Create an lv  -  ('lvcreate -Lsmallsize  vg'
> mount this lv somewhere
> modify this filesystem (via dd)

Do you want me to write to the block device here? I am not surprised if
anything break modifying a mounted filesystem.

> 'fsfreeze --freeze  mountpoint'
> 
> now  - copy whole frozen device somewhere (via dd)
> 
> 'fsfreeze --unfreeze mountpoint'
> 
> umount mountpoint
> 
> losetup -r  /dev/loop_free_number    frozen_copy_of_device
> 
> and now - try to mount your  read-only loop device copy.
> 
> Does it work for you ?
> 
> I've been testing this - and it seem fsfreeze API in kernel is not working 
> properly and you need to reply journal.
> 
> Then try to repeat the same with 3.10 kernel and 3.4 kernel.
> 
> So far I'm not convinced lvm2 version has anything to do with this problem
> 
> For 3.10  ext4 seems to work, but xfs is broken.

As said this is perfectly stable with lvm2 2.02.104. It breaks reliable with
lvm2 2.02.105. So I am sure lvm userspace tools are at least involved.

Looks like even the parent volume gets corrupted. My GnuPG keyring got
corrupted after creating a snapshot (with lvm2 2.02.105).

Just wondering... Is this mailing list just about the kernel part of lvm or
does it cover userspace as well?

-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] lvm2 2.02.105 breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 5521 bytes --]

Zdenek Kabelac <zkabelac@redhat.com> on Tue, 2014/02/04 17:47:
> Dne 4.2.2014 09:55, Christian Hesse napsal(a):
> > Christian Hesse <list@eworm.de> on Thu, 2014/01/23 14:27:
> >> Hello everybody,
> >>
> >> looks like lvm2 2.02.105 breaks snapshots. This is my block device tree:
> >>
> >> NAME                MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
> >> sda                   8:0    0   477G  0 disk
> >> |-sda1                8:1    0   767M  0 part
> >> | `-vg0-boot        254:0    0    64M  0 lvm   /boot
> >> |-sda2                8:2    0 444,2G  0 part
> >> | `-cvg             254:3    0 444,2G  0 crypt
> >> |   |-cvg-root      254:4    0    40G  0 lvm   /
> >> |   |-cvg-swap      254:5    0     4G  0 lvm   [SWAP]
> >> |   |-cvg-log       254:6    0     1G  0 lvm   /var/log
> >> |   `-cvg-home      254:8    0   320G  0 lvm   /home
> >> |-sda3                8:3    0    32G  0 part
> >> `-sda128            259:0    0     1M  0 part
> >>
> >> Creating a snapshot succeeds, but it is broken and can not be mounted:
> >>
> >> # lvcreate -s -pr -l50%free -n snap-home cvg/home
> >>    Logical volume "snap-home" created
> >> # mount /dev/cvg/snap-home /mnt/tmp
> >> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
> >> mount: wrong fs type, bad option, bad superblock on
> >> /dev/mapper/cvg-snap--home, missing codepage or helper program, or other
> >> error
> >>
> >> Syslog has a lot of these messages:
> >>
> >> [ 4823.002220] EXT4-fs (dm-7): ext4_check_descriptors: Checksum for group
> >> 256 failed (43470!=57954)
> >>
> >> Downgrading to lvm2 2.02.104 fixes the problem:
> >>
> >> # lvcreate -s -pr -l50%free -n snap-home
> >> cvg/home Logical volume "snap-home" created
> >> # mount /dev/cvg/snap-home /mnt/tmp
> >> mount: /dev/mapper/cvg-snap--home is write-protected, mounting read-only
> >>
> >> This is an Arch Linux system with Linux 3.12.8.
> >
> > Hello everybody,
> >
> > did anybody notice my mail? This is a real regression for me and I would
> > like to get this sorted. I will help with whatever is needed to find the
> > problem.
> 
> Ok -  could you test this:
> 
> Create an lv  -  ('lvcreate -Lsmallsize  vg'
> mount this lv somewhere
> modify this filesystem (via dd)
> 
> 'fsfreeze --freeze  mountpoint'
> 
> now  - copy whole frozen device somewhere (via dd)
> 
> 'fsfreeze --unfreeze mountpoint'
> 
> umount mountpoint
> 
> losetup -r  /dev/loop_free_number    frozen_copy_of_device
> 
> and now - try to mount your  read-only loop device copy.
> 
> Does it work for you ?
> 
> I've been testing this - and it seem fsfreeze API in kernel is not working 
> properly and you need to reply journal.
> 
> Then try to repeat the same with 3.10 kernel and 3.4 kernel.
> 
> So far I'm not convinced lvm2 version has anything to do with this problem
> 
> For 3.10  ext4 seems to work, but xfs is broken.

Ok, here we go:

root@leda ~ # lvcreate -n test -L 2G cvg
  Logical volume "test" created
root@leda ~ # mkfs.ext4 -L test /dev/cvg/test
mke2fs 1.42.9 (28-Dec-2013)
Discarding device blocks: done
Filesystem label=test
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
131072 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=536870912
16 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

root # mount /dev/cvg/test /mnt/ext
root # dd if=/dev/urandom bs=1M count=1500 of=/mnt/ext/testfile
1500+0 records in
1500+0 records out
1572864000 bytes (1,6 GB) copied, 116,697 s, 13,5 MB/s
root # fsfreeze --freeze /mnt/ext
root # dd if=/dev/cvg/test of=/tmp/test.img
4194304+0 records in
4194304+0 records out
2147483648 bytes (2,1 GB) copied, 6,15333 s, 349 MB/s
root # fsfreeze --unfreeze /mnt/ext
root # umount /mnt/ext
root # lvremove cvg/test
Do you really want to remove active logical volume test? [y/n]: y
  Logical volume "test" successfully removed
root # file -s /tmp/test.img
/tmp/test.img: Linux rev 1.0 ext4 filesystem data,
UUID=18e5dba2-6d04-4d8f-b448-31bbcfbc1462, volume name "test" (extents)
(large files) (huge files)
root # losetup -f /tmp/test.img
root # mount /dev/loop0 /mnt/ext
root # losetup -f /tmp/test.img
root # mount /dev/loop0 /mnt/ext
root # umount /mnt/ext
root # losetup -d /dev/loop0

Mounting the dumped filesystem image just gives:

kernel: EXT4-fs (loop0): mounted filesystem with ordered data mode. Opts:
(null)

Please note that file does report a clean filesystem, there is no "(needs
journal recovery)" flag set.

And just another hint... If the LVM problem occurs I can not mount the
snapshot at all. Freezer problems should not be that bad, no? (e.g. a journal
replay could be required, but it should not end up with a completely broken
filesystem.)

BTW, I found a second report with exactly this problem:
https://bugs.archlinux.org/task/38710
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

[linux-lvm] blkid_wiping breaks snapshots (was: lvm2 2.02.105 breaks snapshots)

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

Hello everybody,

I think I nailed it down with git bisect. My first bad commit is:

75628f341ad38b68aae33eae0b5700be2a6e5769
configure: enable blkid_wiping by default if the blkid library is present

Looks like this wipes data that is still needed... Building a package with
'--disable-blkid_wipe' now to verify on another system.
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] blkid_wiping breaks snapshots (was: lvm2 2.02.105 breaks snapshots)

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]

Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
> Hello everybody,
> 
> I think I nailed it down with git bisect. My first bad commit is:
> 
> 75628f341ad38b68aae33eae0b5700be2a6e5769
> configure: enable blkid_wiping by default if the blkid library is present
> 
> Looks like this wipes data that is still needed... Building a package with
> '--disable-blkid_wipe' now to verify on another system.

Uh, this only helps part of...

I changed my test setup and used writable snapshots. After that I got:

WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
offset 0. Wipe it? [y/n]

(see https://bbs.archlinux.org/viewtopic.php?id=176504 for another report)

Looks like disabling blkid_wiping fixes this. My snapshot corruption still
occurs though. :-/
Bad thing about it is that the corruption does not occur reliable when doing
simple tests in 'git bisect'... Out of ideas for now - Will go to bed now.
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] blkid_wiping breaks snapshots

[Zdenek Kabelac]

Dne 8.2.2014 00:55, Christian Hesse napsal(a):
> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
>> Hello everybody,
>>
>> I think I nailed it down with git bisect. My first bad commit is:
>>
>> 75628f341ad38b68aae33eae0b5700be2a6e5769
>> configure: enable blkid_wiping by default if the blkid library is present
>>
>> Looks like this wipes data that is still needed... Building a package with
>> '--disable-blkid_wipe' now to verify on another system.
>
> Uh, this only helps part of...
>
> I changed my test setup and used writable snapshots. After that I got:
>
> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
> offset 0. Wipe it? [y/n]
>
> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another report)
>
> Looks like disabling blkid_wiping fixes this. My snapshot corruption still
> occurs though. :-/
> Bad thing about it is that the corruption does not occur reliable when doing
> simple tests in 'git bisect'... Out of ideas for now - Will go to bed now.

Now this was helpful, I guess I think what is going on.

Zdenek

Re: [linux-lvm] blkid_wiping breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
> > Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
> >> Hello everybody,
> >>
> >> I think I nailed it down with git bisect. My first bad commit is:
> >>
> >> 75628f341ad38b68aae33eae0b5700be2a6e5769
> >> configure: enable blkid_wiping by default if the blkid library is present
> >>
> >> Looks like this wipes data that is still needed... Building a package
> >> with '--disable-blkid_wipe' now to verify on another system.
> >
> > Uh, this only helps part of...
> >
> > I changed my test setup and used writable snapshots. After that I got:
> >
> > WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
> > offset 0. Wipe it? [y/n]
> >
> > (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another report)
> >
> > Looks like disabling blkid_wiping fixes this. My snapshot corruption still
> > occurs though. :-/
> > Bad thing about it is that the corruption does not occur reliable when
> > doing simple tests in 'git bisect'... Out of ideas for now - Will go to
> > bed now.
> 
> Now this was helpful, I guess I think what is going on.

Great! Waiting for patches then. :D

I could provide some more information when needed:

* This happens with read only snapshots only.
* I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
  (wiping: add support for blkid wiping), so it was introduced there or
  before.

Should I investigate further or wait for something to test from you?
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] blkid_wiping breaks snapshots

[Zdenek Kabelac]

Dne 10.2.2014 10:30, Christian Hesse napsal(a):
> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
>> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
>>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
>>>> Hello everybody,
>>>>
>>>> I think I nailed it down with git bisect. My first bad commit is:
>>>>
>>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
>>>> configure: enable blkid_wiping by default if the blkid library is present
>>>>
>>>> Looks like this wipes data that is still needed... Building a package
>>>> with '--disable-blkid_wipe' now to verify on another system.
>>>
>>> Uh, this only helps part of...
>>>
>>> I changed my test setup and used writable snapshots. After that I got:
>>>
>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
>>> offset 0. Wipe it? [y/n]
>>>
>>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another report)
>>>
>>> Looks like disabling blkid_wiping fixes this. My snapshot corruption still
>>> occurs though. :-/
>>> Bad thing about it is that the corruption does not occur reliable when
>>> doing simple tests in 'git bisect'... Out of ideas for now - Will go to
>>> bed now.
>>
>> Now this was helpful, I guess I think what is going on.
>
> Great! Waiting for patches then. :D
>
> I could provide some more information when needed:
>
> * This happens with read only snapshots only.
> * I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
>    (wiping: add support for blkid wiping), so it was introduced there or
>    before.
>
> Should I investigate further or wait for something to test from you?

Is the patch bellow fixing your problem ?
(It's still not final - but should help)

Zdenek


diff --git a/tools/lvcreate.c b/tools/lvcreate.c
index 638a868..e8b1a7f 100644
--- a/tools/lvcreate.c
+++ b/tools/lvcreate.c
@@ -772,7 +772,7 @@ static int _read_activation_params(struct lvcreate_params *lp,
                                         LVM_READ | LVM_WRITE);

         /* Must not zero/wipe read only volume */
-       if (!(lp->permission & LVM_WRITE)) {
+       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
                 lp->zero = 0;
                 lp->wipe_signatures = 0;
         }

Re: [linux-lvm] blkid_wiping breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 3058 bytes --]

Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
> Dne 10.2.2014 10:30, Christian Hesse napsal(a):
> > Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
> >> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
> >>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
> >>>> Hello everybody,
> >>>>
> >>>> I think I nailed it down with git bisect. My first bad commit is:
> >>>>
> >>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
> >>>> configure: enable blkid_wiping by default if the blkid library is
> >>>> present
> >>>>
> >>>> Looks like this wipes data that is still needed... Building a package
> >>>> with '--disable-blkid_wipe' now to verify on another system.
> >>>
> >>> Uh, this only helps part of...
> >>>
> >>> I changed my test setup and used writable snapshots. After that I got:
> >>>
> >>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
> >>> offset 0. Wipe it? [y/n]
> >>>
> >>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
> >>> report)
> >>>
> >>> Looks like disabling blkid_wiping fixes this. My snapshot corruption
> >>> still occurs though. :-/
> >>> Bad thing about it is that the corruption does not occur reliable when
> >>> doing simple tests in 'git bisect'... Out of ideas for now - Will go to
> >>> bed now.
> >>
> >> Now this was helpful, I guess I think what is going on.
> >
> > Great! Waiting for patches then. :D
> >
> > I could provide some more information when needed:
> >
> > * This happens with read only snapshots only.
> > * I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
> >    (wiping: add support for blkid wiping), so it was introduced there or
> >    before.
> >
> > Should I investigate further or wait for something to test from you?
> 
> Is the patch bellow fixing your problem ?
> (It's still not final - but should help)
> 
> Zdenek
> 
> 
> diff --git a/tools/lvcreate.c b/tools/lvcreate.c
> index 638a868..e8b1a7f 100644
> --- a/tools/lvcreate.c
> +++ b/tools/lvcreate.c
> @@ -772,7 +772,7 @@ static int _read_activation_params(struct
> lvcreate_params *lp, LVM_READ | LVM_WRITE);
> 
>          /* Must not zero/wipe read only volume */
> -       if (!(lp->permission & LVM_WRITE)) {
> +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
>                  lp->zero = 0;
>                  lp->wipe_signatures = 0;
>          }

This looks good to me. I've made some backups now without any corruption.

But with blkid_wiping enabled I do have this warning/question now for read
only snapshots as well:

WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home
at offset 0. Wipe it? [y/n]

lvm works perfectly only with your patch applied and compiled with
--disable-blkid_wiping.
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] blkid_wiping breaks snapshots

[Peter Rajnoha]

On 02/10/2014 02:48 PM, Christian Hesse wrote:
> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
>> Dne 10.2.2014 10:30, Christian Hesse napsal(a):
>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
>>>> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
>>>>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
>>>>>> Hello everybody,
>>>>>>
>>>>>> I think I nailed it down with git bisect. My first bad commit is:
>>>>>>
>>>>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
>>>>>> configure: enable blkid_wiping by default if the blkid library is
>>>>>> present
>>>>>>
>>>>>> Looks like this wipes data that is still needed... Building a package
>>>>>> with '--disable-blkid_wipe' now to verify on another system.
>>>>>
>>>>> Uh, this only helps part of...
>>>>>
>>>>> I changed my test setup and used writable snapshots. After that I got:
>>>>>
>>>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
>>>>> offset 0. Wipe it? [y/n]
>>>>>
>>>>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
>>>>> report)
>>>>>
>>>>> Looks like disabling blkid_wiping fixes this. My snapshot corruption
>>>>> still occurs though. :-/
>>>>> Bad thing about it is that the corruption does not occur reliable when
>>>>> doing simple tests in 'git bisect'... Out of ideas for now - Will go to
>>>>> bed now.
>>>>
>>>> Now this was helpful, I guess I think what is going on.
>>>
>>> Great! Waiting for patches then. :D
>>>
>>> I could provide some more information when needed:
>>>
>>> * This happens with read only snapshots only.
>>> * I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
>>>    (wiping: add support for blkid wiping), so it was introduced there or
>>>    before.
>>>
>>> Should I investigate further or wait for something to test from you?
>>
>> Is the patch bellow fixing your problem ?
>> (It's still not final - but should help)
>>
>> Zdenek
>>
>>
>> diff --git a/tools/lvcreate.c b/tools/lvcreate.c
>> index 638a868..e8b1a7f 100644
>> --- a/tools/lvcreate.c
>> +++ b/tools/lvcreate.c
>> @@ -772,7 +772,7 @@ static int _read_activation_params(struct
>> lvcreate_params *lp, LVM_READ | LVM_WRITE);
>>
>>          /* Must not zero/wipe read only volume */
>> -       if (!(lp->permission & LVM_WRITE)) {
>> +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
>>                  lp->zero = 0;
>>                  lp->wipe_signatures = 0;
>>          }
> 
> This looks good to me. I've made some backups now without any corruption.
> 
> But with blkid_wiping enabled I do have this warning/question now for read
> only snapshots as well:
> 
> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home
> at offset 0. Wipe it? [y/n]
> 
> lvm works perfectly only with your patch applied and compiled with
> --disable-blkid_wiping.

Also, try this patch in addition:

https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=ed166a3b1d3290ad887d8f83c24a8d8877713d3c

-- 
Peter

Re: [linux-lvm] blkid_wiping breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 3759 bytes --]

Peter Rajnoha <prajnoha@redhat.com> on Mon, 2014/02/10 15:12:
> On 02/10/2014 02:48 PM, Christian Hesse wrote:
> > Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
> >> Dne 10.2.2014 10:30, Christian Hesse napsal(a):
> >>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
> >>>> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
> >>>>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
> >>>>>> Hello everybody,
> >>>>>>
> >>>>>> I think I nailed it down with git bisect. My first bad commit is:
> >>>>>>
> >>>>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
> >>>>>> configure: enable blkid_wiping by default if the blkid library is
> >>>>>> present
> >>>>>>
> >>>>>> Looks like this wipes data that is still needed... Building a package
> >>>>>> with '--disable-blkid_wipe' now to verify on another system.
> >>>>>
> >>>>> Uh, this only helps part of...
> >>>>>
> >>>>> I changed my test setup and used writable snapshots. After that I got:
> >>>>>
> >>>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
> >>>>> offset 0. Wipe it? [y/n]
> >>>>>
> >>>>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
> >>>>> report)
> >>>>>
> >>>>> Looks like disabling blkid_wiping fixes this. My snapshot corruption
> >>>>> still occurs though. :-/
> >>>>> Bad thing about it is that the corruption does not occur reliable when
> >>>>> doing simple tests in 'git bisect'... Out of ideas for now - Will go
> >>>>> to bed now.
> >>>>
> >>>> Now this was helpful, I guess I think what is going on.
> >>>
> >>> Great! Waiting for patches then. :D
> >>>
> >>> I could provide some more information when needed:
> >>>
> >>> * This happens with read only snapshots only.
> >>> * I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
> >>>    (wiping: add support for blkid wiping), so it was introduced there or
> >>>    before.
> >>>
> >>> Should I investigate further or wait for something to test from you?
> >>
> >> Is the patch bellow fixing your problem ?
> >> (It's still not final - but should help)
> >>
> >> Zdenek
> >>
> >>
> >> diff --git a/tools/lvcreate.c b/tools/lvcreate.c
> >> index 638a868..e8b1a7f 100644
> >> --- a/tools/lvcreate.c
> >> +++ b/tools/lvcreate.c
> >> @@ -772,7 +772,7 @@ static int _read_activation_params(struct
> >> lvcreate_params *lp, LVM_READ | LVM_WRITE);
> >>
> >>          /* Must not zero/wipe read only volume */
> >> -       if (!(lp->permission & LVM_WRITE)) {
> >> +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
> >>                  lp->zero = 0;
> >>                  lp->wipe_signatures = 0;
> >>          }
> > 
> > This looks good to me. I've made some backups now without any corruption.
> > 
> > But with blkid_wiping enabled I do have this warning/question now for read
> > only snapshots as well:
> > 
> > WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home
> > at offset 0. Wipe it? [y/n]
> > 
> > lvm works perfectly only with your patch applied and compiled with
> > --disable-blkid_wiping.
> 
> Also, try this patch in addition:
> 
> https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=ed166a3b1d3290ad887d8f83c24a8d8877713d3c

That did the trick. Thanks a lot!

If anybody is interested... These are the patches updated to apply against
version 2.02.105:

http://www.eworm.de/download/linux/lvm2-snapshot.patch
http://www.eworm.de/download/linux/lvm2-snapshot-wiping.patch
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] blkid_wiping breaks snapshots

[Zdenek Kabelac]

Dne 10.2.2014 15:37, Christian Hesse napsal(a):
> Peter Rajnoha <prajnoha@redhat.com> on Mon, 2014/02/10 15:12:
>> On 02/10/2014 02:48 PM, Christian Hesse wrote:
>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
>>>> Dne 10.2.2014 10:30, Christian Hesse napsal(a):
>>>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
>>>>>> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
>>>>>>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
>>>>>>>> Hello everybody,
>>>>>>>>
>>>>>>>> I think I nailed it down with git bisect. My first bad commit is:
>>>>>>>>
>>>>>>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
>>>>>>>> configure: enable blkid_wiping by default if the blkid library is
>>>>>>>> present
>>>>>>>>
>>>>>>>> Looks like this wipes data that is still needed... Building a package
>>>>>>>> with '--disable-blkid_wipe' now to verify on another system.
>>>>>>>
>>>>>>> Uh, this only helps part of...
>>>>>>>
>>>>>>> I changed my test setup and used writable snapshots. After that I got:
>>>>>>>
>>>>>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
>>>>>>> offset 0. Wipe it? [y/n]
>>>>>>>
>>>>>>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
>>>>>>> report)
>>>>>>>
>>>>>>> Looks like disabling blkid_wiping fixes this. My snapshot corruption
>>>>>>> still occurs though. :-/
>>>>>>> Bad thing about it is that the corruption does not occur reliable when
>>>>>>> doing simple tests in 'git bisect'... Out of ideas for now - Will go
>>>>>>> to bed now.
>>>>>>
>>>>>> Now this was helpful, I guess I think what is going on.
>>>>>
>>>>> Great! Waiting for patches then. :D
>>>>>
>>>>> I could provide some more information when needed:
>>>>>
>>>>> * This happens with read only snapshots only.
>>>>> * I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
>>>>>     (wiping: add support for blkid wiping), so it was introduced there or
>>>>>     before.
>>>>>
>>>>> Should I investigate further or wait for something to test from you?
>>>>
>>>> Is the patch bellow fixing your problem ?
>>>> (It's still not final - but should help)
>>>>
>>>> Zdenek
>>>>
>>>>
>>>> diff --git a/tools/lvcreate.c b/tools/lvcreate.c
>>>> index 638a868..e8b1a7f 100644
>>>> --- a/tools/lvcreate.c
>>>> +++ b/tools/lvcreate.c
>>>> @@ -772,7 +772,7 @@ static int _read_activation_params(struct
>>>> lvcreate_params *lp, LVM_READ | LVM_WRITE);
>>>>
>>>>           /* Must not zero/wipe read only volume */
>>>> -       if (!(lp->permission & LVM_WRITE)) {
>>>> +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
>>>>                   lp->zero = 0;
>>>>                   lp->wipe_signatures = 0;
>>>>           }
>>>
>>> This looks good to me. I've made some backups now without any corruption.
>>>
>>> But with blkid_wiping enabled I do have this warning/question now for read
>>> only snapshots as well:
>>>
>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home
>>> at offset 0. Wipe it? [y/n]
>>>
>>> lvm works perfectly only with your patch applied and compiled with
>>> --disable-blkid_wiping.
>>
>> Also, try this patch in addition:
>>
>> https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=ed166a3b1d3290ad887d8f83c24a8d8877713d3c
>
> That did the trick. Thanks a lot!
>
> If anybody is interested... These are the patches updated to apply against
> version 2.02.105:
>
> http://www.eworm.de/download/linux/lvm2-snapshot.patch
> http://www.eworm.de/download/linux/lvm2-snapshot-wiping.patch
>


We need to resolve the logic behind the query for wiping. We will most 
probably go with this logic:

We query for wipe of 'known' signatures, before passing newly allocated 
'normal' LV (visible).
Any other so called 'private/hidden' LV will do either wipe or unconditional 
zeroing of 1st. 4kb - which is usually the only thing needed here.

Current logic is somewhat overcomplicated.

Zdenek

Re: [linux-lvm] blkid_wiping breaks snapshots

[Peter Rajnoha]

On 02/10/2014 04:56 PM, Zdenek Kabelac wrote:
> Dne 10.2.2014 15:37, Christian Hesse napsal(a):
>> Peter Rajnoha <prajnoha@redhat.com> on Mon, 2014/02/10 15:12:
>>> On 02/10/2014 02:48 PM, Christian Hesse wrote:
>>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
>>>>> Dne 10.2.2014 10:30, Christian Hesse napsal(a):
>>>>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
>>>>>>> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
>>>>>>>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
>>>>>>>>> Hello everybody,
>>>>>>>>>
>>>>>>>>> I think I nailed it down with git bisect. My first bad commit is:
>>>>>>>>>
>>>>>>>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
>>>>>>>>> configure: enable blkid_wiping by default if the blkid library is
>>>>>>>>> present
>>>>>>>>>
>>>>>>>>> Looks like this wipes data that is still needed... Building a
>>>>>>>>> package
>>>>>>>>> with '--disable-blkid_wipe' now to verify on another system.
>>>>>>>>
>>>>>>>> Uh, this only helps part of...
>>>>>>>>
>>>>>>>> I changed my test setup and used writable snapshots. After that
>>>>>>>> I got:
>>>>>>>>
>>>>>>>> WARNING: DM_snapshot_cow signature detected on
>>>>>>>> /dev/cvg/snap-home at
>>>>>>>> offset 0. Wipe it? [y/n]
>>>>>>>>
>>>>>>>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
>>>>>>>> report)
>>>>>>>>
>>>>>>>> Looks like disabling blkid_wiping fixes this. My snapshot
>>>>>>>> corruption
>>>>>>>> still occurs though. :-/
>>>>>>>> Bad thing about it is that the corruption does not occur
>>>>>>>> reliable when
>>>>>>>> doing simple tests in 'git bisect'... Out of ideas for now -
>>>>>>>> Will go
>>>>>>>> to bed now.
>>>>>>>
>>>>>>> Now this was helpful, I guess I think what is going on.
>>>>>>
>>>>>> Great! Waiting for patches then. :D
>>>>>>
>>>>>> I could provide some more information when needed:
>>>>>>
>>>>>> * This happens with read only snapshots only.
>>>>>> * I could reproduce with commit
>>>>>> eaa23d32732c9bc3dd4f948781b5764cf21d84ba
>>>>>>     (wiping: add support for blkid wiping), so it was introduced
>>>>>> there or
>>>>>>     before.
>>>>>>
>>>>>> Should I investigate further or wait for something to test from you?
>>>>>
>>>>> Is the patch bellow fixing your problem ?
>>>>> (It's still not final - but should help)
>>>>>
>>>>> Zdenek
>>>>>
>>>>>
>>>>> diff --git a/tools/lvcreate.c b/tools/lvcreate.c
>>>>> index 638a868..e8b1a7f 100644
>>>>> --- a/tools/lvcreate.c
>>>>> +++ b/tools/lvcreate.c
>>>>> @@ -772,7 +772,7 @@ static int _read_activation_params(struct
>>>>> lvcreate_params *lp, LVM_READ | LVM_WRITE);
>>>>>
>>>>>           /* Must not zero/wipe read only volume */
>>>>> -       if (!(lp->permission & LVM_WRITE)) {
>>>>> +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
>>>>>                   lp->zero = 0;
>>>>>                   lp->wipe_signatures = 0;
>>>>>           }
>>>>
>>>> This looks good to me. I've made some backups now without any
>>>> corruption.
>>>>
>>>> But with blkid_wiping enabled I do have this warning/question now
>>>> for read
>>>> only snapshots as well:
>>>>
>>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home
>>>> at offset 0. Wipe it? [y/n]
>>>>
>>>> lvm works perfectly only with your patch applied and compiled with
>>>> --disable-blkid_wiping.
>>>
>>> Also, try this patch in addition:
>>>
>>> https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=ed166a3b1d3290ad887d8f83c24a8d8877713d3c
>>>
>>
>> That did the trick. Thanks a lot!
>>
>> If anybody is interested... These are the patches updated to apply
>> against
>> version 2.02.105:
>>
>> http://www.eworm.de/download/linux/lvm2-snapshot.patch
>> http://www.eworm.de/download/linux/lvm2-snapshot-wiping.patch
>>
> 
> 
> We need to resolve the logic behind the query for wiping. We will most
> probably go with this logic:
> 
> We query for wipe of 'known' signatures, before passing newly allocated
> 'normal' LV (visible).
> Any other so called 'private/hidden' LV will do either wipe or
> unconditional zeroing of 1st. 4kb - which is usually the only thing
> needed here.
> 

MD can write its signature beyond 4K. Just an example...

> Current logic is somewhat overcomplicated.

Current logic is simple - we do the wiping first (not asking about
any special LV signatures that we directly produce - for now its only
the DM_snapshot_cow that libblkid recognizes).

And after wiping, zeroing happens.

Simple as that, nothing overcomplicated actually...

-- 
Peter

Re: [linux-lvm] blkid_wiping breaks snapshots

[Zdenek Kabelac]

Dne 10.2.2014 19:15, Peter Rajnoha napsal(a):
> On 02/10/2014 04:56 PM, Zdenek Kabelac wrote:
>> Dne 10.2.2014 15:37, Christian Hesse napsal(a):
>>> Peter Rajnoha <prajnoha@redhat.com> on Mon, 2014/02/10 15:12:
>>>> On 02/10/2014 02:48 PM, Christian Hesse wrote:
>>>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
>>>>>> Dne 10.2.2014 10:30, Christian Hesse napsal(a):
>>>>>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
>>>>>>>> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
>>>>>>>>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
>>>>>>>>>> Hello everybody,
>>>>>>>>>>
>>>>>>>>>> I think I nailed it down with git bisect. My first bad commit is:
>>>>>>>>>>
>>>>>>>>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
>>>>>>>>>> configure: enable blkid_wiping by default if the blkid library is
>>>>>>>>>> present
>>>>>>>>>>
>>>>>>>>>> Looks like this wipes data that is still needed... Building a
>>>>>>>>>> package
>>>>>>>>>> with '--disable-blkid_wipe' now to verify on another system.
>>>>>>>>>
>>>>>>>>> Uh, this only helps part of...
>>>>>>>>>
>>>>>>>>> I changed my test setup and used writable snapshots. After that
>>>>>>>>> I got:
>>>>>>>>>
>>>>>>>>> WARNING: DM_snapshot_cow signature detected on
>>>>>>>>> /dev/cvg/snap-home at
>>>>>>>>> offset 0. Wipe it? [y/n]
>>>>>>>>>
>>>>>>>>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
>>>>>>>>> report)
>>>>>>>>>
>>>>>>>>> Looks like disabling blkid_wiping fixes this. My snapshot
>>>>>>>>> corruption
>>>>>>>>> still occurs though. :-/
>>>>>>>>> Bad thing about it is that the corruption does not occur
>>>>>>>>> reliable when
>>>>>>>>> doing simple tests in 'git bisect'... Out of ideas for now -
>>>>>>>>> Will go
>>>>>>>>> to bed now.
>>>>>>>>
>>>>>>>> Now this was helpful, I guess I think what is going on.
>>>>>>>
>>>>>>> Great! Waiting for patches then. :D
>>>>>>>
>>>>>>> I could provide some more information when needed:
>>>>>>>
>>>>>>> * This happens with read only snapshots only.
>>>>>>> * I could reproduce with commit
>>>>>>> eaa23d32732c9bc3dd4f948781b5764cf21d84ba
>>>>>>>      (wiping: add support for blkid wiping), so it was introduced
>>>>>>> there or
>>>>>>>      before.
>>>>>>>
>>>>>>> Should I investigate further or wait for something to test from you?
>>>>>>
>>>>>> Is the patch bellow fixing your problem ?
>>>>>> (It's still not final - but should help)
>>>>>>
>>>>>> Zdenek
>>>>>>
>>>>>>
>>>>>> diff --git a/tools/lvcreate.c b/tools/lvcreate.c
>>>>>> index 638a868..e8b1a7f 100644
>>>>>> --- a/tools/lvcreate.c
>>>>>> +++ b/tools/lvcreate.c
>>>>>> @@ -772,7 +772,7 @@ static int _read_activation_params(struct
>>>>>> lvcreate_params *lp, LVM_READ | LVM_WRITE);
>>>>>>
>>>>>>            /* Must not zero/wipe read only volume */
>>>>>> -       if (!(lp->permission & LVM_WRITE)) {
>>>>>> +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
>>>>>>                    lp->zero = 0;
>>>>>>                    lp->wipe_signatures = 0;
>>>>>>            }
>>>>>
>>>>> This looks good to me. I've made some backups now without any
>>>>> corruption.
>>>>>
>>>>> But with blkid_wiping enabled I do have this warning/question now
>>>>> for read
>>>>> only snapshots as well:
>>>>>
>>>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home
>>>>> at offset 0. Wipe it? [y/n]
>>>>>
>>>>> lvm works perfectly only with your patch applied and compiled with
>>>>> --disable-blkid_wiping.
>>>>
>>>> Also, try this patch in addition:
>>>>
>>>> https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=ed166a3b1d3290ad887d8f83c24a8d8877713d3c
>>>>
>>>
>>> That did the trick. Thanks a lot!
>>>
>>> If anybody is interested... These are the patches updated to apply
>>> against
>>> version 2.02.105:
>>>
>>> http://www.eworm.de/download/linux/lvm2-snapshot.patch
>>> http://www.eworm.de/download/linux/lvm2-snapshot-wiping.patch
>>>
>>
>>
>> We need to resolve the logic behind the query for wiping. We will most
>> probably go with this logic:
>>
>> We query for wipe of 'known' signatures, before passing newly allocated
>> 'normal' LV (visible).
>> Any other so called 'private/hidden' LV will do either wipe or
>> unconditional zeroing of 1st. 4kb - which is usually the only thing
>> needed here.
>>
>
> MD can write its signature beyond 4K. Just an example...

Which is exactly the thing which should not matter for case like '-cow'

Since this LV is 'private' and nobody should ever touch/use/read this device
except snapshot target.

By removing all signatures could actually mask bugs in i.e. udev rules,
where  md rules would read our own private device.

Zdenek.

Re: [linux-lvm] blkid_wiping breaks snapshots

[Christian Hesse]

[-- Attachment #1: Type: text/plain, Size: 3039 bytes --]

Christian Hesse <list@eworm.de> on Mon, 2014/02/10 14:48:
> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
> > Dne 10.2.2014 10:30, Christian Hesse napsal(a):
> > > Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
> > >> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
> > >>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
> > >>>> Hello everybody,
> > >>>>
> > >>>> I think I nailed it down with git bisect. My first bad commit is:
> > >>>>
> > >>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
> > >>>> configure: enable blkid_wiping by default if the blkid library is
> > >>>> present
> > >>>>
> > >>>> Looks like this wipes data that is still needed... Building a package
> > >>>> with '--disable-blkid_wipe' now to verify on another system.
> > >>>
> > >>> Uh, this only helps part of...
> > >>>
> > >>> I changed my test setup and used writable snapshots. After that I got:
> > >>>
> > >>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
> > >>> offset 0. Wipe it? [y/n]
> > >>>
> > >>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
> > >>> report)
> > >>>
> > >>> Looks like disabling blkid_wiping fixes this. My snapshot corruption
> > >>> still occurs though. :-/
> > >>> Bad thing about it is that the corruption does not occur reliable when
> > >>> doing simple tests in 'git bisect'... Out of ideas for now - Will go
> > >>> to bed now.
> > >>
> > >> Now this was helpful, I guess I think what is going on.
> > >
> > > Great! Waiting for patches then. :D
> > >
> > > I could provide some more information when needed:
> > >
> > > * This happens with read only snapshots only.
> > > * I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
> > >    (wiping: add support for blkid wiping), so it was introduced there or
> > >    before.
> > >
> > > Should I investigate further or wait for something to test from you?
> > 
> > Is the patch bellow fixing your problem ?
> > (It's still not final - but should help)
> > 
> > Zdenek
> > 
> > 
> > diff --git a/tools/lvcreate.c b/tools/lvcreate.c
> > index 638a868..e8b1a7f 100644
> > --- a/tools/lvcreate.c
> > +++ b/tools/lvcreate.c
> > @@ -772,7 +772,7 @@ static int _read_activation_params(struct
> > lvcreate_params *lp, LVM_READ | LVM_WRITE);
> > 
> >          /* Must not zero/wipe read only volume */
> > -       if (!(lp->permission & LVM_WRITE)) {
> > +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
> >                  lp->zero = 0;
> >                  lp->wipe_signatures = 0;
> >          }
> 
> This looks good to me. I've made some backups now without any corruption.

I have not seen anything show up in git master. Or did I miss the final
solution?
-- 
main(a){char*c=/*    Schoene Gruesse                         */"B?IJj;MEH"
"CX:;",b;for(a/*    Chris           get my mail address:    */=0;b=c[a++];)
putchar(b-1/(/*               gcc -o sig sig.c && ./sig    */b/42*2-3)*42);}

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 490 bytes --]

Re: [linux-lvm] blkid_wiping breaks snapshots

[Zdenek Kabelac]

Dne 18.2.2014 09:15, Christian Hesse napsal(a):
> Christian Hesse <list@eworm.de> on Mon, 2014/02/10 14:48:
>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 13:20:
>>> Dne 10.2.2014 10:30, Christian Hesse napsal(a):
>>>> Zdenek Kabelac <zkabelac@redhat.com> on Mon, 2014/02/10 09:33:
>>>>> Dne 8.2.2014 00:55, Christian Hesse napsal(a):
>>>>>> Christian Hesse <list@eworm.de> on Sat, 2014/02/08 00:36:
>>>>>>> Hello everybody,
>>>>>>>
>>>>>>> I think I nailed it down with git bisect. My first bad commit is:
>>>>>>>
>>>>>>> 75628f341ad38b68aae33eae0b5700be2a6e5769
>>>>>>> configure: enable blkid_wiping by default if the blkid library is
>>>>>>> present
>>>>>>>
>>>>>>> Looks like this wipes data that is still needed... Building a package
>>>>>>> with '--disable-blkid_wipe' now to verify on another system.
>>>>>>
>>>>>> Uh, this only helps part of...
>>>>>>
>>>>>> I changed my test setup and used writable snapshots. After that I got:
>>>>>>
>>>>>> WARNING: DM_snapshot_cow signature detected on /dev/cvg/snap-home at
>>>>>> offset 0. Wipe it? [y/n]
>>>>>>
>>>>>> (see https://bbs.archlinux.org/viewtopic.php?id=176504 for another
>>>>>> report)
>>>>>>
>>>>>> Looks like disabling blkid_wiping fixes this. My snapshot corruption
>>>>>> still occurs though. :-/
>>>>>> Bad thing about it is that the corruption does not occur reliable when
>>>>>> doing simple tests in 'git bisect'... Out of ideas for now - Will go
>>>>>> to bed now.
>>>>>
>>>>> Now this was helpful, I guess I think what is going on.
>>>>
>>>> Great! Waiting for patches then. :D
>>>>
>>>> I could provide some more information when needed:
>>>>
>>>> * This happens with read only snapshots only.
>>>> * I could reproduce with commit eaa23d32732c9bc3dd4f948781b5764cf21d84ba
>>>>     (wiping: add support for blkid wiping), so it was introduced there or
>>>>     before.
>>>>
>>>> Should I investigate further or wait for something to test from you?
>>>
>>> Is the patch bellow fixing your problem ?
>>> (It's still not final - but should help)
>>>
>>> Zdenek
>>>
>>>
>>> diff --git a/tools/lvcreate.c b/tools/lvcreate.c
>>> index 638a868..e8b1a7f 100644
>>> --- a/tools/lvcreate.c
>>> +++ b/tools/lvcreate.c
>>> @@ -772,7 +772,7 @@ static int _read_activation_params(struct
>>> lvcreate_params *lp, LVM_READ | LVM_WRITE);
>>>
>>>           /* Must not zero/wipe read only volume */
>>> -       if (!(lp->permission & LVM_WRITE)) {
>>> +       if (!lp->snapshot && !(lp->permission & LVM_WRITE)) {
>>>                   lp->zero = 0;
>>>                   lp->wipe_signatures = 0;
>>>           }
>>
>> This looks good to me. I've made some backups now without any corruption.
>
> I have not seen anything show up in git master. Or did I miss the final
> solution?

No yet committed upstream - there are more ongoing fixes in this area - I'll
commit them all together.

There will also fix for lvcreate -an

Zdenek