Re: [dm-crypt] Filesystem corruption with LVM's pvmove onto an encrypted volume with LUKS2 and a sector size of 4096

[Arno Wagner <arno@wagner.name>]

Hi,

LUKS should still be using 512 Byte sectors. Any mismatch
there should just impact performance,

I suspect you have an offset problem and the sector-numbers
(used as initialization for each secor's encryption) are 
shifted, expecially if this happens in plain mode as well.

Regards,
Arno

On Thu, Feb 21, 2019 at 13:03:51 CET, Ingo Franzki wrote:
> Hi,
> 
> we just encountered an error when using LVM's pvmove command to move the data from an un-encrypted LVM physical volume onto an encrypted volume. 
> After the pvmove has completed, the file system on the logical volume that resides on the moved physical volumes is corrupted.
> 
> It seems to be related to a sector size of 4096 used with LUKS2. Once I use the default sector size (512) then the problem does not happen.
> It happens with LUKS2 and even plain mode, as soon as a sector size of 4096 is used. LUKS1 and the default sector size does not show the problem. 
> 
> Not sure if this is a problem in dm-crypt or LVM, or a combination of both.
> 
> Here is how to reproduce (note the error messages on the very last command): 
> 
> # sudo dd if=/dev/zero of=loopbackfile1.img bs=500M count=1
> 1+0 records in
> 1+0 records out
> 524288000 bytes (524 MB, 500 MiB) copied, 2.32777 s, 225 MB/s
> 
> # sudo dd if=/dev/zero of=loopbackfile2.img bs=500M count=1
> 1+0 records in
> 1+0 records out
> 524288000 bytes (524 MB, 500 MiB) copied, 1.89992 s, 276 MB/s
> 
> # losetup -fP /root/loopbackfile1.img
> 
> # losetup -fP /root/loopbackfile2.img
> 
> # pvcreate /dev/loop0
>   Physical volume "/dev/loop0" successfully created.
> 
> # vgcreate LOOP_VG /dev/loop0
>   Volume group "LOOP_VG" successfully created
> 
> # lvcreate -L 300MB LOOP_VG -n LV /dev/loop0
>   Logical volume "LV" created.
> 
> # mkfs.ext4 /dev/mapper/LOOP_VG-LV
> mke2fs 1.44.1 (24-Mar-2018)
> Discarding device blocks: done
> Creating filesystem with 307200 1k blocks and 76912 inodes
> Filesystem UUID: 344289a3-e251-4d88-b03d-a71a4be2a8ec
> Superblock backups stored on blocks:
>         8193, 24577, 40961, 57345, 73729, 204801, 221185
> 
> Allocating group tables: done
> Writing inode tables: done
> Creating journal (8192 blocks): done
> Writing superblocks and filesystem accounting information: done
> 
> # mount /dev/mapper/LOOP_VG-LV /mnt
> 
> # cryptsetup luksFormat --type luks2 --sector-size 4096 /dev/loop1
> 
> WARNING!
> ========
> This will overwrite data on /dev/loop1 irrevocably.
> 
> Are you sure? (Type uppercase yes): YES
> Enter passphrase for /dev/loop1: loop
> Verify passphrase: loop
> 
> # cryptsetup luksOpen /dev/loop1 enc-loop
> Enter passphrase for /dev/loop1: loop
> 
> # pvcreate /dev/mapper/enc-loop
>   Physical volume "/dev/mapper/enc-loop" successfully created.
> 
> # vgextend LOOP_VG /dev/mapper/enc-loop
>   Volume group "LOOP_VG" successfully extended
> 
> # pvs
>   PV                   VG      Fmt  Attr PSize   PFree
>   /dev/loop0           LOOP_VG lvm2 a--  496.00m 196.00m
>   /dev/mapper/enc-loop LOOP_VG lvm2 a--  492.00m 492.00m
> 
> # pvmove /dev/loop0 /dev/mapper/enc-loop
>   /dev/loop0: Moved: 30.67%
>   /dev/loop0: Moved: 100.00%
> 
> # pvs
>   /dev/LOOP_VG/LV: read failed after 0 of 1024 at 0: Invalid argument
>   /dev/LOOP_VG/LV: read failed after 0 of 1024 at 314507264: Invalid argument
>   /dev/LOOP_VG/LV: read failed after 0 of 1024 at 314564608: Invalid argument
>   /dev/LOOP_VG/LV: read failed after 0 of 1024 at 4096: Invalid argument
>   PV                   VG      Fmt  Attr PSize   PFree
>   /dev/loop0           LOOP_VG lvm2 a--  496.00m 496.00m
>   /dev/mapper/enc-loop LOOP_VG lvm2 a--  492.00m 192.00m
> 
> In case the filesystem of the logical volume is not mounted at the time of pvmove, it gets corrupted anyway, but you only see errors when trying to mount it.
> 
> -- 
> Ingo Franzki
> eMail: ifranzki@linux.ibm.com  
> Tel: ++49 (0)7031-16-4648
> Fax: ++49 (0)7031-16-3456
> Linux on IBM Z Development, Schoenaicher Str. 220, 71032 Boeblingen, Germany
> 
> IBM Deutschland Research & Development GmbH / Vorsitzender des Aufsichtsrats: Matthias Hartmann
> Geschäftsführung: Dirk Wittkopp
> Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
> IBM DATA Privacy Statement: https://www.ibm.com/privacy/us/en/
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> https://www.saout.de/mailman/listinfo/dm-crypt

-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno@wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier