[dm-crypt] Testing TRIM with hdparm on dm-crypt

[dm-crypt] Testing TRIM with hdparm on dm-crypt

[Alexander Koch]

Hello everyone,

I'm trying to verify that TRIM is actually working on my dm-crypt (LUKS) 
encrypted LVM volume with the following layers:

   SSD (Crucial M4 128GB) -> LVM -> LUKS -> ext4

I followed the guide available at [1], which includes these steps:

  1) create a random-filled file, sync to disk
  2) fetch LBAs via 'hdparm --fibmap'
  3) pick one sector, raw-read contents with 'hdparm --read-sector'
  4) delete the file, sync to disk
  5) re-read the sector chosen above, verify it contains zeroes

Unfortunately I already get a zeroed sector in step three:

 > $ hdparm --read-sector 271984 /dev/sdd
 >
 > /dev/sdd:
 > reading sector 271984: succeeded
 > 0000 0000 0000 0000 0000 0000 0000 0000
 > 0000 0000 0000 0000 0000 0000 0000 0000
 > 0000 0000 0000 0000 0000 0000 0000 0000
 > 0000 0000 0000 0000 0000 0000 0000 0000
 > (...)


Is there any LBA-calculation I must do because of the LVM layer?


Regards,

lynix



[1] 
http://techgage.com/article/enabling_and_testing_ssd_trim_support_under_linux/2

Re: [dm-crypt] Testing TRIM with hdparm on dm-crypt

[Milan Broz]

On 11/09/2011 07:14 PM, Alexander Koch wrote:
> Hello everyone,
> 
> I'm trying to verify that TRIM is actually working on my dm-crypt (LUKS) 
> encrypted LVM volume with the following layers:
> 
>    SSD (Crucial M4 128GB) -> LVM -> LUKS -> ext4
> 
> I followed the guide available at [1], which includes these steps:
> 
>   1) create a random-filled file, sync to disk
>   2) fetch LBAs via 'hdparm --fibmap'
>   3) pick one sector, raw-read contents with 'hdparm --read-sector'
>   4) delete the file, sync to disk
>   5) re-read the sector chosen above, verify it contains zeroes
> 
> Unfortunately I already get a zeroed sector in step three:
> 
>  > $ hdparm --read-sector 271984 /dev/sdd
>  >
>  > /dev/sdd:
>  > reading sector 271984: succeeded
>  > 0000 0000 0000 0000 0000 0000 0000 0000
>  > 0000 0000 0000 0000 0000 0000 0000 0000
>  > 0000 0000 0000 0000 0000 0000 0000 0000
>  > 0000 0000 0000 0000 0000 0000 0000 0000
>  > (...)
> 
> 
> Is there any LBA-calculation I must do because of the LVM layer?

If LVM provides just linear mapping of LV, it is about calculating
proper sector offset you need to add to calculated value here.

hdparm uses just top level device offset.

Also note you will NEVER get zeroes on top level device if dm-crypt
is involved. TRIM is bypassed to lower level device, so zeroes are
on hw, but dm-crypt interprets it as ciphertext, so you get "random"
noise on top level device (decrypted zeroes).

Maybe better an example - this is test config, ext4 on LUKS on LVM on sdb

# lsblk /dev/sdb -f
NAME                FSTYPE      LABEL MOUNTPOINT
sdb                 LVM2_member       
`-vg_test-lv (dm-0) crypto_LUKS       
  `-crypt_lv (dm-1) ext4              /mnt/tst

Here are the mapping tables:

# dmsetup table
crypt_lv: 0 405504 crypt aes-cbc-essiv:sha256 <key> 0 254:0 4096
vg_test-lv: 0 409600 linear 8:16 2048

So, crypt_lv has data offset 4096 sectors (LUKS metadata) and LV under
it has data offset 2048 sectors (LVM metadata).

So sector 0 of crypt_lv is mapped to sector 6144 (4096 + 2048) of /dev/sdb.
This is the offset you should use in calculations. Note this is always
in 512 bytes sectors but drive can use 4096 sectors.

So TRIM for sector 10 on /dev/mapper/crypt_lv should be forwarded
as TRIM for sector 6154 (6144 + 10) on /dev/sdb.

Moreover, if you have TRIM enabled, you should see "allow_discards"
in mapping table for LUKS. Otherwise TRIM is blocked.

You should be also able to trace discard (and other) block requests
using blktrace/btrace, this should be really more user friendly :)

Milan

Re: [dm-crypt] Testing TRIM with hdparm on dm-crypt

[Alexander Koch]

Hi Milan,

thanks for your detailed explanations :)

After adding the offsets for LUKS and LVM I realized that I have another 
partition residing logically "in front" of the PV:

$ lsblk /dev/sdd -f
NAME                    FSTYPE      LABEL      MOUNTPOINT
sdd
├─sdd1                  ext2        boot       /boot
└─sdd2                  LVM2_member
   ├─m4vg0-root (dm-0)   crypto_LUKS
   │ └─root-crypt (dm-4) ext4        root-crypt /
(..)

Adding its size in blocks finally resulted in the correct range, and I 
saw the random noise of the encrypted testfile disappear after synced 
`rm` invocation.


> You should be also able to trace discard (and other) block requests
> using blktrace/btrace, this should be really more user friendly :)

I tried blktrace but didn't know which event to track. Supported list 
includes READ, WRITE, BARRIER, SYNC, QUEUE, REQUEUE, ISSUE, COMPLETE, FS 
and PC.


Thanks again for your help! :)


Kind regards,

lynix

Re: [dm-crypt] Testing TRIM with hdparm on dm-crypt

[Milan Broz]

On 11/09/2011 09:25 PM, Alexander Koch wrote:
>> You should be also able to trace discard (and other) block requests
>> using blktrace/btrace, this should be really more user friendly :)
> 
> I tried blktrace but didn't know which event to track. Supported list 
> includes READ, WRITE, BARRIER, SYNC, QUEUE, REQUEUE, ISSUE, COMPLETE, FS 
> and PC.

man blkparse and search for "discard", it should be marked 'D'

something like here, here using simulated scsi_debug device

# modprobe scsi_debug dev_size_mb=16 sector_size=512 num_tgts=1 lbpu=1

# btrace /dev/<scsi_debug device> -a issue
...
  8,144  1       19     0.002461433  2146  D   D 0 + 32768 [mkfs.ext4]

# fstrim <mountpoint>
...
  8,144  1       52     8.372211281  2239  D   D 162 + 2 [fstrim]
  8,144  1       53     8.382095450  2239  D   D 168 + 28 [fstrim]
  8,144  1       54     8.391463509  2239  D   D 1222 + 15164 [fstrim]
  8,144  1       55     8.402445119  2239  D   D 18564 + 14204 [fstrim]

Milan