Incorrect NVMe DLFEAT?

Incorrect NVMe DLFEAT?

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

Hi,

I’m seeing SPDK test failures with QEMU NVMe controllers that I’ve bisected to QEMU commit 2605257a26 (“hw/block/nvme: add the dataset management command”).

The failing tests are related to write zeroes handling.  If an NVMe controller supports DSM, and DLFEAT indicates that deallocated blocks will read back as zeroes, then SPDK uses DEALLOCATE to implement the write zeroes operation.  (Note: SPDK prefers this method to using NVMe WRITE_ZEROES, since the latter is limited to a 16-bit block count.)

QEMU sets DLFEAT = 0x9 – and actually set it to 0x9 even before this commit.  Since the lower 3 bits are 0b001, it is reporting that deallocated blocks will read back later as 0.  This does not actually seem to be the case however – reading previously deallocated blocks do not actually return 0s.

It seems DLFEAT is being set incorrectly here – should probably be 0x8 instead?

Thanks,

Jim




[-- Attachment #2: Type: text/html, Size: 3364 bytes --]

Re: Incorrect NVMe DLFEAT?

[Klaus Jensen]

On Apr 29 16:51, Harris, James R wrote:
>Hi,
>

Hi Jim,

>I’m seeing SPDK test failures with QEMU NVMe controllers that I’ve 
>bisected to QEMU commit 2605257a26 (“hw/block/nvme: add the dataset 
>management command”).
>
>The failing tests are related to write zeroes handling.  If an NVMe 
>controller supports DSM, and DLFEAT indicates that deallocated blocks 
>will read back as zeroes, then SPDK uses DEALLOCATE to implement the 
>write zeroes operation.  (Note: SPDK prefers this method to using NVMe 
>WRITE_ZEROES, since the latter is limited to a 16-bit block count.)
>

The Dataset Management command is advisory, the controller gives no 
guarantee that it will actually deallocate anything. You cannot use DSM 
as a realiable alternative to Write Zeroes. The QEMU emulated nvme 
device exploits this and in many cases wont deallocate anything if it 
does not fit nicely with the underlying block device setup.

>QEMU sets DLFEAT = 0x9 – and actually set it to 0x9 even before this 
>commit.  Since the lower 3 bits are 0b001, it is reporting that 
>deallocated blocks will read back later as 0.  This does not actually 
>seem to be the case however – reading previously deallocated blocks do 
>not actually return 0s.
>

Can you share the configuration you use for QEMU? DSM works best with 
4096 byte LBAs (logical_block_size=4096) and the raw format driver.

Also, please test with the Error Recovery feature (and set DULBE) to 
test if the device reports that the block is actually deallocated.

>It seems DLFEAT is being set incorrectly here – should probably be 0x8 
>instead?
>
>Thanks,
>
>Jim
>
>
>

-- 
k

Re: Incorrect NVMe DLFEAT?

[Harris, James R]

On 4/29/21, 10:22 AM, "Klaus Jensen" <k.jensen@samsung.com> wrote:

    On Apr 29 16:51, Harris, James R wrote:
    >Hi,
    >

    Hi Jim,

    >I’m seeing SPDK test failures with QEMU NVMe controllers that I’ve 
    >bisected to QEMU commit 2605257a26 (“hw/block/nvme: add the dataset 
    >management command”).
    >
    >The failing tests are related to write zeroes handling.  If an NVMe 
    >controller supports DSM, and DLFEAT indicates that deallocated blocks 
    >will read back as zeroes, then SPDK uses DEALLOCATE to implement the 
    >write zeroes operation.  (Note: SPDK prefers this method to using NVMe 
    >WRITE_ZEROES, since the latter is limited to a 16-bit block count.)
    >

    The Dataset Management command is advisory, the controller gives no 
    guarantee that it will actually deallocate anything. You cannot use DSM 
    as a realiable alternative to Write Zeroes. The QEMU emulated nvme 
    device exploits this and in many cases wont deallocate anything if it 
    does not fit nicely with the underlying block device setup.

[JH] Hmmm, looks like we may need some SPDK changes for this case.

    >QEMU sets DLFEAT = 0x9 – and actually set it to 0x9 even before this 
    >commit.  Since the lower 3 bits are 0b001, it is reporting that 
    >deallocated blocks will read back later as 0.  This does not actually 
    >seem to be the case however – reading previously deallocated blocks do 
    >not actually return 0s.
    >

    Can you share the configuration you use for QEMU? DSM works best with 
    4096 byte LBAs (logical_block_size=4096) and the raw format driver.

    Also, please test with the Error Recovery feature (and set DULBE) to 
    test if the device reports that the block is actually deallocated.

[JH] I've tested both 512 and 4096 LBAs and hacked something up to enable
DULBE.  I am not seeing any errors reading blocks that were part of a
previous DSM command.  So at least for the cases in the SPDK test, these
blocks are not actually being deallocated.

    >It seems DLFEAT is being set incorrectly here – should probably be 0x8 
    >instead?
    >
    >Thanks,
    >
    >Jim
    >
    >
    >

    -- 
    k