Re: [PATCH] Make btrfs_prepare_device parallel during mkfs.btrfs

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

On 2022/8/28 16:53, li zhang wrote:
> Hi, I'm a bit confused, do you mean if you open a zoned device
> without O_DIRECT it will fail?

Not a zoned device expert, but to my understanding, if we write into
zoned device, without O_DIRECT, there is no guarantee that the data you
submitted will end at the same bytenr you specified.

E.g. if you do a pwrite() with a 1M buffer, at device bytenr 4M.

Without O_DIRECT, the zoned code can re-locate the bytenr to any range
after the write pointer inside the same zone.

AKA, for zoned device, without O_DIRECT (queue length 1), you can only
known the real physical bytenr after the write has fully finished.

(The final physical bytenr is determined by the zoned device, no longer
the write initiator).

>
> I tested and found that if I open a device with the O_DIRECT flag
> on a virtual device like a loop device, the device cannot be written
> to, but with or without O_DIRECT, it works fine on a real
> device (for me, I only test A normal block device since I don't have
> any zoned devices)

IIRC currently there is no zoned emulation for loop device.

If you want to test zoned device, you can use null block kernel module,
with fully memory backed storage:

https://zonedstorage.io/docs/getting-started/nullblk


Or go a little further, using tcmu-runner to create file backed zoned
device:

https://zonedstorage.io/docs/tools/tcmu-runner

>
> If we use the same flags for all devices,
> does that mean we can't use mkfs.btrfs
> on both real and virtual devices at the same time.
>
>
> Below is my test program and test results.
>
> code(main idea):
> printf("filename:%s.\n", argv[1]);
> int fd = open(argv[1], O_RDWR | O_DIRECT);
> if (fd < 0) {
>       printf("fd:error.\n");
>       return -1;
> }
> int num = write(fd, "123", 3);
> printf("num:%d.\n", num);

O_DIRECT requires strict memory alignment, obviously the length 3 is not
properly aligned.

Please check open(2p) for the full requirement.

For mkfs usage, all of our write is at least 4K aligned, thus O_DIRECT
can work correctly.


Back to btrfs-progs work, I'd say before we do anything, let's check all
the devices passed in to determine if we want zoned mode (any zoned
device should make it zoned).

Then we can determine the open flags for all devices, and for regular
devices, O_DIRECT mostly makes no difference (maybe a little slower, but
may not even be observable).

Thanks,
Qu


> close(fd);
>
> result:
> $ sudo losetup /dev/loop1 loopDev/loop1
> $ sudo ./a.out /dev/loop1
> filename:/dev/loop1.
> num:-1.
> # cannot write to loop1
>
>
> Thanks,
> Li Zhang
>
> Johannes Thumshirn <Johannes.Thumshirn@wdc.com> 于2022年8月25日周四 16:31写道：
>>
>> On 25.08.22 07:20, Qu Wenruo wrote:
>>>> +                    if (zoned && zoned_model(file) == ZONED_HOST_MANAGED)
>>>> +                            prepare_ctx[i].oflags = O_RDWR | O_DIRECT;
>>> Do we need to treat the initial and other devices differently?
>>>
>>> Can't we use the same flags for all devices?
>>>
>>>
>>
>> Yep we need to have the same flags for all devices. Otherwise only
>> device 0 will be opened with O_DIRECT, in case of a host-managed one and
>> the subsequent will be opened without O_DIRECT causing mkfs to fail.