Re: status of bugzilla #99171 - mdraid broken for O_DIRECT

[Roland <devzero@web.de>]

hello,

>> 7. check for inconsistencies  via "cat 
>> /sys/block/md127/md/mismatch_cnt" again
>>
>> i am getting:
>>
>> cat /sys/block/md127/md/mismatch_cnt
>> 1048832
>>
>> so , we see that even with recent kernel (pve9 kernel is 6.14 based 
>> on ubuntu kernel),  we can break mdraid from non-root user inside a 
>> qemu VM on top ext4 on top of mdraid.
>>
>
> And what would happen if you use 'xfs' instead of 'ext4'?
> ext4 has some nasty requirements regarding 'flush', and that might well
> explain the issue here.
>
> Cheers,
>
> Hannes 


thanks for feedback and for this hint.

i tested with xfs today , and it seems to make no difference.

i can inject inconsistency into the raid with the mentioned tool via 
debian-vm with xfs inside debian and hosted on xfs formatted md raid1 on 
proxmox.

root@pve-hpmini-gen8:~# cat /sys/block/md126/md/mismatch_cnt
59648

# cat /proc/mdstat
Personalities : [raid0] [raid1] [raid4] [raid5] [raid6] [raid10] [linear]
md126 : active raid1 sdb3[1] sda3[0]
       48794624 blocks super 1.2 [2/2] [UU]
       [==================>..]  check = 90.5% (44183744/48794624) 
finish=0.7min speed=107987K/sec
       bitmap: 1/1 pages [4KB], 65536KB chunk

md127 : active raid1 sdb1[2] sda1[3]
       48794624 blocks super 1.2 [2/2] [UU]
       bitmap: 0/1 pages [0KB], 65536KB chunk

unused devices: <none>


roland


Am 13.10.25 um 08:48 schrieb Hannes Reinecke:
> On 10/11/25 21:25, Roland wrote:
>> hello,
>>
>> some late reply for this...
>>
>>  > Am 10.10.24 um 10:34 schrieb Hannes Reinecke:
>>  > Which I really would love
>>  > to see reproduced, especially with recent kernels, as there is a 
>> lot of
>>  > vagueness around it (add part of the disk on the raid as swap? How?
>>  > In the host? On the guest?).
>>
>> here is a reproducer everybody should be able to follow/reproduce.
>>
>> 1. install proxmox pve9 on a system with two empty disks for mdraid. 
>> build mdraid and format with ext4 .
>>
>> 2. add that ext4 mountpoint as a datastore type "dir" for file/vm 
>> storage in proxmox.
>>
>> 3. install a debian13 in a normal/default (cache=none, i.e. O_DIRECT 
>> = on)  linux VM. the virtual disk should be backed by that 
>> mdraid/ext4 datastore created above.
>>
>> 4. inside the vm as an ordinary user get break-raid-odirect.c from 
>> https://forum.proxmox.com/threads/mdraid-o_direct.156036/post-713543 , 
>> compile that and let that run for a while. then terminate with ctrl-c.
>>
>> 5. on the pve host, check if your raid did not throw any error or has 
>> mismatch_count >0 ( cat /sys/block/md127/md/mismatch_cnt ) in the 
>> meantime.
>>
>> 6. on the pve host start raid check with  "echo check > /sys/block/ 
>> md127/md/sync_action"
>>
>> 6. let that check run and wait until it finishes (/proc/mdstat)
>>
>> 7. check for inconsistencies  via "cat 
>> /sys/block/md127/md/mismatch_cnt" again
>>
>> i am getting:
>>
>> cat /sys/block/md127/md/mismatch_cnt
>> 1048832
>>
>> so , we see that even with recent kernel (pve9 kernel is 6.14 based 
>> on ubuntu kernel),  we can break mdraid from non-root user inside a 
>> qemu VM on top ext4 on top of mdraid.
>>
>
> And what would happen if you use 'xfs' instead of 'ext4'?
> ext4 has some nasty requirements regarding 'flush', and that might well
> explain the issue here.
>
> Cheers,
>
> Hannes