Bug in reshape+discard?

[Benjamin Sonntag <benjamin@octopuce.fr>]

Hi everyone, 

I think we found a bug in the mdadm code here at Octopuce. I'm reporting it here, please tell me if that's not the right place to report it, or if you need any other information. 

This bug "hangs" processes in the Device-busy (D) state forever, until we reboot. It has been tested on both a debian 5.10 an 6.0 Linux kernel 

How to trigger the bug: 

- create a raid5 or raid6 block device using mdadm 
mdadm --create /dev/md0 -l 5 -n 3 /dev/sd{a,b,c}2 

- create a partition on it and mount it USING DISCARD/TRIM (important) (the underlying device must support trim) 
mkfs.ext4 /dev/md0 
mount /dev/md0 /mnt -o discard 

- create a few files 
for i in $( seq 1 1000 ) ; do dd if=/dev/zero of=/mnt/$i bs=10M count=1 ; done 

- expand the raid by adding a new drive
mdadm --add /dev/md0 /dev/sdd2 
mdadm --grow /dev/md0 -n 4 

- the last command will start a "reshape" operation on md0 
- DURING THE RESHAPE (it's important) erase some file (it goes fine) 
rm /mnt/* -rf 

- THEN, still during the reshape (important) try to sync or fsync 
sync 

- the sync process get stuck in the D state. no way to kill it until reboot 
(in fact, any process that does sync during the reshape after some files were deleted will get stuck, such as mariadbd or rsyslog...) 

- If you don't mount with discard your partition, the 'sync' works properly 


An easy way to circumvent this problem: 

- before reshaping, remount without discard 
mount /mnt -o remount,nodiscard 

- after the reshaping ends, remount with discard 
mount /mnt -o remount,discard 


We don't really know how to start searching for a solution, since it requires knowing the internal of MD & Discard pretty well :/ (and I'm definitely not a kernel coder ;) ) 

thanks for your help on this issue, 

cheers, 

Benjamin Sonntag 
Octopuce, Paris.