fstrim on full stripes

fstrim on full stripes

[Swami Kevala]

I have a RAID6 array made up of 8 NVMe SSD drives. I have noticed that
the write speed for the array is slower than the write speed for an
individual drive!

It is not possible for me to use trim on my array since my drives (WD
SN650) report "Write Zeroes Not Supported"

As per my understanding, the reason why trim on raid is complex to
implement is due to the need to recalculate the parity blocks whenever
data blocks are discarded.

My question is: Would it be possible (or a good idea), to make a
version of fstrim (e.g. fstrimraid) that could discard at the stripe
level? i.e. Discard only those blocks for which all blocks in the
stripe can be discarded.

I guess this would need to call the md api to know which file system
blocks are stored on which stripes.

Our server is used for editing large video files, so I would expect
that a significant percentage of discard operations would result in
entire stripes being discarded at once. So I wonder if this would be a
relatively simple and effective way of improving write performance on
SSD RAIDs without having to worry about parity.

Would be interested to know what people think.

-- 
- 9442504660

-- 
The information contained in this electronic message and any attachments to 
this message are intended for the exclusive use of the addressee(s) and may 
contain proprietary, confidential or privileged information. If you are not 
the intended recipient, you should not disseminate, distribute or copy this 
e-mail. Please notify the sender immediately and destroy all copies of this 
message and any attachments. WARNING: Computer viruses can be transmitted 
via email. The recipient should check this email and any attachments for 
the presence of viruses. The Organisation accepts no liability for any 
damage caused by any virus transmitted by this email. 
www.ishafoundation.org <http://www.ishafoundation.org>

-- 

Re: fstrim on full stripes

[Piergiorgio Sartor]

On Sat, Mar 16, 2024 at 02:27:02PM +0530, Swami Kevala wrote:
> I have a RAID6 array made up of 8 NVMe SSD drives. I have noticed that
> the write speed for the array is slower than the write speed for an
> individual drive!

This could be.
There is some cache tuning in the folder:

/sys/class/block/mdXXX/md/stripe_cache_*

Two files, check for detailed docs.

It might help, or not.

> It is not possible for me to use trim on my array since my drives (WD
> SN650) report "Write Zeroes Not Supported"

So, don't trim.

Apart that, "Write Zeroes Not Supported" is new to me.
I guess it is always possible to write zeroes...
I know about "Return Zeroes After Trim" or similar.

> As per my understanding, the reason why trim on raid is complex to
> implement is due to the need to recalculate the parity blocks whenever
> data blocks are discarded.

It is implemented if 1) the SSD support RZAT or DRAT and
2) it is enable (under user responsability).
At least for RAID5, but it seems to me RAID6 as well.
The module parameter: devices_handle_discard_safely
Also in this case, check for detailed docs.

> My question is: Would it be possible (or a good idea), to make a
> version of fstrim (e.g. fstrimraid) that could discard at the stripe
> level? i.e. Discard only those blocks for which all blocks in the
> stripe can be discarded.
> 
> I guess this would need to call the md api to know which file system
> blocks are stored on which stripes.
> 
> Our server is used for editing large video files, so I would expect
> that a significant percentage of discard operations would result in
> entire stripes being discarded at once. So I wonder if this would be a
> relatively simple and effective way of improving write performance on
> SSD RAIDs without having to worry about parity.
> 
> Would be interested to know what people think.

It's not clear to me the connection you see
between "trim" and "write speed" in RAID6.

I mean, if the SSDs have speed problems due to
missing trims, then maybe better to change SSDs.
On the other hand, as wrote at the beginning,
maybe playing with the RAID stripe cache could
help more (or not, as wrote).

Hope this helps a bit,

bye,

pg


> -- 
> - 9442504660
> 
> -- 
> The information contained in this electronic message and any attachments to 
> this message are intended for the exclusive use of the addressee(s) and may 
> contain proprietary, confidential or privileged information. If you are not 
> the intended recipient, you should not disseminate, distribute or copy this 
> e-mail. Please notify the sender immediately and destroy all copies of this 
> message and any attachments. WARNING: Computer viruses can be transmitted 
> via email. The recipient should check this email and any attachments for 
> the presence of viruses. The Organisation accepts no liability for any 
> damage caused by any virus transmitted by this email. 
> www.ishafoundation.org <http://www.ishafoundation.org>
> 
> -- 
> 
> 

-- 

piergiorgio