RAID5 TRIM/DISCARD user land support (fstriim and mkfs)

RAID5 TRIM/DISCARD user land support (fstriim and mkfs)

[Christian Balzer]

Hello,

Debian system, tested both Wheezy and Jessie, thus 3.16 kernel with
backports.

8 SSD RAID5, of course with raid456.devices_handle_discard_safely set to 1.
I can re-mount it with discard w/o a problem:
---
EXT4-fs (md4): re-mounted. Opts: stripe=112,data=ordered,discard
---

However fstrim whines about "FITRIM ioctl failed: Operation not supported"
and mkfs.ext4 unsurprisingly doesn't discard things when invoked.

The same HW/OS combo works fine with RAID10.

When looking at the sysfs entries for that MD device the fact
that /sys/class/block/md4/queue/discard_zeroes_data is 0 doesn't instill
me with hope or confidence 

Now while the SSDs in question (Intel DC S3610s) don't really need TRIM
really would have liked them to be able to start with a clean slate so to
speak.

Any insights?

Christian
-- 
Christian Balzer        Network/Systems Engineer                
chibi@gol.com   	Global OnLine Japan/Fusion Communications
http://www.gol.com/

Re: RAID5 TRIM/DISCARD user land support (fstriim and mkfs)

[Holger Kiehl]

Hello,

On Mon, 10 Aug 2015, Christian Balzer wrote:

>
> Hello,
>
> Debian system, tested both Wheezy and Jessie, thus 3.16 kernel with
> backports.
>
> 8 SSD RAID5, of course with raid456.devices_handle_discard_safely set to 1.
> I can re-mount it with discard w/o a problem:
> ---
> EXT4-fs (md4): re-mounted. Opts: stripe=112,data=ordered,discard
> ---
>
> However fstrim whines about "FITRIM ioctl failed: Operation not supported"
> and mkfs.ext4 unsurprisingly doesn't discard things when invoked.
>
> The same HW/OS combo works fine with RAID10.
>
> When looking at the sysfs entries for that MD device the fact
> that /sys/class/block/md4/queue/discard_zeroes_data is 0 doesn't instill
> me with hope or confidence
>
> Now while the SSDs in question (Intel DC S3610s) don't really need TRIM
> really would have liked them to be able to start with a clean slate so to
> speak.
>
> Any insights?
>
Discard for raid456 has been disabled by default because some devices in
some cases don't do what it says on the label.

You can enable it by setting the module parameter

    raid456.devices_handle_discard_safely=Y

Ensure that your drives can handle discard safely.

Regards,
Holger

Re: RAID5 TRIM/DISCARD user land support (fstriim and mkfs)

[Christian Balzer]

On Mon, 10 Aug 2015 08:28:39 +0000 (UTC) Holger Kiehl wrote:

> Hello,
> 
> On Mon, 10 Aug 2015, Christian Balzer wrote:
> 
> >
> > Hello,
> >
> > Debian system, tested both Wheezy and Jessie, thus 3.16 kernel with
> > backports.
> >
> > 8 SSD RAID5, of course with raid456.devices_handle_discard_safely set
> > to 1. I can re-mount it with discard w/o a problem:
> > ---
> > EXT4-fs (md4): re-mounted. Opts: stripe=112,data=ordered,discard
> > ---
> >
> > However fstrim whines about "FITRIM ioctl failed: Operation not
> > supported" and mkfs.ext4 unsurprisingly doesn't discard things when
> > invoked.
> >
> > The same HW/OS combo works fine with RAID10.
> >
> > When looking at the sysfs entries for that MD device the fact
> > that /sys/class/block/md4/queue/discard_zeroes_data is 0 doesn't
> > instill me with hope or confidence
> >
> > Now while the SSDs in question (Intel DC S3610s) don't really need TRIM
> > really would have liked them to be able to start with a clean slate so
> > to speak.
> >
> > Any insights?
> >
> Discard for raid456 has been disabled by default because some devices in
> some cases don't do what it says on the label.
> 
> You can enable it by setting the module parameter
> 
>     raid456.devices_handle_discard_safely=Y
> 
That's what I said I did in the 2nd sentence up there.

If it were unset, a kernel warning
"md/raid456: discard support disabled due to uncertainty."
would be issued during RAID startup/assembly.

Christian

> Ensure that your drives can handle discard safely.
> 
> Regards,
> Holger
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
Christian Balzer        Network/Systems Engineer                
chibi@gol.com   	Global OnLine Japan/Fusion Communications
http://www.gol.com/

Re: RAID5 TRIM/DISCARD user land support (fstriim and mkfs)

[Holger Kiehl]

On Mon, 10 Aug 2015, Christian Balzer wrote:

> On Mon, 10 Aug 2015 08:28:39 +0000 (UTC) Holger Kiehl wrote:
>
>> Hello,
>>
>> On Mon, 10 Aug 2015, Christian Balzer wrote:
>>
>>>
>>> Hello,
>>>
>>> Debian system, tested both Wheezy and Jessie, thus 3.16 kernel with
>>> backports.
>>>
>>> 8 SSD RAID5, of course with raid456.devices_handle_discard_safely set
>>> to 1. I can re-mount it with discard w/o a problem:
>>> ---
>>> EXT4-fs (md4): re-mounted. Opts: stripe=112,data=ordered,discard
>>> ---
>>>
>>> However fstrim whines about "FITRIM ioctl failed: Operation not
>>> supported" and mkfs.ext4 unsurprisingly doesn't discard things when
>>> invoked.
>>>
>>> The same HW/OS combo works fine with RAID10.
>>>
>>> When looking at the sysfs entries for that MD device the fact
>>> that /sys/class/block/md4/queue/discard_zeroes_data is 0 doesn't
>>> instill me with hope or confidence
>>>
>>> Now while the SSDs in question (Intel DC S3610s) don't really need TRIM
>>> really would have liked them to be able to start with a clean slate so
>>> to speak.
>>>
>>> Any insights?
>>>
>> Discard for raid456 has been disabled by default because some devices in
>> some cases don't do what it says on the label.
>>
>> You can enable it by setting the module parameter
>>
>>     raid456.devices_handle_discard_safely=Y
>>
> That's what I said I did in the 2nd sentence up there.
>
> If it were unset, a kernel warning
> "md/raid456: discard support disabled due to uncertainty."
> would be issued during RAID startup/assembly.
>
Rigth. Sorry, I did not read your mail properly.

Holger

Re: RAID5 TRIM/DISCARD user land support (fstriim and mkfs)

[Christian Balzer]

Gotta love talking to oneself.

This seems to be related to discard_granularity, the RAID5s I tested this
with had 32KB or 64KB chunks.
When using 8KB or 4KB chunks the resulting discard_granularity is 64KB or
32KB respectively and at that size all of a sudden mkfs does the
initial discard and fstrim starts working as well.

However this is a pretty slow process, at about 3.5GB/s, whereas a single
of those SSDs can do 12GB/s TRIM.

Christian

On Mon, 10 Aug 2015 16:46:57 +0900 Christian Balzer wrote:

> 
> Hello,
> 
> Debian system, tested both Wheezy and Jessie, thus 3.16 kernel with
> backports.
> 
> 8 SSD RAID5, of course with raid456.devices_handle_discard_safely set to
> 1. I can re-mount it with discard w/o a problem:
> ---
> EXT4-fs (md4): re-mounted. Opts: stripe=112,data=ordered,discard
> ---
> 
> However fstrim whines about "FITRIM ioctl failed: Operation not
> supported" and mkfs.ext4 unsurprisingly doesn't discard things when
> invoked.
> 
> The same HW/OS combo works fine with RAID10.
> 
> When looking at the sysfs entries for that MD device the fact
> that /sys/class/block/md4/queue/discard_zeroes_data is 0 doesn't instill
> me with hope or confidence 
> 
> Now while the SSDs in question (Intel DC S3610s) don't really need TRIM
> really would have liked them to be able to start with a clean slate so to
> speak.
> 
> Any insights?
> 
> Christian


-- 
Christian Balzer        Network/Systems Engineer                
chibi@gol.com   	Global OnLine Japan/Fusion Communications
http://www.gol.com/