Feature Request: Device Manager Fake Trim / Zero Trim

Feature Request: Device Manager Fake Trim / Zero Trim

[charlesfdotz]

Hello,

I would like to request a new device manager layer be added that accepts trim requests for sectors and instead writes zeros to those sectors.

This would be useful to deal with SMR (shingled magnetic recording) drives that do not support trim. Currently after an SMR drive has had enough data written to it the performance drops dramatically because the disk must shuffle around data as if it were full and without trim support there is no way to inform the disk which sectors are no longer used. Currently there's no way to "fix" or reset this without doing an ATA secure erase despite many of these disk being sold without informing customers that they were SMR drives (western digital was sued for selling SMR drives as NAS drives).

However it seems that some of these drives are smart enough to mark sectors as unused if a sector is filled with zeros (https://old.reddit.com/r/DataHoarder/comments/g7lqaz/). So I think a device mapper that could emulate trim on these devices by simply accepting trim requests then filling those sectors with zeros. It would make a whole class of hard drives significantly more usable.

Sincerely,
Chuck

P.S. I was directed here after mailing the cryptsetup mailing list so I hope this is the right place to ask about this. Thank you.

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Hannes Reinecke]

On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
> Hello,
> 
> I would like to request a new device manager layer be added that accepts trim requests for sectors and instead
> writes zeros to those sectors.
> 
> This would be useful to deal with SMR (shingled magnetic recording) drives that do not support trim. Currently
> after an SMR drive has had enough data written to it the performance drops dramatically because the disk must
> shuffle around data as if it were full and without trim support there is no way to inform the disk which sectors
> are no longer used. Currently there's no way to "fix" or reset this without doing an ATA secure erase despite
> many of these disk being sold without informing customers that they were SMR drives (western digital was sued
> for selling SMR drives as NAS drives).
> 
Gosh, no, please don't.
SMR drives have a write pointer, and if the zone needs to be reset you 
just reset the write pointer. Writing zeroes will result in the 
opposite; the zone continues to be full, and no writes can happen there.

Which drive is this?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew
Myers, Andrew McDonald, Martje Boudien Moerman

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Mikulas Patocka]

On Mon, 9 Oct 2023, Hannes Reinecke wrote:

> On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
> > Hello,
> > 
> > I would like to request a new device manager layer be added that accepts
> > trim requests for sectors and instead
> > writes zeros to those sectors.
> > 
> > This would be useful to deal with SMR (shingled magnetic recording) drives
> > that do not support trim. Currently
> > after an SMR drive has had enough data written to it the performance drops
> > dramatically because the disk must
> > shuffle around data as if it were full and without trim support there is no
> > way to inform the disk which sectors
> > are no longer used. Currently there's no way to "fix" or reset this without
> > doing an ATA secure erase despite
> > many of these disk being sold without informing customers that they were SMR
> > drives (western digital was sued
> > for selling SMR drives as NAS drives).
> > 
> Gosh, no, please don't.
> SMR drives have a write pointer, and if the zone needs to be reset you just
> reset the write pointer. Writing zeroes will result in the opposite; the zone
> continues to be full, and no writes can happen there.
> 
> Which drive is this?
> 
> Cheers,
> 
> Hannes

BTW. what about converting trims to zone reset? If the trim spans a whole 
zone, the kernel could convert it to REQ_OP_ZONE_RESET on host-aware zoned 
devices.

Mikulas

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[charlesfdotz]

Oct 9, 2023, 09:43 by mpatocka@redhat.com:

>
>
> On Mon, 9 Oct 2023, Hannes Reinecke wrote:
>
>> On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
>> > Hello,
>> >
>> > I would like to request a new device manager layer be added that accepts
>> > trim requests for sectors and instead
>> > writes zeros to those sectors.
>> >
>> > This would be useful to deal with SMR (shingled magnetic recording) drives
>> > that do not support trim. Currently
>> > after an SMR drive has had enough data written to it the performance drops
>> > dramatically because the disk must
>> > shuffle around data as if it were full and without trim support there is no
>> > way to inform the disk which sectors
>> > are no longer used. Currently there's no way to "fix" or reset this without
>> > doing an ATA secure erase despite
>> > many of these disk being sold without informing customers that they were SMR
>> > drives (western digital was sued
>> > for selling SMR drives as NAS drives).
>> >
>> Gosh, no, please don't.
>> SMR drives have a write pointer, and if the zone needs to be reset you just
>> reset the write pointer. Writing zeroes will result in the opposite; the zone
>> continues to be full, and no writes can happen there.
>>
>> Which drive is this?
>>
>> Cheers,
>>
>> Hannes
>>
The drive in the link is a ST8000DM004-2CX188 according to the poster. To be clear I'm talking about device managed SMR drives that don't support trim and do not report being SMR *not* host aware drives. The DM-SMR drives work fine until you've written their total capacity once then the write performance craters permanently because the drive thinks all the data on it is important and has to reshuffle it constantly during writes. We are talking sub-1MiB/s and hangs waiting on the controller to move things around that can be several minutes. WD was recently sued for selling some of these as "WD Red NAS" drives.

I'm not sure why writing zeros appears to work so this just speculation on my part but I was thinking the drive might be smart enough to mark sectors full of zeros as unused and then return any request for an unused sector as zeros. 
>
> BTW. what about converting trims to zone reset? If the trim spans a whole
> zone, the kernel could convert it to REQ_OP_ZONE_RESET on host-aware zoned
> devices.
>
> Mikulas
>
This particular request was to try and make (some) device managed SMR drives that don't report to the host that they are SMR more usable. I was thinking if they are a model where zero sectors appear to "trim" one could just put this "fake-trim" device mapper layer over it and then it would act like a device managed SMR drive that supports trim.

Sincerely,
Chuck

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Damien Le Moal]

On 10/9/23 15:15, Hannes Reinecke wrote:
> On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
>> Hello,
>> 
>> I would like to request a new device manager layer be added that accepts
>> trim requests for sectors and instead writes zeros to those sectors.
>> 
>> This would be useful to deal with SMR (shingled magnetic recording) drives
>> that do not support trim. Currently after an SMR drive has had enough data
>> written to it the performance drops dramatically because the disk must 
>> shuffle around data as if it were full and without trim support there is no
>> way to inform the disk which sectors are no longer used. Currently there's
>> no way to "fix" or reset this without doing an ATA secure erase despite 
>> many of these disk being sold without informing customers that they were
>> SMR drives (western digital was sued for selling SMR drives as NAS
>> drives).
>> 
> Gosh, no, please don't. SMR drives have a write pointer, and if the zone
> needs to be reset you just reset the write pointer. Writing zeroes will
> result in the opposite; the zone continues to be full, and no writes can
> happen there.

Yes. And zone reset *is* a trim also since after a zone reset, the sectors in
the zone cannot be read.

> Which drive is this?

From reading the description of the "issue", it seems that this is related to
drive managed SMR disks, which may or may not be detectable from the host (if
the drive is honest and reports it, the scsi_disk zoned_cap attribute will say
"drive-managed"). So no zone reset for a drive managed SMR disk since they are
"normal" drives from the interface perspective.

A drive managed SMR disk may or may not benefit from TRIM. That completely
depends on the device internal implementation to handle the SMR constraint. If
the drive does not have trim/discard support, it is likely because that serves
no purpose or would be useless performance-wise...

-- 
Damien Le Moal
Western Digital Research

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Damien Le Moal]

On 10/10/23 01:25, charlesfdotz@tutanota.com wrote:
> 
> Oct 9, 2023, 09:43 by mpatocka@redhat.com:
> 
>> 
>> 
>> On Mon, 9 Oct 2023, Hannes Reinecke wrote:
>> 
>>> On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
>>>> Hello,
>>>> 
>>>> I would like to request a new device manager layer be added that
>>>> accepts trim requests for sectors and instead writes zeros to those
>>>> sectors.
>>>> 
>>>> This would be useful to deal with SMR (shingled magnetic recording)
>>>> drives that do not support trim. Currently after an SMR drive has had
>>>> enough data written to it the performance drops dramatically because
>>>> the disk must shuffle around data as if it were full and without trim
>>>> support there is no way to inform the disk which sectors are no longer
>>>> used. Currently there's no way to "fix" or reset this without doing an
>>>> ATA secure erase despite many of these disk being sold without
>>>> informing customers that they were SMR drives (western digital was
>>>> sued for selling SMR drives as NAS drives).
>>>> 
>>> Gosh, no, please don't. SMR drives have a write pointer, and if the zone
>>> needs to be reset you just reset the write pointer. Writing zeroes will
>>> result in the opposite; the zone continues to be full, and no writes can
>>> happen there.
>>> 
>>> Which drive is this?
>>> 
>>> Cheers,
>>> 
>>> Hannes
>>> 
> The drive in the link is a ST8000DM004-2CX188 according to the poster. To be
> clear I'm talking about device managed SMR drives that don't support trim and
> do not report being SMR *not* host aware drives. The DM-SMR drives work fine
> until you've written their total capacity once then the write performance
> craters permanently because the drive thinks all the data on it is important
> and has to reshuffle it constantly during writes. We are talking sub-1MiB/s
> and hangs waiting on the controller to move things around that can be several
> minutes. WD was recently sued for selling some of these as "WD Red NAS"
> drives.
> 
> I'm not sure why writing zeros appears to work so this just speculation on my
> part but I was thinking the drive might be smart enough to mark sectors full
> of zeros as unused and then return any request for an unused sector as zeros.

If the drive does not even support trim/discard, thinking it has such
intelligence is a long shot... I would not bet on it :)

> 
>> 
>> BTW. what about converting trims to zone reset? If the trim spans a whole 
>> zone, the kernel could convert it to REQ_OP_ZONE_RESET on host-aware zoned 
>> devices.
>> 
>> Mikulas
>> 
> This particular request was to try and make (some) device managed SMR drives
> that don't report to the host that they are SMR more usable. I was thinking
> if they are a model where zero sectors appear to "trim" one could just put
> this "fake-trim" device mapper layer over it and then it would act like a
> device managed SMR drive that supports trim.

Check /sys/block/sdX/device/scsi_disk/xxxx/zoned_cap. Does it say "drive-managed" ?

I do not think that writing zeroes will help in any way. Drives do not generally
(as far as I know) look at the data they are reading/writing to try to be smart.
Writing zeroes is the same as writing random bytes. For a good reason: sometimes
initializing sectors to contain zeroes *is* important !

-- 
Damien Le Moal
Western Digital Research

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Hannes Reinecke]

On 10/10/23 08:48, Damien Le Moal wrote:
> On 10/9/23 15:15, Hannes Reinecke wrote:
>> On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
>>> Hello,
>>>
>>> I would like to request a new device manager layer be added that accepts
>>> trim requests for sectors and instead writes zeros to those sectors.
>>>
>>> This would be useful to deal with SMR (shingled magnetic recording) drives
>>> that do not support trim. Currently after an SMR drive has had enough data
>>> written to it the performance drops dramatically because the disk must
>>> shuffle around data as if it were full and without trim support there is no
>>> way to inform the disk which sectors are no longer used. Currently there's
>>> no way to "fix" or reset this without doing an ATA secure erase despite
>>> many of these disk being sold without informing customers that they were
>>> SMR drives (western digital was sued for selling SMR drives as NAS
>>> drives).
>>>
>> Gosh, no, please don't. SMR drives have a write pointer, and if the zone
>> needs to be reset you just reset the write pointer. Writing zeroes will
>> result in the opposite; the zone continues to be full, and no writes can
>> happen there.
> 
> Yes. And zone reset *is* a trim also since after a zone reset, the sectors in
> the zone cannot be read.
> 
On the same vein: why didn't we hook up 'wp reset' to trim/unmap?
The code says:

         if (sd_is_zoned(sdkp)) {
                 sd_config_discard(sdkp, SD_LBP_DISABLE);
                 return count;
         }

I distinctly remember to enable discards via 'wp reset' in the
original code; what happened to it?

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew
Myers, Andrew McDonald, Martje Boudien Moerman

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Damien Le Moal]

On 10/10/23 16:15, Hannes Reinecke wrote:
> On 10/10/23 08:48, Damien Le Moal wrote:
>> On 10/9/23 15:15, Hannes Reinecke wrote:
>>> On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
>>>> Hello,
>>>>
>>>> I would like to request a new device manager layer be added that accepts
>>>> trim requests for sectors and instead writes zeros to those sectors.
>>>>
>>>> This would be useful to deal with SMR (shingled magnetic recording) drives
>>>> that do not support trim. Currently after an SMR drive has had enough data
>>>> written to it the performance drops dramatically because the disk must
>>>> shuffle around data as if it were full and without trim support there is no
>>>> way to inform the disk which sectors are no longer used. Currently there's
>>>> no way to "fix" or reset this without doing an ATA secure erase despite
>>>> many of these disk being sold without informing customers that they were
>>>> SMR drives (western digital was sued for selling SMR drives as NAS
>>>> drives).
>>>>
>>> Gosh, no, please don't. SMR drives have a write pointer, and if the zone
>>> needs to be reset you just reset the write pointer. Writing zeroes will
>>> result in the opposite; the zone continues to be full, and no writes can
>>> happen there.
>>
>> Yes. And zone reset *is* a trim also since after a zone reset, the sectors in
>> the zone cannot be read.
>>
> On the same vein: why didn't we hook up 'wp reset' to trim/unmap?
> The code says:
> 
>          if (sd_is_zoned(sdkp)) {
>                  sd_config_discard(sdkp, SD_LBP_DISABLE);
>                  return count;
>          }
> 
> I distinctly remember to enable discards via 'wp reset' in the
> original code; what happened to it?

Hmmm... I do not recall exactly. I think it was all about being clear and to not
overload trim/discard, or scsi unmap, which at the time was not the prettiest
thing in scsi code (it got better though).

But I do prefer having zone reset separate from trim/discard, to be clear about
the fact that things are zone aligned. From the code perspective, there are only
few places were overloading trim with zone reset would simplify a little the
code. F2fs is probably the main one that comes to mind, but simplification would
be minor. So not worth the confusion I think.

> 
> Cheers,
> 
> Hannes

-- 
Damien Le Moal
Western Digital Research

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[charlesfdotz]

Oct 10, 2023, 06:48 by dlemoal@kernel.org:

> On 10/9/23 15:15, Hannes Reinecke wrote:
>
>> On 10/9/23 02:56, charlesfdotz@tutanota.com wrote:
>>
>>> Hello,
>>>
>>> I would like to request a new device manager layer be added that accepts
>>> trim requests for sectors and instead writes zeros to those sectors.
>>>
>>> This would be useful to deal with SMR (shingled magnetic recording) drives
>>> that do not support trim. Currently after an SMR drive has had enough data
>>> written to it the performance drops dramatically because the disk must 
>>> shuffle around data as if it were full and without trim support there is no
>>> way to inform the disk which sectors are no longer used. Currently there's
>>> no way to "fix" or reset this without doing an ATA secure erase despite 
>>> many of these disk being sold without informing customers that they were
>>> SMR drives (western digital was sued for selling SMR drives as NAS
>>> drives).
>>>
>> Gosh, no, please don't. SMR drives have a write pointer, and if the zone
>> needs to be reset you just reset the write pointer. Writing zeroes will
>> result in the opposite; the zone continues to be full, and no writes can
>> happen there.
>>
>
> Yes. And zone reset *is* a trim also since after a zone reset, the sectors in
> the zone cannot be read.
>
>> Which drive is this?
>>
>
> From reading the description of the "issue", it seems that this is related to
> drive managed SMR disks, which may or may not be detectable from the host (if
> the drive is honest and reports it, the scsi_disk zoned_cap attribute will say
> "drive-managed"). So no zone reset for a drive managed SMR disk since they are
> "normal" drives from the interface perspective.
>
> A drive managed SMR disk may or may not benefit from TRIM. That completely
> depends on the device internal implementation to handle the SMR constraint. If
> the drive does not have trim/discard support, it is likely because that serves
> no purpose or would be useless performance-wise...
>
> -- 
> Damien Le Moal
> Western Digital Research
>
As I've said in all of my emails this is regarding drive-managed SMR disks that do no support trim. Most of these aren't detectable by the host because if there were then people wouldn't buy them as they're not fit for purpose. Western digital was sued over this and lost.
https://arstechnica.com/gadgets/2020/05/western-digital-gets-sued-for-sneaking-smr-disks-into-its-nas-channel/
https://www.tomshardware.com/news/wd-red-smr-lawsuit-pays-out-pennies-in-settlement-damages

Oct 10, 2023, 06:53 by dlemoal@kernel.org:

> On 10/10/23 01:25, > charlesfdotz@tutanota.com>  wrote:
>
>>
>> Oct 9, 2023, 09:43 by >> mpatocka@redhat.com>> :
>>
>>>
>>>
>>> On Mon, 9 Oct 2023, Hannes Reinecke wrote:
>>>
>>>> On 10/9/23 02:56, >>>> charlesfdotz@tutanota.com>>>>  wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I would like to request a new device manager layer be added that
>>>>> accepts trim requests for sectors and instead writes zeros to those
>>>>> sectors.
>>>>>
>>>>> This would be useful to deal with SMR (shingled magnetic recording)
>>>>> drives that do not support trim. Currently after an SMR drive has had
>>>>> enough data written to it the performance drops dramatically because
>>>>> the disk must shuffle around data as if it were full and without trim
>>>>> support there is no way to inform the disk which sectors are no longer
>>>>> used. Currently there's no way to "fix" or reset this without doing an
>>>>> ATA secure erase despite many of these disk being sold without
>>>>> informing customers that they were SMR drives (western digital was
>>>>> sued for selling SMR drives as NAS drives).
>>>>>
>>>> Gosh, no, please don't. SMR drives have a write pointer, and if the zone
>>>> needs to be reset you just reset the write pointer. Writing zeroes will
>>>> result in the opposite; the zone continues to be full, and no writes can
>>>> happen there.
>>>>
>>>> Which drive is this?
>>>>
>>>> Cheers,
>>>>
>>>> Hannes
>>>>
>> The drive in the link is a ST8000DM004-2CX188 according to the poster. To be
>> clear I'm talking about device managed SMR drives that don't support trim and
>> do not report being SMR *not* host aware drives. The DM-SMR drives work fine
>> until you've written their total capacity once then the write performance
>> craters permanently because the drive thinks all the data on it is important
>> and has to reshuffle it constantly during writes. We are talking sub-1MiB/s
>> and hangs waiting on the controller to move things around that can be several
>> minutes. WD was recently sued for selling some of these as "WD Red NAS"
>> drives.
>>
>> I'm not sure why writing zeros appears to work so this just speculation on my
>> part but I was thinking the drive might be smart enough to mark sectors full
>> of zeros as unused and then return any request for an unused sector as zeros.
>>
>
> If the drive does not even support trim/discard, thinking it has such
> intelligence is a long shot... I would not bet on it :)
>
>>>
>>> BTW. what about converting trims to zone reset? If the trim spans a whole
>>> zone, the kernel could convert it to REQ_OP_ZONE_RESET on host-aware zoned
>>> devices.
>>>
>>> Mikulas
>>>
>> This particular request was to try and make (some) device managed SMR drives
>> that don't report to the host that they are SMR more usable. I was thinking
>> if they are a model where zero sectors appear to "trim" one could just put
>> this "fake-trim" device mapper layer over it and then it would act like a
>> device managed SMR drive that supports trim.
>>
>
> Check /sys/block/sdX/device/scsi_disk/xxxx/zoned_cap. Does it say "drive-managed" ?
>
> I do not think that writing zeroes will help in any way. Drives do not generally
> (as far as I know) look at the data they are reading/writing to try to be smart.
> Writing zeroes is the same as writing random bytes. For a good reason: sometimes
> initializing sectors to contain zeroes *is* important !
>
> --
> Damien Le Moal
> Western Digital Research
>
You can opine that this unlikely but we have reports (which I've already linked) that this works on some SMR drives. It's also pretty ironic someone from western digital of all places is saying drives wouldn't lie about how they function so they must be writing zeros when told to.

Sincerely,
Chuck

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Damien Le Moal]

On 10/10/23 23:31, charlesfdotz@tutanota.com wrote:
> As I've said in all of my emails this is regarding drive-managed SMR disks that do no support trim. Most of these aren't detectable by the host because if there were then people wouldn't buy them as they're not fit for purpose. Western digital was sued over this and lost.
> https://arstechnica.com/gadgets/2020/05/western-digital-gets-sued-for-sneaking-smr-disks-into-its-nas-channel/
> https://www.tomshardware.com/news/wd-red-smr-lawsuit-pays-out-pennies-in-settlement-damages

There is no need to name names or point fingers here. Kernel open source
development is vendor agnostic and while my employer is indeed Western Digital,
I do not talk as a representative of WD products but as a kernel developer. So
calm down please if you want to keep the discussion open.

> You can opine that this unlikely but we have reports (which I've already linked) that this works on some SMR drives. It's also pretty ironic someone from western digital of all places is saying drives wouldn't lie about how they function so they must be writing zeros when told to.

Again, the finger pointing here is totally inappropriate. My points are all in
good faith and I am talking about the drives I know of.

In any case, the key-word in your statement is "some drives". Given that you are
talking about something that is not standardized and so cannot be safely
generalized, there is nothing we can do. It is a hard NO from me to replace the
lack of trim support with writing zeroes.

> 
> Sincerely,
> Chuck
> 

-- 
Damien Le Moal
Western Digital Research

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[charlesfdotz]

Oct 11, 2023, 00:33 by dlemoal@kernel.org:

> On 10/10/23 23:31, charlesfdotz@tutanota.com wrote:
>
>> As I've said in all of my emails this is regarding drive-managed SMR disks that do no support trim. Most of these aren't detectable by the host because if there were then people wouldn't buy them as they're not fit for purpose. Western digital was sued over this and lost.
>> https://arstechnica.com/gadgets/2020/05/western-digital-gets-sued-for-sneaking-smr-disks-into-its-nas-channel/
>> https://www.tomshardware.com/news/wd-red-smr-lawsuit-pays-out-pennies-in-settlement-damages
>>
>
> There is no need to name names or point fingers here. Kernel open source
> development is vendor agnostic and while my employer is indeed Western Digital,
> I do not talk as a representative of WD products but as a kernel developer. So
> calm down please if you want to keep the discussion open.
>
>> You can opine that this unlikely but we have reports (which I've already linked) that this works on some SMR drives. It's also pretty ironic someone from western digital of all places is saying drives wouldn't lie about how they function so they must be writing zeros when told to.
>>
>
> Again, the finger pointing here is totally inappropriate. My points are all in
> good faith and I am talking about the drives I know of.
>
> In any case, the key-word in your statement is "some drives". Given that you are
> talking about something that is not standardized and so cannot be safely
> generalized, there is nothing we can do. It is a hard NO from me to replace the
> lack of trim support with writing zeroes.
>
>>
>> Sincerely,
>> Chuck
>>
>
> --
> Damien Le Moal
> Western Digital Research
>
Presumably drives of the same model will work the say way so when I say "some drives" we're talking hundreds of thousands or millions if even a few models function this way. In my initial email I included a link where this seems to help on some models so I think it's very possible this could've made quite a lot of drives much more usable. I was requesting a separate device mapper layer with the thought that someone with one of these drives would test it and if zeroing sectors helped then they could add a "dm-trimemulation" layer on top of it. That's not replacing anything only adding an option for users.

I also don't understand why you would say "calm down please" if I "want to keep this discussion open" while in the same breath saying "it is a hard NO". If the answer is "NO" then "keep[ing] this discussion open" is pointless. In any case I brought up the deception by western digital to highlight that many buyers (myself included) were tricked into unintentionally buying drives that were not fit for purpose so trying to regain some functionality would be worthwhile. And someone from WD saying drives don't lie about how they function *is* ironic.

Going on about "good faith" after saying this probably wouldn't "help in any way" while seemingly ignoring the link to discussions where it very much appears to help is shitty.

Sincerely,
Chuck

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Christoph Hellwig]

Hi Charles,

you are doing a whole lot of handwaiving here (and not even following
the Linux mailing list style).

If you actually have a driver with your alleged behavior you can
simply switch the scsi disk driver to zero blocks on discard by
doing an

    echo "writesame_zero" > /sys/.../provisioning_mode

and get your behavior.  I very much doubt you'll find such a drive.
There are exactly three hard drive manufactures left, with just a few
models each, and they are pretty well understood.  You've even got
feedback from the prime expert on hard drive handling in Linux who
works for one of them.

Re: Feature Request: Device Manager Fake Trim / Zero Trim

[Hannes Reinecke]

On 10/11/23 21:07, charlesfdotz@tutanota.com wrote:
> Oct 11, 2023, 00:33 by dlemoal@kernel.org:
> 
>> On 10/10/23 23:31, charlesfdotz@tutanota.com wrote:
>>
>>> As I've said in all of my emails this is regarding drive-managed SMR disks that do no support trim. Most of these aren't detectable by the host because if there were then people wouldn't buy them as they're not fit for purpose. Western digital was sued over this and lost.
>>> https://arstechnica.com/gadgets/2020/05/western-digital-gets-sued-for-sneaking-smr-disks-into-its-nas-channel/
>>> https://www.tomshardware.com/news/wd-red-smr-lawsuit-pays-out-pennies-in-settlement-damages
>>>
>>
>> There is no need to name names or point fingers here. Kernel open source
>> development is vendor agnostic and while my employer is indeed Western Digital,
>> I do not talk as a representative of WD products but as a kernel developer. So
>> calm down please if you want to keep the discussion open.
>>
>>> You can opine that this unlikely but we have reports (which I've already linked) that this works on some SMR drives. It's also pretty ironic someone from western digital of all places is saying drives wouldn't lie about how they function so they must be writing zeros when told to.
>>>
>>
>> Again, the finger pointing here is totally inappropriate. My points are all in
>> good faith and I am talking about the drives I know of.
>>
>> In any case, the key-word in your statement is "some drives". Given that you are
>> talking about something that is not standardized and so cannot be safely
>> generalized, there is nothing we can do. It is a hard NO from me to replace the
>> lack of trim support with writing zeroes.
>>
>>>
>>> Sincerely,
>>> Chuck
>>>
>>
>> --
>> Damien Le Moal
>> Western Digital Research
>>
> Presumably drives of the same model will work the say way so when I say "some drives" we're talking hundreds of
> thousands or millions if even a few models function this way. In my initial email I included a link where this
> seems to help on some models so I think it's very possible this could've made quite a lot of drives much more
> usable. I was requesting a separate device mapper layer with the thought that someone with one of these drives
> would test it and if zeroing sectors helped then they could add a "dm-trimemulation" layer on top of it. That's
> not replacing anything only adding an option for users.
> 
Oh, sure, they could. Problem here is that we don't have any specifics 
on why this 'dm-trimemulation' layer helps in this situation.
What we do know, though, is that it works exactly _against_ how SMR 
drives work internally. On SMR drives writing zeroes is about the 
worst-case scenario, as it'll fill the disk with meaningless data but 
_not_ reset zones such that you can write to it. IE essentially making 
the drive unusable.

Understandably that we're a little concerned here.

Even more so as this is a firmware emulation issue on the drive, so
it _could_ be fixed by the drive manufacturer in updating the firmware
to actually support TRIM.

But sure, if you found that there are drives for this such an emulation 
might help, by all means, send patches. I'd be happily reviewing them.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                Kernel Storage Architect
hare@suse.de                              +49 911 74053 688
SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
HRB 36809 (AG Nürnberg), Geschäftsführer: Ivo Totev, Andrew
Myers, Andrew McDonald, Martje Boudien Moerman