hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Mark Lord]

hdparm-9.17 is now available from sourceforge.net.

New in this release:
        - added wiper/ subdirectory with preliminary SSD-wiper (TRIM) scripts
        - nuked kernel_patches/ subdirectory
        - Capitalize first word of each line of help output
        - added --fallocate flag
        - added --trim-sector-ranges flag
        - removed --trim-sectors flag
        - removed --fibmap-sector flag
        - embed VERSION string for easier digging out with strings(1)

This release is mostly in support of enabling user-initiated DSM/TRIM operations
for solid-state drives (SSDs).  The in-kernel support for automatic-TRIM seems
a ways off yet, so I'm providing scripts to do it auto/manually instead.  :)

These basically scan for free blocks in a filesystem, and batch them together
into huge DSM/TRIM commands to the drive, letting it know about them all for
garbage-collection and wear-leveling purposes.

These scripts are similar (but being Linux, superior) to an existing wiper.exe
utility that MS Win users have been enjoying for some brands of SSD.

But they won't actually do anything until drives ship with TRIM-enabled firmware.

Cheers 

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Mark Lord]

Mark Lord wrote:
> hdparm-9.17 is now available from sourceforge.net.
> 
> New in this release:
>        - added wiper/ subdirectory with preliminary SSD-wiper (TRIM) 
..

Tejun:  on a related note, my Sil-3132 PCIe SATA controller doesn't
seem to like DSM/TRIM commands passing through it.

[ 3479.010001] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[ 3479.010008] ata4.00: irq_stat 0x00020002, protocol mismatch
[ 3479.010019] ata4.00: cmd 06/01:01:00:00:00/00:00:00:00:00/e0 tag 0 dma 512 out
[ 3479.010021]          res dc/33:00:00:00:00/00:00:00:00:dc/00 Emask 0x2 (HSM violation)
[ 3479.010025] ata4.00: status: { Busy }
[ 3479.010029] ata4.00: error: { IDNF }
[ 3479.010041] ata4: hard resetting link
[ 3481.193102] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
[ 3481.197043] ata4.00: configured for UDMA/100
[ 3481.197082] ata4: EH complete

I haven't taken time to dig further yet, but perhaps somebody here
might know if they've hardcoded all of the ATA opcodes in silicon or
something, and now get confused by DSM/TRIM ?

I'll dig further into that later this summer.

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Robert Hancock]

On 07/29/2009 05:39 PM, Mark Lord wrote:
> Mark Lord wrote:
>> hdparm-9.17 is now available from sourceforge.net.
>>
>> New in this release:
>> - added wiper/ subdirectory with preliminary SSD-wiper (TRIM)
> ..
>
> Tejun: on a related note, my Sil-3132 PCIe SATA controller doesn't
> seem to like DSM/TRIM commands passing through it.
>
> [ 3479.010001] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
> frozen
> [ 3479.010008] ata4.00: irq_stat 0x00020002, protocol mismatch
> [ 3479.010019] ata4.00: cmd 06/01:01:00:00:00/00:00:00:00:00/e0 tag 0
> dma 512 out
> [ 3479.010021] res dc/33:00:00:00:00/00:00:00:00:dc/00 Emask 0x2 (HSM
> violation)
> [ 3479.010025] ata4.00: status: { Busy }
> [ 3479.010029] ata4.00: error: { IDNF }
> [ 3479.010041] ata4: hard resetting link
> [ 3481.193102] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
> [ 3481.197043] ata4.00: configured for UDMA/100
> [ 3481.197082] ata4: EH complete
>
> I haven't taken time to dig further yet, but perhaps somebody here
> might know if they've hardcoded all of the ATA opcodes in silicon or
> something, and now get confused by DSM/TRIM ?
>
> I'll dig further into that later this summer.

Yeah, according to the datasheet "The SiI3124 will decode the 8-bit ATA
command at PRB offset 0x0a and automatically execute the default 
protocol for the command. In certain cases it might be desirable to 
specify a non-default protocol to be used, such as with vendor specific 
device commands." The DSM command seems to be DMA data-out and the chip 
likely doesn't know that command. I have to wonder why they decided to 
use that design instead of just making the driver indicate the protocol 
explicitly. In any case, it looks like the driver needs code to override 
the protocol setting for this command. (Maybe we should just set the 
protocol override for what we know the command is supposed to be in all 
cases?)

Sil311x will have the same problem. The solution there seems to be to 
execute a vendor-specific command to tell the controller what protocol 
that command code uses. Some other controllers may have similar issues 
if they are parsing the ATA command codes.. it's possible that some of 
them might not support DSM/TRIM commands properly.

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Jeff Garzik]

Robert Hancock wrote:
> Yeah, according to the datasheet "The SiI3124 will decode the 8-bit ATA
> command at PRB offset 0x0a and automatically execute the default 
> protocol for the command. In certain cases it might be desirable to 
> specify a non-default protocol to be used, such as with vendor specific 
> device commands." The DSM command seems to be DMA data-out and the chip 
> likely doesn't know that command. I have to wonder why they decided to 
> use that design instead of just making the driver indicate the protocol 
> explicitly. In any case, it looks like the driver needs code to override 
> the protocol setting for this command. (Maybe we should just set the 
> protocol override for what we know the command is supposed to be in all 
> cases?)
> 
> Sil311x will have the same problem. The solution there seems to be to 
> execute a vendor-specific command to tell the controller what protocol 
> that command code uses. Some other controllers may have similar issues 
> if they are parsing the ATA command codes.. it's possible that some of 
> them might not support DSM/TRIM commands properly.


Yes, this is a common problem for many SATA controllers, for any new 
opcode -- and for SATA<->PATA bridges too, which also snoop the opcode 
to determine certain behaviors.

	Jeff

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Mark Lord]

Robert Hancock wrote:
> On 07/29/2009 05:39 PM, Mark Lord wrote:
>> Mark Lord wrote:
>>> hdparm-9.17 is now available from sourceforge.net.
>>>
>>> New in this release:
>>> - added wiper/ subdirectory with preliminary SSD-wiper (TRIM)
>> ..
>>
>> Tejun: on a related note, my Sil-3132 PCIe SATA controller doesn't
>> seem to like DSM/TRIM commands passing through it.
>>
>> [ 3479.010001] ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6
>> frozen
>> [ 3479.010008] ata4.00: irq_stat 0x00020002, protocol mismatch
>> [ 3479.010019] ata4.00: cmd 06/01:01:00:00:00/00:00:00:00:00/e0 tag 0
>> dma 512 out
>> [ 3479.010021] res dc/33:00:00:00:00/00:00:00:00:dc/00 Emask 0x2 (HSM
>> violation)
>> [ 3479.010025] ata4.00: status: { Busy }
>> [ 3479.010029] ata4.00: error: { IDNF }
>> [ 3479.010041] ata4: hard resetting link
>> [ 3481.193102] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 0)
>> [ 3481.197043] ata4.00: configured for UDMA/100
>> [ 3481.197082] ata4: EH complete
>>
>> I haven't taken time to dig further yet, but perhaps somebody here
>> might know if they've hardcoded all of the ATA opcodes in silicon or
>> something, and now get confused by DSM/TRIM ?
>>
>> I'll dig further into that later this summer.
> 
> Yeah, according to the datasheet "The SiI3124 will decode the 8-bit ATA
> command at PRB offset 0x0a and automatically execute the default 
> protocol for the command. In certain cases it might be desirable to 
> specify a non-default protocol to be used, such as with vendor specific 
> device commands." The DSM command seems to be DMA data-out and the chip 
> likely doesn't know that command. I have to wonder why they decided to 
> use that design instead of just making the driver indicate the protocol 
> explicitly. In any case, it looks like the driver needs code to override 
> the protocol setting for this command. (Maybe we should just set the 
> protocol override for what we know the command is supposed to be in all 
> cases?)
..

If you can puzzle out how to do that, and post a quick(?) patch,
it would certainly make testing SSDs easier for me here.

I would like to use the Sil3124 ExpressCard controller with my notebook
for this stuff, rather than having to power up one of the noisy
full-size systems under the table.  ;)

On a related note.. anyone else out there have access to a TRIM-capable
SSD yet? If you do, you probably are under NDA, but that shouldn't stop
you from at least trying out the scripts in the hdparm package and
letting me know how things go.. in a very non vendor/model specific fashion. :)

Thanks

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Robert Hancock]

On 07/30/2009 06:54 AM, Mark Lord wrote:
>> Yeah, according to the datasheet "The SiI3124 will decode the 8-bit ATA
>> command at PRB offset 0x0a and automatically execute the default
>> protocol for the command. In certain cases it might be desirable to
>> specify a non-default protocol to be used, such as with vendor
>> specific device commands." The DSM command seems to be DMA data-out
>> and the chip likely doesn't know that command. I have to wonder why
>> they decided to use that design instead of just making the driver
>> indicate the protocol explicitly. In any case, it looks like the
>> driver needs code to override the protocol setting for this command.
>> (Maybe we should just set the protocol override for what we know the
>> command is supposed to be in all cases?)
> ..
> 
> If you can puzzle out how to do that, and post a quick(?) patch,
> it would certainly make testing SSDs easier for me here.
> 
> I would like to use the Sil3124 ExpressCard controller with my notebook
> for this stuff, rather than having to power up one of the noisy
> full-size systems under the table.  ;)

You can try this patch (totally untested) which basically just bludgeons it
into doing what we want for all non-packet commands:

diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
index 77aa8d7..e6946fc 100644
--- a/drivers/ata/sata_sil24.c
+++ b/drivers/ata/sata_sil24.c
@@ -846,6 +846,17 @@ static void sil24_qc_prep(struct ata_queued_cmd *qc)
 	if (!ata_is_atapi(qc->tf.protocol)) {
 		prb = &cb->ata.prb;
 		sge = cb->ata.sge;
+		if (ata_is_data(qc->tf.protocol)) {
+			u16 prot = 0;
+			ctrl = PRB_CTRL_PROTOCOL;
+			if (ata_is_ncq(qc->tf.protocol))
+				prot |= PRB_PROT_NCQ;
+			if (qc->tf.flags & ATA_TFLAG_WRITE)
+				prot |= PRB_PROT_WRITE;
+			else
+				prot |= PRB_PROT_READ;
+			prb->prot = cpu_to_le16(prot);
+		}
 	} else {
 		prb = &cb->atapi.prb;
 		sge = cb->atapi.sge;

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Mark Lord]

Robert Hancock wrote:
> On 07/30/2009 06:54 AM, Mark Lord wrote:
>>> Yeah, according to the datasheet "The SiI3124 will decode the 8-bit ATA
>>> command at PRB offset 0x0a and automatically execute the default
>>> protocol for the command. In certain cases it might be desirable to
>>> specify a non-default protocol to be used, such as with vendor
>>> specific device commands." The DSM command seems to be DMA data-out
>>> and the chip likely doesn't know that command. I have to wonder why
>>> they decided to use that design instead of just making the driver
>>> indicate the protocol explicitly. In any case, it looks like the
>>> driver needs code to override the protocol setting for this command.
>>> (Maybe we should just set the protocol override for what we know the
>>> command is supposed to be in all cases?)
>> ..
>>
>> If you can puzzle out how to do that, and post a quick(?) patch,
>> it would certainly make testing SSDs easier for me here.
>>
>> I would like to use the Sil3124 ExpressCard controller with my notebook
>> for this stuff, rather than having to power up one of the noisy
>> full-size systems under the table.  ;)
> 
> You can try this patch (totally untested) which basically just bludgeons it
> into doing what we want for all non-packet commands:
> 
> diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
> index 77aa8d7..e6946fc 100644
> --- a/drivers/ata/sata_sil24.c
> +++ b/drivers/ata/sata_sil24.c
> @@ -846,6 +846,17 @@ static void sil24_qc_prep(struct ata_queued_cmd *qc)
>  	if (!ata_is_atapi(qc->tf.protocol)) {
..

Mmm.. that test is failing.  I'll see if I can find where
those bits should be getting set.

Also, the driver now never recovers from a failed TRIM.
It just sits there retrying it over and over and over and over,
even after I unplug the drive, or unplug the controller.
But that might be old behaviour.  :)

Cheers

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Mark Lord]

Mark Lord wrote:
>
>> @@ -846,6 +846,17 @@ static void sil24_qc_prep(struct ata_queued_cmd *qc)
>>      if (!ata_is_atapi(qc->tf.protocol)) {
> ..
> 
> Mmm.. that test is failing.  I'll see if I can find where
> those bits should be getting set.
..

Pilot error.  Rebuilding the kernel again now.  :)

Thanks!

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Mark Lord]

Robert Hancock wrote:
> On 07/30/2009 06:54 AM, Mark Lord wrote:
>>> Yeah, according to the datasheet "The SiI3124 will decode the 8-bit ATA
>>> command at PRB offset 0x0a and automatically execute the default
>>> protocol for the command. In certain cases it might be desirable to
>>> specify a non-default protocol to be used, such as with vendor
>>> specific device commands." The DSM command seems to be DMA data-out
>>> and the chip likely doesn't know that command. I have to wonder why
>>> they decided to use that design instead of just making the driver
>>> indicate the protocol explicitly. In any case, it looks like the
>>> driver needs code to override the protocol setting for this command.
>>> (Maybe we should just set the protocol override for what we know the
>>> command is supposed to be in all cases?)
>> ..
>>
>> If you can puzzle out how to do that, and post a quick(?) patch,
>> it would certainly make testing SSDs easier for me here.
>>
>> I would like to use the Sil3124 ExpressCard controller with my notebook
>> for this stuff, rather than having to power up one of the noisy
>> full-size systems under the table.  ;)
> 
> You can try this patch (totally untested) which basically just bludgeons it
> into doing what we want for all non-packet commands:
> 
> diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
> index 77aa8d7..e6946fc 100644
> --- a/drivers/ata/sata_sil24.c
> +++ b/drivers/ata/sata_sil24.c
> @@ -846,6 +846,17 @@ static void sil24_qc_prep(struct ata_queued_cmd *qc)
>  	if (!ata_is_atapi(qc->tf.protocol)) {
>  		prb = &cb->ata.prb;
>  		sge = cb->ata.sge;
> +		if (ata_is_data(qc->tf.protocol)) {
> +			u16 prot = 0;
> +			ctrl = PRB_CTRL_PROTOCOL;
> +			if (ata_is_ncq(qc->tf.protocol))
> +				prot |= PRB_PROT_NCQ;
> +			if (qc->tf.flags & ATA_TFLAG_WRITE)
> +				prot |= PRB_PROT_WRITE;
> +			else
> +				prot |= PRB_PROT_READ;
> +			prb->prot = cpu_to_le16(prot);
> +		}
>  	} else {
>  		prb = &cb->atapi.prb;
>  		sge = cb->atapi.sge;
..

Okay, that patch works perfectly here on my Sil24 controller card.
I've run all sorts of commands through it now, including TRIM,
and everything works without any hitches or glitches.

Can we push this upstream for Jeff now ?

Thanks again!

Re: hdparm-9.17 released, with experimental trim/wiper scripts for SSDs

[Robert Hancock]

On Thu, Jul 30, 2009 at 1:30 PM, Mark Lord<liml@rtr.ca> wrote:
> Okay, that patch works perfectly here on my Sil24 controller card.
> I've run all sorts of commands through it now, including TRIM,
> and everything works without any hitches or glitches.
>
> Can we push this upstream for Jeff now ?
>
> Thanks again!

OK, I'll submit with sign-off.