[RFC] AHCI Command Completion Coalescing(CCC) proposal

[RFC] AHCI Command Completion Coalescing(CCC) proposal

[zhao, forrest]

Hello, all

0 Why this RFC?
Although AHCI spec 1.1 provides a detailed explanation about how to play
with CCC-related registers to enable CCC, several CCC-policy-related
parameters need to be defined(or the consensus need to be achieved)
before we start to write the code.

1 What is CCC used for?
As described in AHCI spec 1.1, "CCC is a feature designed to reduce the
interrupt and command completion overhead in a heavily loaded system.
The feature enables the number of interrupts taken per completion to be
reduced significantly, while ensuring a minimum quality of service for
command completions. When a software specified number of commands have
completed or a software specified timeout has expired, an interrupt is
generated by hardware to allow software to process completed commands."

2 When is CCC activated?
As stated above, CCC is useful only if the system is heavily loaded. So
CCC should be activated when the system is heavily loaded. Then the
question is how to determine whether the system is heavily-loaded or
not? In other words, how many interrupts generated per second can be
defined as "heavily-loaded system"? Does it make sense to define "1000
IRQs per second" as a heavily-loaded system?

3 What should the software specified number of commands be?
>From my understanding, the measurement of "IRQ numbers per second"
should be based on per-port instead of all ports of a SATA controller.
For NCQ, the usable command slots for each port is 31(the 32nd command
slot is reserved for internal command), so the software specified number
of commands should be 31*n (n is the number of ports, which is selected
to join CCC).
For non-NCQ, the usable command slots for each port is 32, so the
software specified number of commands should be 32*n.

4 What should the software specified timeout be?
I don't have the strong reasoning of a specific timeout value. 500ms? or
1000ms? We should trade-off between the delay and overhead.

5 When is CCC de-activated?
When the port becomes lightly-loaded, we should de-activate CCC of this
port. Otherwise the unnecessary delay would be introduced. However we
should not de-activate CCC of a port immediately when IRQ's per second
drops down the threshold in order to avoid jitter. My suggestion is that
if consecutive 3 timeout occurs, then we de-activate CCC of a port with
least "IRQ's per second".

Your comments are welcome.

Thanks,
Forrest

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Jeff Garzik]

zhao, forrest wrote:
> Hello, all
> 
> 0 Why this RFC?
> Although AHCI spec 1.1 provides a detailed explanation about how to play
> with CCC-related registers to enable CCC, several CCC-policy-related
> parameters need to be defined(or the consensus need to be achieved)
> before we start to write the code.

To brag a bit, I pushed Intel heavily for this feature, in the 
pre-AHCI-1.0 development days.


>>From my understanding, the measurement of "IRQ numbers per second"
> should be based on per-port instead of all ports of a SATA controller.

No, it should be all ports of a SATA controller.

If an interrupt arrives while CCC is active, we should take the 
opportunity to check all ports for activity -- as the standard code does 
now.


> 4 What should the software specified timeout be?
> I don't have the strong reasoning of a specific timeout value. 500ms? or
> 1000ms? We should trade-off between the delay and overhead.

500ms is a lot of latency.

	Jeff

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[zhao, forrest]

On Thu, 2006-06-08 at 11:01 -0400, Jeff Garzik wrote: 
> >>From my understanding, the measurement of "IRQ numbers per second"
> > should be based on per-port instead of all ports of a SATA controller.
> 
> No, it should be all ports of a SATA controller.

Maybe I didn't state my ideas clearly. Let me explain it by an example.

1 Assume there are 2 active ports(P0 and P1) on a system, they all run
under non-NCQ mode
2 During a certain period, P0 is heavily loaded, which generates >1000
interrupts per second; P1 is idle, which generates no interrupt
3
3.1 If the measurement of "IRQ numbers per second" is based on all
active ports of a SATA controller, CCC is activated by CCC_PORTS being
set to 0x3, CCC_CTL.CC being set to 64(32*2). Then the problem comes:
the CCC interrupt will be raised only when the timeout expires, this is
because P1 is in idle, thus hCccComplete can never be greater than or
equal to 64, the maximum of hCccComplete is 32.
3.2 If the measurement of "IRQ numbers per second" is based on per-port,
we can know that P0 is heavily-loaded, then CCC is activated by
CCC_PORTS being set to 0x1, CCC_CTL.CC being set to 32. Then CCC can
take effect as we have expected :)
NOTE: hCccComplete is the term used in section 11 of AHCI spec1.1

> > 4 What should the software specified timeout be?
> > I don't have the strong reasoning of a specific timeout value. 500ms? or
> > 1000ms? We should trade-off between the delay and overhead.
> 
> 500ms is a lot of latency.

I'll use 100ms as timeout value in the code.

Thanks,
Forrest

Another project for you... :)

[Jeff Garzik]

Forrest,

BTW, if you are looking for useful libata projects, it would really be 
nice to resurrect Randy Dunlap's SATA ACPI patches, update those for the 
current libata-dev.git#upstream, and get those in.

libata needs to execute the SATA taskfiles passed to us from ACPI BIOS 
tables, in order to properly set up the hard drive in a way the user 
expects (hard drive password, acoustic settings, etc.).  There should be 
a module option that allows the user to skip this step, and preserve 
current behavior.


Also, a feature Alan requests on occasion:  Call the ATA "set max" 
command to fully address the hard drive, including HPA.  The Linux 
standard is to export the raw hardware directly, making 100% of the 
hardware capability available to the user (and, in this case, 
Linux-based BIOS and recovery tools).

	Jeff

Re: Another project for you... :)

[zhao, forrest]

On Thu, 2006-06-08 at 23:11 -0400, Jeff Garzik wrote:
> Forrest,
> 
> BTW, if you are looking for useful libata projects, it would really be 
> nice to resurrect Randy Dunlap's SATA ACPI patches, update those for the 
> current libata-dev.git#upstream, and get those in.
> 
> libata needs to execute the SATA taskfiles passed to us from ACPI BIOS 
> tables, in order to properly set up the hard drive in a way the user 
> expects (hard drive password, acoustic settings, etc.).  There should be 
> a module option that allows the user to skip this step, and preserve 
> current behavior.

OK. I'll work on Randy's SATA ACPI patches first, then CCC.

> Also, a feature Alan requests on occasion:  Call the ATA "set max" 
> command to fully address the hard drive, including HPA.  The Linux 
> standard is to export the raw hardware directly, making 100% of the 
> hardware capability available to the user (and, in this case, 
> Linux-based BIOS and recovery tools).

I'll first study what this means, then start to work on it.

Jeff and Tejun,

Thank you very much for helping me get involved in libata development.
I'm very happy to make contribution to open source project :)

Best wishes,
Forrest

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Jeff Garzik]

zhao, forrest wrote:
> On Thu, 2006-06-08 at 11:01 -0400, Jeff Garzik wrote: 
>>> >From my understanding, the measurement of "IRQ numbers per second"
>>> should be based on per-port instead of all ports of a SATA controller.
>> No, it should be all ports of a SATA controller.
> 
> Maybe I didn't state my ideas clearly. Let me explain it by an example.
> 
> 1 Assume there are 2 active ports(P0 and P1) on a system, they all run
> under non-NCQ mode
> 2 During a certain period, P0 is heavily loaded, which generates >1000
> interrupts per second; P1 is idle, which generates no interrupt
> 3
> 3.1 If the measurement of "IRQ numbers per second" is based on all
> active ports of a SATA controller, CCC is activated by CCC_PORTS being
> set to 0x3, CCC_CTL.CC being set to 64(32*2). Then the problem comes:
> the CCC interrupt will be raised only when the timeout expires, this is
> because P1 is in idle, thus hCccComplete can never be greater than or
> equal to 64, the maximum of hCccComplete is 32.
> 3.2 If the measurement of "IRQ numbers per second" is based on per-port,
> we can know that P0 is heavily-loaded, then CCC is activated by
> CCC_PORTS being set to 0x1, CCC_CTL.CC being set to 32. Then CCC can
> take effect as we have expected :)
> NOTE: hCccComplete is the term used in section 11 of AHCI spec1.1

I'm still not sure I follow you?

When AHCI runs out of commands to execute, it transitions from H:Idle to 
Ccc:SetIS.

IMPORTANT NOTE:  In order for CCC to be effective on AHCI, ahci.c and 
libata (and sata_sil24) must be updated to support queuing a list of 
non-NCQ commands onto the controller, and recovering from errors in the 
case where a command list full of non-NCQ commands is present.

Also, you should scale up CCC_CTL.CC really based on "commands in 
flight" across all ports.

	Jeff

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[zhao, forrest]

On Thu, 2006-06-08 at 23:30 -0400, Jeff Garzik wrote:
> When AHCI runs out of commands to execute, it transitions from H:Idle to 
> Ccc:SetIS.
> 

Oh, that's my misunderstanding of the spec. Thanks for your patience.

Forrest

[RFC] ATA host-protected area (HPA) device mapper?

[Jeff Garzik]

As I just mentioned on linux-ide in another email:
libata should -- like drivers/ide -- call the ATA "set max" command to 
fully address the hard drive, including the special "host-protected 
area" (HPA).  We should do this because the Linux standard is to export 
the raw hardware directly, making 100% of the hardware capability 
available to the user (and, in this case, Linux-based BIOS and recovery 
tools).

However, there are rare bug reports and general paranoia related to 
presenting 100% of the ATA hard drive "native" space, rather than the 
possibly-smaller space that the BIOS chose to present to the user.

My thinking is that [someone] should create an optional, ATA-specific 
device mapper module.  This module would layer on top of an ATA block 
device, and present two block devices:  the BIOS-presented space, and 
the HPA.

Such a module would make it trivial for users to ensure that partition 
tables and RAID metadata formats know what the BIOS (rather than 
underlying hard drive) considers to be end-of-disk.

Comments?  Questions?  Am I completely insane?  ;-)

	Jeff

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Tejun Heo]

Jeff Garzik wrote:

> I'm still not sure I follow you?
> 
> When AHCI runs out of commands to execute, it transitions from H:Idle to 
> Ccc:SetIS.
> 
> IMPORTANT NOTE:  In order for CCC to be effective on AHCI, ahci.c and 
> libata (and sata_sil24) must be updated to support queuing a list of 
> non-NCQ commands onto the controller, and recovering from errors in the 
> case where a command list full of non-NCQ commands is present.

I thought about it but am not really sure whether it's worth the 
trouble.  We'll be saving on inter-command latency and interrupt 
handling which is great but not so sure how noticeable the improvement 
would be.  NCQ is already all around.  How about doing CCC only during 
NCQ command phase?

Hmmm... maybe those SSDs would benefit from CCC during non-NCQ commands 
though if they don't support NCQ, which they don't really need.

-- 
tejun

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Tejun Heo]

Tejun Heo wrote:
> Jeff Garzik wrote:
> 
>> I'm still not sure I follow you?
>>
>> When AHCI runs out of commands to execute, it transitions from H:Idle 
>> to Ccc:SetIS.
>>
>> IMPORTANT NOTE:  In order for CCC to be effective on AHCI, ahci.c and 
>> libata (and sata_sil24) must be updated to support queuing a list of 
>> non-NCQ commands onto the controller, and recovering from errors in 
>> the case where a command list full of non-NCQ commands is present.
> 
> I thought about it but am not really sure whether it's worth the 
> trouble.  We'll be saving on inter-command latency and interrupt 
> handling which is great but not so sure how noticeable the improvement 
> would be.  NCQ is already all around.  How about doing CCC only during 
> NCQ command phase?
> 
> Hmmm... maybe those SSDs would benefit from CCC during non-NCQ commands 
> though if they don't support NCQ, which they don't really need.
> 

If we're gonna do it.  EH needs only a few changes probably during 
autopsy and report.  Fixing up command issue path and implementing 
command exclusion (NCQ vs. non-NCQ, sil24 does it in hardware, ahci 
doesn't) will be a bit complex though.

-- 
tejun

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[zhao, forrest]

On Fri, 2006-06-09 at 12:47 +0900, Tejun Heo wrote:
> If we're gonna do it.  EH needs only a few changes probably during 
> autopsy and report.  Fixing up command issue path and implementing 
> command exclusion (NCQ vs. non-NCQ, sil24 does it in hardware, ahci 
> doesn't) will be a bit complex though.

Would you please elaborate on command exclusion? Why NCQ commands need
to be excluded from non-NCQ commands?

Thanks,
Forrest

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Jeff Garzik]

Tejun Heo wrote:
> Jeff Garzik wrote:
> 
>> I'm still not sure I follow you?
>>
>> When AHCI runs out of commands to execute, it transitions from H:Idle 
>> to Ccc:SetIS.
>>
>> IMPORTANT NOTE:  In order for CCC to be effective on AHCI, ahci.c and 
>> libata (and sata_sil24) must be updated to support queuing a list of 
>> non-NCQ commands onto the controller, and recovering from errors in 
>> the case where a command list full of non-NCQ commands is present.
> 
> I thought about it but am not really sure whether it's worth the 
> trouble.  We'll be saving on inter-command latency and interrupt 
> handling which is great but not so sure how noticeable the improvement 
> would be.  NCQ is already all around.  How about doing CCC only during 
> NCQ command phase?

Agreed with all these observations.

In general, the non-NCQ case shares characteristics with host-queue 
controllers like sx8.  The benefit is nowhere near true command queueing 
like NCQ, but the benefits (which you list) are real.

Think of this as a long term requirement.  libata _should_ eventually 
support this, simply because the gains are there to be had.  Several 
SATA controllers support queueing of non-NCQ commands.

In the short term, only turning on CCC for NCQ ports makes a lot of sense.


> Hmmm... maybe those SSDs would benefit from CCC during non-NCQ commands 
> though if they don't support NCQ, which they don't really need.

My Gigabyte i-Ram reliably corrupts data within seconds, regardless of 
SATA controller :/

	Jeff

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Jeff Garzik]

Tejun Heo wrote:
> If we're gonna do it.  EH needs only a few changes probably during 
> autopsy and report.  Fixing up command issue path and implementing 
> command exclusion (NCQ vs. non-NCQ, sil24 does it in hardware, ahci 
> doesn't) will be a bit complex though.

FWIW:  Promise, Marvell, and ServerWorks can all queue non-NCQ commands.

Oh yeah, ADMA too, IIRC.

	Jeff

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Jeff Garzik]

zhao, forrest wrote:
> On Fri, 2006-06-09 at 12:47 +0900, Tejun Heo wrote:
>> If we're gonna do it.  EH needs only a few changes probably during 
>> autopsy and report.  Fixing up command issue path and implementing 
>> command exclusion (NCQ vs. non-NCQ, sil24 does it in hardware, ahci 
>> doesn't) will be a bit complex though.
> 
> Would you please elaborate on command exclusion? Why NCQ commands need
> to be excluded from non-NCQ commands?

For one thing, getting the command ordering right (particularly with 
regards to barriers) isn't trivial...  also, you need exclusion for 
major feature changes like set xfer mode.

	Jeff

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Matthew Frost]

Jeff Garzik wrote:
> As I just mentioned on linux-ide in another email:
> libata should -- like drivers/ide -- call the ATA "set max" command to 
> fully address the hard drive, including the special "host-protected 
> area" (HPA).  We should do this because the Linux standard is to export 
> the raw hardware directly, making 100% of the hardware capability 
> available to the user (and, in this case, Linux-based BIOS and recovery 
> tools).
> 

Yay for exposing absolute potential functionality; yay for recognizing 
the havok possible, and proposing strategies for channeling that 
possibility.

> However, there are rare bug reports and general paranoia related to 
> presenting 100% of the ATA hard drive "native" space, rather than the 
> possibly-smaller space that the BIOS chose to present to the user.
> 

I've grepped through several old discussions of HPA handling, and it 
doesn't seem like everyone has the same idea of exactly what this will 
do, possibly because of the delta in BIOS behavior over original design 
restrictions.

> My thinking is that [someone] should create an optional, ATA-specific 
> device mapper module.  This module would layer on top of an ATA block 
> device, and present two block devices:  the BIOS-presented space, and 
> the HPA.
> 
> Such a module would make it trivial for users to ensure that partition 
> tables and RAID metadata formats know what the BIOS (rather than 
> underlying hard drive) considers to be end-of-disk.
> 
> Comments?  Questions?  Am I completely insane?  ;-)
> 

Tools with which to lay waste to systems, or save them.

What I like about your proposal is that it doesn't go back to "Do we 
blow away the HPA or reserve it?"; you suggest conserving both options. 
  Make the kernel aware of the existence of the HPA, and thereby the 
whole capacity of the disk, and simultaneously of what it should see and 
expose for usage 'safely'.  Doesn't sound insane to me; it sounds like 
you're planning on [having someone] teach the kernel to respect the 
actual disk limitations.

Whether the implementation will be sane ... 'nother story.  :)  Thence 
the question of teaching userspace to sanely use what is exposed, though 
if the 'old' (non-HPA) space is presented, it shouldn't be a hard 
reorientation.  Would we be talking about a new sysfs entry parallel to 
the existing information?  If I understand it right -- and I might not 
-- the HPA doesn't get included in the partitioning schemes, because it 
is protected.  Even nuking the disk will/should bypass it.  So the 
system will tend to ignore it under normal conditions, until you decide 
to get fancy and trip over its shadow.  So making the kernel aware that 
this disk has this spot that must be respected should be a no-brainer. 
What better way to make the kernel aware of it, than by acknowledging it 
as a block device among other block devices?  It just needs a good 
molly-guard to cover the respect portion of the problem.

Of course, I don't hack ATA, so my opinions may have limited validity 
after a certain level of specificity.  I can always be enlightened as to 
why you really are insane.  ;)

>     Jeff
> 

Matt

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Tejun Heo]

zhao, forrest wrote:
> On Fri, 2006-06-09 at 12:47 +0900, Tejun Heo wrote:
>> If we're gonna do it.  EH needs only a few changes probably during 
>> autopsy and report.  Fixing up command issue path and implementing 
>> command exclusion (NCQ vs. non-NCQ, sil24 does it in hardware, ahci 
>> doesn't) will be a bit complex though.
> 
> Would you please elaborate on command exclusion? Why NCQ commands need
> to be excluded from non-NCQ commands?

AHCI spec rev 1.1, sect 1.7.  The last paragraph says.

"This multiple-use of the command list is achieved by the HBA only 
moving its command list pointer when the BSY, DRQ, and ERR bits are 
cleared by the device. System software is responsible to ensure that 
queued and non-queued commands are not mixed in the command list."

-- 
tejun

[RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

Jeff Garzik wrote:
> libata should -- like drivers/ide -- call the ATA "set max" command to 
> fully address the hard drive, including the special "host-protected 
> area" (HPA).  We should do this because the Linux standard is to export 
> the raw hardware directly, making 100% of the hardware capability 
> available to the user (and, in this case, Linux-based BIOS and recovery 
> tools).
> .....
> Comments?  Questions?  Am I completely insane?  ;-)

  Your hard disk is a lot more powerfull than what you think, only very old
 hard disks only have ATA set max command. Nowadays, you can not only set the
 maximum size of the hard disk by HPA, but you can also protect the change of
 the HPA by password and even freeze the HPA (i.e. unmodifiable without power
 cycle).
  Changing the accessible size of the disk using the HPA is relatively safe
 because the disk is still reporting it complete size, so the BIOS and Linux
 do not have disks changing their size during use.

  You can also read and write the configuration of the hard disk, and so hide
 the fact that your hard disk has the HPA feature, change the real size of
 the hard disk (so hide the end, and none of Linux or the BIOS will know the
 hidden part) independantly of the HPA, make the HPA area appear as a complete
 hard disk by offseting the LBA (for a safety recovery), manage the noise
 level and protect the content of the disk by password.

  All this is documented in the ATA specification, available for free at least
 at http://www.t13.org/. If you want a (GPL only) source code showing how to
 use it, have a look at the Gujin bootloader at http://gujin.org : in the way
 I am using it, the bootloader installs itself into an HPA and keep a copy of
 the current MBR in the HPA so is nearly indestructible, if the MBR is
 overwritten another floppy or CDROM booting with Gujin will propose to restore
 the MBR. You may want to run a DOS floppy, run the gujin provided dbgdisk.exe,
 go to the setup menu and enable IDE probing, go back to the kernel selection,
 and exit the dbgdisk.exe software by ^C. Then, have a look at the "A:\DBG" file
 created to see what your hard disk is reporting.
 Gujin can also install in a partition if there is no unused space after your
 extended partition at the end of the disk to create a HPA.

 Gujin also do the absolutely needed setup of the IDE hard disk which is to freeze
 the password system _and_ the config system of all the IDE hard disks present, so
 that no virus can put a random password and send you an E-mail with the address
 where to send the money to get the password to unlock the hard disk and so access
 again your data. Again, freezing means no more modifiable until next power cycle,
 so IMO it is the job of the bootloader to setup the hard disk, before running
 anything like Linux, a commercial OS, a bootable CDROM...

 Gujin is assuming that your hard disk are accessible by the documented ATA ide
 system, and some (or all?) IDE SATA interface have (volumtary?) broken
 implementation: they are not IDE register compatible.
 If you buy broken hardware, Gujin will not help you and cannot take care of the
 details for you - it is another win{modem,video,sensor} device.

 Etienne.

__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Jens Axboe]

On Fri, Jun 09 2006, Tejun Heo wrote:
> zhao, forrest wrote:
> >On Fri, 2006-06-09 at 12:47 +0900, Tejun Heo wrote:
> >>If we're gonna do it.  EH needs only a few changes probably during 
> >>autopsy and report.  Fixing up command issue path and implementing 
> >>command exclusion (NCQ vs. non-NCQ, sil24 does it in hardware, ahci 
> >>doesn't) will be a bit complex though.
> >
> >Would you please elaborate on command exclusion? Why NCQ commands need
> >to be excluded from non-NCQ commands?
> 
> AHCI spec rev 1.1, sect 1.7.  The last paragraph says.
> 
> "This multiple-use of the command list is achieved by the HBA only 
> moving its command list pointer when the BSY, DRQ, and ERR bits are 
> cleared by the device. System software is responsible to ensure that 
> queued and non-queued commands are not mixed in the command list."

This, btw, was also the case with the legacy TCQ.

-- 
Jens Axboe

Re: [RFC] AHCI Command Completion Coalescing(CCC) proposal

[Jens Axboe]

On Fri, Jun 09 2006, Tejun Heo wrote:
> Jeff Garzik wrote:
> 
> >I'm still not sure I follow you?
> >
> >When AHCI runs out of commands to execute, it transitions from H:Idle to 
> >Ccc:SetIS.
> >
> >IMPORTANT NOTE:  In order for CCC to be effective on AHCI, ahci.c and 
> >libata (and sata_sil24) must be updated to support queuing a list of 
> >non-NCQ commands onto the controller, and recovering from errors in the 
> >case where a command list full of non-NCQ commands is present.
> 
> I thought about it but am not really sure whether it's worth the 
> trouble.  We'll be saving on inter-command latency and interrupt 
> handling which is great but not so sure how noticeable the improvement 
> would be.  NCQ is already all around.  How about doing CCC only during 
> NCQ command phase?

Fully agree, doing CCC on non-ncq sounds like a very silly thing to do.
Ugly complexity for very little (and questionable) gain.

I'm not a big fan of CCC in generel, to me it seems to have bigger
potential to cause you latency than save you interrupt processing time.
I'm not saying there aren't cases where CCC would be a win, but I see a
lot more cases where it definitely wont be. And it's a classic case of
having to implement policy, so it's surely get it wrong here and there.

-- 
Jens Axboe

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Jeff Garzik]

Etienne Lorrain wrote:
>   Your hard disk is a lot more powerfull than what you think, only very old

No, it's not.  I am well aware of what's in the ATA spec.


>  hard disks only have ATA set max command. Nowadays, you can not only set the

Not true.


>  Gujin also do the absolutely needed setup of the IDE hard disk which is to freeze
>  the password system _and_ the config system of all the IDE hard disks present, so
>  that no virus can put a random password and send you an E-mail with the address
>  where to send the money to get the password to unlock the hard disk and so access
>  again your data. Again, freezing means no more modifiable until next power cycle,
>  so IMO it is the job of the bootloader to setup the hard disk, before running
>  anything like Linux, a commercial OS, a bootable CDROM...

This is totally broken, and I am going to strongly recommend that no one 
use this software.

It is the OS responsibility to do this.  As a simple example, when the 
libata ACPI patches are merged (soon), libata will send BIOS-specified 
taskfiles to the device -- including the hard drive password, if any. 
Then it will freeze the settings.

Gujin's behavior will prevent the user from accessing their data, if 
they have protected it via BIOS.


>  Gujin is assuming that your hard disk are accessible by the documented ATA ide
>  system, and some (or all?) IDE SATA interface have (volumtary?) broken
>  implementation: they are not IDE register compatible.

More evidence that Gujin is completely broken.

Host controller programming interfaces have _always_ been variable.  PCI 
IDE standard was never a requirement for all host controllers, indeed 
such a requirement would be stupid, and widely ignored.

Modern SATA controllers are all FIS-based, and are not (and should not 
be) limited by the legacy IDE register programming interface.

	Jeff

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Alan Cox]

>  Gujin is assuming that your hard disk are accessible by the documented ATA ide
>  system, and some (or all?) IDE SATA interface have (volumtary?) broken
>  implementation: they are not IDE register compatible.

SFF was never a formal standard, and ST-506 was a random vendor
interface copying exercise that caught on. ACPI permits the firmware to
provide ATA taskfiles but afaik not the boot loader unfortunately.

A lot of newer SATA hardware uses a common standard defined interface
called AHCI, and it appears most vendors are migrating in the direction
of using AHCI. If so then we are in the same kind of flux as the VLB
world before SFF and PCI settled the standard interfaces down for a
while.

Don't see how your HPA code protects versus password locking however. If
I hack the OS I write a new boot block which locks the disk then reboot
into it. By the time you go for your floppy its too late.

If thats not the case I'm most interested how you set it up to avoid
this.

Alan

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

--- XXXXXXXX wrote:
> It tends to be preferred around here to implement hardware features so 
> that hardware actually works, rather than to implement spec and blame 
> the hardware for failing to work to the spec.

 Well, the IDE hardware works, and conform to the ATA specs.
 Gujin is using those ATA command for quite a long time.
 I just have a problem with SATA chipset not being ATA1-7 compatible.

>  If, as you say, most BIOS 
> don't obey the spec, then implementing a spec that bears little 
> resemblance to actual behavior only helps the small number of 
> spec-compliant implementations.

  The main problem for BIOS manufacturer is the keyboard mapping
 recognition to enter the password. And there is no real point
 in freezing the password system of a locked drive.
  Gujin try to solve most of those problems, but because it is
 located on the hard disk itself limit the choice - it would be
 so nice to have a big enough BIOS FLASH and put Gujin there.
 Second best choice is having two hard disks (compact flash as
 first HD with compact flash <-> IDE adapter).

> >   The complete ATA specification is describing a register interface since ATA1,
> >  SATA chipsets should be software compatible, even if they add commands/interfaces:
> > http://www.sata-io.org/interopfaq.asp    say:
> >     Are there any known interoperability issues with SATA?
> >     One of the primary requirements of the SATA 1.0 specification was to
> >     maintain backward compatibility with existing operating system drivers
> >     to eliminate incompatibility issues.
> > 
> 
> Again with the "The spec says it works this way, so I can do it this 
> way."  I realize that you can be perfectly, legally, safe implementing 
> specified behavior, but it leaves you doing what you are doing here, 
> which is blaming the hardware for not living up to the specification. 

  Well the ATA specs are quite old, Gujin still work with the BIOS hard disk
 so still act as a bootloader, but it can not protecting you against a
 1 milisecond attack setting password.
 By the way the SATA specs are not free.

> It is generally the choice in the linux kernel to make the hardware 
> work, and ignore the specification where it is irrelevant to actual 
> functionality.  Read up on the SAS Transport Layer.

  Oh yes, Gujin has some strange code to make things work at the end,
 look for ATAPI stuff - I can and do ignore the specs to make the software
 work on real hardware.

  Cheers,
  Etienne.



__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: Another project for you... :)

[Greg Freemyer]

> > Also, a feature Alan requests on occasion:  Call the ATA "set max"
> > command to fully address the hard drive, including HPA.  The Linux
> > standard is to export the raw hardware directly, making 100% of the
> > hardware capability available to the user (and, in this case,
> > Linux-based BIOS and recovery tools).
>
> I'll first study what this means, then start to work on it.
>
> Best wishes,
> Forrest

The below relates to PATA and HPA (Host Protected Area), but I assume
it would be relevant to SATA as well.

====
This has been discussed before in the archives.  I think the desired
behavior was to have the Linux Kernel look for HPAs by default and to
open the drive up to the full size via "set max" on boot, but to have
a boot parameter to disable this behavior if desired.

FYI: There is a userland tool called setmax floating around.  (I can
find if you're interested.)  The userland tool works fine for LBA28
drives, but does not have LBA48 support.  That may get to be an issue,
but at least from my perspective, I have not yet seen any 128+ GiB
drives with HPAs setup.  I do occasionally see smaller drives with
HPAs.

FYI2: I do computer forensics and make dd images of drives routinely.
As part of the process I double check if a HPA is present or not.  I
typically use the userland setmax tool to open it up if the linux
kernel failed to do so on bootup for some reason.  (I've only seen
that a few times so far.)

Greg
-- 
Greg Freemyer
The Norcross Group
Forensics for the 21st Century

Re: Another project for you... :)

[Alan Cox]

Ar Gwe, 2006-06-09 am 18:29 -0400, ysgrifennodd Greg Freemyer:
> behavior was to have the Linux Kernel look for HPAs by default and to
> open the drive up to the full size via "set max" on boot, but to have
> a boot parameter to disable this behavior if desired.

It doesn't really work very well, even less so when you hit SATA and
have hot plug disks. HPA on which disks, inserted when.

Thus it needs, as Jeff says, to be cleanly exposed dynamically at
runtime.

Alan

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

Alan wrote:
> Don't see how your HPA code protects versus password locking however. If
> I hack the OS I write a new boot block which locks the disk then reboot
> into it. By the time you go for your floppy its too late.
> If thats not the case I'm most interested how you set it up to avoid this.

 I was a bit too quick to answer this one yesterday.
 Basically, you want to protect against an attack replacing the bootloader,
 whatever happens later is bad anyway.
 The only way to get that is if the bootloader cannot be overwritten,
 preferably even by root, at the time of the attack.
 You have to know that whatever put Gujin in memory is independant of how
 it will probe the disks, partitions and files.

 The simple solution is: you never boot from the hard disk, but from a
 physically write protected device (write protected floppy, non writeable
 CD or CDRW in a non writing CDROM drive, USB thumb drive with a physical
 write protect switch). It may be interresting to be able to enable
 writing to the boot device at setup (for instance to select a kernel
 to "autoboot" after few seconds or set up the keyboard language and
 the default kernel command line), Gujin can itself save few data in
 its second 512 bytes sector if write is enabled, but that is not
 100 % needed because the installer "instboot" can set all those
 parameter in file "boot.bcd" for CDROM install.
 Note that, if you are using a "write once" CDROM, you should take care
 of not enabling an attacker to add a session with a new El-torito
 boot sector.

 The complex solution is described in the ATA standard T13 D1367,
 Protect Area Run Time Interface Extension Services, skip to "4 Overview",
 and depends on the hard drive supporting the "Address Offset Reserved Area Boot",
 only IBM drives seem to have such a support.
 Basically, at boot, the LBA is offseted by a constant and you boot this
 "minidisk" with its own MBR. You can then send a command to the HD
 to stop this offset, and protect your "minidisk" by the HPA or by
 reducing the visible size of the disk (latter hurts the BIOS).
 The real MBR is never executed, the executed MBR is write protected
 even by root at the time of the attack, and if you boot with a DOS floppy
 you will see a small hard disk with the really used MBR.
 Gujin is designed to produce a complete disk mapping with FAT12/16
 filesystem (no partition like a floppy) to be able to handle such a boot,
 but I did not finish to support it, mainly because the only IBM drive
 I have is my main one, and because my hobby time is limited (I am not
 working in the PC industry). It should not be difficult to support
 that completely, the details are how to upgrade Gujin, how to behave
 with non booted but present disks...

 If you are using one of these two solutions, an attacker will have to
 take control of the system before Gujin is executed - so the only
 attack is either the main BIOS FLASH (check that you have a jumper
 to write protect it physically) and other PCI/AGP BIOS (video BIOS,
 network BIOS) which may be present on FLASH and may be overwritten.
 Modifying the firmware of the HD may be another attack path.

 All this remember me the software write protection of 5 1/4 inch floppy...
 Also, never try to test IDE passwords at 2:00 in the morning, even for
 a quick test, you may type a random password and get a 160GB brick.
 Someone knows the master password of Maxtor 4G160J8 code GAK819K0
 drive S/N G80CNBME FG08A - I promise not to repeat it!

 Cheers,
 Etienne.

__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Arjan van de Ven]

>  The simple solution is: you never boot from the hard disk, but from a
>  physically write protected device (write protected floppy, non writeable
>  CD or CDRW in a non writing CDROM drive, USB thumb drive with a physical
>  write protect switch).


that's not totally fool proof either; after all an attacker can adjust
the nvram to change the boot device order.

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

--- Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> If I hack the OS I write a new boot block which locks the disk then reboot into it.

 Another solution to be safe, because you need a full power cycle of the HD,
 is to keep the HDs on a separate non interruptible power supply (not stoppable
 by APM/ACPI).

__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: Another project for you... :)

[zhao, forrest]

On Thu, 2006-06-08 at 23:11 -0400, Jeff Garzik wrote:
> Forrest,
> 
> BTW, if you are looking for useful libata projects, it would really be 
> nice to resurrect Randy Dunlap's SATA ACPI patches, update those for the 
> current libata-dev.git#upstream, and get those in.

Randy,

Could you confirm if your latest SATA-ACPI patch is at this URL?
http://www.xenotime.net/linux/SATA/2.6.16-rc4/libata-rollup-2616-rc4.patch

Thanks,
Forrest

Re: Another project for you... :)

[Randy.Dunlap]

On Wed, 14 Jun 2006 16:01:54 +0800 zhao, forrest wrote:

> On Thu, 2006-06-08 at 23:11 -0400, Jeff Garzik wrote:
> > Forrest,
> > 
> > BTW, if you are looking for useful libata projects, it would really be 
> > nice to resurrect Randy Dunlap's SATA ACPI patches, update those for the 
> > current libata-dev.git#upstream, and get those in.
> 
> Randy,
> 
> Could you confirm if your latest SATA-ACPI patch is at this URL?
> http://www.xenotime.net/linux/SATA/2.6.16-rc4/libata-rollup-2616-rc4.patch

Yes, correct.

---
~Randy

Re: Another project for you... :)

[zhao, forrest]

On Wed, 2006-06-14 at 08:19 -0700, Randy.Dunlap wrote:
> On Wed, 14 Jun 2006 16:01:54 +0800 zhao, forrest wrote:
> 
> > On Thu, 2006-06-08 at 23:11 -0400, Jeff Garzik wrote:
> > > Forrest,
> > > 
> > > BTW, if you are looking for useful libata projects, it would really be 
> > > nice to resurrect Randy Dunlap's SATA ACPI patches, update those for the 
> > > current libata-dev.git#upstream, and get those in.
> > 
> > Randy,
> > 
> > Could you confirm if your latest SATA-ACPI patch is at this URL?
> > http://www.xenotime.net/linux/SATA/2.6.16-rc4/libata-rollup-2616-rc4.patch
> 
> Yes, correct.

According to ACPI spec 3.0, _GTM and _STM are IDE-only objects, _SDD is
SATA-only object. And in your patch you used field "legacy_mode" of
"struct ata_probe_ent" in order to distinguish between IDE and SATA.
But after reading the code of ata_pci_init_one(), I found that
"legacy_mode" is used to distinguish between legacy mode and native PCI
mode of IDE controller. Am I right? Or did I miss anything?
If I'm right, I'll fix it during the porting of your patch.

Thanks,
Forrest

Re: Another project for you... :)

[Jeff Garzik]

zhao, forrest wrote:
> According to ACPI spec 3.0, _GTM and _STM are IDE-only objects, _SDD is
> SATA-only object. And in your patch you used field "legacy_mode" of
> "struct ata_probe_ent" in order to distinguish between IDE and SATA.
> But after reading the code of ata_pci_init_one(), I found that
> "legacy_mode" is used to distinguish between legacy mode and native PCI
> mode of IDE controller. Am I right? Or did I miss anything?
> If I'm right, I'll fix it during the porting of your patch.

You are correct, you should use cbl==SATA or ATA_FLAG_SATA to check for 
SATA.  legacy-vs-native is not an accurate check.

	Jeff