[RFC] ATA host-protected area (HPA) device mapper?

[RFC] ATA host-protected area (HPA) device mapper?

[Jeff Garzik]

As I just mentioned on linux-ide in another email:
libata should -- like drivers/ide -- call the ATA "set max" command to 
fully address the hard drive, including the special "host-protected 
area" (HPA).  We should do this because the Linux standard is to export 
the raw hardware directly, making 100% of the hardware capability 
available to the user (and, in this case, Linux-based BIOS and recovery 
tools).

However, there are rare bug reports and general paranoia related to 
presenting 100% of the ATA hard drive "native" space, rather than the 
possibly-smaller space that the BIOS chose to present to the user.

My thinking is that [someone] should create an optional, ATA-specific 
device mapper module.  This module would layer on top of an ATA block 
device, and present two block devices:  the BIOS-presented space, and 
the HPA.

Such a module would make it trivial for users to ensure that partition 
tables and RAID metadata formats know what the BIOS (rather than 
underlying hard drive) considers to be end-of-disk.

Comments?  Questions?  Am I completely insane?  ;-)

	Jeff

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Matthew Frost]

Jeff Garzik wrote:
> As I just mentioned on linux-ide in another email:
> libata should -- like drivers/ide -- call the ATA "set max" command to 
> fully address the hard drive, including the special "host-protected 
> area" (HPA).  We should do this because the Linux standard is to export 
> the raw hardware directly, making 100% of the hardware capability 
> available to the user (and, in this case, Linux-based BIOS and recovery 
> tools).
> 

Yay for exposing absolute potential functionality; yay for recognizing 
the havok possible, and proposing strategies for channeling that 
possibility.

> However, there are rare bug reports and general paranoia related to 
> presenting 100% of the ATA hard drive "native" space, rather than the 
> possibly-smaller space that the BIOS chose to present to the user.
> 

I've grepped through several old discussions of HPA handling, and it 
doesn't seem like everyone has the same idea of exactly what this will 
do, possibly because of the delta in BIOS behavior over original design 
restrictions.

> My thinking is that [someone] should create an optional, ATA-specific 
> device mapper module.  This module would layer on top of an ATA block 
> device, and present two block devices:  the BIOS-presented space, and 
> the HPA.
> 
> Such a module would make it trivial for users to ensure that partition 
> tables and RAID metadata formats know what the BIOS (rather than 
> underlying hard drive) considers to be end-of-disk.
> 
> Comments?  Questions?  Am I completely insane?  ;-)
> 

Tools with which to lay waste to systems, or save them.

What I like about your proposal is that it doesn't go back to "Do we 
blow away the HPA or reserve it?"; you suggest conserving both options. 
  Make the kernel aware of the existence of the HPA, and thereby the 
whole capacity of the disk, and simultaneously of what it should see and 
expose for usage 'safely'.  Doesn't sound insane to me; it sounds like 
you're planning on [having someone] teach the kernel to respect the 
actual disk limitations.

Whether the implementation will be sane ... 'nother story.  :)  Thence 
the question of teaching userspace to sanely use what is exposed, though 
if the 'old' (non-HPA) space is presented, it shouldn't be a hard 
reorientation.  Would we be talking about a new sysfs entry parallel to 
the existing information?  If I understand it right -- and I might not 
-- the HPA doesn't get included in the partitioning schemes, because it 
is protected.  Even nuking the disk will/should bypass it.  So the 
system will tend to ignore it under normal conditions, until you decide 
to get fancy and trip over its shadow.  So making the kernel aware that 
this disk has this spot that must be respected should be a no-brainer. 
What better way to make the kernel aware of it, than by acknowledging it 
as a block device among other block devices?  It just needs a good 
molly-guard to cover the respect portion of the problem.

Of course, I don't hack ATA, so my opinions may have limited validity 
after a certain level of specificity.  I can always be enlightened as to 
why you really are insane.  ;)

>     Jeff
> 

Matt

[RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

Jeff Garzik wrote:
> libata should -- like drivers/ide -- call the ATA "set max" command to 
> fully address the hard drive, including the special "host-protected 
> area" (HPA).  We should do this because the Linux standard is to export 
> the raw hardware directly, making 100% of the hardware capability 
> available to the user (and, in this case, Linux-based BIOS and recovery 
> tools).
> .....
> Comments?  Questions?  Am I completely insane?  ;-)

  Your hard disk is a lot more powerfull than what you think, only very old
 hard disks only have ATA set max command. Nowadays, you can not only set the
 maximum size of the hard disk by HPA, but you can also protect the change of
 the HPA by password and even freeze the HPA (i.e. unmodifiable without power
 cycle).
  Changing the accessible size of the disk using the HPA is relatively safe
 because the disk is still reporting it complete size, so the BIOS and Linux
 do not have disks changing their size during use.

  You can also read and write the configuration of the hard disk, and so hide
 the fact that your hard disk has the HPA feature, change the real size of
 the hard disk (so hide the end, and none of Linux or the BIOS will know the
 hidden part) independantly of the HPA, make the HPA area appear as a complete
 hard disk by offseting the LBA (for a safety recovery), manage the noise
 level and protect the content of the disk by password.

  All this is documented in the ATA specification, available for free at least
 at http://www.t13.org/. If you want a (GPL only) source code showing how to
 use it, have a look at the Gujin bootloader at http://gujin.org : in the way
 I am using it, the bootloader installs itself into an HPA and keep a copy of
 the current MBR in the HPA so is nearly indestructible, if the MBR is
 overwritten another floppy or CDROM booting with Gujin will propose to restore
 the MBR. You may want to run a DOS floppy, run the gujin provided dbgdisk.exe,
 go to the setup menu and enable IDE probing, go back to the kernel selection,
 and exit the dbgdisk.exe software by ^C. Then, have a look at the "A:\DBG" file
 created to see what your hard disk is reporting.
 Gujin can also install in a partition if there is no unused space after your
 extended partition at the end of the disk to create a HPA.

 Gujin also do the absolutely needed setup of the IDE hard disk which is to freeze
 the password system _and_ the config system of all the IDE hard disks present, so
 that no virus can put a random password and send you an E-mail with the address
 where to send the money to get the password to unlock the hard disk and so access
 again your data. Again, freezing means no more modifiable until next power cycle,
 so IMO it is the job of the bootloader to setup the hard disk, before running
 anything like Linux, a commercial OS, a bootable CDROM...

 Gujin is assuming that your hard disk are accessible by the documented ATA ide
 system, and some (or all?) IDE SATA interface have (volumtary?) broken
 implementation: they are not IDE register compatible.
 If you buy broken hardware, Gujin will not help you and cannot take care of the
 details for you - it is another win{modem,video,sensor} device.

 Etienne.

__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Jeff Garzik]

Etienne Lorrain wrote:
>   Your hard disk is a lot more powerfull than what you think, only very old

No, it's not.  I am well aware of what's in the ATA spec.


>  hard disks only have ATA set max command. Nowadays, you can not only set the

Not true.


>  Gujin also do the absolutely needed setup of the IDE hard disk which is to freeze
>  the password system _and_ the config system of all the IDE hard disks present, so
>  that no virus can put a random password and send you an E-mail with the address
>  where to send the money to get the password to unlock the hard disk and so access
>  again your data. Again, freezing means no more modifiable until next power cycle,
>  so IMO it is the job of the bootloader to setup the hard disk, before running
>  anything like Linux, a commercial OS, a bootable CDROM...

This is totally broken, and I am going to strongly recommend that no one 
use this software.

It is the OS responsibility to do this.  As a simple example, when the 
libata ACPI patches are merged (soon), libata will send BIOS-specified 
taskfiles to the device -- including the hard drive password, if any. 
Then it will freeze the settings.

Gujin's behavior will prevent the user from accessing their data, if 
they have protected it via BIOS.


>  Gujin is assuming that your hard disk are accessible by the documented ATA ide
>  system, and some (or all?) IDE SATA interface have (volumtary?) broken
>  implementation: they are not IDE register compatible.

More evidence that Gujin is completely broken.

Host controller programming interfaces have _always_ been variable.  PCI 
IDE standard was never a requirement for all host controllers, indeed 
such a requirement would be stupid, and widely ignored.

Modern SATA controllers are all FIS-based, and are not (and should not 
be) limited by the legacy IDE register programming interface.

	Jeff

RE : Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

--- Jeff Garzik wrote:
> >   Your hard disk is a lot more powerfull than what you think, only very old
> 
> No, it's not.  I am well aware of what's in the ATA spec.
>
> >  hard disks only have ATA set max command. Nowadays, you can not only set the
> 
> Not true.

  If you want more info without reading the spec, here is some more:
http://www.heise.de/ct/english/05/08/172/

> >  Gujin also do the absolutely needed setup of the IDE hard disk which is to freeze
> >  the password system _and_ the config system of all the IDE hard disks present, so
> >  that no virus can put a random password and send you an E-mail with the address
> >  where to send the money to get the password to unlock the hard disk and so access
> >  again your data. Again, freezing means no more modifiable until next power cycle,
> >  so IMO it is the job of the bootloader to setup the hard disk, before running
> >  anything like Linux, a commercial OS, a bootable CDROM...
> 
> This is totally broken, and I am going to strongly recommend that no one 
> use this software.

  But that is the recommended way to do in the ATA specification, since ATA4:

http://www.t13.org/project/d1153r18-ATA-ATAPI-4.pdf
and read "6.10 Security Mode feature set" around page 30 / page 46.
Pay attentiong to the last sentense of "6.10.4 Frozen mode".

  And some BIOS correctly implement it - most BIOS don't.

> It is the OS responsibility to do this.  As a simple example, when the 
> libata ACPI patches are merged (soon), libata will send BIOS-specified 
> taskfiles to the device -- including the hard drive password, if any. 
> Then it will freeze the settings.

  Which setting are you talking about: freezing the HPA, the configuration
 or the password - that is different ATA commands.
 Gujin can run a kernel (or some other software) without freezing each of
 them independantly but the user has to explicitely request it - and should
 not try to boot a random CDROM with the password unfrozen.

> Gujin's behavior will prevent the user from accessing their data, if 
> they have protected it via BIOS.

  Some BIOS have support for password, but Gujin is not using BIOS
 services for "protection" because BIOS is not providing such services
 with a documented interface.

> >  Gujin is assuming that your hard disk are accessible by the documented ATA ide
> >  system, and some (or all?) IDE SATA interface have (volumtary?) broken
> >  implementation: they are not IDE register compatible.
> 
> More evidence that Gujin is completely broken.
> 
> Host controller programming interfaces have _always_ been variable.  PCI 
> IDE standard was never a requirement for all host controllers, indeed 
> such a requirement would be stupid, and widely ignored.
> 
> Modern SATA controllers are all FIS-based, and are not (and should not 
> be) limited by the legacy IDE register programming interface.

  The complete ATA specification is describing a register interface since ATA1,
 SATA chipsets should be software compatible, even if they add commands/interfaces:
http://www.sata-io.org/interopfaq.asp    say:
    Are there any known interoperability issues with SATA?
    One of the primary requirements of the SATA 1.0 specification was to
    maintain backward compatibility with existing operating system drivers
    to eliminate incompatibility issues.


  Etienne.


__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Alan Cox]

>  Gujin is assuming that your hard disk are accessible by the documented ATA ide
>  system, and some (or all?) IDE SATA interface have (volumtary?) broken
>  implementation: they are not IDE register compatible.

SFF was never a formal standard, and ST-506 was a random vendor
interface copying exercise that caught on. ACPI permits the firmware to
provide ATA taskfiles but afaik not the boot loader unfortunately.

A lot of newer SATA hardware uses a common standard defined interface
called AHCI, and it appears most vendors are migrating in the direction
of using AHCI. If so then we are in the same kind of flux as the VLB
world before SFF and PCI settled the standard interfaces down for a
while.

Don't see how your HPA code protects versus password locking however. If
I hack the OS I write a new boot block which locks the disk then reboot
into it. By the time you go for your floppy its too late.

If thats not the case I'm most interested how you set it up to avoid
this.

Alan

RE : Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

--- Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> >  Gujin is assuming that your hard disk are accessible by the documented ATA ide
> >  system, and some (or all?) IDE SATA interface have (volumtary?) broken
> >  implementation: they are not IDE register compatible.
> 
> SFF was never a formal standard, and ST-506 was a random vendor
> interface copying exercise that caught on. ACPI permits the firmware to
> provide ATA taskfiles but afaik not the boot loader unfortunately.

  I can understand that 10+ years old interface can be updated, or even redesigned,
 but when you buy the hardware you should be aware that it is not at all compatible
 (maybe some chipset only), so you can decide what to buy. If the spec of your
 hardware say that it is 100% compatible and the chipset do not implement its spec,
 you have a problem.
  It is mainly to increase the bandwidth of the pipe linking the hard disk to the
 memory, but anyway the hard disk cannot reach this bandwidth for ATA anyway (unless
 reading it own memory cache).
  And there is still the current hardware ATA + ATA disk to manage.

> A lot of newer SATA hardware uses a common standard defined interface
> called AHCI, and it appears most vendors are migrating in the direction
> of using AHCI. If so then we are in the same kind of flux as the VLB
> world before SFF and PCI settled the standard interfaces down for a
> while.
> Don't see how your HPA code protects versus password locking however. If
> I hack the OS I write a new boot block which locks the disk then reboot
> into it. By the time you go for your floppy its too late.

 If someone has write access to the MBR and the power supply, well he can destroy
 the system in so many ways. Note that if the hard disk has an IDE password
 that you know and that the disk is frozzen, your new boot block has to enter
 the known-by-you password first...
 Gujin cannot set an IDE password (use other utilities for that), but will ask
 you the password of locked hard disks (after keyboard mapping/language recognition)
 before probing them for partiton.

> If thats not the case I'm most interested how you set it up to avoid
> this.
> 
> Alan
> 

  Etienne.

__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

--- XXXXXXXX wrote:
> It tends to be preferred around here to implement hardware features so 
> that hardware actually works, rather than to implement spec and blame 
> the hardware for failing to work to the spec.

 Well, the IDE hardware works, and conform to the ATA specs.
 Gujin is using those ATA command for quite a long time.
 I just have a problem with SATA chipset not being ATA1-7 compatible.

>  If, as you say, most BIOS 
> don't obey the spec, then implementing a spec that bears little 
> resemblance to actual behavior only helps the small number of 
> spec-compliant implementations.

  The main problem for BIOS manufacturer is the keyboard mapping
 recognition to enter the password. And there is no real point
 in freezing the password system of a locked drive.
  Gujin try to solve most of those problems, but because it is
 located on the hard disk itself limit the choice - it would be
 so nice to have a big enough BIOS FLASH and put Gujin there.
 Second best choice is having two hard disks (compact flash as
 first HD with compact flash <-> IDE adapter).

> >   The complete ATA specification is describing a register interface since ATA1,
> >  SATA chipsets should be software compatible, even if they add commands/interfaces:
> > http://www.sata-io.org/interopfaq.asp    say:
> >     Are there any known interoperability issues with SATA?
> >     One of the primary requirements of the SATA 1.0 specification was to
> >     maintain backward compatibility with existing operating system drivers
> >     to eliminate incompatibility issues.
> > 
> 
> Again with the "The spec says it works this way, so I can do it this 
> way."  I realize that you can be perfectly, legally, safe implementing 
> specified behavior, but it leaves you doing what you are doing here, 
> which is blaming the hardware for not living up to the specification. 

  Well the ATA specs are quite old, Gujin still work with the BIOS hard disk
 so still act as a bootloader, but it can not protecting you against a
 1 milisecond attack setting password.
 By the way the SATA specs are not free.

> It is generally the choice in the linux kernel to make the hardware 
> work, and ignore the specification where it is irrelevant to actual 
> functionality.  Read up on the SAS Transport Layer.

  Oh yes, Gujin has some strange code to make things work at the end,
 look for ATAPI stuff - I can and do ignore the specs to make the software
 work on real hardware.

  Cheers,
  Etienne.



__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

Alan wrote:
> Don't see how your HPA code protects versus password locking however. If
> I hack the OS I write a new boot block which locks the disk then reboot
> into it. By the time you go for your floppy its too late.
> If thats not the case I'm most interested how you set it up to avoid this.

 I was a bit too quick to answer this one yesterday.
 Basically, you want to protect against an attack replacing the bootloader,
 whatever happens later is bad anyway.
 The only way to get that is if the bootloader cannot be overwritten,
 preferably even by root, at the time of the attack.
 You have to know that whatever put Gujin in memory is independant of how
 it will probe the disks, partitions and files.

 The simple solution is: you never boot from the hard disk, but from a
 physically write protected device (write protected floppy, non writeable
 CD or CDRW in a non writing CDROM drive, USB thumb drive with a physical
 write protect switch). It may be interresting to be able to enable
 writing to the boot device at setup (for instance to select a kernel
 to "autoboot" after few seconds or set up the keyboard language and
 the default kernel command line), Gujin can itself save few data in
 its second 512 bytes sector if write is enabled, but that is not
 100 % needed because the installer "instboot" can set all those
 parameter in file "boot.bcd" for CDROM install.
 Note that, if you are using a "write once" CDROM, you should take care
 of not enabling an attacker to add a session with a new El-torito
 boot sector.

 The complex solution is described in the ATA standard T13 D1367,
 Protect Area Run Time Interface Extension Services, skip to "4 Overview",
 and depends on the hard drive supporting the "Address Offset Reserved Area Boot",
 only IBM drives seem to have such a support.
 Basically, at boot, the LBA is offseted by a constant and you boot this
 "minidisk" with its own MBR. You can then send a command to the HD
 to stop this offset, and protect your "minidisk" by the HPA or by
 reducing the visible size of the disk (latter hurts the BIOS).
 The real MBR is never executed, the executed MBR is write protected
 even by root at the time of the attack, and if you boot with a DOS floppy
 you will see a small hard disk with the really used MBR.
 Gujin is designed to produce a complete disk mapping with FAT12/16
 filesystem (no partition like a floppy) to be able to handle such a boot,
 but I did not finish to support it, mainly because the only IBM drive
 I have is my main one, and because my hobby time is limited (I am not
 working in the PC industry). It should not be difficult to support
 that completely, the details are how to upgrade Gujin, how to behave
 with non booted but present disks...

 If you are using one of these two solutions, an attacker will have to
 take control of the system before Gujin is executed - so the only
 attack is either the main BIOS FLASH (check that you have a jumper
 to write protect it physically) and other PCI/AGP BIOS (video BIOS,
 network BIOS) which may be present on FLASH and may be overwritten.
 Modifying the firmware of the HD may be another attack path.

 All this remember me the software write protection of 5 1/4 inch floppy...
 Also, never try to test IDE passwords at 2:00 in the morning, even for
 a quick test, you may type a random password and get a 160GB brick.
 Someone knows the master password of Maxtor 4G160J8 code GAK819K0
 drive S/N G80CNBME FG08A - I promise not to repeat it!

 Cheers,
 Etienne.

__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail 

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Arjan van de Ven]

>  The simple solution is: you never boot from the hard disk, but from a
>  physically write protected device (write protected floppy, non writeable
>  CD or CDRW in a non writing CDROM drive, USB thumb drive with a physical
>  write protect switch).


that's not totally fool proof either; after all an attacker can adjust
the nvram to change the boot device order.

Re: [RFC] ATA host-protected area (HPA) device mapper?

[Etienne Lorrain]

--- Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> If I hack the OS I write a new boot block which locks the disk then reboot into it.

 Another solution to be safe, because you need a full power cycle of the HD,
 is to keep the HDs on a separate non interruptible power supply (not stoppable
 by APM/ACPI).

__________________________________________________
Do You Yahoo!?
En finir avec le spam? Yahoo! Mail vous offre la meilleure protection possible contre les messages non sollicités 
http://mail.yahoo.fr Yahoo! Mail