Re: HPA unlock during partition scan of RAID components

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

Hello,

(cc'ing dm-devel)

On Thu, Nov 03, 2011 at 01:02:37AM +0000, Hawrylewicz Czarnowski, Przemyslaw wrote:
> Recently we have encountered a problem with unlocking of HPA on disks belonging to RAID array.
> The scenario in the simplest form is as follows:
> * take HPA resized drives
> * use SW Raid solution with metadata located at the end of disk (eg. md v1.0, IMSM)
> * create raid0/10/5 using whole devices (eg. /dev/sda) (raid size must extend capacity of single device; raid1 is not the case here) 
> * create any partition with size exceeding HPA limit of single component
> 
> According to the code of rescan_partitions(), if disk has capacity limit enabled (HPA) and boundary of partition extends beyond that limit, it is bypassed/unlocked (regardless of libata.ignore_hpa state).
> As for single block device it is proper action - for raid using whole block device it is not. The partition table despite the fact it is read from single disk belongs to raid array. Values in RAID context are set properly. But from the point of view of single device - they are not.
> Problem is that in general rescan_partition() has no knowledge of any raid array using that block device. Moreover, that raid is not assembled yet.
> And the result: as HPA limit is unlocked and metadata on this device in not recognizable anymore - in worst case raid is not assembled or failed.
> 
> The simplest resolution is to conditionally forbid HPA unlocking (eg. extending ignore_hpa parameter) but I suppose a better and more 'intelligent' solution can be found.

This has come up a couple times and I think the proper solution is to
always unlock HPA and expose both sizes - locked and unlocked and let
dm, md or whatever do whatever is approriate.  Block or driver layer
doesn't have any way to determine which one is the right bet - it
simply doesn't have enough information.  I tried to bounce this idea
off people who whould actually be using this (dm/md) but haven't heard
back yet.

Thanks.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Phillip Susi]

On 11/3/2011 11:54 AM, Tejun Heo wrote:
> This has come up a couple times and I think the proper solution is to
> always unlock HPA and expose both sizes - locked and unlocked and let
> dm, md or whatever do whatever is approriate.  Block or driver layer
> doesn't have any way to determine which one is the right bet - it
> simply doesn't have enough information.  I tried to bounce this idea
> off people who whould actually be using this (dm/md) but haven't heard
> back yet.

Simply making the smaller size available does not solve the problem of 
making the part of the drive that is supposed to remain hidden 
accessible to user space, and it remains unlocked across a reboot, which 
usually makes the bios fail to recognize such drives.

The only reason I am aware of for unlocking the hpa is to avoid problems 
caused by upgrading an old system that was installed using the unlock 
behavior and thus, incorrectly extended its partition into the protected 
area.  It seems the appropriate fix for that it for distribution upgrade 
scripts to test for this and configure the boot loader to pass the 
unlock flag ( or maybe fix the problem by shrinking the partition ), 
rather than have the kernel continue to try unlocking things by default.

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

Hello,

On Thu, Nov 3, 2011 at 12:00 PM, Phillip Susi <psusi@cfl.rr.com> wrote:
> Simply making the smaller size available does not solve the problem of
> making the part of the drive that is supposed to remain hidden accessible to
> user space, and it remains unlocked across a reboot, which usually makes the
> bios fail to recognize such drives.
>
> The only reason I am aware of for unlocking the hpa is to avoid problems
> caused by upgrading an old system that was installed using the unlock
> behavior and thus, incorrectly extended its partition into the protected
> area.  It seems the appropriate fix for that it for distribution upgrade
> scripts to test for this and configure the boot loader to pass the unlock
> flag ( or maybe fix the problem by shrinking the partition ), rather than
> have the kernel continue to try unlocking things by default.

I frankly don't care about BIOSen locking up afterwards and I don't
think they are more common than the issues from not unlocking (e.g.
moving harddrive to another machine, hot-plugging - partition -
reboot).  There will be a kernel param to disable unlocking behavior
and that would be it.  On top of that if bios is that screwed it's
probably a good idea to not use bios raid which is a silly thing to
begin with.  Currently the problem is that the kernel doesn't unlock
by default and the heuristics can't be turned off, so we have unhappy
customers on both sides.  Anyways, if md/dm people are on board, I'm
gonna push for that but otherwise there isn't much point.

Thanks.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Phillip Susi]

On 11/4/2011 10:48 AM, Tejun Heo wrote:
> I frankly don't care about BIOSen locking up afterwards and I don't
> think they are more common than the issues from not unlocking (e.g.
> moving harddrive to another machine, hot-plugging - partition -
> reboot).  There will be a kernel param to disable unlocking behavior

Could you clarify what issues you refer to?  Moving a drive to another 
machine?

> and that would be it.  On top of that if bios is that screwed it's
> probably a good idea to not use bios raid which is a silly thing to
> begin with.  Currently the problem is that the kernel doesn't unlock
> by default and the heuristics can't be turned off, so we have unhappy
> customers on both sides.  Anyways, if md/dm people are on board, I'm
> gonna push for that but otherwise there isn't much point.

It isn't only a problem for bios raid, it also affects mdadm raid. 
Being able to defeat the heuristic would help, but I still don't see why 
the heuristic should be implemented in the kernel instead of user space?

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

Hello,

On Fri, Nov 04, 2011 at 11:42:20AM -0400, Phillip Susi wrote:
> On 11/4/2011 10:48 AM, Tejun Heo wrote:
> >I frankly don't care about BIOSen locking up afterwards and I don't
> >think they are more common than the issues from not unlocking (e.g.
> >moving harddrive to another machine, hot-plugging - partition -
> >reboot).  There will be a kernel param to disable unlocking behavior
> 
> Could you clarify what issues you refer to?  Moving a drive to
> another machine?

Yeah, that or hotplugging, or changing BIOS config, updating BIOS,
and, some (rare) BIOSen even fail to set up things consistently across
reboots.

> >and that would be it.  On top of that if bios is that screwed it's
> >probably a good idea to not use bios raid which is a silly thing to
> >begin with.  Currently the problem is that the kernel doesn't unlock
> >by default and the heuristics can't be turned off, so we have unhappy
> >customers on both sides.  Anyways, if md/dm people are on board, I'm
> >gonna push for that but otherwise there isn't much point.
> 
> It isn't only a problem for bios raid, it also affects mdadm raid.

Exposing alt size should be enough for md/dm no matter how hpa is
setup.

> Being able to defeat the heuristic would help, but I still don't see
> why the heuristic should be implemented in the kernel instead of
> user space?

Because sans raids, it was good enough and doesn't recent md's have
metadata at the beginning anyway?

The thing is that you have your problem, and other people have other
issues w/ HPA.  At this point, it's a misfeature abused by some silly
BIOSen mostly for silly BIOS raids.  I think we should expose the most
we can to our users and if some BIOSen are screwed on soft reboot,
well, just accept it or turn off HPA unlocking manually.

Thanks.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Phillip Susi]

On 11/4/2011 11:52 AM, Tejun Heo wrote:
>> Could you clarify what issues you refer to?  Moving a drive to
>> another machine?
>
> Yeah, that or hotplugging, or changing BIOS config, updating BIOS,
> and, some (rare) BIOSen even fail to set up things consistently across
> reboots.

I'm asking for a clarification not just a restatement.  What problem 
occurs when you move a drive to another machine, or hotplug the drive?

> Exposing alt size should be enough for md/dm no matter how hpa is
> setup.

So you are saying that mdadm should store the metadata for 0.9 and 1.0 
relative to the locked end of the disk, even if it is currently 
unlocked?  I suppose then grub also will need to be able to detect if 
the disk has been unlocked and adjust to using the locked size.

>> Being able to defeat the heuristic would help, but I still don't see
>> why the heuristic should be implemented in the kernel instead of
>> user space?
>
> Because sans raids, it was good enough and doesn't recent md's have
> metadata at the beginning anyway?

It isn't about whether it is good enough or not, it's about whether the 
kernel should be using heuristics to make decisions, or leaving it up to 
user space, where distros can come up with all kinds of complex logic to 
handle all of the goofy cases.  You complain that the block layer does 
not have enough information to get it right, so why not let user space 
decide, where it has access to all of the information it needs.

> The thing is that you have your problem, and other people have other
> issues w/ HPA.  At this point, it's a misfeature abused by some silly
> BIOSen mostly for silly BIOS raids.  I think we should expose the most
> we can to our users and if some BIOSen are screwed on soft reboot,
> well, just accept it or turn off HPA unlocking manually.

It is actually not used FOR the bios raid.  The bios is using the HPA 
for unrelated reasons and it just happens that unlocking it breaks the 
bios raid ( and mdadm with format 0.9 or 1.0 ).

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

Hello,

On Fri, Nov 04, 2011 at 12:26:02PM -0400, Phillip Susi wrote:
> I'm asking for a clarification not just a restatement.  What problem
> occurs when you move a drive to another machine, or hotplug the
> drive?

Umm... partioned / mkfs'd while unlocked and bios (on whatever
machine) locks it on later boots.

> >Exposing alt size should be enough for md/dm no matter how hpa is
> >setup.
> 
> So you are saying that mdadm should store the metadata for 0.9 and
> 1.0 relative to the locked end of the disk, even if it is currently
> unlocked?  I suppose then grub also will need to be able to detect
> if the disk has been unlocked and adjust to using the locked size.

No, with both sizes exposed, mdadm should be able to detect metadata
at the end even if it was created w/ hpa locked.  This is still
fundamentally flawed as if you move it to another machine, it may be
locked in completely different way, but that's as far as we can go.

> >Because sans raids, it was good enough and doesn't recent md's have
> >metadata at the beginning anyway?
> 
> It isn't about whether it is good enough or not, it's about whether
> the kernel should be using heuristics to make decisions, or leaving
> it up to user space, where distros can come up with all kinds of
> complex logic to handle all of the goofy cases.  You complain that
> the block layer does not have enough information to get it right, so
> why not let user space decide, where it has access to all of the
> information it needs.

Upto userspace how?  Unlock it after root is mounted?

> >The thing is that you have your problem, and other people have other
> >issues w/ HPA.  At this point, it's a misfeature abused by some silly
> >BIOSen mostly for silly BIOS raids.  I think we should expose the most
> >we can to our users and if some BIOSen are screwed on soft reboot,
> >well, just accept it or turn off HPA unlocking manually.
> 
> It is actually not used FOR the bios raid.  The bios is using the
> HPA for unrelated reasons and it just happens that unlocking it
> breaks the bios raid ( and mdadm with format 0.9 or 1.0 ).

It is also used for bios raids.  Some BIOSen lock it for no reason.
Some lock it if BIOS raid is enabled.  Some lock it for recovery
partition.  Some are just crazy.  This already has been discussed to
death.

Thanks.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Phillip Susi]

On 11/4/2011 12:32 PM, Tejun Heo wrote:
> Umm... partioned / mkfs'd while unlocked and bios (on whatever
> machine) locks it on later boots.

So the first machine does not set an HPA at all, you format the drive, 
then move it to a machine whose bios blindly adds an HPA to the drive? 
A bios certainly shouldn't be blindly adding an HPA to an already 
formatted drive that did not have one before.  If it does, why is that 
our problem?

> No, with both sizes exposed, mdadm should be able to detect metadata
> at the end even if it was created w/ hpa locked.  This is still
> fundamentally flawed as if you move it to another machine, it may be
> locked in completely different way, but that's as far as we can go.

So mdadm could check both places after the kernel always unlocks, 
without any heuristic?  And when creating an array, it would write it to 
the real, unlocked end of the disk, possibly trashing whatever the bios 
has stored there?

> Upto userspace how?  Unlock it after root is mounted?

Or in the initramfs.  Or just deal with it as part of the upgrade 
process, as I suggested before.

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

On Fri, Nov 04, 2011 at 05:08:55PM -0400, Phillip Susi wrote:
> On 11/4/2011 12:32 PM, Tejun Heo wrote:
> >Umm... partioned / mkfs'd while unlocked and bios (on whatever
> >machine) locks it on later boots.
> 
> So the first machine does not set an HPA at all, you format the
> drive, then move it to a machine whose bios blindly adds an HPA to
> the drive? A bios certainly shouldn't be blindly adding an HPA to an
> already formatted drive that did not have one before.  If it does,
> why is that our problem?

Sigh, yes, it is as much our problem as any other HPA related issues
and it can be much more common because it can happen with hot
plugging/swapping.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/04/2011 05:50 PM, Tejun Heo wrote:
>> So the first machine does not set an HPA at all, you format the 
>> drive, then move it to a machine whose bios blindly adds an HPA
>> to the drive? A bios certainly shouldn't be blindly adding an HPA
>> to an already formatted drive that did not have one before.  If
>> it does, why is that our problem?
> 
> Sigh, yes, it is as much our problem as any other HPA related
> issues and it can be much more common because it can happen with
> hot plugging/swapping.

That would also break a Windows drive.  Are there any such biosen in
the wild?  They should only add an HPA to a drive that is unformatted.

In other words, if you had a drive with Windows on it, and
transplanted it into such a broken machine, Windows would no longer be
able to mount it.  I can't imagine there are such broken biosen in the
wild, and if there are, we can at least point and say "see, it breaks
when you do that with windows too so it is the fault of the bios, not
Linux".
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk60kZEACgkQJ4UciIs+XuITagCfZKTvfglv1jUl3VOsfoZC04/P
Ia8AoJAZykXjJvkz1OHY1zqAVetXYBPn
=5V5N
-----END PGP SIGNATURE-----

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

Hello,

On Fri, Nov 4, 2011 at 6:29 PM, Phillip Susi <psusi@cfl.rr.com> wrote:
> That would also break a Windows drive.  Are there any such biosen in
> the wild?  They should only add an HPA to a drive that is unformatted.

Yes, it would break under windows and it does.  Get a motherboard
which always locks HPA, do hotplug, create filesystem filling the full
disk and reboot. BIOS doesn't have a reliable way of determining
whether the disk is "unformatted" or not. Some try to scan dos
partition table but filesystem may be created on the whole device or
software raid could be using the whole drive. There is no reliable way
to tell. Some BIOSen may try to do HPA locking after hotplug using
ACPI _GTF which in theory can work but in practice many motherboards
either don't implement them or are horridly broken. So, yeah, it isn't
too difficult to break whether the os is windows or linux.

The only argument against unlocking by default and providing both
sizes is that some BIOSen may act incorrectly after soft reset, which
seems acceptable provided there's an override to disable the automatic
unlocking.

Maybe we should just add yet another param to suppress the block layer
heuristics and be done with it. I don't know.

Thanks.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/04/2011 09:43 PM, Tejun Heo wrote:
> Yes, it would break under windows and it does.  Get a motherboard 
> which always locks HPA, do hotplug, create filesystem filling the
> full disk and reboot. BIOS doesn't have a reliable way of
> determining whether the disk is "unformatted" or not. Some try to
> scan dos partition table but filesystem may be created on the whole
> device or software raid could be using the whole drive. There is no
> reliable way to tell. Some BIOSen may try to do HPA locking after
> hotplug using ACPI _GTF which in theory can work but in practice
> many motherboards either don't implement them or are horridly
> broken. So, yeah, it isn't too difficult to break whether the os is
> windows or linux.

Well if it breaks under Windows, then we can't be faulted for having
the same results now can we?

> The only argument against unlocking by default and providing both 
> sizes is that some BIOSen may act incorrectly after soft reset,
> which seems acceptable provided there's an override to disable the
> automatic unlocking.

The other argument is that unlocking by default trashes whatever data
the bios was trying to hide in the HPA, and deviates from the behavior
of Windows.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk60nt0ACgkQJ4UciIs+XuIdkACfVPbcO/Tl0bJ2g/wEb2HQ6Gpl
0u4AoL67GnVVZZgW98tGElJtYpNENXaU
=bxzR
-----END PGP SIGNATURE-----

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

Hello,

> Well if it breaks under Windows, then we can't be faulted for having
> the same results now can we?

Heh, let's just remove all hpa features and screw all the current
users which depend on them!

> The other argument is that unlocking by default trashes whatever data
> the bios was trying to hide in the HPA, and deviates from the behavior
> of Windows.

On the former point, I don't think it matters. If the user doesn't
know what it is, [s]he can leave it alone. Maybe userland can do
something smarter if it knows the partition lives in hpa locked area,
like not mounting it by default or hiding it from browser.

On the point of not deviating from windows, I kinda agree and my
previous "let's just remove all hpa stuff" isn't all that sarcastic,
but it really isn't that simple. On BIOS raid configs, the vendor
controls both bios and driver and those drivers are very self
contained. They implement all from bit pushing driver itself to raid
stack and high level device management, and actually assumes that they
have full control of the whole stack and do good number of crazy
things. It would be surprising if no driver is unlocking hpa during
driver init. And they do all sorts of crazy and broken stuff. Try to
follow windows bios raid troubleshooting threads, it often ends up
"umm... I lost everything and had to reinitialized the whole array
from BIOS and reinstall".

So... um... I don't know. The dual size thing was the best solution
that I know of (don't remember who came up with it first) and at least
some of the ATA people seem to agree with it. Anyways, I don't think
we're making any useful progress in this thread, so this probably will
be my last message unless something new comes up.

If you have a great new idea to solve them all, please go ahead. I'll
be happy to review.

Good luck.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Phillip Susi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/04/2011 10:52 PM, Tejun Heo wrote:
> Hello,
> 
>> Well if it breaks under Windows, then we can't be faulted for
>> having the same results now can we?
> 
> Heh, let's just remove all hpa features and screw all the current 
> users which depend on them!

The only users that depend on them are those who originally installed
a version of Linux that did it wrong, and those can be handled with
proper distribution upgrade scripts, so yes, why not stop unlocking by
default, and leave it up to the distro upgrade scripts to enable the
override when backward compatibility requires it?

>> The other argument is that unlocking by default trashes whatever
>> data the bios was trying to hide in the HPA, and deviates from
>> the behavior of Windows.
> 
> On the former point, I don't think it matters. If the user doesn't 
> know what it is, [s]he can leave it alone. Maybe userland can do 
> something smarter if it knows the partition lives in hpa locked
> area, like not mounting it by default or hiding it from browser.

You don't think it matters that we are overwriting data that the bios
marked as reserved for its use only, and Windows respects that?  And
user land certainly can do things smarter, the question is what the
kernel default should be.

When upgrading, user land scripts can decide that the previous kernel
was doing it wring, and force the new kernel to continue doing it
wrong for the sake of backward compatibility, but I see no reason why
the default on new installs should deviate from Windows and refuse to
respect the will of the bios.

In other words, if a new install of Windows respects the HPA, then so
should a new install of Linux.

> On the point of not deviating from windows, I kinda agree and my 
> previous "let's just remove all hpa stuff" isn't all that
> sarcastic, but it really isn't that simple. On BIOS raid configs,
> the vendor controls both bios and driver and those drivers are very
> self contained. They implement all from bit pushing driver itself
> to raid stack and high level device management, and actually
> assumes that they have full control of the whole stack and do good
> number of crazy things. It would be surprising if no driver is
> unlocking hpa during driver init. And they do all sorts of crazy
> and broken stuff. Try to follow windows bios raid troubleshooting
> threads, it often ends up "umm... I lost everything and had to
> reinitialized the whole array from BIOS and reinstall".

Again though, it doesn't really have anything to do with fakeraid.  If
there is an HPA without a fakeraid, and Linux violates that HPA, then
Linux is causing damage that Windows does not.  The only reason that
fakeraid has anything to do with it is that for reasons unknown, there
seems to be a general overlap between boards that create an HPA and
boards that do fakeraid.  If fakeraid is out of the picture, and you
have a drive that has an HPA for whatever reason, Windows respects it,
and Linux trashes it.  That's not a good policy.

> If you have a great new idea to solve them all, please go ahead.
> I'll be happy to review.

I'm still looking for a problem that is not both caused by a broken
Linux policy in the past ( installing with an older kernel that
automatically unlocked ), and is solved by unlocking now.  The fact
that the broken Linux policy seems to mostly affect fakeraid users
does not justify continuing that broken policy.

In other words, Windows respects the protected area, so we should as
well, unless we have very good reason to believe that doing so would
cause loss of existing user data.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk60sqEACgkQJ4UciIs+XuLtuwCgg2fBXxE+PjW9i38jYvxFUrqa
qAMAn0+KBgzMnstTZpaOUKZXZeXQ3oU3
=106j
-----END PGP SIGNATURE-----

Re: HPA unlock during partition scan of RAID components

[NeilBrown]

[-- Attachment #1.1: Type: text/plain, Size: 2609 bytes --]

On Thu, 3 Nov 2011 08:54:16 -0700 Tejun Heo <tj@kernel.org> wrote:

> Hello,
> 
> (cc'ing dm-devel)
> 
> On Thu, Nov 03, 2011 at 01:02:37AM +0000, Hawrylewicz Czarnowski, Przemyslaw wrote:
> > Recently we have encountered a problem with unlocking of HPA on disks belonging to RAID array.
> > The scenario in the simplest form is as follows:
> > * take HPA resized drives
> > * use SW Raid solution with metadata located at the end of disk (eg. md v1.0, IMSM)
> > * create raid0/10/5 using whole devices (eg. /dev/sda) (raid size must extend capacity of single device; raid1 is not the case here) 
> > * create any partition with size exceeding HPA limit of single component
> > 
> > According to the code of rescan_partitions(), if disk has capacity limit enabled (HPA) and boundary of partition extends beyond that limit, it is bypassed/unlocked (regardless of libata.ignore_hpa state).
> > As for single block device it is proper action - for raid using whole block device it is not. The partition table despite the fact it is read from single disk belongs to raid array. Values in RAID context are set properly. But from the point of view of single device - they are not.
> > Problem is that in general rescan_partition() has no knowledge of any raid array using that block device. Moreover, that raid is not assembled yet.
> > And the result: as HPA limit is unlocked and metadata on this device in not recognizable anymore - in worst case raid is not assembled or failed.
> > 
> > The simplest resolution is to conditionally forbid HPA unlocking (eg. extending ignore_hpa parameter) but I suppose a better and more 'intelligent' solution can be found.
> 
> This has come up a couple times and I think the proper solution is to
> always unlock HPA and expose both sizes - locked and unlocked and let
> dm, md or whatever do whatever is approriate.  Block or driver layer
> doesn't have any way to determine which one is the right bet - it
> simply doesn't have enough information.  I tried to bounce this idea
> off people who whould actually be using this (dm/md) but haven't heard
> back yet.
> 
> Thanks.
> 

What exactly do you mean by "expose both sizes" ??
A new ioctl - BLKGETHPASIZE64 ??

That might work, but I think it would be good if there were also an ioctl
BLKHBALOCK which changed BLKGETSIZE64 to match BLKGETHPASIZE64.
Then some user-space tools could examine the device with a full understanding
of md, dm, dmraid, partitions, filesystems etc etc and make a reasonably
informed decision.  And then put that decision into effect.

??

NeilBrown

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: HPA unlock during partition scan of RAID components

[Tejun Heo]

On Fri, Nov 04, 2011 at 07:46:52AM +1100, NeilBrown wrote:
> What exactly do you mean by "expose both sizes" ??
> A new ioctl - BLKGETHPASIZE64 ??
> 
> That might work, but I think it would be good if there were also an ioctl
> BLKHBALOCK which changed BLKGETSIZE64 to match BLKGETHPASIZE64.
> Then some user-space tools could examine the device with a full understanding
> of md, dm, dmraid, partitions, filesystems etc etc and make a reasonably
> informed decision.  And then put that decision into effect.

In kernel, just another size field.  Out of kernel, I was thinking
more along the line of a new /sysfs field, but yeah maybe another
ioctl.  At this point, I don't really think making unlocking
selectable is a good idea.  That has to go through device detach /
attach cycle and what if someone else is already using first half of
the disk?  We can try to be sneaky and slip in device size change
underneath it but that just sounds too crazy to me.  IMHO, we should
unlock by default and just let everyone know what the size before
unlocking was in case that could be useful.

Thank you.

-- 
tejun

Re: HPA unlock during partition scan of RAID components

[Charles Nordlund]

Tejun Heo <tj <at> kernel.org> writes:

> 
> On Fri, Nov 04, 2011 at 07:46:52AM +1100, NeilBrown wrote:
> > What exactly do you mean by "expose both sizes" ??
> > A new ioctl - BLKGETHPASIZE64 ??
> > 
> > That might work, but I think it would be good if there were also an ioctl
> > BLKHBALOCK which changed BLKGETSIZE64 to match BLKGETHPASIZE64.
> > Then some user-space tools could examine the device with a full understanding
> > of md, dm, dmraid, partitions, filesystems etc etc and make a reasonably
> > informed decision.  And then put that decision into effect.
> 
> In kernel, just another size field.  Out of kernel, I was thinking
> more along the line of a new /sysfs field, but yeah maybe another
> ioctl.  At this point, I don't really think making unlocking
> selectable is a good idea.  That has to go through device detach /
> attach cycle and what if someone else is already using first half of
> the disk?  We can try to be sneaky and slip in device size change
> underneath it but that just sounds too crazy to me.  IMHO, we should
> unlock by default and just let everyone know what the size before
> unlocking was in case that could be useful.
> 
> Thank you.
> 


Just wanted to pop in and say that thanks to your "intelligent HPA unlock" code
you submitted to the kernel, I am completely unable to mount and use an NTFS
RAID0. The way it ignores the ignore_hpa=0 parameter causes the fakeraid to
fail; since the first disk in the array reports a partition size that goes
beyond EOD, the code you submitted goes ahead and unlocks it to native -- not
HPA limited -- capacity, with the expected result that it completely trashes the
array as far as Linux is concerned. What was the point of the huge debate over
locking/unlocking by default and the associated parameters if the kernel is
going to simply ignore all that -- and my explicit parameter to NOT unlock past
HPA size -- and unlock it anyway (which was determined to be a very very bad
idea way back in 2009)?