Trouble booting from a large USB hard drive

Trouble booting from a large USB hard drive

[Daniel Richard G.]

I recently installed Ubuntu Linux to a 500GB external hard drive, with GRUB2, 
and at first could not boot the system due to a GRUB "unknown filesystem" 
error. Resolving this problem led to some interesting revelations that I 
wanted to share here.

Initially, the hard drive in question was partitioned to have a large (>400GB) 
FAT32 partition at the beginning. Knowing that Windows would only mount the 
first partition when the drive was plugged in, I figured this was the way to 
make the USB drive usable as a data vehicle on non-Linux systems. After that 
partition came ones for swap, root, and /home.

After installing Ubuntu, and rebooting, I got the GRUB error. This was on an 
HP laptop; later on, I connected the same drive to a Dell desktop 
machine---and lo, Ubuntu booted!

I believe that the cause of the problem with the HP laptop was a good 
ol'-fashioned BIOS barrier (a la "don't put /boot more than 8.4GB/33GB/137GB 
from the start of the disk"). The laptop was a fairly recent model---an HP 
EliteBook 6930p---but I'm guessing the code to boot from USB devices didn't 
have all the LBA workarounds and whatnot that have been standard in IDE/SATA 
boot code for years.

What all but confirmed it for me was an ingenious solution I saw posted 
somewhere: an out-of-order partition table. Put the Linux partitions first on 
the disk, then the big FAT32 partition---but the FAT32 partition is /dev/sdx1, 
and the Linux partitions are /dev/sdx{2,3,4}. Windows mounts the FAT32 
partition and ignores the others, just as intended, and Ubuntu boots 
flawlessly on the HP laptop.

Aside: All this is also covered in a GRUB bug report at

    https://savannah.gnu.org/bugs/?27069


With all that said, I don't consider this to be a bug in GRUB, and this is not 
meant to be taken as a bug report. (I presume grub-pc can't work around such 
limitations in the BIOS, because there isn't enough room in the MBR to stuff 
in a disk-reading library that makes BIOS disk calls unnecessary.) Rather, I 
think it is a corner case of which users should be aware---and perhaps GRUB 
and/or the Ubuntu installer could do a better job of warning the user if the 
disk layout may lead to that. Beyond that, broader awareness/support/use of 
out-of-order partition tables [for large USB drives in similar usage 
scenarios] doesn't seem like a bad idea.

(Which is not to say I'm certain of all this; that it's a BIOS barrier issue 
is only my hypothesis, consistent with what I've observed. Folks here might be 
aware of something else that may have been going on.)


--Daniel


-- 
Daniel Richard G. || danielg@teragram.com || Software Developer
Teragram Linguistic Technologies (a division of SAS)
http://www.teragram.com/

Re: Trouble booting from a large USB hard drive

[Isaac Dupree]

Daniel Richard G. wrote:
> What all but confirmed it for me was an ingenious solution I saw posted 
> somewhere: an out-of-order partition table. Put the Linux partitions first on 

I accidentally have an out-of-order partition table and I was surprised 
that such a thing is possible (vs. that everything gets automatically 
numbered in order).  Nevertheless, it is a useful feature, though not 
obvious how to control with 'gparted' and the like.  "extended 
partitions" probably add a bit of complication too.

> With all that said, I don't consider this to be a bug in GRUB, and this is not 
> meant to be taken as a bug report. (I presume grub-pc can't work around such 
> limitations in the BIOS, because there isn't enough room in the MBR to stuff 
> in a disk-reading library that makes BIOS disk calls unnecessary.) Rather, I 
> think it is a corner case of which users should be aware---and perhaps GRUB 
> and/or the Ubuntu installer could do a better job of warning the user

if grub-install, which doesn't have code space limitations, could 
possibly check for it, it would be a great help.  But is it possible for 
  a running system to check the BIOS like that? (or possible for grub to 
check somehow on a non-running system in a safe testing sort of manner?)

RE: Trouble booting from a large USB hard drive

[Daniel Richard G.]

[Replying with Outlook, please forgive the atrocious formatting]

Isaac Dupree wrote:
>
> I accidentally have an out-of-order partition table
> and I was surprised
> that such a thing is possible (vs. that everything
> gets automatically
> numbered in order).  Nevertheless, it is a useful
> feature, though not
> obvious how to control with 'gparted' and the like.
> "extended
> partitions" probably add a bit of complication too.

Yes, I used fdisk(8) to get the effect, but with most "user-friendly" 
partitioning tools I'd have been SOL. I'm thinking about the implications for 
something like the Ubuntu installer/partitioner, which has to be dead simple 
for the user, and yet producing an out-of-order table would be a helpful 
measure of interoperability with Windows systems.

> if grub-install, which doesn't have code space
> limitations, could
> possibly check for it, it would be a great help.  But
> is it possible for
>   a running system to check the BIOS like that? (or
> possible for grub to
> check somehow on a non-running system in a safe
> testing sort of manner?)

Consider, too, that GRUB may be installed on a system where the BIOS is good, 
and then booted on a system where it borks. USB drives have a natural mobility 
to them, of course, and even IDE/SATA drives may be transplanted from one 
computer to another.


--Daniel

Re: Trouble booting from a large USB hard drive

[Colin Watson]

On Sun, Jan 17, 2010 at 03:45:06PM -0500, Daniel Richard G. wrote:
> Isaac Dupree wrote:
> > I accidentally have an out-of-order partition table and I was
> > surprised that such a thing is possible (vs. that everything gets
> > automatically numbered in order).  Nevertheless, it is a useful
> > feature, though not obvious how to control with 'gparted' and the
> > like.  "extended partitions" probably add a bit of complication too.
> 
> Yes, I used fdisk(8) to get the effect, but with most "user-friendly" 
> partitioning tools I'd have been SOL. I'm thinking about the implications for 
> something like the Ubuntu installer/partitioner, which has to be dead simple 
> for the user, and yet producing an out-of-order table would be a helpful 
> measure of interoperability with Windows systems.

The Ubuntu installer's partitioner (essentially due to how libparted
works, so I expect any libparted-based partitioner is similar) will
produce out-of-order partition tables when it's convenient for itself to
do so: it avoids renumbering partitions since that tends to break
existing operating systems relying on those partitions, so when you
resize a partition to make some space and then create new partitions in
that space you'll typically get an out-of-order partition table.

Unfortunately, I rather suspect that the very problem being addressed
there means that we won't be able to implement your idea automatically.
Artificially renumbering the partitions of an existing operating system
while installing Ubuntu would be likely to break that operating system
in many cases (as a convenient simple example, consider a Unix-based
system referring to that partition in /etc/fstab using traditional
device names rather than UUIDs), and we have no way I can imagine to
tell whether that's going to be the case or not.  There are many
different possible BIOS limitations (see
http://tldp.org/HOWTO/Large-Disk-HOWTO-4.html) and I doubt that we can
sensibly warn about all of them or even necessarily evaluate accurately
which is the most likely.

GRUB might be able to avoid the problem you describe by using ata.mod,
provided that its core.img is embedded just before the first partition,
as recommended, rather than being installed in a partition boot record
past the BIOS barrier.


For all kinds of reasons, I would love to have a way to detect these
kinds of BIOS limitation in software, but I've never found a sensible
way to do it.  The best suggestion I've heard is to read the BIOS
software version using DMI or similar and then check it against some
kind of giant database, but I don't have the resources (time or
expertise, really) to build up such a database, and I don't even know
whether BIOS version numbering is reliable enough to make it practical.

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Trouble booting from a large USB hard drive

[Isaac Dupree]

Colin Watson wrote:
> Unfortunately, I rather suspect that the very problem being addressed
> there means that we won't be able to implement your idea automatically.

Me too, but for this reason:

putting /boot near the beginning will be normal partitioner stuff, but 
this person's reason for caring about the partition order is that he 
knows which partition he wants to be mounted by Windows, and that's "#1".

Which is an idea that could be implemented in a GUI, given the right 
libraries, (potentially though, if there's a different current 
partition-#1, at the expense of anything that refers to *that* partition 
by number) (and it might be too hacky an idea, i.e. being 
Windows-related and all, for gnu/linux developers to want to support so 
cleanly anyway)

RE: Trouble booting from a large USB hard drive

[Daniel Richard G.]

Colin Watson wrote:
>
> Unfortunately, I rather suspect that the very problem
> being addressed
> there means that we won't be able to implement your
> idea automatically.
> Artificially renumbering the partitions of an existing
> operating system
> while installing Ubuntu would be likely to break that
> operating system
> in many cases (as a convenient simple example,
> consider a Unix-based
> system referring to that partition in /etc/fstab using
> traditional
> device names rather than UUIDs), and we have no way I
> can imagine to
> tell whether that's going to be the case or not.

I was just thinking of the narrow case of a new partition table, perhaps 
narrowed even further to USB drives (unless there's an advantage to the odd 
order with IDE/SATA/SCSI disks). If the user is only modifying an existing 
table, I agree that the numbering shouldn't be touched.

> There are many
> different possible BIOS limitations (see
> http://tldp.org/HOWTO/Large-Disk-HOWTO-4.html) and I
> doubt that we can
> sensibly warn about all of them or even necessarily
> evaluate accurately
> which is the most likely.

Well, the most recent barrier (aside from the one at 2TiB that's just around 
the corner) of 137GB dates back to 2001... I suppose you could check the BIOS 
release date via DMI and warn about smaller limits if it's anywhere near that 
old. Are machines of this vintage even a worthwhile concern? I thought I'd 
read somewhere that not even overseas digital-divide charities will bother 
with such hardware.

> GRUB might be able to avoid the problem you describe
> by using ata.mod,
> provided that its core.img is embedded just before the
> first partition,
> as recommended, rather than being installed in a
> partition boot record
> past the BIOS barrier.

This isn't the norm? Was I wrong about there not being enough space before the 
first partition to store non-BIOS disk-reading routines? This wasn't the case 
with the Karmic install, so I presumed there wasn't a way to do it.

> For all kinds of reasons, I would love to have a way
> to detect these
> kinds of BIOS limitation in software, but I've never
> found a sensible
> way to do it.  The best suggestion I've heard is to
> read the BIOS
> software version using DMI or similar and then check
> it against some
> kind of giant database, but I don't have the resources
> (time or
> expertise, really) to build up such a database, and I
> don't even know
> whether BIOS version numbering is reliable enough to
> make it practical.

And it's somewhat moot anyway, since disks can be moved between systems 
(though this is less likely with fixed disks, of course). I'm thinking the 
reasonable thing to do would be to show a warning if /boot is beyond 137GB (or 
maybe 33GB? 8.4GB?) on a USB disk unconditionally, and maybe on fixed disks if 
the BIOS is detected as being older than a certain date (much as how the 
kernel doesn't use ACPI if the BIOS predates 2000/2001).

Beyond that, having the partitioner support out-of-order numbering would be 
great, though I don't see an obvious way of doing it in the 
manual-partitioning dialog. It would be easier to support it as a canned 
layout option, perhaps available only for USB disks. But I also like to have a 
separate /home, and that means manual partitioning, so....

(I'll be happy to file a wishlist bug report against ubuntu-installer if we 
can figure what would be reasonable for it to do.)


--Daniel

Re: Trouble booting from a large USB hard drive

[Colin Watson]

On Sun, Jan 17, 2010 at 08:54:42PM -0500, Daniel Richard G. wrote:
> Colin Watson wrote:
> > There are many different possible BIOS limitations (see
> > http://tldp.org/HOWTO/Large-Disk-HOWTO-4.html) and I doubt that we
> > can sensibly warn about all of them or even necessarily evaluate
> > accurately which is the most likely.
> 
> Well, the most recent barrier (aside from the one at 2TiB that's just around 
> the corner) of 137GB dates back to 2001... I suppose you could check the BIOS 
> release date via DMI and warn about smaller limits if it's anywhere near that 
> old. Are machines of this vintage even a worthwhile concern? I thought I'd 
> read somewhere that not even overseas digital-divide charities will bother 
> with such hardware.

Unfortunately it seems that often even recent machines suffer from them.
The date doesn't appear to be a good guideline.

> > GRUB might be able to avoid the problem you describe by using
> > ata.mod, provided that its core.img is embedded just before the
> > first partition, as recommended, rather than being installed in a
> > partition boot record past the BIOS barrier.
> 
> This isn't the norm? Was I wrong about there not being enough space before the 
> first partition to store non-BIOS disk-reading routines? This wasn't the case 
> with the Karmic install, so I presumed there wasn't a way to do it.

I believe, but am not sure, that ata.mod is not quite stable enough for
universal use yet.  If it were I imagine we'd be using it instead of
biosdisk.mod.

I don't believe there's room for both methods at once, but one or the
other should just about fit.

> > For all kinds of reasons, I would love to have a way to detect these
> > kinds of BIOS limitation in software, but I've never found a
> > sensible way to do it.  The best suggestion I've heard is to read
> > the BIOS software version using DMI or similar and then check it
> > against some kind of giant database, but I don't have the resources
> > (time or expertise, really) to build up such a database, and I don't
> > even know whether BIOS version numbering is reliable enough to make
> > it practical.
> 
> And it's somewhat moot anyway, since disks can be moved between systems 
> (though this is less likely with fixed disks, of course). I'm thinking the 
> reasonable thing to do would be to show a warning if /boot is beyond 137GB (or 
> maybe 33GB? 8.4GB?) on a USB disk unconditionally, and maybe on fixed disks if 
> the BIOS is detected as being older than a certain date (much as how the 
> kernel doesn't use ACPI if the BIOS predates 2000/2001).

I just have no idea even how to assess what is reasonable to warn about
here, and am reluctant to make changes based on guesswork.  I also
really, *really* don't want to scare people into attempting to make
complicated and perhaps even risky partitioning changes when in fact
their BIOS would support their current layout just fine.  This is why
I've never done anything about this problem, although it and its friends
have been around for some time.

> Beyond that, having the partitioner support out-of-order numbering would be 
> great, though I don't see an obvious way of doing it in the 
> manual-partitioning dialog. It would be easier to support it as a canned 
> layout option, perhaps available only for USB disks. But I also like to have a 
> separate /home, and that means manual partitioning, so....

I'd much rather do this in manual partitioning.  Canned layout options
are highly contended and I want to reserve them only for the most
important and widespread options.

> (I'll be happy to file a wishlist bug report against ubuntu-installer if we 
> can figure what would be reasonable for it to do.)

I'm not sure what's reasonable yet, but feel free to file a wishlist bug
on the partman-base package in Ubuntu for some kind of way to renumber
partitions.  If you referenced this thread, that would be good for when
we come back to this in the future (as I'm not sure I'll be able to
attend to this straight away).

Thanks,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

RE: Trouble booting from a large USB hard drive

[Daniel Richard G.]

Colin Watson wrote:
>
> Unfortunately it seems that often even recent machines
> suffer from them.
> The date doesn't appear to be a good guideline.

BIOS barriers for IDE/SATA/SCSI disks, still? Nine years on since 137GB?

Now that's just laziness...

> I believe, but am not sure, that ata.mod is not quite
> stable enough for
> universal use yet.  If it were I imagine we'd be using
> it instead of
> biosdisk.mod.
>
> I don't believe there's room for both methods at once,
> but one or the
> other should just about fit.

There's also the need to interface with the USB bus. If all that can be 
squeezed in before the first partition, then heck, this whole issue goes away.

> I just have no idea even how to assess what is
> reasonable to warn about
> here, and am reluctant to make changes based on
> guesswork.  I also
> really, *really* don't want to scare people into
> attempting to make
> complicated and perhaps even risky partitioning
> changes when in fact
> their BIOS would support their current layout just
> fine.  This is why
> I've never done anything about this problem, although
> it and its friends
> have been around for some time.

For that reason, I would lean toward only doing it for USB disks (and only 
then for new partition tables). Aside from those being inherently mobile, an 
installation of Ubuntu to a USB disk can be presumed to be a "portable 
desktop" use case (the larger cousin of the live-desktop thumbdrive scenario), 
and guiding the user toward a partition layout that is compatible with some 
least-common-denominator BIOS would be to their benefit.

Consider a traveler, hooking up their trusty USB-based Ubuntu system to a 
hotel PC in Tijuana, booting it up, and getting the "unknown filesystem" 
error...

> I'd much rather do this in manual partitioning.
> Canned layout options
> are highly contended and I want to reserve them only
> for the most
> important and widespread options.

Okay. It'll take some thinking to figure out what widget(s) would best expose 
that functionality.

> I'm not sure what's reasonable yet, but feel free to
> file a wishlist bug
> on the partman-base package in Ubuntu for some kind of
> way to renumber
> partitions.  If you referenced this thread, that would
> be good for when
> we come back to this in the future (as I'm not sure
> I'll be able to
> attend to this straight away).

Okay, I'll put something together for that.


--Daniel

Re: Trouble booting from a large USB hard drive

[Grégoire Sutre]

Daniel Richard G. wrote:

> Knowing that Windows would only mount the 
> first partition when the drive was plugged in

Does this only apply to external hard drives?  I've always put Linux as 
my first partition on my single hard drive without any problem (now it's 
with XP, but AFAIR also with 98SE before).  I mean first partition both 
``logically'' and ``physically'', e.g. what I have now is:

    Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        2611    20972826   83  Linux
/dev/sda2            2612        4569    15727635    7  HPFS/NTFS

And if I remember right, the Windows installer (booting from a windows 
install CD) had the bad habit of renumbering the primary partitions if 
their logical order was not consistent with their physical order.  At 
least it did it once, and I've never played with out-of-order partition 
tables since ;-)

Grégoire

RE: Trouble booting from a large USB hard drive

[Daniel Richard G.]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1417 bytes --]

Grégoire Sutre wrote:
>
> > Knowing that Windows would only mount the
> > first partition when the drive was plugged in
>
> Does this only apply to external hard drives?

Windows treats removable and fixed drives (as distinguished by what is called 
the "removable media bit") differently, the difference being that it will 
mount all the partitions of a fixed disk (presuming they are of a compatible 
partition type and filesystem), whereas for a removable disk, it will mount 
only the first one and ignore all the others. I've seen Windows tools out 
there that can flip the removable-media bit on a USB drive, with the raison 
d'etre of allowing users to work with multiple partitions on same.

(Note: For the purposes of this discussion, Windows "mounting a partition" 
means "an icon for it appears in My Computer when I plug the disk in")

> I've always put Linux as
> my first partition on my single hard drive without any
> problem (now it's
> with XP, but AFAIR also with 98SE before).  I mean
> first partition both
> ``logically'' and ``physically'', e.g. what I have now
> is:

That's a different scenario, that of Windows "skipping over" partitions that 
it doesn't know about to get to those that it does. I don't know anything 
about that---I thought Windows just gave up at the first non-Windows partition 
it encounters, because that would be such a Windows-y thing for it to do :-]


--Daniel