Guidance on conflicts between GNU GRUB and proprietary software

Guidance on conflicts between GNU GRUB and proprietary software

[Colin Watson]

Hi Richard,

We've had some (amicable!) disagreements in the GRUB development team
regarding a question of how far we should go out of our way to avoid
conflicts with certain types of mainly proprietary software, and
Vladimir Serbinenko, our current maintainer, suggested that I ask you
for guidance.

http://lists.gnu.org/archive/html/grub-devel/2010-08/msg00137.html
offers a rough outline of the root of the issue, although not of all the
ensuing discussion.  I seem to remember that you often don't have
convenient web access, so I'll try to summarise here.


On systems using MBR partition tables, GRUB generally makes use of the
boot track (sectors 1-62) to store some of its code, as well as the boot
sector itself.  In GRUB Legacy, this was Stage 1.5; in GRUB 2, this is
core.img.  In either case, it contains enough code to read the remainder
of GRUB from an ordinary filesystem.  There are alternatives to this
strategy: we could embed core.img in reserved space at the start of a
filesystem instead, except that the most common filesystems in current
use don't provide enough space for us; or we could store core.img (or
Stage 2 in GRUB Legacy) as an ordinary file and hardcode its disk
offsets, and indeed we do this in some situations where there's no other
choice.  This tends to be fragile, though, in that filesystem repair
operations and the like could mean you have to reinstall your boot
loader, so we encourage people to embed core.img in the boot track
wherever possible.

Unfortunately, the use of the boot track was never really very clearly
defined by any of what pass for the standards in the MBR world.  The
GRUB developers of course tend to take the position that it should be
obvious that the boot track is owned by the boot loader, but others have
taken different positions.  From time to time, we get bug reports of the
general form "I boot Windows and then GRUB doesn't start any more", and
there often ensues an argument about just whose fault it is.  We've been
getting these reports rather more now that GRUB 2 is being widely
deployed, because GRUB 2's core image is somewhat more featureful than
GRUB Legacy's Stage 1.5 and is thus a few sectors larger, increasing the
probability of problems.

It appears that, rather than the operating system itself being at fault,
a number of Windows applications take over a sector in the boot track
and store bits and pieces of data there.  These applications include
"licence managers" (for which I have precious little sympathy, as
they're trying to enforce proprietary licences in sneaky ways by making
it hard for uninstallation tools to operate properly) and certain backup
and recovery tools.


Along with some others who e-mailed me when I blogged about this issue,
Vladimir suggested that GRUB should use an error-correcting code to
protect itself: codes such as Reed-Solomon can be implemented in not
much space, and storing two extra sectors of data would allow us to
correct an entire corrupted sector.  We have no disagreement here; I
think it's a good idea to make this critical area of code as robust as
reasonably possible.

I proposed that we spend a little time assembling (on a best-effort
basis) a list of signatures of foreign software using the boot track.
It's straightforward for grub-setup to notice that one of the sectors it
needs matches a particular signature, and to write the core image in
such a way as to avoid it; in fact I was pleased to find that this
didn't require changing the boot sector and hard disk bootstrap assembly
code at all!  I have posted a complete working patch for this containing
a handful of known signatures, which needs only to be fleshed out with
some more signatures gathered from users having this kind of trouble;
and I have been working on gathering this information from bug reports.
This is where our disagreement arises.


Vladimir's position, as stated on IRC today, is that:

  * Trying to avoid sectors in this way goes beyond simply protecting
    ourselves, yielding territory that ought to be ours.  In some sense,
    it recognises the legitimacy of the other software in question.

  * It is only worth doing this if the companies producing this software
    give a valid justification.  If they get in contact with us to do
    so, then we should be able to find a better way to collaborate
    anyway.

  * Users may even welcome our inadvertent interference with the time
    counter on some piece of proprietary software ...

My position (and I'm sorry that this is unbalanced because I've gone to
more length - perhaps Vladimir can expand on his) is that:

  * Regrettably, given that I see no practical alternative, I think our
    own justification for using the boot track isn't quite as absolutely
    solid as we'd like to think, given the lack of standardisation; it's
    a manifest fact that we're not the only children trying to play in
    this sandpit.

  * It's not clear that all the companies doing this are using the boot
    track as a tool to restrict users' freedom, even though some
    certainly are.  Some of the programs known to cause issues are
    backup and recovery tools, and I can imagine reasonable if arguably
    misguided reasons for them to want to store data outside any
    partition; maybe I'm naïve but it seems more likely that their
    purpose is purely technical rather than to restrict freedom.

  * The fact that this problem occurs with increased probability when
    using GRUB 2 vs. GRUB Legacy means that some users have gone back to
    GRUB Legacy to avoid this problem, commonly saying something like
    "you must have changed something in GRUB 2 to cause this; why don't
    you put it back the way it was?".  As a GRUB 2 developer who finds
    the new source base an order of magnitude more hackable and
    maintainable than the old, I want to fix any reason why somebody
    would feel they needed to use GRUB Legacy.

  * Most importantly, I feel that this is a situation where insisting on
    our total ownership of the boot track may be self-limiting.  Some of
    the programs appear to be popular ones with many users who genuinely
    want to use them despite their non-freedom, and they've been known
    to get upset when their sector in the boot track vanishes, to the
    point where users have to reinstall the program in question; for
    people with dual-boot systems this is a severe problem.

    On one hand, this casts proprietary software in a bad light: it's
    the awkward thing you have to reinstall all the time.  Searching the
    web for anecdotes I do find some more thoughtful users taking this
    position.  On the other hand, their proprietary software only breaks
    when they install free software, and many users don't think through
    the reasons behind that.  I'm concerned that this casts us as the
    villain: we look like the vandals who break things, and when we
    aren't involved there's no problem.  I find it difficult to defend
    our rightness when we're not the ones sitting there with a broken
    system.


There are various technical arguments to be made, but I think the core
issue under dispute is an ethical/strategic one: to what extent should
free software go out of its way here to avoid inadvertently limiting its
usefulness due to conflicts with proprietary software?  Are we setting
an undesirable precedent if we come down on one or other side of this,
or should this kind of thing be considered case-by-case?  We'd value
your comments on this.

Thanks,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Lennart Sorensen]

On Thu, Sep 23, 2010 at 11:19:24PM +0100, Colin Watson wrote:
> Hi Richard,
> 
> We've had some (amicable!) disagreements in the GRUB development team
> regarding a question of how far we should go out of our way to avoid
> conflicts with certain types of mainly proprietary software, and
> Vladimir Serbinenko, our current maintainer, suggested that I ask you
> for guidance.
> 
> http://lists.gnu.org/archive/html/grub-devel/2010-08/msg00137.html
> offers a rough outline of the root of the issue, although not of all the
> ensuing discussion.  I seem to remember that you often don't have
> convenient web access, so I'll try to summarise here.
> 
> 
> On systems using MBR partition tables, GRUB generally makes use of the
> boot track (sectors 1-62) to store some of its code, as well as the boot
> sector itself.  In GRUB Legacy, this was Stage 1.5; in GRUB 2, this is
> core.img.  In either case, it contains enough code to read the remainder
> of GRUB from an ordinary filesystem.  There are alternatives to this
> strategy: we could embed core.img in reserved space at the start of a
> filesystem instead, except that the most common filesystems in current
> use don't provide enough space for us; or we could store core.img (or
> Stage 2 in GRUB Legacy) as an ordinary file and hardcode its disk
> offsets, and indeed we do this in some situations where there's no other
> choice.  This tends to be fragile, though, in that filesystem repair
> operations and the like could mean you have to reinstall your boot
> loader, so we encourage people to embed core.img in the boot track
> wherever possible.
> 
> Unfortunately, the use of the boot track was never really very clearly
> defined by any of what pass for the standards in the MBR world.  The
> GRUB developers of course tend to take the position that it should be
> obvious that the boot track is owned by the boot loader, but others have
> taken different positions.  From time to time, we get bug reports of the
> general form "I boot Windows and then GRUB doesn't start any more", and
> there often ensues an argument about just whose fault it is.  We've been
> getting these reports rather more now that GRUB 2 is being widely
> deployed, because GRUB 2's core image is somewhat more featureful than
> GRUB Legacy's Stage 1.5 and is thus a few sectors larger, increasing the
> probability of problems.
> 
> It appears that, rather than the operating system itself being at fault,
> a number of Windows applications take over a sector in the boot track
> and store bits and pieces of data there.  These applications include
> "licence managers" (for which I have precious little sympathy, as
> they're trying to enforce proprietary licences in sneaky ways by making
> it hard for uninstallation tools to operate properly) and certain backup
> and recovery tools.
> 
> 
> Along with some others who e-mailed me when I blogged about this issue,
> Vladimir suggested that GRUB should use an error-correcting code to
> protect itself: codes such as Reed-Solomon can be implemented in not
> much space, and storing two extra sectors of data would allow us to
> correct an entire corrupted sector.  We have no disagreement here; I
> think it's a good idea to make this critical area of code as robust as
> reasonably possible.
> 
> I proposed that we spend a little time assembling (on a best-effort
> basis) a list of signatures of foreign software using the boot track.
> It's straightforward for grub-setup to notice that one of the sectors it
> needs matches a particular signature, and to write the core image in
> such a way as to avoid it; in fact I was pleased to find that this
> didn't require changing the boot sector and hard disk bootstrap assembly
> code at all!  I have posted a complete working patch for this containing
> a handful of known signatures, which needs only to be fleshed out with
> some more signatures gathered from users having this kind of trouble;
> and I have been working on gathering this information from bug reports.
> This is where our disagreement arises.
> 
> 
> Vladimir's position, as stated on IRC today, is that:
> 
>   * Trying to avoid sectors in this way goes beyond simply protecting
>     ourselves, yielding territory that ought to be ours.  In some sense,
>     it recognises the legitimacy of the other software in question.
> 
>   * It is only worth doing this if the companies producing this software
>     give a valid justification.  If they get in contact with us to do
>     so, then we should be able to find a better way to collaborate
>     anyway.
> 
>   * Users may even welcome our inadvertent interference with the time
>     counter on some piece of proprietary software ...
> 
> My position (and I'm sorry that this is unbalanced because I've gone to
> more length - perhaps Vladimir can expand on his) is that:
> 
>   * Regrettably, given that I see no practical alternative, I think our
>     own justification for using the boot track isn't quite as absolutely
>     solid as we'd like to think, given the lack of standardisation; it's
>     a manifest fact that we're not the only children trying to play in
>     this sandpit.
> 
>   * It's not clear that all the companies doing this are using the boot
>     track as a tool to restrict users' freedom, even though some
>     certainly are.  Some of the programs known to cause issues are
>     backup and recovery tools, and I can imagine reasonable if arguably
>     misguided reasons for them to want to store data outside any
>     partition; maybe I'm naïve but it seems more likely that their
>     purpose is purely technical rather than to restrict freedom.
> 
>   * The fact that this problem occurs with increased probability when
>     using GRUB 2 vs. GRUB Legacy means that some users have gone back to
>     GRUB Legacy to avoid this problem, commonly saying something like
>     "you must have changed something in GRUB 2 to cause this; why don't
>     you put it back the way it was?".  As a GRUB 2 developer who finds
>     the new source base an order of magnitude more hackable and
>     maintainable than the old, I want to fix any reason why somebody
>     would feel they needed to use GRUB Legacy.
> 
>   * Most importantly, I feel that this is a situation where insisting on
>     our total ownership of the boot track may be self-limiting.  Some of
>     the programs appear to be popular ones with many users who genuinely
>     want to use them despite their non-freedom, and they've been known
>     to get upset when their sector in the boot track vanishes, to the
>     point where users have to reinstall the program in question; for
>     people with dual-boot systems this is a severe problem.
> 
>     On one hand, this casts proprietary software in a bad light: it's
>     the awkward thing you have to reinstall all the time.  Searching the
>     web for anecdotes I do find some more thoughtful users taking this
>     position.  On the other hand, their proprietary software only breaks
>     when they install free software, and many users don't think through
>     the reasons behind that.  I'm concerned that this casts us as the
>     villain: we look like the vandals who break things, and when we
>     aren't involved there's no problem.  I find it difficult to defend
>     our rightness when we're not the ones sitting there with a broken
>     system.
> 
> 
> There are various technical arguments to be made, but I think the core
> issue under dispute is an ethical/strategic one: to what extent should
> free software go out of its way here to avoid inadvertently limiting its
> usefulness due to conflicts with proprietary software?  Are we setting
> an undesirable precedent if we come down on one or other side of this,
> or should this kind of thing be considered case-by-case?  We'd value
> your comments on this.

Well perhaps grub could keep a list of known cases to look for and warn
the user if it sees one and perhaps suggest using the crappy blocklist
method instead if they insist on keeping their license manager or whatever
the problem cause is.

I personally think the software that writes to the disk outside a
filesystem should be banned and shunned, but just because I don't run
any doesn't mean other people don't.

Making grub more robust sounds nice, even if it is just to ensure you
can still boot in case of disk read errors.  Working even if some stupid
software does something it shouldn't, well that's just a bonus.  The
detection and avoidance is certainly debateable, although "do no harm"
would probably be appreciated by users of multiple operating systems on
their machines, so aborting a grub-install if a conflict is detected and
suggesting a (crappy?) workaround of using the blocklist instead saves
the user some hassle.

I wonder what those license managers do on a GPT partitioned disk.
I guess on those grub would be either in an EFI system partition or in
a bios boot partition, and in neither case would it be in unpartitioned
space.

-- 
Len Sorensen

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Brendan Trotter]

Hi,

Just thought I'd throw my 2 cents in..

Any software (except the software that "owns" the MBR) that uses any
sectors that are in the first track with the MBR and outside of any
partition (e.g. before the first partition) is broken. Not only will
this broken software (potentially) conflict with the software that
"owns" the MBR (not just GRUB), but it will also (potentially)
conflict with any other pieces of broken software.

It does not matter if the broken software is proprietory of not, or if
broken software restrict users' freedom or not, or if broken software
is popular or well known or not, or if the broken software is
extremely useful and/or has an "excuse". All of these considerations
are irrelevant. The software touches something it shouldn't, therefore
it is broken.

I doubt that Windows is *directly* at fault. However, allowing broken
software access to these sectors is a security flaw. If Windows allows
software to tamper with these sectors, then what else does Windows
allow software to tamper with? Does Windows allow software to install
a virus in the MBR? Does Windows allow software to install a virus in
one or more sectors that are loaded (and executed) by the MBR? At a
minimum Windows should have a UAC warning (a dialog box requiring
admin privileges that alerts the user to the attempted access, and
gives them the option to deny permission), but I wouldn't be surprised
if there's none. If there is no such warning, then the issue should be
reported to Microsoft as a security vulnerability, because that's
exactly what it is.

While the situation is unfortunate, Vladimir's suggested use of
error-correcting code is a very good idea. However, I think it should
go one step further. If the user has several OSs, and each OS happens
to have several pieces of broken software that trash different
sectors, then simply avoiding those trashed sectors won't stop the
system from becoming unbootable. GRUB has to restore the rightful
contents of trashed sectors during boot to minimise the total number
of sectors that are trashed at any point in time. In addition to
increasing GRUB's ability to tolerate trashed sectors, this would also
help to discourage broken software (and possibly, make it easier for
users to identify which piece/s of software is broken).

As a service to end-users; it would also be very nice if GRUB
displayed a "high visibility" warning that something is tampering with
the system's security and attempted to identify which pieces of
software may be tampering with the system's security (so the user can
more easily identify the cause). This warning could/should include a
URL for a web page that explains the issue in more detail, and maybe a
hex dump from each trashed sector so that signatures can be obtained
more easily.

Also, as a service to end-users; if/when broken software is correctly
identified the publisher of that software should be informed of their
mistake - something like a formal email that (politely) explains to
them why their software is broken and (even more politely) requests
them to fix their broken software.


Cheers,

Brendan

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Richard Stallman]

    > It appears that, rather than the operating system itself being at fault,
    > a number of Windows applications take over a sector in the boot track
    > and store bits and pieces of data there.

I am surprised applications can do that.  Isn't that a security hole
in Windows?

As for the decision at hand, I don't have an opinion.

Re: Guidance on conflicts between GNU GRUB and proprietary software

[richardvoigt]

[-- Attachment #1: Type: text/plain, Size: 1563 bytes --]

On Fri, Sep 24, 2010 at 9:09 AM, Richard Stallman wrote:

>    > It appears that, rather than the operating system itself being at
> fault,
>    > a number of Windows applications take over a sector in the boot track
>    > and store bits and pieces of data there.
>
> I am surprised applications can do that.  Isn't that a security hole
> in Windows?
>

Is it a security hole if the linux superuser can write to /dev/sda ?  If you
block this level of access, how's fdisk (or any number of other partition
managers) supposed to do its job?  How's one supposed to install grub in the
first place, if access to those blocks is forbidden by every OS?

We are talking about the owner of the machine, and software they choose to
run.  An OS that prevented the owner from having full control over his own
machine would be something to complain about, letting the owner write to his
boot track is not.

If we think there's a real security hole here, like unprivileged
applications able to overwrite grub code, I'll go advocate with Microsoft to
have it fixed.  Until then I just agree that it's a tragedy that
applications which aren't bootloaders or partition managers mess around in
this area, but it shouldn't be up to the OS to decide which applications run
by the superuser are specially privileged to manage partitions, and which
aren't.

Yours truly,
R Benjamin Voigt
Microsoft Visual C++ MVP and Windows private beta tester

P.S. Does anyone know if the Linux versions of those same proprietary
license managers abuse the boot track like their Windows behavior?

[-- Attachment #2: Type: text/html, Size: 1967 bytes --]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Bogdan]

Another potential solution that I have not seen proposed - GRUB could create its 
own partition in order to reserve space.

> P.S. Does anyone know if the Linux versions of those same proprietary license 
> managers abuse the boot track like their Windows behavior?

That's a good point although I don't consider this to be an affront to Windows, 
rather it was an affront to proprietary software (which I fully agree with - but 
this particular problem is very silly).

Cheers,
Bogdan



      

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Colin Watson]

On Mon, Sep 27, 2010 at 09:55:17PM -0700, Bogdan wrote:
> Another potential solution that I have not seen proposed - GRUB could create its 
> own partition in order to reserve space.

We do this - well, we expect that the user or the OS installer does this
- on GPT, as it's obviously the right and proper thing to do there.  On
MBR, it's unfortunately mostly impractical for a couple of reasons (the
first is the most important, though the second does come back to bite
you):

  * The BIOS can often only read from relatively near the start of the
    disk, and core.img must be readable by the BIOS.  If some other
    operating system is already installed - the common dual-boot case,
    and the case where this problem is overwhelmingly most likely to
    matter - it's likely to occupy a large stretch of partitioned space
    right after the boot track.

  * The MBR format has so many irritating restrictions on primary
    partitions that the more partitions an operating system needs to
    create by default, the more stress we put on partitioning
    algorithms.  (Most people don't notice any of this until they try to
    install on a machine whose OEM helpfully created three or four
    primary partitions already.)

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Vladimir 'φ-coder/phcoder' Serbinenko]

On 09/24/2010 04:09 PM, Richard Stallman wrote:
>     > It appears that, rather than the operating system itself being at fault,
>     > a number of Windows applications take over a sector in the boot track
>     > and store bits and pieces of data there.
>
> I am surprised applications can do that.  Isn't that a security hole
> in Windows?
>
>   
The windows users only relatively recently started discovering the
privilege separation as on windows till XP default user had complete
root privilegies. From Vista on, microsoft introduced gksudo-like
mechanism but most users have a reflex to press "accept" without even
reading the message since too many programs have the old habit of doing
unnecessary operations requiring root privilegies (like saving
configuration system-wide, rather than user-wide). Various backup
programs can validly be runned as root. Whereas it's possible that they
may have a relatively sane reason to write to MBR gap, I still have to
see a such. The ones I've seen use it to avoid restoring Windows to an
"unlicensed" ("untatooed") disk. Fortunately this use faded out since
many years (when "tatooing" moved to ACPI tables). Other programs
install a backdoor in the installer (which itself is run as root). Here
the problem is that users accept the backdoors running for DRM purposes.
An unrelated but similar example is FreeOTFE which installs a driver
which among things allows any unprivelegied user to read and write
sectors on the disk. I informed the author (who pretends to be a
security expert), but she doesn't see it as a security hole or anything
that should be fixed. Although FreeOTFE doesn't write in MBR gap, this
example shows that most of windows users and even some "security
experts" couldn't care less about security models (but they do care when
marketers say "security"-related buzzwords).
> As for the decision at hand, I don't have an opinion.
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>
>   


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Bogdan]

>  * The BIOS can often only read from relatively near the start of the

>    disk, and core.img must be readable by the BIOS.  If some other
>    operating system is already installed - the common dual-boot case,
>    and the case where this problem is overwhelmingly most likely to
>    matter - it's likely to occupy a large stretch of partitioned space
>    right after the boot track.

Are you talking about the conventional INT 13h interface?

>  * The MBR format has so many irritating restrictions on primary
>    partitions that the more partitions an operating system needs to
>    create by default, the more stress we put on partitioning
>    algorithms.  (Most people don't notice any of this until they try to
>    install on a machine whose OEM helpfully created three or four
>    primary partitions already.)

Only one of the 4 primary partitions can be active. So, turn whatever is not 
needed into an extended partition. What tools with this screw over? Or, are we 
talking about OSes that don't support extended partitions here?

Cheers,
Bogdan



      

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Colin Watson]

On Tue, Sep 28, 2010 at 02:10:20AM -0700, Bogdan wrote:
> >  * The BIOS can often only read from relatively near the start of the
> >    disk, and core.img must be readable by the BIOS.  If some other
> >    operating system is already installed - the common dual-boot case,
> >    and the case where this problem is overwhelmingly most likely to
> >    matter - it's likely to occupy a large stretch of partitioned space
> >    right after the boot track.
> 
> Are you talking about the conventional INT 13h interface?

Yes, which is the only thing that will fit into the boot sector which
needs to read everything else.

> >  * The MBR format has so many irritating restrictions on primary
> >    partitions that the more partitions an operating system needs to
> >    create by default, the more stress we put on partitioning
> >    algorithms.  (Most people don't notice any of this until they try to
> >    install on a machine whose OEM helpfully created three or four
> >    primary partitions already.)
> 
> Only one of the 4 primary partitions can be active. So, turn whatever is not 
> needed into an extended partition. What tools with this screw over? Or, are we 
> talking about OSes that don't support extended partitions here?

It rather depends on the pattern of free space that's available;
consider for example the case where there's already a logical partition
but there is another primary partition between it and the space you
actually want to use.  The precise details are off-topic for this list,
though.

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Bogdan]

> On Tue, Sep 28, 2010 at 02:10:20AM -0700, Bogdan wrote:

> > >  * The BIOS can often only read from relatively near the start of the
> > >    disk, and core.img must be readable by the BIOS.  If some other
> > >    operating system is already installed - the common dual-boot case,
> > >    and the case where this problem is overwhelmingly most likely to
> > >    matter - it's likely to occupy a large stretch of partitioned space
> > >    right after the boot track.
> > 
> > Are you talking about the conventional INT 13h interface?
>
> Yes, which is the only thing that will fit into the boot sector which
> needs to read everything else.

I disagree. You have more than enough space for both EDD and conventional INT 
13h. If you're thinking about something like ATA/ATAPI, sure, that's just crazy. 
Then, you chainload the boot sector of the GRUB partition and that does all the 
work that's necessary. If, for whatever reason, EDD is not available, the only 
sane thing to do is to move all partitions to the end of the disk and have the 
GRUB partition first (yes, I'm aware that's slow but at least it's sane).

> > >  * The MBR format has so many irritating restrictions on primary
> > >    partitions that the more partitions an operating system needs to
> > >    create by default, the more stress we put on partitioning
> > >    algorithms.  (Most people don't notice any of this until they try to
> > >    install on a machine whose OEM helpfully created three or four
> > >    primary partitions already.)
> > 
> > Only one of the 4 primary partitions can be active. So, turn whatever is not 

> > needed into an extended partition. What tools with this screw over? Or, are 
>we 
>
> > talking about OSes that don't support extended partitions here?
>
> It rather depends on the pattern of free space that's available;
> consider for example the case where there's already a logical partition
> but there is another primary partition between it and the space you
> actually want to use.  The precise details are off-topic for this list,
> though.

Here too, you can just move partitions around a bit (the same comment as above 
applies).

I can't see why not since the precise details influence the design of GRUB. 
However, if there's a better place for this, I will happily talk about this 
there.

Cheers,
Bogdan



      

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Colin Watson]

On Tue, Sep 28, 2010 at 02:51:01AM -0700, Bogdan wrote:
> > On Tue, Sep 28, 2010 at 02:10:20AM -0700, Bogdan wrote:
> > Yes, which is the only thing that will fit into the boot sector which
> > needs to read everything else.
> 
> I disagree. You have more than enough space for both EDD and conventional INT 
> 13h. If you're thinking about something like ATA/ATAPI, sure, that's just crazy. 
> Then, you chainload the boot sector of the GRUB partition and that does all the 
> work that's necessary. If, for whatever reason, EDD is not available, the only 
> sane thing to do is to move all partitions to the end of the disk and have the 
> GRUB partition first (yes, I'm aware that's slow but at least it's sane).

EDD is still INT 13h, last I checked - just function 42h rather than 2h.
We use that.  Nevertheless, we still get occasional reports of problems
due to BIOS limitations.

> > It rather depends on the pattern of free space that's available;
> > consider for example the case where there's already a logical partition
> > but there is another primary partition between it and the space you
> > actually want to use.  The precise details are off-topic for this list,
> > though.
> 
> Here too, you can just move partitions around a bit (the same comment as above 
> applies).

Moving partitions in an OS installer as a prerequisite for actually
working properly (which is a large part of what I care about given my
work on Debian and Ubuntu) is far too unsafe, especially if the size of
the partition exceeds half the size of the disk, as it often will.
Users can of course ask for it if they're clueful, but clueful users are
unlikely to run into this problem in the first place!

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Bogdan]

> EDD is still INT 13h, last I checked - just function 42h rather than 2h.

> We use that.  Nevertheless, we still get occasional reports of problems
> due to BIOS limitations.

That's why I used the term "conventional INT 13h interface."

> Moving partitions in an OS installer as a prerequisite for actually
> working properly (which is a large part of what I care about given my
> work on Debian and Ubuntu) is far too unsafe, especially if the size of
> the partition exceeds half the size of the disk, as it often will.

Unfortunately, it's the only solution I can think of at the present time. GRUB 
can't interpret all file systems and having a data base of all applications that 
mess with the boot track is even worse IMO.

> Users can of course ask for it if they're clueful, but clueful users are
> unlikely to run into this problem in the first place!

That's a good point. 

Cheers,
Bogdan



      

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Colin Watson]

On Tue, Sep 28, 2010 at 03:40:34AM -0700, Bogdan wrote:
> > Moving partitions in an OS installer as a prerequisite for actually
> > working properly (which is a large part of what I care about given my
> > work on Debian and Ubuntu) is far too unsafe, especially if the size of
> > the partition exceeds half the size of the disk, as it often will.
> 
> Unfortunately, it's the only solution I can think of at the present time. GRUB 
> can't interpret all file systems and having a data base of all applications that 
> mess with the boot track is even worse IMO.

Well, as a developer of an OS installer I can tell you that it's
certainly not workable for us.  Sorry.

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Lennart Sorensen]

On Tue, Sep 28, 2010 at 02:10:20AM -0700, Bogdan wrote:
> Only one of the 4 primary partitions can be active. So, turn whatever is not 
> needed into an extended partition. What tools with this screw over? Or, are we 
> talking about OSes that don't support extended partitions here?

That would probably require resizing the filesystem since a logical
partition needs to leave space for the extended partition table, not to
mentioned screw up alignment of partitions too.

-- 
Len Sorensen

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Lennart Sorensen]

On Mon, Sep 27, 2010 at 11:44:10PM -0500, richardvoigt@gmail.com wrote:
> Is it a security hole if the linux superuser can write to /dev/sda ?  If you
> block this level of access, how's fdisk (or any number of other partition
> managers) supposed to do its job?  How's one supposed to install grub in the
> first place, if access to those blocks is forbidden by every OS?
> 
> We are talking about the owner of the machine, and software they choose to
> run.  An OS that prevented the owner from having full control over his own
> machine would be something to complain about, letting the owner write to his
> boot track is not.
> 
> If we think there's a real security hole here, like unprivileged
> applications able to overwrite grub code, I'll go advocate with Microsoft to
> have it fixed.  Until then I just agree that it's a tragedy that
> applications which aren't bootloaders or partition managers mess around in
> this area, but it shouldn't be up to the OS to decide which applications run
> by the superuser are specially privileged to manage partitions, and which
> aren't.

Given the majority of Windows user's skill level, it probably is a
security hole.  Asking for permission to do administrator work is one
thing.  Asking for permission to do raw disk access is quite another
(and windows makes no distinction at all between the two).  After all
installing shared files and services requires admin.  This should not
give the installer permission to do far more than that.

If Microsoft cared about their users they would not give out 'windows
compatible' logos to anything that pulled of dirty tricks like writing
to track 0.  Of course that would never happen.

After all this has already killed full drive encryption for some people,
and it is killing boot loaders for some people too.  It is obviously a
bad idea and should be stopped.  Microsoft could probably put a stop to
it, but I doubt they care.  Wouldn't want to offend Adobe or Autodesk
or some other big important windows software provider.

> Yours truly,
> R Benjamin Voigt
> Microsoft Visual C++ MVP and Windows private beta tester
> 
> P.S. Does anyone know if the Linux versions of those same proprietary
> license managers abuse the boot track like their Windows behavior?

Based on what I have managed to find out about it, no, the Linux versions
don't do this.  Perhaps the developers figured linux users were smarter
and knew how to spot stupid misuse of sectors on their disks and work
around it, so it wouldn't be effective anyhow.

-- 
Len Sorensen

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Bogdan]

> On Tue, Sep 28, 2010 at 02:10:20AM -0700, Bogdan wrote:

> > Only one of the 4 primary partitions can be active. So, turn whatever is not 

> > needed into an extended partition. What tools with this screw over? Or, are 
>we 
>
> > talking about OSes that don't support extended partitions here?
>
> That would probably require resizing the filesystem since a logical
> partition needs to leave space for the extended partition table, not to
> mentioned screw up alignment of partitions too.

I'd be more worried about the space required by the actual partition, not the 
space required for the partition table (that requires 1 sector only). Either 
way, it the user has no more free space when (s)he wishes to install GRUB, (s)he 
would be asked to resize a partition on his/her own (since otherwise GRUB would 
need to interpret every file system out there).

Cheers,
Bogdan



      

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Phillip Susi]

On 9/28/2010 4:04 AM, Colin Watson wrote:
>   * The BIOS can often only read from relatively near the start of the
>     disk, and core.img must be readable by the BIOS.  If some other
>     operating system is already installed - the common dual-boot case,
>     and the case where this problem is overwhelmingly most likely to
>     matter - it's likely to occupy a large stretch of partitioned space
>     right after the boot track.

If int13 can't access the whole disk then it does not really matter
where the grub core image is since it still has to use int13 to load the
kernel.  If the bios is broken, then grub might load, but can't load
your kernel.  Worse yet, it might all work fine when you install, then
you later upgrade something that rebuilds your initrd, and it happens to
get allocated further down the disk and suddenly you can't boot.  The
only sane way to deal with such a broken bios is to put all of /boot
near the start of the disk.

>   * The MBR format has so many irritating restrictions on primary
>     partitions that the more partitions an operating system needs to
>     create by default, the more stress we put on partitioning
>     algorithms.  (Most people don't notice any of this until they try to
>     install on a machine whose OEM helpfully created three or four
>     primary partitions already.)

With 3 partitions you should be fine since you only need one for the
extended partition.  What does the installer do now when all 4 are used?
 Doesn't it just bail out?  As long as you are creating an extended
partition to hold the root and swap logical partitions anyhow, you may
as well make one more for grub.

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Colin Watson]

On Tue, Sep 28, 2010 at 11:40:13AM -0400, Phillip Susi wrote:
> On 9/28/2010 4:04 AM, Colin Watson wrote:
> >   * The BIOS can often only read from relatively near the start of the
> >     disk, and core.img must be readable by the BIOS.  If some other
> >     operating system is already installed - the common dual-boot case,
> >     and the case where this problem is overwhelmingly most likely to
> >     matter - it's likely to occupy a large stretch of partitioned space
> >     right after the boot track.
> 
> If int13 can't access the whole disk then it does not really matter
> where the grub core image is since it still has to use int13 to load the
> kernel.

This is not true.  GRUB has an ata module which can be built into
core.img if necessary to read data from parts of the disk the BIOS can't
read.  It's not really solid enough to use by default, but people do
occasionally report success with it.

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Phillip Susi]

On 9/28/2010 12:18 PM, Colin Watson wrote:
> This is not true.  GRUB has an ata module which can be built into
> core.img if necessary to read data from parts of the disk the BIOS can't
> read.  It's not really solid enough to use by default, but people do
> occasionally report success with it.

But when it isn't done by default, or automatically when needed, then
you aren't any better off putting the core in the embed area by default.
 If they are smart enough to build a custom core with the ata driver,
surely they can also put the core in the embed area as well.

To put that another way, just because an ata core needs to be in the
embed area to get around broken bios, does not mean that a biosdisk core
does.

Grub2 Install Image

[Dee Sharpe]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]

Hello all,

Is there an install disk image for Grub2 as there was for Grub Legacy?  I find it hard to install Grub2 onto a VMWare partition. If not, is there another way to do this? I'm doing this from a machine running Win7.

Dee Sharpe

Dee Sharpe

People deserve more,
from themselves & each other!

Sent from my iPhone

[-- Attachment #2: Type: text/html, Size: 741 bytes --]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1837 bytes --]

On 09/28/2010 10:04 AM, Colin Watson wrote:
> On Mon, Sep 27, 2010 at 09:55:17PM -0700, Bogdan wrote:
>   
>> Another potential solution that I have not seen proposed - GRUB could create its 
>> own partition in order to reserve space.
>>     
> We do this - well, we expect that the user or the OS installer does this
> - on GPT, as it's obviously the right and proper thing to do there.  On
> MBR, it's unfortunately mostly impractical for a couple of reasons (the
> first is the most important, though the second does come back to bite
> you):
>
>   * The BIOS can often only read from relatively near the start of the
>     disk, and core.img must be readable by the BIOS.  If some other
>     operating system is already installed - the common dual-boot case,
>     and the case where this problem is overwhelmingly most likely to
>     matter - it's likely to occupy a large stretch of partitioned space
>     right after the boot track.
>
>   * The MBR format has so many irritating restrictions on primary
>     partitions that the more partitions an operating system needs to
>     create by default, the more stress we put on partitioning
>     algorithms.  (Most people don't notice any of this until they try to
>     install on a machine whose OEM helpfully created three or four
>     primary partitions already.)
>
>   
Using an embedding partition on msdos as an optional alternative (not as
replacement) to MBR gap is a clear possibility, good idea and was
proposed before but details are very unclear. Like:
- How to create such partition
- How does grub find it and ensures that it's an embedding partition?
Any false positive will result in data loss. Obviously no msdos type is
completely unused by now so it's not a way.

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1837 bytes --]

On 09/28/2010 10:04 AM, Colin Watson wrote:
> On Mon, Sep 27, 2010 at 09:55:17PM -0700, Bogdan wrote:
>   
>> Another potential solution that I have not seen proposed - GRUB could create its 
>> own partition in order to reserve space.
>>     
> We do this - well, we expect that the user or the OS installer does this
> - on GPT, as it's obviously the right and proper thing to do there.  On
> MBR, it's unfortunately mostly impractical for a couple of reasons (the
> first is the most important, though the second does come back to bite
> you):
>
>   * The BIOS can often only read from relatively near the start of the
>     disk, and core.img must be readable by the BIOS.  If some other
>     operating system is already installed - the common dual-boot case,
>     and the case where this problem is overwhelmingly most likely to
>     matter - it's likely to occupy a large stretch of partitioned space
>     right after the boot track.
>
>   * The MBR format has so many irritating restrictions on primary
>     partitions that the more partitions an operating system needs to
>     create by default, the more stress we put on partitioning
>     algorithms.  (Most people don't notice any of this until they try to
>     install on a machine whose OEM helpfully created three or four
>     primary partitions already.)
>
>   
Using an embedding partition on msdos as an optional alternative (not as
replacement) to MBR gap is a clear possibility, good idea and was
proposed before but details are very unclear. Like:
- How to create such partition
- How does grub find it and ensures that it's an embedding partition?
Any false positive will result in data loss. Obviously no msdos type is
completely unused by now so it's not a way.

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Lennart Sorensen]

On Tue, Sep 28, 2010 at 09:05:21PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> Using an embedding partition on msdos as an optional alternative (not as
> replacement) to MBR gap is a clear possibility, good idea and was
> proposed before but details are very unclear. Like:
> - How to create such partition
> - How does grub find it and ensures that it's an embedding partition?
> Any false positive will result in data loss. Obviously no msdos type is
> completely unused by now so it's not a way.

There are NO unused ones?  How did GPT get a hold of one then?
There should be plenty.

Much harder is making room for yet another partition.

Of course when using grub2 on an IBM powerpc, you have no choice about
where to put grub.  It must be written directly to a PReP boot partition,
since that's the only thing the firmware will attempt to boot from.
And it too uses msdos partitions it seems.  Now of course the PReP boot
partition type could not possibly have any meaning on an x86 system.
Could borrow that one. :)

-- 
Len Sorensen

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Phillip Susi]

On 9/28/2010 3:05 PM, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> Using an embedding partition on msdos as an optional alternative (not as
> replacement) to MBR gap is a clear possibility, good idea and was
> proposed before but details are very unclear. Like:
> - How to create such partition

The same way you create any other partition?

> - How does grub find it and ensures that it's an embedding partition?
> Any false positive will result in data loss. Obviously no msdos type is
> completely unused by now so it's not a way.

There seem to be quite a few left that are at least unknown to linux
fdisk.  If you really want to make sure, then you can require a grub
signature in the starting sector of the partition.  Also this step would
usually be done by the installer, which would know that the partition is
a grub partition since it just created it.

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1898 bytes --]

On 09/28/2010 09:15 PM, Lennart Sorensen wrote:
> On Tue, Sep 28, 2010 at 09:05:21PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>   
>> Using an embedding partition on msdos as an optional alternative (not as
>> replacement) to MBR gap is a clear possibility, good idea and was
>> proposed before but details are very unclear. Like:
>> - How to create such partition
>> - How does grub find it and ensures that it's an embedding partition?
>> Any false positive will result in data loss. Obviously no msdos type is
>> completely unused by now so it's not a way.
>>     
> There are NO unused ones?  How did GPT get a hold of one then?
> There should be plenty.
>   
GPT has new types.
in msdos there are only 255 possible types. Some are known to be used by
well known software but during 10 years of no central body for this and
everybody willing a type just taking one in self-service I think every
single type is used by either widely known or mostly unknown software.
And destroying someone's hobby OS would be a bad thing.
> Much harder is making room for yet another partition.
>
> Of course when using grub2 on an IBM powerpc, you have no choice about
> where to put grub.  It must be written directly to a PReP boot partition,
> since that's the only thing the firmware will attempt to boot from.
> And it too uses msdos partitions it seems.  Now of course the PReP boot
> partition type could not possibly have any meaning on an x86 system.
> Could borrow that one. :)
>
>   
GRUB has a design principle of being cross-platform installable.
Moreover the same disk can contain multiple grub installation. I
personally regularly move the disk between yeeloong and amd64 laptop,
well it has only one GRUB since on yeeloong my GRUB is in flash but it
could easily have one on disk too.


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Lennart Sorensen]

On Tue, Sep 28, 2010 at 09:43:25PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> GPT has new types.

GPT has an msdos partition type for itself for use in hybrid setups.
I know GPT partition tables have new types, but GPT itself has a type
reserved in the old dos partition table.

> in msdos there are only 255 possible types. Some are known to be used by
> well known software but during 10 years of no central body for this and
> everybody willing a type just taking one in self-service I think every
> single type is used by either widely known or mostly unknown software.
> And destroying someone's hobby OS would be a bad thing.

And not having a reliable grub on actual used OSs isn't a bad thing?
Someone's hobby OS could change if it was an issue.

> GRUB has a design principle of being cross-platform installable.
> Moreover the same disk can contain multiple grub installation. I
> personally regularly move the disk between yeeloong and amd64 laptop,
> well it has only one GRUB since on yeeloong my GRUB is in flash but it
> could easily have one on disk too.

If two architectures expect sector 0 to contain boot code, then that
can't work.  I certainly would not consider that a worthy design goal
compared to lots of other things.

-- 
Len Sorensen

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2996 bytes --]

On 09/28/2010 10:07 PM, Lennart Sorensen wrote:
> On Tue, Sep 28, 2010 at 09:43:25PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>   
>> GPT has new types.
>>     
> GPT has an msdos partition type for itself for use in hybrid setups.
> I know GPT partition tables have new types, but GPT itself has a type
> reserved in the old dos partition table.
>   
You probably mean the 0xee type. But it's used only to mark the whole
space as used. In our case it's a partition which is identified to have
all the data deleted. Let's just take a famous collision between Solaris
and Linux swap. I doubt that any of them willingly choose the type in
order to collide with other. If Linux relied solely on the partition
type to identify its swap it would be a disaster for dual-boot system.
>   
>> in msdos there are only 255 possible types. Some are known to be used by
>> well known software but during 10 years of no central body for this and
>> everybody willing a type just taking one in self-service I think every
>> single type is used by either widely known or mostly unknown software.
>> And destroying someone's hobby OS would be a bad thing.
>>     
> And not having a reliable grub on actual used OSs isn't a bad thing?
> Someone's hobby OS could change if it was an issue.
>
>   
Destroying the data which is on its rightful place is bad independently
what you use the place for, how important your usage is or how
"unimportant" you judge the current occupant.
I believe it's possible to have something something much more reliable.
We could have a tool grub-mkembed (analog of mkswap) which would mark
the partition as available for GRUB embedding (perhaps in addition of
checking type). This signature must be written in a way to be
overwritten when formatted in known filesystems
>> GRUB has a design principle of being cross-platform installable.
>> Moreover the same disk can contain multiple grub installation. I
>> personally regularly move the disk between yeeloong and amd64 laptop,
>> well it has only one GRUB since on yeeloong my GRUB is in flash but it
>> could easily have one on disk too.
>>     
> If two architectures expect sector 0 to contain boot code, then that
> can't work.  I certainly would not consider that a worthy design goal
> compared to lots of other things.
>
>   
Some architestures are incompatible because of such reasons but many
others don't conflict in such ways.
When you abandon a design goal or give an exception you first have to
make sure that there is no way to reconcile the given features.
Just one example: I'm ready to give an exception to multiterm design in
order to get the features required for ubuntu CDs but first I discussed
in order to find compromise which would result in less mess on codepath
intersections and it looks like there is actually one.
In this case taking PReP partition type would be unfounded.


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Lennart Sorensen]

On Tue, Sep 28, 2010 at 10:58:31PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> On 09/28/2010 10:07 PM, Lennart Sorensen wrote:
> > On Tue, Sep 28, 2010 at 09:43:25PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> >   
> >> GPT has new types.
> >>     
> > GPT has an msdos partition type for itself for use in hybrid setups.
> > I know GPT partition tables have new types, but GPT itself has a type
> > reserved in the old dos partition table.
> >   
> You probably mean the 0xee type. But it's used only to mark the whole
> space as used. In our case it's a partition which is identified to have
> all the data deleted. Let's just take a famous collision between Solaris
> and Linux swap. I doubt that any of them willingly choose the type in
> order to collide with other. If Linux relied solely on the partition
> type to identify its swap it would be a disaster for dual-boot system.
 
Certainly true. Now there clearly are unused types.  On the other
hand given the lack of partition entries in the first place, needing a
partition isn't very convinient at all.  It might be a nice option to
support though.  Of course I doubt anything prevents the user of a
partition for grub already, given you could use an MBR that just goes
to the active partition (ie: standard DOS/Windows behaviour), and then
have grub be on that active partition, whatever the type may be.

> Destroying the data which is on its rightful place is bad independently
> what you use the place for, how important your usage is or how
> "unimportant" you judge the current occupant.

Well grub should only install where someone tells it to.

> I believe it's possible to have something something much more reliable.
> We could have a tool grub-mkembed (analog of mkswap) which would mark
> the partition as available for GRUB embedding (perhaps in addition of
> checking type). This signature must be written in a way to be
> overwritten when formatted in known filesystems

Not sure you can pick a place and be certain all filesystems will
overwrite it on format.  You can try, but it won't always work.

> >> GRUB has a design principle of being cross-platform installable.
> >> Moreover the same disk can contain multiple grub installation. I
> >> personally regularly move the disk between yeeloong and amd64 laptop,
> >> well it has only one GRUB since on yeeloong my GRUB is in flash but it
> >> could easily have one on disk too.
> >>     
> > If two architectures expect sector 0 to contain boot code, then that
> > can't work.  I certainly would not consider that a worthy design goal
> > compared to lots of other things.
> >
> >   
> Some architestures are incompatible because of such reasons but many
> others don't conflict in such ways.
> When you abandon a design goal or give an exception you first have to
> make sure that there is no way to reconcile the given features.

Nice to support if possible, although given how short on partitions you
are already with msdos partitions is really seems futile.

> Just one example: I'm ready to give an exception to multiterm design in
> order to get the features required for ubuntu CDs but first I discussed
> in order to find compromise which would result in less mess on codepath
> intersections and it looks like there is actually one.
> In this case taking PReP partition type would be unfounded.

Well I think using a partition at all in the case of the msdos partition
table is a huge inconvinience to people, and I suspect many won't be
able to.

it has become annoyingly common to see:

System restore partition
Windows System partition
Windows partition

That leaves one primary partition on a typical system these days.  So to
make more than one partition, that one has to be extended.  Now where
can grub go?

If the system maker had been nice, they would have used GPT instead and
those 3 partitions would not have been a problem.  But of course windows
doesn't know how to boot from GPT on a normal BIOS based system (unlike
most other OSs that have no such problem).

If people are dual booting, using the track 0 area may be a bad thing.
Unfortunately people that a dual booting are most likely to have partition
limitations making it the only option that works.

-- 
Len Sorensen

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 5295 bytes --]

On 09/28/2010 11:15 PM, Lennart Sorensen wrote:
> On Tue, Sep 28, 2010 at 10:58:31PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>   
>> On 09/28/2010 10:07 PM, Lennart Sorensen wrote:
>>     
>>> On Tue, Sep 28, 2010 at 09:43:25PM +0200, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>>>   
>>>       
>>>> GPT has new types.
>>>>     
>>>>         
>>> GPT has an msdos partition type for itself for use in hybrid setups.
>>> I know GPT partition tables have new types, but GPT itself has a type
>>> reserved in the old dos partition table.
>>>   
>>>       
>> You probably mean the 0xee type. But it's used only to mark the whole
>> space as used. In our case it's a partition which is identified to have
>> all the data deleted. Let's just take a famous collision between Solaris
>> and Linux swap. I doubt that any of them willingly choose the type in
>> order to collide with other. If Linux relied solely on the partition
>> type to identify its swap it would be a disaster for dual-boot system.
>>     
>  
> Certainly true. Now there clearly are unused types.
Rather "not widely known to be used".
>   On the other
> hand given the lack of partition entries in the first place, needing a
> partition isn't very convinient at all.  
I never said "replace current method with another one" but "add another
one as an option"
> It might be a nice option to
> support though.  Of course I doubt anything prevents the user of a
> partition for grub already, given you could use an MBR that just goes
> to the active partition (ie: standard DOS/Windows behaviour), and then
> have grub be on that active partition, whatever the type may be.
>
>   
You confuse /boot and embedding.
>> Destroying the data which is on its rightful place is bad independently
>> what you use the place for, how important your usage is or how
>> "unimportant" you judge the current occupant.
>>     
> Well grub should only install where someone tells it to.
>
>   
You confuse again. Where boot code goes is specificied on command line.
Embedding zone is chosed in function of partition map. See "Re:
[grub-setup] New procedure to choose the embedding area" on 09/15/2010
10:11 PM +0200
>> I believe it's possible to have something something much more reliable.
>> We could have a tool grub-mkembed (analog of mkswap) which would mark
>> the partition as available for GRUB embedding (perhaps in addition of
>> checking type). This signature must be written in a way to be
>> overwritten when formatted in known filesystems
>>     
> Not sure you can pick a place and be certain all filesystems will
> overwrite it on format.  You can try, but it won't always work.
>
>   
We can put multiple signatures.
>>>> GRUB has a design principle of being cross-platform installable.
>>>> Moreover the same disk can contain multiple grub installation. I
>>>> personally regularly move the disk between yeeloong and amd64 laptop,
>>>> well it has only one GRUB since on yeeloong my GRUB is in flash but it
>>>> could easily have one on disk too.
>>>>     
>>>>         
>>> If two architectures expect sector 0 to contain boot code, then that
>>> can't work.  I certainly would not consider that a worthy design goal
>>> compared to lots of other things.
>>>
>>>   
>>>       
>> Some architestures are incompatible because of such reasons but many
>> others don't conflict in such ways.
>> When you abandon a design goal or give an exception you first have to
>> make sure that there is no way to reconcile the given features.
>>     
> Nice to support if possible, although given how short on partitions you
> are already with msdos partitions is really seems futile.
>
>   
Logical partitions are fine for most platforms.
>> Just one example: I'm ready to give an exception to multiterm design in
>> order to get the features required for ubuntu CDs but first I discussed
>> in order to find compromise which would result in less mess on codepath
>> intersections and it looks like there is actually one.
>> In this case taking PReP partition type would be unfounded.
>>     
> Well I think using a partition at all in the case of the msdos partition
> table is a huge inconvinience to people, and I suspect many won't be
> able to.
>
> it has become annoyingly common to see:
>
> System restore partition
> Windows System partition
> Windows partition
>
> That leaves one primary partition on a typical system these days.  So to
> make more than one partition, that one has to be extended.  Now where
> can grub go?
>
>   
Logical partition for embedding is fine too.
> If the system maker had been nice, they would have used GPT instead and
> those 3 partitions would not have been a problem.  But of course windows
> doesn't know how to boot from GPT on a normal BIOS based system (unlike
> most other OSs that have no such problem).
>   
see "gptsync"
> If people are dual booting, using the track 0 area may be a bad thing.
> Unfortunately people that a dual booting are most likely to have partition
> limitations making it the only option that works.
>
>   
s,people dualbooting, people having crapware + some other cases.


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Grub2 Install Image

[Dmitry Ilyin]

> 
> Hello all,
> Is there an install disk image for Grub2 as there was for Grub Legacy? I find
> it hard to install Grub2 onto a VMWare partition. If not, is there another way
> to do this? I'm doing this from a machine running Win7.

It's very easy, just boot your virtual machine from cd image. You need to boot into OS that can install 
grub2. Prepare your disk, mount on /mnt, run grub-install --root-directory=/mnt /your/disk.

You can use this:
http://www.supergrubdisk.org/ Very helpful with grub
http://rip.7bf.de/current/ RIP Linux best linux rescue system ever! Boots into RAM and you can do any disk 
related stuff with it, including grub2 install.

Any other linux live cd with grub will do the job.

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Grégoire Sutre]

On 09/28/2010 09:05 PM, Vladimir 'φ-coder/phcoder' Serbinenko wrote:

> Using an embedding partition on msdos as an optional alternative (not as
> replacement) to MBR gap is a clear possibility, good idea and was
> proposed before but details are very unclear. Like:
> - How to create such partition
> - How does grub find it and ensures that it's an embedding partition?
> Any false positive will result in data loss. Obviously no msdos type is
> completely unused by now so it's not a way.

We could borrow a partition type that is reliably used only by boot 
managers (for similar purposes).  Ok, we may overwrite some boot manager 
code sitting in that partition.  But would this be different from what 
happens in the GPT case?

Regarding the question of whether such an MSDOS partition type exists, I 
did not look very hard, but 45h comes to mind.  This type is used by the 
Boot-US boot manager.  It is reported as such by NetBSD fdisk.  It is 
not listed in the known types of Linux fdisk.  Are you aware of other 
uses for this partition type?

If we really want to embed in an MSDOS partition, selecting a partition 
type that is already used for similar purposes is IMHO our best option. 
This would be a step in a direction to set a ``standard'' MSDOS type for 
boot partitions.  We don't need one partition type per boot-loader 
software anyway.

Grégoire

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1757 bytes --]

On 09/28/2010 11:46 PM, Grégoire Sutre wrote:
> Regarding the question of whether such an MSDOS partition type exists,
> I did not look very hard, but 45h comes to mind.  This type is used by
> the Boot-US boot manager.  It is reported as such by NetBSD fdisk.  It
> is not listed in the known types of Linux fdisk.  Are you aware of
> other uses for this partition type?
Eumel according to
http://www.win.tue.nl/~aeb/partitions/partition_types-1.html.
>
> If we really want to embed in an MSDOS partition, selecting a
> partition type that is already used for similar purposes is IMHO our
> best option. This would be a step in a direction to set a ``standard''
> MSDOS type for boot partitions.  We don't need one partition type per
> boot-loader software anyway.
OS/2 used type 0x0a. If someone wants to install the OS/2 on his
computer (I wouldn't attempt it on anything newer than Pentium 3) then
he probably wants to chainload it (direct loading OS/2 would require
much work and has no point, adapting ntldr would be possible though, but
I wouldn't spend any time on it). So he needs 2 boot managers. Probably
there are other similar cases as well.
Deleting some other bootloader may also appear unfair and lead to data
loss if its partition contained anything useful. I don't mind sharing
the embedding partition on the rule "who gets MBR gets this bonus, all
possible occupants have to implement multiboot to be chainloaded if
necessary" but former users of partition type may disagree.
>
> Grégoire
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Guidance on conflicts between GNU GRUB and proprietary software

[Grégoire Sutre]

On 09/29/2010 00:11, Vladimir 'φ-coder/phcoder' Serbinenko wrote:

Let me try to explain more clearly what I have in mind.  In the GPT
case, we have a dedicated partition ID for bootloader software.  This
makes things simpler.  If we had such a dedicated partition ID in the
MSDOS case, then a lot of embedding problems would (a priori) go away.
So it's worth trying to ``establish'' such a partition ID for MSDOS
partitions.  And good candidates are partition IDs that are already
(almost exclusively) used by some bootloader software(s), because

- those partitions are very likely to only contain bootloader code
   and configuration, so the risk of data loss when installing GRUB
   is minimized.

- OSes and partitioning software may already know about them, which
   would limit the risk of losing the embedded GRUB code.


>> If we really want to embed in an MSDOS partition, selecting a
>> partition type that is already used for similar purposes is IMHO
>> our best option. This would be a step in a direction to set a
>> ``standard'' MSDOS type for boot partitions.  We don't need one
>> partition type per boot-loader software anyway.
> OS/2 used type 0x0a. If someone wants to install the OS/2 on his
> computer (I wouldn't attempt it on anything newer than Pentium 3)
> then he probably wants to chainload it (direct loading OS/2 would
> require much work and has no point, adapting ntldr would be possible
>  though, but I wouldn't spend any time on it). So he needs 2 boot
> managers. Probably there are other similar cases as well.

I didn't mean that we only need one partition for bootloaders.  What I
mean is that we only need one partition type.  Such as, if you install
two Linux distributions, you would have the same ID for all your Linux
partitions.

But I agree that, if we had two MSDOS partitions with the same
``bootloader'' type, then grub-install shouldn't blindly take the first
one.  Still, isn't this what happens in GPT case, when there are two
BIOS Boot partitions?  Or maybe, in the GPT case, there is no point in
having several BIOS Boot partitions?

Anyway, I'm sure that we can find solutions to make sure that, in a
non-interactive mode, a new GRUB installation only overwrites a previous
GRUB installation.


> Deleting some other bootloader may also appear unfair and lead to
> data loss if its partition contained anything useful.

In the GPT case, doesn't grub-install delete existing data in the GPT
BIOS Boot partition?


> I don't mind sharing the embedding partition on the rule "who gets
> MBR gets this bonus,

I didn't see it this way.  I assumed we could have different bootloader
software in different partitions of the ``bootloader type''.


> all possible occupants have to implement multiboot to be chainloaded
> if necessary" but former users of partition type may disagree.

Yes, of course, if we try to go this route, we would have to discuss
it with former users of the partition type.

Grégoire