chained grub2 derivative bootauto system

chained grub2 derivative bootauto system

[ivo welch]

[-- Attachment #1: Type: text/plain, Size: 3137 bytes --]

dear grub2 developers:

I have wrestled often with the problems of setting up grub2 on new systems.
  I know booting is a low-level science in itself, so I don't dare to
pretend that I know anything.  but I wanted to put up a small starting
bounty of $500 for a grub2 derivative  type of boot loader, to be made
available GPL, of course, in the main linux distributions (such as ubuntu)
if one knowledgeable developer finds this interesting.


from the user perspective, this booting system should work as follows:

if the user holds any key during the boot process, the new "B" loader (call
it bootauto.bin) would scan all available partitions for bootable systems
(such as Windows, linux, freebsd, etc.) and all root partitions for *.iso
files, and present the user with a list of what it found where, and put the
default selection line on the OS that was most recently booted.  the user
should be able to select one of these, and then proceed booting from them.
 the user presumably could also enter command line options at this stage,
choose a common option (such as "rescue", "single user", or "single user
read-only"), or possibly see all kernels, including older ones.
 bootauto.bin obviously needs a whole lot more intelligence at boot time
than what grub2 has.

if the user does not hold down a key, then bootauto.bin would boot whatever
it booted last, without delay.

the setup is similar to an OSX boot, where holding down an "ALT" key
presents all bootable OS's that are found.

there would be no more grub configuration files, grub-install commands,
etc.  bootauto.bin would do it all.  bootauto.bin would presumably always
reside in a fixed spot, such as /bootauto.bin, and all that the boot sector
would have to do would be to find it and pass control to it.

from a user perspective, creating live USB flash sticks with multiple OS's,
or booting from another hard disk now becomes much simpler.  end users only
need to connect the bootable device or connect USB stick with a couple of
ISOs on them, and it just works.

the system-wide first-time installation of the bootloader would consist of
one command that copies the bootauto.bin file to a designated partition and
writes the bootsector.  "bootauto-install /dev/sda /mnt/sda1" would install
the boot sector on /dev/sda that chain loads the B loader bootauto.bin on
mnt/sda1/bootauto.bin (whatever file system /mnt/sda1/ uses; could be ntfs,
ext4, etc).  the only error should be that /mnt/sda1 cannot be written.  no
mysterious chroots, no --binds, no uuid's, no grub configuration file
consultation.  no problems if disks get rearranged on the next boot. simple.

it doesn't have to work on legacy systems more than 5 years old, either.
 this is to move forward. /bootauto.bin can be big.


if interested, send me a personal email, please.  I will pay upon
completion (or put it into an escrow account at the FSF or another
reasonable place).  maybe some others will supplement the funding---I know
that $500 won't pay for it all.  I just wanted to start the ball rolling,
and put my money where my mouth is.

sincerely,

/iaw
----
Ivo Welch (ivo.welch@gmail.com)

[-- Attachment #2: Type: text/html, Size: 3670 bytes --]

Re: chained grub2 derivative bootauto system

[Philip Rhoades]

Ivo,

Interesting idea - I particularly like the idea of booting from 
arbitrary isos.

Regards,

Phil.


On 2012-08-31 07:20, ivo welch wrote:
> dear grub2 developers:
>
> I have wrestled often with the problems of setting up grub2 on new
> systems.   I know booting is a low-level science in itself, so I dont
> dare to pretend that I know anything.  but I wanted to put up a small
> starting bounty of $500 for a grub2 derivative  type of boot loader,
> to be made available GPL, of course, in the main linux distributions
> (such as ubuntu) if one knowledgeable developer finds this
> interesting.
>
> from the user perspective, this booting system should work as 
> follows:
>
> if the user holds any key during the boot process, the new "B" loader
> (call it bootauto.bin) would scan all available partitions for
> bootable systems (such as Windows, linux, freebsd, etc.) and all root
> partitions for *.iso files, and present the user with a list of what
> it found where, and put the default selection line on the OS that was
> most recently booted.  the user should be able to select one of 
> these,
> and then proceed booting from them.  the user presumably could also
> enter command line options at this stage, choose a common option 
> (such
> as "rescue", "single user", or "single user read-only"), or possibly
> see all kernels, including older ones.  bootauto.bin obviously needs 
> a
> whole lot more intelligence at boot time than what grub2 has.
>
> if the user does not hold down a key, then bootauto.bin would boot
> whatever it booted last, without delay.
>
> the setup is similar to an OSX boot, where holding down an "ALT" key
> presents all bootable OSs that are found.
>
> there would be no more grub configuration files, grub-install
> commands, etc.  bootauto.bin would do it all.  bootauto.bin would
> presumably always reside in a fixed spot, such as /bootauto.bin, and
> all that the boot sector would have to do would be to find it and 
> pass
> control to it.
>
> from a user perspective, creating live USB flash sticks with multiple
> OSs, or booting from another hard disk now becomes much simpler.  end
> users only need to connect the bootable device or connect USB stick
> with a couple of ISOs on them, and it just works.
>
> the system-wide first-time installation of the bootloader would
> consist of one command that copies the bootauto.bin file to a
> designated partition and writes the bootsector.  "bootauto-install
> /dev/sda /mnt/sda1" would install the boot sector on /dev/sda that
> chain loads the B loader bootauto.bin on mnt/sda1/bootauto.bin
> (whatever file system /mnt/sda1/ uses; could be ntfs, ext4, etc). 
>  the
> only error should be that /mnt/sda1 cannot be written.  no mysterious
> chroots, no --binds, no uuids, no grub configuration file
> consultation.  no problems if disks get rearranged on the next boot.
> simple.
>
> it doesnt have to work on legacy systems more than 5 years old,
> either.  this is to move forward. /bootauto.bin can be big.
>
> if interested, send me a personal email, please.  I will pay upon
> completion (or put it into an escrow account at the FSF or another
> reasonable place).  maybe some others will supplement the funding---I
> know that $500 wont pay for it all.  I just wanted to start the ball
> rolling, and put my money where my mouth is.
>
> sincerely,
>
> /iaw----
> Ivo Welch (ivo.welch@gmail.com)
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel

-- 
Philip Rhoades

GPO Box 3411
Sydney NSW	2001
Australia
E-mail:  phil@pricom.com.au

Re: chained grub2 derivative bootauto system

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 3763 bytes --]

On 30.08.2012 23:20, ivo welch wrote:

> 
> dear grub2 developers:
> 
> I have wrestled often with the problems of setting up grub2 on new
> systems.   I know booting is a low-level science in itself, so I don't
> dare to pretend that I know anything.  but I wanted to put up a small
> starting bounty of $500 for a grub2 derivative  type of boot loader, to
> be made available GPL, of course, in the main linux distributions (such
> as ubuntu) if one knowledgeable developer finds this interesting.
> 
> 

You miss the fact that GRUB2 already has the abilities you describe with
corresponding config files (see e.g. isoscan.cfg).

> from the user perspective, this booting system should work as follows:
> 
> if the user holds any key during the boot process, the new "B" loader
> (call it bootauto.bin) would scan all available partitions for bootable
> systems (such as Windows, linux, freebsd, etc.) and all root partitions
> for *.iso files, and present the user with a list of what it found
> where, and put the default selection line on the OS that was most
> recently booted.  the user should be able to select one of these, and
> then proceed booting from them.  the user presumably could also enter
> command line options at this stage, choose a common option (such as
> "rescue", "single user", or "single user read-only"), or possibly see
> all kernels, including older ones.  bootauto.bin obviously needs a whole
> lot more intelligence at boot time than what grub2 has.
> 
> if the user does not hold down a key, then bootauto.bin would boot
> whatever it booted last, without delay.
> 
> the setup is similar to an OSX boot, where holding down an "ALT" key
> presents all bootable OS's that are found.
> 
> there would be no more grub configuration files, grub-install commands,
> etc.  bootauto.bin would do it all.  bootauto.bin would presumably
> always reside in a fixed spot, such as /bootauto.bin, and all that the
> boot sector would have to do would be to find it and pass control to it.
> 
> from a user perspective, creating live USB flash sticks with multiple
> OS's, or booting from another hard disk now becomes much simpler.  end
> users only need to connect the bootable device or connect USB stick with
> a couple of ISOs on them, and it just works.
> 
> the system-wide first-time installation of the bootloader would consist
> of one command that copies the bootauto.bin file to a designated
> partition and writes the bootsector.  "bootauto-install /dev/sda
> /mnt/sda1" would install the boot sector on /dev/sda that chain loads
> the B loader bootauto.bin on mnt/sda1/bootauto.bin (whatever file system
> /mnt/sda1/ uses; could be ntfs, ext4, etc).  the only error should be
> that /mnt/sda1 cannot be written.  no mysterious chroots, no --binds, no
> uuid's, no grub configuration file consultation.  no problems if disks
> get rearranged on the next boot. simple.
> 
> it doesn't have to work on legacy systems more than 5 years old, either.
>  this is to move forward. /bootauto.bin can be big.
> 
> 
> if interested, send me a personal email, please.  I will pay upon
> completion (or put it into an escrow account at the FSF or another
> reasonable place).  maybe some others will supplement the funding---I
> know that $500 won't pay for it all.  I just wanted to start the ball
> rolling, and put my money where my mouth is.
> 
> sincerely,
> 
> /iaw
> ----
> Ivo Welch (ivo.welch@gmail.com <mailto:ivo.welch@gmail.com>)
> 
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel



-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: chained grub2 derivative bootauto system

[Lennart Sorensen]

On Fri, Aug 31, 2012 at 09:19:43AM +1000, Philip Rhoades wrote:
> Ivo,
> 
> Interesting idea - I particularly like the idea of booting from
> arbitrary isos.

Too bad that accessing an iso on a usb key is nothing like an actual cd
or dvd and the sytem you boot must have support for the fact you put the
iso on the usb key rather than as a real disc.  Nothing the boot loader
can do to change that.

If it was really that simple, someone would already have made such a
boot loader for use on usb keys.  The reason it doesn't exist yet is
that it can't exist.

-- 
Len Sorensen

Re: chained grub2 derivative bootauto system

[Lennart Sorensen]

On Thu, Aug 30, 2012 at 02:20:22PM -0700, ivo welch wrote:
> dear grub2 developers:
> 
> I have wrestled often with the problems of setting up grub2 on new systems.
>   I know booting is a low-level science in itself, so I don't dare to
> pretend that I know anything.  but I wanted to put up a small starting
> bounty of $500 for a grub2 derivative  type of boot loader, to be made
> available GPL, of course, in the main linux distributions (such as ubuntu)
> if one knowledgeable developer finds this interesting.

I consider most of it impossible.  The bits that could be done are
already possible in grub2 as far as I know.

> from the user perspective, this booting system should work as follows:

If you want to change the user perspective, go get a time machine and
go get IBM/Microsoft/etc to design a much better boot interface than
the PC has had since the begining.

> if the user holds any key during the boot process, the new "B" loader (call
> it bootauto.bin) would scan all available partitions for bootable systems
> (such as Windows, linux, freebsd, etc.) and all root partitions for *.iso
> files, and present the user with a list of what it found where, and put the
> default selection line on the OS that was most recently booted.  the user
> should be able to select one of these, and then proceed booting from them.
>  the user presumably could also enter command line options at this stage,
> choose a common option (such as "rescue", "single user", or "single user
> read-only"), or possibly see all kernels, including older ones.
>  bootauto.bin obviously needs a whole lot more intelligence at boot time
> than what grub2 has.

You can't boot iso files in general.  You can have special handlers
for specific isos (such as debian or ubuntu or fedora install images)
that knows which files to look for in the iso and which arguments to
pass in order to get that installer to work with an iso rather than an
actual cd/dvd.  There is no way for the boot loader to make an iso on a
disk look like a cd/dvd drive in such a way that whatever you boot can
actually use it as a real cd/dvd.  Once you leave the boot loader it is
all up to the OS you boot to take care of things.  At best you get a
kernel, some ramdisk image, maybe a few modules and some arguments.
The OS is then on its own.

> if the user does not hold down a key, then bootauto.bin would boot whatever
> it booted last, without delay.

Great so now the boot loader has to store data somewhere.  Boot loaders
try to avoid writing.  Writing is much harder (and more dangerous)
than reading.  Especially to filesystems.

> the setup is similar to an OSX boot, where holding down an "ALT" key
> presents all bootable OS's that are found.

The Mac only handles the OSs it knows about, and it has pretty nice
firmware to take care of it (including persistent storage in the firmware,
rather than having to write to disk to remember things).  Also according
to what I read recently, Macs have only VERY recently started to allow
booting from USB.

> there would be no more grub configuration files, grub-install commands,
> etc.  bootauto.bin would do it all.  bootauto.bin would presumably always
> reside in a fixed spot, such as /bootauto.bin, and all that the boot sector
> would have to do would be to find it and pass control to it.

You can not fit code in a boot sector to find anything.  The boot sector
can only store enough to get something from a known location.
You typically have 448 bytes total in the boot sector.

This is why sensible system designs (IBM powerpc, EFI, etc) have a
specific boot partition instead of a boot sector.  That way you actually
have enough space for a boot loader.  The old PC BIOS based system is
awful that way.  The standard boot sector on a PC was simply a bit of
code that would read from the partition marked 'bootable' on drive 0x80
(first hard disk) and look in the root directory for a specific filename
to get the cluster chain to read.  FAT file system is VERY simple to read,
and by not supporting subdirectories it gets simpler yet.  And it just
barely fits in the boot sector.

> from a user perspective, creating live USB flash sticks with multiple OS's,
> or booting from another hard disk now becomes much simpler.  end users only
> need to connect the bootable device or connect USB stick with a couple of
> ISOs on them, and it just works.
> 
> the system-wide first-time installation of the bootloader would consist of
> one command that copies the bootauto.bin file to a designated partition and
> writes the bootsector.  "bootauto-install /dev/sda /mnt/sda1" would install
> the boot sector on /dev/sda that chain loads the B loader bootauto.bin on
> mnt/sda1/bootauto.bin (whatever file system /mnt/sda1/ uses; could be ntfs,
> ext4, etc).  the only error should be that /mnt/sda1 cannot be written.  no
> mysterious chroots, no --binds, no uuid's, no grub configuration file
> consultation.  no problems if disks get rearranged on the next boot. simple.

Actually rearranging would very much be a problem.  The boot sector
is tiny.  About the only thing it can do is have a hardcoded sector
location to go read the actual boot loader from.  Since it would be
hard coded at install time into the boot sector, you can't move the boot
loader file(s) or partition in any way.

> it doesn't have to work on legacy systems more than 5 years old, either.
>  this is to move forward. /bootauto.bin can be big.

In that case, use EFI and make a boot partition for the boot loader to
be in as the spec says you should.  That solves the above problems.

> if interested, send me a personal email, please.  I will pay upon
> completion (or put it into an escrow account at the FSF or another
> reasonable place).  maybe some others will supplement the funding---I know
> that $500 won't pay for it all.  I just wanted to start the ball rolling,
> and put my money where my mouth is.

-- 
Len Sorensen

Re: chained grub2 derivative bootauto system

[Brendan Trotter]

Hi,

On Fri, Aug 31, 2012 at 11:59 PM, Lennart Sorensen
<lsorense@csclub.uwaterloo.ca> wrote:
> On Fri, Aug 31, 2012 at 09:19:43AM +1000, Philip Rhoades wrote:
>> Ivo,
>>
>> Interesting idea - I particularly like the idea of booting from
>> arbitrary isos.
>
> Too bad that accessing an iso on a usb key is nothing like an actual cd
> or dvd and the sytem you boot must have support for the fact you put the
> iso on the usb key rather than as a real disc.  Nothing the boot loader
> can do to change that.
>
> If it was really that simple, someone would already have made such a
> boot loader for use on usb keys.  The reason it doesn't exist yet is
> that it can't exist.

For "PC BIOS", having a USB with multiple ISO images would actually be
relatively easy.

The USB's boot code would need to hook Int 0x15 to steal some memory,
then install code that emulates a CD (and El Torito) into the stolen memory.
Obviously you'd also hook "Int 0x13" so that the BIOS disk services are
redirected to your CD emulation code.

I'd expect that similar would be possible for UEFI (e.g. create a special
UEFI driver that emulates CD/s).

The main problem is after the OS on the ISO tries to take control of hardware
and fails to find its (emulated) CD. For some OSs this may not be a problem -
e.g. MS-DOS and FreeDOS (which continue to use the BIOS services
and don't take control of hardware), Linux (as long as the root
partition isn't on
the emulated CD), etc. For other OS's (Windows) it can't work.

However, someone that wants to create a USB like this might not care about
those "unsupportable" OSs anyway.

Finally; I'm not sure how a scheme like this would involve GRUB - you'd want
relatively specialised USB boot code (not GRUB). GRUB could (potentially)
be installed inside one or more of the ISOs, but this would be a normal
"El Torito" (or UEFI) boot as far as GRUB would know.


Cheers,

Brendan

Re: chained grub2 derivative bootauto system

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2494 bytes --]

On 01.09.2012 08:06, Brendan Trotter wrote:

> Hi,
> 
> On Fri, Aug 31, 2012 at 11:59 PM, Lennart Sorensen
> <lsorense@csclub.uwaterloo.ca> wrote:
>> On Fri, Aug 31, 2012 at 09:19:43AM +1000, Philip Rhoades wrote:
>>> Ivo,
>>>
>>> Interesting idea - I particularly like the idea of booting from
>>> arbitrary isos.
>>
>> Too bad that accessing an iso on a usb key is nothing like an actual cd
>> or dvd and the sytem you boot must have support for the fact you put the
>> iso on the usb key rather than as a real disc.  Nothing the boot loader
>> can do to change that.
>>
>> If it was really that simple, someone would already have made such a
>> boot loader for use on usb keys.  The reason it doesn't exist yet is
>> that it can't exist.
> 
> For "PC BIOS", having a USB with multiple ISO images would actually be
> relatively easy.
> 
> The USB's boot code would need to hook Int 0x15 to steal some memory,
> then install code that emulates a CD (and El Torito) into the stolen memory.
> Obviously you'd also hook "Int 0x13" so that the BIOS disk services are
> redirected to your CD emulation code.
> 
> I'd expect that similar would be possible for UEFI (e.g. create a special
> UEFI driver that emulates CD/s).
> 
> The main problem is after the OS on the ISO tries to take control of hardware
> and fails to find its (emulated) CD. For some OSs this may not be a problem -
> e.g. MS-DOS and FreeDOS (which continue to use the BIOS services
> and don't take control of hardware), Linux (as long as the root
> partition isn't on
> the emulated CD), etc. For other OS's (Windows) it can't work.
> 
> However, someone that wants to create a USB like this might not care about
> those "unsupportable" OSs anyway.
> 
> Finally; I'm not sure how a scheme like this would involve GRUB - you'd want
> relatively specialised USB boot code (not GRUB). GRUB could (potentially)
> be installed inside one or more of the ISOs, but this would be a normal
> "El Torito" (or UEFI) boot as far as GRUB would know.
> 

This is something that can be included in GRUB. Just the amount of
needed work isn't comparable with the number of additional scenarios
supported (namely mostly only *DOS on ISO).

> 
> Cheers,
> 
> Brendan
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
> 



-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]