Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[Christian Koenig]

Hi,

On Monday 17 December 2001 23:10, Grover, Andrew wrote:
> > From: Alexander Viro [mailto:viro@math.psu.edu]
> >
> > On Mon, 17 Dec 2001, Grover, Andrew wrote:
> > > I don't think multiple streams is a good idea, but did you
> >
> > all see the patch
> >
> > > by Christian Koenig to let the bootloader load modules?
> >
> > That seems to solve
> >
> > > the problem nicely.
> >
> > That puts an awful lot of smarts into bootloader and creates
> > code duplication
> > for no good reason.

I agree, but the problem I have with Initrd is that you could only have 
1 single initrd-image on your hard disk / loaded by the boot-loader.

Imaging my situation, I have some removeable hard-disks, changing it between 
a large amount of Systems all with individual Hardware-configuration.
(Some SCSI, some ide, and a cluster booting with pxegrub/Floppy-Disks).

With the last relleas of my patch I'm now capable to have a grub menu.lst 
looking like:

tile Kernel 2.4.14 Adaptec xyz/ne2000
kernel /boot/vmlinuz-2.4.14 root=/dev/sdxyz ...

# Loading SCSI-Drivers
modulesfromfile /etc/modules-adaptec /lib/modules/2.4.14/modules.dep 

# Loading Network-Drivers
modulesfromfile /etc/modules-ne2000 /lib/modules/2.4.14/modules.dep

tile Kernel 2.4.14 Tecram xyz / Inter EExpresPro 

# Loading SCSI-Drivers
modulesfromfile /etc/modules-tecram /lib/modules/2.4.14/modules.dep 

# Loading Network-Drivers
modulesfromfile /etc/modules-eepro /lib/modules/2.4.14/modules.dep 
......

Could you Imaging what work it would be to create an individual initrd-Image 
/ Kernel Image for every system I have. Beside this I sometimes have the 
Problem that I come to a system never seen bevore and have to boot with my 
Floppy Disk's. (I'm now looking forward that grub someday gets capable to use 
cd-roms directly).

I know this it isn't the best solution to add a module-loader to the Kernel, 
but did you have a better idea ?

MfG, Christian Koenig.

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[Jesse Pollard]

---------  Received message begins Here  ---------

> 
> Hi,
> 
> On Monday 17 December 2001 23:10, Grover, Andrew wrote:
> > > From: Alexander Viro [mailto:viro@math.psu.edu]
> > >
> > > On Mon, 17 Dec 2001, Grover, Andrew wrote:
> > > > I don't think multiple streams is a good idea, but did you
> > >
> > > all see the patch
> > >
> > > > by Christian Koenig to let the bootloader load modules?
> > >
> > > That seems to solve
> > >
> > > > the problem nicely.
> > >
> > > That puts an awful lot of smarts into bootloader and creates
> > > code duplication
> > > for no good reason.
> 
> I agree, but the problem I have with Initrd is that you could only have 
> 1 single initrd-image on your hard disk / loaded by the boot-loader.
> 
> Imaging my situation, I have some removeable hard-disks, changing it between 
> a large amount of Systems all with individual Hardware-configuration.
> (Some SCSI, some ide, and a cluster booting with pxegrub/Floppy-Disks).
> 
> With the last relleas of my patch I'm now capable to have a grub menu.lst 
> looking like:
> 
> tile Kernel 2.4.14 Adaptec xyz/ne2000
> kernel /boot/vmlinuz-2.4.14 root=/dev/sdxyz ...
> 
> # Loading SCSI-Drivers
> modulesfromfile /etc/modules-adaptec /lib/modules/2.4.14/modules.dep 
> 
> # Loading Network-Drivers
> modulesfromfile /etc/modules-ne2000 /lib/modules/2.4.14/modules.dep
> 
> tile Kernel 2.4.14 Tecram xyz / Inter EExpresPro 
> 
> # Loading SCSI-Drivers
> modulesfromfile /etc/modules-tecram /lib/modules/2.4.14/modules.dep 
> 
> # Loading Network-Drivers
> modulesfromfile /etc/modules-eepro /lib/modules/2.4.14/modules.dep 
> ......
> 
> Could you Imaging what work it would be to create an individual initrd-Image 
> / Kernel Image for every system I have. Beside this I sometimes have the 
> Problem that I come to a system never seen bevore and have to boot with my 
> Floppy Disk's. (I'm now looking forward that grub someday gets capable to use 
> cd-roms directly).
> 
> I know this it isn't the best solution to add a module-loader to the Kernel, 
> but did you have a better idea ?

How about a three step loader:
1. primary boot (PROM) loads secondary boot <any boot program>
2. the <any boot program> loads a "superboot"
3. superboot has filesystem knowlege of a boot device (this is different) and
   Loads base kernel
4. installs required modules from boot device (this is not done currently)
5. starts kernel

The kernel should not need any installed drivers. All of them could be
installed after the kernel is loaded, but before being started.

This used to be/is done by many systems (VAX, Sun, IBM, SGI...) The PROM loader
loads a boot loader which loads boot program.

There are several ways to implement this: The "superboot" program doesn't
even have to know more than one filesystem type. A very simple file system
using contigueous, variable length segments, each segment containing one file.
The only requirement is that the /boot filesystem be accessable to both
Linux AND the loader. and yes, the /lib/modules/... should be moved to
/boot/modules directory. The /boot filesystem should need be no larger
than 2-5 MB depending on how many kernels and modules are to be present
at one time.

And yes, a tar formatted file should be sufficient if it were on a partition
of it's own (below cylender 1024 for the antique systems). If desired, the
tar file(fs?) could even be compressed (in which case, the kernel wouldn't
need to be). Even an ISO9660 format might work, with some modifications
(it WOULD be desirable to be able to write directly to it, and it would
 have to allow for fragmentation of the free space)

I think it would be much easier to use a block device though.. first block
is a directory block, directory containing the minimum of 
"name:filetype:starting block:length (in blocks)". If the filetype is a
directory, then the blocks pointed to are a subdirectory, treated in the
same manner as the first directory. True, there are no inodes in this,
but they could be added as just a unique number in an additional field
in a directory entry. NO hard links, NO symbolic links, NO owner (mounted
ONLY as uid=0,gid=0) intended for /boot. The superblock would only need to
know where the first filesystem block is, the length of the file name array,
and the size of the partition (for limit error checking).

Even an ISO9660 format might work, with some modifications.

The block nature of the file boundaries would allow for a possible DMA load
of an entire file into memory directly. Installing a module might require a
copy operation though. The contigueous nature of the files eliminates the
need for an fsck (not possible to corrupt the files, and it is easy to
recreate the entire filesystem). This also simplifies the secondary boot -
there is only one starting block number and length needed (file starting
block number + FS offset).

It may not even be necessary that the filesystem be known to linux, provided
some tools similar to the "mtools" were available to put files into the
partition, defragment the filesystem (move files to put all contigueous free
space at the end), get files out, make directory listings... It would just
make support simpler to mount it.

Oh - another thing - this could eliminate the need for the kernel to include
a "decompress" startup; this should be done by the "superboot" loader,
allowing a choice of different kinds of compression without kernel modification.

Who knows - it may be possible to merge <any boot program> with the "superboot"
and produce the equivalent to what is being done now, but without requiring
any built-in modules. Personally, I don't think so because such a loader
would become more usefull as a "mini-os" capable of being used to run
diagnostics as well as booting the system.

Suggested contents of the boot fs:

boot	- the superboot program
vmlinux	- the kernel (one or more instances)
<version> - directory containing modules for kernel <version>
	    multiple instances, one per kernel version. The contents of the
	    directory could even be in the same as in /lib/modules/<vers>
	    with the necessary modules dependancy files and directories of
	    for the drivers.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[H. Peter Anvin]

Christian Koenig wrote:
> 
> I agree, but the problem I have with Initrd is that you could only have 
> 1 single initrd-image on your hard disk / loaded by the boot-loader.
> 

Al and I have already discussed an improved approach.  What we want is 
rather than initrd an initramfs.  I have specifically discussed the 
initramfs protocol with Al so it can be combined from multiple sources.

	-hpa

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[Eric W. Biederman]

"H. Peter Anvin" <hpa@zytor.com> writes:

> Eric W. Biederman wrote:
> 
> > I have a personally dislike for using firmware calls to drive hardware
> > devices, so I won't be picking that up.  But I am interested in what
> > it requires to not burn bridges.  So I can make certain a linux kernel
> > loaded with a linux booting linux patch can requery the firmware.
> >
> 
> > My impression is that the linux kernel already does the important
> > things by not smashing firmware reserved memory, (assuming you aren't
> > loaded with loadlin).  So all that is required is to switch the idt
> > back to address 0, and switch the cpu back to 16bit real mode.
> > But if you know of other cases that need to be handled I would be
> > happy to hear about it.
> >
> 
> 
> Unfortunately that's not the case.  The big issue is "who owns the interrupt
> controller", and "who owns the interrupts." 
[snip]



> You can check out the BIOS extender I wrote for genesis at
> ftp://ftp.zytor.com/pub/linux/genesis/

Thanks.  I've got genesis-1.10 now looking and digesting to see if you
have any unexpected tricks takes a little longer.

For my purposes I intend to fully disable the BIOS and then after I
have done all of my work, reenable the BIOS.  Which should be a little
easier and have a slightly different set of issues. 

>From the 10,000 foot level it looks like I am pretty safe already
except for those BIOS functions that drive the hardware.  For those I
need to setup the legacy PIC back to it's default setting, and
possibly a few other hardware things.   I wonder just how sensitive
the an x86 BIOS really is to changing those things...

For the most part I find it perfectly acceptable if I break all of the
firmware hardware drivers.  As long as the information callbacks are
preserved.  But preserver enough so I could load dos from linux would
be nice.

Eric

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[H. Peter Anvin]

Eric W. Biederman wrote:
> 
>>From the 10,000 foot level it looks like I am pretty safe already
> except for those BIOS functions that drive the hardware.  For those I
> need to setup the legacy PIC back to it's default setting, and
> possibly a few other hardware things.   I wonder just how sensitive
> the an x86 BIOS really is to changing those things...
> 

You never know, especially since part of the BIOS might be an external 
SCSI or network card BIOS...

	-hpa

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[Eric W. Biederman]

"H. Peter Anvin" <hpa@zytor.com> writes:

> Eric W. Biederman wrote:
> >
> >>From the 10,000 foot level it looks like I am pretty safe already
> > except for those BIOS functions that drive the hardware.  For those I
> > need to setup the legacy PIC back to it's default setting, and
> > possibly a few other hardware things.   I wonder just how sensitive
> > the an x86 BIOS really is to changing those things...
> >
> 
> You never know, especially since part of the BIOS might be an external SCSI or
> network card BIOS...

Which just goes to show what a fragile firmware design it is, to have
firmware callbacks doing device I/O.  I think the whole approach of
having firmware callbacks is fundamentally flawed but I'll do my best
to keep it working, for those things that care.  If it works over 50%
of the time I'm happy...

The much more interesting challenge is to get the linux kernel drivers
to shutdown their hardware well enough that they can still drive the
hardware when linux restarts.  And if the hardware can't do that it is
buggy...


Eric

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[H. Peter Anvin]

Followup to:  <m1zo4fursh.fsf@frodo.biederman.org>
By author:    ebiederm@xmission.com (Eric W. Biederman)
In newsgroup: linux.dev.kernel
> 
> Which just goes to show what a fragile firmware design it is, to have
> firmware callbacks doing device I/O.  I think the whole approach of
> having firmware callbacks is fundamentally flawed but I'll do my best
> to keep it working, for those things that care.  If it works over 50%
> of the time I'm happy...
> 

NAK.  You can make it perfectly robust thankyouverymuch, as long as
you don't try to *mix* firmware and poking directly at the
hardware... this is a classic "who owns what" class problem.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[Eric W. Biederman]

"H. Peter Anvin" <hpa@zytor.com> writes:

> Followup to:  <m1zo4fursh.fsf@frodo.biederman.org>
> By author:    ebiederm@xmission.com (Eric W. Biederman)
> In newsgroup: linux.dev.kernel
> > 
> > Which just goes to show what a fragile firmware design it is, to have
> > firmware callbacks doing device I/O.  I think the whole approach of
> > having firmware callbacks is fundamentally flawed but I'll do my best
> > to keep it working, for those things that care.  If it works over 50%
> > of the time I'm happy...
> > 
> 
> NAK.  You can make it perfectly robust thankyouverymuch, as long as
> you don't try to *mix* firmware and poking directly at the
> hardware... this is a classic "who owns what" class problem.

I agree that I could keep it working as well as it ever would.  Not
that x86 firmware or any software is ever perfectly working.

At this point in time I live in a world where 99+% of the time the
hardware is owned by the operating system, and the firmware is just
there to get the operating system loaded, and to hold details about
the motherboard that the operating system can not find out by probing
the hardware.

For the cases I find important I get better reliability and
portability by never involving the firmware at all.  If there is a
problem with a driver I can fix it.  If I want to switch cpus I can.
Admittedly the cost for native drivers is high, but if I don't have to
pay that cost twice and can actually reuse my OS drivers.  It isn't a
price I mind paying.

I care about not trashing the firmware so a newer probe routine can
find out more precisely or robustly what is on a motherboard.  Having
a reasonable chance that the firmware can also still drive the
hardware is a plus. 

I criticize firmware designers, not to attack anyones dependence on
the firmware.  But more to make certain I never implement anything
like that.  

I don't think I have seen a firmware design where someone has designed
it with the assumption that humans mess up.  Instead every firmware
interface I have seen seems to be designed by asking how can I include
every possible desirable feature.  Since it is painful to fix or
replace firmware this is a real issue. 

I have seen alpha firmware getting confused when the operating system
uses the hardware, when rebooting on the alpha.  Which is why I am
sensitive to it.

Eric

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[H. Peter Anvin]

Eric W. Biederman wrote:

> 
>>Followup to:  <m1zo4fursh.fsf@frodo.biederman.org>
>>By author:    ebiederm@xmission.com (Eric W. Biederman)
>>In newsgroup: linux.dev.kernel
>>
>>>Which just goes to show what a fragile firmware design it is, to have
>>>firmware callbacks doing device I/O.  I think the whole approach of
>>>having firmware callbacks is fundamentally flawed but I'll do my best
>>>to keep it working, for those things that care.  If it works over 50%
>>>of the time I'm happy...
>>>
>>>
>>NAK.  You can make it perfectly robust thankyouverymuch, as long as
>>you don't try to *mix* firmware and poking directly at the
>>hardware... this is a classic "who owns what" class problem.
>>
> 
> I agree that I could keep it working as well as it ever would.  Not
> that x86 firmware or any software is ever perfectly working.
> 


True enough; I guess what I meant is you can design it so that it is *no 
less unreliable than the underlying firmware*.


> At this point in time I live in a world where 99+% of the time the
> hardware is owned by the operating system, and the firmware is just
> there to get the operating system loaded, and to hold details about
> the motherboard that the operating system can not find out by probing
> the hardware.


Right.  My point was that you want to treat the transition as modal -- 
you have "boot mode" and you have "runtime".  My work was to build a 
Linux kernel derivative which could run in "boot mode" and therefore not 
require drivers.  The intent was that this "boot mode" kernel would then 
load the real "runtime" kernel and transition to it.

My comment was mostly that there is a persistent myth that you can't use 
the x86 firmware from protected mode.  You can't, directly, but you can 
get the functionality through other means.  This is hardly a new 
technique; in fact, it's very similar to the DOS extenders of old times; 
in fact, it's somewhat simpler.

	-hpa

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[Eric W. Biederman]

"H. Peter Anvin" <hpa@zytor.com> writes:

> My comment was mostly that there is a persistent myth that you can't use the x86
> firmware from protected mode.  You can't, directly, but you can get the
> functionality through other means.  This is hardly a new technique; in fact,
> it's very similar to the DOS extenders of old times; in fact, it's somewhat
> simpler.

Agreed.  And to completely dispel the myth.  Etherboot has been doing
something similar for years.  It disables interrupts in 32bit mode so
it doesn't have quite as much work to do but otherwise it is pretty
much the same picture.

I finally tracked down the reason why Setup.S is run in real mode,
instead of being called from protected mode.  And that is in extremely
hostile environments (like loadlin works in) loading the kernel wrecks
the firmware callbacks.  So you must do your BIOS calls as a special
case before you switch to protected mode.

Eric

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[H. Peter Anvin]

Eric W. Biederman wrote:

> 
> Agreed.  And to completely dispel the myth.  Etherboot has been doing
> something similar for years.  It disables interrupts in 32bit mode so
> it doesn't have quite as much work to do but otherwise it is pretty
> much the same picture.
> 


If you disable interrupts in 32-bit mode a lot of things will not work.


> I finally tracked down the reason why Setup.S is run in real mode,
> instead of being called from protected mode.  And that is in extremely
> hostile environments (like loadlin works in) loading the kernel wrecks
> the firmware callbacks.  So you must do your BIOS calls as a special
> case before you switch to protected mode.


No, it's because it was easier to do it that way -- do all BIOS calls 
once and for all in the early part of the execution of the kernel, and 
then forget about it.

	-hpa

Re: Booting a modular kernel through a multiple streams file / Making Linux multiboot capable and grub loading kernel modules at boot time.

[Eric W. Biederman]

"H. Peter Anvin" <hpa@zytor.com> writes:

> Eric W. Biederman wrote:
> 
> > Agreed.  And to completely dispel the myth.  Etherboot has been doing
> > something similar for years.  It disables interrupts in 32bit mode so
> > it doesn't have quite as much work to do but otherwise it is pretty
> > much the same picture.
> >
> 
> 
> If you disable interrupts in 32-bit mode a lot of things will not work.

The basic technique use by etherboot is:
In 32bit mode handle no interrupts.  When you want to do a BIOS call
switch to 16bit mode enable interrupts do the call.  disable interrupts
and switch back to 32bit mode.

As far as I can tell this is a similar idea to what you are doing in
genesis.  etherboot doesn't need everything so it doesn't handle the
general case.  But given that it is public program, I mentioned it
because to help dispell the myth that you can do bios call from 32 bit
protected mode..

> > I finally tracked down the reason why Setup.S is run in real mode,
> > instead of being called from protected mode.  And that is in extremely
> > hostile environments (like loadlin works in) loading the kernel wrecks
> > the firmware callbacks.  So you must do your BIOS calls as a special
> > case before you switch to protected mode.
> 
> 
> No, it's because it was easier to do it that way -- do all BIOS calls once and
> for all in the early part of the execution of the kernel, and then forget about
> it.

That may have been the original reason.  But the reason to keep the
code that way is so we work with loadlin, (and any kin it might have).  

I have tested it and after you are loaded by loadlin you cannot go
back and make BIOS calls.  The problem is that dos TSR's and device
drivers have intercepted BIOS interrupts, and we stomp all over
those.

It may be possible to overcome this by saving the state of the entire
dos session but to be safe we would need to take a core dump of the
entire machine.  And for such little gain I don't think that is worth
it.

Eric