[RFC] general-usage real-mode loader

[RFC] general-usage real-mode loader

[phcoder]

Hello. A long time ago I written a C+asm code which loads any given code
at any real-mode location, then puts machine in correct state and then
launches the kernel. I can modify this code to suit GRUB2. Then loading
realmode kernel would work like this:
1) copy helper asm to last kb of lower memory
2) jump to the helper
3) helper copies from upper memory the kernel
4) turn A10 bug back on if necessary
5) go to RM
6) prepare registers
7) jump to the kernel

This protocol is very flexible and as such could be used by all loaders
which load kernel in realmode or even in PM (skip step 5, do steps 6-7
in 32-bit mode) except for changing page tables. Such a helper can be
easily implemented as module and so help us removing asm-parts of
loaders from the kernel. If I recieve greenlight for it, I implement it.
Vladimir 'phcoder' Serbinenko

Re: [RFC] general-usage real-mode loader

[phcoder]

BTW. This helper would allow easy addition of many different kernels
(freedos, haiku, ntldr, ...)
Vladimir 'phcoder' Serbinenko
phcoder wrote:
> Hello. A long time ago I written a C+asm code which loads any given code
> at any real-mode location, then puts machine in correct state and then
> launches the kernel. I can modify this code to suit GRUB2. Then loading
> realmode kernel would work like this:
> 1) copy helper asm to last kb of lower memory
> 2) jump to the helper
> 3) helper copies from upper memory the kernel
> 4) turn A10 bug back on if necessary
> 5) go to RM
> 6) prepare registers
> 7) jump to the kernel
> 
> This protocol is very flexible and as such could be used by all loaders
> which load kernel in realmode or even in PM (skip step 5, do steps 6-7
> in 32-bit mode) except for changing page tables. Such a helper can be
> easily implemented as module and so help us removing asm-parts of
> loaders from the kernel. If I recieve greenlight for it, I implement it.
> Vladimir 'phcoder' Serbinenko

Re: [RFC] general-usage real-mode loader

[Robert Millan]

On Fri, Sep 05, 2008 at 07:27:51PM +0200, phcoder wrote:
> Hello. A long time ago I written a C+asm code which loads any given code
> at any real-mode location, then puts machine in correct state and then
> launches the kernel. I can modify this code to suit GRUB2. Then loading
> realmode kernel would work like this:
> 1) copy helper asm to last kb of lower memory
> 2) jump to the helper
> 3) helper copies from upper memory the kernel
> 4) turn A10 bug back on if necessary
> 5) go to RM
> 6) prepare registers
> 7) jump to the kernel
> 
> This protocol is very flexible and as such could be used by all loaders
> which load kernel in realmode or even in PM (skip step 5, do steps 6-7
> in 32-bit mode) except for changing page tables. Such a helper can be
> easily implemented as module and so help us removing asm-parts of
> loaders from the kernel. If I recieve greenlight for it, I implement it.

Maybe I'm confusing this with something else, but isn't this what both Bean
and Vesa implemented separately, and are currently discussing in another
thread?

(the goal there was to move BIOS wrappers out of kern/i386/pc/startup.S)

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: [RFC] general-usage real-mode loader

[phcoder]

I checked the archive found the thread "Idea: Move kernel to upper
memory". But what is discussed there is much more general about general
memory layout. Here I wanted to speak about just one function.
Vladimir 'phcoder' Serbinenko
Robert Millan wrote:
> On Fri, Sep 05, 2008 at 07:27:51PM +0200, phcoder wrote:
>> Hello. A long time ago I written a C+asm code which loads any given code
>> at any real-mode location, then puts machine in correct state and then
>> launches the kernel. I can modify this code to suit GRUB2. Then loading
>> realmode kernel would work like this:
>> 1) copy helper asm to last kb of lower memory
>> 2) jump to the helper
>> 3) helper copies from upper memory the kernel
>> 4) turn A10 bug back on if necessary
>> 5) go to RM
>> 6) prepare registers
>> 7) jump to the kernel
>>
>> This protocol is very flexible and as such could be used by all loaders
>> which load kernel in realmode or even in PM (skip step 5, do steps 6-7
>> in 32-bit mode) except for changing page tables. Such a helper can be
>> easily implemented as module and so help us removing asm-parts of
>> loaders from the kernel. If I recieve greenlight for it, I implement it.
> 
> Maybe I'm confusing this with something else, but isn't this what both Bean
> and Vesa implemented separately, and are currently discussing in another
> thread?
> 
> (the goal there was to move BIOS wrappers out of kern/i386/pc/startup.S)
> 

Re: [RFC] general-usage real-mode loader

[Vesa Jääskeläinen]

phcoder wrote:
> I checked the archive found the thread "Idea: Move kernel to upper
> memory". But what is discussed there is much more general about general
> memory layout. Here I wanted to speak about just one function.
> Vladimir 'phcoder' Serbinenko

Try: [PATCH] Move assembly code out of the kernel

Re: [RFC] general-usage real-mode loader

[phcoder]

Well sometimes it's similar but most of the time isn't because your code
runs a code in RM but with hope returning back to PM and normal grub
operation. And it also loads to quite fixed place. My code would load a
kernel, that means that it will not return and so code can be loaded to
any place not needed by kernel.
Vladimir 'phcoder' Serbinenko
Vesa Jääskeläinen wrote:
> phcoder wrote:
>> I checked the archive found the thread "Idea: Move kernel to upper
>> memory". But what is discussed there is much more general about general
>> memory layout. Here I wanted to speak about just one function.
>> Vladimir 'phcoder' Serbinenko
> 
> Try: [PATCH] Move assembly code out of the kernel
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel

Re: [RFC] general-usage real-mode loader

[Vesa Jääskeläinen]

phcoder wrote:
> Well sometimes it's similar but most of the time isn't because your code
> runs a code in RM but with hope returning back to PM and normal grub
> operation. And it also loads to quite fixed place. My code would load a
> kernel, that means that it will not return and so code can be loaded to
> any place not needed by kernel.

Do you mind replying inline...?

It is true that code is withing grub's i386 kernel. But other than that
same mechanism can be used to load kernel. After all kernel will take
over the computer after it has started and it is not returning back
anymore (eg. stack is not anymore there to be restored).

I would prefer that if we are going to give custom register
configurations for kernel then they all should use same structures for
storing registers.

But anyway... Did you have some good examples where this functionaly
would be needed :) ? For now we have specialized loaders that prepare
system for any special OSes that needs to be supported. Especially how
do you know what registers to fill or better yet, how user will provide
this information so OS can be loaded.

If there is a need for specialized loader please provide some details
where those are needed and what is required from GRUB in order to
support that. If there is possibility to use multiboot then that would
supersede need for specialized loader.

Thanks,
Vesa Jääskeläinen

Re: [RFC] general-usage real-mode loader

[phcoder]

Vesa Jääskeläinen wrote:
 > It is true that code is withing grub's i386 kernel. But other than that
> same mechanism can be used to load kernel. After all kernel will take
> over the computer after it has started and it is not returning back
> anymore (eg. stack is not anymore there to be restored).
> 
In this case my rmkern helper can use this mechanism but the problem is
then that the rmkern should be able to choose where to put RM helper
since some OS need specific places in memory to be filled with specific
values
> I would prefer that if we are going to give custom register
> configurations for kernel then they all should use same structures for
> storing registers.
> 
> But anyway... Did you have some good examples where this functionaly
> would be needed :) ? For now we have specialized loaders that prepare
> system for any special OSes that needs to be supported. Especially how
> do you know what registers to fill or better yet, how user will provide
> this information so OS can be loaded.
I thought that this function can be used as general helper for any
kernel that needs to be launched in RM. In that cases RMkern
functionality is used by OS-specific wrapper which knows what to put in
the registers
Usage cases:
1) chainloader
2) zbeos: BeOS and haiku. BeOS has a good functionality for booting from
loopback but for this you need floppy, windows 9x/me or dos. On my
laptop I have neither. But zbeos (BeOS and Haiku middle-loader) is
really easy to load.
3) ecomstation: eCS is able to load from JFS but needs a specialized
loader to be installed on its own primary active partition. I think GRUB
should be able to load it directly. But perhaps also exporting of GRUB
functions to ecomstation is needed (this is discussed in another thread,
I'll write there later)
4) DOS/Win9x and freedos. Especially the second one. They use extended
chainloader functionality found in grub4dos. With rmkern loading freedos
kernel should be really easy.
5) Loading winnt (ntldr) and winvista (bootmgr)
6) Many other different proprietary and free OS. I don't know enough
about them to say that this or that one can benefit from it but suppose
many OS can be booted easier and more reliable with this functionality.
7) rmkern command. Experienced users, hobbysts and OS-developpers can
enter manually the state in which they want their kernel to be loaded.
May be handy also for reverse engineering
> 
> If there is a need for specialized loader please provide some details
> where those are needed and what is required from GRUB in order to
> support that. If there is possibility to use multiboot then that would
> supersede need for specialized loader.
Some OS have their reasons not to support multiboot. E.g. haiku and
freedos stick very near to their goal of reimplementation beos/dos and
as such they don't want anything that would be better

Vladimir 'phcoder' Serbinenko

Re: [RFC] general-usage real-mode loader

[Robert Millan]

On Sun, Sep 07, 2008 at 02:22:57PM +0200, phcoder wrote:
> 4) DOS/Win9x and freedos. Especially the second one. They use extended
> chainloader functionality found in grub4dos. With rmkern loading freedos
> kernel should be really easy.

Since FreeDOS is free software, we could fix it to use Multiboot instead.

> 5) Loading winnt (ntldr) and winvista (bootmgr)

Ah, and there's a free replacement for NTLDR as well, see:

  http://www.reactos.org/wiki/index.php/FreeLoader

> 6) Many other different proprietary and free OS. I don't know enough
> about them to say that this or that one can benefit from it but suppose
> many OS can be booted easier and more reliable with this functionality.
> 7) rmkern command. Experienced users, hobbysts and OS-developpers can
> enter manually the state in which they want their kernel to be loaded.
> May be handy also for reverse engineering

I think people (specially OS developers) should contemplate obsoleting the
i8086 someday...

Some of the examples you give are things we should indeed support (most
specially the chainloader), but I'm not sure if it's worthy to spend so
much effort in extending them.  IMHO we should push for a sane standard
like Multiboot instead.

But I can't tell you what to do with your time, of course...

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."