Idea: Move kernel to upper memory

Idea: Move kernel to upper memory

[Bean]

Hi,

The traditional memory is quite small, and is used by real mode apps.
By moving grub2 kernel to upper memory, it's possible to keep it after
dos started, and we can invoke grub2 service using interrupts. One
important usage is to provide disk related function to dos via int 13,
for example, loopback device, ata/usb disk or linux software raid. It
would also be possible to reenter grub2 at any time.

I have thought of a method to implement this. First, we put kernel
code in a module kernel.mod. The platform initialization code is
separated and placed in startup.img. Startup.img would do things like
getting memory map, relocate real mode trunk and save the information
in a platform dependent structure. It then pass it to the first
module, which would be kernel.mod. kernel.mod relocates itself and
other embedded modules to upper memory, then calls the entry point for
further tasks.

One advantage of this scheme is that symlist.c is not longer needed.
Kernel is a module, other module can use its exported function. And
platform dependent data and function, such as efi system table, x86
interrupt call, etc, can be passed using a structure.

-- 
Bean

Re: Idea: Move kernel to upper memory

[Robert Millan]

On Fri, Aug 29, 2008 at 07:15:43PM +0800, Bean wrote:
> Hi,
> 
> The traditional memory is quite small, and is used by real mode apps.
> By moving grub2 kernel to upper memory, it's possible to keep it after
> dos started, and we can invoke grub2 service using interrupts. One
> important usage is to provide disk related function to dos via int 13,
> for example, loopback device, ata/usb disk or linux software raid. It
> would also be possible to reenter grub2 at any time.
> 
> I have thought of a method to implement this. First, we put kernel
> code in a module kernel.mod. The platform initialization code is
> separated and placed in startup.img. Startup.img would do things like
> getting memory map, relocate real mode trunk and save the information
> in a platform dependent structure. It then pass it to the first
> module, which would be kernel.mod. kernel.mod relocates itself and
> other embedded modules to upper memory, then calls the entry point for
> further tasks.
> 
> One advantage of this scheme is that symlist.c is not longer needed.
> Kernel is a module, other module can use its exported function. And
> platform dependent data and function, such as efi system table, x86
> interrupt call, etc, can be passed using a structure.

On one hand, I think being useful as a kind of "driver library" for DOS-based
systems (specially FreeDOS) would be a cool thing.  Our hardware drivers
would become more exposed and we could get more people to test them.

On the other, I think we should be careful about going too much out of our
way.  GRUB is a bootloader, and its main strength is that it works well at
what it does.  For example, some have tried to make Linux work as a
bootloader, but it doesn't work well, because it hasn't been built from
the ground up for this purpose.

Similarly, other projects strive to provide a free BIOS [1], like SeaBIOS.
For the same reasons, I think SeaBIOS is better suited for being a BIOS than
GRUB is.

The first concern that comes to mind is how would GRUB coexist with the
payload area which precisely starts at 0x100000.  But I expect we'd face
many unexpected issues.

Another approach with similar result could be to ditch the legacy interface
and provide the glue that would allow FreeDOS to link directly with GRUB.
Maybe that would be as much useful?

[1] Not to be confused with a core initialization program, such as coreboot

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: Idea: Move kernel to upper memory

[Bean]

On Sat, Aug 30, 2008 at 8:40 PM, Robert Millan <rmh@aybabtu.com> wrote:
> On Fri, Aug 29, 2008 at 07:15:43PM +0800, Bean wrote:
>> Hi,
>>
>> The traditional memory is quite small, and is used by real mode apps.
>> By moving grub2 kernel to upper memory, it's possible to keep it after
>> dos started, and we can invoke grub2 service using interrupts. One
>> important usage is to provide disk related function to dos via int 13,
>> for example, loopback device, ata/usb disk or linux software raid. It
>> would also be possible to reenter grub2 at any time.
>>
>> I have thought of a method to implement this. First, we put kernel
>> code in a module kernel.mod. The platform initialization code is
>> separated and placed in startup.img. Startup.img would do things like
>> getting memory map, relocate real mode trunk and save the information
>> in a platform dependent structure. It then pass it to the first
>> module, which would be kernel.mod. kernel.mod relocates itself and
>> other embedded modules to upper memory, then calls the entry point for
>> further tasks.
>>
>> One advantage of this scheme is that symlist.c is not longer needed.
>> Kernel is a module, other module can use its exported function. And
>> platform dependent data and function, such as efi system table, x86
>> interrupt call, etc, can be passed using a structure.
>
> On one hand, I think being useful as a kind of "driver library" for DOS-based
> systems (specially FreeDOS) would be a cool thing.  Our hardware drivers
> would become more exposed and we could get more people to test them.
>
> On the other, I think we should be careful about going too much out of our
> way.  GRUB is a bootloader, and its main strength is that it works well at
> what it does.  For example, some have tried to make Linux work as a
> bootloader, but it doesn't work well, because it hasn't been built from
> the ground up for this purpose.
>
> Similarly, other projects strive to provide a free BIOS [1], like SeaBIOS.
> For the same reasons, I think SeaBIOS is better suited for being a BIOS than
> GRUB is.
>
> The first concern that comes to mind is how would GRUB coexist with the
> payload area which precisely starts at 0x100000.  But I expect we'd face
> many unexpected issues.
>
> Another approach with similar result could be to ditch the legacy interface
> and provide the glue that would allow FreeDOS to link directly with GRUB.
> Maybe that would be as much useful?
>
> [1] Not to be confused with a core initialization program, such as coreboot

Hi,

Yep, come to think of it, this method may be too complicated to be
useful.There are easier way to do similar thing.

Speaking of library, I think we can provide a dynamic library for core
grub2 function. Apps written properly can be executed in both native
os and grub2. For example:

int main()
{
  grub_printf ("Hello world\n");
  return 0;
}

gcc hello.c -o hello -lgrub2

-- 
Bean

Re: Idea: Move kernel to upper memory

[Robert Millan]

On Sat, Aug 30, 2008 at 11:15:35PM +0800, Bean wrote:
> 
> Speaking of library, I think we can provide a dynamic library for core
> grub2 function. Apps written properly can be executed in both native
> os and grub2. For example:
> 
> int main()
> {
>   grub_printf ("Hello world\n");
>   return 0;
> }
> 
> gcc hello.c -o hello -lgrub2

Or it could even be POSIX/C compliant and provide printf() etc.  We discussed
something like this for the Parted GSoC [1].

[1] http://www.gnu.org/software/grub/grub-soc.html

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: Idea: Move kernel to upper memory

[Robert Millan]

On Sat, Aug 30, 2008 at 11:15:35PM +0800, Bean wrote:
> 
> gcc hello.c -o hello -lgrub2

Ah, what I meant about linking was to make it easy to link external code into
GRUB modules.  Standalone executables sound more overkill to me.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: Idea: Move kernel to upper memory

[Bean]

On Sun, Aug 31, 2008 at 9:30 PM, Robert Millan <rmh@aybabtu.com> wrote:
> On Sat, Aug 30, 2008 at 11:15:35PM +0800, Bean wrote:
>>
>> gcc hello.c -o hello -lgrub2
>
> Ah, what I meant about linking was to make it easy to link external code into
> GRUB modules.  Standalone executables sound more overkill to me.

Hi,

Yep, I know. My point is, if we provide a properly wrapper, external
tool can be built as grub module as well as standalone executable.
This make it easier to debug the code (similar to grub-emu).

-- 
Bean

Re: Idea: Move kernel to upper memory

[BVK Chaitanya]

Hi,


What is the conclusion of this thread?  is this idea still explored?


Robert Millan wrote:
> 
> The first concern that comes to mind is how would GRUB coexist with the
> payload area which precisely starts at 0x100000.  But I expect we'd face
> many unexpected issues.
> 

Does this mean, GRUB needs to fit within 1MB limit when all its modules 
are loaded?  More clearly, are there any limits on where (and how many) 
grub modules can be dynamically loaded currently?

Since payload(s) needs to be present at their precise positions only 
after the boot command, we can always deterministically /schedule/ 
memmove operation to be executed as part of boot command.

Since boot command is the last command executed by GRUB (correct me if 
not), we can overwrite any GRUB code/data at that point.  This also 
needs boot command to exist in low memory.

The main advantage i see here is that it removes any arbitrary limits on 
number of modules or their placement, if at all limits exists.


thanks,
--
bvk-chaitanya

Re: Idea: Move kernel to upper memory

[Robert Millan]

On Wed, Sep 03, 2008 at 04:15:33PM +0530, BVK Chaitanya wrote:
> Hi,
> 
> 
> What is the conclusion of this thread?  is this idea still explored?
> 
> 
> Robert Millan wrote:
> >
> >The first concern that comes to mind is how would GRUB coexist with the
> >payload area which precisely starts at 0x100000.  But I expect we'd face
> >many unexpected issues.
> >
> 
> Does this mean, GRUB needs to fit within 1MB limit when all its modules 
> are loaded?  More clearly, are there any limits on where (and how many) 
> grub modules can be dynamically loaded currently?
> 
> Since payload(s) needs to be present at their precise positions only 
> after the boot command, we can always deterministically /schedule/ 
> memmove operation to be executed as part of boot command.
> 
> Since boot command is the last command executed by GRUB (correct me if 
> not), we can overwrite any GRUB code/data at that point.  This also 
> needs boot command to exist in low memory.

This is already done.  See grub_multiboot_{forward,backward}_relocator
variables and how they're used in the multiboot loader.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: Idea: Move kernel to upper memory

[BVK]

On Fri, Sep 5, 2008 at 12:49 AM, Robert Millan <rmh@aybabtu.com> wrote:
> On Wed, Sep 03, 2008 at 04:15:33PM +0530, BVK Chaitanya wrote:
>>
>> Since payload(s) needs to be present at their precise positions only
>> after the boot command, we can always deterministically /schedule/
>> memmove operation to be executed as part of boot command.
>>
>> Since boot command is the last command executed by GRUB (correct me if
>> not), we can overwrite any GRUB code/data at that point.  This also
>> needs boot command to exist in low memory.
>
> This is already done.  See grub_multiboot_{forward,backward}_relocator
> variables and how they're used in the multiboot loader.
>

OK, fine :-)


-- 
bvk-chaitanya

Re: Idea: Move kernel to upper memory

[Vladimir Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1564 bytes --]

Hello, I resurrect this thread because of the recent developement in
preboot, mmap and drivemap this subject becomes very actual

On Fri, Aug 29, 2008 at 1:15 PM, Bean <bean123ch@gmail.com> wrote:

> Hi,
>
> The traditional memory is quite small, and is used by real mode apps.
> By moving grub2 kernel to upper memory, it's possible to keep it after
> dos started, and we can invoke grub2 service using interrupts. One
> important usage is to provide disk related function to dos via int 13,
> for example, loopback device, ata/usb disk or linux software raid. It
> would also be possible to reenter grub2 at any time.
>
> I have thought of a method to implement this. First, we put kernel
> code in a module kernel.mod. The platform initialization code is
> separated and placed in startup.img. Startup.img would do things like
> getting memory map, relocate real mode trunk and save the information
> in a platform dependent structure. It then pass it to the first
> module, which would be kernel.mod. kernel.mod relocates itself and
> other embedded modules to upper memory, then calls the entry point for
> further tasks.
>
> One advantage of this scheme is that symlist.c is not longer needed.
> Kernel is a module, other module can use its exported function. And
> platform dependent data and function, such as efi system table, x86
> interrupt call, etc, can be passed using a structure.
>
> --
> Bean
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>

[-- Attachment #2: Type: text/html, Size: 2072 bytes --]