Replacing the legacy "map" command

Replacing the legacy "map" command

[Javier Martín]

First of all, hi there! I'm a 19-old Spanish student and this is my
first post in this list. On to GRUB now:

I'm interested in reimplementing the "map" command from GRUB legacy,
since it may be required by some OSes, specifically M$ Windows, to
boot if they are installed on an HD that was the "first drive" (for
the BIOS) but that is no longer. In particular, my setup is:

SATA channel 1: Samsung 500GB, Windows XP Pro x64
SATA channel 2: Maxtor 300GB, Ubuntu Linux, GRUB installation

However, the BIOS is set to boot from the 300GB drive, and thus at
boot time the Windows drive becomes (hd1): while in GRUB Legacy I
succeeded in booting if I swapped the two BIOS drives with the "map"
command, in GRUB 2 the best result I've got is an "Error loading
operating system" from the Windows boot sector (confirmed after dding
it and grepping for the message).

This message is mainly concerned with "how should I implement it"?
Since this functionality is AFAIK exclusive to the x86/amd64 PC-BIOS
target, it should be confined there, but there are at least two paths
of action that I can think of:
 1.- Given that only the chain OS loader makes use of such a switch, I
could add it as an option to the "chainloader" command. This is the
simplest path, as it would only require modifying
loader/i386/pc/chainloader* and kern/i386/pc/startup.S (where
grub_chainloader_real_boot resides). Such an option would most
probably be named something like "mapdrivefirst", as in "chainloader
--mapdrivefirst (hd1)+1".
 2.- The other option would be to add a new command that managed the
BIOS drive mappings (like, "biosdrivesmap", duh), with three main
options: "show", "map" and "reset". I'm still unsure of what files
should be modified for this change, and this might be overkill.
No matter how the UI (the commands) actually ends up, the mappings
would just be stored in a variable, and not applied until the "boot"
command is issued. Thus, actual functionality should probably be
implemented in grub_chainloader_real_boot, and the logic is pretty
simple, with most of it available in the GRUB Legacy code.

All in all, I'm asking for your thoughts, opinions, advices and
whatever, since I'm the new guy here and I'd like to learn from this.
I might not be able to start right away, since I'm flooded with exams
right now: I could take it as a summer project, though I don't expect
it to take too long - an experienced GRUB hacker could even do it in
the day, but I'm not included in that set, so I think I could have a
testable version within a week if I don't find any problems.
Well, if you have read through all this, thanks for your attention and cheers!
Habbit

Re: Replacing the legacy "map" command

[Pavel Roskin]

On Tue, 2008-05-27 at 03:36 +0200, Javier Martín wrote:

>  1.- Given that only the chain OS loader makes use of such a switch, I
> could add it as an option to the "chainloader" command. This is the
> simplest path, as it would only require modifying
> loader/i386/pc/chainloader* and kern/i386/pc/startup.S (where
> grub_chainloader_real_boot resides). Such an option would most
> probably be named something like "mapdrivefirst", as in "chainloader
> --mapdrivefirst (hd1)+1".

I suggest that you avoid excessively long option names.  --mapdrive
would mean the same unless you envision --mapdrivelater or something.
Moreover, --map should be fine, unless you think that something else can
be mapped, and users may want to map the drives, but not that other
thing.  And if the mapping is created by another command, then even
"--map" would be excessive, because there is no point in creating a
mapping if it's not used.

Or maybe "first" means to remap the drive used in the chainloader
command to appear the first drive?  I don't like it, because the
bootsector may reside on a different drive, perhaps in a file as a
backup.  And it doesn't provide an option to remap the original first
drive.

>  2.- The other option would be to add a new command that managed the
> BIOS drive mappings (like, "biosdrivesmap", duh), with three main
> options: "show", "map" and "reset". I'm still unsure of what files
> should be modified for this change, and this might be overkill.
> No matter how the UI (the commands) actually ends up, the mappings
> would just be stored in a variable, and not applied until the "boot"
> command is issued. Thus, actual functionality should probably be
> implemented in grub_chainloader_real_boot, and the logic is pretty
> simple, with most of it available in the GRUB Legacy code.

I think it would be a cleaner solution.

By the way, I can imagine that some kernels not loaded with
"chainloader" may benefit from the remapping as well.

Again, I'd like to see the command name shortened, perhaps to "drivemap"
or "map".  It's not like users will need to distinguish BIOS and
non-BIOS mappings.

-- 
Regards,
Pavel Roskin

Re: Replacing the legacy "map" command

[Urja Rannikko]

>
> By the way, I can imagine that some kernels not loaded with
> "chainloader" may benefit from the remapping as well.
>
> Again, I'd like to see the command name shortened, perhaps to "drivemap"
> or "map".  It's not like users will need to distinguish BIOS and
> non-BIOS mappings.
>

[/lurkermode]

Since the mapping (internally) works on bios drive numbers, the
simplest syntax i think of would be
map [oldhex newhex] ...
eg. map 0x81 0x80 0x80 0x81
would swap the first two drives.

For grub, i'd suggest
map [olddrive newdrive] ...
eg. map (hd1) (hd0) (hd0) (hd1)

I don't know whether the brevity of this format is very user friendly,
but it would work very well for me

The map command would just store the given parameters (converted to
bios hex drive numbers) to a table/array.
When the boot command is issued, grub would install an int 0x13
handler that uses this data as an lookup table
to convert requests and "forward" them to the original int 0x13 handler.


[lurkermode]

-- 
urjaman

Re: Replacing the legacy "map" command

[Robert Millan]

On Tue, May 27, 2008 at 11:59:22AM -0400, Pavel Roskin wrote:
> 
> Again, I'd like to see the command name shortened, perhaps to "drivemap"
> or "map".  It's not like users will need to distinguish BIOS and
> non-BIOS mappings.

But then we're occupping generic namespace with arch-specific features.

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Replacing the legacy "map" command

[Pavel Roskin]

On Wed, 2008-05-28 at 16:24 +0200, Robert Millan wrote:
> On Tue, May 27, 2008 at 11:59:22AM -0400, Pavel Roskin wrote:
> > 
> > Again, I'd like to see the command name shortened, perhaps to "drivemap"
> > or "map".  It's not like users will need to distinguish BIOS and
> > non-BIOS mappings.
> 
> But then we're occupping generic namespace with arch-specific features.

I think it's OK.  If another architecture supports similar
functionality, then "drivemap" would still be fine, even if the
implementation is different.  If an architecture doesn't support it, the
command won't exist.

It's very unlikely that there will be two different mechanisms for drive
remapping, each of which should be available to the users.  And even
then, a switch could be added to force the preferred mechanism.

-- 
Regards,
Pavel Roskin

Re: Replacing the legacy "map" command

[Javier Martín]

[-- Attachment #1: Type: text/plain, Size: 1863 bytes --]

Then "drivemap" it is. I've already been delving into the depths of The
Source, though I'd have preferred it to be commented a bit more
exhaustively. Some times it's difficult to guess which methods to call,
and it took me a bit to realize that I had to check with the disk
"device" to see if it was a BIOS drive. Seems I had the wrong conception
of what the disk "device" should be.

By the way, floppies are usually also "BIOS drives". Should the command
filter them out as sources for mapping? I think so, because OS loaders
usually have different code for reading floppies and HDs. Concretely,
floppy code usually makes some assumptions about drive geometry, uses
only old CHS-like interfaces, and some things more I cannot remember
now... it's been long since I wrote anything distantly similar to a
bootloader.

Anyways, the "drivemap" module is just above the phase of a working
"hello world" - I'm happy! ^.^

Habbit

El jue, 29-05-2008 a las 20:55 -0400, Pavel Roskin escribió:
> On Wed, 2008-05-28 at 16:24 +0200, Robert Millan wrote:
> > On Tue, May 27, 2008 at 11:59:22AM -0400, Pavel Roskin wrote:
> > > 
> > > Again, I'd like to see the command name shortened, perhaps to "drivemap"
> > > or "map".  It's not like users will need to distinguish BIOS and
> > > non-BIOS mappings.
> > 
> > But then we're occupping generic namespace with arch-specific features.
> 
> I think it's OK.  If another architecture supports similar
> functionality, then "drivemap" would still be fine, even if the
> implementation is different.  If an architecture doesn't support it, the
> command won't exist.
> 
> It's very unlikely that there will be two different mechanisms for drive
> remapping, each of which should be available to the users.  And even
> then, a switch could be added to force the preferred mechanism.
> 

[-- Attachment #2: Esta parte del mensaje está firmada digitalmente --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: Replacing the legacy "map" command

[Pavel Roskin]

Quoting Javier Martín <lordhabbit@gmail.com>:

> Then "drivemap" it is. I've already been delving into the depths of The
> Source, though I'd have preferred it to be commented a bit more
> exhaustively. Some times it's difficult to guess which methods to call,
> and it took me a bit to realize that I had to check with the disk
> "device" to see if it was a BIOS drive. Seems I had the wrong conception
> of what the disk "device" should be.

I suggest a simple approach starting from what can be done.  There  
will be a small resident piece of code installed to intercept  
interrupt 0x13 and replace the driver number according to the map.  We  
should keep that code simple.

The resident code should go through the list of the mapped drives, and  
if there is an entry in the table, replace the drive number.  That's  
two bytes for an entry.

I think the user interface should operate directly on that  
unidirectional map rather that with some complex swap rules that are  
translated to the map.  Sure, it won't be nice if some device is seen  
as two devices, especially if the OS uses BIOS.  But GRUB cannot  
prevent all stupid actions by users.

> By the way, floppies are usually also "BIOS drives". Should the command
> filter them out as sources for mapping? I think so, because OS loaders
> usually have different code for reading floppies and HDs. Concretely,
> floppy code usually makes some assumptions about drive geometry, uses
> only old CHS-like interfaces, and some things more I cannot remember
> now... it's been long since I wrote anything distantly similar to a
> bootloader.

We may still want to map a hard drive as a floppy.  I can imagine that  
some floppy readers may be seen as hard drives by the BIOS.

I think it's not the most important thing at the moment.  If there is  
a substantial risk of data corruption due to floppy-to-hdd remapping  
or duplicate drives, we can introduce extra safety checks.

-- 
Regards,
Pavel Roskin

Re: Replacing the legacy "map" command

[Robert Millan]

On Thu, May 29, 2008 at 11:48:33PM -0400, Pavel Roskin wrote:
> 
> We may still want to map a hard drive as a floppy.  I can imagine that  
> some floppy readers may be seen as hard drives by the BIOS.

And some BIOSes see USB drives as floppies too :-/

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What good is a phone call… if you are unable to speak?
(as seen on /.)

Re: Replacing the legacy "map" command

[Javier Martín]

Bit a bit, progress continues: I have some crappy UI semi-ready, to
improve on later, and I'm now battling with the INT13h handler to
perform the actual mapping - had to rescue the big old "Architecture
Programmer's Manual" in order to do PIC code without the amd64
rip-relative addressing. However, I haven't yet found a way to install
some kind of "hook" so that it will be called when the boot command is
issued, preferable just before calling grub_X_real_boot. However, I
didn't go to great lengths to find it... Anyone here willing to
enlighten me or should I find my own way through the darkness (again
¬¬)? I'd like to change/add the smallest amount of code possible, but
for the sake of modularity I prefer a pre-boot hook system rather than
a one-time solution.

Habbit

2008/5/30 Pavel Roskin <proski@gnu.org>:
> Quoting Javier Martín <lordhabbit@gmail.com>:
>
>> Then "drivemap" it is. I've already been delving into the depths of The
>> Source, though I'd have preferred it to be commented a bit more
>> exhaustively. Some times it's difficult to guess which methods to call,
>> and it took me a bit to realize that I had to check with the disk
>> "device" to see if it was a BIOS drive. Seems I had the wrong conception
>> of what the disk "device" should be.
>
> I suggest a simple approach starting from what can be done.  There will be a
> small resident piece of code installed to intercept interrupt 0x13 and
> replace the driver number according to the map.  We should keep that code
> simple.
>
> The resident code should go through the list of the mapped drives, and if
> there is an entry in the table, replace the drive number.  That's two bytes
> for an entry.
>
> I think the user interface should operate directly on that unidirectional
> map rather that with some complex swap rules that are translated to the map.
>  Sure, it won't be nice if some device is seen as two devices, especially if
> the OS uses BIOS.  But GRUB cannot prevent all stupid actions by users.
>
>> By the way, floppies are usually also "BIOS drives". Should the command
>> filter them out as sources for mapping? I think so, because OS loaders
>> usually have different code for reading floppies and HDs. Concretely,
>> floppy code usually makes some assumptions about drive geometry, uses
>> only old CHS-like interfaces, and some things more I cannot remember
>> now... it's been long since I wrote anything distantly similar to a
>> bootloader.
>
> We may still want to map a hard drive as a floppy.  I can imagine that some
> floppy readers may be seen as hard drives by the BIOS.
>
> I think it's not the most important thing at the moment.  If there is a
> substantial risk of data corruption due to floppy-to-hdd remapping or
> duplicate drives, we can introduce extra safety checks.
>
> --
> Regards,
> Pavel Roskin
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>

Re: Replacing the legacy "map" command

[Vesa Jääskeläinen]

Javier Martín wrote:
> Bit a bit, progress continues: I have some crappy UI semi-ready, to
> improve on later, and I'm now battling with the INT13h handler to
> perform the actual mapping - had to rescue the big old "Architecture
> Programmer's Manual" in order to do PIC code without the amd64
> rip-relative addressing. However, I haven't yet found a way to install
> some kind of "hook" so that it will be called when the boot command is
> issued, preferable just before calling grub_X_real_boot. However, I
> didn't go to great lengths to find it... Anyone here willing to
> enlighten me or should I find my own way through the darkness (again
> ¬¬)? I'd like to change/add the smallest amount of code possible, but
> for the sake of modularity I prefer a pre-boot hook system rather than
> a one-time solution.

Hi Javier,

There is no existing hook for this kind of stuff. So why not make one 
pointer to boot area where you can register one?. Code would then check 
if it is non-zero and call it before starting OS. Pointer would reside 
on same asm module as low level boot code is to keep it close by and 
helper on C-level to modify that pointer. You may want to support also 
multiple callbacks some way.

Thanks,
Vesa Jääskeläinen

Re: Replacing the legacy "map" command

[Vesa Jääskeläinen]

Hi Javier,

Welcome!

Javier Martín wrote:
> This message is mainly concerned with "how should I implement it"?
> Since this functionality is AFAIK exclusive to the x86/amd64 PC-BIOS
> target, it should be confined there, but there are at least two paths
> of action that I can think of:
>  1.- Given that only the chain OS loader makes use of such a switch, I
> could add it as an option to the "chainloader" command. This is the
> simplest path, as it would only require modifying
> loader/i386/pc/chainloader* and kern/i386/pc/startup.S (where
> grub_chainloader_real_boot resides). Such an option would most
> probably be named something like "mapdrivefirst", as in "chainloader
> --mapdrivefirst (hd1)+1".

Do not got to this route.

>  2.- The other option would be to add a new command that managed the
> BIOS drive mappings (like, "biosdrivesmap", duh), with three main
> options: "show", "map" and "reset". I'm still unsure of what files
> should be modified for this change, and this might be overkill.
> No matter how the UI (the commands) actually ends up, the mappings
> would just be stored in a variable, and not applied until the "boot"
> command is issued. Thus, actual functionality should probably be
> implemented in grub_chainloader_real_boot, and the logic is pretty
> simple, with most of it available in the GRUB Legacy code.

Well what we actually want is to have support to install interrup 
service chains for real mode. There are couple of users for this 
service, one of them being drive mapping (to int 13h) and then El Torito 
support (to int 13h).

So you would have function to install interrupt chain and this would be 
installed right away, or at point when you are leaving grub. You may 
want to have those services even if you are not chainloading. In example 
you have multiboot capable kernel, but you want to still use BIOS services.

In case of drive mapping it would work in user interface in same way as 
in GRUB Legacy. But what happens under the hood:

1. You load drive mapping module (map.mod?)
2. User configures mapping with map command (or similar)
3. If even one map command is issued this installs custom software 
interrupt handler to tweak int13h requests. Map command data is 
installed to this handler.

Thanks,
Vesa Jääskeläinen

Re: Replacing the legacy "map" command

[Urja Rannikko]

>
> Well what we actually want is to have support to install interrup service
> chains for real mode. There are couple of users for this service, one of
> them being drive mapping (to int 13h) and then El Torito support (to int
> 13h).
>
> So you would have function to install interrupt chain and this would be
> installed right away, or at point when you are leaving grub. You may want to
> have those services even if you are not chainloading. In example you have
> multiboot capable kernel, but you want to still use BIOS services.
>
> In case of drive mapping it would work in user interface in same way as in
> GRUB Legacy. But what happens under the hood:
>
> 1. You load drive mapping module (map.mod?)
> 2. User configures mapping with map command (or similar)
> 3. If even one map command is issued this installs custom software interrupt
> handler to tweak int13h requests. Map command data is installed to this
> handler.
>

This is exactly what i meant in my message, just took me a while to
write it so you beat me to it.

-- 
urjaman

Re: Replacing the legacy "map" command

[Javier Martín]

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

> > For grub, i'd suggest
> > map [olddrive newdrive] ...
> > eg. map (hd1) (hd0) (hd0) (hd1)
That was the syntax used in GRUB Legacy, it was successful and could be
reused, but I'd prefer the "drivemap" command I suggested earlier, with
options not just to map drives, but also to show the mappings and to
reset them to the BIOS defaults. I'd still consider a "--automap" flag
in the "chainloader" command quite useful, but I don't know how could
this be set up, since there would be two modules involved: "chain" and
the new "map" module. Chainloader would need to check whether the "map"
module is available and/or loaded ~_~

> > The map command would just store the given parameters (converted to
> > bios hex drive numbers) to a table/array.
> > When the boot command is issued, grub would install an int 0x13
> > handler that uses this data as an lookup table
> > to convert requests and "forward" them to the original int 0x13
> > handler.
Again, that was the method used in GRUB Legacy and my original plan:
contrary to what is suggested in other posts, the mappings would not
apply immediately, but just be stored in an appropriate table. The
required ISRs would be installed only after the boot order is issued,
and the logic in grub_chainloader_real_boot.

> > By the way, I can imagine that some kernels not loaded with
> > "chainloader" may benefit from the remapping as well. 
If we want the mappings to be applied to loaders other than chain,
however, we need to make sure the function installing the ISRs is run
before grub_X_real_boot. I'm not sure if installing such a "pre-boot"
handler is possible without significantly modifying the in-place
structure of GRUB.

Cheers!
Habbit



[-- Attachment #2: Esta parte del mensaje está firmada digitalmente --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: Replacing the legacy 'map' command

[Tomáš Ebenlendr]

Dne 27 Květen 2008, 23:47, Javier Martín napsal(a):
>>> For grub, i'd suggest
>>> map [olddrive newdrive] ... eg. map (hd1) (hd0) (hd0) (hd1)
> That was the syntax used in GRUB Legacy, it was successful and could be
> reused, but I'd prefer the "drivemap" command I suggested earlier, with
> options not just to map drives, but also to show the mappings and to reset
> them to the BIOS defaults. I'd still consider a "--automap" flag in the
> "chainloader" command quite useful, but I don't know how could
> this be set up, since there would be two modules involved: "chain" and the
> new "map" module. Chainloader would need to check whether the "map" module
> is available and/or loaded ~_~

The cooperation between modules is not hard, but I think
map -first (hdx) will be more useful. Such command just swaps hd0 and hdx
drives. This also covers the scenarios when we chainload something else
than first sector of given partition.

And maybe we can also consider following syntax:
map -pos (hd2) (hd0) (hd4)
with meaning:
  hd0->hd1,
  hd1->nothing
  hd2->hd0,
  hd3 not touch,
  hd4->hd2
I.e. positional syntax, first argument is mapped to (hd0), second to (hd1),
  and so on.

This syntax will be sufficient most times and writing
  chainloader --automap (hd1,0)+1
is same as writing
  map -first (hd1)
  chainloader (hd1,0)+1
or
  setting grub root to (hd1,0)
  map -root
  chainloader +1

-- 
Tomas Ebenlendr
                            http://drak.ucw.cz/~ebik

Re: Replacing the legacy 'map' command

[Javier Martín]

> The cooperation between modules is not hard, but I think
> map -first (hdx) will be more useful. Such command just swaps hd0 and hdx
> drives. This also covers the scenarios when we chainload something else
> than first sector of given partition.
>
> And maybe we can also consider following syntax:
> map -pos (hd2) (hd0) (hd4)
> with meaning:
>   hd0->hd1,
>   hd1->nothing
>   hd2->hd0,
>   hd3 not touch,
>   hd4->hd2
> I.e. positional syntax, first argument is mapped to (hd0), second to (hd1),
>   and so on.
>
> This syntax will be sufficient most times and writing
>   chainloader --automap (hd1,0)+1
> is same as writing
>   map -first (hd1)
>   chainloader (hd1,0)+1
> or
>   setting grub root to (hd1,0)
>   map -root
>   chainloader +1
>
> --
> Tomas Ebenlendr

Well, such "additional" UIs will be useful, and there seems to be some
consensus on how this should be done:
   1.- There should be a new command called somethink like "drivemap"
and based on a new module that would provide the drive mapping
capabilities.
   2.- This module would be available in x86/PC (and, if it is ever
split from it, AMD64/PC), and would work by saving the user requested
mappings and installing a pre-boot GRUB routine.
   3.- Such a routine would, before the execution of grub_X_real_boot,
install the appropate ISRs (which are mostly available in GRUB Legacy
code) to make the mappings effective, no matter what the OS loader is
- chain, multiboot, etc.

Therefore, if there are no objections to this plan, I can start the
implementation (when I finish my exams, of course u_u). I think I will
initially place the ISR installation code in
grub_chainloader_real_boot in order to have a working version right
away, then move it away into its own function and, however the way,
"register" it with the boot routines so that it is executed as
mentioned above. Other modules/commands would be later modified to add
drivemap-related functionality. Is there any site/document (apart from
the sources themselves) in which I can learn more about the internals
of GRUB2 and, specifically, the "boot" command sequence?.

Habbit

---no PGP signature, I'm writing from the uni---

Re: Replacing the legacy 'map' command

[Vesa Jääskeläinen]

Javier Martín wrote:
> drivemap-related functionality. Is there any site/document (apart from
> the sources themselves) in which I can learn more about the internals
> of GRUB2 and, specifically, the "boot" command sequence?.

GRUB 2 Wiki will serve the purpose for developer documentation:

http://grub.enbug.org/

If not there, then use the source luke ;)