network support : memory management problem

network support : memory management problem

[Vincent Guffens]

Hi,

I sent a similar e-mail yesterday but I think it didn't get through.

I have a working version of the netboot support in grub2. I can issue 
commands like (seeking in file is not yet possible):

cat (nd0)test.txt
linux (nd0)linux24

but depending on where in the code I free my data blocks, I sometimes 
get a  "free magic is broken" fatal error msg from grub_free().

I found out that if I use the grub_printf() function just before the 
call to grub_free(), the problem disappears.

That is to say that in my grub_net_close function (the close file 
function associated with the net binding file system), I do something like:

struct grub_netfs_data * priv =  (struct grub_netfs_data *) file->data;
struct grub_netfs_block *pp, * p = priv->head;

grub_printf("FREEING\n");

if (p)
   pp = p->next;

   while (p) {
     if ((p->data)){
       grub_free(p->data);
     }


If I remove the FREEING msg, I have the panic error message, otherwise, 
everything looks fine. The exact error message is

free magic is broken at 0x85900: 0x0

Does someone has an idea ? Is there some documentation available about 
the mm in grub2 ?

The full code is available on this web page:
http://www.auto.ucl.ac.be/~guffens/grub2_netboot/index.html


-- 
				Vincent Guffens
				PhD Student UCL/CESAME
				tel:   +32 10 47 80 30
Value your freedom, or you will lose it, teaches history.
"Don't bother us with politics," respond those who don't want to learn.
	      	-- Richard M. Stallman

Re: network support : memory management problem

[Vincent Guffens]

it looks like the problem is gone. I changed the way I allocate the 
memory to hold a block. I changed from:

int load_block ()
	char * p = (char *) grub_malloc(len*sizeof(char));

to

int load_block ()
	char * p;
[...]
p = (char *) grub_malloc(len*sizeof(char));


but still, I would be interested in documentation about the memory 
layout and the memory management of grub2, if available.

Thanks !



Vincent Guffens wrote:
> Hi,
> 
> I sent a similar e-mail yesterday but I think it didn't get through.
> 
> I have a working version of the netboot support in grub2. I can issue 
> commands like (seeking in file is not yet possible):
> 
> cat (nd0)test.txt
> linux (nd0)linux24
> 
> but depending on where in the code I free my data blocks, I sometimes 
> get a  "free magic is broken" fatal error msg from grub_free().
> 
> I found out that if I use the grub_printf() function just before the 
> call to grub_free(), the problem disappears.
> 
> That is to say that in my grub_net_close function (the close file 
> function associated with the net binding file system), I do something like:
> 
> struct grub_netfs_data * priv =  (struct grub_netfs_data *) file->data;
> struct grub_netfs_block *pp, * p = priv->head;
> 
> grub_printf("FREEING\n");
> 
> if (p)
>   pp = p->next;
> 
>   while (p) {
>     if ((p->data)){
>       grub_free(p->data);
>     }
> 
> 
> If I remove the FREEING msg, I have the panic error message, otherwise, 
> everything looks fine. The exact error message is
> 
> free magic is broken at 0x85900: 0x0
> 
> Does someone has an idea ? Is there some documentation available about 
> the mm in grub2 ?
> 
> The full code is available on this web page:
> http://www.auto.ucl.ac.be/~guffens/grub2_netboot/index.html
> 
> 


-- 
				Vincent Guffens
				PhD Student UCL/CESAME
				tel:   +32 10 47 80 30
Value your freedom, or you will lose it, teaches history.
"Don't bother us with politics," respond those who don't want to learn.
	      	-- Richard M. Stallman

Re: network support : memory management problem

[Marco Gerards]

Vincent Guffens <guffens@inma.ucl.ac.be> writes:

> it looks like the problem is gone. I changed the way I allocate the
> memory to hold a block. I changed from:
>
> int load_block ()
> 	char * p = (char *) grub_malloc(len*sizeof(char));
>
> to
>
> int load_block ()
> 	char * p;
> [...]
> p = (char *) grub_malloc(len*sizeof(char));

This code is just the same.  I think the problem is still there but
just does not show up anymore.  I have seen a similar error before,
but I did not have the time to debug it.

> but still, I would be interested in documentation about the memory
> layout and the memory management of grub2, if available.

AFAIK there is none.  But you can ask specific questions of course.

--
Marco

Re: network support : memory management problem

[Vincent Guffens]

Marco Gerards wrote:
> Vincent Guffens <guffens@inma.ucl.ac.be> writes:
> 
> 
>>it looks like the problem is gone. I changed the way I allocate the
>>memory to hold a block. I changed from:
>>
>>int load_block ()
>>	char * p = (char *) grub_malloc(len*sizeof(char));
>>
>>to
>>
>>int load_block ()
>>	char * p;
>>[...]
>>p = (char *) grub_malloc(len*sizeof(char));
> 
> 
> This code is just the same.  I think the problem is still there but
> just does not show up anymore.  I have seen a similar error before,
> but I did not have the time to debug it.

Indead the problem is still there. It is now possible to do something like:

	grub> ifconfig
	[...]
	grub> loopback lo (nd)tfile
	[...]
	grub> ls (lo)/
	[...]
	grub> cat (lo)/file.txt
	[...]
	

I have tested three different card: Tulip, e1000, nforce2 as well as 
three compilers: gcc-2.95, gcc-3.0, gcc-3.3 However, the memory problem 
is still there, and I have no idea where it comes from. It varies 
depending on the compiler, the optimization flag, and some very small 
changes in the code. But it is always possible to find one combination 
of these for which the problem appears after loading a linux kernel or 
after the command boot or after a cat of a text file in the loopback device.



-- 
				Vincent Guffens
				PhD Student UCL/CESAME
				tel:   +32 10 47 80 30
Value your freedom, or you will lose it, teaches history.
"Don't bother us with politics," respond those who don't want to learn.
	      	-- Richard M. Stallman

Re: network support : memory management problem

[Marco Gerards]

Vincent Guffens <guffens@inma.ucl.ac.be> writes:

> I have tested three different card: Tulip, e1000, nforce2 as well as
> three compilers: gcc-2.95, gcc-3.0, gcc-3.3 However, the memory
> problem is still there, and I have no idea where it comes from. It
> varies depending on the compiler, the optimization flag, and some very
> small changes in the code. But it is always possible to find one
> combination of these for which the problem appears after loading a
> linux kernel or after the command boot or after a cat of a text file
> in the loopback device.

It can even be in some other code you did not write...  If you can
reproduce this problem easily, it would be very nice.

--
Marco

Re: network support : memory management problem

[Vincent Guffens]

On Wed, 08 Jun 2005 21:38:20 +0200, Marco Gerards wrote
> Vincent Guffens <guffens@inma.ucl.ac.be> writes:
> 
> > I have tested three different card: Tulip, e1000, nforce2 as well as
> > three compilers: gcc-2.95, gcc-3.0, gcc-3.3 However, the memory
> > problem is still there, and I have no idea where it comes from. It
> > varies depending on the compiler, the optimization flag, and some very
> > small changes in the code. But it is always possible to find one
> > combination of these for which the problem appears after loading a
> > linux kernel or after the command boot or after a cat of a text file
> > in the loopback device.
> 
> It can even be in some other code you did not write...  If you can
> reproduce this problem easily, it would be very nice.

yes, it is easy to reproduce, if you want to try by yourslef, you can find 
the code here:

http://www.auto.ucl.ac.be/~guffens/grub2_netboot/index.html

I will try again next week to find where the problem comes from.

> 
> --
> Marco
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel




--
         Vincent Guffens
         UCL/CESAME  +32 10 47 80 30 
         Euler Building A017

Scripting and keystrokes

[Serbinenko Vladimir]

Hello all. I'm back. I wrote nothing last time because I was quite busy.
2 themes I'd like to speak of:
1) Scripting: how make it: three possibilities
    a) Hand written
    b) Bison with 2 branches: one executes directly and other saves the
commands that are for later execution
             **UGLY**
    c) Convert with bison to the structure like

    enum grub_script_cmd_type
{GRUB_SCRIPT_CMD_TYPE_COMMAND,GRUB_SCRIPT_CMD_TYPE_FOR_IN, <...>};

    struct grub_script_for_in
       {
          char *var;
          char **list;
          grub_script_cmd *loop_begin;
          grub_script_cmd *skip_loop;
       }
  
    struct grub_script_cmd
        {
             grub_script_cmd_type type;
             union
                {
                   char *command;
                   grub_script_for_in for_in;
                   <...>
                }
        }

*GOOD SOLUTION IMHO*

Which scripting: I propose to make bash-like scripting and make it so
close as possible to bash

2) Keystrokes: emulation of keypress: just write keys in buffer before
booting. I propose to add array of functions grub_preboot that will be
launched between grub_machine_fini and grub_loader_boot_func and
functions grub_add_preboot and grub_remove_preboot
And command syntax like:
keystroke [<first key> [<second key> [...]]]
Example
keystroke h e l l o
Space is needed because some keys can't be written with one symbol like:
F1, insert, ...
ex:
keystroke F8 down enter

I'm ready to write both scripting and keystroke.


                                                                        
                                                            Vladimir
Serbinenko

Re: Scripting and keystrokes

[Serbinenko Vladimir]

Other idea: change menu syntax to select like command (see man bash) but
instead of prompting for number GRUB will write it to menu. What do you
think about it?
                                                                     
Vladimir

Re: Scripting and keystrokes

[Yoshinori K. Okuji]

On Monday 13 June 2005 20:09, Serbinenko Vladimir wrote:
> Other idea: change menu syntax to select like command (see man bash) but
> instead of prompting for number GRUB will write it to menu. What do you
> think about it?

I am too stupid to imagine it. Can you show an example?

Okuji

Re: Scripting and keystrokes

[Serbinenko Vladimir]

Yoshinori K. Okuji wrote:

>On Monday 13 June 2005 20:09, Serbinenko Vladimir wrote:
>  
>
>>Other idea: change menu syntax to select like command (see man bash) but
>>instead of prompting for number GRUB will write it to menu. What do you
>>think about it?
>>    
>>
>
>I am too stupid to imagine it. Can you show an example?
>
>
>  
>
I'm sorry that I wrote this letter too fast and w/o examples.
Example
If you write in normal bash:
select x in "Debian Linux" "Gentoo Linux" "Mandrake Linux";
do
    case "$x" in
        ("Debian Linux") 
            echo lets boot debian;
            break;;
       ("Gentoo Linux")
            echo lets boot gentoo;
            break;;
      ("Mandrake Linux")
            echo lets drake;
            break;;
    esac;
done
It will show
1) Debian Linux
2) Gentoo Linux
3) Mandrake Linux
And ask for number and then execute corresponding entry. If GRUB adapts
this syntax it will show menu
Debian Linux
Gentoo Linux
Mandrake Linux
ask use to choose and then boot corresponding entry.
Disadvantages:
    You must write two times the name.
Advantages:
    It's close to bash
    Sub-menus.
    "dynamic" menus (created by scripts)
    Easy creating of similar entries

select x in "Debian Linux" "Gentoo Linux" "Mandrake Linux network1"
"Mandrake Linux network2";
do
    case "$x" in
        ("Debian Linux") 
            echo lets boot debian;
            break;;
      ("Mandrake Linux "* )
            echo lets drake with PROFILE=${x:15};
            break;;
       ("Gentoo Linux")
            echo lets boot gentoo;
            break;;
    esac;
done

In this case script determines which value to pass in PROFILE variable.
Here it's not particularly advantaging but if you have more parameters.
like:
Mandrake Linux network1
Mandrake Linux network2
Mandrake Linux console network1
Mandrake Linux console network2
Mandrake Linux console nonfb network1
Mandrake Linux console nonfb network2
Mandrake Linux console nonfb failsafe network1
Mandrake Linux console nonfb failsafe network2

Script can check if keywords console, network1, network2, nonfb and
failsafe are present and boot corresponding config.
   
                                                                        
                                 Vladimir

Re: Scripting and keystrokes

[Yoshinori K. Okuji]

On Sunday 12 June 2005 10:54, Serbinenko Vladimir wrote:
> Which scripting: I propose to make bash-like scripting and make it so
> close as possible to bash

Agreed.

> 2) Keystrokes: emulation of keypress: just write keys in buffer before
> booting. I propose to add array of functions grub_preboot that will be
> launched between grub_machine_fini and grub_loader_boot_func and
> functions grub_add_preboot and grub_remove_preboot
> And command syntax like:
> keystroke [<first key> [<second key> [...]]]
> Example
> keystroke h e l l o
> Space is needed because some keys can't be written with one symbol like:
> F1, insert, ...
> ex:
> keystroke F8 down enter

Can you describe the expected usage of this? For chain-loading Windows?

Okuji

Re: Scripting and keystrokes

[Serbinenko Vladimir]

Yoshinori K. Okuji wrote:

>On Sunday 12 June 2005 10:54, Serbinenko Vladimir wrote:
>  
>
>>Which scripting: I propose to make bash-like scripting and make it so
>>close as possible to bash
>>    
>>
>
>Agreed.
>
>  
>
OK. But question about realization is still open. All ideas are welcome

>>2) Keystrokes: emulation of keypress: just write keys in buffer before
>>booting. I propose to add array of functions grub_preboot that will be
>>launched between grub_machine_fini and grub_loader_boot_func and
>>functions grub_add_preboot and grub_remove_preboot
>>And command syntax like:
>>keystroke [<first key> [<second key> [...]]]
>>Example
>>keystroke h e l l o
>>Space is needed because some keys can't be written with one symbol like:
>>F1, insert, ...
>>ex:
>>keystroke F8 down enter
>>    
>>
>
>Can you describe the expected usage of this? For chain-loading Windows?
>  
>
It's useful for loading some OS in special modes when it checks if you
press special key to enter special mode but it goes too fast and you
can't press it fast enough after pressing enter in GRUB.
Often it's failsafe modes. Sometimes I had this problem with windows
when it bugs and I can't enter in failsafe-mode because it goes too fast.
Another usage: chainloading boot-managers.
If you install other manager for special OS you can choose from GRUB
directly the configuration like: down down enter.
Not only boot-managers but also some OS like DOS including FreeDos if
multiple configurations are present in config file.
Ex: As I heard GRUB has problems loading FreeBSD. So you can install
FreeBSD native bootloader even have multiple BSD and/or configurations
and choose it from GRUB.

>Okuji
>
>
>_______________________________________________
>Grub-devel mailing list
>Grub-devel@gnu.org
>http://lists.gnu.org/mailman/listinfo/grub-devel
>
>
>  
>

Re: network support : memory management problem

[Yoshinori K. Okuji]

On Tuesday 31 May 2005 15:48, Vincent Guffens wrote:
> but depending on where in the code I free my data blocks, I sometimes
> get a  "free magic is broken" fatal error msg from grub_free().

Usually, it is due to a buffer overrun or underrun.

> Does someone has an idea ? Is there some documentation available about
> the mm in grub2 ?

It is a typical (simple) malloc implementation. You can look at any textbook 
or web site about heap management to get ideas. The extensions are to always 
use magic to detect overrun / underrun, and to support multiple 
non-contiguous free regions. Have a look at the code. It is very small.

Okuji