Question: how to add new menu functionality

Question: how to add new menu functionality

[Stephen Torri]

I am using the released grub-1.98 code for my development. The problem I
am having is how to add new functionality to the 'normal' module. I
would like to add a new feature to the code using some functions that
don't presently exist in the code. When I coded the functions I had to
manually edit def-normal.lst to get the compiler to be happy and
successfully complete the build. This was not enough because when used
the new grub2 loader on a drive I got an error saying "error: the symbol
'work' not found" is displayed. This error code is reported because the
symbol cannot be found in a symbol table. So to summarize what I am
asking:

1. How do you configure the build system to compile with new files (e.g.
new command for command line)?

2. How do you configure the build system to compile with new functions
contained in existing files?

3. Is there a tutorial on how to develop for Grub2? If I missed that in
my search I am sorry for not reading that first.

4. How do you debug your Grub2 builds? The web had references to a patch
to add in GDB functionality.

Stephen

Re: Question: how to add new menu functionality

[Colin Watson]

On Fri, Apr 22, 2011 at 06:03:41AM -0400, Stephen Torri wrote:
> I am using the released grub-1.98 code for my development.

I recommend using 1.99~rc2 or a current bzr checkout, if you can.  Not
just on general principles, but because its rewritten build system is
really a great deal easier to modify in the kinds of ways you need.
1.98's was generally a pain.

> The problem I am having is how to add new functionality to the
> 'normal' module. I would like to add a new feature to the code using
> some functions that don't presently exist in the code. When I coded
> the functions I had to manually edit def-normal.lst to get the
> compiler to be happy and successfully complete the build.

That's a sign that you've missed something; you should never have to
edit that file.

> This was not enough because when used the new grub2 loader on a drive
> I got an error saying "error: the symbol 'work' not found" is
> displayed. This error code is reported because the symbol cannot be
> found in a symbol table. So to summarize what I am asking:
> 
> 1. How do you configure the build system to compile with new files (e.g.
> new command for command line)?

In 1.98, you need to find the appropriate targets in conf/*.rmk (they'll
either be in common.rmk or a platform-specific file - you may need to
edit both), and add the new files there.

In 1.99, you need to edit grub-core/Makefile.core.def and add lines for
the files you're adding to the appropriate target block, either with
'common =' if they're platform-independent, or with a platform-specific
prefix.  Follow the patterns of the surrounding lines.

In either case, you need to run ./autogen.sh and (re-)run ./configure
before building.  This may require additional build system dependencies
beyond those required to build GRUB itself; see the INSTALL file.

It helps to work in a bzr checkout, so that you can use the usual
version control commands to keep track of what you've changed (even if
you never commit anything).

> 2. How do you configure the build system to compile with new functions
> contained in existing files?

Using new functions entirely within a single file requires no special
actions, in either 1.98 or 1.99.

If you're using functions in file A that are defined in file B, you'll
need to make sure that file A includes any necessary headers
(conventionally in include/grub/, using '#include <grub/foo.h>'), and
that any targets that involve file A also involve file B, much as you
would for any normal C project.  See my answer to your previous question
for the exact build system mechanics.

> 3. Is there a tutorial on how to develop for Grub2? If I missed that in
> my search I am sorry for not reading that first.

I've seen the odd one floating around, but they've generally had
substantial flaws (even one of the best I saw involved editing generated
build system files rather than the source files, which is unwise) and/or
not been up to date.  We recently started writing a developer manual,
which is in docs/grub-dev.texi in 1.99~rc2 (or a current bzr checkout)
although not yet on the web.  It may well not yet be what you need, but
it should be a useful start and we'd appreciate feedback from people
trying to use it.

> 4. How do you debug your Grub2 builds? The web had references to a patch
> to add in GDB functionality.

That patch is pretty old and won't apply to current builds.  I recently
started attempting to bring it up to date, although I've been very busy
and so don't have anything to show yet.  I have a few train journeys and
hotel stays in my near future, and may well hack on it then.

For most purposes, using grub_dprintf and 'set debug=all' (or a more
limited set of debug options) is good enough: add debugging output until
it shows what you need.  It helps enormously if your code can be run in
a rescue image that you can run in a virtual machine.  In a built 1.98
tree, where the 'rescue-disk' directory is one you've created with any
other files you need in the rescue image:

  ./grub-mkrescue --override-directory=. --output=grub.iso rescue-disk

In 1.99:

  ./grub-mkrescue --grub-mkimage=./grub-mkimage --override-directory=grub-core --output=grub.iso rescue-disk

You can then boot this in 'kvm -cdrom grub.iso' or similar.  This
greatly speeds up the debugging cycle, versus having to boot it on real
hardware.

In some cases, particularly if you're working on a failure very early
on, you need to resort to stronger techniques.  For instance, a serial
console means that you can see more debugging output, and can help if
the debugging output you need is from the video subsystem (since of
course changing video modes loses screen contents).  GDB-over-serial
will no doubt be great when we get that far.  It's also possible to use
GDB in conjunction with QEMU to debug some parts of GRUB even without
special support in GRUB, if you're sufficiently determined and can cope
with the lack of symbols.  I wrote about this on my blog recently:

  http://www.chiark.greenend.org.uk/ucgi/~cjwatson/blosxom/ubuntu/2011-03-14-wubi-bug-693671.html

Depending on what you're doing, you can often debug large parts of GRUB
in userspace, which makes hacking on GRUB a lot more fun.  For instance,
you can use grub-probe or grub-fstest to debug filesystem drivers, and
you can use grub-emu to debug some other things if you can live with its
eccentricities.  As general advice, it's worth spending a little time
up-front to attempt to move your problem into the most congenial
debugging environment available; GRUB offers developers several ways to
do that, and so it is rarely necessary to resort to heroic techniques
unless you are working on the lowest levels of machine-dependent code.

Cheers,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Re: Question: how to add new menu functionality

[Stephen Torri]

Thanks for the help. The build code for 1.99 is much easier to navigate
than for 1.98. I downloaded the 1.99-rc2 source from the ftp site.

As I worked on the code today I came up with four more questions on
developing grub2:

1. This relates to the use of the kvm for testing a grub2 iso. How do
you scroll up and down in the view in the kvm window?

	When using the grub.iso with the livecd image in it I noticed that I
got a lot of messages that scrolled up and I could not see them with
debug=all.

2. How do you do unblocking I/O from stdin? There are ways to do it with
the select system call but it seems that there are only grub functions
that are substituted for system calls.

3. How do you call a  standard C library routine or system call to work
in grub? For example called read().

	In working with the code today I noticed that the standard C library
headers were not being found because of two compiler flags (-inostdinc
and -isystem). The path to the system headers, /usr/include, was not in
the compiler strings for each file.

4. Lets say you manually compiled certain files by copying the Makefile
command and removed the -inostdinc and -isystem. When I was working on
1.98 and used a system header to get a function I had to manually
include /usr/include and alter the appropriate def-<name>.lst file to
get the code to compile. When I installed that code to a bootable device
and restarted the system I saw a message that said:

     <function name> in <module> is not defined

Often the culprit was the system calls being used (e.g. select). I would
get a message when grub started on the bootable device that said:
 
     select in normal is not defined

This probably relates to the third question. This message tells me what
is wrong but does not offer any information on how to resolve it. What
step(s) did I miss that resulted in this message?

I appreciate the help.

Stephen

Re: Question: how to add new menu functionality

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 650 bytes --]


> 1. This relates to the use of the kvm for testing a grub2 iso. How do
> you scroll up and down in the view in the kvm window?
>
pager=1
> 2. How do you do unblocking I/O from stdin? There are ways to do it with
> the select system call but it seems that there are only grub functions
> that are substituted for system calls.
>
grub_checkkey
> 3. How do you call a  standard C library routine or system call to work
> in grub? For example called read().
>
You don't. You're in an inter-OS space and the only functions you have
are the ones you have in your oxygen bottles.

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: Question: how to add new menu functionality

[Colin Watson]

On Fri, Apr 22, 2011 at 06:25:26PM -0400, Stephen Torri wrote:
> Thanks for the help. The build code for 1.99 is much easier to navigate
> than for 1.98. I downloaded the 1.99-rc2 source from the ftp site.
> 
> As I worked on the code today I came up with four more questions on
> developing grub2:

Vladimir has answered the first three.  While the answer to the fourth
does follow from his answer to the third, I'll be explicit about it:

> 4. Lets say you manually compiled certain files by copying the Makefile
> command and removed the -inostdinc and -isystem. When I was working on
> 1.98 and used a system header to get a function I had to manually
> include /usr/include and alter the appropriate def-<name>.lst file to
> get the code to compile. When I installed that code to a bootable device

Don't do this - really!  The build system made this an error because
what you're trying to do is not possible; your hacks may have made it
limp on anyway and compile, but the only thing that disregarding the
errors from the build system achieves is that it will fail at run-time
instead.  The errors you got were correct.

> When I installed that code to a bootable device and restarted the
> system I saw a message that said:
> 
>      <function name> in <module> is not defined
> 
> Often the culprit was the system calls being used (e.g. select). I would
> get a message when grub started on the bootable device that said:
>  
>      select in normal is not defined

You can't do this.  select and other functions you may be familiar with
from the standard C library are implemented on top of your operating
system's kernel.  They simply do not exist in GRUB.

In the terms of the C standard and many books on C, GRUB (the boot
loader environment, as opposed to the userspace utilities) is more like
a freestanding implementation than a hosted one.  Programs written for
freestanding implementations may only use an extremely limited subset of
the standard library, essentially limited to macro definitions.

A small number of familiar function names (memcmp, memmove, memcpy,
memset) are present in GRUB because GCC may generate code that expects
them to be available, but they should not be used in manually-written
code.  Aside from those, no C library functions are available in GRUB,
and you must not attempt to use them.  You need to find grub_*
equivalents instead.

(This does not apply to the GRUB userspace utilities - grub-probe,
grub-mkimage, etc. - which run on top of an operating system.  Those can
and should generally use OS functions where appropriate.  Code that is
used in both boot and userspace contexts uses grub_* functions, and we
provide a compatibility layer for userspace.  This doesn't apply to
normal.mod, though.)

-- 
Colin Watson                                       [cjwatson@ubuntu.com]