My Summer of Code Project

My Summer of Code Project

[Alex Roman]

Hello list,

My name is Alex Roman and I have been selected as a Summer of Code
student for the GRUB2 project. I thought I'd introduce myself and my
project. I am a second year student at the University of Waterloo
(just finished 2A, first term of second year). I have experience
developing in low-level environments in C and assembly.

Now about my project. I will be adding CD-ROM booting functionality to
GRUB2. The first stage of the project will attempt to use BIOS calls
to achieve this functionality (which is, of course, dependent on the
BIOS's capabilities, standards compliance, etc.). The second stage
will involve writing a complete ATA/ATAPI driver which would bypass
the BIOS completely, thus allowing it to work on virtually any system
out there.

Unfortunately I only have access to ATAPI CD-ROM drives, so I won't be
able to work on SCSI and/or USB, however I will try to separate the
system to load the boot image from the driver itself as much as
possible so that future driver developments for SCSI and USB will just
be sort of drop-in replacements.

So, just thought I'd talk a bit about what I'll be doing...

Thank you for your attention and time. I'm sure I'll have lots of
questions in the months to come. :)

-- 
Alex Roman <alex.roman@gmail.com>

Re: My Summer of Code Project

[Devils-Hawk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

You should probably have a look at http://www.linuxbios.org/FILO maybe
you can salvage something. FILO supports booting from ElTorito bootable
CD media, uses 32-bit mode, and it is free of bios calls. Unfortunately
its x86 only, although some work has been done to get it running on PPC.

regards DevH

Alex Roman wrote:
> Hello list,
> 
> My name is Alex Roman and I have been selected as a Summer of Code
> student for the GRUB2 project. I thought I'd introduce myself and my
> project. I am a second year student at the University of Waterloo
> (just finished 2A, first term of second year). I have experience
> developing in low-level environments in C and assembly.
> 
> Now about my project. I will be adding CD-ROM booting functionality to
> GRUB2. The first stage of the project will attempt to use BIOS calls
> to achieve this functionality (which is, of course, dependent on the
> BIOS's capabilities, standards compliance, etc.). The second stage
> will involve writing a complete ATA/ATAPI driver which would bypass
> the BIOS completely, thus allowing it to work on virtually any system
> out there.
> 
> Unfortunately I only have access to ATAPI CD-ROM drives, so I won't be
> able to work on SCSI and/or USB, however I will try to separate the
> system to load the boot image from the driver itself as much as
> possible so that future driver developments for SCSI and USB will just
> be sort of drop-in replacements.
> 
> So, just thought I'd talk a bit about what I'll be doing...
> 
> Thank you for your attention and time. I'm sure I'll have lots of
> questions in the months to come. :)
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFGLmM6CbmO6P7mspMRCkGqAJ9I6u7BlaiHWjJ1T5qBrwuQ3wyy0QCbBE+Q
QsyJ7MVm3zNEfmA8LuVqcGE=
=L3H4
-----END PGP SIGNATURE-----

Re: My Summer of Code Project

[Stefan Reinauer]

* Devils-Hawk <devilshawk@aon.at> [070424 22:06]:
> You should probably have a look at http://www.linuxbios.org/FILO maybe
> you can salvage something. FILO supports booting from ElTorito bootable
> CD media, uses 32-bit mode, and it is free of bios calls. Unfortunately
> its x86 only, although some work has been done to get it running on PPC.

FILO has been used on PPC for quite some time. For simplicity, that
version has been integrated into LinuxBIOS directly though. 

There is a GSoC project for porting Grub2 to work in LinuxBIOS. It will
be realized by Patrick Georgi <patrick@georgi-clan.de>.

Maybe you guys can join efforts on solving this issue?

I would like to see FILO die. Which will happen once GRUB2 will work on
an x86 machine without 16bit real mode legacy BIOS. 

Stefan

-- 
coresystems GmbH • Brahmsstr. 16 • D-79104 Freiburg i. Br.
      Tel.: +49 761 7668825 • Fax: +49 761 7664613
Email: info@coresystems.de  • http://www.coresystems.de/

Re: My Summer of Code Project

[Marco Gerards]

Stefan Reinauer <stepan@coresystems.de> writes:

> * Devils-Hawk <devilshawk@aon.at> [070424 22:06]:
>> You should probably have a look at http://www.linuxbios.org/FILO maybe
>> you can salvage something. FILO supports booting from ElTorito bootable
>> CD media, uses 32-bit mode, and it is free of bios calls. Unfortunately
>> its x86 only, although some work has been done to get it running on PPC.
>
> FILO has been used on PPC for quite some time. For simplicity, that
> version has been integrated into LinuxBIOS directly though. 
>
> There is a GSoC project for porting Grub2 to work in LinuxBIOS. It will
> be realized by Patrick Georgi <patrick@georgi-clan.de>.
>
> Maybe you guys can join efforts on solving this issue?
>
> I would like to see FILO die. Which will happen once GRUB2 will work on
> an x86 machine without 16bit real mode legacy BIOS. 

Hopefully your student will send in patches to grub-devel.  Although I
will not have a lot of time next few months, I will make sure I will
review these patches and answer questions from both students.  Or do
you want to wait with this until the project is finished?  Whatever
you prefer to do, I think it is wise that we cooperate closely to make
sure both SoC projects succeed (as in: code applied to CVS at the
end).

--
Marco

Re: My Summer of Code Project

[Alex Roman]

Interesting piece of code... Yet another example of showing how to
implement an ATA/IDE controller driver :)

Right now I'm pondering whether it is worth implementing a BIOS calls
CDROM boot support, or whether I should just go right ahead and do it
all with the ATA controller.

I'm not sure how IDE controllers work on PPC, since I've unfortunately
never used the architecture...

Technically, if the ATA/IDE driver is there, implementing the ATAPI
command set to "talk" to the CDROM and interpreting the El-Torito spec
shouldn't be that hard.

If CD-ROM drives are ATAPI on all platforms (where they use ATA), and
the ATA code is there, the ATAPI and El Torito layer should stay
cross-platform. The interesting bit will be making it so that
"plugging in" a SCSI driver will require the least amount of code
change.

Anyways, I'm kind of rambling for now... Until the official "code
start" day I'll do some more reading and investigating how it would be
best to tie the code into GRUB2 to give the most elegant solution.


Cheers,
Alex

On 24/04/07, Devils-Hawk <devilshawk@aon.at> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> You should probably have a look at http://www.linuxbios.org/FILO maybe
> you can salvage something. FILO supports booting from ElTorito bootable
> CD media, uses 32-bit mode, and it is free of bios calls. Unfortunately
> its x86 only, although some work has been done to get it running on PPC.
>
> regards DevH
>
> Alex Roman wrote:
> > Hello list,
> >
> > My name is Alex Roman and I have been selected as a Summer of Code
> > student for the GRUB2 project. I thought I'd introduce myself and my
> > project. I am a second year student at the University of Waterloo
> > (just finished 2A, first term of second year). I have experience
> > developing in low-level environments in C and assembly.
> >
> > Now about my project. I will be adding CD-ROM booting functionality to
> > GRUB2. The first stage of the project will attempt to use BIOS calls
> > to achieve this functionality (which is, of course, dependent on the
> > BIOS's capabilities, standards compliance, etc.). The second stage
> > will involve writing a complete ATA/ATAPI driver which would bypass
> > the BIOS completely, thus allowing it to work on virtually any system
> > out there.
> >
> > Unfortunately I only have access to ATAPI CD-ROM drives, so I won't be
> > able to work on SCSI and/or USB, however I will try to separate the
> > system to load the boot image from the driver itself as much as
> > possible so that future driver developments for SCSI and USB will just
> > be sort of drop-in replacements.
> >
> > So, just thought I'd talk a bit about what I'll be doing...
> >
> > Thank you for your attention and time. I'm sure I'll have lots of
> > questions in the months to come. :)
> >
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD8DBQFGLmM6CbmO6P7mspMRCkGqAJ9I6u7BlaiHWjJ1T5qBrwuQ3wyy0QCbBE+Q
> QsyJ7MVm3zNEfmA8LuVqcGE=
> =L3H4
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>


-- 
Alex Roman <alex.roman@gmail.com>

Re: My Summer of Code Project

[Andrei E. Warkentin]

[-- Attachment #1: Type: text/plain, Size: 3910 bytes --]

I think it's safe to say that IDE is pretty much the same  
anywhere.... on PowerPC there is no dedicated PIO address space, so  
the IDE registers are memory mapped someplace (so don't forget to  
eieio) (read PCI BARs or interpret device tree?).

Andrei Evgenievich Warkentin
andrey.warkentin@gmail.com
Cell: (+1) (847) 321-15-55


On 24.04.2007, at 23:49, Alex Roman wrote:

> Interesting piece of code... Yet another example of showing how to
> implement an ATA/IDE controller driver :)
>
> Right now I'm pondering whether it is worth implementing a BIOS calls
> CDROM boot support, or whether I should just go right ahead and do it
> all with the ATA controller.
>
> I'm not sure how IDE controllers work on PPC, since I've unfortunately
> never used the architecture...
>
> Technically, if the ATA/IDE driver is there, implementing the ATAPI
> command set to "talk" to the CDROM and interpreting the El-Torito spec
> shouldn't be that hard.
>
> If CD-ROM drives are ATAPI on all platforms (where they use ATA), and
> the ATA code is there, the ATAPI and El Torito layer should stay
> cross-platform. The interesting bit will be making it so that
> "plugging in" a SCSI driver will require the least amount of code
> change.
>
> Anyways, I'm kind of rambling for now... Until the official "code
> start" day I'll do some more reading and investigating how it would be
> best to tie the code into GRUB2 to give the most elegant solution.
>
>
> Cheers,
> Alex
>
> On 24/04/07, Devils-Hawk <devilshawk@aon.at> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA512
>>
>> You should probably have a look at http://www.linuxbios.org/FILO  
>> maybe
>> you can salvage something. FILO supports booting from ElTorito  
>> bootable
>> CD media, uses 32-bit mode, and it is free of bios calls.  
>> Unfortunately
>> its x86 only, although some work has been done to get it running  
>> on PPC.
>>
>> regards DevH
>>
>> Alex Roman wrote:
>> > Hello list,
>> >
>> > My name is Alex Roman and I have been selected as a Summer of Code
>> > student for the GRUB2 project. I thought I'd introduce myself  
>> and my
>> > project. I am a second year student at the University of Waterloo
>> > (just finished 2A, first term of second year). I have experience
>> > developing in low-level environments in C and assembly.
>> >
>> > Now about my project. I will be adding CD-ROM booting  
>> functionality to
>> > GRUB2. The first stage of the project will attempt to use BIOS  
>> calls
>> > to achieve this functionality (which is, of course, dependent on  
>> the
>> > BIOS's capabilities, standards compliance, etc.). The second stage
>> > will involve writing a complete ATA/ATAPI driver which would bypass
>> > the BIOS completely, thus allowing it to work on virtually any  
>> system
>> > out there.
>> >
>> > Unfortunately I only have access to ATAPI CD-ROM drives, so I  
>> won't be
>> > able to work on SCSI and/or USB, however I will try to separate the
>> > system to load the boot image from the driver itself as much as
>> > possible so that future driver developments for SCSI and USB  
>> will just
>> > be sort of drop-in replacements.
>> >
>> > So, just thought I'd talk a bit about what I'll be doing...
>> >
>> > Thank you for your attention and time. I'm sure I'll have lots of
>> > questions in the months to come. :)
>> >
>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.6 (GNU/Linux)
>>
>> iD8DBQFGLmM6CbmO6P7mspMRCkGqAJ9I6u7BlaiHWjJ1T5qBrwuQ3wyy0QCbBE+Q
>> QsyJ7MVm3zNEfmA8LuVqcGE=
>> =L3H4
>> -----END PGP SIGNATURE-----
>>
>>
>> _______________________________________________
>> Grub-devel mailing list
>> Grub-devel@gnu.org
>> http://lists.gnu.org/mailman/listinfo/grub-devel
>>
>
>
> -- 
> Alex Roman <alex.roman@gmail.com>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


[-- Attachment #2: Type: text/html, Size: 14015 bytes --]

Re: My Summer of Code Project

[Vesa Jääskeläinen]

Alex Roman wrote:
> Interesting piece of code... Yet another example of showing how to
> implement an ATA/IDE controller driver :)
> 
> Right now I'm pondering whether it is worth implementing a BIOS calls
> CDROM boot support, or whether I should just go right ahead and do it
> all with the ATA controller.
> 
> I'm not sure how IDE controllers work on PPC, since I've unfortunately
> never used the architecture...
> 
> Technically, if the ATA/IDE driver is there, implementing the ATAPI
> command set to "talk" to the CDROM and interpreting the El-Torito spec
> shouldn't be that hard.
> 
> If CD-ROM drives are ATAPI on all platforms (where they use ATA), and
> the ATA code is there, the ATAPI and El Torito layer should stay
> cross-platform. The interesting bit will be making it so that
> "plugging in" a SCSI driver will require the least amount of code
> change.
> 
> Anyways, I'm kind of rambling for now... Until the official "code
> start" day I'll do some more reading and investigating how it would be
> best to tie the code into GRUB2 to give the most elegant solution.

There are couple of things to think of.

1. BIOS method seems to be somewhat unusable as if device is booted from
other medium like hard disk, first the CD-ROM device is hidden and it
cannot be accessed through normal means. I haven't tried packet
interface from BIOS but that could be one solution, but I vaguely
remember that there was some issue in that one too.

2. ATA/ATAPI would be a nice addition but it should be developed in a
way that it allows also other buses to be used later on, like SCSI, USB, ...

3. In order to boot OSes which need to load boot sectors and such there
is a need to provide El Torito emulation layer and it would then
communicate with developed CD-ROM driver. This emulation layer and
driver would then need to be installed on system and that should be used
to boot specific device. GRUB 2's CD-ROM driver and El Torito emulation
layer may use different code in order to make it compatible for El
Torito emulation.

4. Only concentrate on architecture that you know and have access. Just
try to abstract it a bit so that it first nicely in GRUB 2's software
architecture.

Doing all tasks related to CD-ROM support might not be possible in time
reserved for GSoC, but performance may vary depending on person.

Re: My Summer of Code Project

[Marco Gerards]

"Alex Roman" <alex.roman@gmail.com> writes:

Hi Alex,

Please don't top quote, it is confusing ;-).

> Interesting piece of code... Yet another example of showing how to
> implement an ATA/IDE controller driver :)

Yeah.

btw, do you know http://www.osdever.net?

If you are looking for even more information, I could have a look at
which books about this subject I can recommend.

> Right now I'm pondering whether it is worth implementing a BIOS calls
> CDROM boot support, or whether I should just go right ahead and do it
> all with the ATA controller.

Both are important!  Really...

You should know GRUB 2 is designed to be portable and runs on lots of
different systems.

When the firmware gives access to the CDROM drive, this is usually a
nice feature to have.  One thing is that you have support for it, even
if it is SCSI (which you won't implement) or something else which is
terribly obscure :-).

Another nice reason you want this is that usually the firmware tells
you which drive you booted from.  For example, when you boot from a
CDROM drive it is easy to figure out which drive to load all files
from.  Think of livecds, etc.

Actually, an IDE driver is a terrible thing to have.  But it is
required for broken systems like the PC.  Some BIOSes do not have
CDROM support.  Other don't give you access to the CDROM drive when
you do not use it for booting.

> I'm not sure how IDE controllers work on PPC, since I've unfortunately
> never used the architecture...

If you make sure you keep endianess in mind, you usually will be doing
the right thing.  Besides that, on the PPC all IO is memory mapped.
You can have a look at linux to see how it deals with IO access.

> Technically, if the ATA/IDE driver is there, implementing the ATAPI
> command set to "talk" to the CDROM and interpreting the El-Torito spec
> shouldn't be that hard.

Yeah :)

> If CD-ROM drives are ATAPI on all platforms (where they use ATA), and
> the ATA code is there, the ATAPI and El Torito layer should stay
> cross-platform. The interesting bit will be making it so that
> "plugging in" a SCSI driver will require the least amount of code
> change.

El Torito is PC only, no?

> Anyways, I'm kind of rambling for now... Until the official "code
> start" day I'll do some more reading and investigating how it would be
> best to tie the code into GRUB2 to give the most elegant solution.

One good way to start is by just fixing random things in GRUB 2.  Play
with grub-emu, make a floppy image with GRUB 2 on it and boot it from
qemu.  Setup a good development environment (talking about this is
perhaps a smart thing to do, if you lack the experience).

For some other summer of code projects (like ffmpeg), the students we
required to send in patched before they were qualified.  IOW: you
couldn't be accepted, unless you sent in patches that were good enough
to make it into SVN.  By doing this they showed they are capable
enough to work on the project.  But besides that, and that is what I
am trying to say, is that it usually helps if you poke around a bit
and play with the code, you don't even have to do something useful.
And ask lots and lots of questions :-)

--
Marco

Re: My Summer of Code Project

[Alex Roman]

On 25/04/07, Marco Gerards <mgerards@xs4all.nl> wrote:
> "Alex Roman" <alex.roman@gmail.com> writes:
>
> Hi Alex,
>
> Please don't top quote, it is confusing ;-).
GMail is doing it... Sorry :(
>
> > Interesting piece of code... Yet another example of showing how to
> > implement an ATA/IDE controller driver :)
>
> Yeah.
>
> btw, do you know http://www.osdever.net?

The site looks familiar, I think I've been there before sometime... It
looks interesting, thank you for the link!

>
> If you are looking for even more information, I could have a look at
> which books about this subject I can recommend.

IIRC someone was saying that GSoC students will get an O'Reilly Safari
books account. If that's so, hopefully books there will be useful.

>
> > Right now I'm pondering whether it is worth implementing a BIOS calls
> > CDROM boot support, or whether I should just go right ahead and do it
> > all with the ATA controller.
>
> Both are important!  Really...
>
> You should know GRUB 2 is designed to be portable and runs on lots of
> different systems.
>
> When the firmware gives access to the CDROM drive, this is usually a
> nice feature to have.  One thing is that you have support for it, even
> if it is SCSI (which you won't implement) or something else which is
> terribly obscure :-).
>
> Another nice reason you want this is that usually the firmware tells
> you which drive you booted from.  For example, when you boot from a
> CDROM drive it is easy to figure out which drive to load all files
> from.  Think of livecds, etc.
>
> Actually, an IDE driver is a terrible thing to have.  But it is
> required for broken systems like the PC.  Some BIOSes do not have
> CDROM support.  Other don't give you access to the CDROM drive when
> you do not use it for booting.

I see, yes. It makes sense why we would want a BIOS calls driver too

>
> > I'm not sure how IDE controllers work on PPC, since I've unfortunately
> > never used the architecture...
>
> If you make sure you keep endianess in mind, you usually will be doing
> the right thing.  Besides that, on the PPC all IO is memory mapped.
> You can have a look at linux to see how it deals with IO access.

I might need a bit of help as to how I should keep the endianness at
bay... Are there macros or functions currently in GRUB2 that are
architecture dependent and convert a number to the right endianness?
Are there other mechanisms?

>
> > Technically, if the ATA/IDE driver is there, implementing the ATAPI
> > command set to "talk" to the CDROM and interpreting the El-Torito spec
> > shouldn't be that hard.
>
> Yeah :)
>
> > If CD-ROM drives are ATAPI on all platforms (where they use ATA), and
> > the ATA code is there, the ATAPI and El Torito layer should stay
> > cross-platform. The interesting bit will be making it so that
> > "plugging in" a SCSI driver will require the least amount of code
> > change.
>
> El Torito is PC only, no?

From what I've read, it seems like El Torito is just a standard for
the layout of the ISO, it shows how the boot image is stored in the
iso9660 image. Wikipedia knows all :P
http://en.wikipedia.org/wiki/El_Torito_(CD-ROM_standard)

>
> > Anyways, I'm kind of rambling for now... Until the official "code
> > start" day I'll do some more reading and investigating how it would be
> > best to tie the code into GRUB2 to give the most elegant solution.
>
> One good way to start is by just fixing random things in GRUB 2.  Play
> with grub-emu, make a floppy image with GRUB 2 on it and boot it from
> qemu.  Setup a good development environment (talking about this is
> perhaps a smart thing to do, if you lack the experience).

Yes, I need to do that. Setting one up shouldn't be a problem. I've
built GRUB1 before on linux... And besides vim (don't want flame wars,
to each their own!), GNU devel tools (gcc, make, binutils, etc) and a
shell, what else do I need? :-)

>
> For some other summer of code projects (like ffmpeg), the students we
> required to send in patched before they were qualified.  IOW: you
> couldn't be accepted, unless you sent in patches that were good enough
> to make it into SVN.  By doing this they showed they are capable
> enough to work on the project.  But besides that, and that is what I
> am trying to say, is that it usually helps if you poke around a bit
> and play with the code, you don't even have to do something useful.
> And ask lots and lots of questions :-)

I should perhaps add some debugging messages to see how code is
executed, the general flow of the bootloader...

>
> --
> Marco
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>


-- 
Alex Roman <alex.roman@gmail.com>

Re: My Summer of Code Project

[Marco Gerards]

"Alex Roman" <alex.roman@gmail.com> writes:

Hi,

> On 25/04/07, Marco Gerards <mgerards@xs4all.nl> wrote:
>> "Alex Roman" <alex.roman@gmail.com> writes:
>>
>> Hi Alex,
>>
>> Please don't top quote, it is confusing ;-).
> GMail is doing it... Sorry :(

np, you seem to have fixed it :-)

[...]

>> If you are looking for even more information, I could have a look at
>> which books about this subject I can recommend.
>
> IIRC someone was saying that GSoC students will get an O'Reilly Safari
> books account. If that's so, hopefully books there will be useful.

Perhaps the O'Reilly books about the Linux kernel internals.  Other
than that, I doubt that there are many interesting books...  But who
knows. :-)

[...]

>> > I'm not sure how IDE controllers work on PPC, since I've unfortunately
>> > never used the architecture...
>>
>> If you make sure you keep endianess in mind, you usually will be doing
>> the right thing.  Besides that, on the PPC all IO is memory mapped.
>> You can have a look at linux to see how it deals with IO access.
>
> I might need a bit of help as to how I should keep the endianness at
> bay... Are there macros or functions currently in GRUB2 that are
> architecture dependent and convert a number to the right endianness?
> Are there other mechanisms?

Yeah, for example grub_be_to_cpu32 to convert a 32 bits BE integer to
the host format.  You can find all macros in include/grub/types.h.

>> > Technically, if the ATA/IDE driver is there, implementing the ATAPI
>> > command set to "talk" to the CDROM and interpreting the El-Torito spec
>> > shouldn't be that hard.
>>
>> Yeah :)
>>
>> > If CD-ROM drives are ATAPI on all platforms (where they use ATA), and
>> > the ATA code is there, the ATAPI and El Torito layer should stay
>> > cross-platform. The interesting bit will be making it so that
>> > "plugging in" a SCSI driver will require the least amount of code
>> > change.
>>
>> El Torito is PC only, no?
>
>>From what I've read, it seems like El Torito is just a standard for
> the layout of the ISO, it shows how the boot image is stored in the
> iso9660 image. Wikipedia knows all :P
> http://en.wikipedia.org/wiki/El_Torito_(CD-ROM_standard)

Right, but it is only used on the PC AFAIK.  For example, Open
Firmware (the "BIOS" on the PPC Macs) have ISO 9660 support and can
read kernels and bootloaders directly from the filesystem.

>> > Anyways, I'm kind of rambling for now... Until the official "code
>> > start" day I'll do some more reading and investigating how it would be
>> > best to tie the code into GRUB2 to give the most elegant solution.
>>
>> One good way to start is by just fixing random things in GRUB 2.  Play
>> with grub-emu, make a floppy image with GRUB 2 on it and boot it from
>> qemu.  Setup a good development environment (talking about this is
>> perhaps a smart thing to do, if you lack the experience).
>
> Yes, I need to do that. Setting one up shouldn't be a problem. I've
> built GRUB1 before on linux... And besides vim (don't want flame wars,
> to each their own!), GNU devel tools (gcc, make, binutils, etc) and a
> shell, what else do I need? :-)

For GRUB 2 you will need to install ruby as well.  What I actually
meant with a good development environment is setting up qemu or so.
Or when you want to test things on real hardware you can better use
networking instead of messing around with floppies.  Doing things
right from the start can save you time :-).

>> For some other summer of code projects (like ffmpeg), the students we
>> required to send in patched before they were qualified.  IOW: you
>> couldn't be accepted, unless you sent in patches that were good enough
>> to make it into SVN.  By doing this they showed they are capable
>> enough to work on the project.  But besides that, and that is what I
>> am trying to say, is that it usually helps if you poke around a bit
>> and play with the code, you don't even have to do something useful.
>> And ask lots and lots of questions :-)
>
> I should perhaps add some debugging messages to see how code is
> executed, the general flow of the bootloader...

Have a look at grub_dprintf.  It prints debug messages depending on a
variable.  For example, use "set debug=all" on the GRUB prompt.  I
think it is easier for you to start playing with grub-emu to
understand how things in general work.  Although it will be useless to
you when you start working on your project.

--
Marco

Re: My Summer of Code Project

[Alex Roman]

On 25/04/07, Marco Gerards <mgerards@xs4all.nl> wrote:
...
> Yeah, for example grub_be_to_cpu32 to convert a 32 bits BE integer to
> the host format.  You can find all macros in include/grub/types.h.

Cool, sounds good.

>
...
> Right, but it is only used on the PC AFAIK.  For example, Open
> Firmware (the "BIOS" on the PPC Macs) have ISO 9660 support and can
> read kernels and bootloaders directly from the filesystem.

I see..

...
> For GRUB 2 you will need to install ruby as well.  What I actually
> meant with a good development environment is setting up qemu or so.
> Or when you want to test things on real hardware you can better use
> networking instead of messing around with floppies.  Doing things
> right from the start can save you time :-).

How can I tell qemu to boot grub? last I checked qemu took a hard
drive image that it booted from.

> Have a look at grub_dprintf.  It prints debug messages depending on a
> variable.  For example, use "set debug=all" on the GRUB prompt.  I
> think it is easier for you to start playing with grub-emu to
> understand how things in general work.  Although it will be useless to
> you when you start working on your project.

What is grub-emu?

-- 
Alex Roman <alex.roman@gmail.com>

Re: My Summer of Code Project

[Yoshinori K. Okuji]

On Wednesday 25 April 2007 18:41, Alex Roman wrote:
> > For GRUB 2 you will need to install ruby as well.  What I actually
> > meant with a good development environment is setting up qemu or so.
> > Or when you want to test things on real hardware you can better use
> > networking instead of messing around with floppies.  Doing things
> > right from the start can save you time :-).
>
> How can I tell qemu to boot grub? last I checked qemu took a hard
> drive image that it booted from.

I have documented a howto on the wiki:

http://grub.enbug.org/TestingOnX86

Although the page says bochs, you can do the same with qemu. When testing, I 
myself often use qemu with a floppy image. This is quite easy. I also use a 
hard disk image from time to time, but it is a bit more painful to set up a 
correct image than a floppy image.

For real systems, I usually use PXE or a USB key.

> > Have a look at grub_dprintf.  It prints debug messages depending on a
> > variable.  For example, use "set debug=all" on the GRUB prompt.  I
> > think it is easier for you to start playing with grub-emu to
> > understand how things in general work.  Although it will be useless to
> > you when you start working on your project.
>
> What is grub-emu?

Look at the directory util. In the current version, it is built only if you 
specify --enable-grub-emu to configure.

Okuji

Re: My Summer of Code Project

[Yoshinori K. Okuji]

On Wednesday 25 April 2007 06:49, Alex Roman wrote:
> Right now I'm pondering whether it is worth implementing a BIOS calls
> CDROM boot support, or whether I should just go right ahead and do it
> all with the ATA controller.

It is definitely worth doing. When BIOS works gracefully, you do not have to 
worry about interference with BIOS functions, and it will work, regardless of 
the type of a drive (e.g. SCSI or USB). And, if my understanding is correct, 
recent BIOSes are not that buggy with this regard, so it is the best solution 
in a long term. Also this is the easiest thing to implement. Thus I specified 
this as the first task to do.

Besides that, device drivers would be useful even for some other 
architectures, potentially. So I would say that both are equally important 
for GRUB. But our priority is to motivate GRUB Legacy users to switch to GRUB 
2, so supporting PC BIOS is more urgent, IMO.

> Technically, if the ATA/IDE driver is there, implementing the ATAPI
> command set to "talk" to the CDROM and interpreting the El-Torito spec
> shouldn't be that hard.
>
> If CD-ROM drives are ATAPI on all platforms (where they use ATA), and
> the ATA code is there, the ATAPI and El Torito layer should stay
> cross-platform.

Potentially, yes. But I am not sure if it is feasible, because PC has more 
restriction. Because a drive must be accessible via BIOS calls, the code must 
be small enough to fit into somewhere below 1MB.

In addition, such code needs to be, at least partially, 16-bit code for BIOS 
calls. But I am afraid that 16-bit code support by GCC is quite poor, so you 
would have to implement the code in assembly, thus it would result in 
non-portable code.

Okuji

Re: My Summer of Code Project

[Marco Gerards]

Devils-Hawk <devilshawk@aon.at> writes:

> You should probably have a look at http://www.linuxbios.org/FILO maybe
> you can salvage something. FILO supports booting from ElTorito bootable
> CD media, uses 32-bit mode, and it is free of bios calls. Unfortunately
> its x86 only, although some work has been done to get it running on PPC.

Hah!  There is even a better project to look at ;-)

http://code.google.com/soc/coresystems/appinfo.html?csaid=A64F3E060F47221

It is not yet completed, but these two projects have a great overlap.

That said, I know that for summer of code (I am a student with a SoC
project myself, although I won't work on GRUB 2) it's about writing
code.  It is not a problem if code is duplicated, although you can
better talk about this with Jeroen and Okuji because they are the
mentors.  As maintainer I just prefer the best code and the biggest
amount of features and maintainability.

Don't be afraid to write an IDE driver yourself.  I do not think it is
*that* hard.  I estimate it takes you just a few weeks when you are
completely unexperienced.  There is a lot of documentation about this
and you are certainly not the first person working on this.

--
Marco

Re: My Summer of Code Project

[Marco Gerards]

"Alex Roman" <alex.roman@gmail.com> writes:

Hi Alex,

> My name is Alex Roman and I have been selected as a Summer of Code
> student for the GRUB2 project. I thought I'd introduce myself and my
> project. I am a second year student at the University of Waterloo
> (just finished 2A, first term of second year). I have experience
> developing in low-level environments in C and assembly.

Welcome!  If you have any questions, don't be afraid to ask on the
list and I won't mind if you ask me things about GRUB 2,
etc. off-list.

There is also a GRUB 2 IRC channel (#grub on irc.freenode.net) where
GRUB 2 developers waste some of their time :-).

> Now about my project. I will be adding CD-ROM booting functionality to
> GRUB2. The first stage of the project will attempt to use BIOS calls
> to achieve this functionality (which is, of course, dependent on the
> BIOS's capabilities, standards compliance, etc.). The second stage
> will involve writing a complete ATA/ATAPI driver which would bypass
> the BIOS completely, thus allowing it to work on virtually any system
> out there.

That sounds great!

> Unfortunately I only have access to ATAPI CD-ROM drives, so I won't be
> able to work on SCSI and/or USB, however I will try to separate the
> system to load the boot image from the driver itself as much as
> possible so that future driver developments for SCSI and USB will just
> be sort of drop-in replacements.

There is an iso9660 fs implementation already, so you only need to
provide a block driver.  After that it is easy to test if it works.

If you want to know how to write a block driver for GRUB, have a look
at loopback.c.  Or have a look at the block driver for open firmware.
Both are quite easy to understand.  But don't be afraid to ask
questions about the design of GRUB 2.

> So, just thought I'd talk a bit about what I'll be doing...

Please keep talking :-).  Really, it is a waste of time if you are
heading in the wrong direction.  Share your ideas, send patches to the
list if you want us to review your code, etc.

Feedback is one of the things that makes it great to work on Free
Software projects.  You communicate with lots of people, you learn new
stuff, it improves the quality of your code, etc.

Don't be afraid to look stupid by asking stupid questions or by
sending in patches which don't look really nice.

> Thank you for your attention and time. I'm sure I'll have lots of
> questions in the months to come. :)

:-)

--
Marco

Re: My Summer of Code Project

[adrian15]

> Hello list,
> 
> My name is Alex Roman and I have been selected as a Summer of Code
> student for the GRUB2 project. I will be adding CD-ROM booting functionality to
> GRUB2.
Welcome! Cdrom booting funcionality for grub2 is a good piece of news.
> The first stage of the project will attempt to use BIOS calls
> to achieve this functionality (which is, of course, dependent on the
> BIOS's capabilities, standards compliance, etc.).
Ok. That's how it works in Grub legacy I suppose.
 > The second stage
> will involve writing a complete ATA/ATAPI driver which would bypass
> the BIOS completely, thus allowing it to work on virtually any system
> out there.
I suppose that it is going to be difficult. I give you my encouragement!
> 
> Unfortunately I only have access to ATAPI CD-ROM drives, so I won't be
> able to work on SCSI and/or USB,
I don't have either.
 > however I will try to separate the
> system to load the boot image from the driver itself as much as
> possible so that future driver developments for SCSI and USB will just
> be sort of drop-in replacements.
> 
> So, just thought I'd talk a bit about what I'll be doing...
Comunication is a good thing.
> 
> Thank you for your attention and time. I'm sure I'll have lots of
> questions in the months to come. :)
If this happens, it's a good sign of grub2 development health.

adrian15

Re: My Summer of Code Project

[Yoshinori K. Okuji]

On Monday 23 April 2007 02:01, adrian15 wrote:
> > Hello list,
> >
> > My name is Alex Roman and I have been selected as a Summer of Code
> > student for the GRUB2 project. I will be adding CD-ROM booting
> > functionality to GRUB2.
>
> Welcome! Cdrom booting funcionality for grub2 is a good piece of news.
>
> > The first stage of the project will attempt to use BIOS calls
> > to achieve this functionality (which is, of course, dependent on the
> > BIOS's capabilities, standards compliance, etc.).
>
> Ok. That's how it works in Grub legacy I suppose.

Nope. This is a different thing. This is a new feature which is not present in 
GRUB Legacy.

Although people often mistake this, booting a CD from GRUB is quite different 
from booting a GRUB installed into a CD. GRUB Legacy has the latter feature, 
but not the former one.

GRUB 2 needs to support both cases, but I prefer to see something not present 
in GRUB Legacy implemented in GRUB 2 first.

Okuji