network support

network support

[Vincent Guffens]

Hi,

I started to integrate network support into grub2 using the code from 
etherboot 5.3.14.

Although it is still very basic, I sent it now as I saw the post 
"question" about network support, so it might be time to discuss about it.

What I've done so far is simply to add a "lspci" command as a grub 
module, which seems to work on my pc. Because it uses the pci code from 
etherboot, it should be a good starting point to start porting all their 
drivers.

I was thinking that it could be a good idea to be able to use their 
drivers with no modification at all so that future management would be 
easier. New driver in grub would also mean new drivers in etherboot and 
the other way around.

In grub legacy, there was this problem when compiling a lot of drivers 
in. How do we avoid it here ? I was thinking that it could be possible 
to use the lspci to find out the device id and initialize only the right 
driver, but maybe it is not practical ? Is it done anywhere else like 
that or do they probe the card by trying the iniliazation routine of the 
driver ?

Also, if we add a tftp command, what should we do with the downloaded 
file. Maybe it would be convenient to have some kind of ramfs to be able 
to copy the files there ?

What should I do now ? It is not practical to send a patch as they are 
many news files coming directly from etherboot. Would it be an idea to 
have a new netboot branch in the cvs so that it would be possible to 
experiment with it without breaking anything else ?





-- 
				Vincent Guffens
				PhD Student UCL/CESAME
				tel:   +32 10 47 80 30
Value your freedom, or you will lose it, teaches history.
"Don't bother us with politics," respond those who don't want to learn.
	      	-- Richard M. Stallman

Re: network support

[Tobias Wollgam]

> I started to integrate network support into grub2 using the code from
> etherboot 5.3.14.
>
> Although it is still very basic, I sent it now as I saw the post
> "question" about network support, so it might be time to discuss
> about it.
>
> What I've done so far is simply to add a "lspci" command as a grub
> module, which seems to work on my pc. Because it uses the pci code
> from etherboot, it should be a good starting point to start porting
> all their drivers.

very well, lspci, very usefull tool if one works with nics ...

> I was thinking that it could be a good idea to be able to use their
> drivers with no modification at all so that future management would
> be easier. New driver in grub would also mean new drivers in
> etherboot and the other way around.

Yes, I think not modifying the code would be a nice thing.

> In grub legacy, there was this problem when compiling a lot of
> drivers in. How do we avoid it here ? I was thinking that it could be
> possible to use the lspci to find out the device id and initialize
> only the right driver, but maybe it is not practical ? Is it done
> anywhere else like that or do they probe the card by trying the
> iniliazation routine of the driver ?
>
> Also, if we add a tftp command, what should we do with the downloaded
> file. Maybe it would be convenient to have some kind of ramfs to be
> able to copy the files there ?

I think it should be handled like in grub 1, but I do not know the 
capabilities of grub 2.

Re: network support

[Marco Gerards]

Vincent Guffens <guffens@auto.ucl.ac.be> writes:

Hi Vincent,

> I started to integrate network support into grub2 using the code from
> etherboot 5.3.14.

Oh, neat. :)

> Although it is still very basic, I sent it now as I saw the post
> "question" about network support, so it might be time to discuss about
> it.
>
> What I've done so far is simply to add a "lspci" command as a grub
> module, which seems to work on my pc. Because it uses the pci code
> from etherboot, it should be a good starting point to start porting
> all their drivers.

Cool.  My idea of PCI support in GRUB 2 is that we need to be able to
support every kind of bus using modules.  For example PCI could have
an interface so drivers/commands could query it for PCIIDs, vendor
ids and other information about the hardware, etc.

In that case every PCI driver (also on other architectures) export the
same interface.  The lspci utility could use that interface to get the
PCIIDs and look them up in a database (when available) so it can print
nice readable output.

> I was thinking that it could be a good idea to be able to use their
> drivers with no modification at all so that future management would be
> easier. New driver in grub would also mean new drivers in etherboot
> and the other way around.

How hard would that be and what disadvantages would that approach
have?

> In grub legacy, there was this problem when compiling a lot of drivers
> in. How do we avoid it here ? I was thinking that it could be possible
> to use the lspci to find out the device id and initialize only the
> right driver, but maybe it is not practical ? Is it done anywhere else
> like that or do they probe the card by trying the iniliazation routine
> of the driver ?

The problem with GRUB Legacy has is that extra drivers would mean that
GRUB gets bigger.  All drivers would, AFAIK, mean GRUB is too big.

For GRUB 2 every driver should become a module.  Perhaps we could make
a list of drivers and PCIIDs and lookup which driver to load using
that list.  Perhaps that can be a module called pcimodprobe.mod or so.

> Also, if we add a tftp command, what should we do with the downloaded
> file. Maybe it would be convenient to have some kind of ramfs to be
> able to copy the files there ?

Why?  The loader could load the file into memory, no?

> What should I do now ? It is not practical to send a patch as they are
> many news files coming directly from etherboot. Would it be an idea to
> have a new netboot branch in the cvs so that it would be possible to
> experiment with it without breaking anything else ?

Perhaps it would be better to wait until you have things working?  I
could test rtl8029, rtl8139 and some other cards I forgot about.

Okuji, what do you think of my ideas?  Are they a bit too crazy?

For me portability and flexibility is important, so other kinds of
drivers can be implemented on any architecture.

Thanks,
Marco

Re: network support

[Vincent Guffens]

Marco Gerards wrote:

[...]

>>Also, if we add a tftp command, what should we do with the downloaded
>>file. Maybe it would be convenient to have some kind of ramfs to be
>>able to copy the files there ?
> 
> 
> Why?  The loader could load the file into memory, no?

it is just that I was thinking about tftp'ing module in the filesystem 
and loading them from there or something like that but it not necessary 
now.

> 
>>What should I do now ? It is not practical to send a patch as they are
>>many news files coming directly from etherboot. Would it be an idea to
>>have a new netboot branch in the cvs so that it would be possible to
>>experiment with it without breaking anything else ?
> 
> 
> Perhaps it would be better to wait until you have things working?  I
> could test rtl8029, rtl8139 and some other cards I forgot about.

yes but I was just thinking about other people who would like to work on 
it meanwhile. As I have never done that before, it might take some time 
and I might not be able to do it at all. But, ok it is a good start to 
have this lspci working. I go back to work and see how it goes.

cheers !

-- 
				Vincent Guffens
				PhD Student UCL/CESAME
				tel:   +32 10 47 80 30
Value your freedom, or you will lose it, teaches history.
"Don't bother us with politics," respond those who don't want to learn.
	      	-- Richard M. Stallman

Re: network support

[Marco Gerards]

Vincent Guffens <guffens@auto.ucl.ac.be> writes:

> yes but I was just thinking about other people who would like to work
> on it meanwhile. As I have never done that before, it might take some
> time and I might not be able to do it at all. But, ok it is a good
> start to have this lspci working. I go back to work and see how it
> goes.

Yeah, cool.  I am looking forwards to a patch.  If you need any help
or testers, feel free to contact me over email or IRC.

--
Marco

Re: network support

[Vincent Guffens]

Hi,

I just wanted to make a small update on the network support as I've just 
  had grub2 sending bootp request. Here is the request as seen from my 
laptop directly connected to my test pc:

14:39:44.915056 0:c0:9f:16:f6:4c Broadcast ip 590: 0.0.0.0.bootpc > 
255.255.255.255.bootps:  xid:0x9f16f945 [|bootp]

and it gets its address back as can be seen from the next frame:

14:39:44.915423 0:c0:9f:16:f6:4c Broadcast ip 590: muskar.bootpc > 
255.255.255.255.bootps:  xid:0x9f16f945 [|bootp]


As for now, it compiles with as many warning as there are stars in the 
sky but it at least, it does something.

What I've done is as follows:

     *  module : grub_pci
           o provided grub command : lspci, lspci_driver
           o description : loads the core pci routine, initiliases the 
pci_device list (dev_list), export the symbols need to register new drivers
           o files: grub_pci.c, pci_io.c, i386_timer.c, timer.c


     * module : drv_*
           o provide grub command : none
           o description : driver from etherboot. Uses the 
pci_register_driver to register itself and do nothing else
           o files: depends on the driver


     * module : pcimodprobe
           o provide grub command : ifconfig
           o description : initialises the nic, probes for the config
           o files :


The drivers are nearly no modified at all. I just changed 
printf/grub_printf, added some #include and added the code to register 
themselves.


Cheers,


Marco Gerards wrote:
> Vincent Guffens <guffens@auto.ucl.ac.be> writes:
> 
> 
>>yes but I was just thinking about other people who would like to work
>>on it meanwhile. As I have never done that before, it might take some
>>time and I might not be able to do it at all. But, ok it is a good
>>start to have this lspci working. I go back to work and see how it
>>goes.
> 
> 
> Yeah, cool.  I am looking forwards to a patch.  If you need any help
> or testers, feel free to contact me over email or IRC.
> 
> --
> Marco
> 
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
> 


-- 
				Vincent Guffens
				PhD Student UCL/CESAME
				tel:   +32 10 47 80 30
Value your freedom, or you will lose it, teaches history.
"Don't bother us with politics," respond those who don't want to learn.
	      	-- Richard M. Stallman

Re: network support

[Marco Gerards]

Vincent Guffens <guffens@auto.ucl.ac.be> writes:

Hi Vincent,

> I just wanted to make a small update on the network support as I've
> just had grub2 sending bootp request. Here is the request as seen from
> my laptop directly connected to my test pc:

Cool!  nice to hear about this progress.  I am sorry that I am slow
with my replies.  Please keep hacking, I do care but I just have
little time.

> As for now, it compiles with as many warning as there are stars in the
> sky but it at least, it does something.
>
> What I've done is as follows:
>
>      *  module : grub_pci
>            o provided grub command : lspci, lspci_driver

Is it possible to split this up like I described in a previous email?

>
>      * module : drv_*
>            o provide grub command : none
>            o description : driver from etherboot. Uses the
> pci_register_driver to register itself and do nothing else
>            o files: depends on the driver

Would it be later to register other kinds of devices with
pci_register_driver or is it very specific to NICs?

>      * module : pcimodprobe
>            o provide grub command : ifconfig
>            o description : initialises the nic, probes for the config
>            o files :

Config files?

>
>
> The drivers are nearly no modified at all. I just changed
> printf/grub_printf, added some #include and added the code to register
> themselves.

Ok, cool :)

Thanks,
Marco

Re: network support

[Vincent Guffens]

Marco Gerards wrote:
> Vincent Guffens <guffens@auto.ucl.ac.be> writes:
> 
> Hi Vincent,
> 
> 
>>I just wanted to make a small update on the network support as I've
>>just had grub2 sending bootp request. Here is the request as seen from
>>my laptop directly connected to my test pc:
> 
> 
> Cool!  nice to hear about this progress.  I am sorry that I am slow
> with my replies.  Please keep hacking, I do care but I just have
> little time.
> 
> 
>>As for now, it compiles with as many warning as there are stars in the
>>sky but it at least, it does something.
>>
>>What I've done is as follows:
>>
>>     *  module : grub_pci
>>           o provided grub command : lspci, lspci_driver
> 
> 
> Is it possible to split this up like I described in a previous email?

basically, I think this is it. The lspci_driver does nothing when you 
load this module but the module exports the pci_register_driver which 
take a pci_driver structure and adds it to a linked list. Then, 
lspci_driver just goes through the list and display the driver name. 
Drivers are not yet inited.

> 
>>     * module : drv_*
>>           o provide grub command : none
>>           o description : driver from etherboot. Uses the
>>pci_register_driver to register itself and do nothing else
>>           o files: depends on the driver
> 
> 
> Would it be later to register other kinds of devices with
> pci_register_driver or is it very specific to NICs?

yes, all the etherboot drivers should just be fine after a small 
modification.

> 
>>     * module : pcimodprobe
>>           o provide grub command : ifconfig
>>           o description : initialises the nic, probes for the config
>>           o files :
> 
> 
> Config files?

Probing for the config here means looking for a dhcp server, obtaining 
an IP address, the gateway and the next-server. It does not yet look for 
the dhcp option-150 for the menu.

Now, at that point, the idea is that when you type a filename looking like

(nd0,tftp)/dir/file or (nd0)/dir/file

grub2 uses its IP address to contact the next-server with tftp to 
download the file /dir/file

For that purpose, I register a netfs file system. When the open method 
of the fs is called, the file is downladed and all the tftp blocks are 
stored in a linked list. The read method simply goes throught these 
blocks and reads a given lenght of data.

The tftp option in (nd0,tftp) is there for the future if we want to add 
some more download protocols like http or nfs as they do in etherboot.


> 
>>
>>The drivers are nearly no modified at all. I just changed
>>printf/grub_printf, added some #include and added the code to register
>>themselves.
> 
> 
> Ok, cool :)
> 
> Thanks,
> Marco
> 
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
> 


-- 
				Vincent Guffens
				PhD Student UCL/CESAME
				tel:   +32 10 47 80 30
Value your freedom, or you will lose it, teaches history.
"Don't bother us with politics," respond those who don't want to learn.
	      	-- Richard M. Stallman

Re: network support

[Marco Gerards]

Hi Vincent,

>> pci_register_driver or is it very specific to NICs?
>
> yes, all the etherboot drivers should just be fine after a small
> modification.

What does this have to do with etherboot drivers?

I was not too clear with what I meant, sorry.  GRUB 2 is very
portable.  For example there is a powerpc port already, but it will be
ported to other architectures.

It would be very nice if the PCI driver would work alike on every
architecture.  So you will have a pci bus driver like you have.  The
lspci tool is separate so it works on every arch without modification.

On some architectures we need to write our own video drivers and
perhaps even IDE drivers.  Those drivers also rely on PCI.  That's why
I asked about the support for other kinds of drivers from the PCI
interface.

Linux has a file to lookup which module should be used for which PCI
device.

(from /lib/modules/2.6.10-4-amd64-generic/modules.pcimap: )

# pci module         vendor     device     subvendor  subdevice  class      class_mask driver_data
snd-ymfpci           0x00001073 0x00000004 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-ymfpci           0x00001073 0x0000000d 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-ymfpci           0x00001073 0x0000000a 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-ymfpci           0x00001073 0x0000000c 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-ymfpci           0x00001073 0x00000010 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-ymfpci           0x00001073 0x00000012 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-vx222            0x000010b5 0x00009050 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-vx222            0x000010b5 0x00009030 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
snd-trident          0x00001023 0x00002000 0xffffffff 0xffffffff 0x00000000 0x00000000 0x0
...

We could do the same, I think.  In this case to map the PCIID, etc to
GRUB module names.  pcimodprobe could probe for modules when it is
started, load them and of course the module itself has to do the
(cheap) check again to be sure.


>>
>>>     * module : pcimodprobe
>>>           o provide grub command : ifconfig
>>>           o description : initialises the nic, probes for the config
>>>           o files :
>> Config files?
>
> Probing for the config here means looking for a dhcp server, obtaining
> an IP address, the gateway and the next-server. It does not yet look
> for the dhcp option-150 for the menu.

Okuji, I have seen a discussion about this option 150 on bug-grub.
How should this work for GRUB 2?

> Now, at that point, the idea is that when you type a filename looking like
>
> (nd0,tftp)/dir/file or (nd0)/dir/file
>
> grub2 uses its IP address to contact the next-server with tftp to
> download the file /dir/file
>
> For that purpose, I register a netfs file system. When the open method
> of the fs is called, the file is downladed and all the tftp blocks are
> stored in a linked list. The read method simply goes throught these
> blocks and reads a given lenght of data.
>
> The tftp option in (nd0,tftp) is there for the future if we want to
> add some more download protocols like http or nfs as they do in
> etherboot.

Can't you just the tftp server for the block grub_file_read wants?
This would save some memory I think.

Thanks,
Marco

Re: network support

[Yoshinori K. Okuji]

On Friday 25 March 2005 01:07 pm, Marco Gerards wrote:
> > Probing for the config here means looking for a dhcp server, obtaining
> > an IP address, the gateway and the next-server. It does not yet look
> > for the dhcp option-150 for the menu.
>
> Okuji, I have seen a discussion about this option 150 on bug-grub.
> How should this work for GRUB 2?

As Ted Lemon suggested, a vendor excapsulated option is a clean solution. And, 
we should use a vendor class identifier to request such an option. This would 
be safer especially when PXE is used. Since GRUB 2 has no compatibility 
issue, GRUB 2 may not need to use 150 at all.

But I don't decide how to do for GRUB Legacy yet. I might want to insist on 
keeping the usage of 150, simply because I am not willing to work on GRUB 
Legacy.

Okuji

Re: network support

[Yoshinori K. Okuji]

On Friday 11 March 2005 11:33 am, Vincent Guffens wrote:
> In grub legacy, there was this problem when compiling a lot of drivers
> in. How do we avoid it here ? I was thinking that it could be possible
> to use the lspci to find out the device id and initialize only the right
> driver, but maybe it is not practical ? Is it done anywhere else like
> that or do they probe the card by trying the iniliazation routine of the
> driver ?

Each driver should be one module. You can preload as many modules as you want 
with grub-mkimage. But I think you should not load all modules, because this 
might overflow the loaded memory region.

Normally, it is safe to probe PCI devices, thanks to PCI IDs. So it should be 
enough to iterate all probing routines of device drivers, to detect a device.

> Also, if we add a tftp command, what should we do with the downloaded
> file. Maybe it would be convenient to have some kind of ramfs to be able
> to copy the files there ?

It should not be a command, but a network device should be associated with a 
GRUB device (say, nd0). The interface should be consistent between disk 
devices and network devices for the user.

I don't think ramfs is useful. When you open a file, whether it is on a disk 
or a network, you read data to a buffer, don't you?

> What should I do now ? It is not practical to send a patch as they are
> many news files coming directly from etherboot. Would it be an idea to
> have a new netboot branch in the cvs so that it would be possible to
> experiment with it without breaking anything else ?

Do not hesitate to break the code. Any piece of source code has bugs, anyway.

Network support is divided into two parts: a network device framework and 
device drivers. The framework is more important at the moment, because it is 
not implemented. The current code has a template written by me, but we must 
design it seriously.

The fundamental difference between disk devices and network devices is that 
disks consist of blocks and have unique filesystems, while network devices 
consist of paths and can support multiple filesystems at a time. So, unlike 
disks, the user must specify what protocol should be used to network devices, 
and the cache system must be differently implemented.

The template code supposes that the user "bind"s a filesystem to a network 
device. I don't know if this is really good, but probably it is acceptable. 
The default filesystem should be tftp, because this is the most common for 
netbooting.

The cache system would be designed to use the set of a network device 
identifier, a filesystem idenfier, a path and a starting offset as a key.

Okuji

Re: network support

[Tobias Wollgam]

Hi!

Is there a patch available? There is still nothing in cvs relative to 
networking.

TIA,

	Tobias


-- 
Tobias Wollgam
Softwaredevelopment
Competence Center System Management
Business Unit Information 
_________________________________________________
MATERNA GmbH Information & Communications
Vosskuhle 37 * 44141 Dortmund  
http://www.materna.de 

Re: network support

[Tobias Wollgam]

> Tobias Wollgam wrote:
> > Hi!
> >
> > Is there a patch available? There is still nothing in cvs relative
> > to networking.
> >
> > TIA,
> >
> > 	Tobias
>
> There is no patch, I've been working on it on my own but I have
> nothing ready as I had other things to do meanwhile. I have grub
> sending bootp request and configuring its network interface with the
> reply. The link between the file system and tftp is being done. I can
> send you the code if you want to.

Yes, I would like to test the code or have a look at it. If it is not 
too big (around 5MB is ok) you could mail it.