NetBSD port updated

NetBSD port updated

[Christian Limpach]

Hello!

I've updated the NetBSD port:
- use pagetable pinning (requires Xen1.2 with ChangeSet 1.705 applied).
- fix PG_ZERO page not empty failure
- fix memory leak in network driver
- boot regular/unmangled kernel from NetBSD specific domain builder

NetBSD seems quite stable with these changes now...

See http://lola.pin.lu/netbsd/xen/readme-040207 for details on how to get
started...

Ian/Keir: the patches for the domain builder are at
http://lola.pin.lu/netbsd/xen/xen12-netbsd_domaintools.diff
I had to modify xc_dom_create.py (and called it xc_dom_create_nb.py for now)
because I don't ((feel like I) want to) know how to call function references
in python ;-)  If you make the builder call indirect through builder_fn,
xc_dom_create.py should work for both NetBSD and Linux.  There could
probably be some more code sharing between xc_linux_build.c and
xc_netbsd_build.c, maybe even auto-detection...

   christian



-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: NetBSD port updated

[Keir Fraser]

> Hello!
> Ian/Keir: the patches for the domain builder are at
> http://lola.pin.lu/netbsd/xen/xen12-netbsd_domaintools.diff
> I had to modify xc_dom_create.py (and called it xc_dom_create_nb.py for now)
> because I don't ((feel like I) want to) know how to call function references
> in python ;-)  If you make the builder call indirect through builder_fn,
> xc_dom_create.py should work for both NetBSD and Linux.  There could
> probably be some more code sharing between xc_linux_build.c and
> xc_netbsd_build.c, maybe even auto-detection...

Great!

I think we should try to merge netbsd_build and linux_build into a
'guest_build' that autodetects based on kernel-image format.

I'm not sure whether right now is the time to do that --- I think
there are some other updates to the 1.2 tools soon to be checked
in... Ian?

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: NetBSD port updated

[Ian Pratt]

> I think we should try to merge netbsd_build and linux_build into a
> 'guest_build' that autodetects based on kernel-image format.
> 
> I'm not sure whether right now is the time to do that --- I think
> there are some other updates to the 1.2 tools soon to be checked
> in... Ian?

I've picked up Christian's changes and just made the function
pointer stuff work.  I haven't thought about save/restore, but
its possible the Linux functions may just work. 

Some of the netbsd builder code will have to change slightly when
it moves to 1.3. This is probably the time to clean up the code
sharing and auto detect.

Cheers,
Ian



-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: NetBSD port updated

[Keir Fraser]

> 
> > I think we should try to merge netbsd_build and linux_build into a
> > 'guest_build' that autodetects based on kernel-image format.
> > 
> > I'm not sure whether right now is the time to do that --- I think
> > there are some other updates to the 1.2 tools soon to be checked
> > in... Ian?
> 
> I've picked up Christian's changes and just made the function
> pointer stuff work.  I haven't thought about save/restore, but
> its possible the Linux functions may just work. 
> 
> Some of the netbsd builder code will have to change slightly when
> it moves to 1.3. This is probably the time to clean up the code
> sharing and auto detect.

Hmmm. Okay I guess it makes sense to the merging of the two functions
as part of the forward port of all the 1.2 modifications to 1.3.

Presumably the 1.2 stuff must be just about done?

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: NetBSD port updated

[Christian Limpach]

> I think we should try to merge netbsd_build and linux_build into a
> 'guest_build' that autodetects based on kernel-image format.

yes...  but why are you not using the elf loader for Linux as well?  The
XenoGues format doesn't contain any additional information since the
0xc0000000 is in the elf header...

I think for NetBSD in domain0 I'll use the elf loader code to build an
elf->XenoGues translator...  Or would it be easy enough to add the elf
loader for domain0 as well?  I assume this is the original motivation for
the XenoGues format?

    christian



-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: NetBSD port updated

[Ian Pratt]

> > I think we should try to merge netbsd_build and linux_build into a
> > 'guest_build' that autodetects based on kernel-image format.
> 
> yes...  but why are you not using the elf loader for Linux as well?  The
> XenoGues format doesn't contain any additional information since the
> 0xc0000000 is in the elf header...

The main purpose of the 'XenoGuess' header was to be able to
detect people trying to load standard x86 kernel builds rather
than x86-xeno ones. I guess we could achieve the same effect
within the elf format e.g. by defining some special symbol the
loader looks for.

> I think for NetBSD in domain0 I'll use the elf loader code to build an
> elf->XenoGues translator...  Or would it be easy enough to add the elf
> loader for domain0 as well?  I assume this is the original motivation for
> the XenoGues format?

I think maintaining the same format for domain 0 and for domains
started from user space domains is very important. It might well
make sense to put a minimal elf loader in Xen and make that the
standard format for 1.3.  I don't think there are any issues with
Grub as it doesn't look inside a module loading.

BTW: I've had your latest netbsd build up and running with the
current 1.2 tree -- it works great!  However, it might be worth
updating the web page to reveal the root password for the minimal
root file system ;-)

> I haven't thought about save/restore, but its possible the
> Linux functions may just work.

Having actually thought for 1ns I realised that save/restore is
OS specific, so currently won't work. Also, in xc_dom_control
where we currently do the 'save' we don't know what type the
domain is.  That might be an argument for moving save to
xc_dom_create for the time being.

Of course, this will all be much neater in rev 3 of the domain
control tools that will use a db backend to maintain state about
currently running domains across a cluster...

Ian

 



-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: NetBSD port updated

[Keir Fraser]

> > > I think we should try to merge netbsd_build and linux_build into a
> > > 'guest_build' that autodetects based on kernel-image format.
> > 
> > yes...  but why are you not using the elf loader for Linux as well?  The
> > XenoGues format doesn't contain any additional information since the
> > 0xc0000000 is in the elf header...
> 
> The main purpose of the 'XenoGuess' header was to be able to
> detect people trying to load standard x86 kernel builds rather
> than x86-xeno ones. I guess we could achieve the same effect
> within the elf format e.g. by defining some special symbol the
> loader looks for.

An alternative is to have a modified Multiboot format where we embed a
very simple 'XenoGuest' header inside the Elf object file. We can then
confirm whether a given Elf file is suitable for loading by searching
for this header near the start. We can even add an enumeration, or
well-known text string, for OS type to make auto-detection easier
(assuming that some OSes will want slightly different conventions for
calling in).

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: NetBSD port updated

[Keir Fraser]

> > I think we should try to merge netbsd_build and linux_build into a
> > 'guest_build' that autodetects based on kernel-image format.
> 
> yes...  but why are you not using the elf loader for Linux as well?  The
> XenoGues format doesn't contain any additional information since the
> 0xc0000000 is in the elf header...
> 
> I think for NetBSD in domain0 I'll use the elf loader code to build an
> elf->XenoGues translator...  Or would it be easy enough to add the elf
> loader for domain0 as well?  I assume this is the original motivation for
> the XenoGues format?

Yes, but actually I think simple Elf loading of fully-linked images is
very easy so perhaps we should just do that in all cases.

Does NetBSD expect any special parameters when you jump into the OS
image? I'm wondering whether it is possible to eliminate all
differences between linux_build and netbsd_build. If not we can just
keep separate function calls, or have a 'guest_build' which accepts an
'OS type' argument.

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn