2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[roland]

hi !
concerning the uml "hang" after  "NET: Registered protocol family 2":

after some investigation i found, that the problem seems to have been introduced with
uml-patch-2.6.0-1 (at http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-1.bz2)

uml-patch-2.6.0-test9-1 (at http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-test9-1.bz2?download),
in conjunction with minor compile-fix http://people.0x63.nu/~andersg/uml-minor-fixes.diff works nicely for me.

seems there has been a major change (lots of merges?) in uml-patch-2.6.0-1 because it`s much more bigger (around 2x) than
uml-patch-2.6.0-test9-1

btw:
i use Bryan O'Sullivan`s skas patch on my host kernel. see: http://sourceforge.net/mailarchive/message.php?msg_id=5840691
since this patch isn`t based on steven james patch (which stephen williams patch is based of), it isn`t a skas issue, but a compiler
or host-fs issue !? i`m on suse9 host.

any clues?

regards
roland





----- Original Message ----- 
From: "roland" <for_spam@gmx.de>
To: "Phil Nadeau" <phil@innercite.com>; <user-mode-linux-devel@lists.sourceforge.net>
Sent: Saturday, February 14, 2004 10:45 PM
Subject: Re: [uml-devel] Kernel idles in loop after loading IPv4


> hi !
> i have the same problem.
> my uml is 2.6.3-rc2 with jeff`s latest uml patch.
> host is 2.6.0-test11 with skas patch
> with 2.6.0-test11 + uml patch i didn`t have such problem.
> any hints?
>
> regards
> roland
>
>
> ----- Original Message ----- 
> From: "Phil Nadeau" <phil@innercite.com>
> To: <user-mode-linux-devel@lists.sourceforge.net>
> Sent: Tuesday, January 27, 2004 9:54 PM
> Subject: [uml-devel] Kernel idles in loop after loading IPv4
>
>
> > Hello UML developers,
> >
> > I'm trying to use UML to set up a simulator for a high-availability
> > cluster. I encounter the following unexpected behavior -
> >
> > On any kernel that I compile with IPv4 support, the kernel will stop
> > intializing drivers after loading IPv4 support. The last message I see
> > from the kernel is:
> > NET: Registered protocol family 2
> >
> > Compiling with debug allows me to single-step through the kernel. I've
> > traced execution from inet_init(), to synchronize_net(), to
> > synchronize_kernel(), to a call to wait_for_completion(), to
> > switch_to_tt(), after which the kernel spends all its time in the idle
> > loop. This happens whether I'm using SKAS or TT mode.
> >
> > Host kernel is 2.6.1 with Stephen William's 2.6.1 SKAS patch (thanks
> > Stephen). Guest kernel is 2.6.1 with the latest patch from Sourceforge
> > (January 15th), without modules, and with nearly everything disabled at
> > compile except core functionality and IPv4.
> >
> > I've tried to find similar cases in the FAQ's and the list archives, but
> > I'm not seeing anything. It seems like the kernel is waiting for an
> > event related to IPv4 initialization that it never gets, but I'm not
> > familiar enough with the kernel proper to know what that might be. Does
> > anyone have any ideas?
> >
> > Thanks in advance.
> >
> > -- 
> > Phil Nadeau
> > Software Services Manager
> > phil.nadeau@innercite.com
> > (916)932-3200, (800)921-5513
> >
> >
> >
> > -------------------------------------------------------
> > The SF.Net email is sponsored by EclipseCon 2004
> > Premiere Conference on Open Tools Development and Integration
> > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> > http://www.eclipsecon.org/osdn
> > _______________________________________________
> > User-mode-linux-devel mailing list
> > User-mode-linux-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
> >



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[BlaisorBlade]

Alle 18:47, domenica 15 febbraio 2004, roland ha scritto:
> hi !
> concerning the uml "hang" after  "NET: Registered protocol family 2":
>
> after some investigation i found, that the problem seems to have been
> introduced with uml-patch-2.6.0-1 (at
> http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-1.bz2)
>
> uml-patch-2.6.0-test9-1 (at
> http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-test9-1.
>bz2?download), in conjunction with minor compile-fix
> http://people.0x63.nu/~andersg/uml-minor-fixes.diff works nicely for me.

On what guest kernel do you apply the patch? uml-patch-2.6.0-1 gave strange 
problems with 2.6.1 vanilla kernel.

> seems there has been a major change (lots of merges?) in uml-patch-2.6.0-1
> because it`s much more bigger (around 2x) than uml-patch-2.6.0-test9-1
>
> btw:
> i use Bryan O'Sullivan`s skas patch on my host kernel. see:
> http://sourceforge.net/mailarchive/message.php?msg_id=5840691 since this
> patch isn`t based on steven james patch (which stephen williams patch is
> based of), it isn`t a skas issue, but a compiler or host-fs issue !? i`m on
> suse9 host.
You should try using, as skas patch, 
http://www.user-mode-linux.org/~blaisorblade/patches/2.6.2-v1-host-skas2/00-2.6.2-v1-host-SKAS2.patch.bz2, 
which comes from the latest one from Ingo Molnar. I have not been very 
careful, but when I gave a look to the Bryan's patch, it seemed likely to 
have similar problems as the other one. At least, both do only this change to 
write_ldt (i.e. don't use current->mm but let the caller specify which mm 
use):

-static int write_ldt(void __user * ptr, unsigned long bytecount, int oldmode)
+static int write_ldt(struct task_struct *task, void __user * ptr, unsigned 
long bytecount, int oldmode)
 {
-       struct mm_struct * mm = current->mm;
+       struct mm_struct * mm = task->mm;
        __u32 entry_1, entry_2, *lp;
        int error;
        struct user_desc ldt_info;

While Ingo Molnar's one does also this:

@@ -195,7 +197,7 @@

        down(&mm->context.sem);
        if (ldt_info.entry_number >= mm->context.size) {
-               error = alloc_ldt(&current->mm->context, 
ldt_info.entry_number+1, 1);
+               error = alloc_ldt(&mm->context, ldt_info.entry_number+1, 1);
                if (error < 0)
                        goto out_unlock;
        }

This second change is *needed*, whichever patch do you use: without it part of 
write_ldt uses current->mm instead of mm. That also means that alloc_ldt is 
not called on mm->context, so that probably it remains null and so we get 
Oopses.

Bye
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[roland]

hi!

> > after some investigation i found, that the problem seems to have been
> > introduced with uml-patch-2.6.0-1 (at
> > http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-1.bz2)
> >
> > uml-patch-2.6.0-test9-1 (at
> > http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-test9-1.
> >bz2?download), in conjunction with minor compile-fix
> > http://people.0x63.nu/~andersg/uml-minor-fixes.diff works nicely for me.
> 
> On what guest kernel do you apply the patch? uml-patch-2.6.0-1 gave strange 
> problems with 2.6.1 vanilla kernel.

i applied the patches in the way they are named:
uml-patch-2.6.0-1 to 2.6.0 vanilla kernel and uml-patch-2.6.9-test9-1 to vanilla 2.6.9-test9

> You should try using, as skas patch, 
> http://www.user-mode-linux.org/~blaisorblade/patches/2.6.2-v1-host-skas2/00-2.6.2-v1-host-SKAS2.patch.bz2, 
> which comes from the latest one from Ingo Molnar. I have not been very 
ok - i try this as soon i have some time for this. i will be very busy the next week, so don`t expect an answer too soon.

thanks & regards
roland




-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[roland]

hi !

> You should try using, as skas patch,
> http://www.user-mode-linux.org/~blaisorblade/patches/2.6.2-v1-host-skas2/00-2.6.2-v1-host-SKAS2.patch.bz2,
> which comes from the latest one from Ingo Molnar. I have not been very

the bug ("hang after NET: Registered protocol family 2":) doesn`t seem to have a relation to SKAS, because it also happens in TT
mode.
when i run the binary with "./linux mode=tt" the hang occurs, too.
furthermore - it doesn`t seem to be a compiler issue, because i compiled 2.6.3-rc2 on 2 completely different systems (suse9 and
rh7.2).
i also read about guesses, that this could have to do with "newer" hosts running NPTL.
could someone explain? i thought uml is a static binary, so how can this have a relation to a thread >LIBRARY<??
somewhere i read about "kernel threads". is ntpl related to the kernel itself? (sorry if this sounds stupid - but i remember
switching between different thread-libraries on solaris via envvars. is this just the same with ntpl?)

if someone wants to test - the rh7.2 compiled (and for me also failing) binary is at: http://www.my-vserver.de/uml-2.6.3-rc2.gz

update:
after adding a lot if printk`s to the uml-patched 2.6.3-rc2 source, i found the following(maybe interesting):

in ./kernel/sched.c
in function wait_for_completion(struct completion *x)
the call to "schedule()" never returns and this seems to be the reason for the hanging ipv4 initialisation.
i have no clue about kernel internals and i think it`s nonsense, if i dig further into this....

but as i already said:
2.6.0-test9 with uml-patch-2.6.0-test9-1  _WORKS_ (at least for me)
2.6.0 with uml-patch-2.6.0-1 and following kernel/uml-patch versions seem to have the ipv4 hang-at-init bug.

so maybe the bug has been introduced with uml-patch-2.6.0-1 !?
regards
roland



----- Original Message ----- 
From: "BlaisorBlade" <blaisorblade_spam@yahoo.it>
To: <user-mode-linux-devel@lists.sourceforge.net>
Cc: "roland" <for_spam@gmx.de>
Sent: Sunday, February 15, 2004 8:45 PM
Subject: Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4


> Alle 18:47, domenica 15 febbraio 2004, roland ha scritto:
> > hi !
> > concerning the uml "hang" after  "NET: Registered protocol family 2":
> >
> > after some investigation i found, that the problem seems to have been
> > introduced with uml-patch-2.6.0-1 (at
> > http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-1.bz2)
> >
> > uml-patch-2.6.0-test9-1 (at
> > http://prdownloads.sourceforge.net/user-mode-linux/uml-patch-2.6.0-test9-1.
> >bz2?download), in conjunction with minor compile-fix
> > http://people.0x63.nu/~andersg/uml-minor-fixes.diff works nicely for me.
>
> On what guest kernel do you apply the patch? uml-patch-2.6.0-1 gave strange
> problems with 2.6.1 vanilla kernel.
>
> > seems there has been a major change (lots of merges?) in uml-patch-2.6.0-1
> > because it`s much more bigger (around 2x) than uml-patch-2.6.0-test9-1
> >
> > btw:
> > i use Bryan O'Sullivan`s skas patch on my host kernel. see:
> > http://sourceforge.net/mailarchive/message.php?msg_id=5840691 since this
> > patch isn`t based on steven james patch (which stephen williams patch is
> > based of), it isn`t a skas issue, but a compiler or host-fs issue !? i`m on
> > suse9 host.
> You should try using, as skas patch,
> http://www.user-mode-linux.org/~blaisorblade/patches/2.6.2-v1-host-skas2/00-2.6.2-v1-host-SKAS2.patch.bz2,
> which comes from the latest one from Ingo Molnar. I have not been very
> careful, but when I gave a look to the Bryan's patch, it seemed likely to
> have similar problems as the other one. At least, both do only this change to
> write_ldt (i.e. don't use current->mm but let the caller specify which mm
> use):
>
> -static int write_ldt(void __user * ptr, unsigned long bytecount, int oldmode)
> +static int write_ldt(struct task_struct *task, void __user * ptr, unsigned
> long bytecount, int oldmode)
>  {
> -       struct mm_struct * mm = current->mm;
> +       struct mm_struct * mm = task->mm;
>         __u32 entry_1, entry_2, *lp;
>         int error;
>         struct user_desc ldt_info;
>
> While Ingo Molnar's one does also this:
>
> @@ -195,7 +197,7 @@
>
>         down(&mm->context.sem);
>         if (ldt_info.entry_number >= mm->context.size) {
> -               error = alloc_ldt(&current->mm->context,
> ldt_info.entry_number+1, 1);
> +               error = alloc_ldt(&mm->context, ldt_info.entry_number+1, 1);
>                 if (error < 0)
>                         goto out_unlock;
>         }
>
> This second change is *needed*, whichever patch do you use: without it part of
> write_ldt uses current->mm instead of mm. That also means that alloc_ldt is
> not called on mm->context, so that probably it remains null and so we get
> Oopses.
>
> Bye
> -- 
> Paolo Giarrusso, aka Blaisorblade
> Linux registered user n. 292729
>



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[BlaisorBlade]

[-- Attachment #1: Type: text/plain, Size: 2264 bytes --]

Alle 02:52, mercoledì 18 febbraio 2004, roland ha scritto:
> hi !

For Jeff: see at the end why I cc: to you.

>
> the bug ("hang after NET: Registered protocol family 2":) doesn`t seem to
> have a relation to SKAS, because it also happens in TT mode.
> when i run the binary with "./linux mode=tt" the hang occurs, too.
> furthermore - it doesn`t seem to be a compiler issue, because i compiled
> 2.6.3-rc2 on 2 completely different systems (suse9 and rh7.2).
> i also read about guesses, that this could have to do with "newer" hosts
> running NPTL. could someone explain? i thought uml is a static binary

Yes but when it's compiled it's linked against host's libs (mainly glibc and 
libpthreads). However, IIRC, the problems with NPTL on the host have been 
fixed time ago, and with NPTL in the guest have currently a temporary fix.

>, so
> how can this have a relation to a thread >LIBRARY<?? somewhere i read about
> "kernel threads". is ntpl related to the kernel itself? (sorry if this
> sounds stupid - but i remember switching between different thread-libraries
> on solaris via envvars. is this just the same with ntpl?)

> after adding a lot if printk`s to the uml-patched 2.6.3-rc2 source, i found
> the following(maybe interesting):
>
> in ./kernel/sched.c
> in function wait_for_completion(struct completion *x)
> the call to "schedule()" never returns and this seems to be the reason for
> the hanging ipv4 initialisation. i have no clue about kernel internals and
> i think it`s nonsense, if i dig further into this....
>
> but as i already said:
> 2.6.0-test9 with uml-patch-2.6.0-test9-1  _WORKS_ (at least for me)
> 2.6.0 with uml-patch-2.6.0-1 and following kernel/uml-patch versions seem
> to have the ipv4 hang-at-init bug.
>
> so maybe the bug has been introduced with uml-patch-2.6.0-1 !?

The message about "diagnosed and repeatable kernel panic" is IMHO related to 
this: the sigsetjmp change was actually in 2.6.0. And this broke schedule: on 
my system it happens to broke in one point, on your in another point.

If I guessed right, the attached patch will solve everything.

Jeff, do you think this makes sense?
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729

[-- Attachment #2: Panic_schedule_fix.patch --]
[-- Type: text/x-diff, Size: 328 bytes --]

--- 1.9/arch/um/kernel/skas/process_kern.c	Thu Jan  8 07:43:01 2004
+++ edited/arch/um/kernel/skas/process_kern.c	Sun Feb 15 10:42:31 2004
@@ -76,6 +76,7 @@
 
 void new_thread_proc(void *stack, void (*handler)(int sig))
 {
+       local_irq_disable();
 	init_new_thread_stack(stack, handler);
 	os_usr1_process(os_getpid());
 }

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[Jeff Dike]

blaisorblade_spam@yahoo.it said:
> If I guessed right, the attached patch will solve everything.
> Jeff, do you think this makes sense? 

Yes, it makes all kinds of sense, and looks very similar to the fix I posted
a couple of days ago.

Nice debugging, though.  This was somewhat subtle.

				Jeff



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[roland]

hi !
>If I guessed right, the attached patch will solve everything.
yep - maybe - but it doesn`t seem to fix the "hang after NET: Registered protocol family 2" !
i applied the patch and recompiled - same effect.

regards
roland



----- Original Message ----- 
From: "BlaisorBlade" <blaisorblade_spam@yahoo.it>
To: <user-mode-linux-devel@lists.sourceforge.net>
Cc: "Jeff Dike" <jdike@addtoit.com>; "roland" <for_spam@gmx.de>
Sent: Wednesday, February 18, 2004 8:01 PM
Subject: Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4


Alle 02:52, mercoledì 18 febbraio 2004, roland ha scritto:
> hi !

For Jeff: see at the end why I cc: to you.

>
> the bug ("hang after NET: Registered protocol family 2":) doesn`t seem to
> have a relation to SKAS, because it also happens in TT mode.
> when i run the binary with "./linux mode=tt" the hang occurs, too.
> furthermore - it doesn`t seem to be a compiler issue, because i compiled
> 2.6.3-rc2 on 2 completely different systems (suse9 and rh7.2).
> i also read about guesses, that this could have to do with "newer" hosts
> running NPTL. could someone explain? i thought uml is a static binary

Yes but when it's compiled it's linked against host's libs (mainly glibc and
libpthreads). However, IIRC, the problems with NPTL on the host have been
fixed time ago, and with NPTL in the guest have currently a temporary fix.

>, so
> how can this have a relation to a thread >LIBRARY<?? somewhere i read about
> "kernel threads". is ntpl related to the kernel itself? (sorry if this
> sounds stupid - but i remember switching between different thread-libraries
> on solaris via envvars. is this just the same with ntpl?)

> after adding a lot if printk`s to the uml-patched 2.6.3-rc2 source, i found
> the following(maybe interesting):
>
> in ./kernel/sched.c
> in function wait_for_completion(struct completion *x)
> the call to "schedule()" never returns and this seems to be the reason for
> the hanging ipv4 initialisation. i have no clue about kernel internals and
> i think it`s nonsense, if i dig further into this....
>
> but as i already said:
> 2.6.0-test9 with uml-patch-2.6.0-test9-1  _WORKS_ (at least for me)
> 2.6.0 with uml-patch-2.6.0-1 and following kernel/uml-patch versions seem
> to have the ipv4 hang-at-init bug.
>
> so maybe the bug has been introduced with uml-patch-2.6.0-1 !?

The message about "diagnosed and repeatable kernel panic" is IMHO related to
this: the sigsetjmp change was actually in 2.6.0. And this broke schedule: on
my system it happens to broke in one point, on your in another point.

If I guessed right, the attached patch will solve everything.

Jeff, do you think this makes sense?
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[Jeff Dike]

for_spam@gmx.de said:
> if someone wants to test - the rh7.2 compiled (and for me also
> failing) binary is at: http://www.my-vserver.de/uml-2.6.3-rc2.gz 

As other people have noticed, this is host-dependent somehow.  This image
boots fine on my RH8 laptop.

> in ./kernel/sched.c in function wait_for_completion(struct completion
> *x) 

I got similar information from Phil Nadeau, so it's good to confirm that.

Has anyone seen this with a 2.4 UML?  That would be handy because this broke
in 2.6 when I moved a major set of patches to my 2.6 tree.  2.4 is much more
granular, so knowing which 2.4 patch broke it would narrow it down a lot more.
				Jeff



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: 2.6.3-rc2 failure - was: Re: [uml-devel] Kernel idles in loop after loading IPv4

[BlaisorBlade]

Alle 01:36, giovedì 19 febbraio 2004, roland ha scritto:
> hi !
>
> >If I guessed right, the attached patch will solve everything.
>
> yep - maybe - but it doesn`t seem to fix the "hang after NET: Registered
> protocol family 2" ! i applied the patch and recompiled - same effect.

Sorry - your bug is probably related to mine one, in fact the patch does not 
solve neither one. When I wrote that I didn't yet finished compiling the new 
kernel.
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id\x1356&alloc_id438&opÌk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel