[uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[Matt Zimmerman]

I am receiving many reports from my users that 2.4.26-1um is less functional
than past versions, especially (but not exclusively) where hostfs is
concerned.  Are others experiencing problems as well?  I am considering
reverting the Debian packages to 2.4.24-1um, which was the last version
which was stable for me (it had only one frustrating hostfs bug, where
readdir() would read forever).

----- Forwarded message from Marius Gedminas <mgedmin@b4net.lt> -----

Date: Tue, 29 Jun 2004 16:46:21 +0100
From: Marius Gedminas <mgedmin@b4net.lt>
Resent-From: Marius Gedminas <mgedmin@b4net.lt>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: Bug#256873: user-mode-linux: fails to start (kernel panic)

Package: user-mode-linux
Version: 2.4.26-1um-1
Severity: important

After 2.4.26-1um-1 fell into testing (replacing 2.4.24-1um-2), my UML
instances fail to start with a kernel panic.  Rootstrap also crashes
with the same error.  Last several lines of UML output are

  [...]
  Checking that ptrace can change system call numbers...OK
  openpty failed to allocate a pty
  openpty failed to allocate a pty
  POSIX conformance testing by UNIFIX
  process 553 exited with status 0
  Kernel panic: wait_for_stop failed to wait for 553 to stop with 19

  In idle task - not syncing
   <6>SysRq : Show Regs

  Kernel panic: Segfault with no mm
  In idle task - not syncing
   <6>SysRq : Show Regs

  EIP: 0023:[<a015f2e5>] CPU: 0 Not tainted ESP: 002b:a121feb4 EFLAGS: 00010202
      Not tainted
  EAX: 00000000 EBX: a02adee0 ECX: 00000001 EDX: a121c274
  ESI: 00000070 EDI: 00000007 EBP: a121fecc DS: 002b ES: 002b
  Call Trace: [<a016ecd0>] [<a015589f>] [<a0023b2d>] [<a0014c08>] [<a0154b0b>]
     [<a0264a67>] [<a0201c44>] [<a0152503>] [<a0154f65>] [<a015f2e5>] [<a0159241>]
     [<a0155062>] [<a0201b58>] [<a015f2e5>] [<a015f2c7>] [<a0263345>] [<a016ed6a>]
     [<a016ecd0>] [<a015589f>] [<a0023b2d>] [<a0014c08>] [<a0155e6f>] [<a02814c0>]
     [<a014f3cb>] [<a014f360>] [<a021a49a>]

The process ID (553) seems to vary, but the other numbers stay constant,
at a glance.

The contents of my rootstrap.conf, minus comments and empty lines:

  [global]
  fstype=ext3
  initialsize=512
  freespace=0
  modules=network mkfs mount debian uml umount
  PATH=/bin:/sbin:/usr/bin:/usr/sbin
  [network]
  hostname=schooltool-uml
  transport=tuntap
  interface=eth0
  host=192.168.200.1
  uml=192.168.200.100
  netmask=255.255.255.0
  gateway=192.168.200.1
  domain=localdomain
  nameserver=192.168.200.1
  [debian]
  dist=woody
  mirror=http://ftp.uk.debian.org/debian
  exclude=pcmcia-cs setserial
  purge=base-config console-common console-tools console-data
  console-tools-libs
  [base-config]
  root_password=rootstrap
  [tasksel]
  [uml]
  install_modules=yes

Host kernel is kernel-image-2.4.18-1-686, version 2.4.18-13.1.
Rootstrap is version 0.3.20-1.

Downgrading to user-mode-linux version 2.4.24-1um-2 fixes the problem.

I tried rootstrap on a different machine, also with user-mode-linux
2.4.26-1um-1, and it failed with a slightly different error:

-- System Information:
Debian Release: 3.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Kernel: Linux 2.4.18-1-686
Locale: LANG=C, LC_CTYPE=C

Versions of packages user-mode-linux depends on:
ii  uml-utilities                 20040406-1 User-mode Linux (utility programs)

----- End forwarded message -----

-- 
 - mdz


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[BlaisorBlade]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]

Alle 20:27, martedì 29 giugno 2004, Matt Zimmerman ha scritto:
> I am receiving many reports from my users that 2.4.26-1um is less
> functional than past versions, especially (but not exclusively) where
> hostfs is concerned.  Are others experiencing problems as well?  I am
> considering reverting the Debian packages to 2.4.24-1um, which was the last
> version which was stable for me (it had only one frustrating hostfs bug,
> where readdir() would read forever).

I definitely agree. There are some bugfix from 2.4.24-2 to add there, but they 
can wait for now (they are far less fatal). Also, the attached patch is 
needed as forward-port to 2.4.26: use vanilla 2.4.26+ 2.4.24-1um + the 
attached fixlet. Don't downgrade to the security-bugged 2.4.24/.25.

Bye
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729

[-- Attachment #2: uml-2.4.26-cmpxchg.patch --]
[-- Type: text/x-diff, Size: 1049 bytes --]

--- linux-2.4.26/include/asm-um/system-i386.h	Wed Apr 21 15:33:45 2004
+++ linux-2.4.26-umlfix/include/asm-um/system-i386.h	Wed Apr 21 15:27:14 2004
@@ -5,35 +5,4 @@
 
 #define __HAVE_ARCH_CMPXCHG 1
 
-static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old,
-				      unsigned long new, int size)
-{
-	unsigned long prev;
-	switch (size) {
-	case 1:
-		__asm__ __volatile__(LOCK_PREFIX "cmpxchgb %b1,%2"
-				     : "=a"(prev)
-				     : "q"(new), "m"(*__xg(ptr)), "0"(old)
-				     : "memory");
-		return prev;
-	case 2:
-		__asm__ __volatile__(LOCK_PREFIX "cmpxchgw %w1,%2"
-				     : "=a"(prev)
-				     : "q"(new), "m"(*__xg(ptr)), "0"(old)
-				     : "memory");
-		return prev;
-	case 4:
-		__asm__ __volatile__(LOCK_PREFIX "cmpxchgl %1,%2"
-				     : "=a"(prev)
-				     : "q"(new), "m"(*__xg(ptr)), "0"(old)
-				     : "memory");
-		return prev;
-	}
-	return old;
-}
-
-#define cmpxchg(ptr,o,n)\
-	((__typeof__(*(ptr)))__cmpxchg((ptr),(unsigned long)(o),\
-					(unsigned long)(n),sizeof(*(ptr))))
-    
 #endif

Re: [uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[Stefan Gybas]

[-- Attachment #1: Type: text/plain, Size: 1372 bytes --]

Matt Zimmerman wrote:

> I am receiving many reports from my users that 2.4.26-1um is less functional
> than past versions, especially (but not exclusively) where hostfs is
> concerned.  Are others experiencing problems as well?  I am considering
> reverting the Debian packages to 2.4.24-1um, which was the last version
> which was stable for me (it had only one frustrating hostfs bug, where
> readdir() would read forever).

That's also my experince: I'm currently running Linux 2.4.26 with UML 
patch 2.4.24-1 on a production system (kernel 2.6 with SKAS patch on the 
host) without major problems.

When I used UML patch 2.4.24-2 or higher the guest systems were very 
unstable and crashed after a few hours. The linux processes on the host 
were simply killed by a segmentation fault or I got one of these error 
messages:

Kernel panic: Segfault with no mm
Kernel panic: switch_mm_skas - PTRACE_SWITCH_MM failed, errno = 0

I even tried to build a UML kernel on woody (where the AIO glibc headers 
are not available, so the kernel is built with gcc 2.95 and without AIO 
support) but the resulting kernel also was not very stable.

So I suggest to use kernel 2.4.26 (or higher, when available) with UML 
patch 2.4.24-1 for the user-mode-linux Debian package (which is also 
used by some commercial hosting services like Linode). BTW, thanks for 
packaging!

Stefan

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: [uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[James Neal]

Matt Zimmerman wrote:

>I am receiving many reports from my users that 2.4.26-1um is less functional
>than past versions, especially (but not exclusively) where hostfs is
>concerned.  Are others experiencing problems as well?  I am considering
>reverting the Debian packages to 2.4.24-1um, which was the last version
>which was stable for me (it had only one frustrating hostfs bug, where
>readdir() would read forever).
>  
>
Yeah, I upgraded about a hundred (non-critical) UMLs to a straight 
2.4.26-1um kernel about a two weeks ago, and immediately started getting 
2-3 crashes (like the one below) per hour.   When backups kicked off, 
they pretty much all went down at the same time.  (Unfortunately, I 
hadn't added automatic backtracing to the UMLazi codebase yet, so I 
didn't get backtraces.)

I ended up going with a 2.4.26 kernel, with the 2.4.24-1um UML patch, 
and http://vs165141.v.sectoor.de/uml-2.4.26-cmpxchg.patch . This 
combination is running over 600 UMLs for 8 days without a crash.

-James

>----- Forwarded message from Marius Gedminas <mgedmin@b4net.lt> -----
>
>  [...]
>  Checking that ptrace can change system call numbers...OK
>  openpty failed to allocate a pty
>  openpty failed to allocate a pty
>  POSIX conformance testing by UNIFIX
>  process 553 exited with status 0
>  Kernel panic: wait_for_stop failed to wait for 553 to stop with 19
>
>  In idle task - not syncing
>   <6>SysRq : Show Regs
>
>  Kernel panic: Segfault with no mm
>  In idle task - not syncing
>   <6>SysRq : Show Regs
>
>  EIP: 0023:[<a015f2e5>] CPU: 0 Not tainted ESP: 002b:a121feb4 EFLAGS: 00010202
>      Not tainted
>  EAX: 00000000 EBX: a02adee0 ECX: 00000001 EDX: a121c274
>  ESI: 00000070 EDI: 00000007 EBP: a121fecc DS: 002b ES: 002b
>  Call Trace: [<a016ecd0>] [<a015589f>] [<a0023b2d>] [<a0014c08>] [<a0154b0b>]
>     [<a0264a67>] [<a0201c44>] [<a0152503>] [<a0154f65>] [<a015f2e5>] [<a0159241>]
>     [<a0155062>] [<a0201b58>] [<a015f2e5>] [<a015f2c7>] [<a0263345>] [<a016ed6a>]
>     [<a016ecd0>] [<a015589f>] [<a0023b2d>] [<a0014c08>] [<a0155e6f>] [<a02814c0>]
>     [<a014f3cb>] [<a014f360>] [<a021a49a>]
>
>The process ID (553) seems to vary, but the other numbers stay constant,
>at a glance.
>
>----- End forwarded message -----
>
>  
>



-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[Paul Wagland]

[-- Attachment #1: Type: text/plain, Size: 1314 bytes --]


On Jun 30, 2004, at 1:55, James Neal wrote:

> Matt Zimmerman wrote:
>
>> I am receiving many reports from my users that 2.4.26-1um is less 
>> functional
>> than past versions, especially (but not exclusively) where hostfs is
>> concerned.  Are others experiencing problems as well?  I am 
>> considering
>> reverting the Debian packages to 2.4.24-1um, which was the last 
>> version
>> which was stable for me (it had only one frustrating hostfs bug, where
>> readdir() would read forever).
>>
> Yeah, I upgraded about a hundred (non-critical) UMLs to a straight 
> 2.4.26-1um kernel about a two weeks ago, and immediately started 
> getting 2-3 crashes (like the one below) per hour.   When backups 
> kicked off, they pretty much all went down at the same time.  
> (Unfortunately, I hadn't added automatic backtracing to the UMLazi 
> codebase yet, so I didn't get backtraces.)

I normally hate doing this... but me too!

Even worse, it would appear that UML and reiserfs don't play nice... 
when the UML does crash I tend to end up with some pretty massive 
filesystem corruptions, to the point where my UML will not boot 
anymore, and I spent over an hour just now trying to coax it back to 
life by copying in and replacing all of the corrupted binaries.

This is using UBD, and not hostfs.

Cheers,
Paul

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 186 bytes --]

Re: [uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[BlaisorBlade]

Alle 09:56, mercoledì 30 giugno 2004, Paul Wagland ha scritto:
> On Jun 30, 2004, at 1:55, James Neal wrote:

> I normally hate doing this... but me too!

> Even worse, it would appear that UML and reiserfs don't play nice...
> when the UML does crash I tend to end up with some pretty massive
> filesystem corruptions, to the point where my UML will not boot
> anymore, and I spent over an hour just now trying to coax it back to
> life by copying in and replacing all of the corrupted binaries.
If you didn't enable the synchronous UBD access (near to the UBD options in 
the kernel configuration), that's almost expected, as the help explains 
(IIRC), for journaled filesystems (although some people claim ext3 does not 
need it, and this could be true, even because reiserfs works like ext3 
"data=writeback" mode, which is less safe than the default ext3 
"data=ordered").

In either case, give more info, so we know if this is a "no-bug" or if we 
should investigate it.
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729




-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[Paul Wagland]

On Wed 30 Jun 2004 20:38:41 CEST, BlaisorBlade wrote:

> Alle 09:56, mercoled? 30 giugno 2004, Paul Wagland ha scritto:
>> On Jun 30, 2004, at 1:55, James Neal wrote:
>
>> I normally hate doing this... but me too!
>
>> Even worse, it would appear that UML and reiserfs don't play nice...
>> when the UML does crash I tend to end up with some pretty massive
>> filesystem corruptions, to the point where my UML will not boot
>> anymore, and I spent over an hour just now trying to coax it back to
>> life by copying in and replacing all of the corrupted binaries.
> If you didn't enable the synchronous UBD access (near to the UBD options in
> the kernel configuration), that's almost expected, as the help explains
> (IIRC), for journaled filesystems (although some people claim ext3 does not
> need it, and this could be true, even because reiserfs works like ext3
> "data=writeback" mode, which is less safe than the default ext3
> "data=ordered").

Ah. Ok, I assume by this you mean that I should be doing ubd1s=blah instead of
ubd1=blah. I will try that, it looks like it should at least stop the fs
screwage. On a different sidenote, what is the "recommended" filesystem for
UML?

> In either case, give more info, so we know if this is a "no-bug" or if we
> should investigate it.

If I get another crash, I will send through the dump that comes through, other
than that i am afraid i cannot help at the moment.

Cheers,
Paul


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] [mgedmin@b4net.lt: Bug#256873: user-mode-linux: fails to start (kernel panic)]

[BlaisorBlade]

Alle 23:02, giovedì 1 luglio 2004, Paul Wagland ha scritto:
> On Wed 30 Jun 2004 20:38:41 CEST, BlaisorBlade wrote:
> > Alle 09:56, mercoled? 30 giugno 2004, Paul Wagland ha scritto:
> >> On Jun 30, 2004, at 1:55, James Neal wrote:
> >>
> >> I normally hate doing this... but me too!
> >>
> >> Even worse, it would appear that UML and reiserfs don't play nice...
> >> when the UML does crash I tend to end up with some pretty massive
> >> filesystem corruptions, to the point where my UML will not boot
> >> anymore, and I spent over an hour just now trying to coax it back to
> >> life by copying in and replacing all of the corrupted binaries.
> >
> > If you didn't enable the synchronous UBD access (near to the UBD options
> > in the kernel configuration), that's almost expected, as the help
> > explains (IIRC), for journaled filesystems (although some people claim
> > ext3 does not need it, and this could be true, even because reiserfs
> > works like ext3 "data=writeback" mode, which is less safe than the
> > default ext3
> > "data=ordered").
>
> Ah. Ok, I assume by this you mean that I should be doing ubd1s=blah instead
> of ubd1=blah. I will try that, it looks like it should at least stop the fs
> screwage. On a different sidenote, what is the "recommended" filesystem for
> UML?

Well, I speak about CONFIG_BLK_DEV_UBD_SYNC (Always use synchronous access for 
UBD), but maybe what you suggest is the same thing (the kernel option would 
turn "s(ync)" on for all UBDs, maybe).

I've no idea of a "recommended" filesystem; I only know that ext3 *seems* not 
to require that UBD are accessed in a synchronous way. (I.e. somebody says he 
never experienced data loss with ext3 with async UBDs).

Bye
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729



-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel