* Null pointer deference
@ 2004-02-10 1:25 stevegt
2004-02-10 4:13 ` paging request failures under load (was: Re: Null pointer deference) stevegt
2004-02-10 8:18 ` Null pointer deference Ian Pratt
0 siblings, 2 replies; 21+ messages in thread
From: stevegt @ 2004-02-10 1:25 UTC (permalink / raw)
To: xen-devel
Hi All,
I seem to be able to reproduce a null pointer dereference and paging
request errors in 1.2. Can anyone give me any pointers on tracking down
what is causing it?
This is with a 32Mb virtual domain, running debian woody, NFS root,
256Mb swap in a local VD, while running a process which builds openldap,
python2.2.3, and related packages. I'm not sure which package, if any
in particular, is causing this; could be just anything that causes a
similar workload. This particular set of messages appeared before the
virtual domain locked up during the openldap build...
Steve
DOM26: Unable to handle kernel paging request at virtual address
20000001
DOM26: printing eip:
DOM26: c0007743
DOM26: *pde=00000000(00000000)
DOM26: Oops: 0000
DOM26: CPU: 0
DOM26: EIP: 0819:[<c0007743>] Not tainted
DOM26: EFLAGS: 00010202
DOM26: eax: 00000001 ebx: 20000001 ecx: c0a79e6c edx: c0a79e6c
DOM26: esi: c0a78000 edi: c0114254 ebp: c1e5f580 esp: c0a79ce4
DOM26: ds: 0821 es: 0821 ss: 0821
DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
DOM26: Stack: 20000001 c0a78000 c002c728 c0a78000 c0a79db0 c0114254
ffffffb0 c0a78000
DOM26: c003e789 c0a79e6c c014b5ac c003e314 c0a79e6c 00000000
c0114250 c0a79de8
DOM26: c1419640 80000000 00000000 00000000 00000000 00000000
00000000 00000000
DOM26: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
[<c002cf4a>]
DOM26: [<c002cf61>] [<c0090033>] [<c00914bf>]
DOM26:
DOM26: <1>Unable to handle kernel paging request at virtual address
20000001
DOM26: printing eip:
DOM26: c000af0f
DOM26: *pde=00000000(00000000)
DOM26: Oops: 0002
DOM26: CPU: 0
DOM26: EIP: 0819:[<c000af0f>] Not tainted
DOM26: EFLAGS: 00010282
DOM26: eax: 20000001 ebx: c1ed5b20 ecx: c0a78264 edx: c0a78264
DOM26: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c0a79bb4
DOM26: ds: 0821 es: 0821 ss: 0821
DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
DOM26: Stack: c1ed5b20 00000000 c0a78000 0000000b 0000000b c000b55f
20000001 0000001f
DOM26: 00000000 c140d6c0 20000001 c0091a87 0000000b 00000000
c1ed5b3c c0096305
DOM26: c0129928 c0a79cb0 00000000 c0a78000 00000000 20000001
c1e5f580 00000000
DOM26: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
[<c0018a25>]
DOM26: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
[<c0091768>] [<c0007743>]
DOM26: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
[<c002cf4a>] [<c002cf61>]
DOM26: [<c0090033>] [<c00914bf>]
DOM26:
DOM26: <1>Unable to handle kernel NULL pointer dereference at virtual
address 00000001
DOM26: printing eip:
DOM26: c000b623
DOM26: *pde=00000000(00000000)
DOM26: Oops: 0002
DOM26: CPU: 0
DOM26: EIP: 0819:[<c000b623>] Not tainted
DOM26: EFLAGS: 00010202
DOM26: eax: 00000000 ebx: 00000001 ecx: c0a78264 edx: c0a78264
DOM26: esi: 00000002 edi: c0a78000 ebp: 0000000b esp: c0a79aa0
DOM26: ds: 0821 es: 0821 ss: 0821
DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
DOM26: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
00000000 00000002
DOM26: c0096305 c0129928 c0a79b80 00000002 c0a78000 00000002
20000001 0000000b
DOM26: 63303039 c101fc58 c0a78000 00000002 c101fc58 ffffffff
00030001 c001e621
DOM26: Call Trace: [<c0091a87>] [<c0096305>] [<c001e621>] [<c001f6c0>]
[<c0008996>]
DOM26: [<c00200d7>] [<c00204e1>] [<c001464d>] [<c0014c92>]
[<c0091768>] [<c000af0f>]
DOM26: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
[<c0018a25>] [<c0018c46>]
DOM26: [<c0018feb>] [<c0018ed4>] [<c006e759>] [<c0091768>]
[<c0007743>] [<c002c728>]
DOM26: [<c003e789>] [<c003e314>] [<c002ccc7>] [<c002cf4a>]
[<c002cf61>] [<c0090033>]
DOM26: [<c00914bf>]
DOM26:
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* paging request failures under load (was: Re: Null pointer deference)
2004-02-10 1:25 Null pointer deference stevegt
@ 2004-02-10 4:13 ` stevegt
2004-02-10 6:34 ` stevegt
2004-02-10 8:18 ` Null pointer deference Ian Pratt
1 sibling, 1 reply; 21+ messages in thread
From: stevegt @ 2004-02-10 4:13 UTC (permalink / raw)
To: xen-devel
Okay, the problem still exists when I bump the memory up to 256Mb, and
never swap. I.E. I've found no workaround. Hasn't anyone else hit
anything like this?
Steve
DOM3: xen_console_init
DOM3: Linux version 2.4.24-xeno (stevegt@pathfinder) (gcc version
3.0.4) #16 Mon Feb 2 17:46:41 PST 2004
DOM3: On node 0 totalpages: 65536
DOM3: zone(0): 4096 pages.
DOM3: zone(1): 61440 pages.
DOM3: zone(2): 0 pages.
DOM3: Kernel command line:
ip=64.71.149.20:10.27.2.50:64.71.149.1:255.255.255.0::eth0:off
root=/dev/nfs nfsroot=/export//xen/fs/stevegt/tcx/root 4 DOMID=20
DOM3: Initializing CPU#0
DOM3: Xen reported: 398.780 MHz processor.
DOM3: Calibrating delay loop... 1592.52 BogoMIPS
DOM3: Memory: 257132k/262144k available (1078k kernel code, 5012k
reserved, 308k data, 52k init, 0k highmem)
DOM3: Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
DOM3: Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
DOM3: Mount cache hash table entries: 512 (order: 0, 4096 bytes)
DOM3: Buffer cache hash table entries: 16384 (order: 4, 65536 bytes)
DOM3: Page-cache hash table entries: 65536 (order: 6, 262144 bytes)
DOM3: CPU: L1 I cache: 16K, L1 D cache: 16K
DOM3: CPU: L2 cache: 512K
DOM3: CPU: Intel Pentium II (Deschutes) stepping 01
DOM3: POSIX conformance testing by UNIFIX
DOM3: Linux NET4.0 for Linux 2.4
DOM3: Based upon Swansea University Computer Society NET3.039
DOM3: Initializing RT netlink socket
DOM3: Starting kswapd
DOM3: Journalled Block Device driver loaded
DOM3: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
DOM3: Xeno console successfully installed
DOM3: Starting Xeno Balloon driver
DOM3: pty: 256 Unix98 ptys configured
DOM3: RAMDISK driver initialized: 16 RAM disks of 4096K size 1024
blocksize
DOM3: loop: loaded (max 8 devices)
DOM3: NET4: Linux TCP/IP 1.0 for NET4.0
DOM3: IP Protocols: ICMP, UDP, TCP
DOM3: IP: routing cache hash table of 2048 buckets, 16Kbytes
DOM3: TCP: Hash tables configured (established 16384 bind 16384)
DOM3: IP-Config: Complete:
DOM3: device=eth0, addr=64.71.149.20, mask=255.255.255.0,
gw=64.71.149.1,
DOM3: host=64.71.149.20, domain=, nis-domain=(none),
DOM3: bootserver=10.27.2.50, rootserver=10.27.2.50, rootpath=
DOM3: ip_conntrack version 2.1 (2048 buckets, 16384 max) - 292 bytes
per conntrack
DOM3: ip_tables: (C) 2000-2002 Netfilter core team
DOM3: NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
DOM3: Looking up port of RPC 100003/2 on 10.27.2.50
DOM3: Looking up port of RPC 100005/1 on 10.27.2.50
DOM3: VFS: Mounted root (nfs filesystem).
DOM3: Freeing unused kernel memory: 52k freed
DOM3: INIT: version 2.84 booting
DOM3: Activating swap.
DOM3: Adding Swap: 262136k swap-space (priority -1)
DOM3: Checking root file system...
DOM3: fsck 1.27 (8-Mar-2002)
DOM3: 10.27.2.50:/export/xen/fs/stevegt/tcx: NFS file system.
DOM3: System time was Tue Feb 10 02:14:36 UTC 2004.
DOM3: Setting the System Clock using the Hardware Clock as
reference...
DOM3: modprobe: modprobe: Can't locate module char-major-10-135
DOM3: modprobe: modprobe: Can't locate module char-major-4
DOM3: hwclock is unable to get I/O port access: the iopl(3) call
failed.
DOM3: modprobe: modprobe: Can't locate module char-major-10-135
DOM3: modprobe: modprobe: Can't locate module char-major-4
DOM3: System Clock set. System local time is now Tue Feb 10 02:14:36
UTC 2004.
DOM3: Calculating module dependencies... depmod: cannot read ELF
header from /lib/modules/2.4.24-xeno/modules.dep
DOM3: depmod: cannot read ELF header from
/lib/modules/2.4.24-xeno/modules.generic_string
DOM3: depmod: /lib/modules/2.4.24-xeno/modules.ieee1394map is not an
ELF file
DOM3: depmod: /lib/modules/2.4.24-xeno/modules.isapnpmap is not an ELF
file
DOM3: depmod: cannot read ELF header from
/lib/modules/2.4.24-xeno/modules.parportmap
DOM3: depmod: /lib/modules/2.4.24-xeno/modules.pcimap is not an ELF
file
DOM3: depmod: cannot read ELF header from
/lib/modules/2.4.24-xeno/modules.pnpbiosmap
DOM3: depmod: /lib/modules/2.4.24-xeno/modules.usbmap is not an ELF
file
DOM3: done.
DOM3: Loading modules:
DOM3: Checking all file systems...
DOM3: fsck 1.27 (8-Mar-2002)
DOM3: Setting kernel variables.
DOM3: Loading the saved-state of the serial devices...
DOM3: Mounting local filesystems...
DOM3: nothing was mounted
DOM3: Running 0dns-down to make sure resolv.conf is ok...done.
DOM3: Cleaning: /etc/network/ifstate.
DOM3: Setting up IP spoofing protection: rp_filter.
DOM3: Configuring network interfaces: done.
DOM3: Mounting remote filesystems...
DOM3:
DOM3: Setting the System Clock using the Hardware Clock as
reference...
DOM3: System Clock set. Local time: Tue Feb 10 02:14:37 UTC 2004
DOM3:
DOM3: Cleaning: /tmp /var/lock /var/run.
DOM3: Initializing random number generator... done.
DOM3: Recovering nvi editor sessions... done.
DOM3: INIT: Entering runlevel: 4
DOM3: Starting system log daemon: syslogd.
DOM3: Starting kernel log daemon: klogd.
DOM3: Starting internet superserver: inetd.
DOM3: Starting PCMCIA services: module directory
/lib/modules/2.4.24-xeno/pcmcia not found.
DOM3: Starting OpenBSD Secure Shell server: sshd.
DOM3: Starting deferred execution scheduler: atd.
DOM3: Starting periodic command scheduler: cron.
DOM3: INIT: no more processes left in this runlevel
DOM3: Unable to handle kernel paging request at virtual address
20000001
DOM3: printing eip:
DOM3: c0007743
DOM3: *pde=00000000(00000000)
DOM3: Oops: 0000
DOM3: CPU: 0
DOM3: EIP: 0819:[<c0007743>] Not tainted
DOM3: EFLAGS: 00010202
DOM3: eax: 00000001 ebx: 20000001 ecx: c3ebde6c edx: c3ebde6c
DOM3: esi: c3ebc000 edi: c0114254 ebp: c46c1060 esp: c3ebdce4
DOM3: ds: 0821 es: 0821 ss: 0821
DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
DOM3: Stack: 20000001 c3ebc000 c002c728 c3ebc000 c3ebddb0 c0114254
ffffffb0 c3ebc000
DOM3: c003e789 c3ebde6c c014b5ac c003e314 c3ebde6c 00000000
c0114250 00000000
DOM3: 00000000 00000000 01082003 8d588810 00000000 00000000
00000000 00000000
DOM3: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
[<c002cf4a>]
DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
DOM3:
DOM3: <1>Unable to handle kernel paging request at virtual address
20000001
DOM3: printing eip:
DOM3: c000af0f
DOM3: *pde=00000000(00000000)
DOM3: Oops: 0002
DOM3: CPU: 0
DOM3: EIP: 0819:[<c000af0f>] Not tainted
DOM3: EFLAGS: 00010282
DOM3: eax: 20000001 ebx: c485c0a0 ecx: c3ebc264 edx: c3ebc264
DOM3: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c3ebdbb4
DOM3: ds: 0821 es: 0821 ss: 0821
DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
DOM3: Stack: c485c0a0 00000000 c3ebc000 0000000b 0000000b c000b55f
20000001 0000001f
DOM3: 00000000 cf4227e0 20000001 c0091a87 0000000b 00000000
c485c0bc c0096305
DOM3: c0129928 c3ebdcb0 00000000 c3ebc000 00000000 20000001
c46c1060 00000000
DOM3: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
[<c0018a25>]
DOM3: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
[<c0091768>] [<c0007743>]
DOM3: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
[<c002cf4a>] [<c002cf61>]
DOM3: [<c0090033>] [<c00914bf>]
DOM3:
DOM3: <1>Unable to handle kernel NULL pointer dereference at virtual
address 00000001
DOM3: printing eip:
DOM3: c000b623
DOM3: *pde=00000000(00000000)
DOM3: Oops: 0002
DOM3: CPU: 0
DOM3: EIP: 0819:[<c000b623>] Not tainted
DOM3: EFLAGS: 00010202
DOM3: eax: 00000000 ebx: 00000001 ecx: c3ebc264 edx: c3ebc264
DOM3: esi: 00000002 edi: c3ebc000 ebp: 0000000b esp: c3ebdaa0
DOM3: ds: 0821 es: 0821 ss: 0821
DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
DOM3: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
00000000 00000002
DOM3: c0096305 c0129928 c3ebdb80 00000002 c3ebc000 00000002
20000001 0000000b
DOM3: 63303039 ffffffff c3ebc000 00000002 38383130 38643538
00030001 64303030
DOM3: Call Trace: [<c0091a87>] [<c0096305>] [<c0008996>] [<c000f797>]
[<c000f991>]
DOM3: [<c0091768>] [<c000af0f>] [<c000b55f>] [<c0091a87>]
[<c0096305>] [<c002eb19>]
DOM3: [<c0018a25>] [<c0018c46>] [<c0018feb>] [<c0018ed4>]
[<c006e759>] [<c0091768>]
DOM3: [<c0007743>] [<c002c728>] [<c003e789>] [<c003e314>]
[<c002ccc7>] [<c002cf4a>]
DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
DOM3:
On Mon, Feb 09, 2004 at 05:25:00PM -0800, wrote:
> Hi All,
>
> I seem to be able to reproduce a null pointer dereference and paging
> request errors in 1.2. Can anyone give me any pointers on tracking down
> what is causing it?
>
> This is with a 32Mb virtual domain, running debian woody, NFS root,
> 256Mb swap in a local VD, while running a process which builds openldap,
> python2.2.3, and related packages. I'm not sure which package, if any
> in particular, is causing this; could be just anything that causes a
> similar workload. This particular set of messages appeared before the
> virtual domain locked up during the openldap build...
>
> Steve
>
>
> DOM26: Unable to handle kernel paging request at virtual address
> 20000001
> DOM26: printing eip:
> DOM26: c0007743
> DOM26: *pde=00000000(00000000)
> DOM26: Oops: 0000
> DOM26: CPU: 0
> DOM26: EIP: 0819:[<c0007743>] Not tainted
> DOM26: EFLAGS: 00010202
> DOM26: eax: 00000001 ebx: 20000001 ecx: c0a79e6c edx: c0a79e6c
> DOM26: esi: c0a78000 edi: c0114254 ebp: c1e5f580 esp: c0a79ce4
> DOM26: ds: 0821 es: 0821 ss: 0821
> DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> DOM26: Stack: 20000001 c0a78000 c002c728 c0a78000 c0a79db0 c0114254
> ffffffb0 c0a78000
> DOM26: c003e789 c0a79e6c c014b5ac c003e314 c0a79e6c 00000000
> c0114250 c0a79de8
> DOM26: c1419640 80000000 00000000 00000000 00000000 00000000
> 00000000 00000000
> DOM26: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> [<c002cf4a>]
> DOM26: [<c002cf61>] [<c0090033>] [<c00914bf>]
> DOM26:
> DOM26: <1>Unable to handle kernel paging request at virtual address
> 20000001
> DOM26: printing eip:
> DOM26: c000af0f
> DOM26: *pde=00000000(00000000)
> DOM26: Oops: 0002
> DOM26: CPU: 0
> DOM26: EIP: 0819:[<c000af0f>] Not tainted
> DOM26: EFLAGS: 00010282
> DOM26: eax: 20000001 ebx: c1ed5b20 ecx: c0a78264 edx: c0a78264
> DOM26: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c0a79bb4
> DOM26: ds: 0821 es: 0821 ss: 0821
> DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> DOM26: Stack: c1ed5b20 00000000 c0a78000 0000000b 0000000b c000b55f
> 20000001 0000001f
> DOM26: 00000000 c140d6c0 20000001 c0091a87 0000000b 00000000
> c1ed5b3c c0096305
> DOM26: c0129928 c0a79cb0 00000000 c0a78000 00000000 20000001
> c1e5f580 00000000
> DOM26: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> [<c0018a25>]
> DOM26: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> [<c0091768>] [<c0007743>]
> DOM26: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> [<c002cf4a>] [<c002cf61>]
> DOM26: [<c0090033>] [<c00914bf>]
> DOM26:
> DOM26: <1>Unable to handle kernel NULL pointer dereference at virtual
> address 00000001
> DOM26: printing eip:
> DOM26: c000b623
> DOM26: *pde=00000000(00000000)
> DOM26: Oops: 0002
> DOM26: CPU: 0
> DOM26: EIP: 0819:[<c000b623>] Not tainted
> DOM26: EFLAGS: 00010202
> DOM26: eax: 00000000 ebx: 00000001 ecx: c0a78264 edx: c0a78264
> DOM26: esi: 00000002 edi: c0a78000 ebp: 0000000b esp: c0a79aa0
> DOM26: ds: 0821 es: 0821 ss: 0821
> DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> DOM26: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> 00000000 00000002
> DOM26: c0096305 c0129928 c0a79b80 00000002 c0a78000 00000002
> 20000001 0000000b
> DOM26: 63303039 c101fc58 c0a78000 00000002 c101fc58 ffffffff
> 00030001 c001e621
> DOM26: Call Trace: [<c0091a87>] [<c0096305>] [<c001e621>] [<c001f6c0>]
> [<c0008996>]
> DOM26: [<c00200d7>] [<c00204e1>] [<c001464d>] [<c0014c92>]
> [<c0091768>] [<c000af0f>]
> DOM26: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> [<c0018a25>] [<c0018c46>]
> DOM26: [<c0018feb>] [<c0018ed4>] [<c006e759>] [<c0091768>]
> [<c0007743>] [<c002c728>]
> DOM26: [<c003e789>] [<c003e314>] [<c002ccc7>] [<c002cf4a>]
> [<c002cf61>] [<c0090033>]
> DOM26: [<c00914bf>]
> DOM26:
>
>
>
> --
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org
> http://www.stevegt.com -- http://Infrastructures.Org
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: paging request failures under load (was: Re: Null pointer deference)
2004-02-10 4:13 ` paging request failures under load (was: Re: Null pointer deference) stevegt
@ 2004-02-10 6:34 ` stevegt
2004-02-11 4:12 ` 0-order allocation failed stevegt
2004-02-11 21:23 ` paging request failures under load (was: Re: Null pointer deference) stevegt
0 siblings, 2 replies; 21+ messages in thread
From: stevegt @ 2004-02-10 6:34 UTC (permalink / raw)
To: xen-devel
Before anyone burns too much time on this, hang on -- I wasn't able to
duplicate the problem on another cluster node (both nodes were built
from the same SystemImager image). I'm looking for the reason why, and
will let you know as soon as I do.
Steve
On Mon, Feb 09, 2004 at 08:13:22PM -0800, wrote:
> Okay, the problem still exists when I bump the memory up to 256Mb, and
> never swap. I.E. I've found no workaround. Hasn't anyone else hit
> anything like this?
>
> Steve
>
>
> DOM3: xen_console_init
> DOM3: Linux version 2.4.24-xeno (stevegt@pathfinder) (gcc version
> 3.0.4) #16 Mon Feb 2 17:46:41 PST 2004
> DOM3: On node 0 totalpages: 65536
> DOM3: zone(0): 4096 pages.
> DOM3: zone(1): 61440 pages.
> DOM3: zone(2): 0 pages.
> DOM3: Kernel command line:
> ip=64.71.149.20:10.27.2.50:64.71.149.1:255.255.255.0::eth0:off
> root=/dev/nfs nfsroot=/export//xen/fs/stevegt/tcx/root 4 DOMID=20
> DOM3: Initializing CPU#0
> DOM3: Xen reported: 398.780 MHz processor.
> DOM3: Calibrating delay loop... 1592.52 BogoMIPS
> DOM3: Memory: 257132k/262144k available (1078k kernel code, 5012k
> reserved, 308k data, 52k init, 0k highmem)
> DOM3: Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
> DOM3: Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
> DOM3: Mount cache hash table entries: 512 (order: 0, 4096 bytes)
> DOM3: Buffer cache hash table entries: 16384 (order: 4, 65536 bytes)
> DOM3: Page-cache hash table entries: 65536 (order: 6, 262144 bytes)
> DOM3: CPU: L1 I cache: 16K, L1 D cache: 16K
> DOM3: CPU: L2 cache: 512K
> DOM3: CPU: Intel Pentium II (Deschutes) stepping 01
> DOM3: POSIX conformance testing by UNIFIX
> DOM3: Linux NET4.0 for Linux 2.4
> DOM3: Based upon Swansea University Computer Society NET3.039
> DOM3: Initializing RT netlink socket
> DOM3: Starting kswapd
> DOM3: Journalled Block Device driver loaded
> DOM3: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
> DOM3: Xeno console successfully installed
> DOM3: Starting Xeno Balloon driver
> DOM3: pty: 256 Unix98 ptys configured
> DOM3: RAMDISK driver initialized: 16 RAM disks of 4096K size 1024
> blocksize
> DOM3: loop: loaded (max 8 devices)
> DOM3: NET4: Linux TCP/IP 1.0 for NET4.0
> DOM3: IP Protocols: ICMP, UDP, TCP
> DOM3: IP: routing cache hash table of 2048 buckets, 16Kbytes
> DOM3: TCP: Hash tables configured (established 16384 bind 16384)
> DOM3: IP-Config: Complete:
> DOM3: device=eth0, addr=64.71.149.20, mask=255.255.255.0,
> gw=64.71.149.1,
> DOM3: host=64.71.149.20, domain=, nis-domain=(none),
> DOM3: bootserver=10.27.2.50, rootserver=10.27.2.50, rootpath=
> DOM3: ip_conntrack version 2.1 (2048 buckets, 16384 max) - 292 bytes
> per conntrack
> DOM3: ip_tables: (C) 2000-2002 Netfilter core team
> DOM3: NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> DOM3: Looking up port of RPC 100003/2 on 10.27.2.50
> DOM3: Looking up port of RPC 100005/1 on 10.27.2.50
> DOM3: VFS: Mounted root (nfs filesystem).
> DOM3: Freeing unused kernel memory: 52k freed
> DOM3: INIT: version 2.84 booting
> DOM3: Activating swap.
> DOM3: Adding Swap: 262136k swap-space (priority -1)
> DOM3: Checking root file system...
> DOM3: fsck 1.27 (8-Mar-2002)
> DOM3: 10.27.2.50:/export/xen/fs/stevegt/tcx: NFS file system.
> DOM3: System time was Tue Feb 10 02:14:36 UTC 2004.
> DOM3: Setting the System Clock using the Hardware Clock as
> reference...
> DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> DOM3: modprobe: modprobe: Can't locate module char-major-4
> DOM3: hwclock is unable to get I/O port access: the iopl(3) call
> failed.
> DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> DOM3: modprobe: modprobe: Can't locate module char-major-4
> DOM3: System Clock set. System local time is now Tue Feb 10 02:14:36
> UTC 2004.
> DOM3: Calculating module dependencies... depmod: cannot read ELF
> header from /lib/modules/2.4.24-xeno/modules.dep
> DOM3: depmod: cannot read ELF header from
> /lib/modules/2.4.24-xeno/modules.generic_string
> DOM3: depmod: /lib/modules/2.4.24-xeno/modules.ieee1394map is not an
> ELF file
> DOM3: depmod: /lib/modules/2.4.24-xeno/modules.isapnpmap is not an ELF
> file
> DOM3: depmod: cannot read ELF header from
> /lib/modules/2.4.24-xeno/modules.parportmap
> DOM3: depmod: /lib/modules/2.4.24-xeno/modules.pcimap is not an ELF
> file
> DOM3: depmod: cannot read ELF header from
> /lib/modules/2.4.24-xeno/modules.pnpbiosmap
> DOM3: depmod: /lib/modules/2.4.24-xeno/modules.usbmap is not an ELF
> file
> DOM3: done.
> DOM3: Loading modules:
> DOM3: Checking all file systems...
> DOM3: fsck 1.27 (8-Mar-2002)
> DOM3: Setting kernel variables.
> DOM3: Loading the saved-state of the serial devices...
> DOM3: Mounting local filesystems...
> DOM3: nothing was mounted
> DOM3: Running 0dns-down to make sure resolv.conf is ok...done.
> DOM3: Cleaning: /etc/network/ifstate.
> DOM3: Setting up IP spoofing protection: rp_filter.
> DOM3: Configuring network interfaces: done.
> DOM3: Mounting remote filesystems...
> DOM3:
> DOM3: Setting the System Clock using the Hardware Clock as
> reference...
> DOM3: System Clock set. Local time: Tue Feb 10 02:14:37 UTC 2004
> DOM3:
> DOM3: Cleaning: /tmp /var/lock /var/run.
> DOM3: Initializing random number generator... done.
> DOM3: Recovering nvi editor sessions... done.
> DOM3: INIT: Entering runlevel: 4
> DOM3: Starting system log daemon: syslogd.
> DOM3: Starting kernel log daemon: klogd.
> DOM3: Starting internet superserver: inetd.
> DOM3: Starting PCMCIA services: module directory
> /lib/modules/2.4.24-xeno/pcmcia not found.
> DOM3: Starting OpenBSD Secure Shell server: sshd.
> DOM3: Starting deferred execution scheduler: atd.
> DOM3: Starting periodic command scheduler: cron.
> DOM3: INIT: no more processes left in this runlevel
> DOM3: Unable to handle kernel paging request at virtual address
> 20000001
> DOM3: printing eip:
> DOM3: c0007743
> DOM3: *pde=00000000(00000000)
> DOM3: Oops: 0000
> DOM3: CPU: 0
> DOM3: EIP: 0819:[<c0007743>] Not tainted
> DOM3: EFLAGS: 00010202
> DOM3: eax: 00000001 ebx: 20000001 ecx: c3ebde6c edx: c3ebde6c
> DOM3: esi: c3ebc000 edi: c0114254 ebp: c46c1060 esp: c3ebdce4
> DOM3: ds: 0821 es: 0821 ss: 0821
> DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> DOM3: Stack: 20000001 c3ebc000 c002c728 c3ebc000 c3ebddb0 c0114254
> ffffffb0 c3ebc000
> DOM3: c003e789 c3ebde6c c014b5ac c003e314 c3ebde6c 00000000
> c0114250 00000000
> DOM3: 00000000 00000000 01082003 8d588810 00000000 00000000
> 00000000 00000000
> DOM3: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> [<c002cf4a>]
> DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> DOM3:
> DOM3: <1>Unable to handle kernel paging request at virtual address
> 20000001
> DOM3: printing eip:
> DOM3: c000af0f
> DOM3: *pde=00000000(00000000)
> DOM3: Oops: 0002
> DOM3: CPU: 0
> DOM3: EIP: 0819:[<c000af0f>] Not tainted
> DOM3: EFLAGS: 00010282
> DOM3: eax: 20000001 ebx: c485c0a0 ecx: c3ebc264 edx: c3ebc264
> DOM3: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c3ebdbb4
> DOM3: ds: 0821 es: 0821 ss: 0821
> DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> DOM3: Stack: c485c0a0 00000000 c3ebc000 0000000b 0000000b c000b55f
> 20000001 0000001f
> DOM3: 00000000 cf4227e0 20000001 c0091a87 0000000b 00000000
> c485c0bc c0096305
> DOM3: c0129928 c3ebdcb0 00000000 c3ebc000 00000000 20000001
> c46c1060 00000000
> DOM3: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> [<c0018a25>]
> DOM3: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> [<c0091768>] [<c0007743>]
> DOM3: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> [<c002cf4a>] [<c002cf61>]
> DOM3: [<c0090033>] [<c00914bf>]
> DOM3:
> DOM3: <1>Unable to handle kernel NULL pointer dereference at virtual
> address 00000001
> DOM3: printing eip:
> DOM3: c000b623
> DOM3: *pde=00000000(00000000)
> DOM3: Oops: 0002
> DOM3: CPU: 0
> DOM3: EIP: 0819:[<c000b623>] Not tainted
> DOM3: EFLAGS: 00010202
> DOM3: eax: 00000000 ebx: 00000001 ecx: c3ebc264 edx: c3ebc264
> DOM3: esi: 00000002 edi: c3ebc000 ebp: 0000000b esp: c3ebdaa0
> DOM3: ds: 0821 es: 0821 ss: 0821
> DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> DOM3: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> 00000000 00000002
> DOM3: c0096305 c0129928 c3ebdb80 00000002 c3ebc000 00000002
> 20000001 0000000b
> DOM3: 63303039 ffffffff c3ebc000 00000002 38383130 38643538
> 00030001 64303030
> DOM3: Call Trace: [<c0091a87>] [<c0096305>] [<c0008996>] [<c000f797>]
> [<c000f991>]
> DOM3: [<c0091768>] [<c000af0f>] [<c000b55f>] [<c0091a87>]
> [<c0096305>] [<c002eb19>]
> DOM3: [<c0018a25>] [<c0018c46>] [<c0018feb>] [<c0018ed4>]
> [<c006e759>] [<c0091768>]
> DOM3: [<c0007743>] [<c002c728>] [<c003e789>] [<c003e314>]
> [<c002ccc7>] [<c002cf4a>]
> DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> DOM3:
>
>
>
>
> On Mon, Feb 09, 2004 at 05:25:00PM -0800, wrote:
> > Hi All,
> >
> > I seem to be able to reproduce a null pointer dereference and paging
> > request errors in 1.2. Can anyone give me any pointers on tracking down
> > what is causing it?
> >
> > This is with a 32Mb virtual domain, running debian woody, NFS root,
> > 256Mb swap in a local VD, while running a process which builds openldap,
> > python2.2.3, and related packages. I'm not sure which package, if any
> > in particular, is causing this; could be just anything that causes a
> > similar workload. This particular set of messages appeared before the
> > virtual domain locked up during the openldap build...
> >
> > Steve
> >
> >
> > DOM26: Unable to handle kernel paging request at virtual address
> > 20000001
> > DOM26: printing eip:
> > DOM26: c0007743
> > DOM26: *pde=00000000(00000000)
> > DOM26: Oops: 0000
> > DOM26: CPU: 0
> > DOM26: EIP: 0819:[<c0007743>] Not tainted
> > DOM26: EFLAGS: 00010202
> > DOM26: eax: 00000001 ebx: 20000001 ecx: c0a79e6c edx: c0a79e6c
> > DOM26: esi: c0a78000 edi: c0114254 ebp: c1e5f580 esp: c0a79ce4
> > DOM26: ds: 0821 es: 0821 ss: 0821
> > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > DOM26: Stack: 20000001 c0a78000 c002c728 c0a78000 c0a79db0 c0114254
> > ffffffb0 c0a78000
> > DOM26: c003e789 c0a79e6c c014b5ac c003e314 c0a79e6c 00000000
> > c0114250 c0a79de8
> > DOM26: c1419640 80000000 00000000 00000000 00000000 00000000
> > 00000000 00000000
> > DOM26: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > [<c002cf4a>]
> > DOM26: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > DOM26:
> > DOM26: <1>Unable to handle kernel paging request at virtual address
> > 20000001
> > DOM26: printing eip:
> > DOM26: c000af0f
> > DOM26: *pde=00000000(00000000)
> > DOM26: Oops: 0002
> > DOM26: CPU: 0
> > DOM26: EIP: 0819:[<c000af0f>] Not tainted
> > DOM26: EFLAGS: 00010282
> > DOM26: eax: 20000001 ebx: c1ed5b20 ecx: c0a78264 edx: c0a78264
> > DOM26: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c0a79bb4
> > DOM26: ds: 0821 es: 0821 ss: 0821
> > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > DOM26: Stack: c1ed5b20 00000000 c0a78000 0000000b 0000000b c000b55f
> > 20000001 0000001f
> > DOM26: 00000000 c140d6c0 20000001 c0091a87 0000000b 00000000
> > c1ed5b3c c0096305
> > DOM26: c0129928 c0a79cb0 00000000 c0a78000 00000000 20000001
> > c1e5f580 00000000
> > DOM26: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > [<c0018a25>]
> > DOM26: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > [<c0091768>] [<c0007743>]
> > DOM26: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > [<c002cf4a>] [<c002cf61>]
> > DOM26: [<c0090033>] [<c00914bf>]
> > DOM26:
> > DOM26: <1>Unable to handle kernel NULL pointer dereference at virtual
> > address 00000001
> > DOM26: printing eip:
> > DOM26: c000b623
> > DOM26: *pde=00000000(00000000)
> > DOM26: Oops: 0002
> > DOM26: CPU: 0
> > DOM26: EIP: 0819:[<c000b623>] Not tainted
> > DOM26: EFLAGS: 00010202
> > DOM26: eax: 00000000 ebx: 00000001 ecx: c0a78264 edx: c0a78264
> > DOM26: esi: 00000002 edi: c0a78000 ebp: 0000000b esp: c0a79aa0
> > DOM26: ds: 0821 es: 0821 ss: 0821
> > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > DOM26: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > 00000000 00000002
> > DOM26: c0096305 c0129928 c0a79b80 00000002 c0a78000 00000002
> > 20000001 0000000b
> > DOM26: 63303039 c101fc58 c0a78000 00000002 c101fc58 ffffffff
> > 00030001 c001e621
> > DOM26: Call Trace: [<c0091a87>] [<c0096305>] [<c001e621>] [<c001f6c0>]
> > [<c0008996>]
> > DOM26: [<c00200d7>] [<c00204e1>] [<c001464d>] [<c0014c92>]
> > [<c0091768>] [<c000af0f>]
> > DOM26: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > [<c0018a25>] [<c0018c46>]
> > DOM26: [<c0018feb>] [<c0018ed4>] [<c006e759>] [<c0091768>]
> > [<c0007743>] [<c002c728>]
> > DOM26: [<c003e789>] [<c003e314>] [<c002ccc7>] [<c002cf4a>]
> > [<c002cf61>] [<c0090033>]
> > DOM26: [<c00914bf>]
> > DOM26:
> >
> >
> >
> > --
> > Stephen G. Traugott (KG6HDQ)
> > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > stevegt@TerraLuna.Org
> > http://www.stevegt.com -- http://Infrastructures.Org
>
> --
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org
> http://www.stevegt.com -- http://Infrastructures.Org
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Null pointer deference
2004-02-10 1:25 Null pointer deference stevegt
2004-02-10 4:13 ` paging request failures under load (was: Re: Null pointer deference) stevegt
@ 2004-02-10 8:18 ` Ian Pratt
1 sibling, 0 replies; 21+ messages in thread
From: Ian Pratt @ 2004-02-10 8:18 UTC (permalink / raw)
To: stevegt; +Cc: xen-devel, Ian.Pratt
> I seem to be able to reproduce a null pointer dereference and paging
> request errors in 1.2. Can anyone give me any pointers on tracking down
> what is causing it?
Ouch! We haven't seen one of these in a _very_ long time. Our
systems generally don't make heavy use of swap, so I suspect this
could be the problem.
You should be able to process the Oops message as per a
standard Linux kernel (see Documentation/oops-tracing.txt)
As a quick check, look up the EIP in system.map and see what
function it blew up in.
However, if it is a paging fault, I'm not sure how useful the Oops
message will be.
We'll try and recreate locally.
Thanks,
Ian
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* 0-order allocation failed
2004-02-10 6:34 ` stevegt
@ 2004-02-11 4:12 ` stevegt
2004-02-11 4:51 ` Kip Macy
2004-02-11 8:13 ` Keir Fraser
2004-02-11 21:23 ` paging request failures under load (was: Re: Null pointer deference) stevegt
1 sibling, 2 replies; 21+ messages in thread
From: stevegt @ 2004-02-11 4:12 UTC (permalink / raw)
To: xen-devel
Looks like yesterday's paging problems were an artifact of something on
that particular dom0 disk image -- I haven't been able to reproduce it
on other nodes, and after I re-imaged the same node (with the same
image), the problem has gone away there too, so that rules out hardware.
However, something else did pop up -- while trying to break things with
"perl -e '$a="a"x100000000'", I got the following messages; do we care?
This is in mainstream linux mm/page_alloc.c, and it's hard for me to
tell from the code whether these are outright errors or whether they
were recoverable. Does anyone know?
DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
This was still on last Monday's version of 1.2; I'll see if I can
reproduce it in today's 1.2; I'm deploying that later tonight.
Steve
On Mon, Feb 09, 2004 at 10:34:34PM -0800, wrote:
> Before anyone burns too much time on this, hang on -- I wasn't able to
> duplicate the problem on another cluster node (both nodes were built
> from the same SystemImager image). I'm looking for the reason why, and
> will let you know as soon as I do.
>
> Steve
>
> On Mon, Feb 09, 2004 at 08:13:22PM -0800, wrote:
> > Okay, the problem still exists when I bump the memory up to 256Mb, and
> > never swap. I.E. I've found no workaround. Hasn't anyone else hit
> > anything like this?
> >
> > Steve
> >
> >
> > DOM3: xen_console_init
> > DOM3: Linux version 2.4.24-xeno (stevegt@pathfinder) (gcc version
> > 3.0.4) #16 Mon Feb 2 17:46:41 PST 2004
> > DOM3: On node 0 totalpages: 65536
> > DOM3: zone(0): 4096 pages.
> > DOM3: zone(1): 61440 pages.
> > DOM3: zone(2): 0 pages.
> > DOM3: Kernel command line:
> > ip=64.71.149.20:10.27.2.50:64.71.149.1:255.255.255.0::eth0:off
> > root=/dev/nfs nfsroot=/export//xen/fs/stevegt/tcx/root 4 DOMID=20
> > DOM3: Initializing CPU#0
> > DOM3: Xen reported: 398.780 MHz processor.
> > DOM3: Calibrating delay loop... 1592.52 BogoMIPS
> > DOM3: Memory: 257132k/262144k available (1078k kernel code, 5012k
> > reserved, 308k data, 52k init, 0k highmem)
> > DOM3: Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
> > DOM3: Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
> > DOM3: Mount cache hash table entries: 512 (order: 0, 4096 bytes)
> > DOM3: Buffer cache hash table entries: 16384 (order: 4, 65536 bytes)
> > DOM3: Page-cache hash table entries: 65536 (order: 6, 262144 bytes)
> > DOM3: CPU: L1 I cache: 16K, L1 D cache: 16K
> > DOM3: CPU: L2 cache: 512K
> > DOM3: CPU: Intel Pentium II (Deschutes) stepping 01
> > DOM3: POSIX conformance testing by UNIFIX
> > DOM3: Linux NET4.0 for Linux 2.4
> > DOM3: Based upon Swansea University Computer Society NET3.039
> > DOM3: Initializing RT netlink socket
> > DOM3: Starting kswapd
> > DOM3: Journalled Block Device driver loaded
> > DOM3: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
> > DOM3: Xeno console successfully installed
> > DOM3: Starting Xeno Balloon driver
> > DOM3: pty: 256 Unix98 ptys configured
> > DOM3: RAMDISK driver initialized: 16 RAM disks of 4096K size 1024
> > blocksize
> > DOM3: loop: loaded (max 8 devices)
> > DOM3: NET4: Linux TCP/IP 1.0 for NET4.0
> > DOM3: IP Protocols: ICMP, UDP, TCP
> > DOM3: IP: routing cache hash table of 2048 buckets, 16Kbytes
> > DOM3: TCP: Hash tables configured (established 16384 bind 16384)
> > DOM3: IP-Config: Complete:
> > DOM3: device=eth0, addr=64.71.149.20, mask=255.255.255.0,
> > gw=64.71.149.1,
> > DOM3: host=64.71.149.20, domain=, nis-domain=(none),
> > DOM3: bootserver=10.27.2.50, rootserver=10.27.2.50, rootpath=
> > DOM3: ip_conntrack version 2.1 (2048 buckets, 16384 max) - 292 bytes
> > per conntrack
> > DOM3: ip_tables: (C) 2000-2002 Netfilter core team
> > DOM3: NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> > DOM3: Looking up port of RPC 100003/2 on 10.27.2.50
> > DOM3: Looking up port of RPC 100005/1 on 10.27.2.50
> > DOM3: VFS: Mounted root (nfs filesystem).
> > DOM3: Freeing unused kernel memory: 52k freed
> > DOM3: INIT: version 2.84 booting
> > DOM3: Activating swap.
> > DOM3: Adding Swap: 262136k swap-space (priority -1)
> > DOM3: Checking root file system...
> > DOM3: fsck 1.27 (8-Mar-2002)
> > DOM3: 10.27.2.50:/export/xen/fs/stevegt/tcx: NFS file system.
> > DOM3: System time was Tue Feb 10 02:14:36 UTC 2004.
> > DOM3: Setting the System Clock using the Hardware Clock as
> > reference...
> > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > DOM3: hwclock is unable to get I/O port access: the iopl(3) call
> > failed.
> > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > DOM3: System Clock set. System local time is now Tue Feb 10 02:14:36
> > UTC 2004.
> > DOM3: Calculating module dependencies... depmod: cannot read ELF
> > header from /lib/modules/2.4.24-xeno/modules.dep
> > DOM3: depmod: cannot read ELF header from
> > /lib/modules/2.4.24-xeno/modules.generic_string
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.ieee1394map is not an
> > ELF file
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.isapnpmap is not an ELF
> > file
> > DOM3: depmod: cannot read ELF header from
> > /lib/modules/2.4.24-xeno/modules.parportmap
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.pcimap is not an ELF
> > file
> > DOM3: depmod: cannot read ELF header from
> > /lib/modules/2.4.24-xeno/modules.pnpbiosmap
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.usbmap is not an ELF
> > file
> > DOM3: done.
> > DOM3: Loading modules:
> > DOM3: Checking all file systems...
> > DOM3: fsck 1.27 (8-Mar-2002)
> > DOM3: Setting kernel variables.
> > DOM3: Loading the saved-state of the serial devices...
> > DOM3: Mounting local filesystems...
> > DOM3: nothing was mounted
> > DOM3: Running 0dns-down to make sure resolv.conf is ok...done.
> > DOM3: Cleaning: /etc/network/ifstate.
> > DOM3: Setting up IP spoofing protection: rp_filter.
> > DOM3: Configuring network interfaces: done.
> > DOM3: Mounting remote filesystems...
> > DOM3:
> > DOM3: Setting the System Clock using the Hardware Clock as
> > reference...
> > DOM3: System Clock set. Local time: Tue Feb 10 02:14:37 UTC 2004
> > DOM3:
> > DOM3: Cleaning: /tmp /var/lock /var/run.
> > DOM3: Initializing random number generator... done.
> > DOM3: Recovering nvi editor sessions... done.
> > DOM3: INIT: Entering runlevel: 4
> > DOM3: Starting system log daemon: syslogd.
> > DOM3: Starting kernel log daemon: klogd.
> > DOM3: Starting internet superserver: inetd.
> > DOM3: Starting PCMCIA services: module directory
> > /lib/modules/2.4.24-xeno/pcmcia not found.
> > DOM3: Starting OpenBSD Secure Shell server: sshd.
> > DOM3: Starting deferred execution scheduler: atd.
> > DOM3: Starting periodic command scheduler: cron.
> > DOM3: INIT: no more processes left in this runlevel
> > DOM3: Unable to handle kernel paging request at virtual address
> > 20000001
> > DOM3: printing eip:
> > DOM3: c0007743
> > DOM3: *pde=00000000(00000000)
> > DOM3: Oops: 0000
> > DOM3: CPU: 0
> > DOM3: EIP: 0819:[<c0007743>] Not tainted
> > DOM3: EFLAGS: 00010202
> > DOM3: eax: 00000001 ebx: 20000001 ecx: c3ebde6c edx: c3ebde6c
> > DOM3: esi: c3ebc000 edi: c0114254 ebp: c46c1060 esp: c3ebdce4
> > DOM3: ds: 0821 es: 0821 ss: 0821
> > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > DOM3: Stack: 20000001 c3ebc000 c002c728 c3ebc000 c3ebddb0 c0114254
> > ffffffb0 c3ebc000
> > DOM3: c003e789 c3ebde6c c014b5ac c003e314 c3ebde6c 00000000
> > c0114250 00000000
> > DOM3: 00000000 00000000 01082003 8d588810 00000000 00000000
> > 00000000 00000000
> > DOM3: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > [<c002cf4a>]
> > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > DOM3:
> > DOM3: <1>Unable to handle kernel paging request at virtual address
> > 20000001
> > DOM3: printing eip:
> > DOM3: c000af0f
> > DOM3: *pde=00000000(00000000)
> > DOM3: Oops: 0002
> > DOM3: CPU: 0
> > DOM3: EIP: 0819:[<c000af0f>] Not tainted
> > DOM3: EFLAGS: 00010282
> > DOM3: eax: 20000001 ebx: c485c0a0 ecx: c3ebc264 edx: c3ebc264
> > DOM3: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c3ebdbb4
> > DOM3: ds: 0821 es: 0821 ss: 0821
> > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > DOM3: Stack: c485c0a0 00000000 c3ebc000 0000000b 0000000b c000b55f
> > 20000001 0000001f
> > DOM3: 00000000 cf4227e0 20000001 c0091a87 0000000b 00000000
> > c485c0bc c0096305
> > DOM3: c0129928 c3ebdcb0 00000000 c3ebc000 00000000 20000001
> > c46c1060 00000000
> > DOM3: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > [<c0018a25>]
> > DOM3: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > [<c0091768>] [<c0007743>]
> > DOM3: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > [<c002cf4a>] [<c002cf61>]
> > DOM3: [<c0090033>] [<c00914bf>]
> > DOM3:
> > DOM3: <1>Unable to handle kernel NULL pointer dereference at virtual
> > address 00000001
> > DOM3: printing eip:
> > DOM3: c000b623
> > DOM3: *pde=00000000(00000000)
> > DOM3: Oops: 0002
> > DOM3: CPU: 0
> > DOM3: EIP: 0819:[<c000b623>] Not tainted
> > DOM3: EFLAGS: 00010202
> > DOM3: eax: 00000000 ebx: 00000001 ecx: c3ebc264 edx: c3ebc264
> > DOM3: esi: 00000002 edi: c3ebc000 ebp: 0000000b esp: c3ebdaa0
> > DOM3: ds: 0821 es: 0821 ss: 0821
> > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > DOM3: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > 00000000 00000002
> > DOM3: c0096305 c0129928 c3ebdb80 00000002 c3ebc000 00000002
> > 20000001 0000000b
> > DOM3: 63303039 ffffffff c3ebc000 00000002 38383130 38643538
> > 00030001 64303030
> > DOM3: Call Trace: [<c0091a87>] [<c0096305>] [<c0008996>] [<c000f797>]
> > [<c000f991>]
> > DOM3: [<c0091768>] [<c000af0f>] [<c000b55f>] [<c0091a87>]
> > [<c0096305>] [<c002eb19>]
> > DOM3: [<c0018a25>] [<c0018c46>] [<c0018feb>] [<c0018ed4>]
> > [<c006e759>] [<c0091768>]
> > DOM3: [<c0007743>] [<c002c728>] [<c003e789>] [<c003e314>]
> > [<c002ccc7>] [<c002cf4a>]
> > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > DOM3:
> >
> >
> >
> >
> > On Mon, Feb 09, 2004 at 05:25:00PM -0800, wrote:
> > > Hi All,
> > >
> > > I seem to be able to reproduce a null pointer dereference and paging
> > > request errors in 1.2. Can anyone give me any pointers on tracking down
> > > what is causing it?
> > >
> > > This is with a 32Mb virtual domain, running debian woody, NFS root,
> > > 256Mb swap in a local VD, while running a process which builds openldap,
> > > python2.2.3, and related packages. I'm not sure which package, if any
> > > in particular, is causing this; could be just anything that causes a
> > > similar workload. This particular set of messages appeared before the
> > > virtual domain locked up during the openldap build...
> > >
> > > Steve
> > >
> > >
> > > DOM26: Unable to handle kernel paging request at virtual address
> > > 20000001
> > > DOM26: printing eip:
> > > DOM26: c0007743
> > > DOM26: *pde=00000000(00000000)
> > > DOM26: Oops: 0000
> > > DOM26: CPU: 0
> > > DOM26: EIP: 0819:[<c0007743>] Not tainted
> > > DOM26: EFLAGS: 00010202
> > > DOM26: eax: 00000001 ebx: 20000001 ecx: c0a79e6c edx: c0a79e6c
> > > DOM26: esi: c0a78000 edi: c0114254 ebp: c1e5f580 esp: c0a79ce4
> > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > DOM26: Stack: 20000001 c0a78000 c002c728 c0a78000 c0a79db0 c0114254
> > > ffffffb0 c0a78000
> > > DOM26: c003e789 c0a79e6c c014b5ac c003e314 c0a79e6c 00000000
> > > c0114250 c0a79de8
> > > DOM26: c1419640 80000000 00000000 00000000 00000000 00000000
> > > 00000000 00000000
> > > DOM26: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > [<c002cf4a>]
> > > DOM26: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > DOM26:
> > > DOM26: <1>Unable to handle kernel paging request at virtual address
> > > 20000001
> > > DOM26: printing eip:
> > > DOM26: c000af0f
> > > DOM26: *pde=00000000(00000000)
> > > DOM26: Oops: 0002
> > > DOM26: CPU: 0
> > > DOM26: EIP: 0819:[<c000af0f>] Not tainted
> > > DOM26: EFLAGS: 00010282
> > > DOM26: eax: 20000001 ebx: c1ed5b20 ecx: c0a78264 edx: c0a78264
> > > DOM26: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c0a79bb4
> > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > DOM26: Stack: c1ed5b20 00000000 c0a78000 0000000b 0000000b c000b55f
> > > 20000001 0000001f
> > > DOM26: 00000000 c140d6c0 20000001 c0091a87 0000000b 00000000
> > > c1ed5b3c c0096305
> > > DOM26: c0129928 c0a79cb0 00000000 c0a78000 00000000 20000001
> > > c1e5f580 00000000
> > > DOM26: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > [<c0018a25>]
> > > DOM26: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > > [<c0091768>] [<c0007743>]
> > > DOM26: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > [<c002cf4a>] [<c002cf61>]
> > > DOM26: [<c0090033>] [<c00914bf>]
> > > DOM26:
> > > DOM26: <1>Unable to handle kernel NULL pointer dereference at virtual
> > > address 00000001
> > > DOM26: printing eip:
> > > DOM26: c000b623
> > > DOM26: *pde=00000000(00000000)
> > > DOM26: Oops: 0002
> > > DOM26: CPU: 0
> > > DOM26: EIP: 0819:[<c000b623>] Not tainted
> > > DOM26: EFLAGS: 00010202
> > > DOM26: eax: 00000000 ebx: 00000001 ecx: c0a78264 edx: c0a78264
> > > DOM26: esi: 00000002 edi: c0a78000 ebp: 0000000b esp: c0a79aa0
> > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > DOM26: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > > 00000000 00000002
> > > DOM26: c0096305 c0129928 c0a79b80 00000002 c0a78000 00000002
> > > 20000001 0000000b
> > > DOM26: 63303039 c101fc58 c0a78000 00000002 c101fc58 ffffffff
> > > 00030001 c001e621
> > > DOM26: Call Trace: [<c0091a87>] [<c0096305>] [<c001e621>] [<c001f6c0>]
> > > [<c0008996>]
> > > DOM26: [<c00200d7>] [<c00204e1>] [<c001464d>] [<c0014c92>]
> > > [<c0091768>] [<c000af0f>]
> > > DOM26: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > [<c0018a25>] [<c0018c46>]
> > > DOM26: [<c0018feb>] [<c0018ed4>] [<c006e759>] [<c0091768>]
> > > [<c0007743>] [<c002c728>]
> > > DOM26: [<c003e789>] [<c003e314>] [<c002ccc7>] [<c002cf4a>]
> > > [<c002cf61>] [<c0090033>]
> > > DOM26: [<c00914bf>]
> > > DOM26:
> > >
> > >
> > >
> > > --
> > > Stephen G. Traugott (KG6HDQ)
> > > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > > stevegt@TerraLuna.Org
> > > http://www.stevegt.com -- http://Infrastructures.Org
> >
> > --
> > Stephen G. Traugott (KG6HDQ)
> > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > stevegt@TerraLuna.Org
> > http://www.stevegt.com -- http://Infrastructures.Org
>
> --
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org
> http://www.stevegt.com -- http://Infrastructures.Org
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: 0-order allocation failed
2004-02-11 4:12 ` 0-order allocation failed stevegt
@ 2004-02-11 4:51 ` Kip Macy
2004-02-11 6:36 ` stevegt
2004-02-11 8:13 ` Keir Fraser
1 sibling, 1 reply; 21+ messages in thread
From: Kip Macy @ 2004-02-11 4:51 UTC (permalink / raw)
To: stevegt; +Cc: xen-devel
That is odd. What that means is that something tried to allocate a page
without passing in GFP_WAIT as one of the flags, and there were no pages
that could be freed in any of the zones. Does this not happen when
running 2.4.24 directly on the machine with the same amount of memory?
-Kip
On Tue, 10 Feb 2004 stevegt@TerraLuna.Org wrote:
> Looks like yesterday's paging problems were an artifact of something on
> that particular dom0 disk image -- I haven't been able to reproduce it
> on other nodes, and after I re-imaged the same node (with the same
> image), the problem has gone away there too, so that rules out hardware.
>
> However, something else did pop up -- while trying to break things with
> "perl -e '$a="a"x100000000'", I got the following messages; do we care?
> This is in mainstream linux mm/page_alloc.c, and it's hard for me to
> tell from the code whether these are outright errors or whether they
> were recoverable. Does anyone know?
>
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
>
> This was still on last Monday's version of 1.2; I'll see if I can
> reproduce it in today's 1.2; I'm deploying that later tonight.
>
> Steve
>
>
> On Mon, Feb 09, 2004 at 10:34:34PM -0800, wrote:
> > Before anyone burns too much time on this, hang on -- I wasn't able to
> > duplicate the problem on another cluster node (both nodes were built
> > from the same SystemImager image). I'm looking for the reason why, and
> > will let you know as soon as I do.
> >
> > Steve
> >
> > On Mon, Feb 09, 2004 at 08:13:22PM -0800, wrote:
> > > Okay, the problem still exists when I bump the memory up to 256Mb, and
> > > never swap. I.E. I've found no workaround. Hasn't anyone else hit
> > > anything like this?
> > >
> > > Steve
> > >
> > >
> > > DOM3: xen_console_init
> > > DOM3: Linux version 2.4.24-xeno (stevegt@pathfinder) (gcc version
> > > 3.0.4) #16 Mon Feb 2 17:46:41 PST 2004
> > > DOM3: On node 0 totalpages: 65536
> > > DOM3: zone(0): 4096 pages.
> > > DOM3: zone(1): 61440 pages.
> > > DOM3: zone(2): 0 pages.
> > > DOM3: Kernel command line:
> > > ip=64.71.149.20:10.27.2.50:64.71.149.1:255.255.255.0::eth0:off
> > > root=/dev/nfs nfsroot=/export//xen/fs/stevegt/tcx/root 4 DOMID=20
> > > DOM3: Initializing CPU#0
> > > DOM3: Xen reported: 398.780 MHz processor.
> > > DOM3: Calibrating delay loop... 1592.52 BogoMIPS
> > > DOM3: Memory: 257132k/262144k available (1078k kernel code, 5012k
> > > reserved, 308k data, 52k init, 0k highmem)
> > > DOM3: Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
> > > DOM3: Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
> > > DOM3: Mount cache hash table entries: 512 (order: 0, 4096 bytes)
> > > DOM3: Buffer cache hash table entries: 16384 (order: 4, 65536 bytes)
> > > DOM3: Page-cache hash table entries: 65536 (order: 6, 262144 bytes)
> > > DOM3: CPU: L1 I cache: 16K, L1 D cache: 16K
> > > DOM3: CPU: L2 cache: 512K
> > > DOM3: CPU: Intel Pentium II (Deschutes) stepping 01
> > > DOM3: POSIX conformance testing by UNIFIX
> > > DOM3: Linux NET4.0 for Linux 2.4
> > > DOM3: Based upon Swansea University Computer Society NET3.039
> > > DOM3: Initializing RT netlink socket
> > > DOM3: Starting kswapd
> > > DOM3: Journalled Block Device driver loaded
> > > DOM3: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
> > > DOM3: Xeno console successfully installed
> > > DOM3: Starting Xeno Balloon driver
> > > DOM3: pty: 256 Unix98 ptys configured
> > > DOM3: RAMDISK driver initialized: 16 RAM disks of 4096K size 1024
> > > blocksize
> > > DOM3: loop: loaded (max 8 devices)
> > > DOM3: NET4: Linux TCP/IP 1.0 for NET4.0
> > > DOM3: IP Protocols: ICMP, UDP, TCP
> > > DOM3: IP: routing cache hash table of 2048 buckets, 16Kbytes
> > > DOM3: TCP: Hash tables configured (established 16384 bind 16384)
> > > DOM3: IP-Config: Complete:
> > > DOM3: device=eth0, addr=64.71.149.20, mask=255.255.255.0,
> > > gw=64.71.149.1,
> > > DOM3: host=64.71.149.20, domain=, nis-domain=(none),
> > > DOM3: bootserver=10.27.2.50, rootserver=10.27.2.50, rootpath=
> > > DOM3: ip_conntrack version 2.1 (2048 buckets, 16384 max) - 292 bytes
> > > per conntrack
> > > DOM3: ip_tables: (C) 2000-2002 Netfilter core team
> > > DOM3: NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> > > DOM3: Looking up port of RPC 100003/2 on 10.27.2.50
> > > DOM3: Looking up port of RPC 100005/1 on 10.27.2.50
> > > DOM3: VFS: Mounted root (nfs filesystem).
> > > DOM3: Freeing unused kernel memory: 52k freed
> > > DOM3: INIT: version 2.84 booting
> > > DOM3: Activating swap.
> > > DOM3: Adding Swap: 262136k swap-space (priority -1)
> > > DOM3: Checking root file system...
> > > DOM3: fsck 1.27 (8-Mar-2002)
> > > DOM3: 10.27.2.50:/export/xen/fs/stevegt/tcx: NFS file system.
> > > DOM3: System time was Tue Feb 10 02:14:36 UTC 2004.
> > > DOM3: Setting the System Clock using the Hardware Clock as
> > > reference...
> > > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > > DOM3: hwclock is unable to get I/O port access: the iopl(3) call
> > > failed.
> > > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > > DOM3: System Clock set. System local time is now Tue Feb 10 02:14:36
> > > UTC 2004.
> > > DOM3: Calculating module dependencies... depmod: cannot read ELF
> > > header from /lib/modules/2.4.24-xeno/modules.dep
> > > DOM3: depmod: cannot read ELF header from
> > > /lib/modules/2.4.24-xeno/modules.generic_string
> > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.ieee1394map is not an
> > > ELF file
> > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.isapnpmap is not an ELF
> > > file
> > > DOM3: depmod: cannot read ELF header from
> > > /lib/modules/2.4.24-xeno/modules.parportmap
> > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.pcimap is not an ELF
> > > file
> > > DOM3: depmod: cannot read ELF header from
> > > /lib/modules/2.4.24-xeno/modules.pnpbiosmap
> > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.usbmap is not an ELF
> > > file
> > > DOM3: done.
> > > DOM3: Loading modules:
> > > DOM3: Checking all file systems...
> > > DOM3: fsck 1.27 (8-Mar-2002)
> > > DOM3: Setting kernel variables.
> > > DOM3: Loading the saved-state of the serial devices...
> > > DOM3: Mounting local filesystems...
> > > DOM3: nothing was mounted
> > > DOM3: Running 0dns-down to make sure resolv.conf is ok...done.
> > > DOM3: Cleaning: /etc/network/ifstate.
> > > DOM3: Setting up IP spoofing protection: rp_filter.
> > > DOM3: Configuring network interfaces: done.
> > > DOM3: Mounting remote filesystems...
> > > DOM3:
> > > DOM3: Setting the System Clock using the Hardware Clock as
> > > reference...
> > > DOM3: System Clock set. Local time: Tue Feb 10 02:14:37 UTC 2004
> > > DOM3:
> > > DOM3: Cleaning: /tmp /var/lock /var/run.
> > > DOM3: Initializing random number generator... done.
> > > DOM3: Recovering nvi editor sessions... done.
> > > DOM3: INIT: Entering runlevel: 4
> > > DOM3: Starting system log daemon: syslogd.
> > > DOM3: Starting kernel log daemon: klogd.
> > > DOM3: Starting internet superserver: inetd.
> > > DOM3: Starting PCMCIA services: module directory
> > > /lib/modules/2.4.24-xeno/pcmcia not found.
> > > DOM3: Starting OpenBSD Secure Shell server: sshd.
> > > DOM3: Starting deferred execution scheduler: atd.
> > > DOM3: Starting periodic command scheduler: cron.
> > > DOM3: INIT: no more processes left in this runlevel
> > > DOM3: Unable to handle kernel paging request at virtual address
> > > 20000001
> > > DOM3: printing eip:
> > > DOM3: c0007743
> > > DOM3: *pde=00000000(00000000)
> > > DOM3: Oops: 0000
> > > DOM3: CPU: 0
> > > DOM3: EIP: 0819:[<c0007743>] Not tainted
> > > DOM3: EFLAGS: 00010202
> > > DOM3: eax: 00000001 ebx: 20000001 ecx: c3ebde6c edx: c3ebde6c
> > > DOM3: esi: c3ebc000 edi: c0114254 ebp: c46c1060 esp: c3ebdce4
> > > DOM3: ds: 0821 es: 0821 ss: 0821
> > > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > > DOM3: Stack: 20000001 c3ebc000 c002c728 c3ebc000 c3ebddb0 c0114254
> > > ffffffb0 c3ebc000
> > > DOM3: c003e789 c3ebde6c c014b5ac c003e314 c3ebde6c 00000000
> > > c0114250 00000000
> > > DOM3: 00000000 00000000 01082003 8d588810 00000000 00000000
> > > 00000000 00000000
> > > DOM3: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > [<c002cf4a>]
> > > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > DOM3:
> > > DOM3: <1>Unable to handle kernel paging request at virtual address
> > > 20000001
> > > DOM3: printing eip:
> > > DOM3: c000af0f
> > > DOM3: *pde=00000000(00000000)
> > > DOM3: Oops: 0002
> > > DOM3: CPU: 0
> > > DOM3: EIP: 0819:[<c000af0f>] Not tainted
> > > DOM3: EFLAGS: 00010282
> > > DOM3: eax: 20000001 ebx: c485c0a0 ecx: c3ebc264 edx: c3ebc264
> > > DOM3: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c3ebdbb4
> > > DOM3: ds: 0821 es: 0821 ss: 0821
> > > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > > DOM3: Stack: c485c0a0 00000000 c3ebc000 0000000b 0000000b c000b55f
> > > 20000001 0000001f
> > > DOM3: 00000000 cf4227e0 20000001 c0091a87 0000000b 00000000
> > > c485c0bc c0096305
> > > DOM3: c0129928 c3ebdcb0 00000000 c3ebc000 00000000 20000001
> > > c46c1060 00000000
> > > DOM3: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > [<c0018a25>]
> > > DOM3: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > > [<c0091768>] [<c0007743>]
> > > DOM3: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > [<c002cf4a>] [<c002cf61>]
> > > DOM3: [<c0090033>] [<c00914bf>]
> > > DOM3:
> > > DOM3: <1>Unable to handle kernel NULL pointer dereference at virtual
> > > address 00000001
> > > DOM3: printing eip:
> > > DOM3: c000b623
> > > DOM3: *pde=00000000(00000000)
> > > DOM3: Oops: 0002
> > > DOM3: CPU: 0
> > > DOM3: EIP: 0819:[<c000b623>] Not tainted
> > > DOM3: EFLAGS: 00010202
> > > DOM3: eax: 00000000 ebx: 00000001 ecx: c3ebc264 edx: c3ebc264
> > > DOM3: esi: 00000002 edi: c3ebc000 ebp: 0000000b esp: c3ebdaa0
> > > DOM3: ds: 0821 es: 0821 ss: 0821
> > > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > > DOM3: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > > 00000000 00000002
> > > DOM3: c0096305 c0129928 c3ebdb80 00000002 c3ebc000 00000002
> > > 20000001 0000000b
> > > DOM3: 63303039 ffffffff c3ebc000 00000002 38383130 38643538
> > > 00030001 64303030
> > > DOM3: Call Trace: [<c0091a87>] [<c0096305>] [<c0008996>] [<c000f797>]
> > > [<c000f991>]
> > > DOM3: [<c0091768>] [<c000af0f>] [<c000b55f>] [<c0091a87>]
> > > [<c0096305>] [<c002eb19>]
> > > DOM3: [<c0018a25>] [<c0018c46>] [<c0018feb>] [<c0018ed4>]
> > > [<c006e759>] [<c0091768>]
> > > DOM3: [<c0007743>] [<c002c728>] [<c003e789>] [<c003e314>]
> > > [<c002ccc7>] [<c002cf4a>]
> > > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > DOM3:
> > >
> > >
> > >
> > >
> > > On Mon, Feb 09, 2004 at 05:25:00PM -0800, wrote:
> > > > Hi All,
> > > >
> > > > I seem to be able to reproduce a null pointer dereference and paging
> > > > request errors in 1.2. Can anyone give me any pointers on tracking down
> > > > what is causing it?
> > > >
> > > > This is with a 32Mb virtual domain, running debian woody, NFS root,
> > > > 256Mb swap in a local VD, while running a process which builds openldap,
> > > > python2.2.3, and related packages. I'm not sure which package, if any
> > > > in particular, is causing this; could be just anything that causes a
> > > > similar workload. This particular set of messages appeared before the
> > > > virtual domain locked up during the openldap build...
> > > >
> > > > Steve
> > > >
> > > >
> > > > DOM26: Unable to handle kernel paging request at virtual address
> > > > 20000001
> > > > DOM26: printing eip:
> > > > DOM26: c0007743
> > > > DOM26: *pde=00000000(00000000)
> > > > DOM26: Oops: 0000
> > > > DOM26: CPU: 0
> > > > DOM26: EIP: 0819:[<c0007743>] Not tainted
> > > > DOM26: EFLAGS: 00010202
> > > > DOM26: eax: 00000001 ebx: 20000001 ecx: c0a79e6c edx: c0a79e6c
> > > > DOM26: esi: c0a78000 edi: c0114254 ebp: c1e5f580 esp: c0a79ce4
> > > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > > DOM26: Stack: 20000001 c0a78000 c002c728 c0a78000 c0a79db0 c0114254
> > > > ffffffb0 c0a78000
> > > > DOM26: c003e789 c0a79e6c c014b5ac c003e314 c0a79e6c 00000000
> > > > c0114250 c0a79de8
> > > > DOM26: c1419640 80000000 00000000 00000000 00000000 00000000
> > > > 00000000 00000000
> > > > DOM26: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > > [<c002cf4a>]
> > > > DOM26: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > > DOM26:
> > > > DOM26: <1>Unable to handle kernel paging request at virtual address
> > > > 20000001
> > > > DOM26: printing eip:
> > > > DOM26: c000af0f
> > > > DOM26: *pde=00000000(00000000)
> > > > DOM26: Oops: 0002
> > > > DOM26: CPU: 0
> > > > DOM26: EIP: 0819:[<c000af0f>] Not tainted
> > > > DOM26: EFLAGS: 00010282
> > > > DOM26: eax: 20000001 ebx: c1ed5b20 ecx: c0a78264 edx: c0a78264
> > > > DOM26: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c0a79bb4
> > > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > > DOM26: Stack: c1ed5b20 00000000 c0a78000 0000000b 0000000b c000b55f
> > > > 20000001 0000001f
> > > > DOM26: 00000000 c140d6c0 20000001 c0091a87 0000000b 00000000
> > > > c1ed5b3c c0096305
> > > > DOM26: c0129928 c0a79cb0 00000000 c0a78000 00000000 20000001
> > > > c1e5f580 00000000
> > > > DOM26: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > > [<c0018a25>]
> > > > DOM26: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > > > [<c0091768>] [<c0007743>]
> > > > DOM26: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > > [<c002cf4a>] [<c002cf61>]
> > > > DOM26: [<c0090033>] [<c00914bf>]
> > > > DOM26:
> > > > DOM26: <1>Unable to handle kernel NULL pointer dereference at virtual
> > > > address 00000001
> > > > DOM26: printing eip:
> > > > DOM26: c000b623
> > > > DOM26: *pde=00000000(00000000)
> > > > DOM26: Oops: 0002
> > > > DOM26: CPU: 0
> > > > DOM26: EIP: 0819:[<c000b623>] Not tainted
> > > > DOM26: EFLAGS: 00010202
> > > > DOM26: eax: 00000000 ebx: 00000001 ecx: c0a78264 edx: c0a78264
> > > > DOM26: esi: 00000002 edi: c0a78000 ebp: 0000000b esp: c0a79aa0
> > > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > > DOM26: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > > > 00000000 00000002
> > > > DOM26: c0096305 c0129928 c0a79b80 00000002 c0a78000 00000002
> > > > 20000001 0000000b
> > > > DOM26: 63303039 c101fc58 c0a78000 00000002 c101fc58 ffffffff
> > > > 00030001 c001e621
> > > > DOM26: Call Trace: [<c0091a87>] [<c0096305>] [<c001e621>] [<c001f6c0>]
> > > > [<c0008996>]
> > > > DOM26: [<c00200d7>] [<c00204e1>] [<c001464d>] [<c0014c92>]
> > > > [<c0091768>] [<c000af0f>]
> > > > DOM26: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > > [<c0018a25>] [<c0018c46>]
> > > > DOM26: [<c0018feb>] [<c0018ed4>] [<c006e759>] [<c0091768>]
> > > > [<c0007743>] [<c002c728>]
> > > > DOM26: [<c003e789>] [<c003e314>] [<c002ccc7>] [<c002cf4a>]
> > > > [<c002cf61>] [<c0090033>]
> > > > DOM26: [<c00914bf>]
> > > > DOM26:
> > > >
> > > >
> > > >
> > > > --
> > > > Stephen G. Traugott (KG6HDQ)
> > > > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > > > stevegt@TerraLuna.Org
> > > > http://www.stevegt.com -- http://Infrastructures.Org
> > >
> > > --
> > > Stephen G. Traugott (KG6HDQ)
> > > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > > stevegt@TerraLuna.Org
> > > http://www.stevegt.com -- http://Infrastructures.Org
> >
> > --
> > Stephen G. Traugott (KG6HDQ)
> > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > stevegt@TerraLuna.Org
> > http://www.stevegt.com -- http://Infrastructures.Org
>
> --
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org
> http://www.stevegt.com -- http://Infrastructures.Org
>
>
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: 0-order allocation failed
2004-02-11 4:51 ` Kip Macy
@ 2004-02-11 6:36 ` stevegt
0 siblings, 0 replies; 21+ messages in thread
From: stevegt @ 2004-02-11 6:36 UTC (permalink / raw)
To: Kip Macy; +Cc: xen-devel
I don't know -- I've never seen Linux issue this particular message
before. This is a 32Mb virtual domain, with 256Mb VD swap, and an NFS
root filesystem. I don't have any real machines with only 32Mb RAM. ;-)
I've duplicated it in two different nodes now, using the 02 Feb
1.2 build. I seem to be able to get one every few minutes by running
"perl -e '$a="a"x100000000'" in a while loop on two guests at the same
time.
I was going to try to duplicate it in today's 1.2, but that's not
booting -- I'll start another thread for that.
Steve
On Tue, Feb 10, 2004 at 08:51:42PM -0800, Kip Macy wrote:
>
> That is odd. What that means is that something tried to allocate a page
> without passing in GFP_WAIT as one of the flags, and there were no pages
> that could be freed in any of the zones. Does this not happen when
> running 2.4.24 directly on the machine with the same amount of memory?
>
>
> -Kip
>
>
> On Tue, 10 Feb 2004 stevegt@TerraLuna.Org wrote:
>
> > Looks like yesterday's paging problems were an artifact of something on
> > that particular dom0 disk image -- I haven't been able to reproduce it
> > on other nodes, and after I re-imaged the same node (with the same
> > image), the problem has gone away there too, so that rules out hardware.
> >
> > However, something else did pop up -- while trying to break things with
> > "perl -e '$a="a"x100000000'", I got the following messages; do we care?
> > This is in mainstream linux mm/page_alloc.c, and it's hard for me to
> > tell from the code whether these are outright errors or whether they
> > were recoverable. Does anyone know?
> >
> > DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> > DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> > DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> > DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> >
> > This was still on last Monday's version of 1.2; I'll see if I can
> > reproduce it in today's 1.2; I'm deploying that later tonight.
> >
> > Steve
> >
> >
> > On Mon, Feb 09, 2004 at 10:34:34PM -0800, wrote:
> > > Before anyone burns too much time on this, hang on -- I wasn't able to
> > > duplicate the problem on another cluster node (both nodes were built
> > > from the same SystemImager image). I'm looking for the reason why, and
> > > will let you know as soon as I do.
> > >
> > > Steve
> > >
> > > On Mon, Feb 09, 2004 at 08:13:22PM -0800, wrote:
> > > > Okay, the problem still exists when I bump the memory up to 256Mb, and
> > > > never swap. I.E. I've found no workaround. Hasn't anyone else hit
> > > > anything like this?
> > > >
> > > > Steve
> > > >
> > > >
> > > > DOM3: xen_console_init
> > > > DOM3: Linux version 2.4.24-xeno (stevegt@pathfinder) (gcc version
> > > > 3.0.4) #16 Mon Feb 2 17:46:41 PST 2004
> > > > DOM3: On node 0 totalpages: 65536
> > > > DOM3: zone(0): 4096 pages.
> > > > DOM3: zone(1): 61440 pages.
> > > > DOM3: zone(2): 0 pages.
> > > > DOM3: Kernel command line:
> > > > ip=64.71.149.20:10.27.2.50:64.71.149.1:255.255.255.0::eth0:off
> > > > root=/dev/nfs nfsroot=/export//xen/fs/stevegt/tcx/root 4 DOMID=20
> > > > DOM3: Initializing CPU#0
> > > > DOM3: Xen reported: 398.780 MHz processor.
> > > > DOM3: Calibrating delay loop... 1592.52 BogoMIPS
> > > > DOM3: Memory: 257132k/262144k available (1078k kernel code, 5012k
> > > > reserved, 308k data, 52k init, 0k highmem)
> > > > DOM3: Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
> > > > DOM3: Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
> > > > DOM3: Mount cache hash table entries: 512 (order: 0, 4096 bytes)
> > > > DOM3: Buffer cache hash table entries: 16384 (order: 4, 65536 bytes)
> > > > DOM3: Page-cache hash table entries: 65536 (order: 6, 262144 bytes)
> > > > DOM3: CPU: L1 I cache: 16K, L1 D cache: 16K
> > > > DOM3: CPU: L2 cache: 512K
> > > > DOM3: CPU: Intel Pentium II (Deschutes) stepping 01
> > > > DOM3: POSIX conformance testing by UNIFIX
> > > > DOM3: Linux NET4.0 for Linux 2.4
> > > > DOM3: Based upon Swansea University Computer Society NET3.039
> > > > DOM3: Initializing RT netlink socket
> > > > DOM3: Starting kswapd
> > > > DOM3: Journalled Block Device driver loaded
> > > > DOM3: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
> > > > DOM3: Xeno console successfully installed
> > > > DOM3: Starting Xeno Balloon driver
> > > > DOM3: pty: 256 Unix98 ptys configured
> > > > DOM3: RAMDISK driver initialized: 16 RAM disks of 4096K size 1024
> > > > blocksize
> > > > DOM3: loop: loaded (max 8 devices)
> > > > DOM3: NET4: Linux TCP/IP 1.0 for NET4.0
> > > > DOM3: IP Protocols: ICMP, UDP, TCP
> > > > DOM3: IP: routing cache hash table of 2048 buckets, 16Kbytes
> > > > DOM3: TCP: Hash tables configured (established 16384 bind 16384)
> > > > DOM3: IP-Config: Complete:
> > > > DOM3: device=eth0, addr=64.71.149.20, mask=255.255.255.0,
> > > > gw=64.71.149.1,
> > > > DOM3: host=64.71.149.20, domain=, nis-domain=(none),
> > > > DOM3: bootserver=10.27.2.50, rootserver=10.27.2.50, rootpath=
> > > > DOM3: ip_conntrack version 2.1 (2048 buckets, 16384 max) - 292 bytes
> > > > per conntrack
> > > > DOM3: ip_tables: (C) 2000-2002 Netfilter core team
> > > > DOM3: NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> > > > DOM3: Looking up port of RPC 100003/2 on 10.27.2.50
> > > > DOM3: Looking up port of RPC 100005/1 on 10.27.2.50
> > > > DOM3: VFS: Mounted root (nfs filesystem).
> > > > DOM3: Freeing unused kernel memory: 52k freed
> > > > DOM3: INIT: version 2.84 booting
> > > > DOM3: Activating swap.
> > > > DOM3: Adding Swap: 262136k swap-space (priority -1)
> > > > DOM3: Checking root file system...
> > > > DOM3: fsck 1.27 (8-Mar-2002)
> > > > DOM3: 10.27.2.50:/export/xen/fs/stevegt/tcx: NFS file system.
> > > > DOM3: System time was Tue Feb 10 02:14:36 UTC 2004.
> > > > DOM3: Setting the System Clock using the Hardware Clock as
> > > > reference...
> > > > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > > > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > > > DOM3: hwclock is unable to get I/O port access: the iopl(3) call
> > > > failed.
> > > > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > > > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > > > DOM3: System Clock set. System local time is now Tue Feb 10 02:14:36
> > > > UTC 2004.
> > > > DOM3: Calculating module dependencies... depmod: cannot read ELF
> > > > header from /lib/modules/2.4.24-xeno/modules.dep
> > > > DOM3: depmod: cannot read ELF header from
> > > > /lib/modules/2.4.24-xeno/modules.generic_string
> > > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.ieee1394map is not an
> > > > ELF file
> > > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.isapnpmap is not an ELF
> > > > file
> > > > DOM3: depmod: cannot read ELF header from
> > > > /lib/modules/2.4.24-xeno/modules.parportmap
> > > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.pcimap is not an ELF
> > > > file
> > > > DOM3: depmod: cannot read ELF header from
> > > > /lib/modules/2.4.24-xeno/modules.pnpbiosmap
> > > > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.usbmap is not an ELF
> > > > file
> > > > DOM3: done.
> > > > DOM3: Loading modules:
> > > > DOM3: Checking all file systems...
> > > > DOM3: fsck 1.27 (8-Mar-2002)
> > > > DOM3: Setting kernel variables.
> > > > DOM3: Loading the saved-state of the serial devices...
> > > > DOM3: Mounting local filesystems...
> > > > DOM3: nothing was mounted
> > > > DOM3: Running 0dns-down to make sure resolv.conf is ok...done.
> > > > DOM3: Cleaning: /etc/network/ifstate.
> > > > DOM3: Setting up IP spoofing protection: rp_filter.
> > > > DOM3: Configuring network interfaces: done.
> > > > DOM3: Mounting remote filesystems...
> > > > DOM3:
> > > > DOM3: Setting the System Clock using the Hardware Clock as
> > > > reference...
> > > > DOM3: System Clock set. Local time: Tue Feb 10 02:14:37 UTC 2004
> > > > DOM3:
> > > > DOM3: Cleaning: /tmp /var/lock /var/run.
> > > > DOM3: Initializing random number generator... done.
> > > > DOM3: Recovering nvi editor sessions... done.
> > > > DOM3: INIT: Entering runlevel: 4
> > > > DOM3: Starting system log daemon: syslogd.
> > > > DOM3: Starting kernel log daemon: klogd.
> > > > DOM3: Starting internet superserver: inetd.
> > > > DOM3: Starting PCMCIA services: module directory
> > > > /lib/modules/2.4.24-xeno/pcmcia not found.
> > > > DOM3: Starting OpenBSD Secure Shell server: sshd.
> > > > DOM3: Starting deferred execution scheduler: atd.
> > > > DOM3: Starting periodic command scheduler: cron.
> > > > DOM3: INIT: no more processes left in this runlevel
> > > > DOM3: Unable to handle kernel paging request at virtual address
> > > > 20000001
> > > > DOM3: printing eip:
> > > > DOM3: c0007743
> > > > DOM3: *pde=00000000(00000000)
> > > > DOM3: Oops: 0000
> > > > DOM3: CPU: 0
> > > > DOM3: EIP: 0819:[<c0007743>] Not tainted
> > > > DOM3: EFLAGS: 00010202
> > > > DOM3: eax: 00000001 ebx: 20000001 ecx: c3ebde6c edx: c3ebde6c
> > > > DOM3: esi: c3ebc000 edi: c0114254 ebp: c46c1060 esp: c3ebdce4
> > > > DOM3: ds: 0821 es: 0821 ss: 0821
> > > > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > > > DOM3: Stack: 20000001 c3ebc000 c002c728 c3ebc000 c3ebddb0 c0114254
> > > > ffffffb0 c3ebc000
> > > > DOM3: c003e789 c3ebde6c c014b5ac c003e314 c3ebde6c 00000000
> > > > c0114250 00000000
> > > > DOM3: 00000000 00000000 01082003 8d588810 00000000 00000000
> > > > 00000000 00000000
> > > > DOM3: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > > [<c002cf4a>]
> > > > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > > DOM3:
> > > > DOM3: <1>Unable to handle kernel paging request at virtual address
> > > > 20000001
> > > > DOM3: printing eip:
> > > > DOM3: c000af0f
> > > > DOM3: *pde=00000000(00000000)
> > > > DOM3: Oops: 0002
> > > > DOM3: CPU: 0
> > > > DOM3: EIP: 0819:[<c000af0f>] Not tainted
> > > > DOM3: EFLAGS: 00010282
> > > > DOM3: eax: 20000001 ebx: c485c0a0 ecx: c3ebc264 edx: c3ebc264
> > > > DOM3: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c3ebdbb4
> > > > DOM3: ds: 0821 es: 0821 ss: 0821
> > > > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > > > DOM3: Stack: c485c0a0 00000000 c3ebc000 0000000b 0000000b c000b55f
> > > > 20000001 0000001f
> > > > DOM3: 00000000 cf4227e0 20000001 c0091a87 0000000b 00000000
> > > > c485c0bc c0096305
> > > > DOM3: c0129928 c3ebdcb0 00000000 c3ebc000 00000000 20000001
> > > > c46c1060 00000000
> > > > DOM3: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > > [<c0018a25>]
> > > > DOM3: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > > > [<c0091768>] [<c0007743>]
> > > > DOM3: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > > [<c002cf4a>] [<c002cf61>]
> > > > DOM3: [<c0090033>] [<c00914bf>]
> > > > DOM3:
> > > > DOM3: <1>Unable to handle kernel NULL pointer dereference at virtual
> > > > address 00000001
> > > > DOM3: printing eip:
> > > > DOM3: c000b623
> > > > DOM3: *pde=00000000(00000000)
> > > > DOM3: Oops: 0002
> > > > DOM3: CPU: 0
> > > > DOM3: EIP: 0819:[<c000b623>] Not tainted
> > > > DOM3: EFLAGS: 00010202
> > > > DOM3: eax: 00000000 ebx: 00000001 ecx: c3ebc264 edx: c3ebc264
> > > > DOM3: esi: 00000002 edi: c3ebc000 ebp: 0000000b esp: c3ebdaa0
> > > > DOM3: ds: 0821 es: 0821 ss: 0821
> > > > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > > > DOM3: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > > > 00000000 00000002
> > > > DOM3: c0096305 c0129928 c3ebdb80 00000002 c3ebc000 00000002
> > > > 20000001 0000000b
> > > > DOM3: 63303039 ffffffff c3ebc000 00000002 38383130 38643538
> > > > 00030001 64303030
> > > > DOM3: Call Trace: [<c0091a87>] [<c0096305>] [<c0008996>] [<c000f797>]
> > > > [<c000f991>]
> > > > DOM3: [<c0091768>] [<c000af0f>] [<c000b55f>] [<c0091a87>]
> > > > [<c0096305>] [<c002eb19>]
> > > > DOM3: [<c0018a25>] [<c0018c46>] [<c0018feb>] [<c0018ed4>]
> > > > [<c006e759>] [<c0091768>]
> > > > DOM3: [<c0007743>] [<c002c728>] [<c003e789>] [<c003e314>]
> > > > [<c002ccc7>] [<c002cf4a>]
> > > > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > > DOM3:
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Feb 09, 2004 at 05:25:00PM -0800, wrote:
> > > > > Hi All,
> > > > >
> > > > > I seem to be able to reproduce a null pointer dereference and paging
> > > > > request errors in 1.2. Can anyone give me any pointers on tracking down
> > > > > what is causing it?
> > > > >
> > > > > This is with a 32Mb virtual domain, running debian woody, NFS root,
> > > > > 256Mb swap in a local VD, while running a process which builds openldap,
> > > > > python2.2.3, and related packages. I'm not sure which package, if any
> > > > > in particular, is causing this; could be just anything that causes a
> > > > > similar workload. This particular set of messages appeared before the
> > > > > virtual domain locked up during the openldap build...
> > > > >
> > > > > Steve
> > > > >
> > > > >
> > > > > DOM26: Unable to handle kernel paging request at virtual address
> > > > > 20000001
> > > > > DOM26: printing eip:
> > > > > DOM26: c0007743
> > > > > DOM26: *pde=00000000(00000000)
> > > > > DOM26: Oops: 0000
> > > > > DOM26: CPU: 0
> > > > > DOM26: EIP: 0819:[<c0007743>] Not tainted
> > > > > DOM26: EFLAGS: 00010202
> > > > > DOM26: eax: 00000001 ebx: 20000001 ecx: c0a79e6c edx: c0a79e6c
> > > > > DOM26: esi: c0a78000 edi: c0114254 ebp: c1e5f580 esp: c0a79ce4
> > > > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > > > DOM26: Stack: 20000001 c0a78000 c002c728 c0a78000 c0a79db0 c0114254
> > > > > ffffffb0 c0a78000
> > > > > DOM26: c003e789 c0a79e6c c014b5ac c003e314 c0a79e6c 00000000
> > > > > c0114250 c0a79de8
> > > > > DOM26: c1419640 80000000 00000000 00000000 00000000 00000000
> > > > > 00000000 00000000
> > > > > DOM26: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > > > [<c002cf4a>]
> > > > > DOM26: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > > > DOM26:
> > > > > DOM26: <1>Unable to handle kernel paging request at virtual address
> > > > > 20000001
> > > > > DOM26: printing eip:
> > > > > DOM26: c000af0f
> > > > > DOM26: *pde=00000000(00000000)
> > > > > DOM26: Oops: 0002
> > > > > DOM26: CPU: 0
> > > > > DOM26: EIP: 0819:[<c000af0f>] Not tainted
> > > > > DOM26: EFLAGS: 00010282
> > > > > DOM26: eax: 20000001 ebx: c1ed5b20 ecx: c0a78264 edx: c0a78264
> > > > > DOM26: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c0a79bb4
> > > > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > > > DOM26: Stack: c1ed5b20 00000000 c0a78000 0000000b 0000000b c000b55f
> > > > > 20000001 0000001f
> > > > > DOM26: 00000000 c140d6c0 20000001 c0091a87 0000000b 00000000
> > > > > c1ed5b3c c0096305
> > > > > DOM26: c0129928 c0a79cb0 00000000 c0a78000 00000000 20000001
> > > > > c1e5f580 00000000
> > > > > DOM26: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > > > [<c0018a25>]
> > > > > DOM26: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > > > > [<c0091768>] [<c0007743>]
> > > > > DOM26: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > > > [<c002cf4a>] [<c002cf61>]
> > > > > DOM26: [<c0090033>] [<c00914bf>]
> > > > > DOM26:
> > > > > DOM26: <1>Unable to handle kernel NULL pointer dereference at virtual
> > > > > address 00000001
> > > > > DOM26: printing eip:
> > > > > DOM26: c000b623
> > > > > DOM26: *pde=00000000(00000000)
> > > > > DOM26: Oops: 0002
> > > > > DOM26: CPU: 0
> > > > > DOM26: EIP: 0819:[<c000b623>] Not tainted
> > > > > DOM26: EFLAGS: 00010202
> > > > > DOM26: eax: 00000000 ebx: 00000001 ecx: c0a78264 edx: c0a78264
> > > > > DOM26: esi: 00000002 edi: c0a78000 ebp: 0000000b esp: c0a79aa0
> > > > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > > > DOM26: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > > > > 00000000 00000002
> > > > > DOM26: c0096305 c0129928 c0a79b80 00000002 c0a78000 00000002
> > > > > 20000001 0000000b
> > > > > DOM26: 63303039 c101fc58 c0a78000 00000002 c101fc58 ffffffff
> > > > > 00030001 c001e621
> > > > > DOM26: Call Trace: [<c0091a87>] [<c0096305>] [<c001e621>] [<c001f6c0>]
> > > > > [<c0008996>]
> > > > > DOM26: [<c00200d7>] [<c00204e1>] [<c001464d>] [<c0014c92>]
> > > > > [<c0091768>] [<c000af0f>]
> > > > > DOM26: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > > > [<c0018a25>] [<c0018c46>]
> > > > > DOM26: [<c0018feb>] [<c0018ed4>] [<c006e759>] [<c0091768>]
> > > > > [<c0007743>] [<c002c728>]
> > > > > DOM26: [<c003e789>] [<c003e314>] [<c002ccc7>] [<c002cf4a>]
> > > > > [<c002cf61>] [<c0090033>]
> > > > > DOM26: [<c00914bf>]
> > > > > DOM26:
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Stephen G. Traugott (KG6HDQ)
> > > > > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > > > > stevegt@TerraLuna.Org
> > > > > http://www.stevegt.com -- http://Infrastructures.Org
> > > >
> > > > --
> > > > Stephen G. Traugott (KG6HDQ)
> > > > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > > > stevegt@TerraLuna.Org
> > > > http://www.stevegt.com -- http://Infrastructures.Org
> > >
> > > --
> > > Stephen G. Traugott (KG6HDQ)
> > > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > > stevegt@TerraLuna.Org
> > > http://www.stevegt.com -- http://Infrastructures.Org
> >
> > --
> > Stephen G. Traugott (KG6HDQ)
> > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > stevegt@TerraLuna.Org
> > http://www.stevegt.com -- http://Infrastructures.Org
> >
> >
> > -------------------------------------------------------
> > The SF.Net email is sponsored by EclipseCon 2004
> > Premiere Conference on Open Tools Development and Integration
> > See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> > http://www.eclipsecon.org/osdn
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/xen-devel
> >
>
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: 0-order allocation failed
2004-02-11 4:12 ` 0-order allocation failed stevegt
2004-02-11 4:51 ` Kip Macy
@ 2004-02-11 8:13 ` Keir Fraser
1 sibling, 0 replies; 21+ messages in thread
From: Keir Fraser @ 2004-02-11 8:13 UTC (permalink / raw)
To: stevegt; +Cc: xen-devel
This means that a number of GFP_ATOMIC allocations failed. This is no
surprise in a low-memory location where your root filesystem is
mounted over NFS: Linux isn't able to launder and evict pages quickly
enough to satisfy all non-blocking page-allocation requests.
I can practically guarantee exactly the same behaviour in native Linux
under the same circumstances (we saw various NFS-root weirdnesses on
native Linux under high load when stress-testing the MM code).
-- Keir
> Looks like yesterday's paging problems were an artifact of something on
> that particular dom0 disk image -- I haven't been able to reproduce it
> on other nodes, and after I re-imaged the same node (with the same
> image), the problem has gone away there too, so that rules out hardware.
>
> However, something else did pop up -- while trying to break things with
> "perl -e '$a="a"x100000000'", I got the following messages; do we care?
> This is in mainstream linux mm/page_alloc.c, and it's hard for me to
> tell from the code whether these are outright errors or whether they
> were recoverable. Does anyone know?
>
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
> DOM4: __alloc_pages: 0-order allocation failed (gfp=0xf0/0)
>
> This was still on last Monday's version of 1.2; I'll see if I can
> reproduce it in today's 1.2; I'm deploying that later tonight.
>
> Steve
-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: paging request failures under load (was: Re: Null pointer deference)
2004-02-10 6:34 ` stevegt
2004-02-11 4:12 ` 0-order allocation failed stevegt
@ 2004-02-11 21:23 ` stevegt
2004-02-11 22:35 ` Keir Fraser
1 sibling, 1 reply; 21+ messages in thread
From: stevegt @ 2004-02-11 21:23 UTC (permalink / raw)
To: xen-devel
They're baaaack! ;-} While upgrading a virtual domain to debian
testing so I could get a gcc > 3.0.4, it issued a couple of these oopses
and hung, resulting in a broken upgrade. This is different hardware,
same image; a machine I was totally unable to duplicate these on
yesterday. Xen 02 Feb 1.2 build, gcc 3.0.4.
Also this morning, my first production customer sent me mail saying one
of his guests was "slow". I looked, and, sure enough, he had had these
same oopses -- paging failure followed by null pointer dereference.
This was a third node, same image. Telltale symptom from his end was
that 'top' hangs -- I've noticed this in all cases.
This plus Keir's message about the clock skew (which I'm also seeing on
these guests) makes me suspect gcc 3.0.4. So here's what I'm doing:
- running under native Linux, upgrade an unmounted NFS root filesystem
to debian testing in a chroot
- still in the chroot, build today's 1.2 xen/xenolinux with gcc 3.3.2
- deploy the resulting xen and xenolinux on one or more nodes
- install ksymoops/System.map on those nodes so that we can get
meaningful oops output if it does happen again (per earlier mail from
Ian and Bin)
- test, test, test
I'll let you know how it goes.
The reason I'm doing this in a chroot is that I'm thinking of setting up
an automated Xen regression test environment under Xen, daily pulls,
that sort of thing. This NFS root would be a build server for that
environment. Is anyone already working on something like this?
Steve
On Mon, Feb 09, 2004 at 10:34:34PM -0800, wrote:
> Before anyone burns too much time on this, hang on -- I wasn't able to
> duplicate the problem on another cluster node (both nodes were built
> from the same SystemImager image). I'm looking for the reason why, and
> will let you know as soon as I do.
>
> Steve
>
> On Mon, Feb 09, 2004 at 08:13:22PM -0800, wrote:
> > Okay, the problem still exists when I bump the memory up to 256Mb, and
> > never swap. I.E. I've found no workaround. Hasn't anyone else hit
> > anything like this?
> >
> > Steve
> >
> >
> > DOM3: xen_console_init
> > DOM3: Linux version 2.4.24-xeno (stevegt@pathfinder) (gcc version
> > 3.0.4) #16 Mon Feb 2 17:46:41 PST 2004
> > DOM3: On node 0 totalpages: 65536
> > DOM3: zone(0): 4096 pages.
> > DOM3: zone(1): 61440 pages.
> > DOM3: zone(2): 0 pages.
> > DOM3: Kernel command line:
> > ip=64.71.149.20:10.27.2.50:64.71.149.1:255.255.255.0::eth0:off
> > root=/dev/nfs nfsroot=/export//xen/fs/stevegt/tcx/root 4 DOMID=20
> > DOM3: Initializing CPU#0
> > DOM3: Xen reported: 398.780 MHz processor.
> > DOM3: Calibrating delay loop... 1592.52 BogoMIPS
> > DOM3: Memory: 257132k/262144k available (1078k kernel code, 5012k
> > reserved, 308k data, 52k init, 0k highmem)
> > DOM3: Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
> > DOM3: Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
> > DOM3: Mount cache hash table entries: 512 (order: 0, 4096 bytes)
> > DOM3: Buffer cache hash table entries: 16384 (order: 4, 65536 bytes)
> > DOM3: Page-cache hash table entries: 65536 (order: 6, 262144 bytes)
> > DOM3: CPU: L1 I cache: 16K, L1 D cache: 16K
> > DOM3: CPU: L2 cache: 512K
> > DOM3: CPU: Intel Pentium II (Deschutes) stepping 01
> > DOM3: POSIX conformance testing by UNIFIX
> > DOM3: Linux NET4.0 for Linux 2.4
> > DOM3: Based upon Swansea University Computer Society NET3.039
> > DOM3: Initializing RT netlink socket
> > DOM3: Starting kswapd
> > DOM3: Journalled Block Device driver loaded
> > DOM3: Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
> > DOM3: Xeno console successfully installed
> > DOM3: Starting Xeno Balloon driver
> > DOM3: pty: 256 Unix98 ptys configured
> > DOM3: RAMDISK driver initialized: 16 RAM disks of 4096K size 1024
> > blocksize
> > DOM3: loop: loaded (max 8 devices)
> > DOM3: NET4: Linux TCP/IP 1.0 for NET4.0
> > DOM3: IP Protocols: ICMP, UDP, TCP
> > DOM3: IP: routing cache hash table of 2048 buckets, 16Kbytes
> > DOM3: TCP: Hash tables configured (established 16384 bind 16384)
> > DOM3: IP-Config: Complete:
> > DOM3: device=eth0, addr=64.71.149.20, mask=255.255.255.0,
> > gw=64.71.149.1,
> > DOM3: host=64.71.149.20, domain=, nis-domain=(none),
> > DOM3: bootserver=10.27.2.50, rootserver=10.27.2.50, rootpath=
> > DOM3: ip_conntrack version 2.1 (2048 buckets, 16384 max) - 292 bytes
> > per conntrack
> > DOM3: ip_tables: (C) 2000-2002 Netfilter core team
> > DOM3: NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> > DOM3: Looking up port of RPC 100003/2 on 10.27.2.50
> > DOM3: Looking up port of RPC 100005/1 on 10.27.2.50
> > DOM3: VFS: Mounted root (nfs filesystem).
> > DOM3: Freeing unused kernel memory: 52k freed
> > DOM3: INIT: version 2.84 booting
> > DOM3: Activating swap.
> > DOM3: Adding Swap: 262136k swap-space (priority -1)
> > DOM3: Checking root file system...
> > DOM3: fsck 1.27 (8-Mar-2002)
> > DOM3: 10.27.2.50:/export/xen/fs/stevegt/tcx: NFS file system.
> > DOM3: System time was Tue Feb 10 02:14:36 UTC 2004.
> > DOM3: Setting the System Clock using the Hardware Clock as
> > reference...
> > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > DOM3: hwclock is unable to get I/O port access: the iopl(3) call
> > failed.
> > DOM3: modprobe: modprobe: Can't locate module char-major-10-135
> > DOM3: modprobe: modprobe: Can't locate module char-major-4
> > DOM3: System Clock set. System local time is now Tue Feb 10 02:14:36
> > UTC 2004.
> > DOM3: Calculating module dependencies... depmod: cannot read ELF
> > header from /lib/modules/2.4.24-xeno/modules.dep
> > DOM3: depmod: cannot read ELF header from
> > /lib/modules/2.4.24-xeno/modules.generic_string
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.ieee1394map is not an
> > ELF file
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.isapnpmap is not an ELF
> > file
> > DOM3: depmod: cannot read ELF header from
> > /lib/modules/2.4.24-xeno/modules.parportmap
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.pcimap is not an ELF
> > file
> > DOM3: depmod: cannot read ELF header from
> > /lib/modules/2.4.24-xeno/modules.pnpbiosmap
> > DOM3: depmod: /lib/modules/2.4.24-xeno/modules.usbmap is not an ELF
> > file
> > DOM3: done.
> > DOM3: Loading modules:
> > DOM3: Checking all file systems...
> > DOM3: fsck 1.27 (8-Mar-2002)
> > DOM3: Setting kernel variables.
> > DOM3: Loading the saved-state of the serial devices...
> > DOM3: Mounting local filesystems...
> > DOM3: nothing was mounted
> > DOM3: Running 0dns-down to make sure resolv.conf is ok...done.
> > DOM3: Cleaning: /etc/network/ifstate.
> > DOM3: Setting up IP spoofing protection: rp_filter.
> > DOM3: Configuring network interfaces: done.
> > DOM3: Mounting remote filesystems...
> > DOM3:
> > DOM3: Setting the System Clock using the Hardware Clock as
> > reference...
> > DOM3: System Clock set. Local time: Tue Feb 10 02:14:37 UTC 2004
> > DOM3:
> > DOM3: Cleaning: /tmp /var/lock /var/run.
> > DOM3: Initializing random number generator... done.
> > DOM3: Recovering nvi editor sessions... done.
> > DOM3: INIT: Entering runlevel: 4
> > DOM3: Starting system log daemon: syslogd.
> > DOM3: Starting kernel log daemon: klogd.
> > DOM3: Starting internet superserver: inetd.
> > DOM3: Starting PCMCIA services: module directory
> > /lib/modules/2.4.24-xeno/pcmcia not found.
> > DOM3: Starting OpenBSD Secure Shell server: sshd.
> > DOM3: Starting deferred execution scheduler: atd.
> > DOM3: Starting periodic command scheduler: cron.
> > DOM3: INIT: no more processes left in this runlevel
> > DOM3: Unable to handle kernel paging request at virtual address
> > 20000001
> > DOM3: printing eip:
> > DOM3: c0007743
> > DOM3: *pde=00000000(00000000)
> > DOM3: Oops: 0000
> > DOM3: CPU: 0
> > DOM3: EIP: 0819:[<c0007743>] Not tainted
> > DOM3: EFLAGS: 00010202
> > DOM3: eax: 00000001 ebx: 20000001 ecx: c3ebde6c edx: c3ebde6c
> > DOM3: esi: c3ebc000 edi: c0114254 ebp: c46c1060 esp: c3ebdce4
> > DOM3: ds: 0821 es: 0821 ss: 0821
> > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > DOM3: Stack: 20000001 c3ebc000 c002c728 c3ebc000 c3ebddb0 c0114254
> > ffffffb0 c3ebc000
> > DOM3: c003e789 c3ebde6c c014b5ac c003e314 c3ebde6c 00000000
> > c0114250 00000000
> > DOM3: 00000000 00000000 01082003 8d588810 00000000 00000000
> > 00000000 00000000
> > DOM3: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > [<c002cf4a>]
> > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > DOM3:
> > DOM3: <1>Unable to handle kernel paging request at virtual address
> > 20000001
> > DOM3: printing eip:
> > DOM3: c000af0f
> > DOM3: *pde=00000000(00000000)
> > DOM3: Oops: 0002
> > DOM3: CPU: 0
> > DOM3: EIP: 0819:[<c000af0f>] Not tainted
> > DOM3: EFLAGS: 00010282
> > DOM3: eax: 20000001 ebx: c485c0a0 ecx: c3ebc264 edx: c3ebc264
> > DOM3: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c3ebdbb4
> > DOM3: ds: 0821 es: 0821 ss: 0821
> > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > DOM3: Stack: c485c0a0 00000000 c3ebc000 0000000b 0000000b c000b55f
> > 20000001 0000001f
> > DOM3: 00000000 cf4227e0 20000001 c0091a87 0000000b 00000000
> > c485c0bc c0096305
> > DOM3: c0129928 c3ebdcb0 00000000 c3ebc000 00000000 20000001
> > c46c1060 00000000
> > DOM3: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > [<c0018a25>]
> > DOM3: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > [<c0091768>] [<c0007743>]
> > DOM3: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > [<c002cf4a>] [<c002cf61>]
> > DOM3: [<c0090033>] [<c00914bf>]
> > DOM3:
> > DOM3: <1>Unable to handle kernel NULL pointer dereference at virtual
> > address 00000001
> > DOM3: printing eip:
> > DOM3: c000b623
> > DOM3: *pde=00000000(00000000)
> > DOM3: Oops: 0002
> > DOM3: CPU: 0
> > DOM3: EIP: 0819:[<c000b623>] Not tainted
> > DOM3: EFLAGS: 00010202
> > DOM3: eax: 00000000 ebx: 00000001 ecx: c3ebc264 edx: c3ebc264
> > DOM3: esi: 00000002 edi: c3ebc000 ebp: 0000000b esp: c3ebdaa0
> > DOM3: ds: 0821 es: 0821 ss: 0821
> > DOM3: Process cc (pid: 13793, stackpage=c3ebd000)<1>
> > DOM3: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > 00000000 00000002
> > DOM3: c0096305 c0129928 c3ebdb80 00000002 c3ebc000 00000002
> > 20000001 0000000b
> > DOM3: 63303039 ffffffff c3ebc000 00000002 38383130 38643538
> > 00030001 64303030
> > DOM3: Call Trace: [<c0091a87>] [<c0096305>] [<c0008996>] [<c000f797>]
> > [<c000f991>]
> > DOM3: [<c0091768>] [<c000af0f>] [<c000b55f>] [<c0091a87>]
> > [<c0096305>] [<c002eb19>]
> > DOM3: [<c0018a25>] [<c0018c46>] [<c0018feb>] [<c0018ed4>]
> > [<c006e759>] [<c0091768>]
> > DOM3: [<c0007743>] [<c002c728>] [<c003e789>] [<c003e314>]
> > [<c002ccc7>] [<c002cf4a>]
> > DOM3: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > DOM3:
> >
> >
> >
> >
> > On Mon, Feb 09, 2004 at 05:25:00PM -0800, wrote:
> > > Hi All,
> > >
> > > I seem to be able to reproduce a null pointer dereference and paging
> > > request errors in 1.2. Can anyone give me any pointers on tracking down
> > > what is causing it?
> > >
> > > This is with a 32Mb virtual domain, running debian woody, NFS root,
> > > 256Mb swap in a local VD, while running a process which builds openldap,
> > > python2.2.3, and related packages. I'm not sure which package, if any
> > > in particular, is causing this; could be just anything that causes a
> > > similar workload. This particular set of messages appeared before the
> > > virtual domain locked up during the openldap build...
> > >
> > > Steve
> > >
> > >
> > > DOM26: Unable to handle kernel paging request at virtual address
> > > 20000001
> > > DOM26: printing eip:
> > > DOM26: c0007743
> > > DOM26: *pde=00000000(00000000)
> > > DOM26: Oops: 0000
> > > DOM26: CPU: 0
> > > DOM26: EIP: 0819:[<c0007743>] Not tainted
> > > DOM26: EFLAGS: 00010202
> > > DOM26: eax: 00000001 ebx: 20000001 ecx: c0a79e6c edx: c0a79e6c
> > > DOM26: esi: c0a78000 edi: c0114254 ebp: c1e5f580 esp: c0a79ce4
> > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > DOM26: Stack: 20000001 c0a78000 c002c728 c0a78000 c0a79db0 c0114254
> > > ffffffb0 c0a78000
> > > DOM26: c003e789 c0a79e6c c014b5ac c003e314 c0a79e6c 00000000
> > > c0114250 c0a79de8
> > > DOM26: c1419640 80000000 00000000 00000000 00000000 00000000
> > > 00000000 00000000
> > > DOM26: Call Trace: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > [<c002cf4a>]
> > > DOM26: [<c002cf61>] [<c0090033>] [<c00914bf>]
> > > DOM26:
> > > DOM26: <1>Unable to handle kernel paging request at virtual address
> > > 20000001
> > > DOM26: printing eip:
> > > DOM26: c000af0f
> > > DOM26: *pde=00000000(00000000)
> > > DOM26: Oops: 0002
> > > DOM26: CPU: 0
> > > DOM26: EIP: 0819:[<c000af0f>] Not tainted
> > > DOM26: EFLAGS: 00010282
> > > DOM26: eax: 20000001 ebx: c1ed5b20 ecx: c0a78264 edx: c0a78264
> > > DOM26: esi: 00000000 edi: 20000001 ebp: 0000000b esp: c0a79bb4
> > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > DOM26: Stack: c1ed5b20 00000000 c0a78000 0000000b 0000000b c000b55f
> > > 20000001 0000001f
> > > DOM26: 00000000 c140d6c0 20000001 c0091a87 0000000b 00000000
> > > c1ed5b3c c0096305
> > > DOM26: c0129928 c0a79cb0 00000000 c0a78000 00000000 20000001
> > > c1e5f580 00000000
> > > DOM26: Call Trace: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > [<c0018a25>]
> > > DOM26: [<c0018c46>] [<c0018feb>] [<c0018ed4>] [<c006e759>]
> > > [<c0091768>] [<c0007743>]
> > > DOM26: [<c002c728>] [<c003e789>] [<c003e314>] [<c002ccc7>]
> > > [<c002cf4a>] [<c002cf61>]
> > > DOM26: [<c0090033>] [<c00914bf>]
> > > DOM26:
> > > DOM26: <1>Unable to handle kernel NULL pointer dereference at virtual
> > > address 00000001
> > > DOM26: printing eip:
> > > DOM26: c000b623
> > > DOM26: *pde=00000000(00000000)
> > > DOM26: Oops: 0002
> > > DOM26: CPU: 0
> > > DOM26: EIP: 0819:[<c000b623>] Not tainted
> > > DOM26: EFLAGS: 00010202
> > > DOM26: eax: 00000000 ebx: 00000001 ecx: c0a78264 edx: c0a78264
> > > DOM26: esi: 00000002 edi: c0a78000 ebp: 0000000b esp: c0a79aa0
> > > DOM26: ds: 0821 es: 0821 ss: 0821
> > > DOM26: Process sh (pid: 10086, stackpage=c0a79000)<1>
> > > DOM26: Stack: 0000001f 00000002 20000001 20000001 c0091a87 0000000b
> > > 00000000 00000002
> > > DOM26: c0096305 c0129928 c0a79b80 00000002 c0a78000 00000002
> > > 20000001 0000000b
> > > DOM26: 63303039 c101fc58 c0a78000 00000002 c101fc58 ffffffff
> > > 00030001 c001e621
> > > DOM26: Call Trace: [<c0091a87>] [<c0096305>] [<c001e621>] [<c001f6c0>]
> > > [<c0008996>]
> > > DOM26: [<c00200d7>] [<c00204e1>] [<c001464d>] [<c0014c92>]
> > > [<c0091768>] [<c000af0f>]
> > > DOM26: [<c000b55f>] [<c0091a87>] [<c0096305>] [<c002eb19>]
> > > [<c0018a25>] [<c0018c46>]
> > > DOM26: [<c0018feb>] [<c0018ed4>] [<c006e759>] [<c0091768>]
> > > [<c0007743>] [<c002c728>]
> > > DOM26: [<c003e789>] [<c003e314>] [<c002ccc7>] [<c002cf4a>]
> > > [<c002cf61>] [<c0090033>]
> > > DOM26: [<c00914bf>]
> > > DOM26:
> > >
> > >
> > >
> > > --
> > > Stephen G. Traugott (KG6HDQ)
> > > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > > stevegt@TerraLuna.Org
> > > http://www.stevegt.com -- http://Infrastructures.Org
> >
> > --
> > Stephen G. Traugott (KG6HDQ)
> > UNIX/Linux Infrastructure Architect, TerraLuna LLC
> > stevegt@TerraLuna.Org
> > http://www.stevegt.com -- http://Infrastructures.Org
>
> --
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org
> http://www.stevegt.com -- http://Infrastructures.Org
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Re: paging request failures under load (was: Re: Null pointer deference)
2004-02-11 21:23 ` paging request failures under load (was: Re: Null pointer deference) stevegt
@ 2004-02-11 22:35 ` Keir Fraser
2004-02-21 6:23 ` Paging oopses stevegt
0 siblings, 1 reply; 21+ messages in thread
From: Keir Fraser @ 2004-02-11 22:35 UTC (permalink / raw)
To: stevegt; +Cc: xen-devel
> - install ksymoops/System.map on those nodes so that we can get
> meaningful oops output if it does happen again (per earlier mail from
> Ian and Bin)
Number-one priority for debugging is having access to the kernel
object file (it's the 'vmlinux' file at the root of the build
tree). Given that, and the precise version of Xen/Xenolinux that you
built, I can have a fair stab at unpicking what happened. If the crash
is in Xen itself then the Xen image file is what I need ('xen' file at
teh root of the Xen build tree).
Symbolic backtraces are nice but definitely of secondary importance.
> The reason I'm doing this in a chroot is that I'm thinking of setting up
> an automated Xen regression test environment under Xen, daily pulls,
> that sort of thing. This NFS root would be a build server for that
> environment. Is anyone already working on something like this?
We have a regression test here in the lab, but:
1. It uses some SPEC benchmarks, so it's not publically distributable.
2. It's based on an old Redhat -- a more up-to-date filesystem would
be good.
3. We don't have enough spare machines to do a really large test.
Your setup sounds liek it could be much better!
-- Keir
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Paging oopses
2004-02-11 22:35 ` Keir Fraser
@ 2004-02-21 6:23 ` stevegt
2004-02-21 6:31 ` stevegt
2004-02-21 6:52 ` stevegt
0 siblings, 2 replies; 21+ messages in thread
From: stevegt @ 2004-02-21 6:23 UTC (permalink / raw)
To: Keir Fraser; +Cc: xen-devel
Hi All, bad news.
After running several guests for most of the past week on the 12 Feb
build of 1.2, built with GCC 3.3.2, with 64Mb of RAM and NFS roots, I
finally got another paging oops. The entire console log, xen and
xenolinux binaries, and System.map, are at:
http://t7a.org/tmp/oops1-n2h54/
Let me know if there's anything else I can do.
Steve
On Wed, Feb 11, 2004 at 10:35:15PM +0000, Keir Fraser wrote:
> > - install ksymoops/System.map on those nodes so that we can get
> > meaningful oops output if it does happen again (per earlier mail from
> > Ian and Bin)
>
> Number-one priority for debugging is having access to the kernel
> object file (it's the 'vmlinux' file at the root of the build
> tree). Given that, and the precise version of Xen/Xenolinux that you
> built, I can have a fair stab at unpicking what happened. If the crash
> is in Xen itself then the Xen image file is what I need ('xen' file at
> teh root of the Xen build tree).
>
> Symbolic backtraces are nice but definitely of secondary importance.
>
> > The reason I'm doing this in a chroot is that I'm thinking of setting up
> > an automated Xen regression test environment under Xen, daily pulls,
> > that sort of thing. This NFS root would be a build server for that
> > environment. Is anyone already working on something like this?
>
> We have a regression test here in the lab, but:
> 1. It uses some SPEC benchmarks, so it's not publically distributable.
> 2. It's based on an old Redhat -- a more up-to-date filesystem would
> be good.
> 3. We don't have enough spare machines to do a really large test.
>
> Your setup sounds liek it could be much better!
>
> -- Keir
>
>
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!
> http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Paging oopses
2004-02-21 6:23 ` Paging oopses stevegt
@ 2004-02-21 6:31 ` stevegt
2004-02-21 6:52 ` stevegt
1 sibling, 0 replies; 21+ messages in thread
From: stevegt @ 2004-02-21 6:31 UTC (permalink / raw)
To: Keir Fraser; +Cc: xen-devel
Some thoughts: the only thing I can think of that I might be doing
differently than any of you is that I'm running the NFS root server on
another node, a standard Linux machine, with the traffic going through a
lightly-loaded 100Mb switch. Nevertheless, I still get occasional NFS
timeouts, as you'll see in the console log. These haven't worried me,
but if you find that these paging oopses are happening in mmap code,
that might be worth noting.
Steve
On Fri, Feb 20, 2004 at 10:23:26PM -0800, wrote:
> Hi All, bad news.
>
> After running several guests for most of the past week on the 12 Feb
> build of 1.2, built with GCC 3.3.2, with 64Mb of RAM and NFS roots, I
> finally got another paging oops. The entire console log, xen and
> xenolinux binaries, and System.map, are at:
>
> http://t7a.org/tmp/oops1-n2h54/
>
> Let me know if there's anything else I can do.
>
> Steve
>
>
> On Wed, Feb 11, 2004 at 10:35:15PM +0000, Keir Fraser wrote:
> > > - install ksymoops/System.map on those nodes so that we can get
> > > meaningful oops output if it does happen again (per earlier mail from
> > > Ian and Bin)
> >
> > Number-one priority for debugging is having access to the kernel
> > object file (it's the 'vmlinux' file at the root of the build
> > tree). Given that, and the precise version of Xen/Xenolinux that you
> > built, I can have a fair stab at unpicking what happened. If the crash
> > is in Xen itself then the Xen image file is what I need ('xen' file at
> > teh root of the Xen build tree).
> >
> > Symbolic backtraces are nice but definitely of secondary importance.
> >
> > > The reason I'm doing this in a chroot is that I'm thinking of setting up
> > > an automated Xen regression test environment under Xen, daily pulls,
> > > that sort of thing. This NFS root would be a build server for that
> > > environment. Is anyone already working on something like this?
> >
> > We have a regression test here in the lab, but:
> > 1. It uses some SPEC benchmarks, so it's not publically distributable.
> > 2. It's based on an old Redhat -- a more up-to-date filesystem would
> > be good.
> > 3. We don't have enough spare machines to do a really large test.
> >
> > Your setup sounds liek it could be much better!
> >
> > -- Keir
> >
> >
> > -------------------------------------------------------
> > SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> > Build and deploy apps & Web services for Linux with
> > a free DVD software kit from IBM. Click Now!
> > http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/xen-devel
>
> --
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org
> http://www.stevegt.com -- http://Infrastructures.Org
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Paging oopses
2004-02-21 6:23 ` Paging oopses stevegt
2004-02-21 6:31 ` stevegt
@ 2004-02-21 6:52 ` stevegt
2004-02-21 8:31 ` Keir Fraser
1 sibling, 1 reply; 21+ messages in thread
From: stevegt @ 2004-02-21 6:52 UTC (permalink / raw)
To: Keir Fraser; +Cc: xen-devel
Aargh! Please disregard. While processing ksymoops just now I realized
that the two guest machines that had these oopses were in fact the
*only* two that I hadn't yet upgraded to the 12 Feb/GCC 3.3.2 xenolinux.
These were running the new Xen, old xenolinux. All others have been
fine, with a week of runtime so far.
Steve
On Fri, Feb 20, 2004 at 10:23:26PM -0800, wrote:
> Hi All, bad news.
>
> After running several guests for most of the past week on the 12 Feb
> build of 1.2, built with GCC 3.3.2, with 64Mb of RAM and NFS roots, I
> finally got another paging oops. The entire console log, xen and
> xenolinux binaries, and System.map, are at:
>
> http://t7a.org/tmp/oops1-n2h54/
>
> Let me know if there's anything else I can do.
>
> Steve
>
>
> On Wed, Feb 11, 2004 at 10:35:15PM +0000, Keir Fraser wrote:
> > > - install ksymoops/System.map on those nodes so that we can get
> > > meaningful oops output if it does happen again (per earlier mail from
> > > Ian and Bin)
> >
> > Number-one priority for debugging is having access to the kernel
> > object file (it's the 'vmlinux' file at the root of the build
> > tree). Given that, and the precise version of Xen/Xenolinux that you
> > built, I can have a fair stab at unpicking what happened. If the crash
> > is in Xen itself then the Xen image file is what I need ('xen' file at
> > teh root of the Xen build tree).
> >
> > Symbolic backtraces are nice but definitely of secondary importance.
> >
> > > The reason I'm doing this in a chroot is that I'm thinking of setting up
> > > an automated Xen regression test environment under Xen, daily pulls,
> > > that sort of thing. This NFS root would be a build server for that
> > > environment. Is anyone already working on something like this?
> >
> > We have a regression test here in the lab, but:
> > 1. It uses some SPEC benchmarks, so it's not publically distributable.
> > 2. It's based on an old Redhat -- a more up-to-date filesystem would
> > be good.
> > 3. We don't have enough spare machines to do a really large test.
> >
> > Your setup sounds liek it could be much better!
> >
> > -- Keir
> >
> >
> > -------------------------------------------------------
> > SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> > Build and deploy apps & Web services for Linux with
> > a free DVD software kit from IBM. Click Now!
> > http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/xen-devel
>
> --
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org
> http://www.stevegt.com -- http://Infrastructures.Org
--
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org
http://www.stevegt.com -- http://Infrastructures.Org
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Paging oopses
2004-02-21 6:52 ` stevegt
@ 2004-02-21 8:31 ` Keir Fraser
2004-02-21 22:00 ` Kip Macy
0 siblings, 1 reply; 21+ messages in thread
From: Keir Fraser @ 2004-02-21 8:31 UTC (permalink / raw)
To: stevegt; +Cc: xen-devel
> Aargh! Please disregard. While processing ksymoops just now I realized
> that the two guest machines that had these oopses were in fact the
> *only* two that I hadn't yet upgraded to the 12 Feb/GCC 3.3.2 xenolinux.
> These were running the new Xen, old xenolinux. All others have been
> fine, with a week of runtime so far.
Phew. :-)
Of course, we're interested in any crashes that occur when using GCC
2.95.3 and 3.3.x. We'll also accept crash dumps from 3.2.2 -- I've
heard that this compiler has trouble with Linux in some cases, but
since it's the compiler that we use the most (it ships with RH9), we
have some degree of trust in it!
I've taken a lot of time in the last week to shake bugs out of
Xen/Xenolinux 1.2. Hopefully this will reduce the number of bug
reports, despite the recent upgrade to linux-2.4.25.
-- Keir
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Re: Paging oopses
2004-02-21 8:31 ` Keir Fraser
@ 2004-02-21 22:00 ` Kip Macy
2004-02-21 22:55 ` Mandrake-9.1 and Xen I RATTAN
0 siblings, 1 reply; 21+ messages in thread
From: Kip Macy @ 2004-02-21 22:00 UTC (permalink / raw)
To: Keir Fraser; +Cc: xen-devel
While we're on the topic of compiler idiosyncracies, I'm wondering if it
really makes sense to compile the tools with -O3. They're not
performance critical, so it seems like the risks might outweigh the
gains.
-Kip
On Sat, 21 Feb 2004, Keir Fraser wrote:
> > Aargh! Please disregard. While processing ksymoops just now I realized
> > that the two guest machines that had these oopses were in fact the
> > *only* two that I hadn't yet upgraded to the 12 Feb/GCC 3.3.2 xenolinux.
> > These were running the new Xen, old xenolinux. All others have been
> > fine, with a week of runtime so far.
>
> Phew. :-)
>
> Of course, we're interested in any crashes that occur when using GCC
> 2.95.3 and 3.3.x. We'll also accept crash dumps from 3.2.2 -- I've
> heard that this compiler has trouble with Linux in some cases, but
> since it's the compiler that we use the most (it ships with RH9), we
> have some degree of trust in it!
>
> I've taken a lot of time in the last week to shake bugs out of
> Xen/Xenolinux 1.2. Hopefully this will reduce the number of bug
> reports, despite the recent upgrade to linux-2.4.25.
>
> -- Keir
>
>
>
>
>
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!
> http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Mandrake-9.1 and Xen..
2004-02-21 22:00 ` Kip Macy
@ 2004-02-21 22:55 ` I RATTAN
2004-02-21 23:15 ` Ian Pratt
0 siblings, 1 reply; 21+ messages in thread
From: I RATTAN @ 2004-02-21 22:55 UTC (permalink / raw)
To: xen-devel
I have dual P3/750Mhz box running Mandrake-9.1. I have downloaded
the Xen/Xenolinux 1.3 stuff and compiled. It comes out that I can't
the kernel configuration right -- I was able to compile xen-1.2 and
xenolinux-2.4.24 but when the machine is booted (using a grub floppy)
it starts boot process and then the box keeps on resetting itslef.
My guess: xen boots correctly but xenolinux goes haywire.
Can some kind soul give me a copy of '.config' of a correctly booting
system?
Thanks in advance,
-ishwar
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Mandrake-9.1 and Xen..
2004-02-21 22:55 ` Mandrake-9.1 and Xen I RATTAN
@ 2004-02-21 23:15 ` Ian Pratt
2004-02-23 1:26 ` I RATTAN
0 siblings, 1 reply; 21+ messages in thread
From: Ian Pratt @ 2004-02-21 23:15 UTC (permalink / raw)
To: I RATTAN; +Cc: xen-devel, Ian.Pratt
> I have dual P3/750Mhz box running Mandrake-9.1. I have downloaded
> the Xen/Xenolinux 1.3 stuff and compiled. It comes out that I can't
> the kernel configuration right -- I was able to compile xen-1.2 and
> xenolinux-2.4.24 but when the machine is booted (using a grub floppy)
> it starts boot process and then the box keeps on resetting itslef.
> My guess: xen boots correctly but xenolinux goes haywire.
>
> Can some kind soul give me a copy of '.config' of a correctly booting
> system?
There's a default config (defconfig) in the xenolinux tree.
Just type "make mrproper oldconfig dep bzImage"
['mrproper' deletes the previous .config, 'oldconfig' copies the
default arch xeno config to .config.]
If you want to stop it rebooting if something goes wrong, use the
'noreboot' xen command line option. You should then be able to
examine the error message. Alternatively, connect a serial
console and enable serial output with the 'ser_baud' xen command
line option.
Ian
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Mandrake-9.1 and Xen..
2004-02-21 23:15 ` Ian Pratt
@ 2004-02-23 1:26 ` I RATTAN
2004-02-23 1:39 ` Kip Macy
2004-02-23 2:13 ` Ian Pratt
0 siblings, 2 replies; 21+ messages in thread
From: I RATTAN @ 2004-02-23 1:26 UTC (permalink / raw)
To: xen-devel
On Sat, 21 Feb 2004, Ian Pratt wrote:
> > I have dual P3/750Mhz box running Mandrake-9.1. I have downloaded
> > system?
>
> There's a default config (defconfig) in the xenolinux tree.
What it it's location relative to xenolinux-2.4.24?
-ishwar
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Mandrake-9.1 and Xen..
2004-02-23 1:26 ` I RATTAN
@ 2004-02-23 1:39 ` Kip Macy
2004-02-23 2:13 ` Ian Pratt
1 sibling, 0 replies; 21+ messages in thread
From: Kip Macy @ 2004-02-23 1:39 UTC (permalink / raw)
To: I RATTAN; +Cc: xen-devel
just do
> ARCH=xeno make oldconfig
On Sun, 22 Feb 2004, I RATTAN wrote:
>
>
> On Sat, 21 Feb 2004, Ian Pratt wrote:
>
> > > I have dual P3/750Mhz box running Mandrake-9.1. I have downloaded
> > > system?
> >
> > There's a default config (defconfig) in the xenolinux tree.
> What it it's location relative to xenolinux-2.4.24?
>
> -ishwar
>
>
> -------------------------------------------------------
> SF.Net is sponsored by: Speed Start Your Linux Apps Now.
> Build and deploy apps & Web services for Linux with
> a free DVD software kit from IBM. Click Now!
> http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Mandrake-9.1 and Xen..
2004-02-23 1:26 ` I RATTAN
2004-02-23 1:39 ` Kip Macy
@ 2004-02-23 2:13 ` Ian Pratt
1 sibling, 0 replies; 21+ messages in thread
From: Ian Pratt @ 2004-02-23 2:13 UTC (permalink / raw)
To: I RATTAN; +Cc: xen-devel, Ian.Pratt
>
>
> On Sat, 21 Feb 2004, Ian Pratt wrote:
>
> > > I have dual P3/750Mhz box running Mandrake-9.1. I have downloaded
> > > system?
> >
> > There's a default config (defconfig) in the xenolinux tree.
> What it it's location relative to xenolinux-2.4.24?
It will be generated by running 'make mrproper' 'make oldconfig'.
Alternatively, copy it from arch/xeno/defconfig
Ian
-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
^ permalink raw reply [flat|nested] 21+ messages in thread
* 0-order allocation failed
@ 2004-07-13 0:33 Simon Matthews
0 siblings, 0 replies; 21+ messages in thread
From: Simon Matthews @ 2004-07-13 0:33 UTC (permalink / raw)
To: linux-kernel
My system keeps locking up with this error message.
I am trying to copy from a SCSI disk to an IDE disk when this happens.
What I see is that all the memory is shown as used, and "cached". The
machine has 4GB of memory, which should be sufficient for a mere cp -a.
The IDE disk is a WD 200GB disk on a Promise Ultra133 TX2 controller. I
have enabled PROMISE PDC202{68|69|70|71|75|76|77} support in my kernel
(2.4.24 with the 3.5GB VM patch and 3.5GB of memory enabled in the kernel
config)
The cached memory appears to be about twice the amount of data that has
been copied. dmesg showed nothing until the machine locked up.
Any suggestions?
>From the terminal in which I was running "top":
5:15pm up 19 min, 0 users, load average: 2.91, 1.52, 0.72
Segmentation faultleeping, 5 running, 0 zombie, 0 stopped
[simon@kilea ~]$ % user, 92.0% system, 0.0% nice, 7.5% idle
CPU1 states: 0.0% user, 74.4% system, 0.0% nice, 25.0% idle
Mem: 3751712K av, 3653832K used, 97880K free, 0K shrd, 260K
buff
Swap: 4145120K av, 0K used, 4145120K free 3289248K
cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
1139 root 19 0 6464 6464 480 R 63.3 0.1 0:50 cp
5 root 20 0 0 0 0 RW 36.7 0.0 0:34 kswapd
11 root 20 0 0 0 0 RW 36.7 0.0 0:02 kjournald
6 root 20 0 0 0 0 RW 24.1 0.0 0:17 bdflush
7 root 9 0 0 0 0 SW 2.3 0.0 0:04 kupdated
The results of "free" just before the lockup:
[simon@kilea ~]$ free
total used free shared buffers cached
Mem: 3751712 3628700 123012 0 344 3265952
-/+ buffers/cache: 362404 3389308
Swap: 4145120 0 4145120
dmesg, showing boot up:
]# dmesg
1-9 -> 0x71 -> IRQ 9 Mode:1 Active:0)
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: System [ACPI] (supports S0 S1 S4 S5)
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
Transparent bridge - Intel Corp. 82801BA/CA/DB/EB PCI Bridge
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.AGP_._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.HLB_.P64H._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.SLOT._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 *10 11 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 10 *11 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 9 10 11 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 10 11 14 15)
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 *11 14 15)
ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 10 11 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 10 *11 14 15)
PCI: Probing PCI hardware
IOAPIC[0]: Set PCI routing entry (1-16 -> 0xa9 -> IRQ 16 Mode:1 Active:1)
00:00:1f[A] -> 1-16 -> IRQ 16
IOAPIC[0]: Set PCI routing entry (1-17 -> 0xb1 -> IRQ 17 Mode:1 Active:1)
00:00:1f[B] -> 1-17 -> IRQ 17
IOAPIC[0]: Set PCI routing entry (1-18 -> 0xb9 -> IRQ 18 Mode:1 Active:1)
00:00:1f[C] -> 1-18 -> IRQ 18
IOAPIC[0]: Set PCI routing entry (1-19 -> 0xc1 -> IRQ 19 Mode:1 Active:1)
00:00:1f[D] -> 1-19 -> IRQ 19
Pin 1-16 already programmed
Pin 1-17 already programmed
IOAPIC[0]: Set PCI routing entry (1-22 -> 0xc9 -> IRQ 22 Mode:1 Active:1)
00:03:04[A] -> 1-22 -> IRQ 22
Pin 1-22 already programmed
Pin 1-22 already programmed
Pin 1-22 already programmed
Pin 1-22 already programmed
Pin 1-22 already programmed
Pin 1-22 already programmed
Pin 1-22 already programmed
IOAPIC[0]: Set PCI routing entry (1-23 -> 0xd1 -> IRQ 23 Mode:1 Active:1)
00:03:08[A] -> 1-23 -> IRQ 23
Pin 1-19 already programmed
Pin 1-17 already programmed
Pin 1-18 already programmed
Pin 1-19 already programmed
IOAPIC[0]: Set PCI routing entry (1-20 -> 0xd9 -> IRQ 20 Mode:1 Active:1)
00:04:0a[D] -> 1-20 -> IRQ 20
Pin 1-18 already programmed
Pin 1-19 already programmed
Pin 1-20 already programmed
IOAPIC[0]: Set PCI routing entry (1-21 -> 0xe1 -> IRQ 21 Mode:1 Active:1)
00:04:06[D] -> 1-21 -> IRQ 21
Pin 1-19 already programmed
Pin 1-20 already programmed
Pin 1-21 already programmed
Pin 1-22 already programmed
Pin 1-21 already programmed
number of MP IRQ sources: 15.
number of IO-APIC #1 registers: 24.
number of IO-APIC #2 registers: 24.
testing the IO APIC.......................
IO APIC #1......
.... register #00: 01008000
....... : physical APIC id: 01
....... : Delivery Type: 1
....... : LTS : 0
.... register #01: 00178020
....... : max redirection entries: 0017
....... : PRQ implemented: 1
....... : IO APIC version: 0020
.... register #02: 00000000
....... : arbitration: 00
.... register #03: 00000001
....... : Boot DT : 1
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 000 00 1 0 0 0 0 0 0 00
01 003 03 0 0 0 0 0 1 1 39
02 003 03 0 0 0 0 0 1 1 31
03 003 03 0 0 0 0 0 1 1 41
04 003 03 0 0 0 0 0 1 1 49
05 003 03 0 0 0 0 0 1 1 51
06 003 03 0 0 0 0 0 1 1 59
07 003 03 0 0 0 0 0 1 1 61
08 003 03 0 0 0 0 0 1 1 69
09 003 03 0 1 0 0 0 1 1 71
0a 003 03 0 0 0 0 0 1 1 79
0b 003 03 0 0 0 0 0 1 1 81
0c 003 03 0 0 0 0 0 1 1 89
0d 003 03 0 0 0 0 0 1 1 91
0e 003 03 0 0 0 0 0 1 1 99
0f 003 03 0 0 0 0 0 1 1 A1
10 003 03 1 1 0 1 0 1 1 A9
11 003 03 1 1 0 1 0 1 1 B1
12 003 03 1 1 0 1 0 1 1 B9
13 003 03 1 1 0 1 0 1 1 C1
14 003 03 1 1 0 1 0 1 1 D9
15 003 03 1 1 0 1 0 1 1 E1
16 003 03 1 1 0 1 0 1 1 C9
17 003 03 1 1 0 1 0 1 1 D1
IO APIC #2......
.... register #00: 02000000
....... : physical APIC id: 02
....... : Delivery Type: 0
....... : LTS : 0
.... register #01: 00178020
....... : max redirection entries: 0017
....... : PRQ implemented: 1
....... : IO APIC version: 0020
.... register #02: 02000000
....... : arbitration: 02
.... register #03: 00000001
....... : Boot DT : 1
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 000 00 1 0 0 0 0 0 0 00
01 000 00 1 0 0 0 0 0 0 00
02 000 00 1 0 0 0 0 0 0 00
03 000 00 1 0 0 0 0 0 0 00
04 000 00 1 0 0 0 0 0 0 00
05 000 00 1 0 0 0 0 0 0 00
06 000 00 1 0 0 0 0 0 0 00
07 000 00 1 0 0 0 0 0 0 00
08 000 00 1 0 0 0 0 0 0 00
09 000 00 1 0 0 0 0 0 0 00
0a 000 00 1 0 0 0 0 0 0 00
0b 000 00 1 0 0 0 0 0 0 00
0c 000 00 1 0 0 0 0 0 0 00
0d 000 00 1 0 0 0 0 0 0 00
0e 000 00 1 0 0 0 0 0 0 00
0f 000 00 1 0 0 0 0 0 0 00
10 000 00 1 0 0 0 0 0 0 00
11 000 00 1 0 0 0 0 0 0 00
12 000 00 1 0 0 0 0 0 0 00
13 000 00 1 0 0 0 0 0 0 00
14 000 00 1 0 0 0 0 0 0 00
15 000 00 1 0 0 0 0 0 0 00
16 000 00 1 0 0 0 0 0 0 00
17 000 00 1 0 0 0 0 0 0 00
IRQ to pin mappings:
IRQ0 -> 0:2
IRQ1 -> 0:1
IRQ3 -> 0:3
IRQ4 -> 0:4
IRQ5 -> 0:5
IRQ6 -> 0:6
IRQ7 -> 0:7
IRQ8 -> 0:8
IRQ9 -> 0:9-> 0:9
IRQ10 -> 0:10
IRQ11 -> 0:11
IRQ12 -> 0:12
IRQ13 -> 0:13
IRQ14 -> 0:14
IRQ15 -> 0:15
IRQ16 -> 0:16
IRQ17 -> 0:17
IRQ18 -> 0:18
IRQ19 -> 0:19
IRQ20 -> 0:20
IRQ21 -> 0:21
IRQ22 -> 0:22
IRQ23 -> 0:23
.................................... done.
PCI: Using ACPI for IRQ routing
PCI: if you experience problems, try using option 'pci=noacpi' or even
'acpi=off'
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society NET3.039
Initializing RT netlink socket
Starting kswapd
allocated 32 pages and 32 bhs reserved for the highmem bounces
VFS: Disk quotas vdquot_6.5.1
Journalled Block Device driver loaded
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
pty: 256 Unix98 ptys configured
Serial driver version 5.05c (2001-07-08) with MANY_PORTS SHARE_IRQ
SERIAL_PCI enabled
ttyS00 at 0x03f8 (irq = 4) is a 16550A
ttyS01 at 0x02f8 (irq = 3) is a 16550A
Floppy drive(s): fd0 is 1.44M
FDC 0 is a National Semiconductor PC87306
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
Linux agpgart interface v0.99 (c) Jeff Hartmann
agpgart: Maximum main memory to use for agp memory: 3553M
agpgart: Detected Intel i860 chipset
agpgart: AGP aperture is 64M @ 0xec000000
Uniform Multi-Platform E-IDE driver Revision: 7.00beta4-2.4
ide: Assuming 33MHz system bus speed for PIO modes; override with
idebus=xx
ICH2: IDE controller at PCI slot 00:1f.1
ICH2: chipset revision 4
ICH2: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0x10e0-0x10e7, BIOS settings: hda:pio, hdb:pio
ide1: BM-DMA at 0x10e8-0x10ef, BIOS settings: hdc:pio, hdd:pio
PDC20269: IDE controller at PCI slot 04:06.0
PDC20269: chipset revision 2
PDC20269: not 100% native mode: will probe irqs later
ide2: BM-DMA at 0x3040-0x3047, BIOS settings: hde:pio, hdf:pio
ide3: BM-DMA at 0x3048-0x304f, BIOS settings: hdg:pio, hdh:pio
hde: WDC WD2000JB-00FUA0, ATA DISK drive
blk: queue e04136b8, I/O limit 4095Mb (mask 0xffffffff)
ide2 at 0x3060-0x3067,0x3056 on irq 18
hde: attached ide-disk driver.
hde: host protected area => 1
hde: 390721968 sectors (200050 MB) w/8192KiB Cache, CHS=24321/255/63,
UDMA(100)
Partition check:
hde: hde1 hde2
SCSI subsystem driver Revision: 1.00
NCR53c406a: no available ports found
sym.3.8.0: setting PCI_COMMAND_PARITY...
sym.3.8.1: setting PCI_COMMAND_PARITY...
sym0: <1010-66> rev 0x1 on pci bus 3 device 8 function 1 irq 19
sym0: using 64 bit DMA addressing
sym0: Symbios NVRAM, ID 7, Fast-80, SE, parity checking
sym0: open drain IRQ line driver, using on-chip SRAM
sym0: using LOAD/STORE-based firmware.
sym0: handling phase mismatch from SCRIPTS.
sym0: SCSI BUS has been reset.
sym1: <1010-66> rev 0x1 on pci bus 3 device 8 function 0 irq 23
sym1: using 64 bit DMA addressing
sym1: Symbios NVRAM, ID 7, Fast-80, SE, parity checking
sym1: open drain IRQ line driver, using on-chip SRAM
sym1: using LOAD/STORE-based firmware.
sym1: handling phase mismatch from SCRIPTS.
sym1: SCSI BUS has been reset.
scsi0 : sym-2.1.17a
scsi1 : sym-2.1.17a
blk: queue f7acc218, I/O limit 1048575Mb (mask 0xffffffffff)
Vendor: SEAGATE Model: ST39236LW Rev: 0005
Type: Direct-Access ANSI SCSI revision: 03
blk: queue f77cbe18, I/O limit 1048575Mb (mask 0xffffffffff)
Vendor: IBM Model: DDYS-T36950M Rev: SA2A
Type: Direct-Access ANSI SCSI revision: 03
blk: queue f77cba18, I/O limit 1048575Mb (mask 0xffffffffff)
Vendor: IBM Model: DDYS-T36950M Rev: S96H
Type: Direct-Access ANSI SCSI revision: 03
blk: queue f77cb618, I/O limit 1048575Mb (mask 0xffffffffff)
sym0:0:0: tagged command queuing enabled, command queue depth 16.
sym0:1:0: tagged command queuing enabled, command queue depth 16.
sym0:2:0: tagged command queuing enabled, command queue depth 16.
Vendor: IBM Model: IC35L073UCDY10-0 Rev: S21E
Type: Direct-Access ANSI SCSI revision: 03
blk: queue f7762218, I/O limit 1048575Mb (mask 0xffffffffff)
Vendor: SEAGATE Model: ST1181677LC Rev: 0001
Type: Direct-Access ANSI SCSI revision: 03
blk: queue f773f818, I/O limit 1048575Mb (mask 0xffffffffff)
sym1:4:0: tagged command queuing enabled, command queue depth 16.
sym1:9:0: tagged command queuing enabled, command queue depth 16.
scsi2 : SCSI host adapter emulation for IDE ATAPI devices
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
Attached scsi disk sdb at scsi0, channel 0, id 1, lun 0
Attached scsi disk sdc at scsi0, channel 0, id 2, lun 0
Attached scsi disk sdd at scsi1, channel 0, id 4, lun 0
Attached scsi disk sde at scsi1, channel 0, id 9, lun 0
sym0:0: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 31)
SCSI device sda: 17942584 512-byte hdwr sectors (9187 MB)
sda: sda1 sda2 < sda5 sda6 sda7 >
sym0:1:0:M_REJECT to send for : 1-6-4-c-0-3e-1-0.
sym0:1: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 31)
SCSI device sdb: 71687340 512-byte hdwr sectors (36704 MB)
sdb: sdb1 sdb2 sdb3
sym0:2:0:M_REJECT to send for : 1-6-4-c-0-3e-1-0.
sym0:2: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 31)
SCSI device sdc: 71687340 512-byte hdwr sectors (36704 MB)
sdc: sdc1 sdc2 sdc3
sym1:4:0:M_REJECT to send for : 1-6-4-c-0-3e-1-0.
sym1:4: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 31)
SCSI device sdd: 143374805 512-byte hdwr sectors (73408 MB)
sdd: sdd1 sdd2 sdd3 sdd4 sdd6 sdd7 sdd8
sym1:9:0:M_REJECT to send for : 1-6-4-c-0-3e-1-0.
sym1:9: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 31)
SCSI device sde: 354600001 512-byte hdwr sectors (181555 MB)
sde: sde3 sde8
es1371: version v0.32 time 13:52:44 Jan 5 2004
Linux Kernel Card Services 3.1.22
options: [pci] [cardbus] [pm]
md: raid0 personality registered as nr 2
md: raid1 personality registered as nr 3
md: raid5 personality registered as nr 4
raid5: measuring checksumming speed
8regs : 925.200 MB/sec
32regs : 559.200 MB/sec
pIII_sse : 1045.200 MB/sec
pII_mmx : 934.000 MB/sec
p5_mmx : 909.600 MB/sec
raid5: using function: pIII_sse (1045.200 MB/sec)
md: multipath personality registered as nr 7
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
LVM version 1.0.7(28/03/2003)
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 32768 buckets, 256Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
ds: no socket drivers loaded!
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 136k freed
Adding Swap: 2049016k swap-space (priority 1)
Unable to find swap-space signature
Adding Swap: 2096104k swap-space (priority 1)
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,5), internal journal
[events: 00000038]
[events: 00000038]
md: autorun ...
md: considering sdc3 ...
md: adding sdc3 ...
md: adding sdb3 ...
md: created md0
md: bind<sdb3,1>
md: bind<sdc3,2>
md: running: <sdc3><sdb3>
md: sdc3's event counter: 00000038
md: sdb3's event counter: 00000038
md: md0: raid array is not clean -- starting background reconstruction
md: RAID level 1 does not need chunksize! Continuing anyway.
md0: max total readahead window set to 124k
md0: 1 data-disks, max readahead per data-disk: 124k
raid1: device sdc3 operational as mirror 1
raid1: device sdb3 operational as mirror 0
raid1: raid set md0 not clean; reconstructing mirrors
raid1: raid set md0 active with 2 out of 2 mirrors
md: syncing RAID array md0
md: minimum _guaranteed_ reconstruction speed: 100 KB/sec/disc.
md: using maximum available idle IO bandwith (but not more than 100000
KB/sec) for reconstruction.
md: using 124k window, over a total of 33588160 blocks.
md: updating md0 RAID superblock on device
md: sdc3 [events: 00000039]<6>(write) sdc3's sb offset: 33588160
md: sdb3 [events: 00000039]<6>(write) sdb3's sb offset: 33588160
md: ... autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,1), internal journal
EXT3-fs: mounted filesystem with ordered data mode.
EXT3-fs: Unrecognized mount option default
EXT2-fs warning (device md(9,0)): ext2_read_super: mounting ext3
filesystem as ext2
ufs_read_super: fs needs fsck
ufs_read_super: fs needs fsck
ufs_read_super: fs needs fsck
ufs_read_super: fs needs fsck
ufs_read_super: fs needs fsck
parport0: PC-style at 0x378 (0x778) [PCSPP(,...)]
parport0: irq 7 detected
eepro100.c:v1.09j-t 9/29/99 Donald Becker
http://www.scyld.com/network/eepro100.html
eepro100.c: $Revision: 1.36 $ 2000/11/17 Modified by Andrey V. Savochkin
<saw@saw.sw.com.sg> and others
eth0: OEM i82557/i82558 10/100 Ethernet, 00:E0:81:00:16:F6, IRQ 21.
Board assembly 000000-000, Physical connectors present: RJ45
Primary interface chip i82555 PHY #1.
Secondary interface chip i82555.
General self-test: passed.
Serial sub-system self-test: passed.
Internal registers self-test: passed.
ROM checksum self-test: passed (0x3258698e).
EXT3-fs: Unrecognized mount option default
Simon
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2004-07-13 0:34 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-02-10 1:25 Null pointer deference stevegt
2004-02-10 4:13 ` paging request failures under load (was: Re: Null pointer deference) stevegt
2004-02-10 6:34 ` stevegt
2004-02-11 4:12 ` 0-order allocation failed stevegt
2004-02-11 4:51 ` Kip Macy
2004-02-11 6:36 ` stevegt
2004-02-11 8:13 ` Keir Fraser
2004-02-11 21:23 ` paging request failures under load (was: Re: Null pointer deference) stevegt
2004-02-11 22:35 ` Keir Fraser
2004-02-21 6:23 ` Paging oopses stevegt
2004-02-21 6:31 ` stevegt
2004-02-21 6:52 ` stevegt
2004-02-21 8:31 ` Keir Fraser
2004-02-21 22:00 ` Kip Macy
2004-02-21 22:55 ` Mandrake-9.1 and Xen I RATTAN
2004-02-21 23:15 ` Ian Pratt
2004-02-23 1:26 ` I RATTAN
2004-02-23 1:39 ` Kip Macy
2004-02-23 2:13 ` Ian Pratt
2004-02-10 8:18 ` Null pointer deference Ian Pratt
-- strict thread matches above, loose matches on Subject: below --
2004-07-13 0:33 0-order allocation failed Simon Matthews
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.