* Unable to handle kernel paging request
@ 2004-03-03 16:33 Paulo Marques
0 siblings, 0 replies; 31+ messages in thread
From: Paulo Marques @ 2004-03-03 16:33 UTC (permalink / raw)
To: linux-kernel
I got this shortly after reconnecting a stale smb connection on a 2.6.4-rc1
vanilla kernel:
(removed hostname and date information to make it less verbose)
04:03:31 SMB connection re-established (-5)
04:03:47 Unable to handle kernel paging request at virtual address c1f16f1c
04:03:47 printing eip:
04:03:47 c01fc2f8
04:03:47 ws15t1 syslog.alert klogd: *pde = 00007067
04:03:47 ws15t1 syslog.alert klogd: *pte = 01f16000
04:03:47 Oops: 0000 [#1]
04:03:47 PREEMPT DEBUG_PAGEALLOC
04:03:47 CPU: 0
04:03:47 EIP: 0060:[01fc2f8>] Not tainted
04:03:47 EFLAGS: 00010246
04:03:47 EIP is at smb_add_request+0x238/0x520
04:03:47 eax: 00000000 ebx: c3a22000 ecx: c29a2e60 edx: c6940f78
04:03:47 esi: c1f16eec edi: c29a2e60 ebp: c3a23dc4 esp: c3a23d74
04:03:47 ds: 007b es: 007b ss: 0068
04:03:47 Process bash (pid: 93, threadinfo=c3a22000 task=c3a319e0)
04:03:47 Stack: 0000000b c3cec000 c1f16eec c6dcfe00 c1f16ef8 00000000 00000040
c1f16f24
04:03:47 c29a2df8 00000000 c3a319e0 c011eab0 00100100 00200200 00000401
c3cec006
04:03:47 c29a2df8 00000002 c1f16eec c3cec000 c3a23df0 c01f3434 c2529f38
00000000
04:03:47 Call Trace:
04:03:47 [011eab0>] default_wake_function+0x0/0x10
04:03:47 [01f3434>] smb_proc_getattr_trans2+0x84/0x100
04:03:47 [01f3734>] smb_proc_getattr_trans2_all+0x44/0xe0
04:03:47 [01f114c>] smb_init_dirent+0x4c/0x60
04:03:47 [01f1222>] smb_proc_getattr+0x32/0x50
04:03:47 [01f851c>] smb_refresh_inode+0x1c/0x100
04:03:47 [016d8ac>] __getblk+0x1c/0x40
04:03:47 [01bbc4a>] ext3_getblk+0xca/0x260
04:03:47 [01f86cc>] smb_revalidate_inode+0xcc/0x200
04:03:47 [011c826>] kernel_map_pages+0x16/0x50
04:03:47 [01f8bec>] smb_getattr+0x1c/0x50
04:03:47 [01f8bd0>] smb_getattr+0x0/0x50
04:03:47 [01783f1>] vfs_getattr+0x41/0xc0
04:03:47 [01789bf>] vfs_lstat+0x3f/0x60
04:03:47 [016adb0>] filp_dtor+0x0/0x190
04:03:47 [01789f4>] sys_lstat64+0x14/0x30
04:03:47 [010d12d>] do_IRQ+0x2dd/0x430
04:03:47 [016b193>] __fput+0x83/0xd0
04:03:47 [0109fef>] syscall_call+0x7/0xb
04:03:47
04:03:47 Code: 8b 46 30 c7 04 24 a0 da 41 c0 89 44 24 0c b8 58 a6 3f c0 89
The directory was mounted and was idle (no file activity at all) for about 20
hours before this happened.
I also have the complete syslog (it is only 400 lines), meminfo and slabinfo
from the same machine at the time of the dump, and I can post it if you find it
useful.
This is a somewhat "embedded" system, using a transmeta crusoe processor at
500MHz. The kernel configuration includes BlueZ modules, ethernet bridging and
cardbus (yenta) support. These are the most uncommon features.
The samba server was version 2.2.2 (which is quite old), but the client is the
latest kernel available.
I saw this only once and I don't believe this is going to be easy to reproduce.
After the dump the machine continued to operate normally (including acesing the
samba mounted directory), except for the shell that was doing a tab completion
on the directory name, that exited.
--
Paulo Marques - www.grupopie.com
"In a world without walls and fences who needs windows and gates?"
^ permalink raw reply [flat|nested] 31+ messages in thread
* unable to handle kernel paging request
@ 2004-10-31 18:42 Dennis Grevenstein
2004-10-31 19:15 ` Jan-Benedict Glaw
[not found] ` <Pine.GSO.4.10.10410311947570.9753-100000@helios.et.put.poznan.pl>
0 siblings, 2 replies; 31+ messages in thread
From: Dennis Grevenstein @ 2004-10-31 18:42 UTC (permalink / raw)
To: linux-mips
Hi,
I want to get the current cvs kernel running on
an R5000PC Challenge S. It does compile, but the
kernel does not boot. I get this error repeatedly
printed all over the console until I pull the plug:
<1>CPU 0 Unable to handle kernel paging request at\
virtual address 00000000, epc == 8810da1c, ra == 8810e22c
What could be wrong?
mfg
Dennis
--
There is certainly no purpose in remaining in the dark
except long enough to clear from the mind
the illusion of ever having been in the light.
T.S. Eliot
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 18:42 unable to handle kernel paging request Dennis Grevenstein
@ 2004-10-31 19:15 ` Jan-Benedict Glaw
[not found] ` <Pine.GSO.4.10.10410311947570.9753-100000@helios.et.put.poznan.pl>
1 sibling, 0 replies; 31+ messages in thread
From: Jan-Benedict Glaw @ 2004-10-31 19:15 UTC (permalink / raw)
To: Dennis Grevenstein; +Cc: linux-mips
[-- Attachment #1: Type: text/plain, Size: 878 bytes --]
On Sun, 2004-10-31 19:42:33 +0100, Dennis Grevenstein <dennis@pcde.inka.de>
wrote in message <20041031184233.GA11120@aton.pcde.inka.de>:
> I want to get the current cvs kernel running on
> an R5000PC Challenge S. It does compile, but the
> kernel does not boot. I get this error repeatedly
> printed all over the console until I pull the plug:
>
> <1>CPU 0 Unable to handle kernel paging request at\
> virtual address 00000000, epc == 8810da1c, ra == 8810e22c
Look into your System.map what's at 0x8810da1c and 0x8810e22c
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
[not found] ` <Pine.GSO.4.10.10410311947570.9753-100000@helios.et.put.poznan.pl>
@ 2004-10-31 19:16 ` Dennis Grevenstein
2004-10-31 19:26 ` Jan-Benedict Glaw
0 siblings, 1 reply; 31+ messages in thread
From: Dennis Grevenstein @ 2004-10-31 19:16 UTC (permalink / raw)
To: linux-mips
On Sun, Oct 31, 2004 at 07:48:15PM +0100, Stanislaw Skowronek wrote:
> > <1>CPU 0 Unable to handle kernel paging request at\
> > virtual address 00000000, epc == 8810da1c, ra == 8810e22c
>
> Look into your System.map and tell us what is there.
At 8810da1c or 8810e22c? Nothing directly.
Closest thing is this:
8810d9b0 t ip22zilog_maybe_update_regs
8810d9fc t ip22zilog_receive_chars
8810dd18 t ip22zilog_status_handle
8810def0 t ip22zilog_transmit_chars
8810e0e0 t ip22zilog_interrupt
8810e26c t ip22zilog_tx_empty
8810e2e4 t ip22zilog_get_mctrl
8810e378 t ip22zilog_set_mctrl
8810e3e0 t ip22zilog_stop_tx
8810e3f0 t ip22zilog_start_tx
8810e508 t ip22zilog_stop_rx
8810e544 t ip22zilog_enable_ms
8810e580 t ip22zilog_break_ctl
8810e628 t __ip22zilog_startup
mfg
Dennis
--
There is certainly no purpose in remaining in the dark
except long enough to clear from the mind
the illusion of ever having been in the light.
T.S. Eliot
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
@ 2004-10-31 19:26 ` Jan-Benedict Glaw
0 siblings, 0 replies; 31+ messages in thread
From: Jan-Benedict Glaw @ 2004-10-31 19:26 UTC (permalink / raw)
To: Dennis Grevenstein; +Cc: linux-mips
[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]
On Sun, 2004-10-31 20:16:31 +0100, Dennis Grevenstein <dennis@pcde.inka.de>
wrote in message <20041031191631.GB11681@aton.pcde.inka.de>:
> On Sun, Oct 31, 2004 at 07:48:15PM +0100, Stanislaw Skowronek wrote:
> > > <1>CPU 0 Unable to handle kernel paging request at\
> > > virtual address 00000000, epc == 8810da1c, ra == 8810e22c
> >
> > Look into your System.map and tell us what is there.
>
> At 8810da1c or 8810e22c? Nothing directly.
> Closest thing is this:
System.map is a list of function start addresses. Typically, functions
don't crash at their very first instructions, this is why you don't see
"exact" matches.
> 8810d9b0 t ip22zilog_maybe_update_regs
> 8810d9fc t ip22zilog_receive_chars
This is epc (+0x20).
> 8810dd18 t ip22zilog_status_handle
> 8810def0 t ip22zilog_transmit_chars
> 8810e0e0 t ip22zilog_interrupt
...and this is ra.
> 8810e26c t ip22zilog_tx_empty
>From my fading MIPS knowledge, ip22zilog_interrupt called
ip22zilog_receive_chars and the later one crashed. Now, use objdump and
create a disassembly dump of the object file that contains the IP22
Zilog stuff. There, find the part that's 0x20 bytes away from the start
of ip22zilog_receive_chars. Now you know the cause of this oops. From
here, try to figure out the reason for it...
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
@ 2004-10-31 19:26 ` Jan-Benedict Glaw
0 siblings, 0 replies; 31+ messages in thread
From: Jan-Benedict Glaw @ 2004-10-31 19:26 UTC (permalink / raw)
To: Dennis Grevenstein; +Cc: linux-mips
[-- Attachment #1: Type: text/plain, Size: 1649 bytes --]
On Sun, 2004-10-31 20:16:31 +0100, Dennis Grevenstein <dennis@pcde.inka.de>
wrote in message <20041031191631.GB11681@aton.pcde.inka.de>:
> On Sun, Oct 31, 2004 at 07:48:15PM +0100, Stanislaw Skowronek wrote:
> > > <1>CPU 0 Unable to handle kernel paging request at\
> > > virtual address 00000000, epc == 8810da1c, ra == 8810e22c
> >
> > Look into your System.map and tell us what is there.
>
> At 8810da1c or 8810e22c? Nothing directly.
> Closest thing is this:
System.map is a list of function start addresses. Typically, functions
don't crash at their very first instructions, this is why you don't see
"exact" matches.
> 8810d9b0 t ip22zilog_maybe_update_regs
> 8810d9fc t ip22zilog_receive_chars
This is epc (+0x20).
> 8810dd18 t ip22zilog_status_handle
> 8810def0 t ip22zilog_transmit_chars
> 8810e0e0 t ip22zilog_interrupt
...and this is ra.
> 8810e26c t ip22zilog_tx_empty
From my fading MIPS knowledge, ip22zilog_interrupt called
ip22zilog_receive_chars and the later one crashed. Now, use objdump and
create a disassembly dump of the object file that contains the IP22
Zilog stuff. There, find the part that's 0x20 bytes away from the start
of ip22zilog_receive_chars. Now you know the cause of this oops. From
here, try to figure out the reason for it...
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 19:26 ` Jan-Benedict Glaw
(?)
@ 2004-10-31 19:55 ` Dennis Grevenstein
2004-10-31 20:13 ` Jan-Benedict Glaw
-1 siblings, 1 reply; 31+ messages in thread
From: Dennis Grevenstein @ 2004-10-31 19:55 UTC (permalink / raw)
To: linux-mips
On Sun, Oct 31, 2004 at 08:26:53PM +0100, Jan-Benedict Glaw wrote:
> System.map is a list of function start addresses. Typically, functions
> don't crash at their very first instructions, this is why you don't see
> "exact" matches.
okay.
> From my fading MIPS knowledge, ip22zilog_interrupt called
> ip22zilog_receive_chars and the later one crashed. Now, use objdump and
> create a disassembly dump of the object file that contains the IP22
> Zilog stuff. There, find the part that's 0x20 bytes away from the start
> of ip22zilog_receive_chars. Now you know the cause of this oops.
That's what I found:
8810da14: 8c82001c lw v0,28(a0)
8810da18: 00809021 move s2,a0
8810da1c: 8c510000 lw s1,0(v0)
8810da20: 00a09821 move s3,a1
8810da24: 8e220118 lw v0,280(s1)
and:
8810e224: 0e04367f jal 8810d9fc <ip22zilog_receive_chars>
8810e228: 02803021 move a2,s4
8810e22c: 0a04386e j 8810e1b8 <ip22zilog_interrupt+0xd8>
8810e230: 32020001 andi v0,s0,0x1
8810e234: 0e0437bc jal 8810def0 <ip22zilog_transmit_chars>
> From
> here, try to figure out the reason for it...
Well, I'm sure "MIPS assembly for Dummies" must be available
somewhere. While I keep looking please help me ;-)
mfg
Dennis
--
There is certainly no purpose in remaining in the dark
except long enough to clear from the mind
the illusion of ever having been in the light.
T.S. Eliot
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 19:55 ` Dennis Grevenstein
@ 2004-10-31 20:13 ` Jan-Benedict Glaw
2004-10-31 22:36 ` Dennis Grevenstein
2004-10-31 22:51 ` Dennis Grevenstein
0 siblings, 2 replies; 31+ messages in thread
From: Jan-Benedict Glaw @ 2004-10-31 20:13 UTC (permalink / raw)
To: Dennis Grevenstein; +Cc: linux-mips
[-- Attachment #1: Type: text/plain, Size: 2312 bytes --]
On Sun, 2004-10-31 20:55:50 +0100, Dennis Grevenstein <dennis@pcde.inka.de>
wrote in message <20041031195550.GA12397@aton.pcde.inka.de>:
> On Sun, Oct 31, 2004 at 08:26:53PM +0100, Jan-Benedict Glaw wrote:
> > From my fading MIPS knowledge, ip22zilog_interrupt called
> > ip22zilog_receive_chars and the later one crashed. Now, use objdump and
> > create a disassembly dump of the object file that contains the IP22
> > Zilog stuff. There, find the part that's 0x20 bytes away from the start
> > of ip22zilog_receive_chars. Now you know the cause of this oops.
>
> That's what I found:
> 8810da14: 8c82001c lw v0,28(a0)
> 8810da18: 00809021 move s2,a0
> 8810da1c: 8c510000 lw s1,0(v0)
It's accessing (most probably) a structure's first field, where the
structure is supplied by pointer in v0.
> 8810da20: 00a09821 move s3,a1
> 8810da24: 8e220118 lw v0,280(s1)
So now, find out what v0 belongs to. Maybe compiling the kernel with
debug infos (-g) and using objdump -S (for intermixing sources) will
help you.
Most probably, something wasn't correctly registered, so the pointer to
a struct is just a NULL pointer.
> and:
>
> 8810e224: 0e04367f jal 8810d9fc <ip22zilog_receive_chars>
> 8810e228: 02803021 move a2,s4
> 8810e22c: 0a04386e j 8810e1b8 <ip22zilog_interrupt+0xd8>
> 8810e230: 32020001 andi v0,s0,0x1
> 8810e234: 0e0437bc jal 8810def0 <ip22zilog_transmit_chars>
So this all fits properly :-)
> > From
> > here, try to figure out the reason for it...
>
> Well, I'm sure "MIPS assembly for Dummies" must be available
> somewhere. While I keep looking please help me ;-)
"objdump -S" for starters, but it seems quite straight forward. Maybe
paste the code of ip22zilog_receive_chars, I don't have that at hands
right now...
MfG, JBG
--
Jan-Benedict Glaw jbglaw@lug-owl.de . +49-172-7608481 _ O _
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg _ _ O
fuer einen Freien Staat voll Freier Bürger" | im Internet! | im Irak! O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 20:13 ` Jan-Benedict Glaw
@ 2004-10-31 22:36 ` Dennis Grevenstein
2004-10-31 23:59 ` Maciej W. Rozycki
2004-10-31 22:51 ` Dennis Grevenstein
1 sibling, 1 reply; 31+ messages in thread
From: Dennis Grevenstein @ 2004-10-31 22:36 UTC (permalink / raw)
To: linux-mips
On Sun, Oct 31, 2004 at 09:13:35PM +0100, Jan-Benedict Glaw wrote:
>
> So now, find out what v0 belongs to. Maybe compiling the kernel with
> debug infos (-g) and using objdump -S (for intermixing sources) will
> help you.
recompiling the kernel will take another few hours, but
I may try later. Using "objdump -S" I don't get any more info.
Building a kernel without the driver for the serial port
seems not so good for a Challenge S.
> "objdump -S" for starters, but it seems quite straight forward. Maybe
> paste the code of ip22zilog_receive_chars, I don't have that at hands
> right now...
Okay, here it is:
static void ip22zilog_receive_chars(struct uart_ip22zilog_port *up,
struct zilog_channel *channel,
struct pt_regs *regs)
{
struct tty_struct *tty = up->port.info->tty; /* XXX info==NULL? */
while (1) {
unsigned char ch, r1;
if (unlikely(tty->flip.count >= TTY_FLIPBUF_SIZE)) {
tty->flip.work.func((void *)tty);
if (tty->flip.count >= TTY_FLIPBUF_SIZE)
return; /* XXX Ignores SysRq when we nee
d it most. Fix. */
}
r1 = read_zsreg(channel, R1);
if (r1 & (PAR_ERR | Rx_OVR | CRC_ERR)) {
writeb(ERR_RES, &channel->control);
ZSDELAY();
ZS_WSYNC(channel);
}
ch = readb(&channel->control);
ZSDELAY();
/* This funny hack depends upon BRK_ABRT not interfering
* with the other bits we care about in R1.
*/
if (ch & BRK_ABRT)
r1 |= BRK_ABRT;
ch = readb(&channel->data);
ZSDELAY();
ch &= up->parity_mask;
if (ZS_IS_CONS(up) && (r1 & BRK_ABRT)) {
/* Wait for BREAK to deassert to avoid potentially
* confusing the PROM.
*/
while (1) {
ch = readb(&channel->control);
ZSDELAY();
if (!(ch & BRK_ABRT))
break;
}
ip22_do_break();
return;
}
/* A real serial line, record the character and status. */
*tty->flip.char_buf_ptr = ch;
*tty->flip.flag_buf_ptr = TTY_NORMAL;
up->port.icount.rx++;
if (r1 & (BRK_ABRT | PAR_ERR | Rx_OVR | CRC_ERR)) {
if (r1 & BRK_ABRT) {
r1 &= ~(PAR_ERR | CRC_ERR);
up->port.icount.brk++;
if (uart_handle_break(&up->port))
goto next_char;
}
else if (r1 & PAR_ERR)
up->port.icount.parity++;
else if (r1 & CRC_ERR)
up->port.icount.frame++;
if (r1 & Rx_OVR)
up->port.icount.overrun++;
r1 &= up->port.read_status_mask;
if (r1 & BRK_ABRT)
*tty->flip.flag_buf_ptr = TTY_BREAK;
else if (r1 & PAR_ERR)
*tty->flip.flag_buf_ptr = TTY_PARITY;
else if (r1 & CRC_ERR)
*tty->flip.flag_buf_ptr = TTY_FRAME;
}
if (uart_handle_sysrq_char(&up->port, ch, regs))
goto next_char;
if (up->port.ignore_status_mask == 0xff ||
(r1 & up->port.ignore_status_mask) == 0) {
tty->flip.flag_buf_ptr++;
tty->flip.char_buf_ptr++;
tty->flip.count++;
}
if ((r1 & Rx_OVR) &&
tty->flip.count < TTY_FLIPBUF_SIZE) {
*tty->flip.flag_buf_ptr = TTY_OVERRUN;
tty->flip.flag_buf_ptr++;
tty->flip.char_buf_ptr++;
tty->flip.count++;
}
next_char:
ch = readb(&channel->control);
ZSDELAY();
if (!(ch & Rx_CH_AV))
break;
}
tty_flip_buffer_push(tty);
}
mfg
Dennis
--
There is certainly no purpose in remaining in the dark
except long enough to clear from the mind
the illusion of ever having been in the light.
T.S. Eliot
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 20:13 ` Jan-Benedict Glaw
2004-10-31 22:36 ` Dennis Grevenstein
@ 2004-10-31 22:51 ` Dennis Grevenstein
2004-10-31 23:13 ` Dennis Grevenstein
1 sibling, 1 reply; 31+ messages in thread
From: Dennis Grevenstein @ 2004-10-31 22:51 UTC (permalink / raw)
To: linux-mips
Hi,
I just tried a simple, quick recompile with the ip22zilog
driver. The kernel boots and seems to work.
misato:~# uname -a
Linux misato 2.6.10-rc1 #2 Sun Oct 31 21:56:20 CET 2004 mips GNU/Linux
Any more help is greatly appreciated though.
mfg
Dennis
--
There is certainly no purpose in remaining in the dark
except long enough to clear from the mind
the illusion of ever having been in the light.
T.S. Eliot
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 22:51 ` Dennis Grevenstein
@ 2004-10-31 23:13 ` Dennis Grevenstein
0 siblings, 0 replies; 31+ messages in thread
From: Dennis Grevenstein @ 2004-10-31 23:13 UTC (permalink / raw)
To: linux-mips
On Sun, Oct 31, 2004 at 11:51:33PM +0100, Dennis Grevenstein wrote:
>
> I just tried a simple, quick recompile with the ip22zilog
> driver. The kernel boots and seems to work.
Sorry, of course I meant _without_ the driver.
mfg
Dennis
--
There is certainly no purpose in remaining in the dark
except long enough to clear from the mind
the illusion of ever having been in the light.
T.S. Eliot
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 22:36 ` Dennis Grevenstein
@ 2004-10-31 23:59 ` Maciej W. Rozycki
2004-11-01 21:50 ` Florian Lohoff
0 siblings, 1 reply; 31+ messages in thread
From: Maciej W. Rozycki @ 2004-10-31 23:59 UTC (permalink / raw)
To: Dennis Grevenstein; +Cc: linux-mips
On Sun, 31 Oct 2004, Dennis Grevenstein wrote:
> struct tty_struct *tty = up->port.info->tty; /* XXX info==NULL? */
It looks up->port.info is null for some reason (and unhandled as noted
in the comment).
Maciej
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2004-10-31 23:59 ` Maciej W. Rozycki
@ 2004-11-01 21:50 ` Florian Lohoff
0 siblings, 0 replies; 31+ messages in thread
From: Florian Lohoff @ 2004-11-01 21:50 UTC (permalink / raw)
To: Maciej W. Rozycki; +Cc: Dennis Grevenstein, linux-mips
[-- Attachment #1: Type: text/plain, Size: 3705 bytes --]
On Sun, Oct 31, 2004 at 11:59:31PM +0000, Maciej W. Rozycki wrote:
> On Sun, 31 Oct 2004, Dennis Grevenstein wrote:
>
> > struct tty_struct *tty = up->port.info->tty; /* XXX info==NULL? */
>
> It looks up->port.info is null for some reason (and unhandled as noted
> in the comment).
I had the same problem and fixed it like this - It fixes some other
break/sysrq based problems ...
Index: drivers/serial/ip22zilog.c
===================================================================
RCS file: /home/flo/linux-mips-cvs/linux/drivers/serial/ip22zilog.c,v
retrieving revision 1.15
diff -u -r1.15 ip22zilog.c
--- drivers/serial/ip22zilog.c 28 Sep 2004 19:22:07 -0000 1.15
+++ drivers/serial/ip22zilog.c 3 Oct 2004 19:26:06 -0000
@@ -47,8 +47,6 @@
#include "ip22zilog.h"
-void ip22_do_break(void);
-
/*
* On IP22 we need to delay after register accesses but we do not need to
* flush writes.
@@ -256,17 +254,15 @@
struct zilog_channel *channel,
struct pt_regs *regs)
{
- struct tty_struct *tty = up->port.info->tty; /* XXX info==NULL? */
+ struct tty_struct *tty = NULL;
+
+ if (up->port.info != NULL && /* Unopened serial console */
+ up->port.info->tty != NULL) /* Keyboard || mouse */
+ tty = up->port.info->tty;
while (1) {
unsigned char ch, r1;
- if (unlikely(tty->flip.count >= TTY_FLIPBUF_SIZE)) {
- tty->flip.work.func((void *)tty);
- if (tty->flip.count >= TTY_FLIPBUF_SIZE)
- return; /* XXX Ignores SysRq when we need it most. Fix. */
- }
-
r1 = read_zsreg(channel, R1);
if (r1 & (PAR_ERR | Rx_OVR | CRC_ERR)) {
writeb(ERR_RES, &channel->control);
@@ -288,24 +284,8 @@
ch &= up->parity_mask;
- if (ZS_IS_CONS(up) && (r1 & BRK_ABRT)) {
- /* Wait for BREAK to deassert to avoid potentially
- * confusing the PROM.
- */
- while (1) {
- ch = readb(&channel->control);
- ZSDELAY();
- if (!(ch & BRK_ABRT))
- break;
- }
- ip22_do_break();
- return;
- }
-
- /* A real serial line, record the character and status. */
- *tty->flip.char_buf_ptr = ch;
- *tty->flip.flag_buf_ptr = TTY_NORMAL;
up->port.icount.rx++;
+
if (r1 & (BRK_ABRT | PAR_ERR | Rx_OVR | CRC_ERR)) {
if (r1 & BRK_ABRT) {
r1 &= ~(PAR_ERR | CRC_ERR);
@@ -319,6 +299,25 @@
up->port.icount.frame++;
if (r1 & Rx_OVR)
up->port.icount.overrun++;
+ }
+
+ if (uart_handle_sysrq_char(&up->port, ch, regs))
+ goto next_char;
+
+ if (!tty)
+ goto next_char;
+
+ if (unlikely(tty->flip.count >= TTY_FLIPBUF_SIZE)) {
+ tty->flip.work.func((void *)tty);
+ if (tty->flip.count >= TTY_FLIPBUF_SIZE)
+ goto push_tty; /* XXX We drop characters here - Either read or die */
+ }
+
+ /* A real serial line, record the character and status. */
+ *tty->flip.char_buf_ptr = ch;
+ *tty->flip.flag_buf_ptr = TTY_NORMAL;
+
+ if (r1 & (BRK_ABRT | PAR_ERR | Rx_OVR | CRC_ERR)) {
r1 &= up->port.read_status_mask;
if (r1 & BRK_ABRT)
*tty->flip.flag_buf_ptr = TTY_BREAK;
@@ -327,8 +326,6 @@
else if (r1 & CRC_ERR)
*tty->flip.flag_buf_ptr = TTY_FRAME;
}
- if (uart_handle_sysrq_char(&up->port, ch, regs))
- goto next_char;
if (up->port.ignore_status_mask == 0xff ||
(r1 & up->port.ignore_status_mask) == 0) {
@@ -350,7 +347,9 @@
break;
}
- tty_flip_buffer_push(tty);
+push_tty:
+ if (tty)
+ tty_flip_buffer_push(tty);
}
static void ip22zilog_status_handle(struct uart_ip22zilog_port *up,
--
Florian Lohoff flo@rfc822.org +49-171-2280134
Heisenberg may have been here.
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 31+ messages in thread
* Unable to handle kernel paging request
@ 2005-01-12 18:43 Praedor Atrebates
0 siblings, 0 replies; 31+ messages in thread
From: Praedor Atrebates @ 2005-01-12 18:43 UTC (permalink / raw)
To: linux-kernel
Please CC me any answer you might have to offer me as I am not subscribed to
the list.
I am running Mandrake 10.1 on a Gateway Solo laptop, Pentium M, ATI Radeon
9000, firewire, builtin e1000 ethernet. Kernel is 2.6.8.1-12mdk. I have
lately been running into a tremendous problem with kernel paging request
errors/Oops's at seemingly random moments. Right now I am running a
2.6.0-rc2.2mdk kernel because under the 2.6.8.1 kernel the errors were
happening so often that the system was unusable for any length of time. I am
virtually certain to see one upon shutdown even with the current running
kernel as well. The latest such error produced this:
Jan 12 10:31:09 dhcp18-126 kernel: NET: Unregistered protocol family 23
Jan 12 10:32:19 dhcp18-126 kernel: Unable to handle kernel paging request at
virtual address d56c34ac
Jan 12 10:32:19 dhcp18-126 kernel: printing eip:
Jan 12 10:32:19 dhcp18-126 kernel: c01294a3
Jan 12 10:32:19 dhcp18-126 kernel: *pde = 0e1a6067
Jan 12 10:32:19 dhcp18-126 kernel: *pte = 00000000
Jan 12 10:32:19 dhcp18-126 kernel: Oops: 0000 [#1]
Jan 12 10:32:19 dhcp18-126 kernel: Modules linked in: crc-ccitt sg st sr_mod
sd_mod scsi_mod fglrx exportfs lockd sunrpc md5 ipv6 snd-seq-oss
snd-seq-midi-event snd-seq snd-pcm-oss snd-mixer-oss snd-intel8x0
snd-ac97-codec snd-pcm snd-timer snd-page-alloc gameport snd-mpu401-uart
snd-rawmidi snd-seq-device snd soundcore parport_pc lp parport driverloader
af_packet floppy ds yenta_socket pcmcia_core eth1394 e1000 tulip raw ide-cd
cdrom ohci1394 ieee1394 loop nls_iso8859-1 ntfs ext3 jbd intel-agp agpgart
nvram evdev ehci-hcd uhci-hcd usbcore rtc reiserfs
Jan 12 10:32:19 dhcp18-126 kernel: CPU: 0
Jan 12 10:32:19 dhcp18-126 kernel: EIP: 0060:[notifier_call_chain+35/64]
Tainted: P VLI
Jan 12 10:32:19 dhcp18-126 kernel: EIP: 0060:[<c01294a3>] Tainted: P
VLI
Jan 12 10:32:19 dhcp18-126 kernel: EFLAGS: 00010286 (2.6.8.1-12mdk)
Jan 12 10:32:19 dhcp18-126 kernel: EIP is at notifier_call_chain+0x23/0x40
Jan 12 10:32:19 dhcp18-126 kernel: eax: 00000001 ebx: d56c34ac ecx:
00000000 edx: 00000001
Jan 12 10:32:20 dhcp18-126 kernel: esi: cf264800 edi: 00000006 ebp:
c3eade7c esp: c3eade64
Jan 12 10:32:20 dhcp18-126 kernel: ds: 007b es: 007b ss: 0068
Jan 12 10:32:20 dhcp18-126 kernel: Process rmmod (pid: 3956,
threadinfo=c3eac000 task=c253a0b0)
Jan 12 10:32:20 dhcp18-126 kernel: Stack: d56c34ac 00000006 cf264800 cf264800
cf264800 cf264938 c3eadea4 c02769a2
Jan 12 10:32:20 dhcp18-126 kernel: c03e5520 00000006 cf264800 d0ee1cae
cf264938 cf264800 cc40ec98 ccf64000
Jan 12 10:32:20 dhcp18-126 kernel: c3eadeb4 c0220537 cf264800 cf264a20
c3eaded8 d0f7cb9f cf264800 ccf64000
Jan 12 10:32:20 dhcp18-126 kernel: Call Trace:
Jan 12 10:32:20 dhcp18-126 kernel: [show_stack+127/160] show_stack+0x7f/0xa0
Jan 12 10:32:20 dhcp18-126 kernel: [<c0107bbf>] show_stack+0x7f/0xa0
Jan 12 10:32:20 dhcp18-126 kernel: [show_registers+342/464]
show_registers+0x156/0x1d0
Jan 12 10:32:20 dhcp18-126 kernel: [<c0107d56>] show_registers+0x156/0x1d0
Jan 12 10:32:20 dhcp18-126 kernel: [die+102/208] die+0x66/0xd0
Jan 12 10:32:20 dhcp18-126 kernel: [<c0107ef6>] die+0x66/0xd0
Jan 12 10:32:20 dhcp18-126 kernel: [do_page_fault+966/1456]
do_page_fault+0x3c6/0x5b0
Jan 12 10:32:20 dhcp18-126 kernel: [<c0119b26>] do_page_fault+0x3c6/0x5b0
Jan 12 10:32:20 dhcp18-126 kernel: [error_code+45/56] error_code+0x2d/0x38
Jan 12 10:32:20 dhcp18-126 kernel: [<c0107849>] error_code+0x2d/0x38
Jan 12 10:32:20 dhcp18-126 kernel: [unregister_netdevice+338/606]
unregister_netdevice+0x152/0x25e
Jan 12 10:32:20 dhcp18-126 kernel: [<c02769a2>]
unregister_netdevice+0x152/0x25e
Jan 12 10:32:20 dhcp18-126 kernel: [unregister_netdev+23/32]
unregister_netdev+0x17/0x20
Jan 12 10:32:20 dhcp18-126 kernel: [<c0220537>] unregister_netdev+0x17/0x20
Jan 12 10:32:20 dhcp18-126 kernel: [pg0+280554399/1069613056]
ether1394_remove_host+0x6f/0x90 [eth1394]
Jan 12 10:32:20 dhcp18-126 kernel: [<d0f7cb9f>]
ether1394_remove_host+0x6f/0x90 [eth1394]
Jan 12 10:32:20 dhcp18-126 kernel: [pg0+279897277/1069613056]
__unregister_host+0x8d/0xc0 [ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [<d0edc4bd>] __unregister_host+0x8d/0xc0
[ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [pg0+279897361/1069613056]
highlevel_for_each_host_unreg+0x21/0x30 [ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [<d0edc511>]
highlevel_for_each_host_unreg+0x21/0x30 [ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [pg0+279917671/1069613056]
nodemgr_for_each_host+0x47/0x80 [ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [<d0ee1467>]
nodemgr_for_each_host+0x47/0x80 [ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [pg0+279897499/1069613056]
hpsb_unregister_highlevel+0x7b/0x90 [ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [<d0edc59b>]
hpsb_unregister_highlevel+0x7b/0x90 [ieee1394]
Jan 12 10:32:20 dhcp18-126 kernel: [pg0+280562478/1069613056]
ether1394_exit_module+0x1e/0x2d [eth1394]
Jan 12 10:32:20 dhcp18-126 kernel: [<d0f7eb2e>]
ether1394_exit_module+0x1e/0x2d [eth1394]
Jan 12 10:32:20 dhcp18-126 kernel: [sys_delete_module+306/384]
sys_delete_module+0x132/0x180
Jan 12 10:32:20 dhcp18-126 kernel: [<c0131902>] sys_delete_module+0x132/0x180
Jan 12 10:32:20 dhcp18-126 kernel: [sysenter_past_esp+82/113]
sysenter_past_esp+0x52/0x71
Jan 12 10:32:20 dhcp18-126 kernel: [<c0106dcd>] sysenter_past_esp+0x52/0x71
Jan 12 10:32:20 dhcp18-126 kernel: Code: b8 fe ff ff ff c3 89 f6 55 31 d2 89
e5 57 56 53 83 ec 0c 8b 45 08 8b 7d 0c 8b 75 10 8b 18 eb 19 89 74 24 08 89 7c
24 04 89 1c 24 <ff> 13 a9 00 80 00 00 89 c2 75 07 8b 5b 04 85 db 75 e3 83 c4
0c
A previous such error (very brief) produced:
Jan 6 14:23:17 lapdog kernel: NET: Registered protocol family 4
Jan 6 14:23:17 lapdog kernel: Unable to handle kernel paging request at
virtual
address d0bb24b4
Jan 6 14:23:17 lapdog kernel: printing eip:
Jan 6 14:23:17 lapdog kernel: c012942f
Jan 6 14:23:17 lapdog kernel: *pde = 012f3067
Jan 6 14:23:17 lapdog kernel: *pte = 00000000
Jan 6 14:23:17 lapdog kernel: Oops: 0000 [#2]
The same hex address region is involved. From my system map the above two
addresses fall between inter_module_unregister in my system map
(c0129460) and before inter_module_get (c0129520).
Is this indicative of a hardware problem or a software problem? I have booted
up with memtest86+ and let it run for about an hour (couldn't afford to let
it go longer at that point) and it found no problems at all.
Thank you for any aid you can toss my way.
praedor
^ permalink raw reply [flat|nested] 31+ messages in thread
* Unable to handle kernel paging request
2006-08-05 7:38 blocking Xen 3.X production use: soft lockup bugs Ian Pratt
@ 2006-08-12 1:50 ` Steve Traugott
2006-08-12 4:34 ` Steve Traugott
0 siblings, 1 reply; 31+ messages in thread
From: Steve Traugott @ 2006-08-12 1:50 UTC (permalink / raw)
To: Ian Pratt; +Cc: xen-devel
Uh oh....
On Sat, Aug 05, 2006 at 08:38:03AM +0100, Ian Pratt wrote:
> > So I built -unstable changeset 10868, and ran an even heavier workload
> > (the above, plus 'bonnie' in the guests) on dom0 and two guests
> > overnight, and they experienced no soft lockups; running -unstable,
> > changeset 10868, credit scheduler. This same workload would have
> > caused soft lockups within seconds in -testing changeset 9732 using
> > the sedf scheduler; I may not have been able to get it started at all.
> > Response time remained subsecond under -unstable; -testing would have
> > been on its knees.
>
> That's good to hear. 3.0.3 is going to be a big leap forward in many
> ways.
Same environment as we've discussed earlier in this thread, now on
unstable changeset 10868, Xen on top of DRBD, this one seems to be a
reliable crash when under combined disk and network load, still
isolating but would welcome suggestions...
Steve
root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
kernel /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
[Multiboot-elf, <0x100000:0x7a008:0x55ff8>, shtab=0x1d0078, entry=0x100000]
module /boot/vmlinuz-2.6.16.13-xen root=801 ro console=ttyS0 ramdisk_size=32768
[Multiboot-module @ 0x1d1000, 0x494ed0 bytes]
module /boot/initrd.img-2.6.16.13-xen
[Multiboot-module @ 0x666000, 0x9a6400 bytes]
__ __ _____ ___ _ _ _
\ \/ /___ _ __ |___ / / _ \ _ _ _ __ ___| |_ __ _| |__ | | ___
\ // _ \ '_ \ |_ \| | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
/ \ __/ | | | ___) | |_| |__| |_| | | | \__ \ || (_| | |_) | | __/
/_/\_\___|_| |_| |____(_)___/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
http://www.cl.cam.ac.uk/netos/xen
University of Cambridge Computer Laboratory
Xen version 3.0-unstable (root@prd.terraluna.org) (gcc version 3.3.5 (Debian 1:3.3.5-13)) Fri Aug 4 10:54:50 PDT 2006
Latest ChangeSet: Sat Jul 29 06:05:59 2006 +0100 10868:d2bf1a7cc131
(XEN) Command line: /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
(XEN) Physical RAM map:
(XEN) 0000000000000000 - 000000000009c000 (usable)
(XEN) 000000000009c000 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 000000003ffec340 (usable)
(XEN) 000000003ffec340 - 000000003fff0000 (ACPI data)
(XEN) 000000003fff0000 - 0000000040000000 (reserved)
(XEN) 00000000fec00000 - 0000000100000000 (reserved)
(XEN) System RAM: 1023MB (1048096kB)
(XEN) Xen heap: 10MB (10396kB)
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) PAE disabled.
(XEN) found SMP MP-table at 0009c1d0
(XEN) DMI 2.3 present.
(XEN) Using APIC driver default
(XEN) ACPI: RSDP (v000 IBM ) @ 0x000fdfd0
(XEN) ACPI: RSDT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff80
(XEN) ACPI: FADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff00
(XEN) ACPI: MADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffefe80
(XEN) ACPI: DSDT (v001 IBM SEREMRLD 0x00001000 MSFT 0x0100000b) @ 0x00000000
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
(XEN) Processor #3 6:11 APIC version 17
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:11 APIC version 17
(XEN) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
(XEN) ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
(XEN) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
(XEN) ACPI: IRQ3 used by override.
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Initializing CPU#0
(XEN) Detected 1130.180 MHz processor.
(XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
(XEN) CPU: L2 cache: 512K
(XEN) Intel machine check architecture supported.
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
(XEN) Booting processor 1/0 eip 90000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
(XEN) CPU: L2 cache: 512K
(XEN) Intel machine check architecture supported.
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=0 apic2=-1 pin2=-1
(XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
(XEN) ...trying to set up timer (IRQ0) through the 8259A ... failed.
(XEN) ...trying to set up timer as Virtual Wire IRQ... works.
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 1.193MHz PIT
(XEN) Brought up 2 CPUs
(XEN) Machine check exception polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Domain 0 kernel supports features = { 0000001f }.
(XEN) Domain 0 kernel requires features = { 00000000 }.
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 3c000000->3e000000 (8192 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: c0100000->c05d5d74
(XEN) Init. ramdisk: c05d6000->c0f7c400
(XEN) Phys-Mach map: c0f7d000->c0f8d000
(XEN) Start info: c0f8d000->c0f8e000
(XEN) Page tables: c0f8e000->c0f94000
(XEN) Boot stack: c0f94000->c0f95000
(XEN) TOTAL: c0000000->c1400000
(XEN) ENTRY ADDRESS: c0100000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) Initrd len 0x9a6400, start at 0xc05d6000
(XEN) Scrubbing Free RAM: ...........done.
(XEN) Xen trace buffers: disabled
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen).
Linux version 2.6.16.13-xen (root@n4h8) (gcc version 3.3.5 (Debian 1:3.3.5-13)) #3 SMP Fri Aug 4 10:57:44 PDT 2006
BIOS-provided physical RAM map:
Xen: 0000000000000000 - 0000000004800000 (usable)
0MB HIGHMEM available.
72MB LOWMEM available.
DMI 2.3 present.
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
Enabling APIC mode: Flat. Using 2 I/O APICs
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 50000000 (gap: 40000000:bec00000)
Built 1 zonelists
Kernel command line: root=801 ro console=ttyS0 ramdisk_size=32768
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 512 (order: 9, 8192 bytes)
Xen reported: 1130.113 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Software IO TLB enabled:
Aperture: 2 megabytes
Kernel range: 0x00000000c10f0000 - 0x00000000c12f0000
vmalloc area: c5000000-fb7fe000, maxmem 33ffe000
Memory: 47400k/73728k available (3389k kernel code, 18068k reserved, 1044k data, 208k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 2261.60 BogoMIPS (lpj=11308023)
Security Framework v1.0.0 initialized
Capability LSM initialized
Mount-cache hash table entries: 512
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
Checking 'hlt' instruction... OK.
ENABLING IO-APIC IRQs
Enabling SMP...
Brought up 2 CPUs
Initializing CPU#1
migration_cost=1895
checking if image is initramfs... it is
Freeing initrd memory: 9881k freed
Grant table initialized
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using configuration type 1
ACPI: Subsystem revision 20060127
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
ACPI: PCI Interrupt Link [LPE1] (IRQs *10)
ACPI: PCI Interrupt Link [LPE2] (IRQs *10)
ACPI: PCI Interrupt Link [LPVI] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LPUS] (IRQs *7)
ACPI: PCI Root Bridge [PCI1] (0000:01)
ACPI: PCI Interrupt Link [LPSA] (IRQs *9)
ACPI: PCI Interrupt Link [LP1A] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP1B] (IRQs) *0, disabled.
ACPI: PCI Interrupt Link [LP2A] (IRQs *9)
ACPI: PCI Interrupt Link [LP2B] (IRQs) *0, disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
xen_mem: Initialising balloon driver.
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
IA-32 Microcode Update Driver: v1.14-xen <tigran@veritas.com>
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
JFS: nTxBlock = 512, nTxLock = 4096
SGI XFS with ACLs, security attributes, realtime, large block numbers, no debug enabled
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize
Xen virtual console successfully installed as ttyS0
Event-channel device installed.
blkif_init: reqs=64, pages=704, mmap_vstart=0xc0800000
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
SvrWks OSB4: chipset revision 0
SvrWks OSB4: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0x0700-0x0707, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0x0708-0x070f, BIOS settings: hdc:DMA, hdd:DMA
hda: LG CD-ROM CRN-8245B, ATAPI CD/DVD-ROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hda: ATAPI 24X CD-ROM drive, 128kB Cache, (U)DMA
Uniform CD-ROM driver Revision: 3.20
ide-floppy driver 0.99.newide
ACPI: PCI Interrupt 0000:01:03.0[A] -> GSI 28 (level, low) -> IRQ 16
scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0
<Adaptec aic7892 Ultra160 SCSI adapter>
aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
ACPI: PCI Interrupt 0000:01:05.0[A] -> GSI 20 (level, low) -> IRQ 17
scsi1 : IBM PCI ServeRAID 7.12.05 Build 761 <ServeRAID 4L>
Vendor: IBM Model: SERVERAID Rev: 1.00
Type: Direct-Access ANSI SCSI revision: 02
Vendor: IBM Model: SERVERAID Rev: 1.00
Type: Direct-Access ANSI SCSI revision: 02
Vendor: IBM Model: SERVERAID Rev: 1.00
Type: Processor ANSI SCSI revision: 02
Vendor: IBM Model: FTlV1 S2 Rev: 0
Type: Processor ANSI SCSI revision: 02
Fusion MPT base driver 3.03.07
Copyright (c) 1999-2005 LSI Logic Corporation
Fusion MPT SPI Host driver 3.03.07
Fusion MPT misc device (ioctl) driver 3.03.07
mptctl: Registered with Fusion MPT base driver
mptctl: /dev/mptctl @ (major,minor=10,220)
usbmon: debugfs is not available
usbcore: registered new driver libusual
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
input: AT Translated Set 2 keyboard as /class/input/input0
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 4096 (order: 3, 32768 bytes)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP reno registered
Initializing IPsec netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 8
NET: Registered protocol family 20
Using IPI No-Shortcut mode
Freeing unused kernel memory: 208k freed
Loading, please wait...
Begin: Loading essential drivers... ...
logips2pp: Detected unknown logitech mouse model 0
eepro100.c:v1.09j-t 9/29/99 Donald Becker http://www.scyld.com/network/eepro100.html
eepro100.c: $Revision: 1.36 $ 2000/11/17 Modified by Andrey V. Savochkin <saw@saw.sw.com.sg> and others
ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 27 (level, low) -> IRQ 18
eth0: 0000:00:02.0, 00:02:55:C7:CA:D8, IRQ 18.
Board assembly 754338-001, Physical connectors present: RJ45
Primary interface chip i82555 PHY #1.
General self-test: passed.
Serial sub-system self-test: passed.
Internal registers self-test: passed.
ROM checksum self-test: passed (0x04f4518b).
ACPI: PCI Interrupt 0000:00:0a.0[A] -> GSI 25 (level, low) -> IRQ 19
eth1: 0000:00:0a.0, 00:02:55:C7:CA:D9, IRQ 19.
Board assembly 754338-001, Physical connectors present: RJ45
Primary interface chip i82555 PHY #1.
General self-test: passed.
Serial sub-system self-test: passed.
Internal registers self-test: passed.
ROM checksum self-test: passed (0x04f4518b).
Intel(R) PRO/1000 Network Driver - version 6.3.9-k4
Copyright (c) 1999-2005 Intel Corporation.
Done.
Begin: Running /scripts/init-premount ...
input: PS/2 Logitech Mouse as /class/input/input1
Done.
Begin: Mounting root file system... ...
Begin: Running /scripts/local-top ...
device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@redhat.com
/scripts/local-top/lvm: 36: vgchange: not found
Done.
SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
sda: assuming Write Enabled
sda: assuming drive cache: write through
SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
sda: assuming Write Enabled
sda: assuming drive cache: write through
sda: sda1 sda2 < sda5 >
sd 1:0:0:0: Attached scsi disk sda
SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
sdb: assuming Write Enabled
sdb: assuming drive cache: write through
SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
sdb: assuming Write Enabled
sdb: assuming drive cache: write through
sdb: unknown partition table
sd 1:0:1:0: Attached scsi disk sdb
Begin: Running /scripts/local-premount ...
Done.
EXT3-fs: INFO: recovery required on readonly filesystem.
EXT3-fs: write access will be enabled during recovery.
kjournald starting. Commit interval 5 seconds
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
Begin: Running /scripts/log-bottom ...
Done.
Done.
Begin: Running /scripts/init-bottom ...
Done.
INIT: version 2.86 booting
Starting the hotplug events dispatcher: udevd.
Synthesizing the initial hotplug events...done.
Waiting for /dev to be fully populated...done.
Loading /etc/console/boottime.kmap.gz
Activating swap.
Adding 1019880k swap on /dev/sda5. Priority:-1 extents:1 across:1019880k
Checking root file system...
fsck 1.37 (21-Mar-2005)
/dev/sda1: clean, 58437/512000 files, 445341/1023996 blocks
EXT3 FS on sda1, internal journal
System time was Sat Aug 12 00:22:00 UTC 2006.
Setting the System Clock using the Hardware Clock as reference...
System Clock set. System local time is now Sat Aug 12 00:22:01 UTC 2006.
Calculating module dependencies...done.
Loading modules...
auto
FATAL: Module auto not found.
3c59x
softdog
Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0)
tulip
Linux Tulip driver version 1.1.13 (May 11, 2002)
e100
e100: Intel(R) PRO/100 Network Driver, 3.5.10-k2-NAPI
e100: Copyright(c) 1999-2005 Intel Corporation
All modules loaded.
Loading device-mapper support.
Starting Enterprise Volume Management System: device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
device-mapper: dm-linear: Device lookup failed
device-mapper: error adding target to table
evms.
Checking all file systems...
fsck 1.37 (21-Mar-2005)
Setting kernel variables ...
... done.
Loading the saved-state of the serial devices...
Cannot get serial info: Invalid argument
Cannot get serial info: Invalid argument
/dev/ttyS1: No such file or directory
/dev/ttyS1: No such file or directory
Mounting local filesystems...
Cleaning /tmp /var/run /var/lock.
Cleaning: /etc/network/ifstate.
Setting up IP spoofing protection: rp_filter.
Configuring network interfaces: done.
Starting portmap daemon: portmap.
Setting the System Clock using the Hardware Clock as reference...
System Clock set. Local time: Fri Aug 11 17:22:07 PDT 2006
Recovering jove files ... Done.
Running ntpdate to synchronize clock.
Initializing random number generator...done.
Recovering nvi editor sessions... done.
Setting up X server socket directory /tmp/.X11-unix...done.
Setting up ICE socket directory /tmp/.ICE-unix...done.
INIT: Entering runlevel: 2
Starting system log daemon: syslogd.
Starting kernel log daemon: klogd.
Starting virtual private network daemon:.
Starting portmap daemon: portmap.
Starting NFS common utilities: statd lockd.
Starting internet superserver: inetd.
* The mptctl module is missing. Please have a look at the README.Debian.gz.
.
Exporting directories for NFS kernel daemon...Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
10.27.3.21:/export/xen/fs/stevegt/crashme1/root: No such file or directory
10.27.3.18:/export/xen/fs/stevegt/xentest2/root: No such file or directory
10.27.3.17:/export/xen/fs/stevegt/xentest1/root: No such file or directory
10.27.3.8:/export/xen/fs/cclarke/umltest1/root: No such file or directory
hotaru.chaosring.Org:/export/xen/fs/stevegt/crashme1/root: No such file or directory
rn-2.refactored-networks.com:/export/xen/fs/stevegt/xentest2/root: No such file or directory
xentest1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/xentest1/root: No such file or directory
umltest1.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/umltest1/root: No such file or directory
10.27.3.26:/export/xen/fs/stevegt/gforge/root: No such file or directory
10.27.3.23:/export/xen/fs/baseline/sarge/root: No such file or directory
10.27.3.22:/export/xen/fs/stevegt/build1/root: No such file or directory
10.27.3.16:/export/xen/fs/baseline/woody/root: No such file or directory
bugs.t7a.Org:/export/xen/fs/stevegt/gforge/root: No such file or directory
sarge.baseline.TerraLuna.Org:/export/xen/fs/baseline/sarge/root: No such file or directory
build1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/build1/root: No such file or directory
woody.baseline.TerraLuna.Org:/export/xen/fs/baseline/woody/root: No such file or directory
10.27.3.19:/export/xen/fs/cclarke/ebond/root: No such file or directory
ebond.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/ebond/root: No such file or directory
10.27.3.29:/export/xen/fs/dmasten/mss0/root: No such file or directory
10.27.3.27:/export/xen/fs/cclarke/ccms/root: No such file or directory
10.27.3.25:/export/xen/fs/baseline/sid/root: No such file or directory
10.27.3.24:/export/xen/fs/stevegt/cvs1/root: No such file or directory
sid.stevegt.TerraLuna.Org:/export/xen/fs/baseline/sid/root: No such file or directory
cvs1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/cvs1/root: No such file or directory
10.27.3.28:/export/xen/fs/stevegt/t7a/root: No such file or directory
10.27.3.20:/export/xen/fs/stevegt/tcx/root: No such file or directory
tcx.TerraLuna.Org:/export/xen/fs/stevegt/tcx/root: No such file or directory
n2h54.prd.TerraLuna.Org:/export/xen: No such file or directory
n2h53.prd.TerraLuna.Org:/export/xen: No such file or directory
n2h51.prd.TerraLuna.Org:/export/xen: No such file or directory
n2h41.prd.TerraLuna.Org:/export/xen: No such file or directory
done.
Starting NFS kernel daemon: nfsdNFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
NFSD: starting 90-second grace period
mountd.
Starting OpenBSD Secure Shell server: sshdNET: Registered protocol family 10
lo: Disabled Privacy Extensions
IPv6 over IPv4 tunneling driver
.
Starting NTP server: ntpd.
AFS module /lib/modules/2.6.16.13-xen/fs/openafs.mp.o does not exist. Not starting AFS.
Please consider building kernel modules using instructions in
/usr/share/doc/openafs-client/README.modules
Starting DRBD resources: drbd: initialised. Version: 0.7.20 (api:79/proto:74)
drbd: SVN Revision: 2260 build by root@n4h8, 2006-08-04 11:00:44
drbd: registered as block device major 147
[ d0 drbd7: resync bitmap: bits=1048576 words=32768
drbd7: size = 4095 MB (4194303 KB)
drbd7: 0 KB marked out-of-sync by on disk bit-map.
drbd7: No usable activity log found.
drbd7: drbdsetup [3825]: cstate Unconfigured --> StandAlone
d1 drbd8: resync bitmap: bits=1048576 words=32768
drbd8: size = 4095 MB (4194303 KB)
drbd8: 0 KB marked out-of-sync by on disk bit-map.
drbd8: Found 4 transactions (46 active extents) in activity log.
drbd8: Marked additional 156 MB as out-of-sync based on AL.
drbd8: drbdsetup [3829]: cstate Unconfigured --> StandAlone
d2 drbd9: resync bitmap: bits=2621440 words=81920
drbd9: size = 9 GB (10485759 KB)
drbd9: 0 KB marked out-of-sync by on disk bit-map.
drbd9: Found 4 transactions (192 active extents) in activity log.
drbd9: drbdsetup [3833]: cstate Unconfigured --> StandAlone
d3 drbd10: resync bitmap: bits=1835008 words=57344
drbd10: size = 7167 MB (7340031 KB)
drbd10: 0 KB marked out-of-sync by on disk bit-map.
drbd10: Found 4 transactions (192 active extents) in activity log.
drbd10: Marked additional 508 MB as out-of-sync based on AL.
drbd10: drbdsetup [3837]: cstate Unconfigured --> StandAlone
d4 drbd11: resync bitmap: bits=786432 words=24576
drbd11: size = 3071 MB (3145727 KB)
drbd11: 0 KB marked out-of-sync by on disk bit-map.
drbd11: Found 4 transactions (192 active extents) in activity log.
drbd11: drbdsetup [3841]: cstate Unconfigured --> StandAlone
d5 drbd12: resync bitmap: bits=786432 words=24576
drbd12: size = 3071 MB (3145727 KB)
drbd12: 0 KB marked out-of-sync by on disk bit-map.
drbd12: Found 4 transactions (136 active extents) in activity log.
drbd12: Marked additional 508 MB as out-of-sync based on AL.
drbd12: drbdsetup [3845]: cstate Unconfigured --> StandAlone
s0 s1 s2 s3 s4 s5 n0 drbd7: drbdsetup [3883]: cstate StandAlone --> Unconnected
drbd7: drbd7_receiver [3884]: cstate Unconnected --> WFConnection
n1 drbd8: drbdsetup [3891]: cstate StandAlone --> Unconnected
drbd8: drbd8_receiver [3892]: cstate Unconnected --> WFConnection
n2 drbd9: drbdsetup [3899]: cstate StandAlone --> Unconnected
drbd9: drbd9_receiver [3900]: cstate Unconnected --> WFConnection
n3 drbd10: drbdsetup [3907]: cstate StandAlone --> Unconnected
drbd10: drbd10_receiver [3908]: cstate Unconnected --> WFConnection
n4 drbd11: drbdsetup [3915]: cstate StandAlone --> Unconnected
drbd11: drbd11_receiver [3916]: cstate Unconnected --> WFConnection
n5 drbd12: drbdsetup [3923]: cstate StandAlone --> Unconnected
drbd12: drbd12_receiver [3924]: cstate Unconnected --> WFConnection
].
..........
***************************************************************
DRBD's startup script waits for the peer node(s) to appear.
- In case this node was already a degraded cluster before the
reboot the timeout is 120 seconds. [degr-wfc-timeout]
- If the peer was available before the reboot the timeout will
expire after 0 seconds. [wfc-timeout]
(These values are for resource 'ds1.t7a.org'; 0 sec -> wait forever)
To abort waiting enter 'yes' [ 18]:drbd7: drbd7_receiver [3884]: cstate WFConnection --> WFReportParams
drbd7: Handshake successful: DRBD Network Protocol version 74
drbd7: Connection established.
drbd7: I am(S): 1:00000002:00000001:00000018:00000002:01
drbd7: Peer(S): 1:00000002:00000001:00000019:00000002:10
drbd7: drbd7_receiver [3884]: cstate WFReportParams --> WFBitMapT
drbd7: Secondary/Unknown --> Secondary/Secondary
drbd7: drbd7_receiver [3884]: cstate WFBitMapT --> SyncTarget
drbd7: Resync started as SyncTarget (need to sync 520192 KB [130048 bits set]).
drbd9: drbd9_receiver [3900]: cstate WFConnection --> WFReportParams
drbd9: Handshake successful: DRBD Network Protocol version 74
drbd9: Connection established.
drbd9: I am(S): 1:00000002:00000001:0000000d:00000001:01
drbd9: Peer(S): 1:00000002:00000001:0000000e:00000001:10
drbd9: drbd9_receiver [3900]: cstate WFReportParams --> WFBitMapT
drbd9: Secondary/Unknown --> Secondary/Secondary
drbd9: drbd9_receiver [3900]: cstate WFBitMapT --> SyncTarget
drbd9: Resync started as SyncTarget (need to sync 290816 KB [72704 bits set]).
drbd9: drbd9_receiver [3900]: cstate SyncTarget --> PausedSyncT
drbd9: Syncer waits for sync group.
drbd11: drbd11_receiver [3916]: cstate WFConnection --> WFReportParams
drbd11: Handshake successful: DRBD Network Protocol version 74
drbd11: Connection established.
drbd11: I am(S): 1:00000002:00000001:00000008:00000001:01
drbd11: Peer(S): 1:00000002:00000001:00000009:00000001:10
drbd11: drbd11_receiver [3916]: cstate WFReportParams --> WFBitMapT
drbd11: Secondary/Unknown --> Secondary/Secondary
drbd11: drbd11_receiver [3916]: cstate WFBitMapT --> SyncTarget
drbd11: Resync started as SyncTarget (need to sync 274432 KB [68608 bits set]).
drbd11: drbd11_receiver [3916]: cstate SyncTarget --> PausedSyncT
drbd11: Syncer waits for sync group.
drbd8: drbd8_receiver [3892]: cstate WFConnection --> WFReportParams
drbd8: Handshake successful: DRBD Network Protocol version 74
drbd8: Connection established.
drbd8: I am(S): 1:00000002:00000001:00000017:00000001:11
drbd8: Peer(S): 1:00000002:00000001:00000017:00000001:01
drbd8: drbd8_receiver [3892]: cstate WFReportParams --> WFBitMapS
drbd8: Secondary/Unknown --> Secondary/Secondary
drbd8: drbd8_receiver [3892]: cstate WFBitMapS --> SyncSource
drbd8: Resync started as SyncSource (need to sync 159744 KB [39936 bits set]).
drbd8: drbd8_receiver [3892]: cstate SyncSource --> PausedSyncS
drbd8: Syncer waits for sync group.
drbd12: drbd12_receiver [3924]: cstate WFConnection --> WFReportParams
drbd12: Handshake successful: DRBD Network Protocol version 74
drbd12: Connection established.
drbd12: I am(S): 1:00000002:00000001:00000001:00000001:11
drbd12: Peer(S): 1:00000002:00000001:00000001:00000001:01
drbd12: drbd12_receiver [3924]: cstate WFReportParams --> WFBitMapS
drbd12: Secondary/Unknown --> Secondary/Secondary
drbd12: drbd12_receiver [3924]: cstate WFBitMapS --> SyncSource
drbd12: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]).
drbd12: drbd12_receiver [3924]: cstate SyncSource --> PausedSyncS
drbd12: Syncer waits for sync group.
drbd10: drbd10_receiver [3908]: cstate WFConnection --> WFReportParams
drbd10: Handshake successful: DRBD Network Protocol version 74
drbd10: Connection established.
drbd10: I am(S): 1:00000002:00000001:0000000b:00000001:11
drbd10: Peer(S): 1:00000002:00000001:0000000b:00000001:01
drbd10: drbd10_receiver [3908]: cstate WFReportParams --> WFBitMapS
drbd10: Secondary/Unknown --> Secondary/Secondary
drbd10: drbd10_receiver [3908]: cstate WFBitMapS --> SyncSource
drbd10: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]).
drbd10: drbd10_receiver [3908]: cstate SyncSource --> PausedSyncS
drbd10: Syncer waits for sync group.
Heartbeat not configured: /etc/ha.d/ha.cf not found.
Heartbeat failure [rc=1]. Failed.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting watchdog daemon: watchdog.
openMosix: ERROR: Cannot find the /proc/hpc directory.
openMosix: ERROR: Looks like this is not an openMosix enabled kernel.
openMosix: ERROR: Configuration ABORTED.
openMosix: HINT: need to recompile your kernel with openMosix support enabled?
openMosix: HINT: need to update your boot manager (lilo, grub, etc)?
openMosix: HINT: have you read the docs at http://openMosix.sf.net/ ?
Bridge firewalling registered
device vif0.0 entered promiscuous mode
xenbr0: port 1(vif0.0) entering learning state
xenbr0: topology change detected, propagating
xenbr0: port 1(vif0.0) entering forwarding state
device peth0 entered promiscuous mode
xenbr0: port 2(peth0) entering learning state
xenbr0: topology change detected, propagating
xenbr0: port 2(peth0) entering forwarding state
Starting auto Xen domains: afs1.t7a.orgdrbd9: Secondary/Secondary --> Secondary/Primary
device vif1.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif1.0: link is not ready
ip_tables: (C) 2000-2006 Netfilter Core Team
Error: Device 2049 (vbd) could not be connected. Backend device not found.
xenbr0: port 3(vif1.0) entering disabled state
! build2.t7a.orgdevice vif1.0 left promiscuous mode
xenbr0: port 3(vif1.0) entering disabled state
device vif2.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif2.0: link is not ready
drbd10: Secondary/Secondary --> Primary/Secondary
ADDRCONF(NETDEV_CHANGE): vif2.0: link becomes ready
xenbr0: port 3(vif2.0) entering learning state
xenbr0: topology change detected, propagating
xenbr0: port 3(vif2.0) entering forwarding state
ds1.t7a.orgdrbd7: Secondary/Secondary --> Secondary/Primary
device vif3.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif3.0: link is not ready
Error: Device 2049 (vbd) could not be connected. Backend device not found.
! kdc1.t7a.orgxenbr0: port 4(vif3.0) entering disabled state
device vif3.0 left promiscuous mode
xenbr0: port 4(vif3.0) entering disabled state
drbd8: Secondary/Secondary --> Primary/Secondary
device vif4.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif4.0: link is not ready
ADDRCONF(NETDEV_CHANGE): vif4.0: link becomes ready
xenbr0: port 4(vif4.0) entering learning state
xenbr0: topology change detected, propagating
xenbr0: port 4(vif4.0) entering forwarding state
w1.cdint.comdrbd11: Secondary/Secondary --> Secondary/Primary
device vif5.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif5.0: link is not ready
Error: Device 2049 (vbd) could not be connected. Backend device not found.
xenbr0: port 5(vif5.0) entering disabled state
! webkdc.t7a.orgdevice vif5.0 left promiscuous mode
xenbr0: port 5(vif5.0) entering disabled state
device vif6.0 entered promiscuous mode
ADDRCONF(NETDEV_UP): vif6.0: link is not ready
drbd12: Secondary/Secondary --> Primary/Secondary
ADDRCONF(NETDEV_CHANGE): vif6.0: link becomes ready
xenbr0: port 5(vif6.0) entering learning state
xenbr0: topology change detected, propagating
xenbr0: port 5(vif6.0) entering forwarding state
[done]
_sudZUZ#Z#XZo=_ DDDD EEEEEE BBBB IIIIII AAAA NN NN
_jmZZ2!!~---~!!X##wa DD DD EE BB BB II AA AA NNN NN
.<wdP~~ -!YZL, DD DD EEEEE BBBBB II AAAAAA NNNN NN
.mX2' _%aaa__ XZ[. DD DD EE BB BB II AA AA NN NNNN
oZ[ _jdXY!~?S#wa ]Xb; DDDD EEEEEE BBBBB IIIIII AA AA NN NN
_#e' .]X2( ~Xw| )XXc
.2Z` ]X[. xY| ]oZ( Linux Version 2.6.16.13-xen
.2#; )3k; _s!~ jXf` Compiled #3 SMP Fri Aug 4 10:57:44 PDT 2006
1Z> -]Xb/ ~ __#2( Two 1.13GHz Intel Intel(R) Pentium(R) III CPU family 1133MHz Processors
-Zo; +!4ZwaaaauZZXY' 72M RAM
*#[, ~-?!!!!!!-~ 4523.20 Bogomips Total
XUb;. n4h34
)YXL,,
+3#bc,
-)SSL,,
~~~~~
Updating the Linuxlogo... done.
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login: drbd7: Resync done (total 137 sec; paused 0 sec; 3796 K/sec)
drbd7: drbd7_worker [3826]: cstate SyncTarget --> Connected
drbd8: Syncer continues.
drbd8: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
drbd9: Syncer continues.
drbd9: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
drbd10: Syncer continues.
drbd10: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
drbd11: Syncer continues.
drbd11: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
drbd12: Syncer continues.
drbd12: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
drbd8: Resync done (total 211 sec; paused 137 sec; 2156 K/sec)
drbd8: drbd8_worker [3830]: cstate SyncSource --> Connected
drbd11: Resync done (total 239 sec; paused 137 sec; 2688 K/sec)
drbd11: drbd11_worker [3842]: cstate SyncTarget --> Connected
drbd9: Resync done (total 243 sec; paused 137 sec; 2740 K/sec)
drbd9: drbd9_worker [3834]: cstate SyncTarget --> Connected
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login:
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login:
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login:
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login:
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login:
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login: drbd12: Resync done (total 306 sec; paused 136 sec; 3056 K/sec)
drbd12: drbd12_worker [3846]: cstate SyncSource --> Connected
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login:
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login: drbd10: Resync done (total 310 sec; paused 136 sec; 2988 K/sec)
drbd10: drbd10_worker [3838]: cstate SyncSource --> Connected
Debian GNU/Linux 3.0 n4h34 ttyS0
n4h34 login: Unable to handle kernel paging request at virtual address c0976590
printing eip:
c03d6ed7
*pde = ma 3cf91067 pa 00f91067
*pte = ma 00000000 pa fffff000
Oops: 0000 [#1]
SMP
Modules linked in: iptable_filter ip_tables x_tables bridge drbd ipv6 nfsd lockd sunrpc e100 tulip softdog 3c59x evdev sd_mod dm_mod thermal processor fan e1000 eepro100 mii tg3
CPU: 0
EIP: 0061:[<c03d6ed7>] Not tainted VLI
EFLAGS: 00010206 (2.6.16.13-xen #3)
EIP is at skb_copy_bits+0x127/0x280
eax: c0976000 ebx: 000005a8 ecx: 0000016a edx: c3711720
esi: c0976590 edi: c37110e0 ebp: 000005a8 esp: c00818c8
ds: 007b es: 007b ss: 0069
Process drbd12_receiver (pid: 3924, threadinfo=c0080000 task=c3efda90)
Stack: <0>c1012ec0 00000002 c03d65ef c1d51a00 000005ea 00000042 00000000 00000000
00000020 c3edc800 c376fd64 c03d6b0f c376fd64 00000042 c37110e0 000005a8
c1a70000 c54690c0 00000000 c1d51ac0 c3edc800 c376fd64 c03dc478 c376fd64
Call Trace:
[<c03d65ef>] pskb_expand_head+0xdf/0x140
[<c03d6b0f>] __pskb_pull_tail+0x7f/0x320
[<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
[<c03dc478>] dev_queue_xmit+0x328/0x370
[<c5462f7e>] br_dev_queue_push_xmit+0xbe/0x140 [bridge]
[<c5469212>] br_nf_post_routing+0x102/0x1c0 [bridge]
[<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c5463061>] br_forward_finish+0x61/0x70 [bridge]
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c5468995>] br_nf_forward_finish+0x75/0x130 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5468b38>] br_nf_forward_ip+0xe8/0x190 [bridge]
[<c5468920>] br_nf_forward_finish+0x0/0x130 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5463167>] __br_forward+0x77/0x80 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5463fbf>] br_handle_frame_finish+0xdf/0x160 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c5467d89>] br_nf_pre_routing_finish+0xf9/0x370 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c0322e3a>] loopback_start_xmit+0xba/0x110
[<c0400d70>] ip_finish_output+0x0/0x220
[<c03dc07e>] dev_hard_start_xmit+0x5e/0x130
[<c03dc3b5>] dev_queue_xmit+0x265/0x370
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c54685fc>] br_nf_pre_routing+0x26c/0x520 [bridge]
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c546422d>] br_handle_frame+0x1ed/0x230 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c03dcb21>] netif_receive_skb+0x1a1/0x330
[<c03dcd87>] process_backlog+0xd7/0x190
[<c03dcf2a>] net_rx_action+0xea/0x230
[<c0125915>] __do_softirq+0xf5/0x120
[<c01259d5>] do_softirq+0x95/0xa0
[<c0125a42>] local_bh_enable+0x62/0xa0
[<c0406bd1>] tcp_prequeue_process+0x71/0x80
[<c04070e9>] tcp_recvmsg+0x349/0x750
[<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
[<c03d5085>] sock_common_recvmsg+0x55/0x70
[<c03d11cf>] sock_recvmsg+0xef/0x110
[<c03143fa>] force_evtchn_callback+0xa/0x10
[<c0147163>] mempool_alloc+0x33/0xe0
[<c01367d0>] autoremove_wake_function+0x0/0x60
[<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
[<c02b2ee0>] generic_make_request+0xf0/0x160
[<c529c680>] drbd_recv+0x90/0x190 [drbd]
[<c529cdec>] drbd_recv_header+0x2c/0xf0 [drbd]
[<c529e580>] receive_DataRequest+0x0/0x7d0 [drbd]
[<c52a045c>] drbdd+0x1c/0x150 [drbd]
[<c52a105a>] drbdd_init+0x7a/0x1a0 [drbd]
[<c52a7136>] drbd_thread_setup+0x86/0xf0 [drbd]
[<c52a70b0>] drbd_thread_setup+0x0/0xf0 [drbd]
[<c0102f75>] kernel_thread_helper+0x5/0x10
Code: 8b 4c 24 30 8b 7c 24 34 8b 91 a0 00 00 00 8b 4c 24 18 0f b7 74 ca 18 8b 4c 24 14 8d 34 06 01 fe 29 ce 8b 7c 24 38 89 d9 c1 e9 02 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 89 04 24 ba 02 00 00 00 89 54
<0>Kernel panic - not syncing: Fatal exception in interrupt
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
--
Stephen G. Traugott (KG6HDQ)
Managing Partner, TerraLuna LLC
stevegt@TerraLuna.Org -- http://www.t7a.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
2006-08-12 1:50 ` Unable to handle kernel paging request Steve Traugott
@ 2006-08-12 4:34 ` Steve Traugott
2006-08-12 5:50 ` Steve Traugott
0 siblings, 1 reply; 31+ messages in thread
From: Steve Traugott @ 2006-08-12 4:34 UTC (permalink / raw)
To: Ian Pratt; +Cc: xen-devel
Okay, for some reason I'm now able to crash dom0 pretty reliably in
this configuration (domU root filesystem in DRBD) simply by running
this in domU:
nc -u -b 255.255.255.255 54321 < /dev/zero
Anyone else want to give that a try on 10868 or later, and see if they
can duplicate it, with or without DRBD?
Steve
On Fri, Aug 11, 2006 at 06:50:50PM -0700, Steve Traugott wrote:
> Uh oh....
>
> On Sat, Aug 05, 2006 at 08:38:03AM +0100, Ian Pratt wrote:
> > > So I built -unstable changeset 10868, and ran an even heavier workload
> > > (the above, plus 'bonnie' in the guests) on dom0 and two guests
> > > overnight, and they experienced no soft lockups; running -unstable,
> > > changeset 10868, credit scheduler. This same workload would have
> > > caused soft lockups within seconds in -testing changeset 9732 using
> > > the sedf scheduler; I may not have been able to get it started at all.
> > > Response time remained subsecond under -unstable; -testing would have
> > > been on its knees.
> >
> > That's good to hear. 3.0.3 is going to be a big leap forward in many
> > ways.
>
> Same environment as we've discussed earlier in this thread, now on
> unstable changeset 10868, Xen on top of DRBD, this one seems to be a
> reliable crash when under combined disk and network load, still
> isolating but would welcome suggestions...
>
> Steve
>
>
> root (hd0,0)
> Filesystem type is ext2fs, partition type 0x83
> kernel /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
> [Multiboot-elf, <0x100000:0x7a008:0x55ff8>, shtab=0x1d0078, entry=0x100000]
> module /boot/vmlinuz-2.6.16.13-xen root=801 ro console=ttyS0 ramdisk_size=32768
> [Multiboot-module @ 0x1d1000, 0x494ed0 bytes]
> module /boot/initrd.img-2.6.16.13-xen
> [Multiboot-module @ 0x666000, 0x9a6400 bytes]
> __ __ _____ ___ _ _ _
> \ \/ /___ _ __ |___ / / _ \ _ _ _ __ ___| |_ __ _| |__ | | ___
> \ // _ \ '_ \ |_ \| | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> / \ __/ | | | ___) | |_| |__| |_| | | | \__ \ || (_| | |_) | | __/
> /_/\_\___|_| |_| |____(_)___/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
>
> http://www.cl.cam.ac.uk/netos/xen
> University of Cambridge Computer Laboratory
>
> Xen version 3.0-unstable (root@prd.terraluna.org) (gcc version 3.3.5 (Debian 1:3.3.5-13)) Fri Aug 4 10:54:50 PDT 2006
> Latest ChangeSet: Sat Jul 29 06:05:59 2006 +0100 10868:d2bf1a7cc131
>
> (XEN) Command line: /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
> (XEN) Physical RAM map:
> (XEN) 0000000000000000 - 000000000009c000 (usable)
> (XEN) 000000000009c000 - 00000000000a0000 (reserved)
> (XEN) 00000000000e0000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 000000003ffec340 (usable)
> (XEN) 000000003ffec340 - 000000003fff0000 (ACPI data)
> (XEN) 000000003fff0000 - 0000000040000000 (reserved)
> (XEN) 00000000fec00000 - 0000000100000000 (reserved)
> (XEN) System RAM: 1023MB (1048096kB)
> (XEN) Xen heap: 10MB (10396kB)
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) PAE disabled.
> (XEN) found SMP MP-table at 0009c1d0
> (XEN) DMI 2.3 present.
> (XEN) Using APIC driver default
> (XEN) ACPI: RSDP (v000 IBM ) @ 0x000fdfd0
> (XEN) ACPI: RSDT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff80
> (XEN) ACPI: FADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff00
> (XEN) ACPI: MADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffefe80
> (XEN) ACPI: DSDT (v001 IBM SEREMRLD 0x00001000 MSFT 0x0100000b) @ 0x00000000
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> (XEN) Processor #3 6:11 APIC version 17
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> (XEN) Processor #0 6:11 APIC version 17
> (XEN) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
> (XEN) ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
> (XEN) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
> (XEN) ACPI: IRQ3 used by override.
> (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Initializing CPU#0
> (XEN) Detected 1130.180 MHz processor.
> (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
> (XEN) CPU: L2 cache: 512K
> (XEN) Intel machine check architecture supported.
> (XEN) Intel machine check reporting enabled on CPU#0.
> (XEN) CPU0: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
> (XEN) Booting processor 1/0 eip 90000
> (XEN) Initializing CPU#1
> (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
> (XEN) CPU: L2 cache: 512K
> (XEN) Intel machine check architecture supported.
> (XEN) Intel machine check reporting enabled on CPU#1.
> (XEN) CPU1: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=0 apic2=-1 pin2=-1
> (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> (XEN) ...trying to set up timer (IRQ0) through the 8259A ... failed.
> (XEN) ...trying to set up timer as Virtual Wire IRQ... works.
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 1.193MHz PIT
> (XEN) Brought up 2 CPUs
> (XEN) Machine check exception polling timer started.
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Domain 0 kernel supports features = { 0000001f }.
> (XEN) Domain 0 kernel requires features = { 00000000 }.
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 3c000000->3e000000 (8192 pages to be allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: c0100000->c05d5d74
> (XEN) Init. ramdisk: c05d6000->c0f7c400
> (XEN) Phys-Mach map: c0f7d000->c0f8d000
> (XEN) Start info: c0f8d000->c0f8e000
> (XEN) Page tables: c0f8e000->c0f94000
> (XEN) Boot stack: c0f94000->c0f95000
> (XEN) TOTAL: c0000000->c1400000
> (XEN) ENTRY ADDRESS: c0100000
> (XEN) Dom0 has maximum 2 VCPUs
> (XEN) Initrd len 0x9a6400, start at 0xc05d6000
> (XEN) Scrubbing Free RAM: ...........done.
> (XEN) Xen trace buffers: disabled
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen).
> Linux version 2.6.16.13-xen (root@n4h8) (gcc version 3.3.5 (Debian 1:3.3.5-13)) #3 SMP Fri Aug 4 10:57:44 PDT 2006
> BIOS-provided physical RAM map:
> Xen: 0000000000000000 - 0000000004800000 (usable)
> 0MB HIGHMEM available.
> 72MB LOWMEM available.
> DMI 2.3 present.
> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
> ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
> IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
> ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
> Enabling APIC mode: Flat. Using 2 I/O APICs
> Using ACPI (MADT) for SMP configuration information
> Allocating PCI resources starting at 50000000 (gap: 40000000:bec00000)
> Built 1 zonelists
> Kernel command line: root=801 ro console=ttyS0 ramdisk_size=32768
> Enabling fast FPU save and restore... done.
> Enabling unmasked SIMD FPU exception support... done.
> Initializing CPU#0
> PID hash table entries: 512 (order: 9, 8192 bytes)
> Xen reported: 1130.113 MHz processor.
> Console: colour VGA+ 80x25
> Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
> Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
> Software IO TLB enabled:
> Aperture: 2 megabytes
> Kernel range: 0x00000000c10f0000 - 0x00000000c12f0000
> vmalloc area: c5000000-fb7fe000, maxmem 33ffe000
> Memory: 47400k/73728k available (3389k kernel code, 18068k reserved, 1044k data, 208k init, 0k highmem)
> Checking if this processor honours the WP bit even in supervisor mode... Ok.
> Calibrating delay using timer specific routine.. 2261.60 BogoMIPS (lpj=11308023)
> Security Framework v1.0.0 initialized
> Capability LSM initialized
> Mount-cache hash table entries: 512
> CPU: L1 I cache: 16K, L1 D cache: 16K
> CPU: L2 cache: 512K
> Checking 'hlt' instruction... OK.
> ENABLING IO-APIC IRQs
> Enabling SMP...
> Brought up 2 CPUs
> Initializing CPU#1
> migration_cost=1895
> checking if image is initramfs... it is
> Freeing initrd memory: 9881k freed
> Grant table initialized
> NET: Registered protocol family 16
> ACPI: bus type pci registered
> PCI: Using configuration type 1
> ACPI: Subsystem revision 20060127
> ACPI: Interpreter enabled
> ACPI: Using IOAPIC for interrupt routing
> ACPI: PCI Root Bridge [PCI0] (0000:00)
> ACPI: PCI Interrupt Link [LPE1] (IRQs *10)
> ACPI: PCI Interrupt Link [LPE2] (IRQs *10)
> ACPI: PCI Interrupt Link [LPVI] (IRQs) *0, disabled.
> ACPI: PCI Interrupt Link [LPUS] (IRQs *7)
> ACPI: PCI Root Bridge [PCI1] (0000:01)
> ACPI: PCI Interrupt Link [LPSA] (IRQs *9)
> ACPI: PCI Interrupt Link [LP1A] (IRQs) *0, disabled.
> ACPI: PCI Interrupt Link [LP1B] (IRQs) *0, disabled.
> ACPI: PCI Interrupt Link [LP2A] (IRQs *9)
> ACPI: PCI Interrupt Link [LP2B] (IRQs) *0, disabled.
> Linux Plug and Play Support v0.97 (c) Adam Belay
> xen_mem: Initialising balloon driver.
> SCSI subsystem initialized
> usbcore: registered new driver usbfs
> usbcore: registered new driver hub
> PCI: Using ACPI for IRQ routing
> PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
> IA-32 Microcode Update Driver: v1.14-xen <tigran@veritas.com>
> VFS: Disk quotas dquot_6.5.1
> Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
> JFS: nTxBlock = 512, nTxLock = 4096
> SGI XFS with ACLs, security attributes, realtime, large block numbers, no debug enabled
> Initializing Cryptographic API
> io scheduler noop registered
> io scheduler anticipatory registered (default)
> io scheduler deadline registered
> io scheduler cfq registered
> PNP: No PS/2 controller found. Probing ports directly.
> serio: i8042 AUX port at 0x60,0x64 irq 12
> serio: i8042 KBD port at 0x60,0x64 irq 1
> RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize
> Xen virtual console successfully installed as ttyS0
> Event-channel device installed.
> blkif_init: reqs=64, pages=704, mmap_vstart=0xc0800000
> Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
> ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
> SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
> SvrWks OSB4: chipset revision 0
> SvrWks OSB4: not 100% native mode: will probe irqs later
> ide0: BM-DMA at 0x0700-0x0707, BIOS settings: hda:DMA, hdb:DMA
> ide1: BM-DMA at 0x0708-0x070f, BIOS settings: hdc:DMA, hdd:DMA
> hda: LG CD-ROM CRN-8245B, ATAPI CD/DVD-ROM drive
> ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
> hda: ATAPI 24X CD-ROM drive, 128kB Cache, (U)DMA
> Uniform CD-ROM driver Revision: 3.20
> ide-floppy driver 0.99.newide
> ACPI: PCI Interrupt 0000:01:03.0[A] -> GSI 28 (level, low) -> IRQ 16
> scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0
> <Adaptec aic7892 Ultra160 SCSI adapter>
> aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
>
> ACPI: PCI Interrupt 0000:01:05.0[A] -> GSI 20 (level, low) -> IRQ 17
> scsi1 : IBM PCI ServeRAID 7.12.05 Build 761 <ServeRAID 4L>
> Vendor: IBM Model: SERVERAID Rev: 1.00
> Type: Direct-Access ANSI SCSI revision: 02
> Vendor: IBM Model: SERVERAID Rev: 1.00
> Type: Direct-Access ANSI SCSI revision: 02
> Vendor: IBM Model: SERVERAID Rev: 1.00
> Type: Processor ANSI SCSI revision: 02
> Vendor: IBM Model: FTlV1 S2 Rev: 0
> Type: Processor ANSI SCSI revision: 02
> Fusion MPT base driver 3.03.07
> Copyright (c) 1999-2005 LSI Logic Corporation
> Fusion MPT SPI Host driver 3.03.07
> Fusion MPT misc device (ioctl) driver 3.03.07
> mptctl: Registered with Fusion MPT base driver
> mptctl: /dev/mptctl @ (major,minor=10,220)
> usbmon: debugfs is not available
> usbcore: registered new driver libusual
> mice: PS/2 mouse device common for all mice
> md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
> md: bitmap version 4.39
> NET: Registered protocol family 2
> input: AT Translated Set 2 keyboard as /class/input/input0
> IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
> TCP established hash table entries: 4096 (order: 3, 32768 bytes)
> TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
> TCP: Hash tables configured (established 4096 bind 4096)
> TCP reno registered
> Initializing IPsec netlink socket
> NET: Registered protocol family 1
> NET: Registered protocol family 17
> NET: Registered protocol family 8
> NET: Registered protocol family 20
> Using IPI No-Shortcut mode
> Freeing unused kernel memory: 208k freed
> Loading, please wait...
> Begin: Loading essential drivers... ...
> logips2pp: Detected unknown logitech mouse model 0
> eepro100.c:v1.09j-t 9/29/99 Donald Becker http://www.scyld.com/network/eepro100.html
> eepro100.c: $Revision: 1.36 $ 2000/11/17 Modified by Andrey V. Savochkin <saw@saw.sw.com.sg> and others
> ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 27 (level, low) -> IRQ 18
> eth0: 0000:00:02.0, 00:02:55:C7:CA:D8, IRQ 18.
> Board assembly 754338-001, Physical connectors present: RJ45
> Primary interface chip i82555 PHY #1.
> General self-test: passed.
> Serial sub-system self-test: passed.
> Internal registers self-test: passed.
> ROM checksum self-test: passed (0x04f4518b).
> ACPI: PCI Interrupt 0000:00:0a.0[A] -> GSI 25 (level, low) -> IRQ 19
> eth1: 0000:00:0a.0, 00:02:55:C7:CA:D9, IRQ 19.
> Board assembly 754338-001, Physical connectors present: RJ45
> Primary interface chip i82555 PHY #1.
> General self-test: passed.
> Serial sub-system self-test: passed.
> Internal registers self-test: passed.
> ROM checksum self-test: passed (0x04f4518b).
> Intel(R) PRO/1000 Network Driver - version 6.3.9-k4
> Copyright (c) 1999-2005 Intel Corporation.
> Done.
> Begin: Running /scripts/init-premount ...
> input: PS/2 Logitech Mouse as /class/input/input1
> Done.
> Begin: Mounting root file system... ...
> Begin: Running /scripts/local-top ...
> device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@redhat.com
> /scripts/local-top/lvm: 36: vgchange: not found
> Done.
> SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
> sda: assuming Write Enabled
> sda: assuming drive cache: write through
> SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
> sda: assuming Write Enabled
> sda: assuming drive cache: write through
> sda: sda1 sda2 < sda5 >
> sd 1:0:0:0: Attached scsi disk sda
> SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
> sdb: assuming Write Enabled
> sdb: assuming drive cache: write through
> SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
> sdb: assuming Write Enabled
> sdb: assuming drive cache: write through
> sdb: unknown partition table
> sd 1:0:1:0: Attached scsi disk sdb
> Begin: Running /scripts/local-premount ...
> Done.
> EXT3-fs: INFO: recovery required on readonly filesystem.
> EXT3-fs: write access will be enabled during recovery.
> kjournald starting. Commit interval 5 seconds
> EXT3-fs: recovery complete.
> EXT3-fs: mounted filesystem with ordered data mode.
> Begin: Running /scripts/log-bottom ...
> Done.
> Done.
> Begin: Running /scripts/init-bottom ...
> Done.
> INIT: version 2.86 booting
> Starting the hotplug events dispatcher: udevd.
> Synthesizing the initial hotplug events...done.
> Waiting for /dev to be fully populated...done.
> Loading /etc/console/boottime.kmap.gz
> Activating swap.
> Adding 1019880k swap on /dev/sda5. Priority:-1 extents:1 across:1019880k
> Checking root file system...
> fsck 1.37 (21-Mar-2005)
> /dev/sda1: clean, 58437/512000 files, 445341/1023996 blocks
> EXT3 FS on sda1, internal journal
> System time was Sat Aug 12 00:22:00 UTC 2006.
> Setting the System Clock using the Hardware Clock as reference...
> System Clock set. System local time is now Sat Aug 12 00:22:01 UTC 2006.
> Calculating module dependencies...done.
> Loading modules...
> auto
> FATAL: Module auto not found.
> 3c59x
> softdog
> Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0)
> tulip
> Linux Tulip driver version 1.1.13 (May 11, 2002)
> e100
> e100: Intel(R) PRO/100 Network Driver, 3.5.10-k2-NAPI
> e100: Copyright(c) 1999-2005 Intel Corporation
> All modules loaded.
> Loading device-mapper support.
> Starting Enterprise Volume Management System: device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> device-mapper: dm-linear: Device lookup failed
> device-mapper: error adding target to table
> evms.
> Checking all file systems...
> fsck 1.37 (21-Mar-2005)
> Setting kernel variables ...
> ... done.
> Loading the saved-state of the serial devices...
> Cannot get serial info: Invalid argument
> Cannot get serial info: Invalid argument
> /dev/ttyS1: No such file or directory
> /dev/ttyS1: No such file or directory
> Mounting local filesystems...
> Cleaning /tmp /var/run /var/lock.
> Cleaning: /etc/network/ifstate.
> Setting up IP spoofing protection: rp_filter.
> Configuring network interfaces: done.
> Starting portmap daemon: portmap.
>
> Setting the System Clock using the Hardware Clock as reference...
> System Clock set. Local time: Fri Aug 11 17:22:07 PDT 2006
>
> Recovering jove files ... Done.
> Running ntpdate to synchronize clock.
> Initializing random number generator...done.
> Recovering nvi editor sessions... done.
> Setting up X server socket directory /tmp/.X11-unix...done.
> Setting up ICE socket directory /tmp/.ICE-unix...done.
> INIT: Entering runlevel: 2
> Starting system log daemon: syslogd.
> Starting kernel log daemon: klogd.
> Starting virtual private network daemon:.
> Starting portmap daemon: portmap.
> Starting NFS common utilities: statd lockd.
> Starting internet superserver: inetd.
> * The mptctl module is missing. Please have a look at the README.Debian.gz.
> .
> Exporting directories for NFS kernel daemon...Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
> 10.27.3.21:/export/xen/fs/stevegt/crashme1/root: No such file or directory
> 10.27.3.18:/export/xen/fs/stevegt/xentest2/root: No such file or directory
> 10.27.3.17:/export/xen/fs/stevegt/xentest1/root: No such file or directory
> 10.27.3.8:/export/xen/fs/cclarke/umltest1/root: No such file or directory
> hotaru.chaosring.Org:/export/xen/fs/stevegt/crashme1/root: No such file or directory
> rn-2.refactored-networks.com:/export/xen/fs/stevegt/xentest2/root: No such file or directory
> xentest1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/xentest1/root: No such file or directory
> umltest1.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/umltest1/root: No such file or directory
> 10.27.3.26:/export/xen/fs/stevegt/gforge/root: No such file or directory
> 10.27.3.23:/export/xen/fs/baseline/sarge/root: No such file or directory
> 10.27.3.22:/export/xen/fs/stevegt/build1/root: No such file or directory
> 10.27.3.16:/export/xen/fs/baseline/woody/root: No such file or directory
> bugs.t7a.Org:/export/xen/fs/stevegt/gforge/root: No such file or directory
> sarge.baseline.TerraLuna.Org:/export/xen/fs/baseline/sarge/root: No such file or directory
> build1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/build1/root: No such file or directory
> woody.baseline.TerraLuna.Org:/export/xen/fs/baseline/woody/root: No such file or directory
> 10.27.3.19:/export/xen/fs/cclarke/ebond/root: No such file or directory
> ebond.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/ebond/root: No such file or directory
> 10.27.3.29:/export/xen/fs/dmasten/mss0/root: No such file or directory
> 10.27.3.27:/export/xen/fs/cclarke/ccms/root: No such file or directory
> 10.27.3.25:/export/xen/fs/baseline/sid/root: No such file or directory
> 10.27.3.24:/export/xen/fs/stevegt/cvs1/root: No such file or directory
> sid.stevegt.TerraLuna.Org:/export/xen/fs/baseline/sid/root: No such file or directory
> cvs1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/cvs1/root: No such file or directory
> 10.27.3.28:/export/xen/fs/stevegt/t7a/root: No such file or directory
> 10.27.3.20:/export/xen/fs/stevegt/tcx/root: No such file or directory
> tcx.TerraLuna.Org:/export/xen/fs/stevegt/tcx/root: No such file or directory
> n2h54.prd.TerraLuna.Org:/export/xen: No such file or directory
> n2h53.prd.TerraLuna.Org:/export/xen: No such file or directory
> n2h51.prd.TerraLuna.Org:/export/xen: No such file or directory
> n2h41.prd.TerraLuna.Org:/export/xen: No such file or directory
> done.
> Starting NFS kernel daemon: nfsdNFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
> NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
> NFSD: starting 90-second grace period
> mountd.
> Starting OpenBSD Secure Shell server: sshdNET: Registered protocol family 10
> lo: Disabled Privacy Extensions
> IPv6 over IPv4 tunneling driver
> .
> Starting NTP server: ntpd.
> AFS module /lib/modules/2.6.16.13-xen/fs/openafs.mp.o does not exist. Not starting AFS.
> Please consider building kernel modules using instructions in
> /usr/share/doc/openafs-client/README.modules
> Starting DRBD resources: drbd: initialised. Version: 0.7.20 (api:79/proto:74)
> drbd: SVN Revision: 2260 build by root@n4h8, 2006-08-04 11:00:44
> drbd: registered as block device major 147
> [ d0 drbd7: resync bitmap: bits=1048576 words=32768
> drbd7: size = 4095 MB (4194303 KB)
> drbd7: 0 KB marked out-of-sync by on disk bit-map.
> drbd7: No usable activity log found.
> drbd7: drbdsetup [3825]: cstate Unconfigured --> StandAlone
> d1 drbd8: resync bitmap: bits=1048576 words=32768
> drbd8: size = 4095 MB (4194303 KB)
> drbd8: 0 KB marked out-of-sync by on disk bit-map.
> drbd8: Found 4 transactions (46 active extents) in activity log.
> drbd8: Marked additional 156 MB as out-of-sync based on AL.
> drbd8: drbdsetup [3829]: cstate Unconfigured --> StandAlone
> d2 drbd9: resync bitmap: bits=2621440 words=81920
> drbd9: size = 9 GB (10485759 KB)
> drbd9: 0 KB marked out-of-sync by on disk bit-map.
> drbd9: Found 4 transactions (192 active extents) in activity log.
> drbd9: drbdsetup [3833]: cstate Unconfigured --> StandAlone
> d3 drbd10: resync bitmap: bits=1835008 words=57344
> drbd10: size = 7167 MB (7340031 KB)
> drbd10: 0 KB marked out-of-sync by on disk bit-map.
> drbd10: Found 4 transactions (192 active extents) in activity log.
> drbd10: Marked additional 508 MB as out-of-sync based on AL.
> drbd10: drbdsetup [3837]: cstate Unconfigured --> StandAlone
> d4 drbd11: resync bitmap: bits=786432 words=24576
> drbd11: size = 3071 MB (3145727 KB)
> drbd11: 0 KB marked out-of-sync by on disk bit-map.
> drbd11: Found 4 transactions (192 active extents) in activity log.
> drbd11: drbdsetup [3841]: cstate Unconfigured --> StandAlone
> d5 drbd12: resync bitmap: bits=786432 words=24576
> drbd12: size = 3071 MB (3145727 KB)
> drbd12: 0 KB marked out-of-sync by on disk bit-map.
> drbd12: Found 4 transactions (136 active extents) in activity log.
> drbd12: Marked additional 508 MB as out-of-sync based on AL.
> drbd12: drbdsetup [3845]: cstate Unconfigured --> StandAlone
> s0 s1 s2 s3 s4 s5 n0 drbd7: drbdsetup [3883]: cstate StandAlone --> Unconnected
> drbd7: drbd7_receiver [3884]: cstate Unconnected --> WFConnection
> n1 drbd8: drbdsetup [3891]: cstate StandAlone --> Unconnected
> drbd8: drbd8_receiver [3892]: cstate Unconnected --> WFConnection
> n2 drbd9: drbdsetup [3899]: cstate StandAlone --> Unconnected
> drbd9: drbd9_receiver [3900]: cstate Unconnected --> WFConnection
> n3 drbd10: drbdsetup [3907]: cstate StandAlone --> Unconnected
> drbd10: drbd10_receiver [3908]: cstate Unconnected --> WFConnection
> n4 drbd11: drbdsetup [3915]: cstate StandAlone --> Unconnected
> drbd11: drbd11_receiver [3916]: cstate Unconnected --> WFConnection
> n5 drbd12: drbdsetup [3923]: cstate StandAlone --> Unconnected
> drbd12: drbd12_receiver [3924]: cstate Unconnected --> WFConnection
> ].
> ..........
> ***************************************************************
> DRBD's startup script waits for the peer node(s) to appear.
> - In case this node was already a degraded cluster before the
> reboot the timeout is 120 seconds. [degr-wfc-timeout]
> - If the peer was available before the reboot the timeout will
> expire after 0 seconds. [wfc-timeout]
> (These values are for resource 'ds1.t7a.org'; 0 sec -> wait forever)
> To abort waiting enter 'yes' [ 18]:drbd7: drbd7_receiver [3884]: cstate WFConnection --> WFReportParams
> drbd7: Handshake successful: DRBD Network Protocol version 74
> drbd7: Connection established.
> drbd7: I am(S): 1:00000002:00000001:00000018:00000002:01
> drbd7: Peer(S): 1:00000002:00000001:00000019:00000002:10
> drbd7: drbd7_receiver [3884]: cstate WFReportParams --> WFBitMapT
> drbd7: Secondary/Unknown --> Secondary/Secondary
> drbd7: drbd7_receiver [3884]: cstate WFBitMapT --> SyncTarget
> drbd7: Resync started as SyncTarget (need to sync 520192 KB [130048 bits set]).
> drbd9: drbd9_receiver [3900]: cstate WFConnection --> WFReportParams
> drbd9: Handshake successful: DRBD Network Protocol version 74
> drbd9: Connection established.
> drbd9: I am(S): 1:00000002:00000001:0000000d:00000001:01
> drbd9: Peer(S): 1:00000002:00000001:0000000e:00000001:10
> drbd9: drbd9_receiver [3900]: cstate WFReportParams --> WFBitMapT
> drbd9: Secondary/Unknown --> Secondary/Secondary
> drbd9: drbd9_receiver [3900]: cstate WFBitMapT --> SyncTarget
> drbd9: Resync started as SyncTarget (need to sync 290816 KB [72704 bits set]).
> drbd9: drbd9_receiver [3900]: cstate SyncTarget --> PausedSyncT
> drbd9: Syncer waits for sync group.
> drbd11: drbd11_receiver [3916]: cstate WFConnection --> WFReportParams
> drbd11: Handshake successful: DRBD Network Protocol version 74
> drbd11: Connection established.
> drbd11: I am(S): 1:00000002:00000001:00000008:00000001:01
> drbd11: Peer(S): 1:00000002:00000001:00000009:00000001:10
> drbd11: drbd11_receiver [3916]: cstate WFReportParams --> WFBitMapT
> drbd11: Secondary/Unknown --> Secondary/Secondary
> drbd11: drbd11_receiver [3916]: cstate WFBitMapT --> SyncTarget
> drbd11: Resync started as SyncTarget (need to sync 274432 KB [68608 bits set]).
> drbd11: drbd11_receiver [3916]: cstate SyncTarget --> PausedSyncT
> drbd11: Syncer waits for sync group.
> drbd8: drbd8_receiver [3892]: cstate WFConnection --> WFReportParams
> drbd8: Handshake successful: DRBD Network Protocol version 74
> drbd8: Connection established.
> drbd8: I am(S): 1:00000002:00000001:00000017:00000001:11
> drbd8: Peer(S): 1:00000002:00000001:00000017:00000001:01
> drbd8: drbd8_receiver [3892]: cstate WFReportParams --> WFBitMapS
> drbd8: Secondary/Unknown --> Secondary/Secondary
> drbd8: drbd8_receiver [3892]: cstate WFBitMapS --> SyncSource
> drbd8: Resync started as SyncSource (need to sync 159744 KB [39936 bits set]).
> drbd8: drbd8_receiver [3892]: cstate SyncSource --> PausedSyncS
> drbd8: Syncer waits for sync group.
> drbd12: drbd12_receiver [3924]: cstate WFConnection --> WFReportParams
> drbd12: Handshake successful: DRBD Network Protocol version 74
> drbd12: Connection established.
> drbd12: I am(S): 1:00000002:00000001:00000001:00000001:11
> drbd12: Peer(S): 1:00000002:00000001:00000001:00000001:01
> drbd12: drbd12_receiver [3924]: cstate WFReportParams --> WFBitMapS
> drbd12: Secondary/Unknown --> Secondary/Secondary
> drbd12: drbd12_receiver [3924]: cstate WFBitMapS --> SyncSource
> drbd12: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]).
> drbd12: drbd12_receiver [3924]: cstate SyncSource --> PausedSyncS
> drbd12: Syncer waits for sync group.
> drbd10: drbd10_receiver [3908]: cstate WFConnection --> WFReportParams
> drbd10: Handshake successful: DRBD Network Protocol version 74
> drbd10: Connection established.
> drbd10: I am(S): 1:00000002:00000001:0000000b:00000001:11
> drbd10: Peer(S): 1:00000002:00000001:0000000b:00000001:01
> drbd10: drbd10_receiver [3908]: cstate WFReportParams --> WFBitMapS
>
> drbd10: Secondary/Unknown --> Secondary/Secondary
> drbd10: drbd10_receiver [3908]: cstate WFBitMapS --> SyncSource
> drbd10: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]).
> drbd10: drbd10_receiver [3908]: cstate SyncSource --> PausedSyncS
> drbd10: Syncer waits for sync group.
> Heartbeat not configured: /etc/ha.d/ha.cf not found.
> Heartbeat failure [rc=1]. Failed.
> Starting deferred execution scheduler: atd.
> Starting periodic command scheduler: cron.
> Starting watchdog daemon: watchdog.
> openMosix: ERROR: Cannot find the /proc/hpc directory.
> openMosix: ERROR: Looks like this is not an openMosix enabled kernel.
> openMosix: ERROR: Configuration ABORTED.
> openMosix: HINT: need to recompile your kernel with openMosix support enabled?
> openMosix: HINT: need to update your boot manager (lilo, grub, etc)?
> openMosix: HINT: have you read the docs at http://openMosix.sf.net/ ?
> Bridge firewalling registered
> device vif0.0 entered promiscuous mode
> xenbr0: port 1(vif0.0) entering learning state
> xenbr0: topology change detected, propagating
> xenbr0: port 1(vif0.0) entering forwarding state
> device peth0 entered promiscuous mode
> xenbr0: port 2(peth0) entering learning state
> xenbr0: topology change detected, propagating
> xenbr0: port 2(peth0) entering forwarding state
> Starting auto Xen domains: afs1.t7a.orgdrbd9: Secondary/Secondary --> Secondary/Primary
> device vif1.0 entered promiscuous mode
> ADDRCONF(NETDEV_UP): vif1.0: link is not ready
> ip_tables: (C) 2000-2006 Netfilter Core Team
> Error: Device 2049 (vbd) could not be connected. Backend device not found.
> xenbr0: port 3(vif1.0) entering disabled state
> ! build2.t7a.orgdevice vif1.0 left promiscuous mode
> xenbr0: port 3(vif1.0) entering disabled state
> device vif2.0 entered promiscuous mode
> ADDRCONF(NETDEV_UP): vif2.0: link is not ready
> drbd10: Secondary/Secondary --> Primary/Secondary
> ADDRCONF(NETDEV_CHANGE): vif2.0: link becomes ready
> xenbr0: port 3(vif2.0) entering learning state
> xenbr0: topology change detected, propagating
> xenbr0: port 3(vif2.0) entering forwarding state
> ds1.t7a.orgdrbd7: Secondary/Secondary --> Secondary/Primary
> device vif3.0 entered promiscuous mode
> ADDRCONF(NETDEV_UP): vif3.0: link is not ready
> Error: Device 2049 (vbd) could not be connected. Backend device not found.
> ! kdc1.t7a.orgxenbr0: port 4(vif3.0) entering disabled state
>
> device vif3.0 left promiscuous mode
> xenbr0: port 4(vif3.0) entering disabled state
>
>
>
>
>
>
> drbd8: Secondary/Secondary --> Primary/Secondary
> device vif4.0 entered promiscuous mode
> ADDRCONF(NETDEV_UP): vif4.0: link is not ready
> ADDRCONF(NETDEV_CHANGE): vif4.0: link becomes ready
> xenbr0: port 4(vif4.0) entering learning state
> xenbr0: topology change detected, propagating
> xenbr0: port 4(vif4.0) entering forwarding state
> w1.cdint.comdrbd11: Secondary/Secondary --> Secondary/Primary
> device vif5.0 entered promiscuous mode
> ADDRCONF(NETDEV_UP): vif5.0: link is not ready
> Error: Device 2049 (vbd) could not be connected. Backend device not found.
> xenbr0: port 5(vif5.0) entering disabled state
> ! webkdc.t7a.orgdevice vif5.0 left promiscuous mode
> xenbr0: port 5(vif5.0) entering disabled state
> device vif6.0 entered promiscuous mode
> ADDRCONF(NETDEV_UP): vif6.0: link is not ready
> drbd12: Secondary/Secondary --> Primary/Secondary
> ADDRCONF(NETDEV_CHANGE): vif6.0: link becomes ready
> xenbr0: port 5(vif6.0) entering learning state
> xenbr0: topology change detected, propagating
> xenbr0: port 5(vif6.0) entering forwarding state
> [done]
>
>
> _sudZUZ#Z#XZo=_ DDDD EEEEEE BBBB IIIIII AAAA NN NN
> _jmZZ2!!~---~!!X##wa DD DD EE BB BB II AA AA NNN NN
> .<wdP~~ -!YZL, DD DD EEEEE BBBBB II AAAAAA NNNN NN
> .mX2' _%aaa__ XZ[. DD DD EE BB BB II AA AA NN NNNN
> oZ[ _jdXY!~?S#wa ]Xb; DDDD EEEEEE BBBBB IIIIII AA AA NN NN
> _#e' .]X2( ~Xw| )XXc
> .2Z` ]X[. xY| ]oZ( Linux Version 2.6.16.13-xen
> .2#; )3k; _s!~ jXf` Compiled #3 SMP Fri Aug 4 10:57:44 PDT 2006
> 1Z> -]Xb/ ~ __#2( Two 1.13GHz Intel Intel(R) Pentium(R) III CPU family 1133MHz Processors
> -Zo; +!4ZwaaaauZZXY' 72M RAM
> *#[, ~-?!!!!!!-~ 4523.20 Bogomips Total
> XUb;. n4h34
> )YXL,,
> +3#bc,
> -)SSL,,
> ~~~~~
> Updating the Linuxlogo... done.
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login: drbd7: Resync done (total 137 sec; paused 0 sec; 3796 K/sec)
> drbd7: drbd7_worker [3826]: cstate SyncTarget --> Connected
> drbd8: Syncer continues.
> drbd8: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> drbd9: Syncer continues.
> drbd9: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
> drbd10: Syncer continues.
> drbd10: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> drbd11: Syncer continues.
> drbd11: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
> drbd12: Syncer continues.
> drbd12: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> drbd8: Resync done (total 211 sec; paused 137 sec; 2156 K/sec)
> drbd8: drbd8_worker [3830]: cstate SyncSource --> Connected
> drbd11: Resync done (total 239 sec; paused 137 sec; 2688 K/sec)
> drbd11: drbd11_worker [3842]: cstate SyncTarget --> Connected
> drbd9: Resync done (total 243 sec; paused 137 sec; 2740 K/sec)
> drbd9: drbd9_worker [3834]: cstate SyncTarget --> Connected
>
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login:
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login:
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login:
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login:
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login:
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login: drbd12: Resync done (total 306 sec; paused 136 sec; 3056 K/sec)
> drbd12: drbd12_worker [3846]: cstate SyncSource --> Connected
>
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login:
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login: drbd10: Resync done (total 310 sec; paused 136 sec; 2988 K/sec)
> drbd10: drbd10_worker [3838]: cstate SyncSource --> Connected
>
> Debian GNU/Linux 3.0 n4h34 ttyS0
>
> n4h34 login: Unable to handle kernel paging request at virtual address c0976590
> printing eip:
> c03d6ed7
> *pde = ma 3cf91067 pa 00f91067
> *pte = ma 00000000 pa fffff000
> Oops: 0000 [#1]
> SMP
> Modules linked in: iptable_filter ip_tables x_tables bridge drbd ipv6 nfsd lockd sunrpc e100 tulip softdog 3c59x evdev sd_mod dm_mod thermal processor fan e1000 eepro100 mii tg3
> CPU: 0
> EIP: 0061:[<c03d6ed7>] Not tainted VLI
> EFLAGS: 00010206 (2.6.16.13-xen #3)
> EIP is at skb_copy_bits+0x127/0x280
> eax: c0976000 ebx: 000005a8 ecx: 0000016a edx: c3711720
> esi: c0976590 edi: c37110e0 ebp: 000005a8 esp: c00818c8
> ds: 007b es: 007b ss: 0069
> Process drbd12_receiver (pid: 3924, threadinfo=c0080000 task=c3efda90)
> Stack: <0>c1012ec0 00000002 c03d65ef c1d51a00 000005ea 00000042 00000000 00000000
> 00000020 c3edc800 c376fd64 c03d6b0f c376fd64 00000042 c37110e0 000005a8
> c1a70000 c54690c0 00000000 c1d51ac0 c3edc800 c376fd64 c03dc478 c376fd64
> Call Trace:
> [<c03d65ef>] pskb_expand_head+0xdf/0x140
> [<c03d6b0f>] __pskb_pull_tail+0x7f/0x320
> [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
> [<c03dc478>] dev_queue_xmit+0x328/0x370
> [<c5462f7e>] br_dev_queue_push_xmit+0xbe/0x140 [bridge]
> [<c5469212>] br_nf_post_routing+0x102/0x1c0 [bridge]
> [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
> [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> [<c03f43f8>] nf_iterate+0x78/0x90
> [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> [<c03f447e>] nf_hook_slow+0x6e/0x110
> [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> [<c5463061>] br_forward_finish+0x61/0x70 [bridge]
> [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> [<c5468995>] br_nf_forward_finish+0x75/0x130 [bridge]
> [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> [<c5468b38>] br_nf_forward_ip+0xe8/0x190 [bridge]
> [<c5468920>] br_nf_forward_finish+0x0/0x130 [bridge]
> [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> [<c03f43f8>] nf_iterate+0x78/0x90
> [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> [<c03f447e>] nf_hook_slow+0x6e/0x110
> [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> [<c5463167>] __br_forward+0x77/0x80 [bridge]
> [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> [<c5463fbf>] br_handle_frame_finish+0xdf/0x160 [bridge]
> [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> [<c5467d89>] br_nf_pre_routing_finish+0xf9/0x370 [bridge]
> [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> [<c0322e3a>] loopback_start_xmit+0xba/0x110
> [<c0400d70>] ip_finish_output+0x0/0x220
> [<c03dc07e>] dev_hard_start_xmit+0x5e/0x130
> [<c03dc3b5>] dev_queue_xmit+0x265/0x370
> [<c03f43f8>] nf_iterate+0x78/0x90
> [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> [<c03f447e>] nf_hook_slow+0x6e/0x110
> [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> [<c54685fc>] br_nf_pre_routing+0x26c/0x520 [bridge]
> [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> [<c03f43f8>] nf_iterate+0x78/0x90
> [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> [<c03f447e>] nf_hook_slow+0x6e/0x110
> [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> [<c546422d>] br_handle_frame+0x1ed/0x230 [bridge]
> [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> [<c03dcb21>] netif_receive_skb+0x1a1/0x330
> [<c03dcd87>] process_backlog+0xd7/0x190
> [<c03dcf2a>] net_rx_action+0xea/0x230
> [<c0125915>] __do_softirq+0xf5/0x120
> [<c01259d5>] do_softirq+0x95/0xa0
> [<c0125a42>] local_bh_enable+0x62/0xa0
> [<c0406bd1>] tcp_prequeue_process+0x71/0x80
> [<c04070e9>] tcp_recvmsg+0x349/0x750
> [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
> [<c03d5085>] sock_common_recvmsg+0x55/0x70
> [<c03d11cf>] sock_recvmsg+0xef/0x110
> [<c03143fa>] force_evtchn_callback+0xa/0x10
> [<c0147163>] mempool_alloc+0x33/0xe0
> [<c01367d0>] autoremove_wake_function+0x0/0x60
> [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
> [<c02b2ee0>] generic_make_request+0xf0/0x160
> [<c529c680>] drbd_recv+0x90/0x190 [drbd]
> [<c529cdec>] drbd_recv_header+0x2c/0xf0 [drbd]
> [<c529e580>] receive_DataRequest+0x0/0x7d0 [drbd]
> [<c52a045c>] drbdd+0x1c/0x150 [drbd]
> [<c52a105a>] drbdd_init+0x7a/0x1a0 [drbd]
> [<c52a7136>] drbd_thread_setup+0x86/0xf0 [drbd]
> [<c52a70b0>] drbd_thread_setup+0x0/0xf0 [drbd]
> [<c0102f75>] kernel_thread_helper+0x5/0x10
> Code: 8b 4c 24 30 8b 7c 24 34 8b 91 a0 00 00 00 8b 4c 24 18 0f b7 74 ca 18 8b 4c 24 14 8d 34 06 01 fe 29 ce 8b 7c 24 38 89 d9 c1 e9 02 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 89 04 24 ba 02 00 00 00 89 54
> <0>Kernel panic - not syncing: Fatal exception in interrupt
> (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
>
> --
> Stephen G. Traugott (KG6HDQ)
> Managing Partner, TerraLuna LLC
> stevegt@TerraLuna.Org -- http://www.t7a.org
--
Stephen G. Traugott (KG6HDQ)
Managing Partner, TerraLuna LLC
stevegt@TerraLuna.Org -- http://www.t7a.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
2006-08-12 4:34 ` Steve Traugott
@ 2006-08-12 5:50 ` Steve Traugott
2006-08-12 9:50 ` Keir Fraser
0 siblings, 1 reply; 31+ messages in thread
From: Steve Traugott @ 2006-08-12 5:50 UTC (permalink / raw)
To: Ian Pratt; +Cc: xen-devel
Another data point... Dom0 seems to only want to crash when more than
3 or 4 domU's are running (each with their own DRBD root, with DRBD
running in dom0), and the below 'nc' command is run in the last
domU...
Still looking.
Steve
On Fri, Aug 11, 2006 at 09:34:12PM -0700, Steve Traugott wrote:
> Okay, for some reason I'm now able to crash dom0 pretty reliably in
> this configuration (domU root filesystem in DRBD) simply by running
> this in domU:
>
> nc -u -b 255.255.255.255 54321 < /dev/zero
>
> Anyone else want to give that a try on 10868 or later, and see if they
> can duplicate it, with or without DRBD?
>
> Steve
>
> On Fri, Aug 11, 2006 at 06:50:50PM -0700, Steve Traugott wrote:
> > Uh oh....
> >
> > On Sat, Aug 05, 2006 at 08:38:03AM +0100, Ian Pratt wrote:
> > > > So I built -unstable changeset 10868, and ran an even heavier workload
> > > > (the above, plus 'bonnie' in the guests) on dom0 and two guests
> > > > overnight, and they experienced no soft lockups; running -unstable,
> > > > changeset 10868, credit scheduler. This same workload would have
> > > > caused soft lockups within seconds in -testing changeset 9732 using
> > > > the sedf scheduler; I may not have been able to get it started at all.
> > > > Response time remained subsecond under -unstable; -testing would have
> > > > been on its knees.
> > >
> > > That's good to hear. 3.0.3 is going to be a big leap forward in many
> > > ways.
> >
> > Same environment as we've discussed earlier in this thread, now on
> > unstable changeset 10868, Xen on top of DRBD, this one seems to be a
> > reliable crash when under combined disk and network load, still
> > isolating but would welcome suggestions...
> >
> > Steve
> >
> >
> > root (hd0,0)
> > Filesystem type is ext2fs, partition type 0x83
> > kernel /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
> > [Multiboot-elf, <0x100000:0x7a008:0x55ff8>, shtab=0x1d0078, entry=0x100000]
> > module /boot/vmlinuz-2.6.16.13-xen root=801 ro console=ttyS0 ramdisk_size=32768
> > [Multiboot-module @ 0x1d1000, 0x494ed0 bytes]
> > module /boot/initrd.img-2.6.16.13-xen
> > [Multiboot-module @ 0x666000, 0x9a6400 bytes]
> > __ __ _____ ___ _ _ _
> > \ \/ /___ _ __ |___ / / _ \ _ _ _ __ ___| |_ __ _| |__ | | ___
> > \ // _ \ '_ \ |_ \| | | |__| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> > / \ __/ | | | ___) | |_| |__| |_| | | | \__ \ || (_| | |_) | | __/
> > /_/\_\___|_| |_| |____(_)___/ \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> >
> > http://www.cl.cam.ac.uk/netos/xen
> > University of Cambridge Computer Laboratory
> >
> > Xen version 3.0-unstable (root@prd.terraluna.org) (gcc version 3.3.5 (Debian 1:3.3.5-13)) Fri Aug 4 10:54:50 PDT 2006
> > Latest ChangeSet: Sat Jul 29 06:05:59 2006 +0100 10868:d2bf1a7cc131
> >
> > (XEN) Command line: /boot/xen-3.0.gz dom0_mem=65536 com1=9600,8n1
> > (XEN) Physical RAM map:
> > (XEN) 0000000000000000 - 000000000009c000 (usable)
> > (XEN) 000000000009c000 - 00000000000a0000 (reserved)
> > (XEN) 00000000000e0000 - 0000000000100000 (reserved)
> > (XEN) 0000000000100000 - 000000003ffec340 (usable)
> > (XEN) 000000003ffec340 - 000000003fff0000 (ACPI data)
> > (XEN) 000000003fff0000 - 0000000040000000 (reserved)
> > (XEN) 00000000fec00000 - 0000000100000000 (reserved)
> > (XEN) System RAM: 1023MB (1048096kB)
> > (XEN) Xen heap: 10MB (10396kB)
> > (XEN) Using scheduler: SMP Credit Scheduler (credit)
> > (XEN) PAE disabled.
> > (XEN) found SMP MP-table at 0009c1d0
> > (XEN) DMI 2.3 present.
> > (XEN) Using APIC driver default
> > (XEN) ACPI: RSDP (v000 IBM ) @ 0x000fdfd0
> > (XEN) ACPI: RSDT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff80
> > (XEN) ACPI: FADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffeff00
> > (XEN) ACPI: MADT (v001 IBM SEREMRLD 0x00001000 IBM 0x45444f43) @ 0x3ffefe80
> > (XEN) ACPI: DSDT (v001 IBM SEREMRLD 0x00001000 MSFT 0x0100000b) @ 0x00000000
> > (XEN) ACPI: Local APIC address 0xfee00000
> > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> > (XEN) Processor #3 6:11 APIC version 17
> > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> > (XEN) Processor #0 6:11 APIC version 17
> > (XEN) ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
> > (XEN) IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
> > (XEN) ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
> > (XEN) IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
> > (XEN) ACPI: IRQ3 used by override.
> > (XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
> > (XEN) Using ACPI (MADT) for SMP configuration information
> > (XEN) Initializing CPU#0
> > (XEN) Detected 1130.180 MHz processor.
> > (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
> > (XEN) CPU: L2 cache: 512K
> > (XEN) Intel machine check architecture supported.
> > (XEN) Intel machine check reporting enabled on CPU#0.
> > (XEN) CPU0: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
> > (XEN) Booting processor 1/0 eip 90000
> > (XEN) Initializing CPU#1
> > (XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
> > (XEN) CPU: L2 cache: 512K
> > (XEN) Intel machine check architecture supported.
> > (XEN) Intel machine check reporting enabled on CPU#1.
> > (XEN) CPU1: Intel(R) Pentium(R) III CPU family 1133MHz stepping 01
> > (XEN) Total of 2 processors activated.
> > (XEN) ENABLING IO-APIC IRQs
> > (XEN) -> Using new ACK method
> > (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=0 apic2=-1 pin2=-1
> > (XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
> > (XEN) ...trying to set up timer (IRQ0) through the 8259A ... failed.
> > (XEN) ...trying to set up timer as Virtual Wire IRQ... works.
> > (XEN) checking TSC synchronization across 2 CPUs: passed.
> > (XEN) Platform timer is 1.193MHz PIT
> > (XEN) Brought up 2 CPUs
> > (XEN) Machine check exception polling timer started.
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN) Domain 0 kernel supports features = { 0000001f }.
> > (XEN) Domain 0 kernel requires features = { 00000000 }.
> > (XEN) PHYSICAL MEMORY ARRANGEMENT:
> > (XEN) Dom0 alloc.: 3c000000->3e000000 (8192 pages to be allocated)
> > (XEN) VIRTUAL MEMORY ARRANGEMENT:
> > (XEN) Loaded kernel: c0100000->c05d5d74
> > (XEN) Init. ramdisk: c05d6000->c0f7c400
> > (XEN) Phys-Mach map: c0f7d000->c0f8d000
> > (XEN) Start info: c0f8d000->c0f8e000
> > (XEN) Page tables: c0f8e000->c0f94000
> > (XEN) Boot stack: c0f94000->c0f95000
> > (XEN) TOTAL: c0000000->c1400000
> > (XEN) ENTRY ADDRESS: c0100000
> > (XEN) Dom0 has maximum 2 VCPUs
> > (XEN) Initrd len 0x9a6400, start at 0xc05d6000
> > (XEN) Scrubbing Free RAM: ...........done.
> > (XEN) Xen trace buffers: disabled
> > (XEN) Xen is relinquishing VGA console.
> > (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen).
> > Linux version 2.6.16.13-xen (root@n4h8) (gcc version 3.3.5 (Debian 1:3.3.5-13)) #3 SMP Fri Aug 4 10:57:44 PDT 2006
> > BIOS-provided physical RAM map:
> > Xen: 0000000000000000 - 0000000004800000 (usable)
> > 0MB HIGHMEM available.
> > 72MB LOWMEM available.
> > DMI 2.3 present.
> > ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> > ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> > ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
> > IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
> > ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
> > IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
> > ACPI: INT_SRC_OVR (bus 0 bus_irq 3 global_irq 30 dfl dfl)
> > Enabling APIC mode: Flat. Using 2 I/O APICs
> > Using ACPI (MADT) for SMP configuration information
> > Allocating PCI resources starting at 50000000 (gap: 40000000:bec00000)
> > Built 1 zonelists
> > Kernel command line: root=801 ro console=ttyS0 ramdisk_size=32768
> > Enabling fast FPU save and restore... done.
> > Enabling unmasked SIMD FPU exception support... done.
> > Initializing CPU#0
> > PID hash table entries: 512 (order: 9, 8192 bytes)
> > Xen reported: 1130.113 MHz processor.
> > Console: colour VGA+ 80x25
> > Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
> > Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
> > Software IO TLB enabled:
> > Aperture: 2 megabytes
> > Kernel range: 0x00000000c10f0000 - 0x00000000c12f0000
> > vmalloc area: c5000000-fb7fe000, maxmem 33ffe000
> > Memory: 47400k/73728k available (3389k kernel code, 18068k reserved, 1044k data, 208k init, 0k highmem)
> > Checking if this processor honours the WP bit even in supervisor mode... Ok.
> > Calibrating delay using timer specific routine.. 2261.60 BogoMIPS (lpj=11308023)
> > Security Framework v1.0.0 initialized
> > Capability LSM initialized
> > Mount-cache hash table entries: 512
> > CPU: L1 I cache: 16K, L1 D cache: 16K
> > CPU: L2 cache: 512K
> > Checking 'hlt' instruction... OK.
> > ENABLING IO-APIC IRQs
> > Enabling SMP...
> > Brought up 2 CPUs
> > Initializing CPU#1
> > migration_cost=1895
> > checking if image is initramfs... it is
> > Freeing initrd memory: 9881k freed
> > Grant table initialized
> > NET: Registered protocol family 16
> > ACPI: bus type pci registered
> > PCI: Using configuration type 1
> > ACPI: Subsystem revision 20060127
> > ACPI: Interpreter enabled
> > ACPI: Using IOAPIC for interrupt routing
> > ACPI: PCI Root Bridge [PCI0] (0000:00)
> > ACPI: PCI Interrupt Link [LPE1] (IRQs *10)
> > ACPI: PCI Interrupt Link [LPE2] (IRQs *10)
> > ACPI: PCI Interrupt Link [LPVI] (IRQs) *0, disabled.
> > ACPI: PCI Interrupt Link [LPUS] (IRQs *7)
> > ACPI: PCI Root Bridge [PCI1] (0000:01)
> > ACPI: PCI Interrupt Link [LPSA] (IRQs *9)
> > ACPI: PCI Interrupt Link [LP1A] (IRQs) *0, disabled.
> > ACPI: PCI Interrupt Link [LP1B] (IRQs) *0, disabled.
> > ACPI: PCI Interrupt Link [LP2A] (IRQs *9)
> > ACPI: PCI Interrupt Link [LP2B] (IRQs) *0, disabled.
> > Linux Plug and Play Support v0.97 (c) Adam Belay
> > xen_mem: Initialising balloon driver.
> > SCSI subsystem initialized
> > usbcore: registered new driver usbfs
> > usbcore: registered new driver hub
> > PCI: Using ACPI for IRQ routing
> > PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
> > IA-32 Microcode Update Driver: v1.14-xen <tigran@veritas.com>
> > VFS: Disk quotas dquot_6.5.1
> > Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
> > JFS: nTxBlock = 512, nTxLock = 4096
> > SGI XFS with ACLs, security attributes, realtime, large block numbers, no debug enabled
> > Initializing Cryptographic API
> > io scheduler noop registered
> > io scheduler anticipatory registered (default)
> > io scheduler deadline registered
> > io scheduler cfq registered
> > PNP: No PS/2 controller found. Probing ports directly.
> > serio: i8042 AUX port at 0x60,0x64 irq 12
> > serio: i8042 KBD port at 0x60,0x64 irq 1
> > RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize
> > Xen virtual console successfully installed as ttyS0
> > Event-channel device installed.
> > blkif_init: reqs=64, pages=704, mmap_vstart=0xc0800000
> > Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
> > ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
> > SvrWks OSB4: IDE controller at PCI slot 0000:00:0f.1
> > SvrWks OSB4: chipset revision 0
> > SvrWks OSB4: not 100% native mode: will probe irqs later
> > ide0: BM-DMA at 0x0700-0x0707, BIOS settings: hda:DMA, hdb:DMA
> > ide1: BM-DMA at 0x0708-0x070f, BIOS settings: hdc:DMA, hdd:DMA
> > hda: LG CD-ROM CRN-8245B, ATAPI CD/DVD-ROM drive
> > ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
> > hda: ATAPI 24X CD-ROM drive, 128kB Cache, (U)DMA
> > Uniform CD-ROM driver Revision: 3.20
> > ide-floppy driver 0.99.newide
> > ACPI: PCI Interrupt 0000:01:03.0[A] -> GSI 28 (level, low) -> IRQ 16
> > scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0
> > <Adaptec aic7892 Ultra160 SCSI adapter>
> > aic7892: Ultra160 Wide Channel A, SCSI Id=7, 32/253 SCBs
> >
> > ACPI: PCI Interrupt 0000:01:05.0[A] -> GSI 20 (level, low) -> IRQ 17
> > scsi1 : IBM PCI ServeRAID 7.12.05 Build 761 <ServeRAID 4L>
> > Vendor: IBM Model: SERVERAID Rev: 1.00
> > Type: Direct-Access ANSI SCSI revision: 02
> > Vendor: IBM Model: SERVERAID Rev: 1.00
> > Type: Direct-Access ANSI SCSI revision: 02
> > Vendor: IBM Model: SERVERAID Rev: 1.00
> > Type: Processor ANSI SCSI revision: 02
> > Vendor: IBM Model: FTlV1 S2 Rev: 0
> > Type: Processor ANSI SCSI revision: 02
> > Fusion MPT base driver 3.03.07
> > Copyright (c) 1999-2005 LSI Logic Corporation
> > Fusion MPT SPI Host driver 3.03.07
> > Fusion MPT misc device (ioctl) driver 3.03.07
> > mptctl: Registered with Fusion MPT base driver
> > mptctl: /dev/mptctl @ (major,minor=10,220)
> > usbmon: debugfs is not available
> > usbcore: registered new driver libusual
> > mice: PS/2 mouse device common for all mice
> > md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
> > md: bitmap version 4.39
> > NET: Registered protocol family 2
> > input: AT Translated Set 2 keyboard as /class/input/input0
> > IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
> > TCP established hash table entries: 4096 (order: 3, 32768 bytes)
> > TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
> > TCP: Hash tables configured (established 4096 bind 4096)
> > TCP reno registered
> > Initializing IPsec netlink socket
> > NET: Registered protocol family 1
> > NET: Registered protocol family 17
> > NET: Registered protocol family 8
> > NET: Registered protocol family 20
> > Using IPI No-Shortcut mode
> > Freeing unused kernel memory: 208k freed
> > Loading, please wait...
> > Begin: Loading essential drivers... ...
> > logips2pp: Detected unknown logitech mouse model 0
> > eepro100.c:v1.09j-t 9/29/99 Donald Becker http://www.scyld.com/network/eepro100.html
> > eepro100.c: $Revision: 1.36 $ 2000/11/17 Modified by Andrey V. Savochkin <saw@saw.sw.com.sg> and others
> > ACPI: PCI Interrupt 0000:00:02.0[A] -> GSI 27 (level, low) -> IRQ 18
> > eth0: 0000:00:02.0, 00:02:55:C7:CA:D8, IRQ 18.
> > Board assembly 754338-001, Physical connectors present: RJ45
> > Primary interface chip i82555 PHY #1.
> > General self-test: passed.
> > Serial sub-system self-test: passed.
> > Internal registers self-test: passed.
> > ROM checksum self-test: passed (0x04f4518b).
> > ACPI: PCI Interrupt 0000:00:0a.0[A] -> GSI 25 (level, low) -> IRQ 19
> > eth1: 0000:00:0a.0, 00:02:55:C7:CA:D9, IRQ 19.
> > Board assembly 754338-001, Physical connectors present: RJ45
> > Primary interface chip i82555 PHY #1.
> > General self-test: passed.
> > Serial sub-system self-test: passed.
> > Internal registers self-test: passed.
> > ROM checksum self-test: passed (0x04f4518b).
> > Intel(R) PRO/1000 Network Driver - version 6.3.9-k4
> > Copyright (c) 1999-2005 Intel Corporation.
> > Done.
> > Begin: Running /scripts/init-premount ...
> > input: PS/2 Logitech Mouse as /class/input/input1
> > Done.
> > Begin: Mounting root file system... ...
> > Begin: Running /scripts/local-top ...
> > device-mapper: 4.5.0-ioctl (2005-10-04) initialised: dm-devel@redhat.com
> > /scripts/local-top/lvm: 36: vgchange: not found
> > Done.
> > SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
> > sda: assuming Write Enabled
> > sda: assuming drive cache: write through
> > SCSI device sda: 71096320 512-byte hdwr sectors (36401 MB)
> > sda: assuming Write Enabled
> > sda: assuming drive cache: write through
> > sda: sda1 sda2 < sda5 >
> > sd 1:0:0:0: Attached scsi disk sda
> > SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
> > sdb: assuming Write Enabled
> > sdb: assuming drive cache: write through
> > SCSI device sdb: 71096320 512-byte hdwr sectors (36401 MB)
> > sdb: assuming Write Enabled
> > sdb: assuming drive cache: write through
> > sdb: unknown partition table
> > sd 1:0:1:0: Attached scsi disk sdb
> > Begin: Running /scripts/local-premount ...
> > Done.
> > EXT3-fs: INFO: recovery required on readonly filesystem.
> > EXT3-fs: write access will be enabled during recovery.
> > kjournald starting. Commit interval 5 seconds
> > EXT3-fs: recovery complete.
> > EXT3-fs: mounted filesystem with ordered data mode.
> > Begin: Running /scripts/log-bottom ...
> > Done.
> > Done.
> > Begin: Running /scripts/init-bottom ...
> > Done.
> > INIT: version 2.86 booting
> > Starting the hotplug events dispatcher: udevd.
> > Synthesizing the initial hotplug events...done.
> > Waiting for /dev to be fully populated...done.
> > Loading /etc/console/boottime.kmap.gz
> > Activating swap.
> > Adding 1019880k swap on /dev/sda5. Priority:-1 extents:1 across:1019880k
> > Checking root file system...
> > fsck 1.37 (21-Mar-2005)
> > /dev/sda1: clean, 58437/512000 files, 445341/1023996 blocks
> > EXT3 FS on sda1, internal journal
> > System time was Sat Aug 12 00:22:00 UTC 2006.
> > Setting the System Clock using the Hardware Clock as reference...
> > System Clock set. System local time is now Sat Aug 12 00:22:01 UTC 2006.
> > Calculating module dependencies...done.
> > Loading modules...
> > auto
> > FATAL: Module auto not found.
> > 3c59x
> > softdog
> > Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0)
> > tulip
> > Linux Tulip driver version 1.1.13 (May 11, 2002)
> > e100
> > e100: Intel(R) PRO/100 Network Driver, 3.5.10-k2-NAPI
> > e100: Copyright(c) 1999-2005 Intel Corporation
> > All modules loaded.
> > Loading device-mapper support.
> > Starting Enterprise Volume Management System: device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > device-mapper: dm-linear: Device lookup failed
> > device-mapper: error adding target to table
> > evms.
> > Checking all file systems...
> > fsck 1.37 (21-Mar-2005)
> > Setting kernel variables ...
> > ... done.
> > Loading the saved-state of the serial devices...
> > Cannot get serial info: Invalid argument
> > Cannot get serial info: Invalid argument
> > /dev/ttyS1: No such file or directory
> > /dev/ttyS1: No such file or directory
> > Mounting local filesystems...
> > Cleaning /tmp /var/run /var/lock.
> > Cleaning: /etc/network/ifstate.
> > Setting up IP spoofing protection: rp_filter.
> > Configuring network interfaces: done.
> > Starting portmap daemon: portmap.
> >
> > Setting the System Clock using the Hardware Clock as reference...
> > System Clock set. Local time: Fri Aug 11 17:22:07 PDT 2006
> >
> > Recovering jove files ... Done.
> > Running ntpdate to synchronize clock.
> > Initializing random number generator...done.
> > Recovering nvi editor sessions... done.
> > Setting up X server socket directory /tmp/.X11-unix...done.
> > Setting up ICE socket directory /tmp/.ICE-unix...done.
> > INIT: Entering runlevel: 2
> > Starting system log daemon: syslogd.
> > Starting kernel log daemon: klogd.
> > Starting virtual private network daemon:.
> > Starting portmap daemon: portmap.
> > Starting NFS common utilities: statd lockd.
> > Starting internet superserver: inetd.
> > * The mptctl module is missing. Please have a look at the README.Debian.gz.
> > .
> > Exporting directories for NFS kernel daemon...Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
> > 10.27.3.21:/export/xen/fs/stevegt/crashme1/root: No such file or directory
> > 10.27.3.18:/export/xen/fs/stevegt/xentest2/root: No such file or directory
> > 10.27.3.17:/export/xen/fs/stevegt/xentest1/root: No such file or directory
> > 10.27.3.8:/export/xen/fs/cclarke/umltest1/root: No such file or directory
> > hotaru.chaosring.Org:/export/xen/fs/stevegt/crashme1/root: No such file or directory
> > rn-2.refactored-networks.com:/export/xen/fs/stevegt/xentest2/root: No such file or directory
> > xentest1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/xentest1/root: No such file or directory
> > umltest1.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/umltest1/root: No such file or directory
> > 10.27.3.26:/export/xen/fs/stevegt/gforge/root: No such file or directory
> > 10.27.3.23:/export/xen/fs/baseline/sarge/root: No such file or directory
> > 10.27.3.22:/export/xen/fs/stevegt/build1/root: No such file or directory
> > 10.27.3.16:/export/xen/fs/baseline/woody/root: No such file or directory
> > bugs.t7a.Org:/export/xen/fs/stevegt/gforge/root: No such file or directory
> > sarge.baseline.TerraLuna.Org:/export/xen/fs/baseline/sarge/root: No such file or directory
> > build1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/build1/root: No such file or directory
> > woody.baseline.TerraLuna.Org:/export/xen/fs/baseline/woody/root: No such file or directory
> > 10.27.3.19:/export/xen/fs/cclarke/ebond/root: No such file or directory
> > ebond.cclarke.TerraLuna.Org:/export/xen/fs/cclarke/ebond/root: No such file or directory
> > 10.27.3.29:/export/xen/fs/dmasten/mss0/root: No such file or directory
> > 10.27.3.27:/export/xen/fs/cclarke/ccms/root: No such file or directory
> > 10.27.3.25:/export/xen/fs/baseline/sid/root: No such file or directory
> > 10.27.3.24:/export/xen/fs/stevegt/cvs1/root: No such file or directory
> > sid.stevegt.TerraLuna.Org:/export/xen/fs/baseline/sid/root: No such file or directory
> > cvs1.stevegt.TerraLuna.Org:/export/xen/fs/stevegt/cvs1/root: No such file or directory
> > 10.27.3.28:/export/xen/fs/stevegt/t7a/root: No such file or directory
> > 10.27.3.20:/export/xen/fs/stevegt/tcx/root: No such file or directory
> > tcx.TerraLuna.Org:/export/xen/fs/stevegt/tcx/root: No such file or directory
> > n2h54.prd.TerraLuna.Org:/export/xen: No such file or directory
> > n2h53.prd.TerraLuna.Org:/export/xen: No such file or directory
> > n2h51.prd.TerraLuna.Org:/export/xen: No such file or directory
> > n2h41.prd.TerraLuna.Org:/export/xen: No such file or directory
> > done.
> > Starting NFS kernel daemon: nfsdNFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
> > NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
> > NFSD: starting 90-second grace period
> > mountd.
> > Starting OpenBSD Secure Shell server: sshdNET: Registered protocol family 10
> > lo: Disabled Privacy Extensions
> > IPv6 over IPv4 tunneling driver
> > .
> > Starting NTP server: ntpd.
> > AFS module /lib/modules/2.6.16.13-xen/fs/openafs.mp.o does not exist. Not starting AFS.
> > Please consider building kernel modules using instructions in
> > /usr/share/doc/openafs-client/README.modules
> > Starting DRBD resources: drbd: initialised. Version: 0.7.20 (api:79/proto:74)
> > drbd: SVN Revision: 2260 build by root@n4h8, 2006-08-04 11:00:44
> > drbd: registered as block device major 147
> > [ d0 drbd7: resync bitmap: bits=1048576 words=32768
> > drbd7: size = 4095 MB (4194303 KB)
> > drbd7: 0 KB marked out-of-sync by on disk bit-map.
> > drbd7: No usable activity log found.
> > drbd7: drbdsetup [3825]: cstate Unconfigured --> StandAlone
> > d1 drbd8: resync bitmap: bits=1048576 words=32768
> > drbd8: size = 4095 MB (4194303 KB)
> > drbd8: 0 KB marked out-of-sync by on disk bit-map.
> > drbd8: Found 4 transactions (46 active extents) in activity log.
> > drbd8: Marked additional 156 MB as out-of-sync based on AL.
> > drbd8: drbdsetup [3829]: cstate Unconfigured --> StandAlone
> > d2 drbd9: resync bitmap: bits=2621440 words=81920
> > drbd9: size = 9 GB (10485759 KB)
> > drbd9: 0 KB marked out-of-sync by on disk bit-map.
> > drbd9: Found 4 transactions (192 active extents) in activity log.
> > drbd9: drbdsetup [3833]: cstate Unconfigured --> StandAlone
> > d3 drbd10: resync bitmap: bits=1835008 words=57344
> > drbd10: size = 7167 MB (7340031 KB)
> > drbd10: 0 KB marked out-of-sync by on disk bit-map.
> > drbd10: Found 4 transactions (192 active extents) in activity log.
> > drbd10: Marked additional 508 MB as out-of-sync based on AL.
> > drbd10: drbdsetup [3837]: cstate Unconfigured --> StandAlone
> > d4 drbd11: resync bitmap: bits=786432 words=24576
> > drbd11: size = 3071 MB (3145727 KB)
> > drbd11: 0 KB marked out-of-sync by on disk bit-map.
> > drbd11: Found 4 transactions (192 active extents) in activity log.
> > drbd11: drbdsetup [3841]: cstate Unconfigured --> StandAlone
> > d5 drbd12: resync bitmap: bits=786432 words=24576
> > drbd12: size = 3071 MB (3145727 KB)
> > drbd12: 0 KB marked out-of-sync by on disk bit-map.
> > drbd12: Found 4 transactions (136 active extents) in activity log.
> > drbd12: Marked additional 508 MB as out-of-sync based on AL.
> > drbd12: drbdsetup [3845]: cstate Unconfigured --> StandAlone
> > s0 s1 s2 s3 s4 s5 n0 drbd7: drbdsetup [3883]: cstate StandAlone --> Unconnected
> > drbd7: drbd7_receiver [3884]: cstate Unconnected --> WFConnection
> > n1 drbd8: drbdsetup [3891]: cstate StandAlone --> Unconnected
> > drbd8: drbd8_receiver [3892]: cstate Unconnected --> WFConnection
> > n2 drbd9: drbdsetup [3899]: cstate StandAlone --> Unconnected
> > drbd9: drbd9_receiver [3900]: cstate Unconnected --> WFConnection
> > n3 drbd10: drbdsetup [3907]: cstate StandAlone --> Unconnected
> > drbd10: drbd10_receiver [3908]: cstate Unconnected --> WFConnection
> > n4 drbd11: drbdsetup [3915]: cstate StandAlone --> Unconnected
> > drbd11: drbd11_receiver [3916]: cstate Unconnected --> WFConnection
> > n5 drbd12: drbdsetup [3923]: cstate StandAlone --> Unconnected
> > drbd12: drbd12_receiver [3924]: cstate Unconnected --> WFConnection
> > ].
> > ..........
> > ***************************************************************
> > DRBD's startup script waits for the peer node(s) to appear.
> > - In case this node was already a degraded cluster before the
> > reboot the timeout is 120 seconds. [degr-wfc-timeout]
> > - If the peer was available before the reboot the timeout will
> > expire after 0 seconds. [wfc-timeout]
> > (These values are for resource 'ds1.t7a.org'; 0 sec -> wait forever)
> > To abort waiting enter 'yes' [ 18]:drbd7: drbd7_receiver [3884]: cstate WFConnection --> WFReportParams
> > drbd7: Handshake successful: DRBD Network Protocol version 74
> > drbd7: Connection established.
> > drbd7: I am(S): 1:00000002:00000001:00000018:00000002:01
> > drbd7: Peer(S): 1:00000002:00000001:00000019:00000002:10
> > drbd7: drbd7_receiver [3884]: cstate WFReportParams --> WFBitMapT
> > drbd7: Secondary/Unknown --> Secondary/Secondary
> > drbd7: drbd7_receiver [3884]: cstate WFBitMapT --> SyncTarget
> > drbd7: Resync started as SyncTarget (need to sync 520192 KB [130048 bits set]).
> > drbd9: drbd9_receiver [3900]: cstate WFConnection --> WFReportParams
> > drbd9: Handshake successful: DRBD Network Protocol version 74
> > drbd9: Connection established.
> > drbd9: I am(S): 1:00000002:00000001:0000000d:00000001:01
> > drbd9: Peer(S): 1:00000002:00000001:0000000e:00000001:10
> > drbd9: drbd9_receiver [3900]: cstate WFReportParams --> WFBitMapT
> > drbd9: Secondary/Unknown --> Secondary/Secondary
> > drbd9: drbd9_receiver [3900]: cstate WFBitMapT --> SyncTarget
> > drbd9: Resync started as SyncTarget (need to sync 290816 KB [72704 bits set]).
> > drbd9: drbd9_receiver [3900]: cstate SyncTarget --> PausedSyncT
> > drbd9: Syncer waits for sync group.
> > drbd11: drbd11_receiver [3916]: cstate WFConnection --> WFReportParams
> > drbd11: Handshake successful: DRBD Network Protocol version 74
> > drbd11: Connection established.
> > drbd11: I am(S): 1:00000002:00000001:00000008:00000001:01
> > drbd11: Peer(S): 1:00000002:00000001:00000009:00000001:10
> > drbd11: drbd11_receiver [3916]: cstate WFReportParams --> WFBitMapT
> > drbd11: Secondary/Unknown --> Secondary/Secondary
> > drbd11: drbd11_receiver [3916]: cstate WFBitMapT --> SyncTarget
> > drbd11: Resync started as SyncTarget (need to sync 274432 KB [68608 bits set]).
> > drbd11: drbd11_receiver [3916]: cstate SyncTarget --> PausedSyncT
> > drbd11: Syncer waits for sync group.
> > drbd8: drbd8_receiver [3892]: cstate WFConnection --> WFReportParams
> > drbd8: Handshake successful: DRBD Network Protocol version 74
> > drbd8: Connection established.
> > drbd8: I am(S): 1:00000002:00000001:00000017:00000001:11
> > drbd8: Peer(S): 1:00000002:00000001:00000017:00000001:01
> > drbd8: drbd8_receiver [3892]: cstate WFReportParams --> WFBitMapS
> > drbd8: Secondary/Unknown --> Secondary/Secondary
> > drbd8: drbd8_receiver [3892]: cstate WFBitMapS --> SyncSource
> > drbd8: Resync started as SyncSource (need to sync 159744 KB [39936 bits set]).
> > drbd8: drbd8_receiver [3892]: cstate SyncSource --> PausedSyncS
> > drbd8: Syncer waits for sync group.
> > drbd12: drbd12_receiver [3924]: cstate WFConnection --> WFReportParams
> > drbd12: Handshake successful: DRBD Network Protocol version 74
> > drbd12: Connection established.
> > drbd12: I am(S): 1:00000002:00000001:00000001:00000001:11
> > drbd12: Peer(S): 1:00000002:00000001:00000001:00000001:01
> > drbd12: drbd12_receiver [3924]: cstate WFReportParams --> WFBitMapS
> > drbd12: Secondary/Unknown --> Secondary/Secondary
> > drbd12: drbd12_receiver [3924]: cstate WFBitMapS --> SyncSource
> > drbd12: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]).
> > drbd12: drbd12_receiver [3924]: cstate SyncSource --> PausedSyncS
> > drbd12: Syncer waits for sync group.
> > drbd10: drbd10_receiver [3908]: cstate WFConnection --> WFReportParams
> > drbd10: Handshake successful: DRBD Network Protocol version 74
> > drbd10: Connection established.
> > drbd10: I am(S): 1:00000002:00000001:0000000b:00000001:11
> > drbd10: Peer(S): 1:00000002:00000001:0000000b:00000001:01
> > drbd10: drbd10_receiver [3908]: cstate WFReportParams --> WFBitMapS
> >
> > drbd10: Secondary/Unknown --> Secondary/Secondary
> > drbd10: drbd10_receiver [3908]: cstate WFBitMapS --> SyncSource
> > drbd10: Resync started as SyncSource (need to sync 520192 KB [130048 bits set]).
> > drbd10: drbd10_receiver [3908]: cstate SyncSource --> PausedSyncS
> > drbd10: Syncer waits for sync group.
> > Heartbeat not configured: /etc/ha.d/ha.cf not found.
> > Heartbeat failure [rc=1]. Failed.
> > Starting deferred execution scheduler: atd.
> > Starting periodic command scheduler: cron.
> > Starting watchdog daemon: watchdog.
> > openMosix: ERROR: Cannot find the /proc/hpc directory.
> > openMosix: ERROR: Looks like this is not an openMosix enabled kernel.
> > openMosix: ERROR: Configuration ABORTED.
> > openMosix: HINT: need to recompile your kernel with openMosix support enabled?
> > openMosix: HINT: need to update your boot manager (lilo, grub, etc)?
> > openMosix: HINT: have you read the docs at http://openMosix.sf.net/ ?
> > Bridge firewalling registered
> > device vif0.0 entered promiscuous mode
> > xenbr0: port 1(vif0.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 1(vif0.0) entering forwarding state
> > device peth0 entered promiscuous mode
> > xenbr0: port 2(peth0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 2(peth0) entering forwarding state
> > Starting auto Xen domains: afs1.t7a.orgdrbd9: Secondary/Secondary --> Secondary/Primary
> > device vif1.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif1.0: link is not ready
> > ip_tables: (C) 2000-2006 Netfilter Core Team
> > Error: Device 2049 (vbd) could not be connected. Backend device not found.
> > xenbr0: port 3(vif1.0) entering disabled state
> > ! build2.t7a.orgdevice vif1.0 left promiscuous mode
> > xenbr0: port 3(vif1.0) entering disabled state
> > device vif2.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif2.0: link is not ready
> > drbd10: Secondary/Secondary --> Primary/Secondary
> > ADDRCONF(NETDEV_CHANGE): vif2.0: link becomes ready
> > xenbr0: port 3(vif2.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 3(vif2.0) entering forwarding state
> > ds1.t7a.orgdrbd7: Secondary/Secondary --> Secondary/Primary
> > device vif3.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif3.0: link is not ready
> > Error: Device 2049 (vbd) could not be connected. Backend device not found.
> > ! kdc1.t7a.orgxenbr0: port 4(vif3.0) entering disabled state
> >
> > device vif3.0 left promiscuous mode
> > xenbr0: port 4(vif3.0) entering disabled state
> >
> >
> >
> >
> >
> >
> > drbd8: Secondary/Secondary --> Primary/Secondary
> > device vif4.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif4.0: link is not ready
> > ADDRCONF(NETDEV_CHANGE): vif4.0: link becomes ready
> > xenbr0: port 4(vif4.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 4(vif4.0) entering forwarding state
> > w1.cdint.comdrbd11: Secondary/Secondary --> Secondary/Primary
> > device vif5.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif5.0: link is not ready
> > Error: Device 2049 (vbd) could not be connected. Backend device not found.
> > xenbr0: port 5(vif5.0) entering disabled state
> > ! webkdc.t7a.orgdevice vif5.0 left promiscuous mode
> > xenbr0: port 5(vif5.0) entering disabled state
> > device vif6.0 entered promiscuous mode
> > ADDRCONF(NETDEV_UP): vif6.0: link is not ready
> > drbd12: Secondary/Secondary --> Primary/Secondary
> > ADDRCONF(NETDEV_CHANGE): vif6.0: link becomes ready
> > xenbr0: port 5(vif6.0) entering learning state
> > xenbr0: topology change detected, propagating
> > xenbr0: port 5(vif6.0) entering forwarding state
> > [done]
> >
> >
> > _sudZUZ#Z#XZo=_ DDDD EEEEEE BBBB IIIIII AAAA NN NN
> > _jmZZ2!!~---~!!X##wa DD DD EE BB BB II AA AA NNN NN
> > .<wdP~~ -!YZL, DD DD EEEEE BBBBB II AAAAAA NNNN NN
> > .mX2' _%aaa__ XZ[. DD DD EE BB BB II AA AA NN NNNN
> > oZ[ _jdXY!~?S#wa ]Xb; DDDD EEEEEE BBBBB IIIIII AA AA NN NN
> > _#e' .]X2( ~Xw| )XXc
> > .2Z` ]X[. xY| ]oZ( Linux Version 2.6.16.13-xen
> > .2#; )3k; _s!~ jXf` Compiled #3 SMP Fri Aug 4 10:57:44 PDT 2006
> > 1Z> -]Xb/ ~ __#2( Two 1.13GHz Intel Intel(R) Pentium(R) III CPU family 1133MHz Processors
> > -Zo; +!4ZwaaaauZZXY' 72M RAM
> > *#[, ~-?!!!!!!-~ 4523.20 Bogomips Total
> > XUb;. n4h34
> > )YXL,,
> > +3#bc,
> > -)SSL,,
> > ~~~~~
> > Updating the Linuxlogo... done.
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login: drbd7: Resync done (total 137 sec; paused 0 sec; 3796 K/sec)
> > drbd7: drbd7_worker [3826]: cstate SyncTarget --> Connected
> > drbd8: Syncer continues.
> > drbd8: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> > drbd9: Syncer continues.
> > drbd9: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
> > drbd10: Syncer continues.
> > drbd10: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> > drbd11: Syncer continues.
> > drbd11: drbd7_worker [3826]: cstate PausedSyncT --> SyncTarget
> > drbd12: Syncer continues.
> > drbd12: drbd7_worker [3826]: cstate PausedSyncS --> SyncSource
> > drbd8: Resync done (total 211 sec; paused 137 sec; 2156 K/sec)
> > drbd8: drbd8_worker [3830]: cstate SyncSource --> Connected
> > drbd11: Resync done (total 239 sec; paused 137 sec; 2688 K/sec)
> > drbd11: drbd11_worker [3842]: cstate SyncTarget --> Connected
> > drbd9: Resync done (total 243 sec; paused 137 sec; 2740 K/sec)
> > drbd9: drbd9_worker [3834]: cstate SyncTarget --> Connected
> >
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login: drbd12: Resync done (total 306 sec; paused 136 sec; 3056 K/sec)
> > drbd12: drbd12_worker [3846]: cstate SyncSource --> Connected
> >
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login:
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login: drbd10: Resync done (total 310 sec; paused 136 sec; 2988 K/sec)
> > drbd10: drbd10_worker [3838]: cstate SyncSource --> Connected
> >
> > Debian GNU/Linux 3.0 n4h34 ttyS0
> >
> > n4h34 login: Unable to handle kernel paging request at virtual address c0976590
> > printing eip:
> > c03d6ed7
> > *pde = ma 3cf91067 pa 00f91067
> > *pte = ma 00000000 pa fffff000
> > Oops: 0000 [#1]
> > SMP
> > Modules linked in: iptable_filter ip_tables x_tables bridge drbd ipv6 nfsd lockd sunrpc e100 tulip softdog 3c59x evdev sd_mod dm_mod thermal processor fan e1000 eepro100 mii tg3
> > CPU: 0
> > EIP: 0061:[<c03d6ed7>] Not tainted VLI
> > EFLAGS: 00010206 (2.6.16.13-xen #3)
> > EIP is at skb_copy_bits+0x127/0x280
> > eax: c0976000 ebx: 000005a8 ecx: 0000016a edx: c3711720
> > esi: c0976590 edi: c37110e0 ebp: 000005a8 esp: c00818c8
> > ds: 007b es: 007b ss: 0069
> > Process drbd12_receiver (pid: 3924, threadinfo=c0080000 task=c3efda90)
> > Stack: <0>c1012ec0 00000002 c03d65ef c1d51a00 000005ea 00000042 00000000 00000000
> > 00000020 c3edc800 c376fd64 c03d6b0f c376fd64 00000042 c37110e0 000005a8
> > c1a70000 c54690c0 00000000 c1d51ac0 c3edc800 c376fd64 c03dc478 c376fd64
> > Call Trace:
> > [<c03d65ef>] pskb_expand_head+0xdf/0x140
> > [<c03d6b0f>] __pskb_pull_tail+0x7f/0x320
> > [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
> > [<c03dc478>] dev_queue_xmit+0x328/0x370
> > [<c5462f7e>] br_dev_queue_push_xmit+0xbe/0x140 [bridge]
> > [<c5469212>] br_nf_post_routing+0x102/0x1c0 [bridge]
> > [<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
> > [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> > [<c03f43f8>] nf_iterate+0x78/0x90
> > [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> > [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> > [<c03f447e>] nf_hook_slow+0x6e/0x110
> > [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> > [<c5463061>] br_forward_finish+0x61/0x70 [bridge]
> > [<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
> > [<c5468995>] br_nf_forward_finish+0x75/0x130 [bridge]
> > [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> > [<c5468b38>] br_nf_forward_ip+0xe8/0x190 [bridge]
> > [<c5468920>] br_nf_forward_finish+0x0/0x130 [bridge]
> > [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> > [<c03f43f8>] nf_iterate+0x78/0x90
> > [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> > [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> > [<c03f447e>] nf_hook_slow+0x6e/0x110
> > [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> > [<c5463167>] __br_forward+0x77/0x80 [bridge]
> > [<c5463000>] br_forward_finish+0x0/0x70 [bridge]
> > [<c5463fbf>] br_handle_frame_finish+0xdf/0x160 [bridge]
> > [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> > [<c5467d89>] br_nf_pre_routing_finish+0xf9/0x370 [bridge]
> > [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> > [<c0322e3a>] loopback_start_xmit+0xba/0x110
> > [<c0400d70>] ip_finish_output+0x0/0x220
> > [<c03dc07e>] dev_hard_start_xmit+0x5e/0x130
> > [<c03dc3b5>] dev_queue_xmit+0x265/0x370
> > [<c03f43f8>] nf_iterate+0x78/0x90
> > [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> > [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> > [<c03f447e>] nf_hook_slow+0x6e/0x110
> > [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> > [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> > [<c54685fc>] br_nf_pre_routing+0x26c/0x520 [bridge]
> > [<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
> > [<c03f43f8>] nf_iterate+0x78/0x90
> > [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> > [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> > [<c03f447e>] nf_hook_slow+0x6e/0x110
> > [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> > [<c546422d>] br_handle_frame+0x1ed/0x230 [bridge]
> > [<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
> > [<c03dcb21>] netif_receive_skb+0x1a1/0x330
> > [<c03dcd87>] process_backlog+0xd7/0x190
> > [<c03dcf2a>] net_rx_action+0xea/0x230
> > [<c0125915>] __do_softirq+0xf5/0x120
> > [<c01259d5>] do_softirq+0x95/0xa0
> > [<c0125a42>] local_bh_enable+0x62/0xa0
> > [<c0406bd1>] tcp_prequeue_process+0x71/0x80
> > [<c04070e9>] tcp_recvmsg+0x349/0x750
> > [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
> > [<c03d5085>] sock_common_recvmsg+0x55/0x70
> > [<c03d11cf>] sock_recvmsg+0xef/0x110
> > [<c03143fa>] force_evtchn_callback+0xa/0x10
> > [<c0147163>] mempool_alloc+0x33/0xe0
> > [<c01367d0>] autoremove_wake_function+0x0/0x60
> > [<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
> > [<c02b2ee0>] generic_make_request+0xf0/0x160
> > [<c529c680>] drbd_recv+0x90/0x190 [drbd]
> > [<c529cdec>] drbd_recv_header+0x2c/0xf0 [drbd]
> > [<c529e580>] receive_DataRequest+0x0/0x7d0 [drbd]
> > [<c52a045c>] drbdd+0x1c/0x150 [drbd]
> > [<c52a105a>] drbdd_init+0x7a/0x1a0 [drbd]
> > [<c52a7136>] drbd_thread_setup+0x86/0xf0 [drbd]
> > [<c52a70b0>] drbd_thread_setup+0x0/0xf0 [drbd]
> > [<c0102f75>] kernel_thread_helper+0x5/0x10
> > Code: 8b 4c 24 30 8b 7c 24 34 8b 91 a0 00 00 00 8b 4c 24 18 0f b7 74 ca 18 8b 4c 24 14 8d 34 06 01 fe 29 ce 8b 7c 24 38 89 d9 c1 e9 02 <f3> a5 89 d9 83 e1 03 74 02 f3 a4 89 04 24 ba 02 00 00 00 89 54
> > <0>Kernel panic - not syncing: Fatal exception in interrupt
> > (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
> >
> > --
> > Stephen G. Traugott (KG6HDQ)
> > Managing Partner, TerraLuna LLC
> > stevegt@TerraLuna.Org -- http://www.t7a.org
>
> --
> Stephen G. Traugott (KG6HDQ)
> Managing Partner, TerraLuna LLC
> stevegt@TerraLuna.Org -- http://www.t7a.org
--
Stephen G. Traugott (KG6HDQ)
Managing Partner, TerraLuna LLC
stevegt@TerraLuna.Org -- http://www.t7a.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
2006-08-12 5:50 ` Steve Traugott
@ 2006-08-12 9:50 ` Keir Fraser
0 siblings, 0 replies; 31+ messages in thread
From: Keir Fraser @ 2006-08-12 9:50 UTC (permalink / raw)
To: Steve Traugott, Ian Pratt; +Cc: xen-devel
On 12/8/06 6:50 am, "Steve Traugott" <stevegt@TerraLuna.Org> wrote:
> Another data point... Dom0 seems to only want to crash when more than
> 3 or 4 domU's are running (each with their own DRBD root, with DRBD
> running in dom0), and the below 'nc' command is run in the last
> domU...
>
> Still looking.
What does the crash look like? There was no oops message from domain0 in the
kernel logs that you posted.
-- Keir
^ permalink raw reply [flat|nested] 31+ messages in thread
* RE: Unable to handle kernel paging request
@ 2006-08-12 18:48 Ian Pratt
2006-08-13 0:08 ` Keir Fraser
0 siblings, 1 reply; 31+ messages in thread
From: Ian Pratt @ 2006-08-12 18:48 UTC (permalink / raw)
To: Keir Fraser, Steve Traugott; +Cc: xen-devel
Actually, Steve did post the oops (appended).
It's possible this was a cset before the alignment fix which would have
exercised skb copying more heavily, but that's no excuse for it
crashing.
Ian
n4h34 login: Unable to handle kernel paging request at virtual address
c0976590
printing eip:
c03d6ed7
*pde = ma 3cf91067 pa 00f91067
*pte = ma 00000000 pa fffff000
Oops: 0000 [#1]
SMP
Modules linked in: iptable_filter ip_tables x_tables bridge drbd ipv6
nfsd lockd sunrpc e100 tulip softdog 3c59x evdev sd_mod dm_mod thermal
processor fan e1000 eepro100 mii tg3
CPU: 0
EIP: 0061:[<c03d6ed7>] Not tainted VLI
EFLAGS: 00010206 (2.6.16.13-xen #3)
EIP is at skb_copy_bits+0x127/0x280
eax: c0976000 ebx: 000005a8 ecx: 0000016a edx: c3711720
esi: c0976590 edi: c37110e0 ebp: 000005a8 esp: c00818c8
ds: 007b es: 007b ss: 0069
Process drbd12_receiver (pid: 3924, threadinfo=c0080000 task=c3efda90)
Stack: <0>c1012ec0 00000002 c03d65ef c1d51a00 000005ea 00000042 00000000
00000000
00000020 c3edc800 c376fd64 c03d6b0f c376fd64 00000042 c37110e0
000005a8
c1a70000 c54690c0 00000000 c1d51ac0 c3edc800 c376fd64 c03dc478
c376fd64
Call Trace:
[<c03d65ef>] pskb_expand_head+0xdf/0x140
[<c03d6b0f>] __pskb_pull_tail+0x7f/0x320
[<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
[<c03dc478>] dev_queue_xmit+0x328/0x370
[<c5462f7e>] br_dev_queue_push_xmit+0xbe/0x140 [bridge]
[<c5469212>] br_nf_post_routing+0x102/0x1c0 [bridge]
[<c54690c0>] br_nf_dev_queue_xmit+0x0/0x50 [bridge]
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c5463061>] br_forward_finish+0x61/0x70 [bridge]
[<c5462ec0>] br_dev_queue_push_xmit+0x0/0x140 [bridge]
[<c5468995>] br_nf_forward_finish+0x75/0x130 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5468b38>] br_nf_forward_ip+0xe8/0x190 [bridge]
[<c5468920>] br_nf_forward_finish+0x0/0x130 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5463167>] __br_forward+0x77/0x80 [bridge]
[<c5463000>] br_forward_finish+0x0/0x70 [bridge]
[<c5463fbf>] br_handle_frame_finish+0xdf/0x160 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c5467d89>] br_nf_pre_routing_finish+0xf9/0x370 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c0322e3a>] loopback_start_xmit+0xba/0x110
[<c0400d70>] ip_finish_output+0x0/0x220
[<c03dc07e>] dev_hard_start_xmit+0x5e/0x130
[<c03dc3b5>] dev_queue_xmit+0x265/0x370
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c54685fc>] br_nf_pre_routing+0x26c/0x520 [bridge]
[<c5467c90>] br_nf_pre_routing_finish+0x0/0x370 [bridge]
[<c03f43f8>] nf_iterate+0x78/0x90
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c03f447e>] nf_hook_slow+0x6e/0x110
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c546422d>] br_handle_frame+0x1ed/0x230 [bridge]
[<c5463ee0>] br_handle_frame_finish+0x0/0x160 [bridge]
[<c03dcb21>] netif_receive_skb+0x1a1/0x330
[<c03dcd87>] process_backlog+0xd7/0x190
[<c03dcf2a>] net_rx_action+0xea/0x230
[<c0125915>] __do_softirq+0xf5/0x120
[<c01259d5>] do_softirq+0x95/0xa0
[<c0125a42>] local_bh_enable+0x62/0xa0
[<c0406bd1>] tcp_prequeue_process+0x71/0x80
[<c04070e9>] tcp_recvmsg+0x349/0x750
[<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
[<c03d5085>] sock_common_recvmsg+0x55/0x70
[<c03d11cf>] sock_recvmsg+0xef/0x110
[<c03143fa>] force_evtchn_callback+0xa/0x10
[<c0147163>] mempool_alloc+0x33/0xe0
[<c01367d0>] autoremove_wake_function+0x0/0x60
[<c50eebfc>] dm_request+0xbc/0x100 [dm_mod]
[<c02b2ee0>] generic_make_request+0xf0/0x160
[<c529c680>] drbd_recv+0x90/0x190 [drbd]
[<c529cdec>] drbd_recv_header+0x2c/0xf0 [drbd]
[<c529e580>] receive_DataRequest+0x0/0x7d0 [drbd]
[<c52a045c>] drbdd+0x1c/0x150 [drbd]
[<c52a105a>] drbdd_init+0x7a/0x1a0 [drbd]
[<c52a7136>] drbd_thread_setup+0x86/0xf0 [drbd]
[<c52a70b0>] drbd_thread_setup+0x0/0xf0 [drbd]
[<c0102f75>] kernel_thread_helper+0x5/0x10
Code: 8b 4c 24 30 8b 7c 24 34 8b 91 a0 00 00 00 8b 4c 24 18 0f b7 74 ca
18 8b 4c 24 14 8d 34 06 01 fe 29 ce 8b 7c 24 38 89 d9 c1 e9 02 <f3> a5
89 d9 83 e1 03 74 02 f3 a4 89 04 24 ba 02 00 00 00 89 54
<0>Kernel panic - not syncing: Fatal exception in interrupt
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
> -----Original Message-----
> From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk]
> Sent: 12 August 2006 10:50
> To: Steve Traugott; Ian Pratt
> Cc: xen-devel
> Subject: Re: Unable to handle kernel paging request
>
>
>
>
> On 12/8/06 6:50 am, "Steve Traugott" <stevegt@TerraLuna.Org> wrote:
>
> > Another data point... Dom0 seems to only want to crash
> when more than
> > 3 or 4 domU's are running (each with their own DRBD root, with DRBD
> > running in dom0), and the below 'nc' command is run in the last
> > domU...
> >
> > Still looking.
>
> What does the crash look like? There was no oops message from
> domain0 in the kernel logs that you posted.
>
> -- Keir
>
>
>
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
2006-08-12 18:48 Ian Pratt
@ 2006-08-13 0:08 ` Keir Fraser
2006-08-14 5:44 ` Steve Traugott
0 siblings, 1 reply; 31+ messages in thread
From: Keir Fraser @ 2006-08-13 0:08 UTC (permalink / raw)
To: Ian Pratt, Steve Traugott; +Cc: xen-devel
On 12/8/06 7:48 pm, "Ian Pratt" <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> It's possible this was a cset before the alignment fix which would have
> exercised skb copying more heavily, but that's no excuse for it
> crashing.
If the problem has only appeared with recent changesets then it might be
worth working backwards to find which one introduced the problem. The
network driver changes for GSO would be an obvious candidate.
-- Keir
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
2006-08-13 0:08 ` Keir Fraser
@ 2006-08-14 5:44 ` Steve Traugott
0 siblings, 0 replies; 31+ messages in thread
From: Steve Traugott @ 2006-08-14 5:44 UTC (permalink / raw)
To: Keir Fraser; +Cc: Ian Pratt, xen-devel
On Sun, Aug 13, 2006 at 01:08:53AM +0100, Keir Fraser wrote:
> On 12/8/06 7:48 pm, "Ian Pratt" <m+Ian.Pratt@cl.cam.ac.uk> wrote:
> > It's possible this was a cset before the alignment fix which would have
> > exercised skb copying more heavily, but that's no excuse for it
> > crashing.
>
> If the problem has only appeared with recent changesets then it might be
> worth working backwards to find which one introduced the problem. The
> network driver changes for GSO would be an obvious candidate.
So far -testing tip (changeset 9762) looks like it avoids both the
soft lockups that I was getting earlier in -testing, and the various
crashes I've been seeing in -unstable 10868.
(In addition to the dom0 oops we're talking about in this thread,
10868 also randomly crashes domUs when xendomains restores them during
boot; I haven't captured data on that since it hasn't been as critical
and can be worked around. It should be easy enough for folks to
duplicate if anyone wants to chase it down -- get a few domUs running
on a dual-CPU box, then reboot dom0, then check the console of the
domUs after everything's back up. About half of the domUs wound up
oopsed and hung in my case, possibly the odd-numbered ones but I'm not
sure if it was that consistent. If you can't duplicate it, let me
know and I'll move some boxes back to 10868 and have another go.)
Overall, I'm *really* wishing I had time to set up a stress test suite
that exercizes DRBD, aoe, heavy disk and net I/O, etc., and run daily
or weekly changesets across it on a dedicated set of hardware, posting
the results here. Maybe after the dust settles on this rollout I'm in
the middle of right now...
Steve
--
Stephen G. Traugott (KG6HDQ)
Managing Partner, TerraLuna LLC
stevegt@TerraLuna.Org -- http://www.t7a.org
^ permalink raw reply [flat|nested] 31+ messages in thread
* Unable to handle kernel paging request
@ 2007-09-27 16:30 Laurent Vivier
[not found] ` <46FBDA8A.8030109-6ktuUTfB/bM@public.gmane.org>
0 siblings, 1 reply; 31+ messages in thread
From: Laurent Vivier @ 2007-09-27 16:30 UTC (permalink / raw)
To: kvm-devel
[-- Attachment #1.1: Type: text/plain, Size: 2382 bytes --]
Hi,
booting a FC6 on my intel box (xeon) with a kernel 2.6.22.5 and KVM git, I had
the following error (not reproducible):
# kvm-userspace/qemu/x86_64-softmmu/qemu-system-x86_64 -hda fc6.qcow2 -net nic
-net tap -serial stdio -smp 4
...
INIT: version 2.86 booting
Welcome to Fedora Core
Press 'I' to enter interactive startup.
Setting clock (utc): Thu Sep 27 18:06:27 CEST 2007 [ OK ]
Starting udev: Unable to handle kernel paging request at ffffffff880e9000 RIP:
[<ffffffff8104ebc1>] sys_init_module+0x985/0x1786
PGD 203067 PUD 205063 PMD 7fc4067 PTE 6b50163
Oops: 0002 [1] SMP
CPU 1
Modules linked in: dm_snapshot dm_zero dm_mirror dm_mod ata_piix libata sd_mod s
csi_mod ext3 jbd mbcache ehci_hcd ohci_hcd uhci_hcd
Pid: 641, comm: modprobe Not tainted 2.6.22.5 #1
RIP: 0010:[<ffffffff8104ebc1>] [<ffffffff8104ebc1>] sys_init_module+0x985/0x178
6
RSP: 0018:ffff810006399e68 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffc20000090a20 RCX: 0000000000000f84
RDX: ffffffff880e6000 RSI: 0000000000000163 RDI: ffffffff880e9000
RBP: 0000000000000026 R08: ffff810007d94254 R09: 00000000000050cf
R10: 0000000000000000 R11: 0000000000000001 R12: ffffc2000007c300
R13: 0000000000000004 R14: ffffc200000900e0 R15: 00002ab868ac2010
FS: 00002ab8690096e0(0000) GS:ffff810007d94280(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: ffffffff880e9000 CR3: 0000000007cfd000 CR4: 00000000000006e0
Process modprobe (pid: 641, threadinfo ffff810006398000, task ffff81000788e000)
Stack: 00000000000276d8 0000000000000000 0000000000000000 000000000608f340
0000000000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 ffffc200000a2328 ffff810006402000
Call Trace:
[<ffffffff8108e67d>] vfs_read+0xcb/0x173
[<ffffffff81009b9e>] system_call+0x7e/0x83
Code: f3 aa 49 89 94 24 88 01 00 00 49 8b bc 24 90 01 00 00 e8 32
RIP [<ffffffff8104ebc1>] sys_init_module+0x985/0x1786
RSP <ffff810006399e68>
CR2: ffffffff880e9000
The instruction at [<ffffffff8104ebc1>] sys_init_module+0x985/0x1786 is:
0xffffffff8104ebc1 <sys_init_module+2437>: rep stos %al,%es:(%rdi)
Any idea of what happened ?
Laurent
--
------------- Laurent.Vivier-6ktuUTfB/bM@public.gmane.org --------------
"Software is hard" - Donald Knuth
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
[-- Attachment #2: Type: text/plain, Size: 228 bytes --]
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
[-- Attachment #3: Type: text/plain, Size: 186 bytes --]
_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
[not found] ` <46FBDA8A.8030109-6ktuUTfB/bM@public.gmane.org>
@ 2007-09-27 16:54 ` Laurent Vivier
[not found] ` <46FBE055.2040904-6ktuUTfB/bM@public.gmane.org>
0 siblings, 1 reply; 31+ messages in thread
From: Laurent Vivier @ 2007-09-27 16:54 UTC (permalink / raw)
To: Laurent Vivier; +Cc: kvm-devel
[-- Attachment #1.1: Type: text/plain, Size: 3711 bytes --]
Correct me if I'm wrong, perhaps it's the bug Nitin is hunting.
There is always something wrong in the management in the REP prefix.
I think what is happen is:
- we have a REP prefix, we save ECX and EIP.
- we set c->dst to emulate a "stos"
- goto writeback
- writeback: we try a "write_emulated()" with c->dst
- write_emulated failed() AND WE DON'T RESTORE ECX AND EIP -> it's bad...
- exit to QEMU
- re-enter in x86_emulate_insn() with already modified ECX and EIP.
Any comment ?
(Yes, I know, it's again another bug I've introduced into KVM...)
Laurent
Laurent Vivier wrote:
> Hi,
>
> booting a FC6 on my intel box (xeon) with a kernel 2.6.22.5 and KVM git, I had
> the following error (not reproducible):
>
> # kvm-userspace/qemu/x86_64-softmmu/qemu-system-x86_64 -hda fc6.qcow2 -net nic
> -net tap -serial stdio -smp 4
> ...
> INIT: version 2.86 booting
> Welcome to Fedora Core
> Press 'I' to enter interactive startup.
> Setting clock (utc): Thu Sep 27 18:06:27 CEST 2007 [ OK ]
> Starting udev: Unable to handle kernel paging request at ffffffff880e9000 RIP:
> [<ffffffff8104ebc1>] sys_init_module+0x985/0x1786
> PGD 203067 PUD 205063 PMD 7fc4067 PTE 6b50163
> Oops: 0002 [1] SMP
> CPU 1
> Modules linked in: dm_snapshot dm_zero dm_mirror dm_mod ata_piix libata sd_mod s
> csi_mod ext3 jbd mbcache ehci_hcd ohci_hcd uhci_hcd
> Pid: 641, comm: modprobe Not tainted 2.6.22.5 #1
> RIP: 0010:[<ffffffff8104ebc1>] [<ffffffff8104ebc1>] sys_init_module+0x985/0x178
> 6
> RSP: 0018:ffff810006399e68 EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffffc20000090a20 RCX: 0000000000000f84
> RDX: ffffffff880e6000 RSI: 0000000000000163 RDI: ffffffff880e9000
> RBP: 0000000000000026 R08: ffff810007d94254 R09: 00000000000050cf
> R10: 0000000000000000 R11: 0000000000000001 R12: ffffc2000007c300
> R13: 0000000000000004 R14: ffffc200000900e0 R15: 00002ab868ac2010
> FS: 00002ab8690096e0(0000) GS:ffff810007d94280(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: ffffffff880e9000 CR3: 0000000007cfd000 CR4: 00000000000006e0
> Process modprobe (pid: 641, threadinfo ffff810006398000, task ffff81000788e000)
> Stack: 00000000000276d8 0000000000000000 0000000000000000 000000000608f340
> 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000 0000000000000000 ffffc200000a2328 ffff810006402000
> Call Trace:
> [<ffffffff8108e67d>] vfs_read+0xcb/0x173
> [<ffffffff81009b9e>] system_call+0x7e/0x83
>
>
> Code: f3 aa 49 89 94 24 88 01 00 00 49 8b bc 24 90 01 00 00 e8 32
> RIP [<ffffffff8104ebc1>] sys_init_module+0x985/0x1786
> RSP <ffff810006399e68>
> CR2: ffffffff880e9000
>
>
> The instruction at [<ffffffff8104ebc1>] sys_init_module+0x985/0x1786 is:
>
> 0xffffffff8104ebc1 <sys_init_module+2437>: rep stos %al,%es:(%rdi)
>
> Any idea of what happened ?
>
> Laurent
>
>
> ------------------------------------------------------------------------
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> kvm-devel mailing list
> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
--
------------- Laurent.Vivier-6ktuUTfB/bM@public.gmane.org --------------
"Software is hard" - Donald Knuth
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
[-- Attachment #2: Type: text/plain, Size: 228 bytes --]
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
[-- Attachment #3: Type: text/plain, Size: 186 bytes --]
_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
[not found] ` <46FBE055.2040904-6ktuUTfB/bM@public.gmane.org>
@ 2007-09-30 9:07 ` Avi Kivity
[not found] ` <46FF6764.5090502-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
0 siblings, 1 reply; 31+ messages in thread
From: Avi Kivity @ 2007-09-30 9:07 UTC (permalink / raw)
To: Laurent Vivier, Kamble, Nitin A; +Cc: kvm-devel
Laurent Vivier wrote:
> (Yes, I know, it's again another bug I've introduced into KVM...)
>
>
To avoid this, I suggest that Nitin and yourself review each other's
patches. While I review every patch I commit, it works much better when
someone who's involved daily with the code reviews the patch.
--
error compiling committee.c: too many arguments to function
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: Unable to handle kernel paging request
[not found] ` <46FF6764.5090502-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
@ 2007-10-01 6:58 ` Laurent Vivier
0 siblings, 0 replies; 31+ messages in thread
From: Laurent Vivier @ 2007-10-01 6:58 UTC (permalink / raw)
To: Avi Kivity; +Cc: kvm-devel
[-- Attachment #1.1: Type: text/plain, Size: 494 bytes --]
Avi Kivity wrote:
> Laurent Vivier wrote:
>> (Yes, I know, it's again another bug I've introduced into KVM...)
>>
>>
>
> To avoid this, I suggest that Nitin and yourself review each other's
> patches. While I review every patch I commit, it works much better when
> someone who's involved daily with the code reviews the patch.
I agree...
Laurent
--
------------- Laurent.Vivier-6ktuUTfB/bM@public.gmane.org --------------
"Software is hard" - Donald Knuth
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
[-- Attachment #2: Type: text/plain, Size: 228 bytes --]
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
[-- Attachment #3: Type: text/plain, Size: 186 bytes --]
_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel
^ permalink raw reply [flat|nested] 31+ messages in thread
* unable to handle kernel paging request
@ 2016-09-15 14:08 Mark Gavalda
2016-09-15 16:05 ` Chris Mason
0 siblings, 1 reply; 31+ messages in thread
From: Mark Gavalda @ 2016-09-15 14:08 UTC (permalink / raw)
To: linux-btrfs
Hi,
Bumped into the following one today; kernel 4.4.0-36-generic Ubuntu
16.4.1; CPU went to 100% and only a hard restart solved the issue.
Since then everything's back to normal.
Please let me know how can I help get to the bottom of this?
[239049.350514] BUG: unable to handle kernel paging request at 00000000d3c53de8
[239049.358107] IP: [<ffffffff810eecf9>] hrtimer_active+0x9/0x60
[239049.364127] PGD a688df067 PUD 0
[239049.367828] Oops: 0000 [#2] SMP
[239049.371543] Modules linked in: xt_recent xt_nat xt_multiport
ip6t_REJECT nf_reject_ipv6 xt_hl ip6t_rt nf_conntrack_ipv6
nf_defrag_ipv6 ipt_REJECT nf_reject_ipv4 xt_limit xt_addrtype
xt_conntrack binfmt_misc veth xt_CHECKSUM iptable_mangle xt_tcpudp
ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_comment iptable_nat
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack
bridge stp llc ip6table_filter ip6_tables iptable_filter ip_tables
x_tables ppdev parport_pc parport serio_raw pvpanic ib_iser rdma_cm
iw_cm ib_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp
libiscsi scsi_transport_iscsi autofs4 btrfs raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor
raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul
crc32_pclmul aesni_intel aes_x86_64 lrw gf128mul glue_helper
ablk_helper cryptd psmouse virtio_scsi
[239049.457553] CPU: 24 PID: 1298718 Comm: kworker/u64:24 Tainted: G
D 4.4.0-36-generic #55-Ubuntu
[239049.467497] Hardware name: Google Google/Google, BIOS Google 01/01/2011
[239049.474348] Workqueue: btrfs-endio-write btrfs_endio_write_helper [btrfs]
[239049.481472] task: ffff8801512dee00 ti: ffff8801fd1c8000 task.ti:
ffff8801fd1c8000
[239049.489205] RIP: 0010:[<ffffffff810eecf9>] [<ffffffff810eecf9>]
hrtimer_active+0x9/0x60
[239049.497702] RSP: 0018:ffff8801fd1cbc20 EFLAGS: 00010046
[239049.503230] RAX: 0000000000000000 RBX: 00000000d3c53db8 RCX:
0000000000000000
[239049.510569] RDX: 0000000000000000 RSI: 0000000000000003 RDI:
00000000d3c53db8
[239049.517910] RBP: ffff8801fd1cbc20 R08: ffff88336f416d00 R09:
ffff88333fe57e00
[239049.525250] R10: 00000000000000a0 R11: 0000000001b5675f R12:
ffff8807024cb628
[239049.532595] R13: ffff8802d5e5bd98 R14: 0000000000000000 R15:
0000000000000003
[239049.539938] FS: 0000000000000000(0000) GS:ffff88336f400000(0000)
knlGS:0000000000000000
[239049.548246] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[239049.554234] CR2: 00000000d3c53de8 CR3: 0000000a5e8a1000 CR4:
00000000001406e0
[239049.561576] Stack:
[239049.563812] ffff8801fd1cbc58 ffffffff810ef249 0000000000000003
00000000d3b71a0d
[239049.571936] 00000000d3c53db8 ffff8807024cb628 ffff8802d5e5bd98
ffff8801fd1cbca8
[239049.580141] ffffffff8182d3c1 ffffffff810c35f2 0000000100000001
0000000000000000
[239049.588300] Call Trace:
[239049.590976] [<ffffffff810ef249>] hrtimer_try_to_cancel+0x29/0x130
[239049.597384] [<ffffffff8182d3c1>] schedule_hrtimeout_range_clock+0xd1/0x1b0
[239049.604571] [<ffffffff810c35f2>] ? __wake_up_common+0x52/0x90
[239049.610619] [<ffffffff810c3669>] __wake_up+0x39/0x50
[239049.615906] [<ffffffffc02ce324>]
btrfs_remove_ordered_extent+0x154/0x250 [btrfs]
[239049.623620] [<ffffffffc02bc040>]
btrfs_finish_ordered_io+0x1d0/0x650 [btrfs]
[239049.630993] [<ffffffffc02bc785>] finish_ordered_fn+0x15/0x20 [btrfs]
[239049.637666] [<ffffffffc02e4eba>]
btrfs_scrubparity_helper+0xca/0x2f0 [btrfs]
[239049.645042] [<ffffffffc02e516e>] btrfs_endio_write_helper+0xe/0x10 [btrfs]
[239049.652239] [<ffffffff8109a2c5>] process_one_work+0x165/0x480
[239049.658293] [<ffffffff8109a62b>] worker_thread+0x4b/0x4c0
[239049.663984] [<ffffffff8109a5e0>] ? process_one_work+0x480/0x480
[239049.670196] [<ffffffff8109a5e0>] ? process_one_work+0x480/0x480
[239049.676420] [<ffffffff810a0808>] kthread+0xd8/0xf0
[239049.681514] [<ffffffff810a0730>] ? kthread_create_on_node+0x1e0/0x1e0
[239049.688260] [<ffffffff8182e34f>] ret_from_fork+0x3f/0x70
[239049.693870] [<ffffffff810a0730>] ? kthread_create_on_node+0x1e0/0x1e0
[239049.700599] Code: 00 00 0f 1f 44 00 00 55 48 c7 47 28 10 f0 0e 81
48 89 77 58 48 89 e5 5d c3 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00
55 48 89 e5 <48> 8b 57 30 eb 1d 80 7f 38 00 75 32 48 3b 78 08 74 2c 39
50 04
[239049.727797] RIP [<ffffffff810eecf9>] hrtimer_active+0x9/0x60
[239049.733868] RSP <ffff8801fd1cbc20>
[239049.737557] CR2: 00000000d3c53de8
[239049.741535] ---[ end trace 774da4af66731bb5 ]---
Thanks,
Mark Gavalda
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2016-09-15 14:08 Mark Gavalda
@ 2016-09-15 16:05 ` Chris Mason
2016-09-15 20:12 ` Mark Gavalda
0 siblings, 1 reply; 31+ messages in thread
From: Chris Mason @ 2016-09-15 16:05 UTC (permalink / raw)
To: Mark Gavalda, linux-btrfs
On 09/15/2016 10:08 AM, Mark Gavalda wrote:
> Hi,
>
> Bumped into the following one today; kernel 4.4.0-36-generic Ubuntu
> 16.4.1; CPU went to 100% and only a hard restart solved the issue.
> Since then everything's back to normal.
>
> Please let me know how can I help get to the bottom of this?
I saw similar traces when tracking down this bug:
https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/commit/?h=for-linus-4.8&id=cbd60aa7cd17d81a434234268c55192862147439
It's flagged for stable, so you'll get it with the next stable update,
or you can apply it by hand and rebuild.
-chris
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2016-09-15 16:05 ` Chris Mason
@ 2016-09-15 20:12 ` Mark Gavalda
2016-09-16 2:45 ` Duncan
0 siblings, 1 reply; 31+ messages in thread
From: Mark Gavalda @ 2016-09-15 20:12 UTC (permalink / raw)
To: linux-btrfs
Thanks, I can see it included in 4.8-rc6 but not the other branches.
Will it get pulled later or is this a 4.8 only fix?
Mark
On Thu, Sep 15, 2016 at 6:05 PM, Chris Mason <clm@fb.com> wrote:
> On 09/15/2016 10:08 AM, Mark Gavalda wrote:
>>
>> Hi,
>>
>> Bumped into the following one today; kernel 4.4.0-36-generic Ubuntu
>> 16.4.1; CPU went to 100% and only a hard restart solved the issue.
>> Since then everything's back to normal.
>>
>> Please let me know how can I help get to the bottom of this?
>
>
> I saw similar traces when tracking down this bug:
>
> https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/commit/?h=for-linus-4.8&id=cbd60aa7cd17d81a434234268c55192862147439
>
> It's flagged for stable, so you'll get it with the next stable update, or
> you can apply it by hand and rebuild.
>
> -chris
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2016-09-15 20:12 ` Mark Gavalda
@ 2016-09-16 2:45 ` Duncan
0 siblings, 0 replies; 31+ messages in thread
From: Duncan @ 2016-09-16 2:45 UTC (permalink / raw)
To: linux-btrfs
Mark Gavalda posted on Thu, 15 Sep 2016 22:12:57 +0200 as excerpted:
[Moved to bottom to retain quote/reply order.]
> On Thu, Sep 15, 2016 at 6:05 PM, Chris Mason <clm@fb.com> wrote:
>> On 09/15/2016 10:08 AM, Mark Gavalda wrote:
>>>
>>> Hi,
>>>
>>> Bumped into the following one today; kernel 4.4.0-36-generic Ubuntu
>>> 16.4.1; CPU went to 100% and only a hard restart solved the issue.
>>> Since then everything's back to normal.
>>>
>>> Please let me know how can I help get to the bottom of this?
>>
>>
>> I saw similar traces when tracking down this bug:
>>
>> https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/
commit/?h=for-linus-4.8&id=cbd60aa7cd17d81a434234268c55192862147439
>>
>> It's flagged for stable, so you'll get it with the next stable update,
>> or you can apply it by hand and rebuild.
> Thanks, I can see it included in 4.8-rc6 but not the other branches.
> Will it get pulled later or is this a 4.8 only fix?
Flagged for stable means it's headed to the maintained stable branches
(well, the ones to which the fix applies for regression fixes, but
definitely 4.4 LTS series in this case since Chris Mason indicated it
should apply in your case), not just current.
But stabilization policy says a patch must hit mainline current first,
before it is eligible for older stable as well. So it would be
/expected/ to hit 4.8-rc, current development, first. After that, given
that it's already flagged for stable, it should eventually hit all the
stable kernels to which it applies as well. That can be right away, but
if the stable maintainer (Greg K-H, normally) is backlogged due to just
getting back from vacation or something, as sometimes happens, it can
take a few weeks to work thru the backlog, so it can take a bit, as well.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 31+ messages in thread
* unable to handle kernel paging request
@ 2019-11-15 15:26 Lange Norbert
2019-11-15 15:41 ` Jan Kiszka
0 siblings, 1 reply; 31+ messages in thread
From: Lange Norbert @ 2019-11-15 15:26 UTC (permalink / raw)
To: Xenomai (xenomai@xenomai.org)
Hello,
How can I get to the bottom of bugs that lockup the system completely?
I got that error now the 3rd time.
[ 1643.652566] BUG: unable to handle kernel paging request at 0000000000044540
[ 1643.659546] PGD 1750d1067 P4D 1750d1067 PUD 1775e7067 PMD 0
[ 1643.665237] Oops: 0010 [#1] SMP NOPTI
[ 1643.668911] CPU: 2 PID: 862 Comm: fup.medium Tainted: G W 4.19.75-xeno7-static #1
[ 1643.677703] Hardware name: TQ-Group TQMxE39M/Type2 - Board Product Name, BIOS 5.12.30.21.16 01/31/2019
[ 1643.687013] I-pipe domain: Linux
[ 1643.690250] RIP: 0086:0x1c000
[ 1643.693231] Code: Bad RIP value.
[ 1643.696468] RSP: 3bb00000:0000000000000083 EFLAGS: ffff92103bb25090 ORIG_RAX: 0000000000000046
[ 1643.705092] RAX: ffff92103bb30400 RBX: ffff92103bb30400 RCX: ffff92103bb31bb0
[ 1643.712235] RDX: ffff92103bb31d58 RSI: ffffffff8ab85272 RDI: ffff921039f24cc0
[ 1643.719378] RBP: ffff92103bb305c8 R08: 0000000000000002 R09: ffff92103bb31d58
[ 1643.726517] R10: ffff92103bb31bb0 R11: 0000000000004000 R12: ffffffff8bc3f3c0
[ 1643.733657] R13: ffffffff8ab8284b R14: 000000000001c000 R15: 0000000000000086
[ 1643.740799] FS: 00007fffe81da700 GS: 0000000000000000
[ 1643.746033] Modules linked in: rt_igb plusb usbnet mii
[ 1643.751210] CR2: 0000000000044540
[ 1643.754542] ---[ end trace 7dda9c557e28b024 ]---
[ 1643.759167] RIP: 0086:0x1c000
[ 1643.762146] Code: Bad RIP value.
[ 1643.765379] RSP: 3bb00000:0000000000000083 EFLAGS: ffff92103bb25090 ORIG_RAX: 0000000000000046
[ 1643.774003] RAX: ffff92103bb30400 RBX: ffff92103bb30400 RCX: ffff92103bb31bb0
[ 1643.781147] RDX: ffff92103bb31d58 RSI: ffffffff8ab85272 RDI: ffff921039f24cc0
[ 1643.788289] RBP: ffff92103bb305c8 R08: 0000000000000002 R09: ffff92103bb31d58
[ 1643.795427] R10: ffff92103bb31bb0 R11: 0000000000004000 R12: ffffffff8bc3f3c0
[ 1643.802569] R13: ffffffff8ab8284b R14: 000000000001c000 R15: 0000000000000086
[ 1643.809712] FS: 00007fffe81da700(0000) GS:ffff92103bb00000(0000) knlGS:0000000000000000
[ 1643.817806] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1643.823558] CR2: 000000000001bfd6 CR3: 00000001750e4000 CR4: 00000000003406e0
Mit besten Grüßen / Kind regards
NORBERT LANGE
AT-RD3
ANDRITZ HYDRO GmbH
Eibesbrunnergasse 20
1120 Vienna / AUSTRIA
p: +43 50805 56684
norbert.lange@andritz.com<mailto:norbert.lange@andritz.com>
andritz.com<http://www.andritz.com/>
________________________________
This message and any attachments are solely for the use of the intended recipients. They may contain privileged and/or confidential information or other information protected from disclosure. If you are not an intended recipient, you are hereby notified that you received this email in error and that any review, dissemination, distribution or copying of this email and any attachment is strictly prohibited. If you have received this email in error, please contact the sender and delete the message and any attachment from your system.
ANDRITZ HYDRO GmbH
Rechtsform/ Legal form: Gesellschaft mit beschränkter Haftung / Corporation
Firmensitz/ Registered seat: Wien
Firmenbuchgericht/ Court of registry: Handelsgericht Wien
Firmenbuchnummer/ Company registration: FN 61833 g
DVR: 0605077
UID-Nr.: ATU14756806
Thank You
________________________________
^ permalink raw reply [flat|nested] 31+ messages in thread
* Re: unable to handle kernel paging request
2019-11-15 15:26 Lange Norbert
@ 2019-11-15 15:41 ` Jan Kiszka
0 siblings, 0 replies; 31+ messages in thread
From: Jan Kiszka @ 2019-11-15 15:41 UTC (permalink / raw)
To: Lange Norbert, Xenomai (xenomai@xenomai.org)
On 15.11.19 16:26, Lange Norbert via Xenomai wrote:
> Hello,
>
> How can I get to the bottom of bugs that lockup the system completely?
> I got that error now the 3rd time.
>
> [ 1643.652566] BUG: unable to handle kernel paging request at 0000000000044540
> [ 1643.659546] PGD 1750d1067 P4D 1750d1067 PUD 1775e7067 PMD 0
> [ 1643.665237] Oops: 0010 [#1] SMP NOPTI
> [ 1643.668911] CPU: 2 PID: 862 Comm: fup.medium Tainted: G W 4.19.75-xeno7-static #1
> [ 1643.677703] Hardware name: TQ-Group TQMxE39M/Type2 - Board Product Name, BIOS 5.12.30.21.16 01/31/2019
> [ 1643.687013] I-pipe domain: Linux
> [ 1643.690250] RIP: 0086:0x1c000
> [ 1643.693231] Code: Bad RIP value.
> [ 1643.696468] RSP: 3bb00000:0000000000000083 EFLAGS: ffff92103bb25090 ORIG_RAX: 0000000000000046
> [ 1643.705092] RAX: ffff92103bb30400 RBX: ffff92103bb30400 RCX: ffff92103bb31bb0
> [ 1643.712235] RDX: ffff92103bb31d58 RSI: ffffffff8ab85272 RDI: ffff921039f24cc0
> [ 1643.719378] RBP: ffff92103bb305c8 R08: 0000000000000002 R09: ffff92103bb31d58
> [ 1643.726517] R10: ffff92103bb31bb0 R11: 0000000000004000 R12: ffffffff8bc3f3c0
> [ 1643.733657] R13: ffffffff8ab8284b R14: 000000000001c000 R15: 0000000000000086
> [ 1643.740799] FS: 00007fffe81da700 GS: 0000000000000000
> [ 1643.746033] Modules linked in: rt_igb plusb usbnet mii
> [ 1643.751210] CR2: 0000000000044540
> [ 1643.754542] ---[ end trace 7dda9c557e28b024 ]---
> [ 1643.759167] RIP: 0086:0x1c000
> [ 1643.762146] Code: Bad RIP value.
> [ 1643.765379] RSP: 3bb00000:0000000000000083 EFLAGS: ffff92103bb25090 ORIG_RAX: 0000000000000046
> [ 1643.774003] RAX: ffff92103bb30400 RBX: ffff92103bb30400 RCX: ffff92103bb31bb0
> [ 1643.781147] RDX: ffff92103bb31d58 RSI: ffffffff8ab85272 RDI: ffff921039f24cc0
> [ 1643.788289] RBP: ffff92103bb305c8 R08: 0000000000000002 R09: ffff92103bb31d58
> [ 1643.795427] R10: ffff92103bb31bb0 R11: 0000000000004000 R12: ffffffff8bc3f3c0
> [ 1643.802569] R13: ffffffff8ab8284b R14: 000000000001c000 R15: 0000000000000086
> [ 1643.809712] FS: 00007fffe81da700(0000) GS:ffff92103bb00000(0000) knlGS:0000000000000000
> [ 1643.817806] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 1643.823558] CR2: 000000000001bfd6 CR3: 00000001750e4000 CR4: 00000000003406e0
You need CONFIG_DEBUG_INFO and here likely CONFIG_IPIPE_TRACE_PANIC with
the tracer on when the issue occurs.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 31+ messages in thread
end of thread, other threads:[~2019-11-15 15:41 UTC | newest]
Thread overview: 31+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-10-31 18:42 unable to handle kernel paging request Dennis Grevenstein
2004-10-31 19:15 ` Jan-Benedict Glaw
[not found] ` <Pine.GSO.4.10.10410311947570.9753-100000@helios.et.put.poznan.pl>
2004-10-31 19:16 ` Dennis Grevenstein
2004-10-31 19:26 ` Jan-Benedict Glaw
2004-10-31 19:26 ` Jan-Benedict Glaw
2004-10-31 19:55 ` Dennis Grevenstein
2004-10-31 20:13 ` Jan-Benedict Glaw
2004-10-31 22:36 ` Dennis Grevenstein
2004-10-31 23:59 ` Maciej W. Rozycki
2004-11-01 21:50 ` Florian Lohoff
2004-10-31 22:51 ` Dennis Grevenstein
2004-10-31 23:13 ` Dennis Grevenstein
-- strict thread matches above, loose matches on Subject: below --
2019-11-15 15:26 Lange Norbert
2019-11-15 15:41 ` Jan Kiszka
2016-09-15 14:08 Mark Gavalda
2016-09-15 16:05 ` Chris Mason
2016-09-15 20:12 ` Mark Gavalda
2016-09-16 2:45 ` Duncan
2007-09-27 16:30 Unable " Laurent Vivier
[not found] ` <46FBDA8A.8030109-6ktuUTfB/bM@public.gmane.org>
2007-09-27 16:54 ` Laurent Vivier
[not found] ` <46FBE055.2040904-6ktuUTfB/bM@public.gmane.org>
2007-09-30 9:07 ` Avi Kivity
[not found] ` <46FF6764.5090502-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
2007-10-01 6:58 ` Laurent Vivier
2006-08-12 18:48 Ian Pratt
2006-08-13 0:08 ` Keir Fraser
2006-08-14 5:44 ` Steve Traugott
2006-08-05 7:38 blocking Xen 3.X production use: soft lockup bugs Ian Pratt
2006-08-12 1:50 ` Unable to handle kernel paging request Steve Traugott
2006-08-12 4:34 ` Steve Traugott
2006-08-12 5:50 ` Steve Traugott
2006-08-12 9:50 ` Keir Fraser
2005-01-12 18:43 Praedor Atrebates
2004-03-03 16:33 Paulo Marques
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.