2.4.14: crashing on heavy swap-load with SmartArray

2.4.14: crashing on heavy swap-load with SmartArray

[Ronny Lampert (EED)]

Hello,

Total crash (Aieeee: Killing interrupt handlers...) on an compaq 6400R
Dual PIII Xeon 500, 3GB Ram + 1G Swap, custom kernel (SMP,
smartarray/eepro100 into kernel).

syslog says:
Nov  9 13:05:54 eedn36ls kernel: Invalid request on ida/c0d0 = (cmd=30
sect=5490
904 cnt=112 sg=12 ret=10) 

console:
Invalid request ...(same as above)
invalid operand: 0000
CPU: 1
EIP: 0010: [<c0123b3e>] EFLAGS: 00010046
eax: 0 ebx: edd98500 ecx: 1 edx: c1a80dc0
esi: 2 edi: c1a80dc0 ebp: 1 esp: c4039efc
ds: 0018 es: 0018 ss: 0018

(...)

Process: swapper (PID: 0, stackpage=c4039000) ...
....
Code: 0f 0b 8d 5a 24 8d 42 28 39 42 28 74 11 b9 01 00 00 00 ba 03

On the second crash, there was a slight change:
No reporting about an invalid request on ida/c0d0 but:

CPU: 0
same EIP, EFLAGS

Process: mtest01 (PID: 683, stackpage=f7439000)
....

same Code: ...

It is reproduceable on 2.4.14 (first compile using 2.91.66 gcc, second
2.95.3 gcc) using mtest01 from the Linux Testing Project
(ltp-20010801/ltctests/mem/mtest01).
I also applied the newest microcodeupdate from Intel, stays the same.
2.2.19 is doing fine (with bigmempatch, of course).

I used params: ./mtest01 -w -c $[25*1024*1024] -p 60 and started 3
processes. This will allocate 60% of total system ram on each process in
25MB chunks and will write to it after allocating.

If more infos are required, I will try me best to write down the
kernel-panic :)
Please include me on CC.

Kind regards,
Ronny

Re: 2.4.14: crashing on heavy swap-load with SmartArray

[Linux Kernel Developer]

[-- Attachment #1: Type: text/plain, Size: 2217 bytes --]

The cpqarray driver was updated in 2.4.14.  And the update appears to be
broken.  Your probably encountering the same error I did.  I attached a
relevent message that includes a patch.

----- Original Message -----
From: "Ronny Lampert (EED)" <Ronny.Lampert@eed.ericsson.se>
To: <linux-kernel@vger.kernel.org>
Sent: Friday, November 09, 2001 7:20 AM
Subject: 2.4.14: crashing on heavy swap-load with SmartArray


> Hello,
>
> Total crash (Aieeee: Killing interrupt handlers...) on an compaq 6400R
> Dual PIII Xeon 500, 3GB Ram + 1G Swap, custom kernel (SMP,
> smartarray/eepro100 into kernel).
>
> syslog says:
> Nov  9 13:05:54 eedn36ls kernel: Invalid request on ida/c0d0 = (cmd=30
> sect=5490
> 904 cnt=112 sg=12 ret=10)
>
> console:
> Invalid request ...(same as above)
> invalid operand: 0000
> CPU: 1
> EIP: 0010: [<c0123b3e>] EFLAGS: 00010046
> eax: 0 ebx: edd98500 ecx: 1 edx: c1a80dc0
> esi: 2 edi: c1a80dc0 ebp: 1 esp: c4039efc
> ds: 0018 es: 0018 ss: 0018
>
> (...)
>
> Process: swapper (PID: 0, stackpage=c4039000) ...
> ....
> Code: 0f 0b 8d 5a 24 8d 42 28 39 42 28 74 11 b9 01 00 00 00 ba 03
>
> On the second crash, there was a slight change:
> No reporting about an invalid request on ida/c0d0 but:
>
> CPU: 0
> same EIP, EFLAGS
>
> Process: mtest01 (PID: 683, stackpage=f7439000)
> ....
>
> same Code: ...
>
> It is reproduceable on 2.4.14 (first compile using 2.91.66 gcc, second
> 2.95.3 gcc) using mtest01 from the Linux Testing Project
> (ltp-20010801/ltctests/mem/mtest01).
> I also applied the newest microcodeupdate from Intel, stays the same.
> 2.2.19 is doing fine (with bigmempatch, of course).
>
> I used params: ./mtest01 -w -c $[25*1024*1024] -p 60 and started 3
> processes. This will allocate 60% of total system ram on each process in
> 25MB chunks and will write to it after allocating.
>
> If more infos are required, I will try me best to write down the
> kernel-panic :)
> Please include me on CC.
>
> Kind regards,
> Ronny
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

[-- Attachment #2: Re_ CPQARRAY driver horribly broken in 2_4_14.eml --]
[-- Type: message/rfc822, Size: 3365 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 712 bytes --]

Linux Kernel Developer wrote:

> Hi all,
>
>      I'm using the cpqarray driver for a Compaq Smart Arrat 3100ES
> controller on a Compaq Proliant 7000.  Today I tried upgrading the kernel to
> 2.4.14.  Soon after the upgrade I though about making a small change in the
> kernel however as soon as I tried doing a "make dep" the system oopsed and
> froze.

Been there, done that, bought the t-shirt.

The attached patch courtesy of Jens Axboe
fixed my Compaq 6500 which was giving me
fits - basically in 2.4.14 it had a nasty habit of
scribbling on the disk and then locking up,
requiring a power cycle, manual fsck and
file restoration to get it running again.

With this patch 2.4.14 has been solid.

cu

jjs



[-- Attachment #2.1.2: cciss-dequeue-1 --]
[-- Type: text/plain, Size: 615 bytes --]

--- linux/drivers/block/cciss.c~	Thu Nov  8 11:36:24 2001
+++ linux/drivers/block/cciss.c	Thu Nov  8 11:37:03 2001
@@ -1307,6 +1307,8 @@
 	if (( c = cmd_alloc(h, 1)) == NULL)
 		goto startio;

+	blkdev_dequeue_request(creq);
+
 	spin_unlock_irq(&io_request_lock);

 	c->cmd_type = CMD_RWREQ;
@@ -1386,12 +1388,6 @@

 	spin_lock_irq(&io_request_lock);

-	blkdev_dequeue_request(creq);
-
-        /*
-         * ehh, we can't really end the request here since it's not
-         * even started yet. for now it shouldn't hurt though
-         */
 	addQ(&(h->reqQ),c);
 	h->Qdepth++;
 	if(h->Qdepth > h->maxQsinceinit)

Re: 2.4.14: crashing on heavy swap-load with SmartArray

[Linux Kernel Developer]

    Ok my original response to you may have been wrong.  I think that patch
may not have fixed my problem with the cpqarray driver though I could be
wrong.  If the patch doesn't work for you try downgrading the cpqarray
driver.  Copy the cpqarray.[ch], ida_cmd.h, and ida_ioctl.h files from the
drivers/block directory from an earlier kernel source tree over the ones in
your 2.4.14 kernel source tree.  This fixed the problem I had.

----- Original Message -----
From: "Ronny Lampert (EED)" <Ronny.Lampert@eed.ericsson.se>
To: <linux-kernel@vger.kernel.org>
Sent: Friday, November 09, 2001 7:20 AM
Subject: 2.4.14: crashing on heavy swap-load with SmartArray


> Hello,
>
> Total crash (Aieeee: Killing interrupt handlers...) on an compaq 6400R
> Dual PIII Xeon 500, 3GB Ram + 1G Swap, custom kernel (SMP,
> smartarray/eepro100 into kernel).
>
> syslog says:
> Nov  9 13:05:54 eedn36ls kernel: Invalid request on ida/c0d0 = (cmd=30
> sect=5490
> 904 cnt=112 sg=12 ret=10)
>
> console:
> Invalid request ...(same as above)
> invalid operand: 0000
> CPU: 1
> EIP: 0010: [<c0123b3e>] EFLAGS: 00010046
> eax: 0 ebx: edd98500 ecx: 1 edx: c1a80dc0
> esi: 2 edi: c1a80dc0 ebp: 1 esp: c4039efc
> ds: 0018 es: 0018 ss: 0018
>
> (...)
>
> Process: swapper (PID: 0, stackpage=c4039000) ...
> ....
> Code: 0f 0b 8d 5a 24 8d 42 28 39 42 28 74 11 b9 01 00 00 00 ba 03
>
> On the second crash, there was a slight change:
> No reporting about an invalid request on ida/c0d0 but:
>
> CPU: 0
> same EIP, EFLAGS
>
> Process: mtest01 (PID: 683, stackpage=f7439000)
> ....
>
> same Code: ...
>
> It is reproduceable on 2.4.14 (first compile using 2.91.66 gcc, second
> 2.95.3 gcc) using mtest01 from the Linux Testing Project
> (ltp-20010801/ltctests/mem/mtest01).
> I also applied the newest microcodeupdate from Intel, stays the same.
> 2.2.19 is doing fine (with bigmempatch, of course).
>
> I used params: ./mtest01 -w -c $[25*1024*1024] -p 60 and started 3
> processes. This will allocate 60% of total system ram on each process in
> 25MB chunks and will write to it after allocating.
>
> If more infos are required, I will try me best to write down the
> kernel-panic :)
> Please include me on CC.
>
> Kind regards,
> Ronny
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>