2.6.29.1 pktgen crash

2.6.29.1 pktgen crash

[Vladimir Ivashchenko]

[-- Attachment #1: Type: text/plain, Size: 3868 bytes --]

The attached script leads to a crash when you run it for 2-3 times on 2.6.29.1. The server still remains reachable, but starts misbehaving.

With 2.6.23.1, so far I didn't get any crashes.

May 20 21:09:03 tshape-phome2 kernel: BUG: unable to handle kernel paging request at f8bbc008
May 20 21:09:03 tshape-phome2 kernel: IP: [<f863ab01>] pktgen_remove_device+0x88/0xdc [pktgen]
May 20 21:09:03 tshape-phome2 kernel: *pde = 31030067 *pte = 00000000 
May 20 21:09:03 tshape-phome2 kernel: Oops: 0000 [#1] SMP 
May 20 21:09:03 tshape-phome2 kernel: last sysfs file: /sys/devices/platform/i5k_amb.0/name
May 20 21:09:03 tshape-phome2 kernel: Modules linked in: pktgen act_police cls_u32 sch_ingress autofs4 ip_vs rfcomm l2cap bluetooth bonding nf_conntrack_ipv6 xt_state nf_conntrack xt_tcpudp ip6t_ipv6header ip6t_REJECT ip6table_filter ip6_tables x_tables ipv6 dm_multipath joydev i5k_amb hwmon bnx2 sg e1000e ipmi_si i5000_edac edac_core sr_mod ehci_hcd ipmi_msghandler pcspkr serio_raw uhci_hcd iTCO_wdt iTCO_vendor_support cdrom hpwdt dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ext3 mbcache jbd ata_piix libata cciss sd_mod scsi_mod [last unloaded: microcode]
May 20 21:09:03 tshape-phome2 kernel: 
May 20 21:09:03 tshape-phome2 kernel: Pid: 495, comm: kpktgend_0 Not tainted (2.6.29.1-1hzd.fc8 #1) ProLiant DL360 G5
May 20 21:09:03 tshape-phome2 kernel: EIP: 0060:[<f863ab01>] EFLAGS: 00010286 CPU: 0
May 20 21:09:03 tshape-phome2 kernel: EIP is at pktgen_remove_device+0x88/0xdc [pktgen]
May 20 21:09:03 tshape-phome2 kernel: EAX: f8bbc000 EBX: f4e85000 ECX: f4121500 EDX: 00000000
May 20 21:09:03 tshape-phome2 kernel: ESI: 00100000 EDI: 00010000 EBP: f3011f10 ESP: f3011f04
May 20 21:09:03 tshape-phome2 kernel:  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
May 20 21:09:03 tshape-phome2 kernel: Process kpktgend_0 (pid: 495, ti=f3011000 task=f570e200 task.ti=f3011000)
May 20 21:09:03 tshape-phome2 kernel: Stack:
May 20 21:09:03 tshape-phome2 kernel:  ef9c9000 f4e85000 ef9c9004 f3011f24 f863ab8e ef9c9000 ef9c9000 ef9c9000
May 20 21:09:03 tshape-phome2 kernel:  f3011fd0 f863cba3 ef9c9000 e36f8a15 00046a5b f7eaa74a f41cad80 c065a4f6
May 20 21:09:03 tshape-phome2 kernel:  f3011f60 c065b365 00000286 c040272a 00000000 f416ef40 f4146e40 f3011f78
May 20 21:09:03 tshape-phome2 kernel: Call Trace:
May 20 21:09:03 tshape-phome2 kernel:  [<f863ab8e>] ? pktgen_rem_all_ifs+0x39/0x51 [pktgen]
May 20 21:09:03 tshape-phome2 kernel:  [<f863cba3>] ? pktgen_thread_worker+0x111e/0x1266 [pktgen]
May 20 21:09:03 tshape-phome2 kernel:  [<f7eaa74a>] ? bnx2_start_xmit+0x0/0x3c5 [bnx2]
May 20 21:09:03 tshape-phome2 kernel:  [<c065a4f6>] ? kretprobe_table_unlock+0x12/0x14
May 20 21:09:03 tshape-phome2 kernel:  [<c065b365>] ? kprobe_flush_task+0x58/0x5f
May 20 21:09:03 tshape-phome2 kernel:  [<c040272a>] ? __switch_to+0xde/0x168
May 20 21:09:03 tshape-phome2 kernel:  [<c0426c0f>] ? finish_task_switch+0x93/0xab
May 20 21:09:03 tshape-phome2 kernel:  [<c043d205>] ? autoremove_wake_function+0x0/0x33
May 20 21:09:03 tshape-phome2 kernel:  [<f863ba85>] ? pktgen_thread_worker+0x0/0x1266 [pktgen]
May 20 21:09:03 tshape-phome2 kernel:  [<c043d131>] ? kthread+0x3b/0x62
May 20 21:09:03 tshape-phome2 kernel:  [<c043d0f6>] ? kthread+0x0/0x62
May 20 21:09:03 tshape-phome2 kernel:  [<c0404193>] ? kernel_thread_helper+0x7/0x10
May 20 21:09:03 tshape-phome2 kernel: Code: 8b 03 85 c0 74 0e 8b 40 08 8b 15 7c 1a 64 f8 e8 fb f3 e8 c7 31 ff 31 f6 83 bb 44 02 00 00 00 75 36 eb 3c 8b 83 40 02 00 00 01 f0 <8b> 50 08 85 d2 74 21 f0 ff 4a 18 0f 94 c0 84 c0 74 07 89 d0 e8 
May 20 21:09:03 tshape-phome2 kernel: EIP: [<f863ab01>] pktgen_remove_device+0x88/0xdc [pktgen] SS:ESP 0068:f3011f04
May 20 21:09:03 tshape-phome2 kernel: ---[ end trace 2ad593a542263595 ]---

-- 
Best Regards
Vladimir Ivashchenko
Chief Technology Officer
PrimeTel, Cyprus - www.prime-tel.com

[-- Attachment #2: pktgen.sh --]
[-- Type: application/x-sh, Size: 1565 bytes --]

Re: 2.6.29.1 pktgen crash

[Florian Westphal]

Vladimir Ivashchenko <hazard@francoudi.com> wrote:
> The attached script leads to a crash when you run it for 2-3 times on 2.6.29.1. The server still remains reachable, but starts misbehaving.
>
> May 20 21:09:03 tshape-phome2 kernel: BUG: unable to handle kernel paging request at f8bbc008
> May 20 21:09:03 tshape-phome2 kernel: IP: [<f863ab01>] pktgen_remove_device+0x88/0xdc [pktgen]
> May 20 21:09:03 tshape-phome2 kernel: *pde = 31030067 *pte = 00000000 
> May 20 21:09:03 tshape-phome2 kernel: Oops: 0000 [#1] SMP 
[..]

This should fix it:

>From bca277bb59b865a4e9516953dfc9818af1a7deea Mon Sep 17 00:00:00 2001
From: Florian Westphal <fw@strlen.de>
Date: Thu, 21 May 2009 23:52:17 +0200
Subject: [PATCH] pktgen: do not access flows[] beyond its length

typo -- pkt_dev->nflows is for stats only, the number of concurrent
flows is stored in cflows.

Reported-By: Vladimir Ivashchenko <hazard@francoudi.com>
Signed-off-by: Florian Westphal <fw@strlen.de>
---
 net/core/pktgen.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index 3779c14..8445075 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -2447,7 +2447,7 @@ static inline void free_SAs(struct pktgen_dev *pkt_dev)
 	if (pkt_dev->cflows) {
 		/* let go of the SAs if we have them */
 		int i = 0;
-		for (;  i < pkt_dev->nflows; i++){
+		for (;  i < pkt_dev->cflows; i++) {
 			struct xfrm_state *x = pkt_dev->flows[i].x;
 			if (x) {
 				xfrm_state_put(x);
-- 
1.6.0.6

Re: 2.6.29.1 pktgen crash

[David Miller]

From: Florian Westphal <fw@strlen.de>
Date: Thu, 21 May 2009 23:59:15 +0200

> pktgen: do not access flows[] beyond its length
> 
> typo -- pkt_dev->nflows is for stats only, the number of concurrent
> flows is stored in cflows.
> 
> Reported-By: Vladimir Ivashchenko <hazard@francoudi.com>
> Signed-off-by: Florian Westphal <fw@strlen.de>

Applied and queued up for -stable, thanks.