kernel panic when adding QUEUE to OUTGOING

kernel panic when adding QUEUE to OUTGOING

[François Delawarde]

Hello,

I got a kernel panic trying the following command on two different
computers with vanila kernel 2.6.30.9, and as soon as some traffic is
sent. Curriously I could not reproduce in QEMU.

There is no user space program handling the QUEUE (I did not have the
chance to get to that part).

# iptables -I OUTPUT -j QUEUE

Sorry I could only get a picture of the screen:
http://imagebin.org/70454


Regards,
François.

Re: kernel panic when adding QUEUE to OUTGOING

[Patrick McHardy]

Fran����������������������������������������������� wrote:
> Hello,
> 
> I got a kernel panic trying the following command on two different
> computers with vanila kernel 2.6.30.9, and as soon as some traffic is
> sent. Curriously I could not reproduce in QEMU.
> 
> There is no user space program handling the QUEUE (I did not have the
> chance to get to that part).
> 
> # iptables -I OUTPUT -j QUEUE
> 
> Sorry I could only get a picture of the screen:
> http://imagebin.org/70454

I can't reproduce this here. Unfortunately your screenshot only
includes the end of the backtrace, so its hard to tell what
exactly happened. It crashes in packet_rcv(), so it doesn't
look related to NFQUEUE. If you can reproduce this, please try
to capture a full backtrace.

Re: kernel panic when adding QUEUE to OUTGOING

[François Delawarde]

Tell me if this one helps:
http://imagebin.org/70463

Regards,
François.


On Wed, 2009-11-04 at 12:22 +0100, Patrick McHardy wrote:
> Fran����������������������������������������������� wrote:
> > Hello,
> > 
> > I got a kernel panic trying the following command on two different
> > computers with vanila kernel 2.6.30.9, and as soon as some traffic is
> > sent. Curriously I could not reproduce in QEMU.
> > 
> > There is no user space program handling the QUEUE (I did not have the
> > chance to get to that part).
> > 
> > # iptables -I OUTPUT -j QUEUE
> > 
> > Sorry I could only get a picture of the screen:
> > http://imagebin.org/70454
> 
> I can't reproduce this here. Unfortunately your screenshot only
> includes the end of the backtrace, so its hard to tell what
> exactly happened. It crashes in packet_rcv(), so it doesn't
> look related to NFQUEUE. If you can reproduce this, please try
> to capture a full backtrace.

Re: kernel panic when adding QUEUE to OUTGOING

[Patrick McHardy]

Fran����������������������������������������������� wrote:
> Tell me if this one helps:
> http://imagebin.org/70463

Not sure yet. What does /proc/net/netfilter/nf_queue contain?

Re: kernel panic when adding QUEUE to OUTGOING

[François Delawarde]

On Wed, 2009-11-04 at 14:37 +0100, Patrick McHardy wrote:
> Fran����������������������������������������������� wrote:
> > Tell me if this one helps:
> > http://imagebin.org/70463
> 
> Not sure yet. What does /proc/net/netfilter/nf_queue contain?

# cat /proc/net/netfilter/nf_queue
 0 NONE
 1 NONE
 2 NONE
 3 NONE
 4 NONE
 5 NONE
 6 NONE
 7 NONE
 8 NONE
 9 NONE
10 NONE
11 NONE
12 NONE

Re: kernel panic when adding QUEUE to OUTGOING

[Patrick McHardy]

Fran����������������������������������������������� wrote:
> On Wed, 2009-11-04 at 14:37 +0100, Patrick McHardy wrote:
>> Fran����������������������������������������������� wrote:
>>> Tell me if this one helps:
>>> http://imagebin.org/70463
>> Not sure yet. What does /proc/net/netfilter/nf_queue contain?
> 
> # cat /proc/net/netfilter/nf_queue
>  0 NONE
> ...

I can't find anything which could cause this. Please post full
information (all netfilter rules, ping version, qdiscs and whatever
else might seem important).

Also, your kernel is already tainted (D), please retry with
a cleanly booted kernel.

Re: kernel panic when adding QUEUE to OUTGOING

[François Delawarde]

On Wed, 2009-11-04 at 15:01 +0100, Patrick McHardy wrote:
> I can't find anything which could cause this. Please post full
> information (all netfilter rules, ping version, qdiscs and whatever
> else might seem important).
> 
> Also, your kernel is already tainted (D), please retry with
> a cleanly booted kernel.

I reproduce right after a reboot, and it's still tainted, how do I
remove that? The only strange thing at boot would be some vga= option in
lilo that fails (because of a wrong value) and asks me for a mode.

It might be worth noting that the kernel is patched with IMQ, latest
version from linux-imq.net, but the IMQ device is not used.

Versions:
- Everything Debian Lenny except for the kernel and iptables
- Linux 2.6.30.9 SMP-PREEMPT (also happened with older 2.6.30 versions)
- iptables v1.4.4 patched with IMQ
# ping -V
ping utility, iputils-sss20071127

Hardware:
- Reproduced on an HP-ML110 and some man-made gigabyte motherboards with
old dual-cores and realtek eths.
- Could not reproduce in QEMU/KVM with the exact same SW versions.


My script running right before the command that causes panic is:
# cleanup
iptables -t mangle -F
iptables -t nat -F
iptables -t filter -F
tc qdisc del dev eth0 root
tc qdisc del dev eth1 root
ip address flush dev eth0
ip address flush dev eth1
ip rule flush
ip route flush

# configure
ip rule add prio 32766 from all lookup main
ip rule add prio 32767 from all lookup default
ip address add dev eth0 192.168.10.218/24 brd +
ip route add default via 192.168.10.1

The exact commands I run are:
# iptables -I OUTPUT -j QUEUE
# ping google.com

Appart from a ping command, anything generating traffic will provoque
the panic like an ssh command or just waiting for some incoming
packets. 
It does not happen at the very first packet, as I have results when I do
the following from a remote machine (by ssh):
# iptables -I OUTPUT -j QUEUE; cat /proc/net/netfilter/nf_queue
 0 NONE
 1 NONE
 2 NONE
 3 NONE
 4 NONE
 5 NONE
 6 NONE
 7 NONE
 8 NONE
 9 NONE
10 NONE
11 NONE
12 NONE


Thanks,
François.

Re: kernel panic when adding QUEUE to OUTGOING

[François Delawarde]

On Wed, 2009-11-04 at 15:32 +0100, François Delawarde wrote:
> It might be worth noting that the kernel is patched with IMQ, latest
> version from linux-imq.net, but the IMQ device is not used.

I also have dahdi (digium) and wanpipe (sangoma) drivers loaded for
telephony cards. Could this be the cause of the "taint"

Re: kernel panic when adding QUEUE to OUTGOING

[Patrick McHardy]

Fran��������������������������������������������� wrote:
> On Wed, 2009-11-04 at 15:01 +0100, Patrick McHardy wrote:
>> I can't find anything which could cause this. Please post full
>> information (all netfilter rules, ping version, qdiscs and whatever
>> else might seem important).
>>
>> Also, your kernel is already tainted (D), please retry with
>> a cleanly booted kernel.
> 
> I reproduce right after a reboot, and it's still tainted, how do I
> remove that? The only strange thing at boot would be some vga= option in
> lilo that fails (because of a wrong value) and asks me for a mode.
> 
> It might be worth noting that the kernel is patched with IMQ, latest
> version from linux-imq.net, but the IMQ device is not used.

This is most likely the cause.

Re: kernel panic when adding QUEUE to OUTGOING

[François Delawarde]

On Wed, 2009-11-04 at 15:40 +0100, Patrick McHardy wrote:
> > It might be worth noting that the kernel is patched with IMQ, latest
> > version from linux-imq.net, but the IMQ device is not used.
> 
> This is most likely the cause.

Does it affect even if the devices are not used at all? I can still
reproduce doing an "rmmod imq" before anything.

Re: kernel panic when adding QUEUE to OUTGOING

[Javier Almillategui]

or putting imq in the module blacklist so it is not loaded by default.

On Nov 4, 2009, at 9:44 AM, François Delawarde wrote:

> On Wed, 2009-11-04 at 15:40 +0100, Patrick McHardy wrote:
>>> It might be worth noting that the kernel is patched with IMQ, latest
>>> version from linux-imq.net, but the IMQ device is not used.
>>
>> This is most likely the cause.
>
> Does it affect even if the devices are not used at all? I can still
> reproduce doing an "rmmod imq" before anything.
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: kernel panic when adding QUEUE to OUTGOING

[Patrick McHardy]

François Delawarde wrote:
> On Wed, 2009-11-04 at 15:40 +0100, Patrick McHardy wrote:
>>> It might be worth noting that the kernel is patched with IMQ, latest
>>> version from linux-imq.net, but the IMQ device is not used.
>> This is most likely the cause.
> 
> Does it affect even if the devices are not used at all? I can still
> reproduce doing an "rmmod imq" before anything.
> 

Probably, it adds weird code all over the place and from a quick look
it can't even compile cleanly.

Re: kernel panic when adding QUEUE to OUTGOING

[François Delawarde]

On Wed, 2009-11-04 at 15:54 +0100, Patrick McHardy wrote:
> > Does it affect even if the devices are not used at all? I can still
> > reproduce doing an "rmmod imq" before anything.
> > 
> 
> Probably, it adds weird code all over the place and from a quick look
> it can't even compile cleanly.

Ok, thanks for your answers. If I reproduce without IMQ I will let you
know. Otherwise I guess I'll have to wait for that xtable-addons
implementation of IMQ. :-)

François.

Re: kernel panic when adding QUEUE to OUTGOING

[Slava Garaschenko]

I have the same problem. Kernel 2.6.31 FC12, x86_64
Problem appear ONLY then first packet go into imq, which connected to 
outgoing. If I add IMQ to OUTGOOING to interface which don`t have any 
routing to it, and not connected (no packets sent to this interface) - 
all OK, but then first packet send to this interface - I have "kernel 
trap". If it wiil be help, I can try to make screenshot of screen then 
"kernel trap".

-- 
     Garashchenko Vyacheslav

Re: kernel panic when adding QUEUE to OUTGOING

[Slava Garaschenko]

I have the same problem. Kernel 2.6.31 FC12, x86_64
Problem appear ONLY then first packet go into imq, which connected to
outgoing. If I add IMQ to OUTGOOING to interface which don`t have any
routing to it, and not connected (no packets sent to this interface) -
all OK, but then first packet send to this interface - I have "kernel
trap". If it will be help, I can try to make screen-shot of screen then
"kernel trap".

-- 
     Garashchenko Vyacheslav