Re: kernel 2.6.7 -> page allocation failure. order:1, mode:0x20 (netfilter?)

Re: kernel 2.6.7 -> page allocation failure. order:1, mode:0x20 (netfilter?)

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 2189 bytes --]

On Mon, Jul 19, 2004 at 11:27:26AM +0200, Sebastien ESTIENNE wrote:
> Hello,
> i have some "swapper: page allocation failure. order:1, mode:0x20"
> followed by kernel message,
> 
> the hardware is a dell poweredge 1750
> the kernel is a 2.6.7 vanilla
> 
> here for a dmesg
> http://213.41.75.25/kernel/dmesg.txt
the chunk below seems like a standard code path for a locally-generated
outgoing packet:

> [<c028ac94>] skb_checksum_help+0x52/0x136
> [<e094fb79>] ip_nat_fn+0x269/0x27a [iptable_nat]
> [<e094fcb3>] ip_nat_local_fn+0x7b/0xaa [iptable_nat]
> [<c028708e>] tcp_sendmsg+0x509/0x10f7
> [<c0121872>] tasklet_action+0x65/0xae
> [<c01065da>] apic_timer_interrupt+0x1a/0x20
> [<c02ad977>] dst_output+0x0/0x29
> [<c028708e>] inet_sendmsg+0x4d/0x59
> [<c026498f>] dst_output+0x0/0x29
> [<c0276a40>] sock_aio_write+0xbd/0xdd
> [<c015902c>] do_sync_write+0x8b/0xb7
> [<c01da1ac>] nf_iterate+0x71/0xa2
> [<c028708e>] copy_from_user+0x42/0x6e
> [<c0159140>] vfs_write+0xe8/0x119
> [<c0159216>] sys_write+0x42/0x63
> [<c0105beb>] syscall_call+0x7/0xb

 
what's worrying me is the part above... how would skb_checksum_help
directly end up in copy_from_user()?

> swapper: page allocation failure. order:1, mode:0x20
> [<c013d770>] __alloc_pages+0x2da/0x34a
> [<c013d805>] __get_free_pages+0x25/0x3f
> [<c0140e28>] kmem_getpages+0x2b/0xdc
> [<c0141bfc>] kfree_skbmem+0x24/0x2c
> [<c0141c5d>] cache_grow+0xe5/0x2a4
> [<c0141f8a>] cache_grow+0x146/0x2a4
> [<c0295917>] cache_alloc_refill+0x1cf/0x29f
> [<c014262a>] __kmalloc+0x85/0x8c
> [<c02681f1>] tcp_transmit_skb+0x411/0x68a
> [<c0296621>] alloc_skb+0x47/0xe0
> [<c026875e>] tcp_write_xmit+0x16d/0x2d6
> [<c01da1ac>] skb_copy+0x33/0xde
> [<c026ca5b>] copy_from_user+0x42/0x6e

Does anybody have a clue what's going on?

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: kernel 2.6.7 -> page allocation failure. order:1, mode:0x20 (netfilter?)

[Sebastien ESTIENNE]

Le mar 20/07/2004 à 04:47, Harald Welte a écrit :
> On Mon, Jul 19, 2004 at 11:27:26AM +0200, Sebastien ESTIENNE wrote:
> > Hello,
> > i have some "swapper: page allocation failure. order:1, mode:0x20"
> > followed by kernel message,
> > 
> > the hardware is a dell poweredge 1750
> > the kernel is a 2.6.7 vanilla
> > 
> > here for a dmesg
> > http://213.41.75.25/kernel/dmesg.txt
> the chunk below seems like a standard code path for a locally-generated
> outgoing packet:
> 
> > [<c028ac94>] skb_checksum_help+0x52/0x136
> > [<e094fb79>] ip_nat_fn+0x269/0x27a [iptable_nat]
> > [<e094fcb3>] ip_nat_local_fn+0x7b/0xaa [iptable_nat]
> > [<c028708e>] tcp_sendmsg+0x509/0x10f7
> > [<c0121872>] tasklet_action+0x65/0xae
> > [<c01065da>] apic_timer_interrupt+0x1a/0x20
> > [<c02ad977>] dst_output+0x0/0x29
> > [<c028708e>] inet_sendmsg+0x4d/0x59
> > [<c026498f>] dst_output+0x0/0x29
> > [<c0276a40>] sock_aio_write+0xbd/0xdd
> > [<c015902c>] do_sync_write+0x8b/0xb7
> > [<c01da1ac>] nf_iterate+0x71/0xa2
> > [<c028708e>] copy_from_user+0x42/0x6e
> > [<c0159140>] vfs_write+0xe8/0x119
> > [<c0159216>] sys_write+0x42/0x63
> > [<c0105beb>] syscall_call+0x7/0xb
> 
>  
> what's worrying me is the part above... how would skb_checksum_help
> directly end up in copy_from_user()?
> 
> > swapper: page allocation failure. order:1, mode:0x20
> > [<c013d770>] __alloc_pages+0x2da/0x34a
> > [<c013d805>] __get_free_pages+0x25/0x3f
> > [<c0140e28>] kmem_getpages+0x2b/0xdc
> > [<c0141bfc>] kfree_skbmem+0x24/0x2c
> > [<c0141c5d>] cache_grow+0xe5/0x2a4
> > [<c0141f8a>] cache_grow+0x146/0x2a4
> > [<c0295917>] cache_alloc_refill+0x1cf/0x29f
> > [<c014262a>] __kmalloc+0x85/0x8c
> > [<c02681f1>] tcp_transmit_skb+0x411/0x68a
> > [<c0296621>] alloc_skb+0x47/0xe0
> > [<c026875e>] tcp_write_xmit+0x16d/0x2d6
> > [<c01da1ac>] skb_copy+0x33/0xde
> > [<c026ca5b>] copy_from_user+0x42/0x6e
> 
> Does anybody have a clue what's going on?
I was running apache bench v2 on localhost
And Shorewall firewall script is installed on it (version 2.0.4)
iptables version 2.0.4
hope it can help

Re: kernel 2.6.7 -> page allocation failure. order:1, mode:0x20 (netfilter?)

[Andrew Morton]

Harald Welte <laforge@netfilter.org> wrote:
>
> what's worrying me is the part above... how would skb_checksum_help
>  directly end up in copy_from_user()?

That'll be leftover gunk on the stack.  Enabling frame pointers makes that
go away.

>  > swapper: page allocation failure. order:1, mode:0x20
>  > [<c013d770>] __alloc_pages+0x2da/0x34a
>  > [<c013d805>] __get_free_pages+0x25/0x3f
>  > [<c0140e28>] kmem_getpages+0x2b/0xdc
>  > [<c0141bfc>] kfree_skbmem+0x24/0x2c
>  > [<c0141c5d>] cache_grow+0xe5/0x2a4
>  > [<c0141f8a>] cache_grow+0x146/0x2a4
>  > [<c0295917>] cache_alloc_refill+0x1cf/0x29f
>  > [<c014262a>] __kmalloc+0x85/0x8c
>  > [<c02681f1>] tcp_transmit_skb+0x411/0x68a
>  > [<c0296621>] alloc_skb+0x47/0xe0
>  > [<c026875e>] tcp_write_xmit+0x16d/0x2d6
>  > [<c01da1ac>] skb_copy+0x33/0xde
>  > [<c026ca5b>] copy_from_user+0x42/0x6e
> 
>  Does anybody have a clue what's going on?

Networking tried to do an atomic 1-order allocation and there were no
1-order pages available in the free page pools.  It's pretty much
unavoidable, and the caller simply needs to handle it - presumably by
dropping the packet.

Its frequency can be reduced by increasing /proc/sys/vm/min_free_kbytes.

It can be eliminated by using GFP_KERNEL.  It can be hugely reduced by
sticking to 0-order allocations.

I wouldn't worry about this unless someone is seeing a lot of them (a
significant number of packets are getting dropped)