Re: Why "No buffer space available"?

[Pablo Neira Ayuso <pablo@netfilter.org>]

Medialy wrote:
> Problem solved. Thanks.
> BTW, sometimes the program stops at function nfct_close() and never return!

I don't have an explanation for that, but it should not happen.

> On Wed, Dec 30, 2009 at 8:10 PM, Pablo Neira Ayuso <pablo@netfilter.org 
> <mailto:pablo@netfilter.org>> wrote:
> 
>     Medialy wrote:
> 
>         Hi,
>         I have written a program to log the nat behavior. the program works
>         well when traffic is low. But when the traffic reaches 1Gb, program
>         always error.
>         According to the previous discussions about this problem, I even set
>         the recv buffer size to 50MB and the error still exists.
> 
> 
>     Increasing the buffer size would not solve the problem, that will
>     only delay the ENOBUFS error. There are several reasons why you may
>     hit ENOBUFS:
> 
>     a) your program is too slow to handle the Netlink messages that you
>     receive from the kernel at a given rate. This is easier to trigger
>     if the handling that you perform on every message takes too long.
>     b) the queue size is too small, but this does not seem to be your case.
> 
>     ENOBUFS basically means that the kernel has to drop Netlink messages
>     because your user-space program cannot back-off.
> 
>  
> Medialy:
> Reason: system was overloaded due to the storage capability.  The 
> program (2 threads) was set to use last CPU.  When the traffic was 
> heavy,  most of the computing power of last CPU was occupied by the 
> thread which wrote Netlink messages to the stroage.

Good analysis. It is a good idea to put the thread that digest the 
Netlink message in a spare CPU. That reduces the chances to hit ENOBUFS.

I forgot to say but reducing the nice() value also help to avoid ENOBUFS.