queue buffer size increasing howto?

queue buffer size increasing howto?

[Banszki Gabor]

 
Hi everybody!

Greetings for you! I am a new member of this list. 

Please somebody help me!

I am developing an application for counting and processing packets in a
linux box with using the ip_queue module. 

It is working properly, but sometimes I have a problem, that I get often
this message:

Failed to receive netlink message, No buffer space available

Maybee the cause of this, that the box has a lot of short hardly tasks
sometimes. 
I think the solution to prevent the packet loss If I would increase the
queue length. I did it with this command:

echo 2048 > /proc/sys/net/ipv4/ip_queue_maxlen

#cat /proc/net/ip_queue
Peer PID          : 23985
Copy mode         : 2
Copy range        : 2048
Queue length      : 184
Queue max. length : 2048

The enviroment:
 2.4.26 kernel and  iptables: 1.2.9 with promiscuous patch 1.2.3


With this command: watch -n 0 "cat /proc/net/ip_queue" during my
application running the biggest number of "Queue length" is 255.

I don't understand it, because the "Queue max. length" is 2048.

How can I increase the queue length?

What is the really method of the queue increasing?

Thanx:

Gabor

Re: queue buffer size increasing howto?

[Tobias DiPasquale]

On Sun, 22 Aug 2004 15:40:16 +0200, Banszki Gabor
<banszki.gabor@chello.hu> wrote:
>
> Hi everybody!
>
> Greetings for you! I am a new member of this list.
>
> Please somebody help me!
>
> I am developing an application for counting and processing packets in a
> linux box with using the ip_queue module.
>
> It is working properly, but sometimes I have a problem, that I get often
> this message:
>
> Failed to receive netlink message, No buffer space available

This message indicates the the buffer for the netlink socket was full.
This is different from the queue that ip_queue has internally for
packets to be sent to userspace. You will need to open the netlink
socket with a larger buffer, which can be accomplished by using
setsockopt(2):

int fd, rv, size = LARGE_SOCKET_BUFFER;

fd = socket( PF_NETLINK, SOCK_RAW, NETLINK_FIREWALL);
rv = setsockopt( fd, SOL_SOCKET, SO_RCVBUF, &size, sizeof( size));
...

> With this command: watch -n 0 "cat /proc/net/ip_queue" during my
> application running the biggest number of "Queue length" is 255.
>
> I don't understand it, because the "Queue max. length" is 2048.

See above. You should be able to increase the size of the socket
receive buffer in your userspace process without changing the length
of ip_queue's internal packet queue, but test that out to make sure.
You will probably need to run the process with root privileges to
change your socket receive buffer size, however.

--
[ Tobias DiPasquale ]
0x636f6465736c696e67657240676d61696c2e636f6d

Re: queue buffer size increasing howto?

[Banszki Gabor]

On Wed, 2004-09-08 at 01:39, Tobias DiPasquale wrote:
> On Tue, 07 Sep 2004 18:30:26 +0200, Banszki Gabor
> <banszki.gabor@chello.hu> wrote:
> > Hi everybody!
> > 
> > I solved the problem by using pthread.h
> > 
> > The main thread in my process is reading from the queue buffer and
open
> > new threads for the long time tasks.
> > 
> > In this case I don't need to increase the buffer size of the netlink
> > socket.
> 
> You will still lose some data under high enough load. This is exactly
> what I did, as well, and I still needed to pump up the size of the
> socket recv buffer.

Ok....... Bot how can I increase the  buffer size by more than 215040?




>Hi Everybody!
>
>Thank for answer of Tobias!!
>
>I could get the buffer size by the getsockopt (107520).
>
>After this I was able to increase the size of buffer to 215040 by your
>suggestion:
>
>   int size, size_len;
>   size_len = sizeof(size);
>   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
>   printf ("Oldsize of buffer: %i \n",size); { I've got 107520)
>
>   size = 1256000;
>   rv = setsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, size_len); 
>
>   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
>   printf ("New size of buffer: %i \n",size); ( I've got 215040 )
>
>After this I can see the same situation with this command....
>
> watch -n 0 "cat /proc/net/ip_queue"
>
>...like earlier... 
>The maximum number is around 252 - 258, and I still get the netlink
>error message "No buffer space....."
>
>I think it didn't help... Why?
>Maybee I should increase the SO_RCVBUF by more? But how?
>
>Thnx..

Re: queue buffer size increasing howto?

[Banszki Gabor]

Hi Everybody!

Thank for answer of Tobias!!

I could get the buffer size by the getsockopt (107520).

After this I was able to increase the size of buffer to 215040 by your
suggestion:

   int size, size_len;
   size_len = sizeof(size);
   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
   printf ("Oldsize of buffer: %i \n",size); { I've got 107520)

   size = 1256000;
   rv = setsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, size_len); 

   rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
   printf ("New size of buffer: %i \n",size); ( I've got 215040 )

After this I can see the same situation with this command....

 watch -n 0 "cat /proc/net/ip_queue"

...like earlier... 
The maximum number is around 252 - 258, and I still get the netlink
error message "No buffer space....."

I think it didn't help... Why?
Maybee I should increase the SO_RCVBUF by more? But how?

Thnx..

G.

On Sun, 2004-08-22 at 16:50, Tobias DiPasquale wrote:
> On Sun, 22 Aug 2004 15:40:16 +0200, Banszki Gabor
> setsockopt(2):
> 
> int fd, rv, size = LARGE_SOCKET_BUFFER;
> 
> fd = socket( PF_NETLINK, SOCK_RAW, NETLINK_FIREWALL);
> rv = setsockopt( fd, SOL_SOCKET, SO_RCVBUF, &size, sizeof( size));
> ...
> 
> > With this command: watch -n 0 "cat /proc/net/ip_queue" during my
> > application running the biggest number of "Queue length" is 255.
> > 
> > I don't understand it, because the "Queue max. length" is 2048.
> 
> See above. You should be able to increase the size of the socket
> receive buffer in your userspace process without changing the length
> of ip_queue's internal packet queue, but test that out to make sure.
> You will probably need to run the process with root privileges to
> change your socket receive buffer size, however.

Re: queue buffer size increasing howto? - solved

[Banszki Gabor]

Hi everybody!

I solved the problem by using pthread.h

The main thread in my process is reading from the queue buffer and open
new threads for the long time tasks.

In this case I don't need to increase the buffer size of the netlink
socket.

G.




On Sat, 2004-09-04 at 10:16, Banszki Gabor wrote:
> Hi Everybody!
> 
> Thank for answer of Tobias!!
> 
> I could get the buffer size by the getsockopt (107520).
> 
> After this I was able to increase the size of buffer to 215040 by your
> suggestion:
> 
>    int size, size_len;
>    size_len = sizeof(size);
>    rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
>    printf ("Oldsize of buffer: %i \n",size); { I've got 107520)
> 
>    size = 1256000;
>    rv = setsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, size_len); 
> 
>    rv = getsockopt (fd, SOL_SOCKET, SO_RCVBUF, &size, &size_len); 
>    printf ("New size of buffer: %i \n",size); ( I've got 215040 )
> 
> After this I can see the same situation with this command....
> 
>  watch -n 0 "cat /proc/net/ip_queue"
> 
> ...like earlier... 
> The maximum number is around 252 - 258, and I still get the netlink
> error message "No buffer space....."
> 
> I think it didn't help... Why?
> Maybee I should increase the SO_RCVBUF by more? But how?
> 
> Thnx..
> other tasks 
> G.
> 
> On Sun, 2004-08-22 at 16:50, Tobias DiPasquale wrote:
> > On Sun, 22 Aug 2004 15:40:16 +0200, Banszki Gabor
> > setsockopt(2):
> > 
> > int fd, rv, size = LARGE_SOCKET_BUFFER;
> > 
> > fd = socket( PF_NETLINK, SOCK_RAW, NETLINK_FIREWALL);
> > rv = setsockopt( fd, SOL_SOCKET, SO_RCVBUF, &size, sizeof( size));
> > ...
> > 
> > > With this command: watch -n 0 "cat /proc/net/ip_queue" during my
> > > application running the biggest number of "Queue length" is 255.
> > > 
> > > I don't understand it, because the "Queue max. length" is 2048.
> > 
> > See above. You should be able to increase the size of the socket
> > receive buffer in your userspace process without changing the length
> > of ip_queue's internal packet queue, but test that out to make sure.
> > You will probably need to run the process with root privileges to
> > change your socket receive buffer size, however.
> 

Re: queue buffer size increasing howto? - solved

[Tobias DiPasquale]

On Tue, 07 Sep 2004 18:30:26 +0200, Banszki Gabor
<banszki.gabor@chello.hu> wrote:
> Hi everybody!
> 
> I solved the problem by using pthread.h
> 
> The main thread in my process is reading from the queue buffer and open
> new threads for the long time tasks.
> 
> In this case I don't need to increase the buffer size of the netlink
> socket.

You will still lose some data under high enough load. This is exactly
what I did, as well, and I still needed to pump up the size of the
socket recv buffer.

-- 
[ Tobias DiPasquale ]
0x636f6465736c696e67657240676d61696c2e636f6d