min-write-size for a UDP socket to be POLLOUT cannot be set. (proposed fixes)

min-write-size for a UDP socket to be POLLOUT cannot be set. (proposed fixes)

[Ben Greear]

This relates to my earlier question about setting the threshold
at which select returns that a (UDP) socket is writable.

It appears that UDP sockets are hardwired at 2048 bytes...

 From linux/include/net/sock.h:

....
#define SOCK_MIN_SNDBUF 2048
#define SOCK_MIN_RCVBUF 256
/* Must be less or equal SOCK_MIN_SNDBUF */
#define SOCK_MIN_WRITE_SPACE	SOCK_MIN_SNDBUF

/*
  *	Default write policy as shown to user space via poll/select/SIGIO
  *	Kernel internally doesn't use the MIN_WRITE_SPACE threshold.
  */
static inline int sock_writeable(struct sock *sk)
{
	return sock_wspace(sk) >= SOCK_MIN_WRITE_SPACE;
}


It appears that this issue could be fixed in several ways.  One would be to
make SOCK_MIN_WRITE_SPACE a global variable and set it through the proc
file system (or an IOCTL, etc...).  That is not very flexible, but
easy.  It sounds more likely to break existing programs though..

A more flexible way would be to set the value on a per sock basis (in the sock structure),
which is probably the right way, but may require a new IOCTL defined
in user-space...  This way should be completely backwards compatible, as the
default value can be 2048 as it is now.

Fixing this little problem will be very useful for anyone using larger UDP packet
sizes and poll/select.  Remember that GigE supports >>2048 MTU, so using larger UDP
packets becomes a more useful thing, even in the real world :)


As a side note, it appears that TCP is much smarter about this, taking the
current load into account:  If there is very little in the current write
queue, then it will say the socket is writable at a very small threshold.
However, if the write queue is almost full, tcp will make select/poll wait
longer.  This type of thing could also work for UDP, and the user can indirectly
determine the behaviour by setting the write-queue size appropriately large...

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: min-write-size for a UDP socket to be POLLOUT cannot be set. (proposed fixes)

[Richard B. Johnson]

On Mon, 10 Dec 2001, Ben Greear wrote:

> This relates to my earlier question about setting the threshold
> at which select returns that a (UDP) socket is writable.
> 
> It appears that UDP sockets are hardwired at 2048 bytes...
> 
>  From linux/include/net/sock.h:

    int len = 0x8000;

    setsockopt(s, SOL_SOCKET, SO_SNDBUF, &len, sizeof(len));
    setsockopt(s, SOL_SOCKET, SO_RCVBUF, &len, sizeof(len));


Doesn't this work?

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: min-write-size for a UDP socket to be POLLOUT cannot be set. (proposed fixes)

[Ben Greear]

Richard B. Johnson wrote:

> On Mon, 10 Dec 2001, Ben Greear wrote:
> 
> 
>>This relates to my earlier question about setting the threshold
>>at which select returns that a (UDP) socket is writable.
>>
>>It appears that UDP sockets are hardwired at 2048 bytes...
>>
>> From linux/include/net/sock.h:
>>
> 
>     int len = 0x8000;
> 
>     setsockopt(s, SOL_SOCKET, SO_SNDBUF, &len, sizeof(len));
>     setsockopt(s, SOL_SOCKET, SO_RCVBUF, &len, sizeof(len));
> 
> 
> Doesn't this work?


That sets the queue sizes bigger, but suppose this:

I have 4M queue size.  I have 4M-2k bytes already in the
queue (2k free).  I have a 4k UDP buffer to write.  I call
select and it says the socket is writable.  However, in this
case I cannot actually write to the socket because I have only
2 of the 4k that I need...  Now, I can detect the failure to send
and re-transmit, but that basically gets me into a tight loop because
select keeps saying I can write, and I keep trying.  The tight loop
is doubly bad because the machine is already highly stressed or it's
buffers would never be so full....

I want select to only say I can write when I'm at XX (say, 64k) bytes of
free buffer-queue space...

Ben


> 
> Cheers,
> Dick Johnson
> 
> Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).
> 
>     I was going to compile a list of innovations that could be
>     attributed to Microsoft. Once I realized that Ctrl-Alt-Del
>     was handled in the BIOS, I found that there aren't any.
> 
> 


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: min-write-size for a UDP socket to be POLLOUT cannot be set. (proposed fixes)

[Richard B. Johnson]

On Mon, 10 Dec 2001, Ben Greear wrote:

> 
> 
> Richard B. Johnson wrote:
> 
> > On Mon, 10 Dec 2001, Ben Greear wrote:
> > 
> > 
> >>This relates to my earlier question about setting the threshold
> >>at which select returns that a (UDP) socket is writable.
> >>
> >>It appears that UDP sockets are hardwired at 2048 bytes...
> >>
> >> From linux/include/net/sock.h:
> >>
> > 
> >     int len = 0x8000;
> > 
> >     setsockopt(s, SOL_SOCKET, SO_SNDBUF, &len, sizeof(len));
> >     setsockopt(s, SOL_SOCKET, SO_RCVBUF, &len, sizeof(len));
> > 
> > 
> > Doesn't this work?
> 
> 
> That sets the queue sizes bigger, but suppose this:
> 
> I have 4M queue size.  I have 4M-2k bytes already in the
> queue (2k free).  I have a 4k UDP buffer to write.  I call
> select and it says the socket is writable.  However, in this
> case I cannot actually write to the socket because I have only
> 2 of the 4k that I need...  Now, I can detect the failure to send
> and re-transmit, but that basically gets me into a tight loop because
> select keeps saying I can write, and I keep trying.  The tight loop
> is doubly bad because the machine is already highly stressed or it's
> buffers would never be so full....
> 
> I want select to only say I can write when I'm at XX (say, 64k) bytes of
> free buffer-queue space...
> 
> Ben
> 

If you have a 4M queue size, it appears as though you are trying to
use UDP where TCP should have been used. Normally, what you call the
queue size, is set to contain you largest packet you will ever want to
send. With this in mind, you don't even know if a fragmented packet
can be routed if it's more than 64k in length so you would never try
to send something larger than that under UDP.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: min-write-size for a UDP socket to be POLLOUT cannot be set. (proposed fixes)

[Ben Greear]

Richard B. Johnson wrote:

> On Mon, 10 Dec 2001, Ben Greear wrote:

>>I have 4M queue size.  I have 4M-2k bytes already in the
>>queue (2k free).  I have a 4k UDP buffer to write.  I call
>>select and it says the socket is writable.  However, in this
>>case I cannot actually write to the socket because I have only
>>2 of the 4k that I need...  Now, I can detect the failure to send
>>and re-transmit, but that basically gets me into a tight loop because
>>select keeps saying I can write, and I keep trying.  The tight loop
>>is doubly bad because the machine is already highly stressed or it's
>>buffers would never be so full....
>>
>>I want select to only say I can write when I'm at XX (say, 64k) bytes of
>>free buffer-queue space...
>>
>>Ben
>>
>>
> 
> If you have a 4M queue size, it appears as though you are trying to
> use UDP where TCP should have been used. Normally, what you call the
> queue size, is set to contain you largest packet you will ever want to
> send. With this in mind, you don't even know if a fragmented packet
> can be routed if it's more than 64k in length so you would never try
> to send something larger than that under UDP.


Assume that I actually want to do what I said I did! :)

I never try to send
a packet bigger than 64k, the protocol doesn't handle it.  But, I may try
to send 100000 60k UDP packets in very fast succession...which could fill up my
send queue.



-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear