UDP message boundary

UDP message boundary

[Randi Botse]

Hi All

When using TCP socket, I loop send() or recv() until ALL the data has
been transmitted (or error, disconnect, etc.), because TCP socket
packet is transmitted in stream nature, maybe a byte, bytes or all
bytes in one transfer.

The UDP socket preserve message boundary which TCP socket doesn't.
Does this means single call to sendto() will processed by single call
recvfrom()?, and how about packet that exceeds UDP data MAX size?.

So in code, do I need to loop sendto() or recvfrom() to transmit the data?.

Example codes is:

char packet[100];
size_t nbytes = 0;
int ret;

while (nbytes < sizeof(packet)) {
    ret = recvfrom(socket, packet + nbytes, addr, 0,  sizeof(packet) - nbytes);
    if (ret <= 0) {
        /* deal with recvfrom() error */
    }
    nbytes += ret
}


Thanks

Re: UDP message boundary

[Randi Botse]

Anyone can share? :)

Thanks


On Mon, Oct 15, 2012 at 11:50 AM, Randi Botse <nightdecoder@gmail.com> wrote:
> Hi All
>
> When using TCP socket, I loop send() or recv() until ALL the data has
> been transmitted (or error, disconnect, etc.), because TCP socket
> packet is transmitted in stream nature, maybe a byte, bytes or all
> bytes in one transfer.
>
> The UDP socket preserve message boundary which TCP socket doesn't.
> Does this means single call to sendto() will processed by single call
> recvfrom()?, and how about packet that exceeds UDP data MAX size?.
>
> So in code, do I need to loop sendto() or recvfrom() to transmit the data?.
>
> Example codes is:
>
> char packet[100];
> size_t nbytes = 0;
> int ret;
>
> while (nbytes < sizeof(packet)) {
>     ret = recvfrom(socket, packet + nbytes, addr, 0,  sizeof(packet) - nbytes);
>     if (ret <= 0) {
>         /* deal with recvfrom() error */
>     }
>     nbytes += ret
> }
>
>
> Thanks

Re: UDP message boundary

[Lorenzo Beretta]

On 17/10/2012 05:24, Randi Botse wrote:
> Anyone can share? :)
>
> Thanks
>
>
> On Mon, Oct 15, 2012 at 11:50 AM, Randi Botse<nightdecoder@gmail.com>  wrote:
>> Hi All
>>
>> When using TCP socket, I loop send() or recv() until ALL the data has
>> been transmitted (or error, disconnect, etc.), because TCP socket
>> packet is transmitted in stream nature, maybe a byte, bytes or all
>> bytes in one transfer.
>>
>> The UDP socket preserve message boundary which TCP socket doesn't.
>> Does this means single call to sendto() will processed by single call
>> recvfrom()?, and how about packet that exceeds UDP data MAX size?.
>>
>> So in code, do I need to loop sendto() or recvfrom() to transmit the data?.
>>
>> Example codes is:
>>
>> char packet[100];
>> size_t nbytes = 0;
>> int ret;
>>
>> while (nbytes<  sizeof(packet)) {
>>      ret = recvfrom(socket, packet + nbytes, addr, 0,  sizeof(packet) - nbytes);
>>      if (ret<= 0) {
>>          /* deal with recvfrom() error */
>>      }
>>      nbytes += ret
>> }
>>
>>
>> Thanks
> --
> To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
Almost:
1. udp is not reliable, so transmission may just fail
2. there's a maximum lenght of 64k (16 bit size counter)
3. ip fragmentation: as packet size grows, iirc one fragment lost means 
you loose everything

You can start from the wikipedia page and explore from that, or read the 
source of any program doing serious stuff with udp(*)


(*) there's an ftp alternative called fsp which uses udp, but I can't 
comment on the source code because I've never read it

Re: UDP message boundary

[Randi Botse]

Hi

> On Fri, Oct 19, 2012 at 9:15 PM, Lorenzo Beretta <lory.fulgi@infinito.it> wrote:
> Almost:
> 1. udp is not reliable, so transmission may just fail
> 2. there's a maximum lenght of 64k (16 bit size counter)
> 3. ip fragmentation: as packet size grows, iirc one fragment lost means you
> loose everything

Lorenzo: Yes, I already know what UDP socket is, I'm asking about UDP
message boundary behaviour.






> On 17/10/2012 05:24, Randi Botse wrote:
>>
>> Anyone can share? :)
>>
>> Thanks
>>
>>
>> On Mon, Oct 15, 2012 at 11:50 AM, Randi Botse<nightdecoder@gmail.com>
>> wrote:
>>>
>>> Hi All
>>>
>>> When using TCP socket, I loop send() or recv() until ALL the data has
>>> been transmitted (or error, disconnect, etc.), because TCP socket
>>> packet is transmitted in stream nature, maybe a byte, bytes or all
>>> bytes in one transfer.
>>>
>>> The UDP socket preserve message boundary which TCP socket doesn't.
>>> Does this means single call to sendto() will processed by single call
>>> recvfrom()?, and how about packet that exceeds UDP data MAX size?.
>>>
>>> So in code, do I need to loop sendto() or recvfrom() to transmit the
>>> data?.
>>>
>>> Example codes is:
>>>
>>> char packet[100];
>>> size_t nbytes = 0;
>>> int ret;
>>>
>>> while (nbytes<  sizeof(packet)) {
>>>      ret = recvfrom(socket, packet + nbytes, addr, 0,  sizeof(packet) -
>>> nbytes);
>>>      if (ret<= 0) {
>>>          /* deal with recvfrom() error */
>>>      }
>>>      nbytes += ret
>>> }
>>>
>>>
>>> Thanks
>>
>> --
>>
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-c-programming" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

>
> You can start from the wikipedia page and explore from that, or read the
> source of any program doing serious stuff with udp(*)
>
>
> (*) there's an ftp alternative called fsp which uses udp, but I can't
> comment on the source code because I've never read it
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe
> linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: UDP message boundary

[Hendrik Visage]

On Mon, Oct 15, 2012 at 6:50 AM, Randi Botse <nightdecoder@gmail.com> wrote:
> Hi All
>
> When using TCP socket, I loop send() or recv() until ALL the data has
> been transmitted (or error, disconnect, etc.), because TCP socket
> packet is transmitted in stream nature, maybe a byte, bytes or all
> bytes in one transfer.

TCP is like a water tap (a character device) that you need to "format"
yourself with boundaries/etc.

> The UDP socket preserve message boundary which TCP socket doesn't.
> Does this means single call to sendto() will processed by single call
> recvfrom()?, and how about packet that exceeds UDP data MAX size?.

UDP is more like a block device, a letter/package sent. It's only the
single packet that is sent, and only that same single packet that will
be received (or go missing in the post)

I believe the error you might be refering to is  this one from a Linux
2.6 based Ubuntu 6.0.5 man page for recvfrom:

<quote man recv(2)>
       All  three  routines  return the length of the message on
successful completion.  If a message is too long to fit in the supâ
       plied buffer, excess bytes may be discarded depending on the
type of socket the message is received from.
<snip>
(recv(2) flags orred together:)
       MSG_TRUNC (since Linux 2.2)
              For  raw  (AF_PACKET),  Internet datagram (since Linux
2.4.27/2.6.8), and netlink (since Linux 2.6.22) sockets: return
              the real length of the packet or datagram, even when it
was longer than the passed buffer.  Not implemented  for  Unix
              domain (unix(7)) sockets.

<snip>
(recvmsg(2) flags return status:)
       MSG_TRUNC
              indicates that the trailing portion of a datagram was
discarded because the datagram was larger than the  buffer  supâ
              plied.
</quote>

In other words, you have been sent a 1024byte long packet with
sentto(2), but recvfrom(2) only had a 900 bytes buffer, then sorry,
you've lost 124 bytes.
Looking at the recv(2) manual page, I recall that recv(2)/recvfrom(2)
will return the size of the packter received, so if you've provided a
65535 byte buffer, and was only sent 1024bytes, then
recv(2)/recvfrom(2) will return the 1024bytes answer.

> So in code, do I need to loop sendto() or recvfrom() to transmit the data?.

It depends on the data being sent, but for every sendto(2), you will
need a single recv(2)/recvfrom(2) with a correctly sized buffer to
receive that single message.

> Example codes is:
>
> char packet[100];
> size_t nbytes = 0;
> int ret;
>
> while (nbytes < sizeof(packet)) {
>     ret = recvfrom(socket, packet + nbytes, addr, 0,  sizeof(packet) - nbytes);
>     if (ret <= 0) {
>         /* deal with recvfrom() error */
>     }
>     nbytes += ret
> }

That is TCP, for UDP you will have something like:

sender:

char buffer[1024];
int size_of_data=fill_buffer_withdata(buffer,1024);
sendto(socket,buffer,size_of_data,flags);


receiver:
chat buffer [1024];
int size_of_data=recv(socket,buffer,1024,flags);
if size_of_data>1024
then
 throw_error
else
 do_something_with_data(buffer,size_of_data);
end

there are another method, using MSG_PEEK in the flags before reading
the real data and remove that from the queue,

>
>
> Thanks
> --
> To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: UDP message boundary

[Randi Botse]

This is very clear and complete answer.

Thanks You!

Regards.

On Sun, Oct 21, 2012 at 4:55 AM, Hendrik Visage <hvjunk@gmail.com> wrote:
> On Mon, Oct 15, 2012 at 6:50 AM, Randi Botse <nightdecoder@gmail.com> wrote:
>> Hi All
>>
>> When using TCP socket, I loop send() or recv() until ALL the data has
>> been transmitted (or error, disconnect, etc.), because TCP socket
>> packet is transmitted in stream nature, maybe a byte, bytes or all
>> bytes in one transfer.
>
> TCP is like a water tap (a character device) that you need to "format"
> yourself with boundaries/etc.
>
>> The UDP socket preserve message boundary which TCP socket doesn't.
>> Does this means single call to sendto() will processed by single call
>> recvfrom()?, and how about packet that exceeds UDP data MAX size?.
>
> UDP is more like a block device, a letter/package sent. It's only the
> single packet that is sent, and only that same single packet that will
> be received (or go missing in the post)
>
> I believe the error you might be refering to is  this one from a Linux
> 2.6 based Ubuntu 6.0.5 man page for recvfrom:
>
> <quote man recv(2)>
>        All  three  routines  return the length of the message on
> successful completion.  If a message is too long to fit in the supâ
>        plied buffer, excess bytes may be discarded depending on the
> type of socket the message is received from.
> <snip>
> (recv(2) flags orred together:)
>        MSG_TRUNC (since Linux 2.2)
>               For  raw  (AF_PACKET),  Internet datagram (since Linux
> 2.4.27/2.6.8), and netlink (since Linux 2.6.22) sockets: return
>               the real length of the packet or datagram, even when it
> was longer than the passed buffer.  Not implemented  for  Unix
>               domain (unix(7)) sockets.
>
> <snip>
> (recvmsg(2) flags return status:)
>        MSG_TRUNC
>               indicates that the trailing portion of a datagram was
> discarded because the datagram was larger than the  buffer  supâ
>               plied.
> </quote>
>
> In other words, you have been sent a 1024byte long packet with
> sentto(2), but recvfrom(2) only had a 900 bytes buffer, then sorry,
> you've lost 124 bytes.
> Looking at the recv(2) manual page, I recall that recv(2)/recvfrom(2)
> will return the size of the packter received, so if you've provided a
> 65535 byte buffer, and was only sent 1024bytes, then
> recv(2)/recvfrom(2) will return the 1024bytes answer.
>
>> So in code, do I need to loop sendto() or recvfrom() to transmit the data?.
>
> It depends on the data being sent, but for every sendto(2), you will
> need a single recv(2)/recvfrom(2) with a correctly sized buffer to
> receive that single message.
>
>> Example codes is:
>>
>> char packet[100];
>> size_t nbytes = 0;
>> int ret;
>>
>> while (nbytes < sizeof(packet)) {
>>     ret = recvfrom(socket, packet + nbytes, addr, 0,  sizeof(packet) - nbytes);
>>     if (ret <= 0) {
>>         /* deal with recvfrom() error */
>>     }
>>     nbytes += ret
>> }
>
> That is TCP, for UDP you will have something like:
>
> sender:
>
> char buffer[1024];
> int size_of_data=fill_buffer_withdata(buffer,1024);
> sendto(socket,buffer,size_of_data,flags);
>
>
> receiver:
> chat buffer [1024];
> int size_of_data=recv(socket,buffer,1024,flags);
> if size_of_data>1024
> then
>  throw_error
> else
>  do_something_with_data(buffer,size_of_data);
> end
>
> there are another method, using MSG_PEEK in the flags before reading
> the real data and remove that from the queue,
>
>>
>>
>> Thanks
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html