recvfrom a large packet

recvfrom a large packet

[devendra.aaru]

if i do a recvfrom (sk, buf, 10000, 0, &addr, &len), shall i recv all the data
i mean the 10000 bytes?

since fragmentation happen in the ip layer and assembled happen in the
ip layer it doesnt matter for the upper layer about the packet size.


i wrote a test code and it seems to be working.

is there any problem will come if i turn on firewall.

any ideas?

recvfrom a large packet

[Victor Buciuc]

On Tue, Nov 6, 2012 at 1:35 PM, devendra.aaru <devendra.aaru@gmail.com> wrote:
> if i do a recvfrom (sk, buf, 10000, 0, &addr, &len), shall i recv all the data
> i mean the 10000 bytes?
>
> since fragmentation happen in the ip layer and assembled happen in the
> ip layer it doesnt matter for the upper layer about the packet size.
>
>
> i wrote a test code and it seems to be working.
>
> is there any problem will come if i turn on firewall.
>
> any ideas?
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

   I think a signal can interrupt recvfrom. If you already had some
data copied in the buffer then it will return something. You should
always check what you get in the result returned by recvfrom. If
you're not satisfied with what you got you can always call again. (I
assumed it's UDP we're talking about).

   I'm not an expert, it's my two cents.

Victor.

recvfrom a large packet

[Bernd Petrovitsch]

On Die, 2012-11-06 at 16:08 +0200, Victor Buciuc wrote:
> On Tue, Nov 6, 2012 at 1:35 PM, devendra.aaru <devendra.aaru@gmail.com> wrote:
> > if i do a recvfrom (sk, buf, 10000, 0, &addr, &len), shall i recv all the data
> > i mean the 10000 bytes?

Not necessarily. Consider the case that the other side sends only 9000
bytes.

> > since fragmentation happen in the ip layer and assembled happen in the
> > ip layer it doesnt matter for the upper layer about the packet size.
[....]
> > i wrote a test code and it seems to be working.

You tried one case and then you think it is in all cases the same?
You are very optimistic .....

> > is there any problem will come if i turn on firewall.
[...]

Try it. "Turn on firewall" is also not very precise ....

>    I think a signal can interrupt recvfrom. If you already had some
> data copied in the buffer then it will return something. You should

No, if a signal interrupts the recvfrom(), recvfrom() returns -1 (and
errno == EINTR).
Thus it cannot report any partially received packet and so there is
nothing written to the buffer.

> always check what you get in the result returned by recvfrom. If
> you're not satisfied with what you got you can always call again. (I
> assumed it's UDP we're talking about).

That is actually the only robust way:
1) append the read data to a buffer
2) check if enough is there to handle a packet. If not goto 1)
3) handle the first packet and delete it.
4) goto 2)

Kind regards,
	Bernd
-- 
Bernd Petrovitsch                  Email : bernd at petrovitsch.priv.at
                     LUGA : http://www.luga.at

recvfrom a large packet

[devendra.aaru]

On Tue, Nov 6, 2012 at 5:20 PM, Bernd Petrovitsch
<bernd@petrovitsch.priv.at> wrote:
> On Die, 2012-11-06 at 16:08 +0200, Victor Buciuc wrote:
>> On Tue, Nov 6, 2012 at 1:35 PM, devendra.aaru <devendra.aaru@gmail.com> wrote:
>> > if i do a recvfrom (sk, buf, 10000, 0, &addr, &len), shall i recv all the data
>> > i mean the 10000 bytes?
>
> Not necessarily. Consider the case that the other side sends only 9000
> bytes.
>
I tested with 1000 -- 10000 byte ranges in addition of 1000 bytes or 100 bytes.

i am sure that i recieved the bytes i sent,
i put a known bytes at the starting and ending of the buffer i send
from the other side,
i checked these at the receive end.

i used MSG_TRUNC to get the correct number of bytes that i recieved.

>> > since fragmentation happen in the ip layer and assembled happen in the
>> > ip layer it doesnt matter for the upper layer about the packet size.
> [....]
>> > i wrote a test code and it seems to be working.
>
> You tried one case and then you think it is in all cases the same?
> You are very optimistic .....
>

:-)

>> > is there any problem will come if i turn on firewall.
> [...]
>
> Try it. "Turn on firewall" is also not very precise ....
>

i turned on firewall and its working, but when i do a multicast it
seems to be not recieving
any packets, seems that firewall is dropping the fragments. Figuring
out how to dig this out.


>>    I think a signal can interrupt recvfrom. If you already had some
>> data copied in the buffer then it will return something. You should
>
> No, if a signal interrupts the recvfrom(), recvfrom() returns -1 (and
> errno == EINTR).
> Thus it cannot report any partially received packet and so there is
> nothing written to the buffer.
>

basically i check for EINTR, when it occur i dont need to do anything
for the buffer, just returning out.

>> always check what you get in the result returned by recvfrom. If
>> you're not satisfied with what you got you can always call again. (I
>> assumed it's UDP we're talking about).
>
> That is actually the only robust way:
> 1) append the read data to a buffer
> 2) check if enough is there to handle a packet. If not goto 1)
> 3) handle the first packet and delete it.
> 4) goto 2)
>

you mean to say use the MSG_PEEK flag and move past the buffer you read?

> Kind regards,
>         Bernd
> --
> Bernd Petrovitsch                  Email : bernd at petrovitsch.priv.at
>                      LUGA : http://www.luga.at
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

recvfrom a large packet

[Bernd Petrovitsch]

On Mit, 2012-11-07 at 00:09 -0500, devendra.aaru wrote:
> On Tue, Nov 6, 2012 at 5:20 PM, Bernd Petrovitsch
> <bernd@petrovitsch.priv.at> wrote:
> > On Die, 2012-11-06 at 16:08 +0200, Victor Buciuc wrote:
> >> On Tue, Nov 6, 2012 at 1:35 PM, devendra.aaru <devendra.aaru@gmail.com> wrote:
> >> > if i do a recvfrom (sk, buf, 10000, 0, &addr, &len), shall i recv all the data
> >> > i mean the 10000 bytes?
> >
> > Not necessarily. Consider the case that the other side sends only 9000
> > bytes.
> >
> I tested with 1000 -- 10000 byte ranges in addition of 1000 bytes or 100 bytes.

Ah, you meant you get all the bytes you sent (and not you always want to
get 10000 bytes and return then).

In a LAN, it probably works all the time that you get all the data at
once (and you usually have no packet drops in LANs too).
Over router, firewalls and/or the Internet as such, it may be different
as routers (similar to all other IP hosts) may drop packets and/or
reorder them so it may take some time for a 10k packet to come through.

[...]
> i used MSG_TRUNC to get the correct number of bytes that i recieved.

AFAIK that shouldn't make a difference (in that aspect).

[...]
> >>    I think a signal can interrupt recvfrom. If you already had some
> >> data copied in the buffer then it will return something. You should
> >
> > No, if a signal interrupts the recvfrom(), recvfrom() returns -1 (and
> > errno == EINTR).
> > Thus it cannot report any partially received packet and so there is
> > nothing written to the buffer.
> 
> basically i check for EINTR, when it occur i dont need to do anything
> for the buffer, just returning out.

Yes, the buffer shouldn't change - at least no (sane) user-space program
will expect anything useful in the buffer.

> >> always check what you get in the result returned by recvfrom. If
> >> you're not satisfied with what you got you can always call again. (I
> >> assumed it's UDP we're talking about).
> >
> > That is actually the only robust way:
> > 1) append the read data to a buffer
> > 2) check if enough is there to handle a packet. If not goto 1)
> > 3) handle the first packet and delete it.
> > 4) goto 2)
> 
> you mean to say use the MSG_PEEK flag and move past the buffer you read?

Hmm, I don't understand.
The 4 steps above describe what a userspace application should do
IMHO:-).

I'm not a fan of MSG_PEEK and the like. It leads people to "look what
will come and if we are happy with it, actually get it".

The problem with this approach is, that it implicitly assumes that no
one else is interfering with the filedescriptor/socket/entity.

That may hold for hello-world class programs, school (and university)
exercises, demo application, proof-of-concept prototypes and other small
write-once-and-never-touch-again software.

In real life with large applications, changing developer teams and
maintenance over years, people tend to stumble over all kinds of
implicit assumptions (which are violated over time) - and though someone
writes today in a specification and/or design document (and every
README;-) "we will never use threads and close all fds > 2 immediately
after a fork()", it may be well different in 6 months and the
documentation is changed/updated/fixed and usually no one really checks
the code afterwards.
Add the common copy-and-paste-and-change-a-few-lines-and-never-touch-
old-code programming technique for the next feature and it gets even
worse in the long run.

	Bernd
-- 
Bernd Petrovitsch                  Email : bernd at petrovitsch.priv.at
                     LUGA : http://www.luga.at