Re: want opinions on possible glitch in 2.4 network error reporti ng

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Perches, Joe]

>> I ran into a somewhat related issue on a 2.2.16 system, where I had an
app that 
>> was calling sendto() on 217000 packets/sec, even though the wire could
only 
>> handle about 127000 packets/sec. I got no errors at all in sendto, even
though 
>> over a third of the packets were not actually being sent. 

> That is correct UDP behaviour 

Do you think this is the correct PacketSocket/RAW behaviour?
How does one guarantee a send/sendto/write?

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Richard B. Johnson]

On Thu, 7 Feb 2002, Perches, Joe wrote:
[SNIPPED..]
> > That is correct UDP behaviour 
> 
> Do you think this is the correct PacketSocket/RAW behaviour?

Yes.

> How does one guarantee a send/sendto/write?
> -

Easy, you use send() or write(). These work on stream protocol TCP/IP
where there is a "connection". Connectionless protocols, i.e., UDP are
not guaranteed to do anything useful -- but, because of their speed,
they can be useful with some help from user-mode code.

Example file x-fer:
  Connection says: Here is a N-byte file, sent in numbered blocks over
         UDP port Y.
  UDP sender: sends all the file blocks.
  Receiver:  receives all the UDP data except Z numbered blocks.
  Receiver connection says: send me Z numbered blocks.
  UDP sender: sends the missing blocks
  Receiver: receives all UDP data except Z! numbered blocks.
  Receiver connection says: send me Z! numbered blocks.
  /// This repeats until there are no missing blocks.
  Receiver connection says: Thanks and hangs up.

  Now, since every UDP packet is not ACKed in the network layer,
  the through-put can be very high. Only the missing data needs
  to be replaced. Even with 10 percent missing data, such a transfer
  can be much faster than TCP/IP with high-latency links (like satellite).


Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Chris Friesen]

"Richard B. Johnson" wrote:
> 
> On Thu, 7 Feb 2002, Perches, Joe wrote:
> [SNIPPED..]
> > > That is correct UDP behaviour
> >
> > Do you think this is the correct PacketSocket/RAW behaviour?
> 
> Yes.
> 
> > How does one guarantee a send/sendto/write?
> > -
> 
> Easy, you use send() or write(). These work on stream protocol TCP/IP
> where there is a "connection". Connectionless protocols, i.e., UDP are
> not guaranteed to do anything useful -- but, because of their speed,
> they can be useful with some help from user-mode code.

Is there any syscall that can guarantee that a single packet has been sent out
over the wire?  Suppose I want to broadcast an ARP packet.  If I make a packet
socket and call sendto() on it, I want a guarantee that the packet will make it
out onto the wire, or the sendto() should fail.

UDP failing I can understand (kind of, anyway) but for raw sockets, packet
sockets, etc. I think there should be at least some kind of mechanism to bypass
all the congestion controls and either shove the packet onto the device's tx
buffer or return a failure code.

The possibility of random dropping of packets in the kernel means that an
infinite loop on sendto() will chew up the entire machine even if you've only
got a 10Mbit/s link.  This seems just wrong.

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Jan Harkes]

On Thu, Feb 07, 2002 at 01:44:53PM -0500, Chris Friesen wrote:
> The possibility of random dropping of packets in the kernel means that an
> infinite loop on sendto() will chew up the entire machine even if you've only
> got a 10Mbit/s link.  This seems just wrong.

What happens if you have the 100Mbit/s side of the link and the receiver
has a 9600baud dial-in modem....

The sending side needs to do throttling based on packet loss anyways, it
really doesn't matter that it's lost locally or on the network or at the
receiving host.

Jan

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Chris Friesen]

Jan Harkes wrote:
> 
> On Thu, Feb 07, 2002 at 01:44:53PM -0500, Chris Friesen wrote:
> > The possibility of random dropping of packets in the kernel means that an
> > infinite loop on sendto() will chew up the entire machine even if you've only
> > got a 10Mbit/s link.  This seems just wrong.
> 
> What happens if you have the 100Mbit/s side of the link and the receiver
> has a 9600baud dial-in modem....
> 
> The sending side needs to do throttling based on packet loss anyways, it
> really doesn't matter that it's lost locally or on the network or at the
> receiving host.

Yes, this is true for general use.  However, suppose I want to do an IP takeover
and send out arp packets to force an update of the arp caches of everyone on the
subnet.  The network is tightly controlled and we know everything that's on it. 
How do I guarantee that my packets get out of the local system and onto the
wire?

Currently I send three packets in a row on failover, and another three packets
every 10 seconds.  It would still be nice to be assured that the packets
actually made it onto the wire.

This is linux, we like having absolute control over our system.  Does it make
sense to have no possible way of guaranteeing that a specific packet has made it
onto the wire?

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Richard B. Johnson]

On Thu, 7 Feb 2002, Chris Friesen wrote:

> "Richard B. Johnson" wrote:
> > 
> > On Thu, 7 Feb 2002, Perches, Joe wrote:
> > [SNIPPED..]

> 
> Is there any syscall that can guarantee that a single packet has been sent out
> over the wire?  Suppose I want to broadcast an ARP packet.  If I make a packet
> socket and call sendto() on it, I want a guarantee that the packet will make it
> out onto the wire, or the sendto() should fail.

No. Look at how it works. You can guarantee that a packet gets into
the SNIC (Serial Network Interface Controller). If it is congested
or blocked, the hardware will retry for a number of times (typically 15).

However, after that, all bets are off. Note that it is possible for
a physical link to be disconnected at any time. Your SNIC may be connected
to a hub or switch so it "thinks" everything is fine. The message went
out over-the-wire. However, it just fell out the end of a fibre link
connecting your sub-nets and you will never know it at all.

Physical links are, by definition, not reliable links. To obtain a
reliable link, you need to establish a "connection". A connection uses
the basic unreliable UDP and unreliable physical links for lower-level
transfer, in conjunction with end-points that will continue to do whatever
is necessary to send/receive buffers of data (not packets). The buffer you
provide is guaranteed to get there, intact, if you wait long enough. What
is never guaranteed is the time for the buffer to get to its destination.
This time may be several days (no joke). This is useful! You can have
a ftp transfer in progress and have a router crash. After the router
comes back up, the transfer will continue.

> 
> UDP failing I can understand (kind of, anyway) but for raw sockets, packet
> sockets, etc. I think there should be at least some kind of mechanism to
> bypass
> all the congestion controls and either shove the packet onto the device's tx
> buffer or return a failure code.
> 

See above.

> The possibility of random dropping of packets in the kernel means that an
> infinite loop on sendto() will chew up the entire machine even if you've only
> got a 10Mbit/s link.  This seems just wrong.
>

This is the basic reason why the return-value of sento() should be
ignored, even though Alan doesn't agree. Basically, if you give
valid parameters to sendto() (correct socket, pointer to correct
structure, etc), you can just ignore the return value. Its not useful
in the overall scheme. If you must make sure that a UDP packet got
to a receiver, then the receiver must (somehow) hand-shake with you.
How you do the handshake is entirely up to you. UDP stands for
User Datagram Protocol. It's a quick-fling out the spicket. How you
handle lost messages is up to the user.


Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Chris Friesen]

"Richard B. Johnson" wrote:
> 
> On Thu, 7 Feb 2002, Chris Friesen wrote:

> > The possibility of random dropping of packets in the kernel means that an
> > infinite loop on sendto() will chew up the entire machine even if you've only
> > got a 10Mbit/s link.  This seems just wrong.
> >
> 
> This is the basic reason why the return-value of sento() should be
> ignored, even though Alan doesn't agree. Basically, if you give
> valid parameters to sendto() (correct socket, pointer to correct
> structure, etc), you can just ignore the return value. Its not useful
> in the overall scheme. If you must make sure that a UDP packet got
> to a receiver, then the receiver must (somehow) hand-shake with you.
> How you do the handshake is entirely up to you. UDP stands for
> User Datagram Protocol. It's a quick-fling out the spicket. How you
> handle lost messages is up to the user.

Okay, I buy the fact that sendto can't guarantee that the packet got anywhere. 
Fine.  Now what about sending packets out faster than the output device can
possibly handle them?  We know the device is busy, it's under our control.  It
seems to me to be logical to block the sender until the congestion goes away, or
return with an error code if the sender is non-blocking.  This may not play nice
with the current kernel networking code (qdisc and all that) but doesn't it seem
like a good idea in principle?

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Richard B. Johnson]

On Thu, 7 Feb 2002, Chris Friesen wrote:

> "Richard B. Johnson" wrote:
> > 
> > On Thu, 7 Feb 2002, Chris Friesen wrote:
> 
[SNIPPED]

> 
> Okay, I buy the fact that sendto can't guarantee that the packet got
> anywhere. 
> Fine.  Now what about sending packets out faster than the output device can
> possibly handle them?  We know the device is busy, it's under our control.  It
> seems to me to be logical to block the sender until the congestion goes
> away, or
> return with an error code if the sender is non-blocking.  This may not
> play nice
> with the current kernel networking code (qdisc and all that) but doesn't
> it seem
> like a good idea in principle?
> 
> Chris
>

Basically it has nothing to do with the current networking code. If
you started from scratch and wrote the 'perfect' networking code you
would soon learn that various links run at different transfer speeds.
If you have a 100 Mb/s link that goes to a router that has a 9600-baud
dial-up connection, the only way you can do flow-control is to throw
away packets. So, you thought a single UDP packet got there --into
a full buffer on the router, trying to send all the M$ Netbeui packets
down the 9600 baud pipe. Guess again! Whatever host sends the most
data to the starved router will get the most packets through and no
host will get them all through. It's just that simple.

Since UDP, by definition isn't acknowledged, your perfect networking
software "thought" that everything was fine because it never got any
errors at all.

Just for information. Do `tcpdump -n` on your machine. Look at all
those packets. Now, how big a buffer would you need to save them all
and send them out a 9600-baud dial-up?  The answer is 'N', which
tends towards infinity, the longer the dial-up persists. So, if
you were routing these, in your perfect network, you would just
throw away the ones that you don't have room for. You can't tell
the sender(s); "Stop Sending!". They won't be listening to you.
They will only be listening to the end-point, and that, only if
the end-point is "connected".

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (797.90 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Mark Frazer]

Chris Friesen <cfriesen@nortelnetworks.com> [02/02/07 16:04]:
> under our control.  It seems to me to be logical to block the sender
> until the congestion goes away, or return with an error code if the
> sender is non-blocking.  This may not play nice with the current kernel
> networking code (qdisc and all that) but doesn't it seem like a good
> idea in principle?

If not, it is then possible for a user on a fast machine to hammer the
network interfaces with UDP packets as some sort of denial of service
attack?

Blocking all senders when the qdisc is full and round-robin'ing among
the blocked would prevent this particular attack.

Re: want opinions on possible glitch in 2.4 network error reporti ng

[Ian Molton]

On a sunny Thu, 7 Feb 2002 13:08:24 -0500 (EST) Richard B. Johnson gathered
a sheaf of electrons and etched in their motions the following immortal
words:

> On Thu, 7 Feb 2002, Perches, Joe wrote:
> [SNIPPED..]
> > > That is correct UDP behaviour 
> > 
> > Do you think this is the correct PacketSocket/RAW behaviour?
> 
> Yes.
> 
> > How does one guarantee a send/sendto/write?
> > -
> 
> Easy, you use send() or write(). These work on stream protocol TCP/IP

I know its an extreme case, but consider that something goes wrong and the
kernel ends up thinking its buffer is always full / zero lenngth /
something horrible.

I'd personally like it if it warned me it wasnt even trying to send my
packets, rather than just ignoring them completely...

RE: want opinions on possible glitch in 2.4 network error reporti ng

[Perches, Joe]

How does one guarantee a packet socket send/sendto/write?

-----Original Message-----
From: Richard B. Johnson [mailto:root@chaos.analogic.com]
> > That is correct UDP behaviour 
> Do you think this is the correct PacketSocket/RAW behaviour?
Yes.
> How does one guarantee a send/sendto/write?
Easy, you use send() or write(). These work on stream protocol TCP/IP
where there is a "connection".