Netdev List
 help / color / mirror / Atom feed
From: Josh Hunt <johunt@akamai.com>
To: netdev <netdev@vger.kernel.org>,
	Eric Dumazet <edumazet@google.com>,
	David Miller <davem@davemloft.net>,
	Willem de Bruijn <willemb@google.com>
Subject: udp sendmsg ENOBUFS clarification
Date: Tue, 17 Sep 2019 13:20:21 -0700	[thread overview]
Message-ID: <ce01f024-268d-a44e-8093-91be97f1e8b0@akamai.com> (raw)

I was running some tests recently with the udpgso_bench_tx benchmark in 
selftests and noticed that in some configurations it reported sending 
more than line rate! Looking into it more I found that I was overflowing 
the qdisc queue and so it was sending back NET_XMIT_DROP however this 
error did not propagate back up to the application and so it assumed 
whatever it sent was done successfully. That's when I learned about 
IP_RECVERR and saw that the benchmark isn't using that socket option.

That's all fairly straightforward, but what I was hoping to get 
clarification on is where is the line drawn on when or when not to send 
ENOBUFS back to the application if IP_RECVERR is *not* set? My guess 
based on going through the code is that as long as the packet leaves the 
stack (in this case sent to the qdisc) that's where we stop reporting 
ENOBUFS back to the application, but can someone confirm?

For example, we sanitize the error in udp_send_skb():
send:
         err = ip_send_skb(sock_net(sk), skb);
         if (err) {
                 if (err == -ENOBUFS && !inet->recverr) {
                         UDP_INC_STATS(sock_net(sk),
                                       UDP_MIB_SNDBUFERRORS, is_udplite);
                         err = 0;
                 }
         } else


but in udp_sendmsg() we don't:

         if (err == -ENOBUFS || test_bit(SOCK_NOSPACE, 
&sk->sk_socket->flags)) {
                 UDP_INC_STATS(sock_net(sk),
                               UDP_MIB_SNDBUFERRORS, is_udplite);
         }
         return err;

In the case above it looks like we may only get ENOBUFS for allocation 
failures inside of the stack in udp_sendmsg() and so that's why we 
propagate the error back up to the application?

Somewhat related, while I was trying to find answer to the above I came 
across this thread https://patchwork.ozlabs.org/patch/32857/ It looks 
like the man send() man page still only says the following about -ENOBUFS:

  "The output queue for a network interface was full.
   This generally indicates that the interface has stopped sending,
   but may be caused by transient congestion.
   (Normally, this does not occur in Linux. Packets are just silently
   dropped when a device queue overflows.) "

but as Eric points out that's not true when IP_RECVERR is set on the 
socket. Was there an attempt to update the man page to reflect this, but 
it was rejected? I couldn't find any discussion on this.

Thanks
Josh

             reply	other threads:[~2019-09-17 20:20 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-17 20:20 Josh Hunt [this message]
2019-09-18 15:35 ` udp sendmsg ENOBUFS clarification Willem de Bruijn
2019-09-19  2:59   ` Josh Hunt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ce01f024-268d-a44e-8093-91be97f1e8b0@akamai.com \
    --to=johunt@akamai.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=willemb@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox