From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paolo Abeni Subject: Re: [RFC PATCH 3/4] udp: implement GRO plain UDP sockets. Date: Mon, 17 Sep 2018 12:06:06 +0200 Message-ID: <0cb5a631ce390385729d04fbf16b2ccb2548d8db.camel@redhat.com> References: <027ebc7d404af07b70f8f9816512b2a485e8f420.1536939423.git.pabeni@redhat.com> <1e140072-15f6-755c-28be-61534b243d62@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: "David S. Miller" , Willem de Bruijn , Steffen Klassert To: Eric Dumazet , netdev@vger.kernel.org Return-path: Received: from mx1.redhat.com ([209.132.183.28]:36120 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726753AbeIQPct (ORCPT ); Mon, 17 Sep 2018 11:32:49 -0400 In-Reply-To: <1e140072-15f6-755c-28be-61534b243d62@gmail.com> Sender: netdev-owner@vger.kernel.org List-ID: Hi, On Fri, 2018-09-14 at 09:48 -0700, Eric Dumazet wrote: > Are you sure the data is actually fully copied to user space ? > > tools/testing/selftests/net/udpgso_bench_rx.c > > uses : > > static char rbuf[ETH_DATA_LEN]; > /* MSG_TRUNC will make return value full datagram length */ > ret = recv(fd, rbuf, len, MSG_TRUNC | MSG_DONTWAIT); > > So you need to change this program. Thank for the feedback. You are right, I need to update udpgso_bench_rx. Making it unconditionally read up to 64K bytes, I measure: Before: udp rx: 962 MB/s 685339 calls/s After: udp rx: 1344 MB/s 22812 calls/s Top perf offenders for udpgso_bench_rx: 31.83% [kernel] [k] copy_user_enhanced_fast_string 8.90% [kernel] [k] skb_release_data 7.97% [kernel] [k] free_pcppages_bulk 6.82% [kernel] [k] copy_page_to_iter 3.41% [kernel] [k] skb_copy_datagram_iter 2.01% [kernel] [k] free_unref_page 1.92% [kernel] [k] __entry_SYSCALL_64_trampoline Trivial note: with this even UDP sockets would benefit from remote skb freeing, as the cost of skb_release_data is relevant for the GSO packets. > Also, GRO reception would mean that userspace can retrieve, > not only full bytes of X datagrams, but also the gso_size (or length of individual datagrams) > > You can not know the size of the packets in advance, the sender will decide. Thanks for pointing that out. I guess that implementing something like cmsg(UDP_SEGMENT) as Willem suggests in in 8/8 patch would do, right? I can have a look at that _if_ there is interest in this approch, Cheers, Paolo