From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Miller Subject: Re: [PATCH v3 net-next 0/7] RDS: zerocopy support Date: Fri, 16 Feb 2018 16:13:54 -0500 (EST) Message-ID: <20180216.161354.1005826265047434117.davem@davemloft.net> References: Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: netdev@vger.kernel.org, willemdebruijn.kernel@gmail.com, rds-devel@oss.oracle.com, santosh.shilimkar@oracle.com To: sowmini.varadhan@oracle.com Return-path: Received: from shards.monkeyblade.net ([184.105.139.130]:59064 "EHLO shards.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750727AbeBPVNz (ORCPT ); Fri, 16 Feb 2018 16:13:55 -0500 In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: From: Sowmini Varadhan Date: Thu, 15 Feb 2018 10:49:31 -0800 > This is version 3 of the series, following up on review comments for > http://patchwork.ozlabs.org/project/netdev/list/?series=28530 > > Review comments addressed > Patch 4 > - fix fragile use of skb->cb[], do not set ee_code incorrectly. > Patch 5: > - remove needless bzero of skb->cb[], consolidate err cleanup > > A brief overview of this feature follows. > > This patch series provides support for MSG_ZERCOCOPY > on a PF_RDS socket based on the APIs and infrastructure added > by Commit f214f915e7db ("tcp: enable MSG_ZEROCOPY") > > For single threaded rds-stress testing using rds-tcp with the > ixgbe driver using 1M message sizes (-a 1M -q 1M) preliminary > results show that there is a significant reduction in latency: about > 90 usec with zerocopy, compared with 200 usec without zerocopy. > > This patchset modifies the above for zerocopy in the following manner. > - if the MSG_ZEROCOPY flag is specified with rds_sendmsg(), and, > - if the SO_ZEROCOPY socket option has been set on the PF_RDS socket, > application pages sent down with rds_sendmsg are pinned. The pinning > uses the accounting infrastructure added by a91dbff551a6 ("sock: ulimit > on MSG_ZEROCOPY pages"). The message is unpinned when all references > to the message go down to 0, and the message is freed by rds_message_purge. > > A multithreaded application using this infrastructure must send down > a unique 32 bit cookie as ancillary data with each sendmsg invocation. > The format of this ancillary data is described in Patch 5 of the series. > The cookie is passed up to the application on the sk_error_queue when > the message is unpinned, indicating to the application that it is now > safe to free/reuse the message buffer. The details of the completion > notification are provided in Patch 4 of this series. Series applied, thanks a lot!