From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED1AFC636CC for ; Thu, 16 Feb 2023 13:28:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229934AbjBPN2o (ORCPT ); Thu, 16 Feb 2023 08:28:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42008 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229782AbjBPN2n (ORCPT ); Thu, 16 Feb 2023 08:28:43 -0500 Received: from Chamillionaire.breakpoint.cc (Chamillionaire.breakpoint.cc [IPv6:2a0a:51c0:0:237:300::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E61C564AB for ; Thu, 16 Feb 2023 05:28:42 -0800 (PST) Received: from fw by Chamillionaire.breakpoint.cc with local (Exim 4.92) (envelope-from ) id 1pSeJP-0004du-GY; Thu, 16 Feb 2023 14:28:35 +0100 Date: Thu, 16 Feb 2023 14:28:35 +0100 From: Florian Westphal To: Jakub Kicinski Cc: Florian Westphal , davem@davemloft.net, netdev@vger.kernel.org, edumazet@google.com, pabeni@redhat.com, willemb@google.com Subject: Re: [RFC] net: skbuff: let struct skb_ext live inside the head Message-ID: <20230216132835.GA14032@breakpoint.cc> References: <20230215034444.482178-1-kuba@kernel.org> <20230215094332.GB9908@breakpoint.cc> <20230215101356.3b86c451@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230215101356.3b86c451@kernel.org> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Jakub Kicinski wrote: > On Wed, 15 Feb 2023 10:43:32 +0100 Florian Westphal wrote: > > I think the cleaner solution would be to move the new extension ids > > into sk_buff itself (at the end, uninitialized data unless used). > > > > Those extensions would always reside there and not in the slab object. > > Do you mean the entire extension? 8B of metadata + (possibly) 32B > of the key? 32B is too much if its for something esoteric, but see below. > > Obviously that only makes sense for extensions where we assume > > that typical workload will require them, which might be a hard call to > > make. > > I'm guessing that's the reason why Google is okay with putting the key > in the skb - they know they will use it most of the time. But an > average RHEL user may appreciate the skb growth for an esoteric protocol > to a much smaller extent :( Absolutely, I agree that its a non-starter to place this in sk_buff itself. TX side is less of a problem here because of superpackets. For RX I think your simpler napi-recycle patch is a good start. I feel its better to wait before doing anything further in this direction (e.g. array-of-cached extensions or whatever) until we've a better test case/more realistic workload(s). If we need to look at further allocation avoidances one thing that could be evaluated would be placing an extension struct into sk_buff_fclones (unioned with the fclone skb). Fclone skb is marked busy, extension release clears it again. Just something to keep in mind for later. Only downside I see is that we can't release the extension area anymore before the skb gets queued.