From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx3-rdu2.redhat.com ([66.187.233.73]:43022 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751604AbeCVCRA (ORCPT ); Wed, 21 Mar 2018 22:17:00 -0400 Subject: Re: [bpf-next V2 PATCH 10/15] xdp: rhashtable with allocator ID to pointer mapping To: Jesper Dangaard Brouer Cc: netdev@vger.kernel.org, =?UTF-8?B?QmrDtnJuVMO2cGVs?= , magnus.karlsson@intel.com, eugenia@mellanox.com, John Fastabend , Eran Ben Elisha , Saeed Mahameed , galp@mellanox.com, Daniel Borkmann , Alexei Starovoitov , Tariq Toukan References: <152051439383.7018.11827926732878918934.stgit@firesoul> <152051451209.7018.14564668423312296696.stgit@firesoul> <20180309103725.234ff988@redhat.com> <14397dda-c469-f8bd-a1fb-fe8cc6c28e82@redhat.com> <20180309170735.5573f4e0@redhat.com> <36f68a76-4c73-e0d1-b97b-645393e006ee@redhat.com> <20180319104858.4cba3a6b@redhat.com> <6950eb76-8e3d-60f2-6de1-005a4e4fd3f6@redhat.com> <20180320152725.5ea01d34@redhat.com> From: Jason Wang Message-ID: <01481dbe-ed95-dd2e-67f7-1543d75cc138@redhat.com> Date: Thu, 22 Mar 2018 10:16:47 +0800 MIME-Version: 1.0 In-Reply-To: <20180320152725.5ea01d34@redhat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 2018年03月20日 22:27, Jesper Dangaard Brouer wrote: > On Tue, 20 Mar 2018 10:26:50 +0800 > Jason Wang wrote: > >> On 2018年03月19日 17:48, Jesper Dangaard Brouer wrote: >>> On Fri, 16 Mar 2018 16:45:30 +0800 >>> Jason Wang wrote: >>> >>>> On 2018年03月10日 00:07, Jesper Dangaard Brouer wrote: >>>>> On Fri, 9 Mar 2018 21:07:36 +0800 >>>>> Jason Wang wrote: >>>>> >>>>>>>>> Use the IDA infrastructure for getting a cyclic increasing ID number, >>>>>>>>> that is used for keeping track of each registered allocator per >>>>>>>>> RX-queue xdp_rxq_info. >>>>>>>>> >>>>>>>>> Signed-off-by: Jesper Dangaard Brouer >>>>>>>> A stupid question is, can we manage to unify this ID with NAPI id? >>>>>>> Sorry I don't understand the question? >>>>>> I mean can we associate page poll pointer to napi_struct, record NAPI id >>>>>> in xdp_mem_info and do lookup through NAPI id? >>>>> No. The driver can unreg/reg a new XDP memory model, >>>> Is there an actual use case for this? >>> I believe this is the common use case. When attaching an XDP/bpf prog, >>> then the driver usually want to change the RX-ring memory model >>> (different performance trade off). >> Right, but a single driver should only have one XDP memory model. > No! -- a driver can have multiple XDP memory models, based on different > performance trade offs and hardware capabilities. > > The mlx5 (100Gbit/s) driver/hardware is a good example, which need > different memory models. It already support multiple RX memory models, > depending on HW support. So let me correct my question, not familiar with mlx5e driver but if I understand correctly, driver (mlx5) will not change memory model during runtime for each NAPI. So NAPI id still work in this case? > So, I predict that we hit at performance > limit around 42Mpps on PCIe (I can measure 36Mpps), this is due to > PCI-express translations/sec limit. The mlx5 HW supports a compressed > descriptor format which deliver packets in several pages (based on > offset and len), thus lowering the needed PCIe transactions. The > pitfall is that this comes tail room limitations, which can be okay if > e.g. the users use-case does not involve cpumap. > > Plus, when a driver need to support AF_XDP zero-copy, that also count > as another XDP memory model... Yes or TAP zero-copy XDP. But it looks to me that we don't even need to care about the recycling here since the pages belongs to userspace. Thanks