From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8672BC0044B for ; Wed, 13 Feb 2019 20:49:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 53B0F222B6 for ; Wed, 13 Feb 2019 20:49:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IsfKdXYO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390598AbfBMUtk (ORCPT ); Wed, 13 Feb 2019 15:49:40 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:32960 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727251AbfBMUtj (ORCPT ); Wed, 13 Feb 2019 15:49:39 -0500 Received: by mail-pl1-f194.google.com with SMTP id y10so1786754plp.0 for ; Wed, 13 Feb 2019 12:49:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version; bh=BGgucVYqLh/Y8YaRNU/Q6bmxzGMbFsjvX+XhlaTLL5U=; b=IsfKdXYODo1r206aDnZw6bIJID/i/QlFPtn6XV3YJoatcdvua96rVAwNO0JLB7NRQS ln2YK6/AQSOOgumlJSV97WOPkzS8QphDz72+M9DH5EKbgcRPNVcES220/P0WfkFjOwKN QNcXOrCSbdo6h3kEVt5rIBowO1AK9Ye60O3tcXg5MTRgLb5ziT9m3GxJOXS3C7fggdvm 97e1erjnK231yHhuPzCUllYuwcVeT0LXdcKOdZNkv1k1Xg1kY3Eoc7qXWB5NLyXhBuXJ F62ckosEwmWDhFska7ousxXa//embu2s9MQBjup+nPAw/Hjkms9EUYvoiYjtL5NicEPm C3Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version; bh=BGgucVYqLh/Y8YaRNU/Q6bmxzGMbFsjvX+XhlaTLL5U=; b=eZXtS+lDpB5LFQZc7WMuZM6877x40or3myglTBqA9kiG1Irm8k23iT592rYYgl9z1x 6wRnNnVEMoimDyyRGQRV14qXwkQt7gkwmlzDXKf9j4ep051HDKhhhAS56AJ4lQGO7RYt /DBNwRKY2+hGMuMq/9ZPYQfPKdGYkML9pSiQyclPe+Vp0vIG5tjiWPrGh2xVOSmnv97Y HVNA+Qb1gYSCQRjLZQvvDnKdYVx/C38lYSnh+kuHKuzZaGGrMiiSATIeDeeUXH9/3lt1 zW5WEFrXD5mjRPWcF+Y16t1PhlmwoyyEXQOvxULZsu93iXeWIn2o+jUeZOkbTD/duoF4 fgwQ== X-Gm-Message-State: AHQUAuZXpMsmx/KYbuInp3TNlt5Qb4tC5Cj1EljE3zxKOjI6x6VO5/do wXCLiY0WCINSqmO4NhEEzA0= X-Google-Smtp-Source: AHgI3IbsLN3074en0ltxVCaIPbquEatrLipeKGES81NAuB5yMIF1YyzoScDsURVHjkE0XnUg6DxDng== X-Received: by 2002:a17:902:584:: with SMTP id f4mr132834plf.28.1550090978655; Wed, 13 Feb 2019 12:49:38 -0800 (PST) Received: from [172.26.118.98] ([2620:10d:c090:180::1:6c9f]) by smtp.gmail.com with ESMTPSA id g6sm245737pgq.18.2019.02.13.12.49.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 13 Feb 2019 12:49:37 -0800 (PST) From: "Jonathan Lemon" To: "Magnus Karlsson" Cc: "Magnus Karlsson" , "=?utf-8?b?QmrDtnJuIFTDtnBlbA==?=" , ast@kernel.org, "Daniel Borkmann" , "Network Development" , "Jakub Kicinski" , "=?utf-8?b?QmrDtnJuIFTDtnBlbA==?=" , "Zhang, Qi Z" , "Jesper Dangaard Brouer" , xiaolong.ye@intel.com Subject: Re: [PATCH bpf-next v4 0/2] libbpf: adding AF_XDP support Date: Wed, 13 Feb 2019 12:49:35 -0800 X-Mailer: MailMate (1.12.4r5594) Message-ID: <3213D100-3861-4963-9490-EE445B731E63@gmail.com> In-Reply-To: References: <1549631126-29067-1-git-send-email-magnus.karlsson@intel.com> <36557463-D23A-432E-AA18-7731F43CEBA6@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; format=flowed; markup=markdown Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On 13 Feb 2019, at 3:32, Magnus Karlsson wrote: > On Mon, Feb 11, 2019 at 9:44 PM Jonathan Lemon > wrote: >> >> On 8 Feb 2019, at 5:05, Magnus Karlsson wrote: >> >>> This patch proposes to add AF_XDP support to libbpf. The main reason >>> for this is to facilitate writing applications that use AF_XDP by >>> offering higher-level APIs that hide many of the details of the >>> AF_XDP >>> uapi. This is in the same vein as libbpf facilitates XDP adoption by >>> offering easy-to-use higher level interfaces of XDP >>> functionality. Hopefully this will facilitate adoption of AF_XDP, >>> make >>> applications using it simpler and smaller, and finally also make it >>> possible for applications to benefit from optimizations in the >>> AF_XDP >>> user space access code. Previously, people just copied and pasted >>> the >>> code from the sample application into their application, which is >>> not >>> desirable. >> >> I like the idea of encapsulating the boilerplate logic in a library. >> >> I do think there is an important missing piece though - there should >> be >> some code which queries the netdev for how many queues are attached, >> and >> create the appropriate number of umem/AF_XDP sockets. >> >> I ran into this issue when testing the current AF_XDP code - on my >> test >> boxes, the mlx5 card has 55 channels (aka queues), so when the test >> program >> binds only to channel 0, nothing works as expected, since not all >> traffic >> is being intercepted. While obvious in hindsight, this took a while >> to >> track down. > > Yes, agreed. You are not the first one to stumble upon this problem > :-). Let me think a little bit on how to solve this in a good way. We > need this to be simple and intuitive, as you say. Has any investigation been done on using some variant of MPSC implementation as an intermediate form for AF_XDP? E.g.: something like LCRQ or the bulkQ in bpf devmap/cpumap. I'm aware that this would be slightly slower, as it would introduce a lock in the path, but I'd think that having DEVMAP, CPUMAP and XSKMAP all behave the same way would add more flexibility. Ideally, if the configuration matches the underlying hardware, then the implementation would reduce to the current setup (and allow ZC implementations), but a non-matching configuration would still work - as opposed to the current situation. -- Jonathan