From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AD29C3A5A2 for ; Fri, 23 Aug 2019 10:27:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 29C2322CEC for ; Fri, 23 Aug 2019 10:27:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732420AbfHWK10 convert rfc822-to-8bit (ORCPT ); Fri, 23 Aug 2019 06:27:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:33610 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731002AbfHWK10 (ORCPT ); Fri, 23 Aug 2019 06:27:26 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3E38C3061423; Fri, 23 Aug 2019 10:27:25 +0000 (UTC) Received: from carbon (ovpn-200-29.brq.redhat.com [10.40.200.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 456575D6B2; Fri, 23 Aug 2019 10:27:14 +0000 (UTC) Date: Fri, 23 Aug 2019 12:27:13 +0200 From: Jesper Dangaard Brouer To: Andrii Nakryiko Cc: Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= , Stephen Hemminger , Daniel Borkmann , Alexei Starovoitov , Martin KaFai Lau , Song Liu , Yonghong Song , David Miller , Networking , bpf , brouer@redhat.com, Anton Protopopov , Stanislav Fomichev , Yoel Caspersen Subject: Re: [RFC bpf-next 0/5] Convert iproute2 to use libbpf (WIP) Message-ID: <20190823122713.73450a4b@carbon> In-Reply-To: References: <20190820114706.18546-1-toke@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.45]); Fri, 23 Aug 2019 10:27:25 +0000 (UTC) Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Wed, 21 Aug 2019 13:30:09 -0700 Andrii Nakryiko wrote: > On Tue, Aug 20, 2019 at 4:47 AM Toke Høiland-Jørgensen wrote: > > > > iproute2 uses its own bpf loader to load eBPF programs, which has > > evolved separately from libbpf. Since we are now standardising on > > libbpf, this becomes a problem as iproute2 is slowly accumulating > > feature incompatibilities with libbpf-based loaders. In particular, > > iproute2 has its own (expanded) version of the map definition struct, > > which makes it difficult to write programs that can be loaded with both > > custom loaders and iproute2. > > > > This series seeks to address this by converting iproute2 to using libbpf > > for all its bpf needs. This version is an early proof-of-concept RFC, to > > get some feedback on whether people think this is the right direction. > > > > What this series does is the following: > > > > - Updates the libbpf map definition struct to match that of iproute2 > > (patch 1). > > > Hi Toke, > > Thanks for taking a stab at unifying libbpf and iproute2 loaders. I'm > totally in support of making iproute2 use libbpf to load/initialize > BPF programs. But I'm against adding iproute2-specific fields to > libbpf's bpf_map_def definitions to support this. > > I've proposed the plan of extending libbpf's supported features so > that it can be used to load iproute2-style BPF programs earlier, > please see discussions in [0] and [1]. I think instead of emulating > iproute2 way of matching everything based on user-specified internal > IDs, which doesn't provide good user experience and is quite easy to > get wrong, we should support same scenarios with better declarative > syntax and in a less error-prone way. I believe we can do that by > relying on BTF more heavily (again, please check some of my proposals > in [0], [1], and discussion with Daniel in those threads). It will > feel more natural and be more straightforward to follow. It would be > great if you can lend a hand in implementing pieces of that plan! > > I'm currently on vacation, so my availability is very sparse, but I'd > be happy to discuss this further, if need be. > > [0] https://lore.kernel.org/bpf/CAEf4BzbfdG2ub7gCi0OYqBrUoChVHWsmOntWAkJt47=FE+km+A@mail.gmail.com/ > [1] https://www.spinics.net/lists/bpf/msg03976.html > > > - Adds functionality to libbpf to support automatic pinning of maps when > > loading an eBPF program, while re-using pinned maps if they already > > exist (patches 2-3). For production use-cases, libbpf really need an easier higher-level API for re-using pinned maps, for establishing shared maps between programs. The existing libbpf API bpf_object__pin_maps() and bpf_object__unpin_maps(), which don't re-use pinned maps, are not really usable, because they pin/unpin ALL maps in the ELF file. What users really need is an easy way to specify, on a per map basis, what kind of pinning and reuse/sharing they want. E.g. like iproute2 have, "global", "object-scope", and "no-pinning". ("ifindex-scope" would be nice for XDP). Today users have to split/reimplement bpf_prog_load_xattr(), and use/add bpf_map__reuse_fd(). Which is that I ended doing for xdp-cpumap-tc[2] (used in production at ISP) resulting in 142 lines of extra code[3] that should have been hidden inside libbpf. And worse, in this solution[4] the maps for reuse-pinning is specified in the code by name. Thus, they cannot use a generic loader. That I why, I want to mark the maps via a pinning member, like iproute2. I really hope this moves in a practical direction, as I have the next production request lined up (also from an ISP), and I hate to have to advice them to choose the same route as [3]. [2] https://github.com/xdp-project/xdp-cpumap-tc/ [3] https://github.com/xdp-project/xdp-cpumap-tc/blob/master/src/xdp_iphash_to_cpu_user.c#L262-L403 [4] https://github.com/xdp-project/xdp-cpumap-tc/blob/master/src/xdp_iphash_to_cpu_user.c#L431-L441 -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer