From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B8B5C10F13 for ; Mon, 8 Apr 2019 17:06:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6BD6E20880 for ; Mon, 8 Apr 2019 17:06:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729149AbfDHRGA (ORCPT ); Mon, 8 Apr 2019 13:06:00 -0400 Received: from mail-ed1-f68.google.com ([209.85.208.68]:39270 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727190AbfDHRF7 (ORCPT ); Mon, 8 Apr 2019 13:05:59 -0400 Received: by mail-ed1-f68.google.com with SMTP id n9so3494772edt.6 for ; Mon, 08 Apr 2019 10:05:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:user-agent :mime-version:content-transfer-encoding; bh=UHVyw/2ZEI2fyuT/ai8QzXhR0B4BDD4CvVvIYyrCs3o=; b=TgrXQmoUoMErBGQSIafas2MZerv1Q0vD8cJ1MYNfn0ACJ1V0ApISepCwn3awgnNI1g IkuQL1QyBA7Uk+l5+uUK1Pck3dXVMkGYSzKYvNrytr1CXvs1CScYc7L2q3YLzlRem+TX +sSXIhBl5dzgJGy1xIW9A3+0RSz7hSTORjJaLhNLY8RKdhOvrlafw8weNF2sikCNnJTd wINCke4q+9VY7Gb0vKpT+3NMYb8ySUIGWPfWplXJaG28ETLvvm4AwEszScKuq+I5NETw inzYSd2Qe913azvIFuFciBr8ZFfQEcmvnSiOMuzkSekDAivPlNNGDwwN2wfyxu1Eekup nhWg== X-Gm-Message-State: APjAAAXzKMSGqR9ImQYuVEcnflij1Sy4SjtjzwhiHAJaD0Gg4UdZxwho XswdNHgCLntkbeIVjyTkn3oqtw== X-Google-Smtp-Source: APXvYqxFwZchBcK2PW4kxamxg49qafkOCiJEck5p45yoqSsaggWUwx/UkMUZDZWtJW6GcShDlWqv+Q== X-Received: by 2002:aa7:d819:: with SMTP id v25mr17573051edq.70.1554743157633; Mon, 08 Apr 2019 10:05:57 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk (borgediget.toke.dk. [85.204.121.218]) by smtp.gmail.com with ESMTPSA id c26sm3062265eja.86.2019.04.08.10.05.56 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 08 Apr 2019 10:05:57 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 96EFE1804A4; Mon, 8 Apr 2019 19:05:56 +0200 (CEST) Subject: [PATCH net-next v4 0/6] xdp: Use a default map for xdp_redirect helper From: Toke =?utf-8?q?H=C3=B8iland-J=C3=B8rgensen?= To: David Miller Cc: netdev@vger.kernel.org, Jesper Dangaard Brouer , Daniel Borkmann , Alexei Starovoitov , Jakub Kicinski , =?utf-8?b?QmrDtnJu?= =?utf-8?b?VMO2cGVs?= Date: Mon, 08 Apr 2019 19:05:56 +0200 Message-ID: <155474315642.24432.6179239576879119104.stgit@alrua-x1> User-Agent: StGit/unknown-version MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This series changes the xdp_redirect helper to use a hidden default map. The redirect_map() helper also uses the map structure to batch packets, which results in a significant (around 50%) performance boost for the _map variant. However, the xdp_redirect() API is simpler if one just wants to redirect to another interface, which means people tend to use this interface and then wonder why they getter worse performance than expected. This series seeks to close this performance difference between the two APIs. It achieves this by changing xdp_redirect() to use a hidden devmap for looking up destination interfaces, thus gaining the batching benefit with no visible difference from the user API point of view. Allocation of the default map is done dynamically as programs using the xdp_redirect helper are loaded and attached to interfaces, and the maps are freed again when no longer needed. Because of tail calls, this requires two levels of refcounting: One global level that keeps track of whether any XDP programs using the xdp_redirect() helper are loaded in the system at all. And another that keeps track of whether any programs that could potentially result in a call to xdp_redirect (i.e., either programs using the helper, or programs using tail calls) are loaded in a given namespace. The default maps are dynamically sized to the nearest power-of-two size that can contain all interfaces present in each interface. If allocation fails, the user action that triggered the allocation (either attaching an XDP program, or moving an interface with a program attached) is rejected. If new interfaces appear in a namespace which causes the default map to become too small, a new one is allocated with the correct size; this allocation is the only one that cannot lead to a rejection of the userspace action, so if it fails a warning is emitted instead. The first patch in the series refactors devmap.c to prepare for the subsequent patches. The second patch adds the default map handling using the existing array-based devmap structure. The third patch adds a new map type (devmap_idx) that hashes devices on ifindex. Changelog: v3 -> v4: - Add two patches from Björn to re-factor XDP program tracking for netdevs (patches 1+2). This obviates the need for the bpf_get_prog_by_id() function added in the previous versions. - Move freeing and flushing of maps to a workqueue instead of an RCU callback. - Handle replacing a program using redirect with one that doesn't (and vice versa). - Use a mutex instead of a spinlock for the internal devmap locking, and make sure everything is covered by it. - Add selftest (and /proc/net interface) for checking if the maps are allocated correctly (patch 6). - Fix a bunch of nits from Jakub. v2 -> v3: - Fix compile warnings when CONFIG_BPF_SYSCALL is unset (as pointed out by the kbuild test bot). v1 -> v2: - Add refcounting to only allocate default maps when needed - Using said refcounting, also deallocate default maps - Add dynamic sizing of default maps - Handle moving of interfaces between namespaces - Split out refactoring of devmap.c to separate patch - Use hashmap semantics for update_elem of devmap_idx type maps --- Björn Töpel (2): net: xdp: refactor XDP attach net: xdp: remove XDP_QUERY_PROG Toke Høiland-Jørgensen (4): xdp: Refactor devmap code in preparation for subsequent additions xdp: Always use a devmap for XDP_REDIRECT to a device xdp: Add devmap_idx map type for looking up devices by ifindex selftests/bpf: Add test for default devmap allocation drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 4 drivers/net/ethernet/cavium/thunder/nicvf_main.c | 3 drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 3 drivers/net/ethernet/intel/i40e/i40e_main.c | 3 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 4 drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 4 drivers/net/ethernet/mellanox/mlx4/en_netdev.c | 24 - drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 18 - .../net/ethernet/netronome/nfp/nfp_net_common.c | 2 drivers/net/ethernet/qlogic/qede/qede_filter.c | 3 drivers/net/netdevsim/bpf.c | 2 drivers/net/netdevsim/netdevsim.h | 2 drivers/net/tun.c | 15 drivers/net/veth.c | 15 drivers/net/virtio_net.c | 17 - include/linux/bpf.h | 46 + include/linux/bpf_types.h | 1 include/linux/filter.h | 2 include/linux/netdevice.h | 11 include/net/net_namespace.h | 2 include/net/netns/xdp.h | 11 include/trace/events/xdp.h | 3 include/uapi/linux/bpf.h | 1 kernel/bpf/devmap.c | 668 ++++++++++++++++++-- kernel/bpf/syscall.c | 3 kernel/bpf/verifier.c | 15 net/core/dev.c | 209 ++++-- net/core/filter.c | 69 -- net/core/rtnetlink.c | 14 tools/bpf/bpftool/map.c | 1 tools/include/uapi/linux/bpf.h | 1 tools/lib/bpf/libbpf_probes.c | 1 tools/testing/selftests/bpf/Makefile | 3 .../selftests/bpf/progs/test_xdp_tail_call.c | 39 + tools/testing/selftests/bpf/test_maps.c | 16 .../testing/selftests/bpf/test_xdp_devmap_alloc.sh | 94 +++ 36 files changed, 1000 insertions(+), 329 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/test_xdp_tail_call.c create mode 100755 tools/testing/selftests/bpf/test_xdp_devmap_alloc.sh