From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 45155320F; Tue, 14 May 2024 11:52:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715687575; cv=none; b=bsyJhI5vAMxxf4KaPYkCdoBXXpvV6YghJ5/WYRGHa6A8G++uVtURIwIN/XTbqRkAWalgmuwZHGjqLinuZ5Dh3F1S46C3h/FO5hc0egfW179WeSw/gXO4BwF7I7dcGadNEtVXRSH170jgbSOO4QShhFtuh5XMBCoW9rSNacKSgGo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715687575; c=relaxed/simple; bh=tIBNYjor9NNWBEaqAReljwHoXHR0uBXUU25freWdVbg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=amJRuEoFS76wummU8b1DPdlSwbW+72dtPOY5MHbmjY0A+1aNb/7lHfpggnZ+gsiJ6Lw2fGBkR5iSfEFlM36S+eywocrV5IgPrRevjkw4FbmIhVz9u3IqA902y9EyS4ZB6lSJHPw+bfw94SnOALWP5aDyqAwKrAMaES1CFiCeR1E= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b=H3WjK3pn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.b="H3WjK3pn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C1BCFC32781; Tue, 14 May 2024 11:52:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1715687575; bh=tIBNYjor9NNWBEaqAReljwHoXHR0uBXUU25freWdVbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H3WjK3pnkWCRBVGeb8b7MZg+QfGMtC59oeEbd6hdAckIGtKiJuU/rlOAbcle+tz7U h2KrObLYhF0xbvIlJco5d7QZMAweJ59W5RnkDZCUU+BX5DjFJ54M0rjqvJ/0cay4P7 F5LXQURotDqhk+NddE5j5ybGjsOo1y8ppoT38ZAw= From: Greg Kroah-Hartman To: stable@vger.kernel.org Cc: Greg Kroah-Hartman , patches@lists.linux.dev, =?UTF-8?q?Toke=20H=C3=B8iland-J=C3=B8rgensen?= , Alexei Starovoitov , Sasha Levin Subject: [PATCH 5.15 031/168] xdp: Add xdp_do_redirect_frame() for pre-computed xdp_frames Date: Tue, 14 May 2024 12:18:49 +0200 Message-ID: <20240514101007.865879234@linuxfoundation.org> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240514101006.678521560@linuxfoundation.org> References: <20240514101006.678521560@linuxfoundation.org> User-Agent: quilt/0.67 X-stable: review X-Patchwork-Hint: ignore Precedence: bulk X-Mailing-List: patches@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 5.15-stable review patch. If anyone has any objections, please let me know. ------------------ From: Toke Høiland-Jørgensen [ Upstream commit 1372d34ccf6dd480332b2bcb2fd59a2b9a0df415 ] Add an xdp_do_redirect_frame() variant which supports pre-computed xdp_frame structures. This will be used in bpf_prog_run() to avoid having to write to the xdp_frame structure when the XDP program doesn't modify the frame boundaries. Signed-off-by: Toke Høiland-Jørgensen Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20220103150812.87914-6-toke@redhat.com Stable-dep-of: 5bcf0dcbf906 ("xdp: use flags field to disambiguate broadcast redirect") Signed-off-by: Sasha Levin --- include/linux/filter.h | 4 +++ net/core/filter.c | 65 +++++++++++++++++++++++++++++++++++------- 2 files changed, 58 insertions(+), 11 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index ddaeb2afc022f..af0103bebb7bf 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -1020,6 +1020,10 @@ int xdp_do_generic_redirect(struct net_device *dev, struct sk_buff *skb, int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, struct bpf_prog *prog); +int xdp_do_redirect_frame(struct net_device *dev, + struct xdp_buff *xdp, + struct xdp_frame *xdpf, + struct bpf_prog *prog); void xdp_do_flush(void); /* The xdp_do_flush_map() helper has been renamed to drop the _map suffix, as diff --git a/net/core/filter.c b/net/core/filter.c index 96441da61fca8..b756951c92494 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -3987,26 +3987,44 @@ u32 xdp_master_redirect(struct xdp_buff *xdp) } EXPORT_SYMBOL_GPL(xdp_master_redirect); -int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, - struct bpf_prog *xdp_prog) +static inline int __xdp_do_redirect_xsk(struct bpf_redirect_info *ri, + struct net_device *dev, + struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) { - struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info); enum bpf_map_type map_type = ri->map_type; void *fwd = ri->tgt_value; u32 map_id = ri->map_id; - struct xdp_frame *xdpf; - struct bpf_map *map; int err; ri->map_id = 0; /* Valid map id idr range: [1,INT_MAX[ */ ri->map_type = BPF_MAP_TYPE_UNSPEC; - if (map_type == BPF_MAP_TYPE_XSKMAP) { - err = __xsk_map_redirect(fwd, xdp); - goto out; - } + err = __xsk_map_redirect(fwd, xdp); + if (unlikely(err)) + goto err; + + _trace_xdp_redirect_map(dev, xdp_prog, fwd, map_type, map_id, ri->tgt_index); + return 0; +err: + _trace_xdp_redirect_map_err(dev, xdp_prog, fwd, map_type, map_id, ri->tgt_index, err); + return err; +} + +static __always_inline int __xdp_do_redirect_frame(struct bpf_redirect_info *ri, + struct net_device *dev, + struct xdp_frame *xdpf, + struct bpf_prog *xdp_prog) +{ + enum bpf_map_type map_type = ri->map_type; + void *fwd = ri->tgt_value; + u32 map_id = ri->map_id; + struct bpf_map *map; + int err; + + ri->map_id = 0; /* Valid map id idr range: [1,INT_MAX[ */ + ri->map_type = BPF_MAP_TYPE_UNSPEC; - xdpf = xdp_convert_buff_to_frame(xdp); if (unlikely(!xdpf)) { err = -EOVERFLOW; goto err; @@ -4043,7 +4061,6 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, err = -EBADRQC; } -out: if (unlikely(err)) goto err; @@ -4053,8 +4070,34 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, _trace_xdp_redirect_map_err(dev, xdp_prog, fwd, map_type, map_id, ri->tgt_index, err); return err; } + +int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp, + struct bpf_prog *xdp_prog) +{ + struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info); + enum bpf_map_type map_type = ri->map_type; + + if (map_type == BPF_MAP_TYPE_XSKMAP) + return __xdp_do_redirect_xsk(ri, dev, xdp, xdp_prog); + + return __xdp_do_redirect_frame(ri, dev, xdp_convert_buff_to_frame(xdp), + xdp_prog); +} EXPORT_SYMBOL_GPL(xdp_do_redirect); +int xdp_do_redirect_frame(struct net_device *dev, struct xdp_buff *xdp, + struct xdp_frame *xdpf, struct bpf_prog *xdp_prog) +{ + struct bpf_redirect_info *ri = this_cpu_ptr(&bpf_redirect_info); + enum bpf_map_type map_type = ri->map_type; + + if (map_type == BPF_MAP_TYPE_XSKMAP) + return __xdp_do_redirect_xsk(ri, dev, xdp, xdp_prog); + + return __xdp_do_redirect_frame(ri, dev, xdpf, xdp_prog); +} +EXPORT_SYMBOL_GPL(xdp_do_redirect_frame); + static int xdp_do_generic_redirect_map(struct net_device *dev, struct sk_buff *skb, struct xdp_buff *xdp, -- 2.43.0