* [PATCH v10 bpf-next 05/18] net: xdp: add xdp_update_skb_shared_info utility routine
2021-07-28 9:33 [PATCH v10 bpf-next 00/18] mvneta: introduce XDP multi-buffer support Lorenzo Bianconi
@ 2021-07-28 9:38 ` Lorenzo Bianconi
0 siblings, 0 replies; 2+ messages in thread
From: Lorenzo Bianconi @ 2021-07-28 9:38 UTC (permalink / raw)
To: bpf, netdev
Cc: lorenzo.bianconi, davem, kuba, ast, daniel, shayagr,
john.fastabend, dsahern, brouer, echaudro, jasowang,
alexander.duyck, saeed, maciej.fijalkowski, magnus.karlsson,
tirthendu.sarkar, toke
Introduce xdp_update_skb_shared_info routine to update frags array
metadata in skb_shared_info data structure converting to a skb from
a xdp_buff or xdp_frame.
According to the current skb_shared_info architecture in
xdp_frame/xdp_buff and to the xdp multi-buff support, there is
no need to run skb_add_rx_frag() and reset frags array converting the buffer
to a skb since the frag array will be in the same position for xdp_buff/xdp_frame
and for the skb, we just need to update memory metadata.
Introduce XDP_FLAGS_PF_MEMALLOC flag in xdp_buff_flags in order to mark
the xdp_buff or xdp_frame as under memory-pressure if pages of the frags array
are under memory pressure. Doing so we can avoid looping over all fragments in
xdp_update_skb_shared_info routine. The driver is expected to set the
flag constructing the xdp_buffer using xdp_buff_set_frag_pfmemalloc
utility routine.
Rely on xdp_update_skb_shared_info in __xdp_build_skb_from_frame routine
converting the multi-buff xdp_frame to a skb after performing a XDP_REDIRECT.
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
include/net/xdp.h | 33 ++++++++++++++++++++++++++++++++-
net/core/xdp.c | 17 +++++++++++++++++
2 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/include/net/xdp.h b/include/net/xdp.h
index ed5ea784fd45..53cccdc9528c 100644
--- a/include/net/xdp.h
+++ b/include/net/xdp.h
@@ -67,7 +67,10 @@ struct xdp_txq_info {
};
enum xdp_buff_flags {
- XDP_FLAGS_MULTI_BUFF = BIT(0), /* non-linear xdp buff */
+ XDP_FLAGS_MULTI_BUFF = BIT(0), /* non-linear xdp buff */
+ XDP_FLAGS_FRAGS_PF_MEMALLOC = BIT(1), /* xdp multi-buff paged memory
+ * is under pressure
+ */
};
struct xdp_buff {
@@ -96,6 +99,16 @@ static __always_inline void xdp_buff_clear_mb(struct xdp_buff *xdp)
xdp->flags &= ~XDP_FLAGS_MULTI_BUFF;
}
+static __always_inline bool xdp_buff_is_frag_pfmemalloc(struct xdp_buff *xdp)
+{
+ return !!(xdp->flags & XDP_FLAGS_FRAGS_PF_MEMALLOC);
+}
+
+static __always_inline void xdp_buff_set_frag_pfmemalloc(struct xdp_buff *xdp)
+{
+ xdp->flags |= XDP_FLAGS_FRAGS_PF_MEMALLOC;
+}
+
static __always_inline void
xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
{
@@ -151,6 +164,11 @@ static __always_inline bool xdp_frame_is_mb(struct xdp_frame *frame)
return !!(frame->flags & XDP_FLAGS_MULTI_BUFF);
}
+static __always_inline bool xdp_frame_is_frag_pfmemalloc(struct xdp_frame *frame)
+{
+ return !!(frame->flags & XDP_FLAGS_FRAGS_PF_MEMALLOC);
+}
+
#define XDP_BULK_QUEUE_SIZE 16
struct xdp_frame_bulk {
int count;
@@ -186,6 +204,19 @@ static inline void xdp_scrub_frame(struct xdp_frame *frame)
frame->dev_rx = NULL;
}
+static inline void
+xdp_update_skb_shared_info(struct sk_buff *skb, u8 nr_frags,
+ unsigned int size, unsigned int truesize,
+ bool pfmemalloc)
+{
+ skb_shinfo(skb)->nr_frags = nr_frags;
+
+ skb->len += size;
+ skb->data_len += size;
+ skb->truesize += truesize;
+ skb->pfmemalloc |= pfmemalloc;
+}
+
/* Avoids inlining WARN macro in fast-path */
void xdp_warn(const char *msg, const char *func, const int line);
#define XDP_WARN(msg) xdp_warn(msg, __func__, __LINE__)
diff --git a/net/core/xdp.c b/net/core/xdp.c
index cc92ccb38432..504be3ce3ca9 100644
--- a/net/core/xdp.c
+++ b/net/core/xdp.c
@@ -531,8 +531,20 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf,
struct sk_buff *skb,
struct net_device *dev)
{
+ unsigned int frag_size, frag_tsize;
unsigned int headroom, frame_size;
void *hard_start;
+ u8 nr_frags;
+
+ /* xdp multi-buff frame */
+ if (unlikely(xdp_frame_is_mb(xdpf))) {
+ struct skb_shared_info *sinfo;
+
+ sinfo = xdp_get_shared_info_from_frame(xdpf);
+ frag_tsize = sinfo->xdp_frags_tsize;
+ frag_size = sinfo->xdp_frags_size;
+ nr_frags = sinfo->nr_frags;
+ }
/* Part of headroom was reserved to xdpf */
headroom = sizeof(*xdpf) + xdpf->headroom;
@@ -552,6 +564,11 @@ struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf,
if (xdpf->metasize)
skb_metadata_set(skb, xdpf->metasize);
+ if (unlikely(xdp_frame_is_mb(xdpf)))
+ xdp_update_skb_shared_info(skb, nr_frags,
+ frag_size, frag_tsize,
+ xdp_frame_is_frag_pfmemalloc(xdpf));
+
/* Essential SKB info: protocol and skb->dev */
skb->protocol = eth_type_trans(skb, dev);
--
2.31.1
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH v10 bpf-next 05/18] net: xdp: add xdp_update_skb_shared_info utility routine
@ 2021-08-04 15:28 kernel test robot
0 siblings, 0 replies; 2+ messages in thread
From: kernel test robot @ 2021-08-04 15:28 UTC (permalink / raw)
To: kbuild
[-- Attachment #1: Type: text/plain, Size: 6400 bytes --]
CC: kbuild-all(a)lists.01.org
In-Reply-To: <1c65090b9b379528b18994c9ee5658c75ff1c71b.1627463617.git.lorenzo@kernel.org>
References: <1c65090b9b379528b18994c9ee5658c75ff1c71b.1627463617.git.lorenzo@kernel.org>
TO: Lorenzo Bianconi <lorenzo@kernel.org>
TO: bpf(a)vger.kernel.org
TO: netdev(a)vger.kernel.org
CC: lorenzo.bianconi(a)redhat.com
CC: davem(a)davemloft.net
CC: kuba(a)kernel.org
CC: ast(a)kernel.org
CC: daniel(a)iogearbox.net
CC: shayagr(a)amazon.com
CC: john.fastabend(a)gmail.com
CC: dsahern(a)kernel.org
Hi Lorenzo,
I love your patch! Perhaps something to improve:
[auto build test WARNING on bpf-next/master]
url: https://github.com/0day-ci/linux/commits/Lorenzo-Bianconi/net-skbuff-add-size-metadata-to-skb_shared_info-for-xdp/20210728-183811
base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
:::::: branch date: 7 days ago
:::::: commit date: 7 days ago
config: x86_64-randconfig-m001-20210804 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
smatch warnings:
net/core/xdp.c:568 __xdp_build_skb_from_frame() error: uninitialized symbol 'nr_frags'.
net/core/xdp.c:569 __xdp_build_skb_from_frame() error: uninitialized symbol 'frag_size'.
net/core/xdp.c:569 __xdp_build_skb_from_frame() error: uninitialized symbol 'frag_tsize'.
vim +/nr_frags +568 net/core/xdp.c
65e6dcf73398dd Lorenzo Bianconi 2021-01-29 529
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 530 struct sk_buff *__xdp_build_skb_from_frame(struct xdp_frame *xdpf,
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 531 struct sk_buff *skb,
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 532 struct net_device *dev)
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 533 {
78139a2230d403 Lorenzo Bianconi 2021-07-28 534 unsigned int frag_size, frag_tsize;
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 535 unsigned int headroom, frame_size;
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 536 void *hard_start;
78139a2230d403 Lorenzo Bianconi 2021-07-28 537 u8 nr_frags;
78139a2230d403 Lorenzo Bianconi 2021-07-28 538
78139a2230d403 Lorenzo Bianconi 2021-07-28 539 /* xdp multi-buff frame */
78139a2230d403 Lorenzo Bianconi 2021-07-28 540 if (unlikely(xdp_frame_is_mb(xdpf))) {
78139a2230d403 Lorenzo Bianconi 2021-07-28 541 struct skb_shared_info *sinfo;
78139a2230d403 Lorenzo Bianconi 2021-07-28 542
78139a2230d403 Lorenzo Bianconi 2021-07-28 543 sinfo = xdp_get_shared_info_from_frame(xdpf);
78139a2230d403 Lorenzo Bianconi 2021-07-28 544 frag_tsize = sinfo->xdp_frags_tsize;
78139a2230d403 Lorenzo Bianconi 2021-07-28 545 frag_size = sinfo->xdp_frags_size;
78139a2230d403 Lorenzo Bianconi 2021-07-28 546 nr_frags = sinfo->nr_frags;
78139a2230d403 Lorenzo Bianconi 2021-07-28 547 }
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 548
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 549 /* Part of headroom was reserved to xdpf */
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 550 headroom = sizeof(*xdpf) + xdpf->headroom;
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 551
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 552 /* Memory size backing xdp_frame data already have reserved
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 553 * room for build_skb to place skb_shared_info in tailroom.
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 554 */
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 555 frame_size = xdpf->frame_sz;
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 556
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 557 hard_start = xdpf->data - headroom;
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 558 skb = build_skb_around(skb, hard_start, frame_size);
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 559 if (unlikely(!skb))
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 560 return NULL;
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 561
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 562 skb_reserve(skb, headroom);
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 563 __skb_put(skb, xdpf->len);
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 564 if (xdpf->metasize)
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 565 skb_metadata_set(skb, xdpf->metasize);
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 566
78139a2230d403 Lorenzo Bianconi 2021-07-28 567 if (unlikely(xdp_frame_is_mb(xdpf)))
78139a2230d403 Lorenzo Bianconi 2021-07-28 @568 xdp_update_skb_shared_info(skb, nr_frags,
78139a2230d403 Lorenzo Bianconi 2021-07-28 @569 frag_size, frag_tsize,
78139a2230d403 Lorenzo Bianconi 2021-07-28 570 xdp_frame_is_frag_pfmemalloc(xdpf));
78139a2230d403 Lorenzo Bianconi 2021-07-28 571
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 572 /* Essential SKB info: protocol and skb->dev */
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 573 skb->protocol = eth_type_trans(skb, dev);
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 574
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 575 /* Optional SKB info, currently missing:
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 576 * - HW checksum info (skb->ip_summed)
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 577 * - HW RX hash (skb_set_hash)
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 578 * - RX ring dev queue index (skb_record_rx_queue)
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 579 */
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 580
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 581 /* Until page_pool get SKB return path, release DMA here */
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 582 xdp_release_frame(xdpf);
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 583
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 584 /* Allow SKB to reuse area used by xdp_frame */
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 585 xdp_scrub_frame(xdpf);
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 586
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 587 return skb;
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 588 }
97a0e1ea7b41c2 Lorenzo Bianconi 2021-01-12 589 EXPORT_SYMBOL_GPL(__xdp_build_skb_from_frame);
89f479f0eccfc8 Lorenzo Bianconi 2021-01-12 590
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 32290 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-08-04 15:28 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-08-04 15:28 [PATCH v10 bpf-next 05/18] net: xdp: add xdp_update_skb_shared_info utility routine kernel test robot
-- strict thread matches above, loose matches on Subject: below --
2021-07-28 9:33 [PATCH v10 bpf-next 00/18] mvneta: introduce XDP multi-buffer support Lorenzo Bianconi
2021-07-28 9:38 ` [PATCH v10 bpf-next 05/18] net: xdp: add xdp_update_skb_shared_info utility routine Lorenzo Bianconi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.