[PATCH net-next v3 0/3] Abstract page from net stack

[PATCH net-next v3 0/3] Abstract page from net stack

[Mina Almasry]

Changes in v3:

- Replaced the struct netmem union with an opaque netmem_ref type.
- Added func docs to the netmem helpers and type.
- Renamed the skb_frag_t fields since it's no longer a bio_vec

-----------

Changes in v2:
- Reverted changes to the page_pool. The page pool now retains the same
  API, so that we don't have to touch many existing drivers. The devmem
  TCP series will include the changes to the page pool.

- Addressed comments.

This series is a prerequisite to the devmem TCP series. For a full
snapshot of the code which includes these changes, feel free to check:

https://github.com/mina/linux/commits/tcpdevmem-rfcv5/

-----------

Currently these components in the net stack use the struct page
directly:

1. Drivers.
2. Page pool.
3. skb_frag_t.

To add support for new (non struct page) memory types to the net stack, we
must first abstract the current memory type.

Originally the plan was to reuse struct page* for the new memory types,
and to set the LSB on the page* to indicate it's not really a page.
However, for safe compiler type checking we need to introduce a new type.

struct netmem is introduced to abstract the underlying memory type.
Currently it's a no-op abstraction that is always a struct page underneath.
In parallel there is an undergoing effort to add support for devmem to the
net stack:

https://lore.kernel.org/netdev/20231208005250.2910004-1-almasrymina@google.com/

Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Yunsheng Lin <linyunsheng@huawei.com>
Cc: Willem de Bruijn <willemdebruijn.kernel@gmail.com>

Mina Almasry (3):
  vsock/virtio: use skb_frag_*() helpers
  net: introduce abstraction for network memory
  net: add netmem_ref to skb_frag_t

 include/linux/skbuff.h           | 92 ++++++++++++++++++++++----------
 include/net/netmem.h             | 41 ++++++++++++++
 net/core/skbuff.c                | 22 +++++---
 net/kcm/kcmsock.c                | 10 +++-
 net/vmw_vsock/virtio_transport.c |  6 +--
 5 files changed, 133 insertions(+), 38 deletions(-)
 create mode 100644 include/net/netmem.h

-- 
2.43.0.472.g3155946c3a-goog

[PATCH net-next v3 1/3] vsock/virtio: use skb_frag_*() helpers

[Mina Almasry]

Minor fix for virtio: code wanting to access the fields inside an skb
frag should use the skb_frag_*() helpers, instead of accessing the
fields directly. This allows for extensions where the underlying
memory is not a page.

Signed-off-by: Mina Almasry <almasrymina@google.com>

---

v2:

- Also fix skb_frag_off() + skb_frag_size() (David)
- Did not apply the reviewed-by from Stefano since the patch changed
relatively much.

---
 net/vmw_vsock/virtio_transport.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c
index f495b9e5186b..1748268e0694 100644
--- a/net/vmw_vsock/virtio_transport.c
+++ b/net/vmw_vsock/virtio_transport.c
@@ -153,10 +153,10 @@ virtio_transport_send_pkt_work(struct work_struct *work)
 				 * 'virt_to_phys()' later to fill the buffer descriptor.
 				 * We don't touch memory at "virtual" address of this page.
 				 */
-				va = page_to_virt(skb_frag->bv_page);
+				va = page_to_virt(skb_frag_page(skb_frag));
 				sg_init_one(sgs[out_sg],
-					    va + skb_frag->bv_offset,
-					    skb_frag->bv_len);
+					    va + skb_frag_off(skb_frag),
+					    skb_frag_size(skb_frag));
 				out_sg++;
 			}
 		}
-- 
2.43.0.472.g3155946c3a-goog

Re: [PATCH net-next v3 1/3] vsock/virtio: use skb_frag_*() helpers

[Willem de Bruijn]

Mina Almasry wrote:
> Minor fix for virtio: code wanting to access the fields inside an skb
> frag should use the skb_frag_*() helpers, instead of accessing the
> fields directly. This allows for extensions where the underlying
> memory is not a page.
> 
> Signed-off-by: Mina Almasry <almasrymina@google.com>
> 
> ---
> 
> v2:
> 
> - Also fix skb_frag_off() + skb_frag_size() (David)
> - Did not apply the reviewed-by from Stefano since the patch changed
> relatively much.
> 
> ---
>  net/vmw_vsock/virtio_transport.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c
> index f495b9e5186b..1748268e0694 100644
> --- a/net/vmw_vsock/virtio_transport.c
> +++ b/net/vmw_vsock/virtio_transport.c
> @@ -153,10 +153,10 @@ virtio_transport_send_pkt_work(struct work_struct *work)
>  				 * 'virt_to_phys()' later to fill the buffer descriptor.
>  				 * We don't touch memory at "virtual" address of this page.
>  				 */
> -				va = page_to_virt(skb_frag->bv_page);
> +				va = page_to_virt(skb_frag_page(skb_frag));
>  				sg_init_one(sgs[out_sg],
> -					    va + skb_frag->bv_offset,
> -					    skb_frag->bv_len);
> +					    va + skb_frag_off(skb_frag),
> +					    skb_frag_size(skb_frag));
>  				out_sg++;
>  			}
>  		}

If there are requests for further revision in the series, can send
this virtio cleanup on its own to get it off the stack.

> -- 
> 2.43.0.472.g3155946c3a-goog
> 

Re: [PATCH net-next v3 1/3] vsock/virtio: use skb_frag_*() helpers

[Shakeel Butt]

On Wed, Dec 20, 2023 at 1:45 PM Mina Almasry <almasrymina@google.com> wrote:
>
> Minor fix for virtio: code wanting to access the fields inside an skb
> frag should use the skb_frag_*() helpers, instead of accessing the
> fields directly. This allows for extensions where the underlying
> memory is not a page.
>
> Signed-off-by: Mina Almasry <almasrymina@google.com>

Reviewed-by: Shakeel Butt <shakeelb@google.com>

Re: [PATCH net-next v3 1/3] vsock/virtio: use skb_frag_*() helpers

[Stefano Garzarella]

On Wed, Dec 20, 2023 at 01:45:00PM -0800, Mina Almasry wrote:
>Minor fix for virtio: code wanting to access the fields inside an skb
>frag should use the skb_frag_*() helpers, instead of accessing the
>fields directly. This allows for extensions where the underlying
>memory is not a page.
>
>Signed-off-by: Mina Almasry <almasrymina@google.com>
>
>---
>
>v2:
>
>- Also fix skb_frag_off() + skb_frag_size() (David)
>- Did not apply the reviewed-by from Stefano since the patch changed
>relatively much.

Sorry for the delay, I was off.

LGTM!

Acked-by: Stefano Garzarella <sgarzare@redhat.com>

Possibly we can also send this patch alone if the series is still under
discussion because it's definitely an improvement to the current code.

Thanks,
Stefano

>
>---
> net/vmw_vsock/virtio_transport.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
>diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c
>index f495b9e5186b..1748268e0694 100644
>--- a/net/vmw_vsock/virtio_transport.c
>+++ b/net/vmw_vsock/virtio_transport.c
>@@ -153,10 +153,10 @@ virtio_transport_send_pkt_work(struct work_struct *work)
> 				 * 'virt_to_phys()' later to fill the buffer descriptor.
> 				 * We don't touch memory at "virtual" address of this page.
> 				 */
>-				va = page_to_virt(skb_frag->bv_page);
>+				va = page_to_virt(skb_frag_page(skb_frag));
> 				sg_init_one(sgs[out_sg],
>-					    va + skb_frag->bv_offset,
>-					    skb_frag->bv_len);
>+					    va + skb_frag_off(skb_frag),
>+					    skb_frag_size(skb_frag));
> 				out_sg++;
> 			}
> 		}
>-- 
>2.43.0.472.g3155946c3a-goog
>

[PATCH net-next v3 2/3] net: introduce abstraction for network memory

[Mina Almasry]

Add the netmem_ref type, an abstraction for network memory.

To add support for new memory types to the net stack, we must first
abstract the current memory type. Currently parts of the net stack
use struct page directly:

- page_pool
- drivers
- skb_frag_t

Originally the plan was to reuse struct page* for the new memory types,
and to set the LSB on the page* to indicate it's not really a page.
However, for compiler type checking we need to introduce a new type.

netmem_ref is introduced to abstract the underlying memory type. Currently
it's a no-op abstraction that is always a struct page underneath. In
parallel there is an undergoing effort to add support for devmem to the
net stack:

https://lore.kernel.org/netdev/20231208005250.2910004-1-almasrymina@google.com/

Signed-off-by: Mina Almasry <almasrymina@google.com>

---

v3:

- Modify struct netmem from a union of struct page + new types to an opaque
  netmem_ref type.  I went with:

  +typedef void *__bitwise netmem_ref;

  rather than this that Jakub recommended:

  +typedef unsigned long __bitwise netmem_ref;

  Because with the latter the compiler issues warnings to cast NULL to
  netmem_ref. I hope that's ok.

- Add some function docs.

v2:

- Use container_of instead of a type cast (David).
---
 include/net/netmem.h | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
 create mode 100644 include/net/netmem.h

diff --git a/include/net/netmem.h b/include/net/netmem.h
new file mode 100644
index 000000000000..edd977326203
--- /dev/null
+++ b/include/net/netmem.h
@@ -0,0 +1,41 @@
+/* SPDX-License-Identifier: GPL-2.0
+ *
+ *	Network memory
+ *
+ *	Author:	Mina Almasry <almasrymina@google.com>
+ */
+
+#ifndef _NET_NETMEM_H
+#define _NET_NETMEM_H
+
+/**
+ * typedef netmem_ref - a nonexistent type marking a reference to generic
+ * network memory.
+ *
+ * A netmem_ref currently is always a reference to a struct page. This
+ * abstraction is introduced so support for new memory types can be added.
+ *
+ * Use the supplied helpers to obtain the underlying memory pointer and fields.
+ */
+typedef void *__bitwise netmem_ref;
+
+/* This conversion fails (returns NULL) if the netmem_ref is not struct page
+ * backed.
+ *
+ * Currently struct page is the only possible netmem, and this helper never
+ * fails.
+ */
+static inline struct page *netmem_to_page(netmem_ref netmem)
+{
+	return (struct page *)netmem;
+}
+
+/* Converting from page to netmem is always safe, because a page can always be
+ * a netmem.
+ */
+static inline netmem_ref page_to_netmem(struct page *page)
+{
+	return (netmem_ref)page;
+}
+
+#endif /* _NET_NETMEM_H */
-- 
2.43.0.472.g3155946c3a-goog

Re: [PATCH net-next v3 2/3] net: introduce abstraction for network memory

[Shakeel Butt]

On Wed, Dec 20, 2023 at 01:45:01PM -0800, Mina Almasry wrote:
> Add the netmem_ref type, an abstraction for network memory.
> 
> To add support for new memory types to the net stack, we must first
> abstract the current memory type. Currently parts of the net stack
> use struct page directly:
> 
> - page_pool
> - drivers
> - skb_frag_t
> 
> Originally the plan was to reuse struct page* for the new memory types,
> and to set the LSB on the page* to indicate it's not really a page.
> However, for compiler type checking we need to introduce a new type.
> 
> netmem_ref is introduced to abstract the underlying memory type. Currently
> it's a no-op abstraction that is always a struct page underneath. In
> parallel there is an undergoing effort to add support for devmem to the
> net stack:
> 
> https://lore.kernel.org/netdev/20231208005250.2910004-1-almasrymina@google.com/
> 
> Signed-off-by: Mina Almasry <almasrymina@google.com>
> 
> ---
> 
> v3:
> 
> - Modify struct netmem from a union of struct page + new types to an opaque
>   netmem_ref type.  I went with:
> 
>   +typedef void *__bitwise netmem_ref;
> 
>   rather than this that Jakub recommended:
> 
>   +typedef unsigned long __bitwise netmem_ref;
> 
>   Because with the latter the compiler issues warnings to cast NULL to
>   netmem_ref. I hope that's ok.
> 

Can you share what the warning was? You might just need __force
attribute. However you might need this __force a lot. I wonder if you
can just follow struct encoded_page example verbatim here.

Re: [PATCH net-next v3 2/3] net: introduce abstraction for network memory

[Mina Almasry]

On Thu, Dec 21, 2023 at 3:23 PM Shakeel Butt <shakeelb@google.com> wrote:
>
> On Wed, Dec 20, 2023 at 01:45:01PM -0800, Mina Almasry wrote:
> > Add the netmem_ref type, an abstraction for network memory.
> >
> > To add support for new memory types to the net stack, we must first
> > abstract the current memory type. Currently parts of the net stack
> > use struct page directly:
> >
> > - page_pool
> > - drivers
> > - skb_frag_t
> >
> > Originally the plan was to reuse struct page* for the new memory types,
> > and to set the LSB on the page* to indicate it's not really a page.
> > However, for compiler type checking we need to introduce a new type.
> >
> > netmem_ref is introduced to abstract the underlying memory type. Currently
> > it's a no-op abstraction that is always a struct page underneath. In
> > parallel there is an undergoing effort to add support for devmem to the
> > net stack:
> >
> > https://lore.kernel.org/netdev/20231208005250.2910004-1-almasrymina@google.com/
> >
> > Signed-off-by: Mina Almasry <almasrymina@google.com>
> >
> > ---
> >
> > v3:
> >
> > - Modify struct netmem from a union of struct page + new types to an opaque
> >   netmem_ref type.  I went with:
> >
> >   +typedef void *__bitwise netmem_ref;
> >
> >   rather than this that Jakub recommended:
> >
> >   +typedef unsigned long __bitwise netmem_ref;
> >
> >   Because with the latter the compiler issues warnings to cast NULL to
> >   netmem_ref. I hope that's ok.
> >
>
> Can you share what the warning was? You might just need __force
> attribute. However you might need this __force a lot. I wonder if you
> can just follow struct encoded_page example verbatim here.
>

The warning is like so:

./include/net/page_pool/helpers.h: In function ‘page_pool_alloc’:
./include/linux/stddef.h:8:14: warning: returning ‘void *’ from a
function with return type ‘netmem_ref’ {aka ‘long unsigned int’} makes
integer from pointer without a cast [-Wint-conversion]
    8 | #define NULL ((void *)0)
      |              ^
./include/net/page_pool/helpers.h:132:24: note: in expansion of macro
‘NULL’
  132 |                 return NULL;
      |                        ^~~~

And happens in all the code where:

netmem_ref func()
{
    return NULL;
}

It's fixable by changing the return to `return (netmem_ref NULL);` or
`return 0;`, but I feel like netmem_ref should be some type which
allows a cast from NULL implicitly.

Also as you (and patchwork) noticed, __bitwise should not be used with
void*; it's only meant for integer types. Sorry I missed that in the
docs and was not running make C=2.

-- 
Thanks,
Mina

Re: [PATCH net-next v3 2/3] net: introduce abstraction for network memory

[Jakub Kicinski]

On Thu, 21 Dec 2023 15:44:22 -0800 Mina Almasry wrote:
> The warning is like so:
> 
> ./include/net/page_pool/helpers.h: In function ‘page_pool_alloc’:
> ./include/linux/stddef.h:8:14: warning: returning ‘void *’ from a
> function with return type ‘netmem_ref’ {aka ‘long unsigned int’} makes
> integer from pointer without a cast [-Wint-conversion]
>     8 | #define NULL ((void *)0)
>       |              ^
> ./include/net/page_pool/helpers.h:132:24: note: in expansion of macro
> ‘NULL’
>   132 |                 return NULL;
>       |                        ^~~~
> 
> And happens in all the code where:
> 
> netmem_ref func()
> {
>     return NULL;
> }
> 
> It's fixable by changing the return to `return (netmem_ref NULL);` or
> `return 0;`, but I feel like netmem_ref should be some type which
> allows a cast from NULL implicitly.

Why do you think we should be able to cast NULL implicitly?
netmem_ref is a handle, it could possibly be some form of 
an ID in the future, rather than a pointer. Or have more low
bits stolen for specific use cases.

unsigned long, and returning 0 as "no handle" makes perfect sense to me.

Note that 0 is a special case, bitwise types are allowed to convert
to 0/bool and 0 is implicitly allowed to become a bitwise type.
This will pass without a warning:

typedef unsigned long __bitwise netmem_ref;

netmem_ref some_code(netmem_ref ref)
{
	// direct test is fine
	if (!ref)
		// 0 "upgrades" without casts
		return 0;
	// 1 does not, we need __force
	return (__force netmem_ref)1 | ref;
}

The __bitwise annotation will make catching people trying
to cast to struct page * trivial.

You seem to be trying hard to make struct netmem a thing.
Perhaps you have a reason I'm not getting?

Re: [PATCH net-next v3 2/3] net: introduce abstraction for network memory

[Mina Almasry]

On Thu, Jan 4, 2024 at 1:44 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Thu, 21 Dec 2023 15:44:22 -0800 Mina Almasry wrote:
> > The warning is like so:
> >
> > ./include/net/page_pool/helpers.h: In function ‘page_pool_alloc’:
> > ./include/linux/stddef.h:8:14: warning: returning ‘void *’ from a
> > function with return type ‘netmem_ref’ {aka ‘long unsigned int’} makes
> > integer from pointer without a cast [-Wint-conversion]
> >     8 | #define NULL ((void *)0)
> >       |              ^
> > ./include/net/page_pool/helpers.h:132:24: note: in expansion of macro
> > ‘NULL’
> >   132 |                 return NULL;
> >       |                        ^~~~
> >
> > And happens in all the code where:
> >
> > netmem_ref func()
> > {
> >     return NULL;
> > }
> >
> > It's fixable by changing the return to `return (netmem_ref NULL);` or
> > `return 0;`, but I feel like netmem_ref should be some type which
> > allows a cast from NULL implicitly.
>
> Why do you think we should be able to cast NULL implicitly?
> netmem_ref is a handle, it could possibly be some form of
> an ID in the future, rather than a pointer. Or have more low
> bits stolen for specific use cases.
>
> unsigned long, and returning 0 as "no handle" makes perfect sense to me.
>
> Note that 0 is a special case, bitwise types are allowed to convert
> to 0/bool and 0 is implicitly allowed to become a bitwise type.
> This will pass without a warning:
>
> typedef unsigned long __bitwise netmem_ref;
>
> netmem_ref some_code(netmem_ref ref)
> {
>         // direct test is fine
>         if (!ref)
>                 // 0 "upgrades" without casts
>                 return 0;
>         // 1 does not, we need __force
>         return (__force netmem_ref)1 | ref;
> }
>
> The __bitwise annotation will make catching people trying
> to cast to struct page * trivial.
>
> You seem to be trying hard to make struct netmem a thing.
> Perhaps you have a reason I'm not getting?

There are a number of functions that return struct page* today that I
convert to return struct netmem* later in the child devmem series, one
example is something like:

struct page *page_pool_alloc(...); // returns NULL on failure.

becomes:

struct netmem *page_pool_alloc(...); // also returns NULL on failure.

rather than,

netmem_ref page_pool_alloc(...); // returns 0 on failure.

I guess in my mind having NULL be castable to the new type makes it so
that I can avoid the additional code churn of converting a bunch of
`return NULL;` to `return 0;`, and maybe the transition from page
pointers to netmem pointers can be more easily done if they're both
compatible pointer types.

But that is not any huge blocker or critical point in my mind, I just
thought this approach is preferred. If conversion to unsigned long
makes more sense to you, I'll respin this like that and do the `NULL
-> 0` conversion everywhere as needed.

-- 
Thanks,
Mina

Re: [PATCH net-next v3 2/3] net: introduce abstraction for network memory

[Shakeel Butt]

On Thu, Jan 4, 2024 at 1:44 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
[...]
>
> You seem to be trying hard to make struct netmem a thing.
> Perhaps you have a reason I'm not getting?

Mina already went with your suggestion and that is fine. To me, struct
netmem is more aesthetically aligned with the existing struct
encoded_page approach, but I don't have a strong opinion one way or
the other. However it seems like you have a stronger preference for
__bitwise approach. Is there a technical reason or just aesthetic?

Re: [PATCH net-next v3 2/3] net: introduce abstraction for network memory

[Jakub Kicinski]

On Wed, 10 Jan 2024 09:50:08 -0800 Shakeel Butt wrote:
> On Thu, Jan 4, 2024 at 1:44 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > You seem to be trying hard to make struct netmem a thing.
> > Perhaps you have a reason I'm not getting?  
> 
> Mina already went with your suggestion and that is fine. To me, struct
> netmem is more aesthetically aligned with the existing struct
> encoded_page approach, but I don't have a strong opinion one way or
> the other. However it seems like you have a stronger preference for
> __bitwise approach. Is there a technical reason or just aesthetic?

Yes, right above the text you quoted:

  The __bitwise annotation will make catching people trying
  to cast to struct page * trivial.

https://lore.kernel.org/all/20240104134424.399fee0a@kernel.org/

[PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t

[Mina Almasry]

Use netmem_ref instead of page in skb_frag_t. Currently netmem_ref
is always a struct page underneath, but the abstraction allows efforts
to add support for skb frags not backed by pages.

There is unfortunately 1 instance where the skb_frag_t is assumed to be
a bio_vec in kcm. For this case, add a debug assert that the skb frag is
indeed backed by a page, and do a cast.

Add skb[_frag]_fill_netmem_*() and skb_add_rx_frag_netmem() helpers so
that the API can be used to create netmem skbs.

Signed-off-by: Mina Almasry <almasrymina@google.com>

---

v3;
- Renamed the fields in skb_frag_t.

v2:
- Add skb frag filling helpers.

---
 include/linux/skbuff.h | 92 +++++++++++++++++++++++++++++-------------
 net/core/skbuff.c      | 22 +++++++---
 net/kcm/kcmsock.c      | 10 ++++-
 3 files changed, 89 insertions(+), 35 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 7ce38874dbd1..729c95e97be1 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -37,6 +37,7 @@
 #endif
 #include <net/net_debug.h>
 #include <net/dropreason-core.h>
+#include <net/netmem.h>
 
 /**
  * DOC: skb checksums
@@ -359,7 +360,11 @@ extern int sysctl_max_skb_frags;
  */
 #define GSO_BY_FRAGS	0xFFFF
 
-typedef struct bio_vec skb_frag_t;
+typedef struct skb_frag {
+	netmem_ref netmem;
+	unsigned int len;
+	unsigned int offset;
+} skb_frag_t;
 
 /**
  * skb_frag_size() - Returns the size of a skb fragment
@@ -367,7 +372,7 @@ typedef struct bio_vec skb_frag_t;
  */
 static inline unsigned int skb_frag_size(const skb_frag_t *frag)
 {
-	return frag->bv_len;
+	return frag->len;
 }
 
 /**
@@ -377,7 +382,7 @@ static inline unsigned int skb_frag_size(const skb_frag_t *frag)
  */
 static inline void skb_frag_size_set(skb_frag_t *frag, unsigned int size)
 {
-	frag->bv_len = size;
+	frag->len = size;
 }
 
 /**
@@ -387,7 +392,7 @@ static inline void skb_frag_size_set(skb_frag_t *frag, unsigned int size)
  */
 static inline void skb_frag_size_add(skb_frag_t *frag, int delta)
 {
-	frag->bv_len += delta;
+	frag->len += delta;
 }
 
 /**
@@ -397,7 +402,7 @@ static inline void skb_frag_size_add(skb_frag_t *frag, int delta)
  */
 static inline void skb_frag_size_sub(skb_frag_t *frag, int delta)
 {
-	frag->bv_len -= delta;
+	frag->len -= delta;
 }
 
 /**
@@ -417,7 +422,7 @@ static inline bool skb_frag_must_loop(struct page *p)
  *	skb_frag_foreach_page - loop over pages in a fragment
  *
  *	@f:		skb frag to operate on
- *	@f_off:		offset from start of f->bv_page
+ *	@f_off:		offset from start of f->netmem
  *	@f_len:		length from f_off to loop over
  *	@p:		(temp var) current page
  *	@p_off:		(temp var) offset from start of current page,
@@ -2431,22 +2436,37 @@ static inline unsigned int skb_pagelen(const struct sk_buff *skb)
 	return skb_headlen(skb) + __skb_pagelen(skb);
 }
 
+static inline void skb_frag_fill_netmem_desc(skb_frag_t *frag,
+					     netmem_ref netmem, int off,
+					     int size)
+{
+	frag->netmem = netmem;
+	frag->offset = off;
+	skb_frag_size_set(frag, size);
+}
+
 static inline void skb_frag_fill_page_desc(skb_frag_t *frag,
 					   struct page *page,
 					   int off, int size)
 {
-	frag->bv_page = page;
-	frag->bv_offset = off;
-	skb_frag_size_set(frag, size);
+	skb_frag_fill_netmem_desc(frag, page_to_netmem(page), off, size);
+}
+
+static inline void __skb_fill_netmem_desc_noacc(struct skb_shared_info *shinfo,
+						int i, netmem_ref netmem,
+						int off, int size)
+{
+	skb_frag_t *frag = &shinfo->frags[i];
+
+	skb_frag_fill_netmem_desc(frag, netmem, off, size);
 }
 
 static inline void __skb_fill_page_desc_noacc(struct skb_shared_info *shinfo,
 					      int i, struct page *page,
 					      int off, int size)
 {
-	skb_frag_t *frag = &shinfo->frags[i];
-
-	skb_frag_fill_page_desc(frag, page, off, size);
+	__skb_fill_netmem_desc_noacc(shinfo, i, page_to_netmem(page), off,
+				     size);
 }
 
 /**
@@ -2462,10 +2482,10 @@ static inline void skb_len_add(struct sk_buff *skb, int delta)
 }
 
 /**
- * __skb_fill_page_desc - initialise a paged fragment in an skb
+ * __skb_fill_netmem_desc - initialise a fragment in an skb
  * @skb: buffer containing fragment to be initialised
- * @i: paged fragment index to initialise
- * @page: the page to use for this fragment
+ * @i: fragment index to initialise
+ * @netmem: the netmem to use for this fragment
  * @off: the offset to the data with @page
  * @size: the length of the data
  *
@@ -2474,10 +2494,13 @@ static inline void skb_len_add(struct sk_buff *skb, int delta)
  *
  * Does not take any additional reference on the fragment.
  */
-static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
-					struct page *page, int off, int size)
+static inline void __skb_fill_netmem_desc(struct sk_buff *skb, int i,
+					  netmem_ref netmem, int off,
+					  int size)
 {
-	__skb_fill_page_desc_noacc(skb_shinfo(skb), i, page, off, size);
+	struct page *page = netmem_to_page(netmem);
+
+	__skb_fill_netmem_desc_noacc(skb_shinfo(skb), i, netmem, off, size);
 
 	/* Propagate page pfmemalloc to the skb if we can. The problem is
 	 * that not all callers have unique ownership of the page but rely
@@ -2485,7 +2508,21 @@ static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
 	 */
 	page = compound_head(page);
 	if (page_is_pfmemalloc(page))
-		skb->pfmemalloc	= true;
+		skb->pfmemalloc = true;
+}
+
+static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
+					struct page *page, int off, int size)
+{
+	__skb_fill_netmem_desc(skb, i, page_to_netmem(page), off, size);
+}
+
+static inline void skb_fill_netmem_desc(struct sk_buff *skb, int i,
+					netmem_ref netmem, int off,
+					int size)
+{
+	__skb_fill_netmem_desc(skb, i, netmem, off, size);
+	skb_shinfo(skb)->nr_frags = i + 1;
 }
 
 /**
@@ -2505,8 +2542,7 @@ static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
 static inline void skb_fill_page_desc(struct sk_buff *skb, int i,
 				      struct page *page, int off, int size)
 {
-	__skb_fill_page_desc(skb, i, page, off, size);
-	skb_shinfo(skb)->nr_frags = i + 1;
+	skb_fill_netmem_desc(skb, i, page_to_netmem(page), off, size);
 }
 
 /**
@@ -2532,6 +2568,8 @@ static inline void skb_fill_page_desc_noacc(struct sk_buff *skb, int i,
 
 void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off,
 		     int size, unsigned int truesize);
+void skb_add_rx_frag_netmem(struct sk_buff *skb, int i, netmem_ref netmem,
+			    int off, int size, unsigned int truesize);
 
 void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int size,
 			  unsigned int truesize);
@@ -3380,7 +3418,7 @@ static inline void skb_propagate_pfmemalloc(const struct page *page,
  */
 static inline unsigned int skb_frag_off(const skb_frag_t *frag)
 {
-	return frag->bv_offset;
+	return frag->offset;
 }
 
 /**
@@ -3390,7 +3428,7 @@ static inline unsigned int skb_frag_off(const skb_frag_t *frag)
  */
 static inline void skb_frag_off_add(skb_frag_t *frag, int delta)
 {
-	frag->bv_offset += delta;
+	frag->offset += delta;
 }
 
 /**
@@ -3400,7 +3438,7 @@ static inline void skb_frag_off_add(skb_frag_t *frag, int delta)
  */
 static inline void skb_frag_off_set(skb_frag_t *frag, unsigned int offset)
 {
-	frag->bv_offset = offset;
+	frag->offset = offset;
 }
 
 /**
@@ -3411,7 +3449,7 @@ static inline void skb_frag_off_set(skb_frag_t *frag, unsigned int offset)
 static inline void skb_frag_off_copy(skb_frag_t *fragto,
 				     const skb_frag_t *fragfrom)
 {
-	fragto->bv_offset = fragfrom->bv_offset;
+	fragto->offset = fragfrom->offset;
 }
 
 /**
@@ -3422,7 +3460,7 @@ static inline void skb_frag_off_copy(skb_frag_t *fragto,
  */
 static inline struct page *skb_frag_page(const skb_frag_t *frag)
 {
-	return frag->bv_page;
+	return netmem_to_page(frag->netmem);
 }
 
 /**
@@ -3526,7 +3564,7 @@ static inline void *skb_frag_address_safe(const skb_frag_t *frag)
 static inline void skb_frag_page_copy(skb_frag_t *fragto,
 				      const skb_frag_t *fragfrom)
 {
-	fragto->bv_page = fragfrom->bv_page;
+	fragto->netmem = fragfrom->netmem;
 }
 
 bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t prio);
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 4d4b11b0a83d..8b55e927bbe9 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -845,16 +845,24 @@ struct sk_buff *__napi_alloc_skb(struct napi_struct *napi, unsigned int len,
 }
 EXPORT_SYMBOL(__napi_alloc_skb);
 
-void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off,
-		     int size, unsigned int truesize)
+void skb_add_rx_frag_netmem(struct sk_buff *skb, int i, netmem_ref netmem,
+			    int off, int size, unsigned int truesize)
 {
 	DEBUG_NET_WARN_ON_ONCE(size > truesize);
 
-	skb_fill_page_desc(skb, i, page, off, size);
+	skb_fill_netmem_desc(skb, i, netmem, off, size);
 	skb->len += size;
 	skb->data_len += size;
 	skb->truesize += truesize;
 }
+EXPORT_SYMBOL(skb_add_rx_frag_netmem);
+
+void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off,
+		     int size, unsigned int truesize)
+{
+	skb_add_rx_frag_netmem(skb, i, page_to_netmem(page), off, size,
+			       truesize);
+}
 EXPORT_SYMBOL(skb_add_rx_frag);
 
 void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int size,
@@ -1904,10 +1912,11 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask)
 
 	/* skb frags point to kernel buffers */
 	for (i = 0; i < new_frags - 1; i++) {
-		__skb_fill_page_desc(skb, i, head, 0, psize);
+		__skb_fill_netmem_desc(skb, i, page_to_netmem(head), 0, psize);
 		head = (struct page *)page_private(head);
 	}
-	__skb_fill_page_desc(skb, new_frags - 1, head, 0, d_off);
+	__skb_fill_netmem_desc(skb, new_frags - 1, page_to_netmem(head), 0,
+			       d_off);
 	skb_shinfo(skb)->nr_frags = new_frags;
 
 release:
@@ -3645,7 +3654,8 @@ skb_zerocopy(struct sk_buff *to, struct sk_buff *from, int len, int hlen)
 		if (plen) {
 			page = virt_to_head_page(from->head);
 			offset = from->data - (unsigned char *)page_address(page);
-			__skb_fill_page_desc(to, 0, page, offset, plen);
+			__skb_fill_netmem_desc(to, 0, page_to_netmem(page),
+					       offset, plen);
 			get_page(page);
 			j = 1;
 			len -= plen;
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index 65d1f6755f98..3180a54b2c68 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -636,9 +636,15 @@ static int kcm_write_msgs(struct kcm_sock *kcm)
 		for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
 			msize += skb_shinfo(skb)->frags[i].bv_len;
 
+		/* The cast to struct bio_vec* here assumes the frags are
+		 * struct page based. WARN if there is no page in this skb.
+		 */
+		DEBUG_NET_WARN_ON_ONCE(
+			!skb_frag_page(&skb_shinfo(skb)->frags[0]));
+
 		iov_iter_bvec(&msg.msg_iter, ITER_SOURCE,
-			      skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags,
-			      msize);
+			      (const struct bio_vec *)skb_shinfo(skb)->frags,
+			      skb_shinfo(skb)->nr_frags, msize);
 		iov_iter_advance(&msg.msg_iter, txm->frag_offset);
 
 		do {
-- 
2.43.0.472.g3155946c3a-goog

Re: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t

[Simon Horman]

On Wed, Dec 20, 2023 at 01:45:02PM -0800, Mina Almasry wrote:
> Use netmem_ref instead of page in skb_frag_t. Currently netmem_ref
> is always a struct page underneath, but the abstraction allows efforts
> to add support for skb frags not backed by pages.
> 
> There is unfortunately 1 instance where the skb_frag_t is assumed to be
> a bio_vec in kcm. For this case, add a debug assert that the skb frag is
> indeed backed by a page, and do a cast.
> 
> Add skb[_frag]_fill_netmem_*() and skb_add_rx_frag_netmem() helpers so
> that the API can be used to create netmem skbs.
> 
> Signed-off-by: Mina Almasry <almasrymina@google.com>

...

> diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
> index 65d1f6755f98..3180a54b2c68 100644
> --- a/net/kcm/kcmsock.c
> +++ b/net/kcm/kcmsock.c
> @@ -636,9 +636,15 @@ static int kcm_write_msgs(struct kcm_sock *kcm)
>  		for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
>  			msize += skb_shinfo(skb)->frags[i].bv_len;
>  
> +		/* The cast to struct bio_vec* here assumes the frags are
> +		 * struct page based. WARN if there is no page in this skb.
> +		 */
> +		DEBUG_NET_WARN_ON_ONCE(
> +			!skb_frag_page(&skb_shinfo(skb)->frags[0]));
> +
>  		iov_iter_bvec(&msg.msg_iter, ITER_SOURCE,
> -			      skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags,
> -			      msize);
> +			      (const struct bio_vec *)skb_shinfo(skb)->frags,
> +			      skb_shinfo(skb)->nr_frags, msize);
>  		iov_iter_advance(&msg.msg_iter, txm->frag_offset);
>  
>  		do {

Hi Mina,

something isn't quite right here.

  ...//kcmsock.c:637:39: error: no member named 'bv_len' in 'struct skb_frag'
  637 |                         msize += skb_shinfo(skb)->frags[i].bv_len;
      |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~ ^

-- 
pw-bot: changes-requested

Re: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t

[Willem de Bruijn]

Mina Almasry wrote:
> Use netmem_ref instead of page in skb_frag_t. Currently netmem_ref
> is always a struct page underneath, but the abstraction allows efforts
> to add support for skb frags not backed by pages.
> 
> There is unfortunately 1 instance where the skb_frag_t is assumed to be
> a bio_vec in kcm. For this case, add a debug assert that the skb frag is
> indeed backed by a page, and do a cast.
> 
> Add skb[_frag]_fill_netmem_*() and skb_add_rx_frag_netmem() helpers so
> that the API can be used to create netmem skbs.
> 
> Signed-off-by: Mina Almasry <almasrymina@google.com>
> 
> ---
> 
> v3;
> - Renamed the fields in skb_frag_t.
> 
> v2:
> - Add skb frag filling helpers.
> 
> ---
>  include/linux/skbuff.h | 92 +++++++++++++++++++++++++++++-------------
>  net/core/skbuff.c      | 22 +++++++---
>  net/kcm/kcmsock.c      | 10 ++++-
>  3 files changed, 89 insertions(+), 35 deletions(-)
> 
> diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
> index 7ce38874dbd1..729c95e97be1 100644
> --- a/include/linux/skbuff.h
> +++ b/include/linux/skbuff.h
> @@ -37,6 +37,7 @@
>  #endif
>  #include <net/net_debug.h>
>  #include <net/dropreason-core.h>
> +#include <net/netmem.h>
>  
>  /**
>   * DOC: skb checksums
> @@ -359,7 +360,11 @@ extern int sysctl_max_skb_frags;
>   */
>  #define GSO_BY_FRAGS	0xFFFF
>  
> -typedef struct bio_vec skb_frag_t;
> +typedef struct skb_frag {
> +	netmem_ref netmem;
> +	unsigned int len;
> +	unsigned int offset;
> +} skb_frag_t;
>  
>  /**
>   * skb_frag_size() - Returns the size of a skb fragment
> @@ -367,7 +372,7 @@ typedef struct bio_vec skb_frag_t;
>   */
>  static inline unsigned int skb_frag_size(const skb_frag_t *frag)
>  {
> -	return frag->bv_len;
> +	return frag->len;
>  }
>  
>  /**
> @@ -377,7 +382,7 @@ static inline unsigned int skb_frag_size(const skb_frag_t *frag)
>   */
>  static inline void skb_frag_size_set(skb_frag_t *frag, unsigned int size)
>  {
> -	frag->bv_len = size;
> +	frag->len = size;
>  }
>  
>  /**
> @@ -387,7 +392,7 @@ static inline void skb_frag_size_set(skb_frag_t *frag, unsigned int size)
>   */
>  static inline void skb_frag_size_add(skb_frag_t *frag, int delta)
>  {
> -	frag->bv_len += delta;
> +	frag->len += delta;
>  }
>  
>  /**
> @@ -397,7 +402,7 @@ static inline void skb_frag_size_add(skb_frag_t *frag, int delta)
>   */
>  static inline void skb_frag_size_sub(skb_frag_t *frag, int delta)
>  {
> -	frag->bv_len -= delta;
> +	frag->len -= delta;
>  }
>  
>  /**
> @@ -417,7 +422,7 @@ static inline bool skb_frag_must_loop(struct page *p)
>   *	skb_frag_foreach_page - loop over pages in a fragment
>   *
>   *	@f:		skb frag to operate on
> - *	@f_off:		offset from start of f->bv_page
> + *	@f_off:		offset from start of f->netmem
>   *	@f_len:		length from f_off to loop over
>   *	@p:		(temp var) current page
>   *	@p_off:		(temp var) offset from start of current page,
> @@ -2431,22 +2436,37 @@ static inline unsigned int skb_pagelen(const struct sk_buff *skb)
>  	return skb_headlen(skb) + __skb_pagelen(skb);
>  }
>  
> +static inline void skb_frag_fill_netmem_desc(skb_frag_t *frag,
> +					     netmem_ref netmem, int off,
> +					     int size)
> +{
> +	frag->netmem = netmem;
> +	frag->offset = off;
> +	skb_frag_size_set(frag, size);
> +}
> +
>  static inline void skb_frag_fill_page_desc(skb_frag_t *frag,
>  					   struct page *page,
>  					   int off, int size)
>  {
> -	frag->bv_page = page;
> -	frag->bv_offset = off;
> -	skb_frag_size_set(frag, size);
> +	skb_frag_fill_netmem_desc(frag, page_to_netmem(page), off, size);
> +}
> +
> +static inline void __skb_fill_netmem_desc_noacc(struct skb_shared_info *shinfo,
> +						int i, netmem_ref netmem,
> +						int off, int size)
> +{
> +	skb_frag_t *frag = &shinfo->frags[i];
> +
> +	skb_frag_fill_netmem_desc(frag, netmem, off, size);
>  }
>  
>  static inline void __skb_fill_page_desc_noacc(struct skb_shared_info *shinfo,
>  					      int i, struct page *page,
>  					      int off, int size)
>  {
> -	skb_frag_t *frag = &shinfo->frags[i];
> -
> -	skb_frag_fill_page_desc(frag, page, off, size);
> +	__skb_fill_netmem_desc_noacc(shinfo, i, page_to_netmem(page), off,
> +				     size);
>  }
>  
>  /**
> @@ -2462,10 +2482,10 @@ static inline void skb_len_add(struct sk_buff *skb, int delta)
>  }
>  
>  /**
> - * __skb_fill_page_desc - initialise a paged fragment in an skb
> + * __skb_fill_netmem_desc - initialise a fragment in an skb
>   * @skb: buffer containing fragment to be initialised
> - * @i: paged fragment index to initialise
> - * @page: the page to use for this fragment
> + * @i: fragment index to initialise
> + * @netmem: the netmem to use for this fragment
>   * @off: the offset to the data with @page
>   * @size: the length of the data
>   *
> @@ -2474,10 +2494,13 @@ static inline void skb_len_add(struct sk_buff *skb, int delta)
>   *
>   * Does not take any additional reference on the fragment.
>   */
> -static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
> -					struct page *page, int off, int size)
> +static inline void __skb_fill_netmem_desc(struct sk_buff *skb, int i,
> +					  netmem_ref netmem, int off,
> +					  int size)
>  {
> -	__skb_fill_page_desc_noacc(skb_shinfo(skb), i, page, off, size);
> +	struct page *page = netmem_to_page(netmem);
> +
> +	__skb_fill_netmem_desc_noacc(skb_shinfo(skb), i, netmem, off, size);
>  
>  	/* Propagate page pfmemalloc to the skb if we can. The problem is
>  	 * that not all callers have unique ownership of the page but rely
> @@ -2485,7 +2508,21 @@ static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
>  	 */
>  	page = compound_head(page);
>  	if (page_is_pfmemalloc(page))
> -		skb->pfmemalloc	= true;
> +		skb->pfmemalloc = true;
> +}
> +
> +static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
> +					struct page *page, int off, int size)
> +{
> +	__skb_fill_netmem_desc(skb, i, page_to_netmem(page), off, size);
> +}
> +
> +static inline void skb_fill_netmem_desc(struct sk_buff *skb, int i,
> +					netmem_ref netmem, int off,
> +					int size)
> +{
> +	__skb_fill_netmem_desc(skb, i, netmem, off, size);
> +	skb_shinfo(skb)->nr_frags = i + 1;
>  }
>  
>  /**
> @@ -2505,8 +2542,7 @@ static inline void __skb_fill_page_desc(struct sk_buff *skb, int i,
>  static inline void skb_fill_page_desc(struct sk_buff *skb, int i,
>  				      struct page *page, int off, int size)
>  {
> -	__skb_fill_page_desc(skb, i, page, off, size);
> -	skb_shinfo(skb)->nr_frags = i + 1;
> +	skb_fill_netmem_desc(skb, i, page_to_netmem(page), off, size);
>  }
>  
>  /**
> @@ -2532,6 +2568,8 @@ static inline void skb_fill_page_desc_noacc(struct sk_buff *skb, int i,
>  
>  void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off,
>  		     int size, unsigned int truesize);
> +void skb_add_rx_frag_netmem(struct sk_buff *skb, int i, netmem_ref netmem,
> +			    int off, int size, unsigned int truesize);
>  
>  void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int size,
>  			  unsigned int truesize);
> @@ -3380,7 +3418,7 @@ static inline void skb_propagate_pfmemalloc(const struct page *page,
>   */
>  static inline unsigned int skb_frag_off(const skb_frag_t *frag)
>  {
> -	return frag->bv_offset;
> +	return frag->offset;
>  }
>  
>  /**
> @@ -3390,7 +3428,7 @@ static inline unsigned int skb_frag_off(const skb_frag_t *frag)
>   */
>  static inline void skb_frag_off_add(skb_frag_t *frag, int delta)
>  {
> -	frag->bv_offset += delta;
> +	frag->offset += delta;
>  }
>  
>  /**
> @@ -3400,7 +3438,7 @@ static inline void skb_frag_off_add(skb_frag_t *frag, int delta)
>   */
>  static inline void skb_frag_off_set(skb_frag_t *frag, unsigned int offset)
>  {
> -	frag->bv_offset = offset;
> +	frag->offset = offset;
>  }
>  
>  /**
> @@ -3411,7 +3449,7 @@ static inline void skb_frag_off_set(skb_frag_t *frag, unsigned int offset)
>  static inline void skb_frag_off_copy(skb_frag_t *fragto,
>  				     const skb_frag_t *fragfrom)
>  {
> -	fragto->bv_offset = fragfrom->bv_offset;
> +	fragto->offset = fragfrom->offset;
>  }
>  
>  /**
> @@ -3422,7 +3460,7 @@ static inline void skb_frag_off_copy(skb_frag_t *fragto,
>   */
>  static inline struct page *skb_frag_page(const skb_frag_t *frag)
>  {
> -	return frag->bv_page;
> +	return netmem_to_page(frag->netmem);
>  }
>  
>  /**
> @@ -3526,7 +3564,7 @@ static inline void *skb_frag_address_safe(const skb_frag_t *frag)
>  static inline void skb_frag_page_copy(skb_frag_t *fragto,
>  				      const skb_frag_t *fragfrom)
>  {
> -	fragto->bv_page = fragfrom->bv_page;
> +	fragto->netmem = fragfrom->netmem;
>  }
>  
>  bool skb_page_frag_refill(unsigned int sz, struct page_frag *pfrag, gfp_t prio);
> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
> index 4d4b11b0a83d..8b55e927bbe9 100644
> --- a/net/core/skbuff.c
> +++ b/net/core/skbuff.c
> @@ -845,16 +845,24 @@ struct sk_buff *__napi_alloc_skb(struct napi_struct *napi, unsigned int len,
>  }
>  EXPORT_SYMBOL(__napi_alloc_skb);
>  
> -void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off,
> -		     int size, unsigned int truesize)
> +void skb_add_rx_frag_netmem(struct sk_buff *skb, int i, netmem_ref netmem,
> +			    int off, int size, unsigned int truesize)
>  {
>  	DEBUG_NET_WARN_ON_ONCE(size > truesize);
>  
> -	skb_fill_page_desc(skb, i, page, off, size);
> +	skb_fill_netmem_desc(skb, i, netmem, off, size);
>  	skb->len += size;
>  	skb->data_len += size;
>  	skb->truesize += truesize;
>  }
> +EXPORT_SYMBOL(skb_add_rx_frag_netmem);
> +
> +void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off,
> +		     int size, unsigned int truesize)
> +{
> +	skb_add_rx_frag_netmem(skb, i, page_to_netmem(page), off, size,
> +			       truesize);
> +}
>  EXPORT_SYMBOL(skb_add_rx_frag);
>  
>  void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int size,
> @@ -1904,10 +1912,11 @@ int skb_copy_ubufs(struct sk_buff *skb, gfp_t gfp_mask)
>  
>  	/* skb frags point to kernel buffers */
>  	for (i = 0; i < new_frags - 1; i++) {
> -		__skb_fill_page_desc(skb, i, head, 0, psize);
> +		__skb_fill_netmem_desc(skb, i, page_to_netmem(head), 0, psize);
>  		head = (struct page *)page_private(head);
>  	}
> -	__skb_fill_page_desc(skb, new_frags - 1, head, 0, d_off);
> +	__skb_fill_netmem_desc(skb, new_frags - 1, page_to_netmem(head), 0,
> +			       d_off);
>  	skb_shinfo(skb)->nr_frags = new_frags;
>  
>  release:
> @@ -3645,7 +3654,8 @@ skb_zerocopy(struct sk_buff *to, struct sk_buff *from, int len, int hlen)
>  		if (plen) {
>  			page = virt_to_head_page(from->head);
>  			offset = from->data - (unsigned char *)page_address(page);
> -			__skb_fill_page_desc(to, 0, page, offset, plen);
> +			__skb_fill_netmem_desc(to, 0, page_to_netmem(page),
> +					       offset, plen);
>  			get_page(page);
>  			j = 1;
>  			len -= plen;
> diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
> index 65d1f6755f98..3180a54b2c68 100644
> --- a/net/kcm/kcmsock.c
> +++ b/net/kcm/kcmsock.c
> @@ -636,9 +636,15 @@ static int kcm_write_msgs(struct kcm_sock *kcm)
>  		for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
>  			msize += skb_shinfo(skb)->frags[i].bv_len;
>  
> +		/* The cast to struct bio_vec* here assumes the frags are
> +		 * struct page based. WARN if there is no page in this skb.
> +		 */
> +		DEBUG_NET_WARN_ON_ONCE(
> +			!skb_frag_page(&skb_shinfo(skb)->frags[0]));
> +

It would be unsafe to continue the operation in this case. Even though
we should never get here, test and exit in all codepaths, similar to
other test above?

                if (WARN_ON(!skb_shinfo(skb)->nr_frags)) {
                        ret = -EINVAL;
                        goto out;
                }

>  		iov_iter_bvec(&msg.msg_iter, ITER_SOURCE,
> -			      skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags,
> -			      msize);
> +			      (const struct bio_vec *)skb_shinfo(skb)->frags,
> +			      skb_shinfo(skb)->nr_frags, msize);
>  		iov_iter_advance(&msg.msg_iter, txm->frag_offset);
>  
>  		do {
> -- 
> 2.43.0.472.g3155946c3a-goog
> 

Re: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t

[Shakeel Butt]

On Wed, Dec 20, 2023 at 01:45:02PM -0800, Mina Almasry wrote:
> diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
> index 65d1f6755f98..3180a54b2c68 100644
> --- a/net/kcm/kcmsock.c
> +++ b/net/kcm/kcmsock.c
> @@ -636,9 +636,15 @@ static int kcm_write_msgs(struct kcm_sock *kcm)
>  		for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
>  			msize += skb_shinfo(skb)->frags[i].bv_len;

Don't you need the above to cast to bio_vec to get bv_len? skb_frag_t
does not have bv_len anymore.

>  
> +		/* The cast to struct bio_vec* here assumes the frags are
> +		 * struct page based. WARN if there is no page in this skb.
> +		 */
> +		DEBUG_NET_WARN_ON_ONCE(
> +			!skb_frag_page(&skb_shinfo(skb)->frags[0]));
> +
>  		iov_iter_bvec(&msg.msg_iter, ITER_SOURCE,
> -			      skb_shinfo(skb)->frags, skb_shinfo(skb)->nr_frags,
> -			      msize);
> +			      (const struct bio_vec *)skb_shinfo(skb)->frags,
> +			      skb_shinfo(skb)->nr_frags, msize);
>  		iov_iter_advance(&msg.msg_iter, txm->frag_offset);
>  
>  		do {
> -- 
> 2.43.0.472.g3155946c3a-goog
> 

Re: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t

[kernel test robot]

Hi Mina,

kernel test robot noticed the following build errors:

[auto build test ERROR on net-next/main]

url:    https://github.com/intel-lab-lkp/linux/commits/Mina-Almasry/vsock-virtio-use-skb_frag_-helpers/20231222-164637
base:   net-next/main
patch link:    https://lore.kernel.org/r/20231220214505.2303297-4-almasrymina%40google.com
patch subject: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t
config: powerpc-allmodconfig (https://download.01.org/0day-ci/archive/20231223/202312230340.iCf8sOop-lkp@intel.com/config)
compiler: clang version 18.0.0git (https://github.com/llvm/llvm-project d3ef86708241a3bee902615c190dead1638c4e09)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231223/202312230340.iCf8sOop-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202312230340.iCf8sOop-lkp@intel.com/

All errors (new ones prefixed by >>):

>> net/kcm/kcmsock.c:637:39: error: no member named 'bv_len' in 'struct skb_frag'
     637 |                         msize += skb_shinfo(skb)->frags[i].bv_len;
         |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~ ^
   1 error generated.


vim +637 net/kcm/kcmsock.c

cd6e111bf5be5c Tom Herbert       2016-03-07  578  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  579  /* Write any messages ready on the kcm socket.  Called with kcm sock lock
ab7ac4eb9832e3 Tom Herbert       2016-03-07  580   * held.  Return bytes actually sent or error.
ab7ac4eb9832e3 Tom Herbert       2016-03-07  581   */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  582  static int kcm_write_msgs(struct kcm_sock *kcm)
ab7ac4eb9832e3 Tom Herbert       2016-03-07  583  {
c31a25e1db486f David Howells     2023-06-09  584  	unsigned int total_sent = 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  585  	struct sock *sk = &kcm->sk;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  586  	struct kcm_psock *psock;
c31a25e1db486f David Howells     2023-06-09  587  	struct sk_buff *head;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  588  	int ret = 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  589  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  590  	kcm->tx_wait_more = false;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  591  	psock = kcm->tx_psock;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  592  	if (unlikely(psock && psock->tx_stopped)) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  593  		/* A reserved psock was aborted asynchronously. Unreserve
ab7ac4eb9832e3 Tom Herbert       2016-03-07  594  		 * it and we'll retry the message.
ab7ac4eb9832e3 Tom Herbert       2016-03-07  595  		 */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  596  		unreserve_psock(kcm);
cd6e111bf5be5c Tom Herbert       2016-03-07  597  		kcm_report_tx_retry(kcm);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  598  		if (skb_queue_empty(&sk->sk_write_queue))
ab7ac4eb9832e3 Tom Herbert       2016-03-07  599  			return 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  600  
c31a25e1db486f David Howells     2023-06-09  601  		kcm_tx_msg(skb_peek(&sk->sk_write_queue))->started_tx = false;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  602  	}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  603  
c31a25e1db486f David Howells     2023-06-09  604  retry:
c31a25e1db486f David Howells     2023-06-09  605  	while ((head = skb_peek(&sk->sk_write_queue))) {
c31a25e1db486f David Howells     2023-06-09  606  		struct msghdr msg = {
c31a25e1db486f David Howells     2023-06-09  607  			.msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES,
c31a25e1db486f David Howells     2023-06-09  608  		};
c31a25e1db486f David Howells     2023-06-09  609  		struct kcm_tx_msg *txm = kcm_tx_msg(head);
c31a25e1db486f David Howells     2023-06-09  610  		struct sk_buff *skb;
c31a25e1db486f David Howells     2023-06-09  611  		unsigned int msize;
c31a25e1db486f David Howells     2023-06-09  612  		int i;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  613  
c31a25e1db486f David Howells     2023-06-09  614  		if (!txm->started_tx) {
c31a25e1db486f David Howells     2023-06-09  615  			psock = reserve_psock(kcm);
c31a25e1db486f David Howells     2023-06-09  616  			if (!psock)
c31a25e1db486f David Howells     2023-06-09  617  				goto out;
c31a25e1db486f David Howells     2023-06-09  618  			skb = head;
c31a25e1db486f David Howells     2023-06-09  619  			txm->frag_offset = 0;
c31a25e1db486f David Howells     2023-06-09  620  			txm->sent = 0;
c31a25e1db486f David Howells     2023-06-09  621  			txm->started_tx = true;
c31a25e1db486f David Howells     2023-06-09  622  		} else {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  623  			if (WARN_ON(!psock)) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  624  				ret = -EINVAL;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  625  				goto out;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  626  			}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  627  			skb = txm->frag_skb;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  628  		}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  629  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  630  		if (WARN_ON(!skb_shinfo(skb)->nr_frags)) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  631  			ret = -EINVAL;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  632  			goto out;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  633  		}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  634  
c31a25e1db486f David Howells     2023-06-09  635  		msize = 0;
c31a25e1db486f David Howells     2023-06-09  636  		for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
c31a25e1db486f David Howells     2023-06-09 @637  			msize += skb_shinfo(skb)->frags[i].bv_len;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  638  
b2e5852793b6eb Mina Almasry      2023-12-20  639  		/* The cast to struct bio_vec* here assumes the frags are
b2e5852793b6eb Mina Almasry      2023-12-20  640  		 * struct page based. WARN if there is no page in this skb.
b2e5852793b6eb Mina Almasry      2023-12-20  641  		 */
b2e5852793b6eb Mina Almasry      2023-12-20  642  		DEBUG_NET_WARN_ON_ONCE(
b2e5852793b6eb Mina Almasry      2023-12-20  643  			!skb_frag_page(&skb_shinfo(skb)->frags[0]));
b2e5852793b6eb Mina Almasry      2023-12-20  644  
c31a25e1db486f David Howells     2023-06-09  645  		iov_iter_bvec(&msg.msg_iter, ITER_SOURCE,
b2e5852793b6eb Mina Almasry      2023-12-20  646  			      (const struct bio_vec *)skb_shinfo(skb)->frags,
b2e5852793b6eb Mina Almasry      2023-12-20  647  			      skb_shinfo(skb)->nr_frags, msize);
c31a25e1db486f David Howells     2023-06-09  648  		iov_iter_advance(&msg.msg_iter, txm->frag_offset);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  649  
c31a25e1db486f David Howells     2023-06-09  650  		do {
264ba53fac79b0 David Howells     2023-06-09  651  			ret = sock_sendmsg(psock->sk->sk_socket, &msg);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  652  			if (ret <= 0) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  653  				if (ret == -EAGAIN) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  654  					/* Save state to try again when there's
ab7ac4eb9832e3 Tom Herbert       2016-03-07  655  					 * write space on the socket
ab7ac4eb9832e3 Tom Herbert       2016-03-07  656  					 */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  657  					txm->frag_skb = skb;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  658  					ret = 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  659  					goto out;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  660  				}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  661  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  662  				/* Hard failure in sending message, abort this
ab7ac4eb9832e3 Tom Herbert       2016-03-07  663  				 * psock since it has lost framing
71a2fae50895b3 Bhaskar Chowdhury 2021-03-27  664  				 * synchronization and retry sending the
ab7ac4eb9832e3 Tom Herbert       2016-03-07  665  				 * message from the beginning.
ab7ac4eb9832e3 Tom Herbert       2016-03-07  666  				 */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  667  				kcm_abort_tx_psock(psock, ret ? -ret : EPIPE,
ab7ac4eb9832e3 Tom Herbert       2016-03-07  668  						   true);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  669  				unreserve_psock(kcm);
9f8d0dc0ec4a4b David Howells     2023-06-15  670  				psock = NULL;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  671  
c31a25e1db486f David Howells     2023-06-09  672  				txm->started_tx = false;
cd6e111bf5be5c Tom Herbert       2016-03-07  673  				kcm_report_tx_retry(kcm);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  674  				ret = 0;
c31a25e1db486f David Howells     2023-06-09  675  				goto retry;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  676  			}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  677  
c31a25e1db486f David Howells     2023-06-09  678  			txm->sent += ret;
c31a25e1db486f David Howells     2023-06-09  679  			txm->frag_offset += ret;
cd6e111bf5be5c Tom Herbert       2016-03-07  680  			KCM_STATS_ADD(psock->stats.tx_bytes, ret);
c31a25e1db486f David Howells     2023-06-09  681  		} while (msg.msg_iter.count > 0);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  682  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  683  		if (skb == head) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  684  			if (skb_has_frag_list(skb)) {
c31a25e1db486f David Howells     2023-06-09  685  				txm->frag_skb = skb_shinfo(skb)->frag_list;
c31a25e1db486f David Howells     2023-06-09  686  				txm->frag_offset = 0;
c31a25e1db486f David Howells     2023-06-09  687  				continue;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  688  			}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  689  		} else if (skb->next) {
c31a25e1db486f David Howells     2023-06-09  690  			txm->frag_skb = skb->next;
c31a25e1db486f David Howells     2023-06-09  691  			txm->frag_offset = 0;
c31a25e1db486f David Howells     2023-06-09  692  			continue;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  693  		}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  694  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  695  		/* Successfully sent the whole packet, account for it. */
c31a25e1db486f David Howells     2023-06-09  696  		sk->sk_wmem_queued -= txm->sent;
c31a25e1db486f David Howells     2023-06-09  697  		total_sent += txm->sent;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  698  		skb_dequeue(&sk->sk_write_queue);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  699  		kfree_skb(head);
cd6e111bf5be5c Tom Herbert       2016-03-07  700  		KCM_STATS_INCR(psock->stats.tx_msgs);
c31a25e1db486f David Howells     2023-06-09  701  	}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  702  out:
ab7ac4eb9832e3 Tom Herbert       2016-03-07  703  	if (!head) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  704  		/* Done with all queued messages. */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  705  		WARN_ON(!skb_queue_empty(&sk->sk_write_queue));
9f8d0dc0ec4a4b David Howells     2023-06-15  706  		if (psock)
ab7ac4eb9832e3 Tom Herbert       2016-03-07  707  			unreserve_psock(kcm);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  708  	}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  709  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  710  	/* Check if write space is available */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  711  	sk->sk_write_space(sk);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  712  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  713  	return total_sent ? : ret;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  714  }
ab7ac4eb9832e3 Tom Herbert       2016-03-07  715  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t

[kernel test robot]

Hi Mina,

kernel test robot noticed the following build errors:

[auto build test ERROR on net-next/main]

url:    https://github.com/intel-lab-lkp/linux/commits/Mina-Almasry/vsock-virtio-use-skb_frag_-helpers/20231222-164637
base:   net-next/main
patch link:    https://lore.kernel.org/r/20231220214505.2303297-4-almasrymina%40google.com
patch subject: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t
config: i386-randconfig-141-20231222 (https://download.01.org/0day-ci/archive/20231223/202312230739.g0Tfssdt-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231223/202312230739.g0Tfssdt-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202312230739.g0Tfssdt-lkp@intel.com/

All errors (new ones prefixed by >>):

   net/kcm/kcmsock.c: In function 'kcm_write_msgs':
>> net/kcm/kcmsock.c:637:59: error: 'skb_frag_t' {aka 'struct skb_frag'} has no member named 'bv_len'
     637 |                         msize += skb_shinfo(skb)->frags[i].bv_len;
         |                                                           ^


vim +637 net/kcm/kcmsock.c

cd6e111bf5be5c Tom Herbert       2016-03-07  578  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  579  /* Write any messages ready on the kcm socket.  Called with kcm sock lock
ab7ac4eb9832e3 Tom Herbert       2016-03-07  580   * held.  Return bytes actually sent or error.
ab7ac4eb9832e3 Tom Herbert       2016-03-07  581   */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  582  static int kcm_write_msgs(struct kcm_sock *kcm)
ab7ac4eb9832e3 Tom Herbert       2016-03-07  583  {
c31a25e1db486f David Howells     2023-06-09  584  	unsigned int total_sent = 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  585  	struct sock *sk = &kcm->sk;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  586  	struct kcm_psock *psock;
c31a25e1db486f David Howells     2023-06-09  587  	struct sk_buff *head;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  588  	int ret = 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  589  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  590  	kcm->tx_wait_more = false;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  591  	psock = kcm->tx_psock;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  592  	if (unlikely(psock && psock->tx_stopped)) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  593  		/* A reserved psock was aborted asynchronously. Unreserve
ab7ac4eb9832e3 Tom Herbert       2016-03-07  594  		 * it and we'll retry the message.
ab7ac4eb9832e3 Tom Herbert       2016-03-07  595  		 */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  596  		unreserve_psock(kcm);
cd6e111bf5be5c Tom Herbert       2016-03-07  597  		kcm_report_tx_retry(kcm);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  598  		if (skb_queue_empty(&sk->sk_write_queue))
ab7ac4eb9832e3 Tom Herbert       2016-03-07  599  			return 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  600  
c31a25e1db486f David Howells     2023-06-09  601  		kcm_tx_msg(skb_peek(&sk->sk_write_queue))->started_tx = false;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  602  	}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  603  
c31a25e1db486f David Howells     2023-06-09  604  retry:
c31a25e1db486f David Howells     2023-06-09  605  	while ((head = skb_peek(&sk->sk_write_queue))) {
c31a25e1db486f David Howells     2023-06-09  606  		struct msghdr msg = {
c31a25e1db486f David Howells     2023-06-09  607  			.msg_flags = MSG_DONTWAIT | MSG_SPLICE_PAGES,
c31a25e1db486f David Howells     2023-06-09  608  		};
c31a25e1db486f David Howells     2023-06-09  609  		struct kcm_tx_msg *txm = kcm_tx_msg(head);
c31a25e1db486f David Howells     2023-06-09  610  		struct sk_buff *skb;
c31a25e1db486f David Howells     2023-06-09  611  		unsigned int msize;
c31a25e1db486f David Howells     2023-06-09  612  		int i;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  613  
c31a25e1db486f David Howells     2023-06-09  614  		if (!txm->started_tx) {
c31a25e1db486f David Howells     2023-06-09  615  			psock = reserve_psock(kcm);
c31a25e1db486f David Howells     2023-06-09  616  			if (!psock)
c31a25e1db486f David Howells     2023-06-09  617  				goto out;
c31a25e1db486f David Howells     2023-06-09  618  			skb = head;
c31a25e1db486f David Howells     2023-06-09  619  			txm->frag_offset = 0;
c31a25e1db486f David Howells     2023-06-09  620  			txm->sent = 0;
c31a25e1db486f David Howells     2023-06-09  621  			txm->started_tx = true;
c31a25e1db486f David Howells     2023-06-09  622  		} else {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  623  			if (WARN_ON(!psock)) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  624  				ret = -EINVAL;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  625  				goto out;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  626  			}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  627  			skb = txm->frag_skb;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  628  		}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  629  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  630  		if (WARN_ON(!skb_shinfo(skb)->nr_frags)) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  631  			ret = -EINVAL;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  632  			goto out;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  633  		}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  634  
c31a25e1db486f David Howells     2023-06-09  635  		msize = 0;
c31a25e1db486f David Howells     2023-06-09  636  		for (i = 0; i < skb_shinfo(skb)->nr_frags; i++)
c31a25e1db486f David Howells     2023-06-09 @637  			msize += skb_shinfo(skb)->frags[i].bv_len;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  638  
b2e5852793b6eb Mina Almasry      2023-12-20  639  		/* The cast to struct bio_vec* here assumes the frags are
b2e5852793b6eb Mina Almasry      2023-12-20  640  		 * struct page based. WARN if there is no page in this skb.
b2e5852793b6eb Mina Almasry      2023-12-20  641  		 */
b2e5852793b6eb Mina Almasry      2023-12-20  642  		DEBUG_NET_WARN_ON_ONCE(
b2e5852793b6eb Mina Almasry      2023-12-20  643  			!skb_frag_page(&skb_shinfo(skb)->frags[0]));
b2e5852793b6eb Mina Almasry      2023-12-20  644  
c31a25e1db486f David Howells     2023-06-09  645  		iov_iter_bvec(&msg.msg_iter, ITER_SOURCE,
b2e5852793b6eb Mina Almasry      2023-12-20  646  			      (const struct bio_vec *)skb_shinfo(skb)->frags,
b2e5852793b6eb Mina Almasry      2023-12-20  647  			      skb_shinfo(skb)->nr_frags, msize);
c31a25e1db486f David Howells     2023-06-09  648  		iov_iter_advance(&msg.msg_iter, txm->frag_offset);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  649  
c31a25e1db486f David Howells     2023-06-09  650  		do {
264ba53fac79b0 David Howells     2023-06-09  651  			ret = sock_sendmsg(psock->sk->sk_socket, &msg);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  652  			if (ret <= 0) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  653  				if (ret == -EAGAIN) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  654  					/* Save state to try again when there's
ab7ac4eb9832e3 Tom Herbert       2016-03-07  655  					 * write space on the socket
ab7ac4eb9832e3 Tom Herbert       2016-03-07  656  					 */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  657  					txm->frag_skb = skb;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  658  					ret = 0;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  659  					goto out;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  660  				}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  661  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  662  				/* Hard failure in sending message, abort this
ab7ac4eb9832e3 Tom Herbert       2016-03-07  663  				 * psock since it has lost framing
71a2fae50895b3 Bhaskar Chowdhury 2021-03-27  664  				 * synchronization and retry sending the
ab7ac4eb9832e3 Tom Herbert       2016-03-07  665  				 * message from the beginning.
ab7ac4eb9832e3 Tom Herbert       2016-03-07  666  				 */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  667  				kcm_abort_tx_psock(psock, ret ? -ret : EPIPE,
ab7ac4eb9832e3 Tom Herbert       2016-03-07  668  						   true);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  669  				unreserve_psock(kcm);
9f8d0dc0ec4a4b David Howells     2023-06-15  670  				psock = NULL;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  671  
c31a25e1db486f David Howells     2023-06-09  672  				txm->started_tx = false;
cd6e111bf5be5c Tom Herbert       2016-03-07  673  				kcm_report_tx_retry(kcm);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  674  				ret = 0;
c31a25e1db486f David Howells     2023-06-09  675  				goto retry;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  676  			}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  677  
c31a25e1db486f David Howells     2023-06-09  678  			txm->sent += ret;
c31a25e1db486f David Howells     2023-06-09  679  			txm->frag_offset += ret;
cd6e111bf5be5c Tom Herbert       2016-03-07  680  			KCM_STATS_ADD(psock->stats.tx_bytes, ret);
c31a25e1db486f David Howells     2023-06-09  681  		} while (msg.msg_iter.count > 0);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  682  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  683  		if (skb == head) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  684  			if (skb_has_frag_list(skb)) {
c31a25e1db486f David Howells     2023-06-09  685  				txm->frag_skb = skb_shinfo(skb)->frag_list;
c31a25e1db486f David Howells     2023-06-09  686  				txm->frag_offset = 0;
c31a25e1db486f David Howells     2023-06-09  687  				continue;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  688  			}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  689  		} else if (skb->next) {
c31a25e1db486f David Howells     2023-06-09  690  			txm->frag_skb = skb->next;
c31a25e1db486f David Howells     2023-06-09  691  			txm->frag_offset = 0;
c31a25e1db486f David Howells     2023-06-09  692  			continue;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  693  		}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  694  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  695  		/* Successfully sent the whole packet, account for it. */
c31a25e1db486f David Howells     2023-06-09  696  		sk->sk_wmem_queued -= txm->sent;
c31a25e1db486f David Howells     2023-06-09  697  		total_sent += txm->sent;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  698  		skb_dequeue(&sk->sk_write_queue);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  699  		kfree_skb(head);
cd6e111bf5be5c Tom Herbert       2016-03-07  700  		KCM_STATS_INCR(psock->stats.tx_msgs);
c31a25e1db486f David Howells     2023-06-09  701  	}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  702  out:
ab7ac4eb9832e3 Tom Herbert       2016-03-07  703  	if (!head) {
ab7ac4eb9832e3 Tom Herbert       2016-03-07  704  		/* Done with all queued messages. */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  705  		WARN_ON(!skb_queue_empty(&sk->sk_write_queue));
9f8d0dc0ec4a4b David Howells     2023-06-15  706  		if (psock)
ab7ac4eb9832e3 Tom Herbert       2016-03-07  707  			unreserve_psock(kcm);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  708  	}
ab7ac4eb9832e3 Tom Herbert       2016-03-07  709  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  710  	/* Check if write space is available */
ab7ac4eb9832e3 Tom Herbert       2016-03-07  711  	sk->sk_write_space(sk);
ab7ac4eb9832e3 Tom Herbert       2016-03-07  712  
ab7ac4eb9832e3 Tom Herbert       2016-03-07  713  	return total_sent ? : ret;
ab7ac4eb9832e3 Tom Herbert       2016-03-07  714  }
ab7ac4eb9832e3 Tom Herbert       2016-03-07  715  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t

[kernel test robot]

Hi Mina,

kernel test robot noticed the following build warnings:

[auto build test WARNING on net-next/main]

url:    https://github.com/intel-lab-lkp/linux/commits/Mina-Almasry/vsock-virtio-use-skb_frag_-helpers/20231222-164637
base:   net-next/main
patch link:    https://lore.kernel.org/r/20231220214505.2303297-4-almasrymina%40google.com
patch subject: [PATCH net-next v3 3/3] net: add netmem_ref to skb_frag_t
:::::: branch date: 15 hours ago
:::::: commit date: 15 hours ago
config: x86_64-randconfig-121-20231223 (https://download.01.org/0day-ci/archive/20231223/202312230726.4XaPn84E-lkp@intel.com/config)
compiler: gcc-11 (Debian 11.3.0-12) 11.3.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231223/202312230726.4XaPn84E-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/r/202312230726.4XaPn84E-lkp@intel.com/

sparse warnings: (new ones prefixed by >>)
   net/core/netevent.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/netevent.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/linux/rtnetlink.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/request_sock.c: note: in included file (through include/linux/skbuff.h, include/linux/tcp.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/request_sock.c: note: in included file (through include/linux/tcp.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/utils.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/inet.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/utils.c: note: in included file (through include/net/net_namespace.h, include/linux/inet.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/secure_seq.c: note: in included file (through include/linux/skbuff.h, include/linux/tcp.h, include/net/tcp.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/secure_seq.c: note: in included file (through include/linux/tcp.h, include/net/tcp.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/net_namespace.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/net_namespace.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/linux/rtnetlink.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/stream.c: note: in included file (through include/linux/skbuff.h, include/linux/tcp.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/stream.c: note: in included file (through include/linux/tcp.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/dst.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/dst.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/gen_stats.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/gen_stats.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/linux/rtnetlink.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/gen_estimator.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/gen_estimator.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/scm.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/scm.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/datagram.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/inet.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/datagram.c: note: in included file (through include/net/net_namespace.h, include/linux/inet.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/sysctl_net_core.c: note: in included file (through include/linux/skbuff.h, include/linux/filter.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/sysctl_net_core.c: note: in included file (through include/linux/filter.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/dev_addr_lists.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/dev_addr_lists.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
   net/core/dev_addr_lists.c: note: in included file (through include/linux/hrtimer.h, include/linux/sched.h, include/linux/delay.h, ...):
   include/linux/rbtree.h:74:9: sparse: sparse: incompatible types in comparison expression (different address spaces):
   include/linux/rbtree.h:74:9: sparse:    struct rb_node [noderef] __rcu *
   include/linux/rbtree.h:74:9: sparse:    struct rb_node *
--
   net/core/flow_dissector.c: note: in included file (through include/linux/skbuff.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/flow_dissector.c: note: in included file:
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/link_watch.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/link_watch.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/tso.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/tso.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/linux/if_vlan.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/sock_diag.c: note: in included file (through include/linux/skbuff.h, include/linux/filter.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/sock_diag.c: note: in included file (through include/linux/filter.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/fib_notifier.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/fib_notifier.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/linux/rtnetlink.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/netdev-genl-gen.c: note: in included file (through include/linux/skbuff.h, include/linux/netlink.h, include/net/netlink.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/netdev-genl-gen.c: note: in included file (through include/linux/netlink.h, include/net/netlink.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/dev_ioctl.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/dev_ioctl.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/sock_reuseport.c: note: in included file (through include/linux/skbuff.h, include/linux/ip.h, include/net/ip.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/sock_reuseport.c: note: in included file (through include/linux/ip.h, include/net/ip.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/gso.c: note: in included file (through include/linux/skbuff.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/gso.c: note: in included file:
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/flow_offload.c: note: in included file (through include/linux/skbuff.h, include/linux/netlink.h, include/net/flow_offload.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/flow_offload.c: note: in included file (through include/linux/netlink.h, include/net/flow_offload.h, include/net/act_api.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/netdev-genl.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/netdev-genl.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/net-procfs.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/net-procfs.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/xdp.c: note: in included file (through include/linux/skbuff.h, include/linux/filter.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/xdp.c: note: in included file (through include/linux/filter.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/neighbour.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/neighbour.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/gro.c: note: in included file (through include/linux/skbuff.h, include/linux/ip.h, include/net/gro.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/gro.c: note: in included file (through include/linux/ip.h, include/net/gro.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/sock.c: note: in included file (through include/linux/skbuff.h, include/linux/ip.h, include/net/ip.h, include/linux/errqueue.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/sock.c: note: in included file (through include/linux/ip.h, include/net/ip.h, include/linux/errqueue.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/net-sysfs.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/net-sysfs.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/gro_cells.c: note: in included file (through include/linux/skbuff.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/gro_cells.c: note: in included file:
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/ptp_classifier.c: note: in included file (through include/linux/skbuff.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/ptp_classifier.c: note: in included file:
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/of_net.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/etherdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/of_net.c: note: in included file (through include/linux/if_ether.h, include/linux/etherdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/failover.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/etherdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/failover.c: note: in included file (through include/linux/if_ether.h, include/linux/etherdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/dst_cache.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/dst_cache.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/net/dst.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/rtnetlink.c: note: in included file (through include/linux/skbuff.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/rtnetlink.c: note: in included file:
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/skbuff.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/inet.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/skbuff.c: note: in included file (through include/net/net_namespace.h, include/linux/inet.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
>> net/core/skbuff.c:848:68: sparse: sparse: invalid modifier
--
   net/core/dev.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/dev.c: note: in included file (through include/linux/if_ether.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/net-traces.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/net-traces.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/core/filter.c: note: in included file (through include/linux/skbuff.h, include/linux/filter.h, include/linux/bpf_verifier.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/core/filter.c: note: in included file (through include/linux/filter.h, include/linux/bpf_verifier.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/common.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/common.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/strset.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/strset.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/wol.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/wol.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/privflags.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/privflags.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/linkstate.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/linkstate.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/rss.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/rss.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/debug.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/debug.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/linkinfo.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/linkinfo.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/bitset.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/bitset.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/linkmodes.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/linkmodes.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/features.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/features.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/netlink.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/netlink.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/net/sock.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/rings.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/rings.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/channels.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/channels.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, include/net/sock.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/pause.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/pause.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/eee.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/eee.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/ioctl.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/etherdevice.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/ioctl.c: note: in included file (through include/linux/if_ether.h, include/linux/etherdevice.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/coalesce.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/coalesce.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/tsinfo.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/tsinfo.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/cabletest.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/cabletest.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/linux/phy.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/fec.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/fec.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/tunnels.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/tunnels.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/eeprom.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/eeprom.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/module.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/module.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/plca.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/plca.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/linux/phy.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/phc_vclocks.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/phc_vclocks.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/pse-pd.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/pse-pd.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, net/ethtool/common.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/stats.c: note: in included file (through include/linux/skbuff.h, include/linux/if_ether.h, include/linux/ethtool.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/stats.c: note: in included file (through include/linux/if_ether.h, include/linux/ethtool.h, include/uapi/linux/ethtool_netlink.h, ...):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier
--
   net/ethtool/mm.c: note: in included file (through include/linux/skbuff.h, include/net/net_namespace.h, include/linux/netdevice.h, ...):
>> include/net/netmem.h:28:54: sparse: sparse: invalid modifier
   include/net/netmem.h:36:26: sparse: sparse: invalid modifier
   include/net/netmem.h:38:27: sparse: sparse: invalid modifier
   net/ethtool/mm.c: note: in included file (through include/net/net_namespace.h, include/linux/netdevice.h, net/ethtool/common.h):
>> include/linux/skbuff.h:364:20: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2440:57: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2456:67: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2498:54: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2521:52: sparse: sparse: invalid modifier
   include/linux/skbuff.h:2571:68: sparse: sparse: invalid modifier

vim +364 include/linux/skbuff.h

3953c46c3ac7ee Marcelo Ricardo Leitner 2016-06-02  362  
b2e5852793b6eb Mina Almasry            2023-12-20  363  typedef struct skb_frag {
b2e5852793b6eb Mina Almasry            2023-12-20 @364  	netmem_ref netmem;
b2e5852793b6eb Mina Almasry            2023-12-20  365  	unsigned int len;
b2e5852793b6eb Mina Almasry            2023-12-20  366  	unsigned int offset;
b2e5852793b6eb Mina Almasry            2023-12-20  367  } skb_frag_t;
^1da177e4c3f41 Linus Torvalds          2005-04-16  368  

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki