* Re: [PATCH net] net/xdp: Fix suspicious RCU usage warning
From: Daniel Borkmann @ 2018-08-12 23:04 UTC (permalink / raw)
To: Tariq Toukan, Alexei Starovoitov
Cc: David S. Miller, netdev, Eran Ben Elisha, Jesper Dangaard Brouer
In-Reply-To: <44cb66c6-2542-3f6a-c426-5b1d1665ea61@mellanox.com>
On 08/12/2018 10:45 AM, Tariq Toukan wrote:
>
>
> On 20/07/2018 12:36 AM, Alexei Starovoitov wrote:
>> On Wed, Jul 18, 2018 at 05:13:54PM +0300, Tariq Toukan wrote:
>>>
>>>
>>> On 17/07/2018 10:27 PM, Daniel Borkmann wrote:
>>>> On 07/17/2018 06:47 PM, Alexei Starovoitov wrote:
>>>>> On Tue, Jul 17, 2018 at 06:10:38PM +0300, Tariq Toukan wrote:
>>>>>> Fix the warning below by calling rhashtable_lookup under
>>>>>> RCU read lock.
>>>>>>
>>>
>>> ...
>>>
>>>>>> mutex_lock(&mem_id_lock);
>>>>>> + rcu_read_lock();
>>>>>> xa = rhashtable_lookup(mem_id_ht, &id, mem_id_rht_params);
>>>>>> + rcu_read_unlock();
>>>>>> if (!xa) {
>>>>>
>>>>> if it's an actual bug rcu_read_unlock seems to be misplaced.
>>>>> It silences the warn, but rcu section looks wrong.
>>>>
>>>> I think that whole piece in __xdp_rxq_info_unreg_mem_model() should be:
>>>>
>>>> mutex_lock(&mem_id_lock);
>>>> xa = rhashtable_lookup_fast(mem_id_ht, &id, mem_id_rht_params);
>>>> if (xa && rhashtable_remove_fast(mem_id_ht, &xa->node, mem_id_rht_params) == 0)
>>>> call_rcu(&xa->rcu, __xdp_mem_allocator_rcu_free);
>>>> mutex_unlock(&mem_id_lock);
>>>>
>>>> Technically the RCU read side plus rhashtable_lookup() is the same, but lets
>>>> use proper api. From the doc (https://lwn.net/Articles/751374/) object removal
>>>> is wrapped around the RCU read side additionally, but in our case we're behind
>>>> mem_id_lock for insertion/removal serialization.
>>>>
>>>> Cheers,
>>>> Daniel
>>>>
>>>
>>> Just as Daniel stated, I think there's no actual bug here, but we still want
>>> to silence the RCU warning.
>>>
>>> Alexei, did you mean getting the if statement into the RCU lock critical
>>> section?
>>
>> If what Daniel proposes silences the warn, I'd rather do that.
>> Pattern like:
>> rcu_lock;
>> val = lookup();
>> rcu_unlock;
>> if (val)
>> will cause people to question the quality of the code and whether
>> authors of the code understand rcu.
>> There should be a way to silence the warn without adding
>> "wrong on the first glance" code.
>
> I'm re-spinning this.
> Can it still go to net, or better send it to bpf-next ?
Please rebase against bpf-next and we route it to stable, thanks!
^ permalink raw reply
* Re: [V9fs-developer] [PATCH 2/2] 9p: Add refcount to p9_req_t
From: Dominique Martinet @ 2018-08-13 1:48 UTC (permalink / raw)
To: piaojun
Cc: Tomas Bortoli, ericvh, rminnich, lucho, Dominique Martinet,
netdev, linux-kernel, syzkaller, v9fs-developer, davem
In-Reply-To: <5B70E0D1.2080300@huawei.com>
piaojun wrote on Mon, Aug 13, 2018:
> Could you help paste the reason of the crash bug to help others
> understand more clearly? And I have another question below.
The problem for tcp (but other transports have a similar problem) is
that with a malicious server like syzkaller they can try to submit
replies before the request came in.
This leads in the writer thread trying to write a buffer that has
already been freed, and if memory has been reused could potentially leak
some information.
Now, with the previous patches this is based on this would be a slab and
the likeliness of it being sensitive information is rather low (it would
likely be some other packet being sent twice, or a mix and match of two
packets that would have been sent anyway), but it would nevertheless be
a use after free.
There is a second advantage to this reference counting, that is now we
have this system we will be able to implement flush asynchronously.
This will remove the need for the 'goto again' in p9_client_rpc which
was making 9p threads unkillable in practice if the server would not
reply to the flush requests.
Even if the server replies I've always found myself needing to hit ^C
multiple times to exit a process doing I/Os and I think fixing that
behaviour will make 9p more comfortable to use.
> > diff --git a/net/9p/trans_fd.c b/net/9p/trans_fd.c
> > index 20f46f13fe83..686e24e355d0 100644
> > --- a/net/9p/trans_fd.c
> > +++ b/net/9p/trans_fd.c
> > @@ -132,6 +132,7 @@ struct p9_conn {
> > struct list_head req_list;
> > struct list_head unsent_req_list;
> > struct p9_req_t *req;
> > + struct p9_req_t *wreq;
>
> Why adding a wreq for write work? And I wonder we should rename req to
> rreq?
We need to store a pointer to the request for the write thread because
we need to put the reference to it when we're done writing its content.
Previously, the worker would only store the write buffer there but
that's not enough to figure what request to dereference.
I personally don't think renaming req to rreq would bring much but it
could be done in another patch if you think that'd be helpful; I think
it shouldn't be done here at least to make the patch more readable.
--
Dominique
^ permalink raw reply
* Re: [Patch net] ipv6: fix double refcount of fib6_metrics
From: David Ahern @ 2018-08-12 23:08 UTC (permalink / raw)
To: Cong Wang, netdev; +Cc: Sabrina Dubroca
In-Reply-To: <20180803062038.13272-1-xiyou.wangcong@gmail.com>
On 8/3/18 12:20 AM, Cong Wang wrote:
> All the callers of ip6_rt_copy_init()/rt6_set_from() hold refcnt
> of the "from" fib6_info, so there is no need to hold fib6_metrics
> refcnt again, because fib6_metrics refcnt is only released when
> fib6_info is gone, that is, they have the same life time, so the
> whole fib6_metrics refcnt can be removed actually.
>
> This fixes a kmemleak warning reported by Sabrina.
>
> Fixes: 93531c674315 ("net/ipv6: separate handling of FIB entries from dst based routes")
> Reported-by: Sabrina Dubroca <sd@queasysnail.net>
> Cc: Sabrina Dubroca <sd@queasysnail.net>
> Cc: David Ahern <dsahern@gmail.com>
> Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
> ---
> net/ipv6/route.c | 4 ----
> 1 file changed, 4 deletions(-)
>
> diff --git a/net/ipv6/route.c b/net/ipv6/route.c
> index ec18b3ce8b6d..7208c16302f6 100644
> --- a/net/ipv6/route.c
> +++ b/net/ipv6/route.c
> @@ -978,10 +978,6 @@ static void rt6_set_from(struct rt6_info *rt, struct fib6_info *from)
> rt->rt6i_flags &= ~RTF_EXPIRES;
> rcu_assign_pointer(rt->from, from);
> dst_init_metrics(&rt->dst, from->fib6_metrics->metrics, true);
> - if (from->fib6_metrics != &dst_default_metrics) {
> - rt->dst._metrics |= DST_METRICS_REFCOUNTED;
> - refcount_inc(&from->fib6_metrics->refcnt);
> - }
> }
>
> /* Caller must already hold reference to @ort */
>
This seems like a reasonable fix. Thanks for the patch.
^ permalink raw reply
* Re: [PATCH v2 bpf-next 0/4] bpf_skb_ancestor_cgroup_id helper
From: Daniel Borkmann @ 2018-08-12 23:23 UTC (permalink / raw)
To: Andrey Ignatov, netdev; +Cc: ast, tj, guro, yhs, kernel-team
In-Reply-To: <cover.1534095926.git.rdna@fb.com>
On 08/12/2018 07:49 PM, Andrey Ignatov wrote:
> v1->v2:
> - more reliable check for testing IPv6 to become ready in selftest.
>
> This patch set adds new BPF helper bpf_skb_ancestor_cgroup_id that returns
> id of cgroup v2 that is ancestor of cgroup associated with the skb at the
> ancestor_level.
>
> The helper is useful to implement policies in TC based on cgroups that are
> upper in hierarchy than immediate cgroup associated with skb.
>
> Patch 0001 provides more details and describes use-cases.
> Patch 0002 syncs UAPI changes to tools/.
> Patch 0003 adds skb*cgroup_id helpers to bpf_helper.h header.
> Patch 0004 adds selftest for the new helper and is an example of usage.
>
>
> Andrey Ignatov (4):
> bpf: Introduce bpf_skb_ancestor_cgroup_id helper
> bpf: Sync bpf.h to tools/
> selftests/bpf: Add cgroup id helpers to bpf_helpers.h
> selftests/bpf: Selftest for bpf_skb_ancestor_cgroup_id
Applied to bpf-next, thanks Andrey!
^ permalink raw reply
* [PATCH] net: Change the layout of structure trace_event_raw_fib_table_lookup
From: Zong Li @ 2018-08-13 2:26 UTC (permalink / raw)
To: rostedt, mingo, dsahern, netdev, linux-kernel; +Cc: zongbox, greentime, Zong Li
There is an unalignment access about the structure
'trace_event_raw_fib_table_lookup'.
In include/trace/events/fib.h, there is a memory operation which casting
the 'src' data member to a pointer, and then store a value to this
pointer point to.
p32 = (__be32 *) __entry->src;
*p32 = flp->saddr;
The offset of 'src' in structure trace_event_raw_fib_table_lookup is not
four bytes alignment. On some architectures, they don't permit the
unalignment access, it need to pay the price to handle this situation in
exception handler.
Adjust the layout of structure to avoid this case.
Signed-off-by: Zong Li <zong@andestech.com>
---
include/trace/events/fib.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/trace/events/fib.h b/include/trace/events/fib.h
index 9763cddd0594..6271bab63bfb 100644
--- a/include/trace/events/fib.h
+++ b/include/trace/events/fib.h
@@ -22,6 +22,7 @@ TRACE_EVENT(fib_table_lookup,
__field( int, err )
__field( int, oif )
__field( int, iif )
+ __field( u8, proto )
__field( __u8, tos )
__field( __u8, scope )
__field( __u8, flags )
@@ -31,7 +32,6 @@ TRACE_EVENT(fib_table_lookup,
__array( __u8, saddr, 4 )
__field( u16, sport )
__field( u16, dport )
- __field( u8, proto )
__dynamic_array(char, name, IFNAMSIZ )
),
--
2.16.1
^ permalink raw reply related
* pull-request: bpf-next 2018-08-13
From: Daniel Borkmann @ 2018-08-12 23:49 UTC (permalink / raw)
To: davem; +Cc: daniel, ast, netdev
Hi David,
The following pull-request contains BPF updates for your *net-next* tree.
The main changes are:
1) Add driver XDP support for veth. This can be used in conjunction with
redirect of another XDP program e.g. sitting on NIC so the xdp_frame
can be forwarded to the peer veth directly without modification,
from Toshiaki.
2) Add a new BPF map type REUSEPORT_SOCKARRAY and prog type SK_REUSEPORT
in order to provide more control and visibility on where a SO_REUSEPORT
sk should be located, and the latter enables to directly select a sk
from the bpf map. This also enables map-in-map for application migration
use cases, from Martin.
3) Add a new BPF helper bpf_skb_ancestor_cgroup_id() that returns the id
of cgroup v2 that is the ancestor of the cgroup associated with the
skb at the ancestor_level, from Andrey.
4) Implement BPF fs map pretty-print support based on BTF data for regular
hash table and LRU map, from Yonghong.
5) Decouple the ability to attach BTF for a map from the key and value
pretty-printer in BPF fs, and enable further support of BTF for maps for
percpu and LPM trie, from Daniel.
6) Implement a better BPF sample of using XDP's CPU redirect feature for
load balancing SKB processing to remote CPU. The sample implements the
same XDP load balancing as Suricata does which is symmetric hash based
on IP and L4 protocol, from Jesper.
7) Revert adding NULL pointer check with WARN_ON_ONCE() in __xdp_return()'s
critical path as it is ensured that the allocator is present, from Björn.
Please consider pulling these changes from:
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
Thanks a lot!
----------------------------------------------------------------
The following changes since commit a736e074680745faa5dc6be8dd3c58ad4850aab9:
Merge ra.kernel.org:/pub/scm/linux/kernel/git/davem/net (2018-08-09 11:52:36 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
for you to fetch changes up to 2ce3206b9eb3943de09f3bf4ec9134568420d8b9:
Merge branch 'bpf-ancestor-cgroup-id' (2018-08-13 01:02:41 +0200)
----------------------------------------------------------------
Andrey Ignatov (4):
bpf: Introduce bpf_skb_ancestor_cgroup_id helper
bpf: Sync bpf.h to tools/
selftests/bpf: Add cgroup id helpers to bpf_helpers.h
selftests/bpf: Selftest for bpf_skb_ancestor_cgroup_id
Björn Töpel (1):
Revert "xdp: add NULL pointer check in __xdp_return()"
Daniel Borkmann (6):
Merge branch 'bpf-sample-cpumap-lb'
Merge branch 'bpf-veth-xdp-support'
Merge branch 'bpf-btf-for-htab-lru'
Merge branch 'bpf-reuseport-map'
bpf: decouple btf from seq bpf fs dump and enable more maps
Merge branch 'bpf-ancestor-cgroup-id'
Jesper Dangaard Brouer (2):
samples/bpf: add Paul Hsieh's (LGPL 2.1) hash function SuperFastHash
samples/bpf: xdp_redirect_cpu load balance like Suricata
Martin KaFai Lau (9):
tcp: Avoid TCP syncookie rejected by SO_REUSEPORT socket
net: Add ID (if needed) to sock_reuseport and expose reuseport_lock
bpf: Introduce BPF_MAP_TYPE_REUSEPORT_SOCKARRAY
bpf: Introduce BPF_PROG_TYPE_SK_REUSEPORT
bpf: Enable BPF_PROG_TYPE_SK_REUSEPORT bpf prog in reuseport selection
bpf: Refactor ARRAY_SIZE macro to bpf_util.h
bpf: Sync bpf.h uapi to tools/
bpf: test BPF_MAP_TYPE_REUSEPORT_SOCKARRAY
bpf: Test BPF_PROG_TYPE_SK_REUSEPORT
Toshiaki Makita (10):
net: Export skb_headers_offset_update
veth: Add driver XDP
veth: Avoid drops by oversized packets when XDP is enabled
xdp: Helper function to clear kernel pointers in xdp_frame
veth: Handle xdp_frames in xdp napi ring
veth: Add ndo_xdp_xmit
bpf: Make redirect_info accessible from modules
xdp: Helpers for disabling napi_direct of xdp_return_frame
veth: Add XDP TX and REDIRECT
veth: Support per queue XDP ring
Yonghong Song (3):
bpf: fix bpffs non-array map seq_show issue
bpf: btf: add pretty print for hash/lru_hash maps
tools/bpf: add bpffs pretty print btf test for hash/lru_hash maps
drivers/net/veth.c | 750 ++++++++++++++++++++-
include/linux/bpf.h | 41 +-
include/linux/bpf_types.h | 6 +
include/linux/cgroup.h | 30 +
include/linux/filter.h | 51 ++
include/linux/skbuff.h | 1 +
include/net/addrconf.h | 1 +
include/net/sock_reuseport.h | 19 +-
include/net/tcp.h | 30 +-
include/net/xdp.h | 7 +
include/uapi/linux/bpf.h | 56 +-
kernel/bpf/Makefile | 3 +
kernel/bpf/arraymap.c | 28 +-
kernel/bpf/cpumap.c | 1 +
kernel/bpf/devmap.c | 1 +
kernel/bpf/hashtab.c | 26 +
kernel/bpf/inode.c | 11 +-
kernel/bpf/local_storage.c | 1 +
kernel/bpf/lpm_trie.c | 12 +
kernel/bpf/reuseport_array.c | 363 ++++++++++
kernel/bpf/sockmap.c | 2 +
kernel/bpf/stackmap.c | 1 +
kernel/bpf/syscall.c | 42 +-
kernel/bpf/verifier.c | 9 +
kernel/bpf/xskmap.c | 3 +-
net/core/filter.c | 411 +++++++++--
net/core/skbuff.c | 3 +-
net/core/sock_reuseport.c | 92 ++-
net/core/xdp.c | 9 +-
net/ipv4/inet_connection_sock.c | 9 +
net/ipv4/inet_hashtables.c | 19 +-
net/ipv4/udp.c | 9 +-
net/ipv6/inet6_hashtables.c | 14 +-
net/ipv6/udp.c | 4 +
samples/bpf/hash_func01.h | 55 ++
samples/bpf/xdp_redirect_cpu_kern.c | 103 +++
samples/bpf/xdp_redirect_cpu_user.c | 4 +-
tools/include/uapi/linux/bpf.h | 56 +-
tools/lib/bpf/bpf.c | 1 +
tools/lib/bpf/bpf.h | 1 +
tools/lib/bpf/libbpf.c | 1 +
tools/testing/selftests/bpf/Makefile | 11 +-
tools/testing/selftests/bpf/bpf_helpers.h | 8 +
tools/testing/selftests/bpf/bpf_util.h | 4 +
tools/testing/selftests/bpf/test_align.c | 5 +-
tools/testing/selftests/bpf/test_btf.c | 92 ++-
tools/testing/selftests/bpf/test_maps.c | 262 ++++++-
.../testing/selftests/bpf/test_select_reuseport.c | 688 +++++++++++++++++++
.../selftests/bpf/test_select_reuseport_common.h | 36 +
.../selftests/bpf/test_select_reuseport_kern.c | 180 +++++
tools/testing/selftests/bpf/test_skb_cgroup_id.sh | 62 ++
.../selftests/bpf/test_skb_cgroup_id_kern.c | 47 ++
.../selftests/bpf/test_skb_cgroup_id_user.c | 187 +++++
tools/testing/selftests/bpf/test_sock.c | 5 +-
tools/testing/selftests/bpf/test_sock_addr.c | 5 +-
tools/testing/selftests/bpf/test_verifier.c | 5 +-
56 files changed, 3707 insertions(+), 176 deletions(-)
create mode 100644 kernel/bpf/reuseport_array.c
create mode 100644 samples/bpf/hash_func01.h
create mode 100644 tools/testing/selftests/bpf/test_select_reuseport.c
create mode 100644 tools/testing/selftests/bpf/test_select_reuseport_common.h
create mode 100644 tools/testing/selftests/bpf/test_select_reuseport_kern.c
create mode 100755 tools/testing/selftests/bpf/test_skb_cgroup_id.sh
create mode 100644 tools/testing/selftests/bpf/test_skb_cgroup_id_kern.c
create mode 100644 tools/testing/selftests/bpf/test_skb_cgroup_id_user.c
^ permalink raw reply
* bpf-next is CLOSED
From: Daniel Borkmann @ 2018-08-12 23:50 UTC (permalink / raw)
To: netdev; +Cc: ast
Given the merge window is open right now, please only submit bug
fixes to bpf tree at this time, thank you!
^ permalink raw reply
* Re: [BUG] bpf: syscall: a possible sleep-in-atomic-context bug in map_update_elem()
From: Jia-Ju Bai @ 2018-08-13 2:48 UTC (permalink / raw)
To: Y Song
Cc: Daniel Borkmann, Alexei Starovoitov, netdev,
Linux Kernel Mailing List, Jesper Dangaard Brouer
In-Reply-To: <CAH3MdRWBrEfapBMkK9sOa-e_bBZzdkcxhioEHJoZ31hF6XXuxQ@mail.gmail.com>
On 2018/8/11 13:01, Y Song wrote:
> On Fri, Aug 10, 2018 at 6:57 PM, Jia-Ju Bai <baijiaju1990@gmail.com> wrote:
>>
>> On 2018/8/10 22:22, Daniel Borkmann wrote:
>>> On 08/10/2018 04:07 PM, Jia-Ju Bai wrote:
>>>> The kernel may sleep with holding a rcu read lock.
>>>>
>>>> The function call paths (from bottom to top) in Linux-4.16 are:
>>>>
>>>> [FUNC] kmalloc(GFP_KERNEL)
>>>> kernel/kthread.c, 283: kmalloc in __kthread_create_on_node
>>>> kernel/kthread.c, 365: __kthread_create_on_node in kthread_create_on_node
>>>> kernel/bpf/cpumap.c, 368: kthread_create_on_node in __cpu_map_entry_alloc
>>>> kernel/bpf/cpumap.c, 490: __cpu_map_entry_alloc in cpu_map_update_elem
>>>> kernel/bpf/syscall.c, 724: [FUNC_PTR]cpu_map_update_elem in
>>>> map_update_elem
>>>> kernel/bpf/syscall.c, 723: rcu_read_lock in map_update_elem
>>>>
>>>> Note that [FUNC_PTR] means a function pointer call is used.
>>>>
>>>> I do not find a good way to fix it, so I only report.
>>>> This is found by my static analysis tool (DSAC).
> Maybe your static analysis tool have false positives in this case?
>
>>> Thanks for the report Jia-Ju! In the map_update_elem() from syscall
>>> path there's a check map->map_type == BPF_MAP_TYPE_CPUMAP, where we
>>> call the cpumap's map->ops->map_update_elem() while /not/ being under
>>> rcu_read_lock() as in other cases, so looks okay to me. Could you point
>>> out the case for being under rcu_read_lock() more specifically which
>>> the tool found?
>>
>> Thanks for your reply :)
>> My tool cannot accurately track the case of map->map_type at present...
>>
>> According to my code review, there is a indeed check on line 697 in
>> Linux-4.16:
>> else if (map->map_type == BPF_MAP_TYPE_CPUMAP) {
>> err = map->ops->map_update_elem(map, key, value, attr->flags);
>> goto out;
>> }
>> But there is a call to map->ops->map_update_elem() that is under
>> rcu_read_lock on line 724:
>> rcu_read_lock();
>> err = map->ops->map_update_elem(map, key, value, attr->flags);
>> rcu_read_unlock();
>>
>> So I think if map->map_type is not equal to BPF_MAP_TYPE_CPUMAP,
>> map->ops->map_update_elem() can still be called under rcu_read_lock, is it
>> right?
> map->ops->map_update_elem() can be called under rcu_read_lock(), but
> since it is not type cpumap, the function should not be cpu_map_update_elem().
> Could you double check your static analysis tool?
Thanks for your reply :)
I have checked the code again, and find that my report is not correct here.
It is because that my tool does not handle the context of function
pointer assignment.
Best wishes,
Jia-Ju Bai
^ permalink raw reply
* [RFC PATCH net-next 1/5] net: core: generic XDP support for stacked device
From: Jason Wang @ 2018-08-13 3:05 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534129513-4845-1-git-send-email-jasowang@redhat.com>
Stacked device usually change skb->dev to its own and return
RX_HANDLER_ANOTHER during rx handler processing. But we don't call
generic XDP routine at that time, this means it can't work for stacked
device.
Fixing this by calling netif_do_generic_xdp() if rx handler returns
RX_HANDLER_ANOTHER. This allows us to do generic XDP on stacked device
e.g macvlan.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
net/core/dev.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/net/core/dev.c b/net/core/dev.c
index 605c66e..a77ce08 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4822,6 +4822,11 @@ static int __netif_receive_skb_core(struct sk_buff *skb, bool pfmemalloc,
ret = NET_RX_SUCCESS;
goto out;
case RX_HANDLER_ANOTHER:
+ ret = netif_do_generic_xdp(skb);
+ if (ret != XDP_PASS) {
+ ret = NET_RX_SUCCESS;
+ goto out;
+ }
goto another_round;
case RX_HANDLER_EXACT:
deliver_exact = true;
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next 2/5] net: core: introduce XDP rx handler
From: Jason Wang @ 2018-08-13 3:05 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534129513-4845-1-git-send-email-jasowang@redhat.com>
This patch tries to introduce XDP rx handler. This will be used by
stacked device that depends on rx handler for having a fast packet
processing path based on XDP.
This idea is simple, when XDP program returns XDP_PASS, instead of
building skb immediately, driver will call xdp_do_pass() to check
whether or not there's a XDP rx handler, if yes, it will pass XDP
buffer to XDP rx handler first.
There are two main tasks for XDP rx handler, the first is check
whether or not the setup or packet could be processed through XDP buff
directly. The second task is to run XDP program. An XDP rx handler can
return several different results which was defined by enum
rx_xdp_handler_result_t:
RX_XDP_HANDLER_CONSUMED: This means the XDP buff were consumed.
RX_XDP_HANDLER_DROP: This means XDP rx handler ask to drop the packet.
RX_XDP_HANDLER_PASS_FALLBACK: This means XDP rx handler can not
process the packet (e.g cloning), and we need to fall back to normal
skb path to deal with the packet.
Consider we have the following configuration, Level 0 device which has
a rx handler for Level 1 device which has a rx handler for L2 device.
L2 device
|
L1 device
|
L0 device
With the help of XDP rx handler, we can attach XDP program on each of
the layer or even run native XDP handler for L2 without XDP prog
attached to L1 device:
(XDP prog for L2 device)
|
L2 XDP rx handler for L1
|
(XDP prog for L1 device)
|
L1 XDP rx hanlder for L0
|
XDP prog for L0 device
It works like: When the XDP program for L0 device returns XDP_PASS, we
will first try to check and pass XDP buff to its XDP rx handler if
there's one. Then the L1 XDP rx handler will be called and to run XDP
program for L1. When L1 XDP program returns XDP_PASS or there's no XDP
program attached to L1, we will try to call xdp_do_pass() to pass it
to XDP rx hanlder for L1. Then XDP buff will be passed to L2 XDP rx
handler etc. And it will try to run L2 XDP program if any. And if
there's no L2 XDP program or XDP program returns XDP_PASS. The handler
usually will build skb and call netif_rx() for a local receive. If any
of the XDP rx handlers returns XDP_RX_HANDLER_FALLBACK, the code will
return to L0 device and L0 device will try to build skb and go through
normal rx handler path for skb.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
include/linux/filter.h | 1 +
include/linux/netdevice.h | 12 ++++++++++++
net/core/dev.c | 29 +++++++++++++++++++++++++++++
net/core/filter.c | 28 ++++++++++++++++++++++++++++
4 files changed, 70 insertions(+)
diff --git a/include/linux/filter.h b/include/linux/filter.h
index c73dd73..7cc8e69 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -791,6 +791,7 @@ int xdp_do_generic_redirect(struct net_device *dev, struct sk_buff *skb,
int xdp_do_redirect(struct net_device *dev,
struct xdp_buff *xdp,
struct bpf_prog *prog);
+rx_handler_result_t xdp_do_pass(struct xdp_buff *xdp);
void xdp_do_flush_map(void);
void bpf_warn_invalid_xdp_action(u32 act);
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 282e2e9..21f0a9e 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -421,6 +421,14 @@ enum rx_handler_result {
typedef enum rx_handler_result rx_handler_result_t;
typedef rx_handler_result_t rx_handler_func_t(struct sk_buff **pskb);
+enum rx_xdp_handler_result {
+ RX_XDP_HANDLER_CONSUMED,
+ RX_XDP_HANDLER_DROP,
+ RX_XDP_HANDLER_FALLBACK,
+};
+typedef enum rx_xdp_handler_result rx_xdp_handler_result_t;
+typedef rx_xdp_handler_result_t rx_xdp_handler_func_t(struct net_device *dev,
+ struct xdp_buff *xdp);
void __napi_schedule(struct napi_struct *n);
void __napi_schedule_irqoff(struct napi_struct *n);
@@ -1898,6 +1906,7 @@ struct net_device {
struct bpf_prog __rcu *xdp_prog;
unsigned long gro_flush_timeout;
rx_handler_func_t __rcu *rx_handler;
+ rx_xdp_handler_func_t __rcu *rx_xdp_handler;
void __rcu *rx_handler_data;
#ifdef CONFIG_NET_CLS_ACT
@@ -3530,7 +3539,10 @@ bool netdev_is_rx_handler_busy(struct net_device *dev);
int netdev_rx_handler_register(struct net_device *dev,
rx_handler_func_t *rx_handler,
void *rx_handler_data);
+int netdev_rx_xdp_handler_register(struct net_device *dev,
+ rx_xdp_handler_func_t *rx_xdp_handler);
void netdev_rx_handler_unregister(struct net_device *dev);
+void netdev_rx_xdp_handler_unregister(struct net_device *dev);
bool dev_valid_name(const char *name);
int dev_ioctl(struct net *net, unsigned int cmd, struct ifreq *ifr,
diff --git a/net/core/dev.c b/net/core/dev.c
index a77ce08..b4e8949 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4638,6 +4638,12 @@ bool netdev_is_rx_handler_busy(struct net_device *dev)
}
EXPORT_SYMBOL_GPL(netdev_is_rx_handler_busy);
+static bool netdev_is_rx_xdp_handler_busy(struct net_device *dev)
+{
+ ASSERT_RTNL();
+ return dev && rtnl_dereference(dev->rx_xdp_handler);
+}
+
/**
* netdev_rx_handler_register - register receive handler
* @dev: device to register a handler for
@@ -4670,6 +4676,22 @@ int netdev_rx_handler_register(struct net_device *dev,
}
EXPORT_SYMBOL_GPL(netdev_rx_handler_register);
+int netdev_rx_xdp_handler_register(struct net_device *dev,
+ rx_xdp_handler_func_t *rx_xdp_handler)
+{
+ if (netdev_is_rx_xdp_handler_busy(dev))
+ return -EBUSY;
+
+ if (dev->priv_flags & IFF_NO_RX_HANDLER)
+ return -EINVAL;
+
+ rcu_assign_pointer(dev->rx_xdp_handler, rx_xdp_handler);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(netdev_rx_xdp_handler_register);
+
+
/**
* netdev_rx_handler_unregister - unregister receive handler
* @dev: device to unregister a handler from
@@ -4692,6 +4714,13 @@ void netdev_rx_handler_unregister(struct net_device *dev)
}
EXPORT_SYMBOL_GPL(netdev_rx_handler_unregister);
+void netdev_rx_xdp_handler_unregister(struct net_device *dev)
+{
+ ASSERT_RTNL();
+ RCU_INIT_POINTER(dev->rx_xdp_handler, NULL);
+}
+EXPORT_SYMBOL_GPL(netdev_rx_xdp_handler_unregister);
+
/*
* Limit the use of PFMEMALLOC reserves to those protocols that implement
* the special handling of PFMEMALLOC skbs.
diff --git a/net/core/filter.c b/net/core/filter.c
index 587bbfb..9ea3797 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -3312,6 +3312,34 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
}
EXPORT_SYMBOL_GPL(xdp_do_redirect);
+rx_handler_result_t xdp_do_pass(struct xdp_buff *xdp)
+{
+ rx_xdp_handler_result_t ret;
+ rx_xdp_handler_func_t *rx_xdp_handler;
+ struct net_device *dev = xdp->rxq->dev;
+
+ ret = RX_XDP_HANDLER_FALLBACK;
+ rx_xdp_handler = rcu_dereference(dev->rx_xdp_handler);
+
+ if (rx_xdp_handler) {
+ ret = rx_xdp_handler(dev, xdp);
+ switch (ret) {
+ case RX_XDP_HANDLER_CONSUMED:
+ /* Fall through */
+ case RX_XDP_HANDLER_DROP:
+ /* Fall through */
+ case RX_XDP_HANDLER_FALLBACK:
+ break;
+ default:
+ BUG();
+ break;
+ }
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(xdp_do_pass);
+
static int xdp_do_generic_redirect_map(struct net_device *dev,
struct sk_buff *skb,
struct xdp_buff *xdp,
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next 3/5] macvlan: count the number of vlan in source mode
From: Jason Wang @ 2018-08-13 3:05 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534129513-4845-1-git-send-email-jasowang@redhat.com>
This patch tries to count the number of vlans in source mode. This
will be used for implementing XDP rx handler for macvlan.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/net/macvlan.c | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index cfda146..b7c814d 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -53,6 +53,7 @@ struct macvlan_port {
struct hlist_head vlan_source_hash[MACVLAN_HASH_SIZE];
DECLARE_BITMAP(mc_filter, MACVLAN_MC_FILTER_SZ);
unsigned char perm_addr[ETH_ALEN];
+ unsigned long source_count;
};
struct macvlan_source_entry {
@@ -1433,6 +1434,9 @@ int macvlan_common_newlink(struct net *src_net, struct net_device *dev,
if (err)
goto unregister_netdev;
+ if (vlan->mode == MACVLAN_MODE_SOURCE)
+ port->source_count++;
+
list_add_tail_rcu(&vlan->list, &port->vlans);
netif_stacked_transfer_operstate(lowerdev, dev);
linkwatch_fire_event(dev);
@@ -1477,6 +1481,7 @@ static int macvlan_changelink(struct net_device *dev,
struct netlink_ext_ack *extack)
{
struct macvlan_dev *vlan = netdev_priv(dev);
+ struct macvlan_port *port = vlan->port;
enum macvlan_mode mode;
bool set_mode = false;
enum macvlan_macaddr_mode macmode;
@@ -1491,8 +1496,10 @@ static int macvlan_changelink(struct net_device *dev,
(vlan->mode == MACVLAN_MODE_PASSTHRU))
return -EINVAL;
if (vlan->mode == MACVLAN_MODE_SOURCE &&
- vlan->mode != mode)
+ vlan->mode != mode) {
macvlan_flush_sources(vlan->port, vlan);
+ port->source_count--;
+ }
}
if (data && data[IFLA_MACVLAN_FLAGS]) {
@@ -1510,8 +1517,13 @@ static int macvlan_changelink(struct net_device *dev,
}
vlan->flags = flags;
}
- if (set_mode)
+ if (set_mode) {
vlan->mode = mode;
+ if (mode == MACVLAN_MODE_SOURCE &&
+ vlan->mode != mode) {
+ port->source_count++;
+ }
+ }
if (data && data[IFLA_MACVLAN_MACADDR_MODE]) {
if (vlan->mode != MACVLAN_MODE_SOURCE)
return -EINVAL;
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next 5/5] virtio-net: support XDP rx handler
From: Jason Wang @ 2018-08-13 3:05 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534129513-4845-1-git-send-email-jasowang@redhat.com>
This patch tries to add the support of XDP rx handler to
virtio-net. This is straight-forward, just call xdp_do_pass() and
behave depends on its return value.
Test was done by using XDP_DROP (xdp1) for macvlan on top of
virtio-net. PPS of SKB mode was ~1.2Mpps while PPS of native XDP mode
was ~2.2Mpps. About 83% improvement was measured.
Notes: for RFC, only mergeable buffer case was implemented.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/net/virtio_net.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 62311dd..1e22ad9 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -777,6 +777,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
rcu_read_lock();
xdp_prog = rcu_dereference(rq->xdp_prog);
if (xdp_prog) {
+ rx_xdp_handler_result_t ret;
struct xdp_frame *xdpf;
struct page *xdp_page;
struct xdp_buff xdp;
@@ -825,6 +826,15 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
switch (act) {
case XDP_PASS:
+ ret = xdp_do_pass(&xdp);
+ if (ret == RX_XDP_HANDLER_DROP)
+ goto drop;
+ if (ret != RX_XDP_HANDLER_FALLBACK) {
+ if (unlikely(xdp_page != page))
+ put_page(page);
+ rcu_read_unlock();
+ goto xdp_xmit;
+ }
/* recalculate offset to account for any header
* adjustments. Note other cases do not build an
* skb and avoid using offset
@@ -881,6 +891,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
case XDP_ABORTED:
trace_xdp_exception(vi->dev, xdp_prog, act);
/* fall through */
+drop:
case XDP_DROP:
if (unlikely(xdp_page != page))
__free_pages(xdp_page, 0);
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next V2 0/6] XDP rx handler
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
Hi:
This series tries to implement XDP support for rx hanlder. This would
be useful for doing native XDP on stacked device like macvlan, bridge
or even bond.
The idea is simple, let stacked device register a XDP rx handler. And
when driver return XDP_PASS, it will call a new helper xdp_do_pass()
which will try to pass XDP buff to XDP rx handler directly. XDP rx
handler may then decide how to proceed, it could consume the buff, ask
driver to drop the packet or ask the driver to fallback to normal skb
path.
A sample XDP rx handler was implemented for macvlan. And virtio-net
(mergeable buffer case) was converted to call xdp_do_pass() as an
example. For ease comparision, generic XDP support for rx handler was
also implemented.
Compared to skb mode XDP on macvlan, native XDP on macvlan (XDP_DROP)
shows about 83% improvement.
Please review.
Thanks
Jason Wang (6):
net: core: factor out generic XDP check and process routine
net: core: generic XDP support for stacked device
net: core: introduce XDP rx handler
macvlan: count the number of vlan in source mode
macvlan: basic XDP support
virtio-net: support XDP rx handler
drivers/net/macvlan.c | 189 +++++++++++++++++++++++++++++++++++++++++++--
drivers/net/virtio_net.c | 11 +++
include/linux/filter.h | 1 +
include/linux/if_macvlan.h | 1 +
include/linux/netdevice.h | 12 +++
net/core/dev.c | 69 +++++++++++++----
net/core/filter.c | 28 +++++++
7 files changed, 293 insertions(+), 18 deletions(-)
--
2.7.4
^ permalink raw reply
* [RFC PATCH net-next V2 1/6] net: core: factor out generic XDP check and process routine
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534130250-5302-1-git-send-email-jasowang@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
net/core/dev.c | 35 ++++++++++++++++++++++-------------
1 file changed, 22 insertions(+), 13 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index f68122f..605c66e 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4392,13 +4392,9 @@ int do_xdp_generic(struct bpf_prog *xdp_prog, struct sk_buff *skb)
}
EXPORT_SYMBOL_GPL(do_xdp_generic);
-static int netif_rx_internal(struct sk_buff *skb)
+static int netif_do_generic_xdp(struct sk_buff *skb)
{
- int ret;
-
- net_timestamp_check(netdev_tstamp_prequeue, skb);
-
- trace_netif_rx(skb);
+ int ret = XDP_PASS;
if (static_branch_unlikely(&generic_xdp_needed_key)) {
int ret;
@@ -4408,15 +4404,28 @@ static int netif_rx_internal(struct sk_buff *skb)
ret = do_xdp_generic(rcu_dereference(skb->dev->xdp_prog), skb);
rcu_read_unlock();
preempt_enable();
-
- /* Consider XDP consuming the packet a success from
- * the netdev point of view we do not want to count
- * this as an error.
- */
- if (ret != XDP_PASS)
- return NET_RX_SUCCESS;
}
+ return ret;
+}
+
+static int netif_rx_internal(struct sk_buff *skb)
+{
+ int ret;
+
+ net_timestamp_check(netdev_tstamp_prequeue, skb);
+
+ trace_netif_rx(skb);
+
+ ret = netif_do_generic_xdp(skb);
+
+ /* Consider XDP consuming the packet a success from
+ * the netdev point of view we do not want to count
+ * this as an error.
+ */
+ if (ret != XDP_PASS)
+ return NET_RX_SUCCESS;
+
#ifdef CONFIG_RPS
if (static_key_false(&rps_needed)) {
struct rps_dev_flow voidflow, *rflow = &voidflow;
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next V2 2/6] net: core: generic XDP support for stacked device
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534130250-5302-1-git-send-email-jasowang@redhat.com>
Stacked device usually change skb->dev to its own and return
RX_HANDLER_ANOTHER during rx handler processing. But we don't call
generic XDP routine at that time, this means it can't work for stacked
device.
Fixing this by calling netif_do_generic_xdp() if rx handler returns
RX_HANDLER_ANOTHER. This allows us to do generic XDP on stacked device
e.g macvlan.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
net/core/dev.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/net/core/dev.c b/net/core/dev.c
index 605c66e..a77ce08 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4822,6 +4822,11 @@ static int __netif_receive_skb_core(struct sk_buff *skb, bool pfmemalloc,
ret = NET_RX_SUCCESS;
goto out;
case RX_HANDLER_ANOTHER:
+ ret = netif_do_generic_xdp(skb);
+ if (ret != XDP_PASS) {
+ ret = NET_RX_SUCCESS;
+ goto out;
+ }
goto another_round;
case RX_HANDLER_EXACT:
deliver_exact = true;
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next V2 3/6] net: core: introduce XDP rx handler
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534130250-5302-1-git-send-email-jasowang@redhat.com>
This patch tries to introduce XDP rx handler. This will be used by
stacked device that depends on rx handler for having a fast packet
processing path based on XDP.
This idea is simple, when XDP program returns XDP_PASS, instead of
building skb immediately, driver will call xdp_do_pass() to check
whether or not there's a XDP rx handler, if yes, it will pass XDP
buffer to XDP rx handler first.
There are two main tasks for XDP rx handler, the first is check
whether or not the setup or packet could be processed through XDP buff
directly. The second task is to run XDP program. An XDP rx handler can
return several different results which was defined by enum
rx_xdp_handler_result_t:
RX_XDP_HANDLER_CONSUMED: This means the XDP buff were consumed.
RX_XDP_HANDLER_DROP: This means XDP rx handler ask to drop the packet.
RX_XDP_HANDLER_PASS_FALLBACK: This means XDP rx handler can not
process the packet (e.g cloning), and we need to fall back to normal
skb path to deal with the packet.
Consider we have the following configuration, Level 0 device which has
a rx handler for Level 1 device which has a rx handler for L2 device.
L2 device
|
L1 device
|
L0 device
With the help of XDP rx handler, we can attach XDP program on each of
the layer or even run native XDP handler for L2 without XDP prog
attached to L1 device:
(XDP prog for L2 device)
|
L2 XDP rx handler for L1
|
(XDP prog for L1 device)
|
L1 XDP rx hanlder for L0
|
XDP prog for L0 device
It works like: When the XDP program for L0 device returns XDP_PASS, we
will first try to check and pass XDP buff to its XDP rx handler if
there's one. Then the L1 XDP rx handler will be called and to run XDP
program for L1. When L1 XDP program returns XDP_PASS or there's no XDP
program attached to L1, we will try to call xdp_do_pass() to pass it
to XDP rx hanlder for L1. Then XDP buff will be passed to L2 XDP rx
handler etc. And it will try to run L2 XDP program if any. And if
there's no L2 XDP program or XDP program returns XDP_PASS. The handler
usually will build skb and call netif_rx() for a local receive. If any
of the XDP rx handlers returns XDP_RX_HANDLER_FALLBACK, the code will
return to L0 device and L0 device will try to build skb and go through
normal rx handler path for skb.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
include/linux/filter.h | 1 +
include/linux/netdevice.h | 12 ++++++++++++
net/core/dev.c | 29 +++++++++++++++++++++++++++++
net/core/filter.c | 28 ++++++++++++++++++++++++++++
4 files changed, 70 insertions(+)
diff --git a/include/linux/filter.h b/include/linux/filter.h
index c73dd73..7cc8e69 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -791,6 +791,7 @@ int xdp_do_generic_redirect(struct net_device *dev, struct sk_buff *skb,
int xdp_do_redirect(struct net_device *dev,
struct xdp_buff *xdp,
struct bpf_prog *prog);
+rx_handler_result_t xdp_do_pass(struct xdp_buff *xdp);
void xdp_do_flush_map(void);
void bpf_warn_invalid_xdp_action(u32 act);
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 282e2e9..21f0a9e 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -421,6 +421,14 @@ enum rx_handler_result {
typedef enum rx_handler_result rx_handler_result_t;
typedef rx_handler_result_t rx_handler_func_t(struct sk_buff **pskb);
+enum rx_xdp_handler_result {
+ RX_XDP_HANDLER_CONSUMED,
+ RX_XDP_HANDLER_DROP,
+ RX_XDP_HANDLER_FALLBACK,
+};
+typedef enum rx_xdp_handler_result rx_xdp_handler_result_t;
+typedef rx_xdp_handler_result_t rx_xdp_handler_func_t(struct net_device *dev,
+ struct xdp_buff *xdp);
void __napi_schedule(struct napi_struct *n);
void __napi_schedule_irqoff(struct napi_struct *n);
@@ -1898,6 +1906,7 @@ struct net_device {
struct bpf_prog __rcu *xdp_prog;
unsigned long gro_flush_timeout;
rx_handler_func_t __rcu *rx_handler;
+ rx_xdp_handler_func_t __rcu *rx_xdp_handler;
void __rcu *rx_handler_data;
#ifdef CONFIG_NET_CLS_ACT
@@ -3530,7 +3539,10 @@ bool netdev_is_rx_handler_busy(struct net_device *dev);
int netdev_rx_handler_register(struct net_device *dev,
rx_handler_func_t *rx_handler,
void *rx_handler_data);
+int netdev_rx_xdp_handler_register(struct net_device *dev,
+ rx_xdp_handler_func_t *rx_xdp_handler);
void netdev_rx_handler_unregister(struct net_device *dev);
+void netdev_rx_xdp_handler_unregister(struct net_device *dev);
bool dev_valid_name(const char *name);
int dev_ioctl(struct net *net, unsigned int cmd, struct ifreq *ifr,
diff --git a/net/core/dev.c b/net/core/dev.c
index a77ce08..b4e8949 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4638,6 +4638,12 @@ bool netdev_is_rx_handler_busy(struct net_device *dev)
}
EXPORT_SYMBOL_GPL(netdev_is_rx_handler_busy);
+static bool netdev_is_rx_xdp_handler_busy(struct net_device *dev)
+{
+ ASSERT_RTNL();
+ return dev && rtnl_dereference(dev->rx_xdp_handler);
+}
+
/**
* netdev_rx_handler_register - register receive handler
* @dev: device to register a handler for
@@ -4670,6 +4676,22 @@ int netdev_rx_handler_register(struct net_device *dev,
}
EXPORT_SYMBOL_GPL(netdev_rx_handler_register);
+int netdev_rx_xdp_handler_register(struct net_device *dev,
+ rx_xdp_handler_func_t *rx_xdp_handler)
+{
+ if (netdev_is_rx_xdp_handler_busy(dev))
+ return -EBUSY;
+
+ if (dev->priv_flags & IFF_NO_RX_HANDLER)
+ return -EINVAL;
+
+ rcu_assign_pointer(dev->rx_xdp_handler, rx_xdp_handler);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(netdev_rx_xdp_handler_register);
+
+
/**
* netdev_rx_handler_unregister - unregister receive handler
* @dev: device to unregister a handler from
@@ -4692,6 +4714,13 @@ void netdev_rx_handler_unregister(struct net_device *dev)
}
EXPORT_SYMBOL_GPL(netdev_rx_handler_unregister);
+void netdev_rx_xdp_handler_unregister(struct net_device *dev)
+{
+ ASSERT_RTNL();
+ RCU_INIT_POINTER(dev->rx_xdp_handler, NULL);
+}
+EXPORT_SYMBOL_GPL(netdev_rx_xdp_handler_unregister);
+
/*
* Limit the use of PFMEMALLOC reserves to those protocols that implement
* the special handling of PFMEMALLOC skbs.
diff --git a/net/core/filter.c b/net/core/filter.c
index 587bbfb..9ea3797 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -3312,6 +3312,34 @@ int xdp_do_redirect(struct net_device *dev, struct xdp_buff *xdp,
}
EXPORT_SYMBOL_GPL(xdp_do_redirect);
+rx_handler_result_t xdp_do_pass(struct xdp_buff *xdp)
+{
+ rx_xdp_handler_result_t ret;
+ rx_xdp_handler_func_t *rx_xdp_handler;
+ struct net_device *dev = xdp->rxq->dev;
+
+ ret = RX_XDP_HANDLER_FALLBACK;
+ rx_xdp_handler = rcu_dereference(dev->rx_xdp_handler);
+
+ if (rx_xdp_handler) {
+ ret = rx_xdp_handler(dev, xdp);
+ switch (ret) {
+ case RX_XDP_HANDLER_CONSUMED:
+ /* Fall through */
+ case RX_XDP_HANDLER_DROP:
+ /* Fall through */
+ case RX_XDP_HANDLER_FALLBACK:
+ break;
+ default:
+ BUG();
+ break;
+ }
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(xdp_do_pass);
+
static int xdp_do_generic_redirect_map(struct net_device *dev,
struct sk_buff *skb,
struct xdp_buff *xdp,
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next V2 4/6] macvlan: count the number of vlan in source mode
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534130250-5302-1-git-send-email-jasowang@redhat.com>
This patch tries to count the number of vlans in source mode. This
will be used for implementing XDP rx handler for macvlan.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/net/macvlan.c | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index cfda146..b7c814d 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -53,6 +53,7 @@ struct macvlan_port {
struct hlist_head vlan_source_hash[MACVLAN_HASH_SIZE];
DECLARE_BITMAP(mc_filter, MACVLAN_MC_FILTER_SZ);
unsigned char perm_addr[ETH_ALEN];
+ unsigned long source_count;
};
struct macvlan_source_entry {
@@ -1433,6 +1434,9 @@ int macvlan_common_newlink(struct net *src_net, struct net_device *dev,
if (err)
goto unregister_netdev;
+ if (vlan->mode == MACVLAN_MODE_SOURCE)
+ port->source_count++;
+
list_add_tail_rcu(&vlan->list, &port->vlans);
netif_stacked_transfer_operstate(lowerdev, dev);
linkwatch_fire_event(dev);
@@ -1477,6 +1481,7 @@ static int macvlan_changelink(struct net_device *dev,
struct netlink_ext_ack *extack)
{
struct macvlan_dev *vlan = netdev_priv(dev);
+ struct macvlan_port *port = vlan->port;
enum macvlan_mode mode;
bool set_mode = false;
enum macvlan_macaddr_mode macmode;
@@ -1491,8 +1496,10 @@ static int macvlan_changelink(struct net_device *dev,
(vlan->mode == MACVLAN_MODE_PASSTHRU))
return -EINVAL;
if (vlan->mode == MACVLAN_MODE_SOURCE &&
- vlan->mode != mode)
+ vlan->mode != mode) {
macvlan_flush_sources(vlan->port, vlan);
+ port->source_count--;
+ }
}
if (data && data[IFLA_MACVLAN_FLAGS]) {
@@ -1510,8 +1517,13 @@ static int macvlan_changelink(struct net_device *dev,
}
vlan->flags = flags;
}
- if (set_mode)
+ if (set_mode) {
vlan->mode = mode;
+ if (mode == MACVLAN_MODE_SOURCE &&
+ vlan->mode != mode) {
+ port->source_count++;
+ }
+ }
if (data && data[IFLA_MACVLAN_MACADDR_MODE]) {
if (vlan->mode != MACVLAN_MODE_SOURCE)
return -EINVAL;
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next V2 5/6] macvlan: basic XDP support
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534130250-5302-1-git-send-email-jasowang@redhat.com>
This patch tries to implementing basic XDP support for macvlan. The
implementation was split into two parts:
1) XDP rx handler of underlay device:
We will register an XDP rx handler (macvlan_handle_xdp) to under layer
device. In this handler, we will the following cases to go for slow
path (XDP_RX_HANDLER_PASS):
- The packet is a multicast packet.
- A vlan is source mode
- Destination mac address does not match any vlan
If none of the above cases were true, it means we could go for XDP
path directly. We will change the dev and return
RX_XDP_HANDLER_ANOTHER.
2) If we find a destination vlan, we will try to run XDP prog.
If XDP prog return XDP_PASS, we will call xdp_do_pass() to pass it to
up layer XDP rx handler. This is needed for e.g macvtap to work. If
XDP_RX_HANDLER_FALLBACK is returned, we will build skb and call
netif_rx() to finish the receiving. Otherwise just return the result
to lower device. For XDP_TX, we will build skb and try XDP generic
transmission routine for simplicity. This could be optimized on top.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/net/macvlan.c | 173 ++++++++++++++++++++++++++++++++++++++++++++-
include/linux/if_macvlan.h | 1 +
2 files changed, 171 insertions(+), 3 deletions(-)
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index b7c814d..42b747c 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -34,6 +34,7 @@
#include <net/rtnetlink.h>
#include <net/xfrm.h>
#include <linux/netpoll.h>
+#include <linux/bpf.h>
#define MACVLAN_HASH_BITS 8
#define MACVLAN_HASH_SIZE (1<<MACVLAN_HASH_BITS)
@@ -436,6 +437,122 @@ static void macvlan_forward_source(struct sk_buff *skb,
}
}
+struct sk_buff *macvlan_xdp_build_skb(struct net_device *dev,
+ struct xdp_buff *xdp)
+{
+ int len;
+ int buflen = xdp->data_end - xdp->data_hard_start;
+ int headroom = xdp->data - xdp->data_hard_start;
+ struct sk_buff *skb;
+
+ len = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + headroom +
+ SKB_DATA_ALIGN(buflen);
+
+ skb = build_skb(xdp->data_hard_start, len);
+ if (!skb)
+ return NULL;
+
+ skb_reserve(skb, headroom);
+ __skb_put(skb, xdp->data_end - xdp->data);
+
+ skb->protocol = eth_type_trans(skb, dev);
+ skb->dev = dev;
+
+ return skb;
+}
+
+static rx_xdp_handler_result_t macvlan_receive_xdp(struct net_device *dev,
+ struct xdp_buff *xdp)
+{
+ struct macvlan_dev *vlan = netdev_priv(dev);
+ struct bpf_prog *xdp_prog;
+ struct sk_buff *skb;
+ u32 act = XDP_PASS;
+ rx_xdp_handler_result_t ret;
+ int err;
+
+ rcu_read_lock();
+ xdp_prog = rcu_dereference(vlan->xdp_prog);
+
+ if (xdp_prog)
+ act = bpf_prog_run_xdp(xdp_prog, xdp);
+
+ switch (act) {
+ case XDP_PASS:
+ ret = xdp_do_pass(xdp);
+ if (ret != RX_XDP_HANDLER_FALLBACK) {
+ rcu_read_unlock();
+ return ret;
+ }
+ skb = macvlan_xdp_build_skb(dev, xdp);
+ if (!skb) {
+ act = XDP_DROP;
+ break;
+ }
+ rcu_read_unlock();
+ netif_rx(skb);
+ macvlan_count_rx(vlan, skb->len, true, false);
+ goto out;
+ case XDP_TX:
+ skb = macvlan_xdp_build_skb(dev, xdp);
+ if (!skb) {
+ act = XDP_DROP;
+ break;
+ }
+ generic_xdp_tx(skb, xdp_prog);
+ break;
+ case XDP_REDIRECT:
+ err = xdp_do_redirect(dev, xdp, xdp_prog);
+ xdp_do_flush_map();
+ if (err)
+ act = XDP_DROP;
+ break;
+ case XDP_DROP:
+ break;
+ default:
+ bpf_warn_invalid_xdp_action(act);
+ break;
+ }
+
+ rcu_read_unlock();
+out:
+ if (act == XDP_DROP)
+ return RX_XDP_HANDLER_DROP;
+
+ return RX_XDP_HANDLER_CONSUMED;
+}
+
+/* called under rcu_read_lock() from XDP handler */
+static rx_xdp_handler_result_t macvlan_handle_xdp(struct net_device *dev,
+ struct xdp_buff *xdp)
+{
+ const struct ethhdr *eth = (const struct ethhdr *)xdp->data;
+ struct macvlan_port *port;
+ struct macvlan_dev *vlan;
+
+ if (is_multicast_ether_addr(eth->h_dest))
+ return RX_XDP_HANDLER_FALLBACK;
+
+ port = macvlan_port_get_rcu(dev);
+ if (port->source_count)
+ return RX_XDP_HANDLER_FALLBACK;
+
+ if (macvlan_passthru(port))
+ vlan = list_first_or_null_rcu(&port->vlans,
+ struct macvlan_dev, list);
+ else
+ vlan = macvlan_hash_lookup(port, eth->h_dest);
+
+ if (!vlan)
+ return RX_XDP_HANDLER_FALLBACK;
+
+ dev = vlan->dev;
+ if (unlikely(!(dev->flags & IFF_UP)))
+ return RX_XDP_HANDLER_DROP;
+
+ return macvlan_receive_xdp(dev, xdp);
+}
+
/* called under rcu_read_lock() from netif_receive_skb */
static rx_handler_result_t macvlan_handle_frame(struct sk_buff **pskb)
{
@@ -1089,6 +1206,44 @@ static int macvlan_dev_get_iflink(const struct net_device *dev)
return vlan->lowerdev->ifindex;
}
+static int macvlan_xdp_set(struct net_device *dev, struct bpf_prog *prog,
+ struct netlink_ext_ack *extack)
+{
+ struct macvlan_dev *vlan = netdev_priv(dev);
+ struct bpf_prog *old_prog = rtnl_dereference(vlan->xdp_prog);
+
+ rcu_assign_pointer(vlan->xdp_prog, prog);
+
+ if (old_prog)
+ bpf_prog_put(old_prog);
+
+ return 0;
+}
+
+static u32 macvlan_xdp_query(struct net_device *dev)
+{
+ struct macvlan_dev *vlan = netdev_priv(dev);
+ const struct bpf_prog *xdp_prog = rtnl_dereference(vlan->xdp_prog);
+
+ if (xdp_prog)
+ return xdp_prog->aux->id;
+
+ return 0;
+}
+
+static int macvlan_xdp(struct net_device *dev, struct netdev_bpf *xdp)
+{
+ switch (xdp->command) {
+ case XDP_SETUP_PROG:
+ return macvlan_xdp_set(dev, xdp->prog, xdp->extack);
+ case XDP_QUERY_PROG:
+ xdp->prog_id = macvlan_xdp_query(dev);
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
static const struct ethtool_ops macvlan_ethtool_ops = {
.get_link = ethtool_op_get_link,
.get_link_ksettings = macvlan_ethtool_get_link_ksettings,
@@ -1121,6 +1276,7 @@ static const struct net_device_ops macvlan_netdev_ops = {
#endif
.ndo_get_iflink = macvlan_dev_get_iflink,
.ndo_features_check = passthru_features_check,
+ .ndo_bpf = macvlan_xdp,
};
void macvlan_common_setup(struct net_device *dev)
@@ -1173,10 +1329,20 @@ static int macvlan_port_create(struct net_device *dev)
INIT_WORK(&port->bc_work, macvlan_process_broadcast);
err = netdev_rx_handler_register(dev, macvlan_handle_frame, port);
- if (err)
+ if (err) {
kfree(port);
- else
- dev->priv_flags |= IFF_MACVLAN_PORT;
+ goto out;
+ }
+
+ err = netdev_rx_xdp_handler_register(dev, macvlan_handle_xdp);
+ if (err) {
+ netdev_rx_handler_unregister(dev);
+ kfree(port);
+ goto out;
+ }
+
+ dev->priv_flags |= IFF_MACVLAN_PORT;
+out:
return err;
}
@@ -1187,6 +1353,7 @@ static void macvlan_port_destroy(struct net_device *dev)
dev->priv_flags &= ~IFF_MACVLAN_PORT;
netdev_rx_handler_unregister(dev);
+ netdev_rx_xdp_handler_unregister(dev);
/* After this point, no packet can schedule bc_work anymore,
* but we need to cancel it and purge left skbs if any.
diff --git a/include/linux/if_macvlan.h b/include/linux/if_macvlan.h
index 2e55e4c..7c7059b 100644
--- a/include/linux/if_macvlan.h
+++ b/include/linux/if_macvlan.h
@@ -34,6 +34,7 @@ struct macvlan_dev {
#ifdef CONFIG_NET_POLL_CONTROLLER
struct netpoll *netpoll;
#endif
+ struct bpf_prog __rcu *xdp_prog;
};
static inline void macvlan_count_rx(const struct macvlan_dev *vlan,
--
2.7.4
^ permalink raw reply related
* [RFC PATCH net-next V2 6/6] virtio-net: support XDP rx handler
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534130250-5302-1-git-send-email-jasowang@redhat.com>
This patch tries to add the support of XDP rx handler to
virtio-net. This is straight-forward, just call xdp_do_pass() and
behave depends on its return value.
Test was done by using XDP_DROP (xdp1) for macvlan on top of
virtio-net. PPS of SKB mode was ~1.2Mpps while PPS of native XDP mode
was ~2.2Mpps. About 83% improvement was measured.
Notes: for RFC, only mergeable buffer case was implemented.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/net/virtio_net.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 62311dd..1e22ad9 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -777,6 +777,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
rcu_read_lock();
xdp_prog = rcu_dereference(rq->xdp_prog);
if (xdp_prog) {
+ rx_xdp_handler_result_t ret;
struct xdp_frame *xdpf;
struct page *xdp_page;
struct xdp_buff xdp;
@@ -825,6 +826,15 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
switch (act) {
case XDP_PASS:
+ ret = xdp_do_pass(&xdp);
+ if (ret == RX_XDP_HANDLER_DROP)
+ goto drop;
+ if (ret != RX_XDP_HANDLER_FALLBACK) {
+ if (unlikely(xdp_page != page))
+ put_page(page);
+ rcu_read_unlock();
+ goto xdp_xmit;
+ }
/* recalculate offset to account for any header
* adjustments. Note other cases do not build an
* skb and avoid using offset
@@ -881,6 +891,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
case XDP_ABORTED:
trace_xdp_exception(vi->dev, xdp_prog, act);
/* fall through */
+drop:
case XDP_DROP:
if (unlikely(xdp_page != page))
__free_pages(xdp_page, 0);
--
2.7.4
^ permalink raw reply related
* Re: [RFC PATCH net-next 0/5] XDP rx handler
From: Jason Wang @ 2018-08-13 3:17 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst
In-Reply-To: <1534129513-4845-1-git-send-email-jasowang@redhat.com>
On 2018年08月13日 11:05, Jason Wang wrote:
> Hi:
>
> This series tries to implement XDP support for rx hanlder. This would
> be useful for doing native XDP on stacked device like macvlan, bridge
> or even bond.
>
> The idea is simple, let stacked device register a XDP rx handler. And
> when driver return XDP_PASS, it will call a new helper xdp_do_pass()
> which will try to pass XDP buff to XDP rx handler directly. XDP rx
> handler may then decide how to proceed, it could consume the buff, ask
> driver to drop the packet or ask the driver to fallback to normal skb
> path.
>
> A sample XDP rx handler was implemented for macvlan. And virtio-net
> (mergeable buffer case) was converted to call xdp_do_pass() as an
> example. For ease comparision, generic XDP support for rx handler was
> also implemented.
>
> Compared to skb mode XDP on macvlan, native XDP on macvlan (XDP_DROP)
> shows about 83% improvement.
>
> Please review.
>
> Thanks
>
> Jason Wang (5):
> net: core: generic XDP support for stacked device
> net: core: introduce XDP rx handler
> macvlan: count the number of vlan in source mode
> macvlan: basic XDP support
> virtio-net: support XDP rx handler
>
> drivers/net/macvlan.c | 189 +++++++++++++++++++++++++++++++++++++++++++--
> drivers/net/virtio_net.c | 11 +++
> include/linux/filter.h | 1 +
> include/linux/if_macvlan.h | 1 +
> include/linux/netdevice.h | 12 +++
> net/core/dev.c | 34 ++++++++
> net/core/filter.c | 28 +++++++
> 7 files changed, 271 insertions(+), 5 deletions(-)
>
Looks like a patch is missed. Let me post V2.
Thanks
^ permalink raw reply
* Re: [PATCH net-next] openvswitch: Derive IP protocol number for IPv6 later frags
From: Pravin Shelar @ 2018-08-13 1:09 UTC (permalink / raw)
To: Yi-Hung Wei; +Cc: Linux Kernel Network Developers, William Tu
In-Reply-To: <1533921581-22806-1-git-send-email-yihung.wei@gmail.com>
On Fri, Aug 10, 2018 at 10:19 AM, Yi-Hung Wei <yihung.wei@gmail.com> wrote:
> Currently, OVS only parses the IP protocol number for the first
> IPv6 fragment, but sets the IP protocol number for the later fragments
> to be NEXTHDF_FRAGMENT. This patch tries to derive the IP protocol
> number for the IPV6 later frags so that we can match that.
>
> Signed-off-by: Yi-Hung Wei <yihung.wei@gmail.com>
> ---
> net/openvswitch/flow.c | 8 +++++++-
> 1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/net/openvswitch/flow.c b/net/openvswitch/flow.c
> index 56b8e7167790..3d654c4f71be 100644
> --- a/net/openvswitch/flow.c
> +++ b/net/openvswitch/flow.c
> @@ -297,7 +297,13 @@ static int parse_ipv6hdr(struct sk_buff *skb, struct sw_flow_key *key)
>
> nh_len = payload_ofs - nh_ofs;
> skb_set_transport_header(skb, nh_ofs + nh_len);
> - key->ip.proto = nexthdr;
> + if (key->ip.frag == OVS_FRAG_TYPE_LATER) {
> + unsigned int offset = 0;
> +
> + key->ip.proto = ipv6_find_hdr(skb, &offset, -1, NULL, NULL);
> + } else {
> + key->ip.proto = nexthdr;
> + }
parsing ipv6 ipv6_skip_exthdr() is called to find fragment hdr and
then this patch calls ipv6_find_hdr() to find next protocol. I think
we could call ipv6_find_hdr() to get fragment type and next hdr, that
would save parsing same packet twice in some cases.
Other option would be calling ipv6_find_hdr() after setting OVS_FRAG_TYPE_LATER.
^ permalink raw reply
* Re: [PATCH] mt76: Enable NL80211_EXT_FEATURE_CQM_RSSI_LIST
From: Kristian Evensen @ 2018-08-13 4:58 UTC (permalink / raw)
To: arend.vanspriel; +Cc: kvalo, linux-wireless, Network Development, linux-kernel
In-Reply-To: <5B708025.4090906@broadcom.com>
Hi Kalle & Arnd,
On Sun, Aug 12, 2018 at 8:44 PM Arend van Spriel
<arend.vanspriel@broadcom.com> wrote:
> > So have you tested this and with what devices? For example, does it work
> > with recently added USB devices?
>
> I was looking into this as it looks suspicious to me. From reading the
> description of this ext_feature flag it seems this is an extention of CQM:
Thank you very much for your feedback. My commit message should have
been more detailed, sorry about that. I have checked that the flag
works as intended with mt7602-, mt7603- and mt7612-based wifi cards. I
have not had the opportunity to test with any of the recently added
USB devices, as I don't have access to any of those.
In order to test the flag, I wrote a small program which subscribes to
the CQM-multicast group, passes an RSSI threshold-list to the kernel
and logs the received CQM-events. I then disconnected and connected
the wifi-antennas of the different cards. My threshold list was {-70,
-60, -50, -40} and while unscrewing the antenna I received multiple
below-events. When I attached the antenna again, I received multiple
above-events. As an example, here is the log when I tested with mt7612
(singal level when starting was ~-48 dBm):
Requested nl80211 generic netlink id
nl80211 has generic netlink id: 23
mlme ID is 5
Added socket to mlme group
Sent NL80211_CMD_SET_CQM
No error
Wifi (idx 18) went below threshold. RSSI -52
Wifi (idx 18) went above threshold. RSSI -49
Wifi (idx 18) went below threshold. RSSI -52
Wifi (idx 18) went below threshold. RSSI -62
Wifi (idx 18) went above threshold. RSSI -59
Wifi (idx 18) went above threshold. RSSI -49
Based on how I interpret the output and my understanding of how CQM +
RSSI_LIST works, this output shows that mt76 works fine with
NL80211_EXT_FEATURE_CQM_RSSI_LIST (at least for my cards). The list
was interpreted and handled correctly, as I received events when the
RSSI passed different thresholds in my list (-50, -60).
BR,
Kristian
^ permalink raw reply
* [PATCH][net-next][v2] packet: switch kvzalloc to allocate memory
From: Li RongQing @ 2018-08-13 2:42 UTC (permalink / raw)
To: netdev
The patches includes following change:
*Use modern kvzalloc()/kvfree() instead of custom allocations.
*Remove order argument for alloc_pg_vec, it can get from req.
*Remove order argument for free_pg_vec, free_pg_vec now uses
kvfree which does not need order argument.
*Remove pg_vec_order from struct packet_ring_buffer, no longer
need to save/restore 'order'
*Remove variable 'order' for packet_set_ring, it is now unused
Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
Signed-off-by: Li RongQing <lirongqing@baidu.com>
---
net/packet/af_packet.c | 44 +++++++++++++-------------------------------
net/packet/internal.h | 1 -
2 files changed, 13 insertions(+), 32 deletions(-)
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index 75c92a87e7b2..5610061e7f2e 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -4137,52 +4137,36 @@ static const struct vm_operations_struct packet_mmap_ops = {
.close = packet_mm_close,
};
-static void free_pg_vec(struct pgv *pg_vec, unsigned int order,
- unsigned int len)
+static void free_pg_vec(struct pgv *pg_vec, unsigned int len)
{
int i;
for (i = 0; i < len; i++) {
if (likely(pg_vec[i].buffer)) {
- if (is_vmalloc_addr(pg_vec[i].buffer))
- vfree(pg_vec[i].buffer);
- else
- free_pages((unsigned long)pg_vec[i].buffer,
- order);
+ kvfree(pg_vec[i].buffer);
pg_vec[i].buffer = NULL;
}
}
kfree(pg_vec);
}
-static char *alloc_one_pg_vec_page(unsigned long order)
+static char *alloc_one_pg_vec_page(unsigned long size)
{
char *buffer;
- gfp_t gfp_flags = GFP_KERNEL | __GFP_COMP |
- __GFP_ZERO | __GFP_NOWARN | __GFP_NORETRY;
- buffer = (char *) __get_free_pages(gfp_flags, order);
+ buffer = kvzalloc(size, GFP_KERNEL);
if (buffer)
return buffer;
- /* __get_free_pages failed, fall back to vmalloc */
- buffer = vzalloc(array_size((1 << order), PAGE_SIZE));
- if (buffer)
- return buffer;
+ buffer = kvzalloc(size, GFP_KERNEL | __GFP_RETRY_MAYFAIL);
- /* vmalloc failed, lets dig into swap here */
- gfp_flags &= ~__GFP_NORETRY;
- buffer = (char *) __get_free_pages(gfp_flags, order);
- if (buffer)
- return buffer;
-
- /* complete and utter failure */
- return NULL;
+ return buffer;
}
-static struct pgv *alloc_pg_vec(struct tpacket_req *req, int order)
+static struct pgv *alloc_pg_vec(struct tpacket_req *req)
{
unsigned int block_nr = req->tp_block_nr;
+ unsigned long size = req->tp_block_size;
struct pgv *pg_vec;
int i;
@@ -4191,7 +4175,7 @@ static struct pgv *alloc_pg_vec(struct tpacket_req *req, int order)
goto out;
for (i = 0; i < block_nr; i++) {
- pg_vec[i].buffer = alloc_one_pg_vec_page(order);
+ pg_vec[i].buffer = alloc_one_pg_vec_page(size);
if (unlikely(!pg_vec[i].buffer))
goto out_free_pgvec;
}
@@ -4200,7 +4184,7 @@ static struct pgv *alloc_pg_vec(struct tpacket_req *req, int order)
return pg_vec;
out_free_pgvec:
- free_pg_vec(pg_vec, order, block_nr);
+ free_pg_vec(pg_vec, block_nr);
pg_vec = NULL;
goto out;
}
@@ -4210,9 +4194,9 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u,
{
struct pgv *pg_vec = NULL;
struct packet_sock *po = pkt_sk(sk);
- int was_running, order = 0;
struct packet_ring_buffer *rb;
struct sk_buff_head *rb_queue;
+ int was_running;
__be16 num;
int err = -EINVAL;
/* Added to avoid minimal code churn */
@@ -4274,8 +4258,7 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u,
goto out;
err = -ENOMEM;
- order = get_order(req->tp_block_size);
- pg_vec = alloc_pg_vec(req, order);
+ pg_vec = alloc_pg_vec(req);
if (unlikely(!pg_vec))
goto out;
switch (po->tp_version) {
@@ -4329,7 +4312,6 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u,
rb->frame_size = req->tp_frame_size;
spin_unlock_bh(&rb_queue->lock);
- swap(rb->pg_vec_order, order);
swap(rb->pg_vec_len, req->tp_block_nr);
rb->pg_vec_pages = req->tp_block_size/PAGE_SIZE;
@@ -4355,7 +4337,7 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u,
}
if (pg_vec)
- free_pg_vec(pg_vec, order, req->tp_block_nr);
+ free_pg_vec(pg_vec, req->tp_block_nr);
out:
return err;
}
diff --git a/net/packet/internal.h b/net/packet/internal.h
index 3bb7c5fb3bff..8f50036f62f0 100644
--- a/net/packet/internal.h
+++ b/net/packet/internal.h
@@ -64,7 +64,6 @@ struct packet_ring_buffer {
unsigned int frame_size;
unsigned int frame_max;
- unsigned int pg_vec_order;
unsigned int pg_vec_pages;
unsigned int pg_vec_len;
--
2.16.2
^ permalink raw reply related
* [RFC PATCH net-next 0/5] XDP rx handler
From: Jason Wang @ 2018-08-13 3:05 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
Hi:
This series tries to implement XDP support for rx hanlder. This would
be useful for doing native XDP on stacked device like macvlan, bridge
or even bond.
The idea is simple, let stacked device register a XDP rx handler. And
when driver return XDP_PASS, it will call a new helper xdp_do_pass()
which will try to pass XDP buff to XDP rx handler directly. XDP rx
handler may then decide how to proceed, it could consume the buff, ask
driver to drop the packet or ask the driver to fallback to normal skb
path.
A sample XDP rx handler was implemented for macvlan. And virtio-net
(mergeable buffer case) was converted to call xdp_do_pass() as an
example. For ease comparision, generic XDP support for rx handler was
also implemented.
Compared to skb mode XDP on macvlan, native XDP on macvlan (XDP_DROP)
shows about 83% improvement.
Please review.
Thanks
Jason Wang (5):
net: core: generic XDP support for stacked device
net: core: introduce XDP rx handler
macvlan: count the number of vlan in source mode
macvlan: basic XDP support
virtio-net: support XDP rx handler
drivers/net/macvlan.c | 189 +++++++++++++++++++++++++++++++++++++++++++--
drivers/net/virtio_net.c | 11 +++
include/linux/filter.h | 1 +
include/linux/if_macvlan.h | 1 +
include/linux/netdevice.h | 12 +++
net/core/dev.c | 34 ++++++++
net/core/filter.c | 28 +++++++
7 files changed, 271 insertions(+), 5 deletions(-)
--
2.7.4
^ permalink raw reply
* [RFC PATCH net-next 4/5] macvlan: basic XDP support
From: Jason Wang @ 2018-08-13 3:05 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: ast, daniel, jbrouer, mst, Jason Wang
In-Reply-To: <1534129513-4845-1-git-send-email-jasowang@redhat.com>
This patch tries to implementing basic XDP support for macvlan. The
implementation was split into two parts:
1) XDP rx handler of underlay device:
We will register an XDP rx handler (macvlan_handle_xdp) to under layer
device. In this handler, we will the following cases to go for slow
path (XDP_RX_HANDLER_PASS):
- The packet is a multicast packet.
- A vlan is source mode
- Destination mac address does not match any vlan
If none of the above cases were true, it means we could go for XDP
path directly. We will change the dev and return
RX_XDP_HANDLER_ANOTHER.
2) If we find a destination vlan, we will try to run XDP prog.
If XDP prog return XDP_PASS, we will call xdp_do_pass() to pass it to
up layer XDP rx handler. This is needed for e.g macvtap to work. If
XDP_RX_HANDLER_FALLBACK is returned, we will build skb and call
netif_rx() to finish the receiving. Otherwise just return the result
to lower device. For XDP_TX, we will build skb and try XDP generic
transmission routine for simplicity. This could be optimized on top.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
drivers/net/macvlan.c | 173 ++++++++++++++++++++++++++++++++++++++++++++-
include/linux/if_macvlan.h | 1 +
2 files changed, 171 insertions(+), 3 deletions(-)
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index b7c814d..42b747c 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -34,6 +34,7 @@
#include <net/rtnetlink.h>
#include <net/xfrm.h>
#include <linux/netpoll.h>
+#include <linux/bpf.h>
#define MACVLAN_HASH_BITS 8
#define MACVLAN_HASH_SIZE (1<<MACVLAN_HASH_BITS)
@@ -436,6 +437,122 @@ static void macvlan_forward_source(struct sk_buff *skb,
}
}
+struct sk_buff *macvlan_xdp_build_skb(struct net_device *dev,
+ struct xdp_buff *xdp)
+{
+ int len;
+ int buflen = xdp->data_end - xdp->data_hard_start;
+ int headroom = xdp->data - xdp->data_hard_start;
+ struct sk_buff *skb;
+
+ len = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + headroom +
+ SKB_DATA_ALIGN(buflen);
+
+ skb = build_skb(xdp->data_hard_start, len);
+ if (!skb)
+ return NULL;
+
+ skb_reserve(skb, headroom);
+ __skb_put(skb, xdp->data_end - xdp->data);
+
+ skb->protocol = eth_type_trans(skb, dev);
+ skb->dev = dev;
+
+ return skb;
+}
+
+static rx_xdp_handler_result_t macvlan_receive_xdp(struct net_device *dev,
+ struct xdp_buff *xdp)
+{
+ struct macvlan_dev *vlan = netdev_priv(dev);
+ struct bpf_prog *xdp_prog;
+ struct sk_buff *skb;
+ u32 act = XDP_PASS;
+ rx_xdp_handler_result_t ret;
+ int err;
+
+ rcu_read_lock();
+ xdp_prog = rcu_dereference(vlan->xdp_prog);
+
+ if (xdp_prog)
+ act = bpf_prog_run_xdp(xdp_prog, xdp);
+
+ switch (act) {
+ case XDP_PASS:
+ ret = xdp_do_pass(xdp);
+ if (ret != RX_XDP_HANDLER_FALLBACK) {
+ rcu_read_unlock();
+ return ret;
+ }
+ skb = macvlan_xdp_build_skb(dev, xdp);
+ if (!skb) {
+ act = XDP_DROP;
+ break;
+ }
+ rcu_read_unlock();
+ netif_rx(skb);
+ macvlan_count_rx(vlan, skb->len, true, false);
+ goto out;
+ case XDP_TX:
+ skb = macvlan_xdp_build_skb(dev, xdp);
+ if (!skb) {
+ act = XDP_DROP;
+ break;
+ }
+ generic_xdp_tx(skb, xdp_prog);
+ break;
+ case XDP_REDIRECT:
+ err = xdp_do_redirect(dev, xdp, xdp_prog);
+ xdp_do_flush_map();
+ if (err)
+ act = XDP_DROP;
+ break;
+ case XDP_DROP:
+ break;
+ default:
+ bpf_warn_invalid_xdp_action(act);
+ break;
+ }
+
+ rcu_read_unlock();
+out:
+ if (act == XDP_DROP)
+ return RX_XDP_HANDLER_DROP;
+
+ return RX_XDP_HANDLER_CONSUMED;
+}
+
+/* called under rcu_read_lock() from XDP handler */
+static rx_xdp_handler_result_t macvlan_handle_xdp(struct net_device *dev,
+ struct xdp_buff *xdp)
+{
+ const struct ethhdr *eth = (const struct ethhdr *)xdp->data;
+ struct macvlan_port *port;
+ struct macvlan_dev *vlan;
+
+ if (is_multicast_ether_addr(eth->h_dest))
+ return RX_XDP_HANDLER_FALLBACK;
+
+ port = macvlan_port_get_rcu(dev);
+ if (port->source_count)
+ return RX_XDP_HANDLER_FALLBACK;
+
+ if (macvlan_passthru(port))
+ vlan = list_first_or_null_rcu(&port->vlans,
+ struct macvlan_dev, list);
+ else
+ vlan = macvlan_hash_lookup(port, eth->h_dest);
+
+ if (!vlan)
+ return RX_XDP_HANDLER_FALLBACK;
+
+ dev = vlan->dev;
+ if (unlikely(!(dev->flags & IFF_UP)))
+ return RX_XDP_HANDLER_DROP;
+
+ return macvlan_receive_xdp(dev, xdp);
+}
+
/* called under rcu_read_lock() from netif_receive_skb */
static rx_handler_result_t macvlan_handle_frame(struct sk_buff **pskb)
{
@@ -1089,6 +1206,44 @@ static int macvlan_dev_get_iflink(const struct net_device *dev)
return vlan->lowerdev->ifindex;
}
+static int macvlan_xdp_set(struct net_device *dev, struct bpf_prog *prog,
+ struct netlink_ext_ack *extack)
+{
+ struct macvlan_dev *vlan = netdev_priv(dev);
+ struct bpf_prog *old_prog = rtnl_dereference(vlan->xdp_prog);
+
+ rcu_assign_pointer(vlan->xdp_prog, prog);
+
+ if (old_prog)
+ bpf_prog_put(old_prog);
+
+ return 0;
+}
+
+static u32 macvlan_xdp_query(struct net_device *dev)
+{
+ struct macvlan_dev *vlan = netdev_priv(dev);
+ const struct bpf_prog *xdp_prog = rtnl_dereference(vlan->xdp_prog);
+
+ if (xdp_prog)
+ return xdp_prog->aux->id;
+
+ return 0;
+}
+
+static int macvlan_xdp(struct net_device *dev, struct netdev_bpf *xdp)
+{
+ switch (xdp->command) {
+ case XDP_SETUP_PROG:
+ return macvlan_xdp_set(dev, xdp->prog, xdp->extack);
+ case XDP_QUERY_PROG:
+ xdp->prog_id = macvlan_xdp_query(dev);
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
static const struct ethtool_ops macvlan_ethtool_ops = {
.get_link = ethtool_op_get_link,
.get_link_ksettings = macvlan_ethtool_get_link_ksettings,
@@ -1121,6 +1276,7 @@ static const struct net_device_ops macvlan_netdev_ops = {
#endif
.ndo_get_iflink = macvlan_dev_get_iflink,
.ndo_features_check = passthru_features_check,
+ .ndo_bpf = macvlan_xdp,
};
void macvlan_common_setup(struct net_device *dev)
@@ -1173,10 +1329,20 @@ static int macvlan_port_create(struct net_device *dev)
INIT_WORK(&port->bc_work, macvlan_process_broadcast);
err = netdev_rx_handler_register(dev, macvlan_handle_frame, port);
- if (err)
+ if (err) {
kfree(port);
- else
- dev->priv_flags |= IFF_MACVLAN_PORT;
+ goto out;
+ }
+
+ err = netdev_rx_xdp_handler_register(dev, macvlan_handle_xdp);
+ if (err) {
+ netdev_rx_handler_unregister(dev);
+ kfree(port);
+ goto out;
+ }
+
+ dev->priv_flags |= IFF_MACVLAN_PORT;
+out:
return err;
}
@@ -1187,6 +1353,7 @@ static void macvlan_port_destroy(struct net_device *dev)
dev->priv_flags &= ~IFF_MACVLAN_PORT;
netdev_rx_handler_unregister(dev);
+ netdev_rx_xdp_handler_unregister(dev);
/* After this point, no packet can schedule bc_work anymore,
* but we need to cancel it and purge left skbs if any.
diff --git a/include/linux/if_macvlan.h b/include/linux/if_macvlan.h
index 2e55e4c..7c7059b 100644
--- a/include/linux/if_macvlan.h
+++ b/include/linux/if_macvlan.h
@@ -34,6 +34,7 @@ struct macvlan_dev {
#ifdef CONFIG_NET_POLL_CONTROLLER
struct netpoll *netpoll;
#endif
+ struct bpf_prog __rcu *xdp_prog;
};
static inline void macvlan_count_rx(const struct macvlan_dev *vlan,
--
2.7.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox