* [iproute PATCH] ip-address: Fix filtering by negated address flags
From: Phil Sutter @ 2018-11-13 15:12 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev
When disabling a flag, one needs to AND with the inverse not the flag
itself. Otherwise specifying for instance 'home -nodad' will effectively
clear the flags variable.
While being at it, simplify the code a bit by merging common parts of
negated and non-negated case branches. Also allow for the "special
cases" to be inverted, too.
Fixes: f73ac674d0abf ("ip: change flag names to an array")
Signed-off-by: Phil Sutter <phil@nwl.cc>
---
ip/ipaddress.c | 32 ++++++++++++++++++--------------
1 file changed, 18 insertions(+), 14 deletions(-)
diff --git a/ip/ipaddress.c b/ip/ipaddress.c
index cd8cc76a3473f..7b98877085878 100644
--- a/ip/ipaddress.c
+++ b/ip/ipaddress.c
@@ -1211,31 +1211,35 @@ static void print_ifa_flags(FILE *fp, const struct ifaddrmsg *ifa,
static int get_filter(const char *arg)
{
+ bool inv = false;
unsigned int i;
+ if (arg[0] == '-') {
+ inv = true;
+ arg++;
+ }
/* Special cases */
if (strcmp(arg, "dynamic") == 0) {
- filter.flags &= ~IFA_F_PERMANENT;
+ if (inv)
+ filter.flags |= IFA_F_PERMANENT;
+ else
+ filter.flags &= ~IFA_F_PERMANENT;
filter.flagmask |= IFA_F_PERMANENT;
} else if (strcmp(arg, "primary") == 0) {
- filter.flags &= ~IFA_F_SECONDARY;
+ if (inv)
+ filter.flags |= IFA_F_SECONDARY;
+ else
+ filter.flags &= ~IFA_F_SECONDARY;
filter.flagmask |= IFA_F_SECONDARY;
- } else if (*arg == '-') {
- for (i = 0; i < ARRAY_SIZE(ifa_flag_names); i++) {
- if (strcmp(arg + 1, ifa_flag_names[i].name))
- continue;
-
- filter.flags &= ifa_flag_names[i].value;
- filter.flagmask |= ifa_flag_names[i].value;
- return 0;
- }
-
- return -1;
} else {
for (i = 0; i < ARRAY_SIZE(ifa_flag_names); i++) {
if (strcmp(arg, ifa_flag_names[i].name))
continue;
- filter.flags |= ifa_flag_names[i].value;
+
+ if (inv)
+ filter.flags &= ~ifa_flag_names[i].value;
+ else
+ filter.flags |= ifa_flag_names[i].value;
filter.flagmask |= ifa_flag_names[i].value;
return 0;
}
--
2.19.0
^ permalink raw reply related
* Re: [PATCH bpf-next v2] bpftool: make libbfd optional
From: Stanislav Fomichev @ 2018-11-13 15:45 UTC (permalink / raw)
To: Quentin Monnet
Cc: Jakub Kicinski, Stanislav Fomichev, netdev, ast, daniel,
Jiong Wang
In-Reply-To: <c34fe18a-5f23-7b7d-7062-54bf9f7ad125@netronome.com>
On 11/13, Quentin Monnet wrote:
> 2018-11-12 14:02 UTC-0800 ~ Jakub Kicinski <jakub.kicinski@netronome.com>
> > On Mon, 12 Nov 2018 13:44:10 -0800, Stanislav Fomichev wrote:
> >> Make it possible to build bpftool without libbfd. libbfd and libopcodes are
> >> typically provided in dev/dbg packages (binutils-dev in debian) which we
> >> usually don't have installed on the fleet machines and we'd like a way to have
> >> bpftool version that works without installing any additional packages.
> >> This excludes support for disassembling jit-ted code and prints an error if
> >> the user tries to use these features.
> >>
> >> Tested by:
> >> cat > FEATURES_DUMP.bpftool <<EOF
> >> feature-libbfd=0
> >> feature-disassembler-four-args=1
> >> feature-reallocarray=0
> >> feature-libelf=1
> >> feature-libelf-mmap=1
> >> feature-bpf=1
> >> EOF
> >> FEATURES_DUMP=$PWD/FEATURES_DUMP.bpftool make
> >> ldd bpftool | grep libbfd
> >>
> >> Signed-off-by: Stanislav Fomichev <sdf@google.com>
> >
> > Seems reasonable, thanks!
> >
> > Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
> >
>
> Thanks Stanislav!
>
> There is a problem with this patch on some distributions, Ubuntu at least.
>
> Feature detection for libbfd has been used for perf before being also
> used with bpftool. Since commit 280e7c48c3b8 the feature needs libz and
> libiberty to be present on the system, otherwise the feature would not
> compile (and be detected) on OpenSuse.
>
> On Ubuntu, libiberty is not needed (libbfd might be statically linked
> against it, if I remember correctly?), which means that we are able to
> build bpftool as long as binutils-dev has been installed, even if
> libiberty-dev has not been installed. The BFD feature, in that case,
> will appear as “undetected”. It is a bug. But since the Makefile does
> not stop compilation in that case (another bug), in the end we're good.
>
> With your patch, the problem is that libbpf detection will fail on
> Ubuntu if libiberty-dev is not present, even though all the necessary
> libraries for using the JIT disassembler are available. And in that case
> it _will_ make a difference, since the Makefile will no more compile the
> libbfd-related bits.
>
> So I'm not against the idea, but we have to fix libbfd detection first.
Yeah, libbfd feature detection looks broken on ubuntu, this patch just
exercises this brokenness :-). I can take a look somewhere this week,
thanks for spotting it!
I wonder, how does it work on opensuse currently if we link only against -lbfd
(https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/bpf/bpftool/Makefile#n56)?
It might be broken everywhere...
(Resent with proper formatting and without HTML).
> Thanks,
> Quentin
^ permalink raw reply
* Re: [PATCH net-next 16/17] net: sched: conditionally take rtnl lock on rules update path
From: Stefano Brivio @ 2018-11-13 15:53 UTC (permalink / raw)
To: Vlad Buslov
Cc: netdev@vger.kernel.org, jhs@mojatatu.com,
xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net,
ast@kernel.org, daniel@iogearbox.net
In-Reply-To: <vbfefbpjd3r.fsf@reg-r-vrt-018-180.mtr.labs.mlnx>
On Tue, 13 Nov 2018 13:58:05 +0000
Vlad Buslov <vladbu@mellanox.com> wrote:
> On Tue 13 Nov 2018 at 13:40, Stefano Brivio <sbrivio@redhat.com> wrote:
> > On Tue, 13 Nov 2018 13:25:52 +0000
> > Vlad Buslov <vladbu@mellanox.com> wrote:
> >
> >> On Tue 13 Nov 2018 at 09:40, Stefano Brivio <sbrivio@redhat.com> wrote:
> >> > Hi Vlad,
> >> >
> >> > On Mon, 12 Nov 2018 09:55:45 +0200
> >> > Vlad Buslov <vladbu@mellanox.com> wrote:
> >> >
> >> >> @@ -179,9 +179,25 @@ static void tcf_proto_destroy_work(struct work_struct *work)
> >> >> rtnl_unlock();
> >> >> }
> >> >>
> >> >> +/* Helper function to lock rtnl mutex when specified condition is true and mutex
> >> >> + * hasn't been locked yet. Will set rtnl_held to 'true' before taking rtnl lock.
> >> >> + * Note that this function does nothing if rtnl is already held. This is
> >> >> + * intended to be used by cls API rules update API when multiple conditions
> >> >> + * could require rtnl lock and its state needs to be tracked to prevent trying
> >> >> + * to obtain lock multiple times.
> >> >> + */
> >> >> +
> >> >> +static void tcf_require_rtnl(bool cond, bool *rtnl_held)
> >> >> +{
> >> >> + if (!*rtnl_held && cond) {
> >> >> + *rtnl_held = true;
> >> >> + rtnl_lock();
> >> >> + }
> >> >> +}
> >> >
> >> > I guess calls to this function are supposed to be serialised. If that's
> >> > the case (which is my tentative understanding so far), I would indicate
> >> > that in the comment.
> >> >
> >> > If that's not the case, you would be introducing a race I guess.
> >> >
> >> > Same applies to tcf_block_release() from 17/17.
> >>
> >> Hi Stefano,
> >>
> >> Thank you for reviewing my code!
> >>
> >> I did not intend for this function to be serialized. First argument to
> >> tcf_require_rtnl() is passed by value, and second argument is always a
> >> pointer to local stack-allocated value of the caller.
> >
> > Yes, sorry, I haven't been terribly clear, that's what I meant by
> > serialised: it won't be called concurrently with the same *rtnl_held.
> >
> > Perhaps the risk that somebody uses it that way is close to zero, so
> > I'm not even too sure this is worth a comment, but if you can come up
> > with a concise way of saying this, that would be nice.
>
> I considered my comment that function "Will set rtnl_held to 'true'
> before taking rtnl lock" as a red flag for caller to not pass pointer to
> a variable that can be accessed concurrently. I guess I can add
> additional sentence to explicitly warn potential users. Or I can just
> move rtnl_held assignment in both functions to be performed while
> holding rtnl mutex. I implemented it the way I did as an overzealous
> optimization, but realistically price of an assignment is negligible in
> this case.
But to make that effective, you would need to protect the read too, and
that makes your optimisation not really overzealous I think.
I'd rather go with an additional comment, if that doesn't become
unreadable.
--
Stefano
^ permalink raw reply
* [iproute PATCH v2] man: ip-route.8: Document nexthop limit
From: Phil Sutter @ 2018-11-13 15:55 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev, David Ahern
Add a note to 'nexthop' description stating the maximum number of
nexthops per command and pointing at 'append' command as a workaround.
Signed-off-by: Phil Sutter <phil@nwl.cc>
---
Changes since v1:
- Reviewed text.
- 'route append' trick works with IPv6 only.
---
man/man8/ip-route.8.in | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/man/man8/ip-route.8.in b/man/man8/ip-route.8.in
index a33ce1f0f4006..11dd4ca7abf68 100644
--- a/man/man8/ip-route.8.in
+++ b/man/man8/ip-route.8.in
@@ -589,6 +589,15 @@ argument lists:
route reflecting its relative bandwidth or quality.
.in -8
+The internal buffer used in iproute2 limits the maximum number of nexthops that
+may be specified in one go. If only
+.I ADDRESS
+is given, the current buffer size allows for 144 IPv6 nexthops and 253 IPv4
+ones. For IPv4, this effectively limits the number of nexthops possible per
+route. With IPv6, further nexthops may be appended to the same route via
+.B "ip route append"
+command.
+
.TP
.BI scope " SCOPE_VAL"
the scope of the destinations covered by the route prefix.
--
2.19.0
^ permalink raw reply related
* Re: [PATCH net-next 01/17] net: sched: refactor mini_qdisc_pair_swap() to use workqueue
From: David Miller @ 2018-11-13 16:08 UTC (permalink / raw)
To: vladbu; +Cc: netdev, jhs, xiyou.wangcong, jiri, ast, daniel
In-Reply-To: <vbfk1lhjf2s.fsf@reg-r-vrt-018-180.mtr.labs.mlnx>
From: Vlad Buslov <vladbu@mellanox.com>
Date: Tue, 13 Nov 2018 13:13:19 +0000
>
> On Mon 12 Nov 2018 at 17:28, David Miller <davem@davemloft.net> wrote:
>> From: Vlad Buslov <vladbu@mellanox.com>
>> Date: Mon, 12 Nov 2018 09:55:30 +0200
>>
>>> +void mini_qdisc_pair_swap(struct mini_Qdisc_pair *miniqp,
>>> + struct tcf_proto *tp_head)
>>> +{
>>> + xchg(&miniqp->tp_head, tp_head);
>>
>> If you are not checking the return value of xchg(), then this is
>> simply a store with optionally a memory barrier of some sort
>> either before or after.
>
> That was my intention. What would be a better way to atomically
> reset a pointer? Should I just change this line to explicit
> assignment+barrier?
We have all kinds of helpers in the kernel for doing things like
this, grep for things like "smp_load_acquire", "smp_store_release()"
etc.
^ permalink raw reply
* BUG: sleeping function called from invalid context at mm/slab.h:421
From: Naresh Kamboju @ 2018-11-13 16:33 UTC (permalink / raw)
To: netdev; +Cc: Roman Gushchin, ast, Daniel Borkmann, kafai
While running kernel selftests bpf test_cgroup_storage test this
kernel BUG reported every time on all devices running Linux -next
4.20.0-rc2-next-20181113 (from 4.19.0-rc5-next-20180928).
This kernel BUG log is from x86_64 machine.
Do you see at your end ?
[ 73.047526] BUG: sleeping function called from invalid context at
/srv/oe/build/tmp-rpb-glibc/work-shared/intel-corei7-64/kernel-source/mm/slab.h:421
[ 73.060915] in_atomic(): 1, irqs_disabled(): 0, pid: 3157, name:
test_cgroup_sto
[ 73.068342] INFO: lockdep is turned off.
[ 73.072293] CPU: 2 PID: 3157 Comm: test_cgroup_sto Not tainted
4.20.0-rc2-next-20181113 #1
[ 73.080548] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS
2.0b 07/27/2017
[ 73.088018] Call Trace:
[ 73.090463] dump_stack+0x70/0xa5
[ 73.093783] ___might_sleep+0x152/0x240
[ 73.097619] __might_sleep+0x4a/0x80
[ 73.101191] __kmalloc_node+0x1cf/0x2f0
[ 73.105031] ? cgroup_storage_update_elem+0x46/0x90
[ 73.109909] cgroup_storage_update_elem+0x46/0x90
[ 73.114608] map_update_elem+0x4a1/0x4c0
[ 73.118534] __x64_sys_bpf+0x124/0x280
[ 73.122286] do_syscall_64+0x4f/0x190
[ 73.125952] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[ 73.131004] RIP: 0033:0x7f46b93ea7f9
[ 73.134581] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00
00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24
08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4f e6 2b 00 f7 d8 64 89
01 48
[ 73.153318] RSP: 002b:00007fffc6595858 EFLAGS: 00000206 ORIG_RAX:
0000000000000141
[ 73.160876] RAX: ffffffffffffffda RBX: 00000000014a0260 RCX: 00007f46b93ea7f9
[ 73.167999] RDX: 0000000000000048 RSI: 00007fffc65958a0 RDI: 0000000000000002
[ 73.175124] RBP: 00007fffc6595870 R08: 00007fffc65958a0 R09: 00007fffc65958a0
[ 73.182246] R10: 00007fffc65958a0 R11: 0000000000000206 R12: 0000000000000003
[ 73.189369] R13: 0000000000000004 R14: 0000000000000005 R15: 0000000000000006
selftests: bpf: test_cgroup_storage
Please find full test log,
https://lkft.validation.linaro.org/scheduler/job/506640#L2874
Thank you.
Best regards
Naresh Kamboju
^ permalink raw reply
* [PATCH bpf-next v2] filter: add BPF_ADJ_ROOM_DATA mode to bpf_skb_adjust_room()
From: Nicolas Dichtel @ 2018-11-13 16:35 UTC (permalink / raw)
To: kafai; +Cc: ast, daniel, davem, netdev, Nicolas Dichtel
In-Reply-To: <20181112183939.xhiwgbqeaprvscu7@kafai-mbp.dhcp.thefacebook.com>
This new mode enables to add or remove an l2 header in a programmatic way
with cls_bpf.
For example, it enables to play with mpls headers.
Signed-off-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
---
v2: use skb_set_network_header()
include/uapi/linux/bpf.h | 3 ++
net/core/filter.c | 53 ++++++++++++++++++++++++++++++++++
tools/include/uapi/linux/bpf.h | 3 ++
3 files changed, 59 insertions(+)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 47d606d744cc..9762ef3a536c 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1467,6 +1467,8 @@ union bpf_attr {
*
* * **BPF_ADJ_ROOM_NET**: Adjust room at the network layer
* (room space is added or removed below the layer 3 header).
+ * * **BPF_ADJ_ROOM_DATA**: Adjust room at the beginning of the
+ * packet (room space is added or removed below skb->data).
*
* All values for *flags* are reserved for future usage, and must
* be left at zero.
@@ -2412,6 +2414,7 @@ enum bpf_func_id {
/* Mode for BPF_FUNC_skb_adjust_room helper. */
enum bpf_adj_room_mode {
BPF_ADJ_ROOM_NET,
+ BPF_ADJ_ROOM_DATA,
};
/* Mode for BPF_FUNC_skb_load_bytes_relative helper. */
diff --git a/net/core/filter.c b/net/core/filter.c
index 53d50fb75ea1..e56da3077dca 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -2884,6 +2884,57 @@ static int bpf_skb_adjust_net(struct sk_buff *skb, s32 len_diff)
return ret;
}
+static int bpf_skb_data_shrink(struct sk_buff *skb, u32 len)
+{
+ unsigned short hhlen = skb->dev->header_ops ?
+ skb->dev->hard_header_len : 0;
+ int ret;
+
+ ret = skb_unclone(skb, GFP_ATOMIC);
+ if (unlikely(ret < 0))
+ return ret;
+
+ __skb_pull(skb, len);
+ skb_reset_mac_header(skb);
+ skb_set_network_header(skb, hhlen);
+ skb_reset_transport_header(skb);
+ return 0;
+}
+
+static int bpf_skb_data_grow(struct sk_buff *skb, u32 len)
+{
+ unsigned short hhlen = skb->dev->header_ops ?
+ skb->dev->hard_header_len : 0;
+ int ret;
+
+ ret = skb_cow(skb, len);
+ if (unlikely(ret < 0))
+ return ret;
+
+ skb_push(skb, len);
+ skb_reset_mac_header(skb);
+ return 0;
+}
+
+static int bpf_skb_adjust_data(struct sk_buff *skb, s32 len_diff)
+{
+ u32 len_diff_abs = abs(len_diff);
+ bool shrink = len_diff < 0;
+ int ret;
+
+ if (unlikely(len_diff_abs > 0xfffU))
+ return -EFAULT;
+
+ if (shrink && len_diff_abs >= skb_headlen(skb))
+ return -EFAULT;
+
+ ret = shrink ? bpf_skb_data_shrink(skb, len_diff_abs) :
+ bpf_skb_data_grow(skb, len_diff_abs);
+
+ bpf_compute_data_pointers(skb);
+ return ret;
+}
+
BPF_CALL_4(bpf_skb_adjust_room, struct sk_buff *, skb, s32, len_diff,
u32, mode, u64, flags)
{
@@ -2891,6 +2942,8 @@ BPF_CALL_4(bpf_skb_adjust_room, struct sk_buff *, skb, s32, len_diff,
return -EINVAL;
if (likely(mode == BPF_ADJ_ROOM_NET))
return bpf_skb_adjust_net(skb, len_diff);
+ if (likely(mode == BPF_ADJ_ROOM_DATA))
+ return bpf_skb_adjust_data(skb, len_diff);
return -ENOTSUPP;
}
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 852dc17ab47a..47407fd5162b 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1467,6 +1467,8 @@ union bpf_attr {
*
* * **BPF_ADJ_ROOM_NET**: Adjust room at the network layer
* (room space is added or removed below the layer 3 header).
+ * * **BPF_ADJ_ROOM_DATA**: Adjust room at the beginning of the
+ * packet (room space is added or removed below skb->data).
*
* All values for *flags* are reserved for future usage, and must
* be left at zero.
@@ -2408,6 +2410,7 @@ enum bpf_func_id {
/* Mode for BPF_FUNC_skb_adjust_room helper. */
enum bpf_adj_room_mode {
BPF_ADJ_ROOM_NET,
+ BPF_ADJ_ROOM_DATA,
};
/* Mode for BPF_FUNC_skb_load_bytes_relative helper. */
--
2.18.0
^ permalink raw reply related
* [PATCH net] ipv6: fix a dst leak when removing its exception
From: Xin Long @ 2018-11-13 16:48 UTC (permalink / raw)
To: network dev; +Cc: davem, David Ahern
These is no need to hold dst before calling rt6_remove_exception_rt().
The call to dst_hold_safe() in ip6_link_failure() was for ip6_del_rt(),
which has been removed in Commit 93531c674315 ("net/ipv6: separate
handling of FIB entries from dst based routes"). Otherwise, it will
cause a dst leak.
This patch is to simply remove the dst_hold_safe() call before calling
rt6_remove_exception_rt() and also do the same in ip6_del_cached_rt().
It's safe, because the removal of the exception that holds its dst's
refcnt is protected by rt6_exception_lock.
Fixes: 93531c674315 ("net/ipv6: separate handling of FIB entries from dst based routes")
Fixes: 23fb93a4d3f1 ("net/ipv6: Cleanup exception and cache route handling")
Reported-by: Li Shuang <shuali@redhat.com>
Signed-off-by: Xin Long <lucien.xin@gmail.com>
---
net/ipv6/route.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 2a7423c..14b422f 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -2232,8 +2232,7 @@ static void ip6_link_failure(struct sk_buff *skb)
if (rt) {
rcu_read_lock();
if (rt->rt6i_flags & RTF_CACHE) {
- if (dst_hold_safe(&rt->dst))
- rt6_remove_exception_rt(rt);
+ rt6_remove_exception_rt(rt);
} else {
struct fib6_info *from;
struct fib6_node *fn;
@@ -3214,8 +3213,8 @@ static int ip6_del_cached_rt(struct rt6_info *rt, struct fib6_config *cfg)
if (cfg->fc_flags & RTF_GATEWAY &&
!ipv6_addr_equal(&cfg->fc_gateway, &rt->rt6i_gateway))
goto out;
- if (dst_hold_safe(&rt->dst))
- rc = rt6_remove_exception_rt(rt);
+
+ rc = rt6_remove_exception_rt(rt);
out:
return rc;
}
--
2.1.0
^ permalink raw reply related
* Re: pull-request: can 2018-11-09
From: David Miller @ 2018-11-13 16:43 UTC (permalink / raw)
To: mkl; +Cc: netdev, linux-can, kernel
In-Reply-To: <20181112115728.18331-1-mkl@pengutronix.de>
From: Marc Kleine-Budde <mkl@pengutronix.de>
Date: Mon, 12 Nov 2018 12:57:08 +0100
> this is a pull request of 20 patches for net/master.
Pulled, thanks Marc.
^ permalink raw reply
* Re: [PATCH net v2 0/4] qed: Miscellaneous bug fixes
From: David Miller @ 2018-11-13 16:51 UTC (permalink / raw)
To: denis.bolotin; +Cc: netdev, michal.kalderon, ariel.elior
In-Reply-To: <20181112105023.23842-1-denis.bolotin@cavium.com>
From: Denis Bolotin <denis.bolotin@cavium.com>
Date: Mon, 12 Nov 2018 12:50:19 +0200
> This patch series fixes several unrelated bugs across the driver.
> Please consider applying to net.
>
> V1->V2:
> -------
> Use dma_rmb() instead of rmb().
Series applied, thank you.
^ permalink raw reply
* Re: [PATCH net-next 16/17] net: sched: conditionally take rtnl lock on rules update path
From: Stefano Brivio @ 2018-11-13 16:57 UTC (permalink / raw)
To: Vlad Buslov
Cc: netdev@vger.kernel.org, jhs@mojatatu.com,
xiyou.wangcong@gmail.com, jiri@resnulli.us, davem@davemloft.net,
ast@kernel.org, daniel@iogearbox.net
In-Reply-To: <20181113165307.45987362@redhat.com>
On Tue, 13 Nov 2018 16:53:07 +0100
Stefano Brivio <sbrivio@redhat.com> wrote:
> But to make that effective, you would need to protect the read too, and
> that makes your optimisation not really overzealous I think.
>
> I'd rather go with an additional comment, if that doesn't become
> unreadable.
Oh, and of course, this whole thing makes sense only if for some reason
you end up keeping this function. Maybe ignore my comments on this patch
at least for the moment being ;)
--
Stefano
^ permalink raw reply
* Re: [RFCv3 PATCH 1/6] uacce: Add documents for WarpDrive/uacce
From: Kenneth Lee @ 2018-11-14 2:58 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Alexander Shishkin, Tim Sell, Sanyog Kale, Randy Dunlap,
Uwe Kleine-König, Vinod Koul, David Kershner, Sagar Dharia,
Gavin Schenk, Jens Axboe, Philippe Ombredanne, Cyrille Pitchen,
Johan Hovold, Zhou Wang, Hao Fang, Jonathan Cameron, Zaibo Xu,
linux-doc, linux-kernel
In-Reply-To: <20181113002354.GO3695@mtr-leonro.mtl.com>
在 2018/11/13 上午8:23, Leon Romanovsky 写道:
> On Mon, Nov 12, 2018 at 03:58:02PM +0800, Kenneth Lee wrote:
>> From: Kenneth Lee <liguozhu@hisilicon.com>
>>
>> WarpDrive is a general accelerator framework for the user application to
>> access the hardware without going through the kernel in data path.
>>
>> The kernel component to provide kernel facility to driver for expose the
>> user interface is called uacce. It a short name for
>> "Unified/User-space-access-intended Accelerator Framework".
>>
>> This patch add document to explain how it works.
> + RDMA and netdev folks
>
> Sorry, to be late in the game, I don't see other patches, but from
> the description below it seems like you are reinventing RDMA verbs
> model. I have hard time to see the differences in the proposed
> framework to already implemented in drivers/infiniband/* for the kernel
> space and for the https://github.com/linux-rdma/rdma-core/ for the user
> space parts.
Thanks Leon,
Yes, we tried to solve similar problem in RDMA. We also learned a lot
from the exist code of RDMA. But we we have to make a new one because we
cannot register accelerators such as AI operation, encryption or
compression to the RDMA framework:)
Another problem we tried to address is the way to pin the memory for dma
operation. The RDMA way to pin the memory cannot avoid the page lost due
to copy-on-write operation during the memory is used by the device. This
may not be important to RDMA library. But it is important to accelerator.
Hope this can help the understanding.
Cheers
>
> Hard NAK from RDMA side.
>
> Thanks
>
>> Signed-off-by: Kenneth Lee <liguozhu@hisilicon.com>
>> ---
>> Documentation/warpdrive/warpdrive.rst | 260 +++++++
>> Documentation/warpdrive/wd-arch.svg | 764 ++++++++++++++++++++
>> Documentation/warpdrive/wd.svg | 526 ++++++++++++++
>> Documentation/warpdrive/wd_q_addr_space.svg | 359 +++++++++
>> 4 files changed, 1909 insertions(+)
>> create mode 100644 Documentation/warpdrive/warpdrive.rst
>> create mode 100644 Documentation/warpdrive/wd-arch.svg
>> create mode 100644 Documentation/warpdrive/wd.svg
>> create mode 100644 Documentation/warpdrive/wd_q_addr_space.svg
>>
>> diff --git a/Documentation/warpdrive/warpdrive.rst b/Documentation/warpdrive/warpdrive.rst
>> new file mode 100644
>> index 000000000000..ef84d3a2d462
>> --- /dev/null
>> +++ b/Documentation/warpdrive/warpdrive.rst
>> @@ -0,0 +1,260 @@
>> +Introduction of WarpDrive
>> +=========================
>> +
>> +*WarpDrive* is a general accelerator framework for the user application to
>> +access the hardware without going through the kernel in data path.
>> +
>> +It can be used as the quick channel for accelerators, network adaptors or
>> +other hardware for application in user space.
>> +
>> +This may make some implementation simpler. E.g. you can reuse most of the
>> +*netdev* driver in kernel and just share some ring buffer to the user space
>> +driver for *DPDK* [4] or *ODP* [5]. Or you can combine the RSA accelerator with
>> +the *netdev* in the user space as a https reversed proxy, etc.
>> +
>> +*WarpDrive* takes the hardware accelerator as a heterogeneous processor which
>> +can share particular load from the CPU:
>> +
>> +.. image:: wd.svg
>> + :alt: WarpDrive Concept
>> +
>> +The virtual concept, queue, is used to manage the requests sent to the
>> +accelerator. The application send requests to the queue by writing to some
>> +particular address, while the hardware takes the requests directly from the
>> +address and send feedback accordingly.
>> +
>> +The format of the queue may differ from hardware to hardware. But the
>> +application need not to make any system call for the communication.
>> +
>> +*WarpDrive* tries to create a shared virtual address space for all involved
>> +accelerators. Within this space, the requests sent to queue can refer to any
>> +virtual address, which will be valid to the application and all involved
>> +accelerators.
>> +
>> +The name *WarpDrive* is simply a cool and general name meaning the framework
>> +makes the application faster. It includes general user library, kernel
>> +management module and drivers for the hardware. In kernel, the management
>> +module is called *uacce*, meaning "Unified/User-space-access-intended
>> +Accelerator Framework".
>> +
>> +
>> +How does it work
>> +================
>> +
>> +*WarpDrive* uses *mmap* and *IOMMU* to play the trick.
>> +
>> +*Uacce* creates a chrdev for the device registered to it. A "queue" will be
>> +created when the chrdev is opened. The application access the queue by mmap
>> +different address region of the queue file.
>> +
>> +The following figure demonstrated the queue file address space:
>> +
>> +.. image:: wd_q_addr_space.svg
>> + :alt: WarpDrive Queue Address Space
>> +
>> +The first region of the space, device region, is used for the application to
>> +write request or read answer to or from the hardware.
>> +
>> +Normally, there can be three types of device regions mmio and memory regions.
>> +It is recommended to use common memory for request/answer descriptors and use
>> +the mmio space for device notification, such as doorbell. But of course, this
>> +is all up to the interface designer.
>> +
>> +There can be two types of device memory regions, kernel-only and user-shared.
>> +This will be explained in the "kernel APIs" section.
>> +
>> +The Static Share Virtual Memory region is necessary only when the device IOMMU
>> +does not support "Share Virtual Memory". This will be explained after the
>> +*IOMMU* idea.
>> +
>> +
>> +Architecture
>> +------------
>> +
>> +The full *WarpDrive* architecture is represented in the following class
>> +diagram:
>> +
>> +.. image:: wd-arch.svg
>> + :alt: WarpDrive Architecture
>> +
>> +
>> +The user API
>> +------------
>> +
>> +We adopt a polling style interface in the user space: ::
>> +
>> + int wd_request_queue(struct wd_queue *q);
>> + void wd_release_queue(struct wd_queue *q);
>> +
>> + int wd_send(struct wd_queue *q, void *req);
>> + int wd_recv(struct wd_queue *q, void **req);
>> + int wd_recv_sync(struct wd_queue *q, void **req);
>> + void wd_flush(struct wd_queue *q);
>> +
>> +wd_recv_sync() is a wrapper to its non-sync version. It will trapped into
>> +kernel and waits until the queue become available.
>> +
>> +If the queue do not support SVA/SVM. The following helper function
>> +can be used to create Static Virtual Share Memory: ::
>> +
>> + void *wd_preserve_share_memory(struct wd_queue *q, size_t size);
>> +
>> +The user API is not mandatory. It is simply a suggestion and hint what the
>> +kernel interface is supposed to support.
>> +
>> +
>> +The user driver
>> +---------------
>> +
>> +The queue file mmap space will need a user driver to wrap the communication
>> +protocol. *UACCE* provides some attributes in sysfs for the user driver to
>> +match the right accelerator accordingly.
>> +
>> +The *UACCE* device attribute is under the following directory:
>> +
>> +/sys/class/uacce/<dev-name>/params
>> +
>> +The following attributes is supported:
>> +
>> +nr_queue_remained (ro)
>> + number of queue remained
>> +
>> +api_version (ro)
>> + a string to identify the queue mmap space format and its version
>> +
>> +device_attr (ro)
>> + attributes of the device, see UACCE_DEV_xxx flag defined in uacce.h
>> +
>> +numa_node (ro)
>> + id of numa node
>> +
>> +priority (rw)
>> + Priority or the device, bigger is higher
>> +
>> +(This is not yet implemented in RFC version)
>> +
>> +
>> +The kernel API
>> +--------------
>> +
>> +The *uacce* kernel API is defined in uacce.h. If the hardware support SVM/SVA,
>> +The driver need only the following API functions: ::
>> +
>> + int uacce_register(uacce);
>> + void uacce_unregister(uacce);
>> + void uacce_wake_up(q);
>> +
>> +*uacce_wake_up* is used to notify the process who epoll() on the queue file.
>> +
>> +According to the IOMMU capability, *uacce* categories the devices as follow:
>> +
>> +UACCE_DEV_NOIOMMU
>> + The device has no IOMMU. The user process cannot use VA on the hardware
>> + This mode is not recommended.
>> +
>> +UACCE_DEV_SVA (UACCE_DEV_PASID | UACCE_DEV_FAULT_FROM_DEV)
>> + The device has IOMMU which can share the same page table with user
>> + process
>> +
>> +UACCE_DEV_SHARE_DOMAIN
>> + The device has IOMMU which has no multiple page table and device page
>> + fault support
>> +
>> +If the device works in mode other than UACCE_DEV_NOIOMMU, *uacce* will set its
>> +IOMMU to IOMMU_DOMAIN_UNMANAGED. So the driver must not use any kernel
>> +DMA API but the following ones from *uacce* instead: ::
>> +
>> + uacce_dma_map(q, va, size, prot);
>> + uacce_dma_unmap(q, va, size, prot);
>> +
>> +*uacce_dma_map/unmap* is valid only for UACCE_DEV_SVA device. It creates a
>> +particular PASID and page table for the kernel in the IOMMU (Not yet
>> +implemented in the RFC)
>> +
>> +For the UACCE_DEV_SHARE_DOMAIN device, uacce_dma_map/unmap is not valid.
>> +*Uacce* call back start_queue only when the DUS and DKO region is mmapped. The
>> +accelerator driver must use those dma buffer, via uacce_queue->qfrs[], on
>> +start_queue call back. The size of the queue file region is defined by
>> +uacce->ops->qf_pg_start[].
>> +
>> +We have to do it this way because most of current IOMMU cannot support the
>> +kernel and user virtual address at the same time. So we have to let them both
>> +share the same user virtual address space.
>> +
>> +If the device have to support kernel and user at the same time, both kernel
>> +and the user should use these DMA API. This is not convenient. A better
>> +solution is to change the future DMA/IOMMU design to let them separate the
>> +address space between the user and kernel space. But it is not going to be in
>> +a short time.
>> +
>> +
>> +Multiple processes support
>> +==========================
>> +
>> +In the latest mainline kernel (4.19) when this document is written, the IOMMU
>> +subsystem do not support multiple process page tables yet.
>> +
>> +Most IOMMU hardware implementation support multi-process with the concept
>> +of PASID. But they may use different name, e.g. it is call sub-stream-id in
>> +SMMU of ARM. With PASID or similar design, multi page table can be added to
>> +the IOMMU and referred by its PASID.
>> +
>> +*JPB* has a patchset to enable this[1]_. We have tested it with our hardware
>> +(which is known as *D06*). It works well. *WarpDrive* rely on them to support
>> +UACCE_DEV_SVA. If it is not enabled, *WarpDrive* can still work. But it
>> +support only one process, the device will be set to UACCE_DEV_SHARE_DOMAIN
>> +even it is set to UACCE_DEV_SVA initially.
>> +
>> +Static Share Virtual Memory is mainly used by UACCE_DEV_SHARE_DOMAIN device.
>> +
>> +
>> +Legacy Mode Support
>> +===================
>> +For the hardware without IOMMU, WarpDrive can still work, the only problem is
>> +VA cannot be used in the device. The driver should adopt another strategy for
>> +the shared memory. It is only for testing, and not recommended.
>> +
>> +
>> +The Folk Scenario
>> +=================
>> +For a process with allocated queues and shared memory, what happen if it forks
>> +a child?
>> +
>> +The fd of the queue will be duplicated on folk, so the child can send request
>> +to the same queue as its parent. But the requests which is sent from processes
>> +except for the one who open the queue will be blocked.
>> +
>> +It is recommended to add O_CLOEXEC to the queue file.
>> +
>> +The queue mmap space has a VM_DONTCOPY in its VMA. So the child will lost all
>> +those VMAs.
>> +
>> +This is why *WarpDrive* does not adopt the mode used in *VFIO* and *InfiniBand*.
>> +Both solutions can set any user pointer for hardware sharing. But they cannot
>> +support fork when the dma is in process. Or the "Copy-On-Write" procedure will
>> +make the parent process lost its physical pages.
>> +
>> +
>> +The Sample Code
>> +===============
>> +There is a sample user land implementation with a simple driver for Hisilicon
>> +Hi1620 ZIP Accelerator.
>> +
>> +To test, do the following in samples/warpdrive (for the case of PC host): ::
>> + ./autogen.sh
>> + ./conf.sh # or simply ./configure if you build on target system
>> + make
>> +
>> +Then you can get test_hisi_zip in the test subdirectory. Copy it to the target
>> +system and make sure the hisi_zip driver is enabled (the major and minor of
>> +the uacce chrdev can be gotten from the dmesg or sysfs), and run: ::
>> + mknod /dev/ua1 c <major> <minior>
>> + test/test_hisi_zip -z < data > data.zip
>> + test/test_hisi_zip -g < data > data.gzip
>> +
>> +
>> +References
>> +==========
>> +.. [1] https://patchwork.kernel.org/patch/10394851/
>> +
>> +.. vim: tw=78
>> diff --git a/Documentation/warpdrive/wd-arch.svg b/Documentation/warpdrive/wd-arch.svg
>> new file mode 100644
>> index 000000000000..e59934188443
>> --- /dev/null
>> +++ b/Documentation/warpdrive/wd-arch.svg
>> @@ -0,0 +1,764 @@
>> +<?xml version="1.0" encoding="UTF-8" standalone="no"?>
>> +<!-- Created with Inkscape (http://www.inkscape.org/) -->
>> +
>> +<svg
>> + xmlns:dc="http://purl.org/dc/elements/1.1/"
>> + xmlns:cc="http://creativecommons.org/ns#"
>> + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
>> + xmlns:svg="http://www.w3.org/2000/svg"
>> + xmlns="http://www.w3.org/2000/svg"
>> + xmlns:xlink="http://www.w3.org/1999/xlink"
>> + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
>> + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
>> + width="210mm"
>> + height="193mm"
>> + viewBox="0 0 744.09449 683.85823"
>> + id="svg2"
>> + version="1.1"
>> + inkscape:version="0.92.3 (2405546, 2018-03-11)"
>> + sodipodi:docname="wd-arch.svg">
>> + <defs
>> + id="defs4">
>> + <linearGradient
>> + inkscape:collect="always"
>> + id="linearGradient6830">
>> + <stop
>> + style="stop-color:#000000;stop-opacity:1;"
>> + offset="0"
>> + id="stop6832" />
>> + <stop
>> + style="stop-color:#000000;stop-opacity:0;"
>> + offset="1"
>> + id="stop6834" />
>> + </linearGradient>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="translate(-89.949614,405.94594)" />
>> + <linearGradient
>> + inkscape:collect="always"
>> + id="linearGradient5026">
>> + <stop
>> + style="stop-color:#f2f2f2;stop-opacity:1;"
>> + offset="0"
>> + id="stop5028" />
>> + <stop
>> + style="stop-color:#f2f2f2;stop-opacity:0;"
>> + offset="1"
>> + id="stop5030" />
>> + </linearGradient>
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6" />
>> + </filter>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-1"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="translate(175.77842,400.29111)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-0"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-9" />
>> + </filter>
>> + <marker
>> + markerWidth="18.960653"
>> + markerHeight="11.194658"
>> + refX="9.4803267"
>> + refY="5.5973287"
>> + orient="auto"
>> + id="marker4613">
>> + <rect
>> + y="-5.1589785"
>> + x="5.8504119"
>> + height="10.317957"
>> + width="10.317957"
>> + id="rect4212"
>> + style="fill:#ffffff;stroke:#000000;stroke-width:0.69143367;stroke-miterlimit:4;stroke-dasharray:none"
>> + transform="matrix(0.86111274,0.50841405,-0.86111274,0.50841405,0,0)">
>> + <title
>> + id="title4262">generation</title>
>> + </rect>
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-9"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.2452511,0,0,0.98513016,-190.95632,540.33156)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-8"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-9" />
>> + </filter>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-1">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-9-7"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.3742742,0,0,0.97786398,-234.52617,654.63367)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-8-5"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-9-0" />
>> + </filter>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-6">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-1"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-9-4"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.3742912,0,0,2.0035845,-468.34428,342.56603)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-8-54"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-9-7" />
>> + </filter>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-1-8">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-9-6"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-1-8-8">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-9-6-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-0">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-93"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-0-2">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-93-6"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter5382"
>> + x="-0.089695387"
>> + width="1.1793908"
>> + y="-0.10052069"
>> + height="1.2010413">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="0.86758925"
>> + id="feGaussianBlur5384" />
>> + </filter>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient6830"
>> + id="linearGradient6836"
>> + x1="362.73923"
>> + y1="700.04059"
>> + x2="340.4751"
>> + y2="678.25488"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="translate(-23.771026,-135.76835)" />
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-6-2">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-1-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-9-7-3"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.3742742,0,0,0.97786395,-57.357186,649.55786)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-8-5-0"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-9-0-2" />
>> + </filter>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-1-1">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-9-0"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + </defs>
>> + <sodipodi:namedview
>> + id="base"
>> + pagecolor="#ffffff"
>> + bordercolor="#666666"
>> + borderopacity="1.0"
>> + inkscape:pageopacity="0.0"
>> + inkscape:pageshadow="2"
>> + inkscape:zoom="0.98994949"
>> + inkscape:cx="222.32868"
>> + inkscape:cy="370.44492"
>> + inkscape:document-units="px"
>> + inkscape:current-layer="layer1"
>> + showgrid="false"
>> + inkscape:window-width="1916"
>> + inkscape:window-height="1033"
>> + inkscape:window-x="0"
>> + inkscape:window-y="22"
>> + inkscape:window-maximized="0"
>> + fit-margin-right="0.3"
>> + inkscape:snap-global="false" />
>> + <metadata
>> + id="metadata7">
>> + <rdf:RDF>
>> + <cc:Work
>> + rdf:about="">
>> + <dc:format>image/svg+xml</dc:format>
>> + <dc:type
>> + rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
>> + <dc:title />
>> + </cc:Work>
>> + </rdf:RDF>
>> + </metadata>
>> + <g
>> + inkscape:label="Layer 1"
>> + inkscape:groupmode="layer"
>> + id="layer1"
>> + transform="translate(0,-368.50374)">
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3)"
>> + id="rect4136-3-6"
>> + width="101.07784"
>> + height="31.998148"
>> + x="283.01144"
>> + y="588.80896" />
>> + <rect
>> + style="fill:url(#linearGradient5032);fill-opacity:1;stroke:#000000;stroke-width:0.6465112"
>> + id="rect4136-2"
>> + width="101.07784"
>> + height="31.998148"
>> + x="281.63498"
>> + y="586.75739" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="294.21747"
>> + y="612.50073"
>> + id="text4138-6"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1"
>> + x="294.21747"
>> + y="612.50073"
>> + style="font-size:15px;line-height:1.25">WarpDrive</tspan></text>
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-0)"
>> + id="rect4136-3-6-3"
>> + width="101.07784"
>> + height="31.998148"
>> + x="548.7395"
>> + y="583.15417" />
>> + <rect
>> + style="fill:url(#linearGradient5032-1);fill-opacity:1;stroke:#000000;stroke-width:0.6465112"
>> + id="rect4136-2-60"
>> + width="101.07784"
>> + height="31.998148"
>> + x="547.36304"
>> + y="581.1026" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="557.83484"
>> + y="602.32745"
>> + id="text4138-6-6"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-2"
>> + x="557.83484"
>> + y="602.32745"
>> + style="font-size:15px;line-height:1.25">user_driver</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4613)"
>> + d="m 547.36304,600.78954 -156.58203,0.0691"
>> + id="path4855"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-8)"
>> + id="rect4136-3-6-5-7"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(1.2452511,0,0,0.98513016,113.15182,641.02594)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-9);fill-opacity:1;stroke:#000000;stroke-width:0.71606314"
>> + id="rect4136-2-6-3"
>> + width="125.86729"
>> + height="31.522341"
>> + x="271.75983"
>> + y="718.45435" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="309.13705"
>> + y="745.55371"
>> + id="text4138-6-2-6"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-1"
>> + x="309.13705"
>> + y="745.55371"
>> + style="font-size:15px;line-height:1.25">uacce</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-2)"
>> + d="m 329.57309,619.72453 5.0373,97.14447"
>> + id="path4661-3"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-2-1)"
>> + d="m 342.57219,830.63108 -5.67699,-79.2841"
>> + id="path4661-3-4"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-8-5)"
>> + id="rect4136-3-6-5-7-3"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(1.3742742,0,0,0.97786398,101.09126,754.58534)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-9-7);fill-opacity:1;stroke:#000000;stroke-width:0.74946606"
>> + id="rect4136-2-6-3-6"
>> + width="138.90866"
>> + height="31.289837"
>> + x="276.13297"
>> + y="831.44263" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="295.67819"
>> + y="852.98224"
>> + id="text4138-6-2-6-1"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-1-0"
>> + x="295.67819"
>> + y="852.98224"
>> + style="font-size:15px;line-height:1.25">Device Driver</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-2-6)"
>> + d="m 623.05084,615.00104 0.51369,333.80219"
>> + id="path4661-3-5"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="392.63568"
>> + y="660.83667"
>> + id="text4138-6-2-6-1-6-2-5"><tspan
>> + sodipodi:role="line"
>> + x="392.63568"
>> + y="660.83667"
>> + id="tspan4305"
>> + style="font-size:15px;line-height:1.25"><<anom_file>></tspan><tspan
>> + sodipodi:role="line"
>> + x="392.63568"
>> + y="679.58667"
>> + style="font-size:15px;line-height:1.25"
>> + id="tspan1139">Queue FD</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="389.92969"
>> + y="587.44836"
>> + id="text4138-6-2-6-1-6-2-56"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-1-0-3-0-9"
>> + x="389.92969"
>> + y="587.44836"
>> + style="font-size:15px;line-height:1.25">1</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="528.64813"
>> + y="600.08429"
>> + id="text4138-6-2-6-1-6-3"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-1-0-3-7"
>> + x="528.64813"
>> + y="600.08429"
>> + style="font-size:15px;line-height:1.25">*</tspan></text>
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-8-54)"
>> + id="rect4136-3-6-5-7-4"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(1.3745874,0,0,1.8929066,-132.7754,556.04505)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-9-4);fill-opacity:1;stroke:#000000;stroke-width:1.07280123"
>> + id="rect4136-2-6-3-4"
>> + width="138.91039"
>> + height="64.111"
>> + x="42.321312"
>> + y="704.8371" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="110.30745"
>> + y="722.94025"
>> + id="text4138-6-2-6-3"><tspan
>> + sodipodi:role="line"
>> + x="111.99202"
>> + y="722.94025"
>> + id="tspan4366"
>> + style="font-size:15px;line-height:1.25;text-align:center;text-anchor:middle">other standard </tspan><tspan
>> + sodipodi:role="line"
>> + x="110.30745"
>> + y="741.69025"
>> + id="tspan4368"
>> + style="font-size:15px;line-height:1.25;text-align:center;text-anchor:middle">framework</tspan><tspan
>> + sodipodi:role="line"
>> + x="110.30745"
>> + y="760.44025"
>> + style="font-size:15px;line-height:1.25;text-align:center;text-anchor:middle"
>> + id="tspan6840">(crypto/nic/others)</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-2-1-8)"
>> + d="M 276.29661,849.04109 134.04449,771.90853"
>> + id="path4661-3-4-8"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="313.70813"
>> + y="730.06366"
>> + id="text4138-6-2-6-36"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-1-7"
>> + x="313.70813"
>> + y="730.06366"
>> + style="font-size:10px;line-height:1.25"><<lkm>></tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:start;letter-spacing:0px;word-spacing:0px;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="259.53165"
>> + y="797.8056"
>> + id="text4138-6-2-6-1-6-2-5-7-5"><tspan
>> + sodipodi:role="line"
>> + x="259.53165"
>> + y="797.8056"
>> + style="font-size:15px;line-height:1.25;text-align:start;text-anchor:start"
>> + id="tspan2357">uacce register api</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="29.145819"
>> + y="833.44244"
>> + id="text4138-6-2-6-1-6-2-5-7-5-2"><tspan
>> + sodipodi:role="line"
>> + x="29.145819"
>> + y="833.44244"
>> + id="tspan4301"
>> + style="font-size:15px;line-height:1.25">register to other subsystem</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="301.20813"
>> + y="597.29437"
>> + id="text4138-6-2-6-36-1"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-1-7-2"
>> + x="301.20813"
>> + y="597.29437"
>> + style="font-size:10px;line-height:1.25"><<user_lib>></tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="615.9505"
>> + y="739.44012"
>> + id="text4138-6-2-6-1-6-2-5-3"><tspan
>> + sodipodi:role="line"
>> + x="615.9505"
>> + y="739.44012"
>> + id="tspan4274-7"
>> + style="font-size:15px;line-height:1.25">mmapped memory r/w interface</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="371.01291"
>> + y="529.23682"
>> + id="text4138-6-2-6-1-6-2-5-36"><tspan
>> + sodipodi:role="line"
>> + x="371.01291"
>> + y="529.23682"
>> + id="tspan4305-3"
>> + style="font-size:15px;line-height:1.25">wd user api</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + d="m 328.19325,585.87943 0,-23.57142"
>> + id="path4348"
>> + inkscape:connector-curvature="0" />
>> + <ellipse
>> + style="opacity:1;fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0"
>> + id="path4350"
>> + cx="328.01468"
>> + cy="551.95081"
>> + rx="11.607142"
>> + ry="10.357142" />
>> + <path
>> + style="opacity:0.444;fill:url(#linearGradient6836);fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:1;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;filter:url(#filter5382)"
>> + id="path4350-2"
>> + sodipodi:type="arc"
>> + sodipodi:cx="329.44327"
>> + sodipodi:cy="553.37933"
>> + sodipodi:rx="11.607142"
>> + sodipodi:ry="10.357142"
>> + sodipodi:start="0"
>> + sodipodi:end="6.2509098"
>> + d="m 341.05041,553.37933 a 11.607142,10.357142 0 0 1 -11.51349,10.35681 11.607142,10.357142 0 0 1 -11.69928,-10.18967 11.607142,10.357142 0 0 1 11.32469,-10.52124 11.607142,10.357142 0 0 1 11.88204,10.01988"
>> + sodipodi:open="true" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="619.67596"
>> + y="978.22363"
>> + id="text4138-6-2-6-1-6-2-5-36-3"><tspan
>> + sodipodi:role="line"
>> + x="619.67596"
>> + y="978.22363"
>> + id="tspan4305-3-67"
>> + style="font-size:15px;line-height:1.25">Device(Hardware)</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-2-6-2)"
>> + d="m 347.51164,865.4527 193.91929,99.10053"
>> + id="path4661-3-5-1"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-8-5-0)"
>> + id="rect4136-3-6-5-7-3-1"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(1.3742742,0,0,0.97786395,278.26025,749.50952)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-9-7-3);fill-opacity:1;stroke:#000000;stroke-width:0.74946606"
>> + id="rect4136-2-6-3-6-0"
>> + width="138.90868"
>> + height="31.289839"
>> + x="453.30197"
>> + y="826.36682" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:12px;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="493.68158"
>> + y="847.90643"
>> + id="text4138-6-2-6-1-5"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-1-0-1"
>> + x="493.68158"
>> + y="847.90643"
>> + style="font-size:15px;line-height:1.25;stroke-width:1px">IOMMU</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-2-1-1)"
>> + d="m 389.49372,755.46667 111.75324,68.4507"
>> + id="path4661-3-4-85"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:12px;line-height:0%;font-family:sans-serif;text-align:start;letter-spacing:0px;word-spacing:0px;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="427.70282"
>> + y="776.91418"
>> + id="text4138-6-2-6-1-6-2-5-7-5-0"><tspan
>> + sodipodi:role="line"
>> + x="427.70282"
>> + y="776.91418"
>> + style="font-size:15px;line-height:1.25;text-align:start;text-anchor:start;stroke-width:1px"
>> + id="tspan2357-6">manage the driver iommu state</tspan></text>
>> + </g>
>> +</svg>
>> diff --git a/Documentation/warpdrive/wd.svg b/Documentation/warpdrive/wd.svg
>> new file mode 100644
>> index 000000000000..87ab92ebfbc6
>> --- /dev/null
>> +++ b/Documentation/warpdrive/wd.svg
>> @@ -0,0 +1,526 @@
>> +<?xml version="1.0" encoding="UTF-8" standalone="no"?>
>> +<!-- Created with Inkscape (http://www.inkscape.org/) -->
>> +
>> +<svg
>> + xmlns:dc="http://purl.org/dc/elements/1.1/"
>> + xmlns:cc="http://creativecommons.org/ns#"
>> + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
>> + xmlns:svg="http://www.w3.org/2000/svg"
>> + xmlns="http://www.w3.org/2000/svg"
>> + xmlns:xlink="http://www.w3.org/1999/xlink"
>> + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
>> + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
>> + width="210mm"
>> + height="116mm"
>> + viewBox="0 0 744.09449 411.02338"
>> + id="svg2"
>> + version="1.1"
>> + inkscape:version="0.92.3 (2405546, 2018-03-11)"
>> + sodipodi:docname="wd.svg">
>> + <defs
>> + id="defs4">
>> + <linearGradient
>> + inkscape:collect="always"
>> + id="linearGradient5026">
>> + <stop
>> + style="stop-color:#f2f2f2;stop-opacity:1;"
>> + offset="0"
>> + id="stop5028" />
>> + <stop
>> + style="stop-color:#f2f2f2;stop-opacity:0;"
>> + offset="1"
>> + id="stop5030" />
>> + </linearGradient>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(2.7384117,0,0,0.91666329,-952.8283,571.10143)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3" />
>> + </filter>
>> + <marker
>> + markerWidth="18.960653"
>> + markerHeight="11.194658"
>> + refX="9.4803267"
>> + refY="5.5973287"
>> + orient="auto"
>> + id="marker4613">
>> + <rect
>> + y="-5.1589785"
>> + x="5.8504119"
>> + height="10.317957"
>> + width="10.317957"
>> + id="rect4212"
>> + style="fill:#ffffff;stroke:#000000;stroke-width:0.69143367;stroke-miterlimit:4;stroke-dasharray:none"
>> + transform="matrix(0.86111274,0.50841405,-0.86111274,0.50841405,0,0)">
>> + <title
>> + id="title4262">generation</title>
>> + </rect>
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-9"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.2452511,0,0,0.98513016,-190.95632,540.33156)" />
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-1">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-6">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-1"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-1-8">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-9-6"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-1-8-8">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-9-6-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-0">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-93"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-0-2">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-93-6"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-2-6-2">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-9-1-9"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-8"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.0104674,0,0,1.0052679,-218.642,661.15448)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-8"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-9" />
>> + </filter>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-8-2"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(2.1450559,0,0,1.0052679,-521.97704,740.76422)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-8-5"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-9-1" />
>> + </filter>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-8-0"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.0104674,0,0,1.0052679,83.456748,660.20747)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-8-6"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-9-2" />
>> + </filter>
>> + <linearGradient
>> + inkscape:collect="always"
>> + xlink:href="#linearGradient5026"
>> + id="linearGradient5032-3-84"
>> + x1="353"
>> + y1="211.3622"
>> + x2="565.5"
>> + y2="174.8622"
>> + gradientUnits="userSpaceOnUse"
>> + gradientTransform="matrix(1.9884948,0,0,0.94903536,-318.42665,564.37696)" />
>> + <filter
>> + inkscape:collect="always"
>> + style="color-interpolation-filters:sRGB"
>> + id="filter4169-3-5-4"
>> + x="-0.031597666"
>> + width="1.0631953"
>> + y="-0.099812768"
>> + height="1.1996255">
>> + <feGaussianBlur
>> + inkscape:collect="always"
>> + stdDeviation="1.3307599"
>> + id="feGaussianBlur4171-6-3-0" />
>> + </filter>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-0-0">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-93-8"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + <marker
>> + markerWidth="11.227358"
>> + markerHeight="12.355258"
>> + refX="10"
>> + refY="6.177629"
>> + orient="auto"
>> + id="marker4825-6-3">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path4757-1-1"
>> + d="M 0.42024733,0.42806444 10.231357,6.3500844 0.24347733,11.918544"
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1" />
>> + </marker>
>> + </defs>
>> + <sodipodi:namedview
>> + id="base"
>> + pagecolor="#ffffff"
>> + bordercolor="#666666"
>> + borderopacity="1.0"
>> + inkscape:pageopacity="0.0"
>> + inkscape:pageshadow="2"
>> + inkscape:zoom="0.98994949"
>> + inkscape:cx="457.47339"
>> + inkscape:cy="250.14781"
>> + inkscape:document-units="px"
>> + inkscape:current-layer="layer1"
>> + showgrid="false"
>> + inkscape:window-width="1916"
>> + inkscape:window-height="1033"
>> + inkscape:window-x="0"
>> + inkscape:window-y="22"
>> + inkscape:window-maximized="0"
>> + fit-margin-right="0.3" />
>> + <metadata
>> + id="metadata7">
>> + <rdf:RDF>
>> + <cc:Work
>> + rdf:about="">
>> + <dc:format>image/svg+xml</dc:format>
>> + <dc:type
>> + rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
>> + <dc:title></dc:title>
>> + </cc:Work>
>> + </rdf:RDF>
>> + </metadata>
>> + <g
>> + inkscape:label="Layer 1"
>> + inkscape:groupmode="layer"
>> + id="layer1"
>> + transform="translate(0,-641.33861)">
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5)"
>> + id="rect4136-3-6-5"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(2.7384116,0,0,0.91666328,-284.06895,664.79751)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3);fill-opacity:1;stroke:#000000;stroke-width:1.02430749"
>> + id="rect4136-2-6"
>> + width="276.79272"
>> + height="29.331528"
>> + x="64.723419"
>> + y="736.84473" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="78.223282"
>> + y="756.79803"
>> + id="text4138-6-2"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9"
>> + x="78.223282"
>> + y="756.79803"
>> + style="font-size:15px;line-height:1.25">user application (running by the CPU</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6)"
>> + d="m 217.67507,876.6738 113.40331,45.0758"
>> + id="path4661"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-0)"
>> + d="m 208.10197,767.69811 0.29362,76.03656"
>> + id="path4661-6"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-8)"
>> + id="rect4136-3-6-5-3"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(1.0104673,0,0,1.0052679,28.128628,763.90722)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-8);fill-opacity:1;stroke:#000000;stroke-width:0.65159565"
>> + id="rect4136-2-6-6"
>> + width="102.13586"
>> + height="32.16671"
>> + x="156.83217"
>> + y="842.91852" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:12px;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="188.58519"
>> + y="864.47125"
>> + id="text4138-6-2-8"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-0"
>> + x="188.58519"
>> + y="864.47125"
>> + style="font-size:15px;line-height:1.25;stroke-width:1px">MMU</tspan></text>
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-8-5)"
>> + id="rect4136-3-6-5-3-1"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(2.1450556,0,0,1.0052679,1.87637,843.51696)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-8-2);fill-opacity:1;stroke:#000000;stroke-width:0.94937181"
>> + id="rect4136-2-6-6-0"
>> + width="216.8176"
>> + height="32.16671"
>> + x="275.09283"
>> + y="922.5282" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:12px;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="347.81482"
>> + y="943.23291"
>> + id="text4138-6-2-8-8"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-0-5"
>> + x="347.81482"
>> + y="943.23291"
>> + style="font-size:15px;line-height:1.25;stroke-width:1px">Memory</tspan></text>
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-8-6)"
>> + id="rect4136-3-6-5-3-5"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(1.0104673,0,0,1.0052679,330.22737,762.9602)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-8-0);fill-opacity:1;stroke:#000000;stroke-width:0.65159565"
>> + id="rect4136-2-6-6-8"
>> + width="102.13586"
>> + height="32.16671"
>> + x="458.93091"
>> + y="841.9715" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:12px;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="490.68393"
>> + y="863.52423"
>> + id="text4138-6-2-8-6"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-0-2"
>> + x="490.68393"
>> + y="863.52423"
>> + style="font-size:15px;line-height:1.25;stroke-width:1px">IOMMU</tspan></text>
>> + <rect
>> + style="fill:#000000;stroke:#000000;stroke-width:0.6465112;filter:url(#filter4169-3-5-4)"
>> + id="rect4136-3-6-5-6"
>> + width="101.07784"
>> + height="31.998148"
>> + x="128.74678"
>> + y="80.648842"
>> + transform="matrix(1.9884947,0,0,0.94903537,167.19229,661.38193)" />
>> + <rect
>> + style="fill:url(#linearGradient5032-3-84);fill-opacity:1;stroke:#000000;stroke-width:0.88813609"
>> + id="rect4136-2-6-2"
>> + width="200.99274"
>> + height="30.367374"
>> + x="420.4675"
>> + y="735.97351" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:12px;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="441.95297"
>> + y="755.9068"
>> + id="text4138-6-2-9"><tspan
>> + sodipodi:role="line"
>> + id="tspan4140-1-9-9"
>> + x="441.95297"
>> + y="755.9068"
>> + style="font-size:15px;line-height:1.25;stroke-width:1px">Hardware Accelerator</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-0-0)"
>> + d="m 508.2914,766.55885 0.29362,76.03656"
>> + id="path4661-6-1"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker4825-6-3)"
>> + d="M 499.70201,876.47297 361.38296,920.80258"
>> + id="path4661-1"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + </g>
>> +</svg>
>> diff --git a/Documentation/warpdrive/wd_q_addr_space.svg b/Documentation/warpdrive/wd_q_addr_space.svg
>> new file mode 100644
>> index 000000000000..5e6cf8e89908
>> --- /dev/null
>> +++ b/Documentation/warpdrive/wd_q_addr_space.svg
>> @@ -0,0 +1,359 @@
>> +<?xml version="1.0" encoding="UTF-8" standalone="no"?>
>> +<!-- Created with Inkscape (http://www.inkscape.org/) -->
>> +
>> +<svg
>> + xmlns:dc="http://purl.org/dc/elements/1.1/"
>> + xmlns:cc="http://creativecommons.org/ns#"
>> + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
>> + xmlns:svg="http://www.w3.org/2000/svg"
>> + xmlns="http://www.w3.org/2000/svg"
>> + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
>> + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
>> + width="210mm"
>> + height="124mm"
>> + viewBox="0 0 210 124"
>> + version="1.1"
>> + id="svg8"
>> + inkscape:version="0.92.3 (2405546, 2018-03-11)"
>> + sodipodi:docname="wd_q_addr_space.svg">
>> + <defs
>> + id="defs2">
>> + <marker
>> + inkscape:stockid="Arrow1Mend"
>> + orient="auto"
>> + refY="0"
>> + refX="0"
>> + id="marker5428"
>> + style="overflow:visible"
>> + inkscape:isstock="true">
>> + <path
>> + id="path5426"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + transform="matrix(-0.4,0,0,-0.4,-4,0)"
>> + inkscape:connector-curvature="0" />
>> + </marker>
>> + <marker
>> + inkscape:isstock="true"
>> + style="overflow:visible"
>> + id="marker2922"
>> + refX="0"
>> + refY="0"
>> + orient="auto"
>> + inkscape:stockid="Arrow1Mend"
>> + inkscape:collect="always">
>> + <path
>> + transform="matrix(-0.4,0,0,-0.4,-4,0)"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + id="path2920"
>> + inkscape:connector-curvature="0" />
>> + </marker>
>> + <marker
>> + inkscape:stockid="Arrow1Mstart"
>> + orient="auto"
>> + refY="0"
>> + refX="0"
>> + id="Arrow1Mstart"
>> + style="overflow:visible"
>> + inkscape:isstock="true">
>> + <path
>> + id="path840"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + transform="matrix(0.4,0,0,0.4,4,0)"
>> + inkscape:connector-curvature="0" />
>> + </marker>
>> + <marker
>> + inkscape:stockid="Arrow1Mend"
>> + orient="auto"
>> + refY="0"
>> + refX="0"
>> + id="Arrow1Mend"
>> + style="overflow:visible"
>> + inkscape:isstock="true"
>> + inkscape:collect="always">
>> + <path
>> + id="path843"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + transform="matrix(-0.4,0,0,-0.4,-4,0)"
>> + inkscape:connector-curvature="0" />
>> + </marker>
>> + <marker
>> + inkscape:stockid="Arrow1Mstart"
>> + orient="auto"
>> + refY="0"
>> + refX="0"
>> + id="Arrow1Mstart-5"
>> + style="overflow:visible"
>> + inkscape:isstock="true">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path840-1"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + transform="matrix(0.4,0,0,0.4,4,0)" />
>> + </marker>
>> + <marker
>> + inkscape:stockid="Arrow1Mend"
>> + orient="auto"
>> + refY="0"
>> + refX="0"
>> + id="Arrow1Mend-1"
>> + style="overflow:visible"
>> + inkscape:isstock="true">
>> + <path
>> + inkscape:connector-curvature="0"
>> + id="path843-0"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + transform="matrix(-0.4,0,0,-0.4,-4,0)" />
>> + </marker>
>> + <marker
>> + inkscape:isstock="true"
>> + style="overflow:visible"
>> + id="marker2922-2"
>> + refX="0"
>> + refY="0"
>> + orient="auto"
>> + inkscape:stockid="Arrow1Mend"
>> + inkscape:collect="always">
>> + <path
>> + transform="matrix(-0.4,0,0,-0.4,-4,0)"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + id="path2920-9"
>> + inkscape:connector-curvature="0" />
>> + </marker>
>> + <marker
>> + inkscape:isstock="true"
>> + style="overflow:visible"
>> + id="marker2922-27"
>> + refX="0"
>> + refY="0"
>> + orient="auto"
>> + inkscape:stockid="Arrow1Mend"
>> + inkscape:collect="always">
>> + <path
>> + transform="matrix(-0.4,0,0,-0.4,-4,0)"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + id="path2920-0"
>> + inkscape:connector-curvature="0" />
>> + </marker>
>> + <marker
>> + inkscape:isstock="true"
>> + style="overflow:visible"
>> + id="marker2922-27-8"
>> + refX="0"
>> + refY="0"
>> + orient="auto"
>> + inkscape:stockid="Arrow1Mend"
>> + inkscape:collect="always">
>> + <path
>> + transform="matrix(-0.4,0,0,-0.4,-4,0)"
>> + style="fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:1.00000003pt;stroke-opacity:1"
>> + d="M 0,0 5,-5 -12.5,0 5,5 Z"
>> + id="path2920-0-0"
>> + inkscape:connector-curvature="0" />
>> + </marker>
>> + </defs>
>> + <sodipodi:namedview
>> + id="base"
>> + pagecolor="#ffffff"
>> + bordercolor="#666666"
>> + borderopacity="1.0"
>> + inkscape:pageopacity="0.0"
>> + inkscape:pageshadow="2"
>> + inkscape:zoom="1.4"
>> + inkscape:cx="401.66654"
>> + inkscape:cy="218.12255"
>> + inkscape:document-units="mm"
>> + inkscape:current-layer="layer1"
>> + showgrid="false"
>> + inkscape:window-width="1916"
>> + inkscape:window-height="1033"
>> + inkscape:window-x="0"
>> + inkscape:window-y="22"
>> + inkscape:window-maximized="0" />
>> + <metadata
>> + id="metadata5">
>> + <rdf:RDF>
>> + <cc:Work
>> + rdf:about="">
>> + <dc:format>image/svg+xml</dc:format>
>> + <dc:type
>> + rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
>> + <dc:title />
>> + </cc:Work>
>> + </rdf:RDF>
>> + </metadata>
>> + <g
>> + inkscape:label="Layer 1"
>> + inkscape:groupmode="layer"
>> + id="layer1"
>> + transform="translate(0,-173)">
>> + <rect
>> + style="opacity:0.82999998;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.4;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:0.82745098"
>> + id="rect815"
>> + width="21.262758"
>> + height="40.350552"
>> + x="55.509361"
>> + y="195.00098"
>> + ry="0" />
>> + <rect
>> + style="opacity:0.82999998;fill:none;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.4;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:0.82745098"
>> + id="rect815-1"
>> + width="21.24276"
>> + height="43.732346"
>> + x="55.519352"
>> + y="235.26543"
>> + ry="0" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:2.9765625px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="50.549229"
>> + y="190.6078"
>> + id="text1118"><tspan
>> + sodipodi:role="line"
>> + id="tspan1116"
>> + x="50.549229"
>> + y="190.6078"
>> + style="stroke-width:0.26458332px">queue file address space</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + d="M 76.818568,194.95453 H 97.229281"
>> + id="path1126"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + d="M 76.818568,235.20899 H 96.095361"
>> + id="path1126-8"
>> + inkscape:connector-curvature="0" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + d="m 76.762111,278.99778 h 19.27678"
>> + id="path1126-0"
>> + inkscape:connector-curvature="0" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + d="m 55.519355,265.20165 v 19.27678"
>> + id="path1126-2"
>> + inkscape:connector-curvature="0" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + d="m 76.762111,265.20165 v 19.27678"
>> + id="path1126-2-1"
>> + inkscape:connector-curvature="0" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart);marker-end:url(#Arrow1Mend)"
>> + d="m 87.590896,194.76554 0,39.87648"
>> + id="path1126-2-1-0"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Mstart-5);marker-end:url(#Arrow1Mend-1)"
>> + d="m 82.48822,235.77596 v 42.90029"
>> + id="path1126-2-1-0-8"
>> + inkscape:connector-curvature="0"
>> + sodipodi:nodetypes="cc" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker2922)"
>> + d="M 44.123633,195.3325 H 55.651907"
>> + id="path2912"
>> + inkscape:connector-curvature="0" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:2.9765625px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="32.217381"
>> + y="196.27745"
>> + id="text2968"><tspan
>> + sodipodi:role="line"
>> + id="tspan2966"
>> + x="32.217381"
>> + y="196.27745"
>> + style="stroke-width:0.26458332px">offset 0</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:2.9765625px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="91.199554"
>> + y="216.03946"
>> + id="text1118-5"><tspan
>> + sodipodi:role="line"
>> + id="tspan1116-0"
>> + x="91.199554"
>> + y="216.03946"
>> + style="stroke-width:0.26458332px">device region (mapped to device mmio or shared kernel driver memory)</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:2.9765625px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="86.188072"
>> + y="244.50081"
>> + id="text1118-5-6"><tspan
>> + sodipodi:role="line"
>> + id="tspan1116-0-4"
>> + x="86.188072"
>> + y="244.50081"
>> + style="stroke-width:0.26458332px">static share virtual memory region (for device without share virtual memory)</tspan></text>
>> + <flowRoot
>> + xml:space="preserve"
>> + id="flowRoot5699"
>> + style="font-style:normal;font-weight:normal;font-size:11.25px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion
>> + id="flowRegion5701"><rect
>> + id="rect5703"
>> + width="5182.8569"
>> + height="385.71429"
>> + x="34.285713"
>> + y="71.09111" /></flowRegion><flowPara
>> + id="flowPara5705" /></flowRoot> <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker2922-2)"
>> + d="M 43.679028,206.85268 H 55.207302"
>> + id="path2912-1"
>> + inkscape:connector-curvature="0" />
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker2922-27)"
>> + d="M 44.057004,224.23959 H 55.585278"
>> + id="path2912-9"
>> + inkscape:connector-curvature="0" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:2.9765625px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="24.139778"
>> + y="202.40636"
>> + id="text1118-5-3"><tspan
>> + sodipodi:role="line"
>> + id="tspan1116-0-6"
>> + x="24.139778"
>> + y="202.40636"
>> + style="stroke-width:0.26458332px">device mmio region</tspan></text>
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:2.9765625px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="17.010948"
>> + y="216.73672"
>> + id="text1118-5-3-3"><tspan
>> + sodipodi:role="line"
>> + id="tspan1116-0-6-6"
>> + x="17.010948"
>> + y="216.73672"
>> + style="stroke-width:0.26458332px">device kernel only region</tspan></text>
>> + <path
>> + style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#marker2922-27-8)"
>> + d="M 43.981087,235.35153 H 55.509361"
>> + id="path2912-9-2"
>> + inkscape:connector-curvature="0" />
>> + <text
>> + xml:space="preserve"
>> + style="font-style:normal;font-weight:normal;font-size:2.9765625px;line-height:125%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
>> + x="17.575975"
>> + y="230.53285"
>> + id="text1118-5-3-3-0"><tspan
>> + sodipodi:role="line"
>> + id="tspan1116-0-6-6-5"
>> + x="17.575975"
>> + y="230.53285"
>> + style="stroke-width:0.26458332px">device user share region</tspan></text>
>> + </g>
>> +</svg>
>> --
>> 2.17.1
>>
^ permalink raw reply
* Re: [PATCHv2 net-next 1/4] sctp: define subscribe in sctp_sock as __u16
From: Neil Horman @ 2018-11-13 17:15 UTC (permalink / raw)
To: Xin Long; +Cc: network dev, linux-sctp, Marcelo Ricardo Leitner, davem
In-Reply-To: <82150b4fab9e70377db3db9c62fd10323be3f294.1542089666.git.lucien.xin@gmail.com>
On Tue, Nov 13, 2018 at 02:24:53PM +0800, Xin Long wrote:
>
> /* Default Peer Address Parameters. These defaults can
> * be modified via SCTP_PEER_ADDR_PARAMS
> @@ -5267,14 +5274,24 @@ static int sctp_getsockopt_disable_fragments(struct sock *sk, int len,
> static int sctp_getsockopt_events(struct sock *sk, int len, char __user *optval,
> int __user *optlen)
> {
> + struct sctp_event_subscribe subscribe;
> + __u8 *sn_type = (__u8 *)&subscribe;
> + int i;
> +
> if (len == 0)
> return -EINVAL;
> if (len > sizeof(struct sctp_event_subscribe))
> len = sizeof(struct sctp_event_subscribe);
> if (put_user(len, optlen))
> return -EFAULT;
> - if (copy_to_user(optval, &sctp_sk(sk)->subscribe, len))
> +
> + for (i = 0; i <= len; i++)
> + sn_type[i] = sctp_ulpevent_type_enabled(sctp_sk(sk)->subscribe,
> + SCTP_SN_TYPE_BASE + i);
> +
This seems like an off by one error. sctp_event_subscribe has N bytes in it (1
byte for each event), meaning that that events 0-(N-1) are subscribable.
Iterating this loop imples that you are going to check N events, overrunning the
sctp_event_subscribe struct.
Neil
>
^ permalink raw reply
* [PATCH net] ipvs: call ip_vs_dst_notifier before ipv6_dev_notf
From: Xin Long @ 2018-11-13 17:25 UTC (permalink / raw)
To: network dev, netfilter-devel; +Cc: davem, pablo, Hans Schillstrom
ip_vs_dst_event is supposed to clean up all dst used in ipvs'
destinations when a net dev is going down. But it works only
when the dst's dev is the same as the dev from the event.
Now with the same priority but late registration,
ip_vs_dst_notifier is always called after ipv6_dev_notf where
the dst's dev is set to lo for NETDEV_DOWN event.
As the dst's dev lo is not the same as the dev from the event
in ip_vs_dst_event(), ipv6_dev_notf can actually never work.
Also as these dst have to wait for dest_trash_timer to clean
them up. It would cause some non-permanent kernel warnings:
unregister_netdevice: waiting for br0 to become free. Usage count = 3
To fix it, call ip_vs_dst_notifier before ipv6_dev_notf by
increasing its priority to ADDRCONF_NOTIFY_PRIORITY + 5.
Fixes: 7a4f0761fce3 ("IPVS: init and cleanup restructuring")
Reported-by: Li Shuang <shuali@redhat.com>
Signed-off-by: Xin Long <lucien.xin@gmail.com>
---
net/netfilter/ipvs/ip_vs_ctl.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/net/netfilter/ipvs/ip_vs_ctl.c b/net/netfilter/ipvs/ip_vs_ctl.c
index 83395bf6..aded477 100644
--- a/net/netfilter/ipvs/ip_vs_ctl.c
+++ b/net/netfilter/ipvs/ip_vs_ctl.c
@@ -3980,6 +3980,7 @@ static void __net_exit ip_vs_control_net_cleanup_sysctl(struct netns_ipvs *ipvs)
static struct notifier_block ip_vs_dst_notifier = {
.notifier_call = ip_vs_dst_event,
+ .priority = ADDRCONF_NOTIFY_PRIORITY + 5,
};
int __net_init ip_vs_control_net_init(struct netns_ipvs *ipvs)
--
2.1.0
^ permalink raw reply related
* Re: BUG: sleeping function called from invalid context at mm/slab.h:421
From: Roman Gushchin @ 2018-11-13 17:36 UTC (permalink / raw)
To: Naresh Kamboju
Cc: netdev@vger.kernel.org, ast@kernel.org, Daniel Borkmann,
Martin Lau
In-Reply-To: <CA+G9fYsp4UrrY9t5VZDSQn20SFxePTghsHPa3hOyZ7exvfrmHw@mail.gmail.com>
On Tue, Nov 13, 2018 at 10:03:38PM +0530, Naresh Kamboju wrote:
> While running kernel selftests bpf test_cgroup_storage test this
> kernel BUG reported every time on all devices running Linux -next
> 4.20.0-rc2-next-20181113 (from 4.19.0-rc5-next-20180928).
> This kernel BUG log is from x86_64 machine.
>
> Do you see at your end ?
>
> [ 73.047526] BUG: sleeping function called from invalid context at
> /srv/oe/build/tmp-rpb-glibc/work-shared/intel-corei7-64/kernel-source/mm/slab.h:421
> [ 73.060915] in_atomic(): 1, irqs_disabled(): 0, pid: 3157, name:
> test_cgroup_sto
> [ 73.068342] INFO: lockdep is turned off.
> [ 73.072293] CPU: 2 PID: 3157 Comm: test_cgroup_sto Not tainted
> 4.20.0-rc2-next-20181113 #1
> [ 73.080548] Hardware name: Supermicro SYS-5019S-ML/X11SSH-F, BIOS
> 2.0b 07/27/2017
> [ 73.088018] Call Trace:
> [ 73.090463] dump_stack+0x70/0xa5
> [ 73.093783] ___might_sleep+0x152/0x240
> [ 73.097619] __might_sleep+0x4a/0x80
> [ 73.101191] __kmalloc_node+0x1cf/0x2f0
> [ 73.105031] ? cgroup_storage_update_elem+0x46/0x90
> [ 73.109909] cgroup_storage_update_elem+0x46/0x90
> [ 73.114608] map_update_elem+0x4a1/0x4c0
> [ 73.118534] __x64_sys_bpf+0x124/0x280
> [ 73.122286] do_syscall_64+0x4f/0x190
> [ 73.125952] entry_SYSCALL_64_after_hwframe+0x49/0xbe
> [ 73.131004] RIP: 0033:0x7f46b93ea7f9
> [ 73.134581] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00
> 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24
> 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4f e6 2b 00 f7 d8 64 89
> 01 48
> [ 73.153318] RSP: 002b:00007fffc6595858 EFLAGS: 00000206 ORIG_RAX:
> 0000000000000141
> [ 73.160876] RAX: ffffffffffffffda RBX: 00000000014a0260 RCX: 00007f46b93ea7f9
> [ 73.167999] RDX: 0000000000000048 RSI: 00007fffc65958a0 RDI: 0000000000000002
> [ 73.175124] RBP: 00007fffc6595870 R08: 00007fffc65958a0 R09: 00007fffc65958a0
> [ 73.182246] R10: 00007fffc65958a0 R11: 0000000000000206 R12: 0000000000000003
> [ 73.189369] R13: 0000000000000004 R14: 0000000000000005 R15: 0000000000000006
> selftests: bpf: test_cgroup_storage
Hi Naresh!
Thank you for the report! Can you, please, try the following patch?
Thanks!
^ permalink raw reply
* Re: [Patch net-next v2] net: dump more useful information in netdev_rx_csum_fault()
From: Willem de Bruijn @ 2018-11-13 18:00 UTC (permalink / raw)
To: Cong Wang; +Cc: Network Development
In-Reply-To: <20181112224718.19496-1-xiyou.wangcong@gmail.com>
On Mon, Nov 12, 2018 at 2:49 PM Cong Wang <xiyou.wangcong@gmail.com> wrote:
>
> Currently netdev_rx_csum_fault() only shows a device name,
> we need more information about the skb for debugging csum
> failures.
>
> Sample output:
>
> ens3: hw csum failure
> dev features: 0x0000000000014b89
> skb len=84 data_len=0 pkt_type=0 gso_size=0 gso_type=0 nr_frags=0 ip_summed=0 csum=0 csum_complete_sw=0 csum_valid=0 csum_level=0
Recent issues were protocol dependent, including whether vlan headers
were present. Perhaps also print skb vlan fields and even the first N
bytes of data to inspect protocol headers? Also skb_iif, esp. if this
differs from dev->ifindex.
^ permalink raw reply
* Re: [Patch net-next v2] net: dump more useful information in netdev_rx_csum_fault()
From: Cong Wang @ 2018-11-13 18:14 UTC (permalink / raw)
To: Willem de Bruijn; +Cc: Linux Kernel Network Developers
In-Reply-To: <CAF=yD-KWt2NV5hrDrG09yCHb9jQs8oxm5iQAyEhw7GqFZ4a9eg@mail.gmail.com>
On Tue, Nov 13, 2018 at 10:01 AM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> On Mon, Nov 12, 2018 at 2:49 PM Cong Wang <xiyou.wangcong@gmail.com> wrote:
> >
> > Currently netdev_rx_csum_fault() only shows a device name,
> > we need more information about the skb for debugging csum
> > failures.
> >
> > Sample output:
> >
> > ens3: hw csum failure
> > dev features: 0x0000000000014b89
> > skb len=84 data_len=0 pkt_type=0 gso_size=0 gso_type=0 nr_frags=0 ip_summed=0 csum=0 csum_complete_sw=0 csum_valid=0 csum_level=0
>
> Recent issues were protocol dependent, including whether vlan headers
> were present. Perhaps also print skb vlan fields and even the first N
> bytes of data to inspect protocol headers? Also skb_iif, esp. if this
> differs from dev->ifindex.
Pawel's case seems to be vlan related, however, as I mentioned,
my case is neither vlan nor RXFCS related.
Ideally, we should dump the whole packet in order to verify the
correctness of the checksum. :) It is not easy to do so given
how complex an skb is now. This is why I only select a few skb
fields to dump. I am pretty sure this can't cover all cases, you
can always add more for your need in the future.
Thanks.
^ permalink raw reply
* Re: VETH & AF_PACKET problem
From: Willem de Bruijn @ 2018-11-13 18:20 UTC (permalink / raw)
To: anandhkrishnan; +Cc: Network Development
In-Reply-To: <CACeb84v7xj1PNGJj=Wbq_Nvq1QK32u6kD2=4oreUT0+VU2jbbQ@mail.gmail.com>
On Sun, Nov 11, 2018 at 11:15 PM Anand H. Krishnan
<anandhkrishnan@gmail.com> wrote:
>
> Hello,
>
> We are seeing a problem with AF_PACKET when used along with the
> veth interfaces. SCP complains that message authentication code is
> incorrect.
>
> I was browsing the code and I see that veth_xmit calls ____dev_forward_skb
> which does a skb_scrub_packet, which in turn calls the skb destructor function.
Where does it call this?
On the other hand, it is likely that skb_orphan() is called somewhere on the
receive path, which indeed will release the tx_ring slot.
> skb_orphan_frags, called by ____dev_forward_skb, seems to do the right thing,
> but it probably does not get called for packets from AF_PACKET socket, since the
> skb is not a zero copy skb (SKBTX_DEV_ZEROCOPY is not set).
Yes, an skb_copy_ubufs may be needed if the orphan is unavoidable.
^ permalink raw reply
* [net-next 00/11][pull request] 100GbE Intel Wired LAN Driver Updates 2018-11-13
From: Jeff Kirsher @ 2018-11-13 18:32 UTC (permalink / raw)
To: davem; +Cc: Jeff Kirsher, netdev, nhorman, sassmann
This series contains updates to the ice driver only.
Brett cleans up debug print messages by removing useless or duplicate
messages, and make sure we assign the hardware head pointer to head
instead of the software head pointer. Resolved an issue when disabling
SRIOV we were trying to stop queues multiple times, so make sure we
disable SRIOV before stopping transmit and receive queues for VF.
Tony fixes a potential NULL pointer dereference during a VF reset.
Anirudh resolves an issue where we were releasing the VSI before
removing the VSI scheduler node, which was resulting in an error "Failed
to set LAN Tx queue context, error: -1". Also fixed the guaranteed
number of VSIs available and used by discovering the device
capabilities to determine the 'guar_num_vsi' per function, rather than
always using the theoretical max number of VSIs every time.
Dave avoids a deadlock by nesting RTNL locking, so added a boolean to
determine if the RTNL lock is already held.
Lev fixes bad mask values which would break compilation.
Piotr increases the receive queue disable timeout since it can take
additional time to finish all pending queue requests.
Usha resolves an issue of VLAN priority tagged traffic not appearing on
all traffic classes, which was causing ETS bandwidth shaping to not work
as expected.
Henry fixes the reset path to cleanup the old scheduler tree before
rebuilding it.
Md Fahad removes a unnecessary check which was causing a driver load
error on platforms with more than 128 cores.
The following are changes since commit 3e536cff34244959c81575733c9ca60633f74e1b:
net: phy: check if advertising is zero using linkmode_empty
and are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/next-queue 100GbE
Anirudh Venkataramanan (2):
ice: Remove node before releasing VSI
ice: Calculate guaranteed VSIs per function and use it
Brett Creeley (2):
ice: Fix debug print in ice_tx_timeout
ice: Call pci_disable_sriov before stopping queues for VF
Dave Ertman (1):
ice: Avoid nested RTNL locking in ice_dis_vsi
Henry Tieman (1):
ice: Destroy scheduler tree in reset path
Lev Faerman (1):
ice: Fix NVM mask defines
Md Fahad Iqbal Polash (1):
ice: Remove ICE_MAX_TXQ_PER_TXQG check when configuring Tx queue
Piotr Raczynski (1):
ice: Increase Rx queue disable timeout
Tony Nguyen (1):
ice: Check for q_vector when stopping rings
Usha Ketineni (1):
ice: Fix to make VLAN priority tagged traffic to appear on all TCs
drivers/net/ethernet/intel/ice/ice.h | 5 +-
.../net/ethernet/intel/ice/ice_adminq_cmd.h | 7 +-
drivers/net/ethernet/intel/ice/ice_common.c | 31 +++-
.../net/ethernet/intel/ice/ice_hw_autogen.h | 3 +
drivers/net/ethernet/intel/ice/ice_lib.c | 136 ++++++++++--------
drivers/net/ethernet/intel/ice/ice_main.c | 37 +++--
drivers/net/ethernet/intel/ice/ice_sched.c | 110 +++++++++++++-
drivers/net/ethernet/intel/ice/ice_sched.h | 3 +
drivers/net/ethernet/intel/ice/ice_type.h | 4 +-
.../net/ethernet/intel/ice/ice_virtchnl_pf.c | 18 +--
10 files changed, 265 insertions(+), 89 deletions(-)
--
2.19.1
^ permalink raw reply
* [net-next 01/11] ice: Fix debug print in ice_tx_timeout
From: Jeff Kirsher @ 2018-11-13 18:32 UTC (permalink / raw)
To: davem
Cc: Brett Creeley, netdev, nhorman, sassmann, Anirudh Venkataramanan,
Jeff Kirsher
In-Reply-To: <20181113183239.5861-1-jeffrey.t.kirsher@intel.com>
From: Brett Creeley <brett.creeley@intel.com>
Currently the debug print in ice_tx_timeout is printing useless and
duplicate values. First, head is being assigned to tx_ring->next_to_clean
and we are printing both of those values, but naming them HWB and NTC
respectively. Also, reading tail always returns 0 so remove that as well.
Instead of assigning the SW head (NTC) read to head, use the actual head
register and change the debug print to note that this is HW_HEAD. Also
reduce the scope of a couple variables.
Signed-off-by: Brett Creeley <brett.creeley@intel.com>
Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/ice_hw_autogen.h | 3 +++
drivers/net/ethernet/intel/ice/ice_main.c | 15 +++++++++------
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
index 596b9fb1c510..5507928c8fbe 100644
--- a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
+++ b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
@@ -7,6 +7,9 @@
#define _ICE_HW_AUTOGEN_H_
#define QTX_COMM_DBELL(_DBQM) (0x002C0000 + ((_DBQM) * 4))
+#define QTX_COMM_HEAD(_DBQM) (0x000E0000 + ((_DBQM) * 4))
+#define QTX_COMM_HEAD_HEAD_S 0
+#define QTX_COMM_HEAD_HEAD_M ICE_M(0x1FFF, 0)
#define PF_FW_ARQBAH 0x00080180
#define PF_FW_ARQBAL 0x00080080
#define PF_FW_ARQH 0x00080380
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 333312a1d595..8584061e1bc6 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3691,8 +3691,8 @@ static void ice_tx_timeout(struct net_device *netdev)
struct ice_ring *tx_ring = NULL;
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
- u32 head, val = 0, i;
int hung_queue = -1;
+ u32 i;
pf->tx_timeout_count++;
@@ -3736,17 +3736,20 @@ static void ice_tx_timeout(struct net_device *netdev)
return;
if (tx_ring) {
- head = tx_ring->next_to_clean;
+ struct ice_hw *hw = &pf->hw;
+ u32 head, val = 0;
+
+ head = (rd32(hw, QTX_COMM_HEAD(vsi->txq_map[hung_queue])) &
+ QTX_COMM_HEAD_HEAD_M) >> QTX_COMM_HEAD_HEAD_S;
/* Read interrupt register */
if (test_bit(ICE_FLAG_MSIX_ENA, pf->flags))
- val = rd32(&pf->hw,
+ val = rd32(hw,
GLINT_DYN_CTL(tx_ring->q_vector->v_idx +
tx_ring->vsi->hw_base_vector));
- netdev_info(netdev, "tx_timeout: VSI_num: %d, Q %d, NTC: 0x%x, HWB: 0x%x, NTU: 0x%x, TAIL: 0x%x, INT: 0x%x\n",
+ netdev_info(netdev, "tx_timeout: VSI_num: %d, Q %d, NTC: 0x%x, HW_HEAD: 0x%x, NTU: 0x%x, INT: 0x%x\n",
vsi->vsi_num, hung_queue, tx_ring->next_to_clean,
- head, tx_ring->next_to_use,
- readl(tx_ring->tail), val);
+ head, tx_ring->next_to_use, val);
}
pf->tx_timeout_last_recovery = jiffies;
--
2.19.1
^ permalink raw reply related
* [net-next 03/11] ice: Remove node before releasing VSI
From: Jeff Kirsher @ 2018-11-13 18:32 UTC (permalink / raw)
To: davem; +Cc: Anirudh Venkataramanan, netdev, nhorman, sassmann, Jeff Kirsher
In-Reply-To: <20181113183239.5861-1-jeffrey.t.kirsher@intel.com>
From: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Before releasing the VSI, remove the VSI scheduler node. If not,
the node is left in the scheduler tree and, on subsequent load, the
scheduler tree contains the node so it does not set it in vsi_ctx.
This, later, causes the node to not be found in ice_sched_get_free_qparent
which leads to a "Failed to set LAN Tx queue context, error: -1".
To remove the scheduler node, this patch introduces ice_rm_vsi_lan_cfg
and related helpers.
Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/ice_lib.c | 1 +
drivers/net/ethernet/intel/ice/ice_sched.c | 108 ++++++++++++++++++++-
drivers/net/ethernet/intel/ice/ice_sched.h | 2 +
3 files changed, 110 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 11d0ab185dd2..1efd760debc2 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -2492,6 +2492,7 @@ int ice_vsi_release(struct ice_vsi *vsi)
}
ice_remove_vsi_fltr(&pf->hw, vsi->idx);
+ ice_rm_vsi_lan_cfg(vsi->port_info, vsi->idx);
ice_vsi_delete(vsi);
ice_vsi_free_q_vectors(vsi);
ice_vsi_clear_rings(vsi);
diff --git a/drivers/net/ethernet/intel/ice/ice_sched.c b/drivers/net/ethernet/intel/ice/ice_sched.c
index 7cc8aa18a22b..7e807b0e7514 100644
--- a/drivers/net/ethernet/intel/ice/ice_sched.c
+++ b/drivers/net/ethernet/intel/ice/ice_sched.c
@@ -1527,7 +1527,7 @@ ice_sched_update_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_handle,
}
/**
- * ice_sched_cfg_vsi - configure the new/exisiting VSI
+ * ice_sched_cfg_vsi - configure the new/existing VSI
* @pi: port information structure
* @vsi_handle: software VSI handle
* @tc: TC number
@@ -1605,3 +1605,109 @@ ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 maxqs,
return status;
}
+
+/**
+ * ice_sched_rm_agg_vsi_entry - remove agg related VSI info entry
+ * @pi: port information structure
+ * @vsi_handle: software VSI handle
+ *
+ * This function removes single aggregator VSI info entry from
+ * aggregator list.
+ */
+static void
+ice_sched_rm_agg_vsi_info(struct ice_port_info *pi, u16 vsi_handle)
+{
+ struct ice_sched_agg_info *agg_info;
+ struct ice_sched_agg_info *atmp;
+
+ list_for_each_entry_safe(agg_info, atmp, &pi->agg_list, list_entry) {
+ struct ice_sched_agg_vsi_info *agg_vsi_info;
+ struct ice_sched_agg_vsi_info *vtmp;
+
+ list_for_each_entry_safe(agg_vsi_info, vtmp,
+ &agg_info->agg_vsi_list, list_entry)
+ if (agg_vsi_info->vsi_handle == vsi_handle) {
+ list_del(&agg_vsi_info->list_entry);
+ devm_kfree(ice_hw_to_dev(pi->hw),
+ agg_vsi_info);
+ return;
+ }
+ }
+}
+
+/**
+ * ice_sched_rm_vsi_cfg - remove the VSI and its children nodes
+ * @pi: port information structure
+ * @vsi_handle: software VSI handle
+ * @owner: LAN or RDMA
+ *
+ * This function removes the VSI and its LAN or RDMA children nodes from the
+ * scheduler tree.
+ */
+static enum ice_status
+ice_sched_rm_vsi_cfg(struct ice_port_info *pi, u16 vsi_handle, u8 owner)
+{
+ enum ice_status status = ICE_ERR_PARAM;
+ struct ice_vsi_ctx *vsi_ctx;
+ u8 i, j = 0;
+
+ if (!ice_is_vsi_valid(pi->hw, vsi_handle))
+ return status;
+ mutex_lock(&pi->sched_lock);
+ vsi_ctx = ice_get_vsi_ctx(pi->hw, vsi_handle);
+ if (!vsi_ctx)
+ goto exit_sched_rm_vsi_cfg;
+
+ for (i = 0; i < ICE_MAX_TRAFFIC_CLASS; i++) {
+ struct ice_sched_node *vsi_node, *tc_node;
+
+ tc_node = ice_sched_get_tc_node(pi, i);
+ if (!tc_node)
+ continue;
+
+ vsi_node = ice_sched_get_vsi_node(pi->hw, tc_node, vsi_handle);
+ if (!vsi_node)
+ continue;
+
+ while (j < vsi_node->num_children) {
+ if (vsi_node->children[j]->owner == owner) {
+ ice_free_sched_node(pi, vsi_node->children[j]);
+
+ /* reset the counter again since the num
+ * children will be updated after node removal
+ */
+ j = 0;
+ } else {
+ j++;
+ }
+ }
+ /* remove the VSI if it has no children */
+ if (!vsi_node->num_children) {
+ ice_free_sched_node(pi, vsi_node);
+ vsi_ctx->sched.vsi_node[i] = NULL;
+
+ /* clean up agg related vsi info if any */
+ ice_sched_rm_agg_vsi_info(pi, vsi_handle);
+ }
+ if (owner == ICE_SCHED_NODE_OWNER_LAN)
+ vsi_ctx->sched.max_lanq[i] = 0;
+ }
+ status = 0;
+
+exit_sched_rm_vsi_cfg:
+ mutex_unlock(&pi->sched_lock);
+ return status;
+}
+
+/**
+ * ice_rm_vsi_lan_cfg - remove VSI and its LAN children nodes
+ * @pi: port information structure
+ * @vsi_handle: software VSI handle
+ *
+ * This function clears the VSI and its LAN children nodes from scheduler tree
+ * for all TCs.
+ */
+enum ice_status ice_rm_vsi_lan_cfg(struct ice_port_info *pi, u16 vsi_handle)
+{
+ return ice_sched_rm_vsi_cfg(pi, vsi_handle, ICE_SCHED_NODE_OWNER_LAN);
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_sched.h b/drivers/net/ethernet/intel/ice/ice_sched.h
index 5dc9cfa04c58..dc59fbac7dde 100644
--- a/drivers/net/ethernet/intel/ice/ice_sched.h
+++ b/drivers/net/ethernet/intel/ice/ice_sched.h
@@ -12,6 +12,7 @@
struct ice_sched_agg_vsi_info {
struct list_head list_entry;
DECLARE_BITMAP(tc_bitmap, ICE_MAX_TRAFFIC_CLASS);
+ u16 vsi_handle;
};
struct ice_sched_agg_info {
@@ -39,4 +40,5 @@ ice_sched_get_free_qparent(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
enum ice_status
ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 maxqs,
u8 owner, bool enable);
+enum ice_status ice_rm_vsi_lan_cfg(struct ice_port_info *pi, u16 vsi_handle);
#endif /* _ICE_SCHED_H_ */
--
2.19.1
^ permalink raw reply related
* [net-next 02/11] ice: Check for q_vector when stopping rings
From: Jeff Kirsher @ 2018-11-13 18:32 UTC (permalink / raw)
To: davem
Cc: Tony Nguyen, netdev, nhorman, sassmann, Anirudh Venkataramanan,
Jeff Kirsher
In-Reply-To: <20181113183239.5861-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
There is a gap in time between a VF reset, which sets the q_vector to
NULL, and the VF requesting mapping of the q_vectors. If
ice_vsi_stop_tx_rings() is called during this time, a NULL pointer
dereference is encountered. Add a check in ice_vsi_stop_tx_rings()
to ensure the q_vector is set to avoid this situation from occurring.
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/ice_lib.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 1041fa2a7767..11d0ab185dd2 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1908,7 +1908,8 @@ int ice_vsi_stop_tx_rings(struct ice_vsi *vsi, enum ice_disq_rst_src rst_src,
ice_for_each_txq(vsi, i) {
u16 v_idx;
- if (!vsi->tx_rings || !vsi->tx_rings[i]) {
+ if (!vsi->tx_rings || !vsi->tx_rings[i] ||
+ !vsi->tx_rings[i]->q_vector) {
err = -EINVAL;
goto err_out;
}
--
2.19.1
^ permalink raw reply related
* [net-next 05/11] ice: Avoid nested RTNL locking in ice_dis_vsi
From: Jeff Kirsher @ 2018-11-13 18:32 UTC (permalink / raw)
To: davem
Cc: Dave Ertman, netdev, nhorman, sassmann, Anirudh Venkataramanan,
Jeff Kirsher
In-Reply-To: <20181113183239.5861-1-jeffrey.t.kirsher@intel.com>
From: Dave Ertman <david.m.ertman@intel.com>
ice_dis_vsi() performs an rtnl_lock() if it detects a netdev that is
running on the VSI. In cases where the RTNL lock has already been
acquired, a deadlock results. Add a boolean to pass to ice_dis_vsi to
tell it if the RTNL lock is already held.
Signed-off-by: Dave Ertman <david.m.ertman@intel.com>
Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/ice_main.c | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index ea79e5e1f589..089b0f0b2e71 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3137,8 +3137,9 @@ static void ice_vsi_release_all(struct ice_pf *pf)
/**
* ice_dis_vsi - pause a VSI
* @vsi: the VSI being paused
+ * @locked: is the rtnl_lock already held
*/
-static void ice_dis_vsi(struct ice_vsi *vsi)
+static void ice_dis_vsi(struct ice_vsi *vsi, bool locked)
{
if (test_bit(__ICE_DOWN, vsi->state))
return;
@@ -3147,9 +3148,13 @@ static void ice_dis_vsi(struct ice_vsi *vsi)
if (vsi->type == ICE_VSI_PF && vsi->netdev) {
if (netif_running(vsi->netdev)) {
- rtnl_lock();
- vsi->netdev->netdev_ops->ndo_stop(vsi->netdev);
- rtnl_unlock();
+ if (!locked) {
+ rtnl_lock();
+ vsi->netdev->netdev_ops->ndo_stop(vsi->netdev);
+ rtnl_unlock();
+ } else {
+ vsi->netdev->netdev_ops->ndo_stop(vsi->netdev);
+ }
} else {
ice_vsi_close(vsi);
}
@@ -3188,7 +3193,7 @@ static void ice_pf_dis_all_vsi(struct ice_pf *pf)
ice_for_each_vsi(pf, v)
if (pf->vsi[v])
- ice_dis_vsi(pf->vsi[v]);
+ ice_dis_vsi(pf->vsi[v], false);
}
/**
--
2.19.1
^ permalink raw reply related
* [net-next 04/11] ice: Calculate guaranteed VSIs per function and use it
From: Jeff Kirsher @ 2018-11-13 18:32 UTC (permalink / raw)
To: davem; +Cc: Anirudh Venkataramanan, netdev, nhorman, sassmann, Jeff Kirsher
In-Reply-To: <20181113183239.5861-1-jeffrey.t.kirsher@intel.com>
From: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Currently we are setting the guar_num_vsi to equal to ICE_MAX_VSI
which is the device limit of 768. This is incorrect and could have
unintended consequences. To fix this use the valid_function's 8-bit
bitmap returned from discovering device capabilities to determine the
guar_num_vsi per function. guar_num_vsi value is then passed on to
pf->num_alloc_vsi.
Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 1 -
.../net/ethernet/intel/ice/ice_adminq_cmd.h | 1 +
drivers/net/ethernet/intel/ice/ice_common.c | 31 +++++++++++++++++--
drivers/net/ethernet/intel/ice/ice_main.c | 3 +-
drivers/net/ethernet/intel/ice/ice_type.h | 4 ++-
5 files changed, 34 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index b8548370f1c7..ba03cbd3638e 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -52,7 +52,6 @@ extern const char ice_drv_ver[];
#define ICE_MBXQ_LEN 64
#define ICE_MIN_MSIX 2
#define ICE_NO_VSI 0xffff
-#define ICE_MAX_VSI_ALLOC 130
#define ICE_MAX_TXQS 2048
#define ICE_MAX_RXQS 2048
#define ICE_VSI_MAP_CONTIG 0
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 6653555f55dd..602f02a0a2d1 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -87,6 +87,7 @@ struct ice_aqc_list_caps {
/* Device/Function buffer entry, repeated per reported capability */
struct ice_aqc_list_caps_elem {
__le16 cap;
+#define ICE_AQC_CAPS_VALID_FUNCTIONS 0x0005
#define ICE_AQC_CAPS_SRIOV 0x0012
#define ICE_AQC_CAPS_VF 0x0013
#define ICE_AQC_CAPS_VSI 0x0017
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 554fd707a6d6..9de5a3aac77d 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -1386,6 +1386,27 @@ void ice_release_res(struct ice_hw *hw, enum ice_aq_res_ids res)
}
}
+/**
+ * ice_get_guar_num_vsi - determine number of guar VSI for a PF
+ * @hw: pointer to the hw structure
+ *
+ * Determine the number of valid functions by going through the bitmap returned
+ * from parsing capabilities and use this to calculate the number of VSI per PF.
+ */
+static u32 ice_get_guar_num_vsi(struct ice_hw *hw)
+{
+ u8 funcs;
+
+#define ICE_CAPS_VALID_FUNCS_M 0xFF
+ funcs = hweight8(hw->dev_caps.common_cap.valid_functions &
+ ICE_CAPS_VALID_FUNCS_M);
+
+ if (!funcs)
+ return 0;
+
+ return ICE_MAX_VSI / funcs;
+}
+
/**
* ice_parse_caps - parse function/device capabilities
* @hw: pointer to the hw struct
@@ -1428,6 +1449,12 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,
u16 cap = le16_to_cpu(cap_resp->cap);
switch (cap) {
+ case ICE_AQC_CAPS_VALID_FUNCTIONS:
+ caps->valid_functions = number;
+ ice_debug(hw, ICE_DBG_INIT,
+ "HW caps: Valid Functions = %d\n",
+ caps->valid_functions);
+ break;
case ICE_AQC_CAPS_SRIOV:
caps->sr_iov_1_1 = (number == 1);
ice_debug(hw, ICE_DBG_INIT,
@@ -1457,10 +1484,10 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,
"HW caps: Dev.VSI cnt = %d\n",
dev_p->num_vsi_allocd_to_host);
} else if (func_p) {
- func_p->guaranteed_num_vsi = number;
+ func_p->guar_num_vsi = ice_get_guar_num_vsi(hw);
ice_debug(hw, ICE_DBG_INIT,
"HW caps: Func.VSI cnt = %d\n",
- func_p->guaranteed_num_vsi);
+ number);
}
break;
case ICE_AQC_CAPS_RSS:
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 8584061e1bc6..ea79e5e1f589 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -2091,8 +2091,7 @@ static int ice_probe(struct pci_dev *pdev,
ice_determine_q_usage(pf);
- pf->num_alloc_vsi = min_t(u16, ICE_MAX_VSI_ALLOC,
- hw->func_caps.guaranteed_num_vsi);
+ pf->num_alloc_vsi = hw->func_caps.guar_num_vsi;
if (!pf->num_alloc_vsi) {
err = -EIO;
goto err_init_pf_unroll;
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index f4dbc81c1988..0ea428104215 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -124,6 +124,8 @@ struct ice_phy_info {
/* Common HW capabilities for SW use */
struct ice_hw_common_caps {
+ u32 valid_functions;
+
/* TX/RX queues */
u16 num_rxq; /* Number/Total RX queues */
u16 rxq_first_id; /* First queue ID for RX queues */
@@ -150,7 +152,7 @@ struct ice_hw_func_caps {
struct ice_hw_common_caps common_cap;
u32 num_allocd_vfs; /* Number of allocated VFs */
u32 vf_base_id; /* Logical ID of the first VF */
- u32 guaranteed_num_vsi;
+ u32 guar_num_vsi;
};
/* Device wide capabilities */
--
2.19.1
^ permalink raw reply related
* [net-next 06/11] ice: Fix NVM mask defines
From: Jeff Kirsher @ 2018-11-13 18:32 UTC (permalink / raw)
To: davem
Cc: Lev Faerman, netdev, nhorman, sassmann, Anirudh Venkataramanan,
Jeff Kirsher
In-Reply-To: <20181113183239.5861-1-jeffrey.t.kirsher@intel.com>
From: Lev Faerman <lev.faerman@intel.com>
Fixes bad masks that would break compilation when evaluated.
Signed-off-by: Lev Faerman <lev.faerman@intel.com>
Signed-off-by: Anirudh Venkataramanan <anirudh.venkataramanan@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/ice_adminq_cmd.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 602f02a0a2d1..4078070881ce 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -1066,10 +1066,10 @@ struct ice_aqc_nvm {
#define ICE_AQC_NVM_LAST_CMD BIT(0)
#define ICE_AQC_NVM_PCIR_REQ BIT(0) /* Used by NVM Update reply */
#define ICE_AQC_NVM_PRESERVATION_S 1
-#define ICE_AQC_NVM_PRESERVATION_M (3 << CSR_AQ_NVM_PRESERVATION_S)
-#define ICE_AQC_NVM_NO_PRESERVATION (0 << CSR_AQ_NVM_PRESERVATION_S)
+#define ICE_AQC_NVM_PRESERVATION_M (3 << ICE_AQC_NVM_PRESERVATION_S)
+#define ICE_AQC_NVM_NO_PRESERVATION (0 << ICE_AQC_NVM_PRESERVATION_S)
#define ICE_AQC_NVM_PRESERVE_ALL BIT(1)
-#define ICE_AQC_NVM_PRESERVE_SELECTED (3 << CSR_AQ_NVM_PRESERVATION_S)
+#define ICE_AQC_NVM_PRESERVE_SELECTED (3 << ICE_AQC_NVM_PRESERVATION_S)
#define ICE_AQC_NVM_FLASH_ONLY BIT(7)
__le16 module_typeid;
__le16 length;
--
2.19.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox