* Re: [PATCH] netfilter: ipset: Use is_zero_ether_addr instead of static and memcmp
From: Pablo Neira Ayuso @ 2018-03-30 9:54 UTC (permalink / raw)
To: Joe Perches
Cc: linux-kernel, Jozsef Kadlecsik, Florian Westphal, David S. Miller,
netfilter-devel, coreteam, netdev
In-Reply-To: <687a4edaed07441681d3bfe3b63aa4feb55b9d61.1521567319.git.joe@perches.com>
On Tue, Mar 20, 2018 at 10:35:47AM -0700, Joe Perches wrote:
> To make the test a bit clearer and to reduce object size a little.
>
> Miscellanea:
>
> o remove now unnecessary static const array
>
> $ size ip_set_hash_mac.o*
> text data bss dec hex filename
> 22822 4619 64 27505 6b71 ip_set_hash_mac.o.allyesconfig.new
> 22932 4683 64 27679 6c1f ip_set_hash_mac.o.allyesconfig.old
> 10443 1040 0 11483 2cdb ip_set_hash_mac.o.defconfig.new
> 10507 1040 0 11547 2d1b ip_set_hash_mac.o.defconfig.old
Applied, thanks Joe.
^ permalink raw reply
* Re: [PATCH net-next 0/9] devlink: Add support for region access
From: Rahul Lakkireddy @ 2018-03-30 9:51 UTC (permalink / raw)
To: Andrew Lunn
Cc: Alex Vesker, David S. Miller, netdev@vger.kernel.org,
Tariq Toukan, Jiri Pirko
In-Reply-To: <20180329182343.GB12150@lunn.ch>
On Thursday, March 03/29/18, 2018 at 23:53:43 +0530, Andrew Lunn wrote:
> On Thu, Mar 29, 2018 at 07:07:43PM +0300, Alex Vesker wrote:
> > This is a proposal which will allow access to driver defined address
> > regions using devlink. Each device can create its supported address
> > regions and register them. A device which exposes a region will allow
> > access to it using devlink.
>
> Hi Alex
>
> Did you see the work Rahul Lakkireddy has been doing?
>
> https://patchwork.kernel.org/patch/10305935/
>
> It seems like these are similar, or at least overlapping. We probably
> want one solution for both.
>
We're already collecting hardware snapshots when system is live with
ethtool --getdump (which devlink tool is now trying to do).
We are now in the process of adding support to collect hardware
snapshots during kernel panic.
Thanks,
Rahul
^ permalink raw reply
* Re: [PATCH] bridge: netfilter: ebt_stp: Use generic functions for comparisons
From: Pablo Neira Ayuso @ 2018-03-30 9:52 UTC (permalink / raw)
To: Joe Perches
Cc: Jozsef Kadlecsik, Florian Westphal, Stephen Hemminger,
David S. Miller, netfilter-devel, coreteam, bridge, netdev,
linux-kernel
In-Reply-To: <c22b7398816c4533a65ff57aaa44629729fc7254.1521630169.git.joe@perches.com>
On Wed, Mar 21, 2018 at 04:03:22AM -0700, Joe Perches wrote:
> Instead of unnecessary const declarations, use the generic functions to
> save a little object space.
>
> $ size net/bridge/netfilter/ebt_stp.o*
> text data bss dec hex filename
> 1250 144 0 1394 572 net/bridge/netfilter/ebt_stp.o.new
> 1344 144 0 1488 5d0 net/bridge/netfilter/ebt_stp.o.old
Applied, thanks Joe.
^ permalink raw reply
* Re: [PATCH] net: netfilter: Merge assignment with return
From: Pablo Neira Ayuso @ 2018-03-30 9:51 UTC (permalink / raw)
To: Arushi Singhal
Cc: outreachy-kernel, Jozsef Kadlecsik, Florian Westphal,
David S. Miller, netfilter-devel, coreteam, netdev, linux-kernel
In-Reply-To: <20180328190950.GA3151@seema-Inspiron-15-3567>
On Thu, Mar 29, 2018 at 12:39:50AM +0530, Arushi Singhal wrote:
> Merge assignment with return statement to directly return the value.
Applied, thanks Arushi.
^ permalink raw reply
* Re: [PATCH 2/2] net: mvneta: improve suspend/resume
From: Thomas Petazzoni @ 2018-03-30 9:49 UTC (permalink / raw)
To: Jisheng Zhang; +Cc: David Miller, netdev, linux-arm-kernel, linux-kernel
In-Reply-To: <20180330171547.5f96bbff@xhacker.debian>
Hello,
On Fri, 30 Mar 2018 17:15:47 +0800, Jisheng Zhang wrote:
> > > + cpuhp_state_remove_instance_nocalls(online_hpstate,
> > > + &pp->node_online);
> > > + cpuhp_state_remove_instance_nocalls(CPUHP_NET_MVNETA_DEAD,
> > > + &pp->node_dead);
> >
> > Do we need to remove/add those CPU notifiers when suspending/resuming ?
>
> Take mvneta_cpu_online() as an example, if we don't remove it during
> suspend, when system is resume back, it will touch mac when secondary
> cpu is ON, but at this point the mvneta isn't resumed, this is not safe.
Hm. I'm still a bit confused by this new CPU hotplug API. I understand
the issue you have and indeed unregistering the CPU hotplug callbacks
is a way to solve the problem, but I find it weird that we have to do
this.
Anyway, it's OK to do it, because it's anyway what was done so far. It
is just annoying that there is a duplication of the logic between
mvneta_suspend() and mvneta_stop() on one side, and duplication between
mvneta_resume() and mvnete_start() on the other side.
> > > + for (queue = 0; queue < rxq_number; queue++) {
> > > + struct mvneta_rx_queue *rxq = &pp->rxqs[queue];
> > > +
> > > + mvneta_rxq_drop_pkts(pp, rxq);
> > > + }
> >
> > Wouldn't it make sense to have
> > mvneta_rxq_sw_deinit/mvneta_rxq_hw_deinit(), like you did for the
> > initialization ?
>
> For rxq deinit, we'd like to drop rx pkts, this is both HW and SW operation.
> So we reuse mvneta_rxq_drop_pkts() here.
Hum, OK, indeed. It would have been nicer to have something symmetric,
with the hw/sw parts split in a similar way for the init and deinit of
both txqs and rxqs, but I agree that dropping the RX packets before
going into suspend involves both HW and SW operations.
Thanks!
Thomas Petazzoni
--
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply
* Re: [PATCH net-next 2/3] netfilter: Rework xt_TEE netdevice notifier
From: Pablo Neira Ayuso @ 2018-03-30 9:47 UTC (permalink / raw)
To: Kirill Tkhai
Cc: davem, steffen.klassert, herbert, kadlec, fw, daniel,
jakub.kicinski, ast, brouer, linux, john.fastabend, dsahern,
netdev, netfilter-devel, coreteam
In-Reply-To: <152233221526.1654.14062155454284081692.stgit@localhost.localdomain>
On Thu, Mar 29, 2018 at 05:03:35PM +0300, Kirill Tkhai wrote:
> Register netdevice notifier for every iptable entry
> is not good, since this breaks modularity, and
> the hidden synchronization is based on rtnl_lock().
>
> This patch reworks the synchronization via new lock,
> while the rest of logic remains as it was before.
> This is required for the next patch.
>
> Tested via:
>
> while :; do
> unshare -n iptables -t mangle -A OUTPUT -j TEE --gateway 1.1.1.2 --oif lo;
> done
>
> Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
^ permalink raw reply
* Re: [PATCH net-next] dt-bindings: net: renesas-ravb: Add support for r8a77470 SoC
From: Geert Uytterhoeven @ 2018-03-30 9:26 UTC (permalink / raw)
To: Biju Das
Cc: David S . Miller, Rob Herring, Mark Rutland, Russell King,
Geert Uytterhoeven, Sergei Shtylyov, Simon Horman, Magnus Damm,
Chris Paterson, Fabrizio Castro,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
Linux-Renesas, Linux ARM, netdev
In-Reply-To: <1522317775-35671-1-git-send-email-biju.das@bp.renesas.com>
On Thu, Mar 29, 2018 at 12:02 PM, Biju Das <biju.das@bp.renesas.com> wrote:
> Add a new compatible string for the RZ/G1C (R8A77470) SoC.
>
> Signed-off-by: Biju Das <biju.das@bp.renesas.com>
> Reviewed-by: Fabrizio Castro <fabrizio.castro@bp.renesas.com>
Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* [PATCH net-next] net: Revert net_rwsem
From: Kirill Tkhai @ 2018-03-30 9:23 UTC (permalink / raw)
To: davem, ktkhai, netdev
This reverts:
152f253152cc net: Remove rtnl_lock() in nf_ct_iterate_destroy()
ec9c780925c5 ovs: Remove rtnl_lock() from ovs_exit_net()
350311aab4c0 security: Remove rtnl_lock() in selinux_xfrm_notify_policyload()
10256debb918 net: Don't take rtnl_lock() in wireless_nlevent_flush()
f0b07bb151b0 net: Introduce net_rwsem to protect net_namespace_list
There are missed, that down_read() can't be taken recursive.
This is because of rw_semaphore design, which prevents it
to be occupied by only readers forever.
So, we can't take it in register_netdevice_notifier(), as it's also
taken in wext_netdev_notifier_call()->wireless_nlevent_flush().
The solution is to protect net_namespace_list modifications
in register_netdevice_notifier() via pernet_ops_rwsem, as it's
made in:
https://patchwork.ozlabs.org/project/netdev/list/?series=36495
Then, we won't have to take net_rwsem in call_netevent_notifiers().
But since the patchset is not in kernel, let's just revert net_rwsem
for now, and I'll resubmit it later (after the above patchset).
Signed-off-by: Kirill Tkhai <ktkhai@virtuozzo.com>
---
drivers/infiniband/core/roce_gid_mgmt.c | 2 --
include/linux/rtnetlink.h | 1 -
include/net/net_namespace.h | 1 -
net/core/dev.c | 5 -----
net/core/fib_notifier.c | 2 --
net/core/net_namespace.c | 18 +++++-------------
net/core/rtnetlink.c | 5 -----
net/netfilter/nf_conntrack_core.c | 4 ++--
net/openvswitch/datapath.c | 4 ++--
net/wireless/wext-core.c | 6 ++++--
security/selinux/include/xfrm.h | 4 ++--
11 files changed, 15 insertions(+), 37 deletions(-)
diff --git a/drivers/infiniband/core/roce_gid_mgmt.c b/drivers/infiniband/core/roce_gid_mgmt.c
index cc2966380c0c..5a52ec77940a 100644
--- a/drivers/infiniband/core/roce_gid_mgmt.c
+++ b/drivers/infiniband/core/roce_gid_mgmt.c
@@ -403,12 +403,10 @@ static void enum_all_gids_of_dev_cb(struct ib_device *ib_dev,
* our feet
*/
rtnl_lock();
- down_read(&net_rwsem);
for_each_net(net)
for_each_netdev(net, ndev)
if (is_eth_port_of_netdev(ib_dev, port, rdma_ndev, ndev))
add_netdev_ips(ib_dev, port, rdma_ndev, ndev);
- up_read(&net_rwsem);
rtnl_unlock();
}
diff --git a/include/linux/rtnetlink.h b/include/linux/rtnetlink.h
index 5225832bd6ff..c7d1e4689325 100644
--- a/include/linux/rtnetlink.h
+++ b/include/linux/rtnetlink.h
@@ -37,7 +37,6 @@ extern int rtnl_lock_killable(void);
extern wait_queue_head_t netdev_unregistering_wq;
extern struct rw_semaphore pernet_ops_rwsem;
-extern struct rw_semaphore net_rwsem;
#ifdef CONFIG_PROVE_LOCKING
extern bool lockdep_rtnl_is_held(void);
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index 47e35cce3b64..1ab4f920f109 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -291,7 +291,6 @@ static inline struct net *read_pnet(const possible_net_t *pnet)
#endif
}
-/* Protected by net_rwsem */
#define for_each_net(VAR) \
list_for_each_entry(VAR, &net_namespace_list, list)
diff --git a/net/core/dev.c b/net/core/dev.c
index eca5458b2753..e13807b5c84d 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -1629,7 +1629,6 @@ int register_netdevice_notifier(struct notifier_block *nb)
goto unlock;
if (dev_boot_phase)
goto unlock;
- down_read(&net_rwsem);
for_each_net(net) {
for_each_netdev(net, dev) {
err = call_netdevice_notifier(nb, NETDEV_REGISTER, dev);
@@ -1643,7 +1642,6 @@ int register_netdevice_notifier(struct notifier_block *nb)
call_netdevice_notifier(nb, NETDEV_UP, dev);
}
}
- up_read(&net_rwsem);
unlock:
rtnl_unlock();
@@ -1666,7 +1664,6 @@ int register_netdevice_notifier(struct notifier_block *nb)
}
outroll:
- up_read(&net_rwsem);
raw_notifier_chain_unregister(&netdev_chain, nb);
goto unlock;
}
@@ -1697,7 +1694,6 @@ int unregister_netdevice_notifier(struct notifier_block *nb)
if (err)
goto unlock;
- down_read(&net_rwsem);
for_each_net(net) {
for_each_netdev(net, dev) {
if (dev->flags & IFF_UP) {
@@ -1708,7 +1704,6 @@ int unregister_netdevice_notifier(struct notifier_block *nb)
call_netdevice_notifier(nb, NETDEV_UNREGISTER, dev);
}
}
- up_read(&net_rwsem);
unlock:
rtnl_unlock();
return err;
diff --git a/net/core/fib_notifier.c b/net/core/fib_notifier.c
index 13a40b831d6d..b793b523aba3 100644
--- a/net/core/fib_notifier.c
+++ b/net/core/fib_notifier.c
@@ -39,7 +39,6 @@ static unsigned int fib_seq_sum(void)
struct net *net;
rtnl_lock();
- down_read(&net_rwsem);
for_each_net(net) {
rcu_read_lock();
list_for_each_entry_rcu(ops, &net->fib_notifier_ops, list) {
@@ -50,7 +49,6 @@ static unsigned int fib_seq_sum(void)
}
rcu_read_unlock();
}
- up_read(&net_rwsem);
rtnl_unlock();
return fib_seq;
diff --git a/net/core/net_namespace.c b/net/core/net_namespace.c
index 7fdf321d4997..b5796d17a302 100644
--- a/net/core/net_namespace.c
+++ b/net/core/net_namespace.c
@@ -33,10 +33,6 @@ static struct list_head *first_device = &pernet_list;
LIST_HEAD(net_namespace_list);
EXPORT_SYMBOL_GPL(net_namespace_list);
-/* Protects net_namespace_list. Nests iside rtnl_lock() */
-DECLARE_RWSEM(net_rwsem);
-EXPORT_SYMBOL_GPL(net_rwsem);
-
struct net init_net = {
.count = REFCOUNT_INIT(1),
.dev_base_head = LIST_HEAD_INIT(init_net.dev_base_head),
@@ -313,9 +309,9 @@ static __net_init int setup_net(struct net *net, struct user_namespace *user_ns)
if (error < 0)
goto out_undo;
}
- down_write(&net_rwsem);
+ rtnl_lock();
list_add_tail_rcu(&net->list, &net_namespace_list);
- up_write(&net_rwsem);
+ rtnl_unlock();
out:
return error;
@@ -454,7 +450,7 @@ static void unhash_nsid(struct net *net, struct net *last)
* and this work is the only process, that may delete
* a net from net_namespace_list. So, when the below
* is executing, the list may only grow. Thus, we do not
- * use for_each_net_rcu() or net_rwsem.
+ * use for_each_net_rcu() or rtnl_lock().
*/
for_each_net(tmp) {
int id;
@@ -489,7 +485,7 @@ static void cleanup_net(struct work_struct *work)
down_read(&pernet_ops_rwsem);
/* Don't let anyone else find us. */
- down_write(&net_rwsem);
+ rtnl_lock();
llist_for_each_entry(net, net_kill_list, cleanup_list)
list_del_rcu(&net->list);
/* Cache last net. After we unlock rtnl, no one new net
@@ -503,7 +499,7 @@ static void cleanup_net(struct work_struct *work)
* useless anyway, as netns_ids are destroyed there.
*/
last = list_last_entry(&net_namespace_list, struct net, list);
- up_write(&net_rwsem);
+ rtnl_unlock();
llist_for_each_entry(net, net_kill_list, cleanup_list) {
unhash_nsid(net, last);
@@ -904,9 +900,6 @@ static int __register_pernet_operations(struct list_head *list,
list_add_tail(&ops->list, list);
if (ops->init || (ops->id && ops->size)) {
- /* We held write locked pernet_ops_rwsem, and parallel
- * setup_net() and cleanup_net() are not possible.
- */
for_each_net(net) {
error = ops_init(ops, net);
if (error)
@@ -930,7 +923,6 @@ static void __unregister_pernet_operations(struct pernet_operations *ops)
LIST_HEAD(net_exit_list);
list_del(&ops->list);
- /* See comment in __register_pernet_operations() */
for_each_net(net)
list_add_tail(&net->exit_list, &net_exit_list);
ops_exit_list(ops, &net_exit_list);
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index e86b28482ca7..2d3949789cef 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -418,11 +418,9 @@ void __rtnl_link_unregister(struct rtnl_link_ops *ops)
{
struct net *net;
- down_read(&net_rwsem);
for_each_net(net) {
__rtnl_kill_links(net, ops);
}
- up_read(&net_rwsem);
list_del(&ops->list);
}
EXPORT_SYMBOL_GPL(__rtnl_link_unregister);
@@ -440,9 +438,6 @@ static void rtnl_lock_unregistering_all(void)
for (;;) {
unregistering = false;
rtnl_lock();
- /* We held write locked pernet_ops_rwsem, and parallel
- * setup_net() and cleanup_net() are not possible.
- */
for_each_net(net) {
if (net->dev_unreg_count > 0) {
unregistering = true;
diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index 41ff04ee2554..705198de671d 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -1763,14 +1763,14 @@ nf_ct_iterate_destroy(int (*iter)(struct nf_conn *i, void *data), void *data)
{
struct net *net;
- down_read(&net_rwsem);
+ rtnl_lock();
for_each_net(net) {
if (atomic_read(&net->ct.count) == 0)
continue;
__nf_ct_unconfirmed_destroy(net);
nf_queue_nf_hook_drop(net);
}
- up_read(&net_rwsem);
+ rtnl_unlock();
/* Need to wait for netns cleanup worker to finish, if its
* running -- it might have deleted a net namespace from
diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c
index 015e24e08909..ef38e5aecd28 100644
--- a/net/openvswitch/datapath.c
+++ b/net/openvswitch/datapath.c
@@ -2363,10 +2363,10 @@ static void __net_exit ovs_exit_net(struct net *dnet)
list_for_each_entry_safe(dp, dp_next, &ovs_net->dps, list_node)
__dp_destroy(dp);
- down_read(&net_rwsem);
+ rtnl_lock();
for_each_net(net)
list_vports_from_net(net, dnet, &head);
- up_read(&net_rwsem);
+ rtnl_unlock();
/* Detach all vports from given namespace. */
list_for_each_entry_safe(vport, vport_next, &head, detach_list) {
diff --git a/net/wireless/wext-core.c b/net/wireless/wext-core.c
index 5e677dac2a0c..9efbfc753347 100644
--- a/net/wireless/wext-core.c
+++ b/net/wireless/wext-core.c
@@ -347,13 +347,13 @@ void wireless_nlevent_flush(void)
struct sk_buff *skb;
struct net *net;
- down_read(&net_rwsem);
+ ASSERT_RTNL();
+
for_each_net(net) {
while ((skb = skb_dequeue(&net->wext_nlevents)))
rtnl_notify(skb, net, 0, RTNLGRP_LINK, NULL,
GFP_KERNEL);
}
- up_read(&net_rwsem);
}
EXPORT_SYMBOL_GPL(wireless_nlevent_flush);
@@ -410,7 +410,9 @@ subsys_initcall(wireless_nlevent_init);
/* Process events generated by the wireless layer or the driver. */
static void wireless_nlevent_process(struct work_struct *work)
{
+ rtnl_lock();
wireless_nlevent_flush();
+ rtnl_unlock();
}
static DECLARE_WORK(wireless_nlevent_work, wireless_nlevent_process);
diff --git a/security/selinux/include/xfrm.h b/security/selinux/include/xfrm.h
index a0b465316292..1f173a7a4daa 100644
--- a/security/selinux/include/xfrm.h
+++ b/security/selinux/include/xfrm.h
@@ -47,10 +47,10 @@ static inline void selinux_xfrm_notify_policyload(void)
{
struct net *net;
- down_read(&net_rwsem);
+ rtnl_lock();
for_each_net(net)
rt_genid_bump_all(net);
- up_read(&net_rwsem);
+ rtnl_unlock();
}
#else
static inline int selinux_xfrm_enabled(void)
^ permalink raw reply related
* Re: [PATCH 2/2] net: mvneta: improve suspend/resume
From: Jisheng Zhang @ 2018-03-30 9:15 UTC (permalink / raw)
To: Thomas Petazzoni; +Cc: David Miller, netdev, linux-arm-kernel, linux-kernel
In-Reply-To: <20180329135432.7da1299b@windsurf>
On Thu, 29 Mar 2018 13:54:32 +0200 Thomas Petazzoni wrote:
> Hello Jisheng,
Hi Thomas,
>
> On Thu, 29 Mar 2018 18:15:36 +0800, Jisheng Zhang wrote:
> > Current suspend/resume implementation reuses the mvneta_open() and
> > mvneta_close(), but it could be optimized to take only necessary
> > actions during suspend/resume.
> >
> > One obvious problem of current implementation is: after hundreds of
> > system suspend/resume cycles, the resume of mvneta could fail due to
> > fragmented dma coherent memory. After this patch, the non-necessary
> > memory alloc/free is optimized out.
>
> Indeed, this needs to be fixed, you're totally right.
>
> > Signed-off-by: Jisheng Zhang <Jisheng.Zhang@synaptics.com>
> > ---
> > drivers/net/ethernet/marvell/mvneta.c | 76 ++++++++++++++++++++++++++++++-----
> > 1 file changed, 66 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> > index 4ec69bbd1eb4..1870f1dd7093 100644
> > --- a/drivers/net/ethernet/marvell/mvneta.c
> > +++ b/drivers/net/ethernet/marvell/mvneta.c
> > @@ -4575,14 +4575,46 @@ static int mvneta_remove(struct platform_device *pdev)
> > #ifdef CONFIG_PM_SLEEP
> > static int mvneta_suspend(struct device *device)
> > {
> > + int queue;
> > struct net_device *dev = dev_get_drvdata(device);
> > struct mvneta_port *pp = netdev_priv(dev);
> >
> > - rtnl_lock();
> > - if (netif_running(dev))
> > - mvneta_stop(dev);
> > - rtnl_unlock();
> > + if (!netif_running(dev))
> > + return 0;
>
> This is changing the behavior I believe. The current code is:
>
> rtnl_lock();
> if (netif_running(dev))
> mvneta_stop(dev);
> rtnl_unlock();
> netif_device_detach(dev);
> clk_disable_unprepare(pp->clk_bus);
> clk_disable_unprepare(pp->clk);
> return 0;
>
> So, when netif_running(dev) is false, we're indeed not calling
> mvneta_stop(), but we're still doing netif_device_detach(), and
> disabling the clocks. With your change, we're no longer doing these
> steps.
Indeed, will try to keep the behavior in v2
>
> > +
> > netif_device_detach(dev);
> > +
> > + mvneta_stop_dev(pp);
> > +
> > + if (!pp->neta_armada3700) {
> > + spin_lock(&pp->lock);
> > + pp->is_stopped = true;
> > + spin_unlock(&pp->lock);
>
> Real question: is it OK to set pp->is_stopped *after* calling
> mvneta_stop_dev(), while it was set before calling mvneta_stop_dev() in
> the current code ?
oops, you are right. Fixed in v2
>
> > +
> > + cpuhp_state_remove_instance_nocalls(online_hpstate,
> > + &pp->node_online);
> > + cpuhp_state_remove_instance_nocalls(CPUHP_NET_MVNETA_DEAD,
> > + &pp->node_dead);
>
> Do we need to remove/add those CPU notifiers when suspending/resuming ?
Take mvneta_cpu_online() as an example, if we don't remove it during
suspend, when system is resume back, it will touch mac when secondary
cpu is ON, but at this point the mvneta isn't resumed, this is not safe.
>
> > + }
> > +
> > + for (queue = 0; queue < rxq_number; queue++) {
> > + struct mvneta_rx_queue *rxq = &pp->rxqs[queue];
> > +
> > + mvneta_rxq_drop_pkts(pp, rxq);
> > + }
>
> Wouldn't it make sense to have
> mvneta_rxq_sw_deinit/mvneta_rxq_hw_deinit(), like you did for the
> initialization ?
For rxq deinit, we'd like to drop rx pkts, this is both HW and SW operation.
So we reuse mvneta_rxq_drop_pkts() here.
>
> > +
> > + for (queue = 0; queue < txq_number; queue++) {
> > + struct mvneta_tx_queue *txq = &pp->txqs[queue];
> > +
> > + /* Set minimum bandwidth for disabled TXQs */
> > + mvreg_write(pp, MVETH_TXQ_TOKEN_CFG_REG(txq->id), 0);
> > + mvreg_write(pp, MVETH_TXQ_TOKEN_COUNT_REG(txq->id), 0);
> > +
> > + /* Set Tx descriptors queue starting address and size */
> > + mvreg_write(pp, MVNETA_TXQ_BASE_ADDR_REG(txq->id), 0);
> > + mvreg_write(pp, MVNETA_TXQ_SIZE_REG(txq->id), 0);
> > + }
>
> Same comment here: a mvneta_txq_sw_deinit()/mvneta_txq_hw_deinit()
> would be good, and would avoid duplicating this logic.
yep, will do in v2.
Thanks a lot for the kind review.
^ permalink raw reply
* Re: [PATCH net-next 03/12] dt-bindings: net: dwmac-sun8i: Clean up clock delay chain descriptions
From: Chen-Yu Tsai @ 2018-03-30 9:13 UTC (permalink / raw)
To: Sergei Shtylyov
Cc: Maxime Ripard, Michael Turquette, Stephen Boyd,
Giuseppe Cavallaro, Rob Herring, Mark Rutland, Mark Brown,
linux-arm-kernel, linux-clk, devicetree, netdev, Corentin Labbe,
Icenowy Zheng
In-Reply-To: <eb47a027-c678-7496-ef58-fc8e96be7dd9@cogentembedded.com>
On Sun, Mar 18, 2018 at 5:21 PM, Sergei Shtylyov
<sergei.shtylyov@cogentembedded.com> wrote:
> Hello!
>
>
> On 3/17/2018 12:28 PM, Chen-Yu Tsai wrote:
>
>> The clock delay chains found in the glue layer for dwmac-sun8i are only
>> used with RGMII PHYs. They are not intended for non-RGMII PHYs, such as
>> MII external PHYs or the internal PHY. Also, a recent SoC has a smaller
>> range of possible values for the delay chain.
>>
>> This patch reformats the delay chain section of the device tree binding
>> to make it clear that the delay chains only apply to RGMII PHYs, and
>> make it easier to add the R40-specific bits later.
>>
>> Signed-off-by: Chen-Yu Tsai <wens@csie.org>
>> ---
>> Documentation/devicetree/bindings/net/dwmac-sun8i.txt | 11 +++++++----
>> 1 file changed, 7 insertions(+), 4 deletions(-)
>>
>> diff --git a/Documentation/devicetree/bindings/net/dwmac-sun8i.txt
>> b/Documentation/devicetree/bindings/net/dwmac-sun8i.txt
>> index 3d6d5fa0c4d5..b8a3028d6c30 100644
>> --- a/Documentation/devicetree/bindings/net/dwmac-sun8i.txt
>> +++ b/Documentation/devicetree/bindings/net/dwmac-sun8i.txt
>> @@ -28,10 +28,13 @@ Required properties:
>> - allwinner,sun8i-a83t-system-controller
>> Optional properties:
>> -- allwinner,tx-delay-ps: TX clock delay chain value in ps. Range value is
>> 0-700. Default is 0)
>> -- allwinner,rx-delay-ps: RX clock delay chain value in ps. Range value is
>> 0-3100. Default is 0)
>> -Both delay properties need to be a multiple of 100. They control the
>> delay for
>> -external PHY.
>> +- allwinner,tx-delay-ps: TX clock delay chain value in ps.
>> + Range is 0-700. Default is 0.
>> +- allwinner,rx-delay-ps: RX clock delay chain value in ps.
>> + Range is 0-3100. Default is 0.
>> +Both delay properties need to be a multiple of 100. They control the
>> +clock delay for external RGMII PHY. They are do apply to the internal
>
>
> s/are do/do not/?
Fixed. Thanks!
ChenYu
^ permalink raw reply
* Re: [PATCH 1/2] net: mvneta: split rxq/txq init into SW and HW parts
From: Jisheng Zhang @ 2018-03-30 9:04 UTC (permalink / raw)
To: Thomas Petazzoni; +Cc: David Miller, netdev, linux-arm-kernel, linux-kernel
In-Reply-To: <20180329134259.3fe6165b@windsurf>
Hi,
On Thu, 29 Mar 2018 13:42:59 +0200 Thomas Petazzoni wrote:
> Hello,
>
> On Thu, 29 Mar 2018 18:13:56 +0800, Jisheng Zhang wrote:
> > This is to prepare the suspend/resume improvement in next patch. The
> > SW parts can be optimized out during resume.
> >
> > Signed-off-by: Jisheng Zhang <Jisheng.Zhang@synaptics.com>
>
> Thanks, I have two very minor nits below, but otherwise:
>
> Acked-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Thanks for reviewing.
>
> > +/* Create a specified RX queue */
> > +static int mvneta_rxq_init(struct mvneta_port *pp,
> > + struct mvneta_rx_queue *rxq)
> > +
> > +{
> > + int ret;
> > +
> > + ret = mvneta_rxq_sw_init(pp, rxq);
> > + if (ret)
>
> Here you're testing if (ret), while in mvneta_txq_init(), in the same
> situation, you're doing if (ret < 0). I don't have a preference for one
> or the other, but having them consistent between the two lpaces would
> be nice.
updated in v2.
>
> > -/* Create and initialize a tx queue */
> > -static int mvneta_txq_init(struct mvneta_port *pp,
> > - struct mvneta_tx_queue *txq)
> > +static int mvneta_txq_sw_init(struct mvneta_port *pp,
> > + struct mvneta_tx_queue *txq)
> > {
> > int cpu;
> >
> > @@ -2872,7 +2889,6 @@ static int mvneta_txq_init(struct mvneta_port *pp,
> > txq->tx_stop_threshold = txq->size - MVNETA_MAX_SKB_DESCS;
> > txq->tx_wake_threshold = txq->tx_stop_threshold / 2;
> >
> > -
>
> Spurious change.
There's an extra blank line here, so I removed it ;)
Thanks
^ permalink raw reply
* Re: [PATCH net-next 4/6] inet: frags: use rhashtables for reassembly units
From: Herbert Xu @ 2018-03-30 7:18 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, netdev, Florian Westphal, Thomas Graf,
Jesper Dangaard Brouer, Alexander Aring, Stefan Schmidt,
Eric Dumazet, Nikolay Aleksandrov
In-Reply-To: <20180330052241.206667-5-edumazet@google.com>
On Thu, Mar 29, 2018 at 10:22:39PM -0700, Eric Dumazet wrote:
>
> void inet_frags_exit_net(struct netns_frags *nf)
> {
> - struct inet_frags *f =nf->f;
> - unsigned int seq;
> - int i;
> -
> - nf->low_thresh = 0;
> + struct rhashtable_iter hti;
> + struct inet_frag_queue *fq;
>
> -evict_again:
> - local_bh_disable();
> - seq = read_seqbegin(&f->rnd_seqlock);
> + nf->low_thresh = 0; /* prevent creation of new frags */
>
> - for (i = 0; i < INETFRAGS_HASHSZ ; i++)
> - inet_evict_bucket(f, &f->hash[i]);
> + rhashtable_walk_enter(&nf->rhashtable, &hti);
> + do {
> + rhashtable_walk_start(&hti);
>
> - local_bh_enable();
> - cond_resched();
> + while ((fq = rhashtable_walk_next(&hti)) && !IS_ERR(fq)) {
> + if (refcount_inc_not_zero(&fq->refcnt)) {
> + spin_lock_bh(&fq->lock);
> + inet_frag_kill(fq);
> + spin_unlock_bh(&fq->lock);
> + inet_frag_put(fq);
> + }
> + }
>
> - if (read_seqretry(&f->rnd_seqlock, seq) ||
> - sum_frag_mem_limit(nf))
> - goto evict_again;
> + rhashtable_walk_stop(&hti);
> + } while (cond_resched(), fq == ERR_PTR(-EAGAIN));
> + rhashtable_walk_exit(&hti);
> + rhashtable_destroy(&nf->rhashtable);
> }
> EXPORT_SYMBOL(inet_frags_exit_net);
Instead of using the walk interface, how about
rhashtable_free_and_destroy?
> void inet_frag_kill(struct inet_frag_queue *fq)
> {
> if (del_timer(&fq->timer))
> refcount_dec(&fq->refcnt);
>
> if (!(fq->flags & INET_FRAG_COMPLETE)) {
> - fq_unlink(fq);
> + struct netns_frags *nf = fq->net;
> +
> + fq->flags |= INET_FRAG_COMPLETE;
> + rhashtable_remove_fast(&nf->rhashtable, &fq->node, nf->f->rhash_params);
> refcount_dec(&fq->refcnt);
> }
> }
This means that the hash won't inline properly. Don't know big
of an issue it is to you. But you could fix it by doing the same
hack as rhashtable by making inet_frag_kill an inline function and
take the rhash_params as an explicit argument.
> -static struct inet_frag_queue *inet_frag_create(struct netns_frags *nf,
> - struct inet_frags *f,
> - void *arg)
> -{
> - struct inet_frag_queue *q;
> + timer_setup(&fq->timer, f->frag_expire, 0);
> + spin_lock_init(&fq->lock);
> + refcount_set(&fq->refcnt, 3);
> + mod_timer(&fq->timer, jiffies + nf->timeout);
>
> - q = inet_frag_alloc(nf, f, arg);
> - if (!q)
> + err = rhashtable_insert_fast(&nf->rhashtable, &fq->node,
> + f->rhash_params);
Ditto.
> -struct inet_frag_queue *inet_frag_find(struct netns_frags *nf,
> - struct inet_frags *f, void *key,
> - unsigned int hash)
> +/* TODO : call from rcu_read_lock() and no longer use refcount_inc_not_zero() */
> +struct inet_frag_queue *inet_frag_find(struct netns_frags *nf, void *key)
> {
> - struct inet_frag_bucket *hb;
> - struct inet_frag_queue *q;
> - int depth = 0;
> -
> - if (frag_mem_limit(nf) > nf->low_thresh)
> - inet_frag_schedule_worker(f);
> -
> - hash &= (INETFRAGS_HASHSZ - 1);
> - hb = &f->hash[hash];
> -
> - spin_lock(&hb->chain_lock);
> - hlist_for_each_entry(q, &hb->chain, list) {
> - if (q->net == nf && f->match(q, key)) {
> - refcount_inc(&q->refcnt);
> - spin_unlock(&hb->chain_lock);
> - return q;
> - }
> - depth++;
> - }
> - spin_unlock(&hb->chain_lock);
> + struct inet_frag_queue *fq;
>
> - if (depth <= INETFRAGS_MAXDEPTH)
> - return inet_frag_create(nf, f, key);
> + rcu_read_lock();
>
> - if (inet_frag_may_rebuild(f)) {
> - if (!f->rebuild)
> - f->rebuild = true;
> - inet_frag_schedule_worker(f);
> + fq = rhashtable_lookup(&nf->rhashtable, key, nf->f->rhash_params);
Ditto.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 07/30] aio: add delayed cancel support
From: Christoph Hellwig @ 2018-03-30 7:14 UTC (permalink / raw)
To: Al Viro
Cc: Christoph Hellwig, Avi Kivity, linux-aio, linux-fsdevel, netdev,
linux-api, linux-kernel
In-Reply-To: <20180329223500.GY30522@ZenIV.linux.org.uk>
On Thu, Mar 29, 2018 at 11:35:00PM +0100, Al Viro wrote:
> Uh-oh... What happens to existing users of kiocb_set_cancel_fn() now?
> AFAICS, those guys will *not* get aio_kiocb freed at all in case of
> io_cancel(2). Look: we mark them with AIO_IOCB_CANCELLED and
> call whatever ->ki_cancel() the driver has set. Later the damn
> thing calls ->ki_complete() (i.e. aio_complete_rw()), which calls
> aio_complete(iocb, res, res2, 0) and gets false. Nothing's freed,
> struct file is leaked.
True, we'd need a complete call from it.
> Frankly, the more I look at that, the less I like what you've done
> with ->ki_cancel() overloading. In regular case it's just accelerating
> the call of ->ki_complete(), which will do freeing. Here you have
> ->ki_cancel() free the damn thing, with the resulting need to play
> silly buggers with locking, freeing logics in aio_complete(), etc.
I don't really like it all that much either, but I also think the
current model is pretty broken - called under spinlock with irqs
disabled is not even what the current users expect. Second issue
with the existing ki_cancel is that kiocb_set_cancel_fn operates
on a kiocb, but expects that to be embedded in an aio_kiocb, which
might not always be the case, althought in-kernel I/O is unlikely to
be used on them. And based on all of these I bet gadget aio cancel
is basically untested.
Anyway, I guess splitting poll direct cancel out in a way that doesn't
overload ->ki_cancel might be a good idea. It all is inside aio.c
so simply switching on the opcode similar to the submission path
might be one option, or having separate methods. Moving ki_cancel
to kiocb would also solve above mismatch issue.
--
To unsubscribe, send a message with 'unsubscribe linux-aio' in
the body to majordomo@kvack.org. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"aart@kvack.org">aart@kvack.org</a>
^ permalink raw reply
* RE: [PATCH net-next 5/8] net: mscc: Add initial Ocelot switch support
From: Razvan Stefanescu @ 2018-03-30 7:02 UTC (permalink / raw)
To: Alexandre Belloni, David S . Miller
Cc: Allan Nielsen, Po Liu, Thomas Petazzoni, Andrew Lunn,
Florian Fainelli, netdev@vger.kernel.org,
devicetree@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-mips@linux-mips.org
In-Reply-To: <20180323201117.8416-6-alexandre.belloni@bootlin.com>
Hello Alexandre,
> +
> + register_netdevice_notifier(&ocelot_netdevice_nb);
> +
> + dev_info(&pdev->dev, "Ocelot switch probed\n");
> +
> + return 0;
> +
> +err_probe_ports:
> + return err;
> +}
> +
> +static int mscc_ocelot_remove(struct platform_device *pdev)
> +{
> + unregister_netevent_notifier(&ocelot_netdevice_nb);
This should be replaced with a call to unregister_netdevice_notifier(). And also makes
the inclusion of net/netevent.h not necessary.
Best regards,
Razvan
^ permalink raw reply
* Re: [PATCH net-next 0/4] Introduce adaptive TX interrupt moderation to net DIM
From: Tal Gilboa @ 2018-03-30 6:43 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Tariq Toukan, Andy Gospodarek, Saeed Mahameed,
Florian Fainelli
In-Reply-To: <1522391853-37298-1-git-send-email-talgi@mellanox.com>
On 3/30/2018 9:37 AM, Tal Gilboa wrote:
> Net DIM is a library designed for dynamic interrupt moderation. It was
> implemented and optimized with receive side interrupts in mind, since these
> are usually the CPU expensive ones. This patch-set introduces adaptive transmit
> interrupt moderation to net DIM, complete with a usage in the mlx5e driver.
> Using adaptive TX behavior would reduce interrupt rate for multiple scenarios.
> Furthermore, it is essential for increasing bandwidth on cases where payload
> aggregation is required.
Important note: In order to avoid conflicts, this patch-set is rebased
over Florian Fainelli's "[PATCH net-next v2 3/3] net: bcmgenet: Fix
coalescing settings handling" patch.
>
> Tal Gilboa (4):
> net/dim: Rename *_get_profile() functions to *_get_rx_moderation()
> net/dim: Add "enabled" field to net_dim struct
> net/dim: Support adaptive TX moderation
> net/mlx5e: Enable adaptive-TX moderation
>
> drivers/net/ethernet/broadcom/bcmsysport.c | 2 +-
> drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c | 8 +--
> drivers/net/ethernet/broadcom/genet/bcmgenet.c | 2 +-
> drivers/net/ethernet/mellanox/mlx5/core/en.h | 5 +-
> drivers/net/ethernet/mellanox/mlx5/core/en_dim.c | 28 ++++++---
> .../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 35 +++++++----
> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 34 ++++++++--
> drivers/net/ethernet/mellanox/mlx5/core/en_rep.c | 2 +-
> drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 37 ++++++++---
> include/linux/net_dim.h | 72 +++++++++++++++++-----
> 10 files changed, 169 insertions(+), 56 deletions(-)
>
^ permalink raw reply
* [PATCH net-next 4/4] net/mlx5e: Enable adaptive-TX moderation
From: Tal Gilboa @ 2018-03-30 6:37 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Andy Gospodarek, Saeed Mahameed
In-Reply-To: <1522391853-37298-1-git-send-email-talgi@mellanox.com>
Add support for adaptive TX moderation. This greatly reduces TX interrupt
rate and increases bandwidth, mostly for TCP bandwidth over ARM
architecture (below). There is a slight single stream TCP with very large
message sizes degradation (x86). In this case if there's any moderation on
transmitted packets the bandwidth would reduce due to hitting TCP output
limit. Since this is a synthetic case, this is still worth doing.
Performance improvement (ConnectX-4Lx 40GbE, ARM)
TCP 64B bandwidth with 1-50 streams increased 6-35%.
TCP 64B bandwidth with 100-500 streams increased 20-70%.
Performance improvement (ConnectX-5 100GbE, x86)
Bandwidth: increased up to 40% (1024B with 10s of streams).
Interrupt rate: reduced up to 50% (1024B with 1000s of streams).
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 4 +++
drivers/net/ethernet/mellanox/mlx5/core/en_dim.c | 24 +++++++++++---
.../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 37 ++++++++++++++--------
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 22 +++++++++++++
drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 37 ++++++++++++++++------
5 files changed, 96 insertions(+), 28 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 3e24864..e18574d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -338,6 +338,7 @@ enum {
MLX5E_SQ_STATE_IPSEC,
MLX5E_SQ_STATE_TLS,
MLX5E_SQ_STATE_RECOVERING,
+ MLX5E_SQ_STATE_AM,
};
struct mlx5e_sq_wqe_info {
@@ -350,6 +351,7 @@ struct mlx5e_txqsq {
/* dirtied @completion */
u16 cc;
u32 dma_fifo_cc;
+ struct net_dim dim; /* Adaptive Moderation */
/* dirtied @xmit */
u16 pc ____cacheline_aligned_in_smp;
@@ -387,6 +389,7 @@ struct mlx5e_txqsq {
struct work_struct recover_work;
u64 last_recover;
} recover;
+
} ____cacheline_aligned_in_smp;
struct mlx5e_xdpsq {
@@ -1136,4 +1139,5 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev,
u16 max_channels);
u8 mlx5e_params_calculate_tx_min_inline(struct mlx5_core_dev *mdev);
void mlx5e_rx_dim_work(struct work_struct *work);
+void mlx5e_tx_dim_work(struct work_struct *work);
#endif /* __MLX5_EN_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
index 1b286e1..9cec351 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
@@ -33,16 +33,30 @@
#include <linux/net_dim.h>
#include "en.h"
+static inline void
+mlx5e_complete_dim_work(struct net_dim *dim, struct net_dim_cq_moder moder,
+ struct mlx5_core_dev *mdev, struct mlx5_core_cq *mcq)
+{
+ mlx5_core_modify_cq_moderation(mdev, mcq, moder.usec, moder.pkts);
+ dim->state = NET_DIM_START_MEASURE;
+}
+
void mlx5e_rx_dim_work(struct work_struct *work)
{
- struct net_dim *dim = container_of(work, struct net_dim,
- work);
+ struct net_dim *dim = container_of(work, struct net_dim, work);
struct mlx5e_rq *rq = container_of(dim, struct mlx5e_rq, dim);
struct net_dim_cq_moder cur_moder =
net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
- mlx5_core_modify_cq_moderation(rq->mdev, &rq->cq.mcq,
- cur_moder.usec, cur_moder.pkts);
+ mlx5e_complete_dim_work(dim, cur_moder, rq->mdev, &rq->cq.mcq);
+}
- dim->state = NET_DIM_START_MEASURE;
+void mlx5e_tx_dim_work(struct work_struct *work)
+{
+ struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct mlx5e_txqsq *sq = container_of(dim, struct mlx5e_txqsq, dim);
+ struct net_dim_cq_moder cur_moder =
+ net_dim_get_tx_moderation(dim->mode, dim->profile_ix);
+
+ mlx5e_complete_dim_work(dim, cur_moder, sq->cq.mdev, &sq->cq.mcq);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 4b5d022..05faaf7 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -454,15 +454,20 @@ static int mlx5e_set_channels(struct net_device *dev,
int mlx5e_ethtool_get_coalesce(struct mlx5e_priv *priv,
struct ethtool_coalesce *coal)
{
+ struct net_dim_cq_moder *rx_moder, *tx_moder;
+
if (!MLX5_CAP_GEN(priv->mdev, cq_moderation))
return -EOPNOTSUPP;
- coal->rx_coalesce_usecs = priv->channels.params.rx_cq_moderation.usec;
- coal->rx_max_coalesced_frames = priv->channels.params.rx_cq_moderation.pkts;
- coal->use_adaptive_rx_coalesce =
- priv->channels.params.rx_cq_moderation.enabled;
- coal->tx_coalesce_usecs = priv->channels.params.tx_cq_moderation.usec;
- coal->tx_max_coalesced_frames = priv->channels.params.tx_cq_moderation.pkts;
+ rx_moder = &priv->channels.params.rx_cq_moderation;
+ coal->rx_coalesce_usecs = rx_moder->usec;
+ coal->rx_max_coalesced_frames = rx_moder->pkts;
+ coal->use_adaptive_rx_coalesce = rx_moder->enabled;
+
+ tx_moder = &priv->channels.params.tx_cq_moderation;
+ coal->tx_coalesce_usecs = tx_moder->usec;
+ coal->tx_max_coalesced_frames = tx_moder->pkts;
+ coal->use_adaptive_tx_coalesce = tx_moder->enabled;
return 0;
}
@@ -501,6 +506,7 @@ static int mlx5e_get_coalesce(struct net_device *netdev,
int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
struct ethtool_coalesce *coal)
{
+ struct net_dim_cq_moder *rx_moder, *tx_moder;
struct mlx5_core_dev *mdev = priv->mdev;
struct mlx5e_channels new_channels = {};
int err = 0;
@@ -512,12 +518,15 @@ int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
mutex_lock(&priv->state_lock);
new_channels.params = priv->channels.params;
- new_channels.params.tx_cq_moderation.usec = coal->tx_coalesce_usecs;
- new_channels.params.tx_cq_moderation.pkts = coal->tx_max_coalesced_frames;
- new_channels.params.rx_cq_moderation.usec = coal->rx_coalesce_usecs;
- new_channels.params.rx_cq_moderation.pkts = coal->rx_max_coalesced_frames;
- new_channels.params.rx_cq_moderation.enabled =
- !!coal->use_adaptive_rx_coalesce;
+ rx_moder = &new_channels.params.rx_cq_moderation;
+ rx_moder->usec = coal->rx_coalesce_usecs;
+ rx_moder->pkts = coal->rx_max_coalesced_frames;
+ rx_moder->enabled = !!coal->use_adaptive_rx_coalesce;
+
+ tx_moder = &new_channels.params.tx_cq_moderation;
+ tx_moder->usec = coal->tx_coalesce_usecs;
+ tx_moder->pkts = coal->tx_max_coalesced_frames;
+ tx_moder->enabled = !!coal->use_adaptive_tx_coalesce;
if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) {
priv->channels.params = new_channels.params;
@@ -525,8 +534,8 @@ int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
}
/* we are opened */
- reset = !!coal->use_adaptive_rx_coalesce !=
- priv->channels.params.rx_cq_moderation.enabled;
+ reset = (!!rx_moder->enabled != priv->channels.params.rx_cq_moderation.enabled) ||
+ (!!tx_moder->enabled != priv->channels.params.tx_cq_moderation.enabled);
if (!reset) {
mlx5e_set_priv_channels_coalesce(priv, coal);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index e16a705..651f029 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -993,6 +993,9 @@ static int mlx5e_alloc_txqsq(struct mlx5e_channel *c,
if (err)
goto err_sq_wq_destroy;
+ INIT_WORK(&sq->dim.work, mlx5e_tx_dim_work);
+ sq->dim.mode = params->tx_cq_moderation.cq_period_mode;
+
sq->edge = (sq->wq.sz_m1 + 1) - MLX5_SEND_WQE_MAX_WQEBBS;
return 0;
@@ -1156,6 +1159,9 @@ static int mlx5e_open_txqsq(struct mlx5e_channel *c,
if (tx_rate)
mlx5e_set_sq_maxrate(c->netdev, sq, tx_rate);
+ if (params->tx_cq_moderation.enabled)
+ sq->state |= BIT(MLX5E_SQ_STATE_AM);
+
return 0;
err_free_txqsq:
@@ -4065,6 +4071,21 @@ void mlx5e_set_tx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode)
params->tx_cq_moderation.usec =
MLX5E_PARAMS_DEFAULT_TX_CQ_MODERATION_USEC_FROM_CQE;
+ if (params->tx_cq_moderation.enabled) {
+ switch (cq_period_mode) {
+ case MLX5_CQ_PERIOD_MODE_START_FROM_CQE:
+ params->tx_cq_moderation =
+ net_dim_get_def_tx_moderation(
+ NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE);
+ break;
+ case MLX5_CQ_PERIOD_MODE_START_FROM_EQE:
+ default:
+ params->tx_cq_moderation =
+ net_dim_get_def_tx_moderation(
+ NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE);
+ }
+ }
+
MLX5E_SET_PFLAG(params, MLX5E_PFLAG_TX_CQE_BASED_MODER,
params->tx_cq_moderation.cq_period_mode ==
MLX5_CQ_PERIOD_MODE_START_FROM_CQE);
@@ -4156,6 +4177,7 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev,
MLX5_CQ_PERIOD_MODE_START_FROM_CQE :
MLX5_CQ_PERIOD_MODE_START_FROM_EQE;
params->rx_cq_moderation.enabled = MLX5_CAP_GEN(mdev, cq_moderation);
+ params->tx_cq_moderation.enabled = params->rx_cq_moderation.enabled;
mlx5e_set_rx_cq_mode_params(params, cq_period_mode);
mlx5e_set_tx_cq_mode_params(params, cq_period_mode);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index f292bb3..ff1d5fe 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -44,6 +44,30 @@ static inline bool mlx5e_channel_no_affinity_change(struct mlx5e_channel *c)
return cpumask_test_cpu(current_cpu, aff);
}
+static inline void mlx5e_handle_tx_dim(struct mlx5e_txqsq *sq)
+{
+ struct net_dim_sample dim_sample;
+
+ if (unlikely(!MLX5E_TEST_BIT(sq->state, MLX5E_SQ_STATE_AM)))
+ return;
+
+ net_dim_sample(sq->cq.event_ctr, sq->stats.packets, sq->stats.bytes,
+ &dim_sample);
+ net_dim(&sq->dim, dim_sample);
+}
+
+static inline void mlx5e_handle_rx_dim(struct mlx5e_rq *rq)
+{
+ struct net_dim_sample dim_sample;
+
+ if (unlikely(!MLX5E_TEST_BIT(rq->state, MLX5E_RQ_STATE_AM)))
+ return;
+
+ net_dim_sample(rq->cq.event_ctr, rq->stats.packets, rq->stats.bytes,
+ &dim_sample);
+ net_dim(&rq->dim, dim_sample);
+}
+
int mlx5e_napi_poll(struct napi_struct *napi, int budget)
{
struct mlx5e_channel *c = container_of(napi, struct mlx5e_channel,
@@ -75,18 +99,13 @@ int mlx5e_napi_poll(struct napi_struct *napi, int budget)
if (unlikely(!napi_complete_done(napi, work_done)))
return work_done;
- for (i = 0; i < c->num_tc; i++)
+ for (i = 0; i < c->num_tc; i++) {
+ mlx5e_handle_tx_dim(&c->sq[i]);
mlx5e_cq_arm(&c->sq[i].cq);
-
- if (MLX5E_TEST_BIT(c->rq.state, MLX5E_RQ_STATE_AM)) {
- struct net_dim_sample dim_sample;
- net_dim_sample(c->rq.cq.event_ctr,
- c->rq.stats.packets,
- c->rq.stats.bytes,
- &dim_sample);
- net_dim(&c->rq.dim, dim_sample);
}
+ mlx5e_handle_rx_dim(&c->rq);
+
mlx5e_cq_arm(&c->rq.cq);
mlx5e_cq_arm(&c->icosq.cq);
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 3/4] net/dim: Support adaptive TX moderation
From: Tal Gilboa @ 2018-03-30 6:37 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Andy Gospodarek, Saeed Mahameed
In-Reply-To: <1522391853-37298-1-git-send-email-talgi@mellanox.com>
Interrupt moderation for TX traffic requires different profiles than RX
interrupt moderation. The main goal here is to reduce interrupt rate and
allow better payload aggregation by keeping SKBs in the TX queue a bit
longer. Ping-pong behavior would get a profile with a short timer, so
latency wouldn't increase for these scenarios. There's a slight degradation
in bandwidth for single stream with large message sizes, since
net.ipv4.tcp_limit_output_bytes is limiting the allowed TX traffic, but
with many streams performance is always improved.
Performance improvements (ConnectX-5 100GbE)
Bandwidth: increased up to 40% (1024B with 10s of streams).
Interrupt rate: reduced up to 50% (1024B with 1000s of streams).
Performance degradation (ConnectX-5 100GbE)
Bandwidth: up to 10% decrease single stream TCP (1MB message size from
51Gb/s to 47Gb/s).
*Both cases with TX EQE based moderation enabled.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
include/linux/net_dim.h | 64 +++++++++++++++++++++++++++++++++++++++----------
1 file changed, 51 insertions(+), 13 deletions(-)
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index d34fbfe..9449a61 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -104,11 +104,12 @@ enum {
#define NET_DIM_PARAMS_NUM_PROFILES 5
/* Adaptive moderation profiles */
#define NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE 256
+#define NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE 128
#define NET_DIM_DEF_PROFILE_CQE 1
#define NET_DIM_DEF_PROFILE_EQE 1
/* All profiles sizes must be NET_PARAMS_DIM_NUM_PROFILES */
-#define NET_DIM_EQE_PROFILES { \
+#define NET_DIM_RX_EQE_PROFILES { \
{1, NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE}, \
{8, NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE}, \
{64, NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE}, \
@@ -116,7 +117,7 @@ enum {
{256, NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE}, \
}
-#define NET_DIM_CQE_PROFILES { \
+#define NET_DIM_RX_CQE_PROFILES { \
{2, 256}, \
{8, 128}, \
{16, 64}, \
@@ -124,16 +125,38 @@ enum {
{64, 64} \
}
+#define NET_DIM_TX_EQE_PROFILES { \
+ {1, NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE}, \
+ {8, NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE}, \
+ {32, NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE}, \
+ {64, NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE}, \
+ {128, NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE} \
+}
+
+#define NET_DIM_TX_CQE_PROFILES { \
+ {5, 128}, \
+ {8, 64}, \
+ {16, 32}, \
+ {32, 32}, \
+ {64, 32} \
+}
+
+static const struct net_dim_cq_moder
+rx_profile[NET_DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
+ NET_DIM_RX_EQE_PROFILES,
+ NET_DIM_RX_CQE_PROFILES,
+};
+
static const struct net_dim_cq_moder
-profile[NET_DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
- NET_DIM_EQE_PROFILES,
- NET_DIM_CQE_PROFILES,
+tx_profile[NET_DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
+ NET_DIM_TX_EQE_PROFILES,
+ NET_DIM_TX_CQE_PROFILES,
};
static inline struct net_dim_cq_moder
net_dim_get_rx_moderation(u8 cq_period_mode, int ix)
{
- struct net_dim_cq_moder cq_moder = profile[cq_period_mode][ix];
+ struct net_dim_cq_moder cq_moder = rx_profile[cq_period_mode][ix];
cq_moder.cq_period_mode = cq_period_mode;
cq_moder.enabled = true;
@@ -141,16 +164,31 @@ enum {
}
static inline struct net_dim_cq_moder
-net_dim_get_def_rx_moderation(u8 rx_cq_period_mode)
+net_dim_get_def_rx_moderation(u8 cq_period_mode)
{
- int default_profile_ix;
+ u8 profile_ix = cq_period_mode == NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
+ NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
- if (rx_cq_period_mode == NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE)
- default_profile_ix = NET_DIM_DEF_PROFILE_CQE;
- else /* NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE */
- default_profile_ix = NET_DIM_DEF_PROFILE_EQE;
+ return net_dim_get_rx_moderation(cq_period_mode, profile_ix);
+}
+
+static inline struct net_dim_cq_moder
+net_dim_get_tx_moderation(u8 cq_period_mode, int ix)
+{
+ struct net_dim_cq_moder cq_moder = tx_profile[cq_period_mode][ix];
+
+ cq_moder.cq_period_mode = cq_period_mode;
+ cq_moder.enabled = true;
+ return cq_moder;
+}
+
+static inline struct net_dim_cq_moder
+net_dim_get_def_tx_moderation(u8 cq_period_mode)
+{
+ u8 profile_ix = cq_period_mode == NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
+ NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
- return net_dim_get_rx_moderation(rx_cq_period_mode, default_profile_ix);
+ return net_dim_get_tx_moderation(cq_period_mode, profile_ix);
}
static inline bool net_dim_on_top(struct net_dim *dim)
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 2/4] net/dim: Add "enabled" field to net_dim struct
From: Tal Gilboa @ 2018-03-30 6:37 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Andy Gospodarek, Saeed Mahameed
In-Reply-To: <1522391853-37298-1-git-send-email-talgi@mellanox.com>
Preparation for introducing adaptive TX to net DIM.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 -
drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 10 +++++++---
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 +++---
drivers/net/ethernet/mellanox/mlx5/core/en_rep.c | 2 +-
include/linux/net_dim.h | 2 ++
5 files changed, 13 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 0c2fdf0..3e24864 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -250,7 +250,6 @@ struct mlx5e_params {
u32 indirection_rqt[MLX5E_INDIR_RQT_SIZE];
bool vlan_strip_disable;
bool scatter_fcs_en;
- bool rx_dim_enabled;
u32 lro_timeout;
u32 pflags;
struct bpf_prog *xdp_prog;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index c57c929..4b5d022 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -459,9 +459,10 @@ int mlx5e_ethtool_get_coalesce(struct mlx5e_priv *priv,
coal->rx_coalesce_usecs = priv->channels.params.rx_cq_moderation.usec;
coal->rx_max_coalesced_frames = priv->channels.params.rx_cq_moderation.pkts;
+ coal->use_adaptive_rx_coalesce =
+ priv->channels.params.rx_cq_moderation.enabled;
coal->tx_coalesce_usecs = priv->channels.params.tx_cq_moderation.usec;
coal->tx_max_coalesced_frames = priv->channels.params.tx_cq_moderation.pkts;
- coal->use_adaptive_rx_coalesce = priv->channels.params.rx_dim_enabled;
return 0;
}
@@ -515,7 +516,8 @@ int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
new_channels.params.tx_cq_moderation.pkts = coal->tx_max_coalesced_frames;
new_channels.params.rx_cq_moderation.usec = coal->rx_coalesce_usecs;
new_channels.params.rx_cq_moderation.pkts = coal->rx_max_coalesced_frames;
- new_channels.params.rx_dim_enabled = !!coal->use_adaptive_rx_coalesce;
+ new_channels.params.rx_cq_moderation.enabled =
+ !!coal->use_adaptive_rx_coalesce;
if (!test_bit(MLX5E_STATE_OPENED, &priv->state)) {
priv->channels.params = new_channels.params;
@@ -523,7 +525,9 @@ int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
}
/* we are opened */
- reset = !!coal->use_adaptive_rx_coalesce != priv->channels.params.rx_dim_enabled;
+ reset = !!coal->use_adaptive_rx_coalesce !=
+ priv->channels.params.rx_cq_moderation.enabled;
+
if (!reset) {
mlx5e_set_priv_channels_coalesce(priv, coal);
priv->channels.params = new_channels.params;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 8a9670c..e16a705 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -781,7 +781,7 @@ static int mlx5e_open_rq(struct mlx5e_channel *c,
if (err)
goto err_destroy_rq;
- if (params->rx_dim_enabled)
+ if (params->rx_cq_moderation.enabled)
c->rq.state |= BIT(MLX5E_RQ_STATE_AM);
return 0;
@@ -4083,7 +4083,7 @@ void mlx5e_set_rx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode)
params->rx_cq_moderation.usec =
MLX5E_PARAMS_DEFAULT_RX_CQ_MODERATION_USEC_FROM_CQE;
- if (params->rx_dim_enabled) {
+ if (params->rx_cq_moderation.enabled) {
switch (cq_period_mode) {
case MLX5_CQ_PERIOD_MODE_START_FROM_CQE:
params->rx_cq_moderation =
@@ -4155,7 +4155,7 @@ void mlx5e_build_nic_params(struct mlx5_core_dev *mdev,
cq_period_mode = MLX5_CAP_GEN(mdev, cq_period_start_from_cqe) ?
MLX5_CQ_PERIOD_MODE_START_FROM_CQE :
MLX5_CQ_PERIOD_MODE_START_FROM_EQE;
- params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation);
+ params->rx_cq_moderation.enabled = MLX5_CAP_GEN(mdev, cq_moderation);
mlx5e_set_rx_cq_mode_params(params, cq_period_mode);
mlx5e_set_tx_cq_mode_params(params, cq_period_mode);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
index dd32f3e..f238d4c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rep.c
@@ -881,7 +881,7 @@ static void mlx5e_build_rep_params(struct mlx5_core_dev *mdev,
params->rq_wq_type = MLX5_WQ_TYPE_LINKED_LIST;
params->log_rq_size = MLX5E_PARAMS_MINIMUM_LOG_RQ_SIZE;
- params->rx_dim_enabled = MLX5_CAP_GEN(mdev, cq_moderation);
+ params->rx_cq_moderation.enabled = MLX5_CAP_GEN(mdev, cq_moderation);
mlx5e_set_rx_cq_mode_params(params, cq_period_mode);
params->num_tc = 1;
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index e1cabaf..d34fbfe 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -40,6 +40,7 @@ struct net_dim_cq_moder {
u16 usec;
u16 pkts;
u8 cq_period_mode;
+ bool enabled;
};
struct net_dim_sample {
@@ -135,6 +136,7 @@ enum {
struct net_dim_cq_moder cq_moder = profile[cq_period_mode][ix];
cq_moder.cq_period_mode = cq_period_mode;
+ cq_moder.enabled = true;
return cq_moder;
}
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 1/4] net/dim: Rename *_get_profile() functions to *_get_rx_moderation()
From: Tal Gilboa @ 2018-03-30 6:37 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Andy Gospodarek, Saeed Mahameed
In-Reply-To: <1522391853-37298-1-git-send-email-talgi@mellanox.com>
Preparation for introducing adaptive TX to net DIM.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/net/ethernet/broadcom/bcmsysport.c | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c | 8 ++++----
drivers/net/ethernet/broadcom/genet/bcmgenet.c | 2 +-
drivers/net/ethernet/mellanox/mlx5/core/en_dim.c | 6 +++---
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 ++++--
include/linux/net_dim.h | 12 ++++++------
6 files changed, 19 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index 4a75b1d..53c035f 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -1064,7 +1064,7 @@ static void bcm_sysport_dim_work(struct work_struct *work)
struct bcm_sysport_priv *priv =
container_of(ndim, struct bcm_sysport_priv, dim);
struct net_dim_cq_moder cur_profile =
- net_dim_get_profile(dim->mode, dim->profile_ix);
+ net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
bcm_sysport_set_rx_coalesce(priv, cur_profile.usec, cur_profile.pkts);
dim->state = NET_DIM_START_MEASURE;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
index 408dd19..afa97c8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
@@ -21,11 +21,11 @@ void bnxt_dim_work(struct work_struct *work)
struct bnxt_napi *bnapi = container_of(cpr,
struct bnxt_napi,
cp_ring);
- struct net_dim_cq_moder cur_profile = net_dim_get_profile(dim->mode,
- dim->profile_ix);
+ struct net_dim_cq_moder cur_moder =
+ net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
- cpr->rx_ring_coal.coal_ticks = cur_profile.usec;
- cpr->rx_ring_coal.coal_bufs = cur_profile.pkts;
+ cpr->rx_ring_coal.coal_ticks = cur_moder.usec;
+ cpr->rx_ring_coal.coal_bufs = cur_moder.pkts;
bnxt_hwrm_set_ring_coal(bnapi->bp, bnapi);
dim->state = NET_DIM_START_MEASURE;
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index f8af472..1c9c75b 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -1924,7 +1924,7 @@ static void bcmgenet_dim_work(struct work_struct *work)
struct bcmgenet_rx_ring *ring =
container_of(ndim, struct bcmgenet_rx_ring, dim);
struct net_dim_cq_moder cur_profile =
- net_dim_get_profile(dim->mode, dim->profile_ix);
+ net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
bcmgenet_set_rx_coalesce(ring, cur_profile.usec, cur_profile.pkts);
dim->state = NET_DIM_START_MEASURE;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
index 602851a..1b286e1 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
@@ -38,11 +38,11 @@ void mlx5e_rx_dim_work(struct work_struct *work)
struct net_dim *dim = container_of(work, struct net_dim,
work);
struct mlx5e_rq *rq = container_of(dim, struct mlx5e_rq, dim);
- struct net_dim_cq_moder cur_profile = net_dim_get_profile(dim->mode,
- dim->profile_ix);
+ struct net_dim_cq_moder cur_moder =
+ net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
mlx5_core_modify_cq_moderation(rq->mdev, &rq->cq.mcq,
- cur_profile.usec, cur_profile.pkts);
+ cur_moder.usec, cur_moder.pkts);
dim->state = NET_DIM_START_MEASURE;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 4a675f1..8a9670c 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -4087,12 +4087,14 @@ void mlx5e_set_rx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode)
switch (cq_period_mode) {
case MLX5_CQ_PERIOD_MODE_START_FROM_CQE:
params->rx_cq_moderation =
- net_dim_get_def_profile(NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE);
+ net_dim_get_def_rx_moderation(
+ NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE);
break;
case MLX5_CQ_PERIOD_MODE_START_FROM_EQE:
default:
params->rx_cq_moderation =
- net_dim_get_def_profile(NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE);
+ net_dim_get_def_rx_moderation(
+ NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE);
}
}
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index bebeaad..e1cabaf 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -129,17 +129,17 @@ enum {
NET_DIM_CQE_PROFILES,
};
-static inline struct net_dim_cq_moder net_dim_get_profile(u8 cq_period_mode,
- int ix)
+static inline struct net_dim_cq_moder
+net_dim_get_rx_moderation(u8 cq_period_mode, int ix)
{
- struct net_dim_cq_moder cq_moder;
+ struct net_dim_cq_moder cq_moder = profile[cq_period_mode][ix];
- cq_moder = profile[cq_period_mode][ix];
cq_moder.cq_period_mode = cq_period_mode;
return cq_moder;
}
-static inline struct net_dim_cq_moder net_dim_get_def_profile(u8 rx_cq_period_mode)
+static inline struct net_dim_cq_moder
+net_dim_get_def_rx_moderation(u8 rx_cq_period_mode)
{
int default_profile_ix;
@@ -148,7 +148,7 @@ static inline struct net_dim_cq_moder net_dim_get_def_profile(u8 rx_cq_period_mo
else /* NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE */
default_profile_ix = NET_DIM_DEF_PROFILE_EQE;
- return net_dim_get_profile(rx_cq_period_mode, default_profile_ix);
+ return net_dim_get_rx_moderation(rx_cq_period_mode, default_profile_ix);
}
static inline bool net_dim_on_top(struct net_dim *dim)
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 0/4] Introduce adaptive TX interrupt moderation to net DIM
From: Tal Gilboa @ 2018-03-30 6:37 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Andy Gospodarek, Saeed Mahameed
Net DIM is a library designed for dynamic interrupt moderation. It was
implemented and optimized with receive side interrupts in mind, since these
are usually the CPU expensive ones. This patch-set introduces adaptive transmit
interrupt moderation to net DIM, complete with a usage in the mlx5e driver.
Using adaptive TX behavior would reduce interrupt rate for multiple scenarios.
Furthermore, it is essential for increasing bandwidth on cases where payload
aggregation is required.
Tal Gilboa (4):
net/dim: Rename *_get_profile() functions to *_get_rx_moderation()
net/dim: Add "enabled" field to net_dim struct
net/dim: Support adaptive TX moderation
net/mlx5e: Enable adaptive-TX moderation
drivers/net/ethernet/broadcom/bcmsysport.c | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c | 8 +--
drivers/net/ethernet/broadcom/genet/bcmgenet.c | 2 +-
drivers/net/ethernet/mellanox/mlx5/core/en.h | 5 +-
drivers/net/ethernet/mellanox/mlx5/core/en_dim.c | 28 ++++++---
.../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 35 +++++++----
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 34 ++++++++--
drivers/net/ethernet/mellanox/mlx5/core/en_rep.c | 2 +-
drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 37 ++++++++---
include/linux/net_dim.h | 72 +++++++++++++++++-----
10 files changed, 169 insertions(+), 56 deletions(-)
--
1.8.3.1
^ permalink raw reply
* Re: [PATCH net-next 0/9] devlink: Add support for region access
From: Alex Vesker @ 2018-03-30 5:28 UTC (permalink / raw)
To: Andrew Lunn; +Cc: David S. Miller, netdev, Tariq Toukan, Jiri Pirko
In-Reply-To: <20180329195154.GB15565@lunn.ch>
On 3/29/2018 10:51 PM, Andrew Lunn wrote:
>>>> Show all of the exposed regions with region sizes:
>>>> $ devlink region show
>>>> pci/0000:00:05.0/cr-space: size 1048576 snapshot [1 2]
>>> So you have 2Mbytes of snapshot data. Is this held in the device, or
>>> kernel memory?
>> This is allocated in devlink, the maximum number of snapshots is set by the
>> driver.
> And it seems to want contiguous pages. How well does that work after
> the system has been running for a while and memory is fragmented?
The allocation can be changed, there is no read need for contiguous pages.
It is important to note that we the amount of snapshots is limited by
the driver
this can be based on the dump size or expected frequency of collection.
I also prefer not to pre-allocate this memory.
>>>> Dump a snapshot:
>>>> $ devlink region dump pci/0000:00:05.0/fw-health snapshot 1
>>>> 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
>>>> 0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
>>>> 0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
>>>> 0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
>>>>
>>>> Read a specific part of a snapshot:
>>>> $ devlink region read pci/0000:00:05.0/fw-health snapshot 1 address 0
>>>> length 16
>>>> 0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
>>> Why a separate command? It seems to be just a subset of dump.
>> This is useful when debugging values on specific addresses, this also
>> brings the API one step closer for a read and write API.
> The functionality is useful, yes. But why two commands? Why not one
> command, dump, which takes optional parameters?
Dump in devlink means provide all the data, saying dump address x length
y sounds
confusing. Do you see this as a critical issue?
> Also, i doubt write support will be accepted. That sounds like the
> start of an API to allow a user space driver.
If this will be an issue we will stay with read access only.
>
> Andrew
^ permalink raw reply
* [PATCH net-next 6/6] inet: frags: break the 2GB limit for frags storage
From: Eric Dumazet @ 2018-03-30 5:22 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Florian Westphal, Herbert Xu, Thomas Graf,
Jesper Dangaard Brouer, Alexander Aring, Stefan Schmidt,
Eric Dumazet, Eric Dumazet
In-Reply-To: <20180330052241.206667-2-edumazet@google.com>
Some users are willing to provision huge amounts of memory to be able
to perform reassembly reasonnably well under pressure.
Current memory tracking is using one atomic_t and integers.
Switch to atomic_long_t so that 64bit arches can use more than 2GB,
without any cost for 32bit arches.
Tested:
$ echo 16000000000 >/proc/sys/net/ipv4/ipfrag_high_thresh
<frag DDOS>
$ grep FRAG /proc/net/sockstat
FRAG: inuse 14705885 memory 16000002880
$ nstat -n ; sleep 1 ; nstat | grep Reas
IpReasmReqds 3317150 0.0
IpReasmFails 3317112 0.0
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
Documentation/networking/ip-sysctl.txt | 4 ++--
include/net/inet_frag.h | 20 ++++++++++----------
net/ieee802154/6lowpan/reassembly.c | 10 +++++-----
net/ipv4/ip_fragment.c | 10 +++++-----
net/ipv4/proc.c | 2 +-
net/ipv6/netfilter/nf_conntrack_reasm.c | 10 +++++-----
net/ipv6/proc.c | 2 +-
net/ipv6/reassembly.c | 6 +++---
8 files changed, 32 insertions(+), 32 deletions(-)
diff --git a/Documentation/networking/ip-sysctl.txt b/Documentation/networking/ip-sysctl.txt
index c3b65f24e58aa72b720861d816fb76f9956800f0..1c18aa6606b771fb6afb95e55fb880b2484775dd 100644
--- a/Documentation/networking/ip-sysctl.txt
+++ b/Documentation/networking/ip-sysctl.txt
@@ -133,10 +133,10 @@ min_adv_mss - INTEGER
IP Fragmentation:
-ipfrag_high_thresh - INTEGER
+ipfrag_high_thresh - LONG INTEGER
Maximum memory used to reassemble IP fragments.
-ipfrag_low_thresh - INTEGER
+ipfrag_low_thresh - LONG INTEGER
(Obsolete since linux-4.17)
Maximum memory used to reassemble IP fragments before the kernel
begins to remove incomplete fragment queues to free up resources.
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index fae84c10679c012bddc4367bcd0d44e34bd51372..b68f5d4338b63701587deb5ba0e48a324920c332 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -6,14 +6,14 @@
struct netns_frags {
/* sysctls */
+ long high_thresh;
+ long low_thresh;
int timeout;
- int high_thresh;
- int low_thresh;
int max_dist;
struct inet_frags *f;
/* Keep atomic mem on separate cachelines in structs that include it */
- atomic_t mem ____cacheline_aligned_in_smp;
+ atomic_long_t mem ____cacheline_aligned_in_smp;
struct rhashtable rhashtable ____cacheline_aligned_in_smp;
};
@@ -103,7 +103,7 @@ void inet_frags_fini(struct inet_frags *);
static inline int inet_frags_init_net(struct netns_frags *nf)
{
- atomic_set(&nf->mem, 0);
+ atomic_long_set(&nf->mem, 0);
return rhashtable_init(&nf->rhashtable, &nf->f->rhash_params);
}
void inet_frags_exit_net(struct netns_frags *nf);
@@ -120,19 +120,19 @@ static inline void inet_frag_put(struct inet_frag_queue *q)
/* Memory Tracking Functions. */
-static inline int frag_mem_limit(struct netns_frags *nf)
+static inline long frag_mem_limit(const struct netns_frags *nf)
{
- return atomic_read(&nf->mem);
+ return atomic_long_read(&nf->mem);
}
-static inline void sub_frag_mem_limit(struct netns_frags *nf, int i)
+static inline void sub_frag_mem_limit(struct netns_frags *nf, long val)
{
- atomic_sub(i, &nf->mem);
+ atomic_long_sub(val, &nf->mem);
}
-static inline void add_frag_mem_limit(struct netns_frags *nf, int i)
+static inline void add_frag_mem_limit(struct netns_frags *nf, long val)
{
- atomic_add(i, &nf->mem);
+ atomic_long_add(val, &nf->mem);
}
/* RFC 3168 support :
diff --git a/net/ieee802154/6lowpan/reassembly.c b/net/ieee802154/6lowpan/reassembly.c
index 9ee4d22666c26d6d9796d0f484bb4beb265dea42..d7125507c065cd23501f29664892d24c36050ea7 100644
--- a/net/ieee802154/6lowpan/reassembly.c
+++ b/net/ieee802154/6lowpan/reassembly.c
@@ -411,23 +411,23 @@ int lowpan_frag_rcv(struct sk_buff *skb, u8 frag_type)
}
#ifdef CONFIG_SYSCTL
-static int zero;
+static long zero;
static struct ctl_table lowpan_frags_ns_ctl_table[] = {
{
.procname = "6lowpanfrag_high_thresh",
.data = &init_net.ieee802154_lowpan.frags.high_thresh,
- .maxlen = sizeof(int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
+ .proc_handler = proc_doulongvec_minmax,
.extra1 = &init_net.ieee802154_lowpan.frags.low_thresh
},
{
.procname = "6lowpanfrag_low_thresh",
.data = &init_net.ieee802154_lowpan.frags.low_thresh,
- .maxlen = sizeof(int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
+ .proc_handler = proc_doulongvec_minmax,
.extra1 = &zero,
.extra2 = &init_net.ieee802154_lowpan.frags.high_thresh
},
diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c
index ef864db73613b491fb430ff3b594c7286705a1b3..c1516e4a6a02a62972c2aa5d7a61ec3c26c2b73a 100644
--- a/net/ipv4/ip_fragment.c
+++ b/net/ipv4/ip_fragment.c
@@ -670,23 +670,23 @@ struct sk_buff *ip_check_defrag(struct net *net, struct sk_buff *skb, u32 user)
EXPORT_SYMBOL(ip_check_defrag);
#ifdef CONFIG_SYSCTL
-static int zero;
+static long zero;
static struct ctl_table ip4_frags_ns_ctl_table[] = {
{
.procname = "ipfrag_high_thresh",
.data = &init_net.ipv4.frags.high_thresh,
- .maxlen = sizeof(int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
+ .proc_handler = proc_doulongvec_minmax,
.extra1 = &init_net.ipv4.frags.low_thresh
},
{
.procname = "ipfrag_low_thresh",
.data = &init_net.ipv4.frags.low_thresh,
- .maxlen = sizeof(int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
+ .proc_handler = proc_doulongvec_minmax,
.extra1 = &zero,
.extra2 = &init_net.ipv4.frags.high_thresh
},
diff --git a/net/ipv4/proc.c b/net/ipv4/proc.c
index aacfce0d7d82cf59269a69ef4d6ac8d9955b0bdc..a058de677e947846eb93020e0788148827c8f3cd 100644
--- a/net/ipv4/proc.c
+++ b/net/ipv4/proc.c
@@ -71,7 +71,7 @@ static int sockstat_seq_show(struct seq_file *seq, void *v)
sock_prot_inuse_get(net, &udplite_prot));
seq_printf(seq, "RAW: inuse %d\n",
sock_prot_inuse_get(net, &raw_prot));
- seq_printf(seq, "FRAG: inuse %u memory %u\n",
+ seq_printf(seq, "FRAG: inuse %u memory %lu\n",
atomic_read(&net->ipv4.frags.rhashtable.nelems),
frag_mem_limit(&net->ipv4.frags));
return 0;
diff --git a/net/ipv6/netfilter/nf_conntrack_reasm.c b/net/ipv6/netfilter/nf_conntrack_reasm.c
index 53859311dea96c03fa5ae8456de32de25009efbe..14310372da06b956eefd6dfcbad9168e928b00e5 100644
--- a/net/ipv6/netfilter/nf_conntrack_reasm.c
+++ b/net/ipv6/netfilter/nf_conntrack_reasm.c
@@ -63,7 +63,7 @@ struct nf_ct_frag6_skb_cb
static struct inet_frags nf_frags;
#ifdef CONFIG_SYSCTL
-static int zero;
+static long zero;
static struct ctl_table nf_ct_frag6_sysctl_table[] = {
{
@@ -76,18 +76,18 @@ static struct ctl_table nf_ct_frag6_sysctl_table[] = {
{
.procname = "nf_conntrack_frag6_low_thresh",
.data = &init_net.nf_frag.frags.low_thresh,
- .maxlen = sizeof(unsigned int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
+ .proc_handler = proc_doulongvec_minmax,
.extra1 = &zero,
.extra2 = &init_net.nf_frag.frags.high_thresh
},
{
.procname = "nf_conntrack_frag6_high_thresh",
.data = &init_net.nf_frag.frags.high_thresh,
- .maxlen = sizeof(unsigned int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
+ .proc_handler = proc_doulongvec_minmax,
.extra1 = &init_net.nf_frag.frags.low_thresh
},
{ }
diff --git a/net/ipv6/proc.c b/net/ipv6/proc.c
index 8befeb91e0712ecc4d05c4c0a6ecca1808dcbcac..a85f7e0b14b10f59fdd2ea6901f8e9a95c13654f 100644
--- a/net/ipv6/proc.c
+++ b/net/ipv6/proc.c
@@ -47,7 +47,7 @@ static int sockstat6_seq_show(struct seq_file *seq, void *v)
sock_prot_inuse_get(net, &udplitev6_prot));
seq_printf(seq, "RAW6: inuse %d\n",
sock_prot_inuse_get(net, &rawv6_prot));
- seq_printf(seq, "FRAG6: inuse %u memory %u\n",
+ seq_printf(seq, "FRAG6: inuse %u memory %lu\n",
atomic_read(&net->ipv6.frags.rhashtable.nelems),
frag_mem_limit(&net->ipv6.frags));
return 0;
diff --git a/net/ipv6/reassembly.c b/net/ipv6/reassembly.c
index 737b0921ab0c9af198fefdf06d8f4ede91c7f3f6..bedbded6433b78e603a5b48d6ea493d6d90291a8 100644
--- a/net/ipv6/reassembly.c
+++ b/net/ipv6/reassembly.c
@@ -552,15 +552,15 @@ static struct ctl_table ip6_frags_ns_ctl_table[] = {
{
.procname = "ip6frag_high_thresh",
.data = &init_net.ipv6.frags.high_thresh,
- .maxlen = sizeof(int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
- .proc_handler = proc_dointvec_minmax,
+ .proc_handler = proc_doulongvec_minmax,
.extra1 = &init_net.ipv6.frags.low_thresh
},
{
.procname = "ip6frag_low_thresh",
.data = &init_net.ipv6.frags.low_thresh,
- .maxlen = sizeof(int),
+ .maxlen = sizeof(unsigned long),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
.extra1 = &zero,
--
2.17.0.rc1.321.gba9d0f2565-goog
^ permalink raw reply related
* [PATCH net-next 5/6] inet: frags: remove some helpers
From: Eric Dumazet @ 2018-03-30 5:22 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Florian Westphal, Herbert Xu, Thomas Graf,
Jesper Dangaard Brouer, Alexander Aring, Stefan Schmidt,
Eric Dumazet, Eric Dumazet
In-Reply-To: <20180330052241.206667-2-edumazet@google.com>
Remove sum_frag_mem_limit(), ip_frag_mem() & ip6_frag_mem()
Also since we use rhashtable we can bring back the number of fragments
in "grep FRAG /proc/net/sockstat /proc/net/sockstat6" that was
removed in commit 434d305405ab ("inet: frag: don't account number
of fragment queues")
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
include/net/inet_frag.h | 5 -----
include/net/ip.h | 1 -
include/net/ipv6.h | 7 -------
net/ipv4/ip_fragment.c | 5 -----
net/ipv4/proc.c | 6 +++---
net/ipv6/proc.c | 5 +++--
6 files changed, 6 insertions(+), 23 deletions(-)
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index 05099f9f980e2384c0c8cd7e74659656b585cd22..fae84c10679c012bddc4367bcd0d44e34bd51372 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -135,11 +135,6 @@ static inline void add_frag_mem_limit(struct netns_frags *nf, int i)
atomic_add(i, &nf->mem);
}
-static inline int sum_frag_mem_limit(struct netns_frags *nf)
-{
- return atomic_read(&nf->mem);
-}
-
/* RFC 3168 support :
* We want to check ECN values of all fragments, do detect invalid combinations.
* In ipq->ecn, we store the OR value of each ip4_frag_ecn() fragment value.
diff --git a/include/net/ip.h b/include/net/ip.h
index 36f8f7811093c37de06194dc7410b7596f8bf9fa..ecffd843e7b896a83416847fdaa452be6223f3dc 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -588,7 +588,6 @@ static inline struct sk_buff *ip_check_defrag(struct net *net, struct sk_buff *s
return skb;
}
#endif
-int ip_frag_mem(struct net *net);
/*
* Functions provided by ip_forward.c
diff --git a/include/net/ipv6.h b/include/net/ipv6.h
index 76f84d4be91b92761fb9a26e7f52e2101ee34c0a..abbed2687fbd61cb47e2b6d0164ab6cf4d40a618 100644
--- a/include/net/ipv6.h
+++ b/include/net/ipv6.h
@@ -379,13 +379,6 @@ static inline bool ipv6_accept_ra(struct inet6_dev *idev)
idev->cnf.accept_ra;
}
-#if IS_ENABLED(CONFIG_IPV6)
-static inline int ip6_frag_mem(struct net *net)
-{
- return sum_frag_mem_limit(&net->ipv6.frags);
-}
-#endif
-
#define IPV6_FRAG_HIGH_THRESH (4 * 1024*1024) /* 4194304 */
#define IPV6_FRAG_LOW_THRESH (3 * 1024*1024) /* 3145728 */
#define IPV6_FRAG_TIMEOUT (60 * HZ) /* 60 seconds */
diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c
index 1a7423e8ec0a8f88782ad8c945dc0cd6046f79f0..ef864db73613b491fb430ff3b594c7286705a1b3 100644
--- a/net/ipv4/ip_fragment.c
+++ b/net/ipv4/ip_fragment.c
@@ -83,11 +83,6 @@ static u8 ip4_frag_ecn(u8 tos)
static struct inet_frags ip4_frags;
-int ip_frag_mem(struct net *net)
-{
- return sum_frag_mem_limit(&net->ipv4.frags);
-}
-
static int ip_frag_reasm(struct ipq *qp, struct sk_buff *prev,
struct net_device *dev);
diff --git a/net/ipv4/proc.c b/net/ipv4/proc.c
index adfb75340275d240487574257c10feb295df44fe..aacfce0d7d82cf59269a69ef4d6ac8d9955b0bdc 100644
--- a/net/ipv4/proc.c
+++ b/net/ipv4/proc.c
@@ -54,7 +54,6 @@
static int sockstat_seq_show(struct seq_file *seq, void *v)
{
struct net *net = seq->private;
- unsigned int frag_mem;
int orphans, sockets;
orphans = percpu_counter_sum_positive(&tcp_orphan_count);
@@ -72,8 +71,9 @@ static int sockstat_seq_show(struct seq_file *seq, void *v)
sock_prot_inuse_get(net, &udplite_prot));
seq_printf(seq, "RAW: inuse %d\n",
sock_prot_inuse_get(net, &raw_prot));
- frag_mem = ip_frag_mem(net);
- seq_printf(seq, "FRAG: inuse %u memory %u\n", !!frag_mem, frag_mem);
+ seq_printf(seq, "FRAG: inuse %u memory %u\n",
+ atomic_read(&net->ipv4.frags.rhashtable.nelems),
+ frag_mem_limit(&net->ipv4.frags));
return 0;
}
diff --git a/net/ipv6/proc.c b/net/ipv6/proc.c
index 6e57028d2e9160be264d07f9312658fcb677a568..8befeb91e0712ecc4d05c4c0a6ecca1808dcbcac 100644
--- a/net/ipv6/proc.c
+++ b/net/ipv6/proc.c
@@ -38,7 +38,6 @@
static int sockstat6_seq_show(struct seq_file *seq, void *v)
{
struct net *net = seq->private;
- unsigned int frag_mem = ip6_frag_mem(net);
seq_printf(seq, "TCP6: inuse %d\n",
sock_prot_inuse_get(net, &tcpv6_prot));
@@ -48,7 +47,9 @@ static int sockstat6_seq_show(struct seq_file *seq, void *v)
sock_prot_inuse_get(net, &udplitev6_prot));
seq_printf(seq, "RAW6: inuse %d\n",
sock_prot_inuse_get(net, &rawv6_prot));
- seq_printf(seq, "FRAG6: inuse %u memory %u\n", !!frag_mem, frag_mem);
+ seq_printf(seq, "FRAG6: inuse %u memory %u\n",
+ atomic_read(&net->ipv6.frags.rhashtable.nelems),
+ frag_mem_limit(&net->ipv6.frags));
return 0;
}
--
2.17.0.rc1.321.gba9d0f2565-goog
^ permalink raw reply related
* [PATCH net-next 4/6] inet: frags: use rhashtables for reassembly units
From: Eric Dumazet @ 2018-03-30 5:22 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Florian Westphal, Herbert Xu, Thomas Graf,
Jesper Dangaard Brouer, Alexander Aring, Stefan Schmidt,
Eric Dumazet, Eric Dumazet, Nikolay Aleksandrov
In-Reply-To: <20180330052241.206667-2-edumazet@google.com>
Some applications still rely on IP fragmentation, and to be fair linux
reassembly unit is not working under any serious load.
It uses static hash tables of 1024 buckets, and up to 128 items per bucket (!!!)
A work queue is supposed to garbage collect items when host is under memory
pressure, and doing a hash rebuild, changing seed used in hash computations.
This work queue blocks softirqs for up to 25 ms when doing a hash rebuild,
occurring every 5 seconds if host is under fire.
Then there is the problem of sharing this hash table for all netns.
It is time to switch to rhashtables, and allocate one of them per netns
to speedup netns dismantle, since this is a critical metric these days.
Lookup is now using RCU. A followup patch will even remove
the refcount hold/release left from prior implementation and save
a couple of atomic operations.
Before this patch, 16 cpus (16 RX queue NIC) could not handle more
than 1 Mpps frags DDOS.
After the patch, I reach 7 Mpps without any tuning, and can use up to 2GB
of storage for the fragments.
$ grep FRAG /proc/net/sockstat
FRAG: inuse 1966916 memory 2140004608
A followup patch will change the limits for 64bit arches.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Florian Westphal <fw@strlen.de>
Cc: Nikolay Aleksandrov <nikolay@redhat.com>
Cc: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: Alexander Aring <alex.aring@gmail.com>
Cc: Stefan Schmidt <stefan@osg.samsung.com>
---
Documentation/networking/ip-sysctl.txt | 7 +-
include/net/inet_frag.h | 99 +++---
include/net/ipv6.h | 20 +-
net/ieee802154/6lowpan/6lowpan_i.h | 26 +-
net/ieee802154/6lowpan/reassembly.c | 108 +++----
net/ipv4/inet_fragment.c | 399 +++++-------------------
net/ipv4/ip_fragment.c | 165 +++++-----
net/ipv6/netfilter/nf_conntrack_reasm.c | 62 ++--
net/ipv6/reassembly.c | 152 +++++----
9 files changed, 344 insertions(+), 694 deletions(-)
diff --git a/Documentation/networking/ip-sysctl.txt b/Documentation/networking/ip-sysctl.txt
index 1d1120753ae82d0aee3e934a3d9c074b70dcbca6..c3b65f24e58aa72b720861d816fb76f9956800f0 100644
--- a/Documentation/networking/ip-sysctl.txt
+++ b/Documentation/networking/ip-sysctl.txt
@@ -134,13 +134,10 @@ min_adv_mss - INTEGER
IP Fragmentation:
ipfrag_high_thresh - INTEGER
- Maximum memory used to reassemble IP fragments. When
- ipfrag_high_thresh bytes of memory is allocated for this purpose,
- the fragment handler will toss packets until ipfrag_low_thresh
- is reached. This also serves as a maximum limit to namespaces
- different from the initial one.
+ Maximum memory used to reassemble IP fragments.
ipfrag_low_thresh - INTEGER
+ (Obsolete since linux-4.17)
Maximum memory used to reassemble IP fragments before the kernel
begins to remove incomplete fragment queues to free up resources.
The kernel still accepts new fragments for defragmentation.
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index 69e531ed81894393e07cac9e953825fcb55ef42a..05099f9f980e2384c0c8cd7e74659656b585cd22 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -2,15 +2,20 @@
#ifndef __NET_FRAG_H__
#define __NET_FRAG_H__
+#include <linux/rhashtable.h>
+
struct netns_frags {
- /* Keep atomic mem on separate cachelines in structs that include it */
- atomic_t mem ____cacheline_aligned_in_smp;
/* sysctls */
int timeout;
int high_thresh;
int low_thresh;
int max_dist;
struct inet_frags *f;
+
+ /* Keep atomic mem on separate cachelines in structs that include it */
+ atomic_t mem ____cacheline_aligned_in_smp;
+
+ struct rhashtable rhashtable ____cacheline_aligned_in_smp;
};
/**
@@ -26,12 +31,31 @@ enum {
INET_FRAG_COMPLETE = BIT(2),
};
+struct frag_v4_compare_key {
+ __be32 saddr;
+ __be32 daddr;
+ u32 user;
+ u32 vif;
+ __be16 id;
+ u16 protocol;
+};
+
+struct frag_v6_compare_key {
+ struct in6_addr saddr;
+ struct in6_addr daddr;
+ u32 user;
+ __be32 id;
+ u32 iif;
+};
+
/**
* struct inet_frag_queue - fragment queue
*
- * @lock: spinlock protecting the queue
+ * @node: rhash node
+ * @key: keys identifying this frag.
* @timer: queue expiration timer
- * @list: hash bucket list
+ * @net: namespace that this frag belongs to
+ * @lock: spinlock protecting this frag
* @refcnt: reference count of the queue
* @fragments: received fragments head
* @fragments_tail: received fragments tail
@@ -40,66 +64,38 @@ enum {
* @meat: length of received fragments so far
* @flags: fragment queue flags
* @max_size: maximum received fragment size
- * @net: namespace that this frag belongs to
- * @list_evictor: list of queues to forcefully evict (e.g. due to low memory)
+ * @rcu: rcu head for freeing deferall
*/
struct inet_frag_queue {
- spinlock_t lock;
+ struct rhash_head node;
+ union {
+ struct frag_v4_compare_key v4;
+ struct frag_v6_compare_key v6;
+ } key;
struct timer_list timer;
- struct hlist_node list;
+ struct netns_frags *net;
+ spinlock_t lock;
refcount_t refcnt;
struct sk_buff *fragments;
struct sk_buff *fragments_tail;
ktime_t stamp;
int len;
int meat;
- __u8 flags;
+ u8 flags;
u16 max_size;
- struct netns_frags *net;
- struct hlist_node list_evictor;
-};
-
-#define INETFRAGS_HASHSZ 1024
-
-/* averaged:
- * max_depth = default ipfrag_high_thresh / INETFRAGS_HASHSZ /
- * rounded up (SKB_TRUELEN(0) + sizeof(struct ipq or
- * struct frag_queue))
- */
-#define INETFRAGS_MAXDEPTH 128
-
-struct inet_frag_bucket {
- struct hlist_head chain;
- spinlock_t chain_lock;
+ struct rcu_head rcu;
};
struct inet_frags {
- struct inet_frag_bucket hash[INETFRAGS_HASHSZ];
-
- struct work_struct frags_work;
- unsigned int next_bucket;
- unsigned long last_rebuild_jiffies;
- bool rebuild;
-
- /* The first call to hashfn is responsible to initialize
- * rnd. This is best done with net_get_random_once.
- *
- * rnd_seqlock is used to let hash insertion detect
- * when it needs to re-lookup the hash chain to use.
- */
- u32 rnd;
- seqlock_t rnd_seqlock;
unsigned int qsize;
- unsigned int (*hashfn)(const struct inet_frag_queue *);
- bool (*match)(const struct inet_frag_queue *q,
- const void *arg);
- void (*constructor)(struct inet_frag_queue *q,
+ void (*constructor)(struct inet_frag_queue *fq,
const void *arg);
- void (*destructor)(struct inet_frag_queue *);
+ void (*destructor)(struct inet_frag_queue *fq);
void (*frag_expire)(struct timer_list *t);
struct kmem_cache *frags_cachep;
const char *frags_cache_name;
+ struct rhashtable_params rhash_params;
};
int inet_frags_init(struct inet_frags *);
@@ -108,17 +104,13 @@ void inet_frags_fini(struct inet_frags *);
static inline int inet_frags_init_net(struct netns_frags *nf)
{
atomic_set(&nf->mem, 0);
- return 0;
+ return rhashtable_init(&nf->rhashtable, &nf->f->rhash_params);
}
void inet_frags_exit_net(struct netns_frags *nf);
void inet_frag_kill(struct inet_frag_queue *q);
void inet_frag_destroy(struct inet_frag_queue *q);
-struct inet_frag_queue *inet_frag_find(struct netns_frags *nf,
- struct inet_frags *f, void *key, unsigned int hash);
-
-void inet_frag_maybe_warn_overflow(struct inet_frag_queue *q,
- const char *prefix);
+struct inet_frag_queue *inet_frag_find(struct netns_frags *nf, void *key);
static inline void inet_frag_put(struct inet_frag_queue *q)
{
@@ -126,11 +118,6 @@ static inline void inet_frag_put(struct inet_frag_queue *q)
inet_frag_destroy(q);
}
-static inline bool inet_frag_evicting(struct inet_frag_queue *q)
-{
- return !hlist_unhashed(&q->list_evictor);
-}
-
/* Memory Tracking Functions. */
static inline int frag_mem_limit(struct netns_frags *nf)
diff --git a/include/net/ipv6.h b/include/net/ipv6.h
index 5c18836672e9d1c560cdce15f5b34928c337abfd..76f84d4be91b92761fb9a26e7f52e2101ee34c0a 100644
--- a/include/net/ipv6.h
+++ b/include/net/ipv6.h
@@ -579,36 +579,20 @@ enum ip6_defrag_users {
__IP6_DEFRAG_CONNTRACK_BRIDGE_IN = IP6_DEFRAG_CONNTRACK_BRIDGE_IN + USHRT_MAX,
};
-struct ip6_create_arg {
- __be32 id;
- u32 user;
- const struct in6_addr *src;
- const struct in6_addr *dst;
- int iif;
- u8 ecn;
-};
-
void ip6_frag_init(struct inet_frag_queue *q, const void *a);
-bool ip6_frag_match(const struct inet_frag_queue *q, const void *a);
/*
- * Equivalent of ipv4 struct ip
+ * Equivalent of ipv4 struct ipq
*/
struct frag_queue {
struct inet_frag_queue q;
- __be32 id; /* fragment id */
- u32 user;
- struct in6_addr saddr;
- struct in6_addr daddr;
-
int iif;
__u16 nhoffset;
u8 ecn;
};
-void ip6_expire_frag_queue(struct net *net, struct frag_queue *fq,
- struct inet_frags *frags);
+void ip6_expire_frag_queue(struct net *net, struct frag_queue *fq);
static inline bool ipv6_addr_any(const struct in6_addr *a)
{
diff --git a/net/ieee802154/6lowpan/6lowpan_i.h b/net/ieee802154/6lowpan/6lowpan_i.h
index d8de3bcfb1032a1133402cb2a4c50a2448133846..b8d95cb71c25dd69c8a88b2c886a3f0d2ce1174f 100644
--- a/net/ieee802154/6lowpan/6lowpan_i.h
+++ b/net/ieee802154/6lowpan/6lowpan_i.h
@@ -17,37 +17,19 @@ typedef unsigned __bitwise lowpan_rx_result;
#define LOWPAN_DISPATCH_FRAG1 0xc0
#define LOWPAN_DISPATCH_FRAGN 0xe0
-struct lowpan_create_arg {
+struct frag_lowpan_compare_key {
u16 tag;
u16 d_size;
- const struct ieee802154_addr *src;
- const struct ieee802154_addr *dst;
+ const struct ieee802154_addr src;
+ const struct ieee802154_addr dst;
};
-/* Equivalent of ipv4 struct ip
+/* Equivalent of ipv4 struct ipq
*/
struct lowpan_frag_queue {
struct inet_frag_queue q;
-
- u16 tag;
- u16 d_size;
- struct ieee802154_addr saddr;
- struct ieee802154_addr daddr;
};
-static inline u32 ieee802154_addr_hash(const struct ieee802154_addr *a)
-{
- switch (a->mode) {
- case IEEE802154_ADDR_LONG:
- return (((__force u64)a->extended_addr) >> 32) ^
- (((__force u64)a->extended_addr) & 0xffffffff);
- case IEEE802154_ADDR_SHORT:
- return (__force u32)(a->short_addr + (a->pan_id << 16));
- default:
- return 0;
- }
-}
-
int lowpan_frag_rcv(struct sk_buff *skb, const u8 frag_type);
void lowpan_net_frag_exit(void);
int lowpan_net_frag_init(void);
diff --git a/net/ieee802154/6lowpan/reassembly.c b/net/ieee802154/6lowpan/reassembly.c
index 6badc055555b7baedac2051a1aaea15f9e9b180c..9ee4d22666c26d6d9796d0f484bb4beb265dea42 100644
--- a/net/ieee802154/6lowpan/reassembly.c
+++ b/net/ieee802154/6lowpan/reassembly.c
@@ -37,47 +37,15 @@ static struct inet_frags lowpan_frags;
static int lowpan_frag_reasm(struct lowpan_frag_queue *fq,
struct sk_buff *prev, struct net_device *ldev);
-static unsigned int lowpan_hash_frag(u16 tag, u16 d_size,
- const struct ieee802154_addr *saddr,
- const struct ieee802154_addr *daddr)
-{
- net_get_random_once(&lowpan_frags.rnd, sizeof(lowpan_frags.rnd));
- return jhash_3words(ieee802154_addr_hash(saddr),
- ieee802154_addr_hash(daddr),
- (__force u32)(tag + (d_size << 16)),
- lowpan_frags.rnd);
-}
-
-static unsigned int lowpan_hashfn(const struct inet_frag_queue *q)
-{
- const struct lowpan_frag_queue *fq;
-
- fq = container_of(q, struct lowpan_frag_queue, q);
- return lowpan_hash_frag(fq->tag, fq->d_size, &fq->saddr, &fq->daddr);
-}
-
-static bool lowpan_frag_match(const struct inet_frag_queue *q, const void *a)
-{
- const struct lowpan_frag_queue *fq;
- const struct lowpan_create_arg *arg = a;
-
- fq = container_of(q, struct lowpan_frag_queue, q);
- return fq->tag == arg->tag && fq->d_size == arg->d_size &&
- ieee802154_addr_equal(&fq->saddr, arg->src) &&
- ieee802154_addr_equal(&fq->daddr, arg->dst);
-}
-
static void lowpan_frag_init(struct inet_frag_queue *q, const void *a)
{
- const struct lowpan_create_arg *arg = a;
+ const struct frag_lowpan_compare_key *key = a;
struct lowpan_frag_queue *fq;
fq = container_of(q, struct lowpan_frag_queue, q);
- fq->tag = arg->tag;
- fq->d_size = arg->d_size;
- fq->saddr = *arg->src;
- fq->daddr = *arg->dst;
+ BUILD_BUG_ON(sizeof(*key) > sizeof(q->key));
+ memcpy(&q->key, key, sizeof(*key));
}
static void lowpan_frag_expire(struct timer_list *t)
@@ -105,25 +73,20 @@ fq_find(struct net *net, const struct lowpan_802154_cb *cb,
const struct ieee802154_addr *src,
const struct ieee802154_addr *dst)
{
- struct inet_frag_queue *q;
- struct lowpan_create_arg arg;
- unsigned int hash;
struct netns_ieee802154_lowpan *ieee802154_lowpan =
net_ieee802154_lowpan(net);
+ struct frag_lowpan_compare_key key = {
+ .tag = cb->d_tag,
+ .d_size = cb->d_size,
+ .src = *src,
+ .dst = *dst,
+ };
+ struct inet_frag_queue *q;
- arg.tag = cb->d_tag;
- arg.d_size = cb->d_size;
- arg.src = src;
- arg.dst = dst;
-
- hash = lowpan_hash_frag(cb->d_tag, cb->d_size, src, dst);
-
- q = inet_frag_find(&ieee802154_lowpan->frags,
- &lowpan_frags, &arg, hash);
- if (IS_ERR_OR_NULL(q)) {
- inet_frag_maybe_warn_overflow(q, pr_fmt());
+ q = inet_frag_find(&ieee802154_lowpan->frags, &key);
+ if (IS_ERR_OR_NULL(q))
return NULL;
- }
+
return container_of(q, struct lowpan_frag_queue, q);
}
@@ -588,6 +551,7 @@ static int __net_init lowpan_frags_init_net(struct net *net)
ieee802154_lowpan->frags.timeout = IPV6_FRAG_TIMEOUT;
ieee802154_lowpan->frags.f = &lowpan_frags;
+ ieee802154_lowpan->frags.f = &lowpan_frags;
res = inet_frags_init_net(&ieee802154_lowpan->frags);
if (res < 0)
return res;
@@ -611,6 +575,36 @@ static struct pernet_operations lowpan_frags_ops = {
.exit = lowpan_frags_exit_net,
};
+static u32 lowpan_key_hashfn(const void *data, u32 len, u32 seed)
+{
+ return jhash2(data,
+ sizeof(struct frag_lowpan_compare_key) / sizeof(u32), seed);
+}
+
+static u32 lowpan_obj_hashfn(const void *data, u32 len, u32 seed)
+{
+ const struct inet_frag_queue *fq = data;
+
+ return jhash2((const u32 *)&fq->key,
+ sizeof(struct frag_lowpan_compare_key) / sizeof(u32), seed);
+}
+
+static int lowpan_obj_cmpfn(struct rhashtable_compare_arg *arg, const void *ptr)
+{
+ const struct frag_lowpan_compare_key *key = arg->key;
+ const struct inet_frag_queue *fq = ptr;
+
+ return !!memcmp(&fq->key, key, sizeof(*key));
+}
+
+const struct rhashtable_params lowpan_rhash_params = {
+ .head_offset = offsetof(struct inet_frag_queue, node),
+ .hashfn = lowpan_key_hashfn,
+ .obj_hashfn = lowpan_obj_hashfn,
+ .obj_cmpfn = lowpan_obj_cmpfn,
+ .automatic_shrinking = true,
+};
+
int __init lowpan_net_frag_init(void)
{
int ret;
@@ -619,22 +613,24 @@ int __init lowpan_net_frag_init(void)
if (ret)
return ret;
- ret = register_pernet_subsys(&lowpan_frags_ops);
- if (ret)
- goto err_pernet;
-
- lowpan_frags.hashfn = lowpan_hashfn;
lowpan_frags.constructor = lowpan_frag_init;
lowpan_frags.destructor = NULL;
lowpan_frags.qsize = sizeof(struct frag_queue);
- lowpan_frags.match = lowpan_frag_match;
lowpan_frags.frag_expire = lowpan_frag_expire;
lowpan_frags.frags_cache_name = lowpan_frags_cache_name;
+ lowpan_frags.rhash_params = lowpan_rhash_params;
ret = inet_frags_init(&lowpan_frags);
if (ret)
goto err_pernet;
+ ret = register_pernet_subsys(&lowpan_frags_ops);
+ if (ret)
+ goto err_pernet_frags;
+
return ret;
+
+err_pernet_frags:
+ inet_frags_fini(&lowpan_frags);
err_pernet:
lowpan_frags_sysctl_unregister();
return ret;
diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c
index 1ac69f65d0dee600d0ab4db20ff5942952932c40..8ccaf605630f14270996ee1b5a37376299d78661 100644
--- a/net/ipv4/inet_fragment.c
+++ b/net/ipv4/inet_fragment.c
@@ -25,12 +25,6 @@
#include <net/inet_frag.h>
#include <net/inet_ecn.h>
-#define INETFRAGS_EVICT_BUCKETS 128
-#define INETFRAGS_EVICT_MAX 512
-
-/* don't rebuild inetfrag table with new secret more often than this */
-#define INETFRAGS_MIN_REBUILD_INTERVAL (5 * HZ)
-
/* Given the OR values of all fragments, apply RFC 3168 5.3 requirements
* Value : 0xff if frame should be dropped.
* 0 or INET_ECN_CE value, to be ORed in to final iph->tos field
@@ -52,157 +46,8 @@ const u8 ip_frag_ecn_table[16] = {
};
EXPORT_SYMBOL(ip_frag_ecn_table);
-static unsigned int
-inet_frag_hashfn(const struct inet_frags *f, const struct inet_frag_queue *q)
-{
- return f->hashfn(q) & (INETFRAGS_HASHSZ - 1);
-}
-
-static bool inet_frag_may_rebuild(struct inet_frags *f)
-{
- return time_after(jiffies,
- f->last_rebuild_jiffies + INETFRAGS_MIN_REBUILD_INTERVAL);
-}
-
-static void inet_frag_secret_rebuild(struct inet_frags *f)
-{
- int i;
-
- write_seqlock_bh(&f->rnd_seqlock);
-
- if (!inet_frag_may_rebuild(f))
- goto out;
-
- get_random_bytes(&f->rnd, sizeof(u32));
-
- for (i = 0; i < INETFRAGS_HASHSZ; i++) {
- struct inet_frag_bucket *hb;
- struct inet_frag_queue *q;
- struct hlist_node *n;
-
- hb = &f->hash[i];
- spin_lock(&hb->chain_lock);
-
- hlist_for_each_entry_safe(q, n, &hb->chain, list) {
- unsigned int hval = inet_frag_hashfn(f, q);
-
- if (hval != i) {
- struct inet_frag_bucket *hb_dest;
-
- hlist_del(&q->list);
-
- /* Relink to new hash chain. */
- hb_dest = &f->hash[hval];
-
- /* This is the only place where we take
- * another chain_lock while already holding
- * one. As this will not run concurrently,
- * we cannot deadlock on hb_dest lock below, if its
- * already locked it will be released soon since
- * other caller cannot be waiting for hb lock
- * that we've taken above.
- */
- spin_lock_nested(&hb_dest->chain_lock,
- SINGLE_DEPTH_NESTING);
- hlist_add_head(&q->list, &hb_dest->chain);
- spin_unlock(&hb_dest->chain_lock);
- }
- }
- spin_unlock(&hb->chain_lock);
- }
-
- f->rebuild = false;
- f->last_rebuild_jiffies = jiffies;
-out:
- write_sequnlock_bh(&f->rnd_seqlock);
-}
-
-static bool inet_fragq_should_evict(const struct inet_frag_queue *q)
-{
- if (!hlist_unhashed(&q->list_evictor))
- return false;
-
- return q->net->low_thresh == 0 ||
- frag_mem_limit(q->net) >= q->net->low_thresh;
-}
-
-static unsigned int
-inet_evict_bucket(struct inet_frags *f, struct inet_frag_bucket *hb)
-{
- struct inet_frag_queue *fq;
- struct hlist_node *n;
- unsigned int evicted = 0;
- HLIST_HEAD(expired);
-
- spin_lock(&hb->chain_lock);
-
- hlist_for_each_entry_safe(fq, n, &hb->chain, list) {
- if (!inet_fragq_should_evict(fq))
- continue;
-
- if (!del_timer(&fq->timer))
- continue;
-
- hlist_add_head(&fq->list_evictor, &expired);
- ++evicted;
- }
-
- spin_unlock(&hb->chain_lock);
-
- hlist_for_each_entry_safe(fq, n, &expired, list_evictor)
- f->frag_expire(&fq->timer);
-
- return evicted;
-}
-
-static void inet_frag_worker(struct work_struct *work)
-{
- unsigned int budget = INETFRAGS_EVICT_BUCKETS;
- unsigned int i, evicted = 0;
- struct inet_frags *f;
-
- f = container_of(work, struct inet_frags, frags_work);
-
- BUILD_BUG_ON(INETFRAGS_EVICT_BUCKETS >= INETFRAGS_HASHSZ);
-
- local_bh_disable();
-
- for (i = READ_ONCE(f->next_bucket); budget; --budget) {
- evicted += inet_evict_bucket(f, &f->hash[i]);
- i = (i + 1) & (INETFRAGS_HASHSZ - 1);
- if (evicted > INETFRAGS_EVICT_MAX)
- break;
- }
-
- f->next_bucket = i;
-
- local_bh_enable();
-
- if (f->rebuild && inet_frag_may_rebuild(f))
- inet_frag_secret_rebuild(f);
-}
-
-static void inet_frag_schedule_worker(struct inet_frags *f)
-{
- if (unlikely(!work_pending(&f->frags_work)))
- schedule_work(&f->frags_work);
-}
-
int inet_frags_init(struct inet_frags *f)
{
- int i;
-
- INIT_WORK(&f->frags_work, inet_frag_worker);
-
- for (i = 0; i < INETFRAGS_HASHSZ; i++) {
- struct inet_frag_bucket *hb = &f->hash[i];
-
- spin_lock_init(&hb->chain_lock);
- INIT_HLIST_HEAD(&hb->chain);
- }
-
- seqlock_init(&f->rnd_seqlock);
- f->last_rebuild_jiffies = 0;
f->frags_cachep = kmem_cache_create(f->frags_cache_name, f->qsize, 0, 0,
NULL);
if (!f->frags_cachep)
@@ -214,93 +59,80 @@ EXPORT_SYMBOL(inet_frags_init);
void inet_frags_fini(struct inet_frags *f)
{
- cancel_work_sync(&f->frags_work);
+ rcu_barrier();
kmem_cache_destroy(f->frags_cachep);
+ f->frags_cachep = NULL;
}
EXPORT_SYMBOL(inet_frags_fini);
void inet_frags_exit_net(struct netns_frags *nf)
{
- struct inet_frags *f =nf->f;
- unsigned int seq;
- int i;
-
- nf->low_thresh = 0;
+ struct rhashtable_iter hti;
+ struct inet_frag_queue *fq;
-evict_again:
- local_bh_disable();
- seq = read_seqbegin(&f->rnd_seqlock);
+ nf->low_thresh = 0; /* prevent creation of new frags */
- for (i = 0; i < INETFRAGS_HASHSZ ; i++)
- inet_evict_bucket(f, &f->hash[i]);
+ rhashtable_walk_enter(&nf->rhashtable, &hti);
+ do {
+ rhashtable_walk_start(&hti);
- local_bh_enable();
- cond_resched();
+ while ((fq = rhashtable_walk_next(&hti)) && !IS_ERR(fq)) {
+ if (refcount_inc_not_zero(&fq->refcnt)) {
+ spin_lock_bh(&fq->lock);
+ inet_frag_kill(fq);
+ spin_unlock_bh(&fq->lock);
+ inet_frag_put(fq);
+ }
+ }
- if (read_seqretry(&f->rnd_seqlock, seq) ||
- sum_frag_mem_limit(nf))
- goto evict_again;
+ rhashtable_walk_stop(&hti);
+ } while (cond_resched(), fq == ERR_PTR(-EAGAIN));
+ rhashtable_walk_exit(&hti);
+ rhashtable_destroy(&nf->rhashtable);
}
EXPORT_SYMBOL(inet_frags_exit_net);
-static struct inet_frag_bucket *
-get_frag_bucket_locked(struct inet_frag_queue *fq, struct inet_frags *f)
-__acquires(hb->chain_lock)
-{
- struct inet_frag_bucket *hb;
- unsigned int seq, hash;
-
- restart:
- seq = read_seqbegin(&f->rnd_seqlock);
-
- hash = inet_frag_hashfn(f, fq);
- hb = &f->hash[hash];
-
- spin_lock(&hb->chain_lock);
- if (read_seqretry(&f->rnd_seqlock, seq)) {
- spin_unlock(&hb->chain_lock);
- goto restart;
- }
-
- return hb;
-}
-
-static inline void fq_unlink(struct inet_frag_queue *fq)
-{
- struct inet_frag_bucket *hb;
-
- hb = get_frag_bucket_locked(fq, fq->net->f);
- hlist_del(&fq->list);
- fq->flags |= INET_FRAG_COMPLETE;
- spin_unlock(&hb->chain_lock);
-}
-
void inet_frag_kill(struct inet_frag_queue *fq)
{
if (del_timer(&fq->timer))
refcount_dec(&fq->refcnt);
if (!(fq->flags & INET_FRAG_COMPLETE)) {
- fq_unlink(fq);
+ struct netns_frags *nf = fq->net;
+
+ fq->flags |= INET_FRAG_COMPLETE;
+ rhashtable_remove_fast(&nf->rhashtable, &fq->node, nf->f->rhash_params);
refcount_dec(&fq->refcnt);
}
}
EXPORT_SYMBOL(inet_frag_kill);
-void inet_frag_destroy(struct inet_frag_queue *q)
+static void inet_frag_destroy_rcu(struct rcu_head *head)
{
- struct sk_buff *fp;
+ struct inet_frag_queue *fq = container_of(head, struct inet_frag_queue,
+ rcu);
+ struct inet_frags *f = fq->net->f;
+
+ if (f->destructor)
+ f->destructor(fq);
+ kmem_cache_free(f->frags_cachep, fq);
+}
+
+void inet_frag_destroy(struct inet_frag_queue *fq)
+{
+ unsigned int sum_truesize;
struct netns_frags *nf;
- unsigned int sum, sum_truesize = 0;
struct inet_frags *f;
+ struct sk_buff *fp;
- WARN_ON(!(q->flags & INET_FRAG_COMPLETE));
- WARN_ON(del_timer(&q->timer) != 0);
+ WARN_ON(!(fq->flags & INET_FRAG_COMPLETE));
+ WARN_ON(del_timer(&fq->timer) != 0);
/* Release all fragment data. */
- fp = q->fragments;
- nf = q->net;
+ fp = fq->fragments;
+ nf = fq->net;
f = nf->f;
+ sum_truesize = f->qsize;
while (fp) {
struct sk_buff *xp = fp->next;
@@ -308,136 +140,63 @@ void inet_frag_destroy(struct inet_frag_queue *q)
kfree_skb(fp);
fp = xp;
}
- sum = sum_truesize + f->qsize;
- if (f->destructor)
- f->destructor(q);
- kmem_cache_free(f->frags_cachep, q);
+ call_rcu(&fq->rcu, inet_frag_destroy_rcu);
- sub_frag_mem_limit(nf, sum);
+ sub_frag_mem_limit(nf, sum_truesize);
}
EXPORT_SYMBOL(inet_frag_destroy);
-static struct inet_frag_queue *inet_frag_intern(struct netns_frags *nf,
- struct inet_frag_queue *qp_in,
- struct inet_frags *f,
+static struct inet_frag_queue *inet_frag_create(struct netns_frags *nf,
void *arg)
{
- struct inet_frag_bucket *hb = get_frag_bucket_locked(qp_in, f);
- struct inet_frag_queue *qp;
-
-#ifdef CONFIG_SMP
- /* With SMP race we have to recheck hash table, because
- * such entry could have been created on other cpu before
- * we acquired hash bucket lock.
- */
- hlist_for_each_entry(qp, &hb->chain, list) {
- if (qp->net == nf && f->match(qp, arg)) {
- refcount_inc(&qp->refcnt);
- spin_unlock(&hb->chain_lock);
- qp_in->flags |= INET_FRAG_COMPLETE;
- inet_frag_put(qp_in);
- return qp;
- }
- }
-#endif
- qp = qp_in;
- if (!mod_timer(&qp->timer, jiffies + nf->timeout))
- refcount_inc(&qp->refcnt);
-
- refcount_inc(&qp->refcnt);
- hlist_add_head(&qp->list, &hb->chain);
-
- spin_unlock(&hb->chain_lock);
-
- return qp;
-}
-
-static struct inet_frag_queue *inet_frag_alloc(struct netns_frags *nf,
- struct inet_frags *f,
- void *arg)
-{
- struct inet_frag_queue *q;
+ struct inet_frags *f = nf->f;
+ struct inet_frag_queue *fq;
+ int err;
- if (!nf->high_thresh || frag_mem_limit(nf) > nf->high_thresh) {
- inet_frag_schedule_worker(f);
+ if (!nf->high_thresh || frag_mem_limit(nf) > nf->high_thresh)
return NULL;
- }
- q = kmem_cache_zalloc(f->frags_cachep, GFP_ATOMIC);
- if (!q)
+ fq = kmem_cache_zalloc(f->frags_cachep, GFP_ATOMIC);
+ if (!fq)
return NULL;
- q->net = nf;
- f->constructor(q, arg);
- add_frag_mem_limit(nf, f->qsize);
-
- timer_setup(&q->timer, f->frag_expire, 0);
- spin_lock_init(&q->lock);
- refcount_set(&q->refcnt, 1);
-
- return q;
-}
+ fq->net = nf;
+ f->constructor(fq, arg);
-static struct inet_frag_queue *inet_frag_create(struct netns_frags *nf,
- struct inet_frags *f,
- void *arg)
-{
- struct inet_frag_queue *q;
+ timer_setup(&fq->timer, f->frag_expire, 0);
+ spin_lock_init(&fq->lock);
+ refcount_set(&fq->refcnt, 3);
+ mod_timer(&fq->timer, jiffies + nf->timeout);
- q = inet_frag_alloc(nf, f, arg);
- if (!q)
+ err = rhashtable_insert_fast(&nf->rhashtable, &fq->node,
+ f->rhash_params);
+ add_frag_mem_limit(nf, f->qsize);
+ if (err < 0) {
+ fq->flags |= INET_FRAG_COMPLETE;
+ inet_frag_kill(fq);
+ inet_frag_destroy(fq);
return NULL;
-
- return inet_frag_intern(nf, q, f, arg);
+ }
+ return fq;
}
-struct inet_frag_queue *inet_frag_find(struct netns_frags *nf,
- struct inet_frags *f, void *key,
- unsigned int hash)
+/* TODO : call from rcu_read_lock() and no longer use refcount_inc_not_zero() */
+struct inet_frag_queue *inet_frag_find(struct netns_frags *nf, void *key)
{
- struct inet_frag_bucket *hb;
- struct inet_frag_queue *q;
- int depth = 0;
-
- if (frag_mem_limit(nf) > nf->low_thresh)
- inet_frag_schedule_worker(f);
-
- hash &= (INETFRAGS_HASHSZ - 1);
- hb = &f->hash[hash];
-
- spin_lock(&hb->chain_lock);
- hlist_for_each_entry(q, &hb->chain, list) {
- if (q->net == nf && f->match(q, key)) {
- refcount_inc(&q->refcnt);
- spin_unlock(&hb->chain_lock);
- return q;
- }
- depth++;
- }
- spin_unlock(&hb->chain_lock);
+ struct inet_frag_queue *fq;
- if (depth <= INETFRAGS_MAXDEPTH)
- return inet_frag_create(nf, f, key);
+ rcu_read_lock();
- if (inet_frag_may_rebuild(f)) {
- if (!f->rebuild)
- f->rebuild = true;
- inet_frag_schedule_worker(f);
+ fq = rhashtable_lookup(&nf->rhashtable, key, nf->f->rhash_params);
+ if (fq) {
+ if (!refcount_inc_not_zero(&fq->refcnt))
+ fq = NULL;
+ rcu_read_unlock();
+ return fq;
}
+ rcu_read_unlock();
- return ERR_PTR(-ENOBUFS);
+ return inet_frag_create(nf, key);
}
EXPORT_SYMBOL(inet_frag_find);
-
-void inet_frag_maybe_warn_overflow(struct inet_frag_queue *q,
- const char *prefix)
-{
- static const char msg[] = "inet_frag_find: Fragment hash bucket"
- " list length grew over limit " __stringify(INETFRAGS_MAXDEPTH)
- ". Dropping fragment.\n";
-
- if (PTR_ERR(q) == -ENOBUFS)
- net_dbg_ratelimited("%s%s", prefix, msg);
-}
-EXPORT_SYMBOL(inet_frag_maybe_warn_overflow);
diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c
index cd2b4c9419fc1552d367b572926e314b11cb6c00..1a7423e8ec0a8f88782ad8c945dc0cd6046f79f0 100644
--- a/net/ipv4/ip_fragment.c
+++ b/net/ipv4/ip_fragment.c
@@ -69,15 +69,9 @@ struct ipfrag_skb_cb
struct ipq {
struct inet_frag_queue q;
- u32 user;
- __be32 saddr;
- __be32 daddr;
- __be16 id;
- u8 protocol;
u8 ecn; /* RFC3168 support */
u16 max_df_size; /* largest frag with DF set seen */
int iif;
- int vif; /* L3 master device index */
unsigned int rid;
struct inet_peer *peer;
};
@@ -97,41 +91,6 @@ int ip_frag_mem(struct net *net)
static int ip_frag_reasm(struct ipq *qp, struct sk_buff *prev,
struct net_device *dev);
-struct ip4_create_arg {
- struct iphdr *iph;
- u32 user;
- int vif;
-};
-
-static unsigned int ipqhashfn(__be16 id, __be32 saddr, __be32 daddr, u8 prot)
-{
- net_get_random_once(&ip4_frags.rnd, sizeof(ip4_frags.rnd));
- return jhash_3words((__force u32)id << 16 | prot,
- (__force u32)saddr, (__force u32)daddr,
- ip4_frags.rnd);
-}
-
-static unsigned int ip4_hashfn(const struct inet_frag_queue *q)
-{
- const struct ipq *ipq;
-
- ipq = container_of(q, struct ipq, q);
- return ipqhashfn(ipq->id, ipq->saddr, ipq->daddr, ipq->protocol);
-}
-
-static bool ip4_frag_match(const struct inet_frag_queue *q, const void *a)
-{
- const struct ipq *qp;
- const struct ip4_create_arg *arg = a;
-
- qp = container_of(q, struct ipq, q);
- return qp->id == arg->iph->id &&
- qp->saddr == arg->iph->saddr &&
- qp->daddr == arg->iph->daddr &&
- qp->protocol == arg->iph->protocol &&
- qp->user == arg->user &&
- qp->vif == arg->vif;
-}
static void ip4_frag_init(struct inet_frag_queue *q, const void *a)
{
@@ -140,37 +99,23 @@ static void ip4_frag_init(struct inet_frag_queue *q, const void *a)
frags);
struct net *net = container_of(ipv4, struct net, ipv4);
- const struct ip4_create_arg *arg = a;
+ const struct frag_v4_compare_key *key = a;
- qp->protocol = arg->iph->protocol;
- qp->id = arg->iph->id;
- qp->ecn = ip4_frag_ecn(arg->iph->tos);
- qp->saddr = arg->iph->saddr;
- qp->daddr = arg->iph->daddr;
- qp->vif = arg->vif;
- qp->user = arg->user;
+ q->key.v4 = *key;
+ qp->ecn = 0;
qp->peer = q->net->max_dist ?
- inet_getpeer_v4(net->ipv4.peers, arg->iph->saddr, arg->vif, 1) :
+ inet_getpeer_v4(net->ipv4.peers, key->saddr, key->vif, 1) :
NULL;
}
-static void ip4_frag_free(struct inet_frag_queue *q)
+static void ip4_frag_destructor(struct inet_frag_queue *q)
{
- struct ipq *qp;
+ struct ipq *qp = container_of(q, struct ipq, q);
- qp = container_of(q, struct ipq, q);
if (qp->peer)
inet_putpeer(qp->peer);
}
-
-/* Destruction primitives. */
-
-static void ipq_put(struct ipq *ipq)
-{
- inet_frag_put(&ipq->q);
-}
-
/* Kill ipq entry. It is not destroyed immediately,
* because caller (and someone more) holds reference count.
*/
@@ -198,25 +143,25 @@ static void ip_expire(struct timer_list *t)
struct net *net;
qp = container_of(frag, struct ipq, q);
- net = container_of(qp->q.net, struct net, ipv4.frags);
+ net = container_of(frag->net, struct net, ipv4.frags);
rcu_read_lock();
- spin_lock(&qp->q.lock);
+ spin_lock(&frag->lock);
- if (qp->q.flags & INET_FRAG_COMPLETE)
+ if (frag->flags & INET_FRAG_COMPLETE)
goto out;
ipq_kill(qp);
__IP_INC_STATS(net, IPSTATS_MIB_REASMFAILS);
- if (!inet_frag_evicting(&qp->q)) {
- struct sk_buff *clone, *head = qp->q.fragments;
+ if (true) {
+ struct sk_buff *clone, *head = frag->fragments;
const struct iphdr *iph;
int err;
__IP_INC_STATS(net, IPSTATS_MIB_REASMTIMEOUT);
- if (!(qp->q.flags & INET_FRAG_FIRST_IN) || !qp->q.fragments)
+ if (!(frag->flags & INET_FRAG_FIRST_IN) || !frag->fragments)
goto out;
head->dev = dev_get_by_index_rcu(net, qp->iif);
@@ -234,7 +179,7 @@ static void ip_expire(struct timer_list *t)
/* Only an end host needs to send an ICMP
* "Fragment Reassembly Timeout" message, per RFC792.
*/
- if (frag_expire_skip_icmp(qp->user) &&
+ if (frag_expire_skip_icmp(frag->key.v4.user) &&
(skb_rtable(head)->rt_type != RTN_LOCAL))
goto out;
@@ -242,7 +187,7 @@ static void ip_expire(struct timer_list *t)
/* Send an ICMP "Fragment Reassembly Timeout" message. */
if (clone) {
- spin_unlock(&qp->q.lock);
+ spin_unlock(&frag->lock);
icmp_send(clone, ICMP_TIME_EXCEEDED,
ICMP_EXC_FRAGTIME, 0);
consume_skb(clone);
@@ -250,33 +195,32 @@ static void ip_expire(struct timer_list *t)
}
}
out:
- spin_unlock(&qp->q.lock);
+ spin_unlock(&frag->lock);
out_rcu_unlock:
rcu_read_unlock();
- ipq_put(qp);
+ inet_frag_put(frag);
}
/* Find the correct entry in the "incomplete datagrams" queue for
* this IP datagram, and create new one, if nothing is found.
*/
-static struct ipq *ip_find(struct net *net, struct iphdr *iph,
+static struct ipq *ip_find(struct net *net, const struct iphdr *iph,
u32 user, int vif)
{
+ struct frag_v4_compare_key key = {
+ .saddr = iph->saddr,
+ .daddr = iph->daddr,
+ .user = user,
+ .vif = vif,
+ .id = iph->id,
+ .protocol = iph->protocol,
+ };
struct inet_frag_queue *q;
- struct ip4_create_arg arg;
- unsigned int hash;
- arg.iph = iph;
- arg.user = user;
- arg.vif = vif;
-
- hash = ipqhashfn(iph->id, iph->saddr, iph->daddr, iph->protocol);
-
- q = inet_frag_find(&net->ipv4.frags, &ip4_frags, &arg, hash);
- if (IS_ERR_OR_NULL(q)) {
- inet_frag_maybe_warn_overflow(q, pr_fmt());
+ q = inet_frag_find(&net->ipv4.frags, &key);
+ if (!q)
return NULL;
- }
+
return container_of(q, struct ipq, q);
}
@@ -310,8 +254,8 @@ static int ip_frag_too_far(struct ipq *qp)
static int ip_frag_reinit(struct ipq *qp)
{
- struct sk_buff *fp;
unsigned int sum_truesize = 0;
+ struct sk_buff *fp;
if (!mod_timer(&qp->q.timer, jiffies + qp->q.net->timeout)) {
refcount_inc(&qp->q.refcnt);
@@ -652,11 +596,11 @@ static int ip_frag_reasm(struct ipq *qp, struct sk_buff *prev,
return 0;
out_nomem:
- net_dbg_ratelimited("queue_glue: no memory for gluing queue %p\n", qp);
+ net_dbg_ratelimited("queue_glue: no memory for gluing queue\n");
err = -ENOMEM;
goto out_fail;
out_oversize:
- net_info_ratelimited("Oversized IP packet from %pI4\n", &qp->saddr);
+ net_info_ratelimited("Oversized IP packet from %pI4\n", &qp->q.key.v4.saddr);
out_fail:
__IP_INC_STATS(net, IPSTATS_MIB_REASMFAILS);
return err;
@@ -682,7 +626,7 @@ int ip_defrag(struct net *net, struct sk_buff *skb, u32 user)
ret = ip_frag_queue(qp, skb);
spin_unlock(&qp->q.lock);
- ipq_put(qp);
+ inet_frag_put(&qp->q);
return ret;
}
@@ -894,17 +838,52 @@ static struct pernet_operations ip4_frags_ops = {
.exit = ipv4_frags_exit_net,
};
+
+static u32 ip4_key_hashfn(const void *data, u32 len, u32 seed)
+{
+ return jhash2(data,
+ sizeof(struct frag_v4_compare_key) / sizeof(u32), seed);
+}
+
+static u32 ip4_obj_hashfn(const void *data, u32 len, u32 seed)
+{
+ const struct inet_frag_queue *fq = data;
+
+ return jhash2((const u32 *)&fq->key.v4,
+ sizeof(struct frag_v4_compare_key) / sizeof(u32), seed);
+}
+
+static int ip4_obj_cmpfn(struct rhashtable_compare_arg *arg, const void *ptr)
+{
+ const struct frag_v4_compare_key *key = arg->key;
+ const struct inet_frag_queue *fq = ptr;
+
+ return !!memcmp(&fq->key, key, sizeof(*key));
+}
+
+static const struct rhashtable_params ip4_rhash_params = {
+ .head_offset = offsetof(struct inet_frag_queue, node),
+ .key_offset = offsetof(struct inet_frag_queue, key),
+ .key_len = sizeof(struct frag_v4_compare_key),
+ .hashfn = ip4_key_hashfn,
+ .obj_hashfn = ip4_obj_hashfn,
+ .obj_cmpfn = ip4_obj_cmpfn,
+ .automatic_shrinking = true,
+};
+
void __init ipfrag_init(void)
{
- ip4_frags_ctl_register();
- register_pernet_subsys(&ip4_frags_ops);
- ip4_frags.hashfn = ip4_hashfn;
ip4_frags.constructor = ip4_frag_init;
- ip4_frags.destructor = ip4_frag_free;
+ ip4_frags.destructor = ip4_frag_destructor;
ip4_frags.qsize = sizeof(struct ipq);
- ip4_frags.match = ip4_frag_match;
ip4_frags.frag_expire = ip_expire;
ip4_frags.frags_cache_name = ip_frag_cache_name;
+ ip4_frags.rhash_params = ip4_rhash_params;
+
if (inet_frags_init(&ip4_frags))
panic("IP: failed to allocate ip4_frags cache\n");
+
+ ip4_frags_ctl_register();
+ register_pernet_subsys(&ip4_frags_ops);
+
}
diff --git a/net/ipv6/netfilter/nf_conntrack_reasm.c b/net/ipv6/netfilter/nf_conntrack_reasm.c
index f69b7ca52727c814eb2887c9deb9f356c56e5442..53859311dea96c03fa5ae8456de32de25009efbe 100644
--- a/net/ipv6/netfilter/nf_conntrack_reasm.c
+++ b/net/ipv6/netfilter/nf_conntrack_reasm.c
@@ -152,23 +152,6 @@ static inline u8 ip6_frag_ecn(const struct ipv6hdr *ipv6h)
return 1 << (ipv6_get_dsfield(ipv6h) & INET_ECN_MASK);
}
-static unsigned int nf_hash_frag(__be32 id, const struct in6_addr *saddr,
- const struct in6_addr *daddr)
-{
- net_get_random_once(&nf_frags.rnd, sizeof(nf_frags.rnd));
- return jhash_3words(ipv6_addr_hash(saddr), ipv6_addr_hash(daddr),
- (__force u32)id, nf_frags.rnd);
-}
-
-
-static unsigned int nf_hashfn(const struct inet_frag_queue *q)
-{
- const struct frag_queue *nq;
-
- nq = container_of(q, struct frag_queue, q);
- return nf_hash_frag(nq->id, &nq->saddr, &nq->daddr);
-}
-
static void nf_ct_frag6_expire(struct timer_list *t)
{
struct inet_frag_queue *frag = from_timer(frag, t, timer);
@@ -178,34 +161,26 @@ static void nf_ct_frag6_expire(struct timer_list *t)
fq = container_of(frag, struct frag_queue, q);
net = container_of(fq->q.net, struct net, nf_frag.frags);
- ip6_expire_frag_queue(net, fq, &nf_frags);
+ ip6_expire_frag_queue(net, fq);
}
/* Creation primitives. */
-static inline struct frag_queue *fq_find(struct net *net, __be32 id,
- u32 user, struct in6_addr *src,
- struct in6_addr *dst, int iif, u8 ecn)
+static struct frag_queue *fq_find(struct net *net, __be32 id, u32 user,
+ const struct ipv6hdr *hdr, int iif)
{
+ struct frag_v6_compare_key key = {
+ .id = id,
+ .saddr = hdr->saddr,
+ .daddr = hdr->daddr,
+ .user = user,
+ .iif = iif,
+ };
struct inet_frag_queue *q;
- struct ip6_create_arg arg;
- unsigned int hash;
-
- arg.id = id;
- arg.user = user;
- arg.src = src;
- arg.dst = dst;
- arg.iif = iif;
- arg.ecn = ecn;
-
- local_bh_disable();
- hash = nf_hash_frag(id, src, dst);
-
- q = inet_frag_find(&net->nf_frag.frags, &nf_frags, &arg, hash);
- local_bh_enable();
- if (IS_ERR_OR_NULL(q)) {
- inet_frag_maybe_warn_overflow(q, pr_fmt());
+
+ q = inet_frag_find(&net->nf_frag.frags, &key);
+ if (!q)
return NULL;
- }
+
return container_of(q, struct frag_queue, q);
}
@@ -593,8 +568,8 @@ int nf_ct_frag6_gather(struct net *net, struct sk_buff *skb, u32 user)
fhdr = (struct frag_hdr *)skb_transport_header(skb);
skb_orphan(skb);
- fq = fq_find(net, fhdr->identification, user, &hdr->saddr, &hdr->daddr,
- skb->dev ? skb->dev->ifindex : 0, ip6_frag_ecn(hdr));
+ fq = fq_find(net, fhdr->identification, user, hdr,
+ skb->dev ? skb->dev->ifindex : 0);
if (fq == NULL) {
pr_debug("Can't find and can't create new queue\n");
return -ENOMEM;
@@ -656,17 +631,18 @@ static struct pernet_operations nf_ct_net_ops = {
.exit = nf_ct_net_exit,
};
+extern const struct rhashtable_params ip6_rhash_params;
+
int nf_ct_frag6_init(void)
{
int ret = 0;
- nf_frags.hashfn = nf_hashfn;
nf_frags.constructor = ip6_frag_init;
nf_frags.destructor = NULL;
nf_frags.qsize = sizeof(struct frag_queue);
- nf_frags.match = ip6_frag_match;
nf_frags.frag_expire = nf_ct_frag6_expire;
nf_frags.frags_cache_name = nf_frags_cache_name;
+ nf_frags.rhash_params = ip6_rhash_params;
ret = inet_frags_init(&nf_frags);
if (ret)
goto out;
diff --git a/net/ipv6/reassembly.c b/net/ipv6/reassembly.c
index 8cfea13a179c6f048177ac91fe26c8a5565e5820..737b0921ab0c9af198fefdf06d8f4ede91c7f3f6 100644
--- a/net/ipv6/reassembly.c
+++ b/net/ipv6/reassembly.c
@@ -79,59 +79,19 @@ static struct inet_frags ip6_frags;
static int ip6_frag_reasm(struct frag_queue *fq, struct sk_buff *prev,
struct net_device *dev);
-/*
- * callers should be careful not to use the hash value outside the ipfrag_lock
- * as doing so could race with ipfrag_hash_rnd being recalculated.
- */
-static unsigned int inet6_hash_frag(__be32 id, const struct in6_addr *saddr,
- const struct in6_addr *daddr)
-{
- net_get_random_once(&ip6_frags.rnd, sizeof(ip6_frags.rnd));
- return jhash_3words(ipv6_addr_hash(saddr), ipv6_addr_hash(daddr),
- (__force u32)id, ip6_frags.rnd);
-}
-
-static unsigned int ip6_hashfn(const struct inet_frag_queue *q)
-{
- const struct frag_queue *fq;
-
- fq = container_of(q, struct frag_queue, q);
- return inet6_hash_frag(fq->id, &fq->saddr, &fq->daddr);
-}
-
-bool ip6_frag_match(const struct inet_frag_queue *q, const void *a)
-{
- const struct frag_queue *fq;
- const struct ip6_create_arg *arg = a;
-
- fq = container_of(q, struct frag_queue, q);
- return fq->id == arg->id &&
- fq->user == arg->user &&
- ipv6_addr_equal(&fq->saddr, arg->src) &&
- ipv6_addr_equal(&fq->daddr, arg->dst) &&
- (arg->iif == fq->iif ||
- !(ipv6_addr_type(arg->dst) & (IPV6_ADDR_MULTICAST |
- IPV6_ADDR_LINKLOCAL)));
-}
-EXPORT_SYMBOL(ip6_frag_match);
-
void ip6_frag_init(struct inet_frag_queue *q, const void *a)
{
struct frag_queue *fq = container_of(q, struct frag_queue, q);
- const struct ip6_create_arg *arg = a;
+ const struct frag_v6_compare_key *key = a;
- fq->id = arg->id;
- fq->user = arg->user;
- fq->saddr = *arg->src;
- fq->daddr = *arg->dst;
- fq->ecn = arg->ecn;
+ q->key.v6 = *key;
+ fq->ecn = 0;
}
EXPORT_SYMBOL(ip6_frag_init);
-void ip6_expire_frag_queue(struct net *net, struct frag_queue *fq,
- struct inet_frags *frags)
+void ip6_expire_frag_queue(struct net *net, struct frag_queue *fq)
{
- struct net_device *dev = NULL;
+ struct net_device *dev;
spin_lock(&fq->q.lock);
@@ -146,10 +106,6 @@ void ip6_expire_frag_queue(struct net *net, struct frag_queue *fq,
goto out_rcu_unlock;
__IP6_INC_STATS(net, __in6_dev_get(dev), IPSTATS_MIB_REASMFAILS);
-
- if (inet_frag_evicting(&fq->q))
- goto out_rcu_unlock;
-
__IP6_INC_STATS(net, __in6_dev_get(dev), IPSTATS_MIB_REASMTIMEOUT);
/* Don't send error if the first segment did not arrive. */
@@ -179,31 +135,29 @@ static void ip6_frag_expire(struct timer_list *t)
fq = container_of(frag, struct frag_queue, q);
net = container_of(fq->q.net, struct net, ipv6.frags);
- ip6_expire_frag_queue(net, fq, &ip6_frags);
+ ip6_expire_frag_queue(net, fq);
}
static struct frag_queue *
-fq_find(struct net *net, __be32 id, const struct in6_addr *src,
- const struct in6_addr *dst, int iif, u8 ecn)
+fq_find(struct net *net, __be32 id, const struct ipv6hdr *hdr, int iif)
{
+ struct frag_v6_compare_key key = {
+ .id = id,
+ .saddr = hdr->saddr,
+ .daddr = hdr->daddr,
+ .user = IP6_DEFRAG_LOCAL_DELIVER,
+ .iif = iif,
+ };
struct inet_frag_queue *q;
- struct ip6_create_arg arg;
- unsigned int hash;
- arg.id = id;
- arg.user = IP6_DEFRAG_LOCAL_DELIVER;
- arg.src = src;
- arg.dst = dst;
- arg.iif = iif;
- arg.ecn = ecn;
+ if (!(ipv6_addr_type(&hdr->daddr) & (IPV6_ADDR_MULTICAST |
+ IPV6_ADDR_LINKLOCAL)))
+ key.iif = 0;
- hash = inet6_hash_frag(id, src, dst);
-
- q = inet_frag_find(&net->ipv6.frags, &ip6_frags, &arg, hash);
- if (IS_ERR_OR_NULL(q)) {
- inet_frag_maybe_warn_overflow(q, pr_fmt());
+ q = inet_frag_find(&net->ipv6.frags, &key);
+ if (!q)
return NULL;
- }
+
return container_of(q, struct frag_queue, q);
}
@@ -527,10 +481,11 @@ static int ip6_frag_reasm(struct frag_queue *fq, struct sk_buff *prev,
static int ipv6_frag_rcv(struct sk_buff *skb)
{
+ struct net *net = dev_net(skb_dst(skb)->dev);
+ const struct ipv6hdr *hdr = ipv6_hdr(skb);
struct frag_hdr *fhdr;
struct frag_queue *fq;
- const struct ipv6hdr *hdr = ipv6_hdr(skb);
- struct net *net = dev_net(skb_dst(skb)->dev);
+ int iif;
if (IP6CB(skb)->flags & IP6SKB_FRAGMENTED)
goto fail_hdr;
@@ -559,13 +514,14 @@ static int ipv6_frag_rcv(struct sk_buff *skb)
return 1;
}
- fq = fq_find(net, fhdr->identification, &hdr->saddr, &hdr->daddr,
- skb->dev ? skb->dev->ifindex : 0, ip6_frag_ecn(hdr));
+ iif = skb->dev ? skb->dev->ifindex : 0;
+ fq = fq_find(net, fhdr->identification, hdr, iif);
if (fq) {
int ret;
spin_lock(&fq->q.lock);
+ fq->iif = iif;
ret = ip6_frag_queue(fq, skb, fhdr, IP6CB(skb)->nhoff);
spin_unlock(&fq->q.lock);
@@ -718,6 +674,7 @@ static int __net_init ipv6_frags_init_net(struct net *net)
net->ipv6.frags.timeout = IPV6_FRAG_TIMEOUT;
net->ipv6.frags.f = &ip6_frags;
+ net->ipv6.frags.f = &ip6_frags;
res = inet_frags_init_net(&net->ipv6.frags);
if (res < 0)
return res;
@@ -739,14 +696,55 @@ static struct pernet_operations ip6_frags_ops = {
.exit = ipv6_frags_exit_net,
};
+static u32 ip6_key_hashfn(const void *data, u32 len, u32 seed)
+{
+ return jhash2(data,
+ sizeof(struct frag_v6_compare_key) / sizeof(u32), seed);
+}
+
+static u32 ip6_obj_hashfn(const void *data, u32 len, u32 seed)
+{
+ const struct inet_frag_queue *fq = data;
+
+ return jhash2((const u32 *)&fq->key.v6,
+ sizeof(struct frag_v6_compare_key) / sizeof(u32), seed);
+}
+
+static int ip6_obj_cmpfn(struct rhashtable_compare_arg *arg, const void *ptr)
+{
+ const struct frag_v6_compare_key *key = arg->key;
+ const struct inet_frag_queue *fq = ptr;
+
+ return !!memcmp(&fq->key, key, sizeof(*key));
+}
+
+const struct rhashtable_params ip6_rhash_params = {
+ .head_offset = offsetof(struct inet_frag_queue, node),
+ .hashfn = ip6_key_hashfn,
+ .obj_hashfn = ip6_obj_hashfn,
+ .obj_cmpfn = ip6_obj_cmpfn,
+ .automatic_shrinking = true,
+};
+EXPORT_SYMBOL(ip6_rhash_params);
+
int __init ipv6_frag_init(void)
{
int ret;
- ret = inet6_add_protocol(&frag_protocol, IPPROTO_FRAGMENT);
+ ip6_frags.constructor = ip6_frag_init;
+ ip6_frags.destructor = NULL;
+ ip6_frags.qsize = sizeof(struct frag_queue);
+ ip6_frags.frag_expire = ip6_frag_expire;
+ ip6_frags.frags_cache_name = ip6_frag_cache_name;
+ ip6_frags.rhash_params = ip6_rhash_params;
+ ret = inet_frags_init(&ip6_frags);
if (ret)
goto out;
+ ret = inet6_add_protocol(&frag_protocol, IPPROTO_FRAGMENT);
+ if (ret)
+ goto err_protocol;
+
ret = ip6_frags_sysctl_register();
if (ret)
goto err_sysctl;
@@ -755,16 +753,6 @@ int __init ipv6_frag_init(void)
if (ret)
goto err_pernet;
- ip6_frags.hashfn = ip6_hashfn;
- ip6_frags.constructor = ip6_frag_init;
- ip6_frags.destructor = NULL;
- ip6_frags.qsize = sizeof(struct frag_queue);
- ip6_frags.match = ip6_frag_match;
- ip6_frags.frag_expire = ip6_frag_expire;
- ip6_frags.frags_cache_name = ip6_frag_cache_name;
- ret = inet_frags_init(&ip6_frags);
- if (ret)
- goto err_pernet;
out:
return ret;
@@ -772,6 +760,8 @@ int __init ipv6_frag_init(void)
ip6_frags_sysctl_unregister();
err_sysctl:
inet6_del_protocol(&frag_protocol, IPPROTO_FRAGMENT);
+err_protocol:
+ inet_frags_fini(&ip6_frags);
goto out;
}
--
2.17.0.rc1.321.gba9d0f2565-goog
^ permalink raw reply related
* [PATCH net-next 3/6] inet: frags: add a pointer to struct netns_frags
From: Eric Dumazet @ 2018-03-30 5:22 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Florian Westphal, Herbert Xu, Thomas Graf,
Jesper Dangaard Brouer, Alexander Aring, Stefan Schmidt,
Eric Dumazet, Eric Dumazet
In-Reply-To: <20180330052241.206667-2-edumazet@google.com>
In order to simplify the API, add a pointer to struct inet_frags.
This will allow us to make things less complex.
These functions no longer have a struct inet_frags parameter :
inet_frag_destroy(struct inet_frag_queue *q /*, struct inet_frags *f */)
inet_frag_put(struct inet_frag_queue *q /*, struct inet_frags *f */)
inet_frag_kill(struct inet_frag_queue *q /*, struct inet_frags *f */)
inet_frags_exit_net(struct netns_frags *nf /*, struct inet_frags *f */);
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
include/net/inet_frag.h | 11 ++++++-----
net/ieee802154/6lowpan/reassembly.c | 13 +++++++------
net/ipv4/inet_fragment.c | 17 ++++++++++-------
net/ipv4/ip_fragment.c | 9 +++++----
net/ipv6/netfilter/nf_conntrack_reasm.c | 14 ++++++++------
net/ipv6/reassembly.c | 15 ++++++++-------
6 files changed, 44 insertions(+), 35 deletions(-)
diff --git a/include/net/inet_frag.h b/include/net/inet_frag.h
index b1d62176f3b4fcf100bd263e8eae0db656a3d9b6..69e531ed81894393e07cac9e953825fcb55ef42a 100644
--- a/include/net/inet_frag.h
+++ b/include/net/inet_frag.h
@@ -10,6 +10,7 @@ struct netns_frags {
int high_thresh;
int low_thresh;
int max_dist;
+ struct inet_frags *f;
};
/**
@@ -109,20 +110,20 @@ static inline int inet_frags_init_net(struct netns_frags *nf)
atomic_set(&nf->mem, 0);
return 0;
}
-void inet_frags_exit_net(struct netns_frags *nf, struct inet_frags *f);
+void inet_frags_exit_net(struct netns_frags *nf);
-void inet_frag_kill(struct inet_frag_queue *q, struct inet_frags *f);
-void inet_frag_destroy(struct inet_frag_queue *q, struct inet_frags *f);
+void inet_frag_kill(struct inet_frag_queue *q);
+void inet_frag_destroy(struct inet_frag_queue *q);
struct inet_frag_queue *inet_frag_find(struct netns_frags *nf,
struct inet_frags *f, void *key, unsigned int hash);
void inet_frag_maybe_warn_overflow(struct inet_frag_queue *q,
const char *prefix);
-static inline void inet_frag_put(struct inet_frag_queue *q, struct inet_frags *f)
+static inline void inet_frag_put(struct inet_frag_queue *q)
{
if (refcount_dec_and_test(&q->refcnt))
- inet_frag_destroy(q, f);
+ inet_frag_destroy(q);
}
static inline bool inet_frag_evicting(struct inet_frag_queue *q)
diff --git a/net/ieee802154/6lowpan/reassembly.c b/net/ieee802154/6lowpan/reassembly.c
index 2aaab4bba42961647a4d3d1c0b8497917d5065ce..6badc055555b7baedac2051a1aaea15f9e9b180c 100644
--- a/net/ieee802154/6lowpan/reassembly.c
+++ b/net/ieee802154/6lowpan/reassembly.c
@@ -94,10 +94,10 @@ static void lowpan_frag_expire(struct timer_list *t)
if (fq->q.flags & INET_FRAG_COMPLETE)
goto out;
- inet_frag_kill(&fq->q, &lowpan_frags);
+ inet_frag_kill(&fq->q);
out:
spin_unlock(&fq->q.lock);
- inet_frag_put(&fq->q, &lowpan_frags);
+ inet_frag_put(&fq->q);
}
static inline struct lowpan_frag_queue *
@@ -230,7 +230,7 @@ static int lowpan_frag_reasm(struct lowpan_frag_queue *fq, struct sk_buff *prev,
struct sk_buff *fp, *head = fq->q.fragments;
int sum_truesize;
- inet_frag_kill(&fq->q, &lowpan_frags);
+ inet_frag_kill(&fq->q);
/* Make the one we just received the head. */
if (prev) {
@@ -438,7 +438,7 @@ int lowpan_frag_rcv(struct sk_buff *skb, u8 frag_type)
ret = lowpan_frag_queue(fq, skb, frag_type);
spin_unlock(&fq->q.lock);
- inet_frag_put(&fq->q, &lowpan_frags);
+ inet_frag_put(&fq->q);
return ret;
}
@@ -586,13 +586,14 @@ static int __net_init lowpan_frags_init_net(struct net *net)
ieee802154_lowpan->frags.high_thresh = IPV6_FRAG_HIGH_THRESH;
ieee802154_lowpan->frags.low_thresh = IPV6_FRAG_LOW_THRESH;
ieee802154_lowpan->frags.timeout = IPV6_FRAG_TIMEOUT;
+ ieee802154_lowpan->frags.f = &lowpan_frags;
res = inet_frags_init_net(&ieee802154_lowpan->frags);
if (res < 0)
return res;
res = lowpan_frags_ns_sysctl_register(net);
if (res < 0)
- inet_frags_exit_net(&ieee802154_lowpan->frags, &lowpan_frags);
+ inet_frags_exit_net(&ieee802154_lowpan->frags);
return res;
}
@@ -602,7 +603,7 @@ static void __net_exit lowpan_frags_exit_net(struct net *net)
net_ieee802154_lowpan(net);
lowpan_frags_ns_sysctl_unregister(net);
- inet_frags_exit_net(&ieee802154_lowpan->frags, &lowpan_frags);
+ inet_frags_exit_net(&ieee802154_lowpan->frags);
}
static struct pernet_operations lowpan_frags_ops = {
diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c
index e8ec28999f5ce0c5d496e9a97ca1748b18db0cf0..1ac69f65d0dee600d0ab4db20ff5942952932c40 100644
--- a/net/ipv4/inet_fragment.c
+++ b/net/ipv4/inet_fragment.c
@@ -219,8 +219,9 @@ void inet_frags_fini(struct inet_frags *f)
}
EXPORT_SYMBOL(inet_frags_fini);
-void inet_frags_exit_net(struct netns_frags *nf, struct inet_frags *f)
+void inet_frags_exit_net(struct netns_frags *nf)
{
+ struct inet_frags *f =nf->f;
unsigned int seq;
int i;
@@ -264,33 +265,34 @@ __acquires(hb->chain_lock)
return hb;
}
-static inline void fq_unlink(struct inet_frag_queue *fq, struct inet_frags *f)
+static inline void fq_unlink(struct inet_frag_queue *fq)
{
struct inet_frag_bucket *hb;
- hb = get_frag_bucket_locked(fq, f);
+ hb = get_frag_bucket_locked(fq, fq->net->f);
hlist_del(&fq->list);
fq->flags |= INET_FRAG_COMPLETE;
spin_unlock(&hb->chain_lock);
}
-void inet_frag_kill(struct inet_frag_queue *fq, struct inet_frags *f)
+void inet_frag_kill(struct inet_frag_queue *fq)
{
if (del_timer(&fq->timer))
refcount_dec(&fq->refcnt);
if (!(fq->flags & INET_FRAG_COMPLETE)) {
- fq_unlink(fq, f);
+ fq_unlink(fq);
refcount_dec(&fq->refcnt);
}
}
EXPORT_SYMBOL(inet_frag_kill);
-void inet_frag_destroy(struct inet_frag_queue *q, struct inet_frags *f)
+void inet_frag_destroy(struct inet_frag_queue *q)
{
struct sk_buff *fp;
struct netns_frags *nf;
unsigned int sum, sum_truesize = 0;
+ struct inet_frags *f;
WARN_ON(!(q->flags & INET_FRAG_COMPLETE));
WARN_ON(del_timer(&q->timer) != 0);
@@ -298,6 +300,7 @@ void inet_frag_destroy(struct inet_frag_queue *q, struct inet_frags *f)
/* Release all fragment data. */
fp = q->fragments;
nf = q->net;
+ f = nf->f;
while (fp) {
struct sk_buff *xp = fp->next;
@@ -333,7 +336,7 @@ static struct inet_frag_queue *inet_frag_intern(struct netns_frags *nf,
refcount_inc(&qp->refcnt);
spin_unlock(&hb->chain_lock);
qp_in->flags |= INET_FRAG_COMPLETE;
- inet_frag_put(qp_in, f);
+ inet_frag_put(qp_in);
return qp;
}
}
diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c
index e0b39d4ecbd411ff4bb72d1ed973e45bd6da9ef1..cd2b4c9419fc1552d367b572926e314b11cb6c00 100644
--- a/net/ipv4/ip_fragment.c
+++ b/net/ipv4/ip_fragment.c
@@ -168,7 +168,7 @@ static void ip4_frag_free(struct inet_frag_queue *q)
static void ipq_put(struct ipq *ipq)
{
- inet_frag_put(&ipq->q, &ip4_frags);
+ inet_frag_put(&ipq->q);
}
/* Kill ipq entry. It is not destroyed immediately,
@@ -176,7 +176,7 @@ static void ipq_put(struct ipq *ipq)
*/
static void ipq_kill(struct ipq *ipq)
{
- inet_frag_kill(&ipq->q, &ip4_frags);
+ inet_frag_kill(&ipq->q);
}
static bool frag_expire_skip_icmp(u32 user)
@@ -872,20 +872,21 @@ static int __net_init ipv4_frags_init_net(struct net *net)
net->ipv4.frags.timeout = IP_FRAG_TIME;
net->ipv4.frags.max_dist = 64;
+ net->ipv4.frags.f = &ip4_frags;
res = inet_frags_init_net(&net->ipv4.frags);
if (res < 0)
return res;
res = ip4_frags_ns_ctl_register(net);
if (res < 0)
- inet_frags_exit_net(&net->ipv4.frags, &ip4_frags);
+ inet_frags_exit_net(&net->ipv4.frags);
return res;
}
static void __net_exit ipv4_frags_exit_net(struct net *net)
{
ip4_frags_ns_ctl_unregister(net);
- inet_frags_exit_net(&net->ipv4.frags, &ip4_frags);
+ inet_frags_exit_net(&net->ipv4.frags);
}
static struct pernet_operations ip4_frags_ops = {
diff --git a/net/ipv6/netfilter/nf_conntrack_reasm.c b/net/ipv6/netfilter/nf_conntrack_reasm.c
index 6ff41569134ae36809a8b42d8e46d50d19ffde53..f69b7ca52727c814eb2887c9deb9f356c56e5442 100644
--- a/net/ipv6/netfilter/nf_conntrack_reasm.c
+++ b/net/ipv6/netfilter/nf_conntrack_reasm.c
@@ -264,7 +264,7 @@ static int nf_ct_frag6_queue(struct frag_queue *fq, struct sk_buff *skb,
* this case. -DaveM
*/
pr_debug("end of fragment not rounded to 8 bytes.\n");
- inet_frag_kill(&fq->q, &nf_frags);
+ inet_frag_kill(&fq->q);
return -EPROTO;
}
if (end > fq->q.len) {
@@ -357,7 +357,7 @@ static int nf_ct_frag6_queue(struct frag_queue *fq, struct sk_buff *skb,
return 0;
discard_fq:
- inet_frag_kill(&fq->q, &nf_frags);
+ inet_frag_kill(&fq->q);
err:
return -EINVAL;
}
@@ -379,7 +379,7 @@ nf_ct_frag6_reasm(struct frag_queue *fq, struct sk_buff *prev, struct net_devic
int payload_len;
u8 ecn;
- inet_frag_kill(&fq->q, &nf_frags);
+ inet_frag_kill(&fq->q);
WARN_ON(head == NULL);
WARN_ON(NFCT_FRAG6_CB(head)->offset != 0);
@@ -622,7 +622,7 @@ int nf_ct_frag6_gather(struct net *net, struct sk_buff *skb, u32 user)
out_unlock:
spin_unlock_bh(&fq->q.lock);
- inet_frag_put(&fq->q, &nf_frags);
+ inet_frag_put(&fq->q);
return ret;
}
EXPORT_SYMBOL_GPL(nf_ct_frag6_gather);
@@ -634,19 +634,21 @@ static int nf_ct_net_init(struct net *net)
net->nf_frag.frags.high_thresh = IPV6_FRAG_HIGH_THRESH;
net->nf_frag.frags.low_thresh = IPV6_FRAG_LOW_THRESH;
net->nf_frag.frags.timeout = IPV6_FRAG_TIMEOUT;
+ net->nf_frag.frags.f = &nf_frags;
+
res = inet_frags_init_net(&net->nf_frag.frags);
if (res < 0)
return res;
res = nf_ct_frag6_sysctl_register(net);
if (res < 0)
- inet_frags_exit_net(&net->nf_frag.frags, &nf_frags);
+ inet_frags_exit_net(&net->nf_frag.frags);
return res;
}
static void nf_ct_net_exit(struct net *net)
{
nf_ct_frags6_sysctl_unregister(net);
- inet_frags_exit_net(&net->nf_frag.frags, &nf_frags);
+ inet_frags_exit_net(&net->nf_frag.frags);
}
static struct pernet_operations nf_ct_net_ops = {
diff --git a/net/ipv6/reassembly.c b/net/ipv6/reassembly.c
index a8f7a5f0251a7af0b14cc6de5006b924d9d05672..8cfea13a179c6f048177ac91fe26c8a5565e5820 100644
--- a/net/ipv6/reassembly.c
+++ b/net/ipv6/reassembly.c
@@ -138,7 +138,7 @@ void ip6_expire_frag_queue(struct net *net, struct frag_queue *fq,
if (fq->q.flags & INET_FRAG_COMPLETE)
goto out;
- inet_frag_kill(&fq->q, frags);
+ inet_frag_kill(&fq->q);
rcu_read_lock();
dev = dev_get_by_index_rcu(net, fq->iif);
@@ -166,7 +166,7 @@ void ip6_expire_frag_queue(struct net *net, struct frag_queue *fq,
rcu_read_unlock();
out:
spin_unlock(&fq->q.lock);
- inet_frag_put(&fq->q, frags);
+ inet_frag_put(&fq->q);
}
EXPORT_SYMBOL(ip6_expire_frag_queue);
@@ -364,7 +364,7 @@ static int ip6_frag_queue(struct frag_queue *fq, struct sk_buff *skb,
return -1;
discard_fq:
- inet_frag_kill(&fq->q, &ip6_frags);
+ inet_frag_kill(&fq->q);
err:
__IP6_INC_STATS(net, ip6_dst_idev(skb_dst(skb)),
IPSTATS_MIB_REASMFAILS);
@@ -391,7 +391,7 @@ static int ip6_frag_reasm(struct frag_queue *fq, struct sk_buff *prev,
int sum_truesize;
u8 ecn;
- inet_frag_kill(&fq->q, &ip6_frags);
+ inet_frag_kill(&fq->q);
ecn = ip_frag_ecn_table[fq->ecn];
if (unlikely(ecn == 0xff))
@@ -569,7 +569,7 @@ static int ipv6_frag_rcv(struct sk_buff *skb)
ret = ip6_frag_queue(fq, skb, fhdr, IP6CB(skb)->nhoff);
spin_unlock(&fq->q.lock);
- inet_frag_put(&fq->q, &ip6_frags);
+ inet_frag_put(&fq->q);
return ret;
}
@@ -716,6 +716,7 @@ static int __net_init ipv6_frags_init_net(struct net *net)
net->ipv6.frags.high_thresh = IPV6_FRAG_HIGH_THRESH;
net->ipv6.frags.low_thresh = IPV6_FRAG_LOW_THRESH;
net->ipv6.frags.timeout = IPV6_FRAG_TIMEOUT;
+ net->ipv6.frags.f = &ip6_frags;
res = inet_frags_init_net(&net->ipv6.frags);
if (res < 0)
@@ -723,14 +724,14 @@ static int __net_init ipv6_frags_init_net(struct net *net)
res = ip6_frags_ns_sysctl_register(net);
if (res < 0)
- inet_frags_exit_net(&net->ipv6.frags, &ip6_frags);
+ inet_frags_exit_net(&net->ipv6.frags);
return res;
}
static void __net_exit ipv6_frags_exit_net(struct net *net)
{
ip6_frags_ns_sysctl_unregister(net);
- inet_frags_exit_net(&net->ipv6.frags, &ip6_frags);
+ inet_frags_exit_net(&net->ipv6.frags);
}
static struct pernet_operations ip6_frags_ops = {
--
2.17.0.rc1.321.gba9d0f2565-goog
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox