* Re: [PATCH net-next 0/3] r8152: support setting eee by ethtool
From: David Miller @ 2014-09-28 21:24 UTC (permalink / raw)
To: hayeswang; +Cc: netdev, nic_swsd, linux-kernel, linux-usb
In-Reply-To: <1394712342-15778-46-Taiwan-albertk@realtek.com>
From: Hayes Wang <hayeswang@realtek.com>
Date: Thu, 25 Sep 2014 20:53:59 +0800
> Modify some definitions about EEE, and add the support of setting
> the EEE through ethtool.
Series applied, thanks.
^ permalink raw reply
* Re: [PATCH 0/3] defxx: DEFEA fixes and updates
From: David Miller @ 2014-09-28 21:22 UTC (permalink / raw)
To: macro; +Cc: netdev
In-Reply-To: <alpine.LFD.2.11.1409242337420.6148@eddie.linux-mips.org>
From: "Maciej W. Rozycki" <macro@linux-mips.org>
Date: Thu, 25 Sep 2014 11:06:33 +0100 (BST)
> I have finally got my hands on an EISA variation of the board (DEC
> FDDIcontroller/EISA aka DEFEA) and was able to do some testing. Here are
> initial updates to the driver that address problems I encountered so far.
> More to come later on as I get back to the system that I have in a remote
> location -- I need to double-check MMIO support and see what might have
> been causing spurious interrupts I saw with the 8259A PIC the board's
> interrupt line has been routed to.
Series applied to net-next, thank you.
^ permalink raw reply
* Re: pull request (net-next): ipsec-next 2014-09-25
From: David Miller @ 2014-09-28 21:19 UTC (permalink / raw)
To: steffen.klassert; +Cc: herbert, netdev
In-Reply-To: <1411636225-4433-1-git-send-email-steffen.klassert@secunet.com>
From: Steffen Klassert <steffen.klassert@secunet.com>
Date: Thu, 25 Sep 2014 11:10:21 +0200
> 1) Remove useless hash_resize_mutex in xfrm_hash_resize().
> This mutex is used only there, but xfrm_hash_resize()
> can't be called concurrently at all. From Ying Xue.
>
> 2) Extend policy hashing to prefixed policies based on
> prefix lenght thresholds. From Christophe Gouault.
>
> 3) Make the policy hash table thresholds configurable
> via netlink. From Christophe Gouault.
>
> 4) Remove the maximum authentication length for AH.
> This was needed to limit stack usage. We switched
> already to allocate space, so no need to keep the
> limit. From Herbert Xu.
Pulled, thanks a lot Steffen.
^ permalink raw reply
* Re: [PATCH] neigh: check error pointer instead of NULL for ipv4_neigh_lookup()
From: David Miller @ 2014-09-28 21:16 UTC (permalink / raw)
To: xiyou.wangcong; +Cc: netdev
In-Reply-To: <1411603673-19084-1-git-send-email-xiyou.wangcong@gmail.com>
From: Cong Wang <xiyou.wangcong@gmail.com>
Date: Wed, 24 Sep 2014 17:07:53 -0700
> Fixes: commit f187bc6efb7250afee0e2009b6106 ("ipv4: No need to set generic neighbour pointer")
> Cc: David S. Miller <davem@davemloft.net>
> Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
Applied, thank you.
^ permalink raw reply
* Re: [PATCH net-next 0/6] net: dsa: EEE and other PM features
From: David Miller @ 2014-09-28 21:14 UTC (permalink / raw)
To: f.fainelli; +Cc: netdev
In-Reply-To: <1411603522-8384-1-git-send-email-f.fainelli@gmail.com>
From: Florian Fainelli <f.fainelli@gmail.com>
Date: Wed, 24 Sep 2014 17:05:16 -0700
> This patch set allows DSA switch drivers to enable/disable/query EEE on a
> per-port level, as well as control precisely which switch ports are
> enable/disabled.
>
> I might already be too late for net-next, if that's the case, I will repost
> when the tree re-opens.
Series applied, thanks Florian.
^ permalink raw reply
* Re: [PATCH 0/5] ARCNET: add support for EAE multi interfac card
From: David Miller @ 2014-09-28 21:01 UTC (permalink / raw)
To: m.grzeschik; +Cc: netdev, linux-kernel, kernel
In-Reply-To: <20140928.164421.1247481867383564525.davem@davemloft.net>
From: David Miller <davem@davemloft.net>
Date: Sun, 28 Sep 2014 16:44:21 -0400 (EDT)
> From: Michael Grzeschik <m.grzeschik@pengutronix.de>
> Date: Wed, 24 Sep 2014 15:41:48 +0200
>
>> this series adds support for the PLX Bridge based multi interface
>> pci cards and adds support to change device address on com200xx chips
>> during runtime.
>>
>> This series is based on v3.17-rc6.
>
> Series applied to net-next, thanks.
I had to revert, this breaks the build:
drivers/net/arcnet/com20020_cs.c:115:8: error: redefinition of ‘struct com20020_dev’
struct com20020_dev {
^
In file included from drivers/net/arcnet/com20020_cs.c:43:0:
include/linux/com20020.h:66:8: note: originally defined here
struct com20020_dev {
^
make[3]: *** [drivers/net/arcnet/com20020_cs.o] Error 1
make[2]: *** [drivers/net/arcnet] Error 2
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [drivers/net] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [drivers] Error 2
^ permalink raw reply
* Re: [PATCH net-next] mlx4: optimize xmit path
From: Alexei Starovoitov @ 2014-09-28 20:49 UTC (permalink / raw)
To: Eric Dumazet
Cc: Or Gerlitz, David S. Miller, Jesper Dangaard Brouer, Eric Dumazet,
John Fastabend, Linux Netdev List, Amir Vadai, Or Gerlitz
In-Reply-To: <1411930364.15768.80.camel@edumazet-glaptop2.roam.corp.google.com>
On Sun, Sep 28, 2014 at 11:52 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> On Sun, 2014-09-28 at 11:07 -0700, Alexei Starovoitov wrote:
>
>> this is great improvement!
>> Thank you for leading this effort.
>> 10G line rate is definitely nice :)
>> Hopefully Or can demo similar numbers with 40G nic as well :)
>>
>> > + if (ring->bf_enabled && desc_size <= MAX_BF && !bounce &&
>> > + !vlan_tx_tag_present(skb) && send_doorbell) {
>>
>> feels something wrong here, since it checks for
>> send_doorbell, but iowrite() happens in the 'else' part
>> of this branch with another 'if (send_doorbell)'
>>
>
> If we do not plan to send a doorbell, we should not use blueframe.
>
> Blueframe is always sending a doorbell by design, as it uses a single
> flip buffer.
>
> So if you want to see any improvement thanks to skb->xmit_more, we have
> to use the iowrite32(), not the blueframe.
>
> Therefore , if send_doorbell == false, we do not want the doorbell
I see. So xmit_more=true overrides blueflame=on settings.
I wonder what is the performance difference bf=on vs bf=off,
also whether a burst of N packets via bf is slower than
burst via queue+doorbell.
Some fun exploration for driver experts :)
> Is it making sense now ? ;)
It did, but only after studying this BlueFlame thingy ;)
Thanks!
^ permalink raw reply
* Re: [PATCH 0/5] ARCNET: add support for EAE multi interfac card
From: David Miller @ 2014-09-28 20:44 UTC (permalink / raw)
To: m.grzeschik; +Cc: netdev, linux-kernel, kernel
In-Reply-To: <1411566113-15583-1-git-send-email-m.grzeschik@pengutronix.de>
From: Michael Grzeschik <m.grzeschik@pengutronix.de>
Date: Wed, 24 Sep 2014 15:41:48 +0200
> this series adds support for the PLX Bridge based multi interface
> pci cards and adds support to change device address on com200xx chips
> during runtime.
>
> This series is based on v3.17-rc6.
Series applied to net-next, thanks.
^ permalink raw reply
* Re: [PATCH v1 0/4] net: fec: Code cleanup
From: Florian Fainelli @ 2014-09-28 20:43 UTC (permalink / raw)
To: fugang.duan@freescale.com, David Miller
Cc: netdev@vger.kernel.org, Frank.Li@freescale.com
In-Reply-To: <1779baf41a334eada2d11db0a474641b@BY2PR03MB377.namprd03.prod.outlook.com>
On 09/27/14 20:39, fugang.duan@freescale.com wrote:
> From: David Miller <davem@davemloft.net> Sent: Sunday, September 28, 2014 6:55 AM
>> To: Duan Fugang-B38611
>> Cc: netdev@vger.kernel.org; Li Frank-B20596
>> Subject: Re: [PATCH v1 0/4] net: fec: Code cleanup
>>
>> From: "fugang.duan@freescale.com" <fugang.duan@freescale.com>
>> Date: Sat, 27 Sep 2014 09:51:52 +0000
>>
>>> From: David Miller <davem@davemloft.net> Sent: Saturday, September 27,
>>> 2014 4:06 AM
>>>> To: Duan Fugang-B38611
>>>> Cc: netdev@vger.kernel.org; Li Frank-B20596
>>>> Subject: Re: [PATCH v1 0/4] net: fec: Code cleanup
>>>>
>>>> From: Fugang Duan <b38611@freescale.com>
>>>> Date: Tue, 23 Sep 2014 15:40:54 +0800
>>>>
>>>>> This patches does several things:
>>>>> - Fixing multiqueue issue.
>>>>> - Removing the unnecessary errata workaround.
>>>>> - Aligning the data buffer dma map/unmap size.
>>>>> - Freeing resource after probe failed.
>>>>
>>>> Series applied to net-next, thanks.
>>>>
>>>> Please always explicitly state what tree you expect your patches to be
>>>> applied to in the future.
>>>
>>> No special requirement, just follow the normal rel flow.
>>
>> That doesn't mean anything to me.
>>
>> You must state what tree your patches apply properly to, and thus where
>> you expect them to be intergrated.
>
> I see. It is better let them enter to 3.17 (rc7).
They are not bugfixes, so they should target David's net-next tree,
which will be pulled by Linus during the v3.18 merge window in the next
few days.
Stuff that really needs to be fixed for 3.17 need to target David's net
tree.
--
Florian
^ permalink raw reply
* Re: [PATCH net-next v3] tcp: use tcp_flags in tcp_data_queue()
From: David Miller @ 2014-09-28 20:38 UTC (permalink / raw)
To: panweiping3; +Cc: netdev, edumazet
In-Reply-To: <ba777860fdfc42d53ecd35bdac5d81ca5a9872eb.1411568111.git.panweiping3@gmail.com>
From: Weiping Pan <panweiping3@gmail.com>
Date: Wed, 24 Sep 2014 22:17:02 +0800
> This patch is a cleanup which follows the idea in commit e11ecddf5128 (tcp: use
> TCP_SKB_CB(skb)->tcp_flags in input path),
> and it may reduce register pressure since skb->cb[] access is fast,
> bacause skb is probably in a register.
>
> v2: remove variable th
> v3: reword the changelog
>
> Signed-off-by: Weiping Pan <panweiping3@gmail.com>
Applied, thanks.
^ permalink raw reply
* Re: [PATCH net-next] tcp: change tcp_skb_pcount() location
From: David Miller @ 2014-09-28 20:37 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev, ncardwell, ycheng, willemb
In-Reply-To: <1411557082.15395.24.camel@edumazet-glaptop2.roam.corp.google.com>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Wed, 24 Sep 2014 04:11:22 -0700
> From: Eric Dumazet <edumazet@google.com>
>
> Our goal is to access no more than one cache line access per skb in
> a write or receive queue when doing the various walks.
>
> After recent TCP_SKB_CB() reorganizations, it is almost done.
>
> Last part is tcp_skb_pcount() which currently uses
> skb_shinfo(skb)->gso_segs, which is a terrible choice, because it needs
> 3 cache lines in current kernel (skb->head, skb->end, and
> shinfo->gso_segs are all in 3 different cache lines, far from skb->cb)
>
> This very simple patch reuses space currently taken by tcp_tw_isn
> only in input path, as tcp_skb_pcount is only needed for skb stored in
> write queue.
>
> This considerably speeds up tcp_ack(), granted we avoid shinfo->tx_flags
> to get SKBTX_ACK_TSTAMP, which seems possible.
>
> This also speeds up all sack processing in general.
>
> This speeds up tcp_sendmsg() because it no longer has to access/dirty
> shinfo.
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> ---
> Note : This patch applies cleanly after "tcp: better TCP_SKB_CB layout"
> patch serie.
Also applied, thanks Eric.
^ permalink raw reply
* Re: [PATCH v4 net-next 0/3] tcp: better TCP_SKB_CB layout
From: David Miller @ 2014-09-28 20:36 UTC (permalink / raw)
To: edumazet; +Cc: netdev, ycheng, ncardwell, christoph.paasch
In-Reply-To: <1411836657-9542-1-git-send-email-edumazet@google.com>
From: Eric Dumazet <edumazet@google.com>
Date: Sat, 27 Sep 2014 09:50:54 -0700
> TCP had the assumption that IPCB and IP6CB are first members of skb->cb[]
>
> This is fine, except that IPCB/IP6CB are used in TCP for a very short time
> in input path.
>
> What really matters for TCP stack is to get skb->next,
> TCP_SKB_CB(skb)->seq, and TCP_SKB_CB(skb)->end_seq in the same cache line.
>
> skb that are immediately consumed do not care because whole skb->cb[] is
> hot in cpu cache, while skb that sit in wocket write queue or receive queues
> do not need TCP_SKB_CB(skb)->header at all.
>
> This patch set implements the prereq for IPv4, IPv6, and TCP to make this
> possible. This makes TCP more efficient.
Series applied, thanks.
^ permalink raw reply
* Re: [PATCH v1 5/5] driver-core: add driver asynchronous probe support
From: Dmitry Torokhov @ 2014-09-28 19:22 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: gregkh, tiwai, tj, arjan, teg, rmilasan, werner, oleg, hare,
bpoirier, santosh, pmladek, dbueso, mcgrof, linux-kernel,
Tetsuo Handa, Joseph Salisbury, Kay Sievers, One Thousand Gnomes,
Tim Gardner, Pierre Fersing, Andrew Morton, Nagalakshmi Nandigama,
Praveen Krishnamoorthy, Sreekanth Reddy, Abhijit Mahajan,
Casey Leedom, Hariprasad S
In-Reply-To: <1411768637-6809-6-git-send-email-mcgrof@do-not-panic.com>
Hi Luis,
On Fri, Sep 26, 2014 at 02:57:17PM -0700, Luis R. Rodriguez wrote:
> +static bool drv_enable_async_probe(struct device_driver *drv,
> + struct bus_type *bus)
> +{
> + struct module *mod;
> +
> + if (!drv->owner || drv->sync_probe)
> + return false;
This bit is one of the biggest issues I have with the patch set. Why async
probing is limited to modules only? I mentioned several times that we need
async probing for built-in drivers and the way you are structuring the flags
(async by default for modules, possibly opt-out of async for modules, forcibly
sync for built-in) it is hard to extend the infrastructure for built-in case.
Also, as far as I can see, you are only considering the case where driver is
being bound to already registered devices. If you have a module that creates a
device for a driver that is already loaded and takes long time to probe you
would still be probing synchronously even if driver/module requested async
behavior.
So for me it is NAK in the current form.
Thanks.
--
Dmitry
^ permalink raw reply
* [net-next PATCH v3 4/4] net: sched: enable per cpu qstats
From: John Fastabend @ 2014-09-28 18:54 UTC (permalink / raw)
To: xiyou.wangcong, jhs, eric.dumazet, davem; +Cc: netdev
In-Reply-To: <20140928185255.1976.56179.stgit@nitbit.x32>
After previous patches to simplify qstats the qstats can be
made per cpu with a packed union in Qdisc struct.
Signed-off-by: John Fastabend <john.r.fastabend@intel.com>
---
include/net/gen_stats.h | 3 ++
include/net/sch_generic.h | 12 +++++++++-
net/core/gen_stats.c | 55 +++++++++++++++++++++++++++++++++++++++------
net/sched/act_api.c | 2 +-
net/sched/sch_api.c | 12 ++++++++--
net/sched/sch_atm.c | 2 +-
net/sched/sch_cbq.c | 2 +-
net/sched/sch_drr.c | 2 +-
net/sched/sch_fq_codel.c | 2 +-
net/sched/sch_hfsc.c | 2 +-
net/sched/sch_htb.c | 2 +-
net/sched/sch_mq.c | 2 +-
net/sched/sch_mqprio.c | 5 ++--
net/sched/sch_multiq.c | 2 +-
net/sched/sch_prio.c | 2 +-
net/sched/sch_qfq.c | 3 ++
net/sched/sch_sfq.c | 2 +-
17 files changed, 87 insertions(+), 25 deletions(-)
diff --git a/include/net/gen_stats.h b/include/net/gen_stats.h
index de9b3dd..cbafa37 100644
--- a/include/net/gen_stats.h
+++ b/include/net/gen_stats.h
@@ -41,7 +41,8 @@ int gnet_stats_copy_rate_est(struct gnet_dump *d,
const struct gnet_stats_basic_packed *b,
struct gnet_stats_rate_est64 *r);
int gnet_stats_copy_queue(struct gnet_dump *d,
- struct gnet_stats_queue *q, __u32 len);
+ struct gnet_stats_queue __percpu *cpu_q,
+ struct gnet_stats_queue *q, __u32 qlen);
int gnet_stats_copy_app(struct gnet_dump *d, void *st, int len);
int gnet_stats_finish_copy(struct gnet_dump *d);
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index 23a0f0f..f126698 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -90,7 +90,10 @@ struct Qdisc {
struct gnet_stats_basic_cpu __percpu *cpu_bstats;
} __packed;
unsigned int __state;
- struct gnet_stats_queue qstats;
+ union {
+ struct gnet_stats_queue qstats;
+ struct gnet_stats_queue __percpu *cpu_qstats;
+ } __packed;
struct rcu_head rcu_head;
int padded;
atomic_t refcnt;
@@ -543,6 +546,13 @@ static inline void qdisc_qstats_drop(struct Qdisc *sch)
sch->qstats.drops++;
}
+static inline void qdisc_qstats_drop_cpu(struct Qdisc *sch)
+{
+ struct gnet_stats_queue *qstats = this_cpu_ptr(sch->cpu_qstats);
+
+ qstats->drops++;
+}
+
static inline void qdisc_qstats_overlimit(struct Qdisc *sch)
{
sch->qstats.overlimits++;
diff --git a/net/core/gen_stats.c b/net/core/gen_stats.c
index ad3ecb6..14681b9 100644
--- a/net/core/gen_stats.c
+++ b/net/core/gen_stats.c
@@ -215,33 +215,74 @@ gnet_stats_copy_rate_est(struct gnet_dump *d,
}
EXPORT_SYMBOL(gnet_stats_copy_rate_est);
+static void
+__gnet_stats_copy_queue_cpu(struct gnet_stats_queue *qstats,
+ const struct gnet_stats_queue __percpu *q)
+{
+ int i;
+
+ for_each_possible_cpu(i) {
+ const struct gnet_stats_queue *qcpu = per_cpu_ptr(q, i);
+
+ qstats->qlen = 0;
+ qstats->backlog += qcpu->backlog;
+ qstats->drops += qcpu->drops;
+ qstats->requeues += qcpu->requeues;
+ qstats->overlimits += qcpu->overlimits;
+ }
+}
+
+static void __gnet_stats_copy_queue(struct gnet_stats_queue *qstats,
+ const struct gnet_stats_queue __percpu *cpu,
+ const struct gnet_stats_queue *q,
+ __u32 qlen)
+{
+ if (cpu) {
+ __gnet_stats_copy_queue_cpu(qstats, cpu);
+ } else {
+ qstats->qlen = q->qlen;
+ qstats->backlog = q->backlog;
+ qstats->drops = q->drops;
+ qstats->requeues = q->requeues;
+ qstats->overlimits = q->overlimits;
+ }
+
+ qstats->qlen = qlen;
+}
+
/**
* gnet_stats_copy_queue - copy queue statistics into statistics TLV
* @d: dumping handle
+ * @cpu_q: per cpu queue statistics
* @q: queue statistics
* @qlen: queue length statistics
*
* Appends the queue statistics to the top level TLV created by
- * gnet_stats_start_copy().
+ * gnet_stats_start_copy(). Using per cpu queue statistics if
+ * they are available.
*
* Returns 0 on success or -1 with the statistic lock released
* if the room in the socket buffer was not sufficient.
*/
int
gnet_stats_copy_queue(struct gnet_dump *d,
+ struct gnet_stats_queue __percpu *cpu_q,
struct gnet_stats_queue *q, __u32 qlen)
{
- q->qlen = qlen;
+ struct gnet_stats_queue qstats = {0};
+
+ __gnet_stats_copy_queue(&qstats, cpu_q, q, qlen);
if (d->compat_tc_stats) {
- d->tc_stats.drops = q->drops;
- d->tc_stats.qlen = q->qlen;
- d->tc_stats.backlog = q->backlog;
- d->tc_stats.overlimits = q->overlimits;
+ d->tc_stats.drops = qstats.drops;
+ d->tc_stats.qlen = qstats.qlen;
+ d->tc_stats.backlog = qstats.backlog;
+ d->tc_stats.overlimits = qstats.overlimits;
}
if (d->tail)
- return gnet_stats_copy(d, TCA_STATS_QUEUE, q, sizeof(*q));
+ return gnet_stats_copy(d, TCA_STATS_QUEUE,
+ &qstats, sizeof(qstats));
return 0;
}
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index 2e13409..3d43e49 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -623,7 +623,7 @@ int tcf_action_copy_stats(struct sk_buff *skb, struct tc_action *a,
if (gnet_stats_copy_basic(&d, NULL, &p->tcfc_bstats) < 0 ||
gnet_stats_copy_rate_est(&d, &p->tcfc_bstats,
&p->tcfc_rate_est) < 0 ||
- gnet_stats_copy_queue(&d,
+ gnet_stats_copy_queue(&d, NULL,
&p->tcfc_qstats,
p->tcfc_qstats.qlen) < 0)
goto errout;
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index ca00ea8..aa83295 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -947,6 +947,10 @@ qdisc_create(struct net_device *dev, struct netdev_queue *dev_queue,
alloc_percpu(struct gnet_stats_basic_cpu);
if (!sch->cpu_bstats)
goto err_out4;
+
+ sch->cpu_qstats = alloc_percpu(struct gnet_stats_queue);
+ if (!sch->cpu_qstats)
+ goto err_out4;
}
if (tca[TCA_STAB]) {
@@ -995,6 +999,7 @@ err_out:
err_out4:
free_percpu(sch->cpu_bstats);
+ free_percpu(sch->cpu_qstats);
/*
* Any broken qdiscs that would require a ops->reset() here?
* The qdisc was never in action so it shouldn't be necessary.
@@ -1313,6 +1318,7 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
u32 portid, u32 seq, u16 flags, int event)
{
struct gnet_stats_basic_cpu __percpu *cpu_bstats = NULL;
+ struct gnet_stats_queue __percpu *cpu_qstats = NULL;
struct tcmsg *tcm;
struct nlmsghdr *nlh;
unsigned char *b = skb_tail_pointer(skb);
@@ -1349,12 +1355,14 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
if (q->ops->dump_stats && q->ops->dump_stats(q, &d) < 0)
goto nla_put_failure;
- if (qdisc_is_percpu_stats(q))
+ if (qdisc_is_percpu_stats(q)) {
cpu_bstats = q->cpu_bstats;
+ cpu_qstats = q->cpu_qstats;
+ }
if (gnet_stats_copy_basic(&d, cpu_bstats, &q->bstats) < 0 ||
gnet_stats_copy_rate_est(&d, &q->bstats, &q->rate_est) < 0 ||
- gnet_stats_copy_queue(&d, &q->qstats, qlen) < 0)
+ gnet_stats_copy_queue(&d, cpu_qstats, &q->qstats, qlen) < 0)
goto nla_put_failure;
if (gnet_stats_finish_copy(&d) < 0)
diff --git a/net/sched/sch_atm.c b/net/sched/sch_atm.c
index c145eb6..e3e2cc5 100644
--- a/net/sched/sch_atm.c
+++ b/net/sched/sch_atm.c
@@ -638,7 +638,7 @@ atm_tc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
struct atm_flow_data *flow = (struct atm_flow_data *)arg;
if (gnet_stats_copy_basic(d, NULL, &flow->bstats) < 0 ||
- gnet_stats_copy_queue(d, &flow->qstats, flow->q->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &flow->qstats, flow->q->q.qlen) < 0)
return -1;
return 0;
diff --git a/net/sched/sch_cbq.c b/net/sched/sch_cbq.c
index c610081..beeb75f 100644
--- a/net/sched/sch_cbq.c
+++ b/net/sched/sch_cbq.c
@@ -1602,7 +1602,7 @@ cbq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qstats, cl->q->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &cl->qstats, cl->q->q.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &cl->xstats, sizeof(cl->xstats));
diff --git a/net/sched/sch_drr.c b/net/sched/sch_drr.c
index 5835a93..3387060 100644
--- a/net/sched/sch_drr.c
+++ b/net/sched/sch_drr.c
@@ -284,7 +284,7 @@ static int drr_dump_class_stats(struct Qdisc *sch, unsigned long arg,
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qdisc->qstats, qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &cl->qdisc->qstats, qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index 226d735..b9ca32e 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -550,7 +550,7 @@ static int fq_codel_dump_class_stats(struct Qdisc *sch, unsigned long cl,
qs.backlog = q->backlogs[idx];
qs.drops = flow->dropped;
}
- if (gnet_stats_copy_queue(d, &qs, 0) < 0)
+ if (gnet_stats_copy_queue(d, NULL, &qs, 0) < 0)
return -1;
if (idx < q->flows_cnt)
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c
index d364acb..e6c7416 100644
--- a/net/sched/sch_hfsc.c
+++ b/net/sched/sch_hfsc.c
@@ -1378,7 +1378,7 @@ hfsc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qstats, cl->qdisc->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &cl->qstats, cl->qdisc->q.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index 3a691fd..f1acb0f 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -1147,7 +1147,7 @@ htb_dump_class_stats(struct Qdisc *sch, unsigned long arg, struct gnet_dump *d)
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, NULL, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qstats, qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &cl->qstats, qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &cl->xstats, sizeof(cl->xstats));
diff --git a/net/sched/sch_mq.c b/net/sched/sch_mq.c
index 6416a69..f3cbaec 100644
--- a/net/sched/sch_mq.c
+++ b/net/sched/sch_mq.c
@@ -200,7 +200,7 @@ static int mq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
sch = dev_queue->qdisc_sleeping;
if (gnet_stats_copy_basic(d, NULL, &sch->bstats) < 0 ||
- gnet_stats_copy_queue(d, &sch->qstats, sch->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &sch->qstats, sch->q.qlen) < 0)
return -1;
return 0;
}
diff --git a/net/sched/sch_mqprio.c b/net/sched/sch_mqprio.c
index 03dbeb5..3811a74 100644
--- a/net/sched/sch_mqprio.c
+++ b/net/sched/sch_mqprio.c
@@ -356,14 +356,15 @@ static int mqprio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
/* Reclaim root sleeping lock before completing stats */
spin_lock_bh(d->lock);
if (gnet_stats_copy_basic(d, NULL, &bstats) < 0 ||
- gnet_stats_copy_queue(d, &qstats, qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &qstats, qlen) < 0)
return -1;
} else {
struct netdev_queue *dev_queue = mqprio_queue_get(sch, cl);
sch = dev_queue->qdisc_sleeping;
if (gnet_stats_copy_basic(d, NULL, &sch->bstats) < 0 ||
- gnet_stats_copy_queue(d, &sch->qstats, sch->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL,
+ &sch->qstats, sch->q.qlen) < 0)
return -1;
}
return 0;
diff --git a/net/sched/sch_multiq.c b/net/sched/sch_multiq.c
index 53357b3..42dd218 100644
--- a/net/sched/sch_multiq.c
+++ b/net/sched/sch_multiq.c
@@ -361,7 +361,7 @@ static int multiq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
cl_q = q->queues[cl - 1];
if (gnet_stats_copy_basic(d, NULL, &cl_q->bstats) < 0 ||
- gnet_stats_copy_queue(d, &cl_q->qstats, cl_q->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &cl_q->qstats, cl_q->q.qlen) < 0)
return -1;
return 0;
diff --git a/net/sched/sch_prio.c b/net/sched/sch_prio.c
index 4644f55..8e5cd34 100644
--- a/net/sched/sch_prio.c
+++ b/net/sched/sch_prio.c
@@ -325,7 +325,7 @@ static int prio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
cl_q = q->queues[cl - 1];
if (gnet_stats_copy_basic(d, NULL, &cl_q->bstats) < 0 ||
- gnet_stats_copy_queue(d, &cl_q->qstats, cl_q->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL, &cl_q->qstats, cl_q->q.qlen) < 0)
return -1;
return 0;
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 66df9d9..3ec7e88 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -670,7 +670,8 @@ static int qfq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qdisc->qstats, cl->qdisc->q.qlen) < 0)
+ gnet_stats_copy_queue(d, NULL,
+ &cl->qdisc->qstats, cl->qdisc->q.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c
index d4afcbc..b877140 100644
--- a/net/sched/sch_sfq.c
+++ b/net/sched/sch_sfq.c
@@ -871,7 +871,7 @@ static int sfq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
qs.qlen = slot->qlen;
qs.backlog = slot->backlog;
}
- if (gnet_stats_copy_queue(d, &qs, qs.qlen) < 0)
+ if (gnet_stats_copy_queue(d, NULL, &qs, qs.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
}
^ permalink raw reply related
* [net-next PATCH v3 3/4] net: sched: restrict use of qstats qlen
From: John Fastabend @ 2014-09-28 18:53 UTC (permalink / raw)
To: xiyou.wangcong, jhs, eric.dumazet, davem; +Cc: netdev
In-Reply-To: <20140928185255.1976.56179.stgit@nitbit.x32>
This removes the use of qstats->qlen variable from the classifiers
and makes it an explicit argument to gnet_stats_copy_queue().
The qlen represents the qdisc queue length and is packed into
the qstats at the last moment before passnig to user space. By
handling it explicitely we avoid, in the percpu stats case, having
to figure out which per_cpu variable to put it in.
It would probably be best to remove it from qstats completely
but qstats is a user space ABI and can't be broken. A future
patch could make an internal only qstats structure that would
avoid having to allocate an additional u32 variable on the
Qdisc struct. This would make the qstats struct 128bits instead
of 128+32.
Signed-off-by: John Fastabend <john.r.fastabend@intel.com>
---
include/net/gen_stats.h | 3 ++-
net/core/gen_stats.c | 6 +++++-
net/sched/act_api.c | 4 +++-
net/sched/sch_api.c | 5 +++--
net/sched/sch_atm.c | 4 +---
net/sched/sch_cbq.c | 3 +--
net/sched/sch_drr.c | 7 +++----
net/sched/sch_fq_codel.c | 2 +-
net/sched/sch_hfsc.c | 3 +--
net/sched/sch_htb.c | 5 +++--
net/sched/sch_mq.c | 4 +---
net/sched/sch_mqprio.c | 9 ++++-----
net/sched/sch_multiq.c | 3 +--
net/sched/sch_prio.c | 3 +--
net/sched/sch_qfq.c | 3 +--
net/sched/sch_sfq.c | 2 +-
16 files changed, 32 insertions(+), 34 deletions(-)
diff --git a/include/net/gen_stats.h b/include/net/gen_stats.h
index ce3c128..de9b3dd 100644
--- a/include/net/gen_stats.h
+++ b/include/net/gen_stats.h
@@ -40,7 +40,8 @@ void __gnet_stats_copy_basic(struct gnet_stats_basic_packed *bstats,
int gnet_stats_copy_rate_est(struct gnet_dump *d,
const struct gnet_stats_basic_packed *b,
struct gnet_stats_rate_est64 *r);
-int gnet_stats_copy_queue(struct gnet_dump *d, struct gnet_stats_queue *q);
+int gnet_stats_copy_queue(struct gnet_dump *d,
+ struct gnet_stats_queue *q, __u32 len);
int gnet_stats_copy_app(struct gnet_dump *d, void *st, int len);
int gnet_stats_finish_copy(struct gnet_dump *d);
diff --git a/net/core/gen_stats.c b/net/core/gen_stats.c
index 5ff8e80..ad3ecb6 100644
--- a/net/core/gen_stats.c
+++ b/net/core/gen_stats.c
@@ -219,6 +219,7 @@ EXPORT_SYMBOL(gnet_stats_copy_rate_est);
* gnet_stats_copy_queue - copy queue statistics into statistics TLV
* @d: dumping handle
* @q: queue statistics
+ * @qlen: queue length statistics
*
* Appends the queue statistics to the top level TLV created by
* gnet_stats_start_copy().
@@ -227,8 +228,11 @@ EXPORT_SYMBOL(gnet_stats_copy_rate_est);
* if the room in the socket buffer was not sufficient.
*/
int
-gnet_stats_copy_queue(struct gnet_dump *d, struct gnet_stats_queue *q)
+gnet_stats_copy_queue(struct gnet_dump *d,
+ struct gnet_stats_queue *q, __u32 qlen)
{
+ q->qlen = qlen;
+
if (d->compat_tc_stats) {
d->tc_stats.drops = q->drops;
d->tc_stats.qlen = q->qlen;
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index eca4cf9..2e13409 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -623,7 +623,9 @@ int tcf_action_copy_stats(struct sk_buff *skb, struct tc_action *a,
if (gnet_stats_copy_basic(&d, NULL, &p->tcfc_bstats) < 0 ||
gnet_stats_copy_rate_est(&d, &p->tcfc_bstats,
&p->tcfc_rate_est) < 0 ||
- gnet_stats_copy_queue(&d, &p->tcfc_qstats) < 0)
+ gnet_stats_copy_queue(&d,
+ &p->tcfc_qstats,
+ p->tcfc_qstats.qlen) < 0)
goto errout;
if (gnet_stats_finish_copy(&d) < 0)
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index 2862bc6..ca00ea8 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -1318,6 +1318,7 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
unsigned char *b = skb_tail_pointer(skb);
struct gnet_dump d;
struct qdisc_size_table *stab;
+ __u32 qlen;
cond_resched();
nlh = nlmsg_put(skb, portid, seq, event, sizeof(*tcm), flags);
@@ -1335,7 +1336,7 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
goto nla_put_failure;
if (q->ops->dump && q->ops->dump(q, skb) < 0)
goto nla_put_failure;
- q->qstats.qlen = q->q.qlen;
+ qlen = q->q.qlen;
stab = rtnl_dereference(q->stab);
if (stab && qdisc_dump_stab(skb, stab) < 0)
@@ -1353,7 +1354,7 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
if (gnet_stats_copy_basic(&d, cpu_bstats, &q->bstats) < 0 ||
gnet_stats_copy_rate_est(&d, &q->bstats, &q->rate_est) < 0 ||
- gnet_stats_copy_queue(&d, &q->qstats) < 0)
+ gnet_stats_copy_queue(&d, &q->qstats, qlen) < 0)
goto nla_put_failure;
if (gnet_stats_finish_copy(&d) < 0)
diff --git a/net/sched/sch_atm.c b/net/sched/sch_atm.c
index 040212c..c145eb6 100644
--- a/net/sched/sch_atm.c
+++ b/net/sched/sch_atm.c
@@ -637,10 +637,8 @@ atm_tc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
{
struct atm_flow_data *flow = (struct atm_flow_data *)arg;
- flow->qstats.qlen = flow->q->q.qlen;
-
if (gnet_stats_copy_basic(d, NULL, &flow->bstats) < 0 ||
- gnet_stats_copy_queue(d, &flow->qstats) < 0)
+ gnet_stats_copy_queue(d, &flow->qstats, flow->q->q.qlen) < 0)
return -1;
return 0;
diff --git a/net/sched/sch_cbq.c b/net/sched/sch_cbq.c
index 60432c3..c610081 100644
--- a/net/sched/sch_cbq.c
+++ b/net/sched/sch_cbq.c
@@ -1594,7 +1594,6 @@ cbq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
struct cbq_sched_data *q = qdisc_priv(sch);
struct cbq_class *cl = (struct cbq_class *)arg;
- cl->qstats.qlen = cl->q->q.qlen;
cl->xstats.avgidle = cl->avgidle;
cl->xstats.undertime = 0;
@@ -1603,7 +1602,7 @@ cbq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qstats) < 0)
+ gnet_stats_copy_queue(d, &cl->qstats, cl->q->q.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &cl->xstats, sizeof(cl->xstats));
diff --git a/net/sched/sch_drr.c b/net/sched/sch_drr.c
index 907b12f..5835a93 100644
--- a/net/sched/sch_drr.c
+++ b/net/sched/sch_drr.c
@@ -275,17 +275,16 @@ static int drr_dump_class_stats(struct Qdisc *sch, unsigned long arg,
struct gnet_dump *d)
{
struct drr_class *cl = (struct drr_class *)arg;
+ __u32 qlen = cl->qdisc->q.qlen;
struct tc_drr_stats xstats;
memset(&xstats, 0, sizeof(xstats));
- if (cl->qdisc->q.qlen) {
+ if (qlen)
xstats.deficit = cl->deficit;
- cl->qdisc->qstats.qlen = cl->qdisc->q.qlen;
- }
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qdisc->qstats) < 0)
+ gnet_stats_copy_queue(d, &cl->qdisc->qstats, qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index 9270e1b..226d735 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -550,7 +550,7 @@ static int fq_codel_dump_class_stats(struct Qdisc *sch, unsigned long cl,
qs.backlog = q->backlogs[idx];
qs.drops = flow->dropped;
}
- if (gnet_stats_copy_queue(d, &qs) < 0)
+ if (gnet_stats_copy_queue(d, &qs, 0) < 0)
return -1;
if (idx < q->flows_cnt)
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c
index ad27825..d364acb 100644
--- a/net/sched/sch_hfsc.c
+++ b/net/sched/sch_hfsc.c
@@ -1370,7 +1370,6 @@ hfsc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
struct hfsc_class *cl = (struct hfsc_class *)arg;
struct tc_hfsc_stats xstats;
- cl->qstats.qlen = cl->qdisc->q.qlen;
cl->qstats.backlog = cl->qdisc->qstats.backlog;
xstats.level = cl->level;
xstats.period = cl->cl_vtperiod;
@@ -1379,7 +1378,7 @@ hfsc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qstats) < 0)
+ gnet_stats_copy_queue(d, &cl->qstats, cl->qdisc->q.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index c40ab7a..3a691fd 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -1138,15 +1138,16 @@ static int
htb_dump_class_stats(struct Qdisc *sch, unsigned long arg, struct gnet_dump *d)
{
struct htb_class *cl = (struct htb_class *)arg;
+ __u32 qlen = 0;
if (!cl->level && cl->un.leaf.q)
- cl->qstats.qlen = cl->un.leaf.q->q.qlen;
+ qlen = cl->un.leaf.q->q.qlen;
cl->xstats.tokens = PSCHED_NS2TICKS(cl->tokens);
cl->xstats.ctokens = PSCHED_NS2TICKS(cl->ctokens);
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, NULL, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qstats) < 0)
+ gnet_stats_copy_queue(d, &cl->qstats, qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &cl->xstats, sizeof(cl->xstats));
diff --git a/net/sched/sch_mq.c b/net/sched/sch_mq.c
index d3a27fb..6416a69 100644
--- a/net/sched/sch_mq.c
+++ b/net/sched/sch_mq.c
@@ -112,7 +112,6 @@ static int mq_dump(struct Qdisc *sch, struct sk_buff *skb)
sch->q.qlen += qdisc->q.qlen;
sch->bstats.bytes += qdisc->bstats.bytes;
sch->bstats.packets += qdisc->bstats.packets;
- sch->qstats.qlen += qdisc->qstats.qlen;
sch->qstats.backlog += qdisc->qstats.backlog;
sch->qstats.drops += qdisc->qstats.drops;
sch->qstats.requeues += qdisc->qstats.requeues;
@@ -200,9 +199,8 @@ static int mq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
struct netdev_queue *dev_queue = mq_queue_get(sch, cl);
sch = dev_queue->qdisc_sleeping;
- sch->qstats.qlen = sch->q.qlen;
if (gnet_stats_copy_basic(d, NULL, &sch->bstats) < 0 ||
- gnet_stats_copy_queue(d, &sch->qstats) < 0)
+ gnet_stats_copy_queue(d, &sch->qstats, sch->q.qlen) < 0)
return -1;
return 0;
}
diff --git a/net/sched/sch_mqprio.c b/net/sched/sch_mqprio.c
index 8917372..03dbeb5 100644
--- a/net/sched/sch_mqprio.c
+++ b/net/sched/sch_mqprio.c
@@ -236,7 +236,6 @@ static int mqprio_dump(struct Qdisc *sch, struct sk_buff *skb)
sch->q.qlen += qdisc->q.qlen;
sch->bstats.bytes += qdisc->bstats.bytes;
sch->bstats.packets += qdisc->bstats.packets;
- sch->qstats.qlen += qdisc->qstats.qlen;
sch->qstats.backlog += qdisc->qstats.backlog;
sch->qstats.drops += qdisc->qstats.drops;
sch->qstats.requeues += qdisc->qstats.requeues;
@@ -327,6 +326,7 @@ static int mqprio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
if (cl <= netdev_get_num_tc(dev)) {
int i;
+ __u32 qlen = 0;
struct Qdisc *qdisc;
struct gnet_stats_queue qstats = {0};
struct gnet_stats_basic_packed bstats = {0};
@@ -344,9 +344,9 @@ static int mqprio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
qdisc = rtnl_dereference(q->qdisc);
spin_lock_bh(qdisc_lock(qdisc));
+ qlen += qdisc->q.qlen;
bstats.bytes += qdisc->bstats.bytes;
bstats.packets += qdisc->bstats.packets;
- qstats.qlen += qdisc->qstats.qlen;
qstats.backlog += qdisc->qstats.backlog;
qstats.drops += qdisc->qstats.drops;
qstats.requeues += qdisc->qstats.requeues;
@@ -356,15 +356,14 @@ static int mqprio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
/* Reclaim root sleeping lock before completing stats */
spin_lock_bh(d->lock);
if (gnet_stats_copy_basic(d, NULL, &bstats) < 0 ||
- gnet_stats_copy_queue(d, &qstats) < 0)
+ gnet_stats_copy_queue(d, &qstats, qlen) < 0)
return -1;
} else {
struct netdev_queue *dev_queue = mqprio_queue_get(sch, cl);
sch = dev_queue->qdisc_sleeping;
- sch->qstats.qlen = sch->q.qlen;
if (gnet_stats_copy_basic(d, NULL, &sch->bstats) < 0 ||
- gnet_stats_copy_queue(d, &sch->qstats) < 0)
+ gnet_stats_copy_queue(d, &sch->qstats, sch->q.qlen) < 0)
return -1;
}
return 0;
diff --git a/net/sched/sch_multiq.c b/net/sched/sch_multiq.c
index 7f4e1d8..53357b3 100644
--- a/net/sched/sch_multiq.c
+++ b/net/sched/sch_multiq.c
@@ -360,9 +360,8 @@ static int multiq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
struct Qdisc *cl_q;
cl_q = q->queues[cl - 1];
- cl_q->qstats.qlen = cl_q->q.qlen;
if (gnet_stats_copy_basic(d, NULL, &cl_q->bstats) < 0 ||
- gnet_stats_copy_queue(d, &cl_q->qstats) < 0)
+ gnet_stats_copy_queue(d, &cl_q->qstats, cl_q->q.qlen) < 0)
return -1;
return 0;
diff --git a/net/sched/sch_prio.c b/net/sched/sch_prio.c
index b411e78..4644f55 100644
--- a/net/sched/sch_prio.c
+++ b/net/sched/sch_prio.c
@@ -324,9 +324,8 @@ static int prio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
struct Qdisc *cl_q;
cl_q = q->queues[cl - 1];
- cl_q->qstats.qlen = cl_q->q.qlen;
if (gnet_stats_copy_basic(d, NULL, &cl_q->bstats) < 0 ||
- gnet_stats_copy_queue(d, &cl_q->qstats) < 0)
+ gnet_stats_copy_queue(d, &cl_q->qstats, cl_q->q.qlen) < 0)
return -1;
return 0;
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 3fb2655..66df9d9 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -664,14 +664,13 @@ static int qfq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
struct tc_qfq_stats xstats;
memset(&xstats, 0, sizeof(xstats));
- cl->qdisc->qstats.qlen = cl->qdisc->q.qlen;
xstats.weight = cl->agg->class_weight;
xstats.lmax = cl->agg->lmax;
if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
- gnet_stats_copy_queue(d, &cl->qdisc->qstats) < 0)
+ gnet_stats_copy_queue(d, &cl->qdisc->qstats, cl->qdisc->q.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c
index 158dfa6..d4afcbc 100644
--- a/net/sched/sch_sfq.c
+++ b/net/sched/sch_sfq.c
@@ -871,7 +871,7 @@ static int sfq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
qs.qlen = slot->qlen;
qs.backlog = slot->backlog;
}
- if (gnet_stats_copy_queue(d, &qs) < 0)
+ if (gnet_stats_copy_queue(d, &qs, qs.qlen) < 0)
return -1;
return gnet_stats_copy_app(d, &xstats, sizeof(xstats));
}
^ permalink raw reply related
* [net-next PATCH v3 2/4] net: sched: implement qstat helper routines
From: John Fastabend @ 2014-09-28 18:53 UTC (permalink / raw)
To: xiyou.wangcong, jhs, eric.dumazet, davem; +Cc: netdev
In-Reply-To: <20140928185255.1976.56179.stgit@nitbit.x32>
This adds helpers to manipulate qstats logic and replaces locations
that touch the counters directly. This simplifies future patches
to push qstats onto per cpu counters.
Signed-off-by: John Fastabend <john.r.fastabend@intel.com>
---
include/net/sch_generic.h | 39 +++++++++++++++++++++++++++++++++------
net/sched/sch_api.c | 2 +-
net/sched/sch_atm.c | 2 +-
net/sched/sch_cbq.c | 10 +++++-----
net/sched/sch_choke.c | 14 +++++++-------
net/sched/sch_codel.c | 2 +-
net/sched/sch_drr.c | 4 ++--
net/sched/sch_dsmark.c | 2 +-
net/sched/sch_fifo.c | 2 +-
net/sched/sch_fq.c | 4 ++--
net/sched/sch_fq_codel.c | 8 ++++----
net/sched/sch_gred.c | 4 ++--
net/sched/sch_hfsc.c | 8 ++++----
net/sched/sch_hhf.c | 8 ++++----
net/sched/sch_htb.c | 6 +++---
net/sched/sch_ingress.c | 2 +-
net/sched/sch_multiq.c | 4 ++--
net/sched/sch_netem.c | 15 +++++++--------
net/sched/sch_pie.c | 2 +-
net/sched/sch_prio.c | 4 ++--
net/sched/sch_qfq.c | 4 ++--
net/sched/sch_red.c | 8 ++++----
net/sched/sch_sfb.c | 10 +++++-----
net/sched/sch_sfq.c | 17 +++++++++--------
net/sched/sch_tbf.c | 8 ++++----
25 files changed, 108 insertions(+), 81 deletions(-)
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index 4b93511..23a0f0f 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -521,11 +521,38 @@ static inline void qdisc_bstats_update(struct Qdisc *sch,
bstats_update(&sch->bstats, skb);
}
+static inline void qdisc_qstats_backlog_dec(struct Qdisc *sch,
+ const struct sk_buff *skb)
+{
+ sch->qstats.backlog -= qdisc_pkt_len(skb);
+}
+
+static inline void qdisc_qstats_backlog_inc(struct Qdisc *sch,
+ const struct sk_buff *skb)
+{
+ sch->qstats.backlog += qdisc_pkt_len(skb);
+}
+
+static inline void __qdisc_qstats_drop(struct Qdisc *sch, int count)
+{
+ sch->qstats.drops += count;
+}
+
+static inline void qdisc_qstats_drop(struct Qdisc *sch)
+{
+ sch->qstats.drops++;
+}
+
+static inline void qdisc_qstats_overlimit(struct Qdisc *sch)
+{
+ sch->qstats.overlimits++;
+}
+
static inline int __qdisc_enqueue_tail(struct sk_buff *skb, struct Qdisc *sch,
struct sk_buff_head *list)
{
__skb_queue_tail(list, skb);
- sch->qstats.backlog += qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_inc(sch, skb);
return NET_XMIT_SUCCESS;
}
@@ -541,7 +568,7 @@ static inline struct sk_buff *__qdisc_dequeue_head(struct Qdisc *sch,
struct sk_buff *skb = __skb_dequeue(list);
if (likely(skb != NULL)) {
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
qdisc_bstats_update(sch, skb);
}
@@ -560,7 +587,7 @@ static inline unsigned int __qdisc_queue_drop_head(struct Qdisc *sch,
if (likely(skb != NULL)) {
unsigned int len = qdisc_pkt_len(skb);
- sch->qstats.backlog -= len;
+ qdisc_qstats_backlog_dec(sch, skb);
kfree_skb(skb);
return len;
}
@@ -579,7 +606,7 @@ static inline struct sk_buff *__qdisc_dequeue_tail(struct Qdisc *sch,
struct sk_buff *skb = __skb_dequeue_tail(list);
if (likely(skb != NULL))
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
return skb;
}
@@ -661,14 +688,14 @@ static inline unsigned int qdisc_queue_drop(struct Qdisc *sch)
static inline int qdisc_drop(struct sk_buff *skb, struct Qdisc *sch)
{
kfree_skb(skb);
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return NET_XMIT_DROP;
}
static inline int qdisc_reshape_fail(struct sk_buff *skb, struct Qdisc *sch)
{
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
#ifdef CONFIG_NET_CLS_ACT
if (sch->reshape_fail == NULL || sch->reshape_fail(skb, sch))
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index a95e3b4..2862bc6 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -763,7 +763,7 @@ void qdisc_tree_decrease_qlen(struct Qdisc *sch, unsigned int n)
cops->put(sch, cl);
}
sch->q.qlen -= n;
- sch->qstats.drops += drops;
+ __qdisc_qstats_drop(sch, drops);
}
}
EXPORT_SYMBOL(qdisc_tree_decrease_qlen);
diff --git a/net/sched/sch_atm.c b/net/sched/sch_atm.c
index 0101766..040212c 100644
--- a/net/sched/sch_atm.c
+++ b/net/sched/sch_atm.c
@@ -417,7 +417,7 @@ done:
if (ret != NET_XMIT_SUCCESS) {
drop: __maybe_unused
if (net_xmit_drop_count(ret)) {
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
if (flow)
flow->qstats.drops++;
}
diff --git a/net/sched/sch_cbq.c b/net/sched/sch_cbq.c
index 22a3a02..60432c3 100644
--- a/net/sched/sch_cbq.c
+++ b/net/sched/sch_cbq.c
@@ -377,7 +377,7 @@ cbq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
#endif
if (cl == NULL) {
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
}
@@ -395,7 +395,7 @@ cbq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
}
if (net_xmit_drop_count(ret)) {
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
cbq_mark_toplevel(q, cl);
cl->qstats.drops++;
}
@@ -650,11 +650,11 @@ static int cbq_reshape_fail(struct sk_buff *skb, struct Qdisc *child)
return 0;
}
if (net_xmit_drop_count(ret))
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return 0;
}
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return -1;
}
#endif
@@ -995,7 +995,7 @@ cbq_dequeue(struct Qdisc *sch)
*/
if (sch->q.qlen) {
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (q->wd_expires)
qdisc_watchdog_schedule(&q->watchdog,
now + q->wd_expires);
diff --git a/net/sched/sch_choke.c b/net/sched/sch_choke.c
index 8abc262..c009eb9 100644
--- a/net/sched/sch_choke.c
+++ b/net/sched/sch_choke.c
@@ -127,7 +127,7 @@ static void choke_drop_by_idx(struct Qdisc *sch, unsigned int idx)
if (idx == q->tail)
choke_zap_tail_holes(q);
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
qdisc_drop(skb, sch);
qdisc_tree_decrease_qlen(sch, 1);
--sch->q.qlen;
@@ -302,7 +302,7 @@ static int choke_enqueue(struct sk_buff *skb, struct Qdisc *sch)
if (q->vars.qavg > p->qth_max) {
q->vars.qcount = -1;
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (use_harddrop(q) || !use_ecn(q) ||
!INET_ECN_set_ce(skb)) {
q->stats.forced_drop++;
@@ -315,7 +315,7 @@ static int choke_enqueue(struct sk_buff *skb, struct Qdisc *sch)
q->vars.qcount = 0;
q->vars.qR = red_random(p);
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (!use_ecn(q) || !INET_ECN_set_ce(skb)) {
q->stats.prob_drop++;
goto congestion_drop;
@@ -332,7 +332,7 @@ static int choke_enqueue(struct sk_buff *skb, struct Qdisc *sch)
q->tab[q->tail] = skb;
q->tail = (q->tail + 1) & q->tab_mask;
++sch->q.qlen;
- sch->qstats.backlog += qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_inc(sch, skb);
return NET_XMIT_SUCCESS;
}
@@ -345,7 +345,7 @@ congestion_drop:
other_drop:
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
}
@@ -365,7 +365,7 @@ static struct sk_buff *choke_dequeue(struct Qdisc *sch)
q->tab[q->head] = NULL;
choke_zap_head_holes(q);
--sch->q.qlen;
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
qdisc_bstats_update(sch, skb);
return skb;
@@ -460,7 +460,7 @@ static int choke_change(struct Qdisc *sch, struct nlattr *opt)
ntab[tail++] = skb;
continue;
}
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
--sch->q.qlen;
qdisc_drop(skb, sch);
}
diff --git a/net/sched/sch_codel.c b/net/sched/sch_codel.c
index 2f9ab17..de28f8e 100644
--- a/net/sched/sch_codel.c
+++ b/net/sched/sch_codel.c
@@ -149,7 +149,7 @@ static int codel_change(struct Qdisc *sch, struct nlattr *opt)
while (sch->q.qlen > sch->limit) {
struct sk_buff *skb = __skb_dequeue(&sch->q);
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
qdisc_drop(skb, sch);
}
qdisc_tree_decrease_qlen(sch, qlen - sch->q.qlen);
diff --git a/net/sched/sch_drr.c b/net/sched/sch_drr.c
index 7a6243c..907b12f 100644
--- a/net/sched/sch_drr.c
+++ b/net/sched/sch_drr.c
@@ -360,7 +360,7 @@ static int drr_enqueue(struct sk_buff *skb, struct Qdisc *sch)
cl = drr_classify(skb, sch, &err);
if (cl == NULL) {
if (err & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return err;
}
@@ -369,7 +369,7 @@ static int drr_enqueue(struct sk_buff *skb, struct Qdisc *sch)
if (unlikely(err != NET_XMIT_SUCCESS)) {
if (net_xmit_drop_count(err)) {
cl->qstats.drops++;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
}
return err;
}
diff --git a/net/sched/sch_dsmark.c b/net/sched/sch_dsmark.c
index 485e456..227114f 100644
--- a/net/sched/sch_dsmark.c
+++ b/net/sched/sch_dsmark.c
@@ -258,7 +258,7 @@ static int dsmark_enqueue(struct sk_buff *skb, struct Qdisc *sch)
err = qdisc_enqueue(skb, p->q);
if (err != NET_XMIT_SUCCESS) {
if (net_xmit_drop_count(err))
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return err;
}
diff --git a/net/sched/sch_fifo.c b/net/sched/sch_fifo.c
index e15a9eb..2e2398c 100644
--- a/net/sched/sch_fifo.c
+++ b/net/sched/sch_fifo.c
@@ -42,7 +42,7 @@ static int pfifo_tail_enqueue(struct sk_buff *skb, struct Qdisc *sch)
/* queue full, remove one skb to fulfill the limit */
__qdisc_queue_drop_head(sch, &sch->q);
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
qdisc_enqueue_tail(skb, sch);
return NET_XMIT_CN;
diff --git a/net/sched/sch_fq.c b/net/sched/sch_fq.c
index e12f997..c9b9fcb 100644
--- a/net/sched/sch_fq.c
+++ b/net/sched/sch_fq.c
@@ -290,7 +290,7 @@ static struct sk_buff *fq_dequeue_head(struct Qdisc *sch, struct fq_flow *flow)
flow->head = skb->next;
skb->next = NULL;
flow->qlen--;
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
sch->q.qlen--;
}
return skb;
@@ -371,7 +371,7 @@ static int fq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
f->qlen++;
if (skb_is_retransmit(skb))
q->stat_tcp_retrans++;
- sch->qstats.backlog += qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_inc(sch, skb);
if (fq_flow_is_detached(f)) {
fq_flow_add_tail(&q->new_flows, f);
if (time_after(jiffies, f->age + q->flow_refill_delay))
diff --git a/net/sched/sch_fq_codel.c b/net/sched/sch_fq_codel.c
index 105cf55..9270e1b 100644
--- a/net/sched/sch_fq_codel.c
+++ b/net/sched/sch_fq_codel.c
@@ -164,8 +164,8 @@ static unsigned int fq_codel_drop(struct Qdisc *sch)
q->backlogs[idx] -= len;
kfree_skb(skb);
sch->q.qlen--;
- sch->qstats.drops++;
- sch->qstats.backlog -= len;
+ qdisc_qstats_drop(sch);
+ qdisc_qstats_backlog_dec(sch, skb);
flow->dropped++;
return idx;
}
@@ -180,7 +180,7 @@ static int fq_codel_enqueue(struct sk_buff *skb, struct Qdisc *sch)
idx = fq_codel_classify(skb, sch, &ret);
if (idx == 0) {
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
}
@@ -190,7 +190,7 @@ static int fq_codel_enqueue(struct sk_buff *skb, struct Qdisc *sch)
flow = &q->flows[idx];
flow_queue_add(flow, skb);
q->backlogs[idx] += qdisc_pkt_len(skb);
- sch->qstats.backlog += qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_inc(sch, skb);
if (list_empty(&flow->flowchain)) {
list_add_tail(&flow->flowchain, &q->new_flows);
diff --git a/net/sched/sch_gred.c b/net/sched/sch_gred.c
index 12cbc09..a4ca451 100644
--- a/net/sched/sch_gred.c
+++ b/net/sched/sch_gred.c
@@ -209,7 +209,7 @@ static int gred_enqueue(struct sk_buff *skb, struct Qdisc *sch)
break;
case RED_PROB_MARK:
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (!gred_use_ecn(t) || !INET_ECN_set_ce(skb)) {
q->stats.prob_drop++;
goto congestion_drop;
@@ -219,7 +219,7 @@ static int gred_enqueue(struct sk_buff *skb, struct Qdisc *sch)
break;
case RED_HARD_MARK:
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (gred_use_harddrop(t) || !gred_use_ecn(t) ||
!INET_ECN_set_ce(skb)) {
q->stats.forced_drop++;
diff --git a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c
index 209b966..ad27825 100644
--- a/net/sched/sch_hfsc.c
+++ b/net/sched/sch_hfsc.c
@@ -1591,7 +1591,7 @@ hfsc_enqueue(struct sk_buff *skb, struct Qdisc *sch)
cl = hfsc_classify(skb, sch, &err);
if (cl == NULL) {
if (err & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return err;
}
@@ -1600,7 +1600,7 @@ hfsc_enqueue(struct sk_buff *skb, struct Qdisc *sch)
if (unlikely(err != NET_XMIT_SUCCESS)) {
if (net_xmit_drop_count(err)) {
cl->qstats.drops++;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
}
return err;
}
@@ -1643,7 +1643,7 @@ hfsc_dequeue(struct Qdisc *sch)
*/
cl = vttree_get_minvt(&q->root, cur_time);
if (cl == NULL) {
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
hfsc_schedule_watchdog(sch);
return NULL;
}
@@ -1698,7 +1698,7 @@ hfsc_drop(struct Qdisc *sch)
list_move_tail(&cl->dlist, &q->droplist);
}
cl->qstats.drops++;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
sch->q.qlen--;
return len;
}
diff --git a/net/sched/sch_hhf.c b/net/sched/sch_hhf.c
index d85b681..15d3aab 100644
--- a/net/sched/sch_hhf.c
+++ b/net/sched/sch_hhf.c
@@ -376,8 +376,8 @@ static unsigned int hhf_drop(struct Qdisc *sch)
struct sk_buff *skb = dequeue_head(bucket);
sch->q.qlen--;
- sch->qstats.drops++;
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_drop(sch);
+ qdisc_qstats_backlog_dec(sch, skb);
kfree_skb(skb);
}
@@ -395,7 +395,7 @@ static int hhf_enqueue(struct sk_buff *skb, struct Qdisc *sch)
bucket = &q->buckets[idx];
bucket_add(bucket, skb);
- sch->qstats.backlog += qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_inc(sch, skb);
if (list_empty(&bucket->bucketchain)) {
unsigned int weight;
@@ -457,7 +457,7 @@ begin:
if (bucket->head) {
skb = dequeue_head(bucket);
sch->q.qlen--;
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
}
if (!skb) {
diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index 0256dee..c40ab7a 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -586,13 +586,13 @@ static int htb_enqueue(struct sk_buff *skb, struct Qdisc *sch)
#ifdef CONFIG_NET_CLS_ACT
} else if (!cl) {
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
#endif
} else if ((ret = qdisc_enqueue(skb, cl->un.leaf.q)) != NET_XMIT_SUCCESS) {
if (net_xmit_drop_count(ret)) {
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
cl->qstats.drops++;
}
return ret;
@@ -925,7 +925,7 @@ ok:
goto ok;
}
}
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (likely(next_event > q->now)) {
if (!test_bit(__QDISC_STATE_DEACTIVATED,
&qdisc_root_sleeping(q->watchdog.qdisc)->state)) {
diff --git a/net/sched/sch_ingress.c b/net/sched/sch_ingress.c
index b351125..eb5b844 100644
--- a/net/sched/sch_ingress.c
+++ b/net/sched/sch_ingress.c
@@ -69,7 +69,7 @@ static int ingress_enqueue(struct sk_buff *skb, struct Qdisc *sch)
switch (result) {
case TC_ACT_SHOT:
result = TC_ACT_SHOT;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
break;
case TC_ACT_STOLEN:
case TC_ACT_QUEUED:
diff --git a/net/sched/sch_multiq.c b/net/sched/sch_multiq.c
index 4adbf7f..7f4e1d8 100644
--- a/net/sched/sch_multiq.c
+++ b/net/sched/sch_multiq.c
@@ -75,7 +75,7 @@ multiq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
if (qdisc == NULL) {
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
}
@@ -87,7 +87,7 @@ multiq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
return NET_XMIT_SUCCESS;
}
if (net_xmit_drop_count(ret))
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return ret;
}
diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
index 111d70f..b343319 100644
--- a/net/sched/sch_netem.c
+++ b/net/sched/sch_netem.c
@@ -429,12 +429,12 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch)
/* Drop packet? */
if (loss_event(q)) {
if (q->ecn && INET_ECN_set_ce(skb))
- sch->qstats.drops++; /* mark packet */
+ qdisc_qstats_drop(sch); /* mark packet */
else
--count;
}
if (count == 0) {
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return NET_XMIT_SUCCESS | __NET_XMIT_BYPASS;
}
@@ -478,7 +478,7 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch)
if (unlikely(skb_queue_len(&sch->q) >= sch->limit))
return qdisc_reshape_fail(skb, sch);
- sch->qstats.backlog += qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_inc(sch, skb);
cb = netem_skb_cb(skb);
if (q->gap == 0 || /* not doing reordering */
@@ -549,15 +549,14 @@ static unsigned int netem_drop(struct Qdisc *sch)
sch->q.qlen--;
skb->next = NULL;
skb->prev = NULL;
- len = qdisc_pkt_len(skb);
- sch->qstats.backlog -= len;
+ qdisc_qstats_backlog_dec(sch, skb);
kfree_skb(skb);
}
}
if (!len && q->qdisc && q->qdisc->ops->drop)
len = q->qdisc->ops->drop(q->qdisc);
if (len)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return len;
}
@@ -575,7 +574,7 @@ tfifo_dequeue:
skb = __skb_dequeue(&sch->q);
if (skb) {
deliver:
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
qdisc_unthrottled(sch);
qdisc_bstats_update(sch, skb);
return skb;
@@ -610,7 +609,7 @@ deliver:
if (unlikely(err != NET_XMIT_SUCCESS)) {
if (net_xmit_drop_count(err)) {
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
qdisc_tree_decrease_qlen(sch, 1);
}
}
diff --git a/net/sched/sch_pie.c b/net/sched/sch_pie.c
index fefeeb7..33d7a98 100644
--- a/net/sched/sch_pie.c
+++ b/net/sched/sch_pie.c
@@ -232,7 +232,7 @@ static int pie_change(struct Qdisc *sch, struct nlattr *opt)
while (sch->q.qlen > sch->limit) {
struct sk_buff *skb = __skb_dequeue(&sch->q);
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
qdisc_drop(skb, sch);
}
qdisc_tree_decrease_qlen(sch, qlen - sch->q.qlen);
diff --git a/net/sched/sch_prio.c b/net/sched/sch_prio.c
index 68a8f25..b411e78 100644
--- a/net/sched/sch_prio.c
+++ b/net/sched/sch_prio.c
@@ -77,7 +77,7 @@ prio_enqueue(struct sk_buff *skb, struct Qdisc *sch)
if (qdisc == NULL) {
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
}
@@ -89,7 +89,7 @@ prio_enqueue(struct sk_buff *skb, struct Qdisc *sch)
return NET_XMIT_SUCCESS;
}
if (net_xmit_drop_count(ret))
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return ret;
}
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index d59f857..3fb2655 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -1229,7 +1229,7 @@ static int qfq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
cl = qfq_classify(skb, sch, &err);
if (cl == NULL) {
if (err & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return err;
}
@@ -1249,7 +1249,7 @@ static int qfq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
pr_debug("qfq_enqueue: enqueue failed %d\n", err);
if (net_xmit_drop_count(err)) {
cl->qstats.drops++;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
}
return err;
}
diff --git a/net/sched/sch_red.c b/net/sched/sch_red.c
index 633e32d..6c0534c 100644
--- a/net/sched/sch_red.c
+++ b/net/sched/sch_red.c
@@ -74,7 +74,7 @@ static int red_enqueue(struct sk_buff *skb, struct Qdisc *sch)
break;
case RED_PROB_MARK:
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (!red_use_ecn(q) || !INET_ECN_set_ce(skb)) {
q->stats.prob_drop++;
goto congestion_drop;
@@ -84,7 +84,7 @@ static int red_enqueue(struct sk_buff *skb, struct Qdisc *sch)
break;
case RED_HARD_MARK:
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (red_use_harddrop(q) || !red_use_ecn(q) ||
!INET_ECN_set_ce(skb)) {
q->stats.forced_drop++;
@@ -100,7 +100,7 @@ static int red_enqueue(struct sk_buff *skb, struct Qdisc *sch)
sch->q.qlen++;
} else if (net_xmit_drop_count(ret)) {
q->stats.pdrop++;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
}
return ret;
@@ -142,7 +142,7 @@ static unsigned int red_drop(struct Qdisc *sch)
if (child->ops->drop && (len = child->ops->drop(child)) > 0) {
q->stats.other++;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
sch->q.qlen--;
return len;
}
diff --git a/net/sched/sch_sfb.c b/net/sched/sch_sfb.c
index 1562fb2..5819dd8 100644
--- a/net/sched/sch_sfb.c
+++ b/net/sched/sch_sfb.c
@@ -290,7 +290,7 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch)
struct flow_keys keys;
if (unlikely(sch->q.qlen >= q->limit)) {
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
q->stats.queuedrop++;
goto drop;
}
@@ -348,7 +348,7 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch)
sfb_skb_cb(skb)->hashes[slot] = 0;
if (unlikely(minqlen >= q->max)) {
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
q->stats.bucketdrop++;
goto drop;
}
@@ -376,7 +376,7 @@ static int sfb_enqueue(struct sk_buff *skb, struct Qdisc *sch)
}
}
if (sfb_rate_limit(skb, q)) {
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
q->stats.penaltydrop++;
goto drop;
}
@@ -411,7 +411,7 @@ enqueue:
increment_qlen(skb, q);
} else if (net_xmit_drop_count(ret)) {
q->stats.childdrop++;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
}
return ret;
@@ -420,7 +420,7 @@ drop:
return NET_XMIT_CN;
other_drop:
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
}
diff --git a/net/sched/sch_sfq.c b/net/sched/sch_sfq.c
index 80c36bd..158dfa6 100644
--- a/net/sched/sch_sfq.c
+++ b/net/sched/sch_sfq.c
@@ -331,8 +331,8 @@ drop:
sfq_dec(q, x);
kfree_skb(skb);
sch->q.qlen--;
- sch->qstats.drops++;
- sch->qstats.backlog -= len;
+ qdisc_qstats_drop(sch);
+ qdisc_qstats_backlog_dec(sch, skb);
return len;
}
@@ -379,7 +379,7 @@ sfq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
hash = sfq_classify(skb, sch, &ret);
if (hash == 0) {
if (ret & __NET_XMIT_BYPASS)
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
kfree_skb(skb);
return ret;
}
@@ -409,7 +409,7 @@ sfq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
break;
case RED_PROB_MARK:
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (sfq_prob_mark(q)) {
/* We know we have at least one packet in queue */
if (sfq_headdrop(q) &&
@@ -426,7 +426,7 @@ sfq_enqueue(struct sk_buff *skb, struct Qdisc *sch)
goto congestion_drop;
case RED_HARD_MARK:
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
if (sfq_hard_mark(q)) {
/* We know we have at least one packet in queue */
if (sfq_headdrop(q) &&
@@ -461,7 +461,7 @@ congestion_drop:
}
enqueue:
- sch->qstats.backlog += qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_inc(sch, skb);
slot->backlog += qdisc_pkt_len(skb);
slot_queue_add(slot, skb);
sfq_inc(q, x);
@@ -520,7 +520,7 @@ next_slot:
sfq_dec(q, a);
qdisc_bstats_update(sch, skb);
sch->q.qlen--;
- sch->qstats.backlog -= qdisc_pkt_len(skb);
+ qdisc_qstats_backlog_dec(sch, skb);
slot->backlog -= qdisc_pkt_len(skb);
/* Is the slot empty? */
if (slot->qlen == 0) {
@@ -586,7 +586,8 @@ static void sfq_rehash(struct Qdisc *sch)
if (x == SFQ_EMPTY_SLOT) {
x = q->dep[0].next; /* get a free slot */
if (x >= SFQ_MAX_FLOWS) {
-drop: sch->qstats.backlog -= qdisc_pkt_len(skb);
+drop:
+ qdisc_qstats_backlog_dec(sch, skb);
kfree_skb(skb);
dropped++;
continue;
diff --git a/net/sched/sch_tbf.c b/net/sched/sch_tbf.c
index 0c39b75..77edffe 100644
--- a/net/sched/sch_tbf.c
+++ b/net/sched/sch_tbf.c
@@ -175,7 +175,7 @@ static int tbf_segment(struct sk_buff *skb, struct Qdisc *sch)
ret = qdisc_enqueue(segs, q->qdisc);
if (ret != NET_XMIT_SUCCESS) {
if (net_xmit_drop_count(ret))
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
} else {
nb++;
}
@@ -201,7 +201,7 @@ static int tbf_enqueue(struct sk_buff *skb, struct Qdisc *sch)
ret = qdisc_enqueue(skb, q->qdisc);
if (ret != NET_XMIT_SUCCESS) {
if (net_xmit_drop_count(ret))
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
return ret;
}
@@ -216,7 +216,7 @@ static unsigned int tbf_drop(struct Qdisc *sch)
if (q->qdisc->ops->drop && (len = q->qdisc->ops->drop(q->qdisc)) != 0) {
sch->q.qlen--;
- sch->qstats.drops++;
+ qdisc_qstats_drop(sch);
}
return len;
}
@@ -281,7 +281,7 @@ static struct sk_buff *tbf_dequeue(struct Qdisc *sch)
(cf. CSZ, HPFQ, HFSC)
*/
- sch->qstats.overlimits++;
+ qdisc_qstats_overlimit(sch);
}
return NULL;
}
^ permalink raw reply related
* [net-next PATCH v3 1/4] net: sched: make bstats per cpu and estimator RCU safe
From: John Fastabend @ 2014-09-28 18:52 UTC (permalink / raw)
To: xiyou.wangcong, jhs, eric.dumazet, davem; +Cc: netdev
From: John Fastabend <john.fastabend@gmail.com>
In order to run qdisc's without locking statistics and estimators
need to be handled correctly.
To resolve bstats make the statistics per cpu. And because this is
only needed for qdiscs that are running without locks which is not
the case for most qdiscs in the near future only create percpu
stats when qdiscs set the TCQ_F_CPUSTATS flag.
Next because estimators use the bstats to calculate packets per
second and bytes per second the estimator code paths are updated
to use the per cpu statistics.
Signed-off-by: John Fastabend <john.r.fastabend@intel.com>
---
include/net/gen_stats.h | 11 +++++++++
include/net/sch_generic.h | 22 +++++++++++++++++-
net/core/gen_estimator.c | 29 +++++++++++++++---------
net/core/gen_stats.c | 53 ++++++++++++++++++++++++++++++++++++++++----
net/netfilter/xt_RATEEST.c | 2 +-
net/sched/act_api.c | 5 ++--
net/sched/act_police.c | 2 +-
net/sched/sch_api.c | 29 +++++++++++++++++++-----
net/sched/sch_atm.c | 2 +-
net/sched/sch_cbq.c | 7 +++---
net/sched/sch_drr.c | 7 +++---
net/sched/sch_generic.c | 3 ++
net/sched/sch_hfsc.c | 13 +++++++----
net/sched/sch_htb.c | 12 +++++++---
net/sched/sch_mq.c | 2 +-
net/sched/sch_mqprio.c | 4 ++-
net/sched/sch_multiq.c | 2 +-
net/sched/sch_prio.c | 2 +-
net/sched/sch_qfq.c | 8 ++++---
19 files changed, 164 insertions(+), 51 deletions(-)
diff --git a/include/net/gen_stats.h b/include/net/gen_stats.h
index ea4271d..ce3c128 100644
--- a/include/net/gen_stats.h
+++ b/include/net/gen_stats.h
@@ -6,6 +6,11 @@
#include <linux/rtnetlink.h>
#include <linux/pkt_sched.h>
+struct gnet_stats_basic_cpu {
+ struct gnet_stats_basic_packed bstats;
+ struct u64_stats_sync syncp;
+};
+
struct gnet_dump {
spinlock_t * lock;
struct sk_buff * skb;
@@ -27,7 +32,11 @@ int gnet_stats_start_copy_compat(struct sk_buff *skb, int type,
spinlock_t *lock, struct gnet_dump *d);
int gnet_stats_copy_basic(struct gnet_dump *d,
+ struct gnet_stats_basic_cpu __percpu *cpu,
struct gnet_stats_basic_packed *b);
+void __gnet_stats_copy_basic(struct gnet_stats_basic_packed *bstats,
+ struct gnet_stats_basic_cpu __percpu *cpu,
+ struct gnet_stats_basic_packed *b);
int gnet_stats_copy_rate_est(struct gnet_dump *d,
const struct gnet_stats_basic_packed *b,
struct gnet_stats_rate_est64 *r);
@@ -37,11 +46,13 @@ int gnet_stats_copy_app(struct gnet_dump *d, void *st, int len);
int gnet_stats_finish_copy(struct gnet_dump *d);
int gen_new_estimator(struct gnet_stats_basic_packed *bstats,
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats,
struct gnet_stats_rate_est64 *rate_est,
spinlock_t *stats_lock, struct nlattr *opt);
void gen_kill_estimator(struct gnet_stats_basic_packed *bstats,
struct gnet_stats_rate_est64 *rate_est);
int gen_replace_estimator(struct gnet_stats_basic_packed *bstats,
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats,
struct gnet_stats_rate_est64 *rate_est,
spinlock_t *stats_lock, struct nlattr *opt);
bool gen_estimator_active(const struct gnet_stats_basic_packed *bstats,
diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h
index e65b8e0..4b93511 100644
--- a/include/net/sch_generic.h
+++ b/include/net/sch_generic.h
@@ -6,6 +6,7 @@
#include <linux/rcupdate.h>
#include <linux/pkt_sched.h>
#include <linux/pkt_cls.h>
+#include <linux/percpu.h>
#include <net/gen_stats.h>
#include <net/rtnetlink.h>
@@ -58,6 +59,7 @@ struct Qdisc {
* multiqueue device.
*/
#define TCQ_F_WARN_NONWC (1 << 16)
+#define TCQ_F_CPUSTATS 0x20 /* run using percpu statistics */
u32 limit;
const struct Qdisc_ops *ops;
struct qdisc_size_table __rcu *stab;
@@ -83,7 +85,10 @@ struct Qdisc {
*/
unsigned long state;
struct sk_buff_head q;
- struct gnet_stats_basic_packed bstats;
+ union {
+ struct gnet_stats_basic_packed bstats;
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats;
+ } __packed;
unsigned int __state;
struct gnet_stats_queue qstats;
struct rcu_head rcu_head;
@@ -487,6 +492,10 @@ static inline int qdisc_enqueue_root(struct sk_buff *skb, struct Qdisc *sch)
return qdisc_enqueue(skb, sch) & NET_XMIT_MASK;
}
+static inline bool qdisc_is_percpu_stats(const struct Qdisc *q)
+{
+ return q->flags & TCQ_F_CPUSTATS;
+}
static inline void bstats_update(struct gnet_stats_basic_packed *bstats,
const struct sk_buff *skb)
@@ -495,6 +504,17 @@ static inline void bstats_update(struct gnet_stats_basic_packed *bstats,
bstats->packets += skb_is_gso(skb) ? skb_shinfo(skb)->gso_segs : 1;
}
+static inline void qdisc_bstats_update_cpu(struct Qdisc *sch,
+ const struct sk_buff *skb)
+{
+ struct gnet_stats_basic_cpu *bstats =
+ this_cpu_ptr(sch->cpu_bstats);
+
+ u64_stats_update_begin(&bstats->syncp);
+ bstats_update(&bstats->bstats, skb);
+ u64_stats_update_end(&bstats->syncp);
+}
+
static inline void qdisc_bstats_update(struct Qdisc *sch,
const struct sk_buff *skb)
{
diff --git a/net/core/gen_estimator.c b/net/core/gen_estimator.c
index 9d33dff..9dfb88a 100644
--- a/net/core/gen_estimator.c
+++ b/net/core/gen_estimator.c
@@ -91,6 +91,8 @@ struct gen_estimator
u32 avpps;
struct rcu_head e_rcu;
struct rb_node node;
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats;
+ struct rcu_head head;
};
struct gen_estimator_head
@@ -115,9 +117,8 @@ static void est_timer(unsigned long arg)
rcu_read_lock();
list_for_each_entry_rcu(e, &elist[idx].list, list) {
- u64 nbytes;
+ struct gnet_stats_basic_packed b = {0};
u64 brate;
- u32 npackets;
u32 rate;
spin_lock(e->stats_lock);
@@ -125,15 +126,15 @@ static void est_timer(unsigned long arg)
if (e->bstats == NULL)
goto skip;
- nbytes = e->bstats->bytes;
- npackets = e->bstats->packets;
- brate = (nbytes - e->last_bytes)<<(7 - idx);
- e->last_bytes = nbytes;
+ __gnet_stats_copy_basic(&b, e->cpu_bstats, e->bstats);
+
+ brate = (b.bytes - e->last_bytes)<<(7 - idx);
+ e->last_bytes = b.bytes;
e->avbps += (brate >> e->ewma_log) - (e->avbps >> e->ewma_log);
e->rate_est->bps = (e->avbps+0xF)>>5;
- rate = (npackets - e->last_packets)<<(12 - idx);
- e->last_packets = npackets;
+ rate = (b.packets - e->last_packets)<<(12 - idx);
+ e->last_packets = b.packets;
e->avpps += (rate >> e->ewma_log) - (e->avpps >> e->ewma_log);
e->rate_est->pps = (e->avpps+0x1FF)>>10;
skip:
@@ -203,12 +204,14 @@ struct gen_estimator *gen_find_node(const struct gnet_stats_basic_packed *bstats
*
*/
int gen_new_estimator(struct gnet_stats_basic_packed *bstats,
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats,
struct gnet_stats_rate_est64 *rate_est,
spinlock_t *stats_lock,
struct nlattr *opt)
{
struct gen_estimator *est;
struct gnet_estimator *parm = nla_data(opt);
+ struct gnet_stats_basic_packed b = {0};
int idx;
if (nla_len(opt) < sizeof(*parm))
@@ -221,15 +224,18 @@ int gen_new_estimator(struct gnet_stats_basic_packed *bstats,
if (est == NULL)
return -ENOBUFS;
+ __gnet_stats_copy_basic(&b, cpu_bstats, bstats);
+
idx = parm->interval + 2;
est->bstats = bstats;
est->rate_est = rate_est;
est->stats_lock = stats_lock;
est->ewma_log = parm->ewma_log;
- est->last_bytes = bstats->bytes;
+ est->last_bytes = b.bytes;
est->avbps = rate_est->bps<<5;
- est->last_packets = bstats->packets;
+ est->last_packets = b.packets;
est->avpps = rate_est->pps<<10;
+ est->cpu_bstats = cpu_bstats;
spin_lock_bh(&est_tree_lock);
if (!elist[idx].timer.function) {
@@ -290,11 +296,12 @@ EXPORT_SYMBOL(gen_kill_estimator);
* Returns 0 on success or a negative error code.
*/
int gen_replace_estimator(struct gnet_stats_basic_packed *bstats,
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats,
struct gnet_stats_rate_est64 *rate_est,
spinlock_t *stats_lock, struct nlattr *opt)
{
gen_kill_estimator(bstats, rate_est);
- return gen_new_estimator(bstats, rate_est, stats_lock, opt);
+ return gen_new_estimator(bstats, cpu_bstats, rate_est, stats_lock, opt);
}
EXPORT_SYMBOL(gen_replace_estimator);
diff --git a/net/core/gen_stats.c b/net/core/gen_stats.c
index 2ddbce4..5ff8e80 100644
--- a/net/core/gen_stats.c
+++ b/net/core/gen_stats.c
@@ -97,6 +97,43 @@ gnet_stats_start_copy(struct sk_buff *skb, int type, spinlock_t *lock,
}
EXPORT_SYMBOL(gnet_stats_start_copy);
+static void
+__gnet_stats_copy_basic_cpu(struct gnet_stats_basic_packed *bstats,
+ struct gnet_stats_basic_cpu __percpu *cpu)
+{
+ int i;
+
+ for_each_possible_cpu(i) {
+ struct gnet_stats_basic_cpu *bcpu = per_cpu_ptr(cpu, i);
+ unsigned int start;
+ __u64 bytes;
+ __u32 packets;
+
+ do {
+ start = u64_stats_fetch_begin_irq(&bcpu->syncp);
+ bytes = bcpu->bstats.bytes;
+ packets = bcpu->bstats.packets;
+ } while (u64_stats_fetch_retry_irq(&bcpu->syncp, start));
+
+ bstats->bytes += bcpu->bstats.bytes;
+ bstats->packets += bcpu->bstats.packets;
+ }
+}
+
+void
+__gnet_stats_copy_basic(struct gnet_stats_basic_packed *bstats,
+ struct gnet_stats_basic_cpu __percpu *cpu,
+ struct gnet_stats_basic_packed *b)
+{
+ if (cpu) {
+ __gnet_stats_copy_basic_cpu(bstats, cpu);
+ } else {
+ bstats->bytes = b->bytes;
+ bstats->packets = b->packets;
+ }
+}
+EXPORT_SYMBOL(__gnet_stats_copy_basic);
+
/**
* gnet_stats_copy_basic - copy basic statistics into statistic TLV
* @d: dumping handle
@@ -109,19 +146,25 @@ EXPORT_SYMBOL(gnet_stats_start_copy);
* if the room in the socket buffer was not sufficient.
*/
int
-gnet_stats_copy_basic(struct gnet_dump *d, struct gnet_stats_basic_packed *b)
+gnet_stats_copy_basic(struct gnet_dump *d,
+ struct gnet_stats_basic_cpu __percpu *cpu,
+ struct gnet_stats_basic_packed *b)
{
+ struct gnet_stats_basic_packed bstats = {0};
+
+ __gnet_stats_copy_basic(&bstats, cpu, b);
+
if (d->compat_tc_stats) {
- d->tc_stats.bytes = b->bytes;
- d->tc_stats.packets = b->packets;
+ d->tc_stats.bytes = bstats.bytes;
+ d->tc_stats.packets = bstats.packets;
}
if (d->tail) {
struct gnet_stats_basic sb;
memset(&sb, 0, sizeof(sb));
- sb.bytes = b->bytes;
- sb.packets = b->packets;
+ sb.bytes = bstats.bytes;
+ sb.packets = bstats.packets;
return gnet_stats_copy(d, TCA_STATS_BASIC, &sb, sizeof(sb));
}
return 0;
diff --git a/net/netfilter/xt_RATEEST.c b/net/netfilter/xt_RATEEST.c
index 370adf6..604df6f 100644
--- a/net/netfilter/xt_RATEEST.c
+++ b/net/netfilter/xt_RATEEST.c
@@ -136,7 +136,7 @@ static int xt_rateest_tg_checkentry(const struct xt_tgchk_param *par)
cfg.est.interval = info->interval;
cfg.est.ewma_log = info->ewma_log;
- ret = gen_new_estimator(&est->bstats, &est->rstats,
+ ret = gen_new_estimator(&est->bstats, NULL, &est->rstats,
&est->lock, &cfg.opt);
if (ret < 0)
goto err2;
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index 648778a..eca4cf9 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -252,7 +252,8 @@ int tcf_hash_create(u32 index, struct nlattr *est, struct tc_action *a,
p->tcfc_tm.install = jiffies;
p->tcfc_tm.lastuse = jiffies;
if (est) {
- int err = gen_new_estimator(&p->tcfc_bstats, &p->tcfc_rate_est,
+ int err = gen_new_estimator(&p->tcfc_bstats, NULL,
+ &p->tcfc_rate_est,
&p->tcfc_lock, est);
if (err) {
kfree(p);
@@ -619,7 +620,7 @@ int tcf_action_copy_stats(struct sk_buff *skb, struct tc_action *a,
if (err < 0)
goto errout;
- if (gnet_stats_copy_basic(&d, &p->tcfc_bstats) < 0 ||
+ if (gnet_stats_copy_basic(&d, NULL, &p->tcfc_bstats) < 0 ||
gnet_stats_copy_rate_est(&d, &p->tcfc_bstats,
&p->tcfc_rate_est) < 0 ||
gnet_stats_copy_queue(&d, &p->tcfc_qstats) < 0)
diff --git a/net/sched/act_police.c b/net/sched/act_police.c
index f32bcb0..69791ca 100644
--- a/net/sched/act_police.c
+++ b/net/sched/act_police.c
@@ -178,7 +178,7 @@ override:
spin_lock_bh(&police->tcf_lock);
if (est) {
- err = gen_replace_estimator(&police->tcf_bstats,
+ err = gen_replace_estimator(&police->tcf_bstats, NULL,
&police->tcf_rate_est,
&police->tcf_lock, est);
if (err)
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index 15e7bee..a95e3b4 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -942,6 +942,13 @@ qdisc_create(struct net_device *dev, struct netdev_queue *dev_queue,
sch->handle = handle;
if (!ops->init || (err = ops->init(sch, tca[TCA_OPTIONS])) == 0) {
+ if (qdisc_is_percpu_stats(sch)) {
+ sch->cpu_bstats =
+ alloc_percpu(struct gnet_stats_basic_cpu);
+ if (!sch->cpu_bstats)
+ goto err_out4;
+ }
+
if (tca[TCA_STAB]) {
stab = qdisc_get_stab(tca[TCA_STAB]);
if (IS_ERR(stab)) {
@@ -964,8 +971,11 @@ qdisc_create(struct net_device *dev, struct netdev_queue *dev_queue,
else
root_lock = qdisc_lock(sch);
- err = gen_new_estimator(&sch->bstats, &sch->rate_est,
- root_lock, tca[TCA_RATE]);
+ err = gen_new_estimator(&sch->bstats,
+ sch->cpu_bstats,
+ &sch->rate_est,
+ root_lock,
+ tca[TCA_RATE]);
if (err)
goto err_out4;
}
@@ -984,6 +994,7 @@ err_out:
return NULL;
err_out4:
+ free_percpu(sch->cpu_bstats);
/*
* Any broken qdiscs that would require a ops->reset() here?
* The qdisc was never in action so it shouldn't be necessary.
@@ -1022,9 +1033,11 @@ static int qdisc_change(struct Qdisc *sch, struct nlattr **tca)
because change can't be undone. */
if (sch->flags & TCQ_F_MQROOT)
goto out;
- gen_replace_estimator(&sch->bstats, &sch->rate_est,
- qdisc_root_sleeping_lock(sch),
- tca[TCA_RATE]);
+ gen_replace_estimator(&sch->bstats,
+ sch->cpu_bstats,
+ &sch->rate_est,
+ qdisc_root_sleeping_lock(sch),
+ tca[TCA_RATE]);
}
out:
return 0;
@@ -1299,6 +1312,7 @@ graft:
static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
u32 portid, u32 seq, u16 flags, int event)
{
+ struct gnet_stats_basic_cpu __percpu *cpu_bstats = NULL;
struct tcmsg *tcm;
struct nlmsghdr *nlh;
unsigned char *b = skb_tail_pointer(skb);
@@ -1334,7 +1348,10 @@ static int tc_fill_qdisc(struct sk_buff *skb, struct Qdisc *q, u32 clid,
if (q->ops->dump_stats && q->ops->dump_stats(q, &d) < 0)
goto nla_put_failure;
- if (gnet_stats_copy_basic(&d, &q->bstats) < 0 ||
+ if (qdisc_is_percpu_stats(q))
+ cpu_bstats = q->cpu_bstats;
+
+ if (gnet_stats_copy_basic(&d, cpu_bstats, &q->bstats) < 0 ||
gnet_stats_copy_rate_est(&d, &q->bstats, &q->rate_est) < 0 ||
gnet_stats_copy_queue(&d, &q->qstats) < 0)
goto nla_put_failure;
diff --git a/net/sched/sch_atm.c b/net/sched/sch_atm.c
index c398f9c..0101766 100644
--- a/net/sched/sch_atm.c
+++ b/net/sched/sch_atm.c
@@ -639,7 +639,7 @@ atm_tc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
flow->qstats.qlen = flow->q->q.qlen;
- if (gnet_stats_copy_basic(d, &flow->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &flow->bstats) < 0 ||
gnet_stats_copy_queue(d, &flow->qstats) < 0)
return -1;
diff --git a/net/sched/sch_cbq.c b/net/sched/sch_cbq.c
index d2cd981..22a3a02 100644
--- a/net/sched/sch_cbq.c
+++ b/net/sched/sch_cbq.c
@@ -1601,7 +1601,7 @@ cbq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
if (cl->undertime != PSCHED_PASTPERFECT)
cl->xstats.undertime = cl->undertime - q->now;
- if (gnet_stats_copy_basic(d, &cl->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
gnet_stats_copy_queue(d, &cl->qstats) < 0)
return -1;
@@ -1759,7 +1759,8 @@ cbq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, struct nlattr **t
}
if (tca[TCA_RATE]) {
- err = gen_replace_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_replace_estimator(&cl->bstats, NULL,
+ &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE]);
if (err) {
@@ -1852,7 +1853,7 @@ cbq_change_class(struct Qdisc *sch, u32 classid, u32 parentid, struct nlattr **t
goto failure;
if (tca[TCA_RATE]) {
- err = gen_new_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_new_estimator(&cl->bstats, NULL, &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE]);
if (err) {
diff --git a/net/sched/sch_drr.c b/net/sched/sch_drr.c
index d8b5ccf..7a6243c 100644
--- a/net/sched/sch_drr.c
+++ b/net/sched/sch_drr.c
@@ -88,7 +88,8 @@ static int drr_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
if (cl != NULL) {
if (tca[TCA_RATE]) {
- err = gen_replace_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_replace_estimator(&cl->bstats, NULL,
+ &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE]);
if (err)
@@ -116,7 +117,7 @@ static int drr_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
cl->qdisc = &noop_qdisc;
if (tca[TCA_RATE]) {
- err = gen_replace_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_replace_estimator(&cl->bstats, NULL, &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE]);
if (err) {
@@ -282,7 +283,7 @@ static int drr_dump_class_stats(struct Qdisc *sch, unsigned long arg,
cl->qdisc->qstats.qlen = cl->qdisc->q.qlen;
}
- if (gnet_stats_copy_basic(d, &cl->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
gnet_stats_copy_queue(d, &cl->qdisc->qstats) < 0)
return -1;
diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c
index 11b28f6..7c8e5d7 100644
--- a/net/sched/sch_generic.c
+++ b/net/sched/sch_generic.c
@@ -632,6 +632,9 @@ static void qdisc_rcu_free(struct rcu_head *head)
{
struct Qdisc *qdisc = container_of(head, struct Qdisc, rcu_head);
+ if (qdisc_is_percpu_stats(qdisc))
+ free_percpu(qdisc->cpu_bstats);
+
kfree((char *) qdisc - qdisc->padded);
}
diff --git a/net/sched/sch_hfsc.c b/net/sched/sch_hfsc.c
index 04b0de4..209b966 100644
--- a/net/sched/sch_hfsc.c
+++ b/net/sched/sch_hfsc.c
@@ -1014,9 +1014,12 @@ hfsc_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
cur_time = psched_get_time();
if (tca[TCA_RATE]) {
- err = gen_replace_estimator(&cl->bstats, &cl->rate_est,
- qdisc_root_sleeping_lock(sch),
- tca[TCA_RATE]);
+ spinlock_t *lock = qdisc_root_sleeping_lock(sch);
+
+ err = gen_replace_estimator(&cl->bstats, NULL,
+ &cl->rate_est,
+ lock,
+ tca[TCA_RATE]);
if (err)
return err;
}
@@ -1063,7 +1066,7 @@ hfsc_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
return -ENOBUFS;
if (tca[TCA_RATE]) {
- err = gen_new_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_new_estimator(&cl->bstats, NULL, &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE]);
if (err) {
@@ -1374,7 +1377,7 @@ hfsc_dump_class_stats(struct Qdisc *sch, unsigned long arg,
xstats.work = cl->cl_total;
xstats.rtwork = cl->cl_cumul;
- if (gnet_stats_copy_basic(d, &cl->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
gnet_stats_copy_queue(d, &cl->qstats) < 0)
return -1;
diff --git a/net/sched/sch_htb.c b/net/sched/sch_htb.c
index 063e953..0256dee 100644
--- a/net/sched/sch_htb.c
+++ b/net/sched/sch_htb.c
@@ -1144,7 +1144,7 @@ htb_dump_class_stats(struct Qdisc *sch, unsigned long arg, struct gnet_dump *d)
cl->xstats.tokens = PSCHED_NS2TICKS(cl->tokens);
cl->xstats.ctokens = PSCHED_NS2TICKS(cl->ctokens);
- if (gnet_stats_copy_basic(d, &cl->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, NULL, &cl->rate_est) < 0 ||
gnet_stats_copy_queue(d, &cl->qstats) < 0)
return -1;
@@ -1402,7 +1402,8 @@ static int htb_change_class(struct Qdisc *sch, u32 classid,
goto failure;
if (htb_rate_est || tca[TCA_RATE]) {
- err = gen_new_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_new_estimator(&cl->bstats, NULL,
+ &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE] ? : &est.nla);
if (err) {
@@ -1464,8 +1465,11 @@ static int htb_change_class(struct Qdisc *sch, u32 classid,
parent->children++;
} else {
if (tca[TCA_RATE]) {
- err = gen_replace_estimator(&cl->bstats, &cl->rate_est,
- qdisc_root_sleeping_lock(sch),
+ spinlock_t *lock = qdisc_root_sleeping_lock(sch);
+
+ err = gen_replace_estimator(&cl->bstats, NULL,
+ &cl->rate_est,
+ lock,
tca[TCA_RATE]);
if (err)
return err;
diff --git a/net/sched/sch_mq.c b/net/sched/sch_mq.c
index a8b2864..d3a27fb 100644
--- a/net/sched/sch_mq.c
+++ b/net/sched/sch_mq.c
@@ -201,7 +201,7 @@ static int mq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
sch = dev_queue->qdisc_sleeping;
sch->qstats.qlen = sch->q.qlen;
- if (gnet_stats_copy_basic(d, &sch->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &sch->bstats) < 0 ||
gnet_stats_copy_queue(d, &sch->qstats) < 0)
return -1;
return 0;
diff --git a/net/sched/sch_mqprio.c b/net/sched/sch_mqprio.c
index 37e7d25..8917372 100644
--- a/net/sched/sch_mqprio.c
+++ b/net/sched/sch_mqprio.c
@@ -355,7 +355,7 @@ static int mqprio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
}
/* Reclaim root sleeping lock before completing stats */
spin_lock_bh(d->lock);
- if (gnet_stats_copy_basic(d, &bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &bstats) < 0 ||
gnet_stats_copy_queue(d, &qstats) < 0)
return -1;
} else {
@@ -363,7 +363,7 @@ static int mqprio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
sch = dev_queue->qdisc_sleeping;
sch->qstats.qlen = sch->q.qlen;
- if (gnet_stats_copy_basic(d, &sch->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &sch->bstats) < 0 ||
gnet_stats_copy_queue(d, &sch->qstats) < 0)
return -1;
}
diff --git a/net/sched/sch_multiq.c b/net/sched/sch_multiq.c
index c0466c1..4adbf7f 100644
--- a/net/sched/sch_multiq.c
+++ b/net/sched/sch_multiq.c
@@ -361,7 +361,7 @@ static int multiq_dump_class_stats(struct Qdisc *sch, unsigned long cl,
cl_q = q->queues[cl - 1];
cl_q->qstats.qlen = cl_q->q.qlen;
- if (gnet_stats_copy_basic(d, &cl_q->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &cl_q->bstats) < 0 ||
gnet_stats_copy_queue(d, &cl_q->qstats) < 0)
return -1;
diff --git a/net/sched/sch_prio.c b/net/sched/sch_prio.c
index 03ef99e..68a8f25 100644
--- a/net/sched/sch_prio.c
+++ b/net/sched/sch_prio.c
@@ -325,7 +325,7 @@ static int prio_dump_class_stats(struct Qdisc *sch, unsigned long cl,
cl_q = q->queues[cl - 1];
cl_q->qstats.qlen = cl_q->q.qlen;
- if (gnet_stats_copy_basic(d, &cl_q->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &cl_q->bstats) < 0 ||
gnet_stats_copy_queue(d, &cl_q->qstats) < 0)
return -1;
diff --git a/net/sched/sch_qfq.c b/net/sched/sch_qfq.c
index 602ea01..d59f857 100644
--- a/net/sched/sch_qfq.c
+++ b/net/sched/sch_qfq.c
@@ -459,7 +459,8 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
if (cl != NULL) { /* modify existing class */
if (tca[TCA_RATE]) {
- err = gen_replace_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_replace_estimator(&cl->bstats, NULL,
+ &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE]);
if (err)
@@ -484,7 +485,8 @@ static int qfq_change_class(struct Qdisc *sch, u32 classid, u32 parentid,
cl->qdisc = &noop_qdisc;
if (tca[TCA_RATE]) {
- err = gen_new_estimator(&cl->bstats, &cl->rate_est,
+ err = gen_new_estimator(&cl->bstats, NULL,
+ &cl->rate_est,
qdisc_root_sleeping_lock(sch),
tca[TCA_RATE]);
if (err)
@@ -667,7 +669,7 @@ static int qfq_dump_class_stats(struct Qdisc *sch, unsigned long arg,
xstats.weight = cl->agg->class_weight;
xstats.lmax = cl->agg->lmax;
- if (gnet_stats_copy_basic(d, &cl->bstats) < 0 ||
+ if (gnet_stats_copy_basic(d, NULL, &cl->bstats) < 0 ||
gnet_stats_copy_rate_est(d, &cl->bstats, &cl->rate_est) < 0 ||
gnet_stats_copy_queue(d, &cl->qdisc->qstats) < 0)
return -1;
^ permalink raw reply related
* Re: [PATCH net-next] mlx4: optimize xmit path
From: Eric Dumazet @ 2014-09-28 18:52 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Or Gerlitz, David S. Miller, Jesper Dangaard Brouer, Eric Dumazet,
John Fastabend, Linux Netdev List, Amir Vadai, Or Gerlitz
In-Reply-To: <CAMEtUuxwQ8k1wcT8U0VncnVoN=vZjU3zxswgjYPcQTLPw2pRwA@mail.gmail.com>
On Sun, 2014-09-28 at 11:07 -0700, Alexei Starovoitov wrote:
> this is great improvement!
> Thank you for leading this effort.
> 10G line rate is definitely nice :)
> Hopefully Or can demo similar numbers with 40G nic as well :)
>
> > + if (ring->bf_enabled && desc_size <= MAX_BF && !bounce &&
> > + !vlan_tx_tag_present(skb) && send_doorbell) {
>
> feels something wrong here, since it checks for
> send_doorbell, but iowrite() happens in the 'else' part
> of this branch with another 'if (send_doorbell)'
>
If we do not plan to send a doorbell, we should not use blueframe.
Blueframe is always sending a doorbell by design, as it uses a single
flip buffer.
So if you want to see any improvement thanks to skb->xmit_more, we have
to use the iowrite32(), not the blueframe.
Therefore , if send_doorbell == false, we do not want the doorbell
Is it making sense now ? ;)
^ permalink raw reply
* Re: tc rsvp filter show broke
From: Jamal Hadi Salim @ 2014-09-28 18:48 UTC (permalink / raw)
To: John Fastabend, Cong Wang; +Cc: John Fastabend, netdev
In-Reply-To: <54284A51.5080805@gmail.com>
On 09/28/14 13:50, John Fastabend wrote:
> On 09/28/2014 10:15 AM, Cong Wang wrote:
> act->type = exts->type = TCA_OLD_COMPAT;
> list_add(&act->list, &exts->actions);
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
Right.
Note, this exts->xx thing we should eventually kill. It was intended to
allow for old binary compatibility (for about 10 years now).
In the earlier days, policer was a speacial action and one could invoke
it directly from "tc filter ....."
But over time, the more sane way is:
"tc filter .. action police ..."
We have killed ability to compile in the kernel policer to support to
allow old tools to continue to use it at least 5 years ago.
tc still has support but I doubt if anyone uses it. Otherwise there
would have been a lot of fallout from
commit 4bfb21ca2043dc5251102e4debcc0ae27f7304e7 in iproute2.
So if there are any takers for patches, i think the first thing is
to kill every call in iproute2 which calls police directly.
And if nobody whines, then get rid of most of exts->XXX (at minimal
there would be no need for exts->police)
cheers,
jamal
^ permalink raw reply
* Re: [PATCH net-next] mlx4: optimize xmit path
From: Alexei Starovoitov @ 2014-09-28 18:07 UTC (permalink / raw)
To: Eric Dumazet
Cc: Or Gerlitz, David S. Miller, Jesper Dangaard Brouer, Eric Dumazet,
John Fastabend, Linux Netdev List, Amir Vadai, Or Gerlitz
On Sat, Sep 27, 2014 at 3:56 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> First I implemented skb->xmit_more support, and pktgen throughput
> went from ~5Mpps to ~10Mpps.
>
> Then, looking closely at this driver I found false sharing problems that
> should be addressed by this patch, as my pktgen now reaches 14.7 Mpps
> on a single TX queue, with a burst factor of 8.
>
> So this patch in a whole permits to improve raw performance on a single
> TX queue from about 5 Mpps to 14.7 Mpps.
this is great improvement!
Thank you for leading this effort.
10G line rate is definitely nice :)
Hopefully Or can demo similar numbers with 40G nic as well :)
> + if (ring->bf_enabled && desc_size <= MAX_BF && !bounce &&
> + !vlan_tx_tag_present(skb) && send_doorbell) {
feels something wrong here, since it checks for
send_doorbell, but iowrite() happens in the 'else' part
of this branch with another 'if (send_doorbell)'
The previous code seems equally confusing to me.
> + tx_desc->ctrl.bf_qpn = ring->doorbell_qpn |
> + cpu_to_be32(real_size);
>
> op_own |= htonl((bf_index & 0xffff) << 8);
> - /* Ensure new descirptor hits memory
> - * before setting ownership of this descriptor to HW */
> + /* Ensure new descriptor hits memory
> + * before setting ownership of this descriptor to HW
> + */
> wmb();
> tx_desc->ctrl.owner_opcode = op_own;
>
> wmb();
>
> - mlx4_bf_copy(ring->bf.reg + ring->bf.offset, (unsigned long *) &tx_desc->ctrl,
> - desc_size);
> + mlx4_bf_copy(ring->bf.reg + ring->bf.offset,
> + &tx_desc->ctrl,
> + desc_size);
>
> wmb();
>
> ring->bf.offset ^= ring->bf.buf_size;
> } else {
> - /* Ensure new descirptor hits memory
> - * before setting ownership of this descriptor to HW */
> + tx_desc->ctrl.vlan_tag = cpu_to_be16(vlan_tag);
> + tx_desc->ctrl.ins_vlan = MLX4_WQE_CTRL_INS_VLAN *
> + !!vlan_tx_tag_present(skb);
> + tx_desc->ctrl.fence_size = real_size;
> +
> + /* Ensure new descriptor hits memory
> + * before setting ownership of this descriptor to HW
> + */
> wmb();
> tx_desc->ctrl.owner_opcode = op_own;
> - wmb();
> - iowrite32be(ring->doorbell_qpn, ring->bf.uar->map + MLX4_SEND_DOORBELL);
> +
> + if (send_doorbell) {
> + wmb(); /* ensure owner_opcode is written */
> + iowrite32(ring->doorbell_qpn,
> + ring->bf.uar->map + MLX4_SEND_DOORBELL);
> + }
shinfo, prefetch and access_once additions all look useful to me.
Thanks!
^ permalink raw reply
* Re: tc rsvp filter show broke
From: John Fastabend @ 2014-09-28 17:50 UTC (permalink / raw)
To: Cong Wang; +Cc: Jamal Hadi Salim, John Fastabend, netdev
In-Reply-To: <CAM_iQpXzXHc7o1WOtkNnF+io_WZFP+DP4zWm8PZbwX7SC38=Pw@mail.gmail.com>
On 09/28/2014 10:15 AM, Cong Wang wrote:
> On Sun, Sep 28, 2014 at 8:05 AM, Jamal Hadi Salim <jhs@mojatatu.com> wrote:
>> On 09/26/14 19:25, John Fastabend wrote:
>>>
>>> On 09/26/2014 03:48 PM, Jamal Hadi Salim wrote:
>>
>>
>>>> Hrm - poking inside there and this seems to be one of those
>>>> classifiers that hasnt been getting a lot of love. Doesnt
>>>> support actions well for example.
>>>>
>>>> It seems to work for me:
>>>
>>>
>>> This works for me as well try adding two or more policers and
>>> check if they all get printed.
>>>
>>
>> Multiple actions wont work for this classifier. I have time,
>> will send patch.
>>
>
> Hmm, looks like there is a bug in police dump. Please
> try the attached patch.
>
> Thanks.
>
I don't think we need this change, (or perhaps it needs to be a bit
more complete if something is missing) take a look a
tcf_exts_validate(), notice the policer is added to act->list so the
if block in dump() work out,
int tcf_exts_validate(struct net *net, struct tcf_proto *tp, struct
nlattr **tb,
struct nlattr *rate_tlv, struct tcf_exts *exts, bool
ovr)
{
#ifdef CONFIG_NET_CLS_ACT
{
struct tc_action *act;
INIT_LIST_HEAD(&exts->actions);
if (exts->police && tb[exts->police]) {
act = tcf_action_init_1(net, tb[exts->police],
rate_tlv,
"police", ovr,
TCA_ACT_BIND);
if (IS_ERR(act))
return PTR_ERR(act);
act->type = exts->type = TCA_OLD_COMPAT;
list_add(&act->list, &exts->actions);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
} else if (exts->action && tb[exts->action]) {
int err;
err = tcf_action_init(net, tb[exts->action],
rate_tlv,
NULL, ovr,
TCA_ACT_BIND,
&exts->actions);
if (err)
return err;
}
}
[...]
--
John Fastabend Intel Corporation
^ permalink raw reply
* Re: [PATCH 1/1] iproute2: rsvp classifier support for multiple actions
From: John Fastabend @ 2014-09-28 17:39 UTC (permalink / raw)
To: Jamal Hadi Salim; +Cc: stephen, netdev
In-Reply-To: <1411922993-5828-1-git-send-email-jhs@emojatatu.com>
On 09/28/2014 09:49 AM, Jamal Hadi Salim wrote:
> From: Jamal Hadi Salim <jhs@mojatatu.com>
>
> Example setup:
>
> sudo tc qdisc del dev eth0 root handle 1:0 prio
> sudo tc qdisc add dev eth0 root handle 1:0 prio
>
> sudo tc filter add dev eth0 pref 10 proto ip parent 1:0 \
> rsvp session 10.0.0.1 ipproto icmp \
> classid 1:1 \
> action police rate 1kbit burst 90k pipe \
> action ok
>
> tc -s filter show dev eth0 parent 1:0
>
> filter protocol ip pref 10 rsvp
> filter protocol ip pref 10 rsvp fh 0x0001100a flowid 1:1 session
> 10.0.0.1 ipproto icmp
> action order 1: police 0x5 rate 1Kbit burst 23440b mtu 2Kb
> action pipe overhead 0b
> ref 1 bind 1
> Action statistics:
> Sent 98000 bytes 1000 pkt (dropped 0, overlimits 761 requeues 0)
> backlog 0b 0p requeues 0
>
> action order 2: gact action pass
> random type none pass val 0
> index 2 ref 1 bind 1 installed 60 sec used 3 sec
> Action statistics:
> Sent 74578 bytes 761 pkt (dropped 0, overlimits 0 requeues 0)
> backlog 0b 0p requeues 0
>
> Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
> ---
Thanks, this fixes the rsvp syntax.
Tested-by: John Fastabend <john.r.fastabend@intel.com>
--
John Fastabend Intel Corporation
^ permalink raw reply
* Re: tc rsvp filter show broke
From: Cong Wang @ 2014-09-28 17:15 UTC (permalink / raw)
To: Jamal Hadi Salim; +Cc: John Fastabend, John Fastabend, netdev
In-Reply-To: <542823CE.9020301@mojatatu.com>
[-- Attachment #1: Type: text/plain, Size: 657 bytes --]
On Sun, Sep 28, 2014 at 8:05 AM, Jamal Hadi Salim <jhs@mojatatu.com> wrote:
> On 09/26/14 19:25, John Fastabend wrote:
>>
>> On 09/26/2014 03:48 PM, Jamal Hadi Salim wrote:
>
>
>>> Hrm - poking inside there and this seems to be one of those
>>> classifiers that hasnt been getting a lot of love. Doesnt
>>> support actions well for example.
>>>
>>> It seems to work for me:
>>
>>
>> This works for me as well try adding two or more policers and
>> check if they all get printed.
>>
>
> Multiple actions wont work for this classifier. I have time,
> will send patch.
>
Hmm, looks like there is a bug in police dump. Please
try the attached patch.
Thanks.
[-- Attachment #2: cls-fix.diff --]
[-- Type: text/plain, Size: 949 bytes --]
diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c
index c28b0d3..1218900 100644
--- a/net/sched/cls_api.c
+++ b/net/sched/cls_api.c
@@ -576,16 +576,18 @@ int tcf_exts_dump(struct sk_buff *skb, struct tcf_exts *exts)
if (tcf_action_dump(skb, &exts->actions, 0, 0) < 0)
goto nla_put_failure;
nla_nest_end(skb, nest);
- } else if (exts->police) {
- struct tc_action *act = tcf_exts_first_act(exts);
- nest = nla_nest_start(skb, exts->police);
- if (nest == NULL || !act)
- goto nla_put_failure;
- if (tcf_action_dump_old(skb, act, 0, 0) < 0)
- goto nla_put_failure;
- nla_nest_end(skb, nest);
}
}
+
+ if (exts->police) {
+ struct tc_action *act = tcf_exts_first_act(exts);
+ nest = nla_nest_start(skb, exts->police);
+ if (nest == NULL || !act)
+ goto nla_put_failure;
+ if (tcf_action_dump_old(skb, act, 0, 0) < 0)
+ goto nla_put_failure;
+ nla_nest_end(skb, nest);
+ }
return 0;
nla_put_failure:
^ permalink raw reply related
* Re: [PATCH v1 5/5] driver-core: add driver asynchronous probe support
From: Tom Gundersen @ 2014-09-28 17:07 UTC (permalink / raw)
To: Luis R. Rodriguez
Cc: Greg KH, Dmitry Torokhov, Takashi Iwai, Tejun Heo,
Arjan van de Ven, Robert Milasan, werner, Oleg Nesterov, hare,
Benjamin Poirier, Santosh Rastapur, pmladek, dbueso,
Luis R. Rodriguez, LKML, Tetsuo Handa, Joseph Salisbury,
Kay Sievers, One Thousand Gnomes, Tim Gardner, Pierre Fersing,
Andrew Morton, Nagalakshmi Nandigama,
Praveen Krishnamoorthy <praveen
In-Reply-To: <1411768637-6809-6-git-send-email-mcgrof@do-not-panic.com>
Hi Luis,
Thanks for the patches and the detailed analysis.
Feel free to add
Acked-by: Tom Gundersen <teg@jklm.no>
Minor comments on the commit message below.
On Fri, Sep 26, 2014 at 11:57 PM, Luis R. Rodriguez
<mcgrof@do-not-panic.com> wrote:
> From: "Luis R. Rodriguez" <mcgrof@suse.com>
>
> Some init systems may wish to express the desire to have
> device drivers run their device driver's bus probe() run
> asynchronously. This implements support for this and
> allows userspace to request async probe as a preference
> through a generic shared device driver module parameter,
> async_probe. Implemention for async probe is supported
> through a module parameter given that since synchronous
> probe has been prevalent for years some userspace might
> exist which relies on the fact that the device driver will
> probe synchronously and the assumption that devices it
> provides will be immediately available after this.
>
> Some device driver might not be able to run async probe
> so we enable device drivers to annotate this to prevent
> this module parameter from having any effect on them.
>
> This implementation uses queue_work(system_unbound_wq)
> to queue async probes, this should enable probe to run
> slightly *faster* if the driver's probe path did not
> have much interaction with other workqueues otherwise
> it may run _slightly_ slower. Tests were done with cxgb4,
> which is known to take long on probe, both without
> having to run request_firmware() [0] and then by
> requiring it to use request_firmware() [1]. The
> difference in run time are only measurable in microseconds:
>
> =====================================================================|
> strategy fw (usec) no-fw (usec) |
> ---------------------------------------------------------------------|
> synchronous 24472569 1307563 |
> kthread 25066415.5 1309868.5 |
> queue_work(system_unbound_wq) 24913661.5 1307631 |
> ---------------------------------------------------------------------|
>
> In practice, in seconds, the difference is barely noticeable:
>
> =====================================================================|
> strategy fw (s) no-fw (s) |
> ---------------------------------------------------------------------|
> synchronous 24.47 1.31 |
> kthread 25.07 1.31 |
> queue_work(system_unbound_wq) 24.91 1.31 |
> ---------------------------------------------------------------------|
>
> [0] http://ftp.suse.com/pub/people/mcgrof/async-probe/probe-cgxb4-no-firmware.png
> [1] http://ftp.suse.com/pub/people/mcgrof/async-probe/probe-cgxb4-firmware.png
>
> The rest of the commit log documents why this feature was implemented
> primarily first for systemd and things it should consider next.
>
> Systemd has a general timeout for all workers currently set to 180
> seconds after which it will send a sigkill signal. Systemd now has a
> warning which is issued once it reaches 1/3 of the timeout. The original
> motivation for the systemd timeout was to help track device drivers
> which do not use asynch firmware loading on init() and the timeout was
> originally set to 30 seconds.
Please note that the motivation for the timeout in systemd had nothing
to do with async firmware loading (that was just the case where
problems cropped up). The motivation was to not allow udev-workers to
stay around indefinitely, and hence put an upper-bound on
their duration (initially 180 s). At some point the bound was reduced
to 30 seconds to make sure module-loading would bail out before the
kernel's firmware loading timeout would bail out (60s I believe). That
is no longer relevant, which is why it was safe to reset the timeout
to 180 s.
> Since systemd + kernel are heavily tied in for the purposes of this
> patch it is assumed you have merged on systemd the following
> commits:
>
> 671174136525ddf208cdbe75d6d6bd159afa961f udev: timeout - warn after a third of the timeout before killing
> b5338a19864ac3f5632aee48069a669479621dca udev: timeout - increase timeout
> 2e92633dbae52f5ac9b7b2e068935990d475d2cd udev: bump event timeout to 60 seconds
> be2ea723b1d023b3d385d3b791ee4607cbfb20ca udev: remove userspace firmware loading support
> 9f20a8a376f924c8eb5423cfc1f98644fc1e2d1a udev: fixup commit
> dd5eddd28a74a49607a8fffcaf960040dba98479 udev: unify event timeout handling
> 9719859c07aa13539ed2cd4b31972cd30f678543 udevd: add --event-timeout commandline option
>
> Since we bundle together serially driver init() and probe()
> on module initialiation systemd's imposed timeout put a limit on the
> amount of time a driver init() and probe routines can take. There's a
> few overlooked issues with this and the timeout in general:
>
> 0) Not all drivers are killed, the signal is just sent and
> the kill will only be acted upoon if the driver you loaded
> happens to have some code path that either uses kthreads (which
> as of 786235ee are now killable), or uses some code which checks for
> fatal_signal_pending() on the kernel somewhere -- i.e: pci_read_vpd().
Shouldn't this be seen as something to be fixed in the kernel? I mean,
do we not want userspace to have the possibility to kill udev/modprobe
even disregarding the worker timeouts (say at shutdown, or before
switching from the initrd)?
> 1) Since systemd is the only one logging the sigkill debugging that
> drivers are not loaded or in the worst case *failed to boot* because
> of a sigkill has proven hard to debug.
Care to clarify this a bit? Are the udev logs somehow unclear? If you
think we can improve the logging from udev, please ping me about that
and I'll sort it out.
> 2) When and if the signal is received by the driver somehow
> the driver may fail at different points in its initialization
> and unless all error paths on the driver are implemented
> perfectly this could mean leaving a device in a half
> initialized state.
>
> 3) The timeout is penalizing device drivers that take long on
> probe(), this wasn't the original motivation. Systemd seems
> to have been under assumption that probe was asynchronous,
> this perhaps is true as an *objective* and goal for *some
> subsystems* but by no means is it true that we've been on a wide
> crusade to ensure this for all device drivers. It may be a good
> idea for *many* device drivers but penalizing them with a kill
> for taking long on probe is simply unacceptable specially
> when the timeout is completely arbitrary.
The point is really not to "penalize" anything, we just need to make
sure we put some sort of restrictions on our workers so they don't
hang around forever.
> 4) The driver core calls probe for *all* devices that a driver can
> claim and it does so serially, so if a device driver will need
> to probe 3 devices and if probe on the device driver is synchronous
> the amount of time that module loading will take will be:
>
> driver load time = init() + probe for 3 devices serially
>
> The timeout ultimatley ends up limiting the number of devices that
> *any* device driver can support based on the following formula:
>
> number_devices = systemd_timeout
> -------------------------------------
> max known probe time for driver
>
> Lastly since the error value passed down is the value of
> the probe for the last device probed the module will fail
> to load and all devices will fail to be available.
>
> In the Linux kernel we don't want to work around the timeout,
> instead systemd must be changed to take all the above into
> consideration when issuing any kills on device drivers, ideally
> the sigkill should be considered to be ignored at least for
> kmod. In addition to this we help systemd by giving it what it
> originally considered was there and enable it to ask device
> drivers to use asynchronous probe. This patch addresses that
> feature.
>
> Systemd should consider enabling async probe on device drivers
> it loads through systemd-udev but probably does not want to
> enable it for modules loaded through systemd-modules-load
> (modules-load.d). At least on my booting enablign async probe
> for all modules fails to boot as such in order to make this
> a bit more useful we whitelist a few buses where it should be
> at least in theory safe to try to enable async probe. This
> way even if systemd tried to ask to enable async probe for all
> its device drivers the kernel won't blindly do this. We also
> have the sync_probe flag which device drivers can themselves
> enable *iff* its known the device driver should never async
> probe.
>
> In order to help *test* things folks can use the bus.safe_mod_async_probe=1
> kernel parameter which will work as if userspace would have
> requested all modules to load with async probe. Daring folks can
> also use bus.force_mod_async_probe=1 which will enable asynch probe
> even on buses not tested in any way yet, if you use that though
> you're on your own.
>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
> Cc: Joseph Salisbury <joseph.salisbury@canonical.com>
> Cc: Kay Sievers <kay@vrfy.org>
> Cc: One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk>
> Cc: Tim Gardner <tim.gardner@canonical.com>
> Cc: Pierre Fersing <pierre-fersing@pierref.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Benjamin Poirier <bpoirier@suse.de>
> Cc: Nagalakshmi Nandigama <nagalakshmi.nandigama@avagotech.com>
> Cc: Praveen Krishnamoorthy <praveen.krishnamoorthy@avagotech.com>
> Cc: Sreekanth Reddy <sreekanth.reddy@avagotech.com>
> Cc: Abhijit Mahajan <abhijit.mahajan@avagotech.com>
> Cc: Casey Leedom <leedom@chelsio.com>
> Cc: Hariprasad S <hariprasad@chelsio.com>
> Cc: Santosh Rastapur <santosh@chelsio.com>
> Cc: MPT-FusionLinux.pdl@avagotech.com
> Cc: linux-scsi@vger.kernel.org
> Cc: linux-kernel@vger.kernel.org
> Cc: netdev@vger.kernel.org
> Signed-off-by: Luis R. Rodriguez <mcgrof@suse.com>
> ---
> drivers/base/base.h | 6 +++
> drivers/base/bus.c | 137 +++++++++++++++++++++++++++++++++++++++++++++++--
> drivers/base/dd.c | 7 +++
> include/linux/module.h | 2 +
> kernel/module.c | 12 ++++-
> 5 files changed, 159 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/base/base.h b/drivers/base/base.h
> index 251c5d3..24836f1 100644
> --- a/drivers/base/base.h
> +++ b/drivers/base/base.h
> @@ -43,11 +43,17 @@ struct subsys_private {
> };
> #define to_subsys_private(obj) container_of(obj, struct subsys_private, subsys.kobj)
>
> +struct driver_attach_work {
> + struct work_struct work;
> + struct device_driver *driver;
> +};
> +
> struct driver_private {
> struct kobject kobj;
> struct klist klist_devices;
> struct klist_node knode_bus;
> struct module_kobject *mkobj;
> + struct driver_attach_work *attach_work;
> struct device_driver *driver;
> };
> #define to_driver(obj) container_of(obj, struct driver_private, kobj)
> diff --git a/drivers/base/bus.c b/drivers/base/bus.c
> index a5f41e4..41e321e6 100644
> --- a/drivers/base/bus.c
> +++ b/drivers/base/bus.c
> @@ -85,6 +85,7 @@ static void driver_release(struct kobject *kobj)
> struct driver_private *drv_priv = to_driver(kobj);
>
> pr_debug("driver: '%s': %s\n", kobject_name(kobj), __func__);
> + kfree(drv_priv->attach_work);
> kfree(drv_priv);
> }
>
> @@ -662,10 +663,125 @@ static void remove_driver_private(struct device_driver *drv)
> struct driver_private *priv = drv->p;
>
> kobject_put(&priv->kobj);
> + kfree(priv->attach_work);
> kfree(priv);
> drv->p = NULL;
> }
>
> +static void driver_attach_workfn(struct work_struct *work)
> +{
> + int ret;
> + struct driver_attach_work *attach_work =
> + container_of(work, struct driver_attach_work, work);
> + struct device_driver *drv = attach_work->driver;
> + ktime_t calltime, delta, rettime;
> + unsigned long long duration;
> +
> + calltime = ktime_get();
> +
> + ret = driver_attach(drv);
> + if (ret != 0) {
> + remove_driver_private(drv);
> + bus_put(drv->bus);
> + }
> +
> + rettime = ktime_get();
> + delta = ktime_sub(rettime, calltime);
> + duration = (unsigned long long) ktime_to_ns(delta) >> 10;
> +
> + pr_debug("bus: '%s': add driver %s attach completed after %lld usecs\n",
> + drv->bus->name, drv->name, duration);
> +}
> +
> +int bus_driver_async_probe(struct device_driver *drv)
> +{
> + struct driver_private *priv = drv->p;
> +
> + priv->attach_work = kzalloc(sizeof(struct driver_attach_work),
> + GFP_KERNEL);
> + if (!priv->attach_work)
> + return -ENOMEM;
> +
> + priv->attach_work->driver = drv;
> + INIT_WORK(&priv->attach_work->work, driver_attach_workfn);
> +
> + /* Keep this as pr_info() until this is prevalent */
> + pr_info("bus: '%s': probe for driver %s is run asynchronously\n",
> + drv->bus->name, drv->name);
> +
> + queue_work(system_unbound_wq, &priv->attach_work->work);
> +
> + return 0;
> +}
> +
> +/*
> + */
> +static bool safe_mod_async = false;
> +module_param_named(safe_mod_async_probe, safe_mod_async, bool, 0400);
> +MODULE_PARM_DESC(safe_mod_async_probe,
> + "Enable async probe on all modules safely");
> +
> +static bool force_mod_async = false;
> +module_param_named(force_mod_async_probe, force_mod_async, bool, 0400);
> +MODULE_PARM_DESC(force_mod_async_probe,
> + "Force async probe on all modules");
> +
> +/**
> + * drv_enable_async_probe - evaluates if async probe should be used
> + * @drv: device driver to evaluate
> + * @bus: the bus for the device driver
> + *
> + * The driver core supports enabling asynchronous probe on device drivers
> + * by requiring userspace to pass the module parameter "async_probe".
> + * Currently only modules are enabled to use this feature. If a device
> + * driver is known to not work properly with asynchronous probe they
> + * can force disable asynchronous probe from being enabled through
> + * userspace by adding setting sync_probe to true on the @drv. We require
> + * async probe to be requested from userspace given that we have historically
> + * supported synchronous probe and some userspaces may exist which depend
> + * on this functionality. Userspace may wish to use asynchronous probe for
> + * most device drivers but since this can fail boot in practice we only
> + * enable it currently for a set of buses.
> + *
> + * If you'd like to test enabling async probe for all buses whitelisted
> + * you can enable the safe_mod_async_probe module parameter. Note that its
> + * not a good idea to always enable this, in particular you probably don't
> + * want drivers under modules-load.d to use this. This module parameter should
> + * only be used to help test. If you'd like to test even futher you can
> + * use force_mod_async_probe, that will force enable async probe on all
> + * drivers, regardless if its bus type, it should however be used with
> + * caution.
> + */
> +static bool drv_enable_async_probe(struct device_driver *drv,
> + struct bus_type *bus)
> +{
> + struct module *mod;
> +
> + if (!drv->owner || drv->sync_probe)
> + return false;
> +
> + if (force_mod_async)
> + return true;
> +
> + mod = drv->owner;
> + if (!safe_mod_async && !mod->async_probe_requested)
> + return false;
> +
> + /* For now lets avoid stupid bug reports */
> + if (!strcmp(bus->name, "pci") ||
> + !strcmp(bus->name, "pci_express") ||
> + !strcmp(bus->name, "hid") ||
> + !strcmp(bus->name, "sdio") ||
> + !strcmp(bus->name, "gameport") ||
> + !strcmp(bus->name, "mmc") ||
> + !strcmp(bus->name, "i2c") ||
> + !strcmp(bus->name, "platform") ||
> + !strcmp(bus->name, "usb"))
> + return true;
> +
> + return false;
> +}
> +
> /**
> * bus_add_driver - Add a driver to the bus.
> * @drv: driver.
> @@ -675,6 +791,7 @@ int bus_add_driver(struct device_driver *drv)
> struct bus_type *bus;
> struct driver_private *priv;
> int error = 0;
> + bool async_probe = false;
>
> bus = bus_get(drv->bus);
> if (!bus)
> @@ -696,11 +813,19 @@ int bus_add_driver(struct device_driver *drv)
> if (error)
> goto out_unregister;
>
> + async_probe = drv_enable_async_probe(drv, bus);
> +
> klist_add_tail(&priv->knode_bus, &bus->p->klist_drivers);
> if (drv->bus->p->drivers_autoprobe) {
> - error = driver_attach(drv);
> - if (error)
> - goto out_unregister;
> + if (async_probe) {
> + error = bus_driver_async_probe(drv);
> + if (error)
> + goto out_unregister;
> + } else {
> + error = driver_attach(drv);
> + if (error)
> + goto out_unregister;
> + }
> }
> module_add_driver(drv->owner, drv);
>
> @@ -1267,6 +1392,12 @@ EXPORT_SYMBOL_GPL(subsys_virtual_register);
>
> int __init buses_init(void)
> {
> + if (unlikely(safe_mod_async))
> + pr_info("Enabled safe_mod_async -- you may run into issues\n");
> +
> + if (unlikely(force_mod_async))
> + pr_info("Enabling force_mod_async -- you're on your own!\n");
> +
> bus_kset = kset_create_and_add("bus", &bus_uevent_ops, NULL);
> if (!bus_kset)
> return -ENOMEM;
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index e4ffbcf..7999aba 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -507,6 +507,13 @@ static void __device_release_driver(struct device *dev)
>
> drv = dev->driver;
> if (drv) {
> + if (drv->owner && !drv->sync_probe) {
> + struct module *mod = drv->owner;
> + struct driver_private *priv = drv->p;
> +
> + if (mod->async_probe_requested)
> + flush_work(&priv->attach_work->work);
> + }
> pm_runtime_get_sync(dev);
>
> driver_sysfs_remove(dev);
> diff --git a/include/linux/module.h b/include/linux/module.h
> index 71f282a..1e9e017 100644
> --- a/include/linux/module.h
> +++ b/include/linux/module.h
> @@ -271,6 +271,8 @@ struct module {
> bool sig_ok;
> #endif
>
> + bool async_probe_requested;
> +
> /* symbols that will be GPL-only in the near future. */
> const struct kernel_symbol *gpl_future_syms;
> const unsigned long *gpl_future_crcs;
> diff --git a/kernel/module.c b/kernel/module.c
> index 88f3d6c..31d71ff 100644
> --- a/kernel/module.c
> +++ b/kernel/module.c
> @@ -3175,8 +3175,16 @@ out:
> static int unknown_module_param_cb(char *param, char *val, const char *modname,
> void *arg)
> {
> + int ret;
> + struct module *mod = arg;
> +
> + if (strcmp(param, "async_probe") == 0) {
> + mod->async_probe_requested = true;
> + return 0;
> + }
> +
> /* Check for magic 'dyndbg' arg */
> - int ret = ddebug_dyndbg_module_param_cb(param, val, modname);
> + ret = ddebug_dyndbg_module_param_cb(param, val, modname);
> if (ret != 0)
> pr_warn("%s: unknown parameter '%s' ignored\n", modname, param);
> return 0;
> @@ -3278,7 +3286,7 @@ static int load_module(struct load_info *info, const char __user *uargs,
>
> /* Module is ready to execute: parsing args may do that. */
> after_dashes = parse_args(mod->name, mod->args, mod->kp, mod->num_kp,
> - -32768, 32767, NULL,
> + -32768, 32767, mod,
> unknown_module_param_cb);
> if (IS_ERR(after_dashes)) {
> err = PTR_ERR(after_dashes);
> --
> 2.1.0
>
^ permalink raw reply
* [PATCH 1/1] iproute2: rsvp classifier support for multiple actions
From: Jamal Hadi Salim @ 2014-09-28 16:49 UTC (permalink / raw)
To: stephen; +Cc: john.fastabend, netdev, Jamal Hadi Salim
From: Jamal Hadi Salim <jhs@mojatatu.com>
Example setup:
sudo tc qdisc del dev eth0 root handle 1:0 prio
sudo tc qdisc add dev eth0 root handle 1:0 prio
sudo tc filter add dev eth0 pref 10 proto ip parent 1:0 \
rsvp session 10.0.0.1 ipproto icmp \
classid 1:1 \
action police rate 1kbit burst 90k pipe \
action ok
tc -s filter show dev eth0 parent 1:0
filter protocol ip pref 10 rsvp
filter protocol ip pref 10 rsvp fh 0x0001100a flowid 1:1 session
10.0.0.1 ipproto icmp
action order 1: police 0x5 rate 1Kbit burst 23440b mtu 2Kb
action pipe overhead 0b
ref 1 bind 1
Action statistics:
Sent 98000 bytes 1000 pkt (dropped 0, overlimits 761 requeues 0)
backlog 0b 0p requeues 0
action order 2: gact action pass
random type none pass val 0
index 2 ref 1 bind 1 installed 60 sec used 3 sec
Action statistics:
Sent 74578 bytes 761 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
---
tc/f_rsvp.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/tc/f_rsvp.c b/tc/f_rsvp.c
index 44d3bdf..14f24b1 100644
--- a/tc/f_rsvp.c
+++ b/tc/f_rsvp.c
@@ -261,6 +261,13 @@ static int rsvp_parse_opt(struct filter_util *qu, char *handle, int argc, char *
}
pinfo.tunnelhdr = tid;
pinfo_ok++;
+ } else if (matches(*argv, "action") == 0) {
+ NEXT_ARG();
+ if (parse_action(&argc, &argv, TCA_RSVP_ACT, n)) {
+ fprintf(stderr, "Illegal \"action\"\n");
+ return -1;
+ }
+ continue;
} else if (matches(*argv, "police") == 0) {
NEXT_ARG();
if (parse_police(&argc, &argv, TCA_RSVP_POLICE, n)) {
@@ -385,6 +392,10 @@ static int rsvp_print_opt(struct filter_util *qu, FILE *f, struct rtattr *opt, _
SPRINT_BUF(b2);
fprintf(f, "sender [NONE]%s ", sprint_spi(&pinfo->spi, 0, b2));
}
+
+ if (tb[TCA_RSVP_ACT]) {
+ tc_print_action(f, tb[TCA_RSVP_ACT]);
+ }
if (tb[TCA_RSVP_POLICE])
tc_print_police(f, tb[TCA_RSVP_POLICE]);
return 0;
--
1.7.9.5
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox