From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DE48C282CE for ; Mon, 8 Apr 2019 05:53:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E0EB82083E for ; Mon, 8 Apr 2019 05:53:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UNvIX/vh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726591AbfDHFxG (ORCPT ); Mon, 8 Apr 2019 01:53:06 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:33755 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726343AbfDHFxG (ORCPT ); Mon, 8 Apr 2019 01:53:06 -0400 Received: by mail-qt1-f193.google.com with SMTP id k14so14093029qtb.0 for ; Sun, 07 Apr 2019 22:53:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=aUejlwqhp1wkY+UDZpFso8OedkK7yuwhEj7pLQHsX+Q=; b=UNvIX/vhiQ+yWfiPqhRFWOALO8eLnX+3i/3lbx9Ejrd5ETSmegyFzBgG9gvyVTGfhH 7pwR8b09M0L8IrkQV6VYTrbytTb9ske4gfHc6zkhOCRfZtjkuMUCULdxeOwycJZs+fxI GJX7191UuYpmYuSrEKvHHi7en6bI5l14M3OsK1uxn+oWTdLZ8+LfHmO+ntxmPMo0czX5 zgYZK4RhxRyMym20nstWkKzScrz8299dsBtwWUvCdSu8elMyL5ZR5tLtBQAhpKejUNWI S4MSfFuzipJlK/3Zydy2fetrsWGT1f+77+08HuKquiIq3Ew1M5yTcouxNalUNgLCbIoy t+WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=aUejlwqhp1wkY+UDZpFso8OedkK7yuwhEj7pLQHsX+Q=; b=RhQ53EKI/+wXMqiqEb5ikzcgDf9eBEKp1VRY1lVp8ZraPZuV23q8v07GRPNJN0tt6L FvbVgfKvpXZt6WGD0fwKbCP9+3JV5D8o44YHs2cDxi3SDHHgNrGjcsBQVUElqT4pRrEm hiNbQ6hN7xyHQUZQ34oj4LCgMsqQ/041D98yVlHlFJGObCTmJED+hV4zYNXfzQ8tVpr3 UCT+DmYFvPdJRW6fb5Hd1g7oPBz4eWFpReIANncMNxrqMJwupV7IiqU9wWj9PCLokcR2 k/+IkVyRypj31e1WV4iJFmleni7Es2lzp4B0OjRqQP2Hz4MyJ+leJvKtY2Ilr9/BS6Wd DnnQ== X-Gm-Message-State: APjAAAWWfhszRMXuwU0AI7ZbkxwZw03xXy3efiGsLGZlTkHMjy4wi4kl yzRu0zvqrkh1OSjUfaRt12a0NBvBlELJv7QKHaVnhEhKtfw= X-Google-Smtp-Source: APXvYqzWvRy+RRE8cCdeowJu+qAwEWodlKQn/YYWN6hbgpwkG1ZU2DwXM2O+P+yB7R9NMW/H3XiosFy0J93yT/K8kYY= X-Received: by 2002:a0c:f806:: with SMTP id r6mr21261864qvn.188.1554702784890; Sun, 07 Apr 2019 22:53:04 -0700 (PDT) MIME-Version: 1.0 References: <20190331174005.5841-1-gautamramk@gmail.com> <20190331174005.5841-3-gautamramk@gmail.com> <87bm1optqu.fsf@toke.dk> <87k1gcnwt6.fsf@toke.dk> In-Reply-To: From: Dave Taht Date: Mon, 8 Apr 2019 07:52:51 +0200 Message-ID: Subject: Re: [RFC net-next 2/2] net: sched: fq_pie: Flow Queue PIE AQM To: Gautam Ramakrishnan Cc: =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= , Jamal Hadi Salim , "David S. Miller" , Linux Kernel Network Developers , "Mohit P . Tahiliani" , "Sachin D . Patil" , Mohit Bhasi , "V . Saicharan" , Leslie Monis Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Mon, Apr 8, 2019 at 7:37 AM Dave Taht wrote: > > On Mon, Apr 8, 2019 at 7:31 AM Gautam Ramakrishnan = wrote: > > > > I was trying to refactor the code and I ran into some issues. > > 1. I moved some of the parameters such as flows_cnt into a new struct > > called fq_pie_params, instead of keeping them in fq_pie_sched_data. > > Should I move those parameters back into fq_pie_sched_data? > > 2. fq_codel maintains the backlog variable as a list in the > > fq_codel_sched_data, whereas I maintain a backlog in the struct > > fq_pie_flow. What approach should I follow? > > Hmm. I had made some efforts to speed up fq_codel here: > https://github.com/dtaht/fq_codel_fast > > based on ideas here: > https://lists.bufferbloat.net/pipermail/codel/2018-August/002367.html > > Notably I wanted to get rid of the non O(1) bulk dropper search and > keep the flow backlog closer to the flow stats. > Needed oprofiling to see if it helped. I'm sorry to conflate your fq_pie upstreaming attempt with my own long out of tree improvements for fq_codel. I'd been meaning to upstream multiple fixes from there for ages, but needed to profile them on mips first. Are you in a hurry? Because: hacking in SCE into pie and fq_pie, etc, is WAY more interesting than mainlining stuff right now. :) Our updated "some congestion experienced" internet draft with suggestions for pie and red is here: https://github.com/dtaht/bufferbloat-rfcs/blob/master/sce/draft-morton-taht= -tsvwg-sce.txt#L272 and running code for sce is in that fq_codel_fast repo and in the out of tree sch_cake also: https://github.com/dtaht/sch_cake/commit/755ba4cda56bec45d20f7928f1442fdf83= c12573 > > 3. Would maintaining a per flow pie_stats be useful in the future? I > > do not use the per flow stats anywhere in the code. > > In general I have not found per flow stats useful or accessible and it > was one of the things I eliminated in fq_codel_fast > > On Tue, Apr 2, 2019 at 10:55 PM Toke H=C3=B8iland-J=C3=B8rgensen wrote: > > > > > > Gautam Ramakrishnan writes: > > > > > > > Hello, thanks for the feedback > > > > > > > > On Tue, Apr 2, 2019 at 4:19 PM Toke H=C3=B8iland-J=C3=B8rgensen wrote: > > > >> > > > >> Some suggestions below to make fq_pie and fq_codel more similar (r= ef. my > > > >> previous email). > > > >> > > > >> Also, a few unrelated nits. > > > >> > > > >> > From: Mohit P. Tahiliani > > > >> > > > > >> > FQ-PIE incorporates fair/flow queuing in which every queue > > > >> > is managed by an instance of PIE queueing discipline. > > > >> > The algorithm provides good control over the queueing delay > > > >> > while at the same time, ensures fairness across various > > > >> > flows sharing the same link. > > > >> > > > > >> > Principles: > > > >> > - Packets are classified stochastically on flows. > > > >> > - Each flow has a PIE managed queue. > > > >> > - Flows are linked onto two (Round Robin) lists, > > > >> > so that new flows have priority on old ones. > > > >> > - For a given flow, packets are dropped on arrival according > > > >> > to a drop probability. > > > >> > - Tail drops only. > > > >> > > > >> Why tail drops only? > > > > > > > > I had mentioned this because packets are dropped only at enqueue. > > > > > > Yup, realise that; was just wondering why you went with this design > > > instead of doing the head drop that fq_codel does? > > > > > > >> > > > >> > Usage: > > > >> > tc qdisc ... fq_pie [ limit PACKETS ] [ flows NUMBER ] > > > >> > [ alpha NUMBER ] [ beta NUMBER ] > > > >> > [ target TIME us ] [ tupdate TIME us ] > > > >> > [ bytemode ] [ quantum BYTES ] > > > >> > [ ecn | ecn_prob PERCENTAGE ] > > > >> > > > > >> > defaults: 1024 flows, 10240 packets limit, quantum : device MTU > > > >> > target: 15ms > > > >> > tupdate: 15ms > > > >> > alpha: 2 (on a scale of 0 to 16) > > > >> > beta: 20 (on a scale of 0 to 16) > > > >> > ecn: false > > > >> > ecn_prob: 10% > > > >> > > > > >> > Signed-off-by: Mohit P. Tahiliani > > > >> > Signed-off-by: Sachin D. Patil > > > >> > Signed-off-by: Mohit Bhasi > > > >> > Signed-off-by: V. Saicharan > > > >> > Signed-off-by: Leslie Monis > > > >> > Signed-off-by: Gautam Ramakrishnan > > > >> > Cc: Dave Taht > > > >> > --- > > > >> > include/uapi/linux/pkt_sched.h | 28 ++ > > > >> > net/sched/Kconfig | 14 +- > > > >> > net/sched/Makefile | 1 + > > > >> > net/sched/sch_fq_pie.c | 485 ++++++++++++++++++++++++++= +++++++ > > > >> > 4 files changed, 527 insertions(+), 1 deletion(-) > > > >> > create mode 100644 net/sched/sch_fq_pie.c > > > >> > > > > >> > diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux= /pkt_sched.h > > > >> > index 7ee74c3474bf..005413bd09ee 100644 > > > >> > --- a/include/uapi/linux/pkt_sched.h > > > >> > +++ b/include/uapi/linux/pkt_sched.h > > > >> > @@ -964,6 +964,34 @@ struct tc_pie_xstats { > > > >> > __u32 ecn_mark; /* packets marked with ecn*/ > > > >> > }; > > > >> > > > > >> > +/* FQ PIE */ > > > >> > +enum { > > > >> > + TCA_FQ_PIE_UNSPEC, > > > >> > + TCA_FQ_PIE_TARGET, > > > >> > + TCA_FQ_PIE_LIMIT, > > > >> > + TCA_FQ_PIE_TUPDATE, > > > >> > + TCA_FQ_PIE_ALPHA, > > > >> > + TCA_FQ_PIE_BETA, > > > >> > + TCA_FQ_PIE_ECN, > > > >> > + TCA_FQ_PIE_QUANTUM, > > > >> > + TCA_FQ_PIE_BYTEMODE, > > > >> > + TCA_FQ_PIE_FLOWS, > > > >> > + TCA_FQ_PIE_ECN_PROB, > > > >> > + __TCA_FQ_PIE_MAX > > > >> > +}; > > > >> > +#define TCA_FQ_PIE_MAX (__TCA_FQ_PIE_MAX - 1) > > > >> > + > > > >> > +struct tc_fq_pie_xstats { > > > >> > + __u32 packets_in; /* total number of packets enqueue= d */ > > > >> > + __u32 dropped; /* packets dropped due to fq_pie_a= ction */ > > > >> > + __u32 overlimit; /* dropped due to lack of space in= queue */ > > > >> > + __u32 ecn_mark; /* packets marked with ecn*/ > > > >> > + __u32 new_flow_count; /* number of time packets > > > >> > + created a 'new flow' */ > > > >> > + __u32 new_flows_len; /* count of flows in new list */ > > > >> > + __u32 old_flows_len; /* count of flows in old list */ > > > >> > +}; > > > >> > + > > > >> > /* CBS */ > > > >> > struct tc_cbs_qopt { > > > >> > __u8 offload; > > > >> > diff --git a/net/sched/Kconfig b/net/sched/Kconfig > > > >> > index 5c02ad97ef23..49f4dd9894a0 100644 > > > >> > --- a/net/sched/Kconfig > > > >> > +++ b/net/sched/Kconfig > > > >> > @@ -358,13 +358,25 @@ config NET_SCH_PIE > > > >> > help > > > >> > Say Y here if you want to use the Proportional Integral = controller > > > >> > Enhanced scheduler packet scheduling algorithm. > > > >> > - For more information, please see https://tools.ietf.org/= html/rfc8033 > > > >> > + For more information, please see > > > >> > + http://tools.ietf.org/html/draft-pan-tsvwg-pie-00 > > > >> > > > > >> > To compile this driver as a module, choose M here: the m= odule > > > >> > will be called sch_pie. > > > >> > > > > >> > If unsure, say N. > > > >> > > > > >> > +config NET_SCH_FQ_PIE > > > >> > + tristate "Flow Queue Proportional Integral controller Enha= nced (FQ-PIE) scheduler" > > > >> > + help > > > >> > + Say Y here if you want to use the Flow Queue Proportiona= l Integral controller > > > >> > + Enhanced scheduler packet scheduling algorithm. > > > >> > + > > > >> > + To compile this driver as a module, choose M here: the m= odule > > > >> > + will be called sch_fq_pie. > > > >> > + > > > >> > + If unsure, say N. > > > >> > + > > > >> > config NET_SCH_INGRESS > > > >> > tristate "Ingress/classifier-action Qdisc" > > > >> > depends on NET_CLS_ACT > > > >> > diff --git a/net/sched/Makefile b/net/sched/Makefile > > > >> > index 8a40431d7b5c..fdcd3f7b2fb2 100644 > > > >> > --- a/net/sched/Makefile > > > >> > +++ b/net/sched/Makefile > > > >> > @@ -55,6 +55,7 @@ obj-$(CONFIG_NET_SCH_CAKE) +=3D sch_cake.o > > > >> > obj-$(CONFIG_NET_SCH_FQ) +=3D sch_fq.o > > > >> > obj-$(CONFIG_NET_SCH_HHF) +=3D sch_hhf.o > > > >> > obj-$(CONFIG_NET_SCH_PIE) +=3D sch_pie.o > > > >> > +obj-$(CONFIG_NET_SCH_FQ_PIE) +=3D sch_fq_pie.o > > > >> > obj-$(CONFIG_NET_SCH_CBS) +=3D sch_cbs.o > > > >> > obj-$(CONFIG_NET_SCH_ETF) +=3D sch_etf.o > > > >> > obj-$(CONFIG_NET_SCH_TAPRIO) +=3D sch_taprio.o > > > >> > diff --git a/net/sched/sch_fq_pie.c b/net/sched/sch_fq_pie.c > > > >> > new file mode 100644 > > > >> > index 000000000000..4ccefa0bc7f0 > > > >> > --- /dev/null > > > >> > +++ b/net/sched/sch_fq_pie.c > > > >> > @@ -0,0 +1,485 @@ > > > >> > +/* > > > >> > + * net/sched/sch_fq_pie.c > > > >> > + * > > > >> > + * This program is free software; you can redistribute it and/o= r > > > >> > + * modify it under the terms of the GNU General Public License > > > >> > + * as published by the Free Software Foundation; either version= 2 > > > >> > + * of the License. > > > >> > + * > > > >> > + * This program is distributed in the hope that it will be usef= ul, > > > >> > + * but WITHOUT ANY WARRANTY; without even the implied warranty = of > > > >> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See th= e > > > >> > + * GNU General Public License for more details. > > > >> > > > >> Lose the license boilerplate and replace it with a SPDX header lin= e. > > > > > > > > I shall do that. > > > > > > > >> > > > >> > + * Author: Mohit P. Tahiliani > > > >> > + * Author: Sachin D. Patil > > > >> > + * Author: Mohit Bhasi > > > >> > + * Author: V Saicharan > > > >> > + * Author: Leslie Monis > > > >> > + * Author: Gautam Ramakrishnan > > > >> > + * > > > >> > + * References: > > > >> > + * RFC 8033: https://tools.ietf.org/html/rfc8033 > > > >> > + */ > > > >> > + > > > >> > +#include > > > >> > +#include > > > >> > +#include > > > >> > + > > > >> > +struct fq_pie_params { > > > >> > + struct pie_params p_params; > > > >> > + u32 ecn_prob; > > > >> > + u32 flows_cnt; > > > >> > +}; > > > >> > + > > > >> > +struct fq_pie_stats { > > > >> > + u32 packets_in; /* total number of packets enqueue= d */ > > > >> > + u32 dropped; /* packets dropped due to fq_pie a= ction */ > > > >> > + u32 overlimit; /* dropped due to lack of space in= queue */ > > > >> > + u32 ecn_mark; /* packets marked with ECN */ > > > >> > + u32 new_flow_count; /* number of time packets created = a new flow */ > > > >> > +}; > > > >> > + > > > >> > +struct fq_pie_flow { > > > >> > + s32 deficit; /* number of credits remaining for= the flow */ > > > >> > + u32 backlog; /* size of data in the flow */ > > > >> > + u32 qlen; /* number of packets in the flow *= / > > > >> > + struct sk_buff *head; > > > >> > + struct sk_buff *tail; > > > >> > + struct list_head flowchain; > > > >> > + struct pie_vars vars; /* pie vars for the flow */ > > > >> > + struct pie_stats stats; /* pie stats for the flow */ > > > >> > +}; > > > >> > + > > > >> > +struct fq_pie_sched_data { > > > >> > + u32 quantum; /* number of credits in deficit ro= und robin */ > > > >> > + struct fq_pie_flow *flows; > > > >> > + struct Qdisc *sch; > > > >> > + struct fq_pie_params params; > > > >> > + struct fq_pie_stats stats; > > > >> > + struct list_head old_flows; > > > >> > + struct list_head new_flows; > > > >> > + struct timer_list adapt_timer; > > > >> > +}; > > > >> > > > >> The flow and sched_data structs have quite a bit in common with th= ose in > > > >> fq_codel; but the members are in a different order. > > > >> > > > >> > +static void fq_pie_params_init(struct fq_pie_params *params) > > > >> > +{ > > > >> > + pie_params_init(¶ms->p_params); > > > >> > + params->ecn_prob =3D 10; > > > >> > + params->flows_cnt =3D 1024; > > > >> > +} > > > >> > + > > > >> > +static inline void flow_queue_add(struct fq_pie_flow *flow, > > > >> > + struct sk_buff *skb) > > > >> > +{ > > > >> > + if (!flow->head) > > > >> > + flow->head =3D skb; > > > >> > + else > > > >> > + flow->tail->next =3D skb; > > > >> > + flow->tail =3D skb; > > > >> > + skb->next =3D NULL; > > > >> > +} > > > >> > + > > > >> > +static int fq_pie_qdisc_enqueue(struct sk_buff *skb, struct Qdi= sc *sch, > > > >> > + struct sk_buff **to_free) > > > >> > +{ > > > >> > + struct fq_pie_sched_data *q =3D qdisc_priv(sch); > > > >> > + struct fq_pie_flow *sel_flow; > > > >> > + u32 pkt_len; > > > >> > + u32 idx; > > > >> > + u8 enqueue =3D false; > > > >> > + > > > >> > + /* Classifies packet into corresponding flow */ > > > >> > + idx =3D reciprocal_scale(skb_get_hash(skb), q->params.flow= s_cnt); > > > >> > + sel_flow =3D &q->flows[idx]; > > > >> > > > >> This is missing the ability to override the classification from tc > > > >> filters. See fq_codel_classify(). > > > > > > > > I wanted to keep it simple initially. Shall add that. > > > > > > > >> > > > >> > + > > > >> > + /* Checks if the qdisc is full */ > > > >> > + if (unlikely(qdisc_qlen(sch) >=3D sch->limit)) { > > > >> > + q->stats.overlimit++; > > > >> > + sel_flow->stats.overlimit++; > > > >> > + goto out; > > > >> > + } > > > >> > > > >> The memory_limit checks in fq_codel have turned out to be quite us= eful > > > >> on constrained systems. I'd suggest adding them here as well. > > > > > > > > I shall add that too. > > > > > > > >> > > > >> > + > > > >> > + if (!drop_early(sch, sel_flow->backlog, skb->len, &sel_flo= w->vars, > > > >> > + &q->params.p_params)) { > > > >> > + enqueue =3D true; > > > >> > + } else if (q->params.p_params.ecn && > > > >> > + sel_flow->vars.prob <=3D > > > >> > + (MAX_PROB / 100) * q->params.ecn_prob && > > > >> > + INET_ECN_set_ce(skb)) { > > > >> > + /* If packet is ecn capable, mark it if drop proba= bility > > > >> > + * is lower than the parameter ecn_prob, else drop= it. > > > >> > + */ > > > >> > + q->stats.ecn_mark++; > > > >> > + sel_flow->stats.ecn_mark++; > > > >> > + enqueue =3D true; > > > >> > + } > > > >> > + if (enqueue) { > > > >> > + pkt_len =3D qdisc_pkt_len(skb); > > > >> > + q->stats.packets_in++; > > > >> > + sch->qstats.backlog +=3D pkt_len; > > > >> > + sch->q.qlen++; > > > >> > + flow_queue_add(sel_flow, skb); > > > >> > + if (list_empty(&sel_flow->flowchain)) { > > > >> > + list_add_tail(&sel_flow->flowchain, &q->ne= w_flows); > > > >> > + q->stats.new_flow_count++; > > > >> > + sel_flow->deficit =3D q->quantum; > > > >> > + sel_flow->stats.dropped =3D 0; > > > >> > + sel_flow->qlen =3D 0; > > > >> > + sel_flow->backlog =3D 0; > > > >> > + } > > > >> > + sel_flow->qlen++; > > > >> > + sel_flow->stats.packets_in++; > > > >> > + sel_flow->backlog +=3D pkt_len; > > > >> > + return NET_XMIT_SUCCESS; > > > >> > + } > > > >> > +out: > > > >> > + q->stats.dropped++; > > > >> > + sel_flow->stats.dropped++; > > > >> > + return qdisc_drop(skb, sch, to_free); > > > >> > > > >> You probably want to return NET_XMIT_CN instead of NET_XMIT_DROP h= ere. > > > >> > > > > > > > > Should I replace qdisc_drop with __qdisc_drop and return NET_XMIT_C= N? > > > > > > Yeah, I think that would be more appropriate here, as that would > > > directly throttle sockets on the same host. > > > > > > >> > +} > > > >> > + > > > >> > +static const struct nla_policy fq_pie_policy[TCA_FQ_PIE_MAX + 1= ] =3D { > > > >> > + [TCA_FQ_PIE_TARGET] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_LIMIT] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_TUPDATE] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_ALPHA] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_BETA] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_ECN] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_QUANTUM] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_BYTEMODE] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_FLOWS] =3D {.type =3D NLA_U32}, > > > >> > + [TCA_FQ_PIE_ECN_PROB] =3D {.type =3D NLA_U32} > > > >> > +}; > > > >> > + > > > >> > +static inline struct sk_buff *dequeue_head(struct fq_pie_flow *= flow) > > > >> > +{ > > > >> > + struct sk_buff *skb =3D flow->head; > > > >> > + > > > >> > + flow->head =3D skb->next; > > > >> > + skb->next =3D NULL; > > > >> > + return skb; > > > >> > +} > > > >> > + > > > >> > +static struct sk_buff *fq_pie_qdisc_dequeue(struct Qdisc *sch) > > > >> > +{ > > > >> > + struct fq_pie_sched_data *q =3D qdisc_priv(sch); > > > >> > + struct sk_buff *skb =3D NULL; > > > >> > + struct fq_pie_flow *flow; > > > >> > + struct list_head *head; > > > >> > + u32 pkt_len; > > > >> > + > > > >> > +begin: > > > >> > + head =3D &q->new_flows; > > > >> > + if (list_empty(head)) { > > > >> > + head =3D &q->old_flows; > > > >> > + if (list_empty(head)) > > > >> > + return NULL; > > > >> > + } > > > >> > + > > > >> > + flow =3D list_first_entry(head, struct fq_pie_flow, flowch= ain); > > > >> > + /* Flow has exhausted all its credits */ > > > >> > + if (flow->deficit <=3D 0) { > > > >> > + flow->deficit +=3D q->quantum; > > > >> > + list_move_tail(&flow->flowchain, &q->old_flows); > > > >> > + goto begin; > > > >> > + } > > > >> > + > > > >> > + if (flow->head) { > > > >> > + skb =3D dequeue_head(flow); > > > >> > + pkt_len =3D qdisc_pkt_len(skb); > > > >> > + sch->qstats.backlog -=3D pkt_len; > > > >> > + sch->q.qlen--; > > > >> > + qdisc_bstats_update(sch, skb); > > > >> > + } > > > >> > > > >> If you factor this out into a dequeue_func(), this dequeue() funct= ion is > > > >> very close to identical to the one in fq_codel(). > > > > > > > > Do you suggest I put the if(flow->head) block in a different functi= on? > > > > > > Yeah, that was what I meant. However, without doing the full refactor= ing > > > it's only borderline useful, so feel free to just keep it the way it > > > is... > > > > > > -Toke > > > > > > > > -- > > ------------- > > Gautam | > > > > -- > > Dave T=C3=A4ht > CTO, TekLibre, LLC > http://www.teklibre.com > Tel: 1-831-205-9740 --=20 Dave T=C3=A4ht CTO, TekLibre, LLC http://www.teklibre.com Tel: 1-831-205-9740