* [PATCH v1 net-next 03/14] net: ipv4: Hook into time based transmission
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri, Richard Cochran,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Add a transmit_time field to struct inet_cork, then copy the
timestamp from the CMSG cookie at ip_setup_cork() so we can
safely copy it into the skb later during __ip_make_skb().
For the raw fast path, just perform the copy at raw_send_hdrinc().
Signed-off-by: Richard Cochran <rcochran@linutronix.de>
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
include/net/inet_sock.h | 1 +
net/ipv4/ip_output.c | 3 +++
net/ipv4/raw.c | 2 ++
net/ipv4/udp.c | 1 +
4 files changed, 7 insertions(+)
diff --git a/include/net/inet_sock.h b/include/net/inet_sock.h
index 83d5b3c2ac42..314be484c696 100644
--- a/include/net/inet_sock.h
+++ b/include/net/inet_sock.h
@@ -148,6 +148,7 @@ struct inet_cork {
__s16 tos;
char priority;
__u16 gso_size;
+ u64 transmit_time;
};
struct inet_cork_full {
diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c
index b3308e9d9762..904a54a090e9 100644
--- a/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -1153,6 +1153,7 @@ static int ip_setup_cork(struct sock *sk, struct inet_cork *cork,
cork->tos = ipc->tos;
cork->priority = ipc->priority;
cork->tx_flags = ipc->tx_flags;
+ cork->transmit_time = ipc->sockc.transmit_time;
return 0;
}
@@ -1413,6 +1414,7 @@ struct sk_buff *__ip_make_skb(struct sock *sk,
skb->priority = (cork->tos != -1) ? cork->priority: sk->sk_priority;
skb->mark = sk->sk_mark;
+ skb->tstamp = cork->transmit_time;
/*
* Steal rt from cork.dst to avoid a pair of atomic_inc/atomic_dec
* on dst refcount
@@ -1495,6 +1497,7 @@ struct sk_buff *ip_make_skb(struct sock *sk,
cork->flags = 0;
cork->addr = 0;
cork->opt = NULL;
+ cork->transmit_time = 0;
err = ip_setup_cork(sk, cork, ipc, rtp);
if (err)
return ERR_PTR(err);
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index abb3c9490c55..446af7be2b55 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -381,6 +381,7 @@ static int raw_send_hdrinc(struct sock *sk, struct flowi4 *fl4,
skb->priority = sk->sk_priority;
skb->mark = sk->sk_mark;
+ skb->tstamp = sockc->transmit_time;
skb_dst_set(skb, &rt->dst);
*rtp = NULL;
@@ -562,6 +563,7 @@ static int raw_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
}
ipc.sockc.tsflags = sk->sk_tsflags;
+ ipc.sockc.transmit_time = 0;
ipc.addr = inet->inet_saddr;
ipc.opt = NULL;
ipc.tx_flags = 0;
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 9bb27df4dac5..0ab2c13bc7a1 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -978,6 +978,7 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
}
ipc.sockc.tsflags = sk->sk_tsflags;
+ ipc.sockc.transmit_time = 0;
ipc.addr = inet->inet_saddr;
ipc.oif = sk->sk_bound_dev_if;
ipc.gso_size = up->gso_size;
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 06/14] net/sched: Introduce the ETF Qdisc
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
From: Vinicius Costa Gomes <vinicius.gomes@intel.com>
The ETF (Earliest TxTime First) qdisc uses the information added
earlier in this series (the socket option SO_TXTIME and the new
role of sk_buff->tstamp) to schedule packets transmission based
on absolute time.
For some workloads, just bandwidth enforcement is not enough, and
precise control of the transmission of packets is necessary.
Example:
$ tc qdisc replace dev enp2s0 parent root handle 100 mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 queues 1@0 1@1 2@2 hw 0
$ tc qdisc add dev enp2s0 parent 100:1 etf delta 100000 \
clockid CLOCK_REALTIME
In this example, the Qdisc will provide SW best-effort for the control
of the transmission time to the network adapter, the time stamp in the
socket will be in reference to the clockid CLOCK_REALTIME and packets
will leave the qdisc "delta" (100000) nanoseconds before its transmission
time.
The ETF qdisc will buffer packets sorted by their txtime. It will drop
packets on enqueue() if their skbuff clockid does not match the clock
reference of the Qdisc. Moreover, on dequeue(), a packet will be dropped
if it expires while being enqueued.
The qdisc also supports the SO_TXTIME deadline mode. For this mode, it
will dequeue a packet as soon as possible and change the skb timestamp
to 'now' during etf_dequeue().
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
---
include/linux/netdevice.h | 1 +
include/uapi/linux/pkt_sched.h | 17 ++
net/sched/Kconfig | 11 +
net/sched/Makefile | 1 +
net/sched/sch_etf.c | 385 +++++++++++++++++++++++++++++++++
5 files changed, 415 insertions(+)
create mode 100644 net/sched/sch_etf.c
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index c6b377a15869..7f650bdc6ec3 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -793,6 +793,7 @@ enum tc_setup_type {
TC_SETUP_QDISC_RED,
TC_SETUP_QDISC_PRIO,
TC_SETUP_QDISC_MQ,
+ TC_SETUP_QDISC_ETF,
};
/* These structures hold the attributes of bpf state that are being passed
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index 37b5096ae97b..9d6fd2004a03 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -934,4 +934,21 @@ enum {
#define TCA_CBS_MAX (__TCA_CBS_MAX - 1)
+
+/* ETF */
+struct tc_etf_qopt {
+ __s32 delta;
+ __s32 clockid;
+ __u32 flags;
+#define TC_ETF_DEADLINE_MODE_ON BIT(0)
+};
+
+enum {
+ TCA_ETF_UNSPEC,
+ TCA_ETF_PARMS,
+ __TCA_ETF_MAX,
+};
+
+#define TCA_ETF_MAX (__TCA_ETF_MAX - 1)
+
#endif
diff --git a/net/sched/Kconfig b/net/sched/Kconfig
index a01169fb5325..fcc89706745b 100644
--- a/net/sched/Kconfig
+++ b/net/sched/Kconfig
@@ -183,6 +183,17 @@ config NET_SCH_CBS
To compile this code as a module, choose M here: the
module will be called sch_cbs.
+config NET_SCH_ETF
+ tristate "Earliest TxTime First (ETF)"
+ help
+ Say Y here if you want to use the Earliest TxTime First (ETF) packet
+ scheduling algorithm.
+
+ See the top of <file:net/sched/sch_etf.c> for more details.
+
+ To compile this code as a module, choose M here: the
+ module will be called sch_etf.
+
config NET_SCH_GRED
tristate "Generic Random Early Detection (GRED)"
---help---
diff --git a/net/sched/Makefile b/net/sched/Makefile
index 8811d3804878..9a5a7077d217 100644
--- a/net/sched/Makefile
+++ b/net/sched/Makefile
@@ -54,6 +54,7 @@ obj-$(CONFIG_NET_SCH_FQ) += sch_fq.o
obj-$(CONFIG_NET_SCH_HHF) += sch_hhf.o
obj-$(CONFIG_NET_SCH_PIE) += sch_pie.o
obj-$(CONFIG_NET_SCH_CBS) += sch_cbs.o
+obj-$(CONFIG_NET_SCH_ETF) += sch_etf.o
obj-$(CONFIG_NET_CLS_U32) += cls_u32.o
obj-$(CONFIG_NET_CLS_ROUTE4) += cls_route.o
diff --git a/net/sched/sch_etf.c b/net/sched/sch_etf.c
new file mode 100644
index 000000000000..5f01a285f399
--- /dev/null
+++ b/net/sched/sch_etf.c
@@ -0,0 +1,385 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/* net/sched/sch_etf.c Earliest TxTime First queueing discipline.
+ *
+ * Authors: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
+ * Vinicius Costa Gomes <vinicius.gomes@intel.com>
+ */
+
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <linux/errno.h>
+#include <linux/rbtree.h>
+#include <linux/skbuff.h>
+#include <linux/posix-timers.h>
+#include <net/netlink.h>
+#include <net/sch_generic.h>
+#include <net/pkt_sched.h>
+#include <net/sock.h>
+
+#define DEADLINE_MODE_IS_ON(x) ((x)->flags & TC_ETF_DEADLINE_MODE_ON)
+
+struct etf_sched_data {
+ bool deadline_mode;
+ int clockid;
+ int queue;
+ s32 delta; /* in ns */
+ ktime_t last; /* The txtime of the last skb sent to the netdevice. */
+ struct rb_root head;
+ struct qdisc_watchdog watchdog;
+ ktime_t (*get_time)(void);
+};
+
+static const struct nla_policy etf_policy[TCA_ETF_MAX + 1] = {
+ [TCA_ETF_PARMS] = { .len = sizeof(struct tc_etf_qopt) },
+};
+
+static inline int validate_input_params(struct tc_etf_qopt *qopt,
+ struct netlink_ext_ack *extack)
+{
+ /* Check if params comply to the following rules:
+ * * Clockid and delta must be valid.
+ *
+ * * Dynamic clockids are not supported.
+ *
+ * * Delta must be a positive integer.
+ */
+ if (qopt->clockid < 0) {
+ NL_SET_ERR_MSG(extack, "Dynamic clockids are not supported");
+ return -ENOTSUPP;
+ }
+
+ if (qopt->clockid >= MAX_CLOCKS) {
+ NL_SET_ERR_MSG(extack, "Invalid clockid");
+ return -EINVAL;
+ }
+
+ if (qopt->delta < 0) {
+ NL_SET_ERR_MSG(extack, "Delta must be positive");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static bool is_packet_valid(struct Qdisc *sch, struct sk_buff *nskb)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ ktime_t txtime = nskb->tstamp;
+ struct sock *sk = nskb->sk;
+ ktime_t now;
+
+ if (!sk)
+ return false;
+
+ if (!sock_flag(sk, SOCK_TXTIME))
+ return false;
+
+ /* We don't perform crosstimestamping.
+ * Drop if packet's clockid differs from qdisc's.
+ */
+ if (sk->sk_clockid != q->clockid)
+ return false;
+
+ if ((sk->sk_txtime_flags & SK_TXTIME_DEADLINE_MASK) !=
+ q->deadline_mode)
+ return false;
+
+ now = q->get_time();
+ if (ktime_before(txtime, now) || ktime_before(txtime, q->last))
+ return false;
+
+ return true;
+}
+
+static struct sk_buff *etf_peek_timesortedlist(struct Qdisc *sch)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ struct rb_node *p;
+
+ p = rb_first(&q->head);
+ if (!p)
+ return NULL;
+
+ return rb_to_skb(p);
+}
+
+static void reset_watchdog(struct Qdisc *sch)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ struct sk_buff *skb = etf_peek_timesortedlist(sch);
+ ktime_t next;
+
+ if (!skb)
+ return;
+
+ next = ktime_sub_ns(skb->tstamp, q->delta);
+ qdisc_watchdog_schedule_ns(&q->watchdog, ktime_to_ns(next));
+}
+
+static int etf_enqueue_timesortedlist(struct sk_buff *nskb, struct Qdisc *sch,
+ struct sk_buff **to_free)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ struct rb_node **p = &q->head.rb_node, *parent = NULL;
+ ktime_t txtime = nskb->tstamp;
+
+ if (!is_packet_valid(sch, nskb))
+ return qdisc_drop(nskb, sch, to_free);
+
+ while (*p) {
+ struct sk_buff *skb;
+
+ parent = *p;
+ skb = rb_to_skb(parent);
+ if (ktime_after(txtime, skb->tstamp))
+ p = &parent->rb_right;
+ else
+ p = &parent->rb_left;
+ }
+ rb_link_node(&nskb->rbnode, parent, p);
+ rb_insert_color(&nskb->rbnode, &q->head);
+
+ qdisc_qstats_backlog_inc(sch, nskb);
+ sch->q.qlen++;
+
+ /* Now we may need to re-arm the qdisc watchdog for the next packet. */
+ reset_watchdog(sch);
+
+ return NET_XMIT_SUCCESS;
+}
+
+static void timesortedlist_erase(struct Qdisc *sch, struct sk_buff *skb,
+ bool drop)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+
+ rb_erase(&skb->rbnode, &q->head);
+
+ /* The rbnode field in the skb re-uses these fields, now that
+ * we are done with the rbnode, reset them.
+ */
+ skb->next = NULL;
+ skb->prev = NULL;
+ skb->dev = qdisc_dev(sch);
+
+ qdisc_qstats_backlog_dec(sch, skb);
+
+ if (drop) {
+ struct sk_buff *to_free = NULL;
+
+ qdisc_drop(skb, sch, &to_free);
+ kfree_skb_list(to_free);
+ qdisc_qstats_overlimit(sch);
+ } else {
+ qdisc_bstats_update(sch, skb);
+
+ q->last = skb->tstamp;
+ }
+
+ sch->q.qlen--;
+}
+
+static struct sk_buff *etf_dequeue_timesortedlist(struct Qdisc *sch)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ struct sk_buff *skb;
+ ktime_t now, next;
+
+ skb = etf_peek_timesortedlist(sch);
+ if (!skb)
+ return NULL;
+
+ now = q->get_time();
+
+ /* Drop if packet has expired while in queue. */
+ /* FIXME: Must return error on the socket's error queue */
+ if (ktime_before(skb->tstamp, now)) {
+ timesortedlist_erase(sch, skb, true);
+ skb = NULL;
+ goto out;
+ }
+
+ /* When in deadline mode, dequeue as soon as possible and change the
+ * txtime from deadline to (now + delta).
+ */
+ if (q->deadline_mode) {
+ timesortedlist_erase(sch, skb, false);
+ skb->tstamp = now;
+ goto out;
+ }
+
+ next = ktime_sub_ns(skb->tstamp, q->delta);
+
+ /* Dequeue only if now is within the [txtime - delta, txtime] range. */
+ if (ktime_after(now, next))
+ timesortedlist_erase(sch, skb, false);
+ else
+ skb = NULL;
+
+out:
+ /* Now we may need to re-arm the qdisc watchdog for the next packet. */
+ reset_watchdog(sch);
+
+ return skb;
+}
+
+static int etf_init(struct Qdisc *sch, struct nlattr *opt,
+ struct netlink_ext_ack *extack)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ struct net_device *dev = qdisc_dev(sch);
+ struct nlattr *tb[TCA_ETF_MAX + 1];
+ struct tc_etf_qopt *qopt;
+ int err;
+
+ if (!opt) {
+ NL_SET_ERR_MSG(extack,
+ "Missing ETF qdisc options which are mandatory");
+ return -EINVAL;
+ }
+
+ err = nla_parse_nested(tb, TCA_ETF_MAX, opt, etf_policy, extack);
+ if (err < 0)
+ return err;
+
+ if (!tb[TCA_ETF_PARMS]) {
+ NL_SET_ERR_MSG(extack, "Missing mandatory ETF parameters");
+ return -EINVAL;
+ }
+
+ qopt = nla_data(tb[TCA_ETF_PARMS]);
+
+ pr_debug("delta %d clockid %d deadline %s\n",
+ qopt->delta, qopt->clockid,
+ DEADLINE_MODE_IS_ON(qopt) ? "on" : "off");
+
+ err = validate_input_params(qopt, extack);
+ if (err < 0)
+ return err;
+
+ q->queue = sch->dev_queue - netdev_get_tx_queue(dev, 0);
+
+ /* Everything went OK, save the parameters used. */
+ q->delta = qopt->delta;
+ q->clockid = qopt->clockid;
+ q->deadline_mode = DEADLINE_MODE_IS_ON(qopt);
+
+ switch (q->clockid) {
+ case CLOCK_REALTIME:
+ q->get_time = ktime_get_real;
+ break;
+ case CLOCK_MONOTONIC:
+ q->get_time = ktime_get;
+ break;
+ case CLOCK_BOOTTIME:
+ q->get_time = ktime_get_boottime;
+ break;
+ case CLOCK_TAI:
+ q->get_time = ktime_get_clocktai;
+ break;
+ default:
+ NL_SET_ERR_MSG(extack, "Clockid is not supported");
+ return -ENOTSUPP;
+ }
+
+ qdisc_watchdog_init_clockid(&q->watchdog, sch, q->clockid);
+
+ return 0;
+}
+
+static void timesortedlist_clear(struct Qdisc *sch)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ struct rb_node *p = rb_first(&q->head);
+
+ while (p) {
+ struct sk_buff *skb = rb_to_skb(p);
+
+ p = rb_next(p);
+
+ rb_erase(&skb->rbnode, &q->head);
+ rtnl_kfree_skbs(skb, skb);
+ sch->q.qlen--;
+ }
+}
+
+static void etf_reset(struct Qdisc *sch)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+
+ /* Only cancel watchdog if it's been initialized. */
+ if (q->watchdog.qdisc == sch)
+ qdisc_watchdog_cancel(&q->watchdog);
+
+ /* No matter which mode we are on, it's safe to clear both lists. */
+ timesortedlist_clear(sch);
+ __qdisc_reset_queue(&sch->q);
+
+ sch->qstats.backlog = 0;
+ sch->q.qlen = 0;
+
+ q->last = 0;
+}
+
+static void etf_destroy(struct Qdisc *sch)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+
+ /* Only cancel watchdog if it's been initialized. */
+ if (q->watchdog.qdisc == sch)
+ qdisc_watchdog_cancel(&q->watchdog);
+}
+
+static int etf_dump(struct Qdisc *sch, struct sk_buff *skb)
+{
+ struct etf_sched_data *q = qdisc_priv(sch);
+ struct tc_etf_qopt opt = { };
+ struct nlattr *nest;
+
+ nest = nla_nest_start(skb, TCA_OPTIONS);
+ if (!nest)
+ goto nla_put_failure;
+
+ opt.delta = q->delta;
+ opt.clockid = q->clockid;
+ if (q->deadline_mode)
+ opt.flags |= TC_ETF_DEADLINE_MODE_ON;
+
+ if (nla_put(skb, TCA_ETF_PARMS, sizeof(opt), &opt))
+ goto nla_put_failure;
+
+ return nla_nest_end(skb, nest);
+
+nla_put_failure:
+ nla_nest_cancel(skb, nest);
+ return -1;
+}
+
+static struct Qdisc_ops etf_qdisc_ops __read_mostly = {
+ .id = "etf",
+ .priv_size = sizeof(struct etf_sched_data),
+ .enqueue = etf_enqueue_timesortedlist,
+ .dequeue = etf_dequeue_timesortedlist,
+ .peek = etf_peek_timesortedlist,
+ .init = etf_init,
+ .reset = etf_reset,
+ .destroy = etf_destroy,
+ .dump = etf_dump,
+ .owner = THIS_MODULE,
+};
+
+static int __init etf_module_init(void)
+{
+ return register_qdisc(&etf_qdisc_ops);
+}
+
+static void __exit etf_module_exit(void)
+{
+ unregister_qdisc(&etf_qdisc_ops);
+}
+module_init(etf_module_init)
+module_exit(etf_module_exit)
+MODULE_LICENSE("GPL");
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 08/14] igb: Refactor igb_configure_cbs()
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Make this function retrieve what it needs from the Tx ring being
addressed since it already relies on what had been saved on it before.
Also, since this function will be used by the upcoming Launchtime
patches rename it to better reflect its intention. Note that
Launchtime is not part of what 802.1Qav specifies, but the i210
datasheet refers to this set of functionality as "Qav Transmission
Mode".
Here we also perform a tiny refactor at is_any_cbs_enabled(), and add
further documentation to igb_setup_tx_mode().
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 60 +++++++++++------------
1 file changed, 28 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index f1e3397bd405..15f6b9c57ccf 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -1655,23 +1655,17 @@ static void set_queue_mode(struct e1000_hw *hw, int queue, enum queue_mode mode)
}
/**
- * igb_configure_cbs - Configure Credit-Based Shaper (CBS)
+ * igb_config_tx_modes - Configure "Qav Tx mode" features on igb
* @adapter: pointer to adapter struct
* @queue: queue number
- * @enable: true = enable CBS, false = disable CBS
- * @idleslope: idleSlope in kbps
- * @sendslope: sendSlope in kbps
- * @hicredit: hiCredit in bytes
- * @locredit: loCredit in bytes
*
- * Configure CBS for a given hardware queue. When disabling, idleslope,
- * sendslope, hicredit, locredit arguments are ignored. Returns 0 if
- * success. Negative otherwise.
+ * Configure CBS for a given hardware queue. Parameters are retrieved
+ * from the correct Tx ring, so igb_save_cbs_params() should be used
+ * for setting those correctly prior to this function being called.
**/
-static void igb_configure_cbs(struct igb_adapter *adapter, int queue,
- bool enable, int idleslope, int sendslope,
- int hicredit, int locredit)
+static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
{
+ struct igb_ring *ring = adapter->tx_ring[queue];
struct net_device *netdev = adapter->netdev;
struct e1000_hw *hw = &adapter->hw;
u32 tqavcc;
@@ -1680,7 +1674,7 @@ static void igb_configure_cbs(struct igb_adapter *adapter, int queue,
WARN_ON(hw->mac.type != e1000_i210);
WARN_ON(queue < 0 || queue > 1);
- if (enable || queue == 0) {
+ if (ring->cbs_enable || queue == 0) {
/* i210 does not allow the queue 0 to be in the Strict
* Priority mode while the Qav mode is enabled, so,
* instead of disabling strict priority mode, we give
@@ -1690,10 +1684,10 @@ static void igb_configure_cbs(struct igb_adapter *adapter, int queue,
* Queue0 QueueMode must be set to 1b when
* TransmitMode is set to Qav."
*/
- if (queue == 0 && !enable) {
+ if (queue == 0 && !ring->cbs_enable) {
/* max "linkspeed" idleslope in kbps */
- idleslope = 1000000;
- hicredit = ETH_FRAME_LEN;
+ ring->idleslope = 1000000;
+ ring->hicredit = ETH_FRAME_LEN;
}
set_tx_desc_fetch_prio(hw, queue, TX_QUEUE_PRIO_HIGH);
@@ -1756,14 +1750,15 @@ static void igb_configure_cbs(struct igb_adapter *adapter, int queue,
* calculated value, so the resulting bandwidth might
* be slightly higher for some configurations.
*/
- value = DIV_ROUND_UP_ULL(idleslope * 61034ULL, 1000000);
+ value = DIV_ROUND_UP_ULL(ring->idleslope * 61034ULL, 1000000);
tqavcc = rd32(E1000_I210_TQAVCC(queue));
tqavcc &= ~E1000_TQAVCC_IDLESLOPE_MASK;
tqavcc |= value;
wr32(E1000_I210_TQAVCC(queue), tqavcc);
- wr32(E1000_I210_TQAVHC(queue), 0x80000000 + hicredit * 0x7735);
+ wr32(E1000_I210_TQAVHC(queue),
+ 0x80000000 + ring->hicredit * 0x7735);
} else {
set_tx_desc_fetch_prio(hw, queue, TX_QUEUE_PRIO_LOW);
set_queue_mode(hw, queue, QUEUE_MODE_STRICT_PRIORITY);
@@ -1783,8 +1778,9 @@ static void igb_configure_cbs(struct igb_adapter *adapter, int queue,
*/
netdev_dbg(netdev, "CBS %s: queue %d idleslope %d sendslope %d hiCredit %d locredit %d\n",
- (enable) ? "enabled" : "disabled", queue,
- idleslope, sendslope, hicredit, locredit);
+ (ring->cbs_enable) ? "enabled" : "disabled", queue,
+ ring->idleslope, ring->sendslope, ring->hicredit,
+ ring->locredit);
}
static int igb_save_cbs_params(struct igb_adapter *adapter, int queue,
@@ -1809,19 +1805,25 @@ static int igb_save_cbs_params(struct igb_adapter *adapter, int queue,
static bool is_any_cbs_enabled(struct igb_adapter *adapter)
{
- struct igb_ring *ring;
int i;
for (i = 0; i < adapter->num_tx_queues; i++) {
- ring = adapter->tx_ring[i];
-
- if (ring->cbs_enable)
+ if (adapter->tx_ring[i]->cbs_enable)
return true;
}
return false;
}
+/**
+ * igb_setup_tx_mode - Switch to/from Qav Tx mode when applicable
+ * @adapter: pointer to adapter struct
+ *
+ * Configure TQAVCTRL register switching the controller's Tx mode
+ * if FQTSS mode is enabled or disabled. Additionally, will issue
+ * a call to igb_config_tx_modes() per queue so any previously saved
+ * Tx parameters are applied.
+ **/
static void igb_setup_tx_mode(struct igb_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
@@ -1881,11 +1883,7 @@ static void igb_setup_tx_mode(struct igb_adapter *adapter)
adapter->num_tx_queues : I210_SR_QUEUES_NUM;
for (i = 0; i < max_queue; i++) {
- struct igb_ring *ring = adapter->tx_ring[i];
-
- igb_configure_cbs(adapter, i, ring->cbs_enable,
- ring->idleslope, ring->sendslope,
- ring->hicredit, ring->locredit);
+ igb_config_tx_modes(adapter, i);
}
} else {
wr32(E1000_RXPBS, I210_RXPBSIZE_DEFAULT);
@@ -2480,9 +2478,7 @@ static int igb_offload_cbs(struct igb_adapter *adapter,
return err;
if (is_fqtss_enabled(adapter)) {
- igb_configure_cbs(adapter, qopt->queue, qopt->enable,
- qopt->idleslope, qopt->sendslope,
- qopt->hicredit, qopt->locredit);
+ igb_config_tx_modes(adapter, qopt->queue);
if (!is_any_cbs_enabled(adapter))
enable_fqtss(adapter, false);
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 14/14] net/sched: Make etf report drops on error_queue
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Use the socket error queue for reporting dropped packets if the
socket has enabled that feature through the SO_TXTIME API.
Packets are dropped either on enqueue() if they aren't accepted by the
qdisc or on dequeue() if the system misses their deadline. Those are
reported as different errors so applications can react accordingly.
Userspace can retrieve the errors through the socket error queue and the
corresponding cmsg interfaces. A struct sock_extended_err* is used for
returning the error data, and the packet's timestamp can be retrieved by
adding both ee_data and ee_info fields as e.g.:
((__u64) serr->ee_data << 32) + serr->ee_info
This feature is disabled by default and must be explicitly enabled by
applications. Enabling it can bring some overhead for the Tx cycles
of the application.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
include/linux/socket.h | 4 +++-
include/net/sock.h | 1 +
include/uapi/linux/errqueue.h | 2 ++
net/sched/sch_etf.c | 37 +++++++++++++++++++++++++++++++++--
4 files changed, 41 insertions(+), 3 deletions(-)
diff --git a/include/linux/socket.h b/include/linux/socket.h
index ca476b7a8ff0..75e11d29b32a 100644
--- a/include/linux/socket.h
+++ b/include/linux/socket.h
@@ -85,7 +85,9 @@ struct cmsghdr {
struct sock_txtime {
clockid_t clockid; /* reference clockid */
- u16 flags; /* bit 0: txtime in deadline_mode */
+ u16 flags; /* bit 0: txtime in deadline_mode
+ * bit 1: report drops on sk err queue
+ */
};
/*
diff --git a/include/net/sock.h b/include/net/sock.h
index 73f4404e49e4..e681a45cfe7e 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -473,6 +473,7 @@ struct sock {
u16 sk_clockid;
u16 sk_txtime_flags;
#define SK_TXTIME_DEADLINE_MASK BIT(0)
+#define SK_TXTIME_RECV_ERR_MASK BIT(1)
struct socket *sk_socket;
void *sk_user_data;
diff --git a/include/uapi/linux/errqueue.h b/include/uapi/linux/errqueue.h
index dc64cfaf13da..66fd5e443c94 100644
--- a/include/uapi/linux/errqueue.h
+++ b/include/uapi/linux/errqueue.h
@@ -25,6 +25,8 @@ struct sock_extended_err {
#define SO_EE_OFFENDER(ee) ((struct sockaddr*)((ee)+1))
#define SO_EE_CODE_ZEROCOPY_COPIED 1
+#define SO_EE_CODE_TXTIME_INVALID_PARAM 2
+#define SO_EE_CODE_TXTIME_MISSED 3
/**
* struct scm_timestamping - timestamps exposed through cmsg
diff --git a/net/sched/sch_etf.c b/net/sched/sch_etf.c
index 5514a8aa3bd5..166f4b72875b 100644
--- a/net/sched/sch_etf.c
+++ b/net/sched/sch_etf.c
@@ -11,6 +11,7 @@
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
+#include <linux/errqueue.h>
#include <linux/rbtree.h>
#include <linux/skbuff.h>
#include <linux/posix-timers.h>
@@ -124,6 +125,35 @@ static void reset_watchdog(struct Qdisc *sch)
qdisc_watchdog_schedule_ns(&q->watchdog, ktime_to_ns(next));
}
+static void report_sock_error(struct sk_buff *skb, u32 err, u8 code)
+{
+ struct sock_exterr_skb *serr;
+ ktime_t txtime = skb->tstamp;
+
+ if (!skb->sk || !(skb->sk->sk_txtime_flags & SK_TXTIME_RECV_ERR_MASK))
+ return;
+
+ skb = skb_clone_sk(skb);
+ if (!skb)
+ return;
+
+ sock_hold(skb->sk);
+
+ serr = SKB_EXT_ERR(skb);
+ serr->ee.ee_errno = err;
+ serr->ee.ee_origin = SO_EE_ORIGIN_LOCAL;
+ serr->ee.ee_type = 0;
+ serr->ee.ee_code = code;
+ serr->ee.ee_pad = 0;
+ serr->ee.ee_data = (txtime >> 32); /* high part of tstamp */
+ serr->ee.ee_info = txtime; /* low part of tstamp */
+
+ if (sock_queue_err_skb(skb->sk, skb))
+ kfree_skb(skb);
+
+ sock_put(skb->sk);
+}
+
static int etf_enqueue_timesortedlist(struct sk_buff *nskb, struct Qdisc *sch,
struct sk_buff **to_free)
{
@@ -131,8 +161,10 @@ static int etf_enqueue_timesortedlist(struct sk_buff *nskb, struct Qdisc *sch,
struct rb_node **p = &q->head.rb_node, *parent = NULL;
ktime_t txtime = nskb->tstamp;
- if (!is_packet_valid(sch, nskb))
+ if (!is_packet_valid(sch, nskb)) {
+ report_sock_error(nskb, EINVAL, SO_EE_CODE_TXTIME_INVALID_PARAM);
return qdisc_drop(nskb, sch, to_free);
+ }
while (*p) {
struct sk_buff *skb;
@@ -175,6 +207,8 @@ static void timesortedlist_erase(struct Qdisc *sch, struct sk_buff *skb,
if (drop) {
struct sk_buff *to_free = NULL;
+ report_sock_error(skb, ECANCELED, SO_EE_CODE_TXTIME_MISSED);
+
qdisc_drop(skb, sch, &to_free);
kfree_skb_list(to_free);
qdisc_qstats_overlimit(sch);
@@ -200,7 +234,6 @@ static struct sk_buff *etf_dequeue_timesortedlist(struct Qdisc *sch)
now = q->get_time();
/* Drop if packet has expired while in queue. */
- /* FIXME: Must return error on the socket's error queue */
if (ktime_before(skb->tstamp, now)) {
timesortedlist_erase(sch, skb, true);
skb = NULL;
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 09/14] igb: Only change Tx arbitration when CBS is on
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Currently the data transmission arbitration algorithm - DataTranARB
field on TQAVCTRL reg - is always set to CBS when the Tx mode is
changed from legacy to 'Qav' mode.
Make that configuration a bit more granular in preparation for the
upcoming Launchtime enabling patches, since CBS and Launchtime can be
enabled separately. That is achieved by moving the DataTranARB setup
to igb_config_tx_modes() instead.
Similarly, when disabling CBS we must check if it has been disabled
for all queues, and clear the DataTranARB accordingly.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 49 +++++++++++++++--------
1 file changed, 33 insertions(+), 16 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 15f6b9c57ccf..8c90f1e51add 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -1654,6 +1654,18 @@ static void set_queue_mode(struct e1000_hw *hw, int queue, enum queue_mode mode)
wr32(E1000_I210_TQAVCC(queue), val);
}
+static bool is_any_cbs_enabled(struct igb_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_tx_queues; i++) {
+ if (adapter->tx_ring[i]->cbs_enable)
+ return true;
+ }
+
+ return false;
+}
+
/**
* igb_config_tx_modes - Configure "Qav Tx mode" features on igb
* @adapter: pointer to adapter struct
@@ -1668,7 +1680,7 @@ static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
struct igb_ring *ring = adapter->tx_ring[queue];
struct net_device *netdev = adapter->netdev;
struct e1000_hw *hw = &adapter->hw;
- u32 tqavcc;
+ u32 tqavcc, tqavctrl;
u16 value;
WARN_ON(hw->mac.type != e1000_i210);
@@ -1693,6 +1705,14 @@ static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
set_tx_desc_fetch_prio(hw, queue, TX_QUEUE_PRIO_HIGH);
set_queue_mode(hw, queue, QUEUE_MODE_STREAM_RESERVATION);
+ /* Always set data transfer arbitration to credit-based
+ * shaper algorithm on TQAVCTRL if CBS is enabled for any of
+ * the queues.
+ */
+ tqavctrl = rd32(E1000_I210_TQAVCTRL);
+ tqavctrl |= E1000_TQAVCTRL_DATATRANARB;
+ wr32(E1000_I210_TQAVCTRL, tqavctrl);
+
/* According to i210 datasheet section 7.2.7.7, we should set
* the 'idleSlope' field from TQAVCC register following the
* equation:
@@ -1770,6 +1790,16 @@ static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
/* Set hiCredit to zero. */
wr32(E1000_I210_TQAVHC(queue), 0);
+
+ /* If CBS is not enabled for any queues anymore, then return to
+ * the default state of Data Transmission Arbitration on
+ * TQAVCTRL.
+ */
+ if (!is_any_cbs_enabled(adapter)) {
+ tqavctrl = rd32(E1000_I210_TQAVCTRL);
+ tqavctrl &= ~E1000_TQAVCTRL_DATATRANARB;
+ wr32(E1000_I210_TQAVCTRL, tqavctrl);
+ }
}
/* XXX: In i210 controller the sendSlope and loCredit parameters from
@@ -1803,18 +1833,6 @@ static int igb_save_cbs_params(struct igb_adapter *adapter, int queue,
return 0;
}
-static bool is_any_cbs_enabled(struct igb_adapter *adapter)
-{
- int i;
-
- for (i = 0; i < adapter->num_tx_queues; i++) {
- if (adapter->tx_ring[i]->cbs_enable)
- return true;
- }
-
- return false;
-}
-
/**
* igb_setup_tx_mode - Switch to/from Qav Tx mode when applicable
* @adapter: pointer to adapter struct
@@ -1838,11 +1856,10 @@ static void igb_setup_tx_mode(struct igb_adapter *adapter)
int i, max_queue;
/* Configure TQAVCTRL register: set transmit mode to 'Qav',
- * set data fetch arbitration to 'round robin' and set data
- * transfer arbitration to 'credit shaper algorithm.
+ * set data fetch arbitration to 'round robin'.
*/
val = rd32(E1000_I210_TQAVCTRL);
- val |= E1000_TQAVCTRL_XMIT_MODE | E1000_TQAVCTRL_DATATRANARB;
+ val |= E1000_TQAVCTRL_XMIT_MODE;
val &= ~E1000_TQAVCTRL_DATAFETCHARB;
wr32(E1000_I210_TQAVCTRL, val);
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 13/14] net/sched: Enforce usage of CLOCK_TAI for sch_etf
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
The qdisc and the SO_TXTIME ABIs allow for a clockid to be configured,
but it's been decided that usage of CLOCK_TAI should be enforced until
we decide to allow for other clockids to be used. The rationale here is
that PTP times are usually in the TAI scale, thus no other clocks should
be necessary.
For now, the qdisc will return EINVAL if any clocks other than
CLOCK_TAI are used.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
net/sched/sch_etf.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/sched/sch_etf.c b/net/sched/sch_etf.c
index cd6cb5b69228..5514a8aa3bd5 100644
--- a/net/sched/sch_etf.c
+++ b/net/sched/sch_etf.c
@@ -56,8 +56,8 @@ static inline int validate_input_params(struct tc_etf_qopt *qopt,
return -ENOTSUPP;
}
- if (qopt->clockid >= MAX_CLOCKS) {
- NL_SET_ERR_MSG(extack, "Invalid clockid");
+ if (qopt->clockid != CLOCK_TAI) {
+ NL_SET_ERR_MSG(extack, "Invalid clockid. CLOCK_TAI must be used");
return -EINVAL;
}
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 12/14] igb: Only call skb_tx_timestamp after descriptors are ready
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Currently, skb_tx_timestamp() is being called before the DMA
descriptors are prepared in igb_xmit_frame_ring(), which happens
during either the igb_tso() or igb_tx_csum() calls.
Given that now the skb->tstamp might be used to carry the timestamp
for SO_TXTIME, we must only call skb_tx_timestamp() after the
information has been copied into the DMA tx_ring.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 9b9a6a6227e0..0d72f2417143 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -6138,8 +6138,6 @@ netdev_tx_t igb_xmit_frame_ring(struct sk_buff *skb,
}
}
- skb_tx_timestamp(skb);
-
if (skb_vlan_tag_present(skb)) {
tx_flags |= IGB_TX_FLAGS_VLAN;
tx_flags |= (skb_vlan_tag_get(skb) << IGB_TX_FLAGS_VLAN_SHIFT);
@@ -6155,6 +6153,8 @@ netdev_tx_t igb_xmit_frame_ring(struct sk_buff *skb,
else if (!tso)
igb_tx_csum(tx_ring, first);
+ skb_tx_timestamp(skb);
+
if (igb_tx_map(tx_ring, first, hdr_len))
goto cleanup_tx_tstamp;
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 11/14] igb: Add support for ETF offload
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Implement HW offload support for SO_TXTIME through igb's Launchtime
feature. This is done by extending igb_setup_tc() so it supports
TC_SETUP_QDISC_ETF and configuring i210 so time based transmit
arbitration is enabled.
The FQTSS transmission mode added before is extended so strict
priority (SP) queues wait for stream reservation (SR) ones.
igb_config_tx_modes() is extended so it can support enabling/disabling
Launchtime following the previous approach used for the credit-based
shaper (CBS).
As the previous flow, FQTSS transmission mode is enabled automatically
by the driver once Launchtime (or CBS, as before) is enabled.
Similarly, it's automatically disabled when the feature is disabled
for the last queue that had it setup on.
The driver just consumes the transmit times from the skbuffs directly,
so no special handling is done in case an 'invalid' time is provided.
We assume this has been handled by the ETF qdisc already.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
.../net/ethernet/intel/igb/e1000_defines.h | 16 ++
drivers/net/ethernet/intel/igb/igb.h | 1 +
drivers/net/ethernet/intel/igb/igb_main.c | 139 +++++++++++++++---
3 files changed, 139 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/e1000_defines.h b/drivers/net/ethernet/intel/igb/e1000_defines.h
index 252440a418dc..8a28f3388f69 100644
--- a/drivers/net/ethernet/intel/igb/e1000_defines.h
+++ b/drivers/net/ethernet/intel/igb/e1000_defines.h
@@ -1048,6 +1048,22 @@
#define E1000_TQAVCTRL_XMIT_MODE BIT(0)
#define E1000_TQAVCTRL_DATAFETCHARB BIT(4)
#define E1000_TQAVCTRL_DATATRANARB BIT(8)
+#define E1000_TQAVCTRL_DATATRANTIM BIT(9)
+#define E1000_TQAVCTRL_SP_WAIT_SR BIT(10)
+/* Fetch Time Delta - bits 31:16
+ *
+ * This field holds the value to be reduced from the launch time for
+ * fetch time decision. The FetchTimeDelta value is defined in 32 ns
+ * granularity.
+ *
+ * This field is 16 bits wide, and so the maximum value is:
+ *
+ * 65535 * 32 = 2097120 ~= 2.1 msec
+ *
+ * XXX: We are configuring the max value here since we couldn't come up
+ * with a reason for not doing so.
+ */
+#define E1000_TQAVCTRL_FETCHTIME_DELTA (0xFFFF << 16)
/* TX Qav Credit Control fields */
#define E1000_TQAVCC_IDLESLOPE_MASK 0xFFFF
diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index 9643b5b3d444..ca54e268d157 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -262,6 +262,7 @@ struct igb_ring {
u16 count; /* number of desc. in the ring */
u8 queue_index; /* logical index of the ring*/
u8 reg_idx; /* physical index of the ring */
+ bool launchtime_enable; /* true if LaunchTime is enabled */
bool cbs_enable; /* indicates if CBS is enabled */
s32 idleslope; /* idleSlope in kbps */
s32 sendslope; /* sendSlope in kbps */
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index c30ab7b260cc..9b9a6a6227e0 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -1666,13 +1666,26 @@ static bool is_any_cbs_enabled(struct igb_adapter *adapter)
return false;
}
+static bool is_any_txtime_enabled(struct igb_adapter *adapter)
+{
+ int i;
+
+ for (i = 0; i < adapter->num_tx_queues; i++) {
+ if (adapter->tx_ring[i]->launchtime_enable)
+ return true;
+ }
+
+ return false;
+}
+
/**
* igb_config_tx_modes - Configure "Qav Tx mode" features on igb
* @adapter: pointer to adapter struct
* @queue: queue number
*
- * Configure CBS for a given hardware queue. Parameters are retrieved
- * from the correct Tx ring, so igb_save_cbs_params() should be used
+ * Configure CBS and Launchtime for a given hardware queue.
+ * Parameters are retrieved from the correct Tx ring, so
+ * igb_save_cbs_params() and igb_save_txtime_params() should be used
* for setting those correctly prior to this function being called.
**/
static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
@@ -1686,6 +1699,19 @@ static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
WARN_ON(hw->mac.type != e1000_i210);
WARN_ON(queue < 0 || queue > 1);
+ /* If any of the Qav features is enabled, configure queues as SR and
+ * with HIGH PRIO. If none is, then configure them with LOW PRIO and
+ * as SP.
+ */
+ if (ring->cbs_enable || ring->launchtime_enable) {
+ set_tx_desc_fetch_prio(hw, queue, TX_QUEUE_PRIO_HIGH);
+ set_queue_mode(hw, queue, QUEUE_MODE_STREAM_RESERVATION);
+ } else {
+ set_tx_desc_fetch_prio(hw, queue, TX_QUEUE_PRIO_LOW);
+ set_queue_mode(hw, queue, QUEUE_MODE_STRICT_PRIORITY);
+ }
+
+ /* If CBS is enabled, set DataTranARB and config its parameters. */
if (ring->cbs_enable || queue == 0) {
/* i210 does not allow the queue 0 to be in the Strict
* Priority mode while the Qav mode is enabled, so,
@@ -1702,9 +1728,6 @@ static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
ring->hicredit = ETH_FRAME_LEN;
}
- set_tx_desc_fetch_prio(hw, queue, TX_QUEUE_PRIO_HIGH);
- set_queue_mode(hw, queue, QUEUE_MODE_STREAM_RESERVATION);
-
/* Always set data transfer arbitration to credit-based
* shaper algorithm on TQAVCTRL if CBS is enabled for any of
* the queues.
@@ -1780,8 +1803,6 @@ static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
wr32(E1000_I210_TQAVHC(queue),
0x80000000 + ring->hicredit * 0x7735);
} else {
- set_tx_desc_fetch_prio(hw, queue, TX_QUEUE_PRIO_LOW);
- set_queue_mode(hw, queue, QUEUE_MODE_STRICT_PRIORITY);
/* Set idleSlope to zero. */
tqavcc = rd32(E1000_I210_TQAVCC(queue));
@@ -1802,17 +1823,61 @@ static void igb_config_tx_modes(struct igb_adapter *adapter, int queue)
}
}
+ /* If LaunchTime is enabled, set DataTranTIM. */
+ if (ring->launchtime_enable) {
+ /* Always set DataTranTIM on TQAVCTRL if LaunchTime is enabled
+ * for any of the SR queues, and configure fetchtime delta.
+ * XXX NOTE:
+ * - LaunchTime will be enabled for all SR queues.
+ * - A fixed offset can be added relative to the launch
+ * time of all packets if configured at reg LAUNCH_OS0.
+ * We are keeping it as 0 for now (default value).
+ */
+ tqavctrl = rd32(E1000_I210_TQAVCTRL);
+ tqavctrl |= E1000_TQAVCTRL_DATATRANTIM |
+ E1000_TQAVCTRL_FETCHTIME_DELTA;
+ wr32(E1000_I210_TQAVCTRL, tqavctrl);
+ } else {
+ /* If Launchtime is not enabled for any SR queues anymore,
+ * then clear DataTranTIM on TQAVCTRL and clear fetchtime delta,
+ * effectively disabling Launchtime.
+ */
+ if (!is_any_txtime_enabled(adapter)) {
+ tqavctrl = rd32(E1000_I210_TQAVCTRL);
+ tqavctrl &= ~E1000_TQAVCTRL_DATATRANTIM;
+ tqavctrl &= ~E1000_TQAVCTRL_FETCHTIME_DELTA;
+ wr32(E1000_I210_TQAVCTRL, tqavctrl);
+ }
+ }
+
/* XXX: In i210 controller the sendSlope and loCredit parameters from
* CBS are not configurable by software so we don't do any 'controller
* configuration' in respect to these parameters.
*/
- netdev_dbg(netdev, "CBS %s: queue %d idleslope %d sendslope %d hiCredit %d locredit %d\n",
- (ring->cbs_enable) ? "enabled" : "disabled", queue,
+ netdev_dbg(netdev, "Qav Tx mode: cbs %s, launchtime %s, queue %d \
+ idleslope %d sendslope %d hiCredit %d \
+ locredit %d\n",
+ (ring->cbs_enable) ? "enabled" : "disabled",
+ (ring->launchtime_enable) ? "enabled" : "disabled", queue,
ring->idleslope, ring->sendslope, ring->hicredit,
ring->locredit);
}
+static int igb_save_txtime_params(struct igb_adapter *adapter, int queue,
+ bool enable)
+{
+ struct igb_ring *ring;
+
+ if (queue < 0 || queue > adapter->num_tx_queues)
+ return -EINVAL;
+
+ ring = adapter->tx_ring[queue];
+ ring->launchtime_enable = enable;
+
+ return 0;
+}
+
static int igb_save_cbs_params(struct igb_adapter *adapter, int queue,
bool enable, int idleslope, int sendslope,
int hicredit, int locredit)
@@ -1856,10 +1921,11 @@ static void igb_setup_tx_mode(struct igb_adapter *adapter)
int i, max_queue;
/* Configure TQAVCTRL register: set transmit mode to 'Qav',
- * set data fetch arbitration to 'round robin'.
+ * set data fetch arbitration to 'round robin', set SP_WAIT_SR
+ * so SP queues wait for SR ones.
*/
val = rd32(E1000_I210_TQAVCTRL);
- val |= E1000_TQAVCTRL_XMIT_MODE;
+ val |= E1000_TQAVCTRL_XMIT_MODE | E1000_TQAVCTRL_SP_WAIT_SR;
val &= ~E1000_TQAVCTRL_DATAFETCHARB;
wr32(E1000_I210_TQAVCTRL, val);
@@ -2483,7 +2549,7 @@ static void igb_offload_apply(struct igb_adapter *adapter, s32 queue)
igb_config_tx_modes(adapter, queue);
- if (!is_any_cbs_enabled(adapter))
+ if (!is_any_cbs_enabled(adapter) && !is_any_txtime_enabled(adapter))
enable_fqtss(adapter, false);
}
@@ -2756,6 +2822,30 @@ static int igb_setup_tc_block(struct igb_adapter *adapter,
}
}
+static int igb_offload_txtime(struct igb_adapter *adapter,
+ struct tc_etf_qopt_offload *qopt)
+{
+ struct e1000_hw *hw = &adapter->hw;
+ int err;
+
+ /* Launchtime offloading is only supported by i210 controller. */
+ if (hw->mac.type != e1000_i210)
+ return -EOPNOTSUPP;
+
+ /* Launchtime offloading is only supported by queues 0 and 1. */
+ if (qopt->queue < 0 || qopt->queue > 1)
+ return -EINVAL;
+
+ err = igb_save_txtime_params(adapter, qopt->queue, qopt->enable);
+
+ if (err)
+ return err;
+
+ igb_offload_apply(adapter, qopt->queue);
+
+ return 0;
+}
+
static int igb_setup_tc(struct net_device *dev, enum tc_setup_type type,
void *type_data)
{
@@ -2766,6 +2856,8 @@ static int igb_setup_tc(struct net_device *dev, enum tc_setup_type type,
return igb_offload_cbs(adapter, type_data);
case TC_SETUP_BLOCK:
return igb_setup_tc_block(adapter, type_data);
+ case TC_SETUP_QDISC_ETF:
+ return igb_offload_txtime(adapter, type_data);
default:
return -EOPNOTSUPP;
@@ -5586,11 +5678,14 @@ static void igb_set_itr(struct igb_q_vector *q_vector)
}
}
-static void igb_tx_ctxtdesc(struct igb_ring *tx_ring, u32 vlan_macip_lens,
- u32 type_tucmd, u32 mss_l4len_idx)
+static void igb_tx_ctxtdesc(struct igb_ring *tx_ring,
+ struct igb_tx_buffer *first,
+ u32 vlan_macip_lens, u32 type_tucmd,
+ u32 mss_l4len_idx)
{
struct e1000_adv_tx_context_desc *context_desc;
u16 i = tx_ring->next_to_use;
+ struct timespec64 ts;
context_desc = IGB_TX_CTXTDESC(tx_ring, i);
@@ -5605,9 +5700,18 @@ static void igb_tx_ctxtdesc(struct igb_ring *tx_ring, u32 vlan_macip_lens,
mss_l4len_idx |= tx_ring->reg_idx << 4;
context_desc->vlan_macip_lens = cpu_to_le32(vlan_macip_lens);
- context_desc->seqnum_seed = 0;
context_desc->type_tucmd_mlhl = cpu_to_le32(type_tucmd);
context_desc->mss_l4len_idx = cpu_to_le32(mss_l4len_idx);
+
+ /* We assume there is always a valid tx time available. Invalid times
+ * should have been handled by the upper layers.
+ */
+ if (tx_ring->launchtime_enable) {
+ ts = ns_to_timespec64(first->skb->tstamp);
+ context_desc->seqnum_seed = cpu_to_le32(ts.tv_nsec / 32);
+ } else {
+ context_desc->seqnum_seed = 0;
+ }
}
static int igb_tso(struct igb_ring *tx_ring,
@@ -5690,7 +5794,8 @@ static int igb_tso(struct igb_ring *tx_ring,
vlan_macip_lens |= (ip.hdr - skb->data) << E1000_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IGB_TX_FLAGS_VLAN_MASK;
- igb_tx_ctxtdesc(tx_ring, vlan_macip_lens, type_tucmd, mss_l4len_idx);
+ igb_tx_ctxtdesc(tx_ring, first, vlan_macip_lens,
+ type_tucmd, mss_l4len_idx);
return 1;
}
@@ -5745,7 +5850,7 @@ static void igb_tx_csum(struct igb_ring *tx_ring, struct igb_tx_buffer *first)
vlan_macip_lens |= skb_network_offset(skb) << E1000_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IGB_TX_FLAGS_VLAN_MASK;
- igb_tx_ctxtdesc(tx_ring, vlan_macip_lens, type_tucmd, 0);
+ igb_tx_ctxtdesc(tx_ring, first, vlan_macip_lens, type_tucmd, 0);
}
#define IGB_SET_FLAG(_input, _flag, _result) \
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 10/14] igb: Refactor igb_offload_cbs()
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Split code into a separate function (igb_offload_apply()) that will be
used by ETF offload implementation.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
drivers/net/ethernet/intel/igb/igb_main.c | 23 ++++++++++++++---------
1 file changed, 14 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 8c90f1e51add..c30ab7b260cc 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -2474,6 +2474,19 @@ igb_features_check(struct sk_buff *skb, struct net_device *dev,
return features;
}
+static void igb_offload_apply(struct igb_adapter *adapter, s32 queue)
+{
+ if (!is_fqtss_enabled(adapter)) {
+ enable_fqtss(adapter, true);
+ return;
+ }
+
+ igb_config_tx_modes(adapter, queue);
+
+ if (!is_any_cbs_enabled(adapter))
+ enable_fqtss(adapter, false);
+}
+
static int igb_offload_cbs(struct igb_adapter *adapter,
struct tc_cbs_qopt_offload *qopt)
{
@@ -2494,15 +2507,7 @@ static int igb_offload_cbs(struct igb_adapter *adapter,
if (err)
return err;
- if (is_fqtss_enabled(adapter)) {
- igb_config_tx_modes(adapter, qopt->queue);
-
- if (!is_any_cbs_enabled(adapter))
- enable_fqtss(adapter, false);
-
- } else {
- enable_fqtss(adapter, true);
- }
+ igb_offload_apply(adapter, qopt->queue);
return 0;
}
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 02/14] net: Add a new socket option for a future transmit time.
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri, Richard Cochran,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
From: Richard Cochran <rcochran@linutronix.de>
This patch introduces SO_TXTIME. User space enables this option in
order to pass a desired future transmit time in a CMSG when calling
sendmsg(2). The argument to this socket option is a 6-bytes long struct
defined as:
struct sock_txtime {
clockid_t clockid;
u16 flags;
};
Note that two new fields were added to struct sock by filling a 4-bytes
hole found in the struct. For that reason, neither the struct size or
number of cachelines were altered.
Signed-off-by: Richard Cochran <rcochran@linutronix.de>
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
arch/alpha/include/uapi/asm/socket.h | 3 +++
arch/ia64/include/uapi/asm/socket.h | 3 +++
arch/mips/include/uapi/asm/socket.h | 3 +++
arch/parisc/include/uapi/asm/socket.h | 3 +++
arch/s390/include/uapi/asm/socket.h | 3 +++
arch/sparc/include/uapi/asm/socket.h | 3 +++
arch/xtensa/include/uapi/asm/socket.h | 3 +++
include/linux/socket.h | 5 +++++
include/net/sock.h | 8 +++++++
include/uapi/asm-generic/socket.h | 3 +++
net/core/sock.c | 32 +++++++++++++++++++++++++++
11 files changed, 69 insertions(+)
diff --git a/arch/alpha/include/uapi/asm/socket.h b/arch/alpha/include/uapi/asm/socket.h
index be14f16149d5..065fb372e355 100644
--- a/arch/alpha/include/uapi/asm/socket.h
+++ b/arch/alpha/include/uapi/asm/socket.h
@@ -112,4 +112,7 @@
#define SO_ZEROCOPY 60
+#define SO_TXTIME 61
+#define SCM_TXTIME SO_TXTIME
+
#endif /* _UAPI_ASM_SOCKET_H */
diff --git a/arch/ia64/include/uapi/asm/socket.h b/arch/ia64/include/uapi/asm/socket.h
index 3efba40adc54..c872c4e6bafb 100644
--- a/arch/ia64/include/uapi/asm/socket.h
+++ b/arch/ia64/include/uapi/asm/socket.h
@@ -114,4 +114,7 @@
#define SO_ZEROCOPY 60
+#define SO_TXTIME 61
+#define SCM_TXTIME SO_TXTIME
+
#endif /* _ASM_IA64_SOCKET_H */
diff --git a/arch/mips/include/uapi/asm/socket.h b/arch/mips/include/uapi/asm/socket.h
index 49c3d4795963..71370fb3ceef 100644
--- a/arch/mips/include/uapi/asm/socket.h
+++ b/arch/mips/include/uapi/asm/socket.h
@@ -123,4 +123,7 @@
#define SO_ZEROCOPY 60
+#define SO_TXTIME 61
+#define SCM_TXTIME SO_TXTIME
+
#endif /* _UAPI_ASM_SOCKET_H */
diff --git a/arch/parisc/include/uapi/asm/socket.h b/arch/parisc/include/uapi/asm/socket.h
index 1d0fdc3b5d22..061b9cf2a779 100644
--- a/arch/parisc/include/uapi/asm/socket.h
+++ b/arch/parisc/include/uapi/asm/socket.h
@@ -104,4 +104,7 @@
#define SO_ZEROCOPY 0x4035
+#define SO_TXTIME 0x4036
+#define SCM_TXTIME SO_TXTIME
+
#endif /* _UAPI_ASM_SOCKET_H */
diff --git a/arch/s390/include/uapi/asm/socket.h b/arch/s390/include/uapi/asm/socket.h
index 3510c0fd06f4..39d901476ee5 100644
--- a/arch/s390/include/uapi/asm/socket.h
+++ b/arch/s390/include/uapi/asm/socket.h
@@ -111,4 +111,7 @@
#define SO_ZEROCOPY 60
+#define SO_TXTIME 61
+#define SCM_TXTIME SO_TXTIME
+
#endif /* _ASM_SOCKET_H */
diff --git a/arch/sparc/include/uapi/asm/socket.h b/arch/sparc/include/uapi/asm/socket.h
index d58520c2e6ff..7ea35e5601b6 100644
--- a/arch/sparc/include/uapi/asm/socket.h
+++ b/arch/sparc/include/uapi/asm/socket.h
@@ -101,6 +101,9 @@
#define SO_ZEROCOPY 0x003e
+#define SO_TXTIME 0x003f
+#define SCM_TXTIME SO_TXTIME
+
/* Security levels - as per NRL IPv6 - don't actually do anything */
#define SO_SECURITY_AUTHENTICATION 0x5001
#define SO_SECURITY_ENCRYPTION_TRANSPORT 0x5002
diff --git a/arch/xtensa/include/uapi/asm/socket.h b/arch/xtensa/include/uapi/asm/socket.h
index 75a07b8119a9..1de07a7f7680 100644
--- a/arch/xtensa/include/uapi/asm/socket.h
+++ b/arch/xtensa/include/uapi/asm/socket.h
@@ -116,4 +116,7 @@
#define SO_ZEROCOPY 60
+#define SO_TXTIME 61
+#define SCM_TXTIME SO_TXTIME
+
#endif /* _XTENSA_SOCKET_H */
diff --git a/include/linux/socket.h b/include/linux/socket.h
index 7ed4713d5337..ca476b7a8ff0 100644
--- a/include/linux/socket.h
+++ b/include/linux/socket.h
@@ -83,6 +83,11 @@ struct cmsghdr {
int cmsg_type; /* protocol-specific type */
};
+struct sock_txtime {
+ clockid_t clockid; /* reference clockid */
+ u16 flags; /* bit 0: txtime in deadline_mode */
+};
+
/*
* Ancillary data object information MACROS
* Table 5-14 of POSIX 1003.1g
diff --git a/include/net/sock.h b/include/net/sock.h
index b3b75419eafe..73f4404e49e4 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -315,6 +315,7 @@ struct sock_common {
* @sk_destruct: called at sock freeing time, i.e. when all refcnt == 0
* @sk_reuseport_cb: reuseport group container
* @sk_rcu: used during RCU grace period
+ * @sk_txtime: used by time-based scheduling
*/
struct sock {
/*
@@ -468,6 +469,11 @@ struct sock {
u8 sk_shutdown;
u32 sk_tskey;
atomic_t sk_zckey;
+
+ u16 sk_clockid;
+ u16 sk_txtime_flags;
+#define SK_TXTIME_DEADLINE_MASK BIT(0)
+
struct socket *sk_socket;
void *sk_user_data;
#ifdef CONFIG_SECURITY
@@ -783,6 +789,7 @@ enum sock_flags {
SOCK_FILTER_LOCKED, /* Filter cannot be changed anymore */
SOCK_SELECT_ERR_QUEUE, /* Wake select on error queue */
SOCK_RCU_FREE, /* wait rcu grace period in sk_destruct() */
+ SOCK_TXTIME,
};
#define SK_FLAGS_TIMESTAMP ((1UL << SOCK_TIMESTAMP) | (1UL << SOCK_TIMESTAMPING_RX_SOFTWARE))
@@ -1578,6 +1585,7 @@ void sock_kzfree_s(struct sock *sk, void *mem, int size);
void sk_send_sigurg(struct sock *sk);
struct sockcm_cookie {
+ u64 transmit_time;
u32 mark;
u16 tsflags;
};
diff --git a/include/uapi/asm-generic/socket.h b/include/uapi/asm-generic/socket.h
index 0ae758c90e54..a12692e5f7a8 100644
--- a/include/uapi/asm-generic/socket.h
+++ b/include/uapi/asm-generic/socket.h
@@ -107,4 +107,7 @@
#define SO_ZEROCOPY 60
+#define SO_TXTIME 61
+#define SCM_TXTIME SO_TXTIME
+
#endif /* __ASM_GENERIC_SOCKET_H */
diff --git a/net/core/sock.c b/net/core/sock.c
index bcc41829a16d..ef934d808941 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -91,6 +91,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <asm/unaligned.h>
#include <linux/capability.h>
#include <linux/errno.h>
#include <linux/errqueue.h>
@@ -697,6 +698,7 @@ EXPORT_SYMBOL(sk_mc_loop);
int sock_setsockopt(struct socket *sock, int level, int optname,
char __user *optval, unsigned int optlen)
{
+ struct sock_txtime sk_txtime;
struct sock *sk = sock->sk;
int val;
int valbool;
@@ -1070,6 +1072,22 @@ int sock_setsockopt(struct socket *sock, int level, int optname,
}
break;
+ case SO_TXTIME:
+ if (!ns_capable(sock_net(sk)->user_ns, CAP_NET_ADMIN)) {
+ ret = -EPERM;
+ } else if (optlen != sizeof(struct sock_txtime)) {
+ ret = -EINVAL;
+ } else if (copy_from_user(&sk_txtime, optval,
+ sizeof(struct sock_txtime))) {
+ ret = -EFAULT;
+ sock_valbool_flag(sk, SOCK_TXTIME, false);
+ } else {
+ sock_valbool_flag(sk, SOCK_TXTIME, true);
+ sk->sk_clockid = sk_txtime.clockid;
+ sk->sk_txtime_flags = sk_txtime.flags;
+ }
+ break;
+
default:
ret = -ENOPROTOOPT;
break;
@@ -1115,6 +1133,7 @@ int sock_getsockopt(struct socket *sock, int level, int optname,
u64 val64;
struct linger ling;
struct timeval tm;
+ struct sock_txtime txtime;
} v;
int lv = sizeof(int);
@@ -1403,6 +1422,12 @@ int sock_getsockopt(struct socket *sock, int level, int optname,
v.val = sock_flag(sk, SOCK_ZEROCOPY);
break;
+ case SO_TXTIME:
+ lv = sizeof(v.txtime);
+ v.txtime.clockid = sk->sk_clockid;
+ v.txtime.flags = sk->sk_txtime_flags;
+ break;
+
default:
/* We implement the SO_SNDLOWAT etc to not be settable
* (1003.1g 7).
@@ -2137,6 +2162,13 @@ int __sock_cmsg_send(struct sock *sk, struct msghdr *msg, struct cmsghdr *cmsg,
sockc->tsflags &= ~SOF_TIMESTAMPING_TX_RECORD_MASK;
sockc->tsflags |= tsflags;
break;
+ case SCM_TXTIME:
+ if (!sock_flag(sk, SOCK_TXTIME))
+ return -EINVAL;
+ if (cmsg->cmsg_len != CMSG_LEN(sizeof(u64)))
+ return -EINVAL;
+ sockc->transmit_time = get_unaligned((u64 *)CMSG_DATA(cmsg));
+ break;
/* SCM_RIGHTS and SCM_CREDENTIALS are semantically in SOL_UNIX. */
case SCM_RIGHTS:
case SCM_CREDENTIALS:
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 00/14] Scheduled packet Transmission: ETF
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri
Overview
========
This work consists of a set of kernel interfaces that can be used by
applications that require (time-based) Scheduled Tx of packets.
It is comprised by 3 new components to the kernel:
- SO_TXTIME: socket option + cmsg programming interfaces.
- etf: the "earliest txtime first" qdisc, that provides per-queue
TxTime-based scheduling. This has been renamed from 'tbs' to
'etf' to better describe its functionality.
- taprio: the "time-aware priority scheduler" qdisc, that provides
per-port Time-Aware scheduling;
This patchset is providing the first 2 components, which have been
developed for longer. The taprio qdisc will be shared as an RFC separately
(shortly).
Note that this series is a follow up of the "Time based packet
transmission" RFCv3 [1].
etf (formerly known as 'tbs')
=============================
Changes since the RFC v3:
- removed patch adding CLOCKID_INVALID;
- now we report packet drops through the socket's error queue;
- the usage of CLOCK_TAI is enforced by the qdisc;
- fixed bug on igb driver to avoid timestamps from being overwritten;
- simplified queueing modes by making 'sorting' mandatory;
- renamed qdisc from 'tbs' to 'etf'.
For applications/systems that the concept of time slices isn't precise
enough, the etf qdisc allows applications to control the instant when
a packet should leave the network controller. When used in conjunction
with taprio, it can also be used in case the application needs to
control with greater guarantee the offset into each time slice a packet
will be sent. Another use case of etf, is when only a small number of
applications on a system are time sensitive, so it can then be used
with a more traditional root qdisc (like mqprio).
The etf qdisc is designed so it buffers packets until a configurable
time before their deadline (Tx time). The qdisc uses a rbtree internally
so the buffered packets are always 'ordered' by their txtime (deadline)
and will be dequeued following the earliest txtime first.
The qdisc will drop any packets with a Tx time in the past, or if a
packet expires while waiting for being dequeued. Drops can be reported
as errors back to userspace through the socket's error queue.
Example configuration:
$ tc qdisc add dev enp2s0 parent 100:1 etf offload delta 200000 \
clockid CLOCK_TAI
Here, the Qdisc will use HW offload for the txtime control.
Packets will be dequeued by the qdisc "delta" (200000) nanoseconds before
their transmission time. Because this will be using HW offload and
since dynamic clocks are not supported by hrtimers, the system clock
and the PHC clock must be synchronized for this mode to behave as expected.
A more complete example can be found here, with instructions of how to
test it:
https://gist.github.com/jeez/bd3afeff081ba64a695008dd8215866f [2]
Note that we haven't modified the qdisc so it uses a timerqueue because
the modification needed was increasing the number of cachelines of a sk_buff.
SO_TXTIME
=========
Changes since the RFC v3:
- skb->tstamp is now cleared in skb_scrub_packet();
- transmit time is now set for other send paths, and not only the
"fast" ones as before;
- removed the per-packet parameters (clockid and drop_if_late).
Now just the skb->tstamp is used;
- flags and clockid_t are now set per-socket as a parameter of
SO_TXTIME.
This series is also hosted on github and can be found at [3].
The companion iproute2 patches can be found at [4].
[1] https://patchwork.ozlabs.org/cover/882342/
[2] github doesn't make it clear, but the gist can be cloned like this:
$ git clone https://gist.github.com/jeez/bd3afeff081ba64a695008dd8215866f scheduled-tx-tests
[3] https://github.com/jeez/linux/tree/etf-v1
[4] https://github.com/jeez/iproute2/tree/etf-v1
Jesus Sanchez-Palencia (10):
net: Clear skb->tstamp only on the forwarding path
net: ipv4: Hook into time based transmission
net/sched: Add HW offloading capability to ETF
igb: Refactor igb_configure_cbs()
igb: Only change Tx arbitration when CBS is on
igb: Refactor igb_offload_cbs()
igb: Add support for ETF offload
igb: Only call skb_tx_timestamp after descriptors are ready
net/sched: Enforce usage of CLOCK_TAI for sch_etf
net/sched: Make etf report drops on error_queue
Richard Cochran (2):
net: Add a new socket option for a future transmit time.
net: packet: Hook into time based transmission.
Vinicius Costa Gomes (2):
net/sched: Allow creating a Qdisc watchdog with other clocks
net/sched: Introduce the ETF Qdisc
arch/alpha/include/uapi/asm/socket.h | 3 +
arch/ia64/include/uapi/asm/socket.h | 3 +
arch/mips/include/uapi/asm/socket.h | 3 +
arch/parisc/include/uapi/asm/socket.h | 3 +
arch/s390/include/uapi/asm/socket.h | 3 +
arch/sparc/include/uapi/asm/socket.h | 3 +
arch/xtensa/include/uapi/asm/socket.h | 3 +
.../net/ethernet/intel/igb/e1000_defines.h | 16 +
drivers/net/ethernet/intel/igb/igb.h | 1 +
drivers/net/ethernet/intel/igb/igb_main.c | 257 ++++++---
include/linux/netdevice.h | 1 +
include/linux/socket.h | 7 +
include/net/inet_sock.h | 1 +
include/net/pkt_sched.h | 7 +
include/net/sock.h | 9 +
include/uapi/asm-generic/socket.h | 3 +
include/uapi/linux/errqueue.h | 2 +
include/uapi/linux/pkt_sched.h | 18 +
net/core/skbuff.c | 2 +-
net/core/sock.c | 32 ++
net/ipv4/ip_output.c | 3 +
net/ipv4/raw.c | 2 +
net/ipv4/udp.c | 1 +
net/packet/af_packet.c | 6 +
net/sched/Kconfig | 11 +
net/sched/Makefile | 1 +
net/sched/sch_api.c | 11 +-
net/sched/sch_etf.c | 487 ++++++++++++++++++
28 files changed, 829 insertions(+), 70 deletions(-)
create mode 100644 net/sched/sch_etf.c
--
2.17.1
^ permalink raw reply
* [PATCH v1 net-next 04/14] net: packet: Hook into time based transmission.
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri, Richard Cochran,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
From: Richard Cochran <rcochran@linutronix.de>
For raw layer-2 packets, copy the desired future transmit time from
the CMSG cookie into the skb.
Signed-off-by: Richard Cochran <rcochran@linutronix.de>
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
net/packet/af_packet.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index ff8e7e245c37..255c0164e0aa 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -1951,6 +1951,7 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg,
goto out_unlock;
}
+ sockc.transmit_time = 0;
sockc.tsflags = sk->sk_tsflags;
if (msg->msg_controllen) {
err = sock_cmsg_send(sk, msg, &sockc);
@@ -1962,6 +1963,7 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg,
skb->dev = dev;
skb->priority = sk->sk_priority;
skb->mark = sk->sk_mark;
+ skb->tstamp = sockc.transmit_time;
sock_tx_timestamp(sk, sockc.tsflags, &skb_shinfo(skb)->tx_flags);
@@ -2457,6 +2459,7 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb,
skb->dev = dev;
skb->priority = po->sk.sk_priority;
skb->mark = po->sk.sk_mark;
+ skb->tstamp = sockc->transmit_time;
sock_tx_timestamp(&po->sk, sockc->tsflags, &skb_shinfo(skb)->tx_flags);
skb_shinfo(skb)->destructor_arg = ph.raw;
@@ -2633,6 +2636,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg)
if (unlikely(!(dev->flags & IFF_UP)))
goto out_put;
+ sockc.transmit_time = 0;
sockc.tsflags = po->sk.sk_tsflags;
if (msg->msg_controllen) {
err = sock_cmsg_send(&po->sk, msg, &sockc);
@@ -2829,6 +2833,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
if (unlikely(!(dev->flags & IFF_UP)))
goto out_unlock;
+ sockc.transmit_time = 0;
sockc.tsflags = sk->sk_tsflags;
sockc.mark = sk->sk_mark;
if (msg->msg_controllen) {
@@ -2903,6 +2908,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len)
skb->dev = dev;
skb->priority = sk->sk_priority;
skb->mark = sockc.mark;
+ skb->tstamp = sockc.transmit_time;
if (has_vnet_hdr) {
err = virtio_net_hdr_to_skb(skb, &vnet_hdr, vio_le());
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 05/14] net/sched: Allow creating a Qdisc watchdog with other clocks
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
From: Vinicius Costa Gomes <vinicius.gomes@intel.com>
This adds 'qdisc_watchdog_init_clockid()' that allows a clockid to be
passed, this allows other time references to be used when scheduling
the Qdisc to run.
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
---
include/net/pkt_sched.h | 2 ++
net/sched/sch_api.c | 11 +++++++++--
2 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index 815b92a23936..2466ea143d01 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -72,6 +72,8 @@ struct qdisc_watchdog {
struct Qdisc *qdisc;
};
+void qdisc_watchdog_init_clockid(struct qdisc_watchdog *wd, struct Qdisc *qdisc,
+ clockid_t clockid);
void qdisc_watchdog_init(struct qdisc_watchdog *wd, struct Qdisc *qdisc);
void qdisc_watchdog_schedule_ns(struct qdisc_watchdog *wd, u64 expires);
diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c
index 54eca685420f..98541c6399db 100644
--- a/net/sched/sch_api.c
+++ b/net/sched/sch_api.c
@@ -596,12 +596,19 @@ static enum hrtimer_restart qdisc_watchdog(struct hrtimer *timer)
return HRTIMER_NORESTART;
}
-void qdisc_watchdog_init(struct qdisc_watchdog *wd, struct Qdisc *qdisc)
+void qdisc_watchdog_init_clockid(struct qdisc_watchdog *wd, struct Qdisc *qdisc,
+ clockid_t clockid)
{
- hrtimer_init(&wd->timer, CLOCK_MONOTONIC, HRTIMER_MODE_ABS_PINNED);
+ hrtimer_init(&wd->timer, clockid, HRTIMER_MODE_ABS_PINNED);
wd->timer.function = qdisc_watchdog;
wd->qdisc = qdisc;
}
+EXPORT_SYMBOL(qdisc_watchdog_init_clockid);
+
+void qdisc_watchdog_init(struct qdisc_watchdog *wd, struct Qdisc *qdisc)
+{
+ qdisc_watchdog_init_clockid(wd, qdisc, CLOCK_MONOTONIC);
+}
EXPORT_SYMBOL(qdisc_watchdog_init);
void qdisc_watchdog_schedule_ns(struct qdisc_watchdog *wd, u64 expires)
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 07/14] net/sched: Add HW offloading capability to ETF
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
Add infra so etf qdisc supports HW offload of time-based transmission.
For hw offload, the time sorted list is still used, so packets are
dequeued always in order of txtime.
Example:
$ tc qdisc replace dev enp2s0 parent root handle 100 mqprio num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 queues 1@0 1@1 2@2 hw 0
$ tc qdisc add dev enp2s0 parent 100:1 etf offload delta 100000 \
clockid CLOCK_REALTIME
In this example, the Qdisc will use HW offload for the control of the
transmission time through the network adapter. The hrtimer used for
packets scheduling inside the qdisc will use the clockid CLOCK_REALTIME
as reference and packets leave the Qdisc "delta" (100000) nanoseconds
before their transmission time. Because this will be using HW offload and
since dynamic clocks are not supported by the hrtimer, the system clock
and the PHC clock must be synchronized for this mode to behave as
expected.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
include/net/pkt_sched.h | 5 +++
include/uapi/linux/pkt_sched.h | 1 +
net/sched/sch_etf.c | 71 +++++++++++++++++++++++++++++++++-
3 files changed, 76 insertions(+), 1 deletion(-)
diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index 2466ea143d01..7dc769e5452b 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -155,4 +155,9 @@ struct tc_cbs_qopt_offload {
s32 sendslope;
};
+struct tc_etf_qopt_offload {
+ u8 enable;
+ s32 queue;
+};
+
#endif
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index 9d6fd2004a03..efad482e69d2 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -941,6 +941,7 @@ struct tc_etf_qopt {
__s32 clockid;
__u32 flags;
#define TC_ETF_DEADLINE_MODE_ON BIT(0)
+#define TC_ETF_OFFLOAD_ON BIT(1)
};
enum {
diff --git a/net/sched/sch_etf.c b/net/sched/sch_etf.c
index 5f01a285f399..cd6cb5b69228 100644
--- a/net/sched/sch_etf.c
+++ b/net/sched/sch_etf.c
@@ -20,8 +20,10 @@
#include <net/sock.h>
#define DEADLINE_MODE_IS_ON(x) ((x)->flags & TC_ETF_DEADLINE_MODE_ON)
+#define OFFLOAD_IS_ON(x) ((x)->flags & TC_ETF_OFFLOAD_ON)
struct etf_sched_data {
+ bool offload;
bool deadline_mode;
int clockid;
int queue;
@@ -45,6 +47,9 @@ static inline int validate_input_params(struct tc_etf_qopt *qopt,
* * Dynamic clockids are not supported.
*
* * Delta must be a positive integer.
+ *
+ * Also note that for the HW offload case, we must
+ * expect that system clocks have been synchronized to PHC.
*/
if (qopt->clockid < 0) {
NL_SET_ERR_MSG(extack, "Dynamic clockids are not supported");
@@ -226,6 +231,56 @@ static struct sk_buff *etf_dequeue_timesortedlist(struct Qdisc *sch)
return skb;
}
+static void etf_disable_offload(struct net_device *dev,
+ struct etf_sched_data *q)
+{
+ struct tc_etf_qopt_offload etf = { };
+ const struct net_device_ops *ops;
+ int err;
+
+ if (!q->offload)
+ return;
+
+ ops = dev->netdev_ops;
+ if (!ops->ndo_setup_tc)
+ return;
+
+ etf.queue = q->queue;
+ etf.enable = 0;
+
+ err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_ETF, &etf);
+ if (err < 0)
+ pr_warn("Couldn't disable ETF offload for queue %d\n",
+ etf.queue);
+}
+
+static int etf_enable_offload(struct net_device *dev, struct etf_sched_data *q,
+ struct netlink_ext_ack *extack)
+{
+ const struct net_device_ops *ops = dev->netdev_ops;
+ struct tc_etf_qopt_offload etf = { };
+ int err;
+
+ if (q->offload)
+ return 0;
+
+ if (!ops->ndo_setup_tc) {
+ NL_SET_ERR_MSG(extack, "Specified device does not support ETF offload");
+ return -EOPNOTSUPP;
+ }
+
+ etf.queue = q->queue;
+ etf.enable = 1;
+
+ err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_ETF, &etf);
+ if (err < 0) {
+ NL_SET_ERR_MSG(extack, "Specified device failed to setup ETF hardware offload");
+ return err;
+ }
+
+ return 0;
+}
+
static int etf_init(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
@@ -252,8 +307,9 @@ static int etf_init(struct Qdisc *sch, struct nlattr *opt,
qopt = nla_data(tb[TCA_ETF_PARMS]);
- pr_debug("delta %d clockid %d deadline %s\n",
+ pr_debug("delta %d clockid %d offload %s deadline %s\n",
qopt->delta, qopt->clockid,
+ OFFLOAD_IS_ON(qopt) ? "on" : "off",
DEADLINE_MODE_IS_ON(qopt) ? "on" : "off");
err = validate_input_params(qopt, extack);
@@ -262,9 +318,16 @@ static int etf_init(struct Qdisc *sch, struct nlattr *opt,
q->queue = sch->dev_queue - netdev_get_tx_queue(dev, 0);
+ if (OFFLOAD_IS_ON(qopt)) {
+ err = etf_enable_offload(dev, q, extack);
+ if (err < 0)
+ return err;
+ }
+
/* Everything went OK, save the parameters used. */
q->delta = qopt->delta;
q->clockid = qopt->clockid;
+ q->offload = OFFLOAD_IS_ON(qopt);
q->deadline_mode = DEADLINE_MODE_IS_ON(qopt);
switch (q->clockid) {
@@ -327,10 +390,13 @@ static void etf_reset(struct Qdisc *sch)
static void etf_destroy(struct Qdisc *sch)
{
struct etf_sched_data *q = qdisc_priv(sch);
+ struct net_device *dev = qdisc_dev(sch);
/* Only cancel watchdog if it's been initialized. */
if (q->watchdog.qdisc == sch)
qdisc_watchdog_cancel(&q->watchdog);
+
+ etf_disable_offload(dev, q);
}
static int etf_dump(struct Qdisc *sch, struct sk_buff *skb)
@@ -345,6 +411,9 @@ static int etf_dump(struct Qdisc *sch, struct sk_buff *skb)
opt.delta = q->delta;
opt.clockid = q->clockid;
+ if (q->offload)
+ opt.flags |= TC_ETF_OFFLOAD_ON;
+
if (q->deadline_mode)
opt.flags |= TC_ETF_DEADLINE_MODE_ON;
--
2.17.1
^ permalink raw reply related
* [PATCH v1 net-next 01/14] net: Clear skb->tstamp only on the forwarding path
From: Jesus Sanchez-Palencia @ 2018-06-27 21:59 UTC (permalink / raw)
To: netdev
Cc: tglx, jan.altenberg, vinicius.gomes, kurt.kanzenbach, henrik,
richardcochran, levi.pearson, ilias.apalodimas, ivan.khoronzhuk,
mlichvar, willemb, jhs, xiyou.wangcong, jiri,
Jesus Sanchez-Palencia
In-Reply-To: <20180627215950.6719-1-jesus.sanchez-palencia@intel.com>
This is done in preparation for the upcoming time based transmission
patchset. Now that skb->tstamp will be used to hold packet's txtime,
we must ensure that it is being cleared when traversing namespaces.
Also, doing that from skb_scrub_packet() before the early return would
break our feature when tunnels are used.
Signed-off-by: Jesus Sanchez-Palencia <jesus.sanchez-palencia@intel.com>
---
net/core/skbuff.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index b1f274f22d85..236802b35203 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -4898,7 +4898,6 @@ EXPORT_SYMBOL(skb_try_coalesce);
*/
void skb_scrub_packet(struct sk_buff *skb, bool xnet)
{
- skb->tstamp = 0;
skb->pkt_type = PACKET_HOST;
skb->skb_iif = 0;
skb->ignore_df = 0;
@@ -4913,6 +4912,7 @@ void skb_scrub_packet(struct sk_buff *skb, bool xnet)
ipvs_reset(skb);
skb_orphan(skb);
skb->mark = 0;
+ skb->tstamp = 0;
}
EXPORT_SYMBOL_GPL(skb_scrub_packet);
--
2.17.1
^ permalink raw reply related
* Re: [RFC PATCH] net, mm: account sock objects to kmemcg
From: Shakeel Butt @ 2018-06-27 22:03 UTC (permalink / raw)
To: eric.dumazet
Cc: Andrew Morton, Johannes Weiner, Vladimir Davydov, Greg Thelen,
Roman Gushchin, davem, Eric Dumazet, Kirill Tkhai, LKML, netdev,
Linux MM
In-Reply-To: <f08b2e2c-d4c6-7a80-10d9-104c0aab593b@gmail.com>
On Wed, Jun 27, 2018 at 2:51 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
>
>
> On 06/27/2018 01:41 PM, Shakeel Butt wrote:
> > Currently the kernel accounts the memory for network traffic through
> > mem_cgroup_[un]charge_skmem() interface. However the memory accounted
> > only includes the truesize of sk_buff which does not include the size of
> > sock objects. In our production environment, with opt-out kmem
> > accounting, the sock kmem caches (TCP[v6], UDP[v6], RAW[v6], UNIX) are
> > among the top most charged kmem caches and consume a significant amount
> > of memory which can not be left as system overhead. So, this patch
> > converts the kmem caches of more important sock objects to SLAB_ACCOUNT.
> >
> > Signed-off-by: Shakeel Butt <shakeelb@google.com>
> > ---
> > net/ipv4/raw.c | 1 +
> > net/ipv4/tcp_ipv4.c | 2 +-
> > net/ipv4/udp.c | 1 +
> > net/ipv6/raw.c | 1 +
> > net/ipv6/tcp_ipv6.c | 2 +-
> > net/ipv6/udp.c | 1 +
> > net/unix/af_unix.c | 1 +
> > 7 files changed, 7 insertions(+), 2 deletions(-)
>
>
> Hey, you just disclosed we do not use DCCP ;)
>
Oops.
>
> Joke aside, what about simply factorizing this stuff ?
>
This will opt-in all the sock kmem_caches which I think is better and
much smaller change. Should I resend this or do you want to send the
patch?
> diff --git a/net/core/sock.c b/net/core/sock.c
> index bcc41829a16d50714bdd3c25c976c0b7296fab84..b6714f8d7e9ba313723a6f619799c56230ff5fd4 100644
> --- a/net/core/sock.c
> +++ b/net/core/sock.c
> @@ -3243,7 +3243,8 @@ static int req_prot_init(const struct proto *prot)
>
> rsk_prot->slab = kmem_cache_create(rsk_prot->slab_name,
> rsk_prot->obj_size, 0,
> - prot->slab_flags, NULL);
> + SLAB_ACCOUNT | prot->slab_flags,
> + NULL);
>
> if (!rsk_prot->slab) {
> pr_crit("%s: Can't create request sock SLAB cache!\n",
> @@ -3258,7 +3259,8 @@ int proto_register(struct proto *prot, int alloc_slab)
> if (alloc_slab) {
> prot->slab = kmem_cache_create_usercopy(prot->name,
> prot->obj_size, 0,
> - SLAB_HWCACHE_ALIGN | prot->slab_flags,
> + SLAB_HWCACHE_ALIGN | SLAB_ACCOUNT |
> + prot->slab_flags,
> prot->useroffset, prot->usersize,
> NULL);
>
>
Shakeel
^ permalink raw reply
* Re: [RFC PATCH] net, mm: account sock objects to kmemcg
From: Eric Dumazet @ 2018-06-27 21:51 UTC (permalink / raw)
To: Shakeel Butt, Andrew Morton
Cc: Johannes Weiner, Vladimir Davydov, Greg Thelen, Roman Gushchin,
David S . Miller, Eric Dumazet, Kirill Tkhai, linux-kernel,
netdev, linux-mm
In-Reply-To: <20180627204139.225988-1-shakeelb@google.com>
On 06/27/2018 01:41 PM, Shakeel Butt wrote:
> Currently the kernel accounts the memory for network traffic through
> mem_cgroup_[un]charge_skmem() interface. However the memory accounted
> only includes the truesize of sk_buff which does not include the size of
> sock objects. In our production environment, with opt-out kmem
> accounting, the sock kmem caches (TCP[v6], UDP[v6], RAW[v6], UNIX) are
> among the top most charged kmem caches and consume a significant amount
> of memory which can not be left as system overhead. So, this patch
> converts the kmem caches of more important sock objects to SLAB_ACCOUNT.
>
> Signed-off-by: Shakeel Butt <shakeelb@google.com>
> ---
> net/ipv4/raw.c | 1 +
> net/ipv4/tcp_ipv4.c | 2 +-
> net/ipv4/udp.c | 1 +
> net/ipv6/raw.c | 1 +
> net/ipv6/tcp_ipv6.c | 2 +-
> net/ipv6/udp.c | 1 +
> net/unix/af_unix.c | 1 +
> 7 files changed, 7 insertions(+), 2 deletions(-)
Hey, you just disclosed we do not use DCCP ;)
Joke aside, what about simply factorizing this stuff ?
diff --git a/net/core/sock.c b/net/core/sock.c
index bcc41829a16d50714bdd3c25c976c0b7296fab84..b6714f8d7e9ba313723a6f619799c56230ff5fd4 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -3243,7 +3243,8 @@ static int req_prot_init(const struct proto *prot)
rsk_prot->slab = kmem_cache_create(rsk_prot->slab_name,
rsk_prot->obj_size, 0,
- prot->slab_flags, NULL);
+ SLAB_ACCOUNT | prot->slab_flags,
+ NULL);
if (!rsk_prot->slab) {
pr_crit("%s: Can't create request sock SLAB cache!\n",
@@ -3258,7 +3259,8 @@ int proto_register(struct proto *prot, int alloc_slab)
if (alloc_slab) {
prot->slab = kmem_cache_create_usercopy(prot->name,
prot->obj_size, 0,
- SLAB_HWCACHE_ALIGN | prot->slab_flags,
+ SLAB_HWCACHE_ALIGN | SLAB_ACCOUNT |
+ prot->slab_flags,
prot->useroffset, prot->usersize,
NULL);
^ permalink raw reply related
* [PATCH net-next 4/4] ila: Flush netlink command to clear xlat table
From: Tom Herbert @ 2018-06-27 21:39 UTC (permalink / raw)
To: davem, netdev; +Cc: Tom Herbert
In-Reply-To: <1530135542-10372-1-git-send-email-tom@quantonium.net>
Add ILA_CMD_FLUSH netlink command to clear the ILA translation table.
Signed-off-by: Tom Herbert <tom@quantonium.net>
---
include/uapi/linux/ila.h | 1 +
net/ipv6/ila/ila.h | 1 +
net/ipv6/ila/ila_main.c | 6 +++++
net/ipv6/ila/ila_xlat.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++--
4 files changed, 68 insertions(+), 2 deletions(-)
diff --git a/include/uapi/linux/ila.h b/include/uapi/linux/ila.h
index 483b77af..db45d3e 100644
--- a/include/uapi/linux/ila.h
+++ b/include/uapi/linux/ila.h
@@ -30,6 +30,7 @@ enum {
ILA_CMD_ADD,
ILA_CMD_DEL,
ILA_CMD_GET,
+ ILA_CMD_FLUSH,
__ILA_CMD_MAX,
};
diff --git a/net/ipv6/ila/ila.h b/net/ipv6/ila/ila.h
index faba782..1f747bc 100644
--- a/net/ipv6/ila/ila.h
+++ b/net/ipv6/ila/ila.h
@@ -123,6 +123,7 @@ void ila_xlat_exit_net(struct net *net);
int ila_xlat_nl_cmd_add_mapping(struct sk_buff *skb, struct genl_info *info);
int ila_xlat_nl_cmd_del_mapping(struct sk_buff *skb, struct genl_info *info);
int ila_xlat_nl_cmd_get_mapping(struct sk_buff *skb, struct genl_info *info);
+int ila_xlat_nl_cmd_flush(struct sk_buff *skb, struct genl_info *info);
int ila_xlat_nl_dump_start(struct netlink_callback *cb);
int ila_xlat_nl_dump_done(struct netlink_callback *cb);
int ila_xlat_nl_dump(struct sk_buff *skb, struct netlink_callback *cb);
diff --git a/net/ipv6/ila/ila_main.c b/net/ipv6/ila/ila_main.c
index f6ac6b1..18fac76 100644
--- a/net/ipv6/ila/ila_main.c
+++ b/net/ipv6/ila/ila_main.c
@@ -27,6 +27,12 @@ static const struct genl_ops ila_nl_ops[] = {
.flags = GENL_ADMIN_PERM,
},
{
+ .cmd = ILA_CMD_FLUSH,
+ .doit = ila_xlat_nl_cmd_flush,
+ .policy = ila_nl_policy,
+ .flags = GENL_ADMIN_PERM,
+ },
+ {
.cmd = ILA_CMD_GET,
.doit = ila_xlat_nl_cmd_get_mapping,
.start = ila_xlat_nl_dump_start,
diff --git a/net/ipv6/ila/ila_xlat.c b/net/ipv6/ila/ila_xlat.c
index d05de89..51a15ce 100644
--- a/net/ipv6/ila/ila_xlat.c
+++ b/net/ipv6/ila/ila_xlat.c
@@ -164,9 +164,9 @@ static inline void ila_release(struct ila_map *ila)
kfree_rcu(ila, rcu);
}
-static void ila_free_cb(void *ptr, void *arg)
+static void ila_free_node(struct ila_map *ila)
{
- struct ila_map *ila = (struct ila_map *)ptr, *next;
+ struct ila_map *next;
/* Assume rcu_readlock held */
while (ila) {
@@ -176,6 +176,11 @@ static void ila_free_cb(void *ptr, void *arg)
}
}
+static void ila_free_cb(void *ptr, void *arg)
+{
+ ila_free_node((struct ila_map *)ptr);
+}
+
static int ila_xlat_addr(struct sk_buff *skb, bool sir2ila);
static unsigned int
@@ -365,6 +370,59 @@ int ila_xlat_nl_cmd_del_mapping(struct sk_buff *skb, struct genl_info *info)
return 0;
}
+static inline spinlock_t *lock_from_ila_map(struct ila_net *ilan,
+ struct ila_map *ila)
+{
+ return ila_get_lock(ilan, ila->xp.ip.locator_match);
+}
+
+int ila_xlat_nl_cmd_flush(struct sk_buff *skb, struct genl_info *info)
+{
+ struct net *net = genl_info_net(info);
+ struct ila_net *ilan = net_generic(net, ila_net_id);
+ struct rhashtable_iter iter;
+ struct ila_map *ila;
+ spinlock_t *lock;
+ int ret;
+
+ ret = rhashtable_walk_init(&ilan->xlat.rhash_table, &iter, GFP_KERNEL);
+ if (ret)
+ goto done;
+
+ rhashtable_walk_start(&iter);
+
+ for (;;) {
+ ila = rhashtable_walk_next(&iter);
+
+ if (IS_ERR(ila)) {
+ if (PTR_ERR(ila) == -EAGAIN)
+ continue;
+ ret = PTR_ERR(ila);
+ goto done;
+ } else if (!ila) {
+ break;
+ }
+
+ lock = lock_from_ila_map(ilan, ila);
+
+ spin_lock(lock);
+
+ ret = rhashtable_remove_fast(&ilan->xlat.rhash_table,
+ &ila->node, rht_params);
+ if (!ret)
+ ila_free_node(ila);
+
+ spin_unlock(lock);
+
+ if (ret)
+ break;
+ }
+
+done:
+ rhashtable_walk_stop(&iter);
+ return ret;
+}
+
static int ila_fill_info(struct ila_map *ila, struct sk_buff *msg)
{
if (nla_put_u64_64bit(msg, ILA_ATTR_LOCATOR,
--
2.7.4
^ permalink raw reply related
* [PATCH net-next 3/4] ila: Create main ila source file
From: Tom Herbert @ 2018-06-27 21:39 UTC (permalink / raw)
To: davem, netdev; +Cc: Tom Herbert
In-Reply-To: <1530135542-10372-1-git-send-email-tom@quantonium.net>
Create a main ila file that contains the module initialization functions
as well as netlink definitions. Previously these were defined in
ila_xlat and ila_common. This approach allows better extensibility.
Signed-off-by: Tom Herbert <tom@quantonium.net>
---
net/ipv6/ila/Makefile | 2 +-
net/ipv6/ila/ila.h | 26 ++++++++-
net/ipv6/ila/ila_common.c | 30 ----------
net/ipv6/ila/ila_main.c | 115 +++++++++++++++++++++++++++++++++++++
net/ipv6/ila/ila_xlat.c | 142 +++++++++-------------------------------------
5 files changed, 168 insertions(+), 147 deletions(-)
create mode 100644 net/ipv6/ila/ila_main.c
diff --git a/net/ipv6/ila/Makefile b/net/ipv6/ila/Makefile
index 4b32e59..b7739ab 100644
--- a/net/ipv6/ila/Makefile
+++ b/net/ipv6/ila/Makefile
@@ -4,4 +4,4 @@
obj-$(CONFIG_IPV6_ILA) += ila.o
-ila-objs := ila_common.o ila_lwt.o ila_xlat.o
+ila-objs := ila_main.o ila_common.o ila_lwt.o ila_xlat.o
diff --git a/net/ipv6/ila/ila.h b/net/ipv6/ila/ila.h
index 3c7a11b..faba782 100644
--- a/net/ipv6/ila/ila.h
+++ b/net/ipv6/ila/ila.h
@@ -19,6 +19,7 @@
#include <linux/skbuff.h>
#include <linux/types.h>
#include <net/checksum.h>
+#include <net/genetlink.h>
#include <net/ip.h>
#include <net/protocol.h>
#include <uapi/linux/ila.h>
@@ -104,9 +105,30 @@ void ila_update_ipv6_locator(struct sk_buff *skb, struct ila_params *p,
void ila_init_saved_csum(struct ila_params *p);
+struct ila_net {
+ struct {
+ struct rhashtable rhash_table;
+ spinlock_t *locks; /* Bucket locks for entry manipulation */
+ unsigned int locks_mask;
+ bool hooks_registered;
+ } xlat;
+};
+
int ila_lwt_init(void);
void ila_lwt_fini(void);
-int ila_xlat_init(void);
-void ila_xlat_fini(void);
+
+int ila_xlat_init_net(struct net *net);
+void ila_xlat_exit_net(struct net *net);
+
+int ila_xlat_nl_cmd_add_mapping(struct sk_buff *skb, struct genl_info *info);
+int ila_xlat_nl_cmd_del_mapping(struct sk_buff *skb, struct genl_info *info);
+int ila_xlat_nl_cmd_get_mapping(struct sk_buff *skb, struct genl_info *info);
+int ila_xlat_nl_dump_start(struct netlink_callback *cb);
+int ila_xlat_nl_dump_done(struct netlink_callback *cb);
+int ila_xlat_nl_dump(struct sk_buff *skb, struct netlink_callback *cb);
+
+extern unsigned int ila_net_id;
+
+extern struct genl_family ila_nl_family;
#endif /* __ILA_H */
diff --git a/net/ipv6/ila/ila_common.c b/net/ipv6/ila/ila_common.c
index 8c88ecf..5793104 100644
--- a/net/ipv6/ila/ila_common.c
+++ b/net/ipv6/ila/ila_common.c
@@ -154,33 +154,3 @@ void ila_update_ipv6_locator(struct sk_buff *skb, struct ila_params *p,
iaddr->loc = p->locator;
}
-static int __init ila_init(void)
-{
- int ret;
-
- ret = ila_lwt_init();
-
- if (ret)
- goto fail_lwt;
-
- ret = ila_xlat_init();
- if (ret)
- goto fail_xlat;
-
- return 0;
-fail_xlat:
- ila_lwt_fini();
-fail_lwt:
- return ret;
-}
-
-static void __exit ila_fini(void)
-{
- ila_xlat_fini();
- ila_lwt_fini();
-}
-
-module_init(ila_init);
-module_exit(ila_fini);
-MODULE_AUTHOR("Tom Herbert <tom@herbertland.com>");
-MODULE_LICENSE("GPL");
diff --git a/net/ipv6/ila/ila_main.c b/net/ipv6/ila/ila_main.c
new file mode 100644
index 0000000..f6ac6b1
--- /dev/null
+++ b/net/ipv6/ila/ila_main.c
@@ -0,0 +1,115 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <net/genetlink.h>
+#include <net/ila.h>
+#include <net/netns/generic.h>
+#include <uapi/linux/genetlink.h>
+#include "ila.h"
+
+static const struct nla_policy ila_nl_policy[ILA_ATTR_MAX + 1] = {
+ [ILA_ATTR_LOCATOR] = { .type = NLA_U64, },
+ [ILA_ATTR_LOCATOR_MATCH] = { .type = NLA_U64, },
+ [ILA_ATTR_IFINDEX] = { .type = NLA_U32, },
+ [ILA_ATTR_CSUM_MODE] = { .type = NLA_U8, },
+ [ILA_ATTR_IDENT_TYPE] = { .type = NLA_U8, },
+};
+
+static const struct genl_ops ila_nl_ops[] = {
+ {
+ .cmd = ILA_CMD_ADD,
+ .doit = ila_xlat_nl_cmd_add_mapping,
+ .policy = ila_nl_policy,
+ .flags = GENL_ADMIN_PERM,
+ },
+ {
+ .cmd = ILA_CMD_DEL,
+ .doit = ila_xlat_nl_cmd_del_mapping,
+ .policy = ila_nl_policy,
+ .flags = GENL_ADMIN_PERM,
+ },
+ {
+ .cmd = ILA_CMD_GET,
+ .doit = ila_xlat_nl_cmd_get_mapping,
+ .start = ila_xlat_nl_dump_start,
+ .dumpit = ila_xlat_nl_dump,
+ .done = ila_xlat_nl_dump_done,
+ .policy = ila_nl_policy,
+ },
+};
+
+unsigned int ila_net_id;
+
+struct genl_family ila_nl_family __ro_after_init = {
+ .hdrsize = 0,
+ .name = ILA_GENL_NAME,
+ .version = ILA_GENL_VERSION,
+ .maxattr = ILA_ATTR_MAX,
+ .netnsok = true,
+ .parallel_ops = true,
+ .module = THIS_MODULE,
+ .ops = ila_nl_ops,
+ .n_ops = ARRAY_SIZE(ila_nl_ops),
+};
+
+static __net_init int ila_init_net(struct net *net)
+{
+ int err;
+
+ err = ila_xlat_init_net(net);
+ if (err)
+ goto ila_xlat_init_fail;
+
+ return 0;
+
+ila_xlat_init_fail:
+ return err;
+}
+
+static __net_exit void ila_exit_net(struct net *net)
+{
+ ila_xlat_exit_net(net);
+}
+
+static struct pernet_operations ila_net_ops = {
+ .init = ila_init_net,
+ .exit = ila_exit_net,
+ .id = &ila_net_id,
+ .size = sizeof(struct ila_net),
+};
+
+static int __init ila_init(void)
+{
+ int ret;
+
+ ret = register_pernet_device(&ila_net_ops);
+ if (ret)
+ goto register_device_fail;
+
+ ret = genl_register_family(&ila_nl_family);
+ if (ret)
+ goto register_family_fail;
+
+ ret = ila_lwt_init();
+ if (ret)
+ goto fail_lwt;
+
+ return 0;
+
+fail_lwt:
+ genl_unregister_family(&ila_nl_family);
+register_family_fail:
+ unregister_pernet_device(&ila_net_ops);
+register_device_fail:
+ return ret;
+}
+
+static void __exit ila_fini(void)
+{
+ ila_lwt_fini();
+ genl_unregister_family(&ila_nl_family);
+ unregister_pernet_device(&ila_net_ops);
+}
+
+module_init(ila_init);
+module_exit(ila_fini);
+MODULE_AUTHOR("Tom Herbert <tom@herbertland.com>");
+MODULE_LICENSE("GPL");
diff --git a/net/ipv6/ila/ila_xlat.c b/net/ipv6/ila/ila_xlat.c
index 9cc8bee..d05de89 100644
--- a/net/ipv6/ila/ila_xlat.c
+++ b/net/ipv6/ila/ila_xlat.c
@@ -22,21 +22,12 @@ struct ila_map {
struct rcu_head rcu;
};
-static unsigned int ila_net_id;
-
-struct ila_net {
- struct rhashtable rhash_table;
- spinlock_t *locks; /* Bucket locks for entry manipulation */
- unsigned int locks_mask;
- bool hooks_registered;
-};
-
#define MAX_LOCKS 1024
#define LOCKS_PER_CPU 10
static int alloc_ila_locks(struct ila_net *ilan)
{
- return alloc_bucket_spinlocks(&ilan->locks, &ilan->locks_mask,
+ return alloc_bucket_spinlocks(&ilan->xlat.locks, &ilan->xlat.locks_mask,
MAX_LOCKS, LOCKS_PER_CPU,
GFP_KERNEL);
}
@@ -58,7 +49,7 @@ static inline u32 ila_locator_hash(struct ila_locator loc)
static inline spinlock_t *ila_get_lock(struct ila_net *ilan,
struct ila_locator loc)
{
- return &ilan->locks[ila_locator_hash(loc) & ilan->locks_mask];
+ return &ilan->xlat.locks[ila_locator_hash(loc) & ilan->xlat.locks_mask];
}
static inline int ila_cmp_wildcards(struct ila_map *ila,
@@ -102,16 +93,6 @@ static const struct rhashtable_params rht_params = {
.obj_cmpfn = ila_cmpfn,
};
-static struct genl_family ila_nl_family;
-
-static const struct nla_policy ila_nl_policy[ILA_ATTR_MAX + 1] = {
- [ILA_ATTR_LOCATOR] = { .type = NLA_U64, },
- [ILA_ATTR_LOCATOR_MATCH] = { .type = NLA_U64, },
- [ILA_ATTR_IFINDEX] = { .type = NLA_U32, },
- [ILA_ATTR_CSUM_MODE] = { .type = NLA_U8, },
- [ILA_ATTR_IDENT_TYPE] = { .type = NLA_U8, },
-};
-
static int parse_nl_config(struct genl_info *info,
struct ila_xlat_params *xp)
{
@@ -149,7 +130,7 @@ static inline struct ila_map *ila_lookup_wildcards(struct ila_addr *iaddr,
{
struct ila_map *ila;
- ila = rhashtable_lookup_fast(&ilan->rhash_table, &iaddr->loc,
+ ila = rhashtable_lookup_fast(&ilan->xlat.rhash_table, &iaddr->loc,
rht_params);
while (ila) {
if (!ila_cmp_wildcards(ila, iaddr, ifindex))
@@ -166,7 +147,7 @@ static inline struct ila_map *ila_lookup_by_params(struct ila_xlat_params *xp,
{
struct ila_map *ila;
- ila = rhashtable_lookup_fast(&ilan->rhash_table,
+ ila = rhashtable_lookup_fast(&ilan->xlat.rhash_table,
&xp->ip.locator_match,
rht_params);
while (ila) {
@@ -222,7 +203,7 @@ static int ila_add_mapping(struct net *net, struct ila_xlat_params *xp)
spinlock_t *lock = ila_get_lock(ilan, xp->ip.locator_match);
int err = 0, order;
- if (!ilan->hooks_registered) {
+ if (!ilan->xlat.hooks_registered) {
/* We defer registering net hooks in the namespace until the
* first mapping is added.
*/
@@ -231,7 +212,7 @@ static int ila_add_mapping(struct net *net, struct ila_xlat_params *xp)
if (err)
return err;
- ilan->hooks_registered = true;
+ ilan->xlat.hooks_registered = true;
}
ila = kzalloc(sizeof(*ila), GFP_KERNEL);
@@ -246,12 +227,12 @@ static int ila_add_mapping(struct net *net, struct ila_xlat_params *xp)
spin_lock(lock);
- head = rhashtable_lookup_fast(&ilan->rhash_table,
+ head = rhashtable_lookup_fast(&ilan->xlat.rhash_table,
&xp->ip.locator_match,
rht_params);
if (!head) {
/* New entry for the rhash_table */
- err = rhashtable_lookup_insert_fast(&ilan->rhash_table,
+ err = rhashtable_lookup_insert_fast(&ilan->xlat.rhash_table,
&ila->node, rht_params);
} else {
struct ila_map *tila = head, *prev = NULL;
@@ -277,7 +258,7 @@ static int ila_add_mapping(struct net *net, struct ila_xlat_params *xp)
} else {
/* Make this ila new head */
RCU_INIT_POINTER(ila->next, head);
- err = rhashtable_replace_fast(&ilan->rhash_table,
+ err = rhashtable_replace_fast(&ilan->xlat.rhash_table,
&head->node,
&ila->node, rht_params);
if (err)
@@ -303,7 +284,7 @@ static int ila_del_mapping(struct net *net, struct ila_xlat_params *xp)
spin_lock(lock);
- head = rhashtable_lookup_fast(&ilan->rhash_table,
+ head = rhashtable_lookup_fast(&ilan->xlat.rhash_table,
&xp->ip.locator_match, rht_params);
ila = head;
@@ -333,15 +314,15 @@ static int ila_del_mapping(struct net *net, struct ila_xlat_params *xp)
* table
*/
err = rhashtable_replace_fast(
- &ilan->rhash_table, &ila->node,
+ &ilan->xlat.rhash_table, &ila->node,
&head->node, rht_params);
if (err)
goto out;
} else {
/* Entry no longer used */
- err = rhashtable_remove_fast(&ilan->rhash_table,
- &ila->node,
- rht_params);
+ err = rhashtable_remove_fast(
+ &ilan->xlat.rhash_table,
+ &ila->node, rht_params);
}
}
@@ -356,7 +337,7 @@ static int ila_del_mapping(struct net *net, struct ila_xlat_params *xp)
return err;
}
-static int ila_nl_cmd_add_mapping(struct sk_buff *skb, struct genl_info *info)
+int ila_xlat_nl_cmd_add_mapping(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
struct ila_xlat_params p;
@@ -369,7 +350,7 @@ static int ila_nl_cmd_add_mapping(struct sk_buff *skb, struct genl_info *info)
return ila_add_mapping(net, &p);
}
-static int ila_nl_cmd_del_mapping(struct sk_buff *skb, struct genl_info *info)
+int ila_xlat_nl_cmd_del_mapping(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
struct ila_xlat_params xp;
@@ -421,7 +402,7 @@ static int ila_dump_info(struct ila_map *ila,
return -EMSGSIZE;
}
-static int ila_nl_cmd_get_mapping(struct sk_buff *skb, struct genl_info *info)
+int ila_xlat_nl_cmd_get_mapping(struct sk_buff *skb, struct genl_info *info)
{
struct net *net = genl_info_net(info);
struct ila_net *ilan = net_generic(net, ila_net_id);
@@ -465,7 +446,7 @@ struct ila_dump_iter {
int skip;
};
-static int ila_nl_dump_start(struct netlink_callback *cb)
+int ila_xlat_nl_dump_start(struct netlink_callback *cb)
{
struct net *net = sock_net(cb->skb->sk);
struct ila_net *ilan = net_generic(net, ila_net_id);
@@ -476,7 +457,7 @@ static int ila_nl_dump_start(struct netlink_callback *cb)
if (!iter)
return -ENOMEM;
- ret = rhashtable_walk_init(&ilan->rhash_table, &iter->rhiter,
+ ret = rhashtable_walk_init(&ilan->xlat.rhash_table, &iter->rhiter,
GFP_KERNEL);
if (ret) {
kfree(iter);
@@ -489,7 +470,7 @@ static int ila_nl_dump_start(struct netlink_callback *cb)
return ret;
}
-static int ila_nl_dump_done(struct netlink_callback *cb)
+int ila_xlat_nl_dump_done(struct netlink_callback *cb)
{
struct ila_dump_iter *iter = (struct ila_dump_iter *)cb->args[0];
@@ -500,7 +481,7 @@ static int ila_nl_dump_done(struct netlink_callback *cb)
return 0;
}
-static int ila_nl_dump(struct sk_buff *skb, struct netlink_callback *cb)
+int ila_xlat_nl_dump(struct sk_buff *skb, struct netlink_callback *cb)
{
struct ila_dump_iter *iter = (struct ila_dump_iter *)cb->args[0];
struct rhashtable_iter *rhiter = &iter->rhiter;
@@ -570,77 +551,35 @@ static int ila_nl_dump(struct sk_buff *skb, struct netlink_callback *cb)
return ret;
}
-static const struct genl_ops ila_nl_ops[] = {
- {
- .cmd = ILA_CMD_ADD,
- .doit = ila_nl_cmd_add_mapping,
- .policy = ila_nl_policy,
- .flags = GENL_ADMIN_PERM,
- },
- {
- .cmd = ILA_CMD_DEL,
- .doit = ila_nl_cmd_del_mapping,
- .policy = ila_nl_policy,
- .flags = GENL_ADMIN_PERM,
- },
- {
- .cmd = ILA_CMD_GET,
- .doit = ila_nl_cmd_get_mapping,
- .start = ila_nl_dump_start,
- .dumpit = ila_nl_dump,
- .done = ila_nl_dump_done,
- .policy = ila_nl_policy,
- },
-};
-
-static struct genl_family ila_nl_family __ro_after_init = {
- .hdrsize = 0,
- .name = ILA_GENL_NAME,
- .version = ILA_GENL_VERSION,
- .maxattr = ILA_ATTR_MAX,
- .netnsok = true,
- .parallel_ops = true,
- .module = THIS_MODULE,
- .ops = ila_nl_ops,
- .n_ops = ARRAY_SIZE(ila_nl_ops),
-};
-
#define ILA_HASH_TABLE_SIZE 1024
-static __net_init int ila_init_net(struct net *net)
+int ila_xlat_init_net(struct net *net)
{
- int err;
struct ila_net *ilan = net_generic(net, ila_net_id);
+ int err;
err = alloc_ila_locks(ilan);
if (err)
return err;
- rhashtable_init(&ilan->rhash_table, &rht_params);
+ rhashtable_init(&ilan->xlat.rhash_table, &rht_params);
return 0;
}
-static __net_exit void ila_exit_net(struct net *net)
+void ila_xlat_exit_net(struct net *net)
{
struct ila_net *ilan = net_generic(net, ila_net_id);
- rhashtable_free_and_destroy(&ilan->rhash_table, ila_free_cb, NULL);
+ rhashtable_free_and_destroy(&ilan->xlat.rhash_table, ila_free_cb, NULL);
- free_bucket_spinlocks(ilan->locks);
+ free_bucket_spinlocks(ilan->xlat.locks);
- if (ilan->hooks_registered)
+ if (ilan->xlat.hooks_registered)
nf_unregister_net_hooks(net, ila_nf_hook_ops,
ARRAY_SIZE(ila_nf_hook_ops));
}
-static struct pernet_operations ila_net_ops = {
- .init = ila_init_net,
- .exit = ila_exit_net,
- .id = &ila_net_id,
- .size = sizeof(struct ila_net),
-};
-
static int ila_xlat_addr(struct sk_buff *skb, bool sir2ila)
{
struct ila_map *ila;
@@ -667,28 +606,3 @@ static int ila_xlat_addr(struct sk_buff *skb, bool sir2ila)
return 0;
}
-int __init ila_xlat_init(void)
-{
- int ret;
-
- ret = register_pernet_device(&ila_net_ops);
- if (ret)
- goto exit;
-
- ret = genl_register_family(&ila_nl_family);
- if (ret < 0)
- goto unregister;
-
- return 0;
-
-unregister:
- unregister_pernet_device(&ila_net_ops);
-exit:
- return ret;
-}
-
-void ila_xlat_fini(void)
-{
- genl_unregister_family(&ila_nl_family);
- unregister_pernet_device(&ila_net_ops);
-}
--
2.7.4
^ permalink raw reply related
* [PATCH net-next 2/4] ila: Call library function alloc_bucket_locks
From: Tom Herbert @ 2018-06-27 21:39 UTC (permalink / raw)
To: davem, netdev; +Cc: Tom Herbert
In-Reply-To: <1530135542-10372-1-git-send-email-tom@quantonium.net>
To allocate the array of bucket locks for the hash table we now
call library function alloc_bucket_spinlocks.
Signed-off-by: Tom Herbert <tom@quantonium.net>
---
net/ipv6/ila/ila_xlat.c | 23 +++++------------------
1 file changed, 5 insertions(+), 18 deletions(-)
diff --git a/net/ipv6/ila/ila_xlat.c b/net/ipv6/ila/ila_xlat.c
index 40f3f64..9cc8bee 100644
--- a/net/ipv6/ila/ila_xlat.c
+++ b/net/ipv6/ila/ila_xlat.c
@@ -31,27 +31,14 @@ struct ila_net {
bool hooks_registered;
};
+#define MAX_LOCKS 1024
#define LOCKS_PER_CPU 10
static int alloc_ila_locks(struct ila_net *ilan)
{
- unsigned int i, size;
- unsigned int nr_pcpus = num_possible_cpus();
-
- nr_pcpus = min_t(unsigned int, nr_pcpus, 32UL);
- size = roundup_pow_of_two(nr_pcpus * LOCKS_PER_CPU);
-
- if (sizeof(spinlock_t) != 0) {
- ilan->locks = kvmalloc_array(size, sizeof(spinlock_t),
- GFP_KERNEL);
- if (!ilan->locks)
- return -ENOMEM;
- for (i = 0; i < size; i++)
- spin_lock_init(&ilan->locks[i]);
- }
- ilan->locks_mask = size - 1;
-
- return 0;
+ return alloc_bucket_spinlocks(&ilan->locks, &ilan->locks_mask,
+ MAX_LOCKS, LOCKS_PER_CPU,
+ GFP_KERNEL);
}
static u32 hashrnd __read_mostly;
@@ -640,7 +627,7 @@ static __net_exit void ila_exit_net(struct net *net)
rhashtable_free_and_destroy(&ilan->rhash_table, ila_free_cb, NULL);
- kvfree(ilan->locks);
+ free_bucket_spinlocks(ilan->locks);
if (ilan->hooks_registered)
nf_unregister_net_hooks(net, ila_nf_hook_ops,
--
2.7.4
^ permalink raw reply related
* [PATCH net-next 1/4] ila: Fix use of rhashtable walk in ila_xlat.c
From: Tom Herbert @ 2018-06-27 21:38 UTC (permalink / raw)
To: davem, netdev; +Cc: Tom Herbert
In-Reply-To: <1530135542-10372-1-git-send-email-tom@quantonium.net>
Perform better EAGAIN handling, handle case where ila_dump_info
fails and we missed objects in the dump, and add a skip index
to skip over ila entires in a list on a rhashtable node that have
already been visited (by a previous call to ila_nl_dump).
Signed-off-by: Tom Herbert <tom@quantonium.net>
---
net/ipv6/ila/ila_xlat.c | 70 ++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 54 insertions(+), 16 deletions(-)
diff --git a/net/ipv6/ila/ila_xlat.c b/net/ipv6/ila/ila_xlat.c
index 10ae135..40f3f64 100644
--- a/net/ipv6/ila/ila_xlat.c
+++ b/net/ipv6/ila/ila_xlat.c
@@ -475,24 +475,31 @@ static int ila_nl_cmd_get_mapping(struct sk_buff *skb, struct genl_info *info)
struct ila_dump_iter {
struct rhashtable_iter rhiter;
+ int skip;
};
static int ila_nl_dump_start(struct netlink_callback *cb)
{
struct net *net = sock_net(cb->skb->sk);
struct ila_net *ilan = net_generic(net, ila_net_id);
- struct ila_dump_iter *iter = (struct ila_dump_iter *)cb->args[0];
+ struct ila_dump_iter *iter;
+ int ret;
- if (!iter) {
- iter = kmalloc(sizeof(*iter), GFP_KERNEL);
- if (!iter)
- return -ENOMEM;
+ iter = kmalloc(sizeof(*iter), GFP_KERNEL);
+ if (!iter)
+ return -ENOMEM;
- cb->args[0] = (long)iter;
+ ret = rhashtable_walk_init(&ilan->rhash_table, &iter->rhiter,
+ GFP_KERNEL);
+ if (ret) {
+ kfree(iter);
+ return ret;
}
- return rhashtable_walk_init(&ilan->rhash_table, &iter->rhiter,
- GFP_KERNEL);
+ iter->skip = 0;
+ cb->args[0] = (long)iter;
+
+ return ret;
}
static int ila_nl_dump_done(struct netlink_callback *cb)
@@ -510,20 +517,45 @@ static int ila_nl_dump(struct sk_buff *skb, struct netlink_callback *cb)
{
struct ila_dump_iter *iter = (struct ila_dump_iter *)cb->args[0];
struct rhashtable_iter *rhiter = &iter->rhiter;
+ int skip = iter->skip;
struct ila_map *ila;
int ret;
rhashtable_walk_start(rhiter);
- for (;;) {
- ila = rhashtable_walk_next(rhiter);
+ /* Get first entry */
+ ila = rhashtable_walk_peek(rhiter);
+
+ if (ila && !IS_ERR(ila) && skip) {
+ /* Skip over visited entries */
+
+ while (ila && skip) {
+ /* Skip over any ila entries in this list that we
+ * have already dumped.
+ */
+ ila = rcu_access_pointer(ila->next);
+ skip--;
+ }
+ }
+ skip = 0;
+
+ for (;;) {
if (IS_ERR(ila)) {
- if (PTR_ERR(ila) == -EAGAIN)
- continue;
ret = PTR_ERR(ila);
- goto done;
+ if (ret == -EAGAIN) {
+ /* Table has changed and iter has reset. Return
+ * -EAGAIN to the application even if we have
+ * written data to the skb. The application
+ * needs to deal with this.
+ */
+
+ goto out_ret;
+ } else {
+ break;
+ }
} else if (!ila) {
+ ret = 0;
break;
}
@@ -532,15 +564,21 @@ static int ila_nl_dump(struct sk_buff *skb, struct netlink_callback *cb)
cb->nlh->nlmsg_seq, NLM_F_MULTI,
skb, ILA_CMD_GET);
if (ret)
- goto done;
+ goto out;
+ skip++;
ila = rcu_access_pointer(ila->next);
}
+
+ skip = 0;
+ ila = rhashtable_walk_next(rhiter);
}
- ret = skb->len;
+out:
+ iter->skip = skip;
+ ret = (skb->len ? : ret);
-done:
+out_ret:
rhashtable_walk_stop(rhiter);
return ret;
}
--
2.7.4
^ permalink raw reply related
* [PATCH net-next 0/4] ila: Cleanup
From: Tom Herbert @ 2018-06-27 21:38 UTC (permalink / raw)
To: davem, netdev; +Cc: Tom Herbert
Perform some cleanup in ILA code. This includes:
- Fix rhashtable walk for cases where nl dumps are done with muliple
function calls. Add a skip index to skip over entries in
a node that have been previously visitied. Call rhashtable_walk_peek
to avoid dropping items between calls to ila_nl_dump.
- Call alloc_bucket_spinlocks to create bucket locks.
- Split out module initialization and netlink definitions into
separate files.
- Add ILA_CMD_FLUSH netlink command to clear the ILA translation table.
Tom Herbert (4):
ila: Fix use of rhashtable walk in ila_xlat.c
ila: Call library function alloc_bucket_locks
ila: Create main ila source file
ila: Flush netlink command to clear xlat table
include/uapi/linux/ila.h | 1 +
net/ipv6/ila/Makefile | 2 +-
net/ipv6/ila/ila.h | 27 ++++-
net/ipv6/ila/ila_common.c | 30 -----
net/ipv6/ila/ila_main.c | 121 +++++++++++++++++++
net/ipv6/ila/ila_xlat.c | 291 +++++++++++++++++++++++-----------------------
6 files changed, 292 insertions(+), 180 deletions(-)
create mode 100644 net/ipv6/ila/ila_main.c
--
2.7.4
^ permalink raw reply
* Re: [PATCH rdma-next 08/12] overflow.h: Add arithmetic shift helper
From: Rasmus Villemoes @ 2018-06-27 21:35 UTC (permalink / raw)
To: Leon Romanovsky, Jason Gunthorpe
Cc: Doug Ledford, Kees Cook, RDMA mailing list, Hadar Hen Zion,
Matan Barak, Michael J Ruhl, Noa Osherovich, Raed Salem,
Yishai Hadas, Saeed Mahameed, linux-netdev, linux-kernel
In-Reply-To: <20180627182220.GV30877@mtr-leonro.mtl.com>
On 2018-06-27 20:22, Leon Romanovsky wrote:
> On Wed, Jun 27, 2018 at 12:10:12PM -0600, Jason Gunthorpe wrote:
>> On Wed, Jun 27, 2018 at 11:36:03AM +0200, Rasmus Villemoes wrote:
>>> Well, the types you can check at compile-time, the values not, so you
>>> still have to define the result, i.e. contents of *d, for negative
>>> values (even if we decide that "overflow" should always be signalled in
>>> that case).
>>
>> Why do a need to define a 'result' beyond whatever the not-undefined
>> behavior shift expression produces?
Well, perhaps you don't, it's just that the other check_*_overflow have
that behaviour (which they inherit from gcc's builtins), and it's a
nice-to-have. But I see that it's hard to come up with something
sensible in all the "doesn't make sense" cases. When writing the tests
for test_overflow.c, you can of course just omit the comparison to
the/an expected result in the overflow case.
>> /*
>> * Compute *d = (a << s)
>> *
>> * Returns true if '*d' cannot hold the result or 'a << s' doesn't make sense.
>> * - 'a << s' causes bits to be lost when stored in d
>> * - 's' is garbage (eg negative) or so large that a << s is guarenteed to be 0
>> * - 'a' is negative
>> * - 'a << s' sets the sign bit, if any, in '*d'
>> * *d is not defined if false is returned.
>> */
>> #define check_shift_overflow(a, s, d) \
>> ({ \
>> typeof(a) _a = a; \
>> typeof(s) _s = s; \
>> typeof(d) _d = d; \
>> u64 _a_full = _a; \
>> unsigned int _to_shift = \
>> _s >= 0 && _s < 8 * sizeof(*d) ? _s : 0; \
>> \
>> *_d = (_a_full << _to_shift); \
>> \
>> (_to_shift != _s || *_d < 0 || _a < 0 || \
>> (*_d >> _to_shift) != a); \
>> })
That last a still needs to be _a. Other than that, I don't see anything
wrong with this version.
>> int main(int argc, const char *argv[])
>> {
>> int32_t s32;
>> uint32_t u32;
>>
>> assert(check_shift_overflow(1, 0, &s32) == false && s32 == (1 << 0));
[...]>> assert(check_shift_overflow(0xFFFFFFFF, 1, &s32) == true);
Please add these in some form to test_overflow.c, but do also include
cases where a and *d have different width, e.g. check_shift_overflow(1,
32, &s64) should be ok, while check_shift_overflow(65432, 0, &s16)
should not.
Rasmus
^ permalink raw reply
* [PATCH v12 10/10] MAINTAINERS: Add entry for drivers/net/ethernet/cavium/octeon/octeon3-*
From: Steven J. Hill @ 2018-06-27 21:25 UTC (permalink / raw)
To: netdev; +Cc: David Daney, Chandrakala Chavva
In-Reply-To: <1530134719-19407-1-git-send-email-steven.hill@cavium.com>
From: David Daney <david.daney@cavium.com>
Signed-off-by: David Daney <david.daney@cavium.com>
---
MAINTAINERS | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 99e5cef..378009c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3279,6 +3279,12 @@ W: http://www.cavium.com
S: Supported
F: drivers/mmc/host/cavium*
+CAVIUM OCTEON-III NETWORK DRIVER
+M: Steven J. Hill <Steven.Hill@cavium.com>
+L: netdev@vger.kernel.org
+S: Supported
+F: drivers/net/ethernet/cavium/octeon/octeon3-*
+
CAVIUM OCTEON-TX CRYPTO DRIVER
M: George Cherian <george.cherian@cavium.com>
L: linux-crypto@vger.kernel.org
--
2.1.4
^ permalink raw reply related
* [PATCH v12 09/10] netdev: cavium: octeon: Add Octeon III BGX Ethernet building
From: Steven J. Hill @ 2018-06-27 21:25 UTC (permalink / raw)
To: netdev; +Cc: Carlos Munoz, Chandrakala Chavva, Steven J. Hill
In-Reply-To: <1530134719-19407-1-git-send-email-steven.hill@cavium.com>
From: Carlos Munoz <cmunoz@cavium.com>
Add the build and configuration files for the BGX Ethernet.
Signed-off-by: Carlos Munoz <cmunoz@cavium.com>
Signed-off-by: Steven J. Hill <Steven.Hill@cavium.com>
---
drivers/net/ethernet/cavium/Kconfig | 22 +++++++++++++++++++++-
drivers/net/ethernet/cavium/octeon/Makefile | 8 +++++++-
2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/cavium/Kconfig b/drivers/net/ethernet/cavium/Kconfig
index 043e3c1..3b9709d 100644
--- a/drivers/net/ethernet/cavium/Kconfig
+++ b/drivers/net/ethernet/cavium/Kconfig
@@ -4,7 +4,7 @@
config NET_VENDOR_CAVIUM
bool "Cavium ethernet drivers"
- depends on PCI
+ depends on PCI || CAVIUM_OCTEON_SOC
default y
---help---
Select this option if you want enable Cavium network support.
@@ -100,4 +100,24 @@ config LIQUIDIO_VF
will be called liquidio_vf. MSI-X interrupt support is required
for this driver to work correctly
+config OCTEON3_BGX_PORT
+ tristate "Cavium Octeon III BGX port support"
+ depends on CAVIUM_OCTEON_SOC
+ ---help---
+ This driver adds support for Cavium Octeon III BGX ports. BGX ports
+ support sgmii, rgmii, xaui, rxaui, xlaui, xfi, 10KR and 40KR modes.
+
+ Say Y to use the management port on Octeon III boards or to use
+ any other ethernet port.
+
+config OCTEON3_ETHERNET
+ tristate "Cavium OCTEON III PKI/PKO Ethernet support"
+ depends on CAVIUM_OCTEON_SOC
+ select OCTEON_BGX_PORT
+ select OCTEON_FPA3
+ select FW_LOADER
+ ---help---
+ Support for 'BGX' Ethernet via PKI/PKO units. No support for
+ cn70xx chips, use OCTEON_ETHERNET instead.
+
endif # NET_VENDOR_CAVIUM
diff --git a/drivers/net/ethernet/cavium/octeon/Makefile b/drivers/net/ethernet/cavium/octeon/Makefile
index efa41c1..1939c84 100644
--- a/drivers/net/ethernet/cavium/octeon/Makefile
+++ b/drivers/net/ethernet/cavium/octeon/Makefile
@@ -1,5 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0
#
# Makefile for the Cavium network device drivers.
#
-obj-$(CONFIG_OCTEON_MGMT_ETHERNET) += octeon_mgmt.o
+obj-$(CONFIG_OCTEON_MGMT_ETHERNET) += octeon_mgmt.o
+obj-$(CONFIG_OCTEON3_BGX_PORT) += octeon3-bgx-nexus.o octeon3-bgx-port.o
+obj-$(CONFIG_OCTEON3_ETHERNET) += octeon3-ethernet.o
+
+octeon3-ethernet-objs += octeon3-core.o octeon3-pki.o octeon3-pko.o \
+ octeon3-sso.o
--
2.1.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox