* Re: [PATCH net-next v1 08/11] xdp: tracking page_pool resources and safe removal
From: Ivan Khoronzhuk @ 2019-06-18 13:48 UTC (permalink / raw)
To: Ilias Apalodimas
Cc: Tariq Toukan, Jesper Dangaard Brouer, netdev@vger.kernel.org,
Toke Høiland-Jørgensen, toshiaki.makita1@gmail.com,
grygorii.strashko@ti.com, mcroce@redhat.com
In-Reply-To: <20190618133012.GA2055@apalos>
On Tue, Jun 18, 2019 at 04:30:12PM +0300, Ilias Apalodimas wrote:
>Hi Ivan, Tariq,
>
>> >>>+
>[...]
>> >>
>> >>What would you recommend to do for the following situation:
>> >>
>> >>Same receive queue is shared between 2 network devices. The receive ring is
>> >>filled by pages from page_pool, but you don't know the actual port (ndev)
>> >>filling this ring, because a device is recognized only after packet is
>> >>received.
>> >>
>> >>The API is so that xdp rxq is bind to network device, each frame has
>> >>reference
>> >>on it, so rxq ndev must be static. That means each netdev has it's own rxq
>> >>instance even no need in it. Thus, after your changes, page must be
>> >>returned to
>> >>the pool it was taken from, or released from old pool and recycled in
>> >>new one
>> >>somehow.
>> >>
>> >>And that is inconvenience at least. It's hard to move pages between
>> >>pools w/o
>> >>performance penalty. No way to use common pool either, as unreg_rxq now
>> >>drops
>> >>the pool and 2 rxqa can't reference same pool.
>> >>
>> >
>> >Within the single netdev, separate page_pool instances are anyway
>> >created for different RX rings, working under different NAPI's.
>>
>> The circumstances are so that same RX ring is shared between 2
>> netdevs... and netdev can be known only after descriptor/packet is
>> received. Thus, while filling RX ring, there is no actual device,
>> but when packet is received it has to be recycled to appropriate
>> net device pool. Before this change there were no difference from
>> which pool the page was allocated to fill RX ring, as there were no
>> owner. After this change there is owner - netdev page pool.
>>
>> For cpsw the dma unmap is common for both netdevs and no difference
>> who is freeing the page, but there is difference which pool it's
>> freed to.
>Since 2 netdevs are sharing one queue you'll need locking right?
>(Assuming that the rx-irq per device can end up on a different core)
No, rx-irq is not per device, same queue is shared only for descs,
farther it's separate queue with separate pool. Even rx-irq is per
device, no issues, as I said, it has it's own page pool, every queue
and every ndev, no need in locking, for pools...
>We discussed that ideally page pools should be alocated per hardware queue.
This is about one hw queue shared between ndevs. Page pool is separate
for each hw queues and each ndevs ofc.
>If you indeed need locking (and pay the performance penalty anyway) i wonder if
>there's anything preventing you from keeping the same principle, i.e allocate a
>pool per queue
page pool is per queue.
>pool per queue and handle the recycling to the proper ndev internally.
>That way only the first device will be responsible of
>allocating/recycling/maintaining the pool state.
No. There is more dependencies then it looks like, see rxq_info ...
The final recycling is ended not internally.
--
Regards,
Ivan Khoronzhuk
^ permalink raw reply
* Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver
From: Arnd Bergmann @ 2019-06-18 13:48 UTC (permalink / raw)
To: Alex Elder
Cc: Johannes Berg, abhishek.esse, Ben Chan, Bjorn Andersson, cpratapa,
David Miller, Dan Williams, DTML, Eric Caruso, evgreen,
Ilias Apalodimas, Linux ARM, linux-arm-msm,
Linux Kernel Mailing List, linux-soc, Networking,
Subash Abhinov Kasiviswanathan, syadagir
In-Reply-To: <b3686626-e2d8-bc9c-6dd0-9ebb137715af@linaro.org>
On Tue, Jun 18, 2019 at 3:16 PM Alex Elder <elder@linaro.org> wrote:
> On 6/17/19 6:28 AM, Johannes Berg wrote:
> > On Tue, 2019-06-11 at 13:56 +0200, Arnd Bergmann wrote:
>
> I'm probably missing something, but I think the checksum
> offload could be handled by the IPA driver rather than
> rmnet. It seems to be an add-on that is completely
> independent of the multiplexing and aggregation capabilities
> that QMAP provides.
My best guess is that it is part of rmnet simply because this can
be done in a generic way for any qmap based back-end, and rmnet
was intended as the abstraction for qmap.
A better implementation of the checksumming might be to split
it out into a library that is in turn used by qmap drivers. Since this
should be transparent to the user interface, it can be moved
there later.
> >>> If true though, then I think this would be the killer argument *in
> >>> favour* of *not* merging this - because that would mean we *don't* have
> >>> to actually keep the rmnet API around for all foreseeable future.
>
> This is because it's a user space API? If so I now understand
> what you mean.
Yes, I think agreeing on the general user interface is (as usual) the
one thing that has to be done as a prerequisite. I had originally
hoped that by removing the ioctl interface portion of the driver,
this could be avoided, but that was before I had any idea on the
upper layers.
Arnd
^ permalink raw reply
* Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver
From: Alex Elder @ 2019-06-18 13:45 UTC (permalink / raw)
To: Johannes Berg, Arnd Bergmann, Dan Williams
Cc: Subash Abhinov Kasiviswanathan, abhishek.esse, Ben Chan,
Bjorn Andersson, cpratapa, David Miller, DTML, Eric Caruso,
evgreen, Ilias Apalodimas, Linux ARM, linux-arm-msm,
Linux Kernel Mailing List, linux-soc, Networking, syadagir
In-Reply-To: <dbb32f185d2c3a654083ee0a7188379e1f88d899.camel@sipsolutions.net>
On 6/17/19 6:42 AM, Johannes Berg wrote:
> On Wed, 2019-06-12 at 17:06 +0200, Arnd Bergmann wrote:
>> On Wed, Jun 12, 2019 at 4:28 PM Dan Williams <dcbw@redhat.com> wrote:
>>> On Wed, 2019-06-12 at 10:31 +0200, Arnd Bergmann wrote:
>>>> On Tue, Jun 11, 2019 at 7:23 PM Dan Williams <dcbw@redhat.com> wrote:
>>>
>>> I was trying to make the point that rmnet doesn't need to care about
>>> how the QMAP packets get to the device itself; it can be pretty generic
>>> so that it can be used by IPA/qmi_wwan/rmnet_smd/etc.
>>
>> rmnet at the moment is completely generic in that regard already,
>> however it is implemented as a tunnel driver talking to another
>> device rather than an abstraction layer below that driver.
>
> It doesn't really actually *do* much other than muck with the headers a
> small amount, but even that isn't really much.
>
> You can probably implement that far more efficiently on some devices
> where you have a semi-decent DMA engine that at least supports S/G.
If it had a well-defined way of creating new channels to be
multiplexed over the connection to the modem, the IPA driver
(rather than the rmnet driver) could present network interfaces
for each and perform the multiplexing. As I think Arnd
suggested, this could at least partially be done with library
code (to be shared with other "back-end" interfaces) rather
than using a layered driver. This applies to aggregation,
channel flow control, and checksum offload as well.
But I'm only familiar with IPA; I don't know whether the above
statements make any sense for other "back-end" drivers.
>>>> I understand that the rmnet model was intended to provide a cleaner
>>>> abstraction, but it's not how we normally structure subsystems in
>>>> Linux, and moving to a model more like how wireless_dev works
>>>> would improve both readability and performance, as you describe
>>>> it, it would be more like (ignoring for now the need for multiple
>>>> connections):
>>>>
>>>> ipa_dev
>>>> rmnet_dev
>>>> wwan_dev
>>>> net_device
>>>
>>> Perhaps I'm assuming too much from this diagram but this shows a 1:1
>>> between wwan_dev and "lower" devices.
>
> I guess the fuller picture would be something like
>
> ipa_dev
> rmnet_dev
> wwan_dev
> net_device*
>
> (i.e. with multiple net_devices)
>
>>> What Johannes is proposing (IIRC) is something a bit looser where a
>>> wwan_dev does not necessarily provide netdev itself, but is instead the
>>> central point that various channels (control, data, gps, sim card, etc)
>>> register with. That way the wwan_dev can provide an overall view of the
>>> WWAN device to userspace, and userspace can talk to the wwan_dev to ask
>>> the lower drivers (ipa, rmnet, etc) to create new channels (netdev,
>>> tty, otherwise) when the control channel has told the modem firmware to
>>> expect one.
>
> Yeah, that's more what I had in mind after all our discussions (will
> continue this below).
This is great. The start of a more concrete discussion of the
pieces that are missing...
>> Right, as I noted above, I simplified it a bit. We probably want to
>> have multiple net_device instances for an ipa_dev, so there has
>> to be a 1:n relationship instead of 1:1 at one of the intermediate
>> levels, but it's not obvious which level that should be.
>>
>> In theory we could even have a single net_device instance correspond
>> to the ipa_dev, but then have multiple IP addresses bound to it,
>> so each IP address corresponds to a channel/queue/napi_struct,
>> but the user visible object remains a single device.
>
> I don't think this latter (multiple IP addresses) works well - you want
> a hardware specific header ("ETH_P_MAP") to carry the channel ID,
> without looking up the IP address and all that.
I agree with this. It's not just multiple IP addresses for
an interface, it really is multiplexed--with channel ids.
It's another addressing parameter orthogonal to the IP space.
> But anyway, as I alluded to above, I had something like this in mind:
>
> driver_dev
> struct device *dev (USB, PCI, ...)
> net_device NA
> net_device NB
> tty TA
> ...
>
> (I'm cutting out the rmnet layer here for now)
>
> while having a separate that just links all the pieces together:
>
> wwan_device W
> ---> dev
> ---> NA
> ---> NB
> ---> TA
>
> So the driver is still responsible for creating the netdevs (or can of
> course delegate that to an "rmnet" library), but then all it also does
> is register the netdevs with the WWAN core like
>
> wwan_add_netdev(dev, NA)
>
> and the WWAN core would allocate the wwan_device W for this.
That would be nice. I believe you're saying that (in my case)
the IPA driver creates and owns the netdevices.
But I think the IPA driver would register with the WWAN core as
a "provider," and then the WWAN core would subsequently request
that it instantiate netdevices to represent channels on demand
(rather than registering them).
> That way, the drivers can concentrate on providing all the necessary
> bits, and - crucially - even *different* drivers can end up linking to
> the same wwan_device. For example, if you have a modem that has a multi-
> function USB device, then an ethernet driver might create the netdev and
> a tty driver might create the control channel, but if they both agree on
> using the right "struct device" instance, you can still get the correct
> wwan_device out of it all.
>
> And, in fact, some should then be
>
> wwan_maybe_add_netdev(dev, N)
>
> because the ethernet driver may not know if it attached to a modem or
> not, but if the control channel also attaches it's a modem for sure,
> with that ethernet channel attached to it.
>
> Additionally, I'm thinking API such as
>
> wwan_add(dev, &ops, opsdata)
>
> that doesn't automatically attach any channels, but provides "ops" to
> the core to create appropriate channels. I think this latter would be
> something for IPA/rmnet to use, perhaps for rmnet to offer the right ops
> structure.
Yes, that's more like what I meant above. I see you're thinking
as you write...
-Alex
>
> johannes
>
^ permalink raw reply
* [RFC net-next 3/5] taprio: Add support for hardware offloading
From: Voon Weifeng @ 2019-06-18 21:36 UTC (permalink / raw)
To: David S. Miller, Maxime Coquelin
Cc: netdev, linux-kernel, Jose Abreu, Giuseppe Cavallaro, Andrew Lunn,
Florian Fainelli, Alexandre Torgue, Vinicius Costa Gomes,
Ong Boon Leong, Voon Weifeng
In-Reply-To: <1560893778-6838-1-git-send-email-weifeng.voon@intel.com>
From: Vinicius Costa Gomes <vinicius.gomes@intel.com>
This allows taprio to offload the schedule enforcement to capable
network cards, resulting in more precise windows and less CPU usage.
The important detail here is the difference between the gate_mask in
taprio and gate_mask for the network driver. For the driver, each bit
in gate_mask references a transmission queue: bit 0 for queue 0, bit 1
for queue 1, and so on. This is done so the driver doesn't need to
know about traffic classes.
Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Voon Weifeng <weifeng.voon@intel.com>
---
include/linux/netdevice.h | 1 +
include/net/pkt_sched.h | 18 +++
include/uapi/linux/pkt_sched.h | 4 +
net/sched/sch_taprio.c | 263 ++++++++++++++++++++++++++++++++++++++++-
4 files changed, 284 insertions(+), 2 deletions(-)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index eeacebd7debb..e6bf0ede8cfa 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -845,6 +845,7 @@ enum tc_setup_type {
TC_SETUP_QDISC_ETF,
TC_SETUP_ROOT_QDISC,
TC_SETUP_QDISC_GRED,
+ TC_SETUP_QDISC_TAPRIO,
};
/* These structures hold the attributes of bpf state that are being passed
diff --git a/include/net/pkt_sched.h b/include/net/pkt_sched.h
index a16fbe9a2a67..3333c107f920 100644
--- a/include/net/pkt_sched.h
+++ b/include/net/pkt_sched.h
@@ -161,4 +161,22 @@ struct tc_etf_qopt_offload {
s32 queue;
};
+struct tc_taprio_sched_entry {
+ u8 command; /* TC_TAPRIO_CMD_* */
+
+ /* The gate_mask in the offloading side refers to HW queues */
+ u32 gate_mask;
+ u32 interval;
+};
+
+struct tc_taprio_qopt_offload {
+ u8 enable;
+ ktime_t base_time;
+ u64 cycle_time;
+ u64 cycle_time_extension;
+
+ size_t num_entries;
+ struct tc_taprio_sched_entry entries[0];
+};
+
#endif
diff --git a/include/uapi/linux/pkt_sched.h b/include/uapi/linux/pkt_sched.h
index 8b2f993cbb77..08a260fd7843 100644
--- a/include/uapi/linux/pkt_sched.h
+++ b/include/uapi/linux/pkt_sched.h
@@ -1158,6 +1158,9 @@ enum {
* [TCA_TAPRIO_ATTR_SCHED_ENTRY_INTERVAL]
*/
+#define TCA_TAPRIO_ATTR_OFFLOAD_FLAG_FULL_OFFLOAD 0x1
+#define TCA_TAPRIO_ATTR_OFFLOAD_FLAG_TXTIME_OFFLOAD 0x2
+
enum {
TCA_TAPRIO_ATTR_UNSPEC,
TCA_TAPRIO_ATTR_PRIOMAP, /* struct tc_mqprio_qopt */
@@ -1169,6 +1172,7 @@ enum {
TCA_TAPRIO_ATTR_ADMIN_SCHED, /* The admin sched, only used in dump */
TCA_TAPRIO_ATTR_SCHED_CYCLE_TIME, /* s64 */
TCA_TAPRIO_ATTR_SCHED_CYCLE_TIME_EXTENSION, /* s64 */
+ TCA_TAPRIO_ATTR_OFFLOAD_FLAGS, /* u32 */
__TCA_TAPRIO_ATTR_MAX,
};
diff --git a/net/sched/sch_taprio.c b/net/sched/sch_taprio.c
index 9ecfb8f5902a..9e8f066a2474 100644
--- a/net/sched/sch_taprio.c
+++ b/net/sched/sch_taprio.c
@@ -26,6 +26,9 @@
static DEFINE_SPINLOCK(taprio_list_lock);
#define TAPRIO_ALL_GATES_OPEN -1
+#define FULL_OFFLOAD_IS_ON(flags) ((flags) & TCA_TAPRIO_ATTR_OFFLOAD_FLAG_FULL_OFFLOAD)
+#define TXTIME_OFFLOAD_IS_ON(flags) ((flags) & TCA_TAPRIO_ATTR_OFFLOAD_FLAG_TXTIME_OFFLOAD)
+#define VALID_OFFLOAD(flags) ((flags) != U32_MAX)
struct sched_entry {
struct list_head list;
@@ -55,6 +58,8 @@ struct sched_gate_list {
struct taprio_sched {
struct Qdisc **qdiscs;
struct Qdisc *root;
+ struct tc_mqprio_qopt mqprio;
+ u32 offload_flags;
int clockid;
atomic64_t picos_per_byte; /* Using picoseconds because for 10Gbps+
* speeds it's sub-nanoseconds per byte
@@ -66,6 +71,8 @@ struct taprio_sched {
struct sched_gate_list __rcu *oper_sched;
struct sched_gate_list __rcu *admin_sched;
ktime_t (*get_time)(void);
+ struct sk_buff *(*dequeue)(struct Qdisc *sch);
+ struct sk_buff *(*peek)(struct Qdisc *sch);
struct hrtimer advance_timer;
struct list_head taprio_list;
};
@@ -143,7 +150,30 @@ static int taprio_enqueue(struct sk_buff *skb, struct Qdisc *sch,
return qdisc_enqueue(skb, child, to_free);
}
-static struct sk_buff *taprio_peek(struct Qdisc *sch)
+static struct sk_buff *taprio_peek_offload(struct Qdisc *sch)
+{
+ struct taprio_sched *q = qdisc_priv(sch);
+ struct net_device *dev = qdisc_dev(sch);
+ struct sk_buff *skb;
+ int i;
+
+ for (i = 0; i < dev->num_tx_queues; i++) {
+ struct Qdisc *child = q->qdiscs[i];
+
+ if (unlikely(!child))
+ continue;
+
+ skb = child->ops->peek(child);
+ if (!skb)
+ continue;
+
+ return skb;
+ }
+
+ return NULL;
+}
+
+static struct sk_buff *taprio_peek_soft(struct Qdisc *sch)
{
struct taprio_sched *q = qdisc_priv(sch);
struct net_device *dev = qdisc_dev(sch);
@@ -184,6 +214,13 @@ static struct sk_buff *taprio_peek(struct Qdisc *sch)
return NULL;
}
+static struct sk_buff *taprio_peek(struct Qdisc *sch)
+{
+ struct taprio_sched *q = qdisc_priv(sch);
+
+ return q->peek(sch);
+}
+
static inline int length_to_duration(struct taprio_sched *q, int len)
{
return div_u64(len * atomic64_read(&q->picos_per_byte), 1000);
@@ -196,7 +233,7 @@ static void taprio_set_budget(struct taprio_sched *q, struct sched_entry *entry)
atomic64_read(&q->picos_per_byte)));
}
-static struct sk_buff *taprio_dequeue(struct Qdisc *sch)
+static struct sk_buff *taprio_dequeue_soft(struct Qdisc *sch)
{
struct taprio_sched *q = qdisc_priv(sch);
struct net_device *dev = qdisc_dev(sch);
@@ -275,6 +312,40 @@ static struct sk_buff *taprio_dequeue(struct Qdisc *sch)
return skb;
}
+static struct sk_buff *taprio_dequeue_offload(struct Qdisc *sch)
+{
+ struct taprio_sched *q = qdisc_priv(sch);
+ struct net_device *dev = qdisc_dev(sch);
+ struct sk_buff *skb;
+ int i;
+
+ for (i = 0; i < dev->num_tx_queues; i++) {
+ struct Qdisc *child = q->qdiscs[i];
+
+ if (unlikely(!child))
+ continue;
+
+ skb = child->ops->dequeue(child);
+ if (unlikely(!skb))
+ continue;
+
+ qdisc_bstats_update(sch, skb);
+ qdisc_qstats_backlog_dec(sch, skb);
+ sch->q.qlen--;
+
+ return skb;
+ }
+
+ return NULL;
+}
+
+static struct sk_buff *taprio_dequeue(struct Qdisc *sch)
+{
+ struct taprio_sched *q = qdisc_priv(sch);
+
+ return q->dequeue(sch);
+}
+
static bool should_restart_cycle(const struct sched_gate_list *oper,
const struct sched_entry *entry)
{
@@ -707,6 +778,165 @@ static int taprio_dev_notifier(struct notifier_block *nb, unsigned long event,
return NOTIFY_DONE;
}
+static u32 tc_mask_to_queue_mask(const struct tc_mqprio_qopt *mqprio,
+ u32 tc_mask)
+{
+ u32 i, queue_mask = 0;
+
+ for (i = 0; i < mqprio->num_tc; i++) {
+ u32 offset, count;
+
+ if (!(tc_mask & BIT(i)))
+ continue;
+
+ offset = mqprio->offset[i];
+ count = mqprio->count[i];
+
+ queue_mask |= GENMASK(offset + count - 1, offset);
+ }
+
+ return queue_mask;
+}
+
+static void taprio_sched_to_offload(struct taprio_sched *q,
+ struct sched_gate_list *sched,
+ struct tc_taprio_qopt_offload *taprio)
+{
+ struct sched_entry *entry;
+ int i = 0;
+
+ taprio->base_time = sched->base_time;
+
+ list_for_each_entry(entry, &sched->entries, list) {
+ struct tc_taprio_sched_entry *e = &taprio->entries[i];
+
+ e->command = entry->command;
+ e->interval = entry->interval;
+
+ /* We do this transformation because the NIC
+ * has no knowledge of traffic classes, but it
+ * knows about queues.
+ */
+ e->gate_mask = tc_mask_to_queue_mask(&q->mqprio,
+ entry->gate_mask);
+ i++;
+ }
+
+ taprio->num_entries = i;
+}
+
+static void taprio_disable_offload(struct net_device *dev,
+ struct taprio_sched *q)
+{
+ const struct net_device_ops *ops = dev->netdev_ops;
+ struct tc_taprio_qopt_offload taprio = { };
+ int err;
+
+ if (!q->offload_flags)
+ return;
+
+ if (!ops->ndo_setup_tc)
+ return;
+
+ taprio.enable = 0;
+
+ err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, &taprio);
+ if (err < 0)
+ return;
+
+ /* Just to be sure to keep the function pointers in a
+ * consistent state always.
+ */
+ q->dequeue = taprio_dequeue_soft;
+ q->peek = taprio_peek_soft;
+
+ q->advance_timer.function = advance_sched;
+
+ q->offload_flags = 0;
+}
+
+static enum hrtimer_restart next_sched(struct hrtimer *timer)
+{
+ struct taprio_sched *q = container_of(timer, struct taprio_sched,
+ advance_timer);
+ struct sched_gate_list *oper, *admin;
+
+ spin_lock(&q->current_entry_lock);
+ oper = rcu_dereference_protected(q->oper_sched,
+ lockdep_is_held(&q->current_entry_lock));
+ admin = rcu_dereference_protected(q->admin_sched,
+ lockdep_is_held(&q->current_entry_lock));
+
+ rcu_assign_pointer(q->oper_sched, admin);
+ rcu_assign_pointer(q->admin_sched, NULL);
+
+ if (oper)
+ call_rcu(&oper->rcu, taprio_free_sched_cb);
+
+ spin_unlock(&q->current_entry_lock);
+
+ return HRTIMER_NORESTART;
+}
+
+static int taprio_enable_offload(struct net_device *dev,
+ struct tc_mqprio_qopt *mqprio,
+ struct taprio_sched *q,
+ struct sched_gate_list *sched,
+ struct netlink_ext_ack *extack,
+ u32 offload_flags)
+{
+ const struct net_device_ops *ops = dev->netdev_ops;
+ struct tc_taprio_qopt_offload *taprio;
+ size_t size;
+ int err = 0;
+
+ if (!FULL_OFFLOAD_IS_ON(offload_flags)) {
+ NL_SET_ERR_MSG(extack, "Offload mode is not supported");
+ return -EOPNOTSUPP;
+ }
+
+ if (!ops->ndo_setup_tc) {
+ NL_SET_ERR_MSG(extack, "Specified device does not support taprio offload");
+ return -EOPNOTSUPP;
+ }
+
+ size = sizeof(*taprio) +
+ sched->num_entries * sizeof(struct tc_taprio_sched_entry);
+
+ taprio = kzalloc(size, GFP_ATOMIC);
+ if (!taprio) {
+ NL_SET_ERR_MSG(extack, "Not enough memory for enabling offload mode");
+ return -ENOMEM;
+ }
+
+ taprio->enable = 1;
+ taprio_sched_to_offload(q, sched, taprio);
+
+ err = ops->ndo_setup_tc(dev, TC_SETUP_QDISC_TAPRIO, taprio);
+ if (err < 0) {
+ NL_SET_ERR_MSG(extack, "Specified device failed to setup taprio hardware offload");
+ goto done;
+ }
+
+ q->dequeue = taprio_dequeue_offload;
+ q->peek = taprio_peek_offload;
+
+ /* This function will only serve to keep the pointers to the
+ * "oper" and "admin" schedules valid in relation to their
+ * base times, so when calling dump() the users looks at the
+ * right schedules.
+ */
+ q->advance_timer.function = next_sched;
+
+done:
+ kfree(taprio);
+
+ if (err == 0)
+ q->offload_flags = offload_flags;
+
+ return err;
+}
+
static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
@@ -715,6 +945,7 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
struct taprio_sched *q = qdisc_priv(sch);
struct net_device *dev = qdisc_dev(sch);
struct tc_mqprio_qopt *mqprio = NULL;
+ u32 offload_flags = U32_MAX;
int i, err, clockid;
unsigned long flags;
ktime_t start;
@@ -731,6 +962,9 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
if (err < 0)
return err;
+ if (tb[TCA_TAPRIO_ATTR_OFFLOAD_FLAGS])
+ offload_flags = nla_get_u32(tb[TCA_TAPRIO_ATTR_OFFLOAD_FLAGS]);
+
new_admin = kzalloc(sizeof(*new_admin), GFP_KERNEL);
if (!new_admin) {
NL_SET_ERR_MSG(extack, "Not enough memory for a new schedule");
@@ -749,6 +983,12 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
goto free_sched;
}
+ if (offload_flags != U32_MAX && (oper || admin)) {
+ NL_SET_ERR_MSG(extack, "Changing 'offload' of a running schedule is not supported");
+ err = -ENOTSUPP;
+ goto free_sched;
+ }
+
err = parse_taprio_schedule(tb, new_admin, extack);
if (err < 0)
goto free_sched;
@@ -802,6 +1042,8 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
for (i = 0; i < TC_BITMASK + 1; i++)
netdev_set_prio_tc_map(dev, i,
mqprio->prio_tc_map[i]);
+
+ memcpy(&q->mqprio, mqprio, sizeof(q->mqprio));
}
switch (q->clockid) {
@@ -823,6 +1065,15 @@ static int taprio_change(struct Qdisc *sch, struct nlattr *opt,
goto unlock;
}
+ if (!offload_flags) {
+ taprio_disable_offload(dev, q);
+ } else if (VALID_OFFLOAD(offload_flags) || q->offload_flags) {
+ err = taprio_enable_offload(dev, mqprio, q,
+ new_admin, extack, offload_flags);
+ if (err)
+ goto unlock;
+ }
+
err = taprio_get_start_time(sch, new_admin, &start);
if (err < 0) {
NL_SET_ERR_MSG(extack, "Internal error: failed get start time");
@@ -866,6 +1117,8 @@ static void taprio_destroy(struct Qdisc *sch)
hrtimer_cancel(&q->advance_timer);
+ taprio_disable_offload(dev, q);
+
if (q->qdiscs) {
for (i = 0; i < dev->num_tx_queues && q->qdiscs[i]; i++)
qdisc_put(q->qdiscs[i]);
@@ -895,6 +1148,9 @@ static int taprio_init(struct Qdisc *sch, struct nlattr *opt,
hrtimer_init(&q->advance_timer, CLOCK_TAI, HRTIMER_MODE_ABS);
q->advance_timer.function = advance_sched;
+ q->dequeue = taprio_dequeue_soft;
+ q->peek = taprio_peek_soft;
+
q->root = sch;
/* We only support static clockids. Use an invalid value as default
@@ -1080,6 +1336,9 @@ static int taprio_dump(struct Qdisc *sch, struct sk_buff *skb)
if (nla_put_s32(skb, TCA_TAPRIO_ATTR_SCHED_CLOCKID, q->clockid))
goto options_error;
+ if (nla_put_u32(skb, TCA_TAPRIO_ATTR_OFFLOAD_FLAGS, q->offload_flags))
+ goto options_error;
+
if (oper && dump_schedule(skb, oper))
goto options_error;
--
1.9.1
^ permalink raw reply related
* [RFC net-next 4/5] net: stmmac: enable HW offloading for tc taprio
From: Voon Weifeng @ 2019-06-18 21:36 UTC (permalink / raw)
To: David S. Miller, Maxime Coquelin
Cc: netdev, linux-kernel, Jose Abreu, Giuseppe Cavallaro, Andrew Lunn,
Florian Fainelli, Alexandre Torgue, Vinicius Costa Gomes,
Ong Boon Leong, Voon Weifeng
In-Reply-To: <1560893778-6838-1-git-send-email-weifeng.voon@intel.com>
This patch enable iproute2's tc taprio to run IEEE802.1Qbv using HW.
tc taprio manual can refer to:
http://man7.org/linux/man-pages/man8/tc-taprio.8.html
To enable HW offloading, an extra argument need to be added:
offload 1
Example to run:
$ tc qdisc add dev IFACE parent root handle 100 taprio \
num_tc 3 \
map 2 2 1 0 2 2 2 2 2 2 2 2 2 2 2 2 \
queues 1@0 1@1 2@2 \
base-time 10000000 \
sched-entry S 03 300000 \
sched-entry S 02 300000 \
sched-entry S 06 400000 \
clockid CLOCK_TAI
offload 1
Signed-off-by: Voon Weifeng <weifeng.voon@intel.com>
---
drivers/net/ethernet/stmicro/stmmac/hwif.h | 5 ++
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 2 +
drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c | 96 +++++++++++++++++++++++
3 files changed, 103 insertions(+)
diff --git a/drivers/net/ethernet/stmicro/stmmac/hwif.h b/drivers/net/ethernet/stmicro/stmmac/hwif.h
index dec9b1f5c557..c9fa60934710 100644
--- a/drivers/net/ethernet/stmicro/stmmac/hwif.h
+++ b/drivers/net/ethernet/stmicro/stmmac/hwif.h
@@ -514,6 +514,7 @@ struct stmmac_mode_ops {
struct stmmac_priv;
struct tc_cls_u32_offload;
struct tc_cbs_qopt_offload;
+struct tc_taprio_qopt_offload;
struct stmmac_tc_ops {
int (*init)(struct stmmac_priv *priv);
@@ -521,6 +522,8 @@ struct stmmac_tc_ops {
struct tc_cls_u32_offload *cls);
int (*setup_cbs)(struct stmmac_priv *priv,
struct tc_cbs_qopt_offload *qopt);
+ int (*setup_taprio)(struct stmmac_priv *priv,
+ struct tc_taprio_qopt_offload *qopt);
};
#define stmmac_tc_init(__priv, __args...) \
@@ -529,6 +532,8 @@ struct stmmac_tc_ops {
stmmac_do_callback(__priv, tc, setup_cls_u32, __args)
#define stmmac_tc_setup_cbs(__priv, __args...) \
stmmac_do_callback(__priv, tc, setup_cbs, __args)
+#define stmmac_tc_setup_taprio(__priv, __args...) \
+ stmmac_do_callback(__priv, tc, setup_taprio, __args)
struct stmmac_counters;
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index c28b5e69f2cd..a443c42fa58b 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -3811,6 +3811,8 @@ static int stmmac_setup_tc(struct net_device *ndev, enum tc_setup_type type,
return stmmac_setup_tc_block(priv, type_data);
case TC_SETUP_QDISC_CBS:
return stmmac_tc_setup_cbs(priv, priv, type_data);
+ case TC_SETUP_QDISC_TAPRIO:
+ return stmmac_tc_setup_taprio(priv, priv, type_data);
default:
return -EOPNOTSUPP;
}
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
index 58ea18af9813..d118e3636d50 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c
@@ -7,10 +7,13 @@
#include <net/pkt_cls.h>
#include <net/tc_act/tc_gact.h>
#include "common.h"
+#include "dw_tsn_lib.h"
#include "dwmac4.h"
#include "dwmac5.h"
#include "stmmac.h"
+#define ONE_SEC_IN_NANOSEC 1000000000ULL
+
static void tc_fill_all_pass_entry(struct stmmac_tc_entry *entry)
{
memset(entry, 0, sizeof(*entry));
@@ -349,8 +352,101 @@ static int tc_setup_cbs(struct stmmac_priv *priv,
return 0;
}
+static int tc_setup_taprio(struct stmmac_priv *priv,
+ struct tc_taprio_qopt_offload *qopt)
+{
+ u64 time_extension = qopt->cycle_time_extension;
+ u64 base_time = ktime_to_ns(qopt->base_time);
+ u64 cycle_time = qopt->cycle_time;
+ struct est_gcrr egcrr;
+ u32 extension_ns;
+ u32 extension_s;
+ u32 cycle_ns;
+ u32 cycle_s;
+ u32 base_ns;
+ u32 base_s;
+ int ret;
+ int i;
+
+ if (qopt->enable) {
+ stmmac_set_est_enable(priv, priv->ioaddr, 1);
+ dev_info(priv->device, "taprio: EST enabled\n");
+ } else {
+ stmmac_set_est_enable(priv, priv->ioaddr, 0);
+ dev_info(priv->device, "taprio: EST disabled\n");
+ return 0;
+ }
+
+ dev_dbg(priv->device, "taprio: time_extension %lld, base_time %lld, cycle_time %lld\n",
+ qopt->cycle_time_extension, qopt->base_time, qopt->cycle_time);
+
+ for (i = 0; i < qopt->num_entries; i++) {
+ struct est_gc_entry sgce;
+
+ sgce.gates = qopt->entries[i].gate_mask;
+ sgce.ti_nsec = qopt->entries[i].interval;
+
+ /* cycle_time will be sum of all time interval
+ * of the entries in the schedule if the
+ * cycle_time is not provided
+ */
+ if (!qopt->cycle_time)
+ cycle_time += qopt->entries[i].interval;
+
+ dev_dbg(priv->device, "taprio: gates 0x%x, ti_ns %d, cycle_ns %d\n",
+ sgce.gates, sgce.ti_nsec, cycle_ns);
+
+ ret = stmmac_set_est_gce(priv, priv->ioaddr, &sgce, i, 0, 0);
+
+ if (ret) {
+ dev_err(priv->device,
+ "taprio: fail to program GC entry(%d).\n", i);
+
+ return ret;
+ }
+ }
+
+ ret = stmmac_set_est_gcrr_llr(priv, priv->ioaddr,
+ qopt->num_entries,
+ 0, 0);
+ if (ret) {
+ dev_err(priv->device,
+ "taprio: fail to program GC into HW\n");
+ }
+
+ /* set est_info */
+ base_ns = do_div(base_time, ONE_SEC_IN_NANOSEC);
+ base_s = base_time;
+ dev_dbg(priv->device, "taprio: base_s %d, base_ns %d\n",
+ base_s, base_ns);
+
+ cycle_ns = do_div(cycle_time, ONE_SEC_IN_NANOSEC);
+ cycle_s = cycle_time;
+ dev_dbg(priv->device, "taprio: cycle_s %d, cycle_ns %d\n",
+ cycle_s, cycle_ns);
+
+ extension_ns = do_div(time_extension, ONE_SEC_IN_NANOSEC);
+ extension_s = time_extension;
+ dev_dbg(priv->device, "taprio: extension_s %d, extension_ns %d\n",
+ extension_s, extension_ns);
+
+ if (extension_s) {
+ dev_err(priv->device, "taprio: extension in seconds not supported.\n");
+ return -EINVAL;
+ }
+
+ egcrr.cycle_sec = cycle_s;
+ egcrr.cycle_nsec = cycle_ns;
+ egcrr.base_sec = base_s;
+ egcrr.base_nsec = base_ns;
+ egcrr.ter_nsec = extension_ns;
+
+ return stmmac_set_est_gcrr_times(priv, priv->ioaddr, &egcrr, 0, 0);
+}
+
const struct stmmac_tc_ops dwmac510_tc_ops = {
.init = tc_init,
.setup_cls_u32 = tc_setup_cls_u32,
.setup_cbs = tc_setup_cbs,
+ .setup_taprio = tc_setup_taprio,
};
--
1.9.1
^ permalink raw reply related
* [RFC net-next 5/5] net: stmmac: Set TSN HW tunable after tsn setup
From: Voon Weifeng @ 2019-06-18 21:36 UTC (permalink / raw)
To: David S. Miller, Maxime Coquelin
Cc: netdev, linux-kernel, Jose Abreu, Giuseppe Cavallaro, Andrew Lunn,
Florian Fainelli, Alexandre Torgue, Vinicius Costa Gomes,
Ong Boon Leong, Voon Weifeng
In-Reply-To: <1560893778-6838-1-git-send-email-weifeng.voon@intel.com>
TSN HW tunable data for PTP Time Offset Value(PTOV),
Current Time Offset Value(CTOV) and Time Interval Shift
Amount(TILS) are added as platform data. These platform
data are set after tsn setup.
Signed-off-by: Voon Weifeng <weifeng.voon@intel.com>
---
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 16 ++++++++++++++++
include/linux/stmmac.h | 3 +++
2 files changed, 19 insertions(+)
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index a443c42fa58b..d3ce86abdc69 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -2533,6 +2533,22 @@ static int stmmac_hw_setup(struct net_device *dev, bool init_ptp)
/* Setup for TSN capability */
dwmac_tsn_setup(priv->ioaddr);
+ /* Set TSN HW tunable */
+ if (priv->plat->ptov)
+ stmmac_set_tsn_hwtunable(priv, priv->ioaddr,
+ TSN_HWTUNA_TX_EST_PTOV,
+ &priv->plat->ptov);
+
+ if (priv->plat->ctov)
+ stmmac_set_tsn_hwtunable(priv, priv->ioaddr,
+ TSN_HWTUNA_TX_EST_CTOV,
+ &priv->plat->ctov);
+
+ if (priv->plat->tils)
+ stmmac_set_tsn_hwtunable(priv, priv->ioaddr,
+ TSN_HWTUNA_TX_EST_TILS,
+ &priv->plat->tils);
+
return 0;
}
diff --git a/include/linux/stmmac.h b/include/linux/stmmac.h
index d4a90f48e49b..792aa8b3e138 100644
--- a/include/linux/stmmac.h
+++ b/include/linux/stmmac.h
@@ -176,5 +176,8 @@ struct plat_stmmacenet_data {
int mac_port_sel_speed;
bool en_tx_lpi_clockgating;
int has_xgmac;
+ unsigned int ptov;
+ unsigned int ctov;
+ unsigned int tils;
};
#endif
--
1.9.1
^ permalink raw reply related
* [RFC net-next 2/5] net: stmmac: gcl errors reporting and its interrupt handling
From: Voon Weifeng @ 2019-06-18 21:36 UTC (permalink / raw)
To: David S. Miller, Maxime Coquelin
Cc: netdev, linux-kernel, Jose Abreu, Giuseppe Cavallaro, Andrew Lunn,
Florian Fainelli, Alexandre Torgue, Vinicius Costa Gomes,
Ong Boon Leong, Voon Weifeng
In-Reply-To: <1560893778-6838-1-git-send-email-weifeng.voon@intel.com>
Enabled interrupt for Constant Gate Control Error (CGCE), Head-of-Line
Blocking due to scheduling error (HLBS) and Head-of-Line Blocking due to
frame size error (HLBF).
CGCE should not happen as the driver has already implemented a check
before applying the settings. CGCE handling is added as a safety
check so that we can catch it if there is such error being fired. For
HLBS, the user will get the info of all the queues that shows this
error. For HLBF, the user will get the info of all the queue with the
latest frame size which causes the error. Frame size 0 indicates no
error.
This patch also added functionality to get and clear the gcl errors.
The ISR handling takes place when EST feature is enabled by user.
Signed-off-by: Voon Weifeng <weifeng.voon@intel.com>
Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
---
drivers/net/ethernet/stmicro/stmmac/common.h | 4 +
drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c | 123 ++++++++++++++++++++++
drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h | 45 ++++++++
drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c | 3 +
drivers/net/ethernet/stmicro/stmmac/hwif.h | 9 ++
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 7 ++
6 files changed, 191 insertions(+)
diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h
index ad9e9368535d..9723b0a12110 100644
--- a/drivers/net/ethernet/stmicro/stmmac/common.h
+++ b/drivers/net/ethernet/stmicro/stmmac/common.h
@@ -407,6 +407,9 @@ struct mii_regs {
unsigned int clk_csr_mask;
};
+/* tsn capability, meant for mac_device_info->tsn_cap */
+#define TSN_CAP_EST BIT(0)
+
struct mac_device_info {
const struct stmmac_ops *mac;
const struct stmmac_desc_ops *desc;
@@ -425,6 +428,7 @@ struct mac_device_info {
unsigned int pcs;
unsigned int pmt;
unsigned int ps;
+ u32 tsn_cap;
};
struct stmmac_rx_routing {
diff --git a/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c
index cba27c604cb1..f14e86fcc93c 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c
@@ -11,6 +11,7 @@
static bool dw_tsn_feat_en[TSN_FEAT_ID_MAX];
static unsigned int dw_tsn_hwtunable[TSN_HWTUNA_MAX];
static struct est_gc_config dw_est_gc_config;
+static struct tsn_err_stat dw_err_stat;
static unsigned int est_get_gcl_depth(unsigned int hw_cap)
{
@@ -295,6 +296,24 @@ void dwmac_tsn_init(void *ioaddr)
gcl_depth, ti_wid, tils_max, cap->txqcnt);
}
+/* dwmac_tsn_setup is called within stmmac_hw_setup() after
+ * stmmac_init_dma_engine() which resets MAC controller.
+ * This is so-that MAC registers are not cleared.
+ */
+void dwmac_tsn_setup(void *ioaddr)
+{
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int value;
+
+ if (cap->est_support) {
+ /* Enable EST interrupts */
+ value = (MTL_EST_INT_EN_CGCE | MTL_EST_INT_EN_IEHS |
+ MTL_EST_INT_EN_IEHF | MTL_EST_INT_EN_IEBE |
+ MTL_EST_INT_EN_IECC);
+ TSN_WR32(value, ioaddr + MTL_EST_INT_EN);
+ }
+}
+
void dwmac_get_tsn_hwcap(struct tsn_hw_cap **tsn_hwcap)
{
*tsn_hwcap = &dw_tsn_hwcap;
@@ -788,3 +807,107 @@ int dwmac_get_est_gcc(void *ioaddr,
return 0;
}
+
+int dwmac_est_irq_status(void *ioaddr)
+{
+ struct tsn_err_stat *err_stat = &dw_err_stat;
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int txqcnt_mask = 0;
+ unsigned int status = 0;
+ unsigned int value = 0;
+ unsigned int feqn = 0;
+ unsigned int hbfq = 0;
+ unsigned int hbfs = 0;
+
+ txqcnt_mask = (1 << cap->txqcnt) - 1;
+ status = TSN_RD32(ioaddr + MTL_EST_STATUS);
+
+ value = (MTL_EST_STATUS_CGCE | MTL_EST_STATUS_HLBS |
+ MTL_EST_STATUS_HLBF | MTL_EST_STATUS_BTRE |
+ MTL_EST_STATUS_SWLC);
+
+ /* Return if there is no error */
+ if (!(status & value))
+ return 0;
+
+ /* spin_lock() is not needed here because of BTRE and SWLC
+ * bit will not be altered. Both of the bit will be
+ * polled in dwmac_set_est_gcrr_times()
+ */
+ if (status & MTL_EST_STATUS_CGCE) {
+ /* Clear Interrupt */
+ TSN_WR32(MTL_EST_STATUS_CGCE, ioaddr + MTL_EST_STATUS);
+
+ err_stat->cgce_n++;
+ }
+
+ if (status & MTL_EST_STATUS_HLBS) {
+ value = TSN_RD32(ioaddr + MTL_EST_SCH_ERR);
+ value &= txqcnt_mask;
+
+ /* Clear Interrupt */
+ TSN_WR32(value, ioaddr + MTL_EST_SCH_ERR);
+
+ /* Collecting info to shows all the queues that has HLBS */
+ /* issue. The only way to clear this is to clear the */
+ /* statistic */
+ err_stat->hlbs_q |= value;
+ }
+
+ if (status & MTL_EST_STATUS_HLBF) {
+ value = TSN_RD32(ioaddr + MTL_EST_FRM_SZ_ERR);
+ feqn = value & txqcnt_mask;
+
+ value = TSN_RD32(ioaddr + MTL_EST_FRM_SZ_CAP);
+ hbfq = (value & MTL_EST_FRM_SZ_CAP_HBFQ_MASK(cap->txqcnt))
+ >> MTL_EST_FRM_SZ_CAP_HBFQ_SHIFT;
+ hbfs = value & MTL_EST_FRM_SZ_CAP_HBFS_MASK;
+
+ /* Clear Interrupt */
+ TSN_WR32(feqn, ioaddr + MTL_EST_FRM_SZ_ERR);
+
+ err_stat->hlbf_sz[hbfq] = hbfs;
+ }
+
+ if (status & MTL_EST_STATUS_BTRE) {
+ if ((status & MTL_EST_STATUS_BTRL) ==
+ MTL_EST_STATUS_BTRL_MAX)
+ err_stat->btre_max_n++;
+ else
+ err_stat->btre_n++;
+
+ err_stat->btrl = (status & MTL_EST_STATUS_BTRL) >>
+ MTL_EST_STATUS_BTRL_SHIFT;
+
+ TSN_WR32(MTL_EST_STATUS_BTRE, ioaddr +
+ MTL_EST_STATUS);
+ }
+
+ if (status & MTL_EST_STATUS_SWLC) {
+ TSN_WR32(MTL_EST_STATUS_SWLC, ioaddr +
+ MTL_EST_STATUS);
+ TSN_INFO_NA("SWOL has been switched\n");
+ }
+
+ return status;
+}
+
+int dwmac_get_est_err_stat(struct tsn_err_stat **err_stat)
+{
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ *err_stat = &dw_err_stat;
+
+ return 0;
+}
+
+int dwmac_clr_est_err_stat(void *ioaddr)
+{
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ memset(&dw_err_stat, 0, sizeof(dw_err_stat));
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h
index feb71f7e7031..fa9a06c51a04 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h
+++ b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h
@@ -37,9 +37,30 @@
#define MTL_EST_STATUS_BTRL_MAX (0xF << 8)
#define MTL_EST_STATUS_SWOL BIT(7) /* SW owned list */
#define MTL_EST_STATUS_SWOL_SHIFT 7
+#define MTL_EST_STATUS_CGCE BIT(4) /* Constant gate ctrl err */
+#define MTL_EST_STATUS_HLBS BIT(3) /* HLB due to scheduling */
+#define MTL_EST_STATUS_HLBF BIT(2) /* HLB due to frame size */
#define MTL_EST_STATUS_BTRE BIT(1) /* BTR Error */
#define MTL_EST_STATUS_SWLC BIT(0) /* Switch to SWOL complete */
+/* MTL EST Scheduling error */
+#define MTL_EST_SCH_ERR 0x00000c60
+#define MTL_EST_FRM_SZ_ERR 0x00000c64
+#define MTL_EST_FRM_SZ_CAP 0x00000c68
+#define MTL_EST_FRM_SZ_CAP_HBFS_MASK GENMASK(14, 0)
+#define MTL_EST_FRM_SZ_CAP_HBFQ_SHIFT 16
+#define MTL_EST_FRM_SZ_CAP_HBFQ_MASK(x) (x > 4 ? GENMASK(18, 16) : \
+ x > 2 ? GENMASK(17, 16) : \
+ BIT(16))
+
+/* MTL EST interrupt enable */
+#define MTL_EST_INT_EN 0x00000c70
+#define MTL_EST_INT_EN_CGCE BIT(4)
+#define MTL_EST_INT_EN_IEHS BIT(3)
+#define MTL_EST_INT_EN_IEHF BIT(2)
+#define MTL_EST_INT_EN_IEBE BIT(1)
+#define MTL_EST_INT_EN_IECC BIT(0)
+
/* MTL EST GCL control register */
#define MTL_EST_GCL_CTRL 0x00000c80
#define MTL_EST_GCL_CTRL_ADDR GENMASK(10, 8) /* GCL Address */
@@ -118,6 +139,26 @@ struct tsn_hw_cap {
unsigned int ext_max; /* Max time extension */
};
+/* TSN Error Status */
+struct tsn_err_stat {
+ unsigned int cgce_n; /* Constant gate error
+ * count.
+ */
+ unsigned int hlbs_q; /* Queue with HLB due to
+ * Scheduling
+ */
+ unsigned int hlbf_sz[MTL_MAX_TX_QUEUES];/* Frame size that causes
+ * HLB
+ */
+ unsigned int btre_n; /* BTR error with BTR
+ * renewal
+ */
+ unsigned int btre_max_n; /* BTR error with BTR
+ * renewal fail count
+ */
+ unsigned int btrl; /* BTR error loop count */
+};
+
/* EST Gate Control Entry */
struct est_gc_entry {
unsigned int gates; /* gate control: 0: closed,
@@ -150,6 +191,7 @@ struct est_gc_config {
/* TSN functions */
void dwmac_tsn_init(void *ioaddr);
+void dwmac_tsn_setup(void *ioaddr);
void dwmac_get_tsn_hwcap(struct tsn_hw_cap **tsn_hwcap);
void dwmac_set_est_gcb(struct est_gc_entry *gcl, unsigned int bank);
void dwmac_set_tsn_feat(enum tsn_feat_id featid, bool enable);
@@ -170,4 +212,7 @@ int dwmac_set_est_gcrr_times(void *ioaddr,
int dwmac_set_est_enable(void *ioaddr, bool enable);
int dwmac_get_est_gcc(void *ioaddr,
struct est_gc_config **gcc, bool frmdrv);
+int dwmac_est_irq_status(void *ioaddr);
+int dwmac_get_est_err_stat(struct tsn_err_stat **err_stat);
+int dwmac_clr_est_err_stat(void *ioaddr);
#endif /* __DW_TSN_LIB_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
index 1361807fe802..39a35f7eb91d 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
@@ -830,6 +830,9 @@ static void dwmac4_set_mac_loopback(void __iomem *ioaddr, bool enable)
.set_est_gcrr_times = dwmac_set_est_gcrr_times,
.set_est_enable = dwmac_set_est_enable,
.get_est_gcc = dwmac_get_est_gcc,
+ .est_irq_status = dwmac_est_irq_status,
+ .get_est_err_stat = dwmac_get_est_err_stat,
+ .clr_est_err_stat = dwmac_clr_est_err_stat,
.safety_feat_config = dwmac5_safety_feat_config,
.safety_feat_irq_status = dwmac5_safety_feat_irq_status,
.safety_feat_dump = dwmac5_safety_feat_dump,
diff --git a/drivers/net/ethernet/stmicro/stmmac/hwif.h b/drivers/net/ethernet/stmicro/stmmac/hwif.h
index 518a72805185..dec9b1f5c557 100644
--- a/drivers/net/ethernet/stmicro/stmmac/hwif.h
+++ b/drivers/net/ethernet/stmicro/stmmac/hwif.h
@@ -337,6 +337,9 @@ struct stmmac_ops {
int (*set_est_enable)(void __iomem *ioaddr, bool enable);
int (*get_est_gcc)(void __iomem *ioaddr,
struct est_gc_config **gcc, bool frmdrv);
+ int (*est_irq_status)(void __iomem *ioaddr);
+ int (*get_est_err_stat)(struct tsn_err_stat **err_stat);
+ int (*clr_est_err_stat)(void __iomem *ioaddr);
/* Safety Features */
int (*safety_feat_config)(void __iomem *ioaddr, unsigned int asp);
int (*safety_feat_irq_status)(struct net_device *ndev,
@@ -437,6 +440,12 @@ struct stmmac_ops {
stmmac_do_callback(__priv, mac, set_est_enable, __args)
#define stmmac_get_est_gcc(__priv, __args...) \
stmmac_do_callback(__priv, mac, get_est_gcc, __args)
+#define stmmac_est_irq_status(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, est_irq_status, __args)
+#define stmmac_get_est_err_stat(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, get_est_err_stat, __args)
+#define stmmac_clr_est_err_stat(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, clr_est_err_stat, __args)
#define stmmac_safety_feat_config(__priv, __args...) \
stmmac_do_callback(__priv, mac, safety_feat_config, __args)
#define stmmac_safety_feat_irq_status(__priv, __args...) \
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 91213cd3a668..c28b5e69f2cd 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -2530,6 +2530,9 @@ static int stmmac_hw_setup(struct net_device *dev, bool init_ptp)
/* Start the ball rolling... */
stmmac_start_all_dma(priv);
+ /* Setup for TSN capability */
+ dwmac_tsn_setup(priv->ioaddr);
+
return 0;
}
@@ -3693,6 +3696,9 @@ static irqreturn_t stmmac_interrupt(int irq, void *dev_id)
queue);
}
+ if (priv->hw->tsn_cap & TSN_CAP_EST)
+ stmmac_est_irq_status(priv, priv->ioaddr);
+
/* PCS link status */
if (priv->hw->pcs) {
if (priv->xstats.pcs_link)
@@ -4297,6 +4303,7 @@ int stmmac_dvr_probe(struct device *device,
stmmac_get_tsn_hwcap(priv, &tsn_hwcap);
if (tsn_hwcap && tsn_hwcap->est_support && priv->plat->tsn_est_en) {
stmmac_set_tsn_feat(priv, TSN_FEAT_ID_EST, true);
+ priv->hw->tsn_cap |= TSN_CAP_EST;
dev_info(priv->device, "EST feature enabled\n");
}
--
1.9.1
^ permalink raw reply related
* [RFC net-next 1/5] net: stmmac: introduce IEEE 802.1Qbv configuration functionalities
From: Voon Weifeng @ 2019-06-18 21:36 UTC (permalink / raw)
To: David S. Miller, Maxime Coquelin
Cc: netdev, linux-kernel, Jose Abreu, Giuseppe Cavallaro, Andrew Lunn,
Florian Fainelli, Alexandre Torgue, Vinicius Costa Gomes,
Ong Boon Leong, Voon Weifeng
In-Reply-To: <1560893778-6838-1-git-send-email-weifeng.voon@intel.com>
From: Ong Boon Leong <boon.leong.ong@intel.com>
IEEE 802.1Qbv Enhancements for Scheduled Traffics (EST) is available in
EQoS ver5.xx. The change adds basic EST functionalities:
a) EST initialization with hardware capabilities detection.
b) Setting Gate Control List (GCL), i.e. gate open/close & time intervals,
and all GC Related Registers (GCRR), e.g., base time (BTR), cycle time
(CTR), time extension (TER) and GC List Length (LLR).
c) Setting time interval left shift (TILS), PTP time offset (PTOV) and
current time offset (CTOV).
d) Enable/disable EST.
e) Getting TSN hardware capabilities.
f) Getting Gate Control configuration either from driver data store or
hardware.
We extend the main driver logic to include basic TSN capability discovery,
and setup. We also add EST feature enable/disable control.
Reviewed-by: Chuah Kim Tatt <kim.tatt.chuah@intel.com>
Reviewed-by: Voon Weifeng <weifeng.voon@intel.com>
Reviewed-by: Kweh Hock Leong <hock.leong.kweh@intel.com>
Signed-off-by: Ong Boon Leong <boon.leong.ong@intel.com>
Signed-off-by: Voon Weifeng <weifeng.voon@intel.com>
---
drivers/net/ethernet/stmicro/stmmac/Makefile | 2 +-
drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c | 790 ++++++++++++++++++++++
drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h | 173 +++++
drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c | 13 +
drivers/net/ethernet/stmicro/stmmac/hwif.h | 52 ++
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 46 ++
include/linux/stmmac.h | 1 +
7 files changed, 1076 insertions(+), 1 deletion(-)
create mode 100644 drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c
create mode 100644 drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h
diff --git a/drivers/net/ethernet/stmicro/stmmac/Makefile b/drivers/net/ethernet/stmicro/stmmac/Makefile
index c59926d96bcc..76fb36cb4da7 100644
--- a/drivers/net/ethernet/stmicro/stmmac/Makefile
+++ b/drivers/net/ethernet/stmicro/stmmac/Makefile
@@ -6,7 +6,7 @@ stmmac-objs:= stmmac_main.o stmmac_ethtool.o stmmac_mdio.o ring_mode.o \
mmc_core.o stmmac_hwtstamp.o stmmac_ptp.o dwmac4_descs.o \
dwmac4_dma.o dwmac4_lib.o dwmac4_core.o dwmac5.o hwif.o \
stmmac_tc.o dwxgmac2_core.o dwxgmac2_dma.o dwxgmac2_descs.o \
- $(stmmac-y)
+ dw_tsn_lib.o $(stmmac-y)
stmmac-$(CONFIG_STMMAC_SELFTESTS) += stmmac_selftests.o
diff --git a/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c
new file mode 100644
index 000000000000..cba27c604cb1
--- /dev/null
+++ b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c
@@ -0,0 +1,790 @@
+// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
+/* Copyright (c) 2019, Intel Corporation.
+ * dw_tsn_lib.c: DW EQoS v5.00 TSN capabilities
+ */
+
+#include "dwmac4.h"
+#include "dwmac5.h"
+#include "dw_tsn_lib.h"
+
+static struct tsn_hw_cap dw_tsn_hwcap;
+static bool dw_tsn_feat_en[TSN_FEAT_ID_MAX];
+static unsigned int dw_tsn_hwtunable[TSN_HWTUNA_MAX];
+static struct est_gc_config dw_est_gc_config;
+
+static unsigned int est_get_gcl_depth(unsigned int hw_cap)
+{
+ unsigned int estdep = (hw_cap & GMAC_HW_FEAT_ESTDEP)
+ >> GMAC_HW_FEAT_ESTDEP_SHIFT;
+ unsigned int depth;
+
+ switch (estdep) {
+ case 1:
+ depth = 64;
+ break;
+ case 2:
+ depth = 128;
+ break;
+ case 3:
+ depth = 256;
+ break;
+ case 4:
+ depth = 512;
+ break;
+ case 5:
+ depth = 1024;
+ break;
+ default:
+ depth = 0;
+ }
+
+ return depth;
+}
+
+static unsigned int est_get_ti_width(unsigned int hw_cap)
+{
+ unsigned int estwid = (hw_cap & GMAC_HW_FEAT_ESTWID)
+ >> GMAC_HW_FEAT_ESTWID_SHIFT;
+ unsigned int width;
+
+ switch (estwid) {
+ case 1:
+ width = 16;
+ break;
+ case 2:
+ width = 20;
+ break;
+ case 3:
+ width = 24;
+ break;
+ default:
+ width = 0;
+ }
+
+ return width;
+}
+
+static int est_poll_srwo(void *ioaddr)
+{
+ /* Poll until the EST GCL Control[SRWO] bit clears.
+ * Total wait = 12 x 50ms ~= 0.6s.
+ */
+ unsigned int retries = 12;
+ unsigned int value;
+
+ do {
+ value = TSN_RD32(ioaddr + MTL_EST_GCL_CTRL);
+ if (!(value & MTL_EST_GCL_CTRL_SRWO))
+ return 0;
+ msleep(50);
+ } while (--retries);
+
+ return -ETIMEDOUT;
+}
+
+static int est_set_gcl_addr(void *ioaddr, unsigned int addr,
+ unsigned int gcrr, unsigned int rwops,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ unsigned int value;
+
+ value = MTL_EST_GCL_CTRL_ADDR_VAL(addr) & MTL_EST_GCL_CTRL_ADDR;
+
+ if (dbgm) {
+ if (dbgb)
+ value |= MTL_EST_GCL_CTRL_DBGB1;
+
+ value |= MTL_EST_GCL_CTRL_DBGM;
+ }
+
+ if (gcrr)
+ value |= MTL_EST_GCL_CTRL_GCRR;
+
+ /* This is the only place SRWO is set and driver polls SRWO
+ * for self-cleared before exit. Therefore, caller should
+ * check return status for possible time out error.
+ */
+ value |= (rwops | MTL_EST_GCL_CTRL_SRWO);
+
+ TSN_WR32(value, ioaddr + MTL_EST_GCL_CTRL);
+
+ return est_poll_srwo(ioaddr);
+}
+
+static int est_write_gcl_config(void *ioaddr, unsigned int data,
+ unsigned int addr, unsigned int gcrr,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ TSN_WR32(data, ioaddr + MTL_EST_GCL_DATA);
+
+ return est_set_gcl_addr(ioaddr, addr, gcrr, GCL_OPS_W, dbgb, dbgm);
+}
+
+static int est_read_gcl_config(void *ioaddr, unsigned int *data,
+ unsigned int addr, unsigned int gcrr,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ int ret;
+
+ ret = est_set_gcl_addr(ioaddr, addr, gcrr, GCL_OPS_R, dbgb, dbgm);
+ if (ret)
+ return ret;
+
+ *data = TSN_RD32(ioaddr + MTL_EST_GCL_DATA);
+
+ return ret;
+}
+
+static int est_read_gce(void *ioaddr, unsigned int row,
+ unsigned int *gates, unsigned int *ti_nsec,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int ti_wid = cap->ti_wid;
+ unsigned int gates_mask;
+ unsigned int ti_mask;
+ unsigned int value;
+ int ret;
+
+ gates_mask = (1 << cap->txqcnt) - 1;
+ ti_mask = (1 << ti_wid) - 1;
+
+ ret = est_read_gcl_config(ioaddr, &value, row, 0, dbgb, dbgm);
+ if (ret) {
+ TSN_ERR("Read GCE failed! row=%u\n", row);
+
+ return ret;
+ }
+ *ti_nsec = value & ti_mask;
+ *gates = (value >> ti_wid) & gates_mask;
+
+ return ret;
+}
+
+static unsigned int est_get_gcl_total_intervals_nsec(unsigned int bank,
+ unsigned int gcl_len)
+{
+ struct est_gc_entry *gcl = dw_est_gc_config.gcb[bank].gcl;
+ unsigned int nsec = 0;
+ unsigned int row;
+
+ for (row = 0; row < gcl_len; row++) {
+ nsec += gcl->ti_nsec;
+ gcl++;
+ }
+
+ return nsec;
+}
+
+static int est_set_tils(void *ioaddr, const unsigned int tils)
+{
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int value;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ if (tils > cap->tils_max) {
+ TSN_WARN("EST: invalid tils(%u), max=%u\n",
+ tils, cap->tils_max);
+
+ return -EINVAL;
+ }
+
+ /* Ensure that HW is not in the midst of GCL transition */
+ value = TSN_RD32(ioaddr + MTL_EST_CTRL);
+ value &= ~MTL_EST_CTRL_SSWL;
+
+ /* MTL_EST_CTRL value has been read earlier, if TILS value
+ * differs, we update here.
+ */
+ if (tils != dw_tsn_hwtunable[TSN_HWTUNA_TX_EST_TILS]) {
+ value &= ~MTL_EST_CTRL_TILS;
+ value |= (tils << MTL_EST_CTRL_TILS_SHIFT);
+
+ TSN_WR32(value, ioaddr + MTL_EST_CTRL);
+ dw_tsn_hwtunable[TSN_HWTUNA_TX_EST_TILS] = tils;
+ }
+
+ return 0;
+}
+
+static int est_set_ov(void *ioaddr,
+ const unsigned int *ptov,
+ const unsigned int *ctov)
+{
+ unsigned int value;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ value = TSN_RD32(ioaddr + MTL_EST_CTRL);
+ value &= ~MTL_EST_CTRL_SSWL;
+
+ if (ptov) {
+ if (*ptov > EST_PTOV_MAX) {
+ TSN_WARN("EST: invalid PTOV(%u), max=%u\n",
+ *ptov, EST_PTOV_MAX);
+
+ return -EINVAL;
+ } else if (*ptov !=
+ dw_tsn_hwtunable[TSN_HWTUNA_TX_EST_PTOV]) {
+ value &= ~MTL_EST_CTRL_PTOV;
+ value |= (*ptov << MTL_EST_CTRL_PTOV_SHIFT);
+ dw_tsn_hwtunable[TSN_HWTUNA_TX_EST_PTOV] = *ptov;
+ }
+ }
+
+ if (ctov) {
+ if (*ctov > EST_CTOV_MAX) {
+ TSN_WARN("EST: invalid CTOV(%u), max=%u\n",
+ *ctov, EST_CTOV_MAX);
+
+ return -EINVAL;
+ } else if (*ctov != dw_tsn_hwtunable[TSN_HWTUNA_TX_EST_CTOV]) {
+ value &= ~MTL_EST_CTRL_CTOV;
+ value |= (*ctov << MTL_EST_CTRL_CTOV_SHIFT);
+ dw_tsn_hwtunable[TSN_HWTUNA_TX_EST_CTOV] = *ctov;
+ }
+ }
+
+ TSN_WR32(value, ioaddr + MTL_EST_CTRL);
+
+ return 0;
+}
+
+void dwmac_tsn_init(void *ioaddr)
+{
+ unsigned int hwid = TSN_RD32(ioaddr + GMAC4_VERSION) & TSN_VER_MASK;
+ unsigned int hw_cap2 = TSN_RD32(ioaddr + GMAC_HW_FEATURE2);
+ unsigned int hw_cap3 = TSN_RD32(ioaddr + GMAC_HW_FEATURE3);
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int gcl_depth;
+ unsigned int tils_max;
+ unsigned int ti_wid;
+
+ memset(cap, 0, sizeof(*cap));
+
+ if (hwid < TSN_CORE_VER) {
+ TSN_WARN_NA("IP v5.00 does not support TSN\n");
+ return;
+ }
+
+ if (!(hw_cap3 & GMAC_HW_FEAT_ESTSEL)) {
+ TSN_WARN_NA("EST NOT supported\n");
+ cap->est_support = 0;
+
+ return;
+ }
+
+ gcl_depth = est_get_gcl_depth(hw_cap3);
+ ti_wid = est_get_ti_width(hw_cap3);
+
+ cap->ti_wid = ti_wid;
+ cap->gcl_depth = gcl_depth;
+
+ tils_max = (hw_cap3 & GMAC_HW_FEAT_ESTSEL ? 3 : 0);
+ tils_max = (1 << tils_max) - 1;
+ cap->tils_max = tils_max;
+
+ cap->ext_max = EST_TIWID_TO_EXTMAX(ti_wid);
+ cap->txqcnt = ((hw_cap2 & GMAC_HW_FEAT_TXQCNT) >> 6) + 1;
+ cap->est_support = 1;
+
+ TSN_INFO("EST: depth=%u, ti_wid=%u, tils_max=%u tqcnt=%u\n",
+ gcl_depth, ti_wid, tils_max, cap->txqcnt);
+}
+
+void dwmac_get_tsn_hwcap(struct tsn_hw_cap **tsn_hwcap)
+{
+ *tsn_hwcap = &dw_tsn_hwcap;
+}
+
+void dwmac_set_est_gcb(struct est_gc_entry *gcl, unsigned int bank)
+{
+ if (bank >= 0 && bank < EST_GCL_BANK_MAX)
+ dw_est_gc_config.gcb[bank].gcl = gcl;
+}
+
+void dwmac_set_tsn_feat(enum tsn_feat_id featid, bool enable)
+{
+ if (featid < TSN_FEAT_ID_MAX)
+ dw_tsn_feat_en[featid] = enable;
+}
+
+int dwmac_set_tsn_hwtunable(void *ioaddr,
+ enum tsn_hwtunable_id id,
+ const unsigned int *data)
+{
+ int ret = 0;
+
+ switch (id) {
+ case TSN_HWTUNA_TX_EST_TILS:
+ ret = est_set_tils(ioaddr, *data);
+ break;
+ case TSN_HWTUNA_TX_EST_PTOV:
+ ret = est_set_ov(ioaddr, data, NULL);
+ break;
+ case TSN_HWTUNA_TX_EST_CTOV:
+ ret = est_set_ov(ioaddr, NULL, data);
+ break;
+ default:
+ ret = -EINVAL;
+ };
+
+ return ret;
+}
+
+int dwmac_get_tsn_hwtunable(enum tsn_hwtunable_id id, unsigned int *data)
+{
+ if (id >= TSN_HWTUNA_MAX)
+ return -EINVAL;
+
+ *data = dw_tsn_hwtunable[id];
+
+ return 0;
+}
+
+int dwmac_get_est_bank(void *ioaddr, unsigned int own)
+{
+ int swol;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ swol = TSN_RD32(ioaddr + MTL_EST_STATUS);
+
+ swol = ((swol & MTL_EST_STATUS_SWOL) >>
+ MTL_EST_STATUS_SWOL_SHIFT);
+
+ if (own)
+ return swol;
+ else
+ return (~swol & 0x1);
+}
+
+int dwmac_set_est_gce(void *ioaddr,
+ struct est_gc_entry *gce, unsigned int row,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int ti_nsec = gce->ti_nsec;
+ unsigned int gates = gce->gates;
+ struct est_gc_entry *gcl;
+ unsigned int gates_mask;
+ unsigned int ti_wid;
+ unsigned int ti_max;
+ unsigned int value;
+ unsigned int bank;
+ int ret;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ if (dbgb >= EST_GCL_BANK_MAX)
+ return -EINVAL;
+
+ if (dbgm) {
+ bank = dbgb;
+ } else {
+ value = TSN_RD32(ioaddr + MTL_EST_STATUS);
+ bank = (value & MTL_EST_STATUS_SWOL) >>
+ MTL_EST_STATUS_SWOL_SHIFT;
+ }
+
+ if (!cap->gcl_depth || row > cap->gcl_depth) {
+ TSN_WARN("EST: row(%u) > GCL depth(%u)\n",
+ row, cap->gcl_depth);
+
+ return -EINVAL;
+ }
+
+ ti_wid = cap->ti_wid;
+ ti_max = (1 << ti_wid) - 1;
+ if (ti_nsec > ti_max) {
+ TSN_WARN("EST: ti_nsec(%u) > upper limit(%u)\n",
+ ti_nsec, ti_max);
+
+ return -EINVAL;
+ }
+
+ gates_mask = (1 << cap->txqcnt) - 1;
+ value = ((gates & gates_mask) << ti_wid) | ti_nsec;
+
+ ret = est_write_gcl_config(ioaddr, value, row, 0, dbgb, dbgm);
+ if (ret) {
+ TSN_ERR("EST: GCE write failed: bank=%u row=%u.\n",
+ bank, row);
+
+ return ret;
+ }
+
+ TSN_INFO("EST: GCE write: dbgm=%u bank=%u row=%u, gc=0x%x.\n",
+ dbgm, bank, row, value);
+
+ /* Since GC write is successful, update GCL copy of the driver */
+ gcl = dw_est_gc_config.gcb[bank].gcl + row;
+ gcl->gates = gates;
+ gcl->ti_nsec = ti_nsec;
+
+ return ret;
+}
+
+int dwmac_get_est_gcrr_llr(void *ioaddr, unsigned int *gcl_len,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ unsigned int bank, value;
+ int ret;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ if (dbgb >= EST_GCL_BANK_MAX)
+ return -EINVAL;
+
+ if (dbgm) {
+ bank = dbgb;
+ } else {
+ value = TSN_RD32(ioaddr + MTL_EST_STATUS);
+ bank = (value & MTL_EST_STATUS_SWOL) >>
+ MTL_EST_STATUS_SWOL_SHIFT;
+ }
+
+ ret = est_read_gcl_config(ioaddr, &value,
+ GCL_CTRL_ADDR_LLR, 1,
+ dbgb, dbgm);
+ if (ret) {
+ TSN_ERR("read LLR fail at bank=%u\n", bank);
+
+ return ret;
+ }
+
+ *gcl_len = value;
+
+ return 0;
+}
+
+int dwmac_set_est_gcrr_llr(void *ioaddr, unsigned int gcl_len,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int bank, value;
+ struct est_gcrr *bgcrr;
+ int ret = 0;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ if (dbgb >= EST_GCL_BANK_MAX)
+ return -EINVAL;
+
+ if (dbgm) {
+ bank = dbgb;
+ } else {
+ value = TSN_RD32(ioaddr + MTL_EST_STATUS);
+ bank = (value & MTL_EST_STATUS_SWOL) >>
+ MTL_EST_STATUS_SWOL_SHIFT;
+ }
+
+ if (gcl_len > cap->gcl_depth) {
+ TSN_WARN("EST: GCL length(%u) > depth(%u)\n",
+ gcl_len, cap->gcl_depth);
+
+ return -EINVAL;
+ }
+
+ bgcrr = &dw_est_gc_config.gcb[bank].gcrr;
+
+ if (gcl_len != bgcrr->llr) {
+ ret = est_write_gcl_config(ioaddr, gcl_len,
+ GCL_CTRL_ADDR_LLR, 1,
+ dbgb, dbgm);
+ if (ret) {
+ TSN_ERR_NA("EST: GCRR programming failure!\n");
+
+ return ret;
+ }
+ bgcrr->llr = gcl_len;
+ }
+
+ return 0;
+}
+
+int dwmac_set_est_gcrr_times(void *ioaddr,
+ struct est_gcrr *gcrr,
+ unsigned int dbgb, unsigned int dbgm)
+{
+ unsigned int cycle_nsec = gcrr->cycle_nsec;
+ unsigned int cycle_sec = gcrr->cycle_sec;
+ unsigned int base_nsec = gcrr->base_nsec;
+ unsigned int base_sec = gcrr->base_sec;
+ unsigned int ext_nsec = gcrr->ter_nsec;
+ struct tsn_hw_cap *cap = &dw_tsn_hwcap;
+ unsigned int gcl_len, tti_ns, value;
+ struct est_gcrr *bgcrr;
+ u64 val_ns, sys_ns;
+ unsigned int bank;
+ int ret = 0;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ if (dbgb >= EST_GCL_BANK_MAX)
+ return -EINVAL;
+
+ if (dbgm) {
+ bank = dbgb;
+ } else {
+ value = TSN_RD32(ioaddr + MTL_EST_STATUS);
+ bank = (value & MTL_EST_STATUS_SWOL) >>
+ MTL_EST_STATUS_SWOL_SHIFT;
+ }
+
+ if (base_nsec > 1000000000ULL || cycle_nsec > 1000000000ULL) {
+ TSN_WARN("EST: base(%u) or cycle(%u) nsec > 1s !\n",
+ base_nsec, cycle_nsec);
+
+ return -EINVAL;
+ }
+
+ /* Ensure base time is later than MAC system time */
+ val_ns = (u64)base_nsec;
+ val_ns += (u64)(base_sec * 1000000000ULL);
+
+ /* Get the MAC system time */
+ sys_ns = TSN_RD32(ioaddr + TSN_PTP_STNSR);
+ sys_ns += TSN_RD32(ioaddr + TSN_PTP_STSR) * 1000000000ULL;
+
+ if (val_ns <= sys_ns) {
+ TSN_WARN("EST: base time(%llu) <= system time(%llu)\n",
+ val_ns, sys_ns);
+
+ return -EINVAL;
+ }
+
+ if (cycle_sec > EST_CTR_HI_MAX) {
+ TSN_WARN("EST: cycle time(%u) > 255 seconds\n", cycle_sec);
+
+ return -EINVAL;
+ }
+
+ if (ext_nsec > cap->ext_max) {
+ TSN_WARN("EST: invalid time extension(%u), max=%u\n",
+ ext_nsec, cap->ext_max);
+
+ return -EINVAL;
+ }
+
+ bgcrr = &dw_est_gc_config.gcb[bank].gcrr;
+ gcl_len = bgcrr->llr;
+
+ /* Sanity test on GCL total time intervals against cycle time.
+ * a) For GC length = 1, if its time interval is equal or greater
+ * than cycle time, it is a constant gate error.
+ * b) If total time interval > cycle time, irregardless of GC
+ * length, it is not considered an error that GC list is
+ * truncated. In this case, giving a warning message is
+ * sufficient.
+ * c) If total time interval < cycle time, irregardless of GC
+ * length, all GATES are OPEN after the last GC is processed
+ * until cycle time lapses. This is potentially due to poor
+ * GCL configuration but is not an error, so we inform user
+ * about it.
+ */
+ tti_ns = est_get_gcl_total_intervals_nsec(bank, gcl_len);
+ val_ns = (u64)cycle_nsec;
+ val_ns += (u64)(cycle_sec * 1000000000ULL);
+ if (gcl_len == 1 && tti_ns >= val_ns) {
+ TSN_WARN_NA("EST: Constant gate error!\n");
+
+ return -EINVAL;
+ }
+
+ if (tti_ns > val_ns)
+ TSN_WARN_NA("EST: GCL is truncated!\n");
+
+ if (tti_ns < val_ns) {
+ TSN_INFO("EST: All GCs OPEN at %u of %llu-ns cycle\n",
+ tti_ns, val_ns);
+ }
+
+ /* Finally, start programming GCL related registers if the value
+ * differs from the driver copy for efficiency.
+ */
+
+ if (base_nsec != bgcrr->base_nsec)
+ ret |= est_write_gcl_config(ioaddr, base_nsec,
+ GCL_CTRL_ADDR_BTR_LO, 1,
+ dbgb, dbgm);
+
+ if (base_sec != bgcrr->base_sec)
+ ret |= est_write_gcl_config(ioaddr, base_sec,
+ GCL_CTRL_ADDR_BTR_HI, 1,
+ dbgb, dbgm);
+
+ if (cycle_nsec != bgcrr->cycle_nsec)
+ ret |= est_write_gcl_config(ioaddr, cycle_nsec,
+ GCL_CTRL_ADDR_CTR_LO, 1,
+ dbgb, dbgm);
+
+ if (cycle_sec != bgcrr->cycle_sec)
+ ret |= est_write_gcl_config(ioaddr, cycle_sec,
+ GCL_CTRL_ADDR_CTR_HI, 1,
+ dbgb, dbgm);
+
+ if (ext_nsec != bgcrr->ter_nsec)
+ ret |= est_write_gcl_config(ioaddr, ext_nsec,
+ GCL_CTRL_ADDR_TER, 1,
+ dbgb, dbgm);
+
+ if (ret) {
+ TSN_ERR_NA("EST: GCRR programming failure!\n");
+
+ return ret;
+ }
+
+ /* Finally, we are ready to switch SWOL now. */
+ value = TSN_RD32(ioaddr + MTL_EST_CTRL);
+ value |= MTL_EST_CTRL_SSWL;
+ TSN_WR32(value, ioaddr + MTL_EST_CTRL);
+
+ /* Update driver copy */
+ bgcrr->base_sec = base_sec;
+ bgcrr->base_nsec = base_nsec;
+ bgcrr->cycle_sec = cycle_sec;
+ bgcrr->cycle_nsec = cycle_nsec;
+ bgcrr->ter_nsec = ext_nsec;
+
+ TSN_INFO_NA("EST: gcrr set successful\n");
+
+ return 0;
+}
+
+int dwmac_set_est_enable(void *ioaddr, bool enable)
+{
+ unsigned int value;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ value = TSN_RD32(ioaddr + MTL_EST_CTRL);
+ value &= ~(MTL_EST_CTRL_SSWL | MTL_EST_CTRL_EEST);
+ value |= (enable & MTL_EST_CTRL_EEST);
+ TSN_WR32(value, ioaddr + MTL_EST_CTRL);
+ dw_est_gc_config.enable = enable;
+
+ return 0;
+}
+
+int dwmac_get_est_gcc(void *ioaddr,
+ struct est_gc_config **gcc, bool frmdrv)
+{
+ struct est_gc_config *pgcc;
+ unsigned int bank;
+ unsigned int value;
+ int ret;
+
+ if (!dw_tsn_feat_en[TSN_FEAT_ID_EST])
+ return -ENOTSUPP;
+
+ /* Get GC config from driver */
+ if (frmdrv) {
+ *gcc = &dw_est_gc_config;
+
+ TSN_INFO_NA("EST: read GCL from driver copy done.\n");
+
+ return 0;
+ }
+
+ /* Get GC config from HW */
+ pgcc = &dw_est_gc_config;
+
+ value = TSN_RD32(ioaddr + MTL_EST_CTRL);
+ pgcc->enable = value & MTL_EST_CTRL_EEST;
+
+ for (bank = 0; bank < EST_GCL_BANK_MAX; bank++) {
+ unsigned int llr, row;
+ struct est_gc_bank *gcbc = &pgcc->gcb[bank];
+
+ ret = est_read_gcl_config(ioaddr, &value,
+ GCL_CTRL_ADDR_BTR_LO, 1,
+ bank, 1);
+ if (ret) {
+ TSN_ERR("read BTR(low) fail at bank=%u\n", bank);
+
+ return ret;
+ }
+ gcbc->gcrr.base_nsec = value;
+
+ ret = est_read_gcl_config(ioaddr, &value,
+ GCL_CTRL_ADDR_BTR_HI, 1,
+ bank, 1);
+ if (ret) {
+ TSN_ERR("read BTR(high) fail at bank=%u\n", bank);
+
+ return ret;
+ }
+ gcbc->gcrr.base_sec = value;
+
+ ret = est_read_gcl_config(ioaddr, &value,
+ GCL_CTRL_ADDR_CTR_LO, 1,
+ bank, 1);
+ if (ret) {
+ TSN_ERR("read CTR(low) fail at bank=%u\n", bank);
+
+ return ret;
+ }
+ gcbc->gcrr.cycle_nsec = value;
+
+ ret = est_read_gcl_config(ioaddr, &value,
+ GCL_CTRL_ADDR_CTR_HI, 1,
+ bank, 1);
+ if (ret) {
+ TSN_ERR("read CTR(high) fail at bank=%u\n", bank);
+
+ return ret;
+ }
+ gcbc->gcrr.cycle_sec = value;
+
+ ret = est_read_gcl_config(ioaddr, &value,
+ GCL_CTRL_ADDR_TER, 1,
+ bank, 1);
+ if (ret) {
+ TSN_ERR("read TER fail at bank=%u\n", bank);
+
+ return ret;
+ }
+ gcbc->gcrr.ter_nsec = value;
+
+ ret = est_read_gcl_config(ioaddr, &value,
+ GCL_CTRL_ADDR_LLR, 1,
+ bank, 1);
+ if (ret) {
+ TSN_ERR("read LLR fail at bank=%u\n", bank);
+
+ return ret;
+ }
+ gcbc->gcrr.llr = value;
+ llr = value;
+
+ for (row = 0; row < llr; row++) {
+ unsigned int gates, ti_nsec;
+ struct est_gc_entry *gce = gcbc->gcl + row;
+
+ ret = est_read_gce(ioaddr, row, &gates, &ti_nsec,
+ bank, 1);
+ if (ret) {
+ TSN_ERR("read GCE fail at bank=%u\n", bank);
+
+ return ret;
+ }
+ gce->gates = gates;
+ gce->ti_nsec = ti_nsec;
+ }
+ }
+
+ *gcc = pgcc;
+ TSN_INFO_NA("EST: read GCL from HW done.\n");
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h
new file mode 100644
index 000000000000..feb71f7e7031
--- /dev/null
+++ b/drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h
@@ -0,0 +1,173 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2019, Intel Corporation.
+ * dw_tsn_lib.h: DW EQoS v5.00 TSN capabilities header
+ */
+
+#ifndef __DW_TSN_LIB_H__
+#define __DW_TSN_LIB_H__
+
+#include "linux/printk.h"
+
+/* DWMAC v5.xx supports the following Time Sensitive Networking protocols:
+ * 1) IEEE 802.1Qbv Enhancements for Scheduled Traffic (EST)
+ */
+
+/* MAC HW features3 bitmap */
+#define GMAC_HW_FEAT_ESTWID GENMASK(21, 20)
+#define GMAC_HW_FEAT_ESTWID_SHIFT 20
+#define GMAC_HW_FEAT_ESTDEP GENMASK(19, 17)
+#define GMAC_HW_FEAT_ESTDEP_SHIFT 17
+#define GMAC_HW_FEAT_ESTSEL BIT(16)
+
+/* MTL EST control register */
+#define MTL_EST_CTRL 0x00000c50
+#define MTL_EST_CTRL_PTOV GENMASK(31, 24)
+#define MTL_EST_CTRL_PTOV_SHIFT 24
+#define MTL_EST_CTRL_CTOV GENMASK(23, 12)
+#define MTL_EST_CTRL_CTOV_SHIFT 12
+#define MTL_EST_CTRL_TILS GENMASK(10, 8)
+#define MTL_EST_CTRL_TILS_SHIFT 8
+#define MTL_EST_CTRL_SSWL BIT(1) /* Switch to SWOL */
+#define MTL_EST_CTRL_EEST BIT(0) /* Enable EST */
+
+/* MTL EST status register */
+#define MTL_EST_STATUS 0x00000c58
+#define MTL_EST_STATUS_BTRL GENMASK(11, 8) /* BTR ERR loop cnt */
+#define MTL_EST_STATUS_BTRL_SHIFT 8
+#define MTL_EST_STATUS_BTRL_MAX (0xF << 8)
+#define MTL_EST_STATUS_SWOL BIT(7) /* SW owned list */
+#define MTL_EST_STATUS_SWOL_SHIFT 7
+#define MTL_EST_STATUS_BTRE BIT(1) /* BTR Error */
+#define MTL_EST_STATUS_SWLC BIT(0) /* Switch to SWOL complete */
+
+/* MTL EST GCL control register */
+#define MTL_EST_GCL_CTRL 0x00000c80
+#define MTL_EST_GCL_CTRL_ADDR GENMASK(10, 8) /* GCL Address */
+#define MTL_EST_GCL_CTRL_ADDR_VAL(addr) (addr << 8)
+#define GCL_CTRL_ADDR_BTR_LO 0x0
+#define GCL_CTRL_ADDR_BTR_HI 0x1
+#define GCL_CTRL_ADDR_CTR_LO 0x2
+#define GCL_CTRL_ADDR_CTR_HI 0x3
+#define GCL_CTRL_ADDR_TER 0x4
+#define GCL_CTRL_ADDR_LLR 0x5
+#define MTL_EST_GCL_CTRL_DBGB1 BIT(5) /* Debug Mode Bank Select */
+#define MTL_EST_GCL_CTRL_DBGM BIT(4) /* Debug Mode */
+#define MTL_EST_GCL_CTRL_GCRR BIT(2) /* GC Related Registers */
+#define MTL_EST_GCL_CTRL_R1W0 BIT(1) /* Read / Write Operation */
+#define GCL_OPS_R BIT(1)
+#define GCL_OPS_W 0
+#define MTL_EST_GCL_CTRL_SRWO BIT(0) /* Start R/W Operation */
+
+/* MTL EST GCL data register */
+#define MTL_EST_GCL_DATA 0x00000c84
+
+/* EST Global defines */
+#define EST_CTR_HI_MAX 0xff /* CTR Hi is 8-bit only */
+#define EST_PTOV_MAX 0xff /* Max PTP time offset */
+#define EST_CTOV_MAX 0xfff /* Max Current time offset */
+#define EST_TIWID_TO_EXTMAX(ti_wid) ((1 << (ti_wid + 7)) - 1)
+#define EST_GCL_BANK_MAX (2)
+
+/* MAC Core Version */
+#define TSN_VER_MASK 0xFF
+#define TSN_CORE_VER 0x50
+
+/* MAC PTP clock registers */
+#define TSN_PTP_STSR 0x08
+#define TSN_PTP_STNSR 0x0c
+
+/* Hardware Tunable Enum */
+enum tsn_hwtunable_id {
+ TSN_HWTUNA_TX_EST_TILS = 0,
+ TSN_HWTUNA_TX_EST_PTOV,
+ TSN_HWTUNA_TX_EST_CTOV,
+ TSN_HWTUNA_MAX,
+};
+
+/* TSN Feature Enabled List */
+enum tsn_feat_id {
+ TSN_FEAT_ID_EST = 0,
+ TSN_FEAT_ID_MAX,
+};
+
+/* HW register read & write macros */
+#define TSN_RD32(__addr) readl(__addr)
+#define TSN_WR32(__val, __addr) writel(__val, __addr)
+
+/* Logging macros with no args */
+#define DRVNAME "stmmac"
+#define TSN_INFO_NA(__msg) printk(KERN_INFO DRVNAME ":" __msg)
+#define TSN_WARN_NA(__msg) printk(KERN_WARNING DRVNAME ":" __msg)
+#define TSN_ERR_NA(__msg) printk(KERN_ERR DRVNAME ":" __msg)
+
+/* Logging macros with args */
+#define TSN_INFO(__msg, __arg0, __args...) \
+ printk(KERN_INFO DRVNAME ":" __msg, (__arg0), ##__args)
+#define TSN_WARN(__msg, __arg0, __args...) \
+ printk(KERN_WARNING DRVNAME ":" __msg, (__arg0), ##__args)
+#define TSN_ERR(__msg, __arg0, __args...) \
+ printk(KERN_ERR DRVNAME ":" __msg, (__arg0), ##__args)
+
+/* TSN HW Capabilities */
+struct tsn_hw_cap {
+ bool est_support; /* 1: supported */
+ unsigned int txqcnt; /* Number of TxQ (control gate) */
+ unsigned int gcl_depth; /* GCL depth. */
+ unsigned int ti_wid; /* time interval width */
+ unsigned int tils_max; /* Max time interval left shift */
+ unsigned int ext_max; /* Max time extension */
+};
+
+/* EST Gate Control Entry */
+struct est_gc_entry {
+ unsigned int gates; /* gate control: 0: closed,
+ * 1: open.
+ */
+ unsigned int ti_nsec; /* time interval in nsec */
+};
+
+/* EST GCL Related Registers */
+struct est_gcrr {
+ unsigned int base_nsec; /* base time denominator (nsec) */
+ unsigned int base_sec; /* base time numerator (sec) */
+ unsigned int cycle_nsec; /* cycle time denominator (nsec) */
+ unsigned int cycle_sec; /* cycle time numerator sec)*/
+ unsigned int ter_nsec; /* time extension (nsec) */
+ unsigned int llr; /* GC list length */
+};
+
+/* EST Gate Control bank */
+struct est_gc_bank {
+ struct est_gc_entry *gcl; /* Gate Control List */
+ struct est_gcrr gcrr; /* GCL Related Registers */
+};
+
+/* EST Gate Control Configuration */
+struct est_gc_config {
+ struct est_gc_bank gcb[EST_GCL_BANK_MAX];
+ bool enable; /* 1: enabled */
+};
+
+/* TSN functions */
+void dwmac_tsn_init(void *ioaddr);
+void dwmac_get_tsn_hwcap(struct tsn_hw_cap **tsn_hwcap);
+void dwmac_set_est_gcb(struct est_gc_entry *gcl, unsigned int bank);
+void dwmac_set_tsn_feat(enum tsn_feat_id featid, bool enable);
+int dwmac_set_tsn_hwtunable(void *ioaddr, enum tsn_hwtunable_id id,
+ const unsigned int *data);
+int dwmac_get_tsn_hwtunable(enum tsn_hwtunable_id id, unsigned int *data);
+int dwmac_get_est_bank(void *ioaddr, unsigned int own);
+int dwmac_set_est_gce(void *ioaddr,
+ struct est_gc_entry *gce, unsigned int row,
+ unsigned int dbgb, unsigned int dbgm);
+int dwmac_get_est_gcrr_llr(void *ioaddr, unsigned int *gcl_len,
+ unsigned int dbgb, unsigned int dbgm);
+int dwmac_set_est_gcrr_llr(void *ioaddr, unsigned int gcl_len,
+ unsigned int dbgb, unsigned int dbgm);
+int dwmac_set_est_gcrr_times(void *ioaddr,
+ struct est_gcrr *gcrr,
+ unsigned int dbgb, unsigned int dbgm);
+int dwmac_set_est_enable(void *ioaddr, bool enable);
+int dwmac_get_est_gcc(void *ioaddr,
+ struct est_gc_config **gcc, bool frmdrv);
+#endif /* __DW_TSN_LIB_H__ */
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
index 8d9f6cda4012..1361807fe802 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c
@@ -817,6 +817,19 @@ static void dwmac4_set_mac_loopback(void __iomem *ioaddr, bool enable)
.pcs_get_adv_lp = dwmac4_get_adv_lp,
.debug = dwmac4_debug,
.set_filter = dwmac4_set_filter,
+ .tsn_init = dwmac_tsn_init,
+ .get_tsn_hwcap = dwmac_get_tsn_hwcap,
+ .set_est_gcb = dwmac_set_est_gcb,
+ .set_tsn_feat = dwmac_set_tsn_feat,
+ .set_tsn_hwtunable = dwmac_set_tsn_hwtunable,
+ .get_tsn_hwtunable = dwmac_get_tsn_hwtunable,
+ .get_est_bank = dwmac_get_est_bank,
+ .set_est_gce = dwmac_set_est_gce,
+ .get_est_gcrr_llr = dwmac_get_est_gcrr_llr,
+ .set_est_gcrr_llr = dwmac_set_est_gcrr_llr,
+ .set_est_gcrr_times = dwmac_set_est_gcrr_times,
+ .set_est_enable = dwmac_set_est_enable,
+ .get_est_gcc = dwmac_get_est_gcc,
.safety_feat_config = dwmac5_safety_feat_config,
.safety_feat_irq_status = dwmac5_safety_feat_irq_status,
.safety_feat_dump = dwmac5_safety_feat_dump,
diff --git a/drivers/net/ethernet/stmicro/stmmac/hwif.h b/drivers/net/ethernet/stmicro/stmmac/hwif.h
index 2acfbc70e3c8..518a72805185 100644
--- a/drivers/net/ethernet/stmicro/stmmac/hwif.h
+++ b/drivers/net/ethernet/stmicro/stmmac/hwif.h
@@ -7,6 +7,7 @@
#include <linux/netdevice.h>
#include <linux/stmmac.h>
+#include "dw_tsn_lib.h"
#define stmmac_do_void_callback(__priv, __module, __cname, __arg0, __args...) \
({ \
@@ -311,6 +312,31 @@ struct stmmac_ops {
bool loopback);
void (*pcs_rane)(void __iomem *ioaddr, bool restart);
void (*pcs_get_adv_lp)(void __iomem *ioaddr, struct rgmii_adv *adv);
+ /* TSN functions */
+ void (*tsn_init)(void __iomem *ioaddr);
+ void (*get_tsn_hwcap)(struct tsn_hw_cap **tsn_hwcap);
+ void (*set_est_gcb)(struct est_gc_entry *gcl,
+ u32 bank);
+ void (*set_tsn_feat)(enum tsn_feat_id featid, bool enable);
+ int (*set_tsn_hwtunable)(void __iomem *ioaddr,
+ enum tsn_hwtunable_id id,
+ const unsigned int *data);
+ int (*get_tsn_hwtunable)(enum tsn_hwtunable_id id,
+ unsigned int *data);
+ int (*get_est_bank)(void __iomem *ioaddr, u32 own);
+ int (*set_est_gce)(void __iomem *ioaddr,
+ struct est_gc_entry *gce, u32 row,
+ u32 dbgb, u32 dbgm);
+ int (*get_est_gcrr_llr)(void __iomem *ioaddr, u32 *gcl_len,
+ u32 dbgb, u32 dbgm);
+ int (*set_est_gcrr_llr)(void __iomem *ioaddr, u32 gcl_len,
+ u32 dbgb, u32 dbgm);
+ int (*set_est_gcrr_times)(void __iomem *ioaddr,
+ struct est_gcrr *gcrr,
+ u32 dbgb, u32 dbgm);
+ int (*set_est_enable)(void __iomem *ioaddr, bool enable);
+ int (*get_est_gcc)(void __iomem *ioaddr,
+ struct est_gc_config **gcc, bool frmdrv);
/* Safety Features */
int (*safety_feat_config)(void __iomem *ioaddr, unsigned int asp);
int (*safety_feat_irq_status)(struct net_device *ndev,
@@ -385,6 +411,32 @@ struct stmmac_ops {
stmmac_do_void_callback(__priv, mac, pcs_rane, __args)
#define stmmac_pcs_get_adv_lp(__priv, __args...) \
stmmac_do_void_callback(__priv, mac, pcs_get_adv_lp, __args)
+#define stmmac_tsn_init(__priv, __args...) \
+ stmmac_do_void_callback(__priv, mac, tsn_init, __args)
+#define stmmac_get_tsn_hwcap(__priv, __args...) \
+ stmmac_do_void_callback(__priv, mac, get_tsn_hwcap, __args)
+#define stmmac_set_est_gcb(__priv, __args...) \
+ stmmac_do_void_callback(__priv, mac, set_est_gcb, __args)
+#define stmmac_set_tsn_feat(__priv, __args...) \
+ stmmac_do_void_callback(__priv, mac, set_tsn_feat, __args)
+#define stmmac_set_tsn_hwtunable(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, set_tsn_hwtunable, __args)
+#define stmmac_get_tsn_hwtunable(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, get_tsn_hwtunable, __args)
+#define stmmac_get_est_bank(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, get_est_bank, __args)
+#define stmmac_set_est_gce(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, set_est_gce, __args)
+#define stmmac_get_est_gcrr_llr(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, get_est_gcrr_llr, __args)
+#define stmmac_set_est_gcrr_llr(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, set_est_gcrr_llr, __args)
+#define stmmac_set_est_gcrr_times(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, set_est_gcrr_times, __args)
+#define stmmac_set_est_enable(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, set_est_enable, __args)
+#define stmmac_get_est_gcc(__priv, __args...) \
+ stmmac_do_callback(__priv, mac, get_est_gcc, __args)
#define stmmac_safety_feat_config(__priv, __args...) \
stmmac_do_callback(__priv, mac, safety_feat_config, __args)
#define stmmac_safety_feat_irq_status(__priv, __args...) \
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index a48751989fa6..91213cd3a668 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -41,6 +41,7 @@
#include "stmmac.h"
#include <linux/reset.h>
#include <linux/of_mdio.h>
+#include "dw_tsn_lib.h"
#include "dwmac1000.h"
#include "dwxgmac2.h"
#include "hwif.h"
@@ -3621,6 +3622,8 @@ static int stmmac_set_features(struct net_device *netdev,
*/
stmmac_rx_ipc(priv, priv->hw);
+ netdev->features = features;
+
return 0;
}
@@ -4070,6 +4073,8 @@ static void stmmac_service_task(struct work_struct *work)
*/
static int stmmac_hw_init(struct stmmac_priv *priv)
{
+ struct tsn_hw_cap *tsn_hwcap;
+ int gcl_depth = 0;
int ret;
/* dwmac-sun8i only work in chain mode */
@@ -4082,6 +4087,38 @@ static int stmmac_hw_init(struct stmmac_priv *priv)
if (ret)
return ret;
+ /* Initialize TSN capability */
+ stmmac_tsn_init(priv, priv->ioaddr);
+ stmmac_get_tsn_hwcap(priv, &tsn_hwcap);
+ if (tsn_hwcap)
+ gcl_depth = tsn_hwcap->gcl_depth;
+ if (gcl_depth > 0) {
+ u32 bank;
+ struct est_gc_entry *gcl[EST_GCL_BANK_MAX];
+
+ for (bank = 0; bank < EST_GCL_BANK_MAX; bank++) {
+ gcl[bank] = devm_kzalloc(priv->device,
+ (sizeof(*gcl) * gcl_depth),
+ GFP_KERNEL);
+ if (!gcl[bank]) {
+ ret = -ENOMEM;
+ break;
+ }
+ stmmac_set_est_gcb(priv, gcl[bank], bank);
+ }
+ if (ret) {
+ int i;
+
+ for (i = bank - 1; i >= 0; i--) {
+ devm_kfree(priv->device, gcl[i]);
+ stmmac_set_est_gcb(priv, NULL, bank);
+ }
+ dev_warn(priv->device, "EST: GCL -ENOMEM\n");
+
+ return ret;
+ }
+ }
+
/* Get the HW capability (new GMAC newer than 3.50a) */
priv->hw_cap_support = stmmac_get_hw_features(priv);
if (priv->hw_cap_support) {
@@ -4168,6 +4205,7 @@ int stmmac_dvr_probe(struct device *device,
struct stmmac_resources *res)
{
struct net_device *ndev = NULL;
+ struct tsn_hw_cap *tsn_hwcap;
struct stmmac_priv *priv;
u32 queue, maxq;
int ret = 0;
@@ -4254,6 +4292,14 @@ int stmmac_dvr_probe(struct device *device,
}
ndev->features |= ndev->hw_features | NETIF_F_HIGHDMA;
ndev->watchdog_timeo = msecs_to_jiffies(watchdog);
+
+ /* TSN HW feature setup */
+ stmmac_get_tsn_hwcap(priv, &tsn_hwcap);
+ if (tsn_hwcap && tsn_hwcap->est_support && priv->plat->tsn_est_en) {
+ stmmac_set_tsn_feat(priv, TSN_FEAT_ID_EST, true);
+ dev_info(priv->device, "EST feature enabled\n");
+ }
+
#ifdef STMMAC_VLAN_TAG_USED
/* Both mac100 and gmac support receive VLAN tag detection */
ndev->features |= NETIF_F_HW_VLAN_CTAG_RX | NETIF_F_HW_VLAN_STAG_RX;
diff --git a/include/linux/stmmac.h b/include/linux/stmmac.h
index 7d06241582dd..d4a90f48e49b 100644
--- a/include/linux/stmmac.h
+++ b/include/linux/stmmac.h
@@ -172,6 +172,7 @@ struct plat_stmmacenet_data {
int has_gmac4;
bool has_sun8i;
bool tso_en;
+ bool tsn_est_en;
int mac_port_sel_speed;
bool en_tx_lpi_clockgating;
int has_xgmac;
--
1.9.1
^ permalink raw reply related
* [RFC net-next 0/5] net: stmmac: Introducing IEEE802.1Qbv feature
From: Voon Weifeng @ 2019-06-18 21:36 UTC (permalink / raw)
To: David S. Miller, Maxime Coquelin
Cc: netdev, linux-kernel, Jose Abreu, Giuseppe Cavallaro, Andrew Lunn,
Florian Fainelli, Alexandre Torgue, Vinicius Costa Gomes,
Ong Boon Leong, Voon Weifeng
Enabling IEEE 802.1Qbv Enhancements for Scheduled Traffics (EST) which
is available in EQoS ver5.xx. The EST features can be configured using
tc taprio hw offload which proposed by Vinicius. A few hw tunable data
are configured through platform data.
Ong Boon Leong (1):
net: stmmac: introduce IEEE 802.1Qbv configuration functionalities
Vinicius Costa Gomes (1):
taprio: Add support for hardware offloading
Voon Weifeng (3):
net: stmmac: gcl errors reporting and its interrupt handling
net: stmmac: enable HW offloading for tc taprio
net: stmmac: Set TSN HW tunable after tsn setup
drivers/net/ethernet/stmicro/stmmac/Makefile | 2 +-
drivers/net/ethernet/stmicro/stmmac/common.h | 4 +
drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c | 913 ++++++++++++++++++++++
drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h | 218 ++++++
drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c | 16 +
drivers/net/ethernet/stmicro/stmmac/hwif.h | 66 ++
drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 71 ++
drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c | 96 +++
include/linux/netdevice.h | 1 +
include/linux/stmmac.h | 4 +
include/net/pkt_sched.h | 18 +
include/uapi/linux/pkt_sched.h | 4 +
net/sched/sch_taprio.c | 263 ++++++-
13 files changed, 1673 insertions(+), 3 deletions(-)
create mode 100644 drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.c
create mode 100644 drivers/net/ethernet/stmicro/stmmac/dw_tsn_lib.h
--
1.9.1
^ permalink raw reply
* Re: [PATCH net-next v1 08/11] xdp: tracking page_pool resources and safe removal
From: Ilias Apalodimas @ 2019-06-18 13:30 UTC (permalink / raw)
To: Ivan Khoronzhuk
Cc: Tariq Toukan, Jesper Dangaard Brouer, netdev@vger.kernel.org,
Toke Høiland-Jørgensen, toshiaki.makita1@gmail.com,
grygorii.strashko@ti.com, mcroce@redhat.com
In-Reply-To: <20190618125431.GA5307@khorivan>
Hi Ivan, Tariq,
> >>>+
[...]
> >>
> >>What would you recommend to do for the following situation:
> >>
> >>Same receive queue is shared between 2 network devices. The receive ring is
> >>filled by pages from page_pool, but you don't know the actual port (ndev)
> >>filling this ring, because a device is recognized only after packet is
> >>received.
> >>
> >>The API is so that xdp rxq is bind to network device, each frame has
> >>reference
> >>on it, so rxq ndev must be static. That means each netdev has it's own rxq
> >>instance even no need in it. Thus, after your changes, page must be
> >>returned to
> >>the pool it was taken from, or released from old pool and recycled in
> >>new one
> >>somehow.
> >>
> >>And that is inconvenience at least. It's hard to move pages between
> >>pools w/o
> >>performance penalty. No way to use common pool either, as unreg_rxq now
> >>drops
> >>the pool and 2 rxqa can't reference same pool.
> >>
> >
> >Within the single netdev, separate page_pool instances are anyway
> >created for different RX rings, working under different NAPI's.
>
> The circumstances are so that same RX ring is shared between 2
> netdevs... and netdev can be known only after descriptor/packet is
> received. Thus, while filling RX ring, there is no actual device,
> but when packet is received it has to be recycled to appropriate
> net device pool. Before this change there were no difference from
> which pool the page was allocated to fill RX ring, as there were no
> owner. After this change there is owner - netdev page pool.
>
> For cpsw the dma unmap is common for both netdevs and no difference
> who is freeing the page, but there is difference which pool it's
> freed to.
Since 2 netdevs are sharing one queue you'll need locking right?
(Assuming that the rx-irq per device can end up on a different core)
We discussed that ideally page pools should be alocated per hardware queue.
If you indeed need locking (and pay the performance penalty anyway) i wonder if
there's anything preventing you from keeping the same principle, i.e allocate a
pool per queue and handle the recycling to the proper ndev internally.
That way only the first device will be responsible of
allocating/recycling/maintaining the pool state.
>
> So that, while filling RX ring the page is taken from page pool of
> ndev1, but packet is received for ndev2, it has to be later
> returned/recycled to page pool of ndev1, but when xdp buffer is
> handed over to xdp prog the xdp_rxq_info has reference on ndev2 ...
>
> And no way to predict the final ndev before packet is received, so no
> way to choose appropriate page pool as now it becomes page owner.
>
> So, while RX ring filling, the page/dma recycling is needed but should
> be some way to identify page owner only after receiving packet.
>
> Roughly speaking, something like:
>
> pool->pages_state_hold_cnt++;
>
> outside of page allocation API, after packet is received.
> and free of the counter while allocation (w/o owing the page).
If handling it internally is not an option maybe we can sort something out for
special devices
Thanks
/Ilias
^ permalink raw reply
* [PATCH net v5 6/6] ip6_fib: Don't discard nodes with valid routing information in fib6_locate_1()
From: Stefano Brivio @ 2019-06-18 13:20 UTC (permalink / raw)
To: David Miller, David Ahern
Cc: Jianlin Shi, Wei Wang, Martin KaFai Lau, Eric Dumazet,
Matti Vaittinen, netdev
In-Reply-To: <cover.1560827176.git.sbrivio@redhat.com>
When we perform an inexact match on FIB nodes via fib6_locate_1(), longer
prefixes will be preferred to shorter ones. However, it might happen that
a node, with higher fn_bit value than some other, has no valid routing
information.
In this case, we'll pick that node, but it will be discarded by the check
on RTN_RTINFO in fib6_locate(), and we might miss nodes with valid routing
information but with lower fn_bit value.
This is apparent when a routing exception is created for a default route:
# ip -6 route list
fc00:1::/64 dev veth_A-R1 proto kernel metric 256 pref medium
fc00:2::/64 dev veth_A-R2 proto kernel metric 256 pref medium
fc00:4::1 via fc00:2::2 dev veth_A-R2 metric 1024 pref medium
fe80::/64 dev veth_A-R1 proto kernel metric 256 pref medium
fe80::/64 dev veth_A-R2 proto kernel metric 256 pref medium
default via fc00:1::2 dev veth_A-R1 metric 1024 pref medium
# ip -6 route list cache
fc00:4::1 via fc00:2::2 dev veth_A-R2 metric 1024 expires 593sec mtu 1500 pref medium
fc00:3::1 via fc00:1::2 dev veth_A-R1 metric 1024 expires 593sec mtu 1500 pref medium
# ip -6 route flush cache # node for default route is discarded
Failed to send flush request: No such process
# ip -6 route list cache
fc00:3::1 via fc00:1::2 dev veth_A-R1 metric 1024 expires 586sec mtu 1500 pref medium
Check right away if the node has a RTN_RTINFO flag, before replacing the
'prev' pointer, that indicates the longest matching prefix found so far.
Fixes: 38fbeeeeccdb ("ipv6: prepare fib6_locate() for exception table")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
---
v5: No changes
v4: No changes
v3: No changes
v2: No changes
net/ipv6/ip6_fib.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index fc93e1b439a3..17c75ff2fa63 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -1543,7 +1543,8 @@ static struct fib6_node *fib6_locate_1(struct fib6_node *root,
if (plen == fn->fn_bit)
return fn;
- prev = fn;
+ if (fn->fn_flags & RTN_RTINFO)
+ prev = fn;
next:
/*
--
2.20.1
^ permalink raw reply related
* [PATCH net v5 5/6] ipv6: Dump route exceptions if requested
From: Stefano Brivio @ 2019-06-18 13:20 UTC (permalink / raw)
To: David Miller, David Ahern
Cc: Jianlin Shi, Wei Wang, Martin KaFai Lau, Eric Dumazet,
Matti Vaittinen, netdev
In-Reply-To: <cover.1560827176.git.sbrivio@redhat.com>
Since commit 2b760fcf5cfb ("ipv6: hook up exception table to store dst
cache"), route exceptions reside in a separate hash table, and won't be
found by walking the FIB, so they won't be dumped to userspace on a
RTM_GETROUTE message.
This causes 'ip -6 route list cache' and 'ip -6 route flush cache' to
have no function anymore:
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 539sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 536sec mtu 1500 pref medium
# ip -6 route list cache
# ip -6 route flush cache
# ip -6 route get fc00:3::1
fc00:3::1 via fc00:1::2 dev veth_A-R1 src fc00:1::1 metric 1024 expires 520sec mtu 1400 pref medium
# ip -6 route get fc00:4::1
fc00:4::1 via fc00:2::2 dev veth_A-R2 src fc00:2::1 metric 1024 expires 519sec mtu 1500 pref medium
because iproute2 lists cached routes using RTM_GETROUTE, and flushes them
by listing all the routes, and deleting them with RTM_DELROUTE one by one.
If cached routes are requested using the RTM_F_CLONED flag together with
strict checking, or if no strict checking is requested (and hence we can't
consistently apply filters), look up exceptions in the hash table
associated with the current fib6_info in rt6_dump_route(), and, if present
and not expired, add them to the dump.
We might be unable to dump all the entries for a given node in a single
message, so keep track of how many entries were handled for the current
node in fib6_walker, and skip that amount in case we start from the same
partially dumped node.
Note that, with the current version of iproute2, this only fixes the
'ip -6 route list cache': on a flush command, iproute2 doesn't pass
RTM_F_CLONED and, due to this inconsistency, 'ip -6 route flush cache' is
still unable to fetch the routes to be flushed. This is now addressed in a
patch for iproute2.
To flush cached routes, a procfs entry could be introduced instead: that's
how it works for IPv4. We already have a rt6_flush_exception() function
ready to be wired to it. However, this would not solve the issue for
listing.
Versions of iproute2 and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v5:
- use dump_routes and dump_exceptions from filter, ignore NLM_F_MATCH,
update test results (flushing works with iproute2 < 5.0.0 now)
v4:
- split NLM_F_MATCH and strict check handling in separate patches
- filter routes using RTM_F_CLONED: if it's not set, only return
non-cached routes, and if it's set, only return cached routes:
change requested by David Ahern and Martin Lau. This implies that
iproute2 needs a separate patch to be able to flush IPv6 cached
routes. This is not ideal because we can't fix the breakage caused
by 2b760fcf5cfb entirely in kernel. However, two years have passed
since then, and this makes it more tolerable
v3:
- more descriptive comment about expired exceptions in rt6_dump_route()
- swap return values of rt6_dump_route() (suggested by Martin Lau)
- don't zero skip_in_node in case we don't dump anything in a given pass
(also suggested by Martin Lau)
- remove check on RTM_F_CLONED altogether: in the current UAPI semantic,
it's just a flag to indicate the route was cloned, not to filter on
routes
v2: Add tracking of number of entries to be skipped in current node after
a partial dump. As we restart from the same node, if not all the
exceptions for a given node fit in a single message, the dump will
not terminate, as suggested by Martin Lau. This is a concrete
possibility, setting up a big number of exceptions for the same route
actually causes the issue, suggested by David Ahern.
Reported-by: Jianlin Shi <jishi@redhat.com>
Fixes: 2b760fcf5cfb ("ipv6: hook up exception table to store dst cache")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
---
This will cause a non-trivial conflict with commit cc5c073a693f
("ipv6: Move exception bucket to fib6_nh") on net-next. I can submit
an equivalent patch against net-next, if it helps.
include/net/ip6_fib.h | 1 +
include/net/ip6_route.h | 2 +-
net/ipv6/ip6_fib.c | 14 ++++++-
net/ipv6/route.c | 85 +++++++++++++++++++++++++++++++++++------
4 files changed, 87 insertions(+), 15 deletions(-)
diff --git a/include/net/ip6_fib.h b/include/net/ip6_fib.h
index 855b352b660f..5909a9d8ff67 100644
--- a/include/net/ip6_fib.h
+++ b/include/net/ip6_fib.h
@@ -312,6 +312,7 @@ struct fib6_walker {
enum fib6_walk_state state;
unsigned int skip;
unsigned int count;
+ unsigned int skip_in_node;
int (*func)(struct fib6_walker *);
void *args;
};
diff --git a/include/net/ip6_route.h b/include/net/ip6_route.h
index 4790beaa86e0..b66c4aac56ab 100644
--- a/include/net/ip6_route.h
+++ b/include/net/ip6_route.h
@@ -178,7 +178,7 @@ struct rt6_rtnl_dump_arg {
struct fib_dump_filter filter;
};
-int rt6_dump_route(struct fib6_info *f6i, void *p_arg);
+int rt6_dump_route(struct fib6_info *f6i, void *p_arg, unsigned int skip);
void rt6_mtu_change(struct net_device *dev, unsigned int mtu);
void rt6_remove_prefsrc(struct inet6_ifaddr *ifp);
void rt6_clean_tohost(struct net *net, struct in6_addr *gateway);
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index e846192573b0..fc93e1b439a3 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -469,12 +469,19 @@ static int fib6_dump_node(struct fib6_walker *w)
struct fib6_info *rt;
for_each_fib6_walker_rt(w) {
- res = rt6_dump_route(rt, w->args);
- if (res < 0) {
+ res = rt6_dump_route(rt, w->args, w->skip_in_node);
+ if (res >= 0) {
/* Frame is full, suspend walking */
w->leaf = rt;
+
+ /* We'll restart from this node, so if some routes were
+ * already dumped, skip them next time.
+ */
+ w->skip_in_node += res;
+
return 1;
}
+ w->skip_in_node = 0;
/* Multipath routes are dumped in one route with the
* RTA_MULTIPATH attribute. Jump 'rt' to point to the
@@ -526,6 +533,7 @@ static int fib6_dump_table(struct fib6_table *table, struct sk_buff *skb,
if (cb->args[4] == 0) {
w->count = 0;
w->skip = 0;
+ w->skip_in_node = 0;
spin_lock_bh(&table->tb6_lock);
res = fib6_walk(net, w);
@@ -541,6 +549,7 @@ static int fib6_dump_table(struct fib6_table *table, struct sk_buff *skb,
w->state = FWS_INIT;
w->node = w->root;
w->skip = w->count;
+ w->skip_in_node = 0;
} else
w->skip = 0;
@@ -2039,6 +2048,7 @@ static void fib6_clean_tree(struct net *net, struct fib6_node *root,
c.w.func = fib6_clean_node;
c.w.count = 0;
c.w.skip = 0;
+ c.w.skip_in_node = 0;
c.func = func;
c.sernum = sernum;
c.arg = arg;
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index 0f60eb3a2873..7375f3b7d310 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -4854,33 +4854,94 @@ static bool fib6_info_uses_dev(const struct fib6_info *f6i,
return false;
}
-int rt6_dump_route(struct fib6_info *rt, void *p_arg)
+/* Return -1 if done with node, number of handled routes on partial dump */
+int rt6_dump_route(struct fib6_info *rt, void *p_arg, unsigned int skip)
{
struct rt6_rtnl_dump_arg *arg = (struct rt6_rtnl_dump_arg *) p_arg;
struct fib_dump_filter *filter = &arg->filter;
+ struct rt6_exception_bucket *bucket;
unsigned int flags = NLM_F_MULTI;
+ struct rt6_exception *rt6_ex;
struct net *net = arg->net;
+ int i, count = 0;
if (rt == net->ipv6.fib6_null_entry)
- return 0;
+ return -1;
if ((filter->flags & RTM_F_PREFIX) &&
!(rt->fib6_flags & RTF_PREFIX_RT)) {
/* success since this is not a prefix route */
- return 1;
+ return -1;
}
- if (filter->filter_set) {
- if ((filter->rt_type && rt->fib6_type != filter->rt_type) ||
- (filter->dev && !fib6_info_uses_dev(rt, filter->dev)) ||
- (filter->protocol && rt->fib6_protocol != filter->protocol)) {
- return 1;
- }
+ if (filter->filter_set &&
+ ((filter->rt_type && rt->fib6_type != filter->rt_type) ||
+ (filter->dev && !fib6_info_uses_dev(rt, filter->dev)) ||
+ (filter->protocol && rt->fib6_protocol != filter->protocol))) {
+ return -1;
+ }
+
+ if (filter->filter_set ||
+ !filter->dump_routes || !filter->dump_exceptions) {
flags |= NLM_F_DUMP_FILTERED;
}
- return rt6_fill_node(net, arg->skb, rt, NULL, NULL, NULL, 0,
- RTM_NEWROUTE, NETLINK_CB(arg->cb->skb).portid,
- arg->cb->nlh->nlmsg_seq, flags);
+ if (filter->dump_routes) {
+ if (skip) {
+ skip--;
+ } else {
+ if (rt6_fill_node(net, arg->skb, rt, NULL, NULL, NULL,
+ 0, RTM_NEWROUTE,
+ NETLINK_CB(arg->cb->skb).portid,
+ arg->cb->nlh->nlmsg_seq, flags)) {
+ return 0;
+ }
+ count++;
+ }
+ }
+
+ if (!filter->dump_exceptions)
+ return -1;
+
+ bucket = rcu_dereference(rt->rt6i_exception_bucket);
+ if (!bucket)
+ return -1;
+
+ for (i = 0; i < FIB6_EXCEPTION_BUCKET_SIZE; i++) {
+ hlist_for_each_entry(rt6_ex, &bucket->chain, hlist) {
+ if (skip) {
+ skip--;
+ continue;
+ }
+
+ /* Expiration of entries doesn't bump sernum, insertion
+ * does. Removal is triggered by insertion, so we can
+ * rely on the fact that if entries change between two
+ * partial dumps, this node is scanned again completely,
+ * see rt6_insert_exception() and fib6_dump_table().
+ *
+ * Count expired entries we go through as handled
+ * entries that we'll skip next time, in case of partial
+ * node dump. Otherwise, if entries expire meanwhile,
+ * we'll skip the wrong amount.
+ */
+ if (rt6_check_expired(rt6_ex->rt6i)) {
+ count++;
+ continue;
+ }
+
+ if (rt6_fill_node(net, arg->skb, rt, &rt6_ex->rt6i->dst,
+ NULL, NULL, 0, RTM_NEWROUTE,
+ NETLINK_CB(arg->cb->skb).portid,
+ arg->cb->nlh->nlmsg_seq, flags)) {
+ return count;
+ }
+
+ count++;
+ }
+ bucket++;
+ }
+
+ return -1;
}
static int inet6_rtm_valid_getroute_req(struct sk_buff *skb,
--
2.20.1
^ permalink raw reply related
* [PATCH net v5 4/6] Revert "net/ipv6: Bail early if user only wants cloned entries"
From: Stefano Brivio @ 2019-06-18 13:20 UTC (permalink / raw)
To: David Miller, David Ahern
Cc: Jianlin Shi, Wei Wang, Martin KaFai Lau, Eric Dumazet,
Matti Vaittinen, netdev
In-Reply-To: <cover.1560827176.git.sbrivio@redhat.com>
This reverts commit 08e814c9e8eb5a982cbd1e8f6bd255d97c51026f: as we
are preparing to fix listing and dumping of IPv6 cached routes, we
need to allow RTM_F_CLONED as a flag to match routes against while
dumping them.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
---
v5: No changes
v4: New patch
net/ipv6/ip6_fib.c | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 0f58596fd0b1..e846192573b0 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -578,13 +578,10 @@ static int inet6_dump_fib(struct sk_buff *skb, struct netlink_callback *cb)
} else if (nlmsg_len(nlh) >= sizeof(struct rtmsg)) {
struct rtmsg *rtm = nlmsg_data(nlh);
- arg.filter.flags = rtm->rtm_flags & (RTM_F_PREFIX|RTM_F_CLONED);
+ if (rtm->rtm_flags & RTM_F_PREFIX)
+ arg.filter.flags = RTM_F_PREFIX;
}
- /* fib entries are never clones */
- if (arg.filter.flags & RTM_F_CLONED)
- goto out;
-
w = (void *)cb->args[2];
if (!w) {
/* New dump:
--
2.20.1
^ permalink raw reply related
* [PATCH net v5 3/6] ipv4: Dump route exceptions if requested
From: Stefano Brivio @ 2019-06-18 13:20 UTC (permalink / raw)
To: David Miller, David Ahern
Cc: Jianlin Shi, Wei Wang, Martin KaFai Lau, Eric Dumazet,
Matti Vaittinen, netdev
In-Reply-To: <cover.1560827176.git.sbrivio@redhat.com>
Since commit 4895c771c7f0 ("ipv4: Add FIB nexthop exceptions."), cached
exception routes are stored as a separate entity, so they are not dumped
on a FIB dump, even if the RTM_F_CLONED flag is passed.
This implies that the command 'ip route list cache' doesn't return any
result anymore.
If the RTM_F_CLONED flag is passed, and strict checking requested,
retrieve nexthop exception routes and dump them. If no strict checking
is requested, filtering can't be performed consistently: dump everything
in that case.
With this, we need to add an argument to the netlink callback in order to
track how many entries were already dumped for the last leaf included in
a partial netlink dump.
Note that this is only as accurate as the existing tracking mechanism for
leaves: if a partial dump is restarted after exceptions are removed or
expired, we might skip some non-dumped entries. To improve this, we could
attach a 'sernum' attribute (similar to the one used for IPv6) to nexthop
entities, and bump this counter whenever exceptions change.
Listing of exception routes (pre-3.5: modified routes) was tested against
these versions of kernel and iproute2:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0
3.5-rc4 + + + + +
4.4
4.9
4.14
4.15
4.19
5.0
5.1
fixed + + + + +
Fixes: 4895c771c7f0 ("ipv4: Add FIB nexthop exceptions.")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
---
v5: Trivial adaptation for 1/6
v4: New patch
include/net/route.h | 3 ++
net/ipv4/fib_trie.c | 101 ++++++++++++++++++++++++++++++++++++++------
net/ipv4/route.c | 6 +--
3 files changed, 95 insertions(+), 15 deletions(-)
diff --git a/include/net/route.h b/include/net/route.h
index 065b47754f05..f0d0086e76ce 100644
--- a/include/net/route.h
+++ b/include/net/route.h
@@ -221,6 +221,9 @@ void ip_rt_get_source(u8 *src, struct sk_buff *skb, struct rtable *rt);
struct rtable *rt_dst_alloc(struct net_device *dev,
unsigned int flags, u16 type,
bool nopolicy, bool noxfrm, bool will_cache);
+int rt_fill_info(struct net *net, __be32 dst, __be32 src, struct rtable *rt,
+ u32 table_id, struct flowi4 *fl4, struct sk_buff *skb,
+ u32 portid, u32 seq);
struct in_ifaddr;
void fib_add_ifaddr(struct in_ifaddr *);
diff --git a/net/ipv4/fib_trie.c b/net/ipv4/fib_trie.c
index 868c74771fa9..a00408827ae8 100644
--- a/net/ipv4/fib_trie.c
+++ b/net/ipv4/fib_trie.c
@@ -2000,28 +2000,92 @@ void fib_free_table(struct fib_table *tb)
call_rcu(&tb->rcu, __trie_free_rcu);
}
+static int fib_dump_fnhe_from_leaf(struct fib_alias *fa, struct sk_buff *skb,
+ struct netlink_callback *cb,
+ int *fa_index, int fa_start)
+{
+ struct net *net = sock_net(cb->skb->sk);
+ struct fib_info *fi = fa->fa_info;
+ struct fnhe_hash_bucket *bucket;
+ struct fib_nh_common *nhc;
+ int i, genid;
+
+ if (!fi || fi->fib_flags & RTNH_F_DEAD)
+ return 0;
+
+ nhc = fib_info_nhc(fi, 0);
+ if (nhc->nhc_flags & RTNH_F_DEAD)
+ return 0;
+
+ bucket = rcu_dereference(nhc->nhc_exceptions);
+ if (!bucket)
+ return 0;
+
+ genid = fnhe_genid(net);
+
+ for (i = 0; i < FNHE_HASH_SIZE; i++) {
+ struct fib_nh_exception *fnhe;
+
+ for (fnhe = rcu_dereference(bucket[i].chain); fnhe;
+ fnhe = rcu_dereference(fnhe->fnhe_next)) {
+ struct flowi4 fl4 = {};
+ struct rtable *rt;
+ int err;
+
+ if (*fa_index < fa_start)
+ goto next;
+
+ if (fnhe->fnhe_genid != genid)
+ goto next;
+
+ if (fnhe->fnhe_expires &&
+ time_after(jiffies, fnhe->fnhe_expires))
+ goto next;
+
+ rt = rcu_dereference(fnhe->fnhe_rth_input);
+ if (!rt)
+ rt = rcu_dereference(fnhe->fnhe_rth_output);
+ if (!rt)
+ goto next;
+
+ err = rt_fill_info(net, fnhe->fnhe_daddr, 0, rt,
+ fa->tb_id, &fl4, skb,
+ NETLINK_CB(cb->skb).portid,
+ cb->nlh->nlmsg_seq);
+ if (err)
+ return err;
+next:
+ (*fa_index)++;
+ }
+ }
+
+ return 0;
+}
+
static int fn_trie_dump_leaf(struct key_vector *l, struct fib_table *tb,
struct sk_buff *skb, struct netlink_callback *cb,
struct fib_dump_filter *filter)
{
unsigned int flags = NLM_F_MULTI;
__be32 xkey = htonl(l->key);
+ int i, s_i, i_fa, s_fa, err;
struct fib_alias *fa;
- int i, s_i;
- if (filter->filter_set)
+ if (filter->filter_set ||
+ !filter->dump_exceptions || !filter->dump_routes)
flags |= NLM_F_DUMP_FILTERED;
s_i = cb->args[4];
+ s_fa = cb->args[5];
i = 0;
/* rcu_read_lock is hold by caller */
hlist_for_each_entry_rcu(fa, &l->leaf, fa_list) {
- int err;
-
if (i < s_i)
goto next;
+ i_fa = 0;
+
if (tb->tb_id != fa->tb_id)
goto next;
@@ -2038,21 +2102,34 @@ static int fn_trie_dump_leaf(struct key_vector *l, struct fib_table *tb,
goto next;
}
- err = fib_dump_info(skb, NETLINK_CB(cb->skb).portid,
- cb->nlh->nlmsg_seq, RTM_NEWROUTE,
- tb->tb_id, fa->fa_type,
- xkey, KEYLENGTH - fa->fa_slen,
- fa->fa_tos, fa->fa_info, flags);
- if (err < 0) {
- cb->args[4] = i;
- return err;
+ if (filter->dump_routes && !s_fa) {
+ err = fib_dump_info(skb, NETLINK_CB(cb->skb).portid,
+ cb->nlh->nlmsg_seq, RTM_NEWROUTE,
+ tb->tb_id, fa->fa_type,
+ xkey, KEYLENGTH - fa->fa_slen,
+ fa->fa_tos, fa->fa_info, flags);
+ if (err < 0)
+ goto stop;
+ i_fa++;
+ }
+
+ if (filter->dump_exceptions) {
+ err = fib_dump_fnhe_from_leaf(fa, skb, cb, &i_fa, s_fa);
+ if (err < 0)
+ goto stop;
}
+
next:
i++;
}
cb->args[4] = i;
return skb->len;
+
+stop:
+ cb->args[4] = i;
+ cb->args[5] = i_fa;
+ return err;
}
/* rcu_read_lock needs to be hold by caller from readside */
diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 6cb7cff22db9..cc970fd861e8 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -2663,9 +2663,9 @@ struct rtable *ip_route_output_flow(struct net *net, struct flowi4 *flp4,
EXPORT_SYMBOL_GPL(ip_route_output_flow);
/* called with rcu_read_lock held */
-static int rt_fill_info(struct net *net, __be32 dst, __be32 src,
- struct rtable *rt, u32 table_id, struct flowi4 *fl4,
- struct sk_buff *skb, u32 portid, u32 seq)
+int rt_fill_info(struct net *net, __be32 dst, __be32 src, struct rtable *rt,
+ u32 table_id, struct flowi4 *fl4, struct sk_buff *skb,
+ u32 portid, u32 seq)
{
struct rtmsg *r;
struct nlmsghdr *nlh;
--
2.20.1
^ permalink raw reply related
* [PATCH net v5 2/6] ipv4/fib_frontend: Allow RTM_F_CLONED flag to be used for filtering
From: Stefano Brivio @ 2019-06-18 13:20 UTC (permalink / raw)
To: David Miller, David Ahern
Cc: Jianlin Shi, Wei Wang, Martin KaFai Lau, Eric Dumazet,
Matti Vaittinen, netdev
In-Reply-To: <cover.1560827176.git.sbrivio@redhat.com>
This functionally reverts the check introduced by commit
e8ba330ac0c5 ("rtnetlink: Update fib dumps for strict data checking")
as modified by commit e4e92fb160d7 ("net/ipv4: Bail early if user only
wants prefix entries").
As we are preparing to fix listing of IPv4 cached routes, we need to
give userspace a way to request them.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
---
v5: No changes
v4: New patch
net/ipv4/fib_frontend.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/ipv4/fib_frontend.c b/net/ipv4/fib_frontend.c
index c28d60d6c9d0..fced49e473c7 100644
--- a/net/ipv4/fib_frontend.c
+++ b/net/ipv4/fib_frontend.c
@@ -956,8 +956,8 @@ static int inet_dump_fib(struct sk_buff *skb, struct netlink_callback *cb)
filter.flags = rtm->rtm_flags & (RTM_F_PREFIX | RTM_F_CLONED);
}
- /* fib entries are never clones and ipv4 does not use prefix flag */
- if (filter.flags & (RTM_F_PREFIX | RTM_F_CLONED))
+ /* ipv4 does not use prefix flag */
+ if (filter.flags & RTM_F_PREFIX)
return skb->len;
if (filter.table_id) {
--
2.20.1
^ permalink raw reply related
* [PATCH net v5 1/6] fib_frontend, ip6_fib: Select routes or exceptions dump from RTM_F_CLONED
From: Stefano Brivio @ 2019-06-18 13:20 UTC (permalink / raw)
To: David Miller, David Ahern
Cc: Jianlin Shi, Wei Wang, Martin KaFai Lau, Eric Dumazet,
Matti Vaittinen, netdev
In-Reply-To: <cover.1560827176.git.sbrivio@redhat.com>
The following patches add back the ability to dump IPv4 and IPv6 exception
routes, and we need to allow selection of regular routes or exceptions.
Use RTM_F_CLONED as filter to decide whether to dump routes or exceptions:
iproute2 passes it in dump requests (except for IPv6 cache flush requests,
this will be fixed in iproute2) and this used to work as long as
exceptions were stored directly in the FIB, for both IPv4 and IPv6.
Caveat: if strict checking is not requested (that is, if the dump request
doesn't go through ip_valid_fib_dump_req()), we can't filter on protocol,
tables or route types.
In this case, filtering on RTM_F_CLONED would be inconsistent: we would
fix 'ip route list cache' by returning exception routes and at the same
time introduce another bug in case another selector is present, e.g. on
'ip route list cache table main' we would return all exception routes,
without filtering on tables.
Keep this consistent by applying no filters at all, and dumping both
routes and exceptions, if strict checking is not requested. iproute2
currently filters results anyway, and no unwanted results will be
presented to the user. The kernel will just dump more data than needed.
v5: New patch: add dump_routes and dump_exceptions flags in filter and
simply clear the unwanted one if strict checking is enabled, don't
ignore NLM_F_MATCH and don't set filter_set if NLM_F_MATCH is set.
Skip filtering altogether if no strict checking is requested:
selecting routes or exceptions only would be inconsistent with the
fact we can't filter on tables.
Suggested-by: David Ahern <dsahern@gmail.com>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
---
include/net/ip_fib.h | 2 ++
net/ipv4/fib_frontend.c | 8 +++++++-
net/ipv6/ip6_fib.c | 3 ++-
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/include/net/ip_fib.h b/include/net/ip_fib.h
index bbeff32fb6cb..32a37f1afb8e 100644
--- a/include/net/ip_fib.h
+++ b/include/net/ip_fib.h
@@ -243,6 +243,8 @@ struct fib_dump_filter {
/* filter_set is an optimization that an entry is set */
bool filter_set;
bool dump_all_families;
+ bool dump_routes;
+ bool dump_exceptions;
unsigned char protocol;
unsigned char rt_type;
unsigned int flags;
diff --git a/net/ipv4/fib_frontend.c b/net/ipv4/fib_frontend.c
index e54c2bcbb465..c28d60d6c9d0 100644
--- a/net/ipv4/fib_frontend.c
+++ b/net/ipv4/fib_frontend.c
@@ -881,10 +881,15 @@ int ip_valid_fib_dump_req(struct net *net, const struct nlmsghdr *nlh,
NL_SET_ERR_MSG(extack, "Invalid values in header for FIB dump request");
return -EINVAL;
}
+
if (rtm->rtm_flags & ~(RTM_F_CLONED | RTM_F_PREFIX)) {
NL_SET_ERR_MSG(extack, "Invalid flags for FIB dump request");
return -EINVAL;
}
+ if (rtm->rtm_flags & RTM_F_CLONED)
+ filter->dump_routes = false;
+ else
+ filter->dump_exceptions = false;
filter->dump_all_families = (rtm->rtm_family == AF_UNSPEC);
filter->flags = rtm->rtm_flags;
@@ -931,9 +936,10 @@ EXPORT_SYMBOL_GPL(ip_valid_fib_dump_req);
static int inet_dump_fib(struct sk_buff *skb, struct netlink_callback *cb)
{
+ struct fib_dump_filter filter = { .dump_routes = true,
+ .dump_exceptions = true };
const struct nlmsghdr *nlh = cb->nlh;
struct net *net = sock_net(skb->sk);
- struct fib_dump_filter filter = {};
unsigned int h, s_h;
unsigned int e = 0, s_e;
struct fib_table *tb;
diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c
index 9180c8b6f764..0f58596fd0b1 100644
--- a/net/ipv6/ip6_fib.c
+++ b/net/ipv6/ip6_fib.c
@@ -558,9 +558,10 @@ static int fib6_dump_table(struct fib6_table *table, struct sk_buff *skb,
static int inet6_dump_fib(struct sk_buff *skb, struct netlink_callback *cb)
{
+ struct rt6_rtnl_dump_arg arg = { .filter.dump_exceptions = true,
+ .filter.dump_routes = true };
const struct nlmsghdr *nlh = cb->nlh;
struct net *net = sock_net(skb->sk);
- struct rt6_rtnl_dump_arg arg = {};
unsigned int h, s_h;
unsigned int e = 0, s_e;
struct fib6_walker *w;
--
2.20.1
^ permalink raw reply related
* [PATCH net v5 0/6] Fix listing (IPv4, IPv6) and flushing (IPv6) of cached route exceptions
From: Stefano Brivio @ 2019-06-18 13:20 UTC (permalink / raw)
To: David Miller, David Ahern
Cc: Jianlin Shi, Wei Wang, Martin KaFai Lau, Eric Dumazet,
Matti Vaittinen, netdev
For IPv6 cached routes, the commands 'ip -6 route list cache' and
'ip -6 route flush cache' don't work at all after route exceptions have
been moved to a separate hash table in commit 2b760fcf5cfb ("ipv6: hook
up exception table to store dst cache").
For IPv4 cached routes, the command 'ip route list cache' has also
stopped working in kernel 3.5 after commit 4895c771c7f0 ("ipv4: Add FIB
nexthop exceptions.") introduced storage for route exceptions as a
separate entity.
Fix this by allowing userspace to clearly request cached routes with
the RTM_F_CLONED flag used as a filter (in conjuction with strict
checking) and by retrieving and dumping cached routes if requested.
If strict checking is not requested (iproute2 < 5.0.0), we don't have a
way to consistently filter results on other selectors (e.g. on tables),
so skip filtering entirely and dump both regular routes and exceptions.
I'm submitting this for net as these changes fix rather relevant
breakages. However, the scope might be a bit broad, and said breakages
have been introduced 7 and 2 years ago, respectively, for IPv4 and IPv6.
Let me know if I should rebase this on net-next instead.
For IPv4, cache flushing uses a completely different mechanism, so it
wasn't affected. Listing of exception routes (modified routes pre-3.5) was
tested against these versions of kernel and iproute2:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0
3.5-rc4 + + + + +
4.4
4.9
4.14
4.15
4.19
5.0
5.1
fixed + + + + +
For IPv6, a separate iproute2 patch is required. Versions of iproute2
and kernel tested:
iproute2
kernel 4.14.0 4.15.0 4.19.0 5.0.0 5.1.0 5.1.0, patched
3.18 list + + + + + +
flush + + + + + +
4.4 list + + + + + +
flush + + + + + +
4.9 list + + + + + +
flush + + + + + +
4.14 list + + + + + +
flush + + + + + +
4.15 list
flush
4.19 list
flush
5.0 list
flush
5.1 list
flush
with list + + + + + +
fix flush + + + +
v5: Skip filtering altogether if no strict checking is requested: selecting
routes or exceptions only would be inconsistent with the fact we can't
filter on tables. Drop 1/8 (non-strict dump filter function no longer
needed), replace 2/8 (don't use NLM_F_MATCH, decide to skip routes or
exceptions in filter function), drop 6/8 (2/8 is enough for IPv6 too).
Introduce dump_routes and dump_exceptions flags in filter, adapt other
patches to that.
v4: Fix the listing issue also for IPv4, making the behaviour consistent
with IPv6. Honour NLM_F_MATCH as per RFC 3549 and allow usage of
RTM_F_CLONED filter. Split patches into smaller logical changes.
v3: Drop check on RTM_F_CLONED and rework logic of return values of
rt6_dump_route()
v2: Add count of routes handled in partial dumps, and skip them, in patch 1/2.
Stefano Brivio (6):
fib_frontend, ip6_fib: Select routes or exceptions dump from
RTM_F_CLONED
ipv4/fib_frontend: Allow RTM_F_CLONED flag to be used for filtering
ipv4: Dump route exceptions if requested
Revert "net/ipv6: Bail early if user only wants cloned entries"
ipv6: Dump route exceptions if requested
ip6_fib: Don't discard nodes with valid routing information in
fib6_locate_1()
include/net/ip6_fib.h | 1 +
include/net/ip6_route.h | 2 +-
include/net/ip_fib.h | 2 +
include/net/route.h | 3 ++
net/ipv4/fib_frontend.c | 12 +++--
net/ipv4/fib_trie.c | 101 +++++++++++++++++++++++++++++++++++-----
net/ipv4/route.c | 6 +--
net/ipv6/ip6_fib.c | 27 +++++++----
net/ipv6/route.c | 85 ++++++++++++++++++++++++++++-----
9 files changed, 199 insertions(+), 40 deletions(-)
--
2.20.1
^ permalink raw reply
* Re: [PATCH v2 00/17] net: introduce Qualcomm IPA driver
From: Alex Elder @ 2019-06-18 13:16 UTC (permalink / raw)
To: Johannes Berg, Arnd Bergmann
Cc: abhishek.esse, Ben Chan, Bjorn Andersson, cpratapa, David Miller,
Dan Williams, DTML, Eric Caruso, evgreen, Ilias Apalodimas,
Linux ARM, linux-arm-msm, Linux Kernel Mailing List, linux-soc,
Networking, Subash Abhinov Kasiviswanathan, syadagir
In-Reply-To: <066e9b39f937586f0f922abf801351553ec2ba1d.camel@sipsolutions.net>
On 6/17/19 6:28 AM, Johannes Berg wrote:
> On Tue, 2019-06-11 at 13:56 +0200, Arnd Bergmann wrote:
>> On Tue, Jun 11, 2019 at 10:12 AM Johannes Berg
>> <johannes@sipsolutions.net> wrote:
>>
>>>> As I've made clear before, my work on this has been focused on the IPA transport,
>>>> and some of this higher-level LTE architecture is new to me. But it
>>>> seems pretty clear that an abstracted WWAN subsystem is a good plan,
>>>> because these devices represent a superset of what a "normal" netdev
>>>> implements.
>>>
>>> I'm not sure I'd actually call it a superset. By themselves, these
>>> netdevs are actually completely useless to the network stack, AFAICT.
>>> Therefore, the overlap with netdevs you can really use with the network
>>> stack is pretty small?
>>
>> I think Alex meant the concept of having a type of netdev with a generic
>> user space interface for wwan and similar to a wlan device, as I understood
>> you had suggested as well, as opposed to a stacked device as in
>> rmnet or those drivers it seems to be modeled after (vlan, ip tunnel, ...)/.
Yes, that's pretty much what I meant by "superset." We still need
netdev functionality (though not between rmnet and ipa). And it sounds
like we're talking about a better framework for managing the related
WWAN devices that represent logical modem connections. We're discussing
more than one spot in the networking stack though, so I can see why
"superset" wasn't the right word.
> I guess. It is indeed currently modelled after the stacked devices, but
> those regular netdevs are inherently useful by themselves, you don't
> *have* to tunnel or use VLANs after all.
>
> With rmnet, the underlying netdev *isn't* useful by itself, because
> you're always forced to have the stacked rmnet device on top.
Well I had mentioned earlier that I thought IPA could present just
a single non-rmnet interface that could be used "directly" (i.e.,
without rmnet). But that would be a sort of hard-wired thing, and
would not be part of the general WWAN framework under discussion.
>>>> HOWEVER I disagree with your suggestion that the IPA code should
>>>> not be committed until after that is all sorted out. In part it's
>>>> for selfish reasons, but I think there are legitimate reasons to
>>>> commit IPA now *knowing* that it will need to be adapted to fit
>>>> into the generic model that gets defined and developed. Here
>>>> are some reasons why.
>>>
>>> I can't really argue with those, though I would point out that the
>>> converse also holds - if we commit to this now, then we will have to
>>> actually keep the API offered by IPA/rmnet today, so we cannot actually
>>> remove the netdev again, even if we do migrate it to offer support for a
>>> WWAN framework in the future.
>>
>> Right. The interface to support rmnet might be simple enough to keep
>> next to what becomes the generic interface, but it will always continue
>> to be an annoyance.
>
> Not easily, because fundamentally it requires an underlying netdev to
> have an ifindex, so it wouldn't just be another API to keep around
> (which I'd classify as an annoyance) but also a whole separate netdev
> that's exposed by this IPA driver, for basically this purpose only.
>
>>> I dunno if it really has to be months. I think we can cobble something
>>> together relatively quickly that addresses the needs of IPA more
>>> specifically, and then extend later?
>>>
>>> But OTOH it may make sense to take a more paced approach and think
>>> about the details more carefully than we have over in the other thread so far.
>>
>> I would hope that as soon as we can agree on a general approach, it
>> would also be possible to merge a minimal implementation into the kernel
>> along with IPA. Alex already mentioned that IPA in its current state does
>> not actually support more than one data channel, so the necessary
>> setup for it becomes even simpler.
>
> Interesting, I'm not even sure how the driver can stop multiple channels
> in the rmnet model?
Here's a little background.
The IPA driver was very large, and in an effort to have an initial driver
that was more easily accepted upstream, it was carved down to support
a single, very simple use case. It supports only a single channel for
carrying network data, and does not expose any of the IPA's other
capabilities like filtering and routing (and multiplexing).
Originally the IPA code had an IOCTL interface for adding and removing
multiplexed channel IDs, but the simplified use case expected only one
channel to be used. IOCTLs had to be removed to make the code acceptable
for upstream, and again to simplify things, we went with a hard-wired
configuration, with a single channel with an assumed set of features
in use (TCP offload, basically). Once upstream, we planned to add back
features in layers, including adding a netlink interface to control
things like managing multiplexed channels.
The overall design assumed that the IPA connection between the modem
and AP was carrying QMAP protocol though. And the rmnet driver is
designed to parse and handle that, so for the design I started with
the use of the rmnet driver made sense: it is a shim layer that takes
care of rmnet multiplexing and aggregation (and checksum offload).
So getting back to your question, the IPA in its current form only
has a single "multiplexed" channel carried over the connection
between the AP and modem. Previously (and in the future) there
was a way to add or remove channels.
>> At the moment, the rmnet configuration in include/uapi/linux/if_link.h
>> is almost trivial, with the three pieces of information needed being
>> an IFLA_LINK to point to the real device (not needed if there is only
>> one device per channel, instead of two), the IFLA_RMNET_MUX_ID
>> setting the ID of the muxing channel (not needed if there is only
>> one channel ?), a way to specify software bridging between channels
>> (not useful if there is only one channel)
>
> I think the MUX ID is something we *would* want, and we'd probably want
> a channel type as well, so as to not paint ourselves into a corner where
> the default ends up being whatever IPA supports right now.
Agreed.
> The software bridging is very questionable to start with, I'd advocate
> not supporting that at all but adding tracepoints or similar if needed
> for debugging instead.
To be honest I don't understand the connection between software
bridging and debugging, but that's OK. I'm a fan of tracepoints
and have always intended to make use of them in the IPA driver.
>> and a few flags that I assume
>> must match the remote end:
>>
>> #define RMNET_FLAGS_INGRESS_DEAGGREGATION (1U << 0)
>> #define RMNET_FLAGS_INGRESS_MAP_COMMANDS (1U << 1)
>> #define RMNET_FLAGS_INGRESS_MAP_CKSUMV4 (1U << 2)
>> #define RMNET_FLAGS_EGRESS_MAP_CKSUMV4 (1U << 3)
>
> I don't really know about these.
The hardware can aggregate multiple packets received from the
modem into a single buffer, which the rmnet driver is then able
to deaggregate. This feature is supposed to help performance
but I've always been a little skeptical because it also comes
at a cost. This is used as a flag in an rmnet (QMAP) header,
which to me seems a little odd. (There should be a distinction
between flags needed in a message header and flags that represent
properties of a connection or channel.)
I believe the only QMAP commands are for doing essentially
XON/XOFF flow control on a single channel. In the course of
the e-mail discussion in the past few weeks I've come to see
why that would be necessary.
The checksum offload is done differently, depending on whether
it's ingress (download from modem) or egress. For egress,
a header is inserted that describes what the hardware should
checksum and where it should place the result. For ingress,
the hardware appends a trailer that contains information
about the computed checksum values. The rmnet driver is
currently responsible for inserting the header and parsing
the trailer.
I'm probably missing something, but I think the checksum
offload could be handled by the IPA driver rather than
rmnet. It seems to be an add-on that is completely
independent of the multiplexing and aggregation capabilities
that QMAP provides.
>>> If true though, then I think this would be the killer argument *in
>>> favour* of *not* merging this - because that would mean we *don't* have
>>> to actually keep the rmnet API around for all foreseeable future.
This is because it's a user space API? If so I now understand
what you mean.
As Arnd said (below) this is designed in the way out-of-tree code
works and expects. I don't want to advocate for breaking that,
but if a general model that supports what's required can be used,
I'll adapt the IPA code to suit that.
My goal continues to be getting a baseline IPA driver accepted
upstream as soon as possible, so I can then start building on
that foundation.
-Alex
>> I would agree with that. From the code I can see no other driver
>> including the rmnet protocol header (see the discussion about moving
>> the header to include/linux in order to merge ipa), and I don't see
>> any other driver referencing ETH_P_MAP either. My understanding
>> is that any driver used by rmnet would require both, but they are
>> all out-of-tree at the moment.
>
> I guess that would mean we have more work to do here, but it also means
> we don't have to support these interfaces forever.
>
> I'm not *entirely* convinced though. rmnet in itself doesn't really seem
> to require anything from the underlying netdev, so if there's a driver
> that just blindly passes things through to the hardware expecting the
> right configuration, we wouldn't really see it this way?
>
> OTOH, such a driver would probably blow up completely if somebody tried
> to use it without rmnet on top, and so it would at least have to check
> for ETH_P_MAP?
>
> johannes
>
^ permalink raw reply
* [PATCH v2 0/3] fix bugs when enable route_localnet
From: luoshijie @ 2019-06-18 15:14 UTC (permalink / raw)
To: davem, tgraf, dsahern
Cc: netdev, liuzhiqiang26, wangxiaogang3, mingfangsen, zhoukang7
From: Shijie Luo <luoshijie1@huawei.com>
When enable route_localnet, route of the 127/8 address is enabled.
But in some situations like arp_announce=2, ARP requests or reply
work abnormally.
This patchset fix some bugs when enable route_localnet.
Change History:
V2:
- Change a single patch to a patchset.
- Add bug fix for arp_ignore = 3.
- Add a couple of test for enabling route_localnet in selftests.
Shijie Luo (3):
ipv4: fix inet_select_addr() when enable route_localnet
ipv4: fix confirm_addr_indev() when enable route_localnet
selftests: add route_localnet test script
net/ipv4/devinet.c | 15 +++-
tools/testing/selftests/net/route_localnet.sh | 74 +++++++++++++++++++
2 files changed, 86 insertions(+), 3 deletions(-)
create mode 100755 tools/testing/selftests/net/route_localnet.sh
--
2.19.1
^ permalink raw reply
* [PATCH v2 3/3] selftests: add route_localnet test script
From: luoshijie @ 2019-06-18 15:14 UTC (permalink / raw)
To: davem, tgraf, dsahern
Cc: netdev, liuzhiqiang26, wangxiaogang3, mingfangsen, zhoukang7
In-Reply-To: <1560870845-172395-1-git-send-email-luoshijie1@huawei.com>
From: Shijie Luo <luoshijie1@huawei.com>
Add a simple scripts to exercise several situations when enable
route_localnet.
Signed-off-by: Shijie Luo <luoshijie1@huawei.com>
Signed-off-by: Zhiqiang liu <liuzhiqiang26@huawei.com>
---
tools/testing/selftests/net/route_localnet.sh | 74 +++++++++++++++++++
1 file changed, 74 insertions(+)
create mode 100755 tools/testing/selftests/net/route_localnet.sh
diff --git a/tools/testing/selftests/net/route_localnet.sh b/tools/testing/selftests/net/route_localnet.sh
new file mode 100755
index 000000000000..116bfeab72fa
--- /dev/null
+++ b/tools/testing/selftests/net/route_localnet.sh
@@ -0,0 +1,74 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Run a couple of tests when route_localnet = 1.
+
+readonly PEER_NS="ns-peer-$(mktemp -u XXXXXX)"
+
+setup() {
+ ip netns add "${PEER_NS}"
+ ip -netns "${PEER_NS}" link set dev lo up
+ ip link add name veth0 type veth peer name veth1
+ ip link set dev veth0 up
+ ip link set dev veth1 netns "${PEER_NS}"
+
+ # Enable route_localnet and delete useless route 127.0.0.0/8.
+ sysctl -w net.ipv4.conf.veth0.route_localnet=1
+ ip netns exec "${PEER_NS}" sysctl -w net.ipv4.conf.veth1.route_localnet=1
+ ip route del 127.0.0.0/8 dev lo table local
+ ip netns exec "${PEER_NS}" ip route del 127.0.0.0/8 dev lo table local
+
+ ifconfig veth0 127.25.3.4/24 up
+ ip netns exec "${PEER_NS}" ifconfig veth1 127.25.3.14/24 up
+
+ ip route flush cache
+ ip netns exec "${PEER_NS}" ip route flush cache
+}
+
+cleanup() {
+ ip link del veth0
+ ip route add local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
+ local -r ns="$(ip netns list|grep $PEER_NS)"
+ [ -n "$ns" ] && ip netns del $ns 2>/dev/null
+}
+
+# Run test when arp_announce = 2.
+run_arp_announce_test() {
+ echo "run arp_announce test"
+ setup
+
+ sysctl -w net.ipv4.conf.veth0.arp_announce=2
+ ip netns exec "${PEER_NS}" sysctl -w net.ipv4.conf.veth1.arp_announce=2
+ ping -c5 -I veth0 127.25.3.14
+ if [ $? -ne 0 ];then
+ echo "failed"
+ else
+ echo "ok"
+ fi
+
+ cleanup
+}
+
+# Run test when arp_ignore = 3.
+run_arp_ignore_test() {
+ echo "run arp_ignore test"
+ setup
+
+ sysctl -w net.ipv4.conf.veth0.arp_ignore=3
+ ip netns exec "${PEER_NS}" sysctl -w net.ipv4.conf.veth1.arp_ignore=3
+ ping -c5 -I veth0 127.25.3.14
+ if [ $? -ne 0 ];then
+ echo "failed"
+ else
+ echo "ok"
+ fi
+
+ cleanup
+}
+
+run_all_tests() {
+ run_arp_announce_test
+ run_arp_ignore_test
+}
+
+run_all_tests
--
2.19.1
^ permalink raw reply related
* [PATCH v2 1/3] ipv4: fix inet_select_addr() when enable route_localnet
From: luoshijie @ 2019-06-18 15:14 UTC (permalink / raw)
To: davem, tgraf, dsahern
Cc: netdev, liuzhiqiang26, wangxiaogang3, mingfangsen, zhoukang7
In-Reply-To: <1560870845-172395-1-git-send-email-luoshijie1@huawei.com>
From: Shijie Luo <luoshijie1@huawei.com>
Suppose we have two interfaces eth0 and eth1 in two hosts, follow
the same steps in the two hosts:
# sysctl -w net.ipv4.conf.eth1.route_localnet=1
# sysctl -w net.ipv4.conf.eth1.arp_announce=2
# ip route del 127.0.0.0/8 dev lo table local
and then set ip to eth1 in host1 like:
# ifconfig eth1 127.25.3.4/24
set ip to eth2 in host2 and ping host1:
# ifconfig eth1 127.25.3.14/24
# ping -I eth1 127.25.3.4
Well, host2 cannot connect to host1.
When set a ip address with head 127, the scope of the address defaults
to RT_SCOPE_HOST. In this situation, host2 will use arp_solicit() to
send a arp request for the mac address of host1 with ip
address 127.25.3.14. When arp_announce=2, inet_select_addr() cannot
select a correct saddr with condition ifa->ifa_scope > scope, because
ifa_scope is RT_SCOPE_HOST and scope is RT_SCOPE_LINK. Then,
inet_select_addr() will go to no_in_dev to lookup all interfaces to find
a primary ip and finally get the primary ip of eth0.
Here I add a localnet_scope defaults to RT_SCOPE_HOST, and when
route_localnet is enabled, this value changes to RT_SCOPE_LINK to make
inet_select_addr() find a correct primary ip as saddr of arp request.
Fixes: d0daebc3d622 ("ipv4: Add interface option to enable routing of 127.0.0.0/8")
Signed-off-by: Shijie Luo <luoshijie1@huawei.com>
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
---
net/ipv4/devinet.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c
index c6bd0f7a020a..08c6c7c41749 100644
--- a/net/ipv4/devinet.c
+++ b/net/ipv4/devinet.c
@@ -1254,6 +1254,7 @@ static __be32 in_dev_select_addr(const struct in_device *in_dev,
__be32 inet_select_addr(const struct net_device *dev, __be32 dst, int scope)
{
__be32 addr = 0;
+ unsigned char localnet_scope = RT_SCOPE_HOST;
struct in_device *in_dev;
struct net *net = dev_net(dev);
int master_idx;
@@ -1263,8 +1264,11 @@ __be32 inet_select_addr(const struct net_device *dev, __be32 dst, int scope)
if (!in_dev)
goto no_in_dev;
+ if (unlikely(IN_DEV_ROUTE_LOCALNET(in_dev)))
+ localnet_scope = RT_SCOPE_LINK;
+
for_primary_ifa(in_dev) {
- if (ifa->ifa_scope > scope)
+ if (min(ifa->ifa_scope, localnet_scope) > scope)
continue;
if (!dst || inet_ifa_match(dst, ifa)) {
addr = ifa->ifa_local;
--
2.19.1
^ permalink raw reply related
* [PATCH v2 2/3] ipv4: fix confirm_addr_indev() when enable route_localnet
From: luoshijie @ 2019-06-18 15:14 UTC (permalink / raw)
To: davem, tgraf, dsahern
Cc: netdev, liuzhiqiang26, wangxiaogang3, mingfangsen, zhoukang7
In-Reply-To: <1560870845-172395-1-git-send-email-luoshijie1@huawei.com>
From: Shijie Luo <luoshijie1@huawei.com>
When arp_ignore=3, the NIC won't reply for scope host addresses, but
if enable route_locanet, we need to reply ip address with head 127 and
scope RT_SCOPE_HOST.
Fixes: d0daebc3d622 ("ipv4: Add interface option to enable routing of 127.0.0.0/8")
Signed-off-by: Shijie Luo <luoshijie1@huawei.com>
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
---
net/ipv4/devinet.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c
index 08c6c7c41749..cfef8df59373 100644
--- a/net/ipv4/devinet.c
+++ b/net/ipv4/devinet.c
@@ -1321,13 +1321,18 @@ EXPORT_SYMBOL(inet_select_addr);
static __be32 confirm_addr_indev(struct in_device *in_dev, __be32 dst,
__be32 local, int scope)
{
+ unsigned char localnet_scope = RT_SCOPE_HOST;
int same = 0;
__be32 addr = 0;
+ if (unlikely(IN_DEV_ROUTE_LOCALNET(in_dev)))
+ localnet_scope = RT_SCOPE_LINK;
+
for_ifa(in_dev) {
+ unsigned char min_scope = min(ifa->ifa_scope, localnet_scope);
if (!addr &&
(local == ifa->ifa_local || !local) &&
- ifa->ifa_scope <= scope) {
+ min_scope <= scope) {
addr = ifa->ifa_local;
if (same)
break;
@@ -1342,7 +1347,7 @@ static __be32 confirm_addr_indev(struct in_device *in_dev, __be32 dst,
if (inet_ifa_match(addr, ifa))
break;
/* No, then can we use new local src? */
- if (ifa->ifa_scope <= scope) {
+ if (min_scope <= scope) {
addr = ifa->ifa_local;
break;
}
--
2.19.1
^ permalink raw reply related
* [PATCH net-next v2 12/12] page_pool: make sure struct device is stable
From: Jesper Dangaard Brouer @ 2019-06-18 13:06 UTC (permalink / raw)
To: netdev, Ilias Apalodimas, Toke Høiland-Jørgensen,
Tariq Toukan, Jesper Dangaard Brouer
Cc: toshiaki.makita1, grygorii.strashko, ivan.khoronzhuk, mcroce
In-Reply-To: <156086304827.27760.11339786046465638081.stgit@firesoul>
For DMA mapping use-case the page_pool keeps a pointer
to the struct device, which is used in DMA map/unmap calls.
For our in-flight handling, we also need to make sure that
the struct device have not disappeared. This is assured
via using get_device/put_device API.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Reported-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
net/core/page_pool.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index f55ab055d543..b366f59885c1 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -8,6 +8,7 @@
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/slab.h>
+#include <linux/device.h>
#include <net/page_pool.h>
#include <linux/dma-direction.h>
@@ -48,6 +49,9 @@ static int page_pool_init(struct page_pool *pool,
atomic_set(&pool->pages_state_release_cnt, 0);
+ if (pool->p.flags & PP_FLAG_DMA_MAP)
+ get_device(pool->p.dev);
+
return 0;
}
@@ -360,6 +364,10 @@ void __page_pool_free(struct page_pool *pool)
__warn_in_flight(pool);
ptr_ring_cleanup(&pool->ring, NULL);
+
+ if (pool->p.flags & PP_FLAG_DMA_MAP)
+ put_device(pool->p.dev);
+
kfree(pool);
}
EXPORT_SYMBOL(__page_pool_free);
^ permalink raw reply related
* [PATCH net-next v2 11/12] page_pool: add tracepoints for page_pool with details need by XDP
From: Jesper Dangaard Brouer @ 2019-06-18 13:06 UTC (permalink / raw)
To: netdev, Ilias Apalodimas, Toke Høiland-Jørgensen,
Tariq Toukan, Jesper Dangaard Brouer
Cc: toshiaki.makita1, grygorii.strashko, ivan.khoronzhuk, mcroce
In-Reply-To: <156086304827.27760.11339786046465638081.stgit@firesoul>
The xdp tracepoints for mem id disconnect don't carry information about, why
it was not safe_to_remove. The tracepoint page_pool:page_pool_inflight in
this patch can be used for extract this info for further debugging.
This patchset also adds tracepoint for the pages_state_* release/hold
transitions, including a pointer to the page. This can be used for stats
about in-flight pages, or used to debug page leakage via keeping track of
page pointer and combining this with kprobe for __put_page().
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
---
include/trace/events/page_pool.h | 87 ++++++++++++++++++++++++++++++++++++++
net/core/net-traces.c | 4 ++
net/core/page_pool.c | 9 +++-
3 files changed, 99 insertions(+), 1 deletion(-)
create mode 100644 include/trace/events/page_pool.h
diff --git a/include/trace/events/page_pool.h b/include/trace/events/page_pool.h
new file mode 100644
index 000000000000..47b5ee880aa9
--- /dev/null
+++ b/include/trace/events/page_pool.h
@@ -0,0 +1,87 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM page_pool
+
+#if !defined(_TRACE_PAGE_POOL_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_PAGE_POOL_H
+
+#include <linux/types.h>
+#include <linux/tracepoint.h>
+
+#include <net/page_pool.h>
+
+TRACE_EVENT(page_pool_inflight,
+
+ TP_PROTO(const struct page_pool *pool,
+ s32 inflight, u32 hold, u32 release),
+
+ TP_ARGS(pool, inflight, hold, release),
+
+ TP_STRUCT__entry(
+ __field(const struct page_pool *, pool)
+ __field(s32, inflight)
+ __field(u32, hold)
+ __field(u32, release)
+ ),
+
+ TP_fast_assign(
+ __entry->pool = pool;
+ __entry->inflight = inflight;
+ __entry->hold = hold;
+ __entry->release = release;
+ ),
+
+ TP_printk("page_pool=%p inflight=%d hold=%u release=%u",
+ __entry->pool, __entry->inflight, __entry->hold, __entry->release)
+);
+
+TRACE_EVENT(page_pool_state_release,
+
+ TP_PROTO(const struct page_pool *pool,
+ const struct page *page, u32 release),
+
+ TP_ARGS(pool, page, release),
+
+ TP_STRUCT__entry(
+ __field(const struct page_pool *, pool)
+ __field(const struct page *, page)
+ __field(u32, release)
+ ),
+
+ TP_fast_assign(
+ __entry->pool = pool;
+ __entry->page = page;
+ __entry->release = release;
+ ),
+
+ TP_printk("page_pool=%p page=%p release=%u",
+ __entry->pool, __entry->page, __entry->release)
+);
+
+TRACE_EVENT(page_pool_state_hold,
+
+ TP_PROTO(const struct page_pool *pool,
+ const struct page *page, u32 hold),
+
+ TP_ARGS(pool, page, hold),
+
+ TP_STRUCT__entry(
+ __field(const struct page_pool *, pool)
+ __field(const struct page *, page)
+ __field(u32, hold)
+ ),
+
+ TP_fast_assign(
+ __entry->pool = pool;
+ __entry->page = page;
+ __entry->hold = hold;
+ ),
+
+ TP_printk("page_pool=%p page=%p hold=%u",
+ __entry->pool, __entry->page, __entry->hold)
+);
+
+#endif /* _TRACE_PAGE_POOL_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/net/core/net-traces.c b/net/core/net-traces.c
index 470b179d599e..283ddb2dbc7d 100644
--- a/net/core/net-traces.c
+++ b/net/core/net-traces.c
@@ -43,6 +43,10 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(fdb_delete);
EXPORT_TRACEPOINT_SYMBOL_GPL(br_fdb_update);
#endif
+#if IS_ENABLED(CONFIG_PAGE_POOL)
+#include <trace/events/page_pool.h>
+#endif
+
#include <trace/events/neigh.h>
EXPORT_TRACEPOINT_SYMBOL_GPL(neigh_update);
EXPORT_TRACEPOINT_SYMBOL_GPL(neigh_update_done);
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 42c3b0a5a259..f55ab055d543 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -4,6 +4,7 @@
* Author: Jesper Dangaard Brouer <netoptimizer@brouer.com>
* Copyright (C) 2016 Red Hat, Inc.
*/
+
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/slab.h>
@@ -14,6 +15,8 @@
#include <linux/page-flags.h>
#include <linux/mm.h> /* for __put_page() */
+#include <trace/events/page_pool.h>
+
static int page_pool_init(struct page_pool *pool,
const struct page_pool_params *params)
{
@@ -156,6 +159,8 @@ static struct page *__page_pool_alloc_pages_slow(struct page_pool *pool,
/* Track how many pages are held 'in-flight' */
pool->pages_state_hold_cnt++;
+ trace_page_pool_state_hold(pool, page, pool->pages_state_hold_cnt);
+
/* When page just alloc'ed is should/must have refcnt 1. */
return page;
}
@@ -191,7 +196,7 @@ static s32 page_pool_inflight(struct page_pool *pool)
distance = _distance(hold_cnt, release_cnt);
- /* TODO: Add tracepoint here */
+ trace_page_pool_inflight(pool, distance, hold_cnt, release_cnt);
return distance;
}
@@ -222,6 +227,8 @@ static void __page_pool_clean_page(struct page_pool *pool,
page->dma_addr = 0;
skip_dma_unmap:
atomic_inc(&pool->pages_state_release_cnt);
+ trace_page_pool_state_release(pool, page,
+ atomic_read(&pool->pages_state_release_cnt));
}
/* unmap the page and clean our state */
^ permalink raw reply related
* [PATCH net-next v2 10/12] xdp: add tracepoints for XDP mem
From: Jesper Dangaard Brouer @ 2019-06-18 13:05 UTC (permalink / raw)
To: netdev, Ilias Apalodimas, Toke Høiland-Jørgensen,
Tariq Toukan, Jesper Dangaard Brouer
Cc: toshiaki.makita1, grygorii.strashko, ivan.khoronzhuk, mcroce
In-Reply-To: <156086304827.27760.11339786046465638081.stgit@firesoul>
These tracepoints make it easier to troubleshoot XDP mem id disconnect.
The xdp:mem_disconnect tracepoint cannot be replaced via kprobe. It is
placed at the last stable place for the pointer to struct xdp_mem_allocator,
just before it's scheduled for RCU removal. It also extract info on
'safe_to_remove' and 'force'.
Detailed info about in-flight pages is not available at this layer. The next
patch will added tracepoints needed at the page_pool layer for this.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
---
include/net/xdp_priv.h | 23 +++++++++
include/trace/events/xdp.h | 115 ++++++++++++++++++++++++++++++++++++++++++++
net/core/xdp.c | 21 ++------
3 files changed, 143 insertions(+), 16 deletions(-)
create mode 100644 include/net/xdp_priv.h
diff --git a/include/net/xdp_priv.h b/include/net/xdp_priv.h
new file mode 100644
index 000000000000..6a8cba6ea79a
--- /dev/null
+++ b/include/net/xdp_priv.h
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LINUX_NET_XDP_PRIV_H__
+#define __LINUX_NET_XDP_PRIV_H__
+
+#include <linux/rhashtable.h>
+
+/* Private to net/core/xdp.c, but used by trace/events/xdp.h */
+struct xdp_mem_allocator {
+ struct xdp_mem_info mem;
+ union {
+ void *allocator;
+ struct page_pool *page_pool;
+ struct zero_copy_allocator *zc_alloc;
+ };
+ int disconnect_cnt;
+ unsigned long defer_start;
+ struct rhash_head node;
+ struct rcu_head rcu;
+ struct delayed_work defer_wq;
+ unsigned long defer_warn;
+};
+
+#endif /* __LINUX_NET_XDP_PRIV_H__ */
diff --git a/include/trace/events/xdp.h b/include/trace/events/xdp.h
index e95cb86b65cf..bb5e380e2ef3 100644
--- a/include/trace/events/xdp.h
+++ b/include/trace/events/xdp.h
@@ -269,6 +269,121 @@ TRACE_EVENT(xdp_devmap_xmit,
__entry->from_ifindex, __entry->to_ifindex, __entry->err)
);
+/* Expect users already include <net/xdp.h>, but not xdp_priv.h */
+#include <net/xdp_priv.h>
+
+#define __MEM_TYPE_MAP(FN) \
+ FN(PAGE_SHARED) \
+ FN(PAGE_ORDER0) \
+ FN(PAGE_POOL) \
+ FN(ZERO_COPY)
+
+#define __MEM_TYPE_TP_FN(x) \
+ TRACE_DEFINE_ENUM(MEM_TYPE_##x);
+#define __MEM_TYPE_SYM_FN(x) \
+ { MEM_TYPE_##x, #x },
+#define __MEM_TYPE_SYM_TAB \
+ __MEM_TYPE_MAP(__MEM_TYPE_SYM_FN) { -1, 0 }
+__MEM_TYPE_MAP(__MEM_TYPE_TP_FN)
+
+TRACE_EVENT(mem_disconnect,
+
+ TP_PROTO(const struct xdp_mem_allocator *xa,
+ bool safe_to_remove, bool force),
+
+ TP_ARGS(xa, safe_to_remove, force),
+
+ TP_STRUCT__entry(
+ __field(const struct xdp_mem_allocator *, xa)
+ __field(u32, mem_id)
+ __field(u32, mem_type)
+ __field(const void *, allocator)
+ __field(bool, safe_to_remove)
+ __field(bool, force)
+ __field(int, disconnect_cnt)
+ ),
+
+ TP_fast_assign(
+ __entry->xa = xa;
+ __entry->mem_id = xa->mem.id;
+ __entry->mem_type = xa->mem.type;
+ __entry->allocator = xa->allocator;
+ __entry->safe_to_remove = safe_to_remove;
+ __entry->force = force;
+ __entry->disconnect_cnt = xa->disconnect_cnt;
+ ),
+
+ TP_printk("mem_id=%d mem_type=%s allocator=%p"
+ " safe_to_remove=%s force=%s disconnect_cnt=%d",
+ __entry->mem_id,
+ __print_symbolic(__entry->mem_type, __MEM_TYPE_SYM_TAB),
+ __entry->allocator,
+ __entry->safe_to_remove ? "true" : "false",
+ __entry->force ? "true" : "false",
+ __entry->disconnect_cnt
+ )
+);
+
+TRACE_EVENT(mem_connect,
+
+ TP_PROTO(const struct xdp_mem_allocator *xa,
+ const struct xdp_rxq_info *rxq),
+
+ TP_ARGS(xa, rxq),
+
+ TP_STRUCT__entry(
+ __field(const struct xdp_mem_allocator *, xa)
+ __field(u32, mem_id)
+ __field(u32, mem_type)
+ __field(const void *, allocator)
+ __field(const struct xdp_rxq_info *, rxq)
+ __field(int, ifindex)
+ ),
+
+ TP_fast_assign(
+ __entry->xa = xa;
+ __entry->mem_id = xa->mem.id;
+ __entry->mem_type = xa->mem.type;
+ __entry->allocator = xa->allocator;
+ __entry->rxq = rxq;
+ __entry->ifindex = rxq->dev->ifindex;
+ ),
+
+ TP_printk("mem_id=%d mem_type=%s allocator=%p"
+ " ifindex=%d",
+ __entry->mem_id,
+ __print_symbolic(__entry->mem_type, __MEM_TYPE_SYM_TAB),
+ __entry->allocator,
+ __entry->ifindex
+ )
+);
+
+TRACE_EVENT(mem_return_failed,
+
+ TP_PROTO(const struct xdp_mem_info *mem,
+ const struct page *page),
+
+ TP_ARGS(mem, page),
+
+ TP_STRUCT__entry(
+ __field(const struct page *, page)
+ __field(u32, mem_id)
+ __field(u32, mem_type)
+ ),
+
+ TP_fast_assign(
+ __entry->page = page;
+ __entry->mem_id = mem->id;
+ __entry->mem_type = mem->type;
+ ),
+
+ TP_printk("mem_id=%d mem_type=%s page=%p",
+ __entry->mem_id,
+ __print_symbolic(__entry->mem_type, __MEM_TYPE_SYM_TAB),
+ __entry->page
+ )
+);
+
#endif /* _TRACE_XDP_H */
#include <trace/define_trace.h>
diff --git a/net/core/xdp.c b/net/core/xdp.c
index 53bce4fa776a..4ad9e48b76a8 100644
--- a/net/core/xdp.c
+++ b/net/core/xdp.c
@@ -14,6 +14,8 @@
#include <net/page_pool.h>
#include <net/xdp.h>
+#include <net/xdp_priv.h> /* struct xdp_mem_allocator */
+#include <trace/events/xdp.h>
#define REG_STATE_NEW 0x0
#define REG_STATE_REGISTERED 0x1
@@ -29,21 +31,6 @@ static int mem_id_next = MEM_ID_MIN;
static bool mem_id_init; /* false */
static struct rhashtable *mem_id_ht;
-struct xdp_mem_allocator {
- struct xdp_mem_info mem;
- union {
- void *allocator;
- struct page_pool *page_pool;
- struct zero_copy_allocator *zc_alloc;
- };
- struct rhash_head node;
- struct rcu_head rcu;
- struct delayed_work defer_wq;
- unsigned long defer_start;
- unsigned long defer_warn;
- int disconnect_cnt;
-};
-
static u32 xdp_mem_id_hashfn(const void *data, u32 len, u32 seed)
{
const u32 *k = data;
@@ -117,7 +104,7 @@ bool __mem_id_disconnect(int id, bool force)
if (xa->mem.type == MEM_TYPE_PAGE_POOL)
safe_to_remove = page_pool_request_shutdown(xa->page_pool);
- /* TODO: Tracepoint will be added here in next-patch */
+ trace_mem_disconnect(xa, safe_to_remove, force);
if ((safe_to_remove || force) &&
!rhashtable_remove_fast(mem_id_ht, &xa->node, mem_id_rht_params))
@@ -385,6 +372,7 @@ int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq,
mutex_unlock(&mem_id_lock);
+ trace_mem_connect(xdp_alloc, xdp_rxq);
return 0;
err:
mutex_unlock(&mem_id_lock);
@@ -417,6 +405,7 @@ static void __xdp_return(void *data, struct xdp_mem_info *mem, bool napi_direct,
} else {
/* Hopefully stack show who to blame for late return */
WARN_ONCE(1, "page_pool gone mem.id=%d", mem->id);
+ trace_mem_return_failed(mem, page);
put_page(page);
}
rcu_read_unlock();
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox