* Re: [net-next V2 PATCH 1/5] bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP
From: Jakub Kicinski @ 2017-09-29 18:41 UTC (permalink / raw)
To: Jesper Dangaard Brouer
Cc: netdev, Michael S. Tsirkin, Jason Wang, mchan, John Fastabend,
peter.waskiewicz.jr, Daniel Borkmann, Alexei Starovoitov,
Andy Gospodarek
In-Reply-To: <150670285218.23765.2480801081343646072.stgit@firesoul>
On Fri, 29 Sep 2017 18:34:12 +0200, Jesper Dangaard Brouer wrote:
> The 'cpumap' is primary used as a backend map for XDP BPF helper
> call bpf_redirect_map() and XDP_REDIRECT action, like 'devmap'.
>
> This patch implement the main part of the map. It is not connected to
> the XDP redirect system yet, and no SKB allocation are done yet.
>
> The main concern in this patch is to ensure the datapath can run
> without any locking. This adds complexity to the setup and tear-down
> procedure, which assumptions are extra carefully documented in the
> code comments.
>
> V2: make sure array isn't larger than num possible CPUs
>
> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Few trivial nitpicks, hope you don't mind :)
> @@ -0,0 +1,555 @@
> +/* bpf/cpumap.c
> + *
> + * Copyright (c) 2017 Jesper Dangaard Brouer, Red Hat Inc.
> + * Released under terms in GPL version 2. See COPYING.
> + */
> +
> +/* The 'cpumap' is primary used as a backend map for XDP BPF helper
> + * call bpf_redirect_map() and XDP_REDIRECT action, like 'devmap'.
> + *
> + * Unlike devmap which redirect XDP frames out another NIC device,
> + * this map type redirect raw XDP frames to another CPU. The remote
> + * CPU will do SKB-allocation and call the normal network stack.
> + *
> + * This is a scalability and isolation mechanism, that allow
> + * separating the early driver network XDP layer, from the rest of the
> + * netstack, and assigning dedicated CPUs for this stage. This
> + * basically allows for 10G wirespeed pre-filtering via bpf.
> + */
> +#include <linux/bpf.h>
> +#include <linux/filter.h>
> +#include <linux/ptr_ring.h>
> +
> +#include <linux/sched.h>
> +#include <linux/workqueue.h>
> +#include <linux/kthread.h>
> +
> +/*
> + * General idea: XDP packets getting XDP redirected to another CPU,
> + * will maximum be stored/queued for one driver ->poll() call. It is
> + * guaranteed that setting flush bit and flush operation happen on
> + * same CPU. Thus, cpu_map_flush operation can deduct via this_cpu_ptr()
> + * which queue in bpf_cpu_map_entry contains packets.
> + */
> +
> +#define CPU_MAP_BULK_SIZE 8 /* 8 == one cacheline on 64-bit archs */
> +struct xdp_bulk_queue {
> + void *q[CPU_MAP_BULK_SIZE];
> + unsigned int count;
> +};
Out of curiosity - would it make sense to make sure the entire struct
fits into a cache line? The comment seems to indicate that the array is
sized to fit a cache line, but then there is also the count member...
> +/*
> + * After xchg pointer to bpf_cpu_map_entry, use the call_rcu() to
...
There is a mix for networking and non-networking style comments in this
file, is this intentional?
> +const struct bpf_map_ops cpu_map_ops = {
> + .map_alloc = cpu_map_alloc,
> + .map_free = cpu_map_free,
> + .map_delete_elem = cpu_map_delete_elem,
> + .map_update_elem = cpu_map_update_elem,
> + .map_lookup_elem = cpu_map_lookup_elem,
> + .map_get_next_key = cpu_map_get_next_key,
> +};
> +
> +
Extra new line.
> +/* Runs under RCU-read-side, plus in softirq under NAPI protection.
> + * Thus, safe percpu variable access.
> + */
> +static int bq_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_pkt *xdp_pkt)
> +{
> + struct xdp_bulk_queue *bq = this_cpu_ptr(rcpu->bulkq);
> +
> + if (unlikely(bq->count == CPU_MAP_BULK_SIZE)) {
> + bq_flush_to_queue(rcpu, bq);
> + }
Curly brackets not needed.
^ permalink raw reply
* RE: [PATCH RFC 3/5] Add KSZ8795 switch driver
From: Tristram.Ha @ 2017-09-29 18:45 UTC (permalink / raw)
To: pavel
Cc: andrew, muvarov, nathan.leigh.conrad, vivien.didelot, f.fainelli,
netdev, linux-kernel, Woojung.Huh
In-Reply-To: <20170928152445.GC2482@amd>
> > > > > Similar code will be needed by other drivers, right?
> > > >
> > > > Although KSZ8795 and KSZ8895 may use the same code, the other
> > > > chips will have different code.
> > >
> > > Ok, please make sure code is shared between these two.
> >
> > The exact function probably cannot be shared between KSZ8795
> > and KSZ8895 drivers because although the register constants look
> > the same but their exact locations are different in the 2 chips.
>
> Put the code into header as static inline and include it from both
> places?
>
Although KSZ8795 and KSZ8895 share the same code when simulating
the PHY register access, even though the exact registers are not the
same, this code needs a little modification for another chip. It also looks
too large to put in a header.
^ permalink raw reply
* Re: [PATCH RFC 3/5] Add KSZ8795 switch driver
From: Andrew Lunn @ 2017-09-29 18:53 UTC (permalink / raw)
To: Tristram.Ha
Cc: David.Laight, muvarov, pavel, nathan.leigh.conrad, vivien.didelot,
f.fainelli, netdev, linux-kernel, Woojung.Huh
In-Reply-To: <93AF473E2DA327428DE3D46B72B1E9FD4112CB89@CHN-SV-EXMX02.mchp-main.com>
> My concern is if a task is already running with SPI access to a lot
> of registers like reading the 32 MIB counters in every port of the
> switch, another register access has to wait until they are finished.
Why does it have to wait? Looking at the code in
ksz_get_ethtool_stats(), you don't take any mutex which will prevent
others from using the SPI bus. All there is is a mutex which prevents
two sets of ksz_get_ethtool_stats() at the same time.
So a PTP read could happen in parallel, and will not be blocked by MIB
reads. They should get interleaved access to the SPI bus.
Andrew
^ permalink raw reply
* RE: [PATCH RFC 3/5] Add KSZ8795 switch driver
From: Tristram.Ha @ 2017-09-29 18:56 UTC (permalink / raw)
To: pavel
Cc: andrew, muvarov, nathan.leigh.conrad, vivien.didelot, f.fainelli,
netdev, linux-kernel, Woojung.Huh
In-Reply-To: <20170928184059.GA2825@amd>
> On Mon 2017-09-18 20:27:13, Tristram.Ha@microchip.com wrote:
> > > > +/**
> > > > + * Some counters do not need to be read too often because they are
> less
> > > likely
> > > > + * to increase much.
> > > > + */
> > >
> > > What does comment mean? Are you caching statistics, and updating
> > > different values at different rates?
> > >
> >
> > There are 34 counters. In normal case using generic bus I/O or PCI to read
> them
> > is very quick, but the switch is mostly accessed using SPI, or even I2C. As
> the SPI
> > access is very slow and cannot run in interrupt context I keep worrying
> reading
> > the MIB counters in a loop for 5 or more ports will prevent other critical
> hardware
> > access from executing soon enough. These accesses can be getting 1588
> PTP
> > timestamps and opening/closing ports. (RSTP Conformance Test sends test
> traffic
> > to port supposed to be closed/opened after receiving specific RSTP
> > BPDU.)
>
> Hmm. Ok, interesting.
>
> I wonder how well this is going to work if userspace actively 'does
> something' with the switch.
>
> It seems to me that even if your statistics code is careful not to do
> 'a lot' of accesses at the same time, userspace can use other parts of
> the driver to do the same, and thus cause same unwanted effects...
If the user calls "ethtool -S" in a tight loop the system will waste a lot of
CPU time, but this is more like a user error.
Another solution is not to schedule to read the MIB counters in that
function call. I think I was doing a favor to update the MIB counters
sooner as the user probably wants to find out what is wrong with the
switch by reading the MIB counters and checking them several times.
For system tracking like SNMP I think it is likely a separate mechanism
is used to gather those information. If I am wrong that function definitely
needs to be modified.
^ permalink raw reply
* Re: [PATCH v2] lib: fix multiple strlcpy definition
From: Stephen Hemminger @ 2017-09-29 18:58 UTC (permalink / raw)
To: Baruch Siach; +Cc: netdev, Phil Sutter
In-Reply-To: <6910008ec8db3f63c8120b1624a08328cd203e92.1506621731.git.baruch@tkos.co.il>
On Thu, 28 Sep 2017 21:02:11 +0300
Baruch Siach <baruch@tkos.co.il> wrote:
> Some C libraries, like uClibc and musl, provide BSD compatible
> strlcpy(). Add check_strlcpy() to configure, and avoid defining strlcpy
> and strlcat when the C library provides them.
>
> This fixes the following static link error with uClibc-ng:
>
> .../sysroot/usr/lib/libc.a(strlcpy.os): In function `strlcpy':
> strlcpy.c:(.text+0x0): multiple definition of `strlcpy'
> ../lib/libutil.a(utils.o):utils.c:(.text+0x1ddc): first defined here
> collect2: error: ld returned 1 exit status
>
> Acked-by: Phil Sutter <phil@nwl.cc>
> Signed-off-by: Baruch Siach <baruch@tkos.co.il>
This is OK because it doesn't impact normal glibc too much.
> diff --git a/lib/Makefile b/lib/Makefile
> index 0fbdf4c31f50..132ad00c3335 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -1,5 +1,9 @@
> include ../config.mk
>
> +ifeq ($(NEED_STRLCPY),y)
> + CFLAGS += -DNEED_STRLCPY
> +endif
> +
>
I just removed all the conditional CFLAGS out of subdirectory Makefiles
and moved them into the generated config.mk. Please do that for this
as well and resubmit.
^ permalink raw reply
* Re: [PATCH net-next 2/8] net: dsa: directly fetch switch in lan9303_rcv
From: Florian Fainelli @ 2017-09-29 19:02 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-3-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> In a single-chip switch fabric, there is no need to fetch the dsa_switch
> structure from the tree, directly use the CPU port's "ds" member.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
> ---
> net/dsa/tag_lan9303.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/net/dsa/tag_lan9303.c b/net/dsa/tag_lan9303.c
> index 0b9826105e42..f0b51acf36ac 100644
> --- a/net/dsa/tag_lan9303.c
> +++ b/net/dsa/tag_lan9303.c
> @@ -72,11 +72,10 @@ static struct sk_buff *lan9303_rcv(struct sk_buff *skb, struct net_device *dev,
> {
> u16 *lan9303_tag;
> struct dsa_switch_tree *dst = dev->dsa_ptr;
> - struct dsa_switch *ds;
> + struct dsa_port *cpu_dp = dst->cpu_dp;
> + struct dsa_switch *ds = cpu_dp->ds;
> unsigned int source_port;
>
> - ds = dst->ds[0];
> -
> if (unlikely(!ds)) {
> dev_warn_ratelimited(&dev->dev, "Dropping packet, due to missing DSA switch device\n");
> return NULL;
Similarly to the patch before, you can probably drop this test, I don't
actually see how we could trigger it.
--
Florian
^ permalink raw reply
* Re: [iproute PATCH] ip-route: Fix for listing routes with RTAX_LOCK attribute
From: Stephen Hemminger @ 2017-09-29 19:02 UTC (permalink / raw)
To: Phil Sutter; +Cc: netdev, Thomas Haller, Hangbin Liu
In-Reply-To: <20170928173356.17892-1-phil@nwl.cc>
On Thu, 28 Sep 2017 19:33:56 +0200
Phil Sutter <phil@nwl.cc> wrote:
> This fixes a corner-case for routes with a certain metric locked to
> zero:
>
> | ip route add 192.168.7.0/24 dev eth0 window 0
> | ip route add 192.168.7.0/24 dev eth0 window lock 0
>
> Since the kernel doesn't dump the attribute if it is zero, both routes
> added above would appear as if they were equal although they are not.
>
> Fix this by taking mxlock value for the given metric into account before
> skipping it if it is not present.
>
> Reported-by: Thomas Haller <thaller@redhat.com>
> Signed-off-by: Phil Sutter <phil@nwl.cc>
Looks good, Applied. Thanks Phil
^ permalink raw reply
* Re: [PATCH net-next 2/8] net: dsa: directly fetch switch in lan9303_rcv
From: Vivien Didelot @ 2017-09-29 19:04 UTC (permalink / raw)
To: Florian Fainelli, netdev
Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <e1fcff41-6ee7-e125-c7eb-98f6ba00a62d@gmail.com>
Hi Florian,
Florian Fainelli <f.fainelli@gmail.com> writes:
>> @@ -72,11 +72,10 @@ static struct sk_buff *lan9303_rcv(struct sk_buff *skb, struct net_device *dev,
>> {
>> u16 *lan9303_tag;
>> struct dsa_switch_tree *dst = dev->dsa_ptr;
>> - struct dsa_switch *ds;
>> + struct dsa_port *cpu_dp = dst->cpu_dp;
>> + struct dsa_switch *ds = cpu_dp->ds;
>> unsigned int source_port;
>>
>> - ds = dst->ds[0];
>> -
>> if (unlikely(!ds)) {
>> dev_warn_ratelimited(&dev->dev, "Dropping packet, due to missing DSA switch device\n");
>> return NULL;
>
> Similarly to the patch before, you can probably drop this test, I don't
> actually see how we could trigger it.
Good catch, I'll respin after eventually more reviews.
Thanks,
Vivien
^ permalink raw reply
* Re: [PATCH net-next 3/8] net: dsa: use cpu_dp in master code
From: Florian Fainelli @ 2017-09-29 19:10 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-4-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> Make it clear that the master device is linked to a CPU port by using
> "cpu_dp" for the dsa_port variable in master.c instead of "port", then
> use a "port" variable to describe the port index, as usually seen in
> other places of DSA core.
>
> This will make the future patch touching dsa_ptr more readable. There is
> no functional changes.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* Re: [PATCH net-next 4/8] net: dsa: use temporary dsa_device_ops variable
From: Florian Fainelli @ 2017-09-29 19:11 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-5-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> When resolving the DSA tagging protocol used by a CPU switch, use a
> temporary "tag_ops" variable to store the dsa_device_ops instead of
> using directly dst->tag_ops. This will make the future patches moving
> this pointer around easier to read.
>
> There is no functional changes.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* Re: [PATCH net-next 5/8] net: dsa: add tagging ops to port
From: Florian Fainelli @ 2017-09-29 19:12 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-6-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> The DSA tagging protocol operations are specific to each CPU port,
> thus the dsa_device_ops pointer belongs to the dsa_port structure.
>
> From now on assign a slave's xmit copy from its CPU port tagging
> operations. This will ease the future support for multiple CPU ports.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* [PATCH RFC] flow_dissector: Add FLOW_DISSECTOR_F_FLOWER
From: Tom Herbert @ 2017-09-29 19:13 UTC (permalink / raw)
To: davem; +Cc: hannes, netdev, rohit, Tom Herbert
This patch is RFC and would be applied after "flow_dissector:
Protocol specific flow dissector offload"
In order to maitain uAPI in flower, the FLOW_DISSECTOR_F_FLOWER flag
is added to indicate to flow_dissector that the caller is flower.
As new funtionality is addes to flow_dissector that would break
the flower uAPI, the code can be wrapped in "if (!(flags &
FLOW_DISSECTOR_F_FLOWER)).
In this patch the conditional is use around protocol specific
dissection (e.g. DPI into VXLAN) as well as the code that
enforces a depth of parsing to prevent DPI. The latter was a
recent patch that would introduce a parsing limit to flower that
did not exist before (i.e. would break uAPI).
Signed-off-by: Tom Herbert <tom@quantonium.net>
---
include/net/flow_dissector.h | 1 +
net/core/flow_dissector.c | 17 +++++++++++------
net/sched/cls_flow.c | 3 ++-
3 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/include/net/flow_dissector.h b/include/net/flow_dissector.h
index ad75bbfd1c9c..ca315107d147 100644
--- a/include/net/flow_dissector.h
+++ b/include/net/flow_dissector.h
@@ -214,6 +214,7 @@ enum flow_dissector_key_id {
#define FLOW_DISSECTOR_F_STOP_AT_FLOW_LABEL BIT(2)
#define FLOW_DISSECTOR_F_STOP_AT_ENCAP BIT(3)
#define FLOW_DISSECTOR_F_STOP_AT_L4 BIT(4)
+#define FLOW_DISSECTOR_F_FLOWER BIT(5)
struct flow_dissector_key {
enum flow_dissector_key_id key_id;
diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c
index 84b8eb1f6664..a4d5843ec34e 100644
--- a/net/core/flow_dissector.c
+++ b/net/core/flow_dissector.c
@@ -383,8 +383,11 @@ __skb_flow_dissect_ipv6(const struct sk_buff *skb,
*/
#define MAX_FLOW_DISSECT_HDRS 15
-static bool skb_flow_dissect_allowed(int *num_hdrs)
+static bool skb_flow_dissect_allowed(int *num_hdrs, unsigned int flags)
{
+ if (flags & FLOW_DISSECTOR_F_FLOWER)
+ return true;
+
++*num_hdrs;
return (*num_hdrs <= MAX_FLOW_DISSECT_HDRS);
@@ -722,7 +725,8 @@ bool __skb_flow_dissect(struct sk_buff *skb,
break;
default:
- fdret = flow_dissect_by_type(skb, proto, key_control,
+ if (!(flags & FLOW_DISSECTOR_F_FLOWER))
+ fdret = flow_dissect_by_type(skb, proto, key_control,
flow_dissector,
target_container,
data, &proto, &ip_proto, &nhoff,
@@ -735,7 +739,7 @@ bool __skb_flow_dissect(struct sk_buff *skb,
case FLOW_DISSECT_RET_OUT_GOOD:
goto out_good;
case FLOW_DISSECT_RET_PROTO_AGAIN:
- if (skb_flow_dissect_allowed(&num_hdrs))
+ if (skb_flow_dissect_allowed(&num_hdrs, flags))
goto proto_again;
goto out_good;
case FLOW_DISSECT_RET_CONTINUE:
@@ -843,7 +847,8 @@ bool __skb_flow_dissect(struct sk_buff *skb,
break;
default:
- fdret = flow_dissect_by_type_proto(skb, proto,
+ if (!(flags & FLOW_DISSECTOR_F_FLOWER))
+ fdret = flow_dissect_by_type_proto(skb, proto,
ip_proto, key_control,
flow_dissector,
target_container,
@@ -872,11 +877,11 @@ bool __skb_flow_dissect(struct sk_buff *skb,
/* Process result of IP proto processing */
switch (fdret) {
case FLOW_DISSECT_RET_PROTO_AGAIN:
- if (skb_flow_dissect_allowed(&num_hdrs))
+ if (skb_flow_dissect_allowed(&num_hdrs, flags))
goto proto_again;
break;
case FLOW_DISSECT_RET_IPPROTO_AGAIN:
- if (skb_flow_dissect_allowed(&num_hdrs))
+ if (skb_flow_dissect_allowed(&num_hdrs, flags))
goto ip_proto_again;
break;
case FLOW_DISSECT_RET_OUT_GOOD:
diff --git a/net/sched/cls_flow.c b/net/sched/cls_flow.c
index 2a3a60ec5b86..ad4ac503697e 100644
--- a/net/sched/cls_flow.c
+++ b/net/sched/cls_flow.c
@@ -315,7 +315,8 @@ static int flow_classify(struct sk_buff *skb, const struct tcf_proto *tp,
keymask = f->keymask;
if (keymask & FLOW_KEYS_NEEDED)
- skb_flow_dissect_flow_keys(skb, &flow_keys, 0);
+ skb_flow_dissect_flow_keys(skb, &flow_keys,
+ FLOW_DISSECTOR_F_FLOWER);
for (n = 0; n < f->nkeys; n++) {
key = ffs(keymask) - 1;
--
2.11.0
^ permalink raw reply related
* Re: [PATCH net-next 6/8] net: dsa: prepare master receive hot path
From: Florian Fainelli @ 2017-09-29 19:14 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-7-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> In preparation to make DSA master devices point to their corresponding
> CPU port instead of the whole tree, add copies of dst and rcv in the
> dsa_port structure so that we keep fast access in the receive hot path.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* RE: [PATCH RFC 3/5] Add KSZ8795 switch driver
From: Tristram.Ha @ 2017-09-29 19:19 UTC (permalink / raw)
To: andrew
Cc: David.Laight, muvarov, pavel, nathan.leigh.conrad, vivien.didelot,
f.fainelli, netdev, linux-kernel, Woojung.Huh
In-Reply-To: <20170929185316.GB17713@lunn.ch>
> > My concern is if a task is already running with SPI access to a lot
> > of registers like reading the 32 MIB counters in every port of the
> > switch, another register access has to wait until they are finished.
>
> Why does it have to wait? Looking at the code in
> ksz_get_ethtool_stats(), you don't take any mutex which will prevent
> others from using the SPI bus. All there is is a mutex which prevents
> two sets of ksz_get_ethtool_stats() at the same time.
>
> So a PTP read could happen in parallel, and will not be blocked by MIB
> reads. They should get interleaved access to the SPI bus.
>
The MIB counters are read in the background. For multiple CPU cores 2
tasks may run in the same time allowing SPI access one after another.
For single core I am not sure an SPI access like coming from an interrupt
routine can jump ahead from one in a background task.
I know nowadays SoCs are powerful enough to do amazing things. It is
just I spent a long time using a low-powered SoC doing switch driver
development.
^ permalink raw reply
* Re: [PATCH net-next 7/8] net: dsa: change dsa_ptr for a dsa_port
From: Florian Fainelli @ 2017-09-29 19:20 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-8-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> With DSA, a master net device (CPU facing interface) has a dsa_ptr
> pointer to which hangs a dsa_switch_tree. This is not correct because a
> master interface is wired to a dedicated switch port, and because we can
> theoretically have several master interfaces pointing to several CPU
> ports of the same switch fabric.
>
> Change the master interface's dsa_ptr for the CPU dsa_port pointer.
> This is a step towards supporting multiple CPU ports. Also remove
> dsa_get_cpu_port which is now unneeded.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* Re: [PATCH net-next 8/8] net: dsa: remove tag ops from the switch tree
From: Florian Fainelli @ 2017-09-29 19:20 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-9-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> Now that the dsa_ptr is a dsa_port instance, there is no need to keep
> the tag operations in the dsa_switch_tree structure. Remove it.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* Re: [PATCH net-next 1/8] net: dsa: directly fetch switch in mtk_tag_rcv
From: Florian Fainelli @ 2017-09-29 19:21 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-2-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> In a single-chip switch fabric, there is no need to fetch the dsa_switch
> structure from the tree, directly use the CPU port's "ds" member.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Florian Fainelli <f.fainelli@gmail.com>
--
Florian
^ permalink raw reply
* Re: [PATCH net-next 5/8] net: dsa: add tagging ops to port
From: Florian Fainelli @ 2017-09-29 19:24 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-6-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> The DSA tagging protocol operations are specific to each CPU port,
> thus the dsa_device_ops pointer belongs to the dsa_port structure.
>
> From now on assign a slave's xmit copy from its CPU port tagging
> operations. This will ease the future support for multiple CPU ports.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
> ---
> include/net/dsa.h | 3 +++
> net/dsa/dsa2.c | 1 +
> net/dsa/dsa_priv.h | 2 +-
> net/dsa/legacy.c | 1 +
> net/dsa/slave.c | 3 +--
> 5 files changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/include/net/dsa.h b/include/net/dsa.h
> index 8dee216a5a9b..6cd36dcb65e1 100644
> --- a/include/net/dsa.h
> +++ b/include/net/dsa.h
> @@ -189,6 +189,9 @@ struct dsa_port {
> * Original copy of the master netdev ethtool_ops
> */
> const struct ethtool_ops *orig_ethtool_ops;
> +
> + /* CPU port tagging operations used by master or slave devices */
> + const struct dsa_device_ops *tag_ops;
You might actually want to move this up in the dsa_port structure in
order to keep being in the first cacheline (you can use pahole -C
dsa_port vmlinux).
dsa_switch_tree is currently a 56 bytes structure, thus fitting in a 64b
cache line, but dsa_port is 80bytes, and the hot-path are in the second
cacheline, so less efficient.
--
Florian
^ permalink raw reply
* Re: [PATCH net-next 0/8] net: dsa: change dsa_ptr for a dsa_port
From: Florian Fainelli @ 2017-09-29 19:26 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170929183635.8122-1-vivien.didelot@savoirfairelinux.com>
On 09/29/2017 11:36 AM, Vivien Didelot wrote:
> With DSA, a master net_device is physically wired to a dedicated CPU
> switch port. For interaction with the DSA layer, the struct net_device
> contains a dsa_ptr, which currently points to a dsa_switch_tree object.
>
> This is only valid for a switch fabric with a single CPU port. In order
> to support switch fabrics with multiple CPU ports, we first need to
> change the type of dsa_ptr to what it really is: a dsa_port object.
>
> This is what this patchset does. The first 4 patches cleans up portions
> of DSA core to make the next patches more readable. These next patches
> prepare the xmit and receive hot paths and finally change dsa_ptr.
This looks nice and clean, as mentioned in patch 5, there may be room
for organizing the structure a bit more efficiently such that everything
still fits within the first cacheline .
>
> Vivien Didelot (8):
> net: dsa: directly fetch switch in mtk_tag_rcv
> net: dsa: directly fetch switch in lan9303_rcv
> net: dsa: use cpu_dp in master code
> net: dsa: use temporary dsa_device_ops variable
> net: dsa: add tagging ops to port
> net: dsa: prepare master receive hot path
> net: dsa: change dsa_ptr for a dsa_port
> net: dsa: remove tag ops from the switch tree
>
> include/linux/netdevice.h | 4 ++--
> include/net/dsa.h | 19 ++++++++-----------
> net/dsa/dsa.c | 6 +++---
> net/dsa/dsa2.c | 15 ++++++++++-----
> net/dsa/dsa_priv.h | 7 +------
> net/dsa/legacy.c | 15 ++++++++++-----
> net/dsa/master.c | 47 ++++++++++++++++++++++-------------------------
> net/dsa/slave.c | 3 +--
> net/dsa/tag_brcm.c | 3 +--
> net/dsa/tag_dsa.c | 3 ++-
> net/dsa/tag_edsa.c | 3 ++-
> net/dsa/tag_ksz.c | 3 +--
> net/dsa/tag_lan9303.c | 6 ++----
> net/dsa/tag_mtk.c | 12 ++----------
> net/dsa/tag_qca.c | 3 +--
> net/dsa/tag_trailer.c | 3 +--
> 16 files changed, 69 insertions(+), 83 deletions(-)
>
--
Florian
^ permalink raw reply
* Re: [PATCH net-next 0/8] net: dsa: change dsa_ptr for a dsa_port
From: Vivien Didelot @ 2017-09-29 19:34 UTC (permalink / raw)
To: Florian Fainelli, netdev
Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <03cef84e-546d-d8c2-85fb-afc697ced2cb@gmail.com>
Hi Florian,
Florian Fainelli <f.fainelli@gmail.com> writes:
> On 09/29/2017 11:36 AM, Vivien Didelot wrote:
>> With DSA, a master net_device is physically wired to a dedicated CPU
>> switch port. For interaction with the DSA layer, the struct net_device
>> contains a dsa_ptr, which currently points to a dsa_switch_tree object.
>>
>> This is only valid for a switch fabric with a single CPU port. In order
>> to support switch fabrics with multiple CPU ports, we first need to
>> change the type of dsa_ptr to what it really is: a dsa_port object.
>>
>> This is what this patchset does. The first 4 patches cleans up portions
>> of DSA core to make the next patches more readable. These next patches
>> prepare the xmit and receive hot paths and finally change dsa_ptr.
>
> This looks nice and clean, as mentioned in patch 5, there may be room
> for organizing the structure a bit more efficiently such that everything
> still fits within the first cacheline .
Thanks for this very constructive comment! I'll look into this.
Respinning in a few.
Vivien
^ permalink raw reply
* Re: [PATCH net-next] vhost_net: do not stall on zerocopy depletion
From: Michael S. Tsirkin @ 2017-09-29 19:38 UTC (permalink / raw)
To: Willem de Bruijn
Cc: netdev, davem, jasowang, den, virtualization, Willem de Bruijn
In-Reply-To: <20170928002556.41240-1-willemdebruijn.kernel@gmail.com>
On Wed, Sep 27, 2017 at 08:25:56PM -0400, Willem de Bruijn wrote:
> From: Willem de Bruijn <willemb@google.com>
>
> Vhost-net has a hard limit on the number of zerocopy skbs in flight.
> When reached, transmission stalls. Stalls cause latency, as well as
> head-of-line blocking of other flows that do not use zerocopy.
>
> Instead of stalling, revert to copy-based transmission.
>
> Tested by sending two udp flows from guest to host, one with payload
> of VHOST_GOODCOPY_LEN, the other too small for zerocopy (1B). The
> large flow is redirected to a netem instance with 1MBps rate limit
> and deep 1000 entry queue.
>
> modprobe ifb
> ip link set dev ifb0 up
> tc qdisc add dev ifb0 root netem limit 1000 rate 1MBit
>
> tc qdisc add dev tap0 ingress
> tc filter add dev tap0 parent ffff: protocol ip \
> u32 match ip dport 8000 0xffff \
> action mirred egress redirect dev ifb0
>
> Before the delay, both flows process around 80K pps. With the delay,
> before this patch, both process around 400. After this patch, the
> large flow is still rate limited, while the small reverts to its
> original rate. See also discussion in the first link, below.
>
> The limit in vhost_exceeds_maxpend must be carefully chosen. When
> vq->num >> 1, the flows remain correlated. This value happens to
> correspond to VHOST_MAX_PENDING for vq->num == 256. Allow smaller
> fractions and ensure correctness also for much smaller values of
> vq->num, by testing the min() of both explicitly. See also the
> discussion in the second link below.
>
> Link:http://lkml.kernel.org/r/CAF=yD-+Wk9sc9dXMUq1+x_hh=3ThTXa6BnZkygP3tgVpjbp93g@mail.gmail.com
> Link:http://lkml.kernel.org/r/20170819064129.27272-1-den@klaipeden.com
> Signed-off-by: Willem de Bruijn <willemb@google.com>
I'd like to see the effect on the non rate limited case though.
If guest is quick won't we have lots of copies then?
> ---
> drivers/vhost/net.c | 14 ++++----------
> 1 file changed, 4 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index 58585ec8699e..50758602ae9d 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -436,8 +436,8 @@ static bool vhost_exceeds_maxpend(struct vhost_net *net)
> struct vhost_net_virtqueue *nvq = &net->vqs[VHOST_NET_VQ_TX];
> struct vhost_virtqueue *vq = &nvq->vq;
>
> - return (nvq->upend_idx + vq->num - VHOST_MAX_PEND) % UIO_MAXIOV
> - == nvq->done_idx;
> + return (nvq->upend_idx + UIO_MAXIOV - nvq->done_idx) % UIO_MAXIOV >
> + min(VHOST_MAX_PEND, vq->num >> 2);
> }
>
> /* Expects to be always run from workqueue - which acts as
> @@ -480,12 +480,6 @@ static void handle_tx(struct vhost_net *net)
> if (zcopy)
> vhost_zerocopy_signal_used(net, vq);
>
> - /* If more outstanding DMAs, queue the work.
> - * Handle upend_idx wrap around
> - */
> - if (unlikely(vhost_exceeds_maxpend(net)))
> - break;
> -
> head = vhost_net_tx_get_vq_desc(net, vq, vq->iov,
> ARRAY_SIZE(vq->iov),
> &out, &in);
> @@ -509,6 +503,7 @@ static void handle_tx(struct vhost_net *net)
> len = iov_length(vq->iov, out);
> iov_iter_init(&msg.msg_iter, WRITE, vq->iov, out, len);
> iov_iter_advance(&msg.msg_iter, hdr_size);
> +
> /* Sanity check */
> if (!msg_data_left(&msg)) {
> vq_err(vq, "Unexpected header len for TX: "
> @@ -519,8 +514,7 @@ static void handle_tx(struct vhost_net *net)
> len = msg_data_left(&msg);
>
> zcopy_used = zcopy && len >= VHOST_GOODCOPY_LEN
> - && (nvq->upend_idx + 1) % UIO_MAXIOV !=
> - nvq->done_idx
> + && !vhost_exceeds_maxpend(net)
> && vhost_net_tx_select_zcopy(net);
>
> /* use msg_control to pass vhost zerocopy ubuf info to skb */
> --
> 2.14.2.822.g60be5d43e6-goog
^ permalink raw reply
* [PATCH] net: hns3: fix null pointer dereference before null check
From: Colin King @ 2017-09-29 19:51 UTC (permalink / raw)
To: Yisen Zhuang, Salil Mehta, David S . Miller, netdev
Cc: kernel-janitors, linux-kernel
From: Colin Ian King <colin.king@canonical.com>
pointer ndev is being dereferenced with the call to netif_running
before it is being null checked. Re-order the code to only dereference
ndev after it has been null checked.
Detected by CoverityScan, CID#1457206 ("Dereference before null check")
Fixes: 9df8f79a4d29 ("net: hns3: Add DCB support when interacting with network stack")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
---
drivers/net/ethernet/hisilicon/hns3/hns3pf/hns3_enet.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hns3_enet.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hns3_enet.c
index 4a0890f98b70..c31506514e5d 100644
--- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hns3_enet.c
+++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hns3_enet.c
@@ -2865,7 +2865,7 @@ static int hns3_client_setup_tc(struct hnae3_handle *handle, u8 tc)
{
struct hnae3_knic_private_info *kinfo = &handle->kinfo;
struct net_device *ndev = kinfo->netdev;
- bool if_running = netif_running(ndev);
+ bool if_running;
int ret;
u8 i;
@@ -2875,6 +2875,8 @@ static int hns3_client_setup_tc(struct hnae3_handle *handle, u8 tc)
if (!ndev)
return -ENODEV;
+ if_running = netif_running(ndev);
+
ret = netdev_set_num_tc(ndev, tc);
if (ret)
return ret;
--
2.14.1
^ permalink raw reply related
* Re: [net-next V2 PATCH 1/5] bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP
From: Jesper Dangaard Brouer @ 2017-09-29 19:58 UTC (permalink / raw)
To: Jakub Kicinski
Cc: netdev, Michael S. Tsirkin, Jason Wang, mchan, John Fastabend,
peter.waskiewicz.jr, Daniel Borkmann, Alexei Starovoitov,
Andy Gospodarek, brouer
In-Reply-To: <20170929114154.4b5d5918@cakuba>
On Fri, 29 Sep 2017 11:41:54 -0700
Jakub Kicinski <kubakici@wp.pl> wrote:
> On Fri, 29 Sep 2017 18:34:12 +0200, Jesper Dangaard Brouer wrote:
> > The 'cpumap' is primary used as a backend map for XDP BPF helper
> > call bpf_redirect_map() and XDP_REDIRECT action, like 'devmap'.
> >
> > This patch implement the main part of the map. It is not connected to
> > the XDP redirect system yet, and no SKB allocation are done yet.
> >
> > The main concern in this patch is to ensure the datapath can run
> > without any locking. This adds complexity to the setup and tear-down
> > procedure, which assumptions are extra carefully documented in the
> > code comments.
> >
> > V2: make sure array isn't larger than num possible CPUs
> >
> > Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
>
> Few trivial nitpicks, hope you don't mind :)
>
> > @@ -0,0 +1,555 @@
> > +/* bpf/cpumap.c
> > + *
> > + * Copyright (c) 2017 Jesper Dangaard Brouer, Red Hat Inc.
> > + * Released under terms in GPL version 2. See COPYING.
> > + */
> > +
> > +/* The 'cpumap' is primary used as a backend map for XDP BPF helper
> > + * call bpf_redirect_map() and XDP_REDIRECT action, like 'devmap'.
> > + *
> > + * Unlike devmap which redirect XDP frames out another NIC device,
> > + * this map type redirect raw XDP frames to another CPU. The remote
> > + * CPU will do SKB-allocation and call the normal network stack.
> > + *
> > + * This is a scalability and isolation mechanism, that allow
> > + * separating the early driver network XDP layer, from the rest of the
> > + * netstack, and assigning dedicated CPUs for this stage. This
> > + * basically allows for 10G wirespeed pre-filtering via bpf.
> > + */
> > +#include <linux/bpf.h>
> > +#include <linux/filter.h>
> > +#include <linux/ptr_ring.h>
> > +
> > +#include <linux/sched.h>
> > +#include <linux/workqueue.h>
> > +#include <linux/kthread.h>
> > +
> > +/*
> > + * General idea: XDP packets getting XDP redirected to another CPU,
> > + * will maximum be stored/queued for one driver ->poll() call. It is
> > + * guaranteed that setting flush bit and flush operation happen on
> > + * same CPU. Thus, cpu_map_flush operation can deduct via this_cpu_ptr()
> > + * which queue in bpf_cpu_map_entry contains packets.
> > + */
> > +
> > +#define CPU_MAP_BULK_SIZE 8 /* 8 == one cacheline on 64-bit archs */
> > +struct xdp_bulk_queue {
> > + void *q[CPU_MAP_BULK_SIZE];
> > + unsigned int count;
> > +};
>
> Out of curiosity - would it make sense to make sure the entire struct
> fits into a cache line? The comment seems to indicate that the array is
> sized to fit a cache line, but then there is also the count member...
Nope, it does not make sense to align the struct itself for a cacheline.
The idea is that when I bulk enqueue transfer (from this percpu mem)
into the shared queue (ptr_ring), then I dirty a cacheline, thus I
might as well fill up the cacheline.
> > +/*
> > + * After xchg pointer to bpf_cpu_map_entry, use the call_rcu() to
> ...
>
> There is a mix for networking and non-networking style comments in this
> file, is this intentional?
No
> > +const struct bpf_map_ops cpu_map_ops = {
> > + .map_alloc = cpu_map_alloc,
> > + .map_free = cpu_map_free,
> > + .map_delete_elem = cpu_map_delete_elem,
> > + .map_update_elem = cpu_map_update_elem,
> > + .map_lookup_elem = cpu_map_lookup_elem,
> > + .map_get_next_key = cpu_map_get_next_key,
> > +};
> > +
> > +
>
> Extra new line.
Sorry
> > +/* Runs under RCU-read-side, plus in softirq under NAPI protection.
> > + * Thus, safe percpu variable access.
> > + */
> > +static int bq_enqueue(struct bpf_cpu_map_entry *rcpu, struct xdp_pkt *xdp_pkt)
> > +{
> > + struct xdp_bulk_queue *bq = this_cpu_ptr(rcpu->bulkq);
> > +
> > + if (unlikely(bq->count == CPU_MAP_BULK_SIZE)) {
> > + bq_flush_to_queue(rcpu, bq);
> > + }
>
> Curly brackets not needed.
Sorry
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply
* Re: [PATCH next] bonding: speed/duplex update at NETDEV_UP event
From: Stephen Hemminger @ 2017-09-29 20:08 UTC (permalink / raw)
To: Mahesh Bandewar
Cc: Jay Vosburgh, Andy Gospodarek, Veaceslav Falico, David Miller,
Netdev, Mahesh Bandewar
In-Reply-To: <20170928010349.8988-1-mahesh@bandewar.net>
On Wed, 27 Sep 2017 18:03:49 -0700
Mahesh Bandewar <mahesh@bandewar.net> wrote:
> From: Mahesh Bandewar <maheshb@google.com>
>
> Some NIC drivers don't have correct speed/duplex settings at the
> time they send NETDEV_UP notification and that messes up the
> bonding state. Especially 802.3ad mode which is very sensitive
> to these settings. In the current implementation we invoke
> bond_update_speed_duplex() when we receive NETDEV_UP, however,
> ignore the return value. If the values we get are invalid
> (UNKNOWN), then slave gets removed from the aggregator with
> speed and duplex set to UNKNOWN while link is still marked as UP.
>
> This patch fixes this scenario. Also 802.3ad mode is sensitive to
> these conditions while other modes are not, so making sure that it
> doesn't change the behavior for other modes.
>
> Signed-off-by: Mahesh Bandewar <maheshb@google.com>
> ---
> drivers/net/bonding/bond_main.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
> index b7313c1d9dcd..177be373966b 100644
> --- a/drivers/net/bonding/bond_main.c
> +++ b/drivers/net/bonding/bond_main.c
> @@ -3076,7 +3076,16 @@ static int bond_slave_netdev_event(unsigned long event,
> break;
> case NETDEV_UP:
> case NETDEV_CHANGE:
> - bond_update_speed_duplex(slave);
> + /* For 802.3ad mode only:
> + * Getting invalid Speed/Duplex values here will put slave
> + * in weird state. So mark it as link-down for the time
> + * being and let link-monitoring (miimon) set it right when
> + * correct speeds/duplex are available.
> + */
> + if (bond_update_speed_duplex(slave) &&
> + BOND_MODE(bond) == BOND_MODE_8023AD)
> + slave->link = BOND_LINK_DOWN;
> +
> if (BOND_MODE(bond) == BOND_MODE_8023AD)
> bond_3ad_adapter_speed_duplex_changed(slave);
> /* Fallthrough */
Then fix the drivers. Trying to workaround it here isn't helping.
The problem is that miimon is not required. Bonding can be purely
event driven.
^ permalink raw reply
* Re: [PATCH RFC 3/5] Add KSZ8795 switch driver
From: Andrew Lunn @ 2017-09-29 20:39 UTC (permalink / raw)
To: Tristram.Ha
Cc: David.Laight, muvarov, pavel, nathan.leigh.conrad, vivien.didelot,
f.fainelli, netdev, linux-kernel, Woojung.Huh
In-Reply-To: <93AF473E2DA327428DE3D46B72B1E9FD4112CC1D@CHN-SV-EXMX02.mchp-main.com>
On Fri, Sep 29, 2017 at 07:19:17PM +0000, Tristram.Ha@microchip.com wrote:
> > > My concern is if a task is already running with SPI access to a lot
> > > of registers like reading the 32 MIB counters in every port of the
> > > switch, another register access has to wait until they are finished.
> >
> > Why does it have to wait? Looking at the code in
> > ksz_get_ethtool_stats(), you don't take any mutex which will prevent
> > others from using the SPI bus. All there is is a mutex which prevents
> > two sets of ksz_get_ethtool_stats() at the same time.
> >
> > So a PTP read could happen in parallel, and will not be blocked by MIB
> > reads. They should get interleaved access to the SPI bus.
> >
>
> The MIB counters are read in the background. For multiple CPU cores 2
> tasks may run in the same time allowing SPI access one after another.
> For single core I am not sure an SPI access like coming from an interrupt
> routine can jump ahead from one in a background task.
The SPI subsystem has a mutex per controller. When starting a
transfer, it takes the mutex and release it once the transfer has
completed. There is also a reschedule point at the end of a
transfer. So even on your single core CPU, there can be multi tasking
going on.
Andrew
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox