* Re: [net PATCH v2 3/6] net sched: mirred action fix late binding
From: Cong Wang @ 2016-05-10 21:25 UTC (permalink / raw)
To: Jamal Hadi Salim
Cc: David Miller, Linux Kernel Network Developers,
Jiří Pírko, Alexander Duyck
In-Reply-To: <1462913371-9699-4-git-send-email-jhs@emojatatu.com>
On Tue, May 10, 2016 at 1:49 PM, Jamal Hadi Salim <jhs@mojatatu.com> wrote:
> From: Jamal Hadi Salim <jhs@mojatatu.com>
>
> The process below was broken and is fixed with this patch.
>
> //add an mirred action and give it an instance id of 1
> sudo tc actions add action mirred egress mirror dev $MDEV index 1
> //create a filter which binds to mirred action id 1
> sudo tc filter add dev $DEV parent ffff: protocol ip prio 1 u32\
> match ip dst 17.0.0.1/32 flowid 1:10 action mirred index 1
>
> Message before bug fix was:
> RTNETLINK answers: Invalid argument
> We have an error talking to the kernel
>
> Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
Reviewed-by: Cong Wang <xiyou.wangcong@gmail.com>
^ permalink raw reply
* Re: [net PATCH v2 4/6] net sched: simple action fix late binding
From: Cong Wang @ 2016-05-10 21:26 UTC (permalink / raw)
To: Jamal Hadi Salim
Cc: David Miller, Linux Kernel Network Developers,
Jiří Pírko, Alexander Duyck
In-Reply-To: <1462913371-9699-5-git-send-email-jhs@emojatatu.com>
On Tue, May 10, 2016 at 1:49 PM, Jamal Hadi Salim <jhs@mojatatu.com> wrote:
> From: Jamal Hadi Salim <jhs@mojatatu.com>
>
> The process below was broken and is fixed with this patch.
>
> //add a simple action and give it an instance id of 1
> sudo tc actions add action simple sdata "foobar" index 1
> //create a filter which binds to simple action id 1
> sudo tc filter add dev $DEV parent ffff: protocol ip prio 1 u32\
> match ip dst 17.0.0.1/32 flowid 1:10 action simple index 1
>
> Message before fix was:
> RTNETLINK answers: Invalid argument
> We have an error talking to the kernel
>
> Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
Reviewed-by: Cong Wang <xiyou.wangcong@gmail.com>
^ permalink raw reply
* Re: [net PATCH v2 5/6] net sched: skbedit action fix late binding
From: Cong Wang @ 2016-05-10 21:27 UTC (permalink / raw)
To: Jamal Hadi Salim
Cc: David Miller, Linux Kernel Network Developers,
Jiří Pírko, Alexander Duyck
In-Reply-To: <1462913371-9699-6-git-send-email-jhs@emojatatu.com>
On Tue, May 10, 2016 at 1:49 PM, Jamal Hadi Salim <jhs@mojatatu.com> wrote:
> From: Jamal Hadi Salim <jhs@mojatatu.com>
>
> The process below was broken and is fixed with this patch.
>
> //add a skbedit action and give it an instance id of 1
> sudo tc actions add action skbedit mark 10 index 1
> //create a filter which binds to skbedit action id 1
> sudo tc filter add dev $DEV parent ffff: protocol ip prio 1 u32\
> match ip dst 17.0.0.1/32 flowid 1:10 action skbedit index 1
>
> Message before fix was:
> RTNETLINK answers: Invalid argument
> We have an error talking to the kernel
>
> Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
Reviewed-by: Cong Wang <xiyou.wangcong@gmail.com>
^ permalink raw reply
* [PATCH v1 net-next 4/7] dsa: Move gpio reset into switch driver
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
In-Reply-To: <1462915645-24674-1-git-send-email-andrew@lunn.ch>
Resetting the switch is something the driver does, not the framework.
So move the parsing of this property into the driver.
There are no in kernel users of this property, so moving it does not
break anything. There is however a board which will make use of this
property making its way into the kernel.
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
Documentation/devicetree/bindings/net/dsa/dsa.txt | 2 --
Documentation/devicetree/bindings/net/dsa/marvell.txt | 8 ++++++++
drivers/net/dsa/mv88e6xxx.c | 14 +++++++++++++-
drivers/net/dsa/mv88e6xxx.h | 7 +++++++
include/net/dsa.h | 8 --------
net/dsa/dsa.c | 16 ----------------
6 files changed, 28 insertions(+), 27 deletions(-)
diff --git a/Documentation/devicetree/bindings/net/dsa/dsa.txt b/Documentation/devicetree/bindings/net/dsa/dsa.txt
index 5fdbbcdf8c4b..9f4807f90c31 100644
--- a/Documentation/devicetree/bindings/net/dsa/dsa.txt
+++ b/Documentation/devicetree/bindings/net/dsa/dsa.txt
@@ -31,8 +31,6 @@ A switch child node has the following optional property:
switch. Must be set if the switch can not detect
the presence and/or size of a connected EEPROM,
otherwise optional.
-- reset-gpios : phandle and specifier to a gpio line connected to
- reset pin of the switch chip.
A switch may have multiple "port" children nodes
diff --git a/Documentation/devicetree/bindings/net/dsa/marvell.txt b/Documentation/devicetree/bindings/net/dsa/marvell.txt
index cdd70cebdea7..7629189398aa 100644
--- a/Documentation/devicetree/bindings/net/dsa/marvell.txt
+++ b/Documentation/devicetree/bindings/net/dsa/marvell.txt
@@ -10,10 +10,17 @@ If you need a stable binding, use the old dsa.txt binding.
Marvell Switches are MDIO devices. The following properties should be
placed as a child node of an mdio device.
+The properties described here are those specific to Marvell devices.
+Additional required and optional properties can be found in dsa.txt.
+
Required properties:
- compatible : Should be one of "marvell,mv88e6085",
- reg : Address on the MII bus for the switch.
+Optional properties:
+
+- reset-gpios : Should be a gpio specifier for a reset line
+
Example:
mdio {
@@ -23,5 +30,6 @@ Example:
switch0: switch@0 {
compatible = "marvell,mv88e6085";
reg = <0>;
+ reset-gpios = <&gpio5 1 GPIO_ACTIVE_LOW>;
};
};
diff --git a/drivers/net/dsa/mv88e6xxx.c b/drivers/net/dsa/mv88e6xxx.c
index e5ba04132140..f4bb3bed812f 100644
--- a/drivers/net/dsa/mv88e6xxx.c
+++ b/drivers/net/dsa/mv88e6xxx.c
@@ -2568,7 +2568,7 @@ static int mv88e6xxx_switch_reset(struct mv88e6xxx_priv_state *ps)
{
bool ppu_active = mv88e6xxx_has(ps, MV88E6XXX_FLAG_PPU_ACTIVE);
u16 is_reset = (ppu_active ? 0x8800 : 0xc800);
- struct gpio_desc *gpiod = ps->ds->pd->reset;
+ struct gpio_desc *gpiod = ps->reset;
unsigned long timeout;
int ret;
int i;
@@ -3620,6 +3620,7 @@ int mv88e6xxx_probe(struct mdio_device *mdiodev)
struct mv88e6xxx_priv_state *ps;
int id, prod_num, rev;
struct dsa_switch *ds;
+ int err;
ds = devm_kzalloc(dev, sizeof(*ds) + sizeof(*ps), GFP_KERNEL);
if (!ds)
@@ -3649,6 +3650,17 @@ int mv88e6xxx_probe(struct mdio_device *mdiodev)
if (!ps->info)
return -ENODEV;
+ ps->reset = devm_gpiod_get(&mdiodev->dev, "reset", GPIOD_ASIS);
+ if (IS_ERR(ps->reset)) {
+ err = PTR_ERR(ps->reset);
+ if (err == -ENOENT) {
+ /* Optional, so not an error */
+ ps->reset = NULL;
+ } else {
+ return err;
+ }
+ }
+
dev_set_drvdata(dev, ds);
dev_info(dev, "switch 0x%x probed: %s, revision %u\n",
diff --git a/drivers/net/dsa/mv88e6xxx.h b/drivers/net/dsa/mv88e6xxx.h
index ca69a93a42a0..df226c151d9c 100644
--- a/drivers/net/dsa/mv88e6xxx.h
+++ b/drivers/net/dsa/mv88e6xxx.h
@@ -12,6 +12,7 @@
#define __MV88E6XXX_H
#include <linux/if_vlan.h>
+#include <linux/gpio/consumer.h>
#ifndef UINT64_MAX
#define UINT64_MAX (u64)(~((u64)0))
@@ -583,6 +584,12 @@ struct mv88e6xxx_priv_state {
DECLARE_BITMAP(port_state_update_mask, DSA_MAX_PORTS);
struct work_struct bridge_work;
+
+ /* A switch may have a GPIO line tied to its reset pin. Parse
+ * this from the device tree, and use it before performing
+ * switch soft reset.
+ */
+ struct gpio_desc *reset;
};
enum stat_type {
diff --git a/include/net/dsa.h b/include/net/dsa.h
index 8e86af87c84f..ecb52e265cc3 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -16,7 +16,6 @@
#include <linux/timer.h>
#include <linux/workqueue.h>
#include <linux/of.h>
-#include <linux/of_gpio.h>
#include <linux/phy.h>
#include <linux/phy_fixed.h>
#include <linux/ethtool.h>
@@ -65,13 +64,6 @@ struct dsa_chip_data {
* NULL if there is only one switch chip.
*/
s8 *rtable;
-
- /*
- * A switch may have a GPIO line tied to its reset pin. Parse
- * this from the device tree, and use it before performing
- * switch soft reset.
- */
- struct gpio_desc *reset;
};
struct dsa_platform_data {
diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c
index d61ceed912be..df169811f26d 100644
--- a/net/dsa/dsa.c
+++ b/net/dsa/dsa.c
@@ -659,9 +659,6 @@ static int dsa_of_probe(struct device *dev)
const char *port_name;
int chip_index, port_index;
const unsigned int *sw_addr, *port_reg;
- int gpio;
- enum of_gpio_flags of_flags;
- unsigned long flags;
u32 eeprom_len;
int ret;
@@ -740,19 +737,6 @@ static int dsa_of_probe(struct device *dev)
put_device(cd->host_dev);
cd->host_dev = &mdio_bus_switch->dev;
}
- gpio = of_get_named_gpio_flags(child, "reset-gpios", 0,
- &of_flags);
- if (gpio_is_valid(gpio)) {
- flags = (of_flags == OF_GPIO_ACTIVE_LOW ?
- GPIOF_ACTIVE_LOW : 0);
- ret = devm_gpio_request_one(dev, gpio, flags,
- "switch_reset");
- if (ret)
- goto out_free_chip;
-
- cd->reset = gpio_to_desc(gpio);
- gpiod_direction_output(cd->reset, 0);
- }
for_each_available_child_of_node(child, port) {
port_reg = of_get_property(port, "reg", NULL);
--
2.8.1
^ permalink raw reply related
* [PATCH v1 net-next 3/7] dsa: Add mdio device support to Marvell switches
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
In-Reply-To: <1462915645-24674-1-git-send-email-andrew@lunn.ch>
Allow Marvell switches to be mdio devices. Currently the driver just
allocate the private structure and detects what device is on the
bus. Later patches will make them register with the DSA framework.
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
.../devicetree/bindings/net/dsa/marvell.txt | 27 +++++++
drivers/net/dsa/mv88e6xxx.c | 90 +++++++++++++++++-----
2 files changed, 99 insertions(+), 18 deletions(-)
create mode 100644 Documentation/devicetree/bindings/net/dsa/marvell.txt
diff --git a/Documentation/devicetree/bindings/net/dsa/marvell.txt b/Documentation/devicetree/bindings/net/dsa/marvell.txt
new file mode 100644
index 000000000000..cdd70cebdea7
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/dsa/marvell.txt
@@ -0,0 +1,27 @@
+Marvell DSA Switch Device Tree Bindings
+---------------------------------------
+
+WARNING: This binding is currently unstable. Do not program it into a
+FLASH never to be changed again. Once this binding is stable, this
+warning will be removed.
+
+If you need a stable binding, use the old dsa.txt binding.
+
+Marvell Switches are MDIO devices. The following properties should be
+placed as a child node of an mdio device.
+
+Required properties:
+- compatible : Should be one of "marvell,mv88e6085",
+- reg : Address on the MII bus for the switch.
+
+Example:
+
+ mdio {
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ switch0: switch@0 {
+ compatible = "marvell,mv88e6085";
+ reg = <0>;
+ };
+ };
diff --git a/drivers/net/dsa/mv88e6xxx.c b/drivers/net/dsa/mv88e6xxx.c
index 199a8628df2b..e5ba04132140 100644
--- a/drivers/net/dsa/mv88e6xxx.c
+++ b/drivers/net/dsa/mv88e6xxx.c
@@ -5,6 +5,8 @@
* Copyright (c) 2015 CMC Electronics, Inc.
* Added support for VLAN Table Unit operations
*
+ * Copyright (c) 2016 Andrew Lunn <andrew@lunn.ch>
+ *
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
@@ -17,6 +19,7 @@
#include <linux/if_bridge.h>
#include <linux/jiffies.h>
#include <linux/list.h>
+#include <linux/mdio.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/gpio/consumer.h>
@@ -3611,36 +3614,87 @@ struct dsa_switch_driver mv88e6xxx_switch_driver = {
.port_fdb_dump = mv88e6xxx_port_fdb_dump,
};
-static int __init mv88e6xxx_init(void)
+int mv88e6xxx_probe(struct mdio_device *mdiodev)
{
- register_switch_driver(&mv88e6xxx_switch_driver);
+ struct device *dev = &mdiodev->dev;
+ struct mv88e6xxx_priv_state *ps;
+ int id, prod_num, rev;
+ struct dsa_switch *ds;
+
+ ds = devm_kzalloc(dev, sizeof(*ds) + sizeof(*ps), GFP_KERNEL);
+ if (!ds)
+ return -ENOMEM;
+
+ ps = (struct mv88e6xxx_priv_state *)(ds + 1);
+ ds->priv = ps;
+ ps->dev = dev;
+ ps->ds = ds;
+ ps->bus = mdiodev->bus;
+ ps->sw_addr = mdiodev->addr;
+ mutex_init(&ps->smi_mutex);
+
+ get_device(&ps->bus->dev);
+
+ ds->drv = &mv88e6xxx_switch_driver;
+
+ id = mv88e6xxx_reg_read(ps, REG_PORT(0), PORT_SWITCH_ID);
+ if (id < 0)
+ return id;
+
+ prod_num = (id & 0xfff0) >> 4;
+ rev = id & 0x000f;
+
+ ps->info = mv88e6xxx_lookup_info(prod_num, mv88e6xxx_table,
+ ARRAY_SIZE(mv88e6xxx_table));
+ if (!ps->info)
+ return -ENODEV;
+
+ dev_set_drvdata(dev, ds);
+
+ dev_info(dev, "switch 0x%x probed: %s, revision %u\n",
+ prod_num, ps->info->name, rev);
return 0;
}
+
+static void mv88e6xxx_remove(struct mdio_device *mdiodev)
+{
+ struct dsa_switch *ds = dev_get_drvdata(&mdiodev->dev);
+ struct mv88e6xxx_priv_state *ps = ds_to_priv(ds);
+
+ put_device(&ps->bus->dev);
+}
+
+static const struct of_device_id mv88e6xxx_of_match[] = {
+ { .compatible = "marvell,mv88e6085" },
+ { /* sentinel */ },
+};
+
+MODULE_DEVICE_TABLE(of, mv88e6xxx_of_match);
+
+static struct mdio_driver mv88e6xxx_driver = {
+ .probe = mv88e6xxx_probe,
+ .remove = mv88e6xxx_remove,
+ .mdiodrv.driver = {
+ .name = "mv88e6085",
+ .of_match_table = mv88e6xxx_of_match,
+ },
+};
+
+static int __init mv88e6xxx_init(void)
+{
+ register_switch_driver(&mv88e6xxx_switch_driver);
+ return mdio_driver_register(&mv88e6xxx_driver);
+}
module_init(mv88e6xxx_init);
static void __exit mv88e6xxx_cleanup(void)
{
+ mdio_driver_unregister(&mv88e6xxx_driver);
unregister_switch_driver(&mv88e6xxx_switch_driver);
}
module_exit(mv88e6xxx_cleanup);
-MODULE_ALIAS("platform:mv88e6085");
-MODULE_ALIAS("platform:mv88e6095");
-MODULE_ALIAS("platform:mv88e6095f");
-MODULE_ALIAS("platform:mv88e6123");
-MODULE_ALIAS("platform:mv88e6131");
-MODULE_ALIAS("platform:mv88e6161");
-MODULE_ALIAS("platform:mv88e6165");
-MODULE_ALIAS("platform:mv88e6171");
-MODULE_ALIAS("platform:mv88e6172");
-MODULE_ALIAS("platform:mv88e6175");
-MODULE_ALIAS("platform:mv88e6176");
-MODULE_ALIAS("platform:mv88e6320");
-MODULE_ALIAS("platform:mv88e6321");
-MODULE_ALIAS("platform:mv88e6350");
-MODULE_ALIAS("platform:mv88e6351");
-MODULE_ALIAS("platform:mv88e6352");
MODULE_AUTHOR("Lennert Buytenhek <buytenh@wantstofly.org>");
MODULE_DESCRIPTION("Driver for Marvell 88E6XXX ethernet switch chips");
MODULE_LICENSE("GPL");
--
2.8.1
^ permalink raw reply related
* Re: [net PATCH v2 6/6] net sched: ife action fix late binding
From: Cong Wang @ 2016-05-10 21:28 UTC (permalink / raw)
To: Jamal Hadi Salim
Cc: David Miller, Linux Kernel Network Developers,
Jiří Pírko, Alexander Duyck
In-Reply-To: <1462913371-9699-7-git-send-email-jhs@emojatatu.com>
On Tue, May 10, 2016 at 1:49 PM, Jamal Hadi Salim <jhs@mojatatu.com> wrote:
> From: Jamal Hadi Salim <jhs@mojatatu.com>
>
> The process below was broken and is fixed with this patch.
>
> //add an ife action and give it an instance id of 1
> sudo tc actions add action ife encode \
> type 0xDEAD allow mark dst 02:15:15:15:15:15 index 1
>
> //create a filter which binds to ife action id 1
> sudo tc filter add dev $DEV parent ffff: protocol ip prio 1 u32\
> match ip dst 17.0.0.1/32 flowid 1:11 action ife index 1
>
> Message before fix was:
> RTNETLINK answers: Invalid argument
> We have an error talking to the kernel
>
> Signed-off-by: Jamal Hadi Salim <jhs@mojatatu.com>
Reviewed-by: Cong Wang <xiyou.wangcong@gmail.com>
^ permalink raw reply
* [PATCH v1 net-next 0/7] More enabler patches for DSA probing
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
The complete set of patches for the reworked DSA probing is too big to
post as once. These subset contains some enablers which are easy to
review.
Eventually, the Marvell driver will instantiate its own internal MDIO
bus, rather than have the framework do it, thus allows devices on the
bus to be listed in the device tree. Initialize the main mutex as soon
as it is created, to avoid lifetime issues with the mdio bus.
A previous patch renamed all the DSA probe functions to make room for
a true device probe. However the recent merging of all the Marvell
switch drivers resulted in mv88e6xxx going back to the old probe
name. Rename it again, so we can have a driver probe function.
Add minimum support for the Marvell switch driver to probe as an MDIO
device, as well as an DSA driver. Later patches will then register
this device with the new DSA core framework.
Move the GPIO reset code out of the DSA code. Different drivers may
need different reset mechanisms, e.g. via a reset controller for
memory mapped devices. Don't clutter up the core with this. Let each
driver implement what it needs.
master_dev is no longer needed in the switch drivers, since they have
access to a device pointer from the probe function. Remove it.
Let the switch parse the eeprom length from its one device tree
node. This is required with the new binding when the central DSA
platform device no longer exists.
Andrew Lunn (7):
dsa: mv88e6xxx: Initialise the mutex as soon as it is created
dsa: mv88e6xxx: Rename probe function to fit the normal pattern
dsa: Add mdio device support to Marvell switches
dsa: Move gpio reset into switch driver
dsa: Remove master_dev from switch structure
dsa: Rename switch chip data to cd
dsa: mv88e6xxx: Handle eeprom-length property
Documentation/devicetree/bindings/net/dsa/dsa.txt | 2 -
.../devicetree/bindings/net/dsa/marvell.txt | 35 ++++++
drivers/net/dsa/bcm_sf2.c | 4 +-
drivers/net/dsa/mv88e6xxx.c | 137 +++++++++++++++++----
drivers/net/dsa/mv88e6xxx.h | 10 ++
include/net/dsa.h | 19 +--
net/dsa/dsa.c | 36 ++----
net/dsa/slave.c | 12 +-
8 files changed, 177 insertions(+), 78 deletions(-)
create mode 100644 Documentation/devicetree/bindings/net/dsa/marvell.txt
--
2.8.1
^ permalink raw reply
* [PATCH v1 net-next 2/7] dsa: mv88e6xxx: Rename probe function to fit the normal pattern
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
In-Reply-To: <1462915645-24674-1-git-send-email-andrew@lunn.ch>
All other DSA drivers use _drv_ in there DSA probe function name, thus
allowing for a true linux driver probe function to use the
conventional name. Make mv88e6xxx fit this pattern.
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
drivers/net/dsa/mv88e6xxx.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/net/dsa/mv88e6xxx.c b/drivers/net/dsa/mv88e6xxx.c
index b2d27bfd53c2..199a8628df2b 100644
--- a/drivers/net/dsa/mv88e6xxx.c
+++ b/drivers/net/dsa/mv88e6xxx.c
@@ -3529,9 +3529,9 @@ mv88e6xxx_lookup_info(unsigned int prod_num, const struct mv88e6xxx_info *table,
return NULL;
}
-static const char *mv88e6xxx_probe(struct device *dsa_dev,
- struct device *host_dev, int sw_addr,
- void **priv)
+static const char *mv88e6xxx_drv_probe(struct device *dsa_dev,
+ struct device *host_dev, int sw_addr,
+ void **priv)
{
const struct mv88e6xxx_info *info;
struct mv88e6xxx_priv_state *ps;
@@ -3576,7 +3576,7 @@ static const char *mv88e6xxx_probe(struct device *dsa_dev,
struct dsa_switch_driver mv88e6xxx_switch_driver = {
.tag_protocol = DSA_TAG_PROTO_EDSA,
- .probe = mv88e6xxx_probe,
+ .probe = mv88e6xxx_drv_probe,
.setup = mv88e6xxx_setup,
.set_addr = mv88e6xxx_set_addr,
.phy_read = mv88e6xxx_phy_read,
--
2.8.1
^ permalink raw reply related
* [PATCH v1 net-next 1/7] dsa: mv88e6xxx: Initialise the mutex as soon as it is created
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
In-Reply-To: <1462915645-24674-1-git-send-email-andrew@lunn.ch>
By initialising immediately it, we don't run the danger of using it
before it is initialised.
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
drivers/net/dsa/mv88e6xxx.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/net/dsa/mv88e6xxx.c b/drivers/net/dsa/mv88e6xxx.c
index 1e5ca8e0f48e..b2d27bfd53c2 100644
--- a/drivers/net/dsa/mv88e6xxx.c
+++ b/drivers/net/dsa/mv88e6xxx.c
@@ -3118,8 +3118,6 @@ static int mv88e6xxx_setup(struct dsa_switch *ds)
ps->ds = ds;
- mutex_init(&ps->smi_mutex);
-
INIT_WORK(&ps->bridge_work, mv88e6xxx_bridge_work);
if (mv88e6xxx_has(ps, MV88E6XXX_FLAG_EEPROM))
@@ -3566,6 +3564,7 @@ static const char *mv88e6xxx_probe(struct device *dsa_dev,
ps->bus = bus;
ps->sw_addr = sw_addr;
ps->info = info;
+ mutex_init(&ps->smi_mutex);
*priv = ps;
--
2.8.1
^ permalink raw reply related
* Re: [PATCH net-next 1/2] net: dsa: mv88e6xxx: abstract VTU/STU data access
From: Andrew Lunn @ 2016-05-10 21:29 UTC (permalink / raw)
To: Vivien Didelot; +Cc: netdev, linux-kernel, kernel, David S. Miller
In-Reply-To: <1462909469-4827-1-git-send-email-vivien.didelot@savoirfairelinux.com>
On Tue, May 10, 2016 at 03:44:28PM -0400, Vivien Didelot wrote:
> Both VTU and STU operations use the same routine to access their
> (common) data registers, with a different offset.
>
> Add VTU and STU specific read and write functions to the data registers
> to abstract the required offset.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Hi Vivien
Improves the readability.
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Thanks
Andrew
^ permalink raw reply
* [PATCH v1 net-next 7/7] dsa: mv88e6xxx: Handle eeprom-length property
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
In-Reply-To: <1462915645-24674-1-git-send-email-andrew@lunn.ch>
A switch can export an attached EEPROM using the standard ethtool API.
However the switch itself cannot determine the size of the EEPROM, and
multiple sizes are allowed. Thus a device tree property is supported
to indicate the length of the EEPROM. Parse this property during
device probe, and implement a callback function to retrieve it.
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
drivers/net/dsa/mv88e6xxx.c | 17 +++++++++++++++++
drivers/net/dsa/mv88e6xxx.h | 3 +++
2 files changed, 20 insertions(+)
diff --git a/drivers/net/dsa/mv88e6xxx.c b/drivers/net/dsa/mv88e6xxx.c
index c41ec38613c7..6f9be26d002f 100644
--- a/drivers/net/dsa/mv88e6xxx.c
+++ b/drivers/net/dsa/mv88e6xxx.c
@@ -879,6 +879,16 @@ error:
return ret;
}
+static int mv88e6xxx_get_eeprom_len(struct dsa_switch *ds)
+{
+ struct mv88e6xxx_priv_state *ps = ds_to_priv(ds);
+
+ if (mv88e6xxx_has(ps, MV88E6XXX_FLAG_EEPROM))
+ return ps->eeprom_len;
+
+ return 0;
+}
+
static int mv88e6xxx_get_eeprom(struct dsa_switch *ds,
struct ethtool_eeprom *eeprom, u8 *data)
{
@@ -3596,6 +3606,7 @@ struct dsa_switch_driver mv88e6xxx_switch_driver = {
.set_temp_limit = mv88e6xxx_set_temp_limit,
.get_temp_alarm = mv88e6xxx_get_temp_alarm,
#endif
+ .get_eeprom_len = mv88e6xxx_get_eeprom_len,
.get_eeprom = mv88e6xxx_get_eeprom,
.set_eeprom = mv88e6xxx_set_eeprom,
.get_regs_len = mv88e6xxx_get_regs_len,
@@ -3617,9 +3628,11 @@ struct dsa_switch_driver mv88e6xxx_switch_driver = {
int mv88e6xxx_probe(struct mdio_device *mdiodev)
{
struct device *dev = &mdiodev->dev;
+ struct device_node *np = dev->of_node;
struct mv88e6xxx_priv_state *ps;
int id, prod_num, rev;
struct dsa_switch *ds;
+ u32 eeprom_len;
int err;
ds = devm_kzalloc(dev, sizeof(*ds) + sizeof(*ps), GFP_KERNEL);
@@ -3662,6 +3675,10 @@ int mv88e6xxx_probe(struct mdio_device *mdiodev)
}
}
+ if (mv88e6xxx_has(ps, MV88E6XXX_FLAG_EEPROM) &&
+ !of_property_read_u32(np, "eeprom-length", &eeprom_len))
+ ps->eeprom_len = eeprom_len;
+
dev_set_drvdata(dev, ds);
dev_info(dev, "switch 0x%x probed: %s, revision %u\n",
diff --git a/drivers/net/dsa/mv88e6xxx.h b/drivers/net/dsa/mv88e6xxx.h
index df226c151d9c..d83565ac684e 100644
--- a/drivers/net/dsa/mv88e6xxx.h
+++ b/drivers/net/dsa/mv88e6xxx.h
@@ -590,6 +590,9 @@ struct mv88e6xxx_priv_state {
* switch soft reset.
*/
struct gpio_desc *reset;
+
+ /* set to size of eeprom if supported by the switch */
+ int eeprom_len;
};
enum stat_type {
--
2.8.1
^ permalink raw reply related
* [PATCH v1 net-next 5/7] dsa: Remove master_dev from switch structure
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
In-Reply-To: <1462915645-24674-1-git-send-email-andrew@lunn.ch>
The switch drivers only use the master_dev member for dev_info()
messages. Now that the device is passed to the old style probe, and
new style drivers are probed as true linux drivers, this is no longer
needed.
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
drivers/net/dsa/mv88e6xxx.c | 1 +
include/net/dsa.h | 7 ++-----
net/dsa/dsa.c | 2 +-
net/dsa/slave.c | 2 +-
4 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/drivers/net/dsa/mv88e6xxx.c b/drivers/net/dsa/mv88e6xxx.c
index f4bb3bed812f..428d213ed4fc 100644
--- a/drivers/net/dsa/mv88e6xxx.c
+++ b/drivers/net/dsa/mv88e6xxx.c
@@ -3628,6 +3628,7 @@ int mv88e6xxx_probe(struct mdio_device *mdiodev)
ps = (struct mv88e6xxx_priv_state *)(ds + 1);
ds->priv = ps;
+ ds->dev = dev;
ps->dev = dev;
ps->ds = ds;
ps->bus = mdiodev->bus;
diff --git a/include/net/dsa.h b/include/net/dsa.h
index ecb52e265cc3..f4c0bff8d9d6 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -120,6 +120,8 @@ struct dsa_switch_tree {
};
struct dsa_switch {
+ struct device *dev;
+
/*
* Parent switch tree, and switch index.
*/
@@ -142,11 +144,6 @@ struct dsa_switch {
*/
struct dsa_switch_driver *drv;
- /*
- * Reference to host device to use.
- */
- struct device *master_dev;
-
#ifdef CONFIG_NET_DSA_HWMON
/*
* Hardware monitoring information
diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c
index df169811f26d..5db779c69a68 100644
--- a/net/dsa/dsa.c
+++ b/net/dsa/dsa.c
@@ -411,7 +411,7 @@ dsa_switch_setup(struct dsa_switch_tree *dst, int index,
ds->pd = pd;
ds->drv = drv;
ds->priv = priv;
- ds->master_dev = host_dev;
+ ds->dev = parent;
ret = dsa_switch_setup_one(ds, parent);
if (ret)
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index 5ea8a40c8d33..f25dcd9e814a 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -51,7 +51,7 @@ void dsa_slave_mii_bus_init(struct dsa_switch *ds)
ds->slave_mii_bus->write = dsa_slave_phy_write;
snprintf(ds->slave_mii_bus->id, MII_BUS_ID_SIZE, "dsa-%d:%.2x",
ds->index, ds->pd->sw_addr);
- ds->slave_mii_bus->parent = ds->master_dev;
+ ds->slave_mii_bus->parent = ds->dev;
ds->slave_mii_bus->phy_mask = ~ds->phys_mii_mask;
}
--
2.8.1
^ permalink raw reply related
* [PATCH v1 net-next 6/7] dsa: Rename switch chip data to cd
From: Andrew Lunn @ 2016-05-10 21:27 UTC (permalink / raw)
To: David Miller, Vivien Didelot, Florian Fainelli; +Cc: netdev, Andrew Lunn
In-Reply-To: <1462915645-24674-1-git-send-email-andrew@lunn.ch>
The dsa_switch structure contains a dsa_chip_data member called pd.
However in the rest of the code, pd is used for dsa_platform_data.
This is confusing. Rename it cd, which is already often used in dsa.c
and slave.c for this data type.
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
drivers/net/dsa/bcm_sf2.c | 4 ++--
drivers/net/dsa/mv88e6xxx.c | 4 ++--
include/net/dsa.h | 4 ++--
net/dsa/dsa.c | 18 +++++++++---------
net/dsa/slave.c | 10 +++++-----
5 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/drivers/net/dsa/bcm_sf2.c b/drivers/net/dsa/bcm_sf2.c
index 448deb59b9a4..10ddd5a5dfb6 100644
--- a/drivers/net/dsa/bcm_sf2.c
+++ b/drivers/net/dsa/bcm_sf2.c
@@ -949,8 +949,8 @@ static int bcm_sf2_sw_setup(struct dsa_switch *ds)
/* All the interesting properties are at the parent device_node
* level
*/
- dn = ds->pd->of_node->parent;
- bcm_sf2_identify_ports(priv, ds->pd->of_node);
+ dn = ds->cd->of_node->parent;
+ bcm_sf2_identify_ports(priv, ds->cd->of_node);
priv->irq0 = irq_of_parse_and_map(dn, 0);
priv->irq1 = irq_of_parse_and_map(dn, 1);
diff --git a/drivers/net/dsa/mv88e6xxx.c b/drivers/net/dsa/mv88e6xxx.c
index 428d213ed4fc..c41ec38613c7 100644
--- a/drivers/net/dsa/mv88e6xxx.c
+++ b/drivers/net/dsa/mv88e6xxx.c
@@ -3009,9 +3009,9 @@ static int mv88e6xxx_setup_global(struct mv88e6xxx_priv_state *ps)
for (i = 0; i < 32; i++) {
int nexthop = 0x1f;
- if (ps->ds->pd->rtable &&
+ if (ps->ds->cd->rtable &&
i != ps->ds->index && i < ps->ds->dst->pd->nr_chips)
- nexthop = ps->ds->pd->rtable[i] & 0x1f;
+ nexthop = ps->ds->cd->rtable[i] & 0x1f;
err = _mv88e6xxx_reg_write(
ps, REG_GLOBAL2,
diff --git a/include/net/dsa.h b/include/net/dsa.h
index f4c0bff8d9d6..17c3d37b6779 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -137,7 +137,7 @@ struct dsa_switch {
/*
* Configuration data for this switch.
*/
- struct dsa_chip_data *pd;
+ struct dsa_chip_data *cd;
/*
* The used switch driver.
@@ -190,7 +190,7 @@ static inline u8 dsa_upstream_port(struct dsa_switch *ds)
if (dst->cpu_switch == ds->index)
return dst->cpu_port;
else
- return ds->pd->rtable[dst->cpu_switch];
+ return ds->cd->rtable[dst->cpu_switch];
}
struct switchdev_trans;
diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c
index 5db779c69a68..eff5dfc2e33f 100644
--- a/net/dsa/dsa.c
+++ b/net/dsa/dsa.c
@@ -182,7 +182,7 @@ __ATTRIBUTE_GROUPS(dsa_hwmon);
/* basic switch operations **************************************************/
static int dsa_cpu_dsa_setup(struct dsa_switch *ds, struct net_device *master)
{
- struct dsa_chip_data *cd = ds->pd;
+ struct dsa_chip_data *cd = ds->cd;
struct device_node *port_dn;
struct phy_device *phydev;
int ret, port, mode;
@@ -219,7 +219,7 @@ static int dsa_switch_setup_one(struct dsa_switch *ds, struct device *parent)
{
struct dsa_switch_driver *drv = ds->drv;
struct dsa_switch_tree *dst = ds->dst;
- struct dsa_chip_data *pd = ds->pd;
+ struct dsa_chip_data *cd = ds->cd;
bool valid_name_found = false;
int index = ds->index;
int i, ret;
@@ -230,7 +230,7 @@ static int dsa_switch_setup_one(struct dsa_switch *ds, struct device *parent)
for (i = 0; i < DSA_MAX_PORTS; i++) {
char *name;
- name = pd->port_names[i];
+ name = cd->port_names[i];
if (name == NULL)
continue;
@@ -328,10 +328,10 @@ static int dsa_switch_setup_one(struct dsa_switch *ds, struct device *parent)
if (!(ds->enabled_port_mask & (1 << i)))
continue;
- ret = dsa_slave_create(ds, parent, i, pd->port_names[i]);
+ ret = dsa_slave_create(ds, parent, i, cd->port_names[i]);
if (ret < 0) {
netdev_err(dst->master_netdev, "[%d]: can't create dsa slave device for port %d(%s): %d\n",
- index, i, pd->port_names[i], ret);
+ index, i, cd->port_names[i], ret);
ret = 0;
}
}
@@ -379,7 +379,7 @@ static struct dsa_switch *
dsa_switch_setup(struct dsa_switch_tree *dst, int index,
struct device *parent, struct device *host_dev)
{
- struct dsa_chip_data *pd = dst->pd->chip + index;
+ struct dsa_chip_data *cd = dst->pd->chip + index;
struct dsa_switch_driver *drv;
struct dsa_switch *ds;
int ret;
@@ -389,7 +389,7 @@ dsa_switch_setup(struct dsa_switch_tree *dst, int index,
/*
* Probe for switch model.
*/
- drv = dsa_switch_probe(parent, host_dev, pd->sw_addr, &name, &priv);
+ drv = dsa_switch_probe(parent, host_dev, cd->sw_addr, &name, &priv);
if (drv == NULL) {
netdev_err(dst->master_netdev, "[%d]: could not detect attached switch\n",
index);
@@ -408,7 +408,7 @@ dsa_switch_setup(struct dsa_switch_tree *dst, int index,
ds->dst = dst;
ds->index = index;
- ds->pd = pd;
+ ds->cd = cd;
ds->drv = drv;
ds->priv = priv;
ds->dev = parent;
@@ -424,7 +424,7 @@ static void dsa_switch_destroy(struct dsa_switch *ds)
{
struct device_node *port_dn;
struct phy_device *phydev;
- struct dsa_chip_data *cd = ds->pd;
+ struct dsa_chip_data *cd = ds->cd;
int port;
#ifdef CONFIG_NET_DSA_HWMON
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index f25dcd9e814a..152436cdab30 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -50,7 +50,7 @@ void dsa_slave_mii_bus_init(struct dsa_switch *ds)
ds->slave_mii_bus->read = dsa_slave_phy_read;
ds->slave_mii_bus->write = dsa_slave_phy_write;
snprintf(ds->slave_mii_bus->id, MII_BUS_ID_SIZE, "dsa-%d:%.2x",
- ds->index, ds->pd->sw_addr);
+ ds->index, ds->cd->sw_addr);
ds->slave_mii_bus->parent = ds->dev;
ds->slave_mii_bus->phy_mask = ~ds->phys_mii_mask;
}
@@ -615,8 +615,8 @@ static int dsa_slave_get_eeprom_len(struct net_device *dev)
struct dsa_slave_priv *p = netdev_priv(dev);
struct dsa_switch *ds = p->parent;
- if (ds->pd->eeprom_len)
- return ds->pd->eeprom_len;
+ if (ds->cd->eeprom_len)
+ return ds->cd->eeprom_len;
if (ds->drv->get_eeprom_len)
return ds->drv->get_eeprom_len(ds);
@@ -999,7 +999,7 @@ static int dsa_slave_phy_setup(struct dsa_slave_priv *p,
struct net_device *slave_dev)
{
struct dsa_switch *ds = p->parent;
- struct dsa_chip_data *cd = ds->pd;
+ struct dsa_chip_data *cd = ds->cd;
struct device_node *phy_dn, *port_dn;
bool phy_is_fixed = false;
u32 phy_flags = 0;
@@ -1147,7 +1147,7 @@ int dsa_slave_create(struct dsa_switch *ds, struct device *parent,
NULL);
SET_NETDEV_DEV(slave_dev, parent);
- slave_dev->dev.of_node = ds->pd->port_dn[port];
+ slave_dev->dev.of_node = ds->cd->port_dn[port];
slave_dev->vlan_features = master->vlan_features;
p = netdev_priv(slave_dev);
--
2.8.1
^ permalink raw reply related
* Re: [PATCH net-next 2/2] net: dsa: mv88e6xxx: add STU capability
From: Andrew Lunn @ 2016-05-10 21:31 UTC (permalink / raw)
To: Vivien Didelot; +Cc: netdev, linux-kernel, kernel, David S. Miller
In-Reply-To: <1462909469-4827-2-git-send-email-vivien.didelot@savoirfairelinux.com>
On Tue, May 10, 2016 at 03:44:29PM -0400, Vivien Didelot wrote:
> Some switch models have a STU (per VLAN port state database). Add a new
> capability flag to switches info, instead of checking their family.
>
> Also if the 6165 family has an STU, it must have a VTU, so add the
> MV88E6XXX_FLAG_VTU to its family flags.
>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Andrew
^ permalink raw reply
* Re: [RFC PATCH 0/2] net: threadable napi poll loop
From: Eric Dumazet @ 2016-05-10 21:31 UTC (permalink / raw)
To: Eric Dumazet
Cc: Hannes Frederic Sowa, Paolo Abeni, netdev, David S. Miller,
Jiri Pirko, Daniel Borkmann, Alexei Starovoitov, Alexander Duyck,
Tom Herbert, Peter Zijlstra, Ingo Molnar, Rik van Riel, LKML
In-Reply-To: <CANn89i+YKrVOgfFwW-2psfCzq5_YLdmbV06-H-1FCOwXdMBuzQ@mail.gmail.com>
On Tue, 2016-05-10 at 14:09 -0700, Eric Dumazet wrote:
> On Tue, May 10, 2016 at 1:46 PM, Hannes Frederic Sowa
> <hannes@stressinduktion.org> wrote:
>
> > I agree here, but I don't think this patch particularly is a lot of
> > bloat and something very interesting people can play with and extend upon.
> >
>
> Sure, very rarely patch authors think their stuff is bloat.
>
> I prefer to fix kernel softirq.c, or at least show me that you tried
> hard enough.
>
> I am pretty sure that the following would work :
>
> When ksoftirqd is scheduled, remember this in a per cpu variable
> (ksoftiqd_scheduled)
>
> When enabling BH , do not call do_softirq() if this variable is set.
>
> ksoftirqd would clear the variable at the right place (probably in
> run_ksoftirqd())
>
> Sure, this might add a lot of latency regressions, but lets fix them.
Only to give the idea (it is completely untested and probably buggy)
diff --git a/kernel/softirq.c b/kernel/softirq.c
index 17caf4b63342..cb30cfd76687 100644
--- a/kernel/softirq.c
+++ b/kernel/softirq.c
@@ -56,6 +56,7 @@ EXPORT_SYMBOL(irq_stat);
static struct softirq_action softirq_vec[NR_SOFTIRQS] __cacheline_aligned_in_smp;
DEFINE_PER_CPU(struct task_struct *, ksoftirqd);
+DEFINE_PER_CPU(bool, ksoftirqd_scheduled);
const char * const softirq_to_name[NR_SOFTIRQS] = {
"HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL",
@@ -73,8 +74,10 @@ static void wakeup_softirqd(void)
/* Interrupts are disabled: no need to stop preemption */
struct task_struct *tsk = __this_cpu_read(ksoftirqd);
- if (tsk && tsk->state != TASK_RUNNING)
+ if (tsk && tsk->state != TASK_RUNNING) {
+ __this_cpu_write(ksoftirqd_scheduled, true);
wake_up_process(tsk);
+ }
}
/*
@@ -162,7 +165,9 @@ void __local_bh_enable_ip(unsigned long ip, unsigned int cnt)
*/
preempt_count_sub(cnt - 1);
- if (unlikely(!in_interrupt() && local_softirq_pending())) {
+ if (unlikely(!in_interrupt() &&
+ local_softirq_pending() &&
+ !__this_cpu_read(ksoftirqd_scheduled))) {
/*
* Run softirq if any pending. And do it in its own stack
* as we may be calling this deep in a task call stack already.
@@ -660,6 +665,8 @@ static void run_ksoftirqd(unsigned int cpu)
* in the task stack here.
*/
__do_softirq();
+ if (!local_softirq_pending())
+ __this_cpu_write(ksoftirqd_scheduled, false);
local_irq_enable();
cond_resched_rcu_qs();
return;
^ permalink raw reply related
* [PATCH net-next 1/3] net/mlx5e: CQE compression
From: Saeed Mahameed @ 2016-05-10 21:29 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Or Gerlitz, Tal Alon, Eran Ben Elisha, Tariq Toukan,
Eugenia Emantayev, Saeed Mahameed
In-Reply-To: <1462915756-18180-1-git-send-email-saeedm@mellanox.com>
From: Tariq Toukan <tariqt@mellanox.com>
CQE compression feature is meant to save PCIe bandwidth by
compressing few CQEs into smaller amount of bytes on PCIe.
CQE compression can be selectively enabled per CQ. By default
is disabled for now and will be enabled later on.
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Eugenia Emantayev <eugenia@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 10 ++
drivers/net/ethernet/mellanox/mlx5/core/en_clock.c | 4 +
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 6 +
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 151 ++++++++++++++++++++-
drivers/net/ethernet/mellanox/mlx5/core/en_stats.h | 8 ++
include/linux/mlx5/device.h | 34 +++++
6 files changed, 211 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index bfa5daa..19f0d8d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -157,6 +157,8 @@ struct mlx5e_params {
u8 log_rq_size;
u16 num_channels;
u8 num_tc;
+ bool rx_cqe_compress_admin;
+ bool rx_cqe_compress;
u16 rx_cq_moderation_usec;
u16 rx_cq_moderation_pkts;
u16 tx_cq_moderation_usec;
@@ -202,6 +204,13 @@ struct mlx5e_cq {
struct mlx5e_channel *channel;
struct mlx5e_priv *priv;
+ /* cqe decompression */
+ struct mlx5_cqe64 title;
+ struct mlx5_mini_cqe8 mini_arr[MLX5_MINI_CQE_ARRAY_SIZE];
+ u8 mini_arr_idx;
+ u16 decmprs_left;
+ u16 decmprs_wqe_counter;
+
/* control */
struct mlx5_wq_ctrl wq_ctrl;
} ____cacheline_aligned_in_smp;
@@ -616,6 +625,7 @@ void mlx5e_timestamp_init(struct mlx5e_priv *priv);
void mlx5e_timestamp_cleanup(struct mlx5e_priv *priv);
int mlx5e_hwstamp_set(struct net_device *dev, struct ifreq *ifr);
int mlx5e_hwstamp_get(struct net_device *dev, struct ifreq *ifr);
+void mlx5e_modify_rx_cqe_compression(struct mlx5e_priv *priv, bool val);
int mlx5e_vlan_rx_add_vid(struct net_device *dev, __always_unused __be16 proto,
u16 vid);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_clock.c b/drivers/net/ethernet/mellanox/mlx5/core/en_clock.c
index 2018eeb..847a8f3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_clock.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_clock.c
@@ -93,6 +93,8 @@ int mlx5e_hwstamp_set(struct net_device *dev, struct ifreq *ifr)
/* RX HW timestamp */
switch (config.rx_filter) {
case HWTSTAMP_FILTER_NONE:
+ /* Reset CQE compression to Admin default */
+ mlx5e_modify_rx_cqe_compression(priv, priv->params.rx_cqe_compress_admin);
break;
case HWTSTAMP_FILTER_ALL:
case HWTSTAMP_FILTER_SOME:
@@ -108,6 +110,8 @@ int mlx5e_hwstamp_set(struct net_device *dev, struct ifreq *ifr)
case HWTSTAMP_FILTER_PTP_V2_EVENT:
case HWTSTAMP_FILTER_PTP_V2_SYNC:
case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
+ /* Disable CQE compression */
+ mlx5e_modify_rx_cqe_compression(priv, false);
config.rx_filter = HWTSTAMP_FILTER_ALL;
break;
default:
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 1c70e51..0ea4c03 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -114,6 +114,8 @@ static void mlx5e_update_sw_counters(struct mlx5e_priv *priv)
s->rx_mpwqe_filler += rq_stats->mpwqe_filler;
s->rx_mpwqe_frag += rq_stats->mpwqe_frag;
s->rx_buff_alloc_err += rq_stats->buff_alloc_err;
+ s->rx_cqe_compress_blks += rq_stats->cqe_compress_blks;
+ s->rx_cqe_compress_pkts += rq_stats->cqe_compress_pkts;
for (j = 0; j < priv->params.num_tc; j++) {
sq_stats = &priv->channel[i]->sq[j].stats;
@@ -1204,6 +1206,10 @@ static void mlx5e_build_rx_cq_param(struct mlx5e_priv *priv,
}
MLX5_SET(cqc, cqc, log_cq_size, log_cq_size);
+ if (priv->params.rx_cqe_compress) {
+ MLX5_SET(cqc, cqc, mini_cqe_res_format, MLX5_CQE_FORMAT_CSUM);
+ MLX5_SET(cqc, cqc, cqe_comp_en, 1);
+ }
mlx5e_build_common_cq_param(priv, param);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 23adfe2..c8b8d45 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -42,6 +42,143 @@ static inline bool mlx5e_rx_hw_stamp(struct mlx5e_tstamp *tstamp)
return tstamp->hwtstamp_config.rx_filter == HWTSTAMP_FILTER_ALL;
}
+static inline void mlx5e_read_cqe_slot(struct mlx5e_cq *cq, u32 cqcc,
+ void *data)
+{
+ u32 ci = cqcc & cq->wq.sz_m1;
+
+ memcpy(data, mlx5_cqwq_get_wqe(&cq->wq, ci), sizeof(struct mlx5_cqe64));
+}
+
+static inline void mlx5e_read_title_slot(struct mlx5e_rq *rq,
+ struct mlx5e_cq *cq, u32 cqcc)
+{
+ mlx5e_read_cqe_slot(cq, cqcc, &cq->title);
+ cq->decmprs_left = be32_to_cpu(cq->title.byte_cnt);
+ cq->decmprs_wqe_counter = be16_to_cpu(cq->title.wqe_counter);
+ rq->stats.cqe_compress_blks++;
+}
+
+static inline void mlx5e_read_mini_arr_slot(struct mlx5e_cq *cq, u32 cqcc)
+{
+ mlx5e_read_cqe_slot(cq, cqcc, cq->mini_arr);
+ cq->mini_arr_idx = 0;
+}
+
+static inline void mlx5e_cqes_update_owner(struct mlx5e_cq *cq, u32 cqcc, int n)
+{
+ u8 op_own = (cqcc >> cq->wq.log_sz) & 1;
+ u32 wq_sz = 1 << cq->wq.log_sz;
+ u32 ci = cqcc & cq->wq.sz_m1;
+ u32 ci_top = min_t(u32, wq_sz, ci + n);
+
+ for (; ci < ci_top; ci++, n--) {
+ struct mlx5_cqe64 *cqe = mlx5_cqwq_get_wqe(&cq->wq, ci);
+
+ cqe->op_own = op_own;
+ }
+
+ if (unlikely(ci == wq_sz)) {
+ op_own = !op_own;
+ for (ci = 0; ci < n; ci++) {
+ struct mlx5_cqe64 *cqe = mlx5_cqwq_get_wqe(&cq->wq, ci);
+
+ cqe->op_own = op_own;
+ }
+ }
+}
+
+static inline void mlx5e_decompress_cqe(struct mlx5e_rq *rq,
+ struct mlx5e_cq *cq, u32 cqcc)
+{
+ u16 wqe_cnt_step;
+
+ cq->title.byte_cnt = cq->mini_arr[cq->mini_arr_idx].byte_cnt;
+ cq->title.check_sum = cq->mini_arr[cq->mini_arr_idx].checksum;
+ cq->title.op_own &= 0xf0;
+ cq->title.op_own |= 0x01 & (cqcc >> cq->wq.log_sz);
+ cq->title.wqe_counter = cpu_to_be16(cq->decmprs_wqe_counter);
+
+ wqe_cnt_step =
+ rq->wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ ?
+ mpwrq_get_cqe_consumed_strides(&cq->title) : 1;
+ cq->decmprs_wqe_counter =
+ (cq->decmprs_wqe_counter + wqe_cnt_step) & rq->wq.sz_m1;
+}
+
+static inline void mlx5e_decompress_cqe_no_hash(struct mlx5e_rq *rq,
+ struct mlx5e_cq *cq, u32 cqcc)
+{
+ mlx5e_decompress_cqe(rq, cq, cqcc);
+ cq->title.rss_hash_type = 0;
+ cq->title.rss_hash_result = 0;
+}
+
+static inline u32 mlx5e_decompress_cqes_cont(struct mlx5e_rq *rq,
+ struct mlx5e_cq *cq,
+ int update_owner_only,
+ int budget_rem)
+{
+ u32 cqcc = cq->wq.cc + update_owner_only;
+ u32 cqe_count;
+ u32 i;
+
+ cqe_count = min_t(u32, cq->decmprs_left, budget_rem);
+
+ for (i = update_owner_only; i < cqe_count;
+ i++, cq->mini_arr_idx++, cqcc++) {
+ if (unlikely(cq->mini_arr_idx == MLX5_MINI_CQE_ARRAY_SIZE))
+ mlx5e_read_mini_arr_slot(cq, cqcc);
+
+ mlx5e_decompress_cqe_no_hash(rq, cq, cqcc);
+ rq->handle_rx_cqe(rq, &cq->title);
+ }
+ mlx5e_cqes_update_owner(cq, cq->wq.cc, cqcc - cq->wq.cc);
+ cq->wq.cc = cqcc;
+ cq->decmprs_left -= cqe_count;
+ rq->stats.cqe_compress_pkts += cqe_count;
+
+ return cqe_count;
+}
+
+static inline u32 mlx5e_decompress_cqes_start(struct mlx5e_rq *rq,
+ struct mlx5e_cq *cq,
+ int budget_rem)
+{
+ mlx5e_read_title_slot(rq, cq, cq->wq.cc);
+ mlx5e_read_mini_arr_slot(cq, cq->wq.cc + 1);
+ mlx5e_decompress_cqe(rq, cq, cq->wq.cc);
+ rq->handle_rx_cqe(rq, &cq->title);
+ cq->mini_arr_idx++;
+
+ return mlx5e_decompress_cqes_cont(rq, cq, 1, budget_rem) - 1;
+}
+
+void mlx5e_modify_rx_cqe_compression(struct mlx5e_priv *priv, bool val)
+{
+ bool was_opened;
+
+ if (!MLX5_CAP_GEN(priv->mdev, cqe_compression))
+ return;
+
+ mutex_lock(&priv->state_lock);
+
+ if (priv->params.rx_cqe_compress == val)
+ goto unlock;
+
+ was_opened = test_bit(MLX5E_STATE_OPENED, &priv->state);
+ if (was_opened)
+ mlx5e_close_locked(priv->netdev);
+
+ priv->params.rx_cqe_compress = val;
+
+ if (was_opened)
+ mlx5e_open_locked(priv->netdev);
+
+unlock:
+ mutex_unlock(&priv->state_lock);
+}
+
int mlx5e_alloc_rx_wqe(struct mlx5e_rq *rq, struct mlx5e_rx_wqe *wqe, u16 ix)
{
struct sk_buff *skb;
@@ -738,14 +875,24 @@ mpwrq_cqe_out:
int mlx5e_poll_rx_cq(struct mlx5e_cq *cq, int budget)
{
struct mlx5e_rq *rq = container_of(cq, struct mlx5e_rq, cq);
- int work_done;
+ int work_done = 0;
- for (work_done = 0; work_done < budget; work_done++) {
+ if (cq->decmprs_left)
+ work_done += mlx5e_decompress_cqes_cont(rq, cq, 0, budget);
+
+ for (; work_done < budget; work_done++) {
struct mlx5_cqe64 *cqe = mlx5e_get_cqe(cq);
if (!cqe)
break;
+ if (mlx5_get_cqe_format(cqe) == MLX5_COMPRESSED) {
+ work_done +=
+ mlx5e_decompress_cqes_start(rq, cq,
+ budget - work_done);
+ continue;
+ }
+
mlx5_cqwq_pop(&cq->wq);
rq->handle_rx_cqe(rq, cqe);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
index 115752b..83bc32b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_stats.h
@@ -72,6 +72,8 @@ struct mlx5e_sw_stats {
u64 rx_mpwqe_filler;
u64 rx_mpwqe_frag;
u64 rx_buff_alloc_err;
+ u64 rx_cqe_compress_blks;
+ u64 rx_cqe_compress_pkts;
/* Special handling counters */
u64 link_down_events;
@@ -101,6 +103,8 @@ static const struct counter_desc sw_stats_desc[] = {
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_mpwqe_filler) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_mpwqe_frag) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_buff_alloc_err) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_cqe_compress_blks) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, rx_cqe_compress_pkts) },
{ MLX5E_DECLARE_STAT(struct mlx5e_sw_stats, link_down_events) },
};
@@ -283,6 +287,8 @@ struct mlx5e_rq_stats {
u64 mpwqe_filler;
u64 mpwqe_frag;
u64 buff_alloc_err;
+ u64 cqe_compress_blks;
+ u64 cqe_compress_pkts;
};
static const struct counter_desc rq_stats_desc[] = {
@@ -297,6 +303,8 @@ static const struct counter_desc rq_stats_desc[] = {
{ MLX5E_DECLARE_STAT(struct mlx5e_rq_stats, mpwqe_filler) },
{ MLX5E_DECLARE_STAT(struct mlx5e_rq_stats, mpwqe_frag) },
{ MLX5E_DECLARE_STAT(struct mlx5e_rq_stats, buff_alloc_err) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_rq_stats, cqe_compress_blks) },
+ { MLX5E_DECLARE_STAT(struct mlx5e_rq_stats, cqe_compress_pkts) },
};
struct mlx5e_sq_stats {
diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h
index ee0d5a9..035abdf 100644
--- a/include/linux/mlx5/device.h
+++ b/include/linux/mlx5/device.h
@@ -685,6 +685,40 @@ struct mlx5_cqe64 {
u8 op_own;
};
+struct mlx5_mini_cqe8 {
+ union {
+ __be32 rx_hash_result;
+ struct {
+ __be16 checksum;
+ __be16 rsvd;
+ };
+ struct {
+ __be16 wqe_counter;
+ u8 s_wqe_opcode;
+ u8 reserved;
+ } s_wqe_info;
+ };
+ __be32 byte_cnt;
+};
+
+enum {
+ MLX5_NO_INLINE_DATA,
+ MLX5_INLINE_DATA32_SEG,
+ MLX5_INLINE_DATA64_SEG,
+ MLX5_COMPRESSED,
+};
+
+enum {
+ MLX5_CQE_FORMAT_CSUM = 0x1,
+};
+
+#define MLX5_MINI_CQE_ARRAY_SIZE 8
+
+static inline int mlx5_get_cqe_format(struct mlx5_cqe64 *cqe)
+{
+ return (cqe->op_own >> 2) & 0x3;
+}
+
static inline int get_cqe_lro_tcppsh(struct mlx5_cqe64 *cqe)
{
return (cqe->lro_tcppsh_abort_dupack >> 6) & 1;
--
2.8.0
^ permalink raw reply related
* [PATCH net-next 0/3] Mellanox 100G mlx5 CQE compression
From: Saeed Mahameed @ 2016-05-10 21:29 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Or Gerlitz, Tal Alon, Eran Ben Elisha, Saeed Mahameed
Hi Dave,
Introducing ConnectX-4 CQE (Completion Queue Entry) compression feature
for mlx5 etherent driver.
CQE Compressing reduces PCI overhead by coalescing and compressing multiple CQEs into a
single merged CQE. Successful compressing improves message rate especially for small packet
traffic.
CQE Compressing in details:
Instead of writing full CQEs to memory, multiple almost identical CQEs are merged and compressed.
Information that is shared between the CQEs is written once, regardless of the number of
compressed CQEs. In addition, only the unique information (small amount of bytes compared to
full CQE size) is written per CQE.
CQE Compression Block:
This block contains multiple compressed CQEs. CQE Compression Block contains a single copy
of CQEs properties which are shared between all the compressed CQEs (called Title, see below)
and multiple mini CQEs (CQEs in compressed form).
Title:
The Title holds information which is shared between all the compressed CQEs in the CQE Compression
Block. In each Compression Block there is only a single Title regardless of the number
of compressed CQEs.
Mini CQE:
A CQE in compressed form that holds some data needed to extract a single full CQE, for example
8 Bytes instead of 64 Bytes.
The shared information between all compressed CQEs, which belong to the same CQE Compression
Block called Title, is written once, and only the unique information in each compressed
CQE, for example 8 bytes, is written per compressed CQE, called mini CQE.
Since CQE Compression can add overhead to the software (CPU),
it will be only enabled on "weak/slow" PCI slots, where it can actually help.
Applied on top: c047c3b1af62 ('netfilter: conntrack: remove uninitialized shadow variable')
Thanks,
Saeed.
Saeed Mahameed (1):
net/mlx5e: Enable CQE compression when PCI is slower than link
Tariq Toukan (2):
net/mlx5e: CQE compression
net/mlx5e: Expand WQE stride when CQE compression is enabled
drivers/net/ethernet/mellanox/mlx5/core/en.h | 24 ++-
drivers/net/ethernet/mellanox/mlx5/core/en_clock.c | 4 +
.../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 19 +++
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 81 ++++++++-
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 188 ++++++++++++++++++---
drivers/net/ethernet/mellanox/mlx5/core/en_stats.h | 8 +
include/linux/mlx5/device.h | 34 ++++
7 files changed, 328 insertions(+), 30 deletions(-)
--
2.8.0
^ permalink raw reply
* [PATCH net-next 2/3] net/mlx5e: Expand WQE stride when CQE compression is enabled
From: Saeed Mahameed @ 2016-05-10 21:29 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Or Gerlitz, Tal Alon, Eran Ben Elisha, Tariq Toukan,
Saeed Mahameed
In-Reply-To: <1462915756-18180-1-git-send-email-saeedm@mellanox.com>
From: Tariq Toukan <tariqt@mellanox.com>
Make the MPWQE/Striding RQ default configuration dynamic and not
statically set at compile time. Now at driver load we set
stride size and num strides dynamically.
By default we use same values as before, but when CQE compression
is enabled, we set larger stride size to benefit from CQE
compression for larger packets.
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 13 ++++----
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 23 ++++++++++---
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 39 ++++++++++++-----------
3 files changed, 46 insertions(+), 29 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 19f0d8d..e05abad 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -64,12 +64,9 @@
#define MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE_MPW 0x4
#define MLX5E_PARAMS_MAXIMUM_LOG_RQ_SIZE_MPW 0x6
-#define MLX5_MPWRQ_LOG_NUM_STRIDES 11 /* >= 9, HW restriction */
#define MLX5_MPWRQ_LOG_STRIDE_SIZE 6 /* >= 6, HW restriction */
-#define MLX5_MPWRQ_NUM_STRIDES BIT(MLX5_MPWRQ_LOG_NUM_STRIDES)
-#define MLX5_MPWRQ_STRIDE_SIZE BIT(MLX5_MPWRQ_LOG_STRIDE_SIZE)
-#define MLX5_MPWRQ_LOG_WQE_SZ (MLX5_MPWRQ_LOG_NUM_STRIDES +\
- MLX5_MPWRQ_LOG_STRIDE_SIZE)
+#define MLX5_MPWRQ_LOG_STRIDE_SIZE_CQE_COMPRESS 8 /* >= 6, HW restriction */
+#define MLX5_MPWRQ_LOG_WQE_SZ 17
#define MLX5_MPWRQ_WQE_PAGE_ORDER (MLX5_MPWRQ_LOG_WQE_SZ - PAGE_SHIFT > 0 ? \
MLX5_MPWRQ_LOG_WQE_SZ - PAGE_SHIFT : 0)
#define MLX5_MPWRQ_PAGES_PER_WQE BIT(MLX5_MPWRQ_WQE_PAGE_ORDER)
@@ -154,6 +151,8 @@ struct mlx5e_umr_wqe {
struct mlx5e_params {
u8 log_sq_size;
u8 rq_wq_type;
+ u8 mpwqe_log_stride_sz;
+ u8 mpwqe_log_num_strides;
u8 log_rq_size;
u16 num_channels;
u8 num_tc;
@@ -249,6 +248,8 @@ struct mlx5e_rq {
/* control */
struct mlx5_wq_ctrl wq_ctrl;
u8 wq_type;
+ u32 mpwqe_stride_sz;
+ u32 mpwqe_num_strides;
u32 rqn;
struct mlx5e_channel *channel;
struct mlx5e_priv *priv;
@@ -272,7 +273,7 @@ struct mlx5e_mpw_info {
void (*dma_pre_sync)(struct device *pdev,
struct mlx5e_mpw_info *wi,
u32 wqe_offset, u32 len);
- void (*add_skb_frag)(struct device *pdev,
+ void (*add_skb_frag)(struct mlx5e_rq *rq,
struct sk_buff *skb,
struct mlx5e_mpw_info *wi,
u32 page_idx, u32 frag_offset, u32 len);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 0ea4c03..e40e59a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -307,7 +307,9 @@ static int mlx5e_create_rq(struct mlx5e_channel *c,
rq->handle_rx_cqe = mlx5e_handle_rx_cqe_mpwrq;
rq->alloc_wqe = mlx5e_alloc_rx_mpwqe;
- rq->wqe_sz = MLX5_MPWRQ_NUM_STRIDES * MLX5_MPWRQ_STRIDE_SIZE;
+ rq->mpwqe_stride_sz = BIT(priv->params.mpwqe_log_stride_sz);
+ rq->mpwqe_num_strides = BIT(priv->params.mpwqe_log_num_strides);
+ rq->wqe_sz = rq->mpwqe_stride_sz * rq->mpwqe_num_strides;
byte_count = rq->wqe_sz;
break;
default: /* MLX5_WQ_TYPE_LINKED_LIST */
@@ -1130,9 +1132,9 @@ static void mlx5e_build_rq_param(struct mlx5e_priv *priv,
switch (priv->params.rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
MLX5_SET(wq, wq, log_wqe_num_of_strides,
- MLX5_MPWRQ_LOG_NUM_STRIDES - 9);
+ priv->params.mpwqe_log_num_strides - 9);
MLX5_SET(wq, wq, log_wqe_stride_size,
- MLX5_MPWRQ_LOG_STRIDE_SIZE - 6);
+ priv->params.mpwqe_log_stride_sz - 6);
MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ);
break;
default: /* MLX5_WQ_TYPE_LINKED_LIST */
@@ -1199,7 +1201,7 @@ static void mlx5e_build_rx_cq_param(struct mlx5e_priv *priv,
switch (priv->params.rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
log_cq_size = priv->params.log_rq_size +
- MLX5_MPWRQ_LOG_NUM_STRIDES;
+ priv->params.mpwqe_log_num_strides;
break;
default: /* MLX5_WQ_TYPE_LINKED_LIST */
log_cq_size = priv->params.log_rq_size;
@@ -2729,12 +2731,25 @@ static void mlx5e_build_netdev_priv(struct mlx5_core_dev *mdev,
switch (priv->params.rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
priv->params.log_rq_size = MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE_MPW;
+ priv->params.mpwqe_log_stride_sz =
+ priv->params.rx_cqe_compress ?
+ MLX5_MPWRQ_LOG_STRIDE_SIZE_CQE_COMPRESS :
+ MLX5_MPWRQ_LOG_STRIDE_SIZE;
+ priv->params.mpwqe_log_num_strides = MLX5_MPWRQ_LOG_WQE_SZ -
+ priv->params.mpwqe_log_stride_sz;
priv->params.lro_en = true;
break;
default: /* MLX5_WQ_TYPE_LINKED_LIST */
priv->params.log_rq_size = MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE;
}
+ mlx5_core_info(mdev,
+ "MLX5E: StrdRq(%d) RqSz(%ld) StrdSz(%ld) RxCqeCmprss(%d)\n",
+ priv->params.rq_wq_type == MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ,
+ BIT(priv->params.log_rq_size),
+ BIT(priv->params.mpwqe_log_stride_sz),
+ priv->params.rx_cqe_compress_admin);
+
priv->params.min_rx_wqes = mlx5_min_rx_wqes(priv->params.rq_wq_type,
BIT(priv->params.log_rq_size));
priv->params.rx_cq_moderation_usec =
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index c8b8d45..f345679 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -127,7 +127,7 @@ static inline u32 mlx5e_decompress_cqes_cont(struct mlx5e_rq *rq,
for (i = update_owner_only; i < cqe_count;
i++, cq->mini_arr_idx++, cqcc++) {
- if (unlikely(cq->mini_arr_idx == MLX5_MINI_CQE_ARRAY_SIZE))
+ if (cq->mini_arr_idx == MLX5_MINI_CQE_ARRAY_SIZE)
mlx5e_read_mini_arr_slot(cq, cqcc);
mlx5e_decompress_cqe_no_hash(rq, cq, cqcc);
@@ -212,6 +212,11 @@ err_free_skb:
return -ENOMEM;
}
+static inline int mlx5e_mpwqe_strides_per_page(struct mlx5e_rq *rq)
+{
+ return rq->mpwqe_num_strides >> MLX5_MPWRQ_WQE_PAGE_ORDER;
+}
+
static inline void
mlx5e_dma_pre_sync_linear_mpwqe(struct device *pdev,
struct mlx5e_mpw_info *wi,
@@ -230,13 +235,13 @@ mlx5e_dma_pre_sync_fragmented_mpwqe(struct device *pdev,
}
static inline void
-mlx5e_add_skb_frag_linear_mpwqe(struct device *pdev,
+mlx5e_add_skb_frag_linear_mpwqe(struct mlx5e_rq *rq,
struct sk_buff *skb,
struct mlx5e_mpw_info *wi,
u32 page_idx, u32 frag_offset,
u32 len)
{
- unsigned int truesize = ALIGN(len, MLX5_MPWRQ_STRIDE_SIZE);
+ unsigned int truesize = ALIGN(len, rq->mpwqe_stride_sz);
wi->skbs_frags[page_idx]++;
skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
@@ -245,15 +250,15 @@ mlx5e_add_skb_frag_linear_mpwqe(struct device *pdev,
}
static inline void
-mlx5e_add_skb_frag_fragmented_mpwqe(struct device *pdev,
+mlx5e_add_skb_frag_fragmented_mpwqe(struct mlx5e_rq *rq,
struct sk_buff *skb,
struct mlx5e_mpw_info *wi,
u32 page_idx, u32 frag_offset,
u32 len)
{
- unsigned int truesize = ALIGN(len, MLX5_MPWRQ_STRIDE_SIZE);
+ unsigned int truesize = ALIGN(len, rq->mpwqe_stride_sz);
- dma_sync_single_for_cpu(pdev,
+ dma_sync_single_for_cpu(rq->pdev,
wi->umr.dma_info[page_idx].addr + frag_offset,
len, DMA_FROM_DEVICE);
wi->skbs_frags[page_idx]++;
@@ -293,7 +298,6 @@ mlx5e_copy_skb_header_fragmented_mpwqe(struct device *pdev,
skb_copy_to_linear_data_offset(skb, 0,
page_address(dma_info->page) + offset,
len);
-#if (MLX5_MPWRQ_SMALL_PACKET_THRESHOLD >= MLX5_MPWRQ_STRIDE_SIZE)
if (unlikely(offset + headlen > PAGE_SIZE)) {
dma_info++;
headlen_pg = len;
@@ -304,7 +308,6 @@ mlx5e_copy_skb_header_fragmented_mpwqe(struct device *pdev,
page_address(dma_info->page),
len);
}
-#endif
}
static u16 mlx5e_get_wqe_mtt_offset(u16 rq_ix, u16 wqe_ix)
@@ -430,7 +433,7 @@ static int mlx5e_alloc_rx_fragmented_mpwqe(struct mlx5e_rq *rq,
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
if (unlikely(mlx5e_alloc_and_map_page(rq, wi, i)))
goto err_unmap;
- atomic_add(MLX5_MPWRQ_STRIDES_PER_PAGE,
+ atomic_add(mlx5e_mpwqe_strides_per_page(rq),
&wi->umr.dma_info[i].page->_count);
wi->skbs_frags[i] = 0;
}
@@ -449,7 +452,7 @@ err_unmap:
while (--i >= 0) {
dma_unmap_page(rq->pdev, wi->umr.dma_info[i].addr, PAGE_SIZE,
PCI_DMA_FROMDEVICE);
- atomic_sub(MLX5_MPWRQ_STRIDES_PER_PAGE,
+ atomic_sub(mlx5e_mpwqe_strides_per_page(rq),
&wi->umr.dma_info[i].page->_count);
put_page(wi->umr.dma_info[i].page);
}
@@ -474,7 +477,7 @@ void mlx5e_free_rx_fragmented_mpwqe(struct mlx5e_rq *rq,
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
dma_unmap_page(rq->pdev, wi->umr.dma_info[i].addr, PAGE_SIZE,
PCI_DMA_FROMDEVICE);
- atomic_sub(MLX5_MPWRQ_STRIDES_PER_PAGE - wi->skbs_frags[i],
+ atomic_sub(mlx5e_mpwqe_strides_per_page(rq) - wi->skbs_frags[i],
&wi->umr.dma_info[i].page->_count);
put_page(wi->umr.dma_info[i].page);
}
@@ -524,7 +527,7 @@ static int mlx5e_alloc_rx_linear_mpwqe(struct mlx5e_rq *rq,
*/
split_page(wi->dma_info.page, MLX5_MPWRQ_WQE_PAGE_ORDER);
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
- atomic_add(MLX5_MPWRQ_STRIDES_PER_PAGE,
+ atomic_add(mlx5e_mpwqe_strides_per_page(rq),
&wi->dma_info.page[i]._count);
wi->skbs_frags[i] = 0;
}
@@ -548,7 +551,7 @@ void mlx5e_free_rx_linear_mpwqe(struct mlx5e_rq *rq,
dma_unmap_page(rq->pdev, wi->dma_info.addr, rq->wqe_sz,
PCI_DMA_FROMDEVICE);
for (i = 0; i < MLX5_MPWRQ_PAGES_PER_WQE; i++) {
- atomic_sub(MLX5_MPWRQ_STRIDES_PER_PAGE - wi->skbs_frags[i],
+ atomic_sub(mlx5e_mpwqe_strides_per_page(rq) - wi->skbs_frags[i],
&wi->dma_info.page[i]._count);
put_page(&wi->dma_info.page[i]);
}
@@ -793,9 +796,9 @@ static inline void mlx5e_mpwqe_fill_rx_skb(struct mlx5e_rq *rq,
u32 cqe_bcnt,
struct sk_buff *skb)
{
- u32 consumed_bytes = ALIGN(cqe_bcnt, MLX5_MPWRQ_STRIDE_SIZE);
+ u32 consumed_bytes = ALIGN(cqe_bcnt, rq->mpwqe_stride_sz);
u16 stride_ix = mpwrq_get_cqe_stride_index(cqe);
- u32 wqe_offset = stride_ix * MLX5_MPWRQ_STRIDE_SIZE;
+ u32 wqe_offset = stride_ix * rq->mpwqe_stride_sz;
u32 head_offset = wqe_offset & (PAGE_SIZE - 1);
u32 page_idx = wqe_offset >> PAGE_SHIFT;
u32 head_page_idx = page_idx;
@@ -803,19 +806,17 @@ static inline void mlx5e_mpwqe_fill_rx_skb(struct mlx5e_rq *rq,
u32 frag_offset = head_offset + headlen;
u16 byte_cnt = cqe_bcnt - headlen;
-#if (MLX5_MPWRQ_SMALL_PACKET_THRESHOLD >= MLX5_MPWRQ_STRIDE_SIZE)
if (unlikely(frag_offset >= PAGE_SIZE)) {
page_idx++;
frag_offset -= PAGE_SIZE;
}
-#endif
wi->dma_pre_sync(rq->pdev, wi, wqe_offset, consumed_bytes);
while (byte_cnt) {
u32 pg_consumed_bytes =
min_t(u32, PAGE_SIZE - frag_offset, byte_cnt);
- wi->add_skb_frag(rq->pdev, skb, wi, page_idx, frag_offset,
+ wi->add_skb_frag(rq, skb, wi, page_idx, frag_offset,
pg_consumed_bytes);
byte_cnt -= pg_consumed_bytes;
frag_offset = 0;
@@ -865,7 +866,7 @@ void mlx5e_handle_rx_cqe_mpwrq(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
mlx5e_complete_rx_cqe(rq, cqe, cqe_bcnt, skb);
mpwrq_cqe_out:
- if (likely(wi->consumed_strides < MLX5_MPWRQ_NUM_STRIDES))
+ if (likely(wi->consumed_strides < rq->mpwqe_num_strides))
return;
wi->free_wqe(rq, wi);
--
2.8.0
^ permalink raw reply related
* [PATCH net-next 3/3] net/mlx5e: Enable CQE compression when PCI is slower than link
From: Saeed Mahameed @ 2016-05-10 21:29 UTC (permalink / raw)
To: David S. Miller
Cc: netdev, Or Gerlitz, Tal Alon, Eran Ben Elisha, Saeed Mahameed,
Tariq Toukan
In-Reply-To: <1462915756-18180-1-git-send-email-saeedm@mellanox.com>
We turn the feature ON, only for servers with PCI BW < MAX LINK BW, as it
helps reducing PCI pressure on weak PCI slots, but it adds some software
overhead.
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 +
.../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 19 ++++++++
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 52 ++++++++++++++++++++++
3 files changed, 72 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index e05abad..e8a6c33 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -645,6 +645,7 @@ int mlx5e_close_locked(struct net_device *netdev);
void mlx5e_build_default_indir_rqt(struct mlx5_core_dev *mdev,
u32 *indirection_rqt, int len,
int num_channels);
+int mlx5e_get_max_linkspeed(struct mlx5_core_dev *mdev, u32 *speed);
static inline void mlx5e_tx_notify_hw(struct mlx5e_sq *sq,
struct mlx5_wqe_ctrl_seg *ctrl, int bf_sz)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 534d99e..fc7dcc0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -613,6 +613,25 @@ static u32 ptys2ethtool_supported_port(u32 eth_proto_cap)
return 0;
}
+int mlx5e_get_max_linkspeed(struct mlx5_core_dev *mdev, u32 *speed)
+{
+ u32 max_speed = 0;
+ u32 proto_cap;
+ int err;
+ int i;
+
+ err = mlx5_query_port_proto_cap(mdev, &proto_cap, MLX5_PTYS_EN);
+ if (err)
+ return err;
+
+ for (i = 0; i < MLX5E_LINK_MODES_NUMBER; ++i)
+ if (proto_cap & MLX5E_PROT_MASK(i))
+ max_speed = max(max_speed, ptys2ethtool_table[i].speed);
+
+ *speed = max_speed;
+ return 0;
+}
+
static void get_speed_duplex(struct net_device *netdev,
u32 eth_proto_oper,
struct ethtool_cmd *cmd)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index e40e59a..0804070 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -2716,11 +2716,49 @@ static bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev)
MLX5_CAP_ETH(mdev, reg_umr_sq);
}
+static int mlx5e_get_pci_bw(struct mlx5_core_dev *mdev, u32 *pci_bw)
+{
+ enum pcie_link_width width;
+ enum pci_bus_speed speed;
+ int err = 0;
+
+ err = pcie_get_minimum_link(mdev->pdev, &speed, &width);
+ if (err)
+ return err;
+
+ if (speed == PCI_SPEED_UNKNOWN || width == PCIE_LNK_WIDTH_UNKNOWN)
+ return -EINVAL;
+
+ switch (speed) {
+ case PCIE_SPEED_2_5GT:
+ *pci_bw = 2500 * width;
+ break;
+ case PCIE_SPEED_5_0GT:
+ *pci_bw = 5000 * width;
+ break;
+ case PCIE_SPEED_8_0GT:
+ *pci_bw = 8000 * width;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static bool cqe_compress_heuristic(u32 link_speed, u32 pci_bw)
+{
+ return (link_speed && pci_bw &&
+ (pci_bw < 40000) && (pci_bw < link_speed));
+}
+
static void mlx5e_build_netdev_priv(struct mlx5_core_dev *mdev,
struct net_device *netdev,
int num_channels)
{
struct mlx5e_priv *priv = netdev_priv(netdev);
+ u32 link_speed = 0;
+ u32 pci_bw = 0;
priv->params.log_sq_size =
MLX5E_PARAMS_DEFAULT_LOG_SQ_SIZE;
@@ -2728,6 +2766,20 @@ static void mlx5e_build_netdev_priv(struct mlx5_core_dev *mdev,
MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ :
MLX5_WQ_TYPE_LINKED_LIST;
+ /* set CQE compression */
+ priv->params.rx_cqe_compress_admin = false;
+ if (MLX5_CAP_GEN(mdev, cqe_compression) &&
+ MLX5_CAP_GEN(mdev, vport_group_manager)) {
+ mlx5e_get_max_linkspeed(mdev, &link_speed);
+ mlx5e_get_pci_bw(mdev, &pci_bw);
+ mlx5_core_dbg(mdev, "Max link speed = %d, PCI BW = %d\n",
+ link_speed, pci_bw);
+ priv->params.rx_cqe_compress_admin =
+ cqe_compress_heuristic(link_speed, pci_bw);
+ }
+
+ priv->params.rx_cqe_compress = priv->params.rx_cqe_compress_admin;
+
switch (priv->params.rq_wq_type) {
case MLX5_WQ_TYPE_LINKED_LIST_STRIDING_RQ:
priv->params.log_rq_size = MLX5E_PARAMS_DEFAULT_LOG_RQ_SIZE_MPW;
--
2.8.0
^ permalink raw reply related
* Re: [RFC PATCH 0/2] net: threadable napi poll loop
From: Rik van Riel @ 2016-05-10 21:35 UTC (permalink / raw)
To: Eric Dumazet, Eric Dumazet
Cc: Hannes Frederic Sowa, Paolo Abeni, netdev, David S. Miller,
Jiri Pirko, Daniel Borkmann, Alexei Starovoitov, Alexander Duyck,
Tom Herbert, Peter Zijlstra, Ingo Molnar, LKML
In-Reply-To: <1462915900.23934.99.camel@edumazet-glaptop3.roam.corp.google.com>
[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]
On Tue, 2016-05-10 at 14:31 -0700, Eric Dumazet wrote:
> On Tue, 2016-05-10 at 14:09 -0700, Eric Dumazet wrote:
> >
> > On Tue, May 10, 2016 at 1:46 PM, Hannes Frederic Sowa
> > <hannes@stressinduktion.org> wrote:
> >
> > >
> > > I agree here, but I don't think this patch particularly is a lot
> > > of
> > > bloat and something very interesting people can play with and
> > > extend upon.
> > >
> > Sure, very rarely patch authors think their stuff is bloat.
> >
> > I prefer to fix kernel softirq.c, or at least show me that you
> > tried
> > hard enough.
> >
> > I am pretty sure that the following would work :
> >
> > When ksoftirqd is scheduled, remember this in a per cpu variable
> > (ksoftiqd_scheduled)
> >
> > When enabling BH , do not call do_softirq() if this variable is
> > set.
> >
> > ksoftirqd would clear the variable at the right place (probably in
> > run_ksoftirqd())
> >
> > Sure, this might add a lot of latency regressions, but lets fix
> > them.
> Only to give the idea (it is completely untested and probably buggy)
>
> diff --git a/kernel/softirq.c b/kernel/softirq.c
> index 17caf4b63342..cb30cfd76687 100644
> --- a/kernel/softirq.c
> +++ b/kernel/softirq.c
> @@ -56,6 +56,7 @@ EXPORT_SYMBOL(irq_stat);
> static struct softirq_action softirq_vec[NR_SOFTIRQS]
> __cacheline_aligned_in_smp;
>
> DEFINE_PER_CPU(struct task_struct *, ksoftirqd);
> +DEFINE_PER_CPU(bool, ksoftirqd_scheduled);
>
> const char * const softirq_to_name[NR_SOFTIRQS] = {
> "HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL",
> @@ -73,8 +74,10 @@ static void wakeup_softirqd(void)
> /* Interrupts are disabled: no need to stop preemption */
> struct task_struct *tsk = __this_cpu_read(ksoftirqd);
>
> - if (tsk && tsk->state != TASK_RUNNING)
> + if (tsk && tsk->state != TASK_RUNNING) {
> + __this_cpu_write(ksoftirqd_scheduled, true);
> wake_up_process(tsk);
> + }
> }
>
> /*
> @@ -162,7 +165,9 @@ void __local_bh_enable_ip(unsigned long ip,
> unsigned int cnt)
> */
> preempt_count_sub(cnt - 1);
>
> - if (unlikely(!in_interrupt() && local_softirq_pending())) {
> + if (unlikely(!in_interrupt() &&
> + local_softirq_pending() &&
> + !__this_cpu_read(ksoftirqd_scheduled))) {
> /*
>
You might need another one of these in invoke_softirq()
--
All Rights Reversed.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply
* Re: [RFC PATCH 0/2] net: threadable napi poll loop
From: Eric Dumazet @ 2016-05-10 21:53 UTC (permalink / raw)
To: Rik van Riel
Cc: Eric Dumazet, Hannes Frederic Sowa, Paolo Abeni, netdev,
David S. Miller, Jiri Pirko, Daniel Borkmann, Alexei Starovoitov,
Alexander Duyck, Tom Herbert, Peter Zijlstra, Ingo Molnar, LKML
In-Reply-To: <1462916121.16365.14.camel@redhat.com>
On Tue, 2016-05-10 at 17:35 -0400, Rik van Riel wrote:
> You might need another one of these in invoke_softirq()
>
Excellent.
I gave it a quick try (without your suggestion), and host seems to
survive a stress test.
Of course we do have to fix these problems :
[ 147.781629] NOHZ: local_softirq_pending 48
[ 147.785546] NOHZ: local_softirq_pending 48
[ 147.788344] NOHZ: local_softirq_pending 48
[ 147.788992] NOHZ: local_softirq_pending 48
[ 147.790943] NOHZ: local_softirq_pending 48
[ 147.791232] NOHZ: local_softirq_pending 24a
[ 147.791258] NOHZ: local_softirq_pending 48
[ 147.791366] NOHZ: local_softirq_pending 48
[ 147.792118] NOHZ: local_softirq_pending 48
[ 147.793428] NOHZ: local_softirq_pending 48
Thanks.
^ permalink raw reply
* Re: [RFC PATCH 0/2] net: threadable napi poll loop
From: Eric Dumazet @ 2016-05-10 22:02 UTC (permalink / raw)
To: Rik van Riel
Cc: Eric Dumazet, Hannes Frederic Sowa, Paolo Abeni, netdev,
David S. Miller, Jiri Pirko, Daniel Borkmann, Alexei Starovoitov,
Alexander Duyck, Tom Herbert, Peter Zijlstra, Ingo Molnar, LKML
In-Reply-To: <1462917217.23934.101.camel@edumazet-glaptop3.roam.corp.google.com>
On Tue, 2016-05-10 at 14:53 -0700, Eric Dumazet wrote:
> On Tue, 2016-05-10 at 17:35 -0400, Rik van Riel wrote:
>
> > You might need another one of these in invoke_softirq()
> >
>
> Excellent.
>
> I gave it a quick try (without your suggestion), and host seems to
> survive a stress test.
>
> Of course we do have to fix these problems :
>
> [ 147.781629] NOHZ: local_softirq_pending 48
> [ 147.785546] NOHZ: local_softirq_pending 48
> [ 147.788344] NOHZ: local_softirq_pending 48
> [ 147.788992] NOHZ: local_softirq_pending 48
> [ 147.790943] NOHZ: local_softirq_pending 48
> [ 147.791232] NOHZ: local_softirq_pending 24a
> [ 147.791258] NOHZ: local_softirq_pending 48
> [ 147.791366] NOHZ: local_softirq_pending 48
> [ 147.792118] NOHZ: local_softirq_pending 48
> [ 147.793428] NOHZ: local_softirq_pending 48
Well, with your suggestion, these warnings disappear ;)
^ permalink raw reply
* Re: [RFC PATCH 0/2] net: threadable napi poll loop
From: Rik van Riel @ 2016-05-10 22:02 UTC (permalink / raw)
To: Eric Dumazet
Cc: Eric Dumazet, Hannes Frederic Sowa, Paolo Abeni, netdev,
David S. Miller, Jiri Pirko, Daniel Borkmann, Alexei Starovoitov,
Alexander Duyck, Tom Herbert, Peter Zijlstra, Ingo Molnar, LKML
In-Reply-To: <1462917217.23934.101.camel@edumazet-glaptop3.roam.corp.google.com>
[-- Attachment #1: Type: text/plain, Size: 1001 bytes --]
On Tue, 2016-05-10 at 14:53 -0700, Eric Dumazet wrote:
> On Tue, 2016-05-10 at 17:35 -0400, Rik van Riel wrote:
>
> >
> > You might need another one of these in invoke_softirq()
> >
> Excellent.
>
> I gave it a quick try (without your suggestion), and host seems to
> survive a stress test.
>
> Of course we do have to fix these problems :
>
> [ 147.781629] NOHZ: local_softirq_pending 48
> [ 147.785546] NOHZ: local_softirq_pending 48
> [ 147.788344] NOHZ: local_softirq_pending 48
> [ 147.788992] NOHZ: local_softirq_pending 48
> [ 147.790943] NOHZ: local_softirq_pending 48
> [ 147.791232] NOHZ: local_softirq_pending 24a
> [ 147.791258] NOHZ: local_softirq_pending 48
> [ 147.791366] NOHZ: local_softirq_pending 48
> [ 147.792118] NOHZ: local_softirq_pending 48
> [ 147.793428] NOHZ: local_softirq_pending 48
As long as ksoftirqd is running, that should not be
an actual problem, just a false positive.
--
All Rights Reversed.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply
* Re: [RFC PATCH 0/2] net: threadable napi poll loop
From: Hannes Frederic Sowa @ 2016-05-10 22:32 UTC (permalink / raw)
To: Eric Dumazet
Cc: Eric Dumazet, Paolo Abeni, netdev, David S. Miller, Jiri Pirko,
Daniel Borkmann, Alexei Starovoitov, Alexander Duyck, Tom Herbert,
Peter Zijlstra, Ingo Molnar, Rik van Riel, LKML
In-Reply-To: <CANn89i+YKrVOgfFwW-2psfCzq5_YLdmbV06-H-1FCOwXdMBuzQ@mail.gmail.com>
On 10.05.2016 23:09, Eric Dumazet wrote:
> On Tue, May 10, 2016 at 1:46 PM, Hannes Frederic Sowa
> <hannes@stressinduktion.org> wrote:
>
>> I agree here, but I don't think this patch particularly is a lot of
>> bloat and something very interesting people can play with and extend upon.
>>
>
> Sure, very rarely patch authors think their stuff is bloat.
>
> I prefer to fix kernel softirq.c, or at least show me that you tried
> hard enough.
>
> I am pretty sure that the following would work :
>
> When ksoftirqd is scheduled, remember this in a per cpu variable
> (ksoftiqd_scheduled)
>
> When enabling BH , do not call do_softirq() if this variable is set.
>
> ksoftirqd would clear the variable at the right place (probably in
> run_ksoftirqd())
>
> Sure, this might add a lot of latency regressions, but lets fix them.
Probably, yes.
We had a version which limited the number of restarts if softirqs were
invoked from local_bh_enable (so that at least timers etc. would run)
and would defer all other work to ksoftirqd. That also solved the
initial live lock problem. I do have concerns about the fairness of this
approach, but we now have to investigate this. ;)
Not only did we want to present this solely as a bugfix but also as as
performance enhancements in case of virtio (as you can see in the cover
letter). Given that a long time ago there was a tendency to remove
softirqs completely, we thought it might be very interesting, that a
threaded napi in general seems to be absolutely viable nowadays and
might offer new features.
Bye,
Hannes
^ permalink raw reply
* Re: [RFC PATCH 0/2] net: threadable napi poll loop
From: Eric Dumazet @ 2016-05-10 22:44 UTC (permalink / raw)
To: Rik van Riel
Cc: Eric Dumazet, Hannes Frederic Sowa, Paolo Abeni, netdev,
David S. Miller, Jiri Pirko, Daniel Borkmann, Alexei Starovoitov,
Alexander Duyck, Tom Herbert, Peter Zijlstra, Ingo Molnar, LKML
In-Reply-To: <1462917747.23934.102.camel@edumazet-glaptop3.roam.corp.google.com>
On Tue, 2016-05-10 at 15:02 -0700, Eric Dumazet wrote:
> On Tue, 2016-05-10 at 14:53 -0700, Eric Dumazet wrote:
> > On Tue, 2016-05-10 at 17:35 -0400, Rik van Riel wrote:
> >
> > > You might need another one of these in invoke_softirq()
> > >
> >
> > Excellent.
> >
> > I gave it a quick try (without your suggestion), and host seems to
> > survive a stress test.
> >
> > Of course we do have to fix these problems :
> >
> > [ 147.781629] NOHZ: local_softirq_pending 48
> > [ 147.785546] NOHZ: local_softirq_pending 48
> > [ 147.788344] NOHZ: local_softirq_pending 48
> > [ 147.788992] NOHZ: local_softirq_pending 48
> > [ 147.790943] NOHZ: local_softirq_pending 48
> > [ 147.791232] NOHZ: local_softirq_pending 24a
> > [ 147.791258] NOHZ: local_softirq_pending 48
> > [ 147.791366] NOHZ: local_softirq_pending 48
> > [ 147.792118] NOHZ: local_softirq_pending 48
> > [ 147.793428] NOHZ: local_softirq_pending 48
>
>
> Well, with your suggestion, these warnings disappear ;)
This is really nice.
Under stress number of context switches is really small.
ksoftirqd and my netserver compete equally to get the cpu cycles (on
CPU0)
lpaa23:~# vmstat 1 10
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 0 0 260668416 37240 2414428 0 0 21 0 329 349 0 3 96 0
1 0 0 260667904 37240 2414428 0 0 0 12 193126 1050 0 2 98 0
1 0 0 260667904 37240 2414428 0 0 0 0 194354 1056 0 2 98 0
1 0 0 260669104 37240 2414492 0 0 0 0 200897 1095 0 2 98 0
1 0 0 260668592 37240 2414492 0 0 0 0 205731 964 0 2 98 0
1 0 0 260678832 37240 2414492 0 0 0 0 201689 981 0 2 98 0
1 0 0 260678832 37240 2414492 0 0 0 0 204899 742 0 2 98 0
1 0 0 260678320 37240 2414492 0 0 0 0 199148 792 0 3 97 0
1 0 0 260678832 37240 2414492 0 0 0 0 196398 766 0 2 98 0
1 0 0 260678832 37240 2414492 0 0 0 0 201930 858 0 2 98 0
And we can see that ksoftirqd/0 runs for longer periods (~500 usec),
instead of stupid 4 usec before the patch. Less overhead.
lpaa23:~# cat /proc/3/sched
ksoftirqd/0 (3, #threads: 1)
-------------------------------------------------------------------
se.exec_start : 1552401.399526
se.vruntime : 237599.421560
se.sum_exec_runtime : 75432.494199
se.nr_migrations : 0
nr_switches : 144333
nr_voluntary_switches : 143828
nr_involuntary_switches : 505
se.load.weight : 1024
se.avg.load_sum : 10445
se.avg.util_sum : 10445
se.avg.load_avg : 0
se.avg.util_avg : 0
se.avg.last_update_time : 1552401399526
policy : 0
prio : 120
clock-delta : 47
lpaa23:~# echo 75432.494199/144333|bc -l
.52262818758703830724
And yes indeed, user space can progress way faster under flood.
lpaa23:~# nstat >/dev/null;sleep 1;nstat | grep Udp
UdpInDatagrams 186132 0.0
UdpInErrors 735462 0.0
UdpOutDatagrams 10 0.0
UdpRcvbufErrors 735461 0.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox