From: Herve Codina <herve.codina@bootlin.com>
To: Daniel Machon <daniel.machon@microchip.com>
Cc: Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Horatiu Vultur <horatiu.vultur@microchip.com>,
Steen Hegelund <steen.hegelund@microchip.com>,
<UNGLinuxDriver@microchip.com>,
"Alexei Starovoitov" <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
"Jesper Dangaard Brouer" <hawk@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
Stanislav Fomichev <sdf@fomichev.me>,
Arnd Bergmann <arnd@arndb.de>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
<netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
<bpf@vger.kernel.org>
Subject: Re: [PATCH net-next 00/10] net: lan966x: add support for PCIe FDMA
Date: Wed, 8 Apr 2026 11:51:26 +0200 [thread overview]
Message-ID: <20260408115126.5529c823@bootlin.com> (raw)
In-Reply-To: <20260407132016.4cfivs24ljqneyu7@DEN-DL-M70577>
Hi Daniel,
On Tue, 7 Apr 2026 15:20:16 +0200
Daniel Machon <daniel.machon@microchip.com> wrote:
...
>
> The following diff should fix the FDMA traffic issue, and the FDMA error splat,
> when reloading the lan966x-pci driver, by:
>
> 1. Resetting the FDMA engine on PCI init()
>
> 2. Clearing any rogue FDMA errors that may latch due to the soft reset by the
> reset driver.
>
> diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c
> b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c
> --- a/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c
> +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_fdma_pci.c
> @@ -372,6 +372,9 @@ static int lan966x_fdma_pci_init(struct lan966x *lan966x)
> if (!lan966x->fdma)
> return 0;
>
> + lan_wr(FDMA_CTRL_NRESET_SET(0), lan966x, FDMA_CTRL);
> + lan_wr(FDMA_CTRL_NRESET_SET(1), lan966x, FDMA_CTRL);
> +
> fdma_pci_atu_init(&lan966x->atu, lan966x->regs[TARGET_PCIE_DBI]);
>
> lan966x->rx.lan966x = lan966x;
> diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
> b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
> --- a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
> +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
> @@ -1071,6 +1071,15 @@ static int lan966x_reset_switch(struct lan966x *lan966x)
>
> reset_control_reset(switch_reset);
>
> + /* When in PCI mode, the GCB soft reset issued by the reset
> + * controller can latch spurious bits in the FDMA error stickies.
> + * Clear them before request_irq hooks up the FDMA IRQ line,
> + * otherwise the handler fires immediately on probe.
> + */
> + lan_wr(lan_rd(lan966x, FDMA_ERRORS), lan966x, FDMA_ERRORS);
> + lan_wr(lan_rd(lan966x, FDMA_INTR_ERR), lan966x, FDMA_INTR_ERR);
> + lan_wr(lan_rd(lan966x, FDMA_INTR_DB), lan966x, FDMA_INTR_DB);
> +
> /* Don't reinitialize the switch core, if it is already initialized. In
> * case it is initialized twice, some pointers inside the queue system
> * in HW will get corrupted and then after a while the queue system gets
> diff --git a/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h
> b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h
> --- a/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h
> +++ b/drivers/net/ethernet/microchip/lan966x/lan966x_regs.h
> @@ -1010,6 +1010,15 @@ enum lan966x_target {
> #define FDMA_CH_CFG_CH_MEM_GET(x)\
> FIELD_GET(FDMA_CH_CFG_CH_MEM, x)
>
> +/* FDMA:FDMA:FDMA_CTRL */
> +#define FDMA_CTRL __REG(TARGET_FDMA, 0, 1, 8, 0, 1, 428, 424, 0, 1, 4)
> +
> +#define FDMA_CTRL_NRESET BIT(0)
> +#define FDMA_CTRL_NRESET_SET(x)\
> + FIELD_PREP(FDMA_CTRL_NRESET, x)
> +#define FDMA_CTRL_NRESET_GET(x)\
> + FIELD_GET(FDMA_CTRL_NRESET, x)
> +
> /* FDMA:FDMA:FDMA_PORT_CTRL */
> #define FDMA_PORT_CTRL(r) __REG(TARGET_FDMA, 0, 1, 8, 0, 1, 428, 376, r, 2, 4)
>
> Let me know if it works on your end.
>
> (Btw. I have noticed another issue where TX stops working on lan966x-pci reload.
> It happens more rarely, but is unrelated to this patch series, as it also
> happens in register-based INJ/XTR mode. Whenever that happens, you will see
> "Flush timeout chip port" in the logs. This should also be fixed, but sent as a
> separate fix commit, I believe.)
>
I have tested your proposed modification.
- From a cold boot, module unloading / re-loading:
Tested Ok
- From a cold boot, module loading / no interface configuration / reboot:
Tested Ok
- From a cold boot, module loading / interface configuration / reboot:
Issue present.
- From a cold boot, module loading / interface configuration / module unloading / reboot:
Tested Ok.
The interface configuration was the following command:
ip link set eth2 up && ip addr add 192.168.32.20/16 dev eth2
With the interface configured and without unloading the lan966x_pci module
before the reboot, I have the already known FDMA error splat after the
reboot:
[ 18.908397] ------------[ cut here ]------------
[ 18.913226] Unexpected error: 64, error_type: 1073741824
[ 18.918704] WARNING: drivers/net/ethernet/microchip/lan966x/lan966x_fdma.c:558 at lan966x_fdma_irq_handler+0xe0/0x12c [lan966x_switch], CPU#0: kworker/u8:2/38
...
The issue is not present if I unload the lan966x_pci module before the
reboot. This let me think that something has been cleaned by the module
unloading.
On top of your proposed modification, I use the shutdown() hook in the
lan966x_driver:
--- a/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
+++ b/drivers/net/ethernet/microchip/lan966x/lan966x_main.c
@@ -1326,9 +1326,18 @@ static void lan966x_remove(struct platform_device *pdev)
debugfs_remove_recursive(lan966x->debugfs_root);
}
+static void lan966x_shutdown(struct platform_device *pdev)
+{
+ struct lan966x *lan966x = platform_get_drvdata(pdev);
+
+ lan966x->ops->fdma_deinit(lan966x);
+}
+
static struct platform_driver lan966x_driver = {
.probe = lan966x_probe,
.remove = lan966x_remove,
+ .shutdown = lan966x_shutdown,
.driver = {
.name = "lan966x-switch",
.of_match_table = lan966x_match,
With that done, the issue is no more present.
I am not sure that this is a correct fix but it can give some clues.
Feel free to ask for more tests or more details if you need to.
Best regards,
Hervé
prev parent reply other threads:[~2026-04-08 9:51 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-20 15:00 [PATCH net-next 00/10] net: lan966x: add support for PCIe FDMA Daniel Machon
2026-03-20 15:00 ` [PATCH net-next 01/10] net: microchip: fdma: rename contiguous dataptr helpers Daniel Machon
2026-03-20 15:00 ` [PATCH net-next 02/10] net: microchip: fdma: add PCIe ATU support Daniel Machon
2026-03-20 15:00 ` [PATCH net-next 03/10] net: lan966x: add FDMA LLP register write helper Daniel Machon
2026-03-20 15:01 ` [PATCH net-next 04/10] net: lan966x: export FDMA helpers for reuse Daniel Machon
2026-03-20 15:01 ` [PATCH net-next 05/10] net: lan966x: add FDMA ops dispatch for PCIe support Daniel Machon
2026-03-20 15:01 ` [PATCH net-next 06/10] net: lan966x: add PCIe FDMA support Daniel Machon
2026-03-20 15:01 ` [PATCH net-next 07/10] net: lan966x: add PCIe FDMA MTU change support Daniel Machon
2026-03-20 15:01 ` [PATCH net-next 08/10] net: lan966x: add PCIe FDMA XDP support Daniel Machon
2026-03-22 7:11 ` Mohsin Bashir
2026-03-22 20:30 ` Daniel Machon
2026-03-20 15:01 ` [PATCH net-next 09/10] misc: lan966x-pci: dts: extend cpu reg to cover PCIE DBI space Daniel Machon
2026-03-20 15:01 ` [PATCH net-next 10/10] misc: lan966x-pci: dts: add fdma interrupt to overlay Daniel Machon
2026-03-23 14:52 ` [PATCH net-next 00/10] net: lan966x: add support for PCIe FDMA Herve Codina
2026-03-23 16:26 ` Herve Codina
2026-03-23 19:40 ` Daniel Machon
2026-03-24 8:07 ` Herve Codina
2026-03-26 15:48 ` Daniel Machon
2026-03-27 10:33 ` Herve Codina
2026-03-27 11:07 ` Daniel Machon
2026-04-07 13:20 ` Daniel Machon
2026-04-08 9:51 ` Herve Codina [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260408115126.5529c823@bootlin.com \
--to=herve.codina@bootlin.com \
--cc=UNGLinuxDriver@microchip.com \
--cc=andrew+netdev@lunn.ch \
--cc=arnd@arndb.de \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel.machon@microchip.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=gregkh@linuxfoundation.org \
--cc=hawk@kernel.org \
--cc=horatiu.vultur@microchip.com \
--cc=john.fastabend@gmail.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=sdf@fomichev.me \
--cc=steen.hegelund@microchip.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.