[PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Michał Kępień]

While the qca8327 switch appears to support in-band mgmt Ethernet,
prolonged use of that protocol (e.g. for polling link state) makes the
device unstable: within minutes, ports randomly go down and no traffic
is forwarded anymore.  The same issues do not occur when MDIO is used
exclusively, so ensure mgmt Ethernet is not used on the qca8327.

Signed-off-by: Michał Kępień <kernel@kempniu.pl>
---
I came across this while migrating an AR9344-based router with a QCA8327
rev. 4 switch to a DSA-aware driver.  This glitch is a pain in the neck
to troubleshoot any further as it occurs randomly, anywhere between a
minute to an hour after the switch is set up; traffic load exerted on
the switch does not seem to matter as the problem can be triggered on a
virtually idle device.  Previously working links are reported as going
down (one by one, not all at once), even though port LEDs still blink;
no traffic is forwarded; reloading qca8k does not alleviate the problem,
only power cycling seems to help.  Nothing like this happens when only
MDIO is used.  However, qca8k currently only uses MDIO as a fallback.  I
figured that simpler is better and that mgmt Ethernet should simply be
disabled for the qca8327, but I would be happy to work on some
configurable solution if that would be preferable.

 drivers/net/dsa/qca/qca8k-8xxx.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/net/dsa/qca/qca8k-8xxx.c b/drivers/net/dsa/qca/qca8k-8xxx.c
index 4c928983b8623..1d90a23aba6bb 100644
--- a/drivers/net/dsa/qca/qca8k-8xxx.c
+++ b/drivers/net/dsa/qca/qca8k-8xxx.c
@@ -160,6 +160,11 @@ qca8k_set_page(struct qca8k_priv *priv, u16 page)
 	return 0;
 }
 
+static bool qca8k_mgmt_eth_disabled(const struct qca8k_priv *priv)
+{
+	return priv->switch_id == QCA8K_ID_QCA8327;
+}
+
 static void qca8k_rw_reg_ack_handler(struct dsa_switch *ds, struct sk_buff *skb)
 {
 	struct qca8k_mgmt_eth_data *mgmt_eth_data;
@@ -316,6 +321,9 @@ static int qca8k_read_eth(struct qca8k_priv *priv, u32 reg, u32 *val, int len)
 	bool ack;
 	int ret;
 
+	if (qca8k_mgmt_eth_disabled(priv))
+		return -ENXIO;
+
 	skb = qca8k_alloc_mdio_header(MDIO_READ, reg, NULL,
 				      QCA8K_ETHERNET_MDIO_PRIORITY, len);
 	if (!skb)
@@ -368,6 +376,9 @@ static int qca8k_write_eth(struct qca8k_priv *priv, u32 reg, u32 *val, int len)
 	bool ack;
 	int ret;
 
+	if (qca8k_mgmt_eth_disabled(priv))
+		return -ENXIO;
+
 	skb = qca8k_alloc_mdio_header(MDIO_WRITE, reg, val,
 				      QCA8K_ETHERNET_MDIO_PRIORITY, len);
 	if (!skb)
@@ -630,6 +641,9 @@ qca8k_phy_eth_command(struct qca8k_priv *priv, bool read, int phy,
 	int ret, ret1;
 	bool ack;
 
+	if (qca8k_mgmt_eth_disabled(priv))
+		return -ENXIO;
+
 	if (regnum >= QCA8K_MDIO_MASTER_MAX_REG)
 		return -EINVAL;
 
-- 
2.55.0

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Andrew Lunn]

On Mon, Aug 10, 2026 at 08:15:53AM +0200, Michał Kępień wrote:
> While the qca8327 switch appears to support in-band mgmt Ethernet,
> prolonged use of that protocol (e.g. for polling link state) makes the
> device unstable: within minutes, ports randomly go down and no traffic
> is forwarded anymore.  The same issues do not occur when MDIO is used
> exclusively, so ensure mgmt Ethernet is not used on the qca8327.

Do you have time to narrow down the cause?

I _think_ in band management is used for a few different
things. e.g. statistics, as you said, PHY polling etc. Rather than
turning everything off, could you try just doing PHY polling via MDIO,
but statistics via ethernet, and do an ethtool -S every so often to
see if you can trigger the problem.

Could you also check if the management packets are getting lost? Look
at the results from wait_for_completion_timeout(), is it timing out?
If so, is the retry mechanism working? When i added the Marvell
equivalent for in-band signalling i got the retry mechanism wrong, but
never noticed because i was not loosing packets.

Thanks
	Andrew

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Michał Kępień]

Andrew,

First of all, after taking a closer look at this issue and the driver
code, I realized that the approach I went with in the patch stemmed from
my ignorance and that I submitted it too quickly - apologies about that.

> > While the qca8327 switch appears to support in-band mgmt Ethernet,
> > prolonged use of that protocol (e.g. for polling link state) makes the
> > device unstable: within minutes, ports randomly go down and no traffic
> > is forwarded anymore.  The same issues do not occur when MDIO is used
> > exclusively, so ensure mgmt Ethernet is not used on the qca8327.
> 
> Do you have time to narrow down the cause?

Sure, I should be able to spare a few cycles on this in the upcoming
weeks.

> I _think_ in band management is used for a few different
> things. e.g. statistics, as you said, PHY polling etc. Rather than
> turning everything off, could you try just doing PHY polling via MDIO,
> but statistics via ethernet, and do an ethtool -S every so often to
> see if you can trigger the problem.

Ack, I'll try that.

> Could you also check if the management packets are getting lost? Look
> at the results from wait_for_completion_timeout(), is it timing out?

Ack.

> If so, is the retry mechanism working? When i added the Marvell
> equivalent for in-band signalling i got the retry mechanism wrong, but
> never noticed because i was not loosing packets.

Could you please point me at the exact retry mechanism you had in mind,
either the qca8k one or the Marvell equivalent?  I cannot see any in
drivers/net/dsa/qca/qca8k-8xxx.c.  If I'm reading qca8k code correctly,
when wait_for_completion_timeout() fails, its return value is just
bubbled up the call chain.

Thanks,

-- 
Best regards,
Michał Kępień

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Andrew Lunn]

> Could you please point me at the exact retry mechanism you had in mind,
> either the qca8k one or the Marvell equivalent?  I cannot see any in
> drivers/net/dsa/qca/qca8k-8xxx.c.  If I'm reading qca8k code correctly,
> when wait_for_completion_timeout() fails, its return value is just
> bubbled up the call chain.

With the mv88e6xxx code, if ethernet times out, i repeated the request
using MDIO. The code is not in mainline but a few users are using it.

At minimum, i would put a printk() there, and see if it happens.  It
could be unrelated to your problem...

      Andrew

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Michał Kępień]

> > Could you please point me at the exact retry mechanism you had in mind,
> > either the qca8k one or the Marvell equivalent?  I cannot see any in
> > drivers/net/dsa/qca/qca8k-8xxx.c.  If I'm reading qca8k code correctly,
> > when wait_for_completion_timeout() fails, its return value is just
> > bubbled up the call chain.
> 
> With the mv88e6xxx code, if ethernet times out, i repeated the request
> using MDIO. The code is not in mainline but a few users are using it.

Ah, that way, okay.  It looks like every mgmt Ethernet function in qca8k
has an MDIO fallback path, but I'll keep this in mind, thanks.

> At minimum, i would put a printk() there, and see if it happens.  It
> could be unrelated to your problem...

Ack, thanks.

-- 
Best regards,
Michał Kępień

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Christian Marangi]

On Mon, Aug 10, 2026 at 08:15:53AM +0200, Michał Kępień wrote:
> While the qca8327 switch appears to support in-band mgmt Ethernet,
> prolonged use of that protocol (e.g. for polling link state) makes the
> device unstable: within minutes, ports randomly go down and no traffic
> is forwarded anymore.  The same issues do not occur when MDIO is used
> exclusively, so ensure mgmt Ethernet is not used on the qca8327.
> 
> Signed-off-by: Michał Kępień <kernel@kempniu.pl>
> ---
> I came across this while migrating an AR9344-based router with a QCA8327
> rev. 4 switch to a DSA-aware driver.  This glitch is a pain in the neck
> to troubleshoot any further as it occurs randomly, anywhere between a
> minute to an hour after the switch is set up; traffic load exerted on
> the switch does not seem to matter as the problem can be triggered on a
> virtually idle device.  Previously working links are reported as going
> down (one by one, not all at once), even though port LEDs still blink;
> no traffic is forwarded; reloading qca8k does not alleviate the problem,
> only power cycling seems to help.  Nothing like this happens when only
> MDIO is used.  However, qca8k currently only uses MDIO as a fallback.  I
> figured that simpler is better and that mgmt Ethernet should simply be
> disabled for the qca8327, but I would be happy to work on some
> configurable solution if that would be preferable.
>

This is a long standing issue and it seems to me disabling mgmt is just a
big workaround to a real problem.

Long time ago it was reported that there seems to be a problem with the
mdio master register for external and internall access and how mgmt was
actually sending mdio command... just done by the switch. Could the 2 issue
related?

One idea might be to verify that stuff gets actually written... as Andrew
said to verify if some packets doesn't get lost or just ignored.

Also as Andrew said I would still save this for the MIB part as the 2 thing
should be unrelated.
 
>  drivers/net/dsa/qca/qca8k-8xxx.c | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/drivers/net/dsa/qca/qca8k-8xxx.c b/drivers/net/dsa/qca/qca8k-8xxx.c
> index 4c928983b8623..1d90a23aba6bb 100644
> --- a/drivers/net/dsa/qca/qca8k-8xxx.c
> +++ b/drivers/net/dsa/qca/qca8k-8xxx.c
> @@ -160,6 +160,11 @@ qca8k_set_page(struct qca8k_priv *priv, u16 page)
>  	return 0;
>  }
>  
> +static bool qca8k_mgmt_eth_disabled(const struct qca8k_priv *priv)
> +{
> +	return priv->switch_id == QCA8K_ID_QCA8327;
> +}
> +

Instead of this and return ENXIO I would just not install the relevant OPs
for the tagger and use the mdio path directly...

Makes the code cleaner and less CPU cycle (the target is ath79 and powerpc
stuff)

But as said above disabling the feature is the last solution after all the
verification are done.

>  static void qca8k_rw_reg_ack_handler(struct dsa_switch *ds, struct sk_buff *skb)
>  {
>  	struct qca8k_mgmt_eth_data *mgmt_eth_data;
> @@ -316,6 +321,9 @@ static int qca8k_read_eth(struct qca8k_priv *priv, u32 reg, u32 *val, int len)
>  	bool ack;
>  	int ret;
>  
> +	if (qca8k_mgmt_eth_disabled(priv))
> +		return -ENXIO;
> +
>  	skb = qca8k_alloc_mdio_header(MDIO_READ, reg, NULL,
>  				      QCA8K_ETHERNET_MDIO_PRIORITY, len);
>  	if (!skb)
> @@ -368,6 +376,9 @@ static int qca8k_write_eth(struct qca8k_priv *priv, u32 reg, u32 *val, int len)
>  	bool ack;
>  	int ret;
>  
> +	if (qca8k_mgmt_eth_disabled(priv))
> +		return -ENXIO;
> +
>  	skb = qca8k_alloc_mdio_header(MDIO_WRITE, reg, val,
>  				      QCA8K_ETHERNET_MDIO_PRIORITY, len);
>  	if (!skb)
> @@ -630,6 +641,9 @@ qca8k_phy_eth_command(struct qca8k_priv *priv, bool read, int phy,
>  	int ret, ret1;
>  	bool ack;
>  
> +	if (qca8k_mgmt_eth_disabled(priv))
> +		return -ENXIO;
> +
>  	if (regnum >= QCA8K_MDIO_MASTER_MAX_REG)
>  		return -EINVAL;
>  
> -- 
> 2.55.0
> 

-- 
	Ansuel

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Michał Kępień]

> > I came across this while migrating an AR9344-based router with a QCA8327
> > rev. 4 switch to a DSA-aware driver.  This glitch is a pain in the neck
> > to troubleshoot any further as it occurs randomly, anywhere between a
> > minute to an hour after the switch is set up; traffic load exerted on
> > the switch does not seem to matter as the problem can be triggered on a
> > virtually idle device.  Previously working links are reported as going
> > down (one by one, not all at once), even though port LEDs still blink;
> > no traffic is forwarded; reloading qca8k does not alleviate the problem,
> > only power cycling seems to help.  Nothing like this happens when only
> > MDIO is used.  However, qca8k currently only uses MDIO as a fallback.  I
> > figured that simpler is better and that mgmt Ethernet should simply be
> > disabled for the qca8327, but I would be happy to work on some
> > configurable solution if that would be preferable.
> >
> 
> This is a long standing issue and it seems to me disabling mgmt is just a
> big workaround to a real problem.

Understood.  Is it a long-standing issue for the QCA mgmt Ethernet code
in general or for a specific subset of switches (or devices)?

> Long time ago it was reported that there seems to be a problem with the
> mdio master register for external and internall access and how mgmt was
> actually sending mdio command... just done by the switch. Could the 2 issue
> related?

This thread?

https://lore.kernel.org/netdev/20250425151309.30493-1-kabel@kernel.org/

If so, that was for QCA8337, on a board where the external MDIO bus has
an extra PHY attached, so it did not look like a match for my case.  Of
course, it _might_ be related to the issue I'm running into, it just did
not seem to be at first glance.

> One idea might be to verify that stuff gets actually written... as Andrew
> said to verify if some packets doesn't get lost or just ignored.

Ack.

> Also as Andrew said I would still save this for the MIB part as the 2 thing
> should be unrelated.

Got it, I'll play around with it and see what I can find out, thanks.

-- 
Best regards,
Michał Kępień

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Christian Marangi]

On Wed, Aug 12, 2026 at 11:14:28AM +0200, Michał Kępień wrote:
> > > I came across this while migrating an AR9344-based router with a QCA8327
> > > rev. 4 switch to a DSA-aware driver.  This glitch is a pain in the neck
> > > to troubleshoot any further as it occurs randomly, anywhere between a
> > > minute to an hour after the switch is set up; traffic load exerted on
> > > the switch does not seem to matter as the problem can be triggered on a
> > > virtually idle device.  Previously working links are reported as going
> > > down (one by one, not all at once), even though port LEDs still blink;
> > > no traffic is forwarded; reloading qca8k does not alleviate the problem,
> > > only power cycling seems to help.  Nothing like this happens when only
> > > MDIO is used.  However, qca8k currently only uses MDIO as a fallback.  I
> > > figured that simpler is better and that mgmt Ethernet should simply be
> > > disabled for the qca8327, but I would be happy to work on some
> > > configurable solution if that would be preferable.
> > >
> > 
> > This is a long standing issue and it seems to me disabling mgmt is just a
> > big workaround to a real problem.
> 
> Understood.  Is it a long-standing issue for the QCA mgmt Ethernet code
> in general or for a specific subset of switches (or devices)?
>

specific subset of switches. On ipq806x the 8337 is mounted and mgmt works
correctly without issue.

One thing I notice on a different vendor (Airoha) is that sometimes using
these indirect way to access the Switch register might introduce
interesting HW bug.

One bug that was there was that when PBUS was used to access single port
PHY register (instead of direct MDIO) the link up/down was broken. My
theory was that the Switch chip had some latch logic that was only
triggered with MDIO. Using PBUS didn't trigger such thing.

Could be that the QCA 8327 switch also have some kind of HW bug where specific
register needs to go with MDIO or some refresh/latch logic are not
correctly triggered.

An idea might be to limit the mgmt to vlan and fdb and see if the problem
is still there. (after all those are the path where mgmt would benefit due
to the multiple register access required)
 
> > Long time ago it was reported that there seems to be a problem with the
> > mdio master register for external and internall access and how mgmt was
> > actually sending mdio command... just done by the switch. Could the 2 issue
> > related?
> 
> This thread?
> 
> https://lore.kernel.org/netdev/20250425151309.30493-1-kabel@kernel.org/
> 
> If so, that was for QCA8337, on a board where the external MDIO bus has
> an extra PHY attached, so it did not look like a match for my case.  Of
> course, it _might_ be related to the issue I'm running into, it just did
> not seem to be at first glance.
> 
> > One idea might be to verify that stuff gets actually written... as Andrew
> > said to verify if some packets doesn't get lost or just ignored.
> 
> Ack.
> 
> > Also as Andrew said I would still save this for the MIB part as the 2 thing
> > should be unrelated.
> 
> Got it, I'll play around with it and see what I can find out, thanks.
> 
> -- 
> Best regards,
> Michał Kępień

-- 
	Ansuel

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Michał Kępień]

Christian, Andrew,

Here is what I managed to determine for the QCA8327 switch on the board
I have access to.

> One bug that was there was that when PBUS was used to access single port
> PHY register (instead of direct MDIO) the link up/down was broken. My
> theory was that the Switch chip had some latch logic that was only
> triggered with MDIO. Using PBUS didn't trigger such thing.
> 
> Could be that the QCA 8327 switch also have some kind of HW bug where specific
> register needs to go with MDIO or some refresh/latch logic are not
> correctly triggered.

My problem sounds similar.  The exact symptom I am observing is that at
a random point in time after switch setup, PHY register reads performed
via mgmt Ethernet frames start returning all zeros.  The mgmt Ethernet
transactions themselves are completed correctly, they just return all
zeros in the data part.  When the issue is triggered, _all_ subsequent
PHY register reads performed via mgmt Ethernet frames for the affected
PHY return all zeros, i.e. it never recovers.

The problem is triggered on distinct PHYs one by one, in a seemingly
random order; I did not identify any patterns.  Given enough time, all
PHYs eventually start returning all zeros for all of their register
reads.

At one point, I tweaked qca8k_internal_mdio_read() so that when a PHY
register read fails in this specific way, it is immediately retried over
MDIO.  Those fallback reads over MDIO also returned all zeros.

No timeouts are occurring for mgmt Ethernet frames.  I instrumented all
wait_for_completion_timeout() call sites in qca8k-8xxx.c and have yet to
see any of them log a timeout (or any other error, for that matter).  In
other words, all requests sent via mgmt Ethernet frames are responded to
by the switch in a timely manner, so I have no reason to believe that
the retry logic is involved here in any way.

All of this reproducibly happens on a device that does not even have any
Ethernet cables connected to it.

Manually probing registers 2 and 3 on the external MDIO bus reveals no
other devices except the expected five switch PHYs.

I used tcpdump to capture the traffic going through the DSA conduit
device, from the moment the switch is configured until the failure
occurs.  I found nothing suspicious: the last PHY register read before
the failed one happens about a second earlier and the sequence of mgmt
Ethernet frames sent and received for the last _successful_ read is a
1:1 match against the first _failing_ read, sans the sequence numbers.
No other traffic is interleaved, which is expected given that both the
external MDIO bus mutex and the mgmt Ethernet mutex are held throughout
each distinct PHY management transaction.

Given the above, I am out of ideas for something to latch on to for
further investigations.  It seems that over time, using mgmt Ethernet
for PHY access on this board makes the hardware transition into some
broken state that it cannot recover from.  As I am able to reliably
trigger this problem, I am open to ideas for further experiments that
could possibly shed some light on this.

> An idea might be to limit the mgmt to vlan and fdb and see if the problem
> is still there. (after all those are the path where mgmt would benefit due
> to the multiple register access required)

Indeed, it seems that the simplest workaround for this glitch is to move
the PHY nodes in the DTS file from the ethernet-switch node to its
parent, the mdio node, and use phy-handle for ethernet-ports to
configure qca8k in "external MDIO" mode.  That way, the external MDIO
bus is used for PHY management while mgmt Ethernet is still used for MIB
access and switch configuration, which seems to be stable; with PHY
management moved to MDIO, I ran "ethtool -S <iface>" in a loop for a few
hours and everything still seemingly worked fine.  This seems to be a
cleaner workaround than tweaking the driver code, so I don't plan to
submit any further patches in this spirit.

Thank you for your guidance.  If you have any ideas for further
experiments in this area, I am all ears.

-- 
Best regards,
Michał Kępień

Re: [PATCH] net: dsa: qca8k: Disable mgmt Ethernet for qca8327

[Luiz Angelo Daros de Luca]

Hi everyone,

To add to this thread, there are a couple of other well-documented
issues tied to management Ethernet in qca8k:

1) TP-Link Archer C7 v2 (dual CPU ports 0 and 6): One CPU port becomes
muted/unusable during setup when state transitions alter the
management conduit. (ref:
https://github.com/openwrt/openwrt/pull/24011)
2) CZ.NIC Turris 1.0 and 1.1: Internal PHY access via management
frames leaks onto the external MDC/MDIO pins, corrupting communication
between the kernel and the external WAN PHY (commit 526c8ee was
insufficient to fully fix this). (ref:
https://github.com/openwrt/openwrt/pull/24755)

There are also several historical reports for other TP-Link Archer
devices (single CPU port) where ports randomly drop after some uptime,
exhibiting similar symptoms Michał described. At least for the two
cases I cited before, bypassing Ethernet-based PHY management and
sticking to MDIO works around the issue reliably.

As noted in Turris 1.0 PR patch, when the switch translates an
Ethernet management frame into an internal MDIO transaction, that
signal leaks onto the external MDC/MDIO lines. Because the switch acts
as an MDIO master for that internal query while the host CPU remains
the primary MDIO master, electrical bus conflict might break the
switch's internal MDIO translator state machine. Commit 526c8ee
attempted to solve this by acquiring bus->mdio_lock before
transmitting management frames, but software-level locking cannot
prevent hardware-level bus contention or state corruption if
QCA8K_MDIO_MASTER_EN toggles or leaks while lines are shared.

This raises the question: does Ethernet-based PHY access actually
yield any meaningful performance benefit (or is it worth it)? PHY
polling occurs at a low frequency. Also, maintaining a dual code path
(Ethernet management with silent fallback to direct MDIO)
significantly complicates troubleshooting. By comparison, drivers like
rtl8365mb enforce a single, deterministic code path for register/PHY
accesses to keep behavior predictable across all boards, even when we
could use an optimized path.

Given that Ethernet management frames for PHY access introduce
multiple hard-to-diagnose regressions across different board
topologies, should we consider dropping qca8k_phy_eth_command()
entirely in favor of direct MDIO, or at least providing a mechanism to
disable it? If MIB access becomes an issue, we could introduce a cache
mechanism.

Best regards,

Luiz