[PATCH net-next] net: enetc: teardown CBDR during PF/VF unbind

[PATCH net-next] net: enetc: teardown CBDR during PF/VF unbind

[Vladimir Oltean]

From: Vladimir Oltean <vladimir.oltean@nxp.com>

Michael reports that after the blamed patch, unbinding a VF would cause
these transactions to remain pending, and trigger some warnings with the
DMA API debug:

$ echo 1 > /sys/bus/pci/devices/0000\:00\:00.0/sriov_numvfs
pci 0000:00:01.0: [1957:ef00] type 00 class 0x020001
fsl_enetc_vf 0000:00:01.0: Adding to iommu group 19
fsl_enetc_vf 0000:00:01.0: enabling device (0000 -> 0002)
fsl_enetc_vf 0000:00:01.0 eno0vf0: renamed from eth0

$ echo 0 > /sys/bus/pci/devices/0000\:00\:00.0/sriov_numvfs
DMA-API: pci 0000:00:01.0: device driver has pending DMA allocations while released from device [count=1]
One of leaked entries details: [size=2048 bytes] [mapped with DMA_BIDIRECTIONAL] [mapped as coherent]
WARNING: CPU: 0 PID: 2547 at kernel/dma/debug.c:853 dma_debug_device_change+0x174/0x1c8
(...)
Call trace:
 dma_debug_device_change+0x174/0x1c8
 blocking_notifier_call_chain+0x74/0xa8
 device_release_driver_internal+0x18c/0x1f0
 device_release_driver+0x20/0x30
 pci_stop_bus_device+0x8c/0xe8
 pci_stop_and_remove_bus_device+0x20/0x38
 pci_iov_remove_virtfn+0xb8/0x128
 sriov_disable+0x3c/0x110
 pci_disable_sriov+0x24/0x30
 enetc_sriov_configure+0x4c/0x108
 sriov_numvfs_store+0x11c/0x198
(...)
DMA-API: Mapped at:
 dma_entry_alloc+0xa4/0x130
 debug_dma_alloc_coherent+0xbc/0x138
 dma_alloc_attrs+0xa4/0x108
 enetc_setup_cbdr+0x4c/0x1d0
 enetc_vf_probe+0x11c/0x250
pci 0000:00:01.0: Removing from iommu group 19

This happens because stupid me moved enetc_teardown_cbdr outside of
enetc_free_si_resources, but did not bother to keep calling
enetc_teardown_cbdr from all the places where enetc_free_si_resources
was called. In particular, now it is no longer called from the main
unbind function, just from the probe error path.

Fixes: 4b47c0b81ffd ("net: enetc: don't initialize unused ports from a separate code path")
Reported-by: Michael Walle <michael@walle.cc>
Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>
---
 drivers/net/ethernet/freescale/enetc/enetc_pf.c | 1 +
 drivers/net/ethernet/freescale/enetc/enetc_vf.c | 1 +
 2 files changed, 2 insertions(+)

diff --git a/drivers/net/ethernet/freescale/enetc/enetc_pf.c b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
index bea1d935fa1c..60f20970a79d 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_pf.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_pf.c
@@ -1226,6 +1226,7 @@ static void enetc_pf_remove(struct pci_dev *pdev)
 	enetc_free_msix(priv);
 
 	enetc_free_si_resources(priv);
+	enetc_teardown_cbdr(&si->cbd_ring);
 
 	free_netdev(si->ndev);
 
diff --git a/drivers/net/ethernet/freescale/enetc/enetc_vf.c b/drivers/net/ethernet/freescale/enetc/enetc_vf.c
index 371a34d3c6b4..03090ba7e226 100644
--- a/drivers/net/ethernet/freescale/enetc/enetc_vf.c
+++ b/drivers/net/ethernet/freescale/enetc/enetc_vf.c
@@ -223,6 +223,7 @@ static void enetc_vf_remove(struct pci_dev *pdev)
 	enetc_free_msix(priv);
 
 	enetc_free_si_resources(priv);
+	enetc_teardown_cbdr(&si->cbd_ring);
 
 	free_netdev(si->ndev);
 
-- 
2.25.1

Re: [PATCH net-next] net: enetc: teardown CBDR during PF/VF unbind

[Michael Walle]

Am 2021-03-19 11:08, schrieb Vladimir Oltean:
> From: Vladimir Oltean <vladimir.oltean@nxp.com>
> 
> Michael reports that after the blamed patch, unbinding a VF would cause
> these transactions to remain pending, and trigger some warnings with 
> the
> DMA API debug:
> 
> $ echo 1 > /sys/bus/pci/devices/0000\:00\:00.0/sriov_numvfs
> pci 0000:00:01.0: [1957:ef00] type 00 class 0x020001
> fsl_enetc_vf 0000:00:01.0: Adding to iommu group 19
> fsl_enetc_vf 0000:00:01.0: enabling device (0000 -> 0002)
> fsl_enetc_vf 0000:00:01.0 eno0vf0: renamed from eth0
> 
> $ echo 0 > /sys/bus/pci/devices/0000\:00\:00.0/sriov_numvfs
> DMA-API: pci 0000:00:01.0: device driver has pending DMA allocations
> while released from device [count=1]
> One of leaked entries details: [size=2048 bytes] [mapped with
> DMA_BIDIRECTIONAL] [mapped as coherent]
> WARNING: CPU: 0 PID: 2547 at kernel/dma/debug.c:853
> dma_debug_device_change+0x174/0x1c8
> (...)
> Call trace:
>  dma_debug_device_change+0x174/0x1c8
>  blocking_notifier_call_chain+0x74/0xa8
>  device_release_driver_internal+0x18c/0x1f0
>  device_release_driver+0x20/0x30
>  pci_stop_bus_device+0x8c/0xe8
>  pci_stop_and_remove_bus_device+0x20/0x38
>  pci_iov_remove_virtfn+0xb8/0x128
>  sriov_disable+0x3c/0x110
>  pci_disable_sriov+0x24/0x30
>  enetc_sriov_configure+0x4c/0x108
>  sriov_numvfs_store+0x11c/0x198
> (...)
> DMA-API: Mapped at:
>  dma_entry_alloc+0xa4/0x130
>  debug_dma_alloc_coherent+0xbc/0x138
>  dma_alloc_attrs+0xa4/0x108
>  enetc_setup_cbdr+0x4c/0x1d0
>  enetc_vf_probe+0x11c/0x250
> pci 0000:00:01.0: Removing from iommu group 19
> 
> This happens because stupid me moved enetc_teardown_cbdr outside of
> enetc_free_si_resources, but did not bother to keep calling
> enetc_teardown_cbdr from all the places where enetc_free_si_resources
> was called. In particular, now it is no longer called from the main
> unbind function, just from the probe error path.
> 
> Fixes: 4b47c0b81ffd ("net: enetc: don't initialize unused ports from a
> separate code path")
> Reported-by: Michael Walle <michael@walle.cc>
> Signed-off-by: Vladimir Oltean <vladimir.oltean@nxp.com>

Tested-by: Michael Walle <michael@walle.cc>

Thanks!

-michael

Re: [PATCH net-next] net: enetc: teardown CBDR during PF/VF unbind

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to netdev/net-next.git (refs/heads/master):

On Fri, 19 Mar 2021 12:08:06 +0200 you wrote:
> From: Vladimir Oltean <vladimir.oltean@nxp.com>
> 
> Michael reports that after the blamed patch, unbinding a VF would cause
> these transactions to remain pending, and trigger some warnings with the
> DMA API debug:
> 
> $ echo 1 > /sys/bus/pci/devices/0000\:00\:00.0/sriov_numvfs
> pci 0000:00:01.0: [1957:ef00] type 00 class 0x020001
> fsl_enetc_vf 0000:00:01.0: Adding to iommu group 19
> fsl_enetc_vf 0000:00:01.0: enabling device (0000 -> 0002)
> fsl_enetc_vf 0000:00:01.0 eno0vf0: renamed from eth0
> 
> [...]

Here is the summary with links:
  - [net-next] net: enetc: teardown CBDR during PF/VF unbind
    https://git.kernel.org/netdev/net-next/c/c54f042dcc1b

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html