[PATCH 1/2] net: moxa: do not call dma_unmap_single() with null

[PATCH 1/2] net: moxa: do not call dma_unmap_single() with null

[Sergei Antonov]

It fixes a warning during error unwinding:

WARNING: CPU: 0 PID: 1 at kernel/dma/debug.c:963 check_unmap+0x704/0x980
DMA-API: moxart-ethernet 92000000.mac: device driver tries to free DMA memory it has not allocated [device address=0x0000000000000000] [size=1600 bytes]
CPU: 0 PID: 1 Comm: swapper Not tainted 5.19.0+ #60
Hardware name: Generic DT based system
 unwind_backtrace from show_stack+0x10/0x14
 show_stack from dump_stack_lvl+0x34/0x44
 dump_stack_lvl from __warn+0xbc/0x1f0
 __warn from warn_slowpath_fmt+0x94/0xc8
 warn_slowpath_fmt from check_unmap+0x704/0x980
 check_unmap from debug_dma_unmap_page+0x8c/0x9c
 debug_dma_unmap_page from moxart_mac_free_memory+0x3c/0xa8
 moxart_mac_free_memory from moxart_mac_probe+0x190/0x218
 moxart_mac_probe from platform_probe+0x48/0x88
 platform_probe from really_probe+0xc0/0x2e4

Fixes: 6c821bd9edc9 ("net: Add MOXA ART SoCs ethernet driver")
Signed-off-by: Sergei Antonov <saproj@gmail.com>
---
 drivers/net/ethernet/moxa/moxart_ether.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/moxa/moxart_ether.c b/drivers/net/ethernet/moxa/moxart_ether.c
index f11f1cb92025..edd1dab2ec43 100644
--- a/drivers/net/ethernet/moxa/moxart_ether.c
+++ b/drivers/net/ethernet/moxa/moxart_ether.c
@@ -77,8 +77,9 @@ static void moxart_mac_free_memory(struct net_device *ndev)
 	int i;
 
 	for (i = 0; i < RX_DESC_NUM; i++)
-		dma_unmap_single(&priv->pdev->dev, priv->rx_mapping[i],
-				 priv->rx_buf_size, DMA_FROM_DEVICE);
+		if (priv->rx_mapping[i])
+			dma_unmap_single(&priv->pdev->dev, priv->rx_mapping[i],
+					 priv->rx_buf_size, DMA_FROM_DEVICE);
 
 	if (priv->tx_desc_base)
 		dma_free_coherent(&priv->pdev->dev,
-- 
2.32.0

[PATCH 2/2] net: moxa: prevent double-mapping of DMA areas

[Sergei Antonov]

Fix the warning poping up after bringing the link down, then up:
 ip link set dev eth0 down
 ip link set dev eth0 up

WARNING: CPU: 0 PID: 55 at kernel/dma/debug.c:570 add_dma_entry+0x204/0x2ec
DMA-API: moxart-ethernet 92000000.mac: cacheline tracking EEXIST, overlapping mappings aren't supported
CPU: 0 PID: 55 Comm: ip Not tainted 5.19.0+ #57
Hardware name: Generic DT based system
 unwind_backtrace from show_stack+0x10/0x14
 show_stack from dump_stack_lvl+0x34/0x44
 dump_stack_lvl from __warn+0xbc/0x1f0
 __warn from warn_slowpath_fmt+0x94/0xc8
 warn_slowpath_fmt from add_dma_entry+0x204/0x2ec
 add_dma_entry from dma_map_page_attrs+0x110/0x328
 dma_map_page_attrs from moxart_mac_open+0x134/0x320
 moxart_mac_open from __dev_open+0x11c/0x1ec
 __dev_open from __dev_change_flags+0x194/0x22c
 __dev_change_flags from dev_change_flags+0x14/0x44
 dev_change_flags from devinet_ioctl+0x6d4/0x93c
 devinet_ioctl from inet_ioctl+0x1ac/0x25c

Unmap RX memory areas in moxart_mac_stop(), so that moxart_mac_open()
will map them anew instead of double-mapping. To avoid code duplication,
create a new function moxart_mac_unmap_rx(). Nullify unmapped pointers to
prevent double-unmapping (ex: moxart_mac_stop(), then moxart_remove()).

Fixes: 6c821bd9edc9 ("net: Add MOXA ART SoCs ethernet driver")
Signed-off-by: Sergei Antonov <saproj@gmail.com>
---
 drivers/net/ethernet/moxa/moxart_ether.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/moxa/moxart_ether.c b/drivers/net/ethernet/moxa/moxart_ether.c
index edd1dab2ec43..22d0bf4a21e7 100644
--- a/drivers/net/ethernet/moxa/moxart_ether.c
+++ b/drivers/net/ethernet/moxa/moxart_ether.c
@@ -71,15 +71,23 @@ static int moxart_set_mac_address(struct net_device *ndev, void *addr)
 	return 0;
 }
 
-static void moxart_mac_free_memory(struct net_device *ndev)
+static void moxart_mac_unmap_rx(struct moxart_mac_priv_t *priv)
 {
-	struct moxart_mac_priv_t *priv = netdev_priv(ndev);
 	int i;
 
 	for (i = 0; i < RX_DESC_NUM; i++)
-		if (priv->rx_mapping[i])
+		if (priv->rx_mapping[i]) {
 			dma_unmap_single(&priv->pdev->dev, priv->rx_mapping[i],
 					 priv->rx_buf_size, DMA_FROM_DEVICE);
+			priv->rx_mapping[i] = 0;
+		}
+}
+
+static void moxart_mac_free_memory(struct net_device *ndev)
+{
+	struct moxart_mac_priv_t *priv = netdev_priv(ndev);
+
+	moxart_mac_unmap_rx(priv);
 
 	if (priv->tx_desc_base)
 		dma_free_coherent(&priv->pdev->dev,
@@ -205,6 +213,7 @@ static int moxart_mac_stop(struct net_device *ndev)
 	/* disable all functions */
 	writel(0, priv->base + REG_MAC_CTRL);
 
+	moxart_mac_unmap_rx(priv);
 	return 0;
 }
 
-- 
2.32.0

Re: [PATCH 2/2] net: moxa: prevent double-mapping of DMA areas

[Andrew Lunn]

On Thu, Aug 18, 2022 at 09:29:48PM +0300, Sergei Antonov wrote:
> Fix the warning poping up after bringing the link down, then up:
>  ip link set dev eth0 down
>  ip link set dev eth0 up
> 
> WARNING: CPU: 0 PID: 55 at kernel/dma/debug.c:570 add_dma_entry+0x204/0x2ec
> DMA-API: moxart-ethernet 92000000.mac: cacheline tracking EEXIST, overlapping mappings aren't supported
> CPU: 0 PID: 55 Comm: ip Not tainted 5.19.0+ #57
> Hardware name: Generic DT based system
>  unwind_backtrace from show_stack+0x10/0x14
>  show_stack from dump_stack_lvl+0x34/0x44
>  dump_stack_lvl from __warn+0xbc/0x1f0
>  __warn from warn_slowpath_fmt+0x94/0xc8
>  warn_slowpath_fmt from add_dma_entry+0x204/0x2ec
>  add_dma_entry from dma_map_page_attrs+0x110/0x328
>  dma_map_page_attrs from moxart_mac_open+0x134/0x320
>  moxart_mac_open from __dev_open+0x11c/0x1ec
>  __dev_open from __dev_change_flags+0x194/0x22c
>  __dev_change_flags from dev_change_flags+0x14/0x44
>  dev_change_flags from devinet_ioctl+0x6d4/0x93c
>  devinet_ioctl from inet_ioctl+0x1ac/0x25c
> 
> Unmap RX memory areas in moxart_mac_stop(), so that moxart_mac_open()
> will map them anew instead of double-mapping. To avoid code duplication,
> create a new function moxart_mac_unmap_rx(). Nullify unmapped pointers to
> prevent double-unmapping (ex: moxart_mac_stop(), then moxart_remove()).

This makes the code symmetric, which is good.

However, moxart_mac_free_memory() will also free the descriptors,
which is not required. moxart_remove() should undo what the probe did,
nothing more.

	Andrew

Re: [PATCH 2/2] net: moxa: prevent double-mapping of DMA areas

[Sergei Antonov]

On Thu, 18 Aug 2022 at 22:27, Andrew Lunn <andrew@lunn.ch> wrote:
> > Unmap RX memory areas in moxart_mac_stop(), so that moxart_mac_open()
> > will map them anew instead of double-mapping. To avoid code duplication,
> > create a new function moxart_mac_unmap_rx(). Nullify unmapped pointers to
> > prevent double-unmapping (ex: moxart_mac_stop(), then moxart_remove()).
>
> This makes the code symmetric, which is good.
>
> However, moxart_mac_free_memory() will also free the descriptors,
> which is not required. moxart_remove() should undo what the probe did,
> nothing more.

Having considered your comments, I now think I should make another
patch, which fixes two problems at once. To unmap DMA areas only in
moxart_mac_stop() and not in moxart_remove(). It will fix error
unwinding during probe and double-mapping on link down, link up. It
will be correct if Linux always calls moxart_mac_stop() before calling
moxart_remove().

Re: [PATCH 2/2] net: moxa: prevent double-mapping of DMA areas

[Vladimir Oltean]

On Fri, Aug 19, 2022 at 11:12:11AM +0300, Sergei Antonov wrote:
> On Thu, 18 Aug 2022 at 22:27, Andrew Lunn <andrew@lunn.ch> wrote:
> > > Unmap RX memory areas in moxart_mac_stop(), so that moxart_mac_open()
> > > will map them anew instead of double-mapping. To avoid code duplication,
> > > create a new function moxart_mac_unmap_rx(). Nullify unmapped pointers to
> > > prevent double-unmapping (ex: moxart_mac_stop(), then moxart_remove()).
> >
> > This makes the code symmetric, which is good.
> >
> > However, moxart_mac_free_memory() will also free the descriptors,
> > which is not required. moxart_remove() should undo what the probe did,
> > nothing more.
> 
> Having considered your comments, I now think I should make another
> patch, which fixes two problems at once. To unmap DMA areas only in
> moxart_mac_stop() and not in moxart_remove(). It will fix error
> unwinding during probe and double-mapping on link down, link up. It
> will be correct if Linux always calls moxart_mac_stop() before calling
> moxart_remove().

The way this is supposed to work is that from moxart_remove() you call
unregister_netdev(), and this calls dev_close[_many]() internally. The
dev_close() method calls your ndo_stop callback.

Re: [PATCH 1/2] net: moxa: do not call dma_unmap_single() with null

[Andrew Lunn]

On Thu, Aug 18, 2022 at 09:29:47PM +0300, Sergei Antonov wrote:
> It fixes a warning during error unwinding:
> 
> WARNING: CPU: 0 PID: 1 at kernel/dma/debug.c:963 check_unmap+0x704/0x980
> DMA-API: moxart-ethernet 92000000.mac: device driver tries to free DMA memory it has not allocated [device address=0x0000000000000000] [size=1600 bytes]
> CPU: 0 PID: 1 Comm: swapper Not tainted 5.19.0+ #60
> Hardware name: Generic DT based system
>  unwind_backtrace from show_stack+0x10/0x14
>  show_stack from dump_stack_lvl+0x34/0x44
>  dump_stack_lvl from __warn+0xbc/0x1f0
>  __warn from warn_slowpath_fmt+0x94/0xc8
>  warn_slowpath_fmt from check_unmap+0x704/0x980
>  check_unmap from debug_dma_unmap_page+0x8c/0x9c
>  debug_dma_unmap_page from moxart_mac_free_memory+0x3c/0xa8
>  moxart_mac_free_memory from moxart_mac_probe+0x190/0x218
>  moxart_mac_probe from platform_probe+0x48/0x88
>  platform_probe from really_probe+0xc0/0x2e4
> 
> Fixes: 6c821bd9edc9 ("net: Add MOXA ART SoCs ethernet driver")
> Signed-off-by: Sergei Antonov <saproj@gmail.com>

This looks correct as it is:

Reviewed-by: Andrew Lunn <andrew@lunn.ch>

But i do wonder how it go into the situation:

	for (i = 0; i < RX_DESC_NUM; i++) {
		desc = priv->rx_desc_base + i * RX_REG_DESC_SIZE;
		memset(desc, 0, RX_REG_DESC_SIZE);
		moxart_desc_write(RX_DESC0_DMA_OWN, desc + RX_REG_OFFSET_DESC0);
		moxart_desc_write(RX_BUF_SIZE & RX_DESC1_BUF_SIZE_MASK,
		       desc + RX_REG_OFFSET_DESC1);

		priv->rx_buf[i] = priv->rx_buf_base + priv->rx_buf_size * i;
		priv->rx_mapping[i] = dma_map_single(&ndev->dev,
						     priv->rx_buf[i],
						     priv->rx_buf_size,
						     DMA_FROM_DEVICE);
		if (dma_mapping_error(&ndev->dev, priv->rx_mapping[i]))
			netdev_err(ndev, "DMA mapping error\n");

		moxart_desc_write(priv->rx_mapping[i],
		       desc + RX_REG_OFFSET_DESC2 + RX_DESC2_ADDRESS_PHYS);
		moxart_desc_write((uintptr_t)priv->rx_buf[i],
		       desc + RX_REG_OFFSET_DESC2 + RX_DESC2_ADDRESS_VIRT);
	}

There is no way out of this, such that it only allocates some but not
all. So maybe there was an dma_mapping_error, it printed: DMA mapping
error, but kept going? So maybe another patch would be good, making
moxart_mac_setup_desc_ring() return an error when something goes
wrong?

    Andrew

Re: [PATCH 1/2] net: moxa: do not call dma_unmap_single() with null

[Sergei Antonov]

On Thu, 18 Aug 2022 at 22:18, Andrew Lunn <andrew@lunn.ch> wrote:
>
> On Thu, Aug 18, 2022 at 09:29:47PM +0300, Sergei Antonov wrote:
> > It fixes a warning during error unwinding:
> >
> > WARNING: CPU: 0 PID: 1 at kernel/dma/debug.c:963 check_unmap+0x704/0x980
> > DMA-API: moxart-ethernet 92000000.mac: device driver tries to free DMA memory it has not allocated [device address=0x0000000000000000] [size=1600 bytes]
> > CPU: 0 PID: 1 Comm: swapper Not tainted 5.19.0+ #60
> > Hardware name: Generic DT based system
> >  unwind_backtrace from show_stack+0x10/0x14
> >  show_stack from dump_stack_lvl+0x34/0x44
> >  dump_stack_lvl from __warn+0xbc/0x1f0
> >  __warn from warn_slowpath_fmt+0x94/0xc8
> >  warn_slowpath_fmt from check_unmap+0x704/0x980
> >  check_unmap from debug_dma_unmap_page+0x8c/0x9c
> >  debug_dma_unmap_page from moxart_mac_free_memory+0x3c/0xa8
> >  moxart_mac_free_memory from moxart_mac_probe+0x190/0x218
> >  moxart_mac_probe from platform_probe+0x48/0x88
> >  platform_probe from really_probe+0xc0/0x2e4
> >
> > Fixes: 6c821bd9edc9 ("net: Add MOXA ART SoCs ethernet driver")
> > Signed-off-by: Sergei Antonov <saproj@gmail.com>
>
> This looks correct as it is:
>
> Reviewed-by: Andrew Lunn <andrew@lunn.ch>
>
> But i do wonder how it go into the situation:

An error during moxart_mac_probe() does "goto init_fail;" which calls
moxart_mac_free_memory(). And by that time, priv->rx_mapping[i] are
all zero yet.