WARNING: drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c:364 vchiq_prepare_bulk_data

[Stefan Wahren <wahrenst@gmx.net>]

Hi,
i was able to trigger a flood of WARNINGs with Raspberry Pi 3 B Plus (
arm64/defconfig + CONFIG_BCM2835_VCHIQ=m + CONFIG_VCHIQ_CDEV=y ) running
the following userspace tool (triggers DMA transfer)

vchiq_test -f 1

The command seems to succeed as usual except of the warnings. But these
warnings are not reproducible with 32 bit configuration like
multi_v7_defconfig on the same Raspberry Pi 3 B Plus:

[   64.531826] WARNING: CPU: 3 PID: 867 at
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c:364
vchiq_prepare_bulk_data+0x2cc/0x4e8 [vchiq]
[   64.531880] Modules linked in: aes_neon_blk af_alg 8021q garp mrp stp
llc microchip raspberrypi_cpufreq brcmfmac_wcc snd_bcm2835(C) lan78xx
brcmfmac brcmutil vc4 cfg80211 hci_uart snd_soc_hdmi_codec btqca cec
btbcm onboard_usb_hub drm_display_helper bluetooth crct10dif_ce
drm_dma_helper raspberrypi_hwmon clk_raspberrypi drm_kms_helper
ecdh_generic bcm2835_thermal i2c_bcm2835 ecc pwm_bcm2835 rfkill vchiq(C)
drm fuse ip_tables x_tables ipv6
[   64.532010] CPU: 3 PID: 867 Comm: vchiq_test Tainted: G WC        
6.4.0-rc4-00297-gf999f38b4e6f #24
[   64.532022] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT)
[   64.532029] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS
BTYPE=--)
[   64.532038] pc : vchiq_prepare_bulk_data+0x2cc/0x4e8 [vchiq]
[   64.532060] lr : vchiq_prepare_bulk_data+0x320/0x4e8 [vchiq]
[   64.532080] sp : ffff8000093dbb10
[   64.532086] x29: ffff8000093dbb10 x28: 00000000000000a4 x27:
ffff80000b065000
[   64.532101] x26: ffff80000b065010 x25: ffff80000b065060 x24:
0000000000000000
[   64.532114] x23: 0000000000000001 x22: 0000000000000001 x21:
0000000000000001
[   64.532127] x20: ffff80000b065040 x19: 0000000000000002 x18:
0000000000000800
[   64.532139] x17: 001fffffffffffff x16: ffffdefad16fc6d0 x15:
ffff1449b720713c
[   64.532151] x14: 0000000000000000 x13: ffff1449b7207130 x12:
ffff1449b7207100
[   64.532163] x11: 0000000000000030 x10: 00000000000000ff x9 :
000000003f000000
[   64.532175] x8 : 00000000351ae800 x7 : 00000000351ae800 x6 :
00000000000000ff
[   64.532187] x5 : ffff1449b51ae801 x4 : ffff1449b9efc001 x3 :
0000000000000000
[   64.532201] x2 : 00000000f51ae800 x1 : fffffc5126e7bf02 x0 :
0000000000000001
[   64.532218] Call trace:
[   64.532222]  vchiq_prepare_bulk_data+0x2cc/0x4e8 [vchiq]
[   64.532247]  vchiq_bulk_transfer+0x25c/0x3bc [vchiq]
[   64.532272]  vchiq_irq_queue_bulk_tx_rx+0x6c/0x348 [vchiq]
[   64.532293]  vchiq_ioctl+0x684/0xbe8 [vchiq]
[   64.532313]  __arm64_sys_ioctl+0xa8/0xec
[   64.532327]  invoke_syscall+0x48/0x114
[   64.532341]  el0_svc_common.constprop.0+0x44/0xf4
[   64.532353]  do_el0_svc+0x3c/0xa8
[   64.532362]  el0_svc+0x2c/0x84
[   64.532379]  el0t_64_sync_handler+0xbc/0x138
[   64.532389]  el0t_64_sync+0x190/0x194
[   64.532400] ---[ end trace 0000000000000000 ]---

Since there are a lot of WARN_ON in the function, i applied the
following patch:

diff --git
a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index 9fb8f657cc78..2beeb94f629e 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -192,7 +192,7 @@ cleanup_pagelistinfo(struct vchiq_instance
*instance, struct vchiq_pagelist_info
  }

  static inline bool
-is_adjacent_block(u32 *addrs, u32 addr, unsigned int k)
+is_adjacent_block(u32 *addrs, dma_addr_t addr, unsigned int k)
  {
      u32 tmp;

@@ -357,15 +357,19 @@ create_pagelist(struct vchiq_instance *instance,
char *buf, char __user *ubuf,
      k = 0;
      for_each_sg(scatterlist, sg, dma_buffers, i) {
          u32 len = sg_dma_len(sg);
-        u32 addr = sg_dma_address(sg);
+        dma_addr_t addr = sg_dma_address(sg);

          /* Note: addrs is the address + page_count - 1
           * The firmware expects blocks after the first to be page-
           * aligned and a multiple of the page size
           */
-        WARN_ON(len == 0);
-        WARN_ON(i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK));
-        WARN_ON(i && (addr & ~PAGE_MASK));
+        if (len == 0)
+            pr_warn_once("%s: sg_dma_len() == 0\n", __func__);
+        else if (i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK))
+            pr_warn_once("%s: following block not page aligned\n",
__func__);
+        else if (i && (addr & ~PAGE_MASK))
+            pr_warn_once("%s: block %u, DMA address %pad doesn't align
with PAGE_MASK 0x%lx\n", __func__, i, &addr, PAGE_MASK);
+
          if (is_adjacent_block(addrs, addr, k))
              addrs[k - 1] += ((len + PAGE_SIZE - 1) >> PAGE_SHIFT);
          else
@@ -385,11 +389,15 @@ create_pagelist(struct vchiq_instance *instance,
char *buf, char __user *ubuf,
              return NULL;
          }

-        WARN_ON(!g_free_fragments);
+        if (!g_free_fragments)
+            pr_warn_once("%s: !g_free_fragments\n", __func__);

          down(&g_free_fragments_mutex);
          fragments = g_free_fragments;
-        WARN_ON(!fragments);
+
+        if (!fragments)
+            pr_warn_once("%s: !fragments\n", __func__);
+
          g_free_fragments = *(char **)g_free_fragments;
          up(&g_free_fragments_mutex);
          pagelist->type = PAGELIST_READ_WITH_FRAGMENTS +

After that i'm getting the following warning:

   create_pagelist: block 1, DMA address 0x00000000f5f62800 doesn't
align with PAGE_MASK 0xfffffffffffff000

Then i bisected the issue to this commit:

commit 1c1a429efd4ee8ca244cc2401365c983cda4ed76
Author: Catalin Marinas <catalin.marinas@arm.com>
Date:   Mon Jun 12 16:32:01 2023 +0100

     arm64: enable ARCH_WANT_KMALLOC_DMA_BOUNCE for arm64

     With the DMA bouncing of unaligned kmalloc() buffers now in place,
enable
     it for arm64 to allow the kmalloc-{8,16,32,48,96} caches.  In addition,
     always create the swiotlb buffer even when the end of RAM is within the
     32-bit physical address range (the swiotlb buffer can still be
disabled on
     the kernel command line).

So how can the root cause of these warnings be fixed?
Or is the specific WARN_ON() now obsolete?


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel