Re: [PATCH v4 00/14] drm: Add a driver for CSF-based Mali GPUs

[Boris Brezillon <boris.brezillon@collabora.com>]

On Sun, 4 Feb 2024 09:14:44 +0800 (CST)
"Andy Yan" <andyshrk@163.com> wrote:

> Hi Boris：
> I saw this warning sometimes（Run on a armbain based bookworm），not sure is a know issue or something else。

No it's not, and I didn't manage to reproduce locally. Looks like
you're using a 6.8 kernel, but my panthor-v4/next branches are still
based on drm-misc-next from 2 weeks ago, which was based on a 6.7
kernel. Can you share the kernel branch you're using?

> [15368.293031] systemd-journald[715]: Received client request to relinquish /var/log/journal/1bc4a340506142af9bd31a6a3d2170ba access.
> [37743.040737] ------------[ cut here ]------------
> [37743.040764] panthor fb000000.gpu: drm_WARN_ON(shmem->pages_use_count)
> [37743.040890] WARNING: CPU: 2 PID: 5702 at drivers/gpu/drm/drm_gem_shmem_helper.c:158 drm_gem_shmem_free+0x144/0x14c [drm_shmem_helper]
> [37743.040929] Modules linked in: joydev rfkill sunrpc lz4hc lz4 zram binfmt_misc hantro_vpu crct10dif_ce v4l2_vp9 v4l2_h264 snd_soc_simple_amplifier v4l2_mem2mem videobuf2_dma_contig snd_soc_es8328_i2c videobuf2_memops rk_crypto2 snd_soc_es8328 videobuf2_v4l2 sm3_generic videodev crypto_engine sm3 rockchip_rng videobuf2_common nvmem_rockchip_otp snd_soc_rockchip_i2s_tdm snd_soc_hdmi_codec snd_soc_simple_card mc snd_soc_simple_card_utils snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_pcm snd_timer snd soundcore dm_mod ip_tables x_tables autofs4 dw_hdmi_qp_i2s_audio dw_hdmi_qp_cec rk808_regulator rockchipdrm dw_mipi_dsi dw_hdmi_qp dw_hdmi analogix_dp drm_dma_helper fusb302 display_connector rk8xx_spi drm_display_helper phy_rockchip_snps_pcie3 phy_rockchip_samsung_hdptx_hdmi panthor tcpm rk8xx_core cec drm_gpuvm gpu_sched drm_kms_helper drm_shmem_helper drm_exec r8169 drm pwm_bl adc_keys
> [37743.041108] CPU: 2 PID: 5702 Comm: kworker/u16:8 Not tainted 6.8.0-rc1-edge-rockchip-rk3588 #2
> [37743.041115] Hardware name: Rockchip RK3588 EVB1 V10 Board (DT)
> [37743.041120] Workqueue: panthor-cleanup panthor_vm_bind_job_cleanup_op_ctx_work [panthor]
> [37743.041151] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [37743.041157] pc : drm_gem_shmem_free+0x144/0x14c [drm_shmem_helper]
> [37743.041169] lr : drm_gem_shmem_free+0x144/0x14c [drm_shmem_helper]
> [37743.041181] sp : ffff80008d37bcc0
> [37743.041184] x29: ffff80008d37bcc0 x28: ffff800081d379c0 x27: ffff800081d37000
> [37743.041196] x26: ffff00019909a280 x25: ffff00019909a2c0 x24: ffff0001017a4c05
> [37743.041206] x23: dead000000000100 x22: dead000000000122 x21: ffff0001627ac1a0
> [37743.041217] x20: 0000000000000000 x19: ffff0001627ac000 x18: 0000000000000000
> [37743.041227] x17: 000000040044ffff x16: 005000f2b5503510 x15: fffffffffff91b77
> [37743.041238] x14: 0000000000000001 x13: 00000000000003c5 x12: 00000000ffffffea
> [37743.041248] x11: 00000000ffffdfff x10: 00000000ffffdfff x9 : ffff800081e0e818
> [37743.041259] x8 : 000000000002ffe8 x7 : c0000000ffffdfff x6 : 00000000000affa8
> [37743.041269] x5 : 0000000000001fff x4 : 0000000000000000 x3 : ffff8000819a6008
> [37743.041279] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff00018465e900
> [37743.041290] Call trace:
> [37743.041293]  drm_gem_shmem_free+0x144/0x14c [drm_shmem_helper]
> [37743.041306]  panthor_gem_free_object+0x24/0xa0 [panthor]
> [37743.041321]  drm_gem_object_free+0x1c/0x30 [drm]
> [37743.041452]  panthor_vm_bo_put+0xc4/0x12c [panthor]

I checked the _pin/_unpin calls in panthor, and they seem to be
balanced (we take a ref when we allocate a gpuvm_bo and release it
when the gpuvm_bo is gone). I wonder if something else is calling
_pin_pages() or _get_pages() without holding a GEM ref...

While investigating I found a double-cleanup in the code (see below)
which explains why those memset(0) were required in
panthor_vm_cleanup_op_ctx()), but I doubt it fixes your issue.

--->8---
diff --git a/drivers/gpu/drm/panthor/panthor_mmu.c b/drivers/gpu/drm/panthor/panthor_mmu.c
index d3ce29cd0662..5606ab4d6289 100644
--- a/drivers/gpu/drm/panthor/panthor_mmu.c
+++ b/drivers/gpu/drm/panthor/panthor_mmu.c
@@ -1085,17 +1085,12 @@ static void panthor_vm_cleanup_op_ctx(struct panthor_vm_op_ctx *op_ctx,
        }
 
        kfree(op_ctx->rsvd_page_tables.pages);
-       memset(&op_ctx->rsvd_page_tables, 0, sizeof(op_ctx->rsvd_page_tables));
 
        if (op_ctx->map.vm_bo)
                panthor_vm_bo_put(op_ctx->map.vm_bo);
 
-       memset(&op_ctx->map, 0, sizeof(op_ctx->map));
-
-       for (u32 i = 0; i < ARRAY_SIZE(op_ctx->preallocated_vmas); i++) {
+       for (u32 i = 0; i < ARRAY_SIZE(op_ctx->preallocated_vmas); i++)
                kfree(op_ctx->preallocated_vmas[i]);
-               op_ctx->preallocated_vmas[i] = NULL;
-       }
 
        list_for_each_entry_safe(vma, tmp_vma, &op_ctx->returned_vmas, node) {
                list_del(&vma->node);
@@ -2382,7 +2377,6 @@ static void panthor_vm_bind_job_cleanup_op_ctx_work(struct work_struct *work)
        struct panthor_vm_bind_job *job =
                container_of(work, struct panthor_vm_bind_job, cleanup_op_ctx_work);
 
-       panthor_vm_cleanup_op_ctx(&job->ctx, job->vm);
        panthor_vm_bind_job_put(&job->base);
 }