[6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Wang Yugui]

Hi,

I noticed a WARNING in recent 6.12.y kernel.

This WARNING happen on 6.12.26/6.12.25, but not happen on 6.12.20.

More bisect job need to be done, but reporti it firstly.

[   13.288365] ------------[ cut here ]------------
[   13.288366] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0
void blk_mq_map_hw_queues(struct blk_mq_queue_map *qmap,
              struct device *dev, unsigned int offset)
{
...
...
fallback:
L90：    WARN_ON_ONCE(qmap->nr_queues > 1);
    blk_mq_clear_mq_map(qmap);
}

[   13.288373] Modules linked in: ahci(+) mlxfw libahci pci_hyperv_intf bnx2x(+) i40e(+) mpi3mr(+) psample bnxt_en(+) libata mgag200(+) tls megaraid_sas(+) crc32c_intel scsi_transport_sas mdio libie i2c_algo_bit wmi dm_mirror dm_region_hash dm_log dm_mod
[   13.288386] CPU: 0 UID: 0 PID: 681 Comm: kworker/0:2 Not tainted 6.12.26-1.el7.x86_64 #1
[   13.288388] Hardware name: Dell Inc. PowerEdge T640/0TWW5Y, BIOS 2.22.1 09/12/2024
[   13.288390] Workqueue: events work_for_cpu_fn
[   13.288394] RIP: 0010:blk_mq_map_hw_queues+0xcf/0xe0
[   13.288396] Code: 8b 35 85 e4 9a 02 48 63 d0 48 c7 c7 e0 e5 78 94 e8 26 0b 07 00 39 05 70 e4 9a 02 77 d3 5b 5d 41 5c 41 5d 41 5e c3 cc cc cc cc <0f> 0b eb d2 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 90 90 90 90 90
[   13.288398] RSP: 0018:ffffc0e11b83bd38 EFLAGS: 00010212
[   13.288400] RAX: 0000000000000000 RBX: ffffa0804ad58000 RCX: 0000000000000050
[   13.288402] RDX: 0000000000000050 RSI: ffffa07f8f24d0c8 RDI: ffffa0804ad580e8
[   13.288403] RBP: ffffa0804ad580e0 R08: 000000000000004f R09: 0000000000000000
[   13.288404] R10: ffffc0e11b83bd78 R11: ffffa07f9039da00 R12: 0000000000000000
[   13.288405] R13: 0000000000000001 R14: ffffa0804ad580e8 R15: 0000000000000000
[   13.288406] FS:  0000000000000000(0000) GS:ffffa0dd3fe00000(0000) knlGS:0000000000000000
[   13.288407] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   13.288408] CR2: 00007f1f3226ba20 CR3: 00000002f9780005 CR4: 00000000007706f0
[   13.288409] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   13.288410] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   13.288411] PKRU: 55555554
[   13.288412] Call Trace:
[   13.288413]  <TASK>
[   13.288415]  megasas_map_queues+0x49/0x90 [megaraid_sas]
[   13.288431]  blk_mq_alloc_tag_set+0x150/0x3b0
[   13.288436]  scsi_add_host_with_dma+0xd0/0x350
[   13.288440]  megasas_io_attach+0x15c/0x220 [megaraid_sas]
[   13.288456]  megasas_probe_one+0x1cb/0x570 [megaraid_sas]
[   13.288468]  local_pci_probe+0x47/0xa0
[   13.288474]  work_for_cpu_fn+0x17/0x30
[   13.288475]  process_one_work+0x179/0x3a0
[   13.288480]  worker_thread+0x24b/0x350
[   13.288483]  ? __pfx_worker_thread+0x10/0x10
[   13.288485]  kthread+0xde/0x110
[   13.288489]  ? __pfx_kthread+0x10/0x10
[   13.288491]  ret_from_fork+0x31/0x50
[   13.288494]  ? __pfx_kthread+0x10/0x10
[   13.288496]  ret_from_fork_asm+0x1a/0x30
[   13.288502]  </TASK>
[   13.288502] ---[ end trace 0000000000000000 ]---


Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/05/03

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Wang Yugui]

Hi,
Cc: Greg Kroah-Hartman

> Hi,
> 
> I noticed a WARNING in recent 6.12.y kernel.
> 
> This WARNING happen on 6.12.26/6.12.25, but not happen on 6.12.20.
> 
> More bisect job need to be done, but reporti it firstly.
> 
> [   13.288365] ------------[ cut here ]------------
> [   13.288366] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0
> void blk_mq_map_hw_queues(struct blk_mq_queue_map *qmap,
>               struct device *dev, unsigned int offset)
> {
> ...
> ...
> fallback:
> L90：    WARN_ON_ONCE(qmap->nr_queues > 1);
>     blk_mq_clear_mq_map(qmap);
> }
> 

The following patch fixed this WARNING.

From a9ae6fe1c319c4776c2b11e85e15109cd3f04076 Mon Sep 17 00:00:00 2001
From: Daniel Wagner <wagi@kernel.org>
Date: Thu, 23 Jan 2025 14:08:29 +0100
Subject: [PATCH] blk-mq: create correct map for fallback case

please pull it to 6.12.y.

Best Regards
Wang Yugui (wangyugui@e16-tech.com)
2025/05/03


> [   13.288373] Modules linked in: ahci(+) mlxfw libahci pci_hyperv_intf bnx2x(+) i40e(+) mpi3mr(+) psample bnxt_en(+) libata mgag200(+) tls megaraid_sas(+) crc32c_intel scsi_transport_sas mdio libie i2c_algo_bit wmi dm_mirror dm_region_hash dm_log dm_mod
> [   13.288386] CPU: 0 UID: 0 PID: 681 Comm: kworker/0:2 Not tainted 6.12.26-1.el7.x86_64 #1
> [   13.288388] Hardware name: Dell Inc. PowerEdge T640/0TWW5Y, BIOS 2.22.1 09/12/2024
> [   13.288390] Workqueue: events work_for_cpu_fn
> [   13.288394] RIP: 0010:blk_mq_map_hw_queues+0xcf/0xe0
> [   13.288396] Code: 8b 35 85 e4 9a 02 48 63 d0 48 c7 c7 e0 e5 78 94 e8 26 0b 07 00 39 05 70 e4 9a 02 77 d3 5b 5d 41 5c 41 5d 41 5e c3 cc cc cc cc <0f> 0b eb d2 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 90 90 90 90 90
> [   13.288398] RSP: 0018:ffffc0e11b83bd38 EFLAGS: 00010212
> [   13.288400] RAX: 0000000000000000 RBX: ffffa0804ad58000 RCX: 0000000000000050
> [   13.288402] RDX: 0000000000000050 RSI: ffffa07f8f24d0c8 RDI: ffffa0804ad580e8
> [   13.288403] RBP: ffffa0804ad580e0 R08: 000000000000004f R09: 0000000000000000
> [   13.288404] R10: ffffc0e11b83bd78 R11: ffffa07f9039da00 R12: 0000000000000000
> [   13.288405] R13: 0000000000000001 R14: ffffa0804ad580e8 R15: 0000000000000000
> [   13.288406] FS:  0000000000000000(0000) GS:ffffa0dd3fe00000(0000) knlGS:0000000000000000
> [   13.288407] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   13.288408] CR2: 00007f1f3226ba20 CR3: 00000002f9780005 CR4: 00000000007706f0
> [   13.288409] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   13.288410] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [   13.288411] PKRU: 55555554
> [   13.288412] Call Trace:
> [   13.288413]  <TASK>
> [   13.288415]  megasas_map_queues+0x49/0x90 [megaraid_sas]
> [   13.288431]  blk_mq_alloc_tag_set+0x150/0x3b0
> [   13.288436]  scsi_add_host_with_dma+0xd0/0x350
> [   13.288440]  megasas_io_attach+0x15c/0x220 [megaraid_sas]
> [   13.288456]  megasas_probe_one+0x1cb/0x570 [megaraid_sas]
> [   13.288468]  local_pci_probe+0x47/0xa0
> [   13.288474]  work_for_cpu_fn+0x17/0x30
> [   13.288475]  process_one_work+0x179/0x3a0
> [   13.288480]  worker_thread+0x24b/0x350
> [   13.288483]  ? __pfx_worker_thread+0x10/0x10
> [   13.288485]  kthread+0xde/0x110
> [   13.288489]  ? __pfx_kthread+0x10/0x10
> [   13.288491]  ret_from_fork+0x31/0x50
> [   13.288494]  ? __pfx_kthread+0x10/0x10
> [   13.288496]  ret_from_fork_asm+0x1a/0x30
> [   13.288502]  </TASK>
> [   13.288502] ---[ end trace 0000000000000000 ]---
> 
> 
> Best Regards
> Wang Yugui (wangyugui@e16-tech.com)
> 2025/05/03
> 

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Greg Kroah-Hartman]

On Sat, May 03, 2025 at 10:05:29AM +0800, Wang Yugui wrote:
> Hi,
> Cc: Greg Kroah-Hartman
> 
> > Hi,
> > 
> > I noticed a WARNING in recent 6.12.y kernel.
> > 
> > This WARNING happen on 6.12.26/6.12.25, but not happen on 6.12.20.
> > 
> > More bisect job need to be done, but reporti it firstly.
> > 
> > [   13.288365] ------------[ cut here ]------------
> > [   13.288366] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0
> > void blk_mq_map_hw_queues(struct blk_mq_queue_map *qmap,
> >               struct device *dev, unsigned int offset)
> > {
> > ...
> > ...
> > fallback:
> > L90：    WARN_ON_ONCE(qmap->nr_queues > 1);
> >     blk_mq_clear_mq_map(qmap);
> > }
> > 
> 
> The following patch fixed this WARNING.
> 
> From a9ae6fe1c319c4776c2b11e85e15109cd3f04076 Mon Sep 17 00:00:00 2001
> From: Daniel Wagner <wagi@kernel.org>
> Date: Thu, 23 Jan 2025 14:08:29 +0100
> Subject: [PATCH] blk-mq: create correct map for fallback case
> 
> please pull it to 6.12.y.

But the commit that this one says it fixes is NOT in the 6.12.y tree.
So why would this commit fix the issue?

And why haven't you cc:ed the authors involved in these changes?

confused,

greg k-h

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Jack Wang]

From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

In linux-6.12.y, commit 5e8438fd7f11 ("scsi: replace blk_mq_pci_map_queues with blk_mq_map_hw_queues")
was pulled in as depandency, the fix a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
should have just used 1452e9b470c9 ("blk-mq: introduce blk_mq_map_hw_queues")
as Fixes, not the other conversion IMO.

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Greg KH]

On Mon, May 05, 2025 at 01:36:52PM +0200, Jack Wang wrote:
> From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> 
> In linux-6.12.y, commit 5e8438fd7f11 ("scsi: replace blk_mq_pci_map_queues with blk_mq_map_hw_queues")
> was pulled in as depandency, the fix a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
> should have just used 1452e9b470c9 ("blk-mq: introduce blk_mq_map_hw_queues")
> as Fixes, not the other conversion IMO.

What "other conversion"?  Sorry, I do not understand, did we take a
patch we shouldn't have, or did we miss a patch we should have applied?

confused,

greg k-h

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Daniel Wagner]

On Mon, May 05, 2025 at 03:28:22PM +0200, Greg KH wrote:
> On Mon, May 05, 2025 at 01:36:52PM +0200, Jack Wang wrote:
> > From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > 
> > In linux-6.12.y, commit 5e8438fd7f11 ("scsi: replace blk_mq_pci_map_queues with blk_mq_map_hw_queues")
> > was pulled in as depandency, the fix a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
> > should have just used 1452e9b470c9 ("blk-mq: introduce blk_mq_map_hw_queues")
> > as Fixes, not the other conversion IMO.
> 
> What "other conversion"?  Sorry, I do not understand, did we take a
> patch we shouldn't have, or did we miss a patch we should have applied?

If I understand the situation correctly, the problem is that v6.14.25
ships commit 5e8438fd7f11 ("scsi: replace blk_mq_pci_map_queues with
blk_mq_map_hw_queues") which introduced a regression for certain virtio
configurations. The fixup patch is:

a9ae6fe1c319 ("blk-mq: create correct map for fallback case")

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Jinpu Wang]

On Mon, May 5, 2025 at 3:28 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Mon, May 05, 2025 at 01:36:52PM +0200, Jack Wang wrote:
> > From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> >
> > In linux-6.12.y, commit 5e8438fd7f11 ("scsi: replace blk_mq_pci_map_queues with blk_mq_map_hw_queues")
> > was pulled in as depandency, the fix a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
> > should have just used 1452e9b470c9 ("blk-mq: introduce blk_mq_map_hw_queues")
> > as Fixes, not the other conversion IMO.
>
> What "other conversion"?
Sorry, I meant  in the mentioned Fixes in commit a9ae6fe1c319
("blk-mq: create correct map for fallback case")
"    Fixes: a5665c3d150c ("virtio: blk/scsi: replace
blk_mq_virtio_map_queues with blk_mq_map_hw_queues")
    Reported-by: Steven Rostedt <rostedt@goodmis.org>
" which use virtio: blk/scsi: replace blk_mq_virtio_map_queues with
blk_mq_map_hw_queues, should have
> Sorry, I do not understand, did we take a
> patch we shouldn't have, or did we miss a patch we should have applied?
We missed the fix a9ae6fe1c319 ("blk-mq: create correct map for
fallback case")for stable linux 6.12.y, which should be applied.
>
> confused,
>
> greg k-h
Sorry for the confusion.

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Greg KH]

On Mon, May 05, 2025 at 03:40:51PM +0200, Daniel Wagner wrote:
> On Mon, May 05, 2025 at 03:28:22PM +0200, Greg KH wrote:
> > On Mon, May 05, 2025 at 01:36:52PM +0200, Jack Wang wrote:
> > > From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > 
> > > In linux-6.12.y, commit 5e8438fd7f11 ("scsi: replace blk_mq_pci_map_queues with blk_mq_map_hw_queues")
> > > was pulled in as depandency, the fix a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
> > > should have just used 1452e9b470c9 ("blk-mq: introduce blk_mq_map_hw_queues")
> > > as Fixes, not the other conversion IMO.
> > 
> > What "other conversion"?  Sorry, I do not understand, did we take a
> > patch we shouldn't have, or did we miss a patch we should have applied?
> 
> If I understand the situation correctly, the problem is that v6.14.25
> ships commit 5e8438fd7f11 ("scsi: replace blk_mq_pci_map_queues with
> blk_mq_map_hw_queues") which introduced a regression for certain virtio
> configurations. The fixup patch is:
> 
> a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
> 

But commit a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
says it fixes commit a5665c3d150c ("virtio: blk/scsi: replace
blk_mq_virtio_map_queues with blk_mq_map_hw_queues"), which is ONLY in
the 6.14 tree.

Which is why we didn't pull it into the stable tree here.  Is that
commit just not marked properly?  Will it cause problems if it is
backported?

thanks,

greg k-h

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Daniel Wagner]

On Mon, May 05, 2025 at 03:52:38PM +0200, Greg KH wrote:
> But commit a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
> says it fixes commit a5665c3d150c ("virtio: blk/scsi: replace
> blk_mq_virtio_map_queues with blk_mq_map_hw_queues"), which is ONLY in
> the 6.14 tree.
> 
> Which is why we didn't pull it into the stable tree here.  Is that
> commit just not marked properly?  Will it cause problems if it is
> backported?

The stack trace for this report is from a system which has a
megaraid_sas HBA. I did some testing with this HBA but somehow I
didn't run into the problem. Thus the Fixes is just not complete, it
misses

Fixes: bd326a5ad639 ("scsi: replace blk_mq_pci_map_queues with blk_mq_map_hw_queues")

Sorry about that.

Re: [6.12.y] WARNING: CPU: 0 PID: 681 at block/blk-mq-cpumap.c:90 blk_mq_map_hw_queues+0xcf/0xe0

[Greg KH]

On Mon, May 05, 2025 at 04:05:58PM +0200, Daniel Wagner wrote:
> On Mon, May 05, 2025 at 03:52:38PM +0200, Greg KH wrote:
> > But commit a9ae6fe1c319 ("blk-mq: create correct map for fallback case")
> > says it fixes commit a5665c3d150c ("virtio: blk/scsi: replace
> > blk_mq_virtio_map_queues with blk_mq_map_hw_queues"), which is ONLY in
> > the 6.14 tree.
> > 
> > Which is why we didn't pull it into the stable tree here.  Is that
> > commit just not marked properly?  Will it cause problems if it is
> > backported?
> 
> The stack trace for this report is from a system which has a
> megaraid_sas HBA. I did some testing with this HBA but somehow I
> didn't run into the problem. Thus the Fixes is just not complete, it
> misses
> 
> Fixes: bd326a5ad639 ("scsi: replace blk_mq_pci_map_queues with blk_mq_map_hw_queues")
> 
> Sorry about that.

No problem, thanks for letting me know, now queued up.

greg k-h