* [PATCH] virtio_pci: avoid to request intx irq if irq is zero
@ 2022-09-28 3:57 Angus Chen
2022-09-28 4:03 ` Michael S. Tsirkin
0 siblings, 1 reply; 3+ messages in thread
From: Angus Chen @ 2022-09-28 3:57 UTC (permalink / raw)
To: jasowang, mst; +Cc: virtualization, linux-kernel, Angus Chen
The background is that we use dpu in cloud computing,the arch is x86,80
cores.We will have a lots of virtio devices,like 512 or more.
When we probe about 200 virtio_blk devices,it will fail and
the stack is print as follows:
[25338.485128] virtio-pci 0000:b3:00.0: virtio_pci: leaving for legacy driver
[25338.496174] genirq: Flags mismatch irq 0. 00000080 (virtio418) vs. 00015a00 (timer)
[25338.503822] CPU: 20 PID: 5431 Comm: kworker/20:0 Kdump: loaded Tainted: G OE --------- - - 4.18.0-305.30.1.el8.x86_64
[25338.516403] Hardware name: Inspur NF5280M5/YZMB-00882-10E, BIOS 4.1.21 08/25/2021
[25338.523881] Workqueue: events work_for_cpu_fn
[25338.528235] Call Trace:
[25338.530687] dump_stack+0x5c/0x80
[25338.534000] __setup_irq.cold.53+0x7c/0xd3
[25338.538098] request_threaded_irq+0xf5/0x160
[25338.542371] vp_find_vqs+0xc7/0x190
[25338.545866] init_vq+0x17c/0x2e0 [virtio_blk]
[25338.550223] ? ncpus_cmp_func+0x10/0x10
[25338.554061] virtblk_probe+0xe6/0x8a0 [virtio_blk]
[25338.558846] virtio_dev_probe+0x158/0x1f0
[25338.562861] really_probe+0x255/0x4a0
[25338.566524] ? __driver_attach_async_helper+0x90/0x90
[25338.571567] driver_probe_device+0x49/0xc0
[25338.575660] bus_for_each_drv+0x79/0xc0
[25338.579499] __device_attach+0xdc/0x160
[25338.583337] bus_probe_device+0x9d/0xb0
[25338.587167] device_add+0x418/0x780
[25338.590654] register_virtio_device+0x9e/0xe0
[25338.595011] virtio_pci_probe+0xb3/0x140
[25338.598941] local_pci_probe+0x41/0x90
[25338.602689] work_for_cpu_fn+0x16/0x20
[25338.606443] process_one_work+0x1a7/0x360
[25338.610456] ? create_worker+0x1a0/0x1a0
[25338.614381] worker_thread+0x1cf/0x390
[25338.618132] ? create_worker+0x1a0/0x1a0
[25338.622051] kthread+0x116/0x130
[25338.625283] ? kthread_flush_work_fn+0x10/0x10
[25338.629731] ret_from_fork+0x1f/0x40
[25338.633395] virtio_blk: probe of virtio418 failed with error -16
The log :
"genirq: Flags mismatch irq 0. 00000080 (virtio418) vs. 00015a00 (timer)"
was print because of the irq 0 is used by timer exclusive,and when
vp_find_vqs call vp_find_vqs_msix and return false twice for
whatever reason,then it will call vp_find_vqs_intx for the last try.
Because vp_dev->pci_dev->irq is zero,so it will be request irq 0 with
flag IRQF_SHARED,we will get a backtrace like above.
When the pci device is vf,the intx
is not allowed and it will be 0.Even if the pci device is a pf,
we should remind that
irq 0 is almost be reserved or used for some device fixed
like timer in x86 without IRQF_SHARED,so we can avoid call it.
Signed-off-by: Angus Chen <angus.chen@jaguarmicro.com>
Suggested-by: Michael S. Tsirkin <mst@redhat.com>
---
drivers/virtio/virtio_pci_common.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index ad258a9d3b9f..3208a2b149b9 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -362,6 +362,9 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs,
struct virtio_pci_device *vp_dev = to_vp_device(vdev);
int i, err, queue_idx = 0;
+ if (vp_dev->pci_dev->irq == 0)
+ return -EINVAL;
+
vp_dev->vqs = kcalloc(nvqs, sizeof(*vp_dev->vqs), GFP_KERNEL);
if (!vp_dev->vqs)
return -ENOMEM;
--
2.17.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] virtio_pci: avoid to request intx irq if irq is zero
2022-09-28 3:57 [PATCH] virtio_pci: avoid to request intx irq if irq is zero Angus Chen
@ 2022-09-28 4:03 ` Michael S. Tsirkin
2022-09-28 4:46 ` Angus Chen
0 siblings, 1 reply; 3+ messages in thread
From: Michael S. Tsirkin @ 2022-09-28 4:03 UTC (permalink / raw)
To: Angus Chen; +Cc: jasowang, virtualization, linux-kernel
On Wed, Sep 28, 2022 at 11:57:06AM +0800, Angus Chen wrote:
> The background is that we use dpu in cloud computing,the arch is x86,80
> cores.We will have a lots of virtio devices,like 512 or more.
> When we probe about 200 virtio_blk devices,it will fail and
> the stack is print as follows:
>
> [25338.485128] virtio-pci 0000:b3:00.0: virtio_pci: leaving for legacy driver
> [25338.496174] genirq: Flags mismatch irq 0. 00000080 (virtio418) vs. 00015a00 (timer)
> [25338.503822] CPU: 20 PID: 5431 Comm: kworker/20:0 Kdump: loaded Tainted: G OE --------- - - 4.18.0-305.30.1.el8.x86_64
> [25338.516403] Hardware name: Inspur NF5280M5/YZMB-00882-10E, BIOS 4.1.21 08/25/2021
> [25338.523881] Workqueue: events work_for_cpu_fn
> [25338.528235] Call Trace:
> [25338.530687] dump_stack+0x5c/0x80
> [25338.534000] __setup_irq.cold.53+0x7c/0xd3
> [25338.538098] request_threaded_irq+0xf5/0x160
> [25338.542371] vp_find_vqs+0xc7/0x190
> [25338.545866] init_vq+0x17c/0x2e0 [virtio_blk]
> [25338.550223] ? ncpus_cmp_func+0x10/0x10
> [25338.554061] virtblk_probe+0xe6/0x8a0 [virtio_blk]
> [25338.558846] virtio_dev_probe+0x158/0x1f0
> [25338.562861] really_probe+0x255/0x4a0
> [25338.566524] ? __driver_attach_async_helper+0x90/0x90
> [25338.571567] driver_probe_device+0x49/0xc0
> [25338.575660] bus_for_each_drv+0x79/0xc0
> [25338.579499] __device_attach+0xdc/0x160
> [25338.583337] bus_probe_device+0x9d/0xb0
> [25338.587167] device_add+0x418/0x780
> [25338.590654] register_virtio_device+0x9e/0xe0
> [25338.595011] virtio_pci_probe+0xb3/0x140
> [25338.598941] local_pci_probe+0x41/0x90
> [25338.602689] work_for_cpu_fn+0x16/0x20
> [25338.606443] process_one_work+0x1a7/0x360
> [25338.610456] ? create_worker+0x1a0/0x1a0
> [25338.614381] worker_thread+0x1cf/0x390
> [25338.618132] ? create_worker+0x1a0/0x1a0
> [25338.622051] kthread+0x116/0x130
> [25338.625283] ? kthread_flush_work_fn+0x10/0x10
> [25338.629731] ret_from_fork+0x1f/0x40
> [25338.633395] virtio_blk: probe of virtio418 failed with error -16
>
> The log :
> "genirq: Flags mismatch irq 0. 00000080 (virtio418) vs. 00015a00 (timer)"
> was print because of the irq 0 is used by timer exclusive,and when
> vp_find_vqs call vp_find_vqs_msix and return false twice for
> whatever reason,then it will call vp_find_vqs_intx for the last try.
> Because vp_dev->pci_dev->irq is zero,so it will be request irq 0 with
> flag IRQF_SHARED,we will get a backtrace like above.
>
> When the pci device is vf,the intx
> is not allowed and it will be 0.Even if the pci device is a pf,
> we should remind that
> irq 0 is almost be reserved or used for some device fixed
Where does it say irq 0 is reserved on all platforms?
I am not sure this is right.
> like timer in x86 without IRQF_SHARED,so we can avoid call it.
>
> Signed-off-by: Angus Chen <angus.chen@jaguarmicro.com>
> Suggested-by: Michael S. Tsirkin <mst@redhat.com>
> ---
> drivers/virtio/virtio_pci_common.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index ad258a9d3b9f..3208a2b149b9 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -362,6 +362,9 @@ static int vp_find_vqs_intx(struct virtio_device *vdev, unsigned int nvqs,
> struct virtio_pci_device *vp_dev = to_vp_device(vdev);
> int i, err, queue_idx = 0;
>
> + if (vp_dev->pci_dev->irq == 0)
> + return -EINVAL;
> +
Pls read the PCI spec:
Interrupt Pin
The Interrupt Pin register tells which interrupt pin the device (or device function) uses. A
value of 1 corresponds to INTA#. A value of 2 corresponds to INTB#. A value of 3
corresponds to INTC#. A value of 4 corresponds to INTD#. Devices (or device functions)
that do not use an interrupt pin must put a 0 in this register.
Please check vp_dev->pci_dev->pin accordingly.
> vp_dev->vqs = kcalloc(nvqs, sizeof(*vp_dev->vqs), GFP_KERNEL);
> if (!vp_dev->vqs)
> return -ENOMEM;
> --
> 2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread
* RE: [PATCH] virtio_pci: avoid to request intx irq if irq is zero
2022-09-28 4:03 ` Michael S. Tsirkin
@ 2022-09-28 4:46 ` Angus Chen
0 siblings, 0 replies; 3+ messages in thread
From: Angus Chen @ 2022-09-28 4:46 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: jasowang@redhat.com, virtualization@lists.linux-foundation.org,
linux-kernel@vger.kernel.org
> -----Original Message-----
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, September 28, 2022 12:03 PM
> To: Angus Chen <angus.chen@jaguarmicro.com>
> Cc: jasowang@redhat.com; virtualization@lists.linux-foundation.org;
> linux-kernel@vger.kernel.org
> Subject: Re: [PATCH] virtio_pci: avoid to request intx irq if irq is zero
>
> On Wed, Sep 28, 2022 at 11:57:06AM +0800, Angus Chen wrote:
> > The background is that we use dpu in cloud computing,the arch is x86,80
> > cores.We will have a lots of virtio devices,like 512 or more.
> > When we probe about 200 virtio_blk devices,it will fail and
> > the stack is print as follows:
> >
> > [25338.485128] virtio-pci 0000:b3:00.0: virtio_pci: leaving for legacy driver
> > [25338.496174] genirq: Flags mismatch irq 0. 00000080 (virtio418) vs.
> 00015a00 (timer)
> > [25338.503822] CPU: 20 PID: 5431 Comm: kworker/20:0 Kdump: loaded
> Tainted: G OE --------- - - 4.18.0-305.30.1.el8.x86_64
> > [25338.516403] Hardware name: Inspur NF5280M5/YZMB-00882-10E, BIOS
> 4.1.21 08/25/2021
> > [25338.523881] Workqueue: events work_for_cpu_fn
> > [25338.528235] Call Trace:
> > [25338.530687] dump_stack+0x5c/0x80
> > [25338.534000] __setup_irq.cold.53+0x7c/0xd3
> > [25338.538098] request_threaded_irq+0xf5/0x160
> > [25338.542371] vp_find_vqs+0xc7/0x190
> > [25338.545866] init_vq+0x17c/0x2e0 [virtio_blk]
> > [25338.550223] ? ncpus_cmp_func+0x10/0x10
> > [25338.554061] virtblk_probe+0xe6/0x8a0 [virtio_blk]
> > [25338.558846] virtio_dev_probe+0x158/0x1f0
> > [25338.562861] really_probe+0x255/0x4a0
> > [25338.566524] ? __driver_attach_async_helper+0x90/0x90
> > [25338.571567] driver_probe_device+0x49/0xc0
> > [25338.575660] bus_for_each_drv+0x79/0xc0
> > [25338.579499] __device_attach+0xdc/0x160
> > [25338.583337] bus_probe_device+0x9d/0xb0
> > [25338.587167] device_add+0x418/0x780
> > [25338.590654] register_virtio_device+0x9e/0xe0
> > [25338.595011] virtio_pci_probe+0xb3/0x140
> > [25338.598941] local_pci_probe+0x41/0x90
> > [25338.602689] work_for_cpu_fn+0x16/0x20
> > [25338.606443] process_one_work+0x1a7/0x360
> > [25338.610456] ? create_worker+0x1a0/0x1a0
> > [25338.614381] worker_thread+0x1cf/0x390
> > [25338.618132] ? create_worker+0x1a0/0x1a0
> > [25338.622051] kthread+0x116/0x130
> > [25338.625283] ? kthread_flush_work_fn+0x10/0x10
> > [25338.629731] ret_from_fork+0x1f/0x40
> > [25338.633395] virtio_blk: probe of virtio418 failed with error -16
> >
> > The log :
> > "genirq: Flags mismatch irq 0. 00000080 (virtio418) vs. 00015a00 (timer)"
> > was print because of the irq 0 is used by timer exclusive,and when
> > vp_find_vqs call vp_find_vqs_msix and return false twice for
> > whatever reason,then it will call vp_find_vqs_intx for the last try.
> > Because vp_dev->pci_dev->irq is zero,so it will be request irq 0 with
> > flag IRQF_SHARED,we will get a backtrace like above.
> >
> > When the pci device is vf,the intx
> > is not allowed and it will be 0.Even if the pci device is a pf,
> > we should remind that
> > irq 0 is almost be reserved or used for some device fixed
>
>
> Where does it say irq 0 is reserved on all platforms?
> I am not sure this is right.
>
>
> > like timer in x86 without IRQF_SHARED,so we can avoid call it.
> >
> > Signed-off-by: Angus Chen <angus.chen@jaguarmicro.com>
> > Suggested-by: Michael S. Tsirkin <mst@redhat.com>
> > ---
> > drivers/virtio/virtio_pci_common.c | 3 +++
> > 1 file changed, 3 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_pci_common.c
> b/drivers/virtio/virtio_pci_common.c
> > index ad258a9d3b9f..3208a2b149b9 100644
> > --- a/drivers/virtio/virtio_pci_common.c
> > +++ b/drivers/virtio/virtio_pci_common.c
> > @@ -362,6 +362,9 @@ static int vp_find_vqs_intx(struct virtio_device *vdev,
> unsigned int nvqs,
> > struct virtio_pci_device *vp_dev = to_vp_device(vdev);
> > int i, err, queue_idx = 0;
> >
> > + if (vp_dev->pci_dev->irq == 0)
> > + return -EINVAL;
> > +
>
>
> Pls read the PCI spec:
>
> Interrupt Pin
> The Interrupt Pin register tells which interrupt pin the device (or device function)
> uses. A
> value of 1 corresponds to INTA#. A value of 2 corresponds to INTB#. A value of 3
> corresponds to INTC#. A value of 4 corresponds to INTD#. Devices (or device
> functions)
> that do not use an interrupt pin must put a 0 in this register.
>
>
> Please check vp_dev->pci_dev->pin accordingly.
Ok,I will send this for next version.thank you.
>
>
>
>
> > vp_dev->vqs = kcalloc(nvqs, sizeof(*vp_dev->vqs), GFP_KERNEL);
> > if (!vp_dev->vqs)
> > return -ENOMEM;
> > --
> > 2.17.1
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-09-28 4:46 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-09-28 3:57 [PATCH] virtio_pci: avoid to request intx irq if irq is zero Angus Chen
2022-09-28 4:03 ` Michael S. Tsirkin
2022-09-28 4:46 ` Angus Chen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox