[PATCH v2] virtio_pci: Switch away from deprecated irq_set_affinity_hint

[PATCH v2] virtio_pci: Switch away from deprecated irq_set_affinity_hint

[Jakub Sitnicki]

Since commit 65c7cdedeb30 ("genirq: Provide new interfaces for affinity
hints") irq_set_affinity_hint is being phased out.

Switch to new interfaces for setting and applying irq affinity hints.

Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
---
v2:
 - Leave cpumask_copy as is. We can't pass pointer to stack memory as hint.
   Proposed a change to IRQ affinity interface to address this limitation:
   https://lore.kernel.org/r/20231025141517.375378-1-jakub@cloudflare.com

v1: https://lore.kernel.org/r/20231019101625.412936-2-jakub@cloudflare.com
---
 drivers/virtio/virtio_pci_common.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
index c2524a7207cf..7a5593997e0e 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -242,7 +242,7 @@ void vp_del_vqs(struct virtio_device *vdev)
 			if (v != VIRTIO_MSI_NO_VECTOR) {
 				int irq = pci_irq_vector(vp_dev->pci_dev, v);
 
-				irq_set_affinity_hint(irq, NULL);
+				irq_update_affinity_hint(irq, NULL);
 				free_irq(irq, vq);
 			}
 		}
@@ -443,10 +443,10 @@ int vp_set_vq_affinity(struct virtqueue *vq, const struct cpumask *cpu_mask)
 		mask = vp_dev->msix_affinity_masks[info->msix_vector];
 		irq = pci_irq_vector(vp_dev->pci_dev, info->msix_vector);
 		if (!cpu_mask)
-			irq_set_affinity_hint(irq, NULL);
+			irq_update_affinity_hint(irq, NULL);
 		else {
 			cpumask_copy(mask, cpu_mask);
-			irq_set_affinity_hint(irq, mask);
+			irq_set_affinity_and_hint(irq, mask);
 		}
 	}
 	return 0;
-- 
2.41.0

Re: [PATCH v2] virtio_pci: Switch away from deprecated irq_set_affinity_hint

[Xuan Zhuo]

On Wed, 25 Oct 2023 16:53:19 +0200, Jakub Sitnicki <jakub@cloudflare.com> wrote:
> Since commit 65c7cdedeb30 ("genirq: Provide new interfaces for affinity
> hints") irq_set_affinity_hint is being phased out.
>
> Switch to new interfaces for setting and applying irq affinity hints.
>
> Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>

Reviewed-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>

Thanks.

> ---
> v2:
>  - Leave cpumask_copy as is. We can't pass pointer to stack memory as hint.
>    Proposed a change to IRQ affinity interface to address this limitation:
>    https://lore.kernel.org/r/20231025141517.375378-1-jakub@cloudflare.com
>
> v1: https://lore.kernel.org/r/20231019101625.412936-2-jakub@cloudflare.com
> ---
>  drivers/virtio/virtio_pci_common.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/virtio/virtio_pci_common.c
> index c2524a7207cf..7a5593997e0e 100644
> --- a/drivers/virtio/virtio_pci_common.c
> +++ b/drivers/virtio/virtio_pci_common.c
> @@ -242,7 +242,7 @@ void vp_del_vqs(struct virtio_device *vdev)
>  			if (v != VIRTIO_MSI_NO_VECTOR) {
>  				int irq = pci_irq_vector(vp_dev->pci_dev, v);
>
> -				irq_set_affinity_hint(irq, NULL);
> +				irq_update_affinity_hint(irq, NULL);
>  				free_irq(irq, vq);
>  			}
>  		}
> @@ -443,10 +443,10 @@ int vp_set_vq_affinity(struct virtqueue *vq, const struct cpumask *cpu_mask)
>  		mask = vp_dev->msix_affinity_masks[info->msix_vector];
>  		irq = pci_irq_vector(vp_dev->pci_dev, info->msix_vector);
>  		if (!cpu_mask)
> -			irq_set_affinity_hint(irq, NULL);
> +			irq_update_affinity_hint(irq, NULL);
>  		else {
>  			cpumask_copy(mask, cpu_mask);
> -			irq_set_affinity_hint(irq, mask);
> +			irq_set_affinity_and_hint(irq, mask);
>  		}
>  	}
>  	return 0;
> --
> 2.41.0
>

Re: [PATCH v2] virtio_pci: Switch away from deprecated irq_set_affinity_hint

[Jakub Sitnicki]

On Wed, Oct 25, 2023 at 04:53 PM +02, Jakub Sitnicki wrote:
> Since commit 65c7cdedeb30 ("genirq: Provide new interfaces for affinity
> hints") irq_set_affinity_hint is being phased out.
>
> Switch to new interfaces for setting and applying irq affinity hints.
>
> Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
> ---
> v2:
>  - Leave cpumask_copy as is. We can't pass pointer to stack memory as hint.
>    Proposed a change to IRQ affinity interface to address this limitation:
>    https://lore.kernel.org/r/20231025141517.375378-1-jakub@cloudflare.com

Just a note to the ^ - if we wanted to get rid of msix_affinity_masks,
we could call irq_set_affinity directly, instead of calling it through
irq_set_affinity[_and]_hint.

The hint wouldn't be available any more in /proc/irq/N/affinity_hint,
but the same information can be gathered from /proc/irq/N/smp_affinity.

[...]

Re: [PATCH v2] virtio_pci: Switch away from deprecated irq_set_affinity_hint

[Michael S. Tsirkin]

On Thu, Oct 26, 2023 at 06:25:08PM +0200, Jakub Sitnicki wrote:
> On Wed, Oct 25, 2023 at 04:53 PM +02, Jakub Sitnicki wrote:
> > Since commit 65c7cdedeb30 ("genirq: Provide new interfaces for affinity
> > hints") irq_set_affinity_hint is being phased out.
> >
> > Switch to new interfaces for setting and applying irq affinity hints.
> >
> > Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
> > ---
> > v2:
> >  - Leave cpumask_copy as is. We can't pass pointer to stack memory as hint.
> >    Proposed a change to IRQ affinity interface to address this limitation:
> >    https://lore.kernel.org/r/20231025141517.375378-1-jakub@cloudflare.com
> 
> Just a note to the ^ - if we wanted to get rid of msix_affinity_masks,
> we could call irq_set_affinity directly, instead of calling it through
> irq_set_affinity[_and]_hint.
> 
> The hint wouldn't be available any more in /proc/irq/N/affinity_hint,
> but the same information can be gathered from /proc/irq/N/smp_affinity.
> 
> [...]


So we are potentially breaking some userspace - what's the value we
gain?  Is there some way we can make disable_irq/enable_irq work?
That would have a lot of value.
There is an actual need for that in virtio for coco but we can't use
these APIs with affinity managed IRQs.

-- 
MST

Re: [PATCH v2] virtio_pci: Switch away from deprecated irq_set_affinity_hint

[Jakub Sitnicki]

On Thu, Oct 26, 2023 at 01:20 PM -04, Michael S. Tsirkin wrote:
> On Thu, Oct 26, 2023 at 06:25:08PM +0200, Jakub Sitnicki wrote:
>> On Wed, Oct 25, 2023 at 04:53 PM +02, Jakub Sitnicki wrote:
>> > Since commit 65c7cdedeb30 ("genirq: Provide new interfaces for affinity
>> > hints") irq_set_affinity_hint is being phased out.
>> >
>> > Switch to new interfaces for setting and applying irq affinity hints.
>> >
>> > Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
>> > ---
>> > v2:
>> >  - Leave cpumask_copy as is. We can't pass pointer to stack memory as hint.
>> >    Proposed a change to IRQ affinity interface to address this limitation:
>> >    https://lore.kernel.org/r/20231025141517.375378-1-jakub@cloudflare.com
>> 
>> Just a note to the ^ - if we wanted to get rid of msix_affinity_masks,
>> we could call irq_set_affinity directly, instead of calling it through
>> irq_set_affinity[_and]_hint.
>> 
>> The hint wouldn't be available any more in /proc/irq/N/affinity_hint,
>> but the same information can be gathered from /proc/irq/N/smp_affinity.
>> 
>> [...]
>
>
> So we are potentially breaking some userspace - what's the value we
> gain?  Is there some way we can make disable_irq/enable_irq work?
> That would have a lot of value.
> There is an actual need for that in virtio for coco but we can't use
> these APIs with affinity managed IRQs.

Sorry, that is beyond my ken today.

I saw the comment in vp_modern_disable_vq_and_reset:

	/* For the case where vq has an exclusive irq, call synchronize_irq() to
	 * wait for completion.
	 *
	 * note: We can't use disable_irq() since it conflicts with the affinity
	 * managed IRQ that is used by some drivers.
	 */

... but I fail to follow how the two conflict.

Perhaps Xuah could shed some light here.

Re: [PATCH v2] virtio_pci: Switch away from deprecated irq_set_affinity_hint

[Xuan Zhuo]

On Fri, 27 Oct 2023 12:03:17 +0200, Jakub Sitnicki <jakub@cloudflare.com> wrote:
> On Thu, Oct 26, 2023 at 01:20 PM -04, Michael S. Tsirkin wrote:
> > On Thu, Oct 26, 2023 at 06:25:08PM +0200, Jakub Sitnicki wrote:
> >> On Wed, Oct 25, 2023 at 04:53 PM +02, Jakub Sitnicki wrote:
> >> > Since commit 65c7cdedeb30 ("genirq: Provide new interfaces for affinity
> >> > hints") irq_set_affinity_hint is being phased out.
> >> >
> >> > Switch to new interfaces for setting and applying irq affinity hints.
> >> >
> >> > Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
> >> > ---
> >> > v2:
> >> >  - Leave cpumask_copy as is. We can't pass pointer to stack memory as hint.
> >> >    Proposed a change to IRQ affinity interface to address this limitation:
> >> >    https://lore.kernel.org/r/20231025141517.375378-1-jakub@cloudflare.com
> >>
> >> Just a note to the ^ - if we wanted to get rid of msix_affinity_masks,
> >> we could call irq_set_affinity directly, instead of calling it through
> >> irq_set_affinity[_and]_hint.
> >>
> >> The hint wouldn't be available any more in /proc/irq/N/affinity_hint,
> >> but the same information can be gathered from /proc/irq/N/smp_affinity.
> >>
> >> [...]
> >
> >
> > So we are potentially breaking some userspace - what's the value we
> > gain?  Is there some way we can make disable_irq/enable_irq work?
> > That would have a lot of value.
> > There is an actual need for that in virtio for coco but we can't use
> > these APIs with affinity managed IRQs.
>
> Sorry, that is beyond my ken today.
>
> I saw the comment in vp_modern_disable_vq_and_reset:
>
> 	/* For the case where vq has an exclusive irq, call synchronize_irq() to
> 	 * wait for completion.
> 	 *
> 	 * note: We can't use disable_irq() since it conflicts with the affinity
> 	 * managed IRQ that is used by some drivers.
> 	 */
>
> ... but I fail to follow how the two conflict.
>
> Perhaps Xuah could shed some light here.

I forgot which device that manages the irq by it self. If we disable the irq,
then the device will lost the irq, so we must not call disable_irq().

Thanks.