[PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[Jinank Jain]

In case of nested MSHV, retargeting interrupt hypercall should be sent
to L0 hypervisor instead of L1 hypervisor.

Signed-off-by: Jinank Jain <jinankjain@linux.microsoft.com>
---
 drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index f33370b75628..2123f632ecf7 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
 		}
 	}
 
-	res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
-			      params, NULL);
+	if (hv_nested)
+		res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
+					     (var_size << 17),
+					     params, NULL);
+	else
+		res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
+				      (var_size << 17),
+				      params, NULL);
 
 exit_unlock:
 	spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);
-- 
2.34.1

Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[Nuno Das Neves]

Reviewed-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>

On 4/4/2023 4:35 AM, Jinank Jain wrote:
> In case of nested MSHV, retargeting interrupt hypercall should be sent
> to L0 hypervisor instead of L1 hypervisor.
> 
> Signed-off-by: Jinank Jain <jinankjain@linux.microsoft.com>
> ---
>  drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index f33370b75628..2123f632ecf7 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
>  		}
>  	}
>  
> -	res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
> -			      params, NULL);
> +	if (hv_nested)
> +		res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
> +					     (var_size << 17),
> +					     params, NULL);
> +	else
> +		res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
> +				      (var_size << 17),
> +				      params, NULL);
>  
>  exit_unlock:
>  	spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);

Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[Wei Liu]

On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> In case of nested MSHV, retargeting interrupt hypercall should be sent
> to L0 hypervisor instead of L1 hypervisor.
> 
> Signed-off-by: Jinank Jain <jinankjain@linux.microsoft.com>

While I think this is a sensible change -- how did you discover this?
Can you provide a bit more information?

> ---
>  drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index f33370b75628..2123f632ecf7 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
>  		}
>  	}
>  
> -	res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
> -			      params, NULL);
> +	if (hv_nested)
> +		res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
> +					     (var_size << 17),
> +					     params, NULL);
> +	else
> +		res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
> +				      (var_size << 17),
> +				      params, NULL);
>  
>  exit_unlock:
>  	spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);
> -- 
> 2.34.1
> 

Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[Jinank Jain]

This was observed while testing pass-through PCI devices on the nested 
MSHV setup.

Regards,

Jinank

On 4/6/2023 4:47 AM, Wei Liu wrote:
> On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
>> In case of nested MSHV, retargeting interrupt hypercall should be sent
>> to L0 hypervisor instead of L1 hypervisor.
>>
>> Signed-off-by: Jinank Jain <jinankjain@linux.microsoft.com>
> While I think this is a sensible change -- how did you discover this?
> Can you provide a bit more information?
>
>> ---
>>   drivers/pci/controller/pci-hyperv.c | 10 ++++++++--
>>   1 file changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
>> index f33370b75628..2123f632ecf7 100644
>> --- a/drivers/pci/controller/pci-hyperv.c
>> +++ b/drivers/pci/controller/pci-hyperv.c
>> @@ -704,8 +704,14 @@ static void hv_arch_irq_unmask(struct irq_data *data)
>>   		}
>>   	}
>>   
>> -	res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT | (var_size << 17),
>> -			      params, NULL);
>> +	if (hv_nested)
>> +		res = hv_do_nested_hypercall(HVCALL_RETARGET_INTERRUPT |
>> +					     (var_size << 17),
>> +					     params, NULL);
>> +	else
>> +		res = hv_do_hypercall(HVCALL_RETARGET_INTERRUPT |
>> +				      (var_size << 17),
>> +				      params, NULL);
>>   
>>   exit_unlock:
>>   	spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);
>> -- 
>> 2.34.1
>>

Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[Wei Liu]

On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> In case of nested MSHV, retargeting interrupt hypercall should be sent
> to L0 hypervisor instead of L1 hypervisor.
> 
> Signed-off-by: Jinank Jain <jinankjain@linux.microsoft.com>

Applied to hyperv-next. Thanks.

RE: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[Michael Kelley (LINUX)]

From: Wei Liu <wei.liu@kernel.org> Sent: Wednesday, April 12, 2023 6:23 PM
> 
> On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> > In case of nested MSHV, retargeting interrupt hypercall should be sent
> > to L0 hypervisor instead of L1 hypervisor.
> >
> > Signed-off-by: Jinank Jain <jinankjain@linux.microsoft.com>
> 
> Applied to hyperv-next. Thanks.

I'd like to hold off on taking this change.  Nuno and I are discussing
how best to handle nested hypercalls.  In addition to the proposed
nested changes,  we have hypercall changes coming as part of the
TDX and fully enlightened SNP patch sets.  If possible, I'd like to
avoid adding logic at the hv_do_hypercall() call sites.  It's not clear
whether avoiding such logic will really be feasible, but I'd like to
think about it for a bit before reaching that conclusion.

Michael

Re: [PATCH] PCI: hv: Use nested hypercall for retargeting interrupts

[Wei Liu]

On Thu, Apr 13, 2023 at 03:05:09AM +0000, Michael Kelley (LINUX) wrote:
> From: Wei Liu <wei.liu@kernel.org> Sent: Wednesday, April 12, 2023 6:23 PM
> > 
> > On Tue, Apr 04, 2023 at 11:35:46AM +0000, Jinank Jain wrote:
> > > In case of nested MSHV, retargeting interrupt hypercall should be sent
> > > to L0 hypervisor instead of L1 hypervisor.
> > >
> > > Signed-off-by: Jinank Jain <jinankjain@linux.microsoft.com>
> > 
> > Applied to hyperv-next. Thanks.
> 
> I'd like to hold off on taking this change.  Nuno and I are discussing
> how best to handle nested hypercalls.  In addition to the proposed
> nested changes,  we have hypercall changes coming as part of the
> TDX and fully enlightened SNP patch sets.  If possible, I'd like to
> avoid adding logic at the hv_do_hypercall() call sites.  It's not clear
> whether avoiding such logic will really be feasible, but I'd like to
> think about it for a bit before reaching that conclusion.

I thought that discussion will go on for a while but this patch fixed a
real bug.

Holding off is fine too. I will remove this patch from hyperv-next.

Thanks,
Wei.

> 
> Michael
>