[PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Roger Pau Monne]

There's no reason to force HVM guests to have a valid vcpu_info area when
initializing a vCPU, as the vCPU can also be brought online using the local
APIC, and on that path there's no requirement for vcpu_info to be setup ahead
of the bring up.  Note an HVM vCPU can operate normally without making use of
vcpu_info.

Restrict the check against dummy_vcpu_info to only apply to PV guests.

Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 xen/common/compat/domain.c | 2 +-
 xen/common/domain.c        | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/common/compat/domain.c b/xen/common/compat/domain.c
index 7ff238cc2656..6b4afc823217 100644
--- a/xen/common/compat/domain.c
+++ b/xen/common/compat/domain.c
@@ -49,7 +49,7 @@ int compat_common_vcpu_op(int cmd, struct vcpu *v,
     {
     case VCPUOP_initialise:
     {
-        if ( v->vcpu_info_area.map == &dummy_vcpu_info )
+        if ( is_pv_domain(d) && v->vcpu_info_area.map == &dummy_vcpu_info )
             return -EINVAL;
 
 #ifdef CONFIG_HVM
diff --git a/xen/common/domain.c b/xen/common/domain.c
index f6f557499660..d956dc09eca0 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -1817,7 +1817,7 @@ long common_vcpu_op(int cmd, struct vcpu *v, XEN_GUEST_HANDLE_PARAM(void) arg)
     switch ( cmd )
     {
     case VCPUOP_initialise:
-        if ( v->vcpu_info_area.map == &dummy_vcpu_info )
+        if ( is_pv_domain(d) && v->vcpu_info_area.map == &dummy_vcpu_info )
             return -EINVAL;
 
         rc = arch_initialise_vcpu(v, arg);
-- 
2.44.0

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Andrew Cooper]

On 20/03/2024 1:57 pm, Roger Pau Monne wrote:
> There's no reason to force HVM guests to have a valid vcpu_info area when
> initializing a vCPU, as the vCPU can also be brought online using the local
> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
> of the bring up.  Note an HVM vCPU can operate normally without making use of
> vcpu_info.
>
> Restrict the check against dummy_vcpu_info to only apply to PV guests.
>
> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Thanks for looking into this.  But, do we actually need to force this on
PV either?

The only interesting user of dummy_vcpu_info now is vcpu_info_populate()
which can cope with any arbitrary vCPU.

I have a suspicion that we can (now) delete these two checks, delete the
dummy_vcpu_info object, and use a regular NULL pointer in
vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger
pieces of PV-absurdity from Xen.

I can entirely believe that this is only safe to do with Jan's physical
registration changes in place.

~Andrew

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Roger Pau Monné]

On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote:
> On 20/03/2024 1:57 pm, Roger Pau Monne wrote:
> > There's no reason to force HVM guests to have a valid vcpu_info area when
> > initializing a vCPU, as the vCPU can also be brought online using the local
> > APIC, and on that path there's no requirement for vcpu_info to be setup ahead
> > of the bring up.  Note an HVM vCPU can operate normally without making use of
> > vcpu_info.
> >
> > Restrict the check against dummy_vcpu_info to only apply to PV guests.
> >
> > Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks for looking into this.  But, do we actually need to force this on
> PV either?

Possibly, having now taken a look at the users it does seem they could
cope with unpopulated vcpu_info_area.

Part of my understanding was that this was some kind of courtesy to PV
guests in order to prevent starting them without a vcpu_info, which
strictly speaking is not mandatory, but otherwise the guest vCPU won't
be able to receive interrupts, not even IPIs.

> The only interesting user of dummy_vcpu_info now is vcpu_info_populate()
> which can cope with any arbitrary vCPU.
> 
> I have a suspicion that we can (now) delete these two checks, delete the
> dummy_vcpu_info object, and use a regular NULL pointer in
> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger
> pieces of PV-absurdity from Xen.

I was expecting this to be something we can backport.  OTOH removing
the check completely (or even getting rid of dummy_vcpu_info) is not
something that we would want to backport.

I would rather do the removal of dummy_vcpu_info as followup work.

> I can entirely believe that this is only safe to do with Jan's physical
> registration changes in place.

Quite possible.

Thanks, Roger.

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Andrew Cooper]

On 20/03/2024 2:26 pm, Roger Pau Monné wrote:
> On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote:
>> On 20/03/2024 1:57 pm, Roger Pau Monne wrote:
>>> There's no reason to force HVM guests to have a valid vcpu_info area when
>>> initializing a vCPU, as the vCPU can also be brought online using the local
>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>>> vcpu_info.
>>>
>>> Restrict the check against dummy_vcpu_info to only apply to PV guests.
>>>
>>> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Thanks for looking into this.  But, do we actually need to force this on
>> PV either?
> Possibly, having now taken a look at the users it does seem they could
> cope with unpopulated vcpu_info_area.
>
> Part of my understanding was that this was some kind of courtesy to PV
> guests in order to prevent starting them without a vcpu_info, which
> strictly speaking is not mandatory, but otherwise the guest vCPU won't
> be able to receive interrupts, not even IPIs.

That's more of a stick than a carrot... "you must set up the area of a
CPU before you can bring it online". Except as we've seen, HVM has been
fine all along without it.
>> The only interesting user of dummy_vcpu_info now is vcpu_info_populate()
>> which can cope with any arbitrary vCPU.
>>
>> I have a suspicion that we can (now) delete these two checks, delete the
>> dummy_vcpu_info object, and use a regular NULL pointer in
>> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger
>> pieces of PV-absurdity from Xen.
> I was expecting this to be something we can backport.  OTOH removing
> the check completely (or even getting rid of dummy_vcpu_info) is not
> something that we would want to backport.
>
> I would rather do the removal of dummy_vcpu_info as followup work.

I was worried about ARM with your patch, because it's spelt HVM there,
rather than PV.

However, I'd forgotten that ARM's do_vcpu_op() filters ops down to just
VCPUOP_register_{vcpu_info,runstate_memory_area} so VCPUOP_initialise
isn't reachable.

Therefore, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

It also means ARM can't use any of the PHYS registration yet...

~Andrew

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Julien Grall]

Hi Andrew,

On 20/03/2024 14:39, Andrew Cooper wrote:
> On 20/03/2024 2:26 pm, Roger Pau Monné wrote:
>> On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote:
>>> On 20/03/2024 1:57 pm, Roger Pau Monne wrote:
>>>> There's no reason to force HVM guests to have a valid vcpu_info area when
>>>> initializing a vCPU, as the vCPU can also be brought online using the local
>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>>>> vcpu_info.
>>>>
>>>> Restrict the check against dummy_vcpu_info to only apply to PV guests.
>>>>
>>>> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>> Thanks for looking into this.  But, do we actually need to force this on
>>> PV either?
>> Possibly, having now taken a look at the users it does seem they could
>> cope with unpopulated vcpu_info_area.
>>
>> Part of my understanding was that this was some kind of courtesy to PV
>> guests in order to prevent starting them without a vcpu_info, which
>> strictly speaking is not mandatory, but otherwise the guest vCPU won't
>> be able to receive interrupts, not even IPIs.
> 
> That's more of a stick than a carrot... "you must set up the area of a
> CPU before you can bring it online". Except as we've seen, HVM has been
> fine all along without it.
>>> The only interesting user of dummy_vcpu_info now is vcpu_info_populate()
>>> which can cope with any arbitrary vCPU.
>>>
>>> I have a suspicion that we can (now) delete these two checks, delete the
>>> dummy_vcpu_info object, and use a regular NULL pointer in
>>> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger
>>> pieces of PV-absurdity from Xen.
>> I was expecting this to be something we can backport.  OTOH removing
>> the check completely (or even getting rid of dummy_vcpu_info) is not
>> something that we would want to backport.
>>
>> I would rather do the removal of dummy_vcpu_info as followup work.
> 
> I was worried about ARM with your patch, because it's spelt HVM there,
> rather than PV.
> 
> However, I'd forgotten that ARM's do_vcpu_op() filters ops down to just
> VCPUOP_register_{vcpu_info,runstate_memory_area} so VCPUOP_initialise
> isn't reachable.
> 
> Therefore, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> It also means ARM can't use any of the PHYS registration yet...

Whoops. I don't think this was intended. Jan, I don't seem to find any 
use in Linux. Do you have any patches you could share? I would like to 
give a try on Arm before sending a patch?

Cheers,

-- 
Julien Grall

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 26.03.2024 23:08, Julien Grall wrote:
> Hi Andrew,
> 
> On 20/03/2024 14:39, Andrew Cooper wrote:
>> On 20/03/2024 2:26 pm, Roger Pau Monné wrote:
>>> On Wed, Mar 20, 2024 at 02:06:27PM +0000, Andrew Cooper wrote:
>>>> On 20/03/2024 1:57 pm, Roger Pau Monne wrote:
>>>>> There's no reason to force HVM guests to have a valid vcpu_info area when
>>>>> initializing a vCPU, as the vCPU can also be brought online using the local
>>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>>>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>>>>> vcpu_info.
>>>>>
>>>>> Restrict the check against dummy_vcpu_info to only apply to PV guests.
>>>>>
>>>>> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
>>>>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>>>> Thanks for looking into this.  But, do we actually need to force this on
>>>> PV either?
>>> Possibly, having now taken a look at the users it does seem they could
>>> cope with unpopulated vcpu_info_area.
>>>
>>> Part of my understanding was that this was some kind of courtesy to PV
>>> guests in order to prevent starting them without a vcpu_info, which
>>> strictly speaking is not mandatory, but otherwise the guest vCPU won't
>>> be able to receive interrupts, not even IPIs.
>>
>> That's more of a stick than a carrot... "you must set up the area of a
>> CPU before you can bring it online". Except as we've seen, HVM has been
>> fine all along without it.
>>>> The only interesting user of dummy_vcpu_info now is vcpu_info_populate()
>>>> which can cope with any arbitrary vCPU.
>>>>
>>>> I have a suspicion that we can (now) delete these two checks, delete the
>>>> dummy_vcpu_info object, and use a regular NULL pointer in
>>>> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger
>>>> pieces of PV-absurdity from Xen.
>>> I was expecting this to be something we can backport.  OTOH removing
>>> the check completely (or even getting rid of dummy_vcpu_info) is not
>>> something that we would want to backport.
>>>
>>> I would rather do the removal of dummy_vcpu_info as followup work.
>>
>> I was worried about ARM with your patch, because it's spelt HVM there,
>> rather than PV.
>>
>> However, I'd forgotten that ARM's do_vcpu_op() filters ops down to just
>> VCPUOP_register_{vcpu_info,runstate_memory_area} so VCPUOP_initialise
>> isn't reachable.
>>
>> Therefore, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>
>> It also means ARM can't use any of the PHYS registration yet...
> 
> Whoops. I don't think this was intended. Jan, I don't seem to find any 
> use in Linux. Do you have any patches you could share?

No, I don't. I did all development with hacked up XTF tests, and I was
expecting Linux folks to be looking into making use of the new subops.

Jan

> I would like to 
> give a try on Arm before sending a patch?
> 
> Cheers,
> 

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 20.03.2024 15:06, Andrew Cooper wrote:
> On 20/03/2024 1:57 pm, Roger Pau Monne wrote:
>> There's no reason to force HVM guests to have a valid vcpu_info area when
>> initializing a vCPU, as the vCPU can also be brought online using the local
>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>> vcpu_info.
>>
>> Restrict the check against dummy_vcpu_info to only apply to PV guests.
>>
>> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> Thanks for looking into this.  But, do we actually need to force this on
> PV either?
> 
> The only interesting user of dummy_vcpu_info now is vcpu_info_populate()
> which can cope with any arbitrary vCPU.

Why would that be the only interesting user? If we were to ...

> I have a suspicion that we can (now) delete these two checks, delete the
> dummy_vcpu_info object, and use a regular NULL pointer in
> vcpu_info_{populate,reset}(), and in doing so, remove one of the bigger
> pieces of PV-absurdity from Xen.

... use NULL pointers instead, we'd need to audit all Xen's vcpu_info
accesses for properly avoiding to de-reference that NULL. The dummy
struct instance was really meant as a safety belt (besides also acting
as a sentinel).

> I can entirely believe that this is only safe to do with Jan's physical
> registration changes in place.

Which nobody uses just yet, afaik.

Jan

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 20.03.2024 14:57, Roger Pau Monne wrote:
> There's no reason to force HVM guests to have a valid vcpu_info area when
> initializing a vCPU, as the vCPU can also be brought online using the local
> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
> of the bring up.  Note an HVM vCPU can operate normally without making use of
> vcpu_info.

While I'd agree if you started with "There's no real need to force ...", I
still think there is a reason: If one wants to use paravirt interfaces (i.e.
hypercalls), they would better do so consistently. After all there's also
no need to use VCPUOP_initialise, yet you're not disabling its use.

As said in reply to Andrew's reply, besides acting as a sentinel that
structure instance also acts as a sink for Xen accesses to a vCPU's
vcpu_info. By removing the check, you switch that from being a safeguard to
being something that actually has to be expected to be accessed. Unless
you've audited all uses to prove that no such access exists.

Jan

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Andrew Cooper]

On 20/03/2024 3:09 pm, Jan Beulich wrote:
> On 20.03.2024 14:57, Roger Pau Monne wrote:
>> There's no reason to force HVM guests to have a valid vcpu_info area when
>> initializing a vCPU, as the vCPU can also be brought online using the local
>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>> vcpu_info.
> While I'd agree if you started with "There's no real need to force ...", I
> still think there is a reason: If one wants to use paravirt interfaces (i.e.
> hypercalls), they would better do so consistently. After all there's also
> no need to use VCPUOP_initialise, yet you're not disabling its use.

I firmly disagree.

There are good reasons to use VCPUOP_initialise over INIT-SIPI-SIPI
(like avoiding 16bit mode - in the case we want it here, to fix APIC_ID
enumeration in a way that doesn't involve putting more complexity into
HVMLoader), and forcing us to set up a useless structure before we can
boot vCPU number 32 is just wrong.

It was dumb to design a hypercall like this in the first place for PV
guests, and it definitely isn't OK to keep guests broken because of it.

~Andrew

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 20.03.2024 16:20, Andrew Cooper wrote:
> On 20/03/2024 3:09 pm, Jan Beulich wrote:
>> On 20.03.2024 14:57, Roger Pau Monne wrote:
>>> There's no reason to force HVM guests to have a valid vcpu_info area when
>>> initializing a vCPU, as the vCPU can also be brought online using the local
>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>>> vcpu_info.
>> While I'd agree if you started with "There's no real need to force ...", I
>> still think there is a reason: If one wants to use paravirt interfaces (i.e.
>> hypercalls), they would better do so consistently. After all there's also
>> no need to use VCPUOP_initialise, yet you're not disabling its use.
> 
> I firmly disagree.
> 
> There are good reasons to use VCPUOP_initialise over INIT-SIPI-SIPI
> (like avoiding 16bit mode - in the case we want it here, to fix APIC_ID
> enumeration in a way that doesn't involve putting more complexity into
> HVMLoader), and forcing us to set up a useless structure before we can
> boot vCPU number 32 is just wrong.

Just to mention it: I can certainly accept this as one possible valid
viewpoint, and my looking at it differently is not an objection to the
patch. It's just that the other aspect mentioned needs sorting (perhaps
by just extending the patch description).

> It was dumb to design a hypercall like this in the first place for PV
> guests, and it definitely isn't OK to keep guests broken because of it.

And again just to mention it: Originally, with a vCPU limit of 32, all
vCPU-s would reliably have had vcpu_info (by way of that being embedded
in shared_info). When raising the limit, the goal was to not chance
overlooking any vcpu_info access in Xen. Hence why, instead of putting
a NULL pointer there (or perhaps some non-canonical sentinel), the
dummy approach was chosen. It then seemed quite desirable to prevent
vCPU-s coming online without them first being detached from that dummy
structure. And I think this firmly needs to continue to hold for PV.

Jan

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Roger Pau Monné]

On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote:
> On 20.03.2024 14:57, Roger Pau Monne wrote:
> > There's no reason to force HVM guests to have a valid vcpu_info area when
> > initializing a vCPU, as the vCPU can also be brought online using the local
> > APIC, and on that path there's no requirement for vcpu_info to be setup ahead
> > of the bring up.  Note an HVM vCPU can operate normally without making use of
> > vcpu_info.
> 
> While I'd agree if you started with "There's no real need to force ...", I
> still think there is a reason: If one wants to use paravirt interfaces (i.e.
> hypercalls), they would better do so consistently. After all there's also
> no need to use VCPUOP_initialise, yet you're not disabling its use.
> 
> As said in reply to Andrew's reply, besides acting as a sentinel that
> structure instance also acts as a sink for Xen accesses to a vCPU's
> vcpu_info. By removing the check, you switch that from being a safeguard to
> being something that actually has to be expected to be accessed. Unless
> you've audited all uses to prove that no such access exists.

I'm kind of lost in this last paragraph, how is that different than
what currently happens when an HVM vCPU >= 32 is brought up using the
lapic and has no vpcu_info mapped?

Also, from a quick look it seems like sites do check whether vcpu_info
== dummy_vcpu_info, otherwise we would already be in trouble.

Thanks, Roger.

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 20.03.2024 17:29, Roger Pau Monné wrote:
> On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote:
>> On 20.03.2024 14:57, Roger Pau Monne wrote:
>>> There's no reason to force HVM guests to have a valid vcpu_info area when
>>> initializing a vCPU, as the vCPU can also be brought online using the local
>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>>> vcpu_info.
>>
>> While I'd agree if you started with "There's no real need to force ...", I
>> still think there is a reason: If one wants to use paravirt interfaces (i.e.
>> hypercalls), they would better do so consistently. After all there's also
>> no need to use VCPUOP_initialise, yet you're not disabling its use.
>>
>> As said in reply to Andrew's reply, besides acting as a sentinel that
>> structure instance also acts as a sink for Xen accesses to a vCPU's
>> vcpu_info. By removing the check, you switch that from being a safeguard to
>> being something that actually has to be expected to be accessed. Unless
>> you've audited all uses to prove that no such access exists.
> 
> I'm kind of lost in this last paragraph, how is that different than
> what currently happens when an HVM vCPU >= 32 is brought up using the
> lapic and has no vpcu_info mapped?

I think this aspect was simply missed back at the time. And I think it
wants mentioning explicitly to justify the change.

As said in reply to Andrew, I don't think the dummy structure can be got
rid of. Nor can the checks here be (easily) removed altogether, i.e. your
change cannot (easily) be extended to PV as well. Even conditional removal
of the structure in !PV builds would first require all vcpu_info accesses
to gain a suitable conditional. Which may be undesirable, as some of these
may be deemed fast paths.

> Also, from a quick look it seems like sites do check whether vcpu_info
> == dummy_vcpu_info, otherwise we would already be in trouble.

Such checks exist in code managing vcpu_info, but not - afaics - in places
actually accessing it.

Jan

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Roger Pau Monné]

On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote:
> On 20.03.2024 17:29, Roger Pau Monné wrote:
> > On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote:
> >> On 20.03.2024 14:57, Roger Pau Monne wrote:
> >>> There's no reason to force HVM guests to have a valid vcpu_info area when
> >>> initializing a vCPU, as the vCPU can also be brought online using the local
> >>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
> >>> of the bring up.  Note an HVM vCPU can operate normally without making use of
> >>> vcpu_info.
> >>
> >> While I'd agree if you started with "There's no real need to force ...", I
> >> still think there is a reason: If one wants to use paravirt interfaces (i.e.
> >> hypercalls), they would better do so consistently. After all there's also
> >> no need to use VCPUOP_initialise, yet you're not disabling its use.
> >>
> >> As said in reply to Andrew's reply, besides acting as a sentinel that
> >> structure instance also acts as a sink for Xen accesses to a vCPU's
> >> vcpu_info. By removing the check, you switch that from being a safeguard to
> >> being something that actually has to be expected to be accessed. Unless
> >> you've audited all uses to prove that no such access exists.
> > 
> > I'm kind of lost in this last paragraph, how is that different than
> > what currently happens when an HVM vCPU >= 32 is brought up using the
> > lapic and has no vpcu_info mapped?
> 
> I think this aspect was simply missed back at the time. And I think it
> wants mentioning explicitly to justify the change.

OK, I can add to the commit message:

"Note an HVM vCPU can operate normally without making use of
vcpu_info, and in fact does so when brought up from the local APIC."

> As said in reply to Andrew, I don't think the dummy structure can be got
> rid of. Nor can the checks here be (easily) removed altogether, i.e. your
> change cannot (easily) be extended to PV as well. Even conditional removal
> of the structure in !PV builds would first require all vcpu_info accesses
> to gain a suitable conditional. Which may be undesirable, as some of these
> may be deemed fast paths.

I didn't intended to do this here, as replied to Andrew.  If we want
to get rid of the check for PV also it needs to be done in a different
patch, and with a different justification and analysis.

> > Also, from a quick look it seems like sites do check whether vcpu_info
> > == dummy_vcpu_info, otherwise we would already be in trouble.
> 
> Such checks exist in code managing vcpu_info, but not - afaics - in places
> actually accessing it.

Quite possibly, I didn't look that close TBH, since my intention was
not to remove dummy_vcpu_info.  I've noticed however that
__update_vcpu_system_time() checks for v->vcpu_info_area.map == NULL,
which is fine, but shouldn't it also check for v->vcpu_info_area.map
== &dummy_vcpu_info, as it's pointless to update the vcpu system time
if pointing to the dummy_vcpu_info?

Thanks, Roger.

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 21.03.2024 10:10, Roger Pau Monné wrote:
> On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote:
>> On 20.03.2024 17:29, Roger Pau Monné wrote:
>>> On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote:
>>>> On 20.03.2024 14:57, Roger Pau Monne wrote:
>>>>> There's no reason to force HVM guests to have a valid vcpu_info area when
>>>>> initializing a vCPU, as the vCPU can also be brought online using the local
>>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>>>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>>>>> vcpu_info.
>>>>
>>>> While I'd agree if you started with "There's no real need to force ...", I
>>>> still think there is a reason: If one wants to use paravirt interfaces (i.e.
>>>> hypercalls), they would better do so consistently. After all there's also
>>>> no need to use VCPUOP_initialise, yet you're not disabling its use.
>>>>
>>>> As said in reply to Andrew's reply, besides acting as a sentinel that
>>>> structure instance also acts as a sink for Xen accesses to a vCPU's
>>>> vcpu_info. By removing the check, you switch that from being a safeguard to
>>>> being something that actually has to be expected to be accessed. Unless
>>>> you've audited all uses to prove that no such access exists.
>>>
>>> I'm kind of lost in this last paragraph, how is that different than
>>> what currently happens when an HVM vCPU >= 32 is brought up using the
>>> lapic and has no vpcu_info mapped?
>>
>> I think this aspect was simply missed back at the time. And I think it
>> wants mentioning explicitly to justify the change.
> 
> OK, I can add to the commit message:
> 
> "Note an HVM vCPU can operate normally without making use of
> vcpu_info, and in fact does so when brought up from the local APIC."

I'd be fine adding this (or having this added) while committing.

>> As said in reply to Andrew, I don't think the dummy structure can be got
>> rid of. Nor can the checks here be (easily) removed altogether, i.e. your
>> change cannot (easily) be extended to PV as well. Even conditional removal
>> of the structure in !PV builds would first require all vcpu_info accesses
>> to gain a suitable conditional. Which may be undesirable, as some of these
>> may be deemed fast paths.
> 
> I didn't intended to do this here, as replied to Andrew.  If we want
> to get rid of the check for PV also it needs to be done in a different
> patch, and with a different justification and analysis.
> 
>>> Also, from a quick look it seems like sites do check whether vcpu_info
>>> == dummy_vcpu_info, otherwise we would already be in trouble.
>>
>> Such checks exist in code managing vcpu_info, but not - afaics - in places
>> actually accessing it.
> 
> Quite possibly, I didn't look that close TBH, since my intention was
> not to remove dummy_vcpu_info.  I've noticed however that
> __update_vcpu_system_time() checks for v->vcpu_info_area.map == NULL,
> which is fine, but shouldn't it also check for v->vcpu_info_area.map
> == &dummy_vcpu_info, as it's pointless to update the vcpu system time
> if pointing to the dummy_vcpu_info?

The check is there to guard against NULL deref. As said, the aspect of a
vCPU being brought up the "native" way yet then still using its vCPU info
was, by mistake, neglected earlier on. So yes, such a check could be
added here, but it isn't strictly necessary as long as we don't avoid
accessing the dummy structure uniformly everywhere (which, as said, I'm
not sure we want to do).

Jan

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Roger Pau Monné]

On Thu, Mar 21, 2024 at 10:17:25AM +0100, Jan Beulich wrote:
> On 21.03.2024 10:10, Roger Pau Monné wrote:
> > On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote:
> >> On 20.03.2024 17:29, Roger Pau Monné wrote:
> >>> On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote:
> >>>> On 20.03.2024 14:57, Roger Pau Monne wrote:
> >>>>> There's no reason to force HVM guests to have a valid vcpu_info area when
> >>>>> initializing a vCPU, as the vCPU can also be brought online using the local
> >>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
> >>>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
> >>>>> vcpu_info.
> >>>>
> >>>> While I'd agree if you started with "There's no real need to force ...", I
> >>>> still think there is a reason: If one wants to use paravirt interfaces (i.e.
> >>>> hypercalls), they would better do so consistently. After all there's also
> >>>> no need to use VCPUOP_initialise, yet you're not disabling its use.
> >>>>
> >>>> As said in reply to Andrew's reply, besides acting as a sentinel that
> >>>> structure instance also acts as a sink for Xen accesses to a vCPU's
> >>>> vcpu_info. By removing the check, you switch that from being a safeguard to
> >>>> being something that actually has to be expected to be accessed. Unless
> >>>> you've audited all uses to prove that no such access exists.
> >>>
> >>> I'm kind of lost in this last paragraph, how is that different than
> >>> what currently happens when an HVM vCPU >= 32 is brought up using the
> >>> lapic and has no vpcu_info mapped?
> >>
> >> I think this aspect was simply missed back at the time. And I think it
> >> wants mentioning explicitly to justify the change.
> > 
> > OK, I can add to the commit message:
> > 
> > "Note an HVM vCPU can operate normally without making use of
> > vcpu_info, and in fact does so when brought up from the local APIC."
> 
> I'd be fine adding this (or having this added) while committing.
> 
> >> As said in reply to Andrew, I don't think the dummy structure can be got
> >> rid of. Nor can the checks here be (easily) removed altogether, i.e. your
> >> change cannot (easily) be extended to PV as well. Even conditional removal
> >> of the structure in !PV builds would first require all vcpu_info accesses
> >> to gain a suitable conditional. Which may be undesirable, as some of these
> >> may be deemed fast paths.
> > 
> > I didn't intended to do this here, as replied to Andrew.  If we want
> > to get rid of the check for PV also it needs to be done in a different
> > patch, and with a different justification and analysis.
> > 
> >>> Also, from a quick look it seems like sites do check whether vcpu_info
> >>> == dummy_vcpu_info, otherwise we would already be in trouble.
> >>
> >> Such checks exist in code managing vcpu_info, but not - afaics - in places
> >> actually accessing it.

Is there anything else that needs adjusting then, or are you happy to
pick this up and adjust the commit message?

> > Quite possibly, I didn't look that close TBH, since my intention was
> > not to remove dummy_vcpu_info.  I've noticed however that
> > __update_vcpu_system_time() checks for v->vcpu_info_area.map == NULL,
> > which is fine, but shouldn't it also check for v->vcpu_info_area.map
> > == &dummy_vcpu_info, as it's pointless to update the vcpu system time
> > if pointing to the dummy_vcpu_info?
> 
> The check is there to guard against NULL deref. As said, the aspect of a
> vCPU being brought up the "native" way yet then still using its vCPU info
> was, by mistake, neglected earlier on. So yes, such a check could be
> added here, but it isn't strictly necessary as long as we don't avoid
> accessing the dummy structure uniformly everywhere (which, as said, I'm
> not sure we want to do).

I could add such a check in a separate patch, my main concern with
this is not correctness, it's just that we waste cycles updating the
contents of dummy_vcpu_info which is useless.

Thanks, Roger.

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 21.03.2024 10:57, Roger Pau Monné wrote:
> On Thu, Mar 21, 2024 at 10:17:25AM +0100, Jan Beulich wrote:
>> On 21.03.2024 10:10, Roger Pau Monné wrote:
>>> On Thu, Mar 21, 2024 at 09:07:10AM +0100, Jan Beulich wrote:
>>>> On 20.03.2024 17:29, Roger Pau Monné wrote:
>>>>> On Wed, Mar 20, 2024 at 04:09:33PM +0100, Jan Beulich wrote:
>>>>>> On 20.03.2024 14:57, Roger Pau Monne wrote:
>>>>>>> There's no reason to force HVM guests to have a valid vcpu_info area when
>>>>>>> initializing a vCPU, as the vCPU can also be brought online using the local
>>>>>>> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
>>>>>>> of the bring up.  Note an HVM vCPU can operate normally without making use of
>>>>>>> vcpu_info.
>>>>>>
>>>>>> While I'd agree if you started with "There's no real need to force ...", I
>>>>>> still think there is a reason: If one wants to use paravirt interfaces (i.e.
>>>>>> hypercalls), they would better do so consistently. After all there's also
>>>>>> no need to use VCPUOP_initialise, yet you're not disabling its use.
>>>>>>
>>>>>> As said in reply to Andrew's reply, besides acting as a sentinel that
>>>>>> structure instance also acts as a sink for Xen accesses to a vCPU's
>>>>>> vcpu_info. By removing the check, you switch that from being a safeguard to
>>>>>> being something that actually has to be expected to be accessed. Unless
>>>>>> you've audited all uses to prove that no such access exists.
>>>>>
>>>>> I'm kind of lost in this last paragraph, how is that different than
>>>>> what currently happens when an HVM vCPU >= 32 is brought up using the
>>>>> lapic and has no vpcu_info mapped?
>>>>
>>>> I think this aspect was simply missed back at the time. And I think it
>>>> wants mentioning explicitly to justify the change.
>>>
>>> OK, I can add to the commit message:
>>>
>>> "Note an HVM vCPU can operate normally without making use of
>>> vcpu_info, and in fact does so when brought up from the local APIC."
>>
>> I'd be fine adding this (or having this added) while committing.
>>
>>>> As said in reply to Andrew, I don't think the dummy structure can be got
>>>> rid of. Nor can the checks here be (easily) removed altogether, i.e. your
>>>> change cannot (easily) be extended to PV as well. Even conditional removal
>>>> of the structure in !PV builds would first require all vcpu_info accesses
>>>> to gain a suitable conditional. Which may be undesirable, as some of these
>>>> may be deemed fast paths.
>>>
>>> I didn't intended to do this here, as replied to Andrew.  If we want
>>> to get rid of the check for PV also it needs to be done in a different
>>> patch, and with a different justification and analysis.
>>>
>>>>> Also, from a quick look it seems like sites do check whether vcpu_info
>>>>> == dummy_vcpu_info, otherwise we would already be in trouble.
>>>>
>>>> Such checks exist in code managing vcpu_info, but not - afaics - in places
>>>> actually accessing it.
> 
> Is there anything else that needs adjusting then, or are you happy to
> pick this up and adjust the commit message?

I'll pick this up and adjust (unless Andrew beats me).

Jan

Re: [PATCH] x86/vcpu: relax VCPUOP_initialise restriction for non-PV vCPUs

[Jan Beulich]

On 20.03.2024 14:57, Roger Pau Monne wrote:
> There's no reason to force HVM guests to have a valid vcpu_info area when
> initializing a vCPU, as the vCPU can also be brought online using the local
> APIC, and on that path there's no requirement for vcpu_info to be setup ahead
> of the bring up.  Note an HVM vCPU can operate normally without making use of
> vcpu_info.
> 
> Restrict the check against dummy_vcpu_info to only apply to PV guests.
> 
> Fixes: 192df6f9122d ('x86: allow HVM guests to use hypercalls to bring up vCPUs')
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Despite the Fixes: tag I have to admit I'm uncertain of backporting here.
HVM guests need to be prepared anyway that they may be denied use of this
sub-op without first setting up vCPU info.

Jan