[PATCH] x86/xen: fix percpu vcpu_info allocation

[PATCH] x86/xen: fix percpu vcpu_info allocation

[Juergen Gross]

Today the percpu struct vcpu_info is allocated via DEFINE_PER_CPU(),
meaning that it could cross a page boundary. In this case registering
it with the hypervisor will fail, resulting in a panic().

This can easily be fixed by using DEFINE_PER_CPU_ALIGNED() instead,
as struct vcpu_info is guaranteed to have a size of 64 bytes, matching
the cache line size of x86 64-bit processors (Xen doesn't support
32-bit processors).

Fixes: 5ead97c84fa7 ("xen: Core Xen implementation")
Signed-off-by: Juergen Gross <jgross@suse.com>
---
 arch/x86/xen/enlighten.c | 6 +++++-
 arch/x86/xen/xen-ops.h   | 2 +-
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 0337392a3121..3c61bb98c10e 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -33,9 +33,12 @@ EXPORT_SYMBOL_GPL(hypercall_page);
  * and xen_vcpu_setup for details. By default it points to share_info->vcpu_info
  * but during boot it is switched to point to xen_vcpu_info.
  * The pointer is used in xen_evtchn_do_upcall to acknowledge pending events.
+ * Make sure that xen_vcpu_info doesn't cross a page boundary by making it
+ * cache-line aligned (the struct is guaranteed to have a size of 64 bytes,
+ * which matches the cache line size of 64-bit x86 processors).
  */
 DEFINE_PER_CPU(struct vcpu_info *, xen_vcpu);
-DEFINE_PER_CPU(struct vcpu_info, xen_vcpu_info);
+DEFINE_PER_CPU_ALIGNED(struct vcpu_info, xen_vcpu_info);
 
 /* Linux <-> Xen vCPU id mapping */
 DEFINE_PER_CPU(uint32_t, xen_vcpu_id);
@@ -160,6 +163,7 @@ void xen_vcpu_setup(int cpu)
 	int err;
 	struct vcpu_info *vcpup;
 
+	BUILD_BUG_ON(sizeof(*vcpup) > SMP_CACHE_BYTES);
 	BUG_ON(HYPERVISOR_shared_info == &xen_dummy_shared_info);
 
 	/*
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 408a2aa66c69..a87ab36889e7 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -21,7 +21,7 @@ extern void *xen_initial_gdt;
 struct trap_info;
 void xen_copy_trap_info(struct trap_info *traps);
 
-DECLARE_PER_CPU(struct vcpu_info, xen_vcpu_info);
+DECLARE_PER_CPU_ALIGNED(struct vcpu_info, xen_vcpu_info);
 DECLARE_PER_CPU(unsigned long, xen_cr3);
 DECLARE_PER_CPU(unsigned long, xen_current_cr3);
 
-- 
2.35.3

Re: [PATCH] x86/xen: fix percpu vcpu_info allocation

[Boris Ostrovsky]

On 11/24/23 2:48 AM, Juergen Gross wrote:
> Today the percpu struct vcpu_info is allocated via DEFINE_PER_CPU(),
> meaning that it could cross a page boundary. In this case registering
> it with the hypervisor will fail, resulting in a panic().
> 
> This can easily be fixed by using DEFINE_PER_CPU_ALIGNED() instead,
> as struct vcpu_info is guaranteed to have a size of 64 bytes, matching
> the cache line size of x86 64-bit processors (Xen doesn't support
> 32-bit processors).
> 
> Fixes: 5ead97c84fa7 ("xen: Core Xen implementation")
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.con>

although I am not sure in usefulness of BUILD_BUG_ON --- 64 bytes is part of ABI and hypervisor already has its own BUILD_BUG_ON for this.


-boris


> @@ -160,6 +163,7 @@ void xen_vcpu_setup(int cpu)
>   	int err;
>   	struct vcpu_info *vcpup;
>   
> +	BUILD_BUG_ON(sizeof(*vcpup) > SMP_CACHE_BYTES);
>   	BUG_ON(HYPERVISOR_shared_info == &xen_dummy_shared_info);

Re: [PATCH] x86/xen: fix percpu vcpu_info allocation

[Jan Beulich]

On 27.11.2023 15:57, Boris Ostrovsky wrote:
> 
> 
> On 11/24/23 2:48 AM, Juergen Gross wrote:
>> Today the percpu struct vcpu_info is allocated via DEFINE_PER_CPU(),
>> meaning that it could cross a page boundary. In this case registering
>> it with the hypervisor will fail, resulting in a panic().
>>
>> This can easily be fixed by using DEFINE_PER_CPU_ALIGNED() instead,
>> as struct vcpu_info is guaranteed to have a size of 64 bytes, matching
>> the cache line size of x86 64-bit processors (Xen doesn't support
>> 32-bit processors).
>>
>> Fixes: 5ead97c84fa7 ("xen: Core Xen implementation")
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.con>
> 
> although I am not sure in usefulness of BUILD_BUG_ON --- 64 bytes is part of ABI and hypervisor already has its own BUILD_BUG_ON for this.

I understood the check to guard against SMP_CACHE_BYTES < 64.

Jan

>> @@ -160,6 +163,7 @@ void xen_vcpu_setup(int cpu)
>>   	int err;
>>   	struct vcpu_info *vcpup;
>>   
>> +	BUILD_BUG_ON(sizeof(*vcpup) > SMP_CACHE_BYTES);
>>   	BUG_ON(HYPERVISOR_shared_info == &xen_dummy_shared_info);
> 

Re: [PATCH] x86/xen: fix percpu vcpu_info allocation

[Juergen Gross]

[-- Attachment #1.1.1: Type: text/plain, Size: 1063 bytes --]

On 27.11.23 16:00, Jan Beulich wrote:
> On 27.11.2023 15:57, Boris Ostrovsky wrote:
>>
>>
>> On 11/24/23 2:48 AM, Juergen Gross wrote:
>>> Today the percpu struct vcpu_info is allocated via DEFINE_PER_CPU(),
>>> meaning that it could cross a page boundary. In this case registering
>>> it with the hypervisor will fail, resulting in a panic().
>>>
>>> This can easily be fixed by using DEFINE_PER_CPU_ALIGNED() instead,
>>> as struct vcpu_info is guaranteed to have a size of 64 bytes, matching
>>> the cache line size of x86 64-bit processors (Xen doesn't support
>>> 32-bit processors).
>>>
>>> Fixes: 5ead97c84fa7 ("xen: Core Xen implementation")
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>
>> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.con>
>>
>> although I am not sure in usefulness of BUILD_BUG_ON --- 64 bytes is part of ABI and hypervisor already has its own BUILD_BUG_ON for this.
> 
> I understood the check to guard against SMP_CACHE_BYTES < 64.

Yes, that was the idea. Better safe than sorry.


Juergen

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3743 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [PATCH] x86/xen: fix percpu vcpu_info allocation

[Roger Pau Monné]

On Fri, Nov 24, 2023 at 08:48:52AM +0100, Juergen Gross wrote:
> Today the percpu struct vcpu_info is allocated via DEFINE_PER_CPU(),
> meaning that it could cross a page boundary. In this case registering
> it with the hypervisor will fail, resulting in a panic().
> 
> This can easily be fixed by using DEFINE_PER_CPU_ALIGNED() instead,
> as struct vcpu_info is guaranteed to have a size of 64 bytes, matching
> the cache line size of x86 64-bit processors (Xen doesn't support
> 32-bit processors).
> 
> Fixes: 5ead97c84fa7 ("xen: Core Xen implementation")
> Signed-off-by: Juergen Gross <jgross@suse.com>

FWIW, on FreeBSD we also switched to the same approach quite
recently:

https://cgit.freebsd.org/src/commit/sys/xen/xen_common.c?id=20fc5bf7df1db698f2651eaa04a3bc71290e1636

Should have checked the Linux side, sorry.

Roger.