[PATCH] x86: cpu: Error out if memory exceeds addressable range

[PATCH] x86: cpu: Error out if memory exceeds addressable range

[Nikunj A Dadhania]

Currently it is possible to start a guest with memory that is beyond
the addressable range of CPU and QEMU does not even warn about it.
The default phys_bits is 40 and can address 1TB. However it allows to
start a guest with greater than 1TB memory.

Prevent this by erroring out in such a scenario.

Reported-by: Shaju Abraham <Abraham.Shaju@amd.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
---
 target/i386/cpu.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 6a57ef13af..1afbdbac7d 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -6376,6 +6376,7 @@ static void x86_cpu_hyperv_realize(X86CPU *cpu)
 
 static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
 {
+    MachineState *machine = MACHINE(qdev_get_machine());
     CPUState *cs = CPU(dev);
     X86CPU *cpu = X86_CPU(dev);
     X86CPUClass *xcc = X86_CPU_GET_CLASS(dev);
@@ -6541,6 +6542,15 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
         }
     }
 
+    if (BIT_ULL(cpu->phys_bits) < machine->maxram_size) {
+        error_setg(&local_err, "cannot setup guest memory: "
+                   "%s memory(%lu MiB) exceeds addressable limit(%llu MiB)",
+                   machine->maxram_size == machine->ram_size ? "" : "max",
+                   machine->maxram_size / MiB,
+                   BIT_ULL(cpu->phys_bits) / MiB);
+        goto out;
+    }
+
     /* Cache information initialization */
     if (!cpu->legacy_cache) {
         if (!xcc->model || !xcc->model->cpudef->cache_info) {
-- 
2.32.0

Re: [PATCH] x86: cpu: Error out if memory exceeds addressable range

[Igor Mammedov]

On Mon, 18 Jul 2022 13:47:34 +0530
Nikunj A Dadhania <nikunj@amd.com> wrote:

> Currently it is possible to start a guest with memory that is beyond
> the addressable range of CPU and QEMU does not even warn about it.
> The default phys_bits is 40 and can address 1TB. However it allows to
> start a guest with greater than 1TB memory.
> 
> Prevent this by erroring out in such a scenario.
> 
> Reported-by: Shaju Abraham <Abraham.Shaju@amd.com>
> Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>


Following shall care of your issue:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg900136.html

> ---
>  target/i386/cpu.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index 6a57ef13af..1afbdbac7d 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -6376,6 +6376,7 @@ static void x86_cpu_hyperv_realize(X86CPU *cpu)
>  
>  static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>  {
> +    MachineState *machine = MACHINE(qdev_get_machine());
>      CPUState *cs = CPU(dev);
>      X86CPU *cpu = X86_CPU(dev);
>      X86CPUClass *xcc = X86_CPU_GET_CLASS(dev);
> @@ -6541,6 +6542,15 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
>          }
>      }
>  
> +    if (BIT_ULL(cpu->phys_bits) < machine->maxram_size) {
> +        error_setg(&local_err, "cannot setup guest memory: "
> +                   "%s memory(%lu MiB) exceeds addressable limit(%llu MiB)",
> +                   machine->maxram_size == machine->ram_size ? "" : "max",
> +                   machine->maxram_size / MiB,
> +                   BIT_ULL(cpu->phys_bits) / MiB);
> +        goto out;
> +    }
> +
>      /* Cache information initialization */
>      if (!cpu->legacy_cache) {
>          if (!xcc->model || !xcc->model->cpudef->cache_info) {

Re: [PATCH] x86: cpu: Error out if memory exceeds addressable range

[Nikunj A. Dadhania]

On 7/18/2022 6:12 PM, Igor Mammedov wrote:
> On Mon, 18 Jul 2022 13:47:34 +0530
> Nikunj A Dadhania <nikunj@amd.com> wrote:
> 
>> Currently it is possible to start a guest with memory that is beyond
>> the addressable range of CPU and QEMU does not even warn about it.
>> The default phys_bits is 40 and can address 1TB. However it allows to
>> start a guest with greater than 1TB memory.
>>
>> Prevent this by erroring out in such a scenario.
>>
>> Reported-by: Shaju Abraham <Abraham.Shaju@amd.com>
>> Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
> 
> 
> Following shall care of your issue:
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg900136.html

Thanks, I tried out the patch series, I could start guest till 978G (not sure 
why this magic number yet) and after that I start getting errors:

$ ./build/qemu-system-x86_64 -enable-kvm  -machine q35 -m 979G  -kernel bzImage -initrd initramfs.cpio -vga none -nographic -append "console=ttyS0,115200n8 earlyprintk=serial,ttyS0,115200 debug=1 " -nodefaults -serial stdio
qemu-system-x86_64: Address space limit 0xffffffffff < 0x1fc3fffffff phys-bits too low (40)

Regards
Nikunj

Re: [PATCH] x86: cpu: Error out if memory exceeds addressable range

[Joao Martins]

On 7/18/22 14:10, Nikunj A. Dadhania wrote:
> On 7/18/2022 6:12 PM, Igor Mammedov wrote:
>> On Mon, 18 Jul 2022 13:47:34 +0530
>> Nikunj A Dadhania <nikunj@amd.com> wrote:
>>
>>> Currently it is possible to start a guest with memory that is beyond
>>> the addressable range of CPU and QEMU does not even warn about it.
>>> The default phys_bits is 40 and can address 1TB. However it allows to
>>> start a guest with greater than 1TB memory.
>>>
>>> Prevent this by erroring out in such a scenario.
>>>
>>> Reported-by: Shaju Abraham <Abraham.Shaju@amd.com>
>>> Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
>>
>>
>> Following shall care of your issue:
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg900136.html
> 
> Thanks, I tried out the patch series, I could start guest till 978G (not sure 
> why this magic number yet) and after that I start getting errors:

It's expected. The point of the series is meant to avoid attempting at DMA mapping
over the HyperTransport region. Before it would just fail to either hotplug/boot with VFIO
devices on kernels >= 5.4 (even if older kernels or other configs let you go through you
might still see IOMMU errors at some point). So what we essentially do is to have the
region above 4G to instead start at 1T, thus requiring 1 more phys-bit on cases like this
where the max gpa hits the Hyper Transport reserved region.

The cover-letter and this patch
(https://lore.kernel.org/qemu-devel/20220715171628.21437-11-joao.m.martins@oracle.com/)
should clarify on the logic.

The check you're adding here is essentially patch 9 of the series.

> 
> $ ./build/qemu-system-x86_64 -enable-kvm  -machine q35 -m 979G  -kernel bzImage -initrd initramfs.cpio -vga none -nographic -append "console=ttyS0,115200n8 earlyprintk=serial,ttyS0,115200 debug=1 " -nodefaults -serial stdio
> qemu-system-x86_64: Address space limit 0xffffffffff < 0x1fc3fffffff phys-bits too low (40)
> 
> Regards
> Nikunj

Re: [PATCH] x86: cpu: Error out if memory exceeds addressable range

[Nikunj A. Dadhania]

On 7/18/2022 7:15 PM, Joao Martins wrote:
> On 7/18/22 14:10, Nikunj A. Dadhania wrote:
>> On 7/18/2022 6:12 PM, Igor Mammedov wrote:
>>> On Mon, 18 Jul 2022 13:47:34 +0530
>>> Nikunj A Dadhania <nikunj@amd.com> wrote:
>>>
>>>> Currently it is possible to start a guest with memory that is beyond
>>>> the addressable range of CPU and QEMU does not even warn about it.
>>>> The default phys_bits is 40 and can address 1TB. However it allows to
>>>> start a guest with greater than 1TB memory.
>>>>
>>>> Prevent this by erroring out in such a scenario.
>>>>
>>>> Reported-by: Shaju Abraham <Abraham.Shaju@amd.com>
>>>> Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
>>>
>>>
>>> Following shall care of your issue:
>>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg900136.html
>>
>> Thanks, I tried out the patch series, I could start guest till 978G (not sure 
>> why this magic number yet) and after that I start getting errors:
> 
> It's expected. The point of the series is meant to avoid attempting at DMA mapping
> over the HyperTransport region. Before it would just fail to either hotplug/boot with VFIO
> devices on kernels >= 5.4 (even if older kernels or other configs let you go through you
> might still see IOMMU errors at some point). So what we essentially do is to have the
> region above 4G to instead start at 1T, thus requiring 1 more phys-bit on cases like this
> where the max gpa hits the Hyper Transport reserved region.
> 
> The cover-letter and this patch
> (https://lore.kernel.org/qemu-devel/20220715171628.21437-11-joao.m.martins@oracle.com/
> should clarify on the logic.

Thanks looks good !

> The check you're adding here is essentially patch 9 of the series.

Yes, saw that change.

Regards
Nikunj