Re: [PATCH 2/4] x86/iommu: Make x86-iommu a singleton object

[Peter Xu <peterx@redhat.com>]

On Fri, Oct 25, 2024 at 05:55:59PM -0400, Peter Xu wrote:
> On Fri, Oct 25, 2024 at 11:25:23AM +0200, Markus Armbruster wrote:
> > Peter Xu <peterx@redhat.com> writes:
> > 
> > > X86 IOMMUs cannot be created more than one on a system yet.  Make it a
> > > singleton so it guards the system from accidentally create yet another
> > > IOMMU object when one already presents.
> > >
> > > Now if someone tries to create more than one, e.g., via:
> > >
> > >   ./qemu -M q35 -device intel-iommu -device intel-iommu
> > >
> > > The error will change from:
> > >
> > >   qemu-system-x86_64: -device intel-iommu: QEMU does not support multiple vIOMMUs for x86 yet.
> > >
> > > To:
> > >
> > >   qemu-system-x86_64: -device intel-iommu: Class 'intel-iommu' only supports one instance
> > >
> > > Unfortunately, yet we can't remove the singleton check in the machine
> > > hook (pc_machine_device_pre_plug_cb), because there can also be
> > > virtio-iommu involved, which doesn't share a common parent class yet.
> > >
> > > But with this, it should be closer to reach that goal to check singleton by
> > > QOM one day.
> > >
> > > Signed-off-by: Peter Xu <peterx@redhat.com>
> > 
> > $ qemu-system-x86_64 -device amd-iommu,help
> > /work/armbru/qemu/include/hw/boards.h:24:MACHINE: Object 0x56473906f960 is not an instance of type machine
> > Aborted (core dumped)
> > 
> > (gdb) bt
> > #0  0x00007ffff4e43834 in __pthread_kill_implementation () at /lib64/libc.so.6
> > #1  0x00007ffff4df18ee in raise () at /lib64/libc.so.6
> > #2  0x00007ffff4dd98ff in abort () at /lib64/libc.so.6
> > #3  0x0000555555f75ef3 in object_dynamic_cast_assert
> >     (obj=0x555557e03960, typename=0x5555563c403e "machine", file=0x5555563c4018 "/work/armbru/qemu/include/hw/boards.h", line=24, func=0x5555563c4290 <__func__.7> "MACHINE") at ../qom/object.c:936
> > #4  0x0000555555d5db0f in MACHINE (obj=0x555557e03960)
> >     at /work/armbru/qemu/include/hw/boards.h:24
> > #5  0x0000555555d5e030 in x86_iommu_get_default () at ../hw/i386/x86-iommu.c:83
> > #6  0x0000555555d5e262 in x86_iommu_get_instance
> >     (errp=0x5555573d4918 <error_abort>) at ../hw/i386/x86-iommu.c:139
> > #7  0x0000555555f7c27c in singleton_get_instance (class=0x555557e00320)
> >     at ../qom/object_interfaces.c:371
> > #8  0x000055555612a842 in qmp_device_list_properties
> >     (typename=0x555557e001d0 "amd-iommu", errp=0x7fffffffda38)
> >     at ../qom/qom-qmp-cmds.c:147
> > #9  0x0000555555bf20b2 in qdev_device_help (opts=0x555557e001f0)
> >     at ../system/qdev-monitor.c:314
> > #10 0x0000555555bfe06d in device_help_func
> >     (opaque=0x0, opts=0x555557e001f0, errp=0x0) at ../system/vl.c:1208
> > #11 0x0000555556217186 in qemu_opts_foreach
> >     (list=0x55555729e5c0 <qemu_device_opts>, func=0x555555bfe04d <device_help_func>, opaque=0x0, errp=0x0) at ../util/qemu-option.c:1135
> > #12 0x0000555555c01d56 in qemu_process_help_options () at ../system/vl.c:2555
> > #13 0x0000555555c04d81 in qemu_init (argc=3, argv=0x7fffffffde28)
> >     at ../system/vl.c:3654
> > #14 0x000055555612ffae in main (argc=3, argv=0x7fffffffde28)
> >     at ../system/main.c:47
> > 
> 
> Thanks for the report!
> 
> It turns out that qdev_get_machine() cannot be invoked too early, and the
> singleton code can make it earlier..
> 
> We may want a pre-requisite patch to allow qdev_get_machine() to be invoked
> anytime, like:
> 
> ===8<===
> diff --git a/hw/core/qdev.c b/hw/core/qdev.c
> index db36f54d91..7ceae47139 100644
> --- a/hw/core/qdev.c
> +++ b/hw/core/qdev.c
> @@ -831,6 +831,16 @@ Object *qdev_get_machine(void)
>  {
>      static Object *dev;
>  
> +    if (!phase_check(PHASE_MACHINE_CREATED)) {
> +        /*
> +         * When the machine is not created, below can wrongly create
> +         * /machine to be a container.. this enables qdev_get_machine() to
> +         * be used at any time and return NULL properly when machine is not
> +         * created.
> +         */
> +        return NULL;
> +    }
> +
>      if (dev == NULL) {
>          dev = container_get(object_get_root(), "/machine");
>      }
> ===8<===
> 
> I hope it makes sense on its own.

My apologies, spoke too soon here.  This helper is used too after machine
is created, but right before switching to PHASE_MACHINE_CREATE stage..

So we need another way, like:

===8<===
diff --git a/hw/core/qdev.c b/hw/core/qdev.c
index db36f54d91..36a9fdb428 100644
--- a/hw/core/qdev.c
+++ b/hw/core/qdev.c
@@ -832,7 +832,13 @@ Object *qdev_get_machine(void)
     static Object *dev;
 
     if (dev == NULL) {
-        dev = container_get(object_get_root(), "/machine");
+        /*
+         * NOTE: dev can keep being NULL if machine is not yet created!
+         * In which case the function will properly return NULL.
+         *
+         * Whenever machine object is created and found once, we cache it.
+         */
+        dev = object_resolve_path_component(object_get_root(), "machine");
     }
 
     return dev;
===8<===

The idea is still the same.  Meanwhile I'll test more to see whether it has
other issues.

Thanks,

> Then callers who can be invoked earlier
> could then handle NULL properly, in this case..
> 
> ===8<===
> diff --git a/hw/i386/x86-iommu.c b/hw/i386/x86-iommu.c
> index 4bfeb08705..fceb7adfe0 100644
> --- a/hw/i386/x86-iommu.c
> +++ b/hw/i386/x86-iommu.c
> @@ -80,9 +80,15 @@ void x86_iommu_irq_to_msi_message(X86IOMMUIrq *irq, MSIMessage *msg_out)
>  
>  X86IOMMUState *x86_iommu_get_default(void)
>  {
> -    MachineState *ms = MACHINE(qdev_get_machine());
> -    PCMachineState *pcms =
> -        PC_MACHINE(object_dynamic_cast(OBJECT(ms), TYPE_PC_MACHINE));
> +    Object *machine = qdev_get_machine();
> +    PCMachineState *pcms;
> +
> +    /* If machine has not been created, so is the vIOMMU */
> +    if (!machine) {
> +        return NULL;
> +    }
> +
> +    pcms = PC_MACHINE(object_dynamic_cast(machine, TYPE_PC_MACHINE));
>  
>      if (pcms &&
>          object_dynamic_cast(OBJECT(pcms->iommu), TYPE_X86_IOMMU_DEVICE)) {
> ===8<===
> 
> I'll make sure this works if I'll repost.
> 
> Thanks,
> 
> -- 
> Peter Xu

-- 
Peter Xu