Re: [PATCH v2 3/7] util: Introduce ThreadContext user-creatable object

[Markus Armbruster <armbru@redhat.com>]

David Hildenbrand <david@redhat.com> writes:

>>> But note that due to dynamic library loading this example will not work
>>> before we actually make use of thread_context_create_thread() in QEMU
>>> code, because the type will otherwise not get registered.
>> 
>> What do you mean exactly by "not work"?  It's not "CLI option or HMP
>> command fails":
>
> For me, if I compile patch #1-#3 only, I get:
>
> $ ./build/qemu-system-x86_64 -S -display none -nodefaults -monitor stdio -object thread-context,id=tc1,cpu-affinity=0-1,cpu-affinity=6-7
> qemu-system-x86_64: invalid object type: thread-context
>
>
> Reason is that, without a call to thread_context_create_thread(), we won't trigger type_init(thread_context_register_types) and consequently, 
> the type won't be registered.
>
> Is it really different in your environment? Maybe it depends on the QEMU config?

I just tested again, and get the same result as you.  I figure my
previous test was with the complete series.

PATCH 5 appears to make it work.  Suggest to say something like "The
commit after next will make this work".

>>      $ upstream-qemu -S -display none -nodefaults -monitor stdio -object thread-context,id=tc1,cpu-affinity=0-1,cpu-affinity=6-7
>>      QEMU 7.1.50 monitor - type 'help' for more information
>>      (qemu) qom-get tc1 cpu-affinity
>>      [
>>          0,
>>          1,
>>          6,
>>          7
>>      ]
>>      (qemu) info cpus
>>      * CPU #0: thread_id=1670613
>> 
>> Even though the affinities refer to nonexistent CPUs :)
>
> CPU affinities are CPU numbers on your system (host), not QEMU vCPU numbers. I could talk about physical CPU numbers in the doc here, 
> although I am not sure if that really helps. What about "host CPU numbers" and in patch #4 "host node numbers"?

I think this would reduce the confusion opportunities for noobs like me.

> Seems to match what we document for @MemoryBackendProperties: "@host-nodes: the list of NUMA host nodes to bind the memory to"

Even better.

> But unrelated to that, pthread_setaffinity_np() won't bail out on CPUs that are currently not available in the host -- because one might 
> online/hotplug them later. It only bails out if none of the CPUs is currently available in the host:
>
> https://man7.org/linux/man-pages/man3/pthread_setaffinity_np.3.html
>
>
>        EINVAL (pthread_setaffinity_np()) The affinity bit mask mask
>               contains no processors that are currently physically on
>               the system and permitted to the thread according to any
>               restrictions that may be imposed by the "cpuset" mechanism
>               described in cpuset(7).
>
> It will bail out on CPUs that cannot be available in the host though, because it's impossible due to the kernel config:
>
>
>        EINVAL (pthread_setaffinity_np()) cpuset specified a CPU that was
>               outside the set supported by the kernel.  (The kernel
>               configuration option CONFIG_NR_CPUS defines the range of
>               the set supported by the kernel data type used to
>               represent CPU sets.)
>
>
>>> A ThreadContext can be reused, simply by reconfiguring the CPU affinity.
>> 
>> So, when a thread is created, its affinity comes from its thread context
>> (if any).  When I later change the context's affinity, it does *not*
>> affect existing threads, only future ones.  Correct?
>
> Yes, that's the current state.

Thanks!

>>> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
>>> Signed-off-by: David Hildenbrand <david@redhat.com>

[...]

>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>> index 80dd419b39..67d47f4051 100644
>>> --- a/qapi/qom.json
>>> +++ b/qapi/qom.json
>>> @@ -830,6 +830,21 @@
>>>               'reduced-phys-bits': 'uint32',
>>>               '*kernel-hashes': 'bool' } }
>>>   +##
>>> +# @ThreadContextProperties:
>>> +#
>>> +# Properties for thread context objects.
>>> +#
>>> +# @cpu-affinity: the list of CPU numbers used as CPU affinity for all threads
>>> +#                created in the thread context (default: QEMU main thread
>>> +#                affinity)
>>
>> Another ignorant question: is the QEMU main thread affinity fixed or
>> configurable?  If configurable, how?
>
> AFAIK, it's only configurable externally, for example, via "virsh emulatorpin". There is no QEMU interface to adjust that (because it 
> wouldn't work).
>
> Libvirt will essentially trigger "taskset" on the emulator thread to change its CPU affinity.

I see.

QAPI schema
Acked-by: Markus Armbruster <armbru@redhat.com>