[PATCH] Revert "cputlb: Restrict SavedIOTLB to system emulation"

[PATCH] Revert "cputlb: Restrict SavedIOTLB to system emulation"

[Peter Maydell]

This reverts commit d7ee93e24359703debf4137f4cc632563aa4e8d1.

That commit tries to make a field in the CPUState struct not be
present when CONFIG_USER_ONLY is set.  Unfortunately, you can't
conditionally omit fields in structs like this based on ifdefs that
are set per-target.  If you try it, then code in files compiled
per-target (where CONFIG_USER_ONLY is or can be set) will disagree
about the struct layout with files that are compiled once-only (where
this kind of ifdef is never set).

This manifests specifically in 'make check-tcg' failing, because code
in cpus-common.c that sets up the CPUState::cpu_index field puts it
at a different offset from the code in plugins/core.c in
qemu_plugin_vcpu_init_hook() which reads the cpu_index field.  The
latter then hits an assert because from its point of view every
thread has a 0 cpu_index. There might be other weird behaviour too.

Mostly we catch this kind of bug because the CONFIG_whatever is
listed in include/exec/poison.h and so the reference to it in
build-once source files will then cause a compiler error.
Unfortunately CONFIG_USER_ONLY is an exception to that: we have some
places where we use it in "safe" ways in headers that will be seen by
once-only source files (e.g.  ifdeffing out function prototypes) and
it would be a lot of refactoring to be able to get to a position
where we could poison it.  This leaves us in a "you have to be
careful to walk around the bear trap" situation...

Fixes: d7ee93e243597 ("cputlb: Restrict SavedIOTLB to system emulation")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
 include/hw/core/cpu.h | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index ee8d6b40b3b..4871ad85f07 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -226,7 +226,7 @@ struct CPUWatchpoint {
     QTAILQ_ENTRY(CPUWatchpoint) entry;
 };
 
-#if defined(CONFIG_PLUGIN) && !defined(CONFIG_USER_ONLY)
+#ifdef CONFIG_PLUGIN
 /*
  * For plugins we sometime need to save the resolved iotlb data before
  * the memory regions get moved around  by io_writex.
@@ -410,11 +410,9 @@ struct CPUState {
 
 #ifdef CONFIG_PLUGIN
     GArray *plugin_mem_cbs;
-#if !defined(CONFIG_USER_ONLY)
     /* saved iotlb data from io_writex */
     SavedIOTLB saved_iotlb;
-#endif /* !CONFIG_USER_ONLY */
-#endif /* CONFIG_PLUGIN */
+#endif
 
     /* TODO Move common fields from CPUArchState here. */
     int cpu_index;
-- 
2.34.1

Re: [PATCH] Revert "cputlb: Restrict SavedIOTLB to system emulation"

[Richard Henderson]

On 6/20/23 19:57, Peter Maydell wrote:
> This reverts commit d7ee93e24359703debf4137f4cc632563aa4e8d1.
> 
> That commit tries to make a field in the CPUState struct not be
> present when CONFIG_USER_ONLY is set.  Unfortunately, you can't
> conditionally omit fields in structs like this based on ifdefs that
> are set per-target.  If you try it, then code in files compiled
> per-target (where CONFIG_USER_ONLY is or can be set) will disagree
> about the struct layout with files that are compiled once-only (where
> this kind of ifdef is never set).
> 
> This manifests specifically in 'make check-tcg' failing, because code
> in cpus-common.c that sets up the CPUState::cpu_index field puts it
> at a different offset from the code in plugins/core.c in
> qemu_plugin_vcpu_init_hook() which reads the cpu_index field.  The
> latter then hits an assert because from its point of view every
> thread has a 0 cpu_index. There might be other weird behaviour too.
> 
> Mostly we catch this kind of bug because the CONFIG_whatever is
> listed in include/exec/poison.h and so the reference to it in
> build-once source files will then cause a compiler error.
> Unfortunately CONFIG_USER_ONLY is an exception to that: we have some
> places where we use it in "safe" ways in headers that will be seen by
> once-only source files (e.g.  ifdeffing out function prototypes) and
> it would be a lot of refactoring to be able to get to a position
> where we could poison it.  This leaves us in a "you have to be
> careful to walk around the bear trap" situation...
> 
> Fixes: d7ee93e243597 ("cputlb: Restrict SavedIOTLB to system emulation")
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
>   include/hw/core/cpu.h | 6 ++----
>   1 file changed, 2 insertions(+), 4 deletions(-)

Ho hum, thanks.  I'll apply this directly.

r~

Re: [PATCH] Revert "cputlb: Restrict SavedIOTLB to system emulation"

[Philippe Mathieu-Daudé]

On 21/6/23 07:19, Richard Henderson wrote:
> On 6/20/23 19:57, Peter Maydell wrote:
>> This reverts commit d7ee93e24359703debf4137f4cc632563aa4e8d1.
>>
>> That commit tries to make a field in the CPUState struct not be
>> present when CONFIG_USER_ONLY is set.  Unfortunately, you can't
>> conditionally omit fields in structs like this based on ifdefs that
>> are set per-target.  If you try it, then code in files compiled
>> per-target (where CONFIG_USER_ONLY is or can be set) will disagree
>> about the struct layout with files that are compiled once-only (where
>> this kind of ifdef is never set).

Oops, sorry.

>> This manifests specifically in 'make check-tcg' failing, because code
>> in cpus-common.c that sets up the CPUState::cpu_index field puts it
>> at a different offset from the code in plugins/core.c in
>> qemu_plugin_vcpu_init_hook() which reads the cpu_index field.  The
>> latter then hits an assert because from its point of view every
>> thread has a 0 cpu_index. There might be other weird behaviour too.

Why isn't this covered by CI, and where could we add a such check?

>> Mostly we catch this kind of bug because the CONFIG_whatever is
>> listed in include/exec/poison.h and so the reference to it in
>> build-once source files will then cause a compiler error.
>> Unfortunately CONFIG_USER_ONLY is an exception to that: we have some
>> places where we use it in "safe" ways in headers that will be seen by
>> once-only source files (e.g.  ifdeffing out function prototypes) and
>> it would be a lot of refactoring to be able to get to a position
>> where we could poison it.  This leaves us in a "you have to be
>> careful to walk around the bear trap" situation...
>>
>> Fixes: d7ee93e243597 ("cputlb: Restrict SavedIOTLB to system emulation")
>> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>> ---
>>   include/hw/core/cpu.h | 6 ++----
>>   1 file changed, 2 insertions(+), 4 deletions(-)
> 
> Ho hum, thanks.  I'll apply this directly.

Thanks both.

Re: [PATCH] Revert "cputlb: Restrict SavedIOTLB to system emulation"

[Philippe Mathieu-Daudé]

On 21/6/23 11:39, Philippe Mathieu-Daudé wrote:
> On 21/6/23 07:19, Richard Henderson wrote:
>> On 6/20/23 19:57, Peter Maydell wrote:
>>> This reverts commit d7ee93e24359703debf4137f4cc632563aa4e8d1.
>>>
>>> That commit tries to make a field in the CPUState struct not be
>>> present when CONFIG_USER_ONLY is set.  Unfortunately, you can't
>>> conditionally omit fields in structs like this based on ifdefs that
>>> are set per-target.  If you try it, then code in files compiled
>>> per-target (where CONFIG_USER_ONLY is or can be set) will disagree
>>> about the struct layout with files that are compiled once-only (where
>>> this kind of ifdef is never set).
> 
> Oops, sorry.
> 
>>> This manifests specifically in 'make check-tcg' failing, because code
>>> in cpus-common.c that sets up the CPUState::cpu_index field puts it
>>> at a different offset from the code in plugins/core.c in
>>> qemu_plugin_vcpu_init_hook() which reads the cpu_index field.  The
>>> latter then hits an assert because from its point of view every
>>> thread has a 0 cpu_index. There might be other weird behaviour too.
> 
> Why isn't this covered by CI, and where could we add a such check?

Actually it is covered and failed on staging:
https://gitlab.com/qemu-project/qemu/-/jobs/4503766933

Anyhow, sorry for the mess.

>>> Mostly we catch this kind of bug because the CONFIG_whatever is
>>> listed in include/exec/poison.h and so the reference to it in
>>> build-once source files will then cause a compiler error.
>>> Unfortunately CONFIG_USER_ONLY is an exception to that: we have some
>>> places where we use it in "safe" ways in headers that will be seen by
>>> once-only source files (e.g.  ifdeffing out function prototypes) and
>>> it would be a lot of refactoring to be able to get to a position
>>> where we could poison it.  This leaves us in a "you have to be
>>> careful to walk around the bear trap" situation...
>>>
>>> Fixes: d7ee93e243597 ("cputlb: Restrict SavedIOTLB to system emulation")
>>> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
>>> ---
>>>   include/hw/core/cpu.h | 6 ++----
>>>   1 file changed, 2 insertions(+), 4 deletions(-)
>>
>> Ho hum, thanks.  I'll apply this directly.
> 
> Thanks both.
> 

Re: [PATCH] Revert "cputlb: Restrict SavedIOTLB to system emulation"

[Thiago Jung Bauermann]

Hello,

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> On 21/6/23 11:39, Philippe Mathieu-Daudé wrote:
>> On 21/6/23 07:19, Richard Henderson wrote:
>>> On 6/20/23 19:57, Peter Maydell wrote:
>>>> This manifests specifically in 'make check-tcg' failing, because code
>>>> in cpus-common.c that sets up the CPUState::cpu_index field puts it
>>>> at a different offset from the code in plugins/core.c in
>>>> qemu_plugin_vcpu_init_hook() which reads the cpu_index field.  The
>>>> latter then hits an assert because from its point of view every
>>>> thread has a 0 cpu_index. There might be other weird behaviour too.
>> Why isn't this covered by CI, and where could we add a such check?
>
> Actually it is covered and failed on staging:
> https://gitlab.com/qemu-project/qemu/-/jobs/4503766933

Just for the record, it was also caught yesterday by the TCWG CI:

https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-arm-bisect/87/

-- 
Thiago