[PATCH v3 0/3] plugins: generate list of symbols automatically

[PATCH v3 0/3] plugins: generate list of symbols automatically

[Pierrick Bouvier]

Now that meson build for plugins was merged, we can cleanup another part with
the symbols file.
It has to be kept in sync between the header (qemu-plugin.h) and the symbols
file. This has proved to be error prone and tedious.

We solve this by generating this list from header directly using a python
script. Dependencies are triggered correctly, and the file is regenerated if the
header is modified, as expected.

Pierrick Bouvier (3):
  plugins: add missing export for qemu_plugin_num_vcpus
  plugins: detect qemu plugin API symbols from header
  plugins: eradicate qemu-plugins.symbols static file

 MAINTAINERS                    |  1 +
 include/qemu/qemu-plugin.h     |  1 +
 plugins/meson.build            | 12 +++++--
 plugins/qemu-plugins.symbols   | 59 ----------------------------------
 scripts/qemu-plugin-symbols.py | 45 ++++++++++++++++++++++++++
 5 files changed, 56 insertions(+), 62 deletions(-)
 delete mode 100644 plugins/qemu-plugins.symbols
 create mode 100755 scripts/qemu-plugin-symbols.py

-- 
2.39.5

[PATCH v3 1/3] plugins: add missing export for qemu_plugin_num_vcpus

[Pierrick Bouvier]

Fixes: 4a448b148ca ("plugins: add qemu_plugin_num_vcpus function")
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
---
 include/qemu/qemu-plugin.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/qemu/qemu-plugin.h b/include/qemu/qemu-plugin.h
index 622c9a02327..0fba36ae028 100644
--- a/include/qemu/qemu-plugin.h
+++ b/include/qemu/qemu-plugin.h
@@ -803,6 +803,7 @@ void qemu_plugin_register_atexit_cb(qemu_plugin_id_t id,
                                     qemu_plugin_udata_cb_t cb, void *userdata);
 
 /* returns how many vcpus were started at this point */
+QEMU_PLUGIN_API
 int qemu_plugin_num_vcpus(void);
 
 /**
-- 
2.39.5

[PATCH v3 2/3] plugins: detect qemu plugin API symbols from header

[Pierrick Bouvier]

Instead of using a static file (error prone and hard to keep in sync),
we generate it using a script.

Note: if a symbol is not exported, we'll now notice it when linking for
Windows/MacOS platforms.

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
---
 MAINTAINERS                    |  1 +
 plugins/meson.build            | 12 ++++++---
 scripts/qemu-plugin-symbols.py | 45 ++++++++++++++++++++++++++++++++++
 3 files changed, 55 insertions(+), 3 deletions(-)
 create mode 100755 scripts/qemu-plugin-symbols.py

diff --git a/MAINTAINERS b/MAINTAINERS
index 0844f5da196..022e9f0f57f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3746,6 +3746,7 @@ F: plugins/
 F: tests/tcg/plugins/
 F: tests/functional/test_aarch64_tcg_plugins.py
 F: contrib/plugins/
+F: scripts/qemu-plugin-symbols.py
 
 AArch64 TCG target
 M: Richard Henderson <richard.henderson@linaro.org>
diff --git a/plugins/meson.build b/plugins/meson.build
index 51297582f93..d60be2a4d6d 100644
--- a/plugins/meson.build
+++ b/plugins/meson.build
@@ -2,11 +2,17 @@ if not get_option('plugins')
   subdir_done()
 endif
 
+qemu_plugin_symbols = configure_file(
+  input: files('../include/qemu/qemu-plugin.h'),
+  output: 'qemu-plugin.symbols',
+  capture: true,
+  command: [files('../scripts/qemu-plugin-symbols.py'), '@INPUT@'])
+
 # Modules need more symbols than just those in plugins/qemu-plugins.symbols
 if not enable_modules
   if host_os == 'darwin'
     configure_file(
-      input: files('qemu-plugins.symbols'),
+      input: qemu_plugin_symbols,
       output: 'qemu-plugins-ld64.symbols',
       capture: true,
       command: ['sed', '-ne', 's/^[[:space:]]*\\(qemu_.*\\);/_\\1/p', '@INPUT@'])
@@ -15,7 +21,7 @@ if not enable_modules
     # LLVM/lld does not support exporting specific symbols. However, it works
     # out of the box with dllexport/dllimport attribute we set in the code.
   else
-    emulator_link_args += ['-Xlinker', '--dynamic-list=' + (meson.project_source_root() / 'plugins/qemu-plugins.symbols')]
+    emulator_link_args += ['-Xlinker', '--dynamic-list=' + qemu_plugin_symbols.full_path()]
   endif
 endif
 
@@ -24,7 +30,7 @@ if host_os == 'windows'
   # First, create a .def file listing all the symbols a plugin should expect to have
   # available in qemu
   win32_plugin_def = configure_file(
-    input: files('qemu-plugins.symbols'),
+    input: qemu_plugin_symbols,
     output: 'qemu_plugin_api.def',
     capture: true,
     command: ['sed', '-e', '0,/^/s//EXPORTS/; s/[{};]//g', '@INPUT@'])
diff --git a/scripts/qemu-plugin-symbols.py b/scripts/qemu-plugin-symbols.py
new file mode 100755
index 00000000000..e285ebb8f9e
--- /dev/null
+++ b/scripts/qemu-plugin-symbols.py
@@ -0,0 +1,45 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+#
+# Extract QEMU Plugin API symbols from a header file
+#
+# Copyright 2024 Linaro Ltd
+#
+# Author: Pierrick Bouvier <pierrick.bouvier@linaro.org>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import argparse
+import re
+
+def extract_symbols(plugin_header):
+    with open(plugin_header) as file:
+        content = file.read()
+    # Remove QEMU_PLUGIN_API macro definition.
+    content = content.replace('#define QEMU_PLUGIN_API', '')
+    expected = content.count('QEMU_PLUGIN_API')
+    # Find last word between QEMU_PLUGIN_API and (, matching on several lines.
+    # We use *? non-greedy quantifier.
+    syms = re.findall(r'QEMU_PLUGIN_API.*?(\w+)\s*\(', content, re.DOTALL)
+    syms.sort()
+    # Ensure we found as many symbols as API markers.
+    assert len(syms) == expected
+    return syms
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description='Extract QEMU plugin symbols')
+    parser.add_argument('plugin_header', help='Path to QEMU plugin header.')
+    args = parser.parse_args()
+
+    syms = extract_symbols(args.plugin_header)
+
+    print('{')
+    for s in syms:
+        print("  {};".format(s))
+    print('};')
+
+if __name__ == '__main__':
+    main()
-- 
2.39.5

[PATCH v3 3/3] plugins: eradicate qemu-plugins.symbols static file

[Pierrick Bouvier]

Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
---
 plugins/qemu-plugins.symbols | 59 ------------------------------------
 1 file changed, 59 deletions(-)
 delete mode 100644 plugins/qemu-plugins.symbols

diff --git a/plugins/qemu-plugins.symbols b/plugins/qemu-plugins.symbols
deleted file mode 100644
index 032661f9ea7..00000000000
--- a/plugins/qemu-plugins.symbols
+++ /dev/null
@@ -1,59 +0,0 @@
-{
-  qemu_plugin_bool_parse;
-  qemu_plugin_end_code;
-  qemu_plugin_entry_code;
-  qemu_plugin_get_hwaddr;
-  qemu_plugin_get_registers;
-  qemu_plugin_hwaddr_device_name;
-  qemu_plugin_hwaddr_is_io;
-  qemu_plugin_hwaddr_phys_addr;
-  qemu_plugin_insn_data;
-  qemu_plugin_insn_disas;
-  qemu_plugin_insn_haddr;
-  qemu_plugin_insn_size;
-  qemu_plugin_insn_symbol;
-  qemu_plugin_insn_vaddr;
-  qemu_plugin_mem_get_value;
-  qemu_plugin_mem_is_big_endian;
-  qemu_plugin_mem_is_sign_extended;
-  qemu_plugin_mem_is_store;
-  qemu_plugin_mem_size_shift;
-  qemu_plugin_num_vcpus;
-  qemu_plugin_outs;
-  qemu_plugin_path_to_binary;
-  qemu_plugin_read_memory_vaddr;
-  qemu_plugin_read_register;
-  qemu_plugin_register_atexit_cb;
-  qemu_plugin_register_flush_cb;
-  qemu_plugin_register_vcpu_exit_cb;
-  qemu_plugin_register_vcpu_idle_cb;
-  qemu_plugin_register_vcpu_init_cb;
-  qemu_plugin_register_vcpu_insn_exec_cb;
-  qemu_plugin_register_vcpu_insn_exec_cond_cb;
-  qemu_plugin_register_vcpu_insn_exec_inline_per_vcpu;
-  qemu_plugin_register_vcpu_mem_cb;
-  qemu_plugin_register_vcpu_mem_inline_per_vcpu;
-  qemu_plugin_register_vcpu_resume_cb;
-  qemu_plugin_register_vcpu_syscall_cb;
-  qemu_plugin_register_vcpu_syscall_ret_cb;
-  qemu_plugin_register_vcpu_tb_exec_cb;
-  qemu_plugin_register_vcpu_tb_exec_cond_cb;
-  qemu_plugin_register_vcpu_tb_exec_inline_per_vcpu;
-  qemu_plugin_register_vcpu_tb_trans_cb;
-  qemu_plugin_request_time_control;
-  qemu_plugin_reset;
-  qemu_plugin_scoreboard_free;
-  qemu_plugin_scoreboard_find;
-  qemu_plugin_scoreboard_new;
-  qemu_plugin_start_code;
-  qemu_plugin_tb_get_insn;
-  qemu_plugin_tb_n_insns;
-  qemu_plugin_tb_vaddr;
-  qemu_plugin_u64_add;
-  qemu_plugin_u64_get;
-  qemu_plugin_u64_set;
-  qemu_plugin_u64_sum;
-  qemu_plugin_uninstall;
-  qemu_plugin_update_ns;
-  qemu_plugin_vcpu_for_each;
-};
-- 
2.39.5

Re: [PATCH v3 1/3] plugins: add missing export for qemu_plugin_num_vcpus

[Pierrick Bouvier]

On 11/6/24 09:09, Pierrick Bouvier wrote:
> Fixes: 4a448b148ca ("plugins: add qemu_plugin_num_vcpus function")
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
> ---
>   include/qemu/qemu-plugin.h | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/include/qemu/qemu-plugin.h b/include/qemu/qemu-plugin.h
> index 622c9a02327..0fba36ae028 100644
> --- a/include/qemu/qemu-plugin.h
> +++ b/include/qemu/qemu-plugin.h
> @@ -803,6 +803,7 @@ void qemu_plugin_register_atexit_cb(qemu_plugin_id_t id,
>                                       qemu_plugin_udata_cb_t cb, void *userdata);
>   
>   /* returns how many vcpus were started at this point */
> +QEMU_PLUGIN_API
>   int qemu_plugin_num_vcpus(void);
>   
>   /**

This patch can be applied to qemu-stable, as it's a simple bug fix.

Re: [PATCH v3 0/3] plugins: generate list of symbols automatically

[Alex Bennée]

Pierrick Bouvier <pierrick.bouvier@linaro.org> writes:

> Now that meson build for plugins was merged, we can cleanup another part with
> the symbols file.
> It has to be kept in sync between the header (qemu-plugin.h) and the symbols
> file. This has proved to be error prone and tedious.
>
> We solve this by generating this list from header directly using a python
> script. Dependencies are triggered correctly, and the file is regenerated if the
> header is modified, as expected.
>
> Pierrick Bouvier (3):
>   plugins: add missing export for qemu_plugin_num_vcpus
>   plugins: detect qemu plugin API symbols from header
>   plugins: eradicate qemu-plugins.symbols static file
>
>  MAINTAINERS                    |  1 +
>  include/qemu/qemu-plugin.h     |  1 +
>  plugins/meson.build            | 12 +++++--

this failed to apply cleanly. 

>  plugins/qemu-plugins.symbols   | 59 ----------------------------------
>  scripts/qemu-plugin-symbols.py | 45 ++++++++++++++++++++++++++
>  5 files changed, 56 insertions(+), 62 deletions(-)
>  delete mode 100644 plugins/qemu-plugins.symbols
>  create mode 100755 scripts/qemu-plugin-symbols.py

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [PATCH v3 0/3] plugins: generate list of symbols automatically

[Pierrick Bouvier]

On 11/12/24 13:08, Alex Bennée wrote:
> Pierrick Bouvier <pierrick.bouvier@linaro.org> writes:
> 
>> Now that meson build for plugins was merged, we can cleanup another part with
>> the symbols file.
>> It has to be kept in sync between the header (qemu-plugin.h) and the symbols
>> file. This has proved to be error prone and tedious.
>>
>> We solve this by generating this list from header directly using a python
>> script. Dependencies are triggered correctly, and the file is regenerated if the
>> header is modified, as expected.
>>
>> Pierrick Bouvier (3):
>>    plugins: add missing export for qemu_plugin_num_vcpus
>>    plugins: detect qemu plugin API symbols from header
>>    plugins: eradicate qemu-plugins.symbols static file
>>
>>   MAINTAINERS                    |  1 +
>>   include/qemu/qemu-plugin.h     |  1 +
>>   plugins/meson.build            | 12 +++++--
> 
> this failed to apply cleanly.

Sorry about that. The commit is correct but the patch context is 
incorrect. Fixed in v4.

> 
>>   plugins/qemu-plugins.symbols   | 59 ----------------------------------
>>   scripts/qemu-plugin-symbols.py | 45 ++++++++++++++++++++++++++
>>   5 files changed, 56 insertions(+), 62 deletions(-)
>>   delete mode 100644 plugins/qemu-plugins.symbols
>>   create mode 100755 scripts/qemu-plugin-symbols.py
>