[PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[John B. Wyatt IV]

SWIG is a tool packaged in Fedora and other distros that can generate
bindings from C and C++ code for several languages including Python,
Perl, and Go. We at Red Hat are interested in adding binding support to
libcpupower so Python tools like rteval or tuned can make easy use of it.

This RFC provides a limited subset of bindings as a demonstration. The second
commit provides a Python test script to verify the bindings. I wanted to get
feedback on this before implementing (and possibly testing) the entire library.

The name raw_pylibcpupower is used because this is a demonstration example that
only provides direct bindings for a few functions. A wrapper `pylibcpupower`
may be needed to make the bindings more 'pythonic'. The bindings folder is used
because Go bindings may be useful for Kubernetes or OpenShift in the future.

How should the bindings be built? The current example requires the makefile
in cpupower directory be run first to generate the .o files needed before
running the makefile in the python directory in a seperate step. Would the
maintainers prefer the two makefiles integrated?

Another question is do you want more test files like the .py example? Would
this be used as part of a greater test suite?

Note that while SWIG itself is GPL v3+ licensed; the resulting output, the
bindings code, is permissively licensed. Please see
https://swig.org/legal.html for more details.

John B. Wyatt IV (2):
  Add SWIG bindings files for libcpupower
  Include test_raw_pylibcpupower.py

 .../power/cpupower/bindings/python/.gitignore |  8 ++++
 tools/power/cpupower/bindings/python/Makefile | 31 +++++++++++++
 tools/power/cpupower/bindings/python/README   | 33 ++++++++++++++
 .../bindings/python/raw_pylibcpupower.i       | 45 +++++++++++++++++++
 .../bindings/python/test_raw_pylibcpupower.py | 42 +++++++++++++++++
 5 files changed, 159 insertions(+)
 create mode 100644 tools/power/cpupower/bindings/python/.gitignore
 create mode 100644 tools/power/cpupower/bindings/python/Makefile
 create mode 100644 tools/power/cpupower/bindings/python/README
 create mode 100644 tools/power/cpupower/bindings/python/raw_pylibcpupower.i
 create mode 100755 tools/power/cpupower/bindings/python/test_raw_pylibcpupower.py

-- 
2.45.2

[PATCH 1/2] Add SWIG bindings files for libcpupower

[John B. Wyatt IV]

SWIG is a tool packaged in Fedora and other distros that can generate
bindings from C and C++ code for several languages including Python,
Perl, and Go.

A latter commit will demonstrate taking a SWIG .i file and
the libcpupower object files to generate the bindings.

Note that while SWIG itself is GPL v3+ licensed; the resulting output,
the bindings code, is permissively licensed. Please see
https://swig.org/legal.html for more details.

Signed-off-by: John B. Wyatt IV <jwyatt@redhat.com>
Signed-off-by: John B. Wyatt IV <sageofredondo@gmail.com>
---
 .../power/cpupower/bindings/python/.gitignore |  8 ++++
 tools/power/cpupower/bindings/python/Makefile | 31 +++++++++++++
 tools/power/cpupower/bindings/python/README   | 33 ++++++++++++++
 .../bindings/python/raw_pylibcpupower.i       | 45 +++++++++++++++++++
 4 files changed, 117 insertions(+)
 create mode 100644 tools/power/cpupower/bindings/python/.gitignore
 create mode 100644 tools/power/cpupower/bindings/python/Makefile
 create mode 100644 tools/power/cpupower/bindings/python/README
 create mode 100644 tools/power/cpupower/bindings/python/raw_pylibcpupower.i

diff --git a/tools/power/cpupower/bindings/python/.gitignore b/tools/power/cpupower/bindings/python/.gitignore
new file mode 100644
index 000000000000..b83e0f5e26a0
--- /dev/null
+++ b/tools/power/cpupower/bindings/python/.gitignore
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0-only
+__pycache__/
+*.c
+*.o
+*.so
+*.py
+!test_raw_pylibcpupower.py
+!raw_pylibcpupower.i # git keeps ignoring this file
diff --git a/tools/power/cpupower/bindings/python/Makefile b/tools/power/cpupower/bindings/python/Makefile
new file mode 100644
index 000000000000..a6c779333f8d
--- /dev/null
+++ b/tools/power/cpupower/bindings/python/Makefile
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0-only
+# Makefile for libcpupower Python bindings
+#
+# This Makefile expects you have already run the makefile for cpupower to build
+# the .o files in the lib directory for the bindings to be created.
+
+CC=gcc
+
+LIB_DIR = ../../lib
+BIND_DIR = .
+PY_INCLUDE := $(firstword $(shell python-config --includes))
+#PY_INCLUDE = $(shell python-config --includes | awk '{ print $1 }')
+
+OBJECTS_LIB = $(wildcard $(LIB_DIR)/*.o)
+OBJECTS_BIND = $(wildcard $(BIND_DIR)/*.o)
+
+all: _raw_pylibcpupower.so
+
+_raw_pylibcpupower.so: raw_pylibcpupower_wrap.o
+	$(CC) -shared $(OBJECTS_LIB) raw_pylibcpupower_wrap.o -o _raw_pylibcpupower.so # raw_pylibcpupower_wrap.o
+#	$(CC) -shared $(OBJECTS_BIND) $(OBJECTS_LIB) -o _raw_pylibcpupower.so # raw_pylibcpupower_wrap.o
+
+raw_pylibcpupower_wrap.o: raw_pylibcpupower_wrap.c
+	$(CC) -fPIC -c raw_pylibcpupower_wrap.c $(PY_INCLUDE)
+
+raw_pylibcpupower_wrap.c: raw_pylibcpupower.i
+	swig -python raw_pylibcpupower.i
+
+# Will only clean the bindings folder; will not clean the actual cpupower folder
+clean:
+	rm -f raw_pylibcpupower.py raw_pylibcpupower_wrap.c raw_pylibcpupower_wrap.o _raw_pylibcpupower.so
diff --git a/tools/power/cpupower/bindings/python/README b/tools/power/cpupower/bindings/python/README
new file mode 100644
index 000000000000..a2663a2d23d1
--- /dev/null
+++ b/tools/power/cpupower/bindings/python/README
@@ -0,0 +1,33 @@
+
+This folder contains the necessary files to build Python bindings for libcpupower.
+
+To begin, first build the object files for libcpupower by running make in
+the cpupower directory.
+
+Next you will need to install SWIG. Using Fedora:
+
+sudo dnf install swig
+
+Please check that your version of SWIG is compatible with the version of
+Python installed on your machine by checking the SWIG changelog on their
+website. https://swig.org/
+
+Then return to the directory this README is in to run:
+
+make
+
+To run the test script:
+
+python test_raw_pylibcpupower.py
+
+Note that while SWIG itself is GPL v3+ licensed; the resulting output,
+the bindings code, is permissively licensed. Please see
+https://swig.org/legal.html for more details.
+
+Original Bindings Author:
+John B. Wyatt IV
+jwyatt@redhat.com
+sageofredondo@gmail.com
+
+Copyright (C) 2024 Red Hat
+
diff --git a/tools/power/cpupower/bindings/python/raw_pylibcpupower.i b/tools/power/cpupower/bindings/python/raw_pylibcpupower.i
new file mode 100644
index 000000000000..9a920ecf1f47
--- /dev/null
+++ b/tools/power/cpupower/bindings/python/raw_pylibcpupower.i
@@ -0,0 +1,45 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+%module raw_pylibcpupower
+%{
+#include "../../lib/cpupower_intern.h"
+#include "../../lib/acpi_cppc.h"
+#include "../../lib/cpufreq.h"
+#include "../../lib/cpuidle.h"
+#include "../../lib/cpupower.h"
+#include "../../lib/powercap.h"
+%}
+
+/*
+ * cpuidle.h
+ */
+
+unsigned int cpuidle_state_count(unsigned int cpu);
+int cpuidle_state_disable(unsigned int cpu, unsigned int idlestate, unsigned int disable);
+
+/*
+ * cpupower.h
+ */
+
+struct cpupower_topology {
+	/* Amount of CPU cores, packages and threads per core in the system */
+	unsigned int cores;
+	unsigned int pkgs;
+	unsigned int threads; /* per core */
+
+	/* Array gets mallocated with cores entries, holding per core info */
+	struct cpuid_core_info *core_info;
+};
+
+struct cpuid_core_info {
+	int pkg;
+	int core;
+	int cpu;
+
+	/* flags */
+	unsigned int is_online:1;
+};
+
+int get_cpu_topology(struct cpupower_topology *cpu_top);
+void cpu_topology_release(struct cpupower_topology cpu_top);
+int cpupower_is_cpu_online(unsigned int cpu);
-- 
2.45.2

[PATCH 2/2] Include test_raw_pylibcpupower.py

[John B. Wyatt IV]

This script demonstrates how to make use of, and tests, the bindings.

The .i file currently only binds a limited subsection of functionality
with the bindings.

Signed-off-by: John B. Wyatt IV <jwyatt@redhat.com>
Signed-off-by: John B. Wyatt IV <sageofredondo@gmail.com>
---
 .../bindings/python/test_raw_pylibcpupower.py | 42 +++++++++++++++++++
 1 file changed, 42 insertions(+)
 create mode 100755 tools/power/cpupower/bindings/python/test_raw_pylibcpupower.py

diff --git a/tools/power/cpupower/bindings/python/test_raw_pylibcpupower.py b/tools/power/cpupower/bindings/python/test_raw_pylibcpupower.py
new file mode 100755
index 000000000000..3d6f62b9556a
--- /dev/null
+++ b/tools/power/cpupower/bindings/python/test_raw_pylibcpupower.py
@@ -0,0 +1,42 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+
+import raw_pylibcpupower as p
+
+# Simple function call
+
+"""
+Get cstate count
+"""
+cpu_cstates_count = p.cpuidle_state_count(0)
+if cpu_cstates_count > -1:
+    print(f"CPU 0 has {cpu_cstates_count} c-states")
+else:
+    print(f"cstate count error: return code: {cpu_cstates_count}")
+
+"""
+Disable cstate (will fail if the above is 0, ex: a virtual machine)
+"""
+cstate_disabled = p.cpuidle_state_disable(0, 0, 1)
+if cpu_cstates_count == 0:
+    print(f"CPU 0 has {cpu_cstates_count} c-states")
+else:
+    print(f"cstate count error: return code: {cpu_cstates_count}")
+
+match cstate_disabled:
+    case 0:
+        print(f"CPU state disabled")
+    case -1:
+        print(f"Idlestate not available")
+    case _:
+        print(f"Not documented")
+
+
+# Pointer example
+
+topo = p.cpupower_topology()
+total_cpus = p.get_cpu_topology(topo)
+if total_cpus > 0:
+    print(f"Number of total cpus: {total_cpus} and number of cores: {topo.cores}")
+else:
+    print(f"Error: could not get cpu topology")
-- 
2.45.2

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[Shuah Khan]

On 7/24/24 16:11, John B. Wyatt IV wrote:
> SWIG is a tool packaged in Fedora and other distros that can generate
> bindings from C and C++ code for several languages including Python,
> Perl, and Go. We at Red Hat are interested in adding binding support to
> libcpupower so Python tools like rteval or tuned can make easy use of it.
> 
> This RFC provides a limited subset of bindings as a demonstration. The second
> commit provides a Python test script to verify the bindings. I wanted to get
> feedback on this before implementing (and possibly testing) the entire library.
> 
> The name raw_pylibcpupower is used because this is a demonstration example that
> only provides direct bindings for a few functions. A wrapper `pylibcpupower`
> may be needed to make the bindings more 'pythonic'. The bindings folder is used
> because Go bindings may be useful for Kubernetes or OpenShift in the future.
> 
> How should the bindings be built? The current example requires the makefile
> in cpupower directory be run first to generate the .o files needed before
> running the makefile in the python directory in a seperate step. Would the
> maintainers prefer the two makefiles integrated?
> 
> Another question is do you want more test files like the .py example? Would
> this be used as part of a greater test suite?
> 
> Note that while SWIG itself is GPL v3+ licensed; the resulting output, the
> bindings code, is permissively licensed. Please see
> https://swig.org/legal.html for more details.
> 
> John B. Wyatt IV (2):
>    Add SWIG bindings files for libcpupower
>    Include test_raw_pylibcpupower.py
> 
>

Quick note to let you know that I will get to this once the merge window closes.

thanks,
-- Shuah

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[Shuah Khan]

Hi,

On 7/24/24 16:11, John B. Wyatt IV wrote:
> SWIG is a tool packaged in Fedora and other distros that can generate
> bindings from C and C++ code for several languages including Python,
> Perl, and Go. We at Red Hat are interested in adding binding support to
> libcpupower so Python tools like rteval or tuned can make easy use of it.
> 

Can you elaborate on the use-case and what rteval currently does and
how it could benefit from using libcpupower with the bindings?

> This RFC provides a limited subset of bindings as a demonstration. The second
> commit provides a Python test script to verify the bindings. I wanted to get
> feedback on this before implementing (and possibly testing) the entire library.
> 
> The name raw_pylibcpupower is used because this is a demonstration example that
> only provides direct bindings for a few functions. A wrapper `pylibcpupower`
> may be needed to make the bindings more 'pythonic'. The bindings folder is used
> because Go bindings may be useful for Kubernetes or OpenShift in the future.
> 
> How should the bindings be built? The current example requires the makefile
> in cpupower directory be run first to generate the .o files needed before
> running the makefile in the python directory in a seperate step. Would the
> maintainers prefer the two makefiles integrated?

I can't answer this question until I understand the licensing.
However, I would lean towards keeping them separate.

> 
> Another question is do you want more test files like the .py example? Would
> this be used as part of a greater test suite?

I would like to see document outlining the dependencies and examples of how
this would be used. I see the README which says that

"Next you will need to install SWIG. Using Fedora:"

The document will have to include more than Fedora instructions. Instead
of a README I would like to see a document.

Before we go any further - we have the licensing implications to figure out.

> 
> Note that while SWIG itself is GPL v3+ licensed; the resulting output, the
> bindings code, is permissively licensed. Please see
> https://swig.org/legal.html for more details.

Adding Linus and Greg for their feedback and input on this proposal.

What does it mean by "the resulting output, the bindings code is
permissively licensed."

I would like to get a better understanding of the licensing angle
since this code adds dependency on SWIG which is GPL v3+ to build
the proposed python bindings.

thanks,
-- Shuah

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[John B. Wyatt IV]

Addressing the licensing first:

> On Tue, Jul 30, 2024 at 02:48:41PM -0600, Shuah Khan wrote:
> > On 7/24/24 16:11, John B. Wyatt IV wrote:
> > Note that while SWIG itself is GPL v3+ licensed; the resulting output, the
> > bindings code, is permissively licensed. Please see
> > https://swig.org/legal.html for more details.
> 
> Adding Linus and Greg for their feedback and input on this proposal.
> 
> What does it mean by "the resulting output, the bindings code is
> permissively licensed."
> 
> I would like to get a better understanding of the licensing angle
> since this code adds dependency on SWIG which is GPL v3+ to build
> the proposed python bindings.

Copying and pasting from the link above:

/begin

The intention of the SWIG license is to ensure that the SWIG source code (the
code that is compiled into the SWIG executable) remains free software by using
the GPL license on the SWIG source code. SWIG is a code generator and
the intention of the SWIG license is also to enable distribution of the output
code under license terms of the user's choice/requirements. 

[snip html license links]

When SWIG is used as it is distributed by the SWIG developers, its output is
not governed by SWIG's license (including the GPL). SWIG's output contains
code from three sources:

* code generated by SWIG, which is not governed by copyright;
* code copied from the SWIG library which is permissively licensed to be
redistributed without restriction;
* code derived from the user's input, which may be governed by the license of 
he code supplied by the user.

So, while the input supplied to SWIG may affect the license of SWIG's output
(e.g. if the input code is licensed under a copyleft or proprietary license),
SWIG's license does not affect the license of the output. This is consistent
with the FSF's FAQ entries on this subject (GPLOutput and 
WhatCaseIsOutputGPL), because the SWIG code copied into the output by 
SWIG is not GPL-licensed. 

/end

The output of SWIG depends on SWIG's library which is permissively
licensed + the license of the .o files used, which is libcpupower, which is
GPLv2 licensed. Therefore, any bindings generated from libcpupower is GPL v2.

SWIG Library and Examples license:

 The SWIG library and examples, under the Lib and Examples top level 
 directories, are distributed under the following terms:

  You may copy, modify, distribute, and make derivative works based on
  this software, in source code or object code form, without
  restriction. If you distribute the software to others, you may do
  so according to the terms of your choice. This software is offered as
  is, without warranty of any kind.

The snipped license links link to the GPL v3, SWIG's declaration of the GPL 3 or
a later version with the above permissive declaration for output, and a link
to four university licenses that all look permissive.

https://swig.org/Release/LICENSE-UNIVERSITIES

The resulting generated code can be linked and distributed with GPL v2
binaries. I am not a lawyer, but this is my understanding of what they
declare on their website above.

Please let me know if this clarifies this.

>
> On 7/24/24 16:11, John B. Wyatt IV wrote:
> > SWIG is a tool packaged in Fedora and other distros that can generate
> > bindings from C and C++ code for several languages including Python,
> > Perl, and Go. We at Red Hat are interested in adding binding support to
> > libcpupower so Python tools like rteval or tuned can make easy use of it.
> > 
> 
> Can you elaborate on the use-case and what rteval currently does and
> how it could benefit from using libcpupower with the bindings?

rteval is a Python program used to measure realtime performance. We wanted to
test the effect of enabling some levels of idle-stat to see how it affects
latency, and didn't want to reinvent the wheel. We thought that the Python
bindings could be useful to other people as well who might want to call
cpupower too from Python. I did some testing and was able to achieve this with
SWIG. We sent the patchset to see what folks thought about this.

> 
> > 
> > Another question is do you want more test files like the .py example? Would
> > this be used as part of a greater test suite?
> 
> I would like to see document outlining the dependencies and examples of how
> this would be used. I see the README which says that
> 
> "Next you will need to install SWIG. Using Fedora:"
> 
> The document will have to include more than Fedora instructions. Instead
> of a README I would like to see a document.

Understood. I wanted to get an idea of how we would structure the makefile
first.

-- 
Sincerely,
John Wyatt
Software Engineer, Core Kernel
Red Hat

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[Greg Kroah-Hartman]

On Thu, Aug 01, 2024 at 05:24:20PM -0400, John B. Wyatt IV wrote:
> > On 7/24/24 16:11, John B. Wyatt IV wrote:
> > > SWIG is a tool packaged in Fedora and other distros that can generate
> > > bindings from C and C++ code for several languages including Python,
> > > Perl, and Go. We at Red Hat are interested in adding binding support to
> > > libcpupower so Python tools like rteval or tuned can make easy use of it.
> > > 
> > 
> > Can you elaborate on the use-case and what rteval currently does and
> > how it could benefit from using libcpupower with the bindings?
> 
> rteval is a Python program used to measure realtime performance. We wanted to
> test the effect of enabling some levels of idle-stat to see how it affects
> latency, and didn't want to reinvent the wheel. We thought that the Python
> bindings could be useful to other people as well who might want to call
> cpupower too from Python. I did some testing and was able to achieve this with
> SWIG. We sent the patchset to see what folks thought about this.

Is this going to require a built-time dependency on SWIG?  If not, when
would it be run, and who will be in charge of running it and updating
the bindings?

And finally, why do we need these at all?  You are saying these are new
tests that external tools will be using, but why, if external tools are
required to run them, are they needed in the kernel tree at all?  Why
isn't this just another external test-suite that people who care about
measuring this type of thing going to just run on their own if desired?

thanks,

greg k-h

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[John B. Wyatt IV]

Adding Arnaldo to the CC list.

On Sun, Aug 04, 2024 at 10:54:10AM +0200, Greg Kroah-Hartman wrote:
> On Thu, Aug 01, 2024 at 05:24:20PM -0400, John B. Wyatt IV wrote:
> > > On 7/24/24 16:11, John B. Wyatt IV wrote:
> > > > SWIG is a tool packaged in Fedora and other distros that can generate
> > > > bindings from C and C++ code for several languages including Python,
> > > > Perl, and Go. We at Red Hat are interested in adding binding support to
> > > > libcpupower so Python tools like rteval or tuned can make easy use of it.
> > > > 
> > > 
> > > Can you elaborate on the use-case and what rteval currently does and
> > > how it could benefit from using libcpupower with the bindings?
> > 
> > rteval is a Python program used to measure realtime performance. We wanted to
> > test the effect of enabling some levels of idle-stat to see how it affects
> > latency, and didn't want to reinvent the wheel. We thought that the Python
> > bindings could be useful to other people as well who might want to call
> > cpupower too from Python. I did some testing and was able to achieve this with
> > SWIG. We sent the patchset to see what folks thought about this.
> 
> Is this going to require a built-time dependency on SWIG?  If not, when
> would it be run

It is optional, and based on my conversation with Shuah; the bindings will be
in a seperate makefile. It would be ran after running cpupower's
makefile, seperately. (But one can call the other.)

> and who will be in charge of running it and updating the bindings?

That would be myself. If I no longer wish to continue I would reassign it to
another person here on the real-time team at Red Hat. John Kacur (whom I am
working with on rteval) is fine with being listed as a backup contact.

The bindings would need to be updated every time one of the functions or data
structures listed in the .i file changes. But it can be as simple as copying the
changed declaration from the header file to the .i file to resolve.

> And finally, why do we need these at all?

To provide bindings for Python programs like rteval to easily interface with
libcpupower. It is very common for userspace programs to include bindings to
scripting languages.

> You are saying these are new
> tests that external tools will be using, but why, if external tools are
> required to run them, are they needed in the kernel tree at all?  Why
> isn't this just another external test-suite that people who care about
> measuring this type of thing going to just run on their own if desired?

SWIG the tool requires the .o files compiled from libcpupower to generate
bindings. Since we need these artifacts from a packaging and usability perspective
it makes sense to include the bindings source code along in a seperate directory
with the cpupower source code to generate the builds for them at the same time.
The source code is: the .i definitions file, any future Python wrapper helpers
around the bindings needed by SWIG, and the documentation.

Why SWIG? It was chosen for this project because of:

1) Many Python bindings generators are strickly Apache2 or GPL v3 licensed; and
they may inject code into the bindings binary that will create a license conflict
with the GPL v2. Swig was far more clear about the license status of the
generated bindings. As described above:
https://lore.kernel.org/linux-pm/1f5c24b6-f3ee-4863-8b7a-49344a550206@linuxfoundation.org/T/#mb7170232fb429fc242bb45c8d3d4d5ed47f0c59f

2) It's extensive documentation. This is the Python section:
https://www.swig.org/Doc4.2/Python.html#Python

3) Support: SWIG has been around since 1996 and had a release 5 months ago. It's
length of time and the acitvity in it's community makes it unlikely SWIG will
lose support by it's developers anytime soon.

4) This RFC demonstrates how quickly and with little work you can generate
bindings for an existing C project with no code changes to that project. :-)
There were issues getting other Python bindings generators to work as
described in their documentation.

-- 
Sincerely,
John Wyatt
Software Engineer, Core Kernel
Red Hat

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[John B. Wyatt IV]

I wanted to follow up on this since I am close to sending out the v2 of
this patchset.

3 points I wanted to raise:

1) Does everyone understand, is okay with the SWIG license, and wants to
proceed with me sending in a more complete version of this as a candidate for
upstreaming?

2) About maintainership: if I am to be the maintainer of this, how would
myself and John Kacur be listed? As a CPU POWER MONITORING SUBSYSTEM
maintainer, a separate category below it called CPU POWER MONITORING SUBSYSTEM
PYTHON BINDINGS maintainer, or is not needed to be listed at this time?

A quick search for bindings shows what I believe to be all of them for device
tree. This may establish a new precedent.

If I was to be added, I assume it would be a separate commit in the v2
submission?

3) I had to comment out powercap_set_enabled

SWIG reported this symbol not being found despite being in powercap.h. I did a
quick search and was not able to find it's implementation in powercap.c. The
get equivalent powercap_get_enabled is in powercap.c. Wanted to check on this
just in case it is a bug or part of future functionality. I am assuming the
latter; I would send it v2 with that declaration commented out with a note
explaining it for users if there is no objection.
-- 
Sincerely,
John Wyatt
Software Engineer, Core Kernel
Red Hat

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[Shuah Khan]

On 8/20/24 00:40, John B. Wyatt IV wrote:
> I wanted to follow up on this since I am close to sending out the v2 of
> this patchset.
> 
> 3 points I wanted to raise:
> 
> 1) Does everyone understand, is okay with the SWIG license, and wants to
> proceed with me sending in a more complete version of this as a candidate for
> upstreaming?
> 
> 2) About maintainership: if I am to be the maintainer of this, how would
> myself and John Kacur be listed? As a CPU POWER MONITORING SUBSYSTEM
> maintainer, a separate category below it called CPU POWER MONITORING SUBSYSTEM
> PYTHON BINDINGS maintainer, or is not needed to be listed at this time?
> 
> A quick search for bindings shows what I believe to be all of them for device
> tree. This may establish a new precedent.
> 
> If I was to be added, I assume it would be a separate commit in the v2
> submission?
> 
> 3) I had to comment out powercap_set_enabled
> 
> SWIG reported this symbol not being found despite being in powercap.h. I did a
> quick search and was not able to find it's implementation in powercap.c. The
> get equivalent powercap_get_enabled is in powercap.c. Wanted to check on this
> just in case it is a bug or part of future functionality. I am assuming the
> latter; I would send it v2 with that declaration commented out with a note
> explaining it for users if there is no objection.

Yes please send a complete version for review.

thanks,
-- Shuah

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[Shuah Khan]

On 8/21/24 01:08, Shuah Khan wrote:
> On 8/20/24 00:40, John B. Wyatt IV wrote:
>> I wanted to follow up on this since I am close to sending out the v2 of
>> this patchset.
>>

Here is more complete response after my quick response.

>> 3 points I wanted to raise:
>>
>> 1) Does everyone understand, is okay with the SWIG license, and wants to
>> proceed with me sending in a more complete version of this as a candidate for
>> upstreaming?
>>

Yes - send patch series without RFC tag.

>> 2) About maintainership: if I am to be the maintainer of this, how would
>> myself and John Kacur be listed? As a CPU POWER MONITORING SUBSYSTEM
>> maintainer, a separate category below it called CPU POWER MONITORING SUBSYSTEM
>> PYTHON BINDINGS maintainer, or is not needed to be listed at this time?
>>

Let's not add a new category at this time. You can add yourself and John Kacur
be Reviewers under CPU POWER MONITORING SUBSYSTEM for now. We can revisit as
we go forward.

>> A quick search for bindings shows what I believe to be all of them for device
>> tree. This may establish a new precedent.
>>

I don't fully understand the above. Will this patch series include device tree
changes? I didn't get that from the RFC.

>> If I was to be added, I assume it would be a separate commit in the v2
>> submission?
>>
>> 3) I had to comment out powercap_set_enabled
>>
>> SWIG reported this symbol not being found despite being in powercap.h. I did a
>> quick search and was not able to find it's implementation in powercap.c. The
>> get equivalent powercap_get_enabled is in powercap.c. Wanted to check on this
>> just in case it is a bug or part of future functionality. I am assuming the
>> latter; I would send it v2 with that declaration commented out with a note
>> explaining it for users if there is no objection.

Good find. It is missing functionality. For now you can define it in powercap.c to
simply return 0 with a TODO

thanks,
-- Shuah

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[John B. Wyatt IV]

On Wed, Aug 21, 2024 at 09:25:52PM -0600, Shuah Khan wrote:
> On 8/21/24 01:08, Shuah Khan wrote:
> > On 8/20/24 00:40, John B. Wyatt IV wrote:
> > > A quick search for bindings shows what I believe to be all of them for device
> > > tree. This may establish a new precedent.
> > > 
> 
> I don't fully understand the above. Will this patch series include device tree
> changes? I didn't get that from the RFC.

I miswrote. I meant I could not find anything related to bindings for a
user-space program in the kernel repo. Device-bindings was the closest I
could find and it is not relevant. My apologies. Thank you for asking about
that.

-- 
Sincerely,
John Wyatt
Software Engineer, Core Kernel
Red Hat

Re: [PATCH 0/2][RFC] Add SWIG Bindings to libcpupower

[Arnaldo Carvalho de Melo]

On Sun, Aug 04, 2024 at 10:54:10AM +0200, Greg Kroah-Hartman wrote:
> On Thu, Aug 01, 2024 at 05:24:20PM -0400, John B. Wyatt IV wrote:
> > > On 7/24/24 16:11, John B. Wyatt IV wrote:
> > > > SWIG is a tool packaged in Fedora and other distros that can generate
> > > > bindings from C and C++ code for several languages including Python,
> > > > Perl, and Go. We at Red Hat are interested in adding binding support to
> > > > libcpupower so Python tools like rteval or tuned can make easy use of it.
> > > > 
> > > 
> > > Can you elaborate on the use-case and what rteval currently does and
> > > how it could benefit from using libcpupower with the bindings?
> > 
> > rteval is a Python program used to measure realtime performance. We wanted to
> > test the effect of enabling some levels of idle-stat to see how it affects
> > latency, and didn't want to reinvent the wheel. We thought that the Python
> > bindings could be useful to other people as well who might want to call
> > cpupower too from Python. I did some testing and was able to achieve this with
> > SWIG. We sent the patchset to see what folks thought about this.
 
> Is this going to require a built-time dependency on SWIG?  If not, when
> would it be run, and who will be in charge of running it and updating
> the bindings?

> And finally, why do we need these at all?  You are saying these are new
> tests that external tools will be using, but why, if external tools are
> required to run them, are they needed in the kernel tree at all?  Why
> isn't this just another external test-suite that people who care about
> measuring this type of thing going to just run on their own if desired?

We have a python binding for perf, it was done manually, didn't use
something like swig, don't recall why I made that decision at the time,
maybe not to get one extra build dependency, as you mention.

I did it at the time as another tool developed at Red Hat, tuna, a top
like tool, using GTK+, could use the perf infrastructure to get
notifications about new threads and threads exiting, as exemplified by
test tools in the kernel sources:

root@number:/home/acme/git/linux# tools/perf/python/twatch.py 
cpu: 24, pid: 2787405, tid: 2787405 { type: fork, pid: 2787405, ppid: 2787405, tid: 673409, ptid: 2787405, time: 300460407945015}
cpu: 23, pid: 2787405, tid: 673409 { type: comm, pid: 2787405, tid: 673409, comm: StreamT~s #1624 }
cpu: 4, pid: 3923381, tid: 3923381 { type: fork, pid: 3923381, ppid: 3923381, tid: 673410, ptid: 3923381, time: 300460862312442}
cpu: 23, pid: 2787405, tid: 673409 { type: comm, pid: 2787405, tid: 673409, comm: StreamT~s #1624 }
cpu: 23, pid: 3923381, tid: 673410 { type: comm, pid: 3923381, tid: 673410, comm: StreamT~s #3197 }
cpu: 23, pid: 3923381, tid: 673410 { type: comm, pid: 3923381, tid: 673410, comm: StreamT~s #3197 }
cpu: 13, pid: 2787464, tid: 2787464 { type: fork, pid: 2787464, ppid: 2787464, tid: 673411, ptid: 2787464, time: 300461205531219}
cpu: 26, pid: 2787464, tid: 673411 { type: comm, pid: 2787464, tid: 673411, comm: StreamT~s #1624 }
cpu: 26, pid: 2787464, tid: 673411 { type: comm, pid: 2787464, tid: 673411, comm: StreamT~s #1624 }
^CTraceback (most recent call last):
  File "/home/acme/git/linux/tools/perf/python/twatch.py", line 61, in <module>
    main()
  File "/home/acme/git/linux/tools/perf/python/twatch.py", line 33, in main
    evlist.poll(timeout = -1)
KeyboardInterrupt

root@number:/home/acme/git/linux# 

root@number:/home/acme/git/linux# head tools/perf/python/twatch.py 
#! /usr/bin/env python
# SPDX-License-Identifier: GPL-2.0-only
# -*- python -*-
# -*- coding: utf-8 -*-
#   twatch - Experimental use of the perf python interface
#   Copyright (C) 2011 Arnaldo Carvalho de Melo <acme@redhat.com>
#

import perf

root@number:/home/acme/git/linux#

Another

root@number:/home/acme/git/linux# tools/perf/python/tracepoint.py | head -50 | tail
time 302243176736731 prev_comm=chromium-browse prev_pid=633048 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/8 next_pid=0 next_prio=120
time 302243177970076 prev_comm=swapper/10 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=python next_pid=676159 next_prio=120
time 302243177952044 prev_comm=Compositor prev_pid=660727 prev_prio=120 prev_state=0x1 ==> next_comm=migration/17 next_pid=74 next_prio=0
time 302243176742113 prev_comm=swapper/8 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=chromium-browse next_pid=633048 next_prio=120
time 302243177991642 prev_comm=python prev_pid=676159 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/10 next_pid=0 next_prio=120
time 302243177962820 prev_comm=migration/17 prev_pid=74 prev_prio=0 prev_state=0x1 ==> next_comm=swapper/17 next_pid=0 next_prio=120
time 302243176784747 prev_comm=chromium-browse prev_pid=633048 prev_prio=120 prev_state=0x1 ==> next_comm=swapper/8 next_pid=0 next_prio=120
time 302243177993435 prev_comm=swapper/10 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=python next_pid=676159 next_prio=120
time 302243177966620 prev_comm=swapper/17 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=Compositor next_pid=660727 next_prio=120
time 302243176999285 prev_comm=swapper/8 prev_pid=0 prev_prio=120 prev_state=0x0 ==> next_comm=chromium-browse next_pid=633048 next_prio=120
root@number:/home/acme/git/linux#

By now I think there are other tools that use that python binding out in
the wild and it is packaged by distros as:

root@number:/home/acme/git/linux# dnf search python3-perf
============================================================================= Name Exactly Matched: python3-perf ==============================================================================
python3-perf.x86_64 : Python bindings for apps which will manipulate perf events
root@number:/home/acme/git/linux# 

Having these bindings opens the doors for projects to use these
libraries provided via the kernel sources, allowing other languages to
be used than C.

Now we have multiple libraries in the kernel sources, if we could have a
convenient way to provide those bindings for python, Go, etc, that could
help more projects while reducing the cost of supporting those bindings
by having more people capable of fixing up when needed as we would be
using a common way of providing such bindings.

Cheers,

- Arnaldo