[PATCH 0/6] Address some issues related to Python version

[PATCH 0/6] Address some issues related to Python version

[Mauro Carvalho Chehab]

This series remove compatibility with Python 2.x from scripts that have some
backward compatibility logic on it. The rationale is that, since 
commit 627395716cc3 ("docs: document python version used for compilation"),
the minimal Python version was set to 3.x. Also, Python 2.x is EOL since Jan, 2020.

Patch 1: fix a script that was compatible only with Python 2.x;
Patches 2-4: remove backward-compat code;
Patches 5-6 solves forward-compat with modern Python which warns about using
 raw strings without using "r" format.

Mauro Carvalho Chehab (6):
  docs: trace: decode_msr.py: make it compatible with python 3
  tools: perf: exported-sql-viewer: drop support for Python 2
  tools: perf: tools: perf: exported-sql-viewer: drop support for Python
    2
  tools: perf: task-analyzer: drop support for Python 2
  tools: selftests/bpf: test_bpftool_synctypes: escape raw symbols
  comedi: convert_csv_to_c.py: use r-string for a regex expression

 Documentation/trace/postprocess/decode_msr.py |  2 +-
 .../ni_routing/tools/convert_csv_to_c.py      |  2 +-
 .../scripts/python/exported-sql-viewer.py     |  5 ++--
 tools/perf/scripts/python/task-analyzer.py    | 23 ++++----------
 tools/perf/tests/shell/lib/attr.py            |  6 +---
 .../selftests/bpf/test_bpftool_synctypes.py   | 30 +++++++++----------
 6 files changed, 25 insertions(+), 43 deletions(-)

-- 
2.48.1

[PATCH 1/6] docs: trace: decode_msr.py: make it compatible with python 3

[Mauro Carvalho Chehab]

This script uses print <foo> instead of print(foo), which is
incompatible with Python 3.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/trace/postprocess/decode_msr.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/trace/postprocess/decode_msr.py b/Documentation/trace/postprocess/decode_msr.py
index aa9cc7abd5c2..f5609b16f589 100644
--- a/Documentation/trace/postprocess/decode_msr.py
+++ b/Documentation/trace/postprocess/decode_msr.py
@@ -32,6 +32,6 @@ for j in sys.stdin:
 					break
 		if r:
 			j = j.replace(" " + m.group(2), " " + r + "(" + m.group(2) + ")")
-	print j,
+	print(j)
 
 
-- 
2.48.1

[PATCH 2/6] tools: perf: exported-sql-viewer: drop support for Python 2

[Mauro Carvalho Chehab]

As stated at process/changes.rsy doc, the current minimal Python
version is 3.x, so drop support for EOL python 2.x.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/perf/tests/shell/lib/attr.py | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/tools/perf/tests/shell/lib/attr.py b/tools/perf/tests/shell/lib/attr.py
index 3db9a7d78715..3ca405f45d10 100644
--- a/tools/perf/tests/shell/lib/attr.py
+++ b/tools/perf/tests/shell/lib/attr.py
@@ -12,11 +12,7 @@ import logging
 import re
 import shutil
 import subprocess
-
-try:
-    import configparser
-except ImportError:
-    import ConfigParser as configparser
+import configparser
 
 def data_equal(a, b):
     # Allow multiple values in assignment separated by '|'
-- 
2.48.1

[PATCH 3/6] tools: perf: tools: perf: exported-sql-viewer: drop support for Python 2

[Mauro Carvalho Chehab]

As stated at process/changes.rsy doc, the current minimal Python
version is 3.x, so drop support for EOL python 2.x.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/perf/scripts/python/exported-sql-viewer.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/tools/perf/scripts/python/exported-sql-viewer.py b/tools/perf/scripts/python/exported-sql-viewer.py
index 121cf61ba1b3..b096592fd35f 100755
--- a/tools/perf/scripts/python/exported-sql-viewer.py
+++ b/tools/perf/scripts/python/exported-sql-viewer.py
@@ -3939,9 +3939,8 @@ def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
 	indent_str = " " * indent_sz
 
 	expanded_mark_sz = 2
-	if sys.version_info[0] == 3:
-		expanded_mark = "\u25BC "
-		not_expanded_mark = "\u25B6 "
+    expanded_mark = "\u25BC "
+    not_expanded_mark = "\u25B6 "
 	else:
 		expanded_mark = unicode(chr(0xE2) + chr(0x96) + chr(0xBC) + " ", "utf-8")
 		not_expanded_mark =  unicode(chr(0xE2) + chr(0x96) + chr(0xB6) + " ", "utf-8")
-- 
2.48.1

Re: [PATCH 3/6] tools: perf: tools: perf: exported-sql-viewer: drop support for Python 2

[Adrian Hunter]

On 29/01/25 19:39, Mauro Carvalho Chehab wrote:
> As stated at process/changes.rsy doc, the current minimal Python
> version is 3.x, so drop support for EOL python 2.x.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> ---
>  tools/perf/scripts/python/exported-sql-viewer.py | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/perf/scripts/python/exported-sql-viewer.py b/tools/perf/scripts/python/exported-sql-viewer.py
> index 121cf61ba1b3..b096592fd35f 100755
> --- a/tools/perf/scripts/python/exported-sql-viewer.py
> +++ b/tools/perf/scripts/python/exported-sql-viewer.py
> @@ -3939,9 +3939,8 @@ def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
>  	indent_str = " " * indent_sz
>  
>  	expanded_mark_sz = 2
> -	if sys.version_info[0] == 3:
> -		expanded_mark = "\u25BC "
> -		not_expanded_mark = "\u25B6 "
> +    expanded_mark = "\u25BC "
> +    not_expanded_mark = "\u25B6 "

You removed the 'if' but left the 'else'.

The white is messed up.

So presumably not tested at all:

  python3 tools/perf/scripts/python/exported-sql-viewer.py
    File "/home/user/git/review2/tools/perf/scripts/python/exported-sql-viewer.py", line 3942
      expanded_mark = "\u25BC "
                             ^
  IndentationError: unindent does not match any outer indentation level

Posting untested patches, especially tidy-ups,
should really be discouraged.

There are many other obvious python2 things in this
file that have not been addressed.  When asked, AI
listed 10 things.

>  	else:
>  		expanded_mark = unicode(chr(0xE2) + chr(0x96) + chr(0xBC) + " ", "utf-8")
>  		not_expanded_mark =  unicode(chr(0xE2) + chr(0x96) + chr(0xB6) + " ", "utf-8")

Re: [PATCH 3/6] tools: perf: tools: perf: exported-sql-viewer: drop support for Python 2

[Mauro Carvalho Chehab]

Em Thu, 30 Jan 2025 09:00:26 +0200
Adrian Hunter <adrian.hunter@intel.com> escreveu:

> On 29/01/25 19:39, Mauro Carvalho Chehab wrote:
> > As stated at process/changes.rsy doc, the current minimal Python
> > version is 3.x, so drop support for EOL python 2.x.
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
> > ---
> >  tools/perf/scripts/python/exported-sql-viewer.py | 5 ++---
> >  1 file changed, 2 insertions(+), 3 deletions(-)
> > 
> > diff --git a/tools/perf/scripts/python/exported-sql-viewer.py b/tools/perf/scripts/python/exported-sql-viewer.py
> > index 121cf61ba1b3..b096592fd35f 100755
> > --- a/tools/perf/scripts/python/exported-sql-viewer.py
> > +++ b/tools/perf/scripts/python/exported-sql-viewer.py
> > @@ -3939,9 +3939,8 @@ def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
> >  	indent_str = " " * indent_sz
> >  
> >  	expanded_mark_sz = 2
> > -	if sys.version_info[0] == 3:
> > -		expanded_mark = "\u25BC "
> > -		not_expanded_mark = "\u25B6 "
> > +    expanded_mark = "\u25BC "
> > +    not_expanded_mark = "\u25B6 "  
> 
> You removed the 'if' but left the 'else'.
> 
> The white is messed up.
> 
> So presumably not tested at all:
> 
>   python3 tools/perf/scripts/python/exported-sql-viewer.py
>     File "/home/user/git/review2/tools/perf/scripts/python/exported-sql-viewer.py", line 3942
>       expanded_mark = "\u25BC "
>                              ^
>   IndentationError: unindent does not match any outer indentation level
> 
> Posting untested patches, especially tidy-ups,
> should really be discouraged.
> 
> There are many other obvious python2 things in this
> file that have not been addressed.  When asked, AI
> listed 10 things.

You're right. Btw my main goal was just to make vermin to detect the
minimal version of this script.

I tried to run this script on Fedora 41, installing PySide with
pip there (*):

	$ pip install --user PySide2
	...
	$ perf record  ls
	...
	$ perf script -s tools/perf/scripts/python/export-to-sqlite.py pt_example branches calls
	Segmentation fault (core dumped)


but it gives me segmentation fault on a required library:

	Program terminated with signal SIGSEGV, Segmentation fault.
	#0  0x00007fcab2e0d1ec in SbkObject_TypeF () from ~/.local/lib/python3.13/site-packages/shiboken2/libshiboken2.abi3.so.5.13

(*) Fedora 41 doesn't have pyside2 packages anymore. 

Thanks,
Mauro

[PATCH 4/6] tools: perf: task-analyzer: drop support for Python 2

[Mauro Carvalho Chehab]

As stated at process/changes.rsy doc, the current minimal Python
version is 3.x, so drop support for EOL python 2.x.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/perf/scripts/python/task-analyzer.py | 23 +++++-----------------
 1 file changed, 5 insertions(+), 18 deletions(-)

diff --git a/tools/perf/scripts/python/task-analyzer.py b/tools/perf/scripts/python/task-analyzer.py
index 3f1df9894246..9dc18875630c 100755
--- a/tools/perf/scripts/python/task-analyzer.py
+++ b/tools/perf/scripts/python/task-analyzer.py
@@ -50,25 +50,12 @@ LEN_IN_IN = len("99999999999.999")  # 15
 LEN_IN_OUT = len("99999999999.999")  # 15
 
 
-# py2/py3 compatibility layer, see PEP469
-try:
-    dict.iteritems
-except AttributeError:
-    # py3
-    def itervalues(d):
-        return iter(d.values())
-
-    def iteritems(d):
-        return iter(d.items())
-
-else:
-    # py2
-    def itervalues(d):
-        return d.itervalues()
-
-    def iteritems(d):
-        return d.iteritems()
+# py3
+def itervalues(d):
+    return iter(d.values())
 
+def iteritems(d):
+    return iter(d.items())
 
 def _check_color():
     global _COLORS
-- 
2.48.1

[PATCH 5/6] tools: selftests/bpf: test_bpftool_synctypes: escape raw symbols

[Mauro Carvalho Chehab]

Modern Python versions complain about usage of "\" inside normal
strings, as they should use r-string notation.

Change the annotations there to avoid such warnings:

	tools/testing/selftests/bpf/test_bpftool_synctypes.py:319: SyntaxWarning: invalid escape sequence '\w' pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 .../selftests/bpf/test_bpftool_synctypes.py   | 30 +++++++++----------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_bpftool_synctypes.py b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
index 0ed67b6b31dd..81f286991012 100755
--- a/tools/testing/selftests/bpf/test_bpftool_synctypes.py
+++ b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
@@ -66,7 +66,7 @@ class ArrayParser(BlockParser):
 
     def __init__(self, reader, array_name):
         self.array_name = array_name
-        self.start_marker = re.compile(f'(static )?const bool {self.array_name}\[.*\] = {{\n')
+        self.start_marker = re.compile(fr'(static )?const bool {self.array_name}\[.*\] = {{\n')
         super().__init__(reader)
 
     def search_block(self):
@@ -80,7 +80,7 @@ class ArrayParser(BlockParser):
         Parse a block and return data as a dictionary. Items to extract must be
         on separate lines in the file.
         """
-        pattern = re.compile('\[(BPF_\w*)\]\s*= (true|false),?$')
+        pattern = re.compile(r'\[(BPF_\w*)\]\s*= (true|false),?$')
         entries = set()
         while True:
             line = self.reader.readline()
@@ -177,9 +177,9 @@ class FileExtractor(object):
 
         @enum_name: name of the enum to parse
         """
-        start_marker = re.compile(f'enum {enum_name} {{\n')
-        pattern = re.compile('^\s*(BPF_\w+),?(\s+/\*.*\*/)?$')
-        end_marker = re.compile('^};')
+        start_marker = re.compile(fr'enum {enum_name} {{\n')
+        pattern = re.compile(r'^\s*(BPF_\w+),?(\s+/\*.*\*/)?$')
+        end_marker = re.compile(r'^};')
         parser = BlockParser(self.reader)
         parser.search_block(start_marker)
         return parser.parse(pattern, end_marker)
@@ -226,8 +226,8 @@ class FileExtractor(object):
 
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
-        start_marker = re.compile(f'\*{block_name}\* := {{')
-        pattern = re.compile('\*\*([\w/-]+)\*\*')
+        start_marker = re.compile(fr'\*{block_name}\* := {{')
+        pattern = re.compile(r'\*\*([\w/-]+)\*\*')
         end_marker = re.compile('}\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -245,8 +245,8 @@ class FileExtractor(object):
 
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
-        start_marker = re.compile(f'"\s*{block_name} := {{')
-        pattern = re.compile('([\w/]+) [|}]')
+        start_marker = re.compile(fr'"\s*{block_name} := {{')
+        pattern = re.compile(r'([\w/]+) [|}]')
         end_marker = re.compile('}')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -264,8 +264,8 @@ class FileExtractor(object):
 
         @macro: macro starting the block, 'HELP_SPEC_OPTIONS' in the example
         """
-        start_marker = re.compile(f'"\s*{macro}\s*" [|}}]')
-        pattern = re.compile('([\w-]+) ?(?:\||}[ }\]])')
+        start_marker = re.compile(fr'"\s*{macro}\s*" [|}}]')
+        pattern = re.compile(r'([\w-]+) ?(?:\||}[ }\]])')
         end_marker = re.compile('}\\\\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -284,7 +284,7 @@ class FileExtractor(object):
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
         start_marker = re.compile(f'local {block_name}=\'')
-        pattern = re.compile('(?:.*=\')?([\w/]+)')
+        pattern = re.compile(r'(?:.*=\')?([\w/]+)')
         end_marker = re.compile('\'$')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -316,7 +316,7 @@ class MainHeaderFileExtractor(SourceFileExtractor):
             {'-p', '-d', '--pretty', '--debug', '--json', '-j'}
         """
         start_marker = re.compile(f'"OPTIONS :=')
-        pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')
+        pattern = re.compile(r'([\w-]+) ?(?:\||}[ }\]"])')
         end_marker = re.compile('#define')
 
         parser = InlineListParser(self.reader)
@@ -338,8 +338,8 @@ class ManSubstitutionsExtractor(SourceFileExtractor):
 
             {'-p', '-d', '--pretty', '--debug', '--json', '-j'}
         """
-        start_marker = re.compile('\|COMMON_OPTIONS\| replace:: {')
-        pattern = re.compile('\*\*([\w/-]+)\*\*')
+        start_marker = re.compile(r'\|COMMON_OPTIONS\| replace:: {')
+        pattern = re.compile(r'\*\*([\w/-]+)\*\*')
         end_marker = re.compile('}$')
 
         parser = InlineListParser(self.reader)
-- 
2.48.1

Re: [PATCH 5/6] tools: selftests/bpf: test_bpftool_synctypes: escape raw symbols

[Quentin Monnet]

2025-01-29 18:39 UTC+0100 ~ Mauro Carvalho Chehab
<mchehab+huawei@kernel.org>
> Modern Python versions complain about usage of "\" inside normal
> strings, as they should use r-string notation.
> 
> Change the annotations there to avoid such warnings:
> 
> 	tools/testing/selftests/bpf/test_bpftool_synctypes.py:319: SyntaxWarning: invalid escape sequence '\w' pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

Hi, and thanks! But please note we have a fix for this in the bpf-next
tree already:

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=c5d2bac978c513e1f22273cba9c55db3778032e5

Thanks,
Quentin

[PATCH 6/6] comedi: convert_csv_to_c.py: use r-string for a regex expression

[Mauro Carvalho Chehab]

Modern Python versions complain about usage of "\" inside normal
strings, as they should use r-string notation.

Change the annotations there to avoid such warnings:

	drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py:47: SyntaxWarning: invalid escape sequence '\('

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py b/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
index 90378fb50580..d19101fc2a94 100755
--- a/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
+++ b/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
@@ -44,7 +44,7 @@ def routedict_to_structinit_single(name, D, return_name=False):
 
     lines.append('\t\t[B({})] = {{'.format(D0_sig))
     for D1_sig, value in D1:
-      if not re.match('[VIU]\([^)]*\)', value):
+      if not re.match(r'[VIU]\([^)]*\)', value):
         sys.stderr.write('Invalid register format: {}\n'.format(repr(value)))
         sys.stderr.write(
           'Register values should be formatted with V(),I(),or U()\n')
-- 
2.48.1