[PATCH 0/8] Fix Python string escapes

[PATCH 0/8] Fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12 (3.12.0rc1), and is
intended to eventually be a syntax error.

This series aims to fix these now to get ahead of it before it's an error.

Most of the patches are generated using the below Python script. It
uses the builtin ast module to parse a Python file, locate all strings,
find the corresponding source code, and check for bad escape sequences.

If it finds things to fix then it applies the fixes and reparses the
file into a fixed AST. It finally compares the original and fixed ASTs
to ensure no semantic difference has been introduced (dumping is done to
remove node location information, which is expected to be different).

There are some limitations of the ast module, in particular it throws
away a lot of information about the string source. f-strings especially
interact poorly here (the slices between formats are presented as
separate strings, but the node range of each is the entire f-string),
so are skipped. f-strings are handled manually in the final patch.

A lot of the fixes are for regex patterns, which could be changed to use
r-strings instead. But that is less easy to automate, so I avoided doing
so in this series. AST verification should still be possible though,
because being a plain or r-string is stripped away in the AST.

---
#!/usr/bin/env python3

"""
Fix all bad escape characters in strings
"""

import ast
from pathlib import Path

def get_offset(source: str, row: int, col: int) -> int:
    """
    Turn a row + column pair into a byte offset
    """
    offset = 0

    cur_row = 1  # 1-indexed rows
    cur_col = 0  # 0-indexed columns

    for c in source:
        if cur_row == row and cur_col == col:
            return offset

        offset += 1

        if c == "\n":
            cur_row += 1
            cur_col = 0
        else:
            cur_col += 1

    raise Exception("Failed to get offset")


parse_failures: list[Path] = []
fix_failures = 0
bad_escapes = 0
fstrings: set[Path] = set()

for pyfile in Path(".").glob("**/*.py"):
    content = pyfile.read_text("utf-8")

    try:
        syntax = ast.parse(content, filename=pyfile)
    except:
        print(f"{pyfile}: ERROR Failed to parse, is it Python3?")
        parse_failures.append(pyfile)
        continue

    fixes: list[tuple[int, int, str]] = []

    for node in ast.walk(syntax):
        if not isinstance(node, ast.Constant):
            continue

        if not isinstance(node.value, str):
            continue

        if node.value.count("\\") == 0:
            continue

        assert(isinstance(node.end_lineno, int))
        assert(isinstance(node.end_col_offset, int))

        start = get_offset(content, node.lineno, node.col_offset)
        end = get_offset(content, node.end_lineno, node.end_col_offset)
        raw = content[start:end]

        # backslashes in r-strings are already literal
        if raw.startswith("r"):
            continue

        # f-strings are difficult to handle correctly
        if raw.startswith("f"):
            fstrings.add(pyfile)
            continue

        fixed = ""  # The fixed representation of the string
        escaped = False  # If the current character is escaped by a previous backslash
        allowed = '\n\\\'"abfnrtv01234567xNuU'  # characters allowed after a backslash

        for c in raw:
            if escaped:
                if c not in allowed:
                        fixed += '\\'

                fixed += c
                escaped = False
                continue

            fixed += c

            if c == '\\':
                escaped = True

        if fixed != raw:
            print(f"{pyfile}:{node.lineno}:{node.col_offset}: FOUND {raw}")
            fixes.append((start, end, fixed))

    if len(fixes) == 0:
        continue

    bad_escapes += len(fixes)

    # Apply fixes in reverse order to keep offsets valid
    for start, end, fix in reversed(sorted(fixes, key=lambda k: k[0])):
        print(f"{pyfile}:[{start}-{end}]: APPLY {fix}")
        content = content[:start] + fix + content[end:]

    fixed_syntax = ast.parse(content, filename=f"{pyfile}+fixed")

    if ast.dump(syntax) != ast.dump(fixed_syntax):
        print(f"{pyfile}: ERROR Fixed syntax tree yields different AST")
        fix_failures += 1
        continue

    pyfile.write_text(content)


print(f"---------------------------------")
print(f"Parse failures:               {len(parse_failures)}")
for f in sorted(parse_failures):
    print(f"  - {f}")

print(f"Bad escapes fixed:            {bad_escapes}")
print(f"Fixes that broke the AST:     {fix_failures}")
print(f"Files with skipped f-strings: {len(fstrings)}")
for f in sorted(fstrings):
    print(f"  - {f}")

---

Benjamin Gray (8):
  ia64: fix Python string escapes
  Documentation/sphinx: fix Python string escapes
  drivers/comedi: fix Python string escapes
  scripts: fix Python string escapes
  tools/perf: fix Python string escapes
  tools/power: fix Python string escapes
  selftests/bpf: fix Python string escapes
  selftests/bpf: fix Python string escapes in f-strings

 Documentation/sphinx/cdomain.py               |  2 +-
 Documentation/sphinx/kernel_abi.py            |  2 +-
 Documentation/sphinx/kernel_feat.py           |  2 +-
 Documentation/sphinx/kerneldoc.py             |  2 +-
 Documentation/sphinx/maintainers_include.py   |  8 +--
 arch/ia64/scripts/unwcheck.py                 |  2 +-
 .../ni_routing/tools/convert_csv_to_c.py      |  2 +-
 scripts/bpf_doc.py                            | 56 +++++++++----------
 scripts/clang-tools/gen_compile_commands.py   |  2 +-
 scripts/gdb/linux/symbols.py                  |  2 +-
 tools/perf/pmu-events/jevents.py              |  2 +-
 .../scripts/python/arm-cs-trace-disasm.py     |  4 +-
 tools/perf/scripts/python/compaction-times.py |  2 +-
 .../scripts/python/exported-sql-viewer.py     |  4 +-
 tools/power/pm-graph/bootgraph.py             | 12 ++--
 .../selftests/bpf/test_bpftool_synctypes.py   | 26 ++++-----
 tools/testing/selftests/bpf/test_offload.py   |  2 +-
 17 files changed, 66 insertions(+), 66 deletions(-)

--
2.41.0

[PATCH 1/8] ia64: fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 arch/ia64/scripts/unwcheck.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/ia64/scripts/unwcheck.py b/arch/ia64/scripts/unwcheck.py
index 9581742f0db2..f66f4015891b 100644
--- a/arch/ia64/scripts/unwcheck.py
+++ b/arch/ia64/scripts/unwcheck.py
@@ -21,7 +21,7 @@ if len(sys.argv) != 2:
 
 readelf = os.getenv("READELF", "readelf")
 
-start_pattern = re.compile("<([^>]*)>: \[0x([0-9a-f]+)-0x([0-9a-f]+)\]")
+start_pattern = re.compile("<([^>]*)>: \\[0x([0-9a-f]+)-0x([0-9a-f]+)\\]")
 rlen_pattern  = re.compile(".*rlen=([0-9]+)")
 
 def check_func (func, slots, rlen_sum):
-- 
2.41.0

[PATCH 2/8] Documentation/sphinx: fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 Documentation/sphinx/cdomain.py             | 2 +-
 Documentation/sphinx/kernel_abi.py          | 2 +-
 Documentation/sphinx/kernel_feat.py         | 2 +-
 Documentation/sphinx/kerneldoc.py           | 2 +-
 Documentation/sphinx/maintainers_include.py | 8 ++++----
 5 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
index ca8ac9e59ded..dbdc74bd0772 100644
--- a/Documentation/sphinx/cdomain.py
+++ b/Documentation/sphinx/cdomain.py
@@ -93,7 +93,7 @@ def markup_ctype_refs(match):
 #
 RE_expr = re.compile(r':c:(expr|texpr):`([^\`]+)`')
 def markup_c_expr(match):
-    return '\ ``' + match.group(2) + '``\ '
+    return '\\ ``' + match.group(2) + '``\\ '
 
 #
 # Parse Sphinx 3.x C markups, replacing them by backward-compatible ones
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index b5feb5b1d905..b9f026f016fd 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -138,7 +138,7 @@ class KernelCmd(Directive):
                 code_block += "\n    " + l
             lines = code_block + "\n\n"
 
-        line_regex = re.compile("^\.\. LINENO (\S+)\#([0-9]+)$")
+        line_regex = re.compile("^\\.\\. LINENO (\\S+)\\#([0-9]+)$")
         ln = 0
         n = 0
         f = fname
diff --git a/Documentation/sphinx/kernel_feat.py b/Documentation/sphinx/kernel_feat.py
index 27b701ed3681..d17adc1a367a 100644
--- a/Documentation/sphinx/kernel_feat.py
+++ b/Documentation/sphinx/kernel_feat.py
@@ -104,7 +104,7 @@ class KernelFeat(Directive):
 
         lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
 
-        line_regex = re.compile("^\.\. FILE (\S+)$")
+        line_regex = re.compile("^\\.\\. FILE (\\S+)$")
 
         out_lines = ""
 
diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index 9395892c7ba3..d6ec34ce2232 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -130,7 +130,7 @@ class KernelDocDirective(Directive):
             result = ViewList()
 
             lineoffset = 0;
-            line_regex = re.compile("^\.\. LINENO ([0-9]+)$")
+            line_regex = re.compile("^\\.\\. LINENO ([0-9]+)$")
             for line in lines:
                 match = line_regex.search(line)
                 if match:
diff --git a/Documentation/sphinx/maintainers_include.py b/Documentation/sphinx/maintainers_include.py
index 328b3631a585..73be47963153 100755
--- a/Documentation/sphinx/maintainers_include.py
+++ b/Documentation/sphinx/maintainers_include.py
@@ -77,7 +77,7 @@ class MaintainersInclude(Include):
             line = line.rstrip()
 
             # Linkify all non-wildcard refs to ReST files in Documentation/.
-            pat = '(Documentation/([^\s\?\*]*)\.rst)'
+            pat = '(Documentation/([^\\s\\?\\*]*)\\.rst)'
             m = re.search(pat, line)
             if m:
                 # maintainers.rst is in a subdirectory, so include "../".
@@ -90,11 +90,11 @@ class MaintainersInclude(Include):
                 output = "| %s" % (line.replace("\\", "\\\\"))
                 # Look for and record field letter to field name mappings:
                 #   R: Designated *reviewer*: FullName <address@domain>
-                m = re.search("\s(\S):\s", line)
+                m = re.search("\\s(\\S):\\s", line)
                 if m:
                     field_letter = m.group(1)
                 if field_letter and not field_letter in fields:
-                    m = re.search("\*([^\*]+)\*", line)
+                    m = re.search("\\*([^\\*]+)\\*", line)
                     if m:
                         fields[field_letter] = m.group(1)
             elif subsystems:
@@ -112,7 +112,7 @@ class MaintainersInclude(Include):
                     field_content = ""
 
                     # Collapse whitespace in subsystem name.
-                    heading = re.sub("\s+", " ", line)
+                    heading = re.sub("\\s+", " ", line)
                     output = output + "%s\n%s" % (heading, "~" * len(heading))
                     field_prev = ""
                 else:
-- 
2.41.0

[PATCH 3/8] drivers/comedi: fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py b/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
index 90378fb50580..3d124a887be9 100755
--- a/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
+++ b/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
@@ -44,7 +44,7 @@ def routedict_to_structinit_single(name, D, return_name=False):
 
     lines.append('\t\t[B({})] = {{'.format(D0_sig))
     for D1_sig, value in D1:
-      if not re.match('[VIU]\([^)]*\)', value):
+      if not re.match('[VIU]\\([^)]*\\)', value):
         sys.stderr.write('Invalid register format: {}\n'.format(repr(value)))
         sys.stderr.write(
           'Register values should be formatted with V(),I(),or U()\n')
-- 
2.41.0

[PATCH 4/8] scripts: fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 scripts/bpf_doc.py                          | 56 ++++++++++-----------
 scripts/clang-tools/gen_compile_commands.py |  2 +-
 scripts/gdb/linux/symbols.py                |  2 +-
 3 files changed, 30 insertions(+), 30 deletions(-)

diff --git a/scripts/bpf_doc.py b/scripts/bpf_doc.py
index eaae2ce78381..40a2643f94d2 100755
--- a/scripts/bpf_doc.py
+++ b/scripts/bpf_doc.py
@@ -59,9 +59,9 @@ class Helper(APIElement):
         Break down helper function protocol into smaller chunks: return type,
         name, distincts arguments.
         """
-        arg_re = re.compile('((\w+ )*?(\w+|...))( (\**)(\w+))?$')
+        arg_re = re.compile('((\\w+ )*?(\\w+|...))( (\\**)(\\w+))?$')
         res = {}
-        proto_re = re.compile('(.+) (\**)(\w+)\(((([^,]+)(, )?){1,5})\)$')
+        proto_re = re.compile('(.+) (\\**)(\\w+)\\(((([^,]+)(, )?){1,5})\\)$')
 
         capture = proto_re.match(self.proto)
         res['ret_type'] = capture.group(1)
@@ -114,11 +114,11 @@ class HeaderParser(object):
         return Helper(proto=proto, desc=desc, ret=ret)
 
     def parse_symbol(self):
-        p = re.compile(' \* ?(BPF\w+)$')
+        p = re.compile(' \\* ?(BPF\\w+)$')
         capture = p.match(self.line)
         if not capture:
             raise NoSyscallCommandFound
-        end_re = re.compile(' \* ?NOTES$')
+        end_re = re.compile(' \\* ?NOTES$')
         end = end_re.match(self.line)
         if end:
             raise NoSyscallCommandFound
@@ -133,7 +133,7 @@ class HeaderParser(object):
         #   - Same as above, with "const" and/or "struct" in front of type
         #   - "..." (undefined number of arguments, for bpf_trace_printk())
         # There is at least one term ("void"), and at most five arguments.
-        p = re.compile(' \* ?((.+) \**\w+\((((const )?(struct )?(\w+|\.\.\.)( \**\w+)?)(, )?){1,5}\))$')
+        p = re.compile(' \\* ?((.+) \\**\\w+\\((((const )?(struct )?(\\w+|\\.\\.\\.)( \\**\\w+)?)(, )?){1,5}\\))$')
         capture = p.match(self.line)
         if not capture:
             raise NoHelperFound
@@ -141,7 +141,7 @@ class HeaderParser(object):
         return capture.group(1)
 
     def parse_desc(self, proto):
-        p = re.compile(' \* ?(?:\t| {5,8})Description$')
+        p = re.compile(' \\* ?(?:\t| {5,8})Description$')
         capture = p.match(self.line)
         if not capture:
             raise Exception("No description section found for " + proto)
@@ -154,7 +154,7 @@ class HeaderParser(object):
             if self.line == ' *\n':
                 desc += '\n'
             else:
-                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
+                p = re.compile(' \\* ?(?:\t| {5,8})(?:\t| {8})(.*)')
                 capture = p.match(self.line)
                 if capture:
                     desc_present = True
@@ -167,7 +167,7 @@ class HeaderParser(object):
         return desc
 
     def parse_ret(self, proto):
-        p = re.compile(' \* ?(?:\t| {5,8})Return$')
+        p = re.compile(' \\* ?(?:\t| {5,8})Return$')
         capture = p.match(self.line)
         if not capture:
             raise Exception("No return section found for " + proto)
@@ -180,7 +180,7 @@ class HeaderParser(object):
             if self.line == ' *\n':
                 ret += '\n'
             else:
-                p = re.compile(' \* ?(?:\t| {5,8})(?:\t| {8})(.*)')
+                p = re.compile(' \\* ?(?:\t| {5,8})(?:\t| {8})(.*)')
                 capture = p.match(self.line)
                 if capture:
                     ret_present = True
@@ -219,12 +219,12 @@ class HeaderParser(object):
         self.seek_to('enum bpf_cmd {',
                      'Could not find start of bpf_cmd enum', 0)
         # Searches for either one or more BPF\w+ enums
-        bpf_p = re.compile('\s*(BPF\w+)+')
+        bpf_p = re.compile('\\s*(BPF\\w+)+')
         # Searches for an enum entry assigned to another entry,
         # for e.g. BPF_PROG_RUN = BPF_PROG_TEST_RUN, which is
         # not documented hence should be skipped in check to
         # determine if the right number of syscalls are documented
-        assign_p = re.compile('\s*(BPF\w+)\s*=\s*(BPF\w+)')
+        assign_p = re.compile('\\s*(BPF\\w+)\\s*=\\s*(BPF\\w+)')
         bpf_cmd_str = ''
         while True:
             capture = assign_p.match(self.line)
@@ -239,7 +239,7 @@ class HeaderParser(object):
                 break
             self.line = self.reader.readline()
         # Find the number of occurences of BPF\w+
-        self.enum_syscalls = re.findall('(BPF\w+)+', bpf_cmd_str)
+        self.enum_syscalls = re.findall('(BPF\\w+)+', bpf_cmd_str)
 
     def parse_desc_helpers(self):
         self.seek_to(helpersDocStart,
@@ -263,7 +263,7 @@ class HeaderParser(object):
         self.seek_to('#define ___BPF_FUNC_MAPPER(FN, ctx...)',
                      'Could not find start of eBPF helper definition list')
         # Searches for one FN(\w+) define or a backslash for newline
-        p = re.compile('\s*FN\((\w+), (\d+), ##ctx\)|\\\\')
+        p = re.compile('\\s*FN\\((\\w+), (\\d+), ##ctx\\)|\\\\')
         fn_defines_str = ''
         i = 0
         while True:
@@ -278,7 +278,7 @@ class HeaderParser(object):
                 break
             self.line = self.reader.readline()
         # Find the number of occurences of FN(\w+)
-        self.define_unique_helpers = re.findall('FN\(\w+, \d+, ##ctx\)', fn_defines_str)
+        self.define_unique_helpers = re.findall('FN\\(\\w+, \\d+, ##ctx\\)', fn_defines_str)
 
     def validate_helpers(self):
         last_helper = ''
@@ -425,7 +425,7 @@ class PrinterRST(Printer):
         try:
             cmd = ['git', 'log', '-1', '--pretty=format:%cs', '--no-patch',
                    '-L',
-                   '/{}/,/\*\//:include/uapi/linux/bpf.h'.format(delimiter)]
+                   '/{}/,/\\*\\//:include/uapi/linux/bpf.h'.format(delimiter)]
             date = subprocess.run(cmd, cwd=linuxRoot,
                                   capture_output=True, check=True)
             return date.stdout.decode().rstrip()
@@ -516,7 +516,7 @@ as "Dual BSD/GPL", may be used). Some helper functions are only accessible to
 programs that are compatible with the GNU Privacy License (GPL).
 
 In order to use such helpers, the eBPF program must be loaded with the correct
-license string passed (via **attr**) to the **bpf**\ () system call, and this
+license string passed (via **attr**) to the **bpf**\\ () system call, and this
 generally translates into the C source code of the program containing a line
 similar to the following:
 
@@ -550,7 +550,7 @@ may be interested in:
 * The bpftool utility can be used to probe the availability of helper functions
   on the system (as well as supported program and map types, and a number of
   other parameters). To do so, run **bpftool feature probe** (see
-  **bpftool-feature**\ (8) for details). Add the **unprivileged** keyword to
+  **bpftool-feature**\\ (8) for details). Add the **unprivileged** keyword to
   list features available to unprivileged users.
 
 Compatibility between helper functions and program types can generally be found
@@ -562,23 +562,23 @@ other functions, themselves allowing access to additional helpers. The
 requirement for GPL license is also in those **struct bpf_func_proto**.
 
 Compatibility between helper functions and map types can be found in the
-**check_map_func_compatibility**\ () function in file *kernel/bpf/verifier.c*.
+**check_map_func_compatibility**\\ () function in file *kernel/bpf/verifier.c*.
 
 Helper functions that invalidate the checks on **data** and **data_end**
 pointers for network processing are listed in function
-**bpf_helper_changes_pkt_data**\ () in file *net/core/filter.c*.
+**bpf_helper_changes_pkt_data**\\ () in file *net/core/filter.c*.
 
 SEE ALSO
 ========
 
-**bpf**\ (2),
-**bpftool**\ (8),
-**cgroups**\ (7),
-**ip**\ (8),
-**perf_event_open**\ (2),
-**sendmsg**\ (2),
-**socket**\ (7),
-**tc-bpf**\ (8)'''
+**bpf**\\ (2),
+**bpftool**\\ (8),
+**cgroups**\\ (7),
+**ip**\\ (8),
+**perf_event_open**\\ (2),
+**sendmsg**\\ (2),
+**socket**\\ (7),
+**tc-bpf**\\ (8)'''
         print(footer)
 
     def print_proto(self, helper):
@@ -598,7 +598,7 @@ SEE ALSO
             one_arg = '{}{}'.format(comma, a['type'])
             if a['name']:
                 if a['star']:
-                    one_arg += ' {}**\ '.format(a['star'].replace('*', '\\*'))
+                    one_arg += ' {}**\\ '.format(a['star'].replace('*', '\\*'))
                 else:
                     one_arg += '** '
                 one_arg += '*{}*\\ **'.format(a['name'])
diff --git a/scripts/clang-tools/gen_compile_commands.py b/scripts/clang-tools/gen_compile_commands.py
index a84cc5737c2c..649a80005c83 100755
--- a/scripts/clang-tools/gen_compile_commands.py
+++ b/scripts/clang-tools/gen_compile_commands.py
@@ -170,7 +170,7 @@ def process_line(root_directory, command_prefix, file_path):
     # escape the pound sign '#', either as '\#' or '$(pound)' (depending on the
     # kernel version). The compile_commands.json file is not interepreted
     # by Make, so this code replaces the escaped version with '#'.
-    prefix = command_prefix.replace('\#', '#').replace('$(pound)', '#')
+    prefix = command_prefix.replace('\\#', '#').replace('$(pound)', '#')
 
     # Use os.path.abspath() to normalize the path resolving '.' and '..' .
     abs_path = os.path.abspath(os.path.join(root_directory, file_path))
diff --git a/scripts/gdb/linux/symbols.py b/scripts/gdb/linux/symbols.py
index fdad3f32c747..bae7cbff187e 100644
--- a/scripts/gdb/linux/symbols.py
+++ b/scripts/gdb/linux/symbols.py
@@ -82,7 +82,7 @@ lx-symbols command."""
         self.module_files_updated = True
 
     def _get_module_file(self, module_name):
-        module_pattern = ".*/{0}\.ko(?:.debug)?$".format(
+        module_pattern = ".*/{0}\\.ko(?:.debug)?$".format(
             module_name.replace("_", r"[_\-]"))
         for name in self.module_files:
             if re.match(module_pattern, name) and os.path.exists(name):
-- 
2.41.0

[PATCH 5/8] tools/perf: fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 tools/perf/pmu-events/jevents.py                 | 2 +-
 tools/perf/scripts/python/arm-cs-trace-disasm.py | 4 ++--
 tools/perf/scripts/python/compaction-times.py    | 2 +-
 tools/perf/scripts/python/exported-sql-viewer.py | 4 ++--
 4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py
index 12e80bb7939b..87964bf32b62 100755
--- a/tools/perf/pmu-events/jevents.py
+++ b/tools/perf/pmu-events/jevents.py
@@ -83,7 +83,7 @@ def c_len(s: str) -> int:
   """Return the length of s a C string
 
   This doesn't handle all escape characters properly. It first assumes
-  all \ are for escaping, it then adjusts as it will have over counted
+  all \\ are for escaping, it then adjusts as it will have over counted
   \\. The code uses \000 rather than \0 as a terminator as an adjacent
   number would be folded into a string of \0 (ie. "\0" + "5" doesn't
   equal a terminator followed by the number 5 but the escape of
diff --git a/tools/perf/scripts/python/arm-cs-trace-disasm.py b/tools/perf/scripts/python/arm-cs-trace-disasm.py
index d59ff53f1d94..cbd866acef2f 100755
--- a/tools/perf/scripts/python/arm-cs-trace-disasm.py
+++ b/tools/perf/scripts/python/arm-cs-trace-disasm.py
@@ -45,8 +45,8 @@ parser = OptionParser(option_list=option_list)
 # Initialize global dicts and regular expression
 disasm_cache = dict()
 cpu_data = dict()
-disasm_re = re.compile("^\s*([0-9a-fA-F]+):")
-disasm_func_re = re.compile("^\s*([0-9a-fA-F]+)\s.*:")
+disasm_re = re.compile("^\\s*([0-9a-fA-F]+):")
+disasm_func_re = re.compile("^\\s*([0-9a-fA-F]+)\\s.*:")
 cache_size = 64*1024
 
 glb_source_file_name	= None
diff --git a/tools/perf/scripts/python/compaction-times.py b/tools/perf/scripts/python/compaction-times.py
index 2560a042dc6f..b75d90ffb194 100644
--- a/tools/perf/scripts/python/compaction-times.py
+++ b/tools/perf/scripts/python/compaction-times.py
@@ -260,7 +260,7 @@ def pr_help():
 
 comm_re = None
 pid_re = None
-pid_regex = "^(\d*)-(\d*)$|^(\d*)$"
+pid_regex = "^(\\d*)-(\\d*)$|^(\\d*)$"
 
 opt_proc = popt.DISP_DFL
 opt_disp = topt.DISP_ALL
diff --git a/tools/perf/scripts/python/exported-sql-viewer.py b/tools/perf/scripts/python/exported-sql-viewer.py
index 13f2d8a81610..121cf61ba1b3 100755
--- a/tools/perf/scripts/python/exported-sql-viewer.py
+++ b/tools/perf/scripts/python/exported-sql-viewer.py
@@ -677,8 +677,8 @@ class CallGraphModelBase(TreeModel):
 			#   sqlite supports GLOB (text only) which uses * and ? and is case sensitive
 			if not self.glb.dbref.is_sqlite3:
 				# Escape % and _
-				s = value.replace("%", "\%")
-				s = s.replace("_", "\_")
+				s = value.replace("%", "\\%")
+				s = s.replace("_", "\\_")
 				# Translate * and ? into SQL LIKE pattern characters % and _
 				trans = string.maketrans("*?", "%_")
 				match = " LIKE '" + str(s).translate(trans) + "'"
-- 
2.41.0

[PATCH 6/8] tools/power: fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 tools/power/pm-graph/bootgraph.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/power/pm-graph/bootgraph.py b/tools/power/pm-graph/bootgraph.py
index f96f50e0c336..e983c4e71d77 100755
--- a/tools/power/pm-graph/bootgraph.py
+++ b/tools/power/pm-graph/bootgraph.py
@@ -77,12 +77,12 @@ class SystemValues(aslib.SystemValues):
 			fp.close()
 		self.testdir = datetime.now().strftime('boot-%y%m%d-%H%M%S')
 	def kernelVersion(self, msg):
-		m = re.match('^[Ll]inux *[Vv]ersion *(?P<v>\S*) .*', msg)
+		m = re.match('^[Ll]inux *[Vv]ersion *(?P<v>\\S*) .*', msg)
 		if m:
 			return m.group('v')
 		return 'unknown'
 	def checkFtraceKernelVersion(self):
-		m = re.match('^(?P<x>[0-9]*)\.(?P<y>[0-9]*)\.(?P<z>[0-9]*).*', self.kernel)
+		m = re.match('^(?P<x>[0-9]*)\\.(?P<y>[0-9]*)\\.(?P<z>[0-9]*).*', self.kernel)
 		if m:
 			val = tuple(map(int, m.groups()))
 			if val >= (4, 10, 0):
@@ -324,7 +324,7 @@ def parseKernelLog():
 		idx = line.find('[')
 		if idx > 1:
 			line = line[idx:]
-		m = re.match('[ \t]*(\[ *)(?P<ktime>[0-9\.]*)(\]) (?P<msg>.*)', line)
+		m = re.match('[ \t]*(\\[ *)(?P<ktime>[0-9\\.]*)(\\]) (?P<msg>.*)', line)
 		if(not m):
 			continue
 		ktime = float(m.group('ktime'))
@@ -336,20 +336,20 @@ def parseKernelLog():
 			if(not sysvals.stamp['kernel']):
 				sysvals.stamp['kernel'] = sysvals.kernelVersion(msg)
 			continue
-		m = re.match('.* setting system clock to (?P<d>[0-9\-]*)[ A-Z](?P<t>[0-9:]*) UTC.*', msg)
+		m = re.match('.* setting system clock to (?P<d>[0-9\\-]*)[ A-Z](?P<t>[0-9:]*) UTC.*', msg)
 		if(m):
 			bt = datetime.strptime(m.group('d')+' '+m.group('t'), '%Y-%m-%d %H:%M:%S')
 			bt = bt - timedelta(seconds=int(ktime))
 			data.boottime = bt.strftime('%Y-%m-%d_%H:%M:%S')
 			sysvals.stamp['time'] = bt.strftime('%B %d %Y, %I:%M:%S %p')
 			continue
-		m = re.match('^calling *(?P<f>.*)\+.* @ (?P<p>[0-9]*)', msg)
+		m = re.match('^calling *(?P<f>.*)\\+.* @ (?P<p>[0-9]*)', msg)
 		if(m):
 			func = m.group('f')
 			pid = int(m.group('p'))
 			devtemp[func] = (ktime, pid)
 			continue
-		m = re.match('^initcall *(?P<f>.*)\+.* returned (?P<r>.*) after (?P<t>.*) usecs', msg)
+		m = re.match('^initcall *(?P<f>.*)\\+.* returned (?P<r>.*) after (?P<t>.*) usecs', msg)
 		if(m):
 			data.valid = True
 			data.end = ktime
-- 
2.41.0

[PATCH 7/8] selftests/bpf: fix Python string escapes

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 .../selftests/bpf/test_bpftool_synctypes.py    | 18 +++++++++---------
 tools/testing/selftests/bpf/test_offload.py    |  2 +-
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_bpftool_synctypes.py b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
index 0cfece7ff4f8..b21bc1a35bf4 100755
--- a/tools/testing/selftests/bpf/test_bpftool_synctypes.py
+++ b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
@@ -80,7 +80,7 @@ class ArrayParser(BlockParser):
         Parse a block and return data as a dictionary. Items to extract must be
         on separate lines in the file.
         """
-        pattern = re.compile('\[(BPF_\w*)\]\s*= (true|false),?$')
+        pattern = re.compile('\\[(BPF_\\w*)\\]\\s*= (true|false),?$')
         entries = set()
         while True:
             line = self.reader.readline()
@@ -178,7 +178,7 @@ class FileExtractor(object):
         @enum_name: name of the enum to parse
         """
         start_marker = re.compile(f'enum {enum_name} {{\n')
-        pattern = re.compile('^\s*(BPF_\w+),?(\s+/\*.*\*/)?$')
+        pattern = re.compile('^\\s*(BPF_\\w+),?(\\s+/\\*.*\\*/)?$')
         end_marker = re.compile('^};')
         parser = BlockParser(self.reader)
         parser.search_block(start_marker)
@@ -227,7 +227,7 @@ class FileExtractor(object):
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
         start_marker = re.compile(f'\*{block_name}\* := {{')
-        pattern = re.compile('\*\*([\w/-]+)\*\*')
+        pattern = re.compile('\\*\\*([\\w/-]+)\\*\\*')
         end_marker = re.compile('}\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -246,7 +246,7 @@ class FileExtractor(object):
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
         start_marker = re.compile(f'"\s*{block_name} := {{')
-        pattern = re.compile('([\w/]+) [|}]')
+        pattern = re.compile('([\\w/]+) [|}]')
         end_marker = re.compile('}')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -265,7 +265,7 @@ class FileExtractor(object):
         @macro: macro starting the block, 'HELP_SPEC_OPTIONS' in the example
         """
         start_marker = re.compile(f'"\s*{macro}\s*" [|}}]')
-        pattern = re.compile('([\w-]+) ?(?:\||}[ }\]])')
+        pattern = re.compile('([\\w-]+) ?(?:\\||}[ }\\]])')
         end_marker = re.compile('}\\\\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -284,7 +284,7 @@ class FileExtractor(object):
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
         start_marker = re.compile(f'local {block_name}=\'')
-        pattern = re.compile('(?:.*=\')?([\w/]+)')
+        pattern = re.compile('(?:.*=\')?([\\w/]+)')
         end_marker = re.compile('\'$')
         return self.__get_description_list(start_marker, pattern, end_marker)
 
@@ -316,7 +316,7 @@ class MainHeaderFileExtractor(SourceFileExtractor):
             {'-p', '-d', '--pretty', '--debug', '--json', '-j'}
         """
         start_marker = re.compile(f'"OPTIONS :=')
-        pattern = re.compile('([\w-]+) ?(?:\||}[ }\]"])')
+        pattern = re.compile('([\\w-]+) ?(?:\\||}[ }\\]"])')
         end_marker = re.compile('#define')
 
         parser = InlineListParser(self.reader)
@@ -338,8 +338,8 @@ class ManSubstitutionsExtractor(SourceFileExtractor):
 
             {'-p', '-d', '--pretty', '--debug', '--json', '-j'}
         """
-        start_marker = re.compile('\|COMMON_OPTIONS\| replace:: {')
-        pattern = re.compile('\*\*([\w/-]+)\*\*')
+        start_marker = re.compile('\\|COMMON_OPTIONS\\| replace:: {')
+        pattern = re.compile('\\*\\*([\\w/-]+)\\*\\*')
         end_marker = re.compile('}$')
 
         parser = InlineListParser(self.reader)
diff --git a/tools/testing/selftests/bpf/test_offload.py b/tools/testing/selftests/bpf/test_offload.py
index 40cba8d368d9..f7cee0d1d36e 100755
--- a/tools/testing/selftests/bpf/test_offload.py
+++ b/tools/testing/selftests/bpf/test_offload.py
@@ -429,7 +429,7 @@ class NetdevSim:
     def __init__(self, nsimdev, port_index, ifname):
         # In case udev renamed the netdev to according to new schema,
         # check if the name matches the port_index.
-        nsimnamere = re.compile("eni\d+np(\d+)")
+        nsimnamere = re.compile("eni\\d+np(\\d+)")
         match = nsimnamere.match(ifname)
         if match and int(match.groups()[0]) != port_index + 1:
             raise Exception("netdevice name mismatches the expected one")
-- 
2.41.0

[PATCH 8/8] selftests/bpf: fix Python string escapes in f-strings

[Benjamin Gray]

Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
be a syntax error.

Fix these now to get ahead of it before it's an error.

Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
---
 tools/testing/selftests/bpf/test_bpftool_synctypes.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/bpf/test_bpftool_synctypes.py b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
index b21bc1a35bf4..a93144b3b2b0 100755
--- a/tools/testing/selftests/bpf/test_bpftool_synctypes.py
+++ b/tools/testing/selftests/bpf/test_bpftool_synctypes.py
@@ -66,7 +66,7 @@ class ArrayParser(BlockParser):
 
     def __init__(self, reader, array_name):
         self.array_name = array_name
-        self.start_marker = re.compile(f'(static )?const bool {self.array_name}\[.*\] = {{\n')
+        self.start_marker = re.compile(f'(static )?const bool {self.array_name}\\[.*\\] = {{\n')
         super().__init__(reader)
 
     def search_block(self):
@@ -226,7 +226,7 @@ class FileExtractor(object):
 
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
-        start_marker = re.compile(f'\*{block_name}\* := {{')
+        start_marker = re.compile(f'\\*{block_name}\\* := {{')
         pattern = re.compile('\\*\\*([\\w/-]+)\\*\\*')
         end_marker = re.compile('}\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
@@ -245,7 +245,7 @@ class FileExtractor(object):
 
         @block_name: name of the blog to parse, 'TYPE' in the example
         """
-        start_marker = re.compile(f'"\s*{block_name} := {{')
+        start_marker = re.compile(f'"\\s*{block_name} := {{')
         pattern = re.compile('([\\w/]+) [|}]')
         end_marker = re.compile('}')
         return self.__get_description_list(start_marker, pattern, end_marker)
@@ -264,7 +264,7 @@ class FileExtractor(object):
 
         @macro: macro starting the block, 'HELP_SPEC_OPTIONS' in the example
         """
-        start_marker = re.compile(f'"\s*{macro}\s*" [|}}]')
+        start_marker = re.compile(f'"\\s*{macro}\\s*" [|}}]')
         pattern = re.compile('([\\w-]+) ?(?:\\||}[ }\\]])')
         end_marker = re.compile('}\\\\n')
         return self.__get_description_list(start_marker, pattern, end_marker)
-- 
2.41.0

Re: [PATCH 2/8] Documentation/sphinx: fix Python string escapes

[Jonathan Corbet]

Benjamin Gray <bgray@linux.ibm.com> writes:

> Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
> This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
> be a syntax error.
>
> Fix these now to get ahead of it before it's an error.
>
> Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
> ---
>  Documentation/sphinx/cdomain.py             | 2 +-
>  Documentation/sphinx/kernel_abi.py          | 2 +-
>  Documentation/sphinx/kernel_feat.py         | 2 +-
>  Documentation/sphinx/kerneldoc.py           | 2 +-
>  Documentation/sphinx/maintainers_include.py | 8 ++++----
>  5 files changed, 8 insertions(+), 8 deletions(-)

So I am the maintainer for this stuff...is there a reason you didn't
copy me on this work?

> diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
> index ca8ac9e59ded..dbdc74bd0772 100644
> --- a/Documentation/sphinx/cdomain.py
> +++ b/Documentation/sphinx/cdomain.py
> @@ -93,7 +93,7 @@ def markup_ctype_refs(match):
>  #
>  RE_expr = re.compile(r':c:(expr|texpr):`([^\`]+)`')
>  def markup_c_expr(match):
> -    return '\ ``' + match.group(2) + '``\ '
> +    return '\\ ``' + match.group(2) + '``\\ '

I have to wonder about this one; I doubt the intent was to insert a
literal backslash.  I have to fire up my ancient build environment to
even try this, but even if it's right...

>  #
>  # Parse Sphinx 3.x C markups, replacing them by backward-compatible ones
> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> index b5feb5b1d905..b9f026f016fd 100644
> --- a/Documentation/sphinx/kernel_abi.py
> +++ b/Documentation/sphinx/kernel_abi.py
> @@ -138,7 +138,7 @@ class KernelCmd(Directive):
>                  code_block += "\n    " + l
>              lines = code_block + "\n\n"
>  
> -        line_regex = re.compile("^\.\. LINENO (\S+)\#([0-9]+)$")
> +        line_regex = re.compile("^\\.\\. LINENO (\\S+)\\#([0-9]+)$")

All of these really just want to be raw strings - a much more minimal
fix that makes the result quite a bit more readable:

     line_regex = re.compile(r"^\.\. LINENO (\S+)\#([0-9]+)$")
                             ^
                             |
  ---------------------------+

That, I think, is how these should be fixed.

Thanks,

jon

Re: [PATCH 2/8] Documentation/sphinx: fix Python string escapes

[Benjamin Gray]

On 14/8/23 11:35 pm, Jonathan Corbet wrote:
> Benjamin Gray <bgray@linux.ibm.com> writes:
> 
>> Python 3.6 introduced a DeprecationWarning for invalid escape sequences.
>> This is upgraded to a SyntaxWarning in Python 3.12, and will eventually
>> be a syntax error.
>>
>> Fix these now to get ahead of it before it's an error.
>>
>> Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
>> ---
>>   Documentation/sphinx/cdomain.py             | 2 +-
>>   Documentation/sphinx/kernel_abi.py          | 2 +-
>>   Documentation/sphinx/kernel_feat.py         | 2 +-
>>   Documentation/sphinx/kerneldoc.py           | 2 +-
>>   Documentation/sphinx/maintainers_include.py | 8 ++++----
>>   5 files changed, 8 insertions(+), 8 deletions(-)
> 
> So I am the maintainer for this stuff...is there a reason you didn't
> copy me on this work?

Sorry, I thought the list linux-doc@vger.kernel.org itself was enough. I 
haven't done a cross tree series before, I was a bit adverse to CC'ing 
everyone that appears as a maintainer for every patch.

> 
>> diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
>> index ca8ac9e59ded..dbdc74bd0772 100644
>> --- a/Documentation/sphinx/cdomain.py
>> +++ b/Documentation/sphinx/cdomain.py
>> @@ -93,7 +93,7 @@ def markup_ctype_refs(match):
>>   #
>>   RE_expr = re.compile(r':c:(expr|texpr):`([^\`]+)`')
>>   def markup_c_expr(match):
>> -    return '\ ``' + match.group(2) + '``\ '
>> +    return '\\ ``' + match.group(2) + '``\\ '
> 
> I have to wonder about this one; I doubt the intent was to insert a
> literal backslash.  I have to fire up my ancient build environment to
> even try this, but even if it's right...

Yeah, there is even a file that just has a syntax error. I don't have a 
way to verify the original script was correct, but I have verified this 
series doesn't change the parsed AST.

In this case though, it's generating reST, so it might just be 
conservatively guarding against generating bad markup[1]

[1]: 
https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#inline-markup 


>>   #
>>   # Parse Sphinx 3.x C markups, replacing them by backward-compatible ones
>> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
>> index b5feb5b1d905..b9f026f016fd 100644
>> --- a/Documentation/sphinx/kernel_abi.py
>> +++ b/Documentation/sphinx/kernel_abi.py
>> @@ -138,7 +138,7 @@ class KernelCmd(Directive):
>>                   code_block += "\n    " + l
>>               lines = code_block + "\n\n"
>>   
>> -        line_regex = re.compile("^\.\. LINENO (\S+)\#([0-9]+)$")
>> +        line_regex = re.compile("^\\.\\. LINENO (\\S+)\\#([0-9]+)$")
> 
> All of these really just want to be raw strings - a much more minimal
> fix that makes the result quite a bit more readable:
> 
>       line_regex = re.compile(r"^\.\. LINENO (\S+)\#([0-9]+)$")
>                               ^
>                               |
>    ---------------------------+
> 
> That, I think, is how these should be fixed.

Yup, I mentioned that at the end of the cover letter. I can automate and 
verify the conversion, but automating what _should_ be treated as a 
'regex' string is fuzzier. Checking if there's a `re.*(` prefix on the 
string should work for most though. I'll give it a shot.

> Thanks,
> 
> jon