[PATCH 0/3] tools/mm/thpmaps: fix THP statistics and mapping names

[PATCH 0/3] tools/mm/thpmaps: fix THP statistics and mapping names

[Weilin Tong]

This series fixes three issues in thpmaps.

Patch 1 corrects anon/file accounting for trailing partial THPs.
Patch 2 includes order-1 file THPs in the per-size statistics.
Patch 3 preserves mapping names containing whitespace, including
paths with a (deleted) suffix.

The three fixes are independent.

Weilin Tong (3):
  tools/mm/thpmaps: fix partial THP anon/file accounting
  tools/mm/thpmaps: fix missing order-1 file THP statistics
  tools/mm/thpmaps: fix missing mapping names

 tools/mm/thpmaps | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)


base-commit: 39274b454e9993cea319f4ac342a8082a9ee1730
-- 
2.43.7

[PATCH 1/3] tools/mm/thpmaps: fix partial THP anon/file accounting

[Weilin Tong]

thp_parse() reduces index_end before looking up the type of the
trailing partially mapped THP. The lookup then uses the preceding
page, or anons[-1] if the THP starts at index zero, so its pages can
be charged to the wrong anon/file counter.

Account the partial THP before reducing index_end.

Fixes: 2444172cfde4 ("tools/mm: add thpmaps script to dump THP usage info")
Assisted-by: LLM
Signed-off-by: Weilin Tong <tongweilin@linux.alibaba.com>
---
 tools/mm/thpmaps | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps
index 803e0318f2fe..d92aace005c1 100644
--- a/tools/mm/thpmaps
+++ b/tools/mm/thpmaps
@@ -263,8 +263,8 @@ def thp_parse(vma, kpageflags, ranges, indexes, vfns, pfns, anons, heads):
             if flags & KPF_COMPOUND_TAIL:
                 nr = index_end - int(folios[-1])
                 folios = folios[:-1]
-                index_end -= nr
                 stats['anon' if anons[index_end - 1] else 'file']['partial'] += nr
+                index_end -= nr
 
         # Account fully mapped THPs in the middle of the range.
         if len(folios):
-- 
2.43.7

[PATCH 2/3] tools/mm/thpmaps: fix missing order-1 file THP statistics

[Weilin Tong]

File-backed THPs can have order 1, but the per-size counters are only
emitted for order 2 and above. Fully and contiguously mapped order-1
file THPs are therefore missing from the output.

Start emitting the per-size counters at order 1.

Fixes: 2444172cfde4 ("tools/mm: add thpmaps script to dump THP usage info")
Assisted-by: LLM
Signed-off-by: Weilin Tong <tongweilin@linux.alibaba.com>
---
 tools/mm/thpmaps | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps
index d92aace005c1..30a8e915d677 100644
--- a/tools/mm/thpmaps
+++ b/tools/mm/thpmaps
@@ -295,7 +295,7 @@ def thp_parse(vma, kpageflags, ranges, indexes, vfns, pfns, anons, heads):
 
     def flatten_sub(type, subtype, stats):
         param = f"{type}-thp-pte-{subtype}-{{}}kB"
-        for od, nr in enumerate(stats[2:], 2):
+        for od, nr in enumerate(stats[1:], 1):
             rstats[param.format(odkb(od))] = {'type': type, 'value': nrkb(nr)}
 
     def flatten_type(type, stats):
-- 
2.43.7

[PATCH 3/3] tools/mm/thpmaps: fix missing mapping names

[Weilin Tong]

VMAList splits smaps lines on whitespace and only keeps the mapping
name when there are exactly six fields. Names containing whitespace,
including paths with a (deleted) suffix, are therefore lost.

Limit splitting to the five fixed fields and preserve the remaining
name.

Fixes: 2444172cfde4 ("tools/mm: add thpmaps script to dump THP usage info")
Assisted-by: LLM
Signed-off-by: Weilin Tong <tongweilin@linux.alibaba.com>
---
 tools/mm/thpmaps | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps
index 30a8e915d677..a1a9bc30514f 100644
--- a/tools/mm/thpmaps
+++ b/tools/mm/thpmaps
@@ -198,7 +198,7 @@ class VMAList:
         self.vmas = []
         with open(f'/proc/{pid}/smaps', 'r') as file:
             for line in file:
-                elements = line.split()
+                elements = line.rstrip('\n').split(maxsplit=5)
                 if '-' in elements[0]:
                     start, end = map(lambda x: int(x, 16), elements[0].split('-'))
                     major, minor = map(lambda x: int(x, 16), elements[3].split(':'))
-- 
2.43.7

Re: [PATCH 3/3] tools/mm/thpmaps: fix missing mapping names

[Andrew Morton]

On Tue, 27 Oct 2026 15:49:04 +0800 Weilin Tong <tongweilin@linux.alibaba.com> wrote:

> VMAList splits smaps lines on whitespace and only keeps the mapping
> name when there are exactly six fields. Names containing whitespace,
> including paths with a (deleted) suffix, are therefore lost.
> 
> Limit splitting to the five fixed fields and preserve the remaining
> name.
> 
> ...
>
> --- a/tools/mm/thpmaps
> +++ b/tools/mm/thpmaps
> @@ -198,7 +198,7 @@ class VMAList:
>          self.vmas = []
>          with open(f'/proc/{pid}/smaps', 'r') as file:
>              for line in file:
> -                elements = line.split()
> +                elements = line.rstrip('\n').split(maxsplit=5)
>                  if '-' in elements[0]:
>                      start, end = map(lambda x: int(x, 16), elements[0].split('-'))
>                      major, minor = map(lambda x: int(x, 16), elements[3].split(':'))

Thanks.  My pedantic reviewbot points out

: Could we also handle carriage returns and non-UTF-8 bytes in filenames?
: Both are valid filename contents.  Python's universal newline handling
: splits the former into separate records, while strict decoding raises
: UnicodeDecodeError for the latter.
: 
: I reproduced both failures with real file mappings, before and after
: this series. Opening smaps this way fixes the tested cases:
: 
:     with open(f'/proc/{pid}/smaps', 'r', newline='\n',
:               errors='surrogateescape') as file:
: 
: These are pre-existing problems, not regressions introduced here.
: They could be included in [3/3] or handled in a follow-up.

and yup, these patterns could indeed be encountered.

But let me grab the series as-is, on the grounds it's much better than
before - spaces in filenames are common whereas \r's and non-UTF8 are
not.

[PATCH] tools/mm/thpmaps: handle carriage returns and non-UTF-8 filenames

[Weilin Tong]

VMAList opens smaps with universal newline handling and strict decoding.
A carriage return in a mapped filename can split a header into multiple
records, while non-UTF-8 bytes cause UnicodeDecodeError.

Read LF-terminated records and use surrogateescape to preserve filename
bytes.

Fixes: 2444172cfde4 ("tools/mm: add thpmaps script to dump THP usage info")
Suggested-by: Andrew Morton <akpm@linux-foundation.org>
Link: https://lore.kernel.org/linux-mm/20261101211644.c63a2028bbede0f763f41f80@linux-foundation.org/
Signed-off-by: Weilin Tong <tongweilin@linux.alibaba.com>
---
This is a follow-up to "tools/mm/thpmaps: fix missing mapping names":
https://lore.kernel.org/linux-mm/20261027074904.3328878-4-tongweilin@linux.alibaba.com/

 tools/mm/thpmaps | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps
index a1a9bc30514f..f89568bd499e 100644
--- a/tools/mm/thpmaps
+++ b/tools/mm/thpmaps
@@ -196,7 +196,8 @@ class VMAList:
     # instance to receive VMAs.
     def __init__(self, pid='self', stats=[]):
         self.vmas = []
-        with open(f'/proc/{pid}/smaps', 'r') as file:
+        with open(f'/proc/{pid}/smaps', 'r', newline='\n',
+                  errors='surrogateescape') as file:
             for line in file:
                 elements = line.rstrip('\n').split(maxsplit=5)
                 if '-' in elements[0]:
-- 
2.43.7

Re: [PATCH 2/3] tools/mm/thpmaps: fix missing order-1 file THP statistics

[David Hildenbrand (Arm)]

On 10/27/26 08:49, Weilin Tong wrote:
> File-backed THPs can have order 1, but the per-size counters are only
> emitted for order 2 and above. Fully and contiguously mapped order-1
> file THPs are therefore missing from the output.
> 
> Start emitting the per-size counters at order 1.
> 
> Fixes: 2444172cfde4 ("tools/mm: add thpmaps script to dump THP usage info")
> Assisted-by: LLM
> Signed-off-by: Weilin Tong <tongweilin@linux.alibaba.com>
> ---
>  tools/mm/thpmaps | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps
> index d92aace005c1..30a8e915d677 100644
> --- a/tools/mm/thpmaps
> +++ b/tools/mm/thpmaps
> @@ -295,7 +295,7 @@ def thp_parse(vma, kpageflags, ranges, indexes, vfns, pfns, anons, heads):
>  
>      def flatten_sub(type, subtype, stats):
>          param = f"{type}-thp-pte-{subtype}-{{}}kB"
> -        for od, nr in enumerate(stats[2:], 2):
> +        for od, nr in enumerate(stats[1:], 1):
>              rstats[param.format(odkb(od))] = {'type': type, 'value': nrkb(nr)}
>  
>      def flatten_type(type, stats):

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

Re: [PATCH 3/3] tools/mm/thpmaps: fix missing mapping names

[David Hildenbrand (Arm)]

On 10/27/26 08:49, Weilin Tong wrote:
> VMAList splits smaps lines on whitespace and only keeps the mapping
> name when there are exactly six fields. Names containing whitespace,
> including paths with a (deleted) suffix, are therefore lost.
> 
> Limit splitting to the five fixed fields and preserve the remaining
> name.
> 
> Fixes: 2444172cfde4 ("tools/mm: add thpmaps script to dump THP usage info")
> Assisted-by: LLM
> Signed-off-by: Weilin Tong <tongweilin@linux.alibaba.com>
> ---

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

Re: [PATCH] tools/mm/thpmaps: handle carriage returns and non-UTF-8 filenames

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] tools/mm/thpmaps: UnicodeEncodeError in vma_print() on non-UTF-8 filenames
--

commit 5f793a93e6fde4bbef2fd7ad6b6bda119aa8b288
Author: Weilin Tong <tongweilin@linux.alibaba.com>

tools/mm/thpmaps: handle carriage returns and non-UTF-8 filenames

This commit updates VMAList.__init__() in tools/mm/thpmaps to open
/proc/<pid>/smaps with newline='\n' and errors='surrogateescape' so that
carriage returns in mapped filenames do not split header lines and
non-UTF-8 bytes do not raise UnicodeDecodeError while reading.

Link: https://lore.kernel.org/linux-mm/20261101211644.c63a2028bbede0f763f41f80@linux-foundation.org/

> diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps
> index a1a9bc30514f3..f89568bd499ec 100644
> --- a/tools/mm/thpmaps
> +++ b/tools/mm/thpmaps
> @@ -196,7 +196,8 @@ class VMAList:
>      # instance to receive VMAs.
>      def __init__(self, pid='self', stats=[]):
>          self.vmas = []
> -        with open(f'/proc/{pid}/smaps', 'r') as file:
> +        with open(f'/proc/{pid}/smaps', 'r', newline='\n',
> +                  errors='surrogateescape') as file:

[Severity: Medium]
When VMAList.__init__() opens /proc/<pid>/smaps with
errors='surrogateescape', non-UTF-8 bytes in a mapped filename are decoded
into lone surrogate code points in vma.name.

In default (non-rollup) mode, do_main() passes each VMA to vma_print():

tools/mm/thpmaps:do_main() {
    ...
                        if args.rollup:
                            ...
                        else:
                            vma_print(vma, pid)
                            stats_print(stats, vma_anon, vma_file, args.inc_empty)
        except (FileNotFoundError, ProcessLookupError, FileIOException):
    ...
}

And vma_print() writes vma.name directly to sys.stdout via print():

tools/mm/thpmaps:vma_print() {
    print("{:010d}: {:016x}-{:016x} {}{}{}{} {:08x} {:02x}:{:02x} {:08x} {}"
        .format(
            pid, vma.start, vma.end,
            'r' if vma.read else '-', 'w' if vma.write else '-',
            'x' if vma.execute else '-', 'p' if vma.private else 's',
            vma.pgoff, vma.major, vma.minor, vma.inode, vma.name
        ))
}

Under standard UTF-8 locales, sys.stdout uses errors='strict' by default,
which rejects lone surrogates. Doesn't printing a VMA with non-UTF-8
filename bytes in vma_print() still terminate the script with an uncaught
UnicodeEncodeError unless sys.stdout is also reconfigured (for example with
errors='surrogateescape' or errors='backslashreplace') or vma.name is
sanitized before printing?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261102085419.696232-1-tongweilin@linux.alibaba.com?part=1

Re: [PATCH] tools/mm/thpmaps: handle carriage returns and non-UTF-8 filenames

[Weilin Tong]

在 2026/11/3 16:54, sashiko-bot@kernel.org 写道:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [Medium] tools/mm/thpmaps: UnicodeEncodeError in vma_print() on non-UTF-8 filenames
> --
>
> commit 5f793a93e6fde4bbef2fd7ad6b6bda119aa8b288
> Author: Weilin Tong <tongweilin@linux.alibaba.com>
>
> tools/mm/thpmaps: handle carriage returns and non-UTF-8 filenames
>
> This commit updates VMAList.__init__() in tools/mm/thpmaps to open
> /proc/<pid>/smaps with newline='\n' and errors='surrogateescape' so that
> carriage returns in mapped filenames do not split header lines and
> non-UTF-8 bytes do not raise UnicodeDecodeError while reading.
>
> Link: https://lore.kernel.org/linux-mm/20261101211644.c63a2028bbede0f763f41f80@linux-foundation.org/
>
>> diff --git a/tools/mm/thpmaps b/tools/mm/thpmaps
>> index a1a9bc30514f3..f89568bd499ec 100644
>> --- a/tools/mm/thpmaps
>> +++ b/tools/mm/thpmaps
>> @@ -196,7 +196,8 @@ class VMAList:
>>       # instance to receive VMAs.
>>       def __init__(self, pid='self', stats=[]):
>>           self.vmas = []
>> -        with open(f'/proc/{pid}/smaps', 'r') as file:
>> +        with open(f'/proc/{pid}/smaps', 'r', newline='\n',
>> +                  errors='surrogateescape') as file:
> [Severity: Medium]
> When VMAList.__init__() opens /proc/<pid>/smaps with
> errors='surrogateescape', non-UTF-8 bytes in a mapped filename are decoded
> into lone surrogate code points in vma.name.
>
> In default (non-rollup) mode, do_main() passes each VMA to vma_print():
>
> tools/mm/thpmaps:do_main() {
>      ...
>                          if args.rollup:
>                              ...
>                          else:
>                              vma_print(vma, pid)
>                              stats_print(stats, vma_anon, vma_file, args.inc_empty)
>          except (FileNotFoundError, ProcessLookupError, FileIOException):
>      ...
> }
>
> And vma_print() writes vma.name directly to sys.stdout via print():
>
> tools/mm/thpmaps:vma_print() {
>      print("{:010d}: {:016x}-{:016x} {}{}{}{} {:08x} {:02x}:{:02x} {:08x} {}"
>          .format(
>              pid, vma.start, vma.end,
>              'r' if vma.read else '-', 'w' if vma.write else '-',
>              'x' if vma.execute else '-', 'p' if vma.private else 's',
>              vma.pgoff, vma.major, vma.minor, vma.inode, vma.name
>          ))
> }
>
> Under standard UTF-8 locales, sys.stdout uses errors='strict' by default,
> which rejects lone surrogates. Doesn't printing a VMA with non-UTF-8
> filename bytes in vma_print() still terminate the script with an uncaught
> UnicodeEncodeError unless sys.stdout is also reconfigured (for example with
> errors='surrogateescape' or errors='backslashreplace') or vma.name is
> sanitized before printing?


Thanks for the report. I reproduced the failure under en_US.UTF-8.
I also confirmed that valid Unicode filenames fail when stdout uses
strict ASCII encoding.

I'll address both output encoding cases in v2 alongside the smaps
reading fixes.

Thanks,
Weilin

>