[PATCH v2 06/28] docs: kdoc: use tokenizer to handle comments on structs

Mauro Carvalho Chehab
2026-03-12T14:54:58Z

Better handle comments inside structs. After those changes,
all unittests now pass:

test_private:
TestPublicPrivate:
test balanced_inner_private: OK
test balanced_non_greddy_private: OK
test balanced_private: OK
test no private: OK
test unbalanced_inner_private: OK
test unbalanced_private: OK
test unbalanced_struct_group_tagged_with_private: OK
test unbalanced_two_struct_group_tagged_first_with_private: OK
test unbalanced_without_end_of_line: OK

Ran 9 tests

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <f83ee9e8c38407eaab6ad10d4ccf155fb36683cc.1773074166.git.mchehab+huawei@kernel.org>
---
tools/lib/python/kdoc/kdoc_parser.py | 14 ++++----------
1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/tools/lib/python/kdoc/kdoc_parser.py b/tools/lib/python/kdoc/kdoc_parser.py
index 4b3c555e6c8e..6b181ead3175 100644
--- a/tools/lib/python/kdoc/kdoc_parser.py
+++ b/tools/lib/python/kdoc/kdoc_parser.py
@@ -13,7 +13,7 @@ import sys
import re
from pprint import pformat

-from kdoc.kdoc_re import NestedMatch, KernRe
+from kdoc.kdoc_re import NestedMatch, KernRe, CTokenizer
from kdoc.kdoc_item import KdocItem

#
@@ -84,15 +84,9 @@ def trim_private_members(text):
"""
Remove ``struct``/``enum`` members that have been marked "private".
"""
- # First look for a "public:" block that ends a private region, then
- # handle the "private until the end" case.
- #
- text = KernRe(r'/\*\s*private:.*?/\*\s*public:.*?\*/', flags=re.S).sub('', text)
- text = KernRe(r'/\*\s*private:.*', flags=re.S).sub('', text)
- #
- # We needed the comments to do the above, but now we can take them out.
- #
- return KernRe(r'\s*/\*.*?\*/\s*', flags=re.S).sub('', text).strip()
+
+ tokens = CTokenizer(text)
+ return str(tokens)

class state:
"""
--
2.52.0