Re: [docs] [PATCH v3 5/5] tools: Add check-confusables pre-commit hook

[Paul Barker <paul@pbarker.dev>]

[-- Attachment #1: Type: text/plain, Size: 2526 bytes --]

On Tue, 2026-07-28 at 19:40 +0300, Niko Mauno wrote:
> 
> On 7/28/26 1:41 PM, Paul Barker wrote:
> > On Tue, 2026-07-28 at 12:07 +0300, Niko Mauno via lists.yoctoproject.org
> > wrote:
> > > From: Niko Mauno <niko.mauno@vaisala.com>
> > > 
> > > Add a check-confusables script, in the same fashion as
> > > check-glossaries, that scans the documentation .rst sources for
> > > non-ASCII "confusable" characters (curly quotes, en/em dashes,
> > > non-breaking and zero-width spaces, etc.) and reports each occurrence
> > > with its location and suggested ASCII replacement, exiting non-zero if
> > > any are found. This guards against the class of breakage fixed in the
> > > preceding commit, e.g. curly quotes causing recipe ParseErrors.
> > > 
> > > Legitimate non-ASCII such as box-drawing characters used in directory
> > > trees, accented letters in contributor names and CJK characters are
> > > intentionally left untouched. No-break spaces are likewise tolerated
> > > on lines containing box-drawing characters, since the tree command
> > > emits them as indentation in directory listings.
> > > 
> > > Wire it up both as a local pre-commit hook, which checks the changed
> > > files, and in the Makefile "checks" target, which scans the whole
> > > tree, alongside check-glossaries.
> > > 
> > > Suggested-by: Quentin Schulz <quentin.schulz@cherry.de>
> > > Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
> > > Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
> > 
> > Hi Niko,
> > 
> > Are there any existing tools we can use instead of implementing this
> > ourselves? Perhaps https://pypi.org/project/confusables/ ?
> > 
> > Some comments below if we do want to merge our own implementation of
> > this...
> > 
> 
> Hi Paul,
> 
> Thank you for these observations. It seems that confusables (and its 
> alternatives) would not directly resolve the exact problems that 
> check-confusables script addresses, therefore retained the 
> dependency-free approach in v4 which I just submitted. It addresses the 
> missing copyright/license header, and changes the literal glyphs to \u 
> escape sequences.
> 
> Also the topmost commit message now contains a segment explaining why 
> the "own short table" is used instead of depending on a 3rd party package.

Hi Niko,

Thanks for following up. If existing Python packages don't meet our
needs then I agree it makes sense to carry our own curated list.

Best regards,

-- 
Paul Barker


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 252 bytes --]