Re: [Buildroot] [PATCH] package/python3: ensure pyc embed rooted paths

From: Thomas Petazzoni via buildroot <buildroot@buildroot.org>
To: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>, buildroot@buildroot.org
Subject: Re: [Buildroot] [PATCH] package/python3: ensure pyc embed rooted paths
Date: Mon, 29 Aug 2022 21:16:05 +0200	[thread overview]
Message-ID: <20220829211605.5cb4b256@windsurf> (raw)
In-Reply-To: <20220825200510.125728-1-yann.morin.1998@free.fr>

On Thu, 25 Aug 2022 22:05:10 +0200
"Yann E. MORIN" <yann.morin.1998@free.fr> wrote:

> To report usable tracebacks, pyc files embed the path of the original py
> files, so that users can more easily try and debug the reported issue.
> 
> We genrate the pyc files by calling the python3-supplied compileall
> script, to scan the directry where python modules are installed. Since
> this is done on the build machine, we tell compileall.py to strip away
> the TARGET_DIR prefix, as that has no meaning at runtime.
> 
> However, compileall.py forgets [0] to keep a leading / in the front of
> the paths, thus generating non-rooted paths., e.g.:
>      /path/buildroot.ouput/targt/usr/lib/python3.10/argparse.py
> gets embedded as:
>      usr/lib/python3.10/argparse.py
> 
> This is a bit confusing but, as far as we could see, should be mostly be
> used for display purposes in tracebacks, and does not seem to impact
> actual functkionality.
> 
> We fix that by instructing compileall.py that the embeeded paths should
> be rooted to / which generates proper paths in tracebacks.
> 
> And alternate solution would be to swith gears, and tell compileall.py
> exactly the resulting runtime "base" directory, wihch replaces the
> stripping and prefixing; i.e. it's either:
>     -s $(TARGET_DIR) -p /
> or
>     -d /usr/lib/python$(PYTHON3_MAJOR_VERSION)
> 
> We choose to keep the first solution, because that is semantically what
> we really want to do: to strip the leading build-time path, rather than
> to force anything.
> 
> Note: the python test-suite was executed with both solutions (in a
> pyc-only setup), and the results were exactly the same; so in practice,
> -d or -s+-p yield the same results.
> 
> Many thanks go to Vincent for reporting the issue and suggesting the
> solutions.
> 
> Reported-by: Vincent Fazio <vfazio@xes-inc.com>
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> ---
>  package/python3/python3.mk | 1 +
>  1 file changed, 1 insertion(+)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot