Re: [PATCH v3 0/2] Don't create Python bytecode when building the kernel

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Mon, 21 Apr 2025 10:35:29 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> writes:
> 
> > On Wed, Apr 16, 2025 at 03:51:03PM +0800, Mauro Carvalho Chehab wrote:  
> >> 
> >> As reported by Andy, the Kernel build system runs kernel-doc script for DRM,
> >> when W=1. Due to Python's normal behavior, its JIT compiler will create
> >> a bytecode and store it under scripts/lib/*/__pycache__. 
> >> 
> >> As one may be using O= and even having the sources on a read-only mount
> >> point, disable its creation during build time.  
> >
> > Would it be possible to properly support O= and create pyc / pycache
> > inside the object/output dir?  
> 
> I have to confess, I've been wondering if we should be treating the .pyc
> files like we treat .o files or other intermediate products.  Rather
> than trying to avoid their creation entirely, perhaps we should just be
> sure they end up in the right place and are properly cleaned up...?

I sent a v4 of the series disabling *.pyc creation, basically placing
*.pyc at the right place on .gitignore.

That's said, I agree that the best would be to use PYTHONPYCACHEPREFIX,
placing the intermediate products altogether with O= results. There is
however something we need to deal with. To follow our building system
to the letter, if we have *.py files at scripts/lib/kdoc, the intermediate
files should be at: either:

	- {outdir}/scripts/lib/kdoc; or: 
	- {outdir}/scripts/lib/kdoc/__pycache__ 

The same shall happen for all other places including Documentation/sphinx.
In the specific case of Sphinx extensions, it would mean that it would
produce *.pyc for both scripts/lib/*/*.py and Documentation/sphinx.
No idea how to make Python to do that without doing some horrible tricks.

An alternate approach would be to store all of them at the same place, like:

	- {outdir}/__pycache__ 

The problem is that, if we end having two scripts with the same name, and
using PYTHONPYCACHEPREFIX would place both of them at the same place, we'll
have troubles.

IMO, let's apply this series first, and then see if we can replace patch 3
with another one using PYTHONPYCACHEPREFIX, after we're sure that there's
a way for it to do the right thing.

> 
> To answer Dmitry's question, it seems that setting PYTHONPYCACHEPREFIX
> should do the trick?

Regards,
Mauro