[Buildroot] [PATCH 0/2 v2] support/size-stats: actually handle .pyc files

[Buildroot] [PATCH 0/2 v2] support/size-stats: actually handle .pyc files

[yann.morin]

From: "Yann E. MORIN" <yann.morin@orange.com>

Hello All!

This small two-patch series properly associates .pyc files to the
corresponding package, to account for the PEP 3147 layout.

Changes v1 -> v2:
  - use python's cache_from_source() instead of open-coding the
   path  (Vincent)


Regards,
Yann E. MORIN.


The following changes since commit 04d7d7bdeb907483e7db0005b403484f379cf84b

  package/mosh: bump gcc version requirement due to protobuf (2026-09-13 21:59:28 +0200)


are available as patches in this mail series,

for you to apply patches up to 9e49dfb4368becd006de1be4977695caea90c237

  support/scripts/size-stats: properly assign pre-compiled .pyc files (2026-09-14 14:29:42 +0200)


----------------------------------------------------------------
Yann E. MORIN (2):
      support/scripts/size-stats: prepare for more .py associated files
      support/scripts/size-stats: properly assign pre-compiled .pyc files

 support/scripts/size-stats | 31 ++++++++++++++++++++++++++++---
 1 file changed, 28 insertions(+), 3 deletions(-)

--
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
|                 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
| yann.morin (at) orange.com           |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 1/2 v2] support/scripts/size-stats: prepare for more .py associated files

[yann.morin]

From: "Yann E. MORIN" <yann.morin@orange.com>

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
---
 support/scripts/size-stats | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/support/scripts/size-stats b/support/scripts/size-stats
index 79c0dc571e..fdcaeaa546 100755
--- a/support/scripts/size-stats
+++ b/support/scripts/size-stats
@@ -54,9 +54,8 @@ class Config:
 # pkg: package to which the file belongs
 #
 def add_file(filesdict, relpath, abspath, pkg):
-    if relpath.endswith(".py"):
-        # also check for compiled .pyc file
-        add_file(filesdict, relpath + "c", abspath + "c", pkg)
+    for relpath2, abspath2 in get_associated_files(relpath, abspath):
+        add_file(filesdict, relpath2, abspath2, pkg)
     if not os.path.exists(abspath):
         return
     if os.path.islink(abspath):
@@ -65,6 +64,22 @@ def add_file(filesdict, relpath, abspath, pkg):
     filesdict[relpath] = (pkg, sz)
 
 
+#
+# Returns an iterator over the files associated with the given one
+# as a 2-tuple of (relpath, abspath).
+#
+# Supported files:
+#  *.py => yields the list of corresponding pre-compiled .pyc files
+#
+# relpath: relative path of the file
+# fullpath: absolute path to the file
+#
+def get_associated_files(relpath, abspath):
+    # also check for compiled .pyc file
+    if relpath.endswith(".py"):
+        yield (relpath + "c", abspath + "c")
+
+
 #
 # This function returns a dict where each key is the path of a file in
 # the root filesystem, and the value is a tuple containing two
-- 
2.43.0

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 2/2 v2] support/scripts/size-stats: properly assign pre-compiled .pyc files

[yann.morin]

From: "Yann E. MORIN" <yann.morin@orange.com>

Since commit 3fed42456693 (package/python3: use the provided pyc
compiler), we abide by the PEP 3147 guidelines on where the pyc files
are located. In practice, this means that:

1. BR2_PACKAGE_PYTHON3_PY_ONLY: there is no precompiled file; this was
   still working so far;

2. BR2_PACKAGE_PYTHON3_PYC_ONLY: there is only the .pyc file in the same
   directory as the original .py file; tjhis was still working so far,
   as it uses the same legacy pre-PEP 3147 layout;

3. BR2_PACKAGE_PYTHON3_PY_PYC: for a .py file /foo/bar.py, the
   corresponding .pyc file will be /foo/__pycache__/bar.pythonX.Y.pyc
   (where X.Y is the current major.minor python version, e.g. 3.14);
   this is what is broken now.

Add support for this layout in the size-stats script.

When we associate files to a package, we only have the original name,
and we must construct the associated filenames. We could construct the
path to the .pyc file manually (like we do for the legacy layout), ut
this is brittel, and will break if/when python changes the layout.
Instead, use importlib's helper cache_from_source(), as suggested by
Vincent.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>

---
Changes v1 -> v2:
  - use cache_from_source() rather than compute the path manually
    (Veincent)
---
 support/scripts/size-stats | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/support/scripts/size-stats b/support/scripts/size-stats
index fdcaeaa546..c54ef7bc2c 100755
--- a/support/scripts/size-stats
+++ b/support/scripts/size-stats
@@ -23,6 +23,7 @@ import argparse
 import csv
 import collections
 import math
+import importlib
 
 try:
     import matplotlib
@@ -77,7 +78,16 @@ def add_file(filesdict, relpath, abspath, pkg):
 def get_associated_files(relpath, abspath):
     # also check for compiled .pyc file
     if relpath.endswith(".py"):
+        # Legacy .pyc next to .py, or .pyc-only setup
         yield (relpath + "c", abspath + "c")
+        # PEP 3147 layout
+        # importlib._bootstrap_external.cache_from_source() works on the
+        # path string, not on actual files being present/absent, and so
+        # it also works for relative paths...
+        yield (
+            importlib._bootstrap_external.cache_from_source(relpath),
+            importlib._bootstrap_external.cache_from_source(abspath),
+        )
 
 
 #
-- 
2.43.0

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2 v2] support/size-stats: actually handle .pyc files

[yann.morin]

All,

+Peter, +Thomas

On 2026-09-14 14:48 +0200, yann.morin@orange.com spake thusly:
> This small two-patch series properly associates .pyc files to the
> corresponding package, to account for the PEP 3147 layout.

Weird: this patch series did not end up in patchwork, although it is on
lore:
    https://lore.kernel.org/buildroot/cover.1789390115.git.yann.morin@orange.com/

Should I resend it?

Regards,
Yann E. MORIN.

-- 
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
|                 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
| yann.morin (at) orange.com           |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2 v2] support/size-stats: actually handle .pyc files

[Peter Korsgaard]

>>>>> "yann" == yann morin <yann.morin@orange.com> writes:

 > All,
 > +Peter, +Thomas

 > On 2026-09-14 14:48 +0200, yann.morin@orange.com spake thusly:
 >> This small two-patch series properly associates .pyc files to the
 >> corresponding package, to account for the PEP 3147 layout.

 > Weird: this patch series did not end up in patchwork, although it is on
 > lore:
 >     https://lore.kernel.org/buildroot/cover.1789390115.git.yann.morin@orange.com/

 > Should I resend it?

It it certainly easier to have it in patchwork, please do.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH 0/2 v2] support/size-stats: actually handle .pyc files

[yann.morin]

From: "Yann E. MORIN" <yann.morin@orange.com>

Hello All!

This small two-patch series properly associates .pyc files to the
corresponding package, to account for the PEP 3147 layout.

[This is a resend of v2, which did not end up on patchwork.]

Changes v1 -> v2:
  - use python's cache_from_source() instead of open-coding the
    path  (Vincent)


Regards,
Yann E. MORIN.


The following changes since commit 2ca7fb5e7bf7b416b72faf709bf1220eef0dc352

  package/wpewebkit: fix build with CMake 4.4 (2026-09-17 17:10:35 +0200)


are available as patches in this mail series,

for you to apply patches up to 71588ea1ac7d7cc7f5f5f30fcb9c02e84ec12ccd

  support/scripts/size-stats: properly assign pre-compiled .pyc files (2026-09-18 06:34:49 +0200)


----------------------------------------------------------------
Yann E. MORIN (2):
      support/scripts/size-stats: prepare for more .py associated files
      support/scripts/size-stats: properly assign pre-compiled .pyc files

 support/scripts/size-stats | 31 ++++++++++++++++++++++++++++---
 1 file changed, 28 insertions(+), 3 deletions(-)

--
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
|                 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
| yann.morin (at) orange.com           |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2 v2] support/size-stats: actually handle .pyc files

[Vincent Fazio]

> -----Original Message-----
> From: yann.morin@orange.com <yann.morin@orange.com>
> Sent: Thursday, September 17, 2026 11:46 PM
> To: buildroot@buildroot.org
> Cc: Vincent Fazio <vfazio@xes-inc.com>; Yann E . MORIN
> <yann.morin@orange.com>
> Subject: [External] - [PATCH 0/2 v2] support/size-stats: actually handle .pyc
> files
> 
> From: "Yann E. MORIN" <yann.morin@orange.com>
> 
> Hello All!
> 
> This small two-patch series properly associates .pyc files to the
> corresponding package, to account for the PEP 3147 layout.
> 
> [This is a resend of v2, which did not end up on patchwork.]
> 
> Changes v1 -> v2:
>   - use python's cache_from_source() instead of open-coding the
>     path  (Vincent)
> 

Look OK to me.

I think we can specify the `optimization` argument to `cache_from_source` in the future if it becomes necessary to account for different optimization levels.

Acked-by: Vincent Fazio <vfazio@xes-inc.com>


_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 0/2 v2] support/size-stats: actually handle .pyc files

[yann.morin]

All,

On 2026-09-18 06:45 +0200, yann.morin@orange.com spake thusly:
> This small two-patch series properly associates .pyc files to the
> corresponding package, to account for the PEP 3147 layout.

Now that Vincent has reviewed and acked that series (thanks! 👍), is
there any more feedback that I should take into account? :-)

Regards,
Yann E. MORIN.

-- 
                                        ____________
.-----------------.--------------------:       _    :------------------.
|  Yann E. MORIN  | Real-Time Embedded |    __/ )   | /"\ ASCII RIBBON |
|                 | Software  Designer |  _/ - /'   | \ / CAMPAIGN     |
| +33 638.411.245 '--------------------: (_    `--, |  X  AGAINST      |
| yann.morin (at) orange.com           |_="    ,--' | / \ HTML MAIL    |
'--------------------------------------:______/_____:------------------'

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot