[Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Sergey Matyukevich]

Linux kernel commit 00facc760903 ("perf jevents: Switch build to use
jevents.py") switched to auto-generation of arch-specific PMU events
using python script. Now custom PMU events for different platforms of
the selected target architecture are not embedded into perf binary if
an appropriate host python interpreter is not present. In practice it
means that perf is successfully built, but 'perf list pmu' will show
no custom events on a target platform even if those events are supported
and properly defined in tools/perf/pmu-events/arch/<target arch>
directory in the kernel source tree.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
---
 package/linux-tools/linux-tool-perf.mk.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/linux-tools/linux-tool-perf.mk.in b/package/linux-tools/linux-tool-perf.mk.in
index 057c6f80e7..44c1bbdf83 100644
--- a/package/linux-tools/linux-tool-perf.mk.in
+++ b/package/linux-tools/linux-tool-perf.mk.in
@@ -6,7 +6,7 @@
 
 LINUX_TOOLS += perf
 
-PERF_DEPENDENCIES = host-flex host-bison
+PERF_DEPENDENCIES = host-flex host-bison host-python3
 
 ifeq ($(NORMALIZED_ARCH),x86_64)
 PERF_ARCH=x86
-- 
2.37.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Yann E. MORIN]

Sergey, All,

On 2022-09-04 23:57 +0300, Sergey Matyukevich spake thusly:
> Linux kernel commit 00facc760903 ("perf jevents: Switch build to use
> jevents.py")

00facc760903 is only parts of 6.0-rc kernels so far, so unless one uses
bleeding edge technology, there is no issue. So, paying the price of a
host-pytohn3 build just for release-candidate kernels is a bit too much
in my opinion.

So, I added a config knob to drive this, like we have for host-openssl
and a few other host programs for building the kernel.

Applied to master with the above change, thanks.

Regards,
Yann E. MORIN.

> switched to auto-generation of arch-specific PMU events
> using python script. Now custom PMU events for different platforms of
> the selected target architecture are not embedded into perf binary if
> an appropriate host python interpreter is not present. In practice it
> means that perf is successfully built, but 'perf list pmu' will show
> no custom events on a target platform even if those events are supported
> and properly defined in tools/perf/pmu-events/arch/<target arch>
> directory in the kernel source tree.
> 
> Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
> ---
>  package/linux-tools/linux-tool-perf.mk.in | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/package/linux-tools/linux-tool-perf.mk.in b/package/linux-tools/linux-tool-perf.mk.in
> index 057c6f80e7..44c1bbdf83 100644
> --- a/package/linux-tools/linux-tool-perf.mk.in
> +++ b/package/linux-tools/linux-tool-perf.mk.in
> @@ -6,7 +6,7 @@
>  
>  LINUX_TOOLS += perf
>  
> -PERF_DEPENDENCIES = host-flex host-bison
> +PERF_DEPENDENCIES = host-flex host-bison host-python3
>  
>  ifeq ($(NORMALIZED_ARCH),x86_64)
>  PERF_ARCH=x86
> -- 
> 2.37.1
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Arnout Vandecappelle]

On 05/09/2022 22:17, Yann E. MORIN wrote:
> Sergey, All,
> 
> On 2022-09-04 23:57 +0300, Sergey Matyukevich spake thusly:
>> Linux kernel commit 00facc760903 ("perf jevents: Switch build to use
>> jevents.py")
> 
> 00facc760903 is only parts of 6.0-rc kernels so far, so unless one uses
> bleeding edge technology, there is no issue. So, paying the price of a
> host-pytohn3 build just for release-candidate kernels is a bit too much
> in my opinion.
> 
> So, I added a config knob to drive this, like we have for host-openssl
> and a few other host programs for building the kernel.

  If only python is needed, no modules, then $(BR2_PYTHON3_HOST_DEPENDENCY) 
should be used instead. That checks for python 3.4 but I think that that is fine 
for the kernel as well. If not, we should probably bump the minimum system 
python3 version.

  Regards,
  Arnout

> 
> Applied to master with the above change, thanks.
> 
> Regards,
> Yann E. MORIN.
> 
>> switched to auto-generation of arch-specific PMU events
>> using python script. Now custom PMU events for different platforms of
>> the selected target architecture are not embedded into perf binary if
>> an appropriate host python interpreter is not present. In practice it
>> means that perf is successfully built, but 'perf list pmu' will show
>> no custom events on a target platform even if those events are supported
>> and properly defined in tools/perf/pmu-events/arch/<target arch>
>> directory in the kernel source tree.
>>
>> Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
>> ---
>>   package/linux-tools/linux-tool-perf.mk.in | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/package/linux-tools/linux-tool-perf.mk.in b/package/linux-tools/linux-tool-perf.mk.in
>> index 057c6f80e7..44c1bbdf83 100644
>> --- a/package/linux-tools/linux-tool-perf.mk.in
>> +++ b/package/linux-tools/linux-tool-perf.mk.in
>> @@ -6,7 +6,7 @@
>>   
>>   LINUX_TOOLS += perf
>>   
>> -PERF_DEPENDENCIES = host-flex host-bison
>> +PERF_DEPENDENCIES = host-flex host-bison host-python3
>>   
>>   ifeq ($(NORMALIZED_ARCH),x86_64)
>>   PERF_ARCH=x86
>> -- 
>> 2.37.1
>>
>> _______________________________________________
>> buildroot mailing list
>> buildroot@buildroot.org
>> https://lists.buildroot.org/mailman/listinfo/buildroot
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Yann E. MORIN]

Arnout, All,

On 2022-09-06 21:46 +0200, Arnout Vandecappelle spake thusly:
> On 05/09/2022 22:17, Yann E. MORIN wrote:
> >On 2022-09-04 23:57 +0300, Sergey Matyukevich spake thusly:
> >>Linux kernel commit 00facc760903 ("perf jevents: Switch build to use
> >>jevents.py")
> >00facc760903 is only parts of 6.0-rc kernels so far, so unless one uses
> >bleeding edge technology, there is no issue. So, paying the price of a
> >host-pytohn3 build just for release-candidate kernels is a bit too much
> >in my opinion.
> >So, I added a config knob to drive this, like we have for host-openssl
> >and a few other host programs for building the kernel.
>  If only python is needed, no modules, then $(BR2_PYTHON3_HOST_DEPENDENCY)

Dang, that's right, I totally forgot about that one...

> should be used instead. That checks for python 3.4 but I think that that is
> fine for the kernel as well. If not, we should probably bump the minimum
> system python3 version.

The imports are:

    import argparse
    import csv
    import json
    import os
    import sys
    from typing import (Callable, Dict, Optional, Sequence, Set, Tuple)
    import collections

I think hey all are from the stdlib, so no external module needed

So, indeed, BR2_PYTHON3_HOST_DEPENDENCY seems a much better fit...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Sergey Matyukevich]

Hi Arnout, Yann, and all

> On 2022-09-06 21:46 +0200, Arnout Vandecappelle spake thusly:
> > On 05/09/2022 22:17, Yann E. MORIN wrote:
> > >On 2022-09-04 23:57 +0300, Sergey Matyukevich spake thusly:
> > >>Linux kernel commit 00facc760903 ("perf jevents: Switch build to use
> > >>jevents.py")
> > >00facc760903 is only parts of 6.0-rc kernels so far, so unless one uses
> > >bleeding edge technology, there is no issue. So, paying the price of a
> > >host-pytohn3 build just for release-candidate kernels is a bit too much
> > >in my opinion.
> > >So, I added a config knob to drive this, like we have for host-openssl
> > >and a few other host programs for building the kernel.
> >  If only python is needed, no modules, then $(BR2_PYTHON3_HOST_DEPENDENCY)
> 
> Dang, that's right, I totally forgot about that one...
> 
> > should be used instead. That checks for python 3.4 but I think that that is
> > fine for the kernel as well. If not, we should probably bump the minimum
> > system python3 version.
> 
> The imports are:
> 
>     import argparse
>     import csv
>     import json
>     import os
>     import sys
>     from typing import (Callable, Dict, Optional, Sequence, Set, Tuple)
>     import collections
> 
> I think hey all are from the stdlib, so no external module needed
> 
> So, indeed, BR2_PYTHON3_HOST_DEPENDENCY seems a much better fit...

It looks like Python 3.4 is not enough in this specific case. Looking
into the kernel commit that introduced jevents.py, it checks for Python
3.6 with the following comment: "jevents.py uses f-strings present in
Python 3.6 released in Dec. 2016."

Regards,
Sergey

Regards,
Sergey
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Yann E. MORIN]

Arnout, All,

On 2022-09-06 22:07 +0200, Yann E. MORIN spake thusly:
> On 2022-09-06 21:46 +0200, Arnout Vandecappelle spake thusly:
> > On 05/09/2022 22:17, Yann E. MORIN wrote:
> > >On 2022-09-04 23:57 +0300, Sergey Matyukevich spake thusly:
> > >>Linux kernel commit 00facc760903 ("perf jevents: Switch build to use
> > >>jevents.py")
> > >00facc760903 is only parts of 6.0-rc kernels so far, so unless one uses
> > >bleeding edge technology, there is no issue. So, paying the price of a
> > >host-pytohn3 build just for release-candidate kernels is a bit too much
> > >in my opinion.
> > >So, I added a config knob to drive this, like we have for host-openssl
> > >and a few other host programs for building the kernel.
> >  If only python is needed, no modules, then $(BR2_PYTHON3_HOST_DEPENDENCY)
> 
> Dang, that's right, I totally forgot about that one...
[--SNIP--]
> So, indeed, BR2_PYTHON3_HOST_DEPENDENCY seems a much better fit...

Err, hold on, no: we still do not want to pay the price for a
host-python3 build if the local system does not have python-3.4+ and the
kernel does not even need it, in fact.

So, I think we want to keep BR2_PACKAGE_LINUX_TOOLS_PERF_NEEDS_HOST_PYTHON3
but switch the _DEPENDENCY to BR2_PYTHON3_HOST_DEPENDENCY.

Thoughts?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH 1/1] package/linux-tools: perf: add host-python3 dependency

[Arnout Vandecappelle]

On 06/09/2022 22:37, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2022-09-06 22:07 +0200, Yann E. MORIN spake thusly:
>> On 2022-09-06 21:46 +0200, Arnout Vandecappelle spake thusly:
>>> On 05/09/2022 22:17, Yann E. MORIN wrote:
>>>> On 2022-09-04 23:57 +0300, Sergey Matyukevich spake thusly:
>>>>> Linux kernel commit 00facc760903 ("perf jevents: Switch build to use
>>>>> jevents.py")
>>>> 00facc760903 is only parts of 6.0-rc kernels so far, so unless one uses
>>>> bleeding edge technology, there is no issue. So, paying the price of a
>>>> host-pytohn3 build just for release-candidate kernels is a bit too much
>>>> in my opinion.
>>>> So, I added a config knob to drive this, like we have for host-openssl
>>>> and a few other host programs for building the kernel.
>>>   If only python is needed, no modules, then $(BR2_PYTHON3_HOST_DEPENDENCY)
>>
>> Dang, that's right, I totally forgot about that one...
> [--SNIP--]
>> So, indeed, BR2_PYTHON3_HOST_DEPENDENCY seems a much better fit...
> 
> Err, hold on, no: we still do not want to pay the price for a
> host-python3 build if the local system does not have python-3.4+ and the
> kernel does not even need it, in fact.
> 
> So, I think we want to keep BR2_PACKAGE_LINUX_TOOLS_PERF_NEEDS_HOST_PYTHON3
> but switch the _DEPENDENCY to BR2_PYTHON3_HOST_DEPENDENCY.

  Yes, that's what I meant. The place where I added my comment was maybe not ideal.

  Regards,
  Arnout

> 
> Thoughts?
> 
> Regards,
> Yann E. MORIN.
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot