* Re: [PATCH v3] perf: Correct the substitution of python shebangs
2020-02-14 3:57 [PATCH v3] perf: Correct the substitution of python shebangs zhe.he
@ 2020-02-14 4:33 ` Chen, Qi
2020-02-14 13:18 ` Bruce Ashfield
2020-02-21 21:43 ` Joshua Watt
2 siblings, 0 replies; 8+ messages in thread
From: Chen, Qi @ 2020-02-14 4:33 UTC (permalink / raw)
To: He, Zhe, openembedded-core@lists.openembedded.org
TA
________________________________________
From: openembedded-core-bounces@lists.openembedded.org [openembedded-core-bounces@lists.openembedded.org] on behalf of zhe.he@windriver.com [zhe.he@windriver.com]
Sent: Friday, February 14, 2020 11:57 AM
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH v3] perf: Correct the substitution of python shebangs
From: He Zhe <zhe.he@windriver.com>
To make the native python3 is always used,
- Use sed one-liner instead
- Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the
following warning.
File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata'
This issue is first reported by Joel Stanley <joel@jms.id.au>
The sed one-liner is credited to Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: He Zhe <zhe.he@windriver.com>
---
meta/recipes-kernel/perf/perf.bb | 7 ++-----
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 6d1b06693d..a6fb51d3db 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -239,11 +239,8 @@ do_configure_prepend () {
fi
# use /usr/bin/env instead of version specific python
- for s in `find ${S}/tools/perf/ -name '*.py'`; do
- sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}"
- sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}"
- sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}"
- sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}"
+ for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do
+ sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s}
done
# unistd.h can be out of sync between libc-headers and the captured version in the perf source
--
2.24.1
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH v3] perf: Correct the substitution of python shebangs
2020-02-14 3:57 [PATCH v3] perf: Correct the substitution of python shebangs zhe.he
2020-02-14 4:33 ` Chen, Qi
@ 2020-02-14 13:18 ` Bruce Ashfield
2020-02-14 18:09 ` Richard Purdie
2020-02-21 21:43 ` Joshua Watt
2 siblings, 1 reply; 8+ messages in thread
From: Bruce Ashfield @ 2020-02-14 13:18 UTC (permalink / raw)
To: He Zhe; +Cc: Patches and discussions about the oe-core layer
On Thu, Feb 13, 2020 at 10:58 PM <zhe.he@windriver.com> wrote:
>
> From: He Zhe <zhe.he@windriver.com>
>
> To make the native python3 is always used,
>
> - Use sed one-liner instead
> - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the
> following warning.
>
> File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
> _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
> ModuleNotFoundError: No module named '_sysconfigdata'
>
> This issue is first reported by Joel Stanley <joel@jms.id.au>
> The sed one-liner is credited to Anuj Mittal <anuj.mittal@intel.com>
Thanks!
This looks good to me.
Cheers,
Bruce
>
> Signed-off-by: He Zhe <zhe.he@windriver.com>
> ---
> meta/recipes-kernel/perf/perf.bb | 7 ++-----
> 1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> index 6d1b06693d..a6fb51d3db 100644
> --- a/meta/recipes-kernel/perf/perf.bb
> +++ b/meta/recipes-kernel/perf/perf.bb
> @@ -239,11 +239,8 @@ do_configure_prepend () {
> fi
>
> # use /usr/bin/env instead of version specific python
> - for s in `find ${S}/tools/perf/ -name '*.py'`; do
> - sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}"
> - sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}"
> - sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}"
> - sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}"
> + for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do
> + sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s}
> done
>
> # unistd.h can be out of sync between libc-headers and the captured version in the perf source
> --
> 2.24.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v3] perf: Correct the substitution of python shebangs
2020-02-14 13:18 ` Bruce Ashfield
@ 2020-02-14 18:09 ` Richard Purdie
0 siblings, 0 replies; 8+ messages in thread
From: Richard Purdie @ 2020-02-14 18:09 UTC (permalink / raw)
To: Bruce Ashfield, He Zhe; +Cc: Patches and discussions about the oe-core layer
On Fri, 2020-02-14 at 08:18 -0500, Bruce Ashfield wrote:
> On Thu, Feb 13, 2020 at 10:58 PM <zhe.he@windriver.com> wrote:
> > From: He Zhe <zhe.he@windriver.com>
> >
> > To make the native python3 is always used,
> >
> > - Use sed one-liner instead
> > - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the
> > following warning.
> >
> > File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
> > _temp = __import__(name, globals(), locals(), ['build_time_vars'],
> > 0)
> > ModuleNotFoundError: No module named '_sysconfigdata'
> >
> > This issue is first reported by Joel Stanley <joel@jms.id.au>
> > The sed one-liner is credited to Anuj Mittal <anuj.mittal@intel.com
> > >
>
> Thanks!
>
> This looks good to me.
I'm not sure how but with this applied we see breaking kernel module
tests:
https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/1555
https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/275
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/1559
(mips/sysvinit, mips/systemd and mips64)
I'll remove this patch from -next and retest see if I can confirm it is
this change somehow.
Cheers,
Richard
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] perf: Correct the substitution of python shebangs
2020-02-14 3:57 [PATCH v3] perf: Correct the substitution of python shebangs zhe.he
2020-02-14 4:33 ` Chen, Qi
2020-02-14 13:18 ` Bruce Ashfield
@ 2020-02-21 21:43 ` Joshua Watt
2020-02-21 22:15 ` Khem Raj
2 siblings, 1 reply; 8+ messages in thread
From: Joshua Watt @ 2020-02-21 21:43 UTC (permalink / raw)
To: zhe.he; +Cc: OE-core
On Thu, Feb 13, 2020 at 9:58 PM <zhe.he@windriver.com> wrote:
>
> From: He Zhe <zhe.he@windriver.com>
>
> To make the native python3 is always used,
>
> - Use sed one-liner instead
> - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the
> following warning.
>
> File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
> _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
> ModuleNotFoundError: No module named '_sysconfigdata'
>
> This issue is first reported by Joel Stanley <joel@jms.id.au>
> The sed one-liner is credited to Anuj Mittal <anuj.mittal@intel.com>
>
> Signed-off-by: He Zhe <zhe.he@windriver.com>
> ---
> meta/recipes-kernel/perf/perf.bb | 7 ++-----
> 1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> index 6d1b06693d..a6fb51d3db 100644
> --- a/meta/recipes-kernel/perf/perf.bb
> +++ b/meta/recipes-kernel/perf/perf.bb
> @@ -239,11 +239,8 @@ do_configure_prepend () {
> fi
>
> # use /usr/bin/env instead of version specific python
> - for s in `find ${S}/tools/perf/ -name '*.py'`; do
> - sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}"
> - sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}"
> - sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}"
> - sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}"
> + for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do
Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older
versions (e.g. 4.14) and causes do_configure to fail.
> + sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s}
> done
>
> # unistd.h can be out of sync between libc-headers and the captured version in the perf source
> --
> 2.24.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v3] perf: Correct the substitution of python shebangs
2020-02-21 21:43 ` Joshua Watt
@ 2020-02-21 22:15 ` Khem Raj
2020-02-21 22:23 ` Bruce Ashfield
0 siblings, 1 reply; 8+ messages in thread
From: Khem Raj @ 2020-02-21 22:15 UTC (permalink / raw)
To: Joshua Watt, zhe.he; +Cc: OE-core
On 2/21/20 1:43 PM, Joshua Watt wrote:
> On Thu, Feb 13, 2020 at 9:58 PM <zhe.he@windriver.com> wrote:
>>
>> From: He Zhe <zhe.he@windriver.com>
>>
>> To make the native python3 is always used,
>>
>> - Use sed one-liner instead
>> - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the
>> following warning.
>>
>> File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
>> _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
>> ModuleNotFoundError: No module named '_sysconfigdata'
>>
>> This issue is first reported by Joel Stanley <joel@jms.id.au>
>> The sed one-liner is credited to Anuj Mittal <anuj.mittal@intel.com>
>>
>> Signed-off-by: He Zhe <zhe.he@windriver.com>
>> ---
>> meta/recipes-kernel/perf/perf.bb | 7 ++-----
>> 1 file changed, 2 insertions(+), 5 deletions(-)
>>
>> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
>> index 6d1b06693d..a6fb51d3db 100644
>> --- a/meta/recipes-kernel/perf/perf.bb
>> +++ b/meta/recipes-kernel/perf/perf.bb
>> @@ -239,11 +239,8 @@ do_configure_prepend () {
>> fi
>>
>> # use /usr/bin/env instead of version specific python
>> - for s in `find ${S}/tools/perf/ -name '*.py'`; do
>> - sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}"
>> - sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}"
>> - sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}"
>> - sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}"
>> + for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do
>
> Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older
> versions (e.g. 4.14) and causes do_configure to fail.
Probably not, I also stumbled into this problem.
>
>> + sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s}
>> done
>>
>> # unistd.h can be out of sync between libc-headers and the captured version in the perf source
>> --
>> 2.24.1
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v3] perf: Correct the substitution of python shebangs
2020-02-21 22:15 ` Khem Raj
@ 2020-02-21 22:23 ` Bruce Ashfield
2020-02-21 22:30 ` Khem Raj
0 siblings, 1 reply; 8+ messages in thread
From: Bruce Ashfield @ 2020-02-21 22:23 UTC (permalink / raw)
To: Khem Raj; +Cc: OE-core
On Fri, Feb 21, 2020 at 5:16 PM Khem Raj <raj.khem@gmail.com> wrote:
>
>
>
> On 2/21/20 1:43 PM, Joshua Watt wrote:
> > On Thu, Feb 13, 2020 at 9:58 PM <zhe.he@windriver.com> wrote:
> >>
> >> From: He Zhe <zhe.he@windriver.com>
> >>
> >> To make the native python3 is always used,
> >>
> >> - Use sed one-liner instead
> >> - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the
> >> following warning.
> >>
> >> File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
> >> _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
> >> ModuleNotFoundError: No module named '_sysconfigdata'
> >>
> >> This issue is first reported by Joel Stanley <joel@jms.id.au>
> >> The sed one-liner is credited to Anuj Mittal <anuj.mittal@intel.com>
> >>
> >> Signed-off-by: He Zhe <zhe.he@windriver.com>
> >> ---
> >> meta/recipes-kernel/perf/perf.bb | 7 ++-----
> >> 1 file changed, 2 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> >> index 6d1b06693d..a6fb51d3db 100644
> >> --- a/meta/recipes-kernel/perf/perf.bb
> >> +++ b/meta/recipes-kernel/perf/perf.bb
> >> @@ -239,11 +239,8 @@ do_configure_prepend () {
> >> fi
> >>
> >> # use /usr/bin/env instead of version specific python
> >> - for s in `find ${S}/tools/perf/ -name '*.py'`; do
> >> - sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}"
> >> - sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}"
> >> - sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}"
> >> - sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}"
> >> + for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do
> >
> > Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older
> > versions (e.g. 4.14) and causes do_configure to fail.
>
> Probably not, I also stumbled into this problem.
Yah,
We can either just do the entire directory (but during the review, we
were talking about the fact that since we aren't using them, we don't
actually know if they are py3 safe, so blindly converting the shebang
isnt' a great idea either).
But the solution we use in the devsrc recipe would work easily enough
here, just do $(find . -name ${S}/scripts/bpf_helpers_doc.py) and let
it return nothing for those older kernels.
Bruce
>
> >
> >> + sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s}
> >> done
> >>
> >> # unistd.h can be out of sync between libc-headers and the captured version in the perf source
> >> --
> >> 2.24.1
> >>
> >> --
> >> _______________________________________________
> >> Openembedded-core mailing list
> >> Openembedded-core@lists.openembedded.org
> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v3] perf: Correct the substitution of python shebangs
2020-02-21 22:23 ` Bruce Ashfield
@ 2020-02-21 22:30 ` Khem Raj
0 siblings, 0 replies; 8+ messages in thread
From: Khem Raj @ 2020-02-21 22:30 UTC (permalink / raw)
To: Bruce Ashfield; +Cc: OE-core
On 2/21/20 2:23 PM, Bruce Ashfield wrote:
> On Fri, Feb 21, 2020 at 5:16 PM Khem Raj <raj.khem@gmail.com> wrote:
>>
>>
>>
>> On 2/21/20 1:43 PM, Joshua Watt wrote:
>>> On Thu, Feb 13, 2020 at 9:58 PM <zhe.he@windriver.com> wrote:
>>>>
>>>> From: He Zhe <zhe.he@windriver.com>
>>>>
>>>> To make the native python3 is always used,
>>>>
>>>> - Use sed one-liner instead
>>>> - Add substitution for ${S}/scripts/bpf_helpers_doc.py to fix the
>>>> following warning.
>>>>
>>>> File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
>>>> _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
>>>> ModuleNotFoundError: No module named '_sysconfigdata'
>>>>
>>>> This issue is first reported by Joel Stanley <joel@jms.id.au>
>>>> The sed one-liner is credited to Anuj Mittal <anuj.mittal@intel.com>
>>>>
>>>> Signed-off-by: He Zhe <zhe.he@windriver.com>
>>>> ---
>>>> meta/recipes-kernel/perf/perf.bb | 7 ++-----
>>>> 1 file changed, 2 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
>>>> index 6d1b06693d..a6fb51d3db 100644
>>>> --- a/meta/recipes-kernel/perf/perf.bb
>>>> +++ b/meta/recipes-kernel/perf/perf.bb
>>>> @@ -239,11 +239,8 @@ do_configure_prepend () {
>>>> fi
>>>>
>>>> # use /usr/bin/env instead of version specific python
>>>> - for s in `find ${S}/tools/perf/ -name '*.py'`; do
>>>> - sed -i 's,/usr/bin/python,/usr/bin/env python3,' "${s}"
>>>> - sed -i 's,/usr/bin/python2,/usr/bin/env python3,' "${s}"
>>>> - sed -i 's,/usr/bin/env python2,/usr/bin/env python3,' "${s}"
>>>> - sed -i 's,/usr/bin/python3,/usr/bin/env python3,' "${s}"
>>>> + for s in `find ${S}/tools/perf/ -name '*.py'` ${S}/scripts/bpf_helpers_doc.py; do
>>>
>>> Do we have to hard code bpf_helpers_doc.py? It doesn't exist in older
>>> versions (e.g. 4.14) and causes do_configure to fail.
>>
>> Probably not, I also stumbled into this problem.
>
> Yah,
>
> We can either just do the entire directory (but during the review, we
> were talking about the fact that since we aren't using them, we don't
> actually know if they are py3 safe, so blindly converting the shebang
> isnt' a great idea either).
>
> But the solution we use in the devsrc recipe would work easily enough
> here, just do $(find . -name ${S}/scripts/bpf_helpers_doc.py) and let
> it return nothing for those older kernels.
then perhaps its better to callout each script by name and check if that
file exists, so older kernels can ignore that
>
> Bruce
>
>>
>>>
>>>> + sed -i -e "s,#!.*python.*,#!${USRBINPATH}/env python3," ${s}
>>>> done
>>>>
>>>> # unistd.h can be out of sync between libc-headers and the captured version in the perf source
>>>> --
>>>> 2.24.1
>>>>
>>>> --
>>>> _______________________________________________
>>>> Openembedded-core mailing list
>>>> Openembedded-core@lists.openembedded.org
>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread