[meta-oe][PATCH] itstool: Use python3 binary fom PATH instead of /usr/bin/python3

[meta-oe][PATCH] itstool: Use python3 binary fom PATH instead of /usr/bin/python3

[Piotr Tworek]

Although /usr/bin/python3 works fine for class-target builds its not
really what we want for class-native. The host system might not even
have python3 installed. The native itstool version should use pythn3
provided in recipe-sysroot-native. To accomplish it modify the default
shebang to use /usr/bin/env. This works as expected for both native and
target builds.

Signed-off-by: Piotr Tworek <tworaz@tworaz.net>
---
 meta-oe/recipes-support/itstool/itstool_2.0.5.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
index a0af44a71..f21e7f39b 100644
--- a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
+++ b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
@@ -13,7 +13,7 @@ SRC_URI[sha256sum] = "100506f8df62cca6225ec3e631a8237e9c04650c77495af4919ac6a100
 
 do_install_append() {
     # fix shebang of main script
-    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/${PYTHON_PN} -s:' ${D}${bindir}/itstool
+    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/env -S ${PYTHON_PN} -s:' ${D}${bindir}/itstool
 }
 
 BBCLASSEXTEND = "native"
-- 
2.21.0

Re: [meta-oe][PATCH] itstool: Use python3 binary fom PATH instead of /usr/bin/python3

[Burton, Ross]

Considering:

1) bitbake is written in Python 3
2) HOSTTOOLS contains python3

There *really* is a host python 3 binary, and the canonical path to it
is ${HOSTTOOLS_DIR}/python3.

Ross

On Fri, 26 Jul 2019 at 16:40, Piotr Tworek <tworaz@tworaz.net> wrote:
>
> Although /usr/bin/python3 works fine for class-target builds its not
> really what we want for class-native. The host system might not even
> have python3 installed. The native itstool version should use pythn3
> provided in recipe-sysroot-native. To accomplish it modify the default
> shebang to use /usr/bin/env. This works as expected for both native and
> target builds.
>
> Signed-off-by: Piotr Tworek <tworaz@tworaz.net>
> ---
>  meta-oe/recipes-support/itstool/itstool_2.0.5.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
> index a0af44a71..f21e7f39b 100644
> --- a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
> +++ b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
> @@ -13,7 +13,7 @@ SRC_URI[sha256sum] = "100506f8df62cca6225ec3e631a8237e9c04650c77495af4919ac6a100
>
>  do_install_append() {
>      # fix shebang of main script
> -    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/${PYTHON_PN} -s:' ${D}${bindir}/itstool
> +    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/env -S ${PYTHON_PN} -s:' ${D}${bindir}/itstool
>  }
>
>  BBCLASSEXTEND = "native"
> --
> 2.21.0
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [meta-oe][PATCH] itstool: Use python3 binary fom PATH instead of /usr/bin/python3

[Piotr Tworek]

Hi Ross,

Thanks for looking at my patch. You're right that my explanation behind 
it is a wrong, I was a bit lazy there. Its not really about python3 not 
being installed on the host system. My patch does fix a real issue with 
itstool-native which is currently unusable. The main problem is we can't 
mix host and OE provided python3 usage in one recipe. The problem I 
found is that if  the recipe inherits python3native bbclass any use of 
system provided python3 will fail. This seems to happen because 
python3native bbclass exports 
_PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata". If I try to invoke host 
python3 binary with this export it fails to start.complaining it can't 
find module named _sysconfigdata.

You might ask why nobody noticed it before. The explanation is pretty 
simple. In meta-oe there are two recipes which depend on itstool-native. 
Those are gnome-desktop3 and gnome-system-monitor. Both recipes inherit 
meson class which in turn inherit python3native so in theory they should 
fail. This does not happen because both recipes actually disable 
document generation steps for which itstool is invoked. 
Gnome-system-monitor has a patch for it while gnome-desktop3 has 
desktop_docs=false in EXTRA_OEMESON. I haven't tried it yet, but with my 
patch the document generation step should probably work fine in both 
cases. This was the case for some other gnome related recipes which I 
have in my own custom layer.

Is there anything besides the explanation that is wrong with my patch?

/ptw

On 26/07/2019 17:57, Burton, Ross wrote:
> Considering:
>
> 1) bitbake is written in Python 3
> 2) HOSTTOOLS contains python3
>
> There *really* is a host python 3 binary, and the canonical path to it
> is ${HOSTTOOLS_DIR}/python3.
>
> Ross
>
> On Fri, 26 Jul 2019 at 16:40, Piotr Tworek <tworaz@tworaz.net> wrote:
>> Although /usr/bin/python3 works fine for class-target builds its not
>> really what we want for class-native. The host system might not even
>> have python3 installed. The native itstool version should use pythn3
>> provided in recipe-sysroot-native. To accomplish it modify the default
>> shebang to use /usr/bin/env. This works as expected for both native and
>> target builds.
>>
>> Signed-off-by: Piotr Tworek <tworaz@tworaz.net>
>> ---
>>   meta-oe/recipes-support/itstool/itstool_2.0.5.bb | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
>> index a0af44a71..f21e7f39b 100644
>> --- a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
>> +++ b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
>> @@ -13,7 +13,7 @@ SRC_URI[sha256sum] = "100506f8df62cca6225ec3e631a8237e9c04650c77495af4919ac6a100
>>
>>   do_install_append() {
>>       # fix shebang of main script
>> -    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/${PYTHON_PN} -s:' ${D}${bindir}/itstool
>> +    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/env -S ${PYTHON_PN} -s:' ${D}${bindir}/itstool
>>   }
>>
>>   BBCLASSEXTEND = "native"
>> --
>> 2.21.0
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [meta-oe][PATCH] itstool: Use python3 binary fom PATH instead of /usr/bin/python3

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 4151 bytes --]

I think this should be fine, as long as you:
1. Modify the commit to fix the original place in the source code via a
SRC_URI patch (instead of doing it after the fact with sed - such ad hoc
patching is hard to read and understand).
2. Write a better description: we need to preserve explanations like this
in commit history.

Alex

On Fri, 26 Jul 2019 at 21:31, Piotr Tworek <tworaz@tworaz.net> wrote:

> Hi Ross,
>
> Thanks for looking at my patch. You're right that my explanation behind
> it is a wrong, I was a bit lazy there. Its not really about python3 not
> being installed on the host system. My patch does fix a real issue with
> itstool-native which is currently unusable. The main problem is we can't
> mix host and OE provided python3 usage in one recipe. The problem I
> found is that if  the recipe inherits python3native bbclass any use of
> system provided python3 will fail. This seems to happen because
> python3native bbclass exports
> _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata". If I try to invoke host
> python3 binary with this export it fails to start.complaining it can't
> find module named _sysconfigdata.
>
> You might ask why nobody noticed it before. The explanation is pretty
> simple. In meta-oe there are two recipes which depend on itstool-native.
> Those are gnome-desktop3 and gnome-system-monitor. Both recipes inherit
> meson class which in turn inherit python3native so in theory they should
> fail. This does not happen because both recipes actually disable
> document generation steps for which itstool is invoked.
> Gnome-system-monitor has a patch for it while gnome-desktop3 has
> desktop_docs=false in EXTRA_OEMESON. I haven't tried it yet, but with my
> patch the document generation step should probably work fine in both
> cases. This was the case for some other gnome related recipes which I
> have in my own custom layer.
>
> Is there anything besides the explanation that is wrong with my patch?
>
> /ptw
>
> On 26/07/2019 17:57, Burton, Ross wrote:
> > Considering:
> >
> > 1) bitbake is written in Python 3
> > 2) HOSTTOOLS contains python3
> >
> > There *really* is a host python 3 binary, and the canonical path to it
> > is ${HOSTTOOLS_DIR}/python3.
> >
> > Ross
> >
> > On Fri, 26 Jul 2019 at 16:40, Piotr Tworek <tworaz@tworaz.net> wrote:
> >> Although /usr/bin/python3 works fine for class-target builds its not
> >> really what we want for class-native. The host system might not even
> >> have python3 installed. The native itstool version should use pythn3
> >> provided in recipe-sysroot-native. To accomplish it modify the default
> >> shebang to use /usr/bin/env. This works as expected for both native and
> >> target builds.
> >>
> >> Signed-off-by: Piotr Tworek <tworaz@tworaz.net>
> >> ---
> >>   meta-oe/recipes-support/itstool/itstool_2.0.5.bb | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
> b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
> >> index a0af44a71..f21e7f39b 100644
> >> --- a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
> >> +++ b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
> >> @@ -13,7 +13,7 @@ SRC_URI[sha256sum] =
> "100506f8df62cca6225ec3e631a8237e9c04650c77495af4919ac6a100
> >>
> >>   do_install_append() {
> >>       # fix shebang of main script
> >> -    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN}
> -s:#!${bindir_native}/${PYTHON_PN} -s:' ${D}${bindir}/itstool
> >> +    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/env -S
> ${PYTHON_PN} -s:' ${D}${bindir}/itstool
> >>   }
> >>
> >>   BBCLASSEXTEND = "native"
> >> --
> >> 2.21.0
> >>
> >> --
> >> _______________________________________________
> >> Openembedded-core mailing list
> >> Openembedded-core@lists.openembedded.org
> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 5838 bytes --]

Re: [meta-oe][PATCH] itstool: Use python3 binary fom PATH instead of /usr/bin/python3

[Khem Raj]

and send to oe-devel ml please

On Fri, Jul 26, 2019 at 11:50 AM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
>
> I think this should be fine, as long as you:
> 1. Modify the commit to fix the original place in the source code via a SRC_URI patch (instead of doing it after the fact with sed - such ad hoc patching is hard to read and understand).
> 2. Write a better description: we need to preserve explanations like this in commit history.
>
> Alex
>
> On Fri, 26 Jul 2019 at 21:31, Piotr Tworek <tworaz@tworaz.net> wrote:
>>
>> Hi Ross,
>>
>> Thanks for looking at my patch. You're right that my explanation behind
>> it is a wrong, I was a bit lazy there. Its not really about python3 not
>> being installed on the host system. My patch does fix a real issue with
>> itstool-native which is currently unusable. The main problem is we can't
>> mix host and OE provided python3 usage in one recipe. The problem I
>> found is that if  the recipe inherits python3native bbclass any use of
>> system provided python3 will fail. This seems to happen because
>> python3native bbclass exports
>> _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata". If I try to invoke host
>> python3 binary with this export it fails to start.complaining it can't
>> find module named _sysconfigdata.
>>
>> You might ask why nobody noticed it before. The explanation is pretty
>> simple. In meta-oe there are two recipes which depend on itstool-native.
>> Those are gnome-desktop3 and gnome-system-monitor. Both recipes inherit
>> meson class which in turn inherit python3native so in theory they should
>> fail. This does not happen because both recipes actually disable
>> document generation steps for which itstool is invoked.
>> Gnome-system-monitor has a patch for it while gnome-desktop3 has
>> desktop_docs=false in EXTRA_OEMESON. I haven't tried it yet, but with my
>> patch the document generation step should probably work fine in both
>> cases. This was the case for some other gnome related recipes which I
>> have in my own custom layer.
>>
>> Is there anything besides the explanation that is wrong with my patch?
>>
>> /ptw
>>
>> On 26/07/2019 17:57, Burton, Ross wrote:
>> > Considering:
>> >
>> > 1) bitbake is written in Python 3
>> > 2) HOSTTOOLS contains python3
>> >
>> > There *really* is a host python 3 binary, and the canonical path to it
>> > is ${HOSTTOOLS_DIR}/python3.
>> >
>> > Ross
>> >
>> > On Fri, 26 Jul 2019 at 16:40, Piotr Tworek <tworaz@tworaz.net> wrote:
>> >> Although /usr/bin/python3 works fine for class-target builds its not
>> >> really what we want for class-native. The host system might not even
>> >> have python3 installed. The native itstool version should use pythn3
>> >> provided in recipe-sysroot-native. To accomplish it modify the default
>> >> shebang to use /usr/bin/env. This works as expected for both native and
>> >> target builds.
>> >>
>> >> Signed-off-by: Piotr Tworek <tworaz@tworaz.net>
>> >> ---
>> >>   meta-oe/recipes-support/itstool/itstool_2.0.5.bb | 2 +-
>> >>   1 file changed, 1 insertion(+), 1 deletion(-)
>> >>
>> >> diff --git a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
>> >> index a0af44a71..f21e7f39b 100644
>> >> --- a/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
>> >> +++ b/meta-oe/recipes-support/itstool/itstool_2.0.5.bb
>> >> @@ -13,7 +13,7 @@ SRC_URI[sha256sum] = "100506f8df62cca6225ec3e631a8237e9c04650c77495af4919ac6a100
>> >>
>> >>   do_install_append() {
>> >>       # fix shebang of main script
>> >> -    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/${PYTHON_PN} -s:' ${D}${bindir}/itstool
>> >> +    sed -i 's:^#!${WORKDIR}.*${PYTHON_PN} -s:#!${bindir_native}/env -S ${PYTHON_PN} -s:' ${D}${bindir}/itstool
>> >>   }
>> >>
>> >>   BBCLASSEXTEND = "native"
>> >> --
>> >> 2.21.0
>> >>
>> >> --
>> >> _______________________________________________
>> >> Openembedded-core mailing list
>> >> Openembedded-core@lists.openembedded.org
>> >> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

✗ patchtest: failure for itstool: Use python3 binary fom PATH instead of /usr/bin/python3

[Patchwork]

== Series Details ==

Series: itstool: Use python3 binary fom PATH instead of /usr/bin/python3
Revision: 1
URL   : https://patchwork.openembedded.org/series/18916/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Patch            [meta-oe] itstool: Use python3 binary fom PATH instead of /usr/bin/python3
 Issue             Series sent to the wrong mailing list [test_target_mailing_list] 
  Suggested fix    Check the project's README (meta-oe) and send the patch to the indicated list

* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 5b9a4214ee)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe