[PATCH v2 0/1] Fix buildtools after git-perltools split

[PATCH v2 0/1] Fix buildtools after git-perltools split

[Paul Eggleton]

The following change since commit f7b342a73661cfff4043fab77ab8e9ef0e48dec4:

  gdb: disable Guile (2014-12-03 12:22:42 +0000)

is available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib paule/git-perltools
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/git-perltools

Paul Eggleton (1):
  buildtools-tarball: restore missing git tools

 meta/recipes-core/meta/buildtools-tarball.bb          |  4 ++++
 .../meta/nativesdk-buildtools-perl-dummy.bb           | 19 +++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb

-- 
1.9.3

[PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Paul Eggleton]

Since the split out of git-perltools, some git tools (such as "git am",
"git send-email" and "git-submodule") have no longer been part of the
buildtools. We need these, so add them back in.

However, adding git-perltools to buildtools triggers perl itself being
brought into buildtools as well, and we don't want that; but we also
don't want to have to hack the git recipe or indeed anything else that
starts depending on perl. Thus, add a dummy package which gets installed
in its place, in a separate package architecture that is only enabled
for buildtools to ensure it doesn't start appearing in place of
nativesdk-perl anywhere else.

Fixes [YOCTO #7033].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/recipes-core/meta/buildtools-tarball.bb          |  4 ++++
 .../meta/nativesdk-buildtools-perl-dummy.bb           | 19 +++++++++++++++++++
 2 files changed, 23 insertions(+)
 create mode 100644 meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb

diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb
index 58e0ffa..302510c 100644
--- a/meta/recipes-core/meta/buildtools-tarball.bb
+++ b/meta/recipes-core/meta/buildtools-tarball.bb
@@ -15,13 +15,17 @@ TOOLCHAIN_HOST_TASK ?= "\
     nativesdk-ncurses-terminfo-base \
     nativesdk-chrpath \
     nativesdk-tar \
+    nativesdk-buildtools-perl-dummy \
     nativesdk-git \
+    nativesdk-git-perltools \
     nativesdk-pigz \
     nativesdk-make \
     nativesdk-wget \
     nativesdk-ca-certificates \
     "
 
+SDK_PACKAGE_ARCHS =+ "buildtools-dummy-${SDKPKGSUFFIX}"
+
 TOOLCHAIN_OUTPUTNAME ?= "${SDK_NAME}-buildtools-nativesdk-standalone-${DISTRO_VERSION}"
 
 RDEPENDS = "${TOOLCHAIN_HOST_TASK}"
diff --git a/meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb b/meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb
new file mode 100644
index 0000000..d971c3c
--- /dev/null
+++ b/meta/recipes-core/meta/nativesdk-buildtools-perl-dummy.bb
@@ -0,0 +1,19 @@
+SUMMARY = "Dummy package which ensures perl is excluded from buildtools"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
+
+inherit nativesdk
+
+# Put it somewhere separate to ensure it's never used except when we want it
+PACKAGE_ARCH = "buildtools-dummy-${SDKPKGSUFFIX}"
+
+PERLPACKAGES = "nativesdk-perl \
+                nativesdk-perl-module-file-path"
+
+ALLOW_EMPTY_${PN} = "1"
+
+python populate_packages_prepend() {
+    d.appendVar('RPROVIDES_${PN}', '${PERLPACKAGES}')
+    d.appendVar('RCONFLICTS_${PN}', '${PERLPACKAGES}')
+}
+
-- 
1.9.3

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Otavio Salvador]

On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> Since the split out of git-perltools, some git tools (such as "git am",
> "git send-email" and "git-submodule") have no longer been part of the
> buildtools. We need these, so add them back in.
>
> However, adding git-perltools to buildtools triggers perl itself being
> brought into buildtools as well, and we don't want that; but we also
> don't want to have to hack the git recipe or indeed anything else that
> starts depending on perl. Thus, add a dummy package which gets installed
> in its place, in a separate package architecture that is only enabled
> for buildtools to ensure it doesn't start appearing in place of
> nativesdk-perl anywhere else.
>
> Fixes [YOCTO #7033].
>
> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>

This dummy thing looks like a hack to me :-(

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Paul Eggleton]

On Friday 05 December 2014 16:30:44 Otavio Salvador wrote:
> On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
> 
> <paul.eggleton@linux.intel.com> wrote:
> > Since the split out of git-perltools, some git tools (such as "git am",
> > "git send-email" and "git-submodule") have no longer been part of the
> > buildtools. We need these, so add them back in.
> > 
> > However, adding git-perltools to buildtools triggers perl itself being
> > brought into buildtools as well, and we don't want that; but we also
> > don't want to have to hack the git recipe or indeed anything else that
> > starts depending on perl. Thus, add a dummy package which gets installed
> > in its place, in a separate package architecture that is only enabled
> > for buildtools to ensure it doesn't start appearing in place of
> > nativesdk-perl anywhere else.
> > 
> > Fixes [YOCTO #7033].
> > 
> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> 
> This dummy thing looks like a hack to me :-(

Perhaps - I'm all ears for an alternative solution, but it absolutely has to 
be fixed, and soon. Shipping an incomplete git (or incomplete perl) in 
buildtools basically defeats a major part of having the thing in the first 
place.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Otavio Salvador]

On Fri, Dec 5, 2014 at 11:39 PM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Friday 05 December 2014 16:30:44 Otavio Salvador wrote:
>> On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
>>
>> <paul.eggleton@linux.intel.com> wrote:
>> > Since the split out of git-perltools, some git tools (such as "git am",
>> > "git send-email" and "git-submodule") have no longer been part of the
>> > buildtools. We need these, so add them back in.
>> >
>> > However, adding git-perltools to buildtools triggers perl itself being
>> > brought into buildtools as well, and we don't want that; but we also
>> > don't want to have to hack the git recipe or indeed anything else that
>> > starts depending on perl. Thus, add a dummy package which gets installed
>> > in its place, in a separate package architecture that is only enabled
>> > for buildtools to ensure it doesn't start appearing in place of
>> > nativesdk-perl anywhere else.
>> >
>> > Fixes [YOCTO #7033].
>> >
>> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
>>
>> This dummy thing looks like a hack to me :-(
>
> Perhaps - I'm all ears for an alternative solution, but it absolutely has to
> be fixed, and soon. Shipping an incomplete git (or incomplete perl) in
> buildtools basically defeats a major part of having the thing in the first
> place.

We should split out the tools we really want (so we skip the perl
runtime dependency) or include perl. Use a dummy package for it is
wrong in my opinion. You cannot guarantee host perl will work as
expected for our git release.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Peter A. Bigot]

On 12/06/2014 07:01 AM, Otavio Salvador wrote:
> On Fri, Dec 5, 2014 at 11:39 PM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
>> On Friday 05 December 2014 16:30:44 Otavio Salvador wrote:
>>> On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
>>>
>>> <paul.eggleton@linux.intel.com> wrote:
>>>> Since the split out of git-perltools, some git tools (such as "git am",
>>>> "git send-email" and "git-submodule") have no longer been part of the
>>>> buildtools. We need these, so add them back in.
>>>>
>>>> However, adding git-perltools to buildtools triggers perl itself being
>>>> brought into buildtools as well, and we don't want that; but we also
>>>> don't want to have to hack the git recipe or indeed anything else that
>>>> starts depending on perl. Thus, add a dummy package which gets installed
>>>> in its place, in a separate package architecture that is only enabled
>>>> for buildtools to ensure it doesn't start appearing in place of
>>>> nativesdk-perl anywhere else.
>>>>
>>>> Fixes [YOCTO #7033].
>>>>
>>>> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
>>> This dummy thing looks like a hack to me :-(
>> Perhaps - I'm all ears for an alternative solution, but it absolutely has to
>> be fixed, and soon. Shipping an incomplete git (or incomplete perl) in
>> buildtools basically defeats a major part of having the thing in the first
>> place.
> We should split out the tools we really want (so we skip the perl
> runtime dependency) or include perl. Use a dummy package for it is
> wrong in my opinion. You cannot guarantee host perl will work as
> expected for our git release.

I vote for "include perl".

I was surprised that I had to submit a patch to get git-perltools to 
include the git Perl module which is required by git add --interactive.

Why are there things in git-perltools that don't have a true dependency 
on perl?  And if they do, then is the issue that some of them are usable 
in restricted ways that don't invoke that dependency?

If so there probably needs to be a 
git-tools-that-could-use-perl-but-dont-have-to package to hold them, if 
it's important enough to provide them in a partly-crippled form instead 
of including perl.  But as a user I expect something called 
"git-perltools" to have everything necessary to use the tools it 
contains in all ways they're documented to be usable, whether it's on 
target or native.

Peter

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Richard Purdie]

On Sat, 2014-12-06 at 11:01 -0200, Otavio Salvador wrote:
> On Fri, Dec 5, 2014 at 11:39 PM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
> > On Friday 05 December 2014 16:30:44 Otavio Salvador wrote:
> >> On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
> >>
> >> <paul.eggleton@linux.intel.com> wrote:
> >> > Since the split out of git-perltools, some git tools (such as "git am",
> >> > "git send-email" and "git-submodule") have no longer been part of the
> >> > buildtools. We need these, so add them back in.
> >> >
> >> > However, adding git-perltools to buildtools triggers perl itself being
> >> > brought into buildtools as well, and we don't want that; but we also
> >> > don't want to have to hack the git recipe or indeed anything else that
> >> > starts depending on perl. Thus, add a dummy package which gets installed
> >> > in its place, in a separate package architecture that is only enabled
> >> > for buildtools to ensure it doesn't start appearing in place of
> >> > nativesdk-perl anywhere else.
> >> >
> >> > Fixes [YOCTO #7033].
> >> >
> >> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> >>
> >> This dummy thing looks like a hack to me :-(
> >
> > Perhaps - I'm all ears for an alternative solution, but it absolutely has to
> > be fixed, and soon. Shipping an incomplete git (or incomplete perl) in
> > buildtools basically defeats a major part of having the thing in the first
> > place.
> 
> We should split out the tools we really want (so we skip the perl
> runtime dependency) or include perl. Use a dummy package for it is
> wrong in my opinion. You cannot guarantee host perl will work as
> expected for our git release.

There needs to be a mechanism where we draw the line on which
dependencies a given thing has. Perl is arguable both ways. In all cases
I'm aware of, the host perl is generally API stable and works fine with
buildtools.

If you don't buy the perl argument, think about bash. Should we ship
bash in case anything in the buildtools tarball? Should we alter all
script paths to point at our own "sh"?

At some point you have to depend on the host system, be it perl, sh or
other things. The exact line should be configurable, Paul is simply
adding a mechanism here that allows us to control that. He's needed a
default and I think perl is API stable enough that we should be ok here.

Cheers,

Richard

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Otavio Salvador]

On Sat, Dec 6, 2014 at 12:37 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Sat, 2014-12-06 at 11:01 -0200, Otavio Salvador wrote:
>> On Fri, Dec 5, 2014 at 11:39 PM, Paul Eggleton
>> <paul.eggleton@linux.intel.com> wrote:
>> > On Friday 05 December 2014 16:30:44 Otavio Salvador wrote:
>> >> On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
>> >>
>> >> <paul.eggleton@linux.intel.com> wrote:
>> >> > Since the split out of git-perltools, some git tools (such as "git am",
>> >> > "git send-email" and "git-submodule") have no longer been part of the
>> >> > buildtools. We need these, so add them back in.
>> >> >
>> >> > However, adding git-perltools to buildtools triggers perl itself being
>> >> > brought into buildtools as well, and we don't want that; but we also
>> >> > don't want to have to hack the git recipe or indeed anything else that
>> >> > starts depending on perl. Thus, add a dummy package which gets installed
>> >> > in its place, in a separate package architecture that is only enabled
>> >> > for buildtools to ensure it doesn't start appearing in place of
>> >> > nativesdk-perl anywhere else.
>> >> >
>> >> > Fixes [YOCTO #7033].
>> >> >
>> >> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
>> >>
>> >> This dummy thing looks like a hack to me :-(
>> >
>> > Perhaps - I'm all ears for an alternative solution, but it absolutely has to
>> > be fixed, and soon. Shipping an incomplete git (or incomplete perl) in
>> > buildtools basically defeats a major part of having the thing in the first
>> > place.
>>
>> We should split out the tools we really want (so we skip the perl
>> runtime dependency) or include perl. Use a dummy package for it is
>> wrong in my opinion. You cannot guarantee host perl will work as
>> expected for our git release.
>
> There needs to be a mechanism where we draw the line on which
> dependencies a given thing has. Perl is arguable both ways. In all cases
> I'm aware of, the host perl is generally API stable and works fine with
> buildtools.
>
> If you don't buy the perl argument, think about bash. Should we ship
> bash in case anything in the buildtools tarball? Should we alter all
> script paths to point at our own "sh"?
>
> At some point you have to depend on the host system, be it perl, sh or
> other things. The exact line should be configurable, Paul is simply
> adding a mechanism here that allows us to control that. He's needed a
> default and I think perl is API stable enough that we should be ok here.

I understand but the provided mechanism is ugly and hackish.

Maybe a mechanism like 'ASSUME_PROVIDED' could be made and use for
this use-case. A dummy package looks like a workaround.

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Richard Purdie]

On Sat, 2014-12-06 at 14:10 -0200, Otavio Salvador wrote:
> On Sat, Dec 6, 2014 at 12:37 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > At some point you have to depend on the host system, be it perl, sh or
> > other things. The exact line should be configurable, Paul is simply
> > adding a mechanism here that allows us to control that. He's needed a
> > default and I think perl is API stable enough that we should be ok here.
> 
> I understand but the provided mechanism is ugly and hackish.
> 
> Maybe a mechanism like 'ASSUME_PROVIDED' could be made and use for
> this use-case. A dummy package looks like a workaround.

The mechanism needs to work at the package manager level, not at bitbake
since that is where the problem is. I'm not aware of mechanisms in any
of the package managers, let alone one common to all of them.

As Paul said, better ideas more than welcome.

Cheers,

Richard

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Paul Eggleton]

On Saturday 06 December 2014 11:01:49 Otavio Salvador wrote:
> On Fri, Dec 5, 2014 at 11:39 PM, Paul Eggleton
> 
> <paul.eggleton@linux.intel.com> wrote:
> > On Friday 05 December 2014 16:30:44 Otavio Salvador wrote:
> >> On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
> >> 
> >> <paul.eggleton@linux.intel.com> wrote:
> >> > Since the split out of git-perltools, some git tools (such as "git am",
> >> > "git send-email" and "git-submodule") have no longer been part of the
> >> > buildtools. We need these, so add them back in.
> >> > 
> >> > However, adding git-perltools to buildtools triggers perl itself being
> >> > brought into buildtools as well, and we don't want that; but we also
> >> > don't want to have to hack the git recipe or indeed anything else that
> >> > starts depending on perl. Thus, add a dummy package which gets
> >> > installed
> >> > in its place, in a separate package architecture that is only enabled
> >> > for buildtools to ensure it doesn't start appearing in place of
> >> > nativesdk-perl anywhere else.
> >> > 
> >> > Fixes [YOCTO #7033].
> >> > 
> >> > Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> >> 
> >> This dummy thing looks like a hack to me :-(
> > 
> > Perhaps - I'm all ears for an alternative solution, but it absolutely has
> > to be fixed, and soon. Shipping an incomplete git (or incomplete perl) in
> > buildtools basically defeats a major part of having the thing in the
> > first place.
> 
> We should split out the tools we really want (so we skip the perl
> runtime dependency) 

This is exactly what caused this problem in the first place. We cannot do 
without the perl-dependent pieces of git.

> or include perl. Use a dummy package for it is
> wrong in my opinion. You cannot guarantee host perl will work as
> expected for our git release.

It hasn't presented a problem up to now, and until it does I'd rather not ship 
perl in buildtools. CentOS 6 is the oldest distro we support, and the latest 
version of git works perfectly fine with the version of perl that's provided by 
the distro there. On the other hand, adding perl means we need to add a bunch 
of additional perl modules as well, and I really don't think we want to do 
that because it just increases the compexity and size of the buildtools and we 
don't actually need it anyway.

I have actually thought about this.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH v2 1/1] buildtools-tarball: restore missing git tools

[Paul Eggleton]

Hi Peter,

On Saturday 06 December 2014 07:40:48 Peter A. Bigot wrote:
> On 12/06/2014 07:01 AM, Otavio Salvador wrote:
> > On Fri, Dec 5, 2014 at 11:39 PM, Paul Eggleton
> > 
> > <paul.eggleton@linux.intel.com> wrote:
> >> On Friday 05 December 2014 16:30:44 Otavio Salvador wrote:
> >>> On Fri, Dec 5, 2014 at 4:09 PM, Paul Eggleton
> >>> 
> >>> <paul.eggleton@linux.intel.com> wrote:
> >>>> Since the split out of git-perltools, some git tools (such as "git am",
> >>>> "git send-email" and "git-submodule") have no longer been part of the
> >>>> buildtools. We need these, so add them back in.
> >>>> 
> >>>> However, adding git-perltools to buildtools triggers perl itself being
> >>>> brought into buildtools as well, and we don't want that; but we also
> >>>> don't want to have to hack the git recipe or indeed anything else that
> >>>> starts depending on perl. Thus, add a dummy package which gets
> >>>> installed
> >>>> in its place, in a separate package architecture that is only enabled
> >>>> for buildtools to ensure it doesn't start appearing in place of
> >>>> nativesdk-perl anywhere else.
> >>>> 
> >>>> Fixes [YOCTO #7033].
> >>>> 
> >>>> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
> >>> 
> >>> This dummy thing looks like a hack to me :-(
> >> 
> >> Perhaps - I'm all ears for an alternative solution, but it absolutely has
> >> to be fixed, and soon. Shipping an incomplete git (or incomplete perl)
> >> in buildtools basically defeats a major part of having the thing in the
> >> first place.
> > 
> > We should split out the tools we really want (so we skip the perl
> > runtime dependency) or include perl. Use a dummy package for it is
> > wrong in my opinion. You cannot guarantee host perl will work as
> > expected for our git release.
> 
> I vote for "include perl".

We'd rather not do that for buildtools unless there are actual problems with 
the host perl. I've heard of no such issues up to this point.

> I was surprised that I had to submit a patch to get git-perltools to
> include the git Perl module which is required by git add --interactive.

Right, we have that dependency now though.

> Why are there things in git-perltools that don't have a true dependency
> on perl?  And if they do, then is the issue that some of them are usable
> in restricted ways that don't invoke that dependency?

There aren't. These require perl, but the host perl should be perfectly 
adequate. Buildtools isn't quite the same as the target.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre