Re: [OE-core] [PATCH v2] bzip2: Add test suite for bzip2

["Peter Kjellerstedt" <peter.kjellerstedt@axis.com>]

> -----Original Message-----
> From: Randy MacLeod <randy.macleod@windriver.com>
> Sent: den 19 maj 2020 23:45
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>; Rahul Kumar
> <rahulk@mvista.com>
> Cc: Alexander Kanavin <alex.kanavin@gmail.com>; Richard Purdie
> <richard.purdie@linuxfoundation.org>; OE-core <openembedded-
> core@lists.openembedded.org>; Trevor Gamblin
> <trevor.gamblin@windriver.com>
> Subject: Re: [OE-core] [PATCH v2] bzip2: Add test suite for bzip2
> 
> On 2020-05-19 12:29 p.m., Peter Kjellerstedt wrote:
> > The jzlib license is a three clause BSD license, and so is the
> > go/LICENSE, so you should be able to set LICENSE as:
> >
> > LICENSE = "bzip2-1.0.6 & GPLv3+ & Apache-2.0 & MS-PL & BSD-3-Clause & Zlib"
> 
> Peter,
> 
> I respectfully disagree.
> 
> The only source that is _executed_ in normal use by bzip2-test is
> the run-tests.sh script that is licensed as GPLv3+

Well, I am definitely not a lawyer either, but I am pretty sure that 
whether something that is distributed under a certain license is 
executed or not is irrelevant (unless of course the license covers 
execution of the code).

> The other files, although they came from source packages,
> are *only* used as  test data. The source code has been completely
> stripped out, hasn't it?

Given that they were originally distributed in an archive together 
with a LICENSE/COPYING/similar file covering the entire archive, 
those files are still covered by that license even if they are not 
source files per se.

> If, instead of compressed data,
> someone were to give me an image file covered by a creative common's
> license and the image had a watermark that when extracted produced
> file with source code with a FOO license, would you change the
> package license to include FOO? In fact, we don't even look at
> image file licensing let alone at embedded watermarks.

Well, if there is a file distributed in the picture (not using the 
word "image" here to avoid confusing the discussion any more) in the 
form of a watermark, then that is no different from distributing it 
as part of a tar.gz file (albeit a bit unusual). Whatever license 
that file is covered by still applies in both cases.

Actually, here is something to think about. I recently read an 
article published by the Linux Foundation called "Docker Containers 
for Legal Professionals". [1] There they discussed the distribution 
of Docker images. Consider this, you take an existing Docker image, 
e.g., fedora:32, and remove everything that is covered by GPL-3.0 
(not sure how to actually accomplish that, but anyway) and then you 
distribute this new Docker image to someone. Does this mean that 
GPL-3.0 covered code is being distributed? The answer is yes. 
Because of the way Docker images are made up from layers, you need 
to adhere to the licenses covering everything in all layers, not 
just the final result. And somewhere down the stack of layers 
making up your Docker image is still all of that GPL-3.0 code 
that you removed in a higher layer.

> I'm not a lawyer so this is simply my opinion on the matter.
> How do we come to an agreement?
> I'd be happy to add a comment in the recipe explaining our position:
> 
> # This package contains a script and test data this is derived from
> # other software packages as explained in the top level README.
> # Although there are COPYING files explaining the license terms
> # for each sub-directory, the files in each directory are only used
> # as test data so the LICENSING terms do not appear to apply to
> # the recipe as we ship it and have therefore NOT been incorporated
> # into the bzip2-ptests recipe licensing tags.

I would be very worried seeing such a statement, and I believe any 
lawyer would too.

> I can consult with some intellectual property experts that I know
> about this particular case if you like.

If you have the possibility to do so, I think it may be a good idea.

> ../Randy

//Peter

[1] https://www.linuxfoundation.org/wp-content/uploads/2020/04/Docker-Containers-for-Legal-Professionals-Whitepaper_v4.ac_-3.pdf

> > You also need to add:
> >
> > LICENSE_${PN} = "bzip2-1.0.6"
> >
> > LICENSE_${PN}-dev = "bzip2-1.0.6"
> >
> > LICENSE_${PN}-dbg = "bzip2-1.0.6"
> >
> > LICENSE_${PN}-doc = "bzip2-1.0.6"
> >
> > LICENSE_${PN}-src = "bzip2-1.0.6"
> >
> > LICENSE_libbz2 = "bzip2-1.0.6"
> >
> > so the non-ptest packages are not affected by the licenses for the
> ptest
> > package.
> >
> > //Peter
> >
> > *From:*Rahul Kumar <rahulk@mvista.com>
> > *Sent:* den 19 maj 2020 13:58
> > *To:* Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> > *Cc:* Randy MacLeod <randy.macleod@windriver.com>; Alexander Kanavin
> > <alex.kanavin@gmail.com>; Richard Purdie
> > <richard.purdie@linuxfoundation.org>; OE-core
> > <openembedded-core@lists.openembedded.org>; Trevor Gamblin
> > <trevor.gamblin@windriver.com>
> > *Subject:* Re: [OE-core] [PATCH v2] bzip2: Add test suite for bzip2
> >
> > Hi Peter,
> >
> > I am sharing a few information related to bzip2-test source and
> > licensing information.
> > As per your suggestion and as per my understanding I added License to
> > the bzip2 recipe file.
> >
> > bzip2-tests.git source tree structure is given below
> > (here showing licence related fle, removed other file names )
> >
> > bzip2-tests.git/
> > ├──commons-compress
> > │ ├──LICENSE.txt
> > │
> > ├──dotnetzip
> > │ ├──License.txt
> > │ ├──License.zlib.txt
> > |
> > ├──go
> > │ ├──LICENSE
> > │
> > ├──lbzip2
> > ├──pyflate
> > ├──README
> > └── run-tests.sh
> >
> > source code link:
> > https://sourceware.org/git/bzip2-tests.git
> >
> > bzip2-tests.git source contains
> > 1) comman-commpress: which is Originally distributed under the Apache
> > License Version 2.0.
> >
> > See LICENSE.txt
> >
> >
> > 2) dotnetzip:
> > (i)  The DotNetZip Library is distributed under the MicroSoft Public
> License
> >      (Ms-PL) see the License.txt file.
> > (ii) The BZ2 related resources are distributed under the licenses in
> > License.zlib.txt.
> >
> > 3) go:
> >
> >   Go is distributed under a BSD-style license.
> >   See the LICENSE file.
> >
> > 4) lbzip2:
> > lbzip2 is free software: you can redistribute it and/or modify it under
> the
> > terms of the GNU General Public License as published by the Free
> Software
> > Foundation, either version 3 of the License, or (at your option) any
> later
> > version
> >
> > 5) pyflate: You may use and distribute this code and documentation
> under any
> >              DFSG-compatible license (eg. BSD, GNU GPLv2).
> >
> > 6) run-tests.sh: you can redistribute it and/or modify it under the
> > terms of the GNU General Public License (GPL); either
> >                   version 3, or (at your option) any later version.
> >
> >
> > SO in recipe I added the License like below
> > LICENSE = "bzip2-1.0.6 & GPLv3+ & Apache-2.0 & MS-PL & BSD "
> > LICENSE_${PN}-ptest = "GPLv3+ & Apache-2.0 & MS-PL & BSD"
> > LIC_FILES_CHKSUM =
> >
> "file://LICENSE;beginline=4;endline=37;md5=600af43c50f1fcb82e32f19b32df46
> 64
> > \
> >
> > file://${WORKDIR}/git/commons-
> compress/LICENSE.txt;md5=86d3f3a95c324c9479bd8986968f4327
> > \ # For comman-commpress (Apache-2.0)
> >
> >
> file://${WORKDIR}/git/dotnetzip/License.txt;md5=9cb56871eed4e748c3bc7e8ff
> 352a54f
> > \      # For dotnetzip (Ms-PL)
> >
> >
> file://${WORKDIR}/git/dotnetzip/License.zlib.txt;md5=cc421ccd22eeb2e5db6b
> 79e6de0a029f
> > \   # For dotnetzip  (jzlib +zlib)
> >
> > file://${WORKDIR}/git/go/LICENSE;md5=5d4950ecb7b26d2c5e4e7b4e0dd74707 \
> >       # For go (BSD-style license)
> >
> > here I need few suggestion:
> >
> > 1) I added the BSD and GPLv3+ license in LICENSE and LICENSE_${PN}-
> ptest.
> >     but not added LIC_FILES_CHKSUM for GPLv3 and BSD because
> > bzip2-tests.git source code doesn't have GPLv3 and BSD License text
> file
> > like other
> >     directories.
> >
> > 2) License.zlib.txt: This is a combination of two licence jzlib and
> > zlib. In meta/files/common-licenses Zlib license is available but not
> jzlib.
> >     In this case how can I proceed?
> >
> > Kindly comment on this and feel free to point out if i am wrong at any
> > place.
> >
> > *Thanks & Regards,*
> >
> > Rahul Kumar
> >
> > Software Engineer,Linux Solutions Engineering
> >
> > Group,Montavista Software LLC
> >
> > Email Id: rahulk@mvista.com <mailto:rahulk@mvista.com>
> > <https://plus.google.com/+CodeTwoSoftware>
> >
> > On Mon, May 18, 2020 at 1:18 AM Peter Kjellerstedt
> > <peter.kjellerstedt@axis.com <mailto:peter.kjellerstedt@axis.com>>
> wrote:
> >
> >     As I wrote in my previous mail, when it comes to licenses, it is
> the
> >     distribution that is important. So if you build a package
> >     (bzip2-ptest) that includes code that is GPL-3.0, it does not
> matter
> >     if that code is only used as a binary blob, it is still distributed
> >     in the package, and thus the package needs to be covered by the
> >     GPL-3.0 license.
> >
> >     //Peter
> >
> >     *From:*openembedded-core@lists.openembedded.org
> >     <mailto:openembedded-core@lists.openembedded.org>
> >     <openembedded-core@lists.openembedded.org
> >     <mailto:openembedded-core@lists.openembedded.org>> *On Behalf Of
> >     *Rahul Kumar
> >     *Sent:* den 17 maj 2020 21:18
> >     *To:* Randy MacLeod <randy.macleod@windriver.com
> >     <mailto:randy.macleod@windriver.com>>
> >     *Cc:* Alexander Kanavin <alex.kanavin@gmail.com
> >     <mailto:alex.kanavin@gmail.com>>; Richard Purdie
> >     <richard.purdie@linuxfoundation.org
> >     <mailto:richard.purdie@linuxfoundation.org>>; OE-core
> >     <openembedded-core@lists.openembedded.org
> >     <mailto:openembedded-core@lists.openembedded.org>>; Trevor Gamblin
> >     <trevor.gamblin@windriver.com
> <mailto:trevor.gamblin@windriver.com>>
> >     *Subject:* Re: [OE-core] [PATCH v2] bzip2: Add test suite for bzip2
> >
> >     Hi Randy,
> >
> >     patch has been accepted by bzip2 community with additional changes.
> >     https://sourceware.org/git/?p=bzip2-
> tests.git;a=commit;h=f9061c030a25de5b6829e1abf373057309c734c0
> >
> >
> >     So I need your suggestion with respect to Yocto.
> >
> >     Since required changes for ptest infrastructure is integrated to
> the
> >     bzip2-tests.git source code.
> >     So now we can just change the SRCREV with the latest commit instead
> >     of applying the patch to the bzip2-tests.git source code.
> >
> >     And since we are just integrating the bzip2-tests source to the
> >     bzip2 recipe and not modifying any file so i think no need to add
> >     any License for it.
> >
> >
> >     Now patch will look like
> >
> =====================================================================
> >     --- a/meta/recipes-extended/bzip2/bzip2/Makefile.am
> >     +++ b/meta/recipes-extended/bzip2/bzip2/Makefile.am
> >     @@ -46,6 +46,7 @@ runtest:
> >              else echo "FAIL: sample2 decompress"; fi
> >              @if cmp sample3.tst sample3.ref; then echo "PASS: sample3
> >     decompress";\
> >              else echo "FAIL: sample3 decompress"; fi
> >     +       ./bzip2-tests/run-tests.sh --tests-dir="$(PWD)/bzip2-tests"
> >
> >       install-ptest:
> >              sed  -n '/^runtest:/,/^install-ptest:/{/^install-
> ptest:/!p}' \
> >     @@ -56,6 +57,7 @@ install-ptest:
> >              cp $(srcdir)/sample1.bz2        $(DESTDIR)/
> >              cp $(srcdir)/sample2.bz2        $(DESTDIR)/
> >              cp $(srcdir)/sample3.bz2        $(DESTDIR)/
> >     +       cp -rf $(srcdir)/../git         $(DESTDIR)/bzip2-tests
> >              ln -s $(bindir)/bzip2           $(DESTDIR)/bzip2
> >
> >       install-exec-hook:
> >     diff --git a/meta/recipes-extended/bzip2/bzip2_1.0.8.bb
> >     <http://bzip2_1.0.8.bb> b/meta/recipes-
> extended/bzip2/bzip2_1.0.8.bb
> >     <http://bzip2_1.0.8.bb>
> >     index d58f553..c5aa690 100644
> >     --- a/meta/recipes-extended/bzip2/bzip2_1.0.8.bb
> <http://bzip2_1.0.8.bb>
> >     +++ b/meta/recipes-extended/bzip2/bzip2_1.0.8.bb
> <http://bzip2_1.0.8.bb>
> >     @@ -8,13 +8,17 @@ LICENSE = "bzip2-1.0.6"
> >       LIC_FILES_CHKSUM =
> >
> "file://LICENSE;beginline=4;endline=37;md5=600af43c50f1fcb82e32f19b32df46
> 64"
> >
> >       SRC_URI = "https://sourceware.org/pub/${BPN}/${BPN}-${PV}.tar.gz
> >     <https://sourceware.org/pub/$%7bBPN%7d/$%7bBPN%7d-$%7bPV%7d.tar.gz>
> \
> >     +
> >     git://sourceware.org/git/bzip2-tests.git;name=bzip2-tests
> >     <http://sourceware.org/git/bzip2-tests.git;name=bzip2-tests> \
> >                  file://configure.ac <http://configure.ac>;subdir=${BP}
> \
> >                  file://Makefile.am;subdir=${BP} \
> >                  file://run-ptest \
> >                  "
> >     +
> >       SRC_URI[md5sum] = "67e051268d0c475ea773822f7500d0e5"
> >       SRC_URI[sha256sum] =
> >     "ab5a03176ee106d3f0fa90e381da478ddae405918153cca248e682cd0c4a2269"
> >
> >     +SRCREV_bzip2-tests = "f9061c030a25de5b6829e1abf373057309c734c0"
> *<<
> >     this commit Id includes latest charges for ptest infrastruct.*
> >     +
> >       UPSTREAM_CHECK_URI = "https://www.sourceware.org/pub/bzip2/"
> >
> >       PACKAGES =+ "libbz2"
> >     @@ -39,7 +43,7 @@ do_install_ptest () {
> >
> >       FILES_libbz2 = "${libdir}/lib*${SOLIBS}"
> >
> >     -RDEPENDS_${PN}-ptest += "make"
> >     +RDEPENDS_${PN}-ptest += "make bash"
> >
> >       PROVIDES_append_class-native = " bzip2-replacement-native"
> >       BBCLASSEXTEND = "native nativesdk"
> >
> =========================================================================
> ========
> >
> >     Kindly comment on it and feel free to point out, if i am wrong at
> >     any place.
> >
> >     *Thanks & Regards,*
> >
> >     Rahul Kumar
> >
> >     Software Engineer,Linux Solutions Engineering
> >
> >     Group,Montavista Software LLC
> >
> >     Email Id: rahulk@mvista.com <mailto:rahulk@mvista.com>
> >     <https://plus.google.com/+CodeTwoSoftware>
> >
> >     On Fri, May 1, 2020 at 6:56 AM Randy MacLeod
> >     <randy.macleod@windriver.com <mailto:randy.macleod@windriver.com>>
> >     wrote:
> >
> >         On 2020-04-27 3:39 p.m., Alexander Kanavin wrote:
> >          > You need to first see from the failure page which
> >         configuration is
> >          > failing, for example non-gpl3 is one such.
> >          >
> >          > Then you find that configuration in config.json. The below
> >         should
> >          > hopefully be self-explanatory in how you should set up the
> build?
> >          >
> >          > |"non-gpl3" : { "NEEDREPOS" : ["poky", "meta-gplv2"],
> >         "MACHINE" :
> >          > "qemux86", "BBTARGETS" : "core-image-minimal
> >         core-image-full-cmdline",
> >          > "extravars" : [ "INCOMPATIBLE_LICENSE = '*GPLv3'",
> >         "WARN_QA_remove =
> >          > 'incompatible-license'" ], "EXTRACMDS" : [
> >          > "../../yocto-autobuilder-helper/scripts/check-gplv3" ] },
> >          >
> >          > |
> >          >
> >          > |
> >          > |
> >          >
> >          > |Alex
> >
> >         Hi Rahul,
> >
> >         Sorry for my late reply.
> >
> >         The commit log for v2 is very good now!
> >         Thanks for incorporating my --pedantic suggestions. ;-)
> >
> >         It seems that you need a perl dependency for something (docs?
> >              $ cd .../bzip2.git
> >              $ grep -r "perl " *
> >              format.pl:#!/usr/bin/perl -w
> >              README.XML.STUFF:It uses format.pl <http://format.pl>, a
> >         perl script...
> >
> >         Then we need to figure out how to deal with the GPLv3 issue.
> >
> >         The buildbot output can be tedious to figure out. I haven't
> really
> >         spent enough time plugging away at it to be proficient yet
> either.
> >         Have you been able to reproduce the problems that Richard
> reported?
> >         If not, and you've tried for a bit, then just say so and I'll
> try to
> >         help tomorrow or early next week.
> >
> >         It looks like you are packaging the test code/data with the
> main
> >         package
> >         not in bzip2-ptest. Have a look at:
> >              meta/recipes-support/libpcre/libpcre_8.44.bb
> >         <http://libpcre_8.44.bb>
> >         for an example. There are many more.
> >         Also, if you look at oe-core.git:
> >              $ rgrep LICENSE_ *  | grep PN
> >         you can see many examples of sub-packages with different
> licenses
> >         than the main package. One example is:
> >              meta/recipes-support/gnutls/gnutls_3.6.13.bb
> >         <http://gnutls_3.6.13.bb>
> >         I hope that can address the buildbot problem but I haven't
> tried it
> >         myself yet.
> >
> >         BTW, Trevor has gotten the YP autobuilder going at Wind River
> and
> >         he'll be sending a few documentation updates next week or so.
> >         That may help in case you want to reproduce the YP AB test
> >         infrastructure. I expect that you don't _have_ to do so but
> >         I think it would be good if more contributing organizations did
> >         have an instance with only limited builders of the YP AB so
> that
> >         we can do more testing before Richard runs our changes through
> >         the main system. Richard has cautioned that the YP AB has lots
> of
> >         builders each of which has many cores but I hope that we can at
> >         least
> >         do some AB checking ourselves.
> >
> >         ../Randy
> >
> >
> >          > |
> >          >
> >          >
> >          > On Mon, 27 Apr 2020 at 20:54, Rahul Kumar <rahulk@mvista.com
> >         <mailto:rahulk@mvista.com>
> >          > <mailto:rahulk@mvista.com <mailto:rahulk@mvista.com>>>
> wrote:
> >          >
> >          >     Hi Richard/Alexander,
> >          >
> >          >     I am not able to understand how I can use the below
> file.
> >          >
> >         http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-
> helper/tree/config.json
> >          >
> >          >     did you mean to say that i have to set MACRO in
> >         local.conf based on
> >          >     this file.
> >          >
> >          >     *Thanks & Regards,*
> >          >     Rahul Kumar
> >          >     Software Engineer,Linux Solutions Engineering
> >          >     Group,Montavista Software LLC
> >          >     Email Id: rahulk@mvista.com <mailto:rahulk@mvista.com>
> >         <mailto:rahulk@mvista.com <mailto:rahulk@mvista.com>>
> >          >     <https://plus.google.com/+CodeTwoSoftware>
> >          >
> >          >
> >          >     On Mon, Apr 27, 2020 at 11:46 PM Richard Purdie
> >          >     <richard.purdie@linuxfoundation.org
> >         <mailto:richard.purdie@linuxfoundation.org>
> >          >     <mailto:richard.purdie@linuxfoundation.org
> >         <mailto:richard.purdie@linuxfoundation.org>>> wrote:
> >          >
> >          >         On Mon, 2020-04-27 at 18:30 +0200, Alexander Kanavin
> >         wrote:
> >          >          > You need to look at configurations defined here:
> >          >          >
> >          >
> >         http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-
> helper/tree/config.json
> >          >          > and replicate them locally. Then you can
> reproduce the
> >          >         failures that
> >          >          > the AB gets in those configurations.
> >          >
> >          >         That start of the failing logs on the autobuilder
> >         also list out the
> >          >         configuration options for that build.
> >          >
> >          >         Cheers,
> >          >
> >          >         Richard
> >          >
> >          >
> >          >
> >          >
> >
> >
> >         --
> >         # Randy MacLeod
> >         # Wind River Linux
> >
> 
> 
> --
> # Randy MacLeod
> # Wind River Linux