Re: [OE-core] [PATCH v2] bzip2: Add test suite for bzip2

["Trevor Gamblin" <trevor.gamblin@windriver.com>]

On 5/12/20 7:22 PM, Randy MacLeod wrote:
> On 2020-05-12 12:58 a.m., Rahul Kumar wrote:
>> Hi,
>>
>> Can any one please help me to figure out how to deal with the GPLv3+ 
>> issue.
>>
>> you can see my Patch at below link
>> https://patchwork.openembedded.org/patch/172134/
>>
>> *Issue:*
>> the new license (GPLv3) causes problems:
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/75/builds/1814
>> *
>> *
>
> Hi Rahul,
>
> I'm having some email problems with the oe-core list so apologies
> if this is redundant.
>
> What happens if you split the license info into two parts like:
>
> $ grep "^LICENSE" recipes-extended/libvirt/libvirt_6.1.0.bb
> LICENSE = "LGPLv2.1+ & GPLv2+"
> LICENSE_${PN}-ptest = "GPLv2+ & LGPLv2.1+"
> except of course with GPLv3.
>
> I might try that tomorrow on our local instance of the YP autobuilder.
> If you'd like to set one up @ mvista, I hear from Trevor that it doesn't
> take all that much time. As others have explained, you can also dig
> through the yocto-autobuilder2/yocto-autobuilder-helper git repos.

Hi Rahul, I can definitely help you get an autobuilder instance running 
locally, if you'd like. At the moment I'd suggest doing so on a host 
that's running Fedora, as that's where I've had the most success 
(Richard has explained to me how they run it on Ubuntu, but I haven't 
gotten it fully functional there yet).

Are you on either the #yocto and/or #oe IRC channels?

>
> ../Randy
>
>> Thanks & Regards,
>> Rahul Kumar
>> Software Engineer,Linux Solutions Engineering
>> Group,Montavista Software LLC
>> Email Id: rahulk@mvista.com <mailto:rahulk@mvista.com>
>> <https://plus.google.com/+CodeTwoSoftware>
>>
>>
>> On Wed, May 6, 2020 at 4:47 PM Rahul Kumar via lists.openembedded.org 
>> <http://lists.openembedded.org> 
>> <rahulk=mvista.com@lists.openembedded.org 
>> <mailto:mvista.com@lists.openembedded.org>> wrote:
>>
>>     Hi Randy,
>>
>>     As per your suggestion I did some progress.
>>
>>     Issue 1:
>>     ========
>>
>>     Configuration for this issue:
>>     =============================
>>        MACHINE = "edgerouter"
>>        DISTRO = "poky"
>>        SDKMACHINE = "i686"
>>        PACKAGE_CLASSES = "package_rpm package_deb package_ipk"
>>        INHERIT += 'image-buildinfo'
>>        IMAGE_BUILDINFO_VARS_append = ' IMAGE_BASENAME IMAGE_NAME'
>>        QEMU_USE_KVM = 'True'
>>        INHERIT += 'report-error'
>>        PREMIRRORS = ''
>>        BB_GENERATE_MIRROR_TARBALLS = '1'
>>        BB_NUMBER_THREADS = '16'
>>        PARALLEL_MAKE = '-j 16'
>>        BB_TASK_NICE_LEVEL = '5'
>>        BB_TASK_NICE_LEVEL_task-testimage = '0'
>>        BB_TASK_IONICE_LEVEL = '2.7'
>>        BB_TASK_IONICE_LEVEL_task-testimage = '2.1'
>>        INHERIT += 'testimage'
>>        TEST_QEMUBOOT_TIMEOUT = '1500'
>>        SANITY_TESTED_DISTROS = ''
>>        SDK_EXT_TYPE = 'minimal'
>>        SDK_INCLUDE_TOOLCHAIN = '1'
>>     Command:
>>     ========
>>     bitbake core-image-sato core-image-sato-dev core-image-sato-sdk
>>     core-image-minimal core-image-minimal-dev core-image-sato-ptest
>>     core-image-sato:do_populate_sdk -k
>>
>>     but could not reproduce the issue.
>>
>>     work-around to reproduce this issue.
>>     ====================================
>>     I am observing since bzip2-tests is a git repo and
>>     fsmonitor-watchman.sample (.git/hooks/fsmonitor-watchman.sample) is
>>     perl script.
>>     that's why I got this error.
>>       so manually I copied fsmonitor-watchman.sample file into the
>>     bzip2-tests/.git/hooks and able to reproduce the issue.
>>     Error:
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/62/builds/1816
>>     step1b: ERROR: bzip2-1.0.8-r0 do_package_qa: QA Issue:
>> /usr/lib/bzip2/ptest/bzip2-tests/.git/hooks/fsmonitor-watchman.sample 
>> contained
>>     in package bzip2-ptest requires /usr/bin/perl, but no providers
>>     found in RDEPENDS_bzip2-ptest? [file-rdeps]
>>     step1b: ERROR: bzip2-1.0.8-r0 do_package_qa: QA run found fatal
>>     errors. Please consider fixing them.
>>
>>     I find out the solution by appending RDEPENDS_${PN}-ptest with perl.
>>     RDEPENDS_${PN}-ptest += "make bash perl"
>>
>>     so this issue got resolved.
>>
>>     Issue2:
>>     =======
>>     Configuration for this issue
>>     ============================
>>        MACHINE = "qemux86"
>>        DISTRO = "poky"
>>        SDKMACHINE = "i686"
>>        PACKAGE_CLASSES = "package_rpm package_deb package_ipk"
>>        INCOMPATIBLE_LICENSE = '*GPLv3'
>>        WARN_QA_remove = 'incompatible-license'
>>        QEMU_USE_KVM = 'True'
>>        INHERIT += 'report-error'
>>        PREMIRRORS = ''
>>        BB_GENERATE_MIRROR_TARBALLS = '1'
>>        BB_NUMBER_THREADS = '16'
>>        PARALLEL_MAKE = '-j 16'
>>        BB_TASK_NICE_LEVEL = '5'
>>        BB_TASK_NICE_LEVEL_task-testimage = '0'
>>        BB_TASK_IONICE_LEVEL = '2.7'
>>        BB_TASK_IONICE_LEVEL_task-testimage = '2.1'
>>        INHERIT += 'testimage'
>>        TEST_QEMUBOOT_TIMEOUT = '1500'
>>        SANITY_TESTED_DISTROS = ''
>>        SDK_EXT_TYPE = 'minimal'
>>        SDK_INCLUDE_TOOLCHAIN = '1'
>>     Command
>>     =======
>>     bitbake core-image-minimal core-image-full-cmdline -k
>>
>>
>>     INCOMPATIBLE_LICENSE = '*GPLv3'
>>     WARN_QA_remove = 'incompatible-license'
>>     My doubt is since above configuration is using during build and we
>>     are using GPLv3+ license then definetly it will report error.
>>
>>     It looks like you are packaging the test code/data with the main 
>> package
>>     not in bzip2-ptest. Have a look at:
>>          meta/recipes-support/libpcre/libpcre_8.44.bb
>>     <http://libpcre_8.44.bb>
>>     for an example. There are many more.
>>     Also, if you look at oe-core.git:
>>          $ rgrep LICENSE_ *  | grep PN
>>     you can see many examples of sub-packages with different licenses
>>     than the main package. One example is:
>>          meta/recipes-support/gnutls/gnutls_3.6.13.bb
>>     <http://gnutls_3.6.13.bb>
>>     I hope that can address the buildbot problem but I haven't tried it
>>     myself yet.
>>
>>     Explanation:
>>     I checked, Here is packaging the test code/data in bzip2-ptest.
>> /opt/opensource/build/tmp/work/mips64-poky-linux/bzip2/1.0.8-r0/packages-split/bzip2-ptest
>>
>>     I tried with the changes below  in the bzip2_1.0.8.bb
>>     <http://bzip2_1.0.8.bb> file.
>>     LICENSE = "bzip2"
>>     LICENSE_${PN}-ptest = "GPLv3+"
>>
>>     WARNING: LICENSE_bzip2-ptest includes licenses (GPLv3+) that are not
>>     listed in LICENSE
>>     To resolve this warning i did below changes.
>>     LICENSE = "bzip2 & GPLv3+"
>>     LICENSE_${PN}-ptest = "GPLv3+"
>>
>>     But I am getting below error in both case
>>
>>     ERROR: Nothing RPROVIDES 'bzip2' (but
>> /opt/opensource/poky/meta/recipes-extended/packagegroups/packagegroup-core-full-cmdline.bb
>>     <http://packagegroup-core-full-cmdline.bb>,
>> /opt/opensource/poky/meta/recipes-devtools/python/python3_3.8.2.bb
>>     <http://python3_3.8.2.bb> RDEPENDS on or otherwise requires it)
>>     bzip2 was skipped: it has incompatible license(s): GPL-3.0+
>>     NOTE: Runtime target 'bzip2' is unbuildable, removing...
>>     Missing or unbuildable dependency chain was: ['bzip2']
>>
>>     So as per my understanding, if we are splitting the package and
>>     assigning Licence to it.
>>     example:
>>     LICENSE = "bzip2"
>>     LICENSE_${PN}-ptest = "GPLv3+"
>>
>>     In this case I have to set LICENSE_PATH where your license file is
>>     located.
>>     or if I am using standard license, I have to set LICENSE first then
>>     we can set LICENSE_${PN}-ptest.
>>
>>     Example:
>>     LICENSE = "bzip2 & GPLv3+"
>>     LICENSE_${PN}-ptest = "GPLv3+"
>>
>>     Kindly comment on it and feel free to point out if i am wrong at any
>>     point.
>>
>>
>>     *Thanks & Regards,*
>>     Rahul Kumar
>>     Software Engineer,Linux Solutions Engineering
>>     Group,Montavista Software LLC
>>     Email Id: rahulk@mvista.com <mailto:rahulk@mvista.com>
>>     <https://plus.google.com/+CodeTwoSoftware>
>>
>>
>>     On Fri, May 1, 2020 at 6:56 AM Randy MacLeod
>>     <randy.macleod@windriver.com <mailto:randy.macleod@windriver.com>>
>>     wrote:
>>
>>         On 2020-04-27 3:39 p.m., Alexander Kanavin wrote:
>>          > You need to first see from the failure page which
>>         configuration is
>>          > failing, for example non-gpl3 is one such.
>>          >
>>          > Then you find that configuration in config.json. The below
>>         should
>>          > hopefully be self-explanatory in how you should set up the 
>> build?
>>          >
>>          > |"non-gpl3" : { "NEEDREPOS" : ["poky", "meta-gplv2"],
>>         "MACHINE" :
>>          > "qemux86", "BBTARGETS" : "core-image-minimal
>>         core-image-full-cmdline",
>>          > "extravars" : [ "INCOMPATIBLE_LICENSE = '*GPLv3'",
>>         "WARN_QA_remove =
>>          > 'incompatible-license'" ], "EXTRACMDS" : [
>>          > "../../yocto-autobuilder-helper/scripts/check-gplv3" ] },
>>          >
>>          > |
>>          >
>>          > |
>>          > |
>>          >
>>          > |Alex
>>
>>         Hi Rahul,
>>
>>         Sorry for my late reply.
>>
>>         The commit log for v2 is very good now!
>>         Thanks for incorporating my --pedantic suggestions. ;-)
>>
>>         It seems that you need a perl dependency for something (docs?
>>              $ cd .../bzip2.git
>>              $ grep -r "perl " *
>>              format.pl:#!/usr/bin/perl -w
>>              README.XML.STUFF:It uses format.pl <http://format.pl>, a
>>         perl script...
>>
>>         Then we need to figure out how to deal with the GPLv3 issue.
>>
>>         The buildbot output can be tedious to figure out. I haven't 
>> really
>>         spent enough time plugging away at it to be proficient yet 
>> either.
>>         Have you been able to reproduce the problems that Richard 
>> reported?
>>         If not, and you've tried for a bit, then just say so and I'll 
>> try to
>>         help tomorrow or early next week.
>>
>>         It looks like you are packaging the test code/data with the main
>>         package
>>         not in bzip2-ptest. Have a look at:
>>              meta/recipes-support/libpcre/libpcre_8.44.bb
>>         <http://libpcre_8.44.bb>
>>         for an example. There are many more.
>>         Also, if you look at oe-core.git:
>>              $ rgrep LICENSE_ *  | grep PN
>>         you can see many examples of sub-packages with different 
>> licenses
>>         than the main package. One example is:
>>              meta/recipes-support/gnutls/gnutls_3.6.13.bb
>>         <http://gnutls_3.6.13.bb>
>>         I hope that can address the buildbot problem but I haven't 
>> tried it
>>         myself yet.
>>
>>         BTW, Trevor has gotten the YP autobuilder going at Wind River 
>> and
>>         he'll be sending a few documentation updates next week or so.
>>         That may help in case you want to reproduce the YP AB test
>>         infrastructure. I expect that you don't _have_ to do so but
>>         I think it would be good if more contributing organizations did
>>         have an instance with only limited builders of the YP AB so that
>>         we can do more testing before Richard runs our changes through
>>         the main system. Richard has cautioned that the YP AB has 
>> lots of
>>         builders each of which has many cores but I hope that we can at
>>         least
>>         do some AB checking ourselves.
>>
>>         ../Randy
>>
>>
>>          > |
>>          >
>>          >
>>          > On Mon, 27 Apr 2020 at 20:54, Rahul Kumar <rahulk@mvista.com
>>         <mailto:rahulk@mvista.com>
>>          > <mailto:rahulk@mvista.com <mailto:rahulk@mvista.com>>> wrote:
>>          >
>>          >     Hi Richard/Alexander,
>>          >
>>          >     I am not able to understand how I can use the below file.
>>          >
>> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-helper/tree/config.json
>>          >
>>          >     did you mean to say that i have to set MACRO in
>>         local.conf based on
>>          >     this file.
>>          >
>>          >     *Thanks & Regards,*
>>          >     Rahul Kumar
>>          >     Software Engineer,Linux Solutions Engineering
>>          >     Group,Montavista Software LLC
>>          >     Email Id: rahulk@mvista.com <mailto:rahulk@mvista.com>
>>         <mailto:rahulk@mvista.com <mailto:rahulk@mvista.com>>
>>          >  <https://plus.google.com/+CodeTwoSoftware>
>>          >
>>          >
>>          >     On Mon, Apr 27, 2020 at 11:46 PM Richard Purdie
>>          >     <richard.purdie@linuxfoundation.org
>>         <mailto:richard.purdie@linuxfoundation.org>
>>          >     <mailto:richard.purdie@linuxfoundation.org
>> <mailto:richard.purdie@linuxfoundation.org>>> wrote:
>>          >
>>          >         On Mon, 2020-04-27 at 18:30 +0200, Alexander Kanavin
>>         wrote:
>>          >          > You need to look at configurations defined here:
>>          >          >
>>          >
>> http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder-helper/tree/config.json
>>          >          > and replicate them locally. Then you can 
>> reproduce the
>>          >         failures that
>>          >          > the AB gets in those configurations.
>>          >
>>          >         That start of the failing logs on the autobuilder
>>         also list out the
>>          >         configuration options for that build.
>>          >
>>          >         Cheers,
>>          >
>>          >         Richard
>>          >
>>          >
>>          >
>>          >
>>
>>
>>         --         # Randy MacLeod
>>         # Wind River Linux
>>
>>     
>>
>
>