* [RFC PATCH] license: Ensure we find multilib packages also @ 2012-09-13 19:26 Saul Wold 2012-09-13 22:31 ` Paul Eggleton 0 siblings, 1 reply; 11+ messages in thread From: Saul Wold @ 2012-09-13 19:26 UTC (permalink / raw) To: openembedded-core Make sure to find -package, this was causing a failure in the multi-lib build license generation during rootfs. Signed-off-by: Saul Wold <sgw@linux.intel.com> --- meta/classes/license.bbclass | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass index 29fe938..b29067c 100644 --- a/meta/classes/license.bbclass +++ b/meta/classes/license.bbclass @@ -88,7 +88,7 @@ license_create_manifest() { # list of installed packages is broken for deb for pkg in ${INSTALLED_PKGS}; do # not the best way to do this but licenses are not arch dependant iirc - filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1` + filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/*${pkg}| head -1` pkged_pn="$(sed -n 's/^PN: //p' ${filename})" # exclude locale recipes -- 1.7.7.6 ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-13 19:26 [RFC PATCH] license: Ensure we find multilib packages also Saul Wold @ 2012-09-13 22:31 ` Paul Eggleton 2012-09-13 22:43 ` Saul Wold 0 siblings, 1 reply; 11+ messages in thread From: Paul Eggleton @ 2012-09-13 22:31 UTC (permalink / raw) To: Saul Wold; +Cc: openembedded-core On Thursday 13 September 2012 12:26:19 Saul Wold wrote: > Make sure to find -package, this was causing a failure > in the multi-lib build license generation during rootfs. > > Signed-off-by: Saul Wold <sgw@linux.intel.com> > --- > meta/classes/license.bbclass | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) > > diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass > index 29fe938..b29067c 100644 > --- a/meta/classes/license.bbclass > +++ b/meta/classes/license.bbclass > @@ -88,7 +88,7 @@ license_create_manifest() { > # list of installed packages is broken for deb > for pkg in ${INSTALLED_PKGS}; do > # not the best way to do this but licenses are not arch dependant iirc > - filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1` > + filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/*${pkg}| head -1` > pkged_pn="$(sed -n 's/^PN: //p' ${filename})" > > # exclude locale recipes Surely this could end up matching a the wrong file when one package name is a substring of another? Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-13 22:31 ` Paul Eggleton @ 2012-09-13 22:43 ` Saul Wold 2012-09-14 16:01 ` Richard Purdie 0 siblings, 1 reply; 11+ messages in thread From: Saul Wold @ 2012-09-13 22:43 UTC (permalink / raw) To: Paul Eggleton; +Cc: openembedded-core On 09/13/2012 03:31 PM, Paul Eggleton wrote: > On Thursday 13 September 2012 12:26:19 Saul Wold wrote: >> Make sure to find -package, this was causing a failure >> in the multi-lib build license generation during rootfs. >> >> Signed-off-by: Saul Wold <sgw@linux.intel.com> >> --- >> meta/classes/license.bbclass | 2 +- >> 1 files changed, 1 insertions(+), 1 deletions(-) >> >> diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass >> index 29fe938..b29067c 100644 >> --- a/meta/classes/license.bbclass >> +++ b/meta/classes/license.bbclass >> @@ -88,7 +88,7 @@ license_create_manifest() { >> # list of installed packages is broken for deb >> for pkg in ${INSTALLED_PKGS}; do >> # not the best way to do this but licenses are not arch dependant iirc >> - filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1` >> + filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/*${pkg}| head -1` >> pkged_pn="$(sed -n 's/^PN: //p' ${filename})" >> >> # exclude locale recipes > > Surely this could end up matching a the wrong file when one package name is a > substring of another? > I am open to ideas here, I tried the ${MLPREFIX}, but it seems to be empty when rootfs runs, that did not help, I thought about adding the '-', but that would fail in the non-multilib case, how can I determine what the prefix will be to get a more accurate match? Sau! > Cheers, > Paul > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-13 22:43 ` Saul Wold @ 2012-09-14 16:01 ` Richard Purdie 2012-09-14 16:48 ` Mark Hatle 0 siblings, 1 reply; 11+ messages in thread From: Richard Purdie @ 2012-09-14 16:01 UTC (permalink / raw) To: Saul Wold; +Cc: Paul Eggleton, openembedded-core On Thu, 2012-09-13 at 15:43 -0700, Saul Wold wrote: > On 09/13/2012 03:31 PM, Paul Eggleton wrote: > > On Thursday 13 September 2012 12:26:19 Saul Wold wrote: > >> Make sure to find -package, this was causing a failure > >> in the multi-lib build license generation during rootfs. > >> > >> Signed-off-by: Saul Wold <sgw@linux.intel.com> > >> --- > >> meta/classes/license.bbclass | 2 +- > >> 1 files changed, 1 insertions(+), 1 deletions(-) > >> > >> diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass > >> index 29fe938..b29067c 100644 > >> --- a/meta/classes/license.bbclass > >> +++ b/meta/classes/license.bbclass > >> @@ -88,7 +88,7 @@ license_create_manifest() { > >> # list of installed packages is broken for deb > >> for pkg in ${INSTALLED_PKGS}; do > >> # not the best way to do this but licenses are not arch dependant iirc > >> - filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1` > >> + filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/*${pkg}| head -1` > >> pkged_pn="$(sed -n 's/^PN: //p' ${filename})" > >> > >> # exclude locale recipes > > > > Surely this could end up matching a the wrong file when one package name is a > > substring of another? > > > I am open to ideas here, I tried the ${MLPREFIX}, but it seems to be > empty when rootfs runs, that did not help, I thought about adding the > '-', but that would fail in the non-multilib case, how can I determine > what the prefix will be to get a more accurate match? I would observe here that this probably works for ipk and multilib and only breaks for multilib+rpm. The reason is that rpm collapses the namespace when it creates the package list so "lib32-bash" becomes bash. I think this might be an error in however we generate the INSTALLED_PKGS list and we might need to revisit the rpm mechanism and ensure the multilib prefixes get added. Cheers, Richard ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-14 16:01 ` Richard Purdie @ 2012-09-14 16:48 ` Mark Hatle 2012-09-14 20:50 ` Flanagan, Elizabeth 0 siblings, 1 reply; 11+ messages in thread From: Mark Hatle @ 2012-09-14 16:48 UTC (permalink / raw) To: openembedded-core On 9/14/12 11:01 AM, Richard Purdie wrote: > On Thu, 2012-09-13 at 15:43 -0700, Saul Wold wrote: >> On 09/13/2012 03:31 PM, Paul Eggleton wrote: >>> On Thursday 13 September 2012 12:26:19 Saul Wold wrote: >>>> Make sure to find -package, this was causing a failure >>>> in the multi-lib build license generation during rootfs. >>>> >>>> Signed-off-by: Saul Wold <sgw@linux.intel.com> >>>> --- >>>> meta/classes/license.bbclass | 2 +- >>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>> >>>> diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass >>>> index 29fe938..b29067c 100644 >>>> --- a/meta/classes/license.bbclass >>>> +++ b/meta/classes/license.bbclass >>>> @@ -88,7 +88,7 @@ license_create_manifest() { >>>> # list of installed packages is broken for deb >>>> for pkg in ${INSTALLED_PKGS}; do >>>> # not the best way to do this but licenses are not arch dependant iirc >>>> - filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1` >>>> + filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/*${pkg}| head -1` >>>> pkged_pn="$(sed -n 's/^PN: //p' ${filename})" >>>> >>>> # exclude locale recipes >>> >>> Surely this could end up matching a the wrong file when one package name is a >>> substring of another? >>> >> I am open to ideas here, I tried the ${MLPREFIX}, but it seems to be >> empty when rootfs runs, that did not help, I thought about adding the >> '-', but that would fail in the non-multilib case, how can I determine >> what the prefix will be to get a more accurate match? > > I would observe here that this probably works for ipk and multilib and > only breaks for multilib+rpm. The reason is that rpm collapses the > namespace when it creates the package list so "lib32-bash" becomes bash. > I think this might be an error in however we generate the INSTALLED_PKGS > list and we might need to revisit the rpm mechanism and ensure the > multilib prefixes get added. RPM has internal mechanisms to tag both the filetypes and architectures as being different, unlike most of the other packaging formats which assume a package with the same name is the same. The way to deal with this is via the mapping actions. You should be able to map from ML_PREFIX-${PN} to ${PN}.${ARCH} and of course the reverse... How it works.... So during package generation we know the package arch is unique and just strip off the mlprefix. To do the reverse iterate over all of the available MULTILIBS, there is code in rootfs_rpm.bbclass already to do this. Then reconstruct the package name based on the multilib configuration. --Mark > Cheers, > > Richard > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-14 16:48 ` Mark Hatle @ 2012-09-14 20:50 ` Flanagan, Elizabeth 2012-09-14 22:09 ` Richard Purdie 2012-09-14 22:11 ` Paul Eggleton 0 siblings, 2 replies; 11+ messages in thread From: Flanagan, Elizabeth @ 2012-09-14 20:50 UTC (permalink / raw) To: Mark Hatle; +Cc: openembedded-core On Fri, Sep 14, 2012 at 9:48 AM, Mark Hatle <mark.hatle@windriver.com> wrote: > On 9/14/12 11:01 AM, Richard Purdie wrote: >> >> On Thu, 2012-09-13 at 15:43 -0700, Saul Wold wrote: >>> >>> On 09/13/2012 03:31 PM, Paul Eggleton wrote: >>>> >>>> On Thursday 13 September 2012 12:26:19 Saul Wold wrote: >>>>> >>>>> Make sure to find -package, this was causing a failure >>>>> in the multi-lib build license generation during rootfs. >>>>> >>>>> Signed-off-by: Saul Wold <sgw@linux.intel.com> >>>>> --- >>>>> meta/classes/license.bbclass | 2 +- >>>>> 1 files changed, 1 insertions(+), 1 deletions(-) >>>>> >>>>> diff --git a/meta/classes/license.bbclass >>>>> b/meta/classes/license.bbclass >>>>> index 29fe938..b29067c 100644 >>>>> --- a/meta/classes/license.bbclass >>>>> +++ b/meta/classes/license.bbclass >>>>> @@ -88,7 +88,7 @@ license_create_manifest() { >>>>> # list of installed packages is broken for deb >>>>> for pkg in ${INSTALLED_PKGS}; do >>>>> # not the best way to do this but licenses are not arch >>>>> dependant iirc >>>>> - filename=`ls >>>>> ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1` >>>>> + filename=`ls >>>>> ${TMPDIR}/pkgdata/*/runtime-reverse/*${pkg}| head -1` >>>>> pkged_pn="$(sed -n 's/^PN: //p' ${filename})" >>>>> >>>>> # exclude locale recipes >>>> >>>> >>>> Surely this could end up matching a the wrong file when one package name >>>> is a >>>> substring of another? >>>> >>> I am open to ideas here, I tried the ${MLPREFIX}, but it seems to be >>> empty when rootfs runs, that did not help, I thought about adding the >>> '-', but that would fail in the non-multilib case, how can I determine >>> what the prefix will be to get a more accurate match? I'm not particularly happy with how we've been doing manifests for a while now and have been hacking at a revamp of it for the past 2 months on and off. We really shouldn't be relying on package data at all for license manifests. As the manifests are relying on list_installed_packages, they end up inaccurate anyway as list_installed_packages does exactly that. List software installed via package. Not installed via the package manager? Then you aren't in the manifest. This ends up missing quite a few things necessary from a release engineering standpoint (like, hey, modutils? The kernel? Not listed in the manifest. IMHO this is wrong.) As a release engineer, I kind of don't really care if the software on my image was installed via the packaging system. I just want a list of what is on the images so I can audit my images to ensure GPL commitments. One way I've been looking at fixing this is via event handlers. I'll have some code for RFC on this hopefully soon, but a preview (which may or may not work it's still a WIP) is at: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=eflanagan/license_manifest One thing the patch I've been working on does other than creates accurate manifests is it also tries to create a semi-compliant SPDX 1.0 manifest. I know this is late in the cycle so it might not make 1.3, but this may be something we want to look at to end up solving this issue once and for all. -b >> >> >> I would observe here that this probably works for ipk and multilib and >> only breaks for multilib+rpm. The reason is that rpm collapses the >> namespace when it creates the package list so "lib32-bash" becomes bash. >> I think this might be an error in however we generate the INSTALLED_PKGS >> list and we might need to revisit the rpm mechanism and ensure the >> multilib prefixes get added. > > > RPM has internal mechanisms to tag both the filetypes and architectures as > being different, unlike most of the other packaging formats which assume a > package with the same name is the same. > > The way to deal with this is via the mapping actions. You should be able to > map from ML_PREFIX-${PN} to ${PN}.${ARCH} and of course the reverse... > > How it works.... > > So during package generation we know the package arch is unique and just > strip off the mlprefix. To do the reverse iterate over all of the available > MULTILIBS, there is code in rootfs_rpm.bbclass already to do this. Then > reconstruct the package name based on the multilib configuration. > > --Mark > > > >> Cheers, >> >> Richard >> >> >> _______________________________________________ >> Openembedded-core mailing list >> Openembedded-core@lists.openembedded.org >> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core >> > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core -- Elizabeth Flanagan Yocto Project Build and Release ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-14 20:50 ` Flanagan, Elizabeth @ 2012-09-14 22:09 ` Richard Purdie 2012-09-14 22:11 ` Paul Eggleton 1 sibling, 0 replies; 11+ messages in thread From: Richard Purdie @ 2012-09-14 22:09 UTC (permalink / raw) To: Flanagan, Elizabeth; +Cc: openembedded-core On Fri, 2012-09-14 at 13:50 -0700, Flanagan, Elizabeth wrote: > On Fri, Sep 14, 2012 at 9:48 AM, Mark Hatle <mark.hatle@windriver.com> wrote: > > On 9/14/12 11:01 AM, Richard Purdie wrote: > >> > >> On Thu, 2012-09-13 at 15:43 -0700, Saul Wold wrote: > >>> > >>> On 09/13/2012 03:31 PM, Paul Eggleton wrote: > >>>> > >>>> On Thursday 13 September 2012 12:26:19 Saul Wold wrote: > >>>>> > >>>>> Make sure to find -package, this was causing a failure > >>>>> in the multi-lib build license generation during rootfs. > >>>>> > >>>>> Signed-off-by: Saul Wold <sgw@linux.intel.com> > >>>>> --- > >>>>> meta/classes/license.bbclass | 2 +- > >>>>> 1 files changed, 1 insertions(+), 1 deletions(-) > >>>>> > >>>>> diff --git a/meta/classes/license.bbclass > >>>>> b/meta/classes/license.bbclass > >>>>> index 29fe938..b29067c 100644 > >>>>> --- a/meta/classes/license.bbclass > >>>>> +++ b/meta/classes/license.bbclass > >>>>> @@ -88,7 +88,7 @@ license_create_manifest() { > >>>>> # list of installed packages is broken for deb > >>>>> for pkg in ${INSTALLED_PKGS}; do > >>>>> # not the best way to do this but licenses are not arch > >>>>> dependant iirc > >>>>> - filename=`ls > >>>>> ${TMPDIR}/pkgdata/*/runtime-reverse/${pkg}| head -1` > >>>>> + filename=`ls > >>>>> ${TMPDIR}/pkgdata/*/runtime-reverse/*${pkg}| head -1` > >>>>> pkged_pn="$(sed -n 's/^PN: //p' ${filename})" > >>>>> > >>>>> # exclude locale recipes > >>>> > >>>> > >>>> Surely this could end up matching a the wrong file when one package name > >>>> is a > >>>> substring of another? > >>>> > >>> I am open to ideas here, I tried the ${MLPREFIX}, but it seems to be > >>> empty when rootfs runs, that did not help, I thought about adding the > >>> '-', but that would fail in the non-multilib case, how can I determine > >>> what the prefix will be to get a more accurate match? > > I'm not particularly happy with how we've been doing manifests for a > while now and have been hacking at a revamp of it for the past 2 > months on and off. > > We really shouldn't be relying on package data at all for license > manifests. As the manifests are relying on list_installed_packages, > they end up inaccurate anyway as list_installed_packages does exactly > that. List software installed via package. Not installed via the > package manager? Then you aren't in the manifest. This ends up missing > quite a few things necessary from a release engineering standpoint > (like, hey, modutils? The kernel? Not listed in the manifest. IMHO > this is wrong.) > > As a release engineer, I kind of don't really care if the software on > my image was installed via the packaging system. I just want a list of > what is on the images so I can audit my images to ensure GPL > commitments. > > One way I've been looking at fixing this is via event handlers. I'll > have some code for RFC on this hopefully soon, but a preview (which > may or may not work it's still a WIP) is at: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=eflanagan/license_manifest > > One thing the patch I've been working on does other than creates > accurate manifests is it also tries to create a semi-compliant SPDX > 1.0 manifest. > > I know this is late in the cycle so it might not make 1.3, but this > may be something we want to look at to end up solving this issue once > and for all. This depends whether this is a release manifest or an image manifest. You likely want *both* under different circumstances. If the kernel is installed in the image, it will be listed in the image manifest. We just need to be really clear about what the manifest is for. As for modutils, we use kmod now. Basically anything doing deployment should have a manifest associated with it. That is do_deploy (kernel, uboot, some small list of others) and do_rootfs. do_package_write_* are also arguably deployment in another form, as are sstate feeds. Cheers, Richard ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-14 20:50 ` Flanagan, Elizabeth 2012-09-14 22:09 ` Richard Purdie @ 2012-09-14 22:11 ` Paul Eggleton 2012-09-14 22:26 ` Flanagan, Elizabeth 1 sibling, 1 reply; 11+ messages in thread From: Paul Eggleton @ 2012-09-14 22:11 UTC (permalink / raw) To: Flanagan, Elizabeth; +Cc: openembedded-core On Friday 14 September 2012 13:50:22 Flanagan, Elizabeth wrote: > We really shouldn't be relying on package data at all for license > manifests. As the manifests are relying on list_installed_packages, > they end up inaccurate anyway as list_installed_packages does exactly > that. List software installed via package. Not installed via the > package manager? Then you aren't in the manifest. This ends up missing > quite a few things necessary from a release engineering standpoint > (like, hey, modutils? The kernel? Not listed in the manifest. IMHO > this is wrong.) What do you mean by modutils? If you mean the earlier suggestion that module- init-tools was missing from the manifest, I'm fairly certain that was a misunderstanding because of it being replaced by kmod. The thing is, we have no other means of accurately determining the contents of the image than the installed package list, given that the package manager is ultimately in charge of resolving and installing dependencies during image creation. The list may not cover additional files copied into tmp/deploy as part of building the image (kernel, bootloader, etc.) - however, that does not make it inaccurate as to the contents of the image, let's be clear about that. I agree we do need to write out the license information for these additional files; however, since they aren't actually *in* the image itself, I think we need to list these in a separate file. What happens for example if the system builds u-boot as a result of building my image, but I don't ever end up distributing that with the image? As to the mechanism for picking these up, the only real possibility I can see is to hook into do_deploy; this is not currently straightforward though since that's not something that is implemented generically at the moment. Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-14 22:11 ` Paul Eggleton @ 2012-09-14 22:26 ` Flanagan, Elizabeth 2012-09-14 22:31 ` Mark Hatle 2012-09-14 23:09 ` Paul Eggleton 0 siblings, 2 replies; 11+ messages in thread From: Flanagan, Elizabeth @ 2012-09-14 22:26 UTC (permalink / raw) To: Paul Eggleton; +Cc: openembedded-core On Fri, Sep 14, 2012 at 3:11 PM, Paul Eggleton <paul.eggleton@linux.intel.com> wrote: > On Friday 14 September 2012 13:50:22 Flanagan, Elizabeth wrote: >> We really shouldn't be relying on package data at all for license >> manifests. As the manifests are relying on list_installed_packages, >> they end up inaccurate anyway as list_installed_packages does exactly >> that. List software installed via package. Not installed via the >> package manager? Then you aren't in the manifest. This ends up missing >> quite a few things necessary from a release engineering standpoint >> (like, hey, modutils? The kernel? Not listed in the manifest. IMHO >> this is wrong.) > > What do you mean by modutils? If you mean the earlier suggestion that module- > init-tools was missing from the manifest, I'm fairly certain that was a > misunderstanding because of it being replaced by kmod. > Correct. Even so, there are things obviously missing from the manifests. Kernel is the most obvious. > The thing is, we have no other means of accurately determining the contents of > the image than the installed package list, given that the package manager is > ultimately in charge of resolving and installing dependencies during image > creation. The list may not cover additional files copied into tmp/deploy as > part of building the image (kernel, bootloader, etc.) - however, that does not > make it inaccurate as to the contents of the image, let's be clear about that. > No, but it's like the joke about the two software managers in a hot air balloon asking the engineer on the group where they are only to be told "In a hot air balloon". It's technically correct but the user case I keep running into here is that people want to know what they're deploying with a product. The expectation people have (and right or wrong, it's what I keep running into) is that everything that's on the hddimg is what is on the manifest. > I agree we do need to write out the license information for these additional > files; however, since they aren't actually *in* the image itself, I think we > need to list these in a separate file. What happens for example if the system > builds u-boot as a result of building my image, but I don't ever end up > distributing that with the image? Then you have a manifest that has software that you end up complying with the GPL (when you don't really need to) verses having a manifest that doesn't list software which you do need to comply with the GPL. Both are wrong, but one is more wrong. RP is right here. We probably need to list a few different use cases here and support the most common ones. > > As to the mechanism for picking these up, the only real possibility I can see > is to hook into do_deploy; this is not currently straightforward though since > that's not something that is implemented generically at the moment. I'm currently hooking into do_package because, as best I can figure it out, everything ends up getting packaged, even if it's not installed via package. I'm happy for other suggestions though. -b > > Cheers, > Paul > > -- > > Paul Eggleton > Intel Open Source Technology Centre -- Elizabeth Flanagan Yocto Project Build and Release ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-14 22:26 ` Flanagan, Elizabeth @ 2012-09-14 22:31 ` Mark Hatle 2012-09-14 23:09 ` Paul Eggleton 1 sibling, 0 replies; 11+ messages in thread From: Mark Hatle @ 2012-09-14 22:31 UTC (permalink / raw) To: Flanagan, Elizabeth; +Cc: Paul Eggleton, openembedded-core On 9/14/12 5:26 PM, Flanagan, Elizabeth wrote: > On Fri, Sep 14, 2012 at 3:11 PM, Paul Eggleton > <paul.eggleton@linux.intel.com> wrote: >> On Friday 14 September 2012 13:50:22 Flanagan, Elizabeth wrote: >>> We really shouldn't be relying on package data at all for license >>> manifests. As the manifests are relying on list_installed_packages, >>> they end up inaccurate anyway as list_installed_packages does exactly >>> that. List software installed via package. Not installed via the >>> package manager? Then you aren't in the manifest. This ends up missing >>> quite a few things necessary from a release engineering standpoint >>> (like, hey, modutils? The kernel? Not listed in the manifest. IMHO >>> this is wrong.) >> >> What do you mean by modutils? If you mean the earlier suggestion that module- >> init-tools was missing from the manifest, I'm fairly certain that was a >> misunderstanding because of it being replaced by kmod. >> > > Correct. Even so, there are things obviously missing from the > manifests. Kernel is the most obvious. > >> The thing is, we have no other means of accurately determining the contents of >> the image than the installed package list, given that the package manager is >> ultimately in charge of resolving and installing dependencies during image >> creation. The list may not cover additional files copied into tmp/deploy as >> part of building the image (kernel, bootloader, etc.) - however, that does not >> make it inaccurate as to the contents of the image, let's be clear about that. >> > > No, but it's like the joke about the two software managers in a hot > air balloon asking the engineer on the group where they are only to be > told "In a hot air balloon". It's technically correct but the user > case I keep running into here is that people want to know what they're > deploying with a product. The expectation people have (and right or > wrong, it's what I keep running into) is that everything that's on the > hddimg is what is on the manifest. From thinking about the deployment of the rootfs through images, it seems to me it should be the responsibility of the components populating the items to produce a manifest of what they've done. Be it install a package, or copy a file. Then at the end collect together these manifest fragments to describe exactly what was done (and where it was done to!). >> I agree we do need to write out the license information for these additional >> files; however, since they aren't actually *in* the image itself, I think we >> need to list these in a separate file. What happens for example if the system >> builds u-boot as a result of building my image, but I don't ever end up >> distributing that with the image? > > Then you have a manifest that has software that you end up complying > with the GPL (when you don't really need to) verses having a manifest > that doesn't list software which you do need to comply with the GPL. > Both are wrong, but one is more wrong. > > RP is right here. We probably need to list a few different use cases > here and support the most common ones. > >> >> As to the mechanism for picking these up, the only real possibility I can see >> is to hook into do_deploy; this is not currently straightforward though since >> that's not something that is implemented generically at the moment. > > I'm currently hooking into do_package because, as best I can figure it > out, everything ends up getting packaged, even if it's not installed > via package. I'm happy for other suggestions though. > > -b > >> >> Cheers, >> Paul >> >> -- >> >> Paul Eggleton >> Intel Open Source Technology Centre > > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC PATCH] license: Ensure we find multilib packages also 2012-09-14 22:26 ` Flanagan, Elizabeth 2012-09-14 22:31 ` Mark Hatle @ 2012-09-14 23:09 ` Paul Eggleton 1 sibling, 0 replies; 11+ messages in thread From: Paul Eggleton @ 2012-09-14 23:09 UTC (permalink / raw) To: Flanagan, Elizabeth; +Cc: openembedded-core On Friday 14 September 2012 15:26:14 Flanagan, Elizabeth wrote: > > The thing is, we have no other means of accurately determining the > > contents of the image than the installed package list, given that the > > package manager is ultimately in charge of resolving and installing > > dependencies during image creation. The list may not cover additional > > files copied into tmp/deploy as part of building the image (kernel, > > bootloader, etc.) - however, that does not make it inaccurate as to the > > contents of the image, let's be clear about that. > > No, but it's like the joke about the two software managers in a hot > air balloon asking the engineer on the group where they are only to be > told "In a hot air balloon". It's technically correct but the user > case I keep running into here is that people want to know what they're > deploying with a product. Again, I absolutely agree we need to report the licenses for these files. I'm just advocating not commingling two separate things. > The expectation people have (and right or > wrong, it's what I keep running into) is that everything that's on the > hddimg is what is on the manifest. hddimg files are a special situation anyway though. To handle those properly we would have to use some special license reporting in the same place where the hddimg is composed, because we cannot really tell for certain anywhere else in the code as to what is in it. > > I agree we do need to write out the license information for these > > additional files; however, since they aren't actually *in* the image > > itself, I think we need to list these in a separate file. What happens > > for example if the system builds u-boot as a result of building my image, > > but I don't ever end up distributing that with the image? > > Then you have a manifest that has software that you end up complying > with the GPL (when you don't really need to) verses having a manifest > that doesn't list software which you do need to comply with the GPL. > Both are wrong, but one is more wrong. > > RP is right here. We probably need to list a few different use cases > here and support the most common ones. I think we can satisfy all of these cases in the manner I described - a manifest for the image (what we already have) and an additional manifest beside it that lists all of the additionally deployed files along with their license information. If the files are right next to eachother with the same base name there can't be any confusion. > > As to the mechanism for picking these up, the only real possibility I can > > see is to hook into do_deploy; this is not currently straightforward > > though since that's not something that is implemented generically at the > > moment. > > I'm currently hooking into do_package because, as best I can figure it > out, everything ends up getting packaged, even if it's not installed > via package. I'm happy for other suggestions though. pkgdata (which writes files into TMPDIR/pkgdata) already hooks into do_package and already lists LICENSE on a per-package basis for every package written out. Clearly we can't just take absolutely every package produced as the list for the manifest though, because not all of that gets installed into the image. Another more serious issue is that it's not guaranteed that everything that gets deployed is packaged, so this probably wouldn't satisfactorily solve the issue anyway. As far as I can see, do_deploy is the only way, since we want to know what is deployed and which recipe deployed it. We'll just have to adjust things so that there is a central point we can tap into for do_deploy in order to collect the desired information - I haven't looked into how yet but I don't anticipate it being too hard. I think we're also going to have to add a specific call that can be made from anywhere during do_rootfs that adds a recipe to be reported in the manifest. We can use this for custom cases such as hddimg and provide it for anyone doing anything similar to call manually (e.g. adding non-packaged files into their image in a post-processing function - not really recommended but I know people do do it). Cheers, Paul -- Paul Eggleton Intel Open Source Technology Centre ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2012-09-14 23:22 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-09-13 19:26 [RFC PATCH] license: Ensure we find multilib packages also Saul Wold 2012-09-13 22:31 ` Paul Eggleton 2012-09-13 22:43 ` Saul Wold 2012-09-14 16:01 ` Richard Purdie 2012-09-14 16:48 ` Mark Hatle 2012-09-14 20:50 ` Flanagan, Elizabeth 2012-09-14 22:09 ` Richard Purdie 2012-09-14 22:11 ` Paul Eggleton 2012-09-14 22:26 ` Flanagan, Elizabeth 2012-09-14 22:31 ` Mark Hatle 2012-09-14 23:09 ` Paul Eggleton
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox