[Buildroot] [PATCH] legal-info: Add site to legal info manifest

[Buildroot] [PATCH] legal-info: Add site to legal info manifest

[Clayton Shotwell]

I would like to propose adding the site to the legal-info manifest
files. This gives a little more information on where the sources came
from without adding much overhead. Please note that is is only for
packages where the source is not local or set with OVERRIDE_SRCDIR.

This patch works for the most part. The only issue I see with it occurs
when the SITE for a package uses one of the common url macros such as
BR2_GNU_MIRROR. The legal info manifest ends up having an extra set of
double quotes in the site string.

Signed-off-by: Clayton Shotwell <clshotwe@rockwellcollins.com>
---
 Makefile               |    6 +++---
 package/pkg-generic.mk |    4 +++-
 package/pkg-utils.mk   |    4 ++--
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 41c51c6..31edf9a 100644
--- a/Makefile
+++ b/Makefile
@@ -638,9 +638,9 @@ legal-info-clean:
 legal-info-prepare: $(LEGAL_INFO_DIR)
 	@$(call MESSAGE,"Collecting legal info")
 	@$(call legal-license-file,buildroot,COPYING,COPYING,HOST)
-	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,TARGET)
-	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,HOST)
-	@$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,HOST)
+	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,SOURCE SITE,TARGET)
+	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,SOURCE SITE,HOST)
+	@$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,not saved,HOST)
 	@$(call legal-warning,the Buildroot source code has not been saved)
 	@$(call legal-warning,the toolchain has not been saved)
 	@cp $(BR2_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 339c3eb..eff021b 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -561,10 +561,12 @@ ifneq ($$($(2)_SITE_METHOD),override)
 # Packages that have a tarball need it downloaded and extracted beforehand
 $(1)-legal-info: $(1)-extract $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(4)))
 $(2)_MANIFEST_TARBALL = $$($(2)_SOURCE)
+$(2)_MANIFEST_SITE = $$($(2)_SITE)
 endif
 endif
 endif
 $(2)_MANIFEST_TARBALL ?= not saved
+$(2)_MANIFEST_SITE ?= not saved
 
 # legal-info: produce legally relevant info.
 $(1)-legal-info:
@@ -599,7 +601,7 @@ ifeq ($$($(2)_REDISTRIBUTE),YES)
 endif # redistribute
 
 endif # other packages
-	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL),$(call UPPERCASE,$(4)))
+	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL),$$($(2)_MANIFEST_SITE),$(call UPPERCASE,$(4)))
 endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
 	$(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))
 
diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
index 91a1981..42d5820 100644
--- a/package/pkg-utils.mk
+++ b/package/pkg-utils.mk
@@ -109,8 +109,8 @@ define legal-warning-pkg-savednothing # pkg, {local|override}
 	$(call legal-warning-pkg,$(1),sources and license files not saved ($(2) packages not handled))
 endef
 
-define legal-manifest # pkg, version, license, license-files, source, {HOST|TARGET}
-	echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6))
+define legal-manifest # pkg, version, license, license-files, source, url, {HOST|TARGET}
+	echo '"$(1)","$(2)","$(3)","$(4)","$(5)","$(6)"' >>$(LEGAL_MANIFEST_CSV_$(7))
 endef
 
 define legal-license-header # pkg, license-file, {HOST|TARGET}
-- 
1.7.9.5

[Buildroot] [PATCH] legal-info: Add site to legal info manifest

[Thomas Petazzoni]

Hello Luca,

Could you have a look at this proposal from Clayton regarding the legal
infrastructure?

Thanks a lot,

Thomas

On Mon,  7 Apr 2014 15:45:22 -0500, Clayton Shotwell wrote:
> I would like to propose adding the site to the legal-info manifest
> files. This gives a little more information on where the sources came
> from without adding much overhead. Please note that is is only for
> packages where the source is not local or set with OVERRIDE_SRCDIR.
> 
> This patch works for the most part. The only issue I see with it occurs
> when the SITE for a package uses one of the common url macros such as
> BR2_GNU_MIRROR. The legal info manifest ends up having an extra set of
> double quotes in the site string.
> 
> Signed-off-by: Clayton Shotwell <clshotwe@rockwellcollins.com>
> ---
>  Makefile               |    6 +++---
>  package/pkg-generic.mk |    4 +++-
>  package/pkg-utils.mk   |    4 ++--
>  3 files changed, 8 insertions(+), 6 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 41c51c6..31edf9a 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -638,9 +638,9 @@ legal-info-clean:
>  legal-info-prepare: $(LEGAL_INFO_DIR)
>  	@$(call MESSAGE,"Collecting legal info")
>  	@$(call legal-license-file,buildroot,COPYING,COPYING,HOST)
> -	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,TARGET)
> -	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,HOST)
> -	@$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,HOST)
> +	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,SOURCE SITE,TARGET)
> +	@$(call legal-manifest,PACKAGE,VERSION,LICENSE,LICENSE FILES,SOURCE ARCHIVE,SOURCE SITE,HOST)
> +	@$(call legal-manifest,buildroot,$(BR2_VERSION_FULL),GPLv2+,COPYING,not saved,not saved,HOST)
>  	@$(call legal-warning,the Buildroot source code has not been saved)
>  	@$(call legal-warning,the toolchain has not been saved)
>  	@cp $(BR2_CONFIG) $(LEGAL_INFO_DIR)/buildroot.config
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 339c3eb..eff021b 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -561,10 +561,12 @@ ifneq ($$($(2)_SITE_METHOD),override)
>  # Packages that have a tarball need it downloaded and extracted beforehand
>  $(1)-legal-info: $(1)-extract $(REDIST_SOURCES_DIR_$(call UPPERCASE,$(4)))
>  $(2)_MANIFEST_TARBALL = $$($(2)_SOURCE)
> +$(2)_MANIFEST_SITE = $$($(2)_SITE)
>  endif
>  endif
>  endif
>  $(2)_MANIFEST_TARBALL ?= not saved
> +$(2)_MANIFEST_SITE ?= not saved
>  
>  # legal-info: produce legally relevant info.
>  $(1)-legal-info:
> @@ -599,7 +601,7 @@ ifeq ($$($(2)_REDISTRIBUTE),YES)
>  endif # redistribute
>  
>  endif # other packages
> -	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL),$(call UPPERCASE,$(4)))
> +	@$(call legal-manifest,$$($(2)_RAWNAME),$$($(2)_VERSION),$$($(2)_LICENSE),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_MANIFEST_TARBALL),$$($(2)_MANIFEST_SITE),$(call UPPERCASE,$(4)))
>  endif # ifneq ($(call qstrip,$$($(2)_SOURCE)),)
>  	$(foreach hook,$($(2)_POST_LEGAL_INFO_HOOKS),$(call $(hook))$(sep))
>  
> diff --git a/package/pkg-utils.mk b/package/pkg-utils.mk
> index 91a1981..42d5820 100644
> --- a/package/pkg-utils.mk
> +++ b/package/pkg-utils.mk
> @@ -109,8 +109,8 @@ define legal-warning-pkg-savednothing # pkg, {local|override}
>  	$(call legal-warning-pkg,$(1),sources and license files not saved ($(2) packages not handled))
>  endef
>  
> -define legal-manifest # pkg, version, license, license-files, source, {HOST|TARGET}
> -	echo '"$(1)","$(2)","$(3)","$(4)","$(5)"' >>$(LEGAL_MANIFEST_CSV_$(6))
> +define legal-manifest # pkg, version, license, license-files, source, url, {HOST|TARGET}
> +	echo '"$(1)","$(2)","$(3)","$(4)","$(5)","$(6)"' >>$(LEGAL_MANIFEST_CSV_$(7))
>  endef
>  
>  define legal-license-header # pkg, license-file, {HOST|TARGET}



-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] legal-info: Add site to legal info manifest

[Luca Ceresoli]

Hi Clayton,

Clayton Shotwell wrote:
> I would like to propose adding the site to the legal-info manifest
> files. This gives a little more information on where the sources came
> from without adding much overhead. Please note that is is only for
> packages where the source is not local or set with OVERRIDE_SRCDIR.
>
> This patch works for the most part. The only issue I see with it occurs
> when the SITE for a package uses one of the common url macros such as
> BR2_GNU_MIRROR. The legal info manifest ends up having an extra set of
> double quotes in the site string.
>
> Signed-off-by: Clayton Shotwell <clshotwe@rockwellcollins.com>

Thanks for the proposal.

The idea is generally good to me: it provides a useful info with a
little effort.

The project website URL would be nice as well, but that's not easily
extracted.

Of course the double-quote issue with some packages needs to be fixed.
But this is just "implementation details". Note that, besides packages
from GNU_MIRROR it also affects the kernel from kernel.org, probably
U-Boot and Busybox, maybe a few others.

Instead, the case of packages downloaded from custom locations deserves
a little more thought on what we want to do.

When using e.g. BR2_LINUX_KERNEL_CUSTOM_GIT=y, the REPO_URL may point
either to a public server (github, gitorious, silicon vendors...) or to
an enterprise server. In the former case the repository URL is
informative. In the latter case it is not, and some companies may not
like exposing their server names to the public. You know how netadmins
are made... :)

So I'm not sure of what is the best policy when using custom (non-wget)
download methods.

-- 
Luca

[Buildroot] [PATCH] legal-info: Add site to legal info manifest

[Clayton Shotwell]

Luca,

Luca Ceresoli <luca@lucaceresoli.net> wrote on 04/17/2014 04:47:42 PM:

> From: Luca Ceresoli <luca@lucaceresoli.net>
> To: Clayton Shotwell <clshotwe@rockwellcollins.com>, 
buildroot at buildroot.org
> Date: 04/17/2014 04:47 PM
> Subject: Re: [Buildroot] [PATCH] legal-info: Add site to legal info 
manifest
> 
> Hi Clayton,
> 
> Clayton Shotwell wrote:
> > I would like to propose adding the site to the legal-info manifest
> > files. This gives a little more information on where the sources came
> > from without adding much overhead. Please note that is is only for
> > packages where the source is not local or set with OVERRIDE_SRCDIR.
> >
> > This patch works for the most part. The only issue I see with it 
occurs
> > when the SITE for a package uses one of the common url macros such as
> > BR2_GNU_MIRROR. The legal info manifest ends up having an extra set of
> > double quotes in the site string.
> >
> > Signed-off-by: Clayton Shotwell <clshotwe@rockwellcollins.com>
> 
> Thanks for the proposal.
> 
> The idea is generally good to me: it provides a useful info with a
> little effort.
> 
> The project website URL would be nice as well, but that's not easily
> extracted.

Agreed

> Of course the double-quote issue with some packages needs to be fixed.
> But this is just "implementation details". Note that, besides packages
> from GNU_MIRROR it also affects the kernel from kernel.org, probably
> U-Boot and Busybox, maybe a few others.

I think this could be solved with a creative strip command but I was not
able to come up with anything that worked.  Maybe you or someone else will
have more luck?

> Instead, the case of packages downloaded from custom locations deserves
> a little more thought on what we want to do.
> 
> When using e.g. BR2_LINUX_KERNEL_CUSTOM_GIT=y, the REPO_URL may point
> either to a public server (github, gitorious, silicon vendors...) or to
> an enterprise server. In the former case the repository URL is
> informative. In the latter case it is not, and some companies may not
> like exposing their server names to the public. You know how netadmins
> are made... :)
> 
> So I'm not sure of what is the best policy when using custom (non-wget)
> download methods.

I was assuming the information could be scrubbed if needed from the 
resulting csv files.

Thanks,
Clayton

Clayton Shotwell
Software Engineer
clshotwe at rockwellcollins.com
www.rockwellcollins.com 

[Buildroot] [PATCH] legal-info: Add site to legal info manifest

[Thomas Petazzoni]

Dear Luca Ceresoli,

Would it be possible to help on converging towards a decision on this
patch? Either ask Clayton to implement some changes, or post an updated
version of the patch.

Thanks,

Thomas

On Thu, 17 Apr 2014 23:47:42 +0200, Luca Ceresoli wrote:
> Hi Clayton,
> 
> Clayton Shotwell wrote:
> > I would like to propose adding the site to the legal-info manifest
> > files. This gives a little more information on where the sources came
> > from without adding much overhead. Please note that is is only for
> > packages where the source is not local or set with OVERRIDE_SRCDIR.
> >
> > This patch works for the most part. The only issue I see with it occurs
> > when the SITE for a package uses one of the common url macros such as
> > BR2_GNU_MIRROR. The legal info manifest ends up having an extra set of
> > double quotes in the site string.
> >
> > Signed-off-by: Clayton Shotwell <clshotwe@rockwellcollins.com>
> 
> Thanks for the proposal.
> 
> The idea is generally good to me: it provides a useful info with a
> little effort.
> 
> The project website URL would be nice as well, but that's not easily
> extracted.
> 
> Of course the double-quote issue with some packages needs to be fixed.
> But this is just "implementation details". Note that, besides packages
> from GNU_MIRROR it also affects the kernel from kernel.org, probably
> U-Boot and Busybox, maybe a few others.
> 
> Instead, the case of packages downloaded from custom locations deserves
> a little more thought on what we want to do.
> 
> When using e.g. BR2_LINUX_KERNEL_CUSTOM_GIT=y, the REPO_URL may point
> either to a public server (github, gitorious, silicon vendors...) or to
> an enterprise server. In the former case the repository URL is
> informative. In the latter case it is not, and some companies may not
> like exposing their server names to the public. You know how netadmins
> are made... :)
> 
> So I'm not sure of what is the best policy when using custom (non-wget)
> download methods.
> 



-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [PATCH] legal-info: Add site to legal info manifest

[Clayton Shotwell]

Thomas,

On Wed, Jul 16, 2014 at 3:06 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Luca Ceresoli,
>
> Would it be possible to help on converging towards a decision on this
> patch? Either ask Clayton to implement some changes, or post an updated
> version of the patch.

I will send a new version of the patch since I fixed my previous issue
with the site path having quotation marks in it.

-- 
Thanks,
Clayton

Clayton Shotwell
Software Engineer, Rockwell Collins
clayton.shotwell at rockwellcollins.com