[Buildroot] [RFC PATCH 0/2] gitorious helper

[Buildroot] [RFC PATCH 0/2] gitorious helper

[Alexandre Belloni]

The following patches introduce a gitroious helper to download tarballs for
gitorious. I currently see two shortcomings:
 - the tarball URL is https://gitorious.org/<project>/<repo>/archive/<version>.tar.gz
   so the downloaded tarball doesn't have any name appart from <version>.tar.gz
 - It needs two separate functions for target and host packages.

Alexandre Belloni (2):
  infra: introduce gitorious helper function
  packages: use gitorious helper when applicable

 boot/xloader/xloader.mk                 |  2 +-
 docs/manual/adding-packages-generic.txt | 10 +++++-----
 docs/manual/adding-packages-tips.txt    | 18 ++++++++++++++++++
 package/can-utils/can-utils.mk          |  2 +-
 package/lpc3250loader/lpc3250loader.mk  |  2 +-
 package/pkg-download.mk                 |  8 ++++++++
 package/ti-uim/ti-uim.mk                |  2 +-
 7 files changed, 35 insertions(+), 9 deletions(-)

-- 
1.8.3.2

[Buildroot] [PATCH 1/2] infra: introduce gitorious helper function

[Alexandre Belloni]

Introduce a gitorious helper function to make it easy to retrieve
gitorious packages without using git whose port is often blocked on
enterprise networks.

usage: $(eval $(call gitorious,project,repository))

Signed-off-by: Alexandre Belloni <alexandre.belloni@free-electrons.com>
---
 docs/manual/adding-packages-generic.txt | 10 +++++-----
 docs/manual/adding-packages-tips.txt    | 18 ++++++++++++++++++
 package/pkg-download.mk                 |  8 ++++++++
 3 files changed, 31 insertions(+), 5 deletions(-)

diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index 14971b502ba9..d92e49bdcdc3 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -231,11 +231,11 @@ information is (assuming the package name is +libfoo+) :
   types for retrieving package tarballs. Git, Subversion, Mercurial,
   and Bazaar are supported URL types for retrieving packages directly
   from source code management systems. There is a helper function to make
-  it easier to download source tarballs from github (refer to
-  xref:github-download-url[] for details). A filesystem path may be used
-  to specify either a tarball or a directory containing the package
-  source code. See +LIBFOO_SITE_METHOD+ below for more details on how
-  retrieval works. +
+  it easier to download source tarballs from github or gitorious (refer
+  to xref:github-download-url[] and xref:gitorious-download-url[] for
+  details). A filesystem path may be used to specify either a tarball or
+  a directory containing the package source code. See
+  +LIBFOO_SITE_METHOD+ below for more details on how retrieval works. +
   Note that SCP URLs should be of the form
   +scp://[user@]host:filepath+, and that filepath is relative to the
   user's home directory, so you may want to prepend the path with a
diff --git a/docs/manual/adding-packages-tips.txt b/docs/manual/adding-packages-tips.txt
index 73c25be015ab..b51105eae9f2 100644
--- a/docs/manual/adding-packages-tips.txt
+++ b/docs/manual/adding-packages-tips.txt
@@ -55,3 +55,21 @@ FOO_SITE = $(call github,<user>,<package>,$(FOO_VERSION))
   Buildroot (e.g.: +foo-f6fb6654af62045239caed5950bc6c7971965e60.tar.gz+),
   so it is not necessary to specify it in the +.mk+ file.
 - When using a commit ID as version, you should use the full 40 hex characters.
+
+
+[[gitorious-download-url]]
+How to add a package from gitorious
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+As with github, you can use the 'gitorious' helper function to download
+tarballs from gitorious:
+
+------------------------
+FOO_VERSION = v1.0 # tag or full commit ID
+$(eval $(call gitorious,<project>,<repo>))
+------------------------
+
+.Notes
+- For host packages, use host-gitorious
+- The FOO_VERSION can either be a tag or a commit ID.
+- When using a commit ID as version, you should use the full 40 hex characters.
diff --git a/package/pkg-download.mk b/package/pkg-download.mk
index 758b211097e8..531df49b60f2 100644
--- a/package/pkg-download.mk
+++ b/package/pkg-download.mk
@@ -58,6 +58,14 @@ domainseparator=$(if $(1),$(1),/)
 # github(user,package,version): returns site of github repository
 github = https://github.com/$(1)/$(2)/tarball/$(3)
 
+# gitorious(project,repo): sets <foo>_SITE and <foo>_SOURCE for a gitorious repository 
+define inner-gitorious
+$(1)_SITE = https://gitorious.org/$(2)/$(3)/archive
+$(1)_SOURCE = $($(1)_VERSION).tar.gz
+endef
+gitorious = $(call inner-gitorious,$(call UPPERCASE,$(call pkgname)),$(1),$(2))
+host-gitorious = $(call inner-gitorious,$(call UPPERCASE,host-$(call pkgname)),$(1),$(2))
+
 ################################################################################
 # The DOWNLOAD_* helpers are in charge of getting a working copy
 # of the source repository for their corresponding SCM,
-- 
1.8.3.2

[Buildroot] [PATCH 2/2] packages: use gitorious helper when applicable

[Alexandre Belloni]

Signed-off-by: Alexandre Belloni <alexandre.belloni@free-electrons.com>
---
 boot/xloader/xloader.mk                | 2 +-
 package/can-utils/can-utils.mk         | 2 +-
 package/lpc3250loader/lpc3250loader.mk | 2 +-
 package/ti-uim/ti-uim.mk               | 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/boot/xloader/xloader.mk b/boot/xloader/xloader.mk
index 00c63b1efb91..0ec11de1d6af 100644
--- a/boot/xloader/xloader.mk
+++ b/boot/xloader/xloader.mk
@@ -5,7 +5,7 @@
 ################################################################################
 
 XLOADER_VERSION    = 6f3a26101303051e0f91b6213735b68ce804e94e
-XLOADER_SITE       = git://gitorious.org/x-loader/x-loader.git
+$(eval $(call gitorious,x-loader,x-loader))
 XLOADER_BOARD_NAME = $(call qstrip,$(BR2_TARGET_XLOADER_BOARDNAME))
 
 XLOADER_LICENSE = GPLv2+
diff --git a/package/can-utils/can-utils.mk b/package/can-utils/can-utils.mk
index 743fe7ce0b2b..d5f6673dd25f 100644
--- a/package/can-utils/can-utils.mk
+++ b/package/can-utils/can-utils.mk
@@ -5,7 +5,7 @@
 ################################################################################
 
 CAN_UTILS_VERSION = 16c970d40e6c19dde705bad4751bab1a3a4f3a0d
-CAN_UTILS_SITE = git://gitorious.org/linux-can/can-utils.git
+$(eval $(call gitorious,linux-can,can-utils))
 CAN_UTILS_AUTORECONF = YES
 
 $(eval $(autotools-package))
diff --git a/package/lpc3250loader/lpc3250loader.mk b/package/lpc3250loader/lpc3250loader.mk
index 42beb7bc84b4..606e446c20bb 100644
--- a/package/lpc3250loader/lpc3250loader.mk
+++ b/package/lpc3250loader/lpc3250loader.mk
@@ -4,8 +4,8 @@
 #
 ################################################################################
 
-HOST_LPC3250LOADER_SITE = git://gitorious.org/lpc3250loader/lpc3250loader.git
 HOST_LPC3250LOADER_VERSION = 1.0
+$(eval $(call host-gitorious,lpc3250loader,lpc3250loader))
 
 define HOST_LPC3250LOADER_INSTALL_CMDS
 	$(INSTALL) -m 0755 -D $(@D)/LPC3250loader.py \
diff --git a/package/ti-uim/ti-uim.mk b/package/ti-uim/ti-uim.mk
index b49818004989..034388a08d1e 100644
--- a/package/ti-uim/ti-uim.mk
+++ b/package/ti-uim/ti-uim.mk
@@ -5,7 +5,7 @@
 ################################################################################
 
 TI_UIM_VERSION = c73894456df5def97111cb33d2106b684b8b7959
-TI_UIM_SITE = git://gitorious.org/uim/uim.git
+$(eval $(call gitorious,uim,uim))
 TI_UIM_LICENSE = GPLv2+
 
 define TI_UIM_BUILD_CMDS
-- 
1.8.3.2

[Buildroot] [RFC PATCH 0/2] gitorious helper

[Thomas De Schampheleire]

Hi Alexandre,

On Wed, Dec 25, 2013 at 5:18 PM, Alexandre Belloni
<alexandre.belloni@free-electrons.com> wrote:
> The following patches introduce a gitroious helper to download tarballs for
> gitorious. I currently see two shortcomings:
>  - the tarball URL is https://gitorious.org/<project>/<repo>/archive/<version>.tar.gz
>    so the downloaded tarball doesn't have any name appart from <version>.tar.gz
>  - It needs two separate functions for target and host packages.
>
> Alexandre Belloni (2):
>   infra: introduce gitorious helper function
>   packages: use gitorious helper when applicable

Just a quick note: I'm going to review this (hopefully soon)...

Best regards,
Thomas

[Buildroot] [RFC PATCH 0/2] gitorious helper

[Thomas De Schampheleire]

Hi Alexandre,


On Wed, Dec 25, 2013 at 5:18 PM, Alexandre Belloni
<alexandre.belloni@free-electrons.com> wrote:
> The following patches introduce a gitroious helper to download tarballs for
> gitorious. I currently see two shortcomings:
>  - the tarball URL is https://gitorious.org/<project>/<repo>/archive/<version>.tar.gz
>    so the downloaded tarball doesn't have any name appart from <version>.tar.gz

This is indeed a pity. Not only because it makes the helper more
complex, but also because the dl/ directory will contain a 'magic'
file called <version>.tar.gz; the package it belongs to is not clear
to the user.

I propose two alternatives:
1. discuss the situation with the Gitorious people, requesting them to
provide a downloadable tarball with the name
<package>-<version>.tar.gz (or whichever extension), like the rest of
the software world does.

2. accept the fact that the URL cannot be changed, but modify the wget
download helper so that it can rename the file after download
(clarifying the list of files in dl/)

In the first alternative, the gitorious helper can be a one-liner just
as for github, and the .mk files only need to specify FOO_SITE.
In the second alternative, we need to set two variables FOO_SITE and
FOO_SOURCE. In your current patchset, you have done this with one
helper that sets both variables. As a result, the usage compared to
the github helper is fundamentally different.
For github:
FOO_SITE = $(call github,username,package,$(FOO_VERSION))
while for gitorious:
$(call gitorious,username,package,$(FOO_VERSION)

I don't like this difference, and suggest two alternatives:
A. split the helper in two, so that:
FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION))
FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))

This is then equivalent to github except the new FOO_SOURCE, which is
simply not present for github (at the moment).

B. Keep the current gitorious helper, but apply the same strategy to
the github helper, so that its usage also becomes
$(call github,username,package,$(FOO_VERSION))


>  - It needs two separate functions for target and host packages.
>

Can you clarify this?
In package/pkg-generic.mk, if HOST_FOO_VERSION/SOURCE is not defined
and we try to build a host package, its value is taken from
FOO_VERSION/SOURCE. Then why do we need to have a separate helper?

Thanks,
Thomas

[Buildroot] [RFC PATCH 0/2] gitorious helper

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Wed, 8 Jan 2014 08:54:37 +0100, Thomas De Schampheleire wrote:

> 1. discuss the situation with the Gitorious people, requesting them to
> provide a downloadable tarball with the name
> <package>-<version>.tar.gz (or whichever extension), like the rest of
> the software world does.

This would be an interesting thing to do, indeed.

> 2. accept the fact that the URL cannot be changed, but modify the wget
> download helper so that it can rename the file after download
> (clarifying the list of files in dl/)
> 
> In the first alternative, the gitorious helper can be a one-liner just
> as for github, and the .mk files only need to specify FOO_SITE.
> In the second alternative, we need to set two variables FOO_SITE and
> FOO_SOURCE. In your current patchset, you have done this with one
> helper that sets both variables. As a result, the usage compared to
> the github helper is fundamentally different.
> For github:
> FOO_SITE = $(call github,username,package,$(FOO_VERSION))
> while for gitorious:
> $(call gitorious,username,package,$(FOO_VERSION)
> 
> I don't like this difference, and suggest two alternatives:
> A. split the helper in two, so that:
> FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION))
> FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))

I very much prefer this solution. I don't really like the helper that
magically defines FOO_SITE and FOO_VERSION.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [RFC PATCH 0/2] gitorious helper

[Alexandre Belloni]

Hi,

On 10/01/2014 at 16:35:10 +0800, Thomas Petazzoni wrote :
> > 2. accept the fact that the URL cannot be changed, but modify the wget
> > download helper so that it can rename the file after download
> > (clarifying the list of files in dl/)
> > 
> > In the first alternative, the gitorious helper can be a one-liner just
> > as for github, and the .mk files only need to specify FOO_SITE.
> > In the second alternative, we need to set two variables FOO_SITE and
> > FOO_SOURCE. In your current patchset, you have done this with one
> > helper that sets both variables. As a result, the usage compared to
> > the github helper is fundamentally different.
> > For github:
> > FOO_SITE = $(call github,username,package,$(FOO_VERSION))
> > while for gitorious:
> > $(call gitorious,username,package,$(FOO_VERSION)
> > 
> > I don't like this difference, and suggest two alternatives:
> > A. split the helper in two, so that:
> > FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION))
> > FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))
> 
> I very much prefer this solution. I don't really like the helper that
> magically defines FOO_SITE and FOO_VERSION.
> 

After discussing quickly with Thomas, we feel that we can drop it as
this would not really help with readability.

Regards,

-- 
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] [RFC PATCH 0/2] gitorious helper

[Thomas De Schampheleire]

Alexandre Belloni <alexandre.belloni@free-electrons.com> schreef:
>Hi,
>
>On 10/01/2014 at 16:35:10 +0800, Thomas Petazzoni wrote :
>> > 2. accept the fact that the URL cannot be changed, but modify the wget
>> > download helper so that it can rename the file after download
>> > (clarifying the list of files in dl/)
>> > 
>> > In the first alternative, the gitorious helper can be a one-liner just
>> > as for github, and the .mk files only need to specify FOO_SITE.
>> > In the second alternative, we need to set two variables FOO_SITE and
>> > FOO_SOURCE. In your current patchset, you have done this with one
>> > helper that sets both variables. As a result, the usage compared to
>> > the github helper is fundamentally different.
>> > For github:
>> > FOO_SITE = $(call github,username,package,$(FOO_VERSION))
>> > while for gitorious:
>> > $(call gitorious,username,package,$(FOO_VERSION)
>> > 
>> > I don't like this difference, and suggest two alternatives:
>> > A. split the helper in two, so that:
>> > FOO_SITE = $(call gitorious-site,username,package,$(FOO_VERSION))
>> > FOO_SOURCE = $(call gitorious-version,username,package,$(FOO_VERSION))
>> 
>> I very much prefer this solution. I don't really like the helper that
>> magically defines FOO_SITE and FOO_VERSION.
>> 
>
>After discussing quickly with Thomas, we feel that we can drop it as
>this would not really help with readability.

Readability wasn't the only reason such helpers were introduced. For github, the download urls changed a few times, and having it coded in only one place made changing this much easier.
So if that us the only reason I prefer to keep this patch. I'm willing to adopt it by the way...

Best regards,
Thomas

[Buildroot] [RFC PATCH 0/2] gitorious helper

[Alexandre Belloni]

On 29/04/2014 at 07:18:08 +0200, Thomas De Schampheleire wrote :
> Alexandre Belloni <alexandre.belloni@free-electrons.com> schreef:
> >After discussing quickly with Thomas, we feel that we can drop it as
> >this would not really help with readability.
> 
> Readability wasn't the only reason such helpers were introduced. For github, the download urls changed a few times, and having it coded in only one place made changing this much easier.
> So if that us the only reason I prefer to keep this patch. I'm willing to adopt it by the way...
> 

Feel free to adopt it but I'm not sure we can have something easy to use
and elegant.


-- 
Alexandre Belloni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com