* [Buildroot] [PATCH 1/2] package infra: add mirror support
@ 2011-10-18 14:14 Gustavo Zacarias
2011-10-18 14:14 ` [Buildroot] [PATCH 2/2] stunnel: add mirror site Gustavo Zacarias
` (3 more replies)
0 siblings, 4 replies; 18+ messages in thread
From: Gustavo Zacarias @ 2011-10-18 14:14 UTC (permalink / raw)
To: buildroot
Add FOO_MIRROR support for packages that have different URLs to fetch
from, for redundancy or when they move to an obsolete/old directory when
a new version pops up.
Only implement for WGET methods.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
package/Makefile.package.in | 26 +++++++++++++++++++++++---
1 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/package/Makefile.package.in b/package/Makefile.package.in
index b5ef57b..b9b5181 100644
--- a/package/Makefile.package.in
+++ b/package/Makefile.package.in
@@ -207,10 +207,12 @@ endef
# DOWNLOAD -- Download helper. Will try to download source from:
# 1) BR2_PRIMARY_SITE if enabled
# 2) Download site
-# 3) BR2_BACKUP_SITE if enabled
+# 3) Download mirror
+# 4) BR2_BACKUP_SITE if enabled
#
# Argument 1 is the source location
# Argument 2 is the source filename
+# Argument 3 is the mirror location
#
# E.G. use like this:
# $(call DOWNLOAD,$(FOO_SITE),$(FOO_SOURCE))
@@ -229,6 +231,15 @@ define DOWNLOAD
*) $(call $(DL_MODE)_WGET,$(1),$(2)) && exit ;; \
esac ; \
fi ; \
+ if test -n "$(3)" ; then \
+ case "$($(PKG)_SITE_METHOD)" in \
+ git) exit ;; \
+ svn) exit ;; \
+ bzr) exit ;; \
+ file) exit ;; \
+ *) $(call $(DL_MODE)_WGET,$(3),$(2)) && exit ;; \
+ esac ; \
+ fi ; \
if test -n "$(call qstrip,$(BR2_BACKUP_SITE))" ; then \
$(call $(DL_MODE)_WGET,$(BR2_BACKUP_SITE),$(2)) && exit ; \
fi ; \
@@ -266,8 +277,8 @@ ifeq ($(DL_MODE),DOWNLOAD)
(test -z $($(PKG)_PATCH) || test -e $(DL_DIR)$($(PKG)_PATCH))) || \
$(call MESSAGE,"Downloading")
endif
- $(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_SOURCE)))
- $(if $($(PKG)_PATCH),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_PATCH)))
+ $(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_SOURCE),$($(PKG)_MIRROR)))
+ $(if $($(PKG)_PATCH),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_PATCH),$($(PKG)_MIRROR)))
$(foreach hook,$($(PKG)_POST_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
ifeq ($(DL_MODE),DOWNLOAD)
$(Q)mkdir -p $(@D)
@@ -463,6 +474,15 @@ ifndef $(2)_SITE
endif
endif
+ifndef $(2)_MIRROR
+ ifdef $(3)_MIRROR
+ $(2)_MIRROR = $($(3)_MIRROR)
+ else
+ $(2)_MIRROR ?= \
+ http://$$(BR2_SOURCEFORGE_MIRROR).dl.sourceforge.net/sourceforge/$(1)
+ endif
+endif
+
ifndef $(2)_SITE_METHOD
ifdef $(3)_SITE_METHOD
$(2)_SITE_METHOD = $($(3)_SITE_METHOD)
--
1.7.3.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [Buildroot] [PATCH 2/2] stunnel: add mirror site
2011-10-18 14:14 [Buildroot] [PATCH 1/2] package infra: add mirror support Gustavo Zacarias
@ 2011-10-18 14:14 ` Gustavo Zacarias
2011-10-18 14:22 ` [Buildroot] [PATCH 1/2] package infra: add mirror support Thomas Petazzoni
` (2 subsequent siblings)
3 siblings, 0 replies; 18+ messages in thread
From: Gustavo Zacarias @ 2011-10-18 14:14 UTC (permalink / raw)
To: buildroot
stunnel is known to move all but the latest version to an obsolete
directory when there's a new release, as pointed out by Maxime Ripard.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
---
package/stunnel/stunnel.mk | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/package/stunnel/stunnel.mk b/package/stunnel/stunnel.mk
index 6088e58..c5ea152 100644
--- a/package/stunnel/stunnel.mk
+++ b/package/stunnel/stunnel.mk
@@ -6,6 +6,7 @@
STUNNEL_VERSION = 4.44
STUNNEL_SITE = http://ftp.nluug.nl/pub/networking/stunnel
+STUNNEL_MIRROR = http://ftp.nluug.nl/pub/networking/stunnel/obsolete/4.x
STUNNEL_DEPENDENCIES = openssl
STUNNEL_CONF_OPT += \
--with-ssl=$(STAGING_DIR)/usr \
--
1.7.3.4
^ permalink raw reply related [flat|nested] 18+ messages in thread* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 14:14 [Buildroot] [PATCH 1/2] package infra: add mirror support Gustavo Zacarias
2011-10-18 14:14 ` [Buildroot] [PATCH 2/2] stunnel: add mirror site Gustavo Zacarias
@ 2011-10-18 14:22 ` Thomas Petazzoni
2011-10-18 14:32 ` Thomas Petazzoni
2011-10-18 19:05 ` Arnout Vandecappelle
3 siblings, 0 replies; 18+ messages in thread
From: Thomas Petazzoni @ 2011-10-18 14:22 UTC (permalink / raw)
To: buildroot
Le Tue, 18 Oct 2011 11:14:54 -0300,
Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
> Add FOO_MIRROR support for packages that have different URLs to fetch
> from, for redundancy or when they move to an obsolete/old directory
> when a new version pops up.
> Only implement for WGET methods.
Just wondering: what about making <pkg>_SITE a space-separated list of
URLs to try, instead of adding <pkg>_MIRROR ?
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 14:14 [Buildroot] [PATCH 1/2] package infra: add mirror support Gustavo Zacarias
2011-10-18 14:14 ` [Buildroot] [PATCH 2/2] stunnel: add mirror site Gustavo Zacarias
2011-10-18 14:22 ` [Buildroot] [PATCH 1/2] package infra: add mirror support Thomas Petazzoni
@ 2011-10-18 14:32 ` Thomas Petazzoni
2011-10-18 15:55 ` Michael S. Zick
2011-10-18 19:05 ` Arnout Vandecappelle
3 siblings, 1 reply; 18+ messages in thread
From: Thomas Petazzoni @ 2011-10-18 14:32 UTC (permalink / raw)
To: buildroot
Le Tue, 18 Oct 2011 11:14:54 -0300,
Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
> + if test -n "$(3)" ; then \
> + case "$($(PKG)_SITE_METHOD)" in \
> + git) exit ;; \
> + svn) exit ;; \
> + bzr) exit ;; \
> + file) exit ;; \
I don't think you want to exit here, you want to continue to evaluate
the BR2_BACKUP_SITE solution below. For example, sources.buildroot.net
has http://sources.buildroot.net/libsvgtiny-12121.tar.gz, which is the
Subversion tarball of libsvgtiny. Same for
http://sources.buildroot.net/tslib-412d99d8b92c.tar.gz.
Regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 14:32 ` Thomas Petazzoni
@ 2011-10-18 15:55 ` Michael S. Zick
2011-10-18 16:00 ` Gustavo Zacarias
2011-10-18 16:02 ` Yann E. MORIN
0 siblings, 2 replies; 18+ messages in thread
From: Michael S. Zick @ 2011-10-18 15:55 UTC (permalink / raw)
To: buildroot
On Tue October 18 2011, Thomas Petazzoni wrote:
> Le Tue, 18 Oct 2011 11:14:54 -0300,
> Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
>
> > + if test -n "$(3)" ; then \
> > + case "$($(PKG)_SITE_METHOD)" in \
> > + git) exit ;; \
> > + svn) exit ;; \
> > + bzr) exit ;; \
> > + file) exit ;; \
>
How about just a nop?
s/exit/:/
Mike
> I don't think you want to exit here, you want to continue to evaluate
> the BR2_BACKUP_SITE solution below. For example, sources.buildroot.net
> has http://sources.buildroot.net/libsvgtiny-12121.tar.gz, which is the
> Subversion tarball of libsvgtiny. Same for
> http://sources.buildroot.net/tslib-412d99d8b92c.tar.gz.
>
> Regards,
>
> Thomas
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 15:55 ` Michael S. Zick
@ 2011-10-18 16:00 ` Gustavo Zacarias
2011-10-18 17:27 ` Thomas Petazzoni
2011-10-18 16:02 ` Yann E. MORIN
1 sibling, 1 reply; 18+ messages in thread
From: Gustavo Zacarias @ 2011-10-18 16:00 UTC (permalink / raw)
To: buildroot
On 18.10.2011 12:55, Michael S. Zick wrote:
> On Tue October 18 2011, Thomas Petazzoni wrote:
>> Le Tue, 18 Oct 2011 11:14:54 -0300,
>> Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
>>
>> > + if test -n "$(3)" ; then \
>> > + case "$($(PKG)_SITE_METHOD)" in \
>> > + git) exit ;; \
>> > + svn) exit ;; \
>> > + bzr) exit ;; \
>> > + file) exit ;; \
>>
>
> How about just a nop?
> s/exit/:/
>
> Mike
Yes that's a solution.
I'm also looking into using multiple PKG_SITE definitions as Thomas
suggested, however it's a bit more complicated since i've got to pull
out the PKG_SITE_METHOD eval from GENTARGETS_INNER since we could have
multiple methods for different mirrors.
That means looking that things don't break by doing so.
Also contemplating how to properly do PKG_SIZE (for file size) and
PKG_HASH (MD5 and/or SHA1) for source consistency.
Regards.
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 16:00 ` Gustavo Zacarias
@ 2011-10-18 17:27 ` Thomas Petazzoni
2011-10-18 18:01 ` Gustavo Zacarias
0 siblings, 1 reply; 18+ messages in thread
From: Thomas Petazzoni @ 2011-10-18 17:27 UTC (permalink / raw)
To: buildroot
Le Tue, 18 Oct 2011 13:00:28 -0300,
Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
> Also contemplating how to properly do PKG_SIZE (for file size) and
> PKG_HASH (MD5 and/or SHA1) for source consistency.
Do we actually want that ? I know OE does this, but tarball size and
hashes will be a pain to keep updated.
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 17:27 ` Thomas Petazzoni
@ 2011-10-18 18:01 ` Gustavo Zacarias
2011-10-18 21:38 ` Arnout Vandecappelle
0 siblings, 1 reply; 18+ messages in thread
From: Gustavo Zacarias @ 2011-10-18 18:01 UTC (permalink / raw)
To: buildroot
On 18.10.2011 14:27, Thomas Petazzoni wrote:
> Le Tue, 18 Oct 2011 13:00:28 -0300,
> Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
>
>> Also contemplating how to properly do PKG_SIZE (for file size) and
>> PKG_HASH (MD5 and/or SHA1) for source consistency.
>
> Do we actually want that ? I know OE does this, but tarball size and
> hashes will be a pain to keep updated.
>
> Thomas
In a non-mandatory way it may be desirable.
That is if a package doesn't define it then we don't care.
Most of the time tarballs shouldn't change size & hash, though it
doesn't hold true for every project around.
It could also be an option to completely disable it.
Regards.
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 18:01 ` Gustavo Zacarias
@ 2011-10-18 21:38 ` Arnout Vandecappelle
2011-10-19 8:11 ` [Buildroot] Adding hashes to package recipes Thomas Petazzoni
2011-10-19 8:12 ` [Buildroot] [PATCH 1/2] package infra: add mirror support Sven Neumann
0 siblings, 2 replies; 18+ messages in thread
From: Arnout Vandecappelle @ 2011-10-18 21:38 UTC (permalink / raw)
To: buildroot
On Tuesday 18 October 2011 20:01:14, Gustavo Zacarias wrote:
> On 18.10.2011 14:27, Thomas Petazzoni wrote:
>
> > Le Tue, 18 Oct 2011 13:00:28 -0300,
> > Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
> >
> >> Also contemplating how to properly do PKG_SIZE (for file size) and
> >> PKG_HASH (MD5 and/or SHA1) for source consistency.
> >
> > Do we actually want that ? I know OE does this, but tarball size and
> > hashes will be a pain to keep updated.
> >
> > Thomas
>
> In a non-mandatory way it may be desirable.
> That is if a package doesn't define it then we don't care.
> Most of the time tarballs shouldn't change size & hash, though it
> doesn't hold true for every project around.
Size and hash should only change when you bump the version, and in that case you anyway download the new tar file for testing so it shouldn't be too hard to recompute the hash. And of course, the hash should be the same regardless of which mirror it is downloaded from...
I'm not sure what good size would do, however.
> It could also be an option to completely disable it.
Definitely!
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286540
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 31BB CF53 8660 6F88 345D 54CC A836 5879 20D7 CF43
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] Adding hashes to package recipes
2011-10-18 21:38 ` Arnout Vandecappelle
@ 2011-10-19 8:11 ` Thomas Petazzoni
2011-10-19 8:12 ` [Buildroot] [PATCH 1/2] package infra: add mirror support Sven Neumann
1 sibling, 0 replies; 18+ messages in thread
From: Thomas Petazzoni @ 2011-10-19 8:11 UTC (permalink / raw)
To: buildroot
Le Tue, 18 Oct 2011 23:38:59 +0200,
Arnout Vandecappelle <arnout@mind.be> a ?crit :
> Size and hash should only change when you bump the version, and in
> that case you anyway download the new tar file for testing so it
> shouldn't be too hard to recompute the hash. And of course, the hash
> should be the same regardless of which mirror it is downloaded from...
True, but what does it bring exactly ?
> > It could also be an option to completely disable it.
>
> Definitely!
Well, the check at runtime can be disabled, but the fact that there
will be hashes in all packages to update every time we bump a package
is not something that can be "disabled". I am not talking from an user
point of view, but from a maintenance point of view.
I think in the past we talked about that with Peter and our opinions
was that it wasn't worth it. Now, I guess that if there are
sufficiently convincing arguments, we can change our minds :)
Regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 21:38 ` Arnout Vandecappelle
2011-10-19 8:11 ` [Buildroot] Adding hashes to package recipes Thomas Petazzoni
@ 2011-10-19 8:12 ` Sven Neumann
2011-10-19 9:35 ` Arnout Vandecappelle
1 sibling, 1 reply; 18+ messages in thread
From: Sven Neumann @ 2011-10-19 8:12 UTC (permalink / raw)
To: buildroot
Hi,
On Tue, 2011-10-18 at 23:38 +0200, Arnout Vandecappelle wrote:
> > In a non-mandatory way it may be desirable.
> > That is if a package doesn't define it then we don't care.
> > Most of the time tarballs shouldn't change size & hash, though it
> > doesn't hold true for every project around.
>
> Size and hash should only change when you bump the version
Unfortunately it is not so uncommon for some projects to re-release with
the same version number if a bug sneaked into the released tarball. I
know that this should not be the case, but it happens every so often.
Not sure though what implications this has for the proposed buildroot
changes...
Regards,
Sven
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-19 8:12 ` [Buildroot] [PATCH 1/2] package infra: add mirror support Sven Neumann
@ 2011-10-19 9:35 ` Arnout Vandecappelle
2011-10-19 12:59 ` Thomas Petazzoni
0 siblings, 1 reply; 18+ messages in thread
From: Arnout Vandecappelle @ 2011-10-19 9:35 UTC (permalink / raw)
To: buildroot
On Wednesday 19 October 2011 10:12:39, Sven Neumann wrote:
> On Tue, 2011-10-18 at 23:38 +0200, Arnout Vandecappelle wrote:
>
> > > In a non-mandatory way it may be desirable.
> > > That is if a package doesn't define it then we don't care.
> > > Most of the time tarballs shouldn't change size & hash, though it
> > > doesn't hold true for every project around.
> >
> > Size and hash should only change when you bump the version
>
> Unfortunately it is not so uncommon for some projects to re-release with
> the same version number if a bug sneaked into the released tarball. I
> know that this should not be the case, but it happens every so often.
>
> Not sure though what implications this has for the proposed buildroot
> changes...
I guess that if we start with this feature, it should be limited to
warnings for the first year or so until it proves its worth.
The recent kernel.org horror has convinced me that some form of
verification is needed, though.
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286540
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 31BB CF53 8660 6F88 345D 54CC A836 5879 20D7 CF43
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-19 9:35 ` Arnout Vandecappelle
@ 2011-10-19 12:59 ` Thomas Petazzoni
2011-10-19 16:37 ` Michael S. Zick
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: Thomas Petazzoni @ 2011-10-19 12:59 UTC (permalink / raw)
To: buildroot
Le Wed, 19 Oct 2011 11:35:52 +0200,
Arnout Vandecappelle <arnout@mind.be> a ?crit :
> The recent kernel.org horror has convinced me that some form of
> verification is needed, though.
Having hashes in Buildroot will not necessarily provide an additional
security. Consider the following scenario:
1. The world exists.
2. Project foo releases foo-2.1.tar.bz2
3. A BR packager bumps package foo to 2.1. He downloads the new
tarball, generates locally its hash and adds this hash to the
foo.mk.
4. The BR packager patch is merged in Buildroot.
Having hashes in the foo.mk will warn you if the foo website has been
cracked after step 3. But if it has been cracked between 2) and 3),
then you're doomed: the BR packager will assume foo-2.1.tar.bz2 is
correct. This packager will quite probably never check a GPG signature
or do any kind of additional security check when bumping foo to 2.1.
Therefore, I fear that this mechanism would give an *impression* of
higher security, but would in fact provide no additional security
compared to not verifying the hashes.
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 18+ messages in thread* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-19 12:59 ` Thomas Petazzoni
@ 2011-10-19 16:37 ` Michael S. Zick
2011-10-19 16:59 ` Eric Bénard
2011-10-19 22:10 ` Arnout Vandecappelle
2 siblings, 0 replies; 18+ messages in thread
From: Michael S. Zick @ 2011-10-19 16:37 UTC (permalink / raw)
To: buildroot
On Wed October 19 2011, Thomas Petazzoni wrote:
> Le Wed, 19 Oct 2011 11:35:52 +0200,
> Arnout Vandecappelle <arnout@mind.be> a ?crit :
>
> > The recent kernel.org horror has convinced me that some form of
> > verification is needed, though.
>
> Having hashes in Buildroot will not necessarily provide an additional
> security. Consider the following scenario:
>
> 1. The world exists.
>
Thomas, I can hardly keep myself from typing a smart remark here.
;-)
> 2. Project foo releases foo-2.1.tar.bz2
> 3. A BR packager bumps package foo to 2.1. He downloads the new
> tarball, generates locally its hash and adds this hash to the
> foo.mk.
> 4. The BR packager patch is merged in Buildroot.
>
> Having hashes in the foo.mk will warn you if the foo website has been
> cracked after step 3. But if it has been cracked between 2) and 3),
> then you're doomed: the BR packager will assume foo-2.1.tar.bz2 is
> correct. This packager will quite probably never check a GPG signature
> or do any kind of additional security check when bumping foo to 2.1.
>
In my local, subjective, reality (#1) - - -
I post for re-distribution on one of my web sites archives built by
others (who don't provide signatures or even checksums).
When I post a package, I include the original posting link,
a checksum and a signature file.
But all that I can claim is that "if the signature matches, you
have got a true copy of what I posted" - which doesn't make any
claims that what I got was "correct" or the "intended" original.
From my server logs, it appears that it is an usual event for someone
to download either the signature or the checksum.
So in practice, it seems that few people bother to check.
> Therefore, I fear that this mechanism would give an *impression* of
> higher security, but would in fact provide no additional security
> compared to not verifying the hashes.
>
When done further down the line of distribution than the original
creator of the package during package creation time...
It does give the "subjective impression" (see #1) that the poster
was following some sort of "best practices" in their re-distribution;
A "good thing".
But I have to agree that it also gives the impression that the contents
are still as the original packager intended;
Which could be a "bad thing".
For instance:
When troubleshooting a problem, having passed this "warm and fuzzy test"
it might be a long time before the user suspects that something is wrong
inside of the "trusted" package contents.
Add to the fact that few even bother to check...
I don't think it is worth the time and trouble to __add__ checksums and
signatures;
Although it would be a benefit to also archive any checksums and signatures
provided by the __original__ package creator.
Mike
> Thomas
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-19 12:59 ` Thomas Petazzoni
2011-10-19 16:37 ` Michael S. Zick
@ 2011-10-19 16:59 ` Eric Bénard
2011-10-19 22:10 ` Arnout Vandecappelle
2 siblings, 0 replies; 18+ messages in thread
From: Eric Bénard @ 2011-10-19 16:59 UTC (permalink / raw)
To: buildroot
Hi,
Le 19/10/2011 14:59, Thomas Petazzoni a ?crit :
> Le Wed, 19 Oct 2011 11:35:52 +0200, Arnout Vandecappelle<arnout@mind.be> a
> ?crit :
>
>> The recent kernel.org horror has convinced me that some form of
>> verification is needed, though.
>
> Having hashes in Buildroot will not necessarily provide an additional
> security. Consider the following scenario:
>
> 1. The world exists. 2. Project foo releases foo-2.1.tar.bz2 3. A BR
> packager bumps package foo to 2.1. He downloads the new tarball, generates
> locally its hash and adds this hash to the foo.mk. 4. The BR packager patch
> is merged in Buildroot.
>
> Having hashes in the foo.mk will warn you if the foo website has been
> cracked after step 3. But if it has been cracked between 2) and 3), then
> you're doomed: the BR packager will assume foo-2.1.tar.bz2 is correct. This
> packager will quite probably never check a GPG signature or do any kind of
> additional security check when bumping foo to 2.1.
>
> Therefore, I fear that this mechanism would give an *impression* of higher
> security, but would in fact provide no additional security compared to not
> verifying the hashes.
>
apart from security, the hash feature gives the user the opportunity to be
sure that he is using the sources which were used by the initial package
creator and thus are the one which are supposed to have been validated at
least once.
This can be useful when projects are changing the source archive without
changing the version number (example : qt 4.7.4 archive which was regenerated
after the first release) or for project where the source archive with a
version number is a symbolic link to a source archive with an other version
number (example : binutils x.y.z which is now x.y.za).
Without a hash check you won't notice these changes (and thus can't analyse
the impact they may have on your build).
Eric
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-19 12:59 ` Thomas Petazzoni
2011-10-19 16:37 ` Michael S. Zick
2011-10-19 16:59 ` Eric Bénard
@ 2011-10-19 22:10 ` Arnout Vandecappelle
2 siblings, 0 replies; 18+ messages in thread
From: Arnout Vandecappelle @ 2011-10-19 22:10 UTC (permalink / raw)
To: buildroot
On Wednesday 19 October 2011 14:59:34, Thomas Petazzoni wrote:
> Le Wed, 19 Oct 2011 11:35:52 +0200,
> Arnout Vandecappelle <arnout@mind.be> a ?crit :
>
> > The recent kernel.org horror has convinced me that some form of
> > verification is needed, though.
>
> Having hashes in Buildroot will not necessarily provide an additional
> security. Consider the following scenario:
>
> 1. The world exists.
> 2. Project foo releases foo-2.1.tar.bz2
> 3. A BR packager bumps package foo to 2.1. He downloads the new
> tarball, generates locally its hash and adds this hash to the
> foo.mk.
> 4. The BR packager patch is merged in Buildroot.
>
> Having hashes in the foo.mk will warn you if the foo website has been
> cracked after step 3. But if it has been cracked between 2) and 3),
> then you're doomed: the BR packager will assume foo-2.1.tar.bz2 is
> correct.
If the upstream source is cracked, then of course you're doomed. What it does protect again is man-in-the-middle attacks (same as https is supposed to protect against but doesn't because of unreliable CAs). If the packager generates a hash of a non-authorized tar, then most users will download packages with a different hash and will (hopefully) report this.
Clearly there is still a vulnerability window, but it is much smaller than in the current situation.
> This packager will quite probably never check a GPG signature
> or do any kind of additional security check when bumping foo to 2.1.
>
> Therefore, I fear that this mechanism would give an *impression* of
> higher security, but would in fact provide no additional security
> compared to not verifying the hashes.
It does give higher security. Perhaps not yet high security, though.
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286540
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 31BB CF53 8660 6F88 345D 54CC A836 5879 20D7 CF43
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 15:55 ` Michael S. Zick
2011-10-18 16:00 ` Gustavo Zacarias
@ 2011-10-18 16:02 ` Yann E. MORIN
1 sibling, 0 replies; 18+ messages in thread
From: Yann E. MORIN @ 2011-10-18 16:02 UTC (permalink / raw)
To: buildroot
Mike, All,
On Tuesday 18 October 2011 17:55:20 Michael S. Zick wrote:
> On Tue October 18 2011, Thomas Petazzoni wrote:
> > Le Tue, 18 Oct 2011 11:14:54 -0300,
> > Gustavo Zacarias <gustavo@zacarias.com.ar> a ?crit :
> >
> > > + if test -n "$(3)" ; then \
> > > + case "$($(PKG)_SITE_METHOD)" in \
> > > + git) exit ;; \
> > > + svn) exit ;; \
> > > + bzr) exit ;; \
> > > + file) exit ;; \
> >
>
> How about just a nop?
> s/exit/:/
Or simply:
foo) ;;
Regards,
Yann E. MORIN.
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Buildroot] [PATCH 1/2] package infra: add mirror support
2011-10-18 14:14 [Buildroot] [PATCH 1/2] package infra: add mirror support Gustavo Zacarias
` (2 preceding siblings ...)
2011-10-18 14:32 ` Thomas Petazzoni
@ 2011-10-18 19:05 ` Arnout Vandecappelle
3 siblings, 0 replies; 18+ messages in thread
From: Arnout Vandecappelle @ 2011-10-18 19:05 UTC (permalink / raw)
To: buildroot
On Tuesday 18 October 2011 16:14:54, Gustavo Zacarias wrote:
> Add FOO_MIRROR support for packages that have different URLs to fetch
> from, for redundancy or when they move to an obsolete/old directory when
> a new version pops up.
> Only implement for WGET methods.
>
> Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
> ---
> package/Makefile.package.in | 26 +++++++++++++++++++++++---
> 1 files changed, 23 insertions(+), 3 deletions(-)
>
> diff --git a/package/Makefile.package.in b/package/Makefile.package.in
> index b5ef57b..b9b5181 100644
> --- a/package/Makefile.package.in
> +++ b/package/Makefile.package.in
> @@ -207,10 +207,12 @@ endef
>
> # DOWNLOAD -- Download helper. Will try to download source from:
> # 1) BR2_PRIMARY_SITE if enabled
> # 2) Download site
>
> -# 3) BR2_BACKUP_SITE if enabled
> +# 3) Download mirror
> +# 4) BR2_BACKUP_SITE if enabled
>
> #
> # Argument 1 is the source location
> # Argument 2 is the source filename
>
> +# Argument 3 is the mirror location
Argument 3 will be $($(PKG)_MIRROR) in all calls, so there is no need to
make it an argument. Just use $($(PKG)_MIRROR) directly, like it is done
for $($(PKG)_SITE_METHOD).
>
> #
> # E.G. use like this:
> # $(call DOWNLOAD,$(FOO_SITE),$(FOO_SOURCE))
> @@ -229,6 +231,15 @@ define DOWNLOAD
> *) $(call $(DL_MODE)_WGET,$(1),$(2)) && exit ;; \
> esac ; \
> fi ; \
> + if test -n "$(3)" ; then \
You can easily support $($(PKG)_MIRRORS) (plural) with
for mirror in $($(PKG)_MIRRORS); do \
... $$mirror ...
done
This will simply skip the loop if $(3) is empty.
> + case "$($(PKG)_SITE_METHOD)" in \
> + git) exit ;; \
> + svn) exit ;; \
> + bzr) exit ;; \
> + file) exit ;; \
> + *) $(call $(DL_MODE)_WGET,$(3),$(2)) && exit ;; \
> + esac ; \
Is there a need to check for $($(PKG)_SITE_METHOD) at all? This is not
done for the BR2_BACKUP_SITE after all.
> + fi ; \
> if test -n "$(call qstrip,$(BR2_BACKUP_SITE))" ; then \
> $(call $(DL_MODE)_WGET,$(BR2_BACKUP_SITE),$(2)) && exit ; \
> fi ; \
> @@ -266,8 +277,8 @@ ifeq ($(DL_MODE),DOWNLOAD)
> (test -z $($(PKG)_PATCH) || test -e $(DL_DIR)$($(PKG)_PATCH))) || \
> $(call MESSAGE,"Downloading")
> endif
> - $(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_SOURCE)))
> - $(if $($(PKG)_PATCH),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_PATCH)))
> + $(if $($(PKG)_SOURCE),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_SOURCE),$($(PKG)_MIRROR)))
> + $(if $($(PKG)_PATCH),$(call DOWNLOAD,$($(PKG)_SITE),$($(PKG)_PATCH),$($(PKG)_MIRROR)))
> $(foreach hook,$($(PKG)_POST_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
> ifeq ($(DL_MODE),DOWNLOAD)
> $(Q)mkdir -p $(@D)
> @@ -463,6 +474,15 @@ ifndef $(2)_SITE
> endif
> endif
>
> +ifndef $(2)_MIRROR
> + ifdef $(3)_MIRROR
> + $(2)_MIRROR = $($(3)_MIRROR)
> + else
> + $(2)_MIRROR ?= \
> + http://$$(BR2_SOURCEFORGE_MIRROR).dl.sourceforge.net/sourceforge/$(1)
Err... Either it is a sourceforge package and then this will already be the
normal URL, or it is not and then this will just fail. Or am I missing
something?
> + endif
> +endif
> +
> ifndef $(2)_SITE_METHOD
> ifdef $(3)_SITE_METHOD
> $(2)_SITE_METHOD = $($(3)_SITE_METHOD)
>
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286540
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 31BB CF53 8660 6F88 345D 54CC A836 5879 20D7 CF43
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2011-10-19 22:10 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-10-18 14:14 [Buildroot] [PATCH 1/2] package infra: add mirror support Gustavo Zacarias
2011-10-18 14:14 ` [Buildroot] [PATCH 2/2] stunnel: add mirror site Gustavo Zacarias
2011-10-18 14:22 ` [Buildroot] [PATCH 1/2] package infra: add mirror support Thomas Petazzoni
2011-10-18 14:32 ` Thomas Petazzoni
2011-10-18 15:55 ` Michael S. Zick
2011-10-18 16:00 ` Gustavo Zacarias
2011-10-18 17:27 ` Thomas Petazzoni
2011-10-18 18:01 ` Gustavo Zacarias
2011-10-18 21:38 ` Arnout Vandecappelle
2011-10-19 8:11 ` [Buildroot] Adding hashes to package recipes Thomas Petazzoni
2011-10-19 8:12 ` [Buildroot] [PATCH 1/2] package infra: add mirror support Sven Neumann
2011-10-19 9:35 ` Arnout Vandecappelle
2011-10-19 12:59 ` Thomas Petazzoni
2011-10-19 16:37 ` Michael S. Zick
2011-10-19 16:59 ` Eric Bénard
2011-10-19 22:10 ` Arnout Vandecappelle
2011-10-18 16:02 ` Yann E. MORIN
2011-10-18 19:05 ` Arnout Vandecappelle
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox