* [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads)
@ 2014-06-01 17:40 Yann E. MORIN
2014-06-01 17:40 ` [Buildroot] [PATCH 01/12] Makefile: add BR2_DL_DIR to EXTRA_ENV Yann E. MORIN
` (11 more replies)
0 siblings, 12 replies; 25+ messages in thread
From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Hello All!
This series contains those download-related changes:
- Patches 1..2 : preparatory patches for upcoming helper scripts and
hash checks
- Patches 3..8 : move download helpers to shell scripts, one
such script for each download helper
- Patch 9 : don't use DL_DIR as scratchpad for temporary VCS
checkouts, since DL_DIR is a precious location
- Patches 10.11 : check hashes of downloaded files, to ensure the
files are genuine (with doc)
- Patch 12 : add hashes for ca-certificates
Changes v5 -> v6:
- ensure we can run multiple parallel downloads from different build
dirs with the same BR2_DL_DIR (Thomas)
- also convert bzr
- fix for stray failed downloads
Changes v4 -> v5:
- update ca-certificates hashes after version bump
- fix detection of comments and empty lines in .hash file
- rebase on top of master
Changes v3 -> v4:
- enhance the manual, typoes (Gustavo, Samuel, Thomas DS)
- remove spurious test bump on package/fis (Samuel)
Changes v2 -> v3:
- avoid partial downloads (Thomas DS)
- store hash-type in .hash file (Gustavo)
- typoes (Samuel, Gustavo)
- add possibility to treat missing hashes as an error
Changes v1 -> v2:
- don't do any behavioural change when switching to shell scripts,
just reproduce exactly what was in the Makefile (Luca, Arnout)
- support more than one hash algorithm (Arnout, Gustavo)
- typoes (Luca, Baruch)
Regards,
Yann E. MORIN.
The following changes since commit 27a5414804f7b8ab41101aad219ebef02d364bde:
Merge branch 'next' (2014-06-01 09:58:54 +0200)
are available in the git repository at:
git://gitorious.org/buildroot/buildroot.git yem/check-downloads
for you to fetch changes up to a4d10289e59a5422d0e4ad101a8e7d20469008a4:
package/ca-certificates: add tarball's hashes (2014-06-01 19:28:30 +0200)
----------------------------------------------------------------
Yann E. MORIN (12):
Makefile: add BR2_DL_DIR to EXTRA_ENV
pkg-infra: also set PKGDIR for the download step
pkg-infra: move the git download helper to a script
pkg-infra: move the svn download helper to a script
pkg-infra: move the cvs download helper to a script
pkg-infra: move the hg download helper to a script
pkg-infra: move the wget download helper to a script
pkg-infra: move the bzr download helper to a script
pkg-infra: don't use DL_DIR as scratchpad for temporary downloads
pkg-infra: add possiblity to check downloaded files against known hashes
manual: add documentation about packages' hashes
package/ca-certificates: add tarball's hashes
docs/manual/adding-packages-directory.txt | 67 ++++++++++++++++++++++
package/Makefile.in | 1 +
package/ca-certificates/ca-certificates.hash | 3 +
package/pkg-download.mk | 83 ++++++++++++----------------
package/pkg-generic.mk | 1 +
support/download/bzr | 37 +++++++++++++
support/download/check-hash | 77 ++++++++++++++++++++++++++
support/download/cvs | 40 ++++++++++++++
support/download/git | 50 +++++++++++++++++
support/download/hg | 38 +++++++++++++
support/download/svn | 37 +++++++++++++
support/download/wget | 32 +++++++++++
12 files changed, 417 insertions(+), 49 deletions(-)
create mode 100644 package/ca-certificates/ca-certificates.hash
create mode 100755 support/download/bzr
create mode 100755 support/download/check-hash
create mode 100755 support/download/cvs
create mode 100755 support/download/git
create mode 100755 support/download/hg
create mode 100755 support/download/svn
create mode 100755 support/download/wget
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 25+ messages in thread* [Buildroot] [PATCH 01/12] Makefile: add BR2_DL_DIR to EXTRA_ENV 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 02/12] pkg-infra: also set PKGDIR for the download step Yann E. MORIN ` (10 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Also export BR2_DL_DIR for incoming download helper scripts. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Samuel Martin <s.martin49@gmail.com> Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> --- package/Makefile.in | 1 + 1 file changed, 1 insertion(+) diff --git a/package/Makefile.in b/package/Makefile.in index 0233506..97053ba 100644 --- a/package/Makefile.in +++ b/package/Makefile.in @@ -305,6 +305,7 @@ HOST_MAKE_ENV=PATH=$(BR_PATH) \ # post-images) EXTRA_ENV=\ PATH=$(BR_PATH) \ + BR2_DL_DIR=$(BR2_DL_DIR) \ BUILD_DIR=$(BUILD_DIR) ################################################################################ -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 02/12] pkg-infra: also set PKGDIR for the download step 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 01/12] Makefile: add BR2_DL_DIR to EXTRA_ENV Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 03/12] pkg-infra: move the git download helper to a script Yann E. MORIN ` (9 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> This will be needed to get the hash file, to check the downloaded files. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Samuel Martin <s.martin49@gmail.com> --- package/pkg-generic.mk | 1 + 1 file changed, 1 insertion(+) diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk index 5116ed9..b97e43c 100644 --- a/package/pkg-generic.mk +++ b/package/pkg-generic.mk @@ -571,6 +571,7 @@ $$($(2)_TARGET_PATCH): RAWNAME=$(patsubst host-%,%,$(1)) $$($(2)_TARGET_PATCH): PKGDIR=$(pkgdir) $$($(2)_TARGET_EXTRACT): PKG=$(2) $$($(2)_TARGET_SOURCE): PKG=$(2) +$$($(2)_TARGET_SOURCE): PKGDIR=$(pkgdir) $$($(2)_TARGET_DIRCLEAN): PKG=$(2) # Compute the name of the Kconfig option that correspond to the -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 03/12] pkg-infra: move the git download helper to a script 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 01/12] Makefile: add BR2_DL_DIR to EXTRA_ENV Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 02/12] pkg-infra: also set PKGDIR for the download step Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 04/12] pkg-infra: move the svn " Yann E. MORIN ` (8 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> The git download helper is getting a bit more complex. Fixing it in the Makefile when it breaks (like the recent breakage with a non-existing sha1-cset) proves to be challenging, to say the least. Move it into a shell script in support/download/git, which will make it much easier to read, maintain, fix and enhance in the future. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Peter Korsgaard <jacmet@uclibc.org> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Acked-by: Luca Ceresoli <luca@lucaceresoli.net> Cc: Arnout Vandecappelle <arnout@mind.be> Reviewed-by: Samuel Martin <s.martin49@gmail.com> Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> --- package/pkg-download.mk | 17 +++-------------- support/download/git | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 40 insertions(+), 14 deletions(-) create mode 100755 support/download/git diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 192b9e0..9603ea8 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -12,7 +12,7 @@ WGET := $(call qstrip,$(BR2_WGET)) $(QUIET) SVN := $(call qstrip,$(BR2_SVN)) CVS := $(call qstrip,$(BR2_CVS)) BZR := $(call qstrip,$(BR2_BZR)) -GIT := $(call qstrip,$(BR2_GIT)) +export GIT := $(call qstrip,$(BR2_GIT)) HG := $(call qstrip,$(BR2_HG)) $(QUIET) SCP := $(call qstrip,$(BR2_SCP)) $(QUIET) SSH := $(call qstrip,$(BR2_SSH)) $(QUIET) @@ -84,19 +84,8 @@ github = https://github.com/$(1)/$(2)/archive/$(3) # problems define DOWNLOAD_GIT test -e $(DL_DIR)/$($(PKG)_SOURCE) || \ - (pushd $(DL_DIR) > /dev/null && \ - ((test "`git ls-remote $($(PKG)_SITE) $($(PKG)_DL_VERSION)`" && \ - echo "Doing shallow clone" && \ - $(GIT) clone --depth 1 -b $($(PKG)_DL_VERSION) --bare $($(PKG)_SITE) $($(PKG)_BASE_NAME)) || \ - (echo "Doing full clone" && \ - $(GIT) clone --bare $($(PKG)_SITE) $($(PKG)_BASE_NAME))) && \ - pushd $($(PKG)_BASE_NAME) > /dev/null && \ - $(GIT) archive --format=tar --prefix=$($(PKG)_BASE_NAME)/ -o $(DL_DIR)/.$($(PKG)_SOURCE).tmp $($(PKG)_DL_VERSION) && \ - gzip -c $(DL_DIR)/.$($(PKG)_SOURCE).tmp > $(DL_DIR)/$($(PKG)_SOURCE) && \ - rm -f $(DL_DIR)/.$($(PKG)_SOURCE).tmp && \ - popd > /dev/null && \ - rm -rf $($(PKG)_DL_DIR) && \ - popd > /dev/null) + $(EXTRA_ENV) support/download/git $($(PKG)_SITE) $($(PKG)_DL_VERSION) \ + $($(PKG)_BASE_NAME) $(DL_DIR)/$($(PKG)_SOURCE) endef # TODO: improve to check that the given PKG_DL_VERSION exists on the remote diff --git a/support/download/git b/support/download/git new file mode 100755 index 0000000..96db3a9 --- /dev/null +++ b/support/download/git @@ -0,0 +1,37 @@ +#!/bin/sh + +# We want to catch any command failure, and exit immediately +set -e + +# Download helper for git +# Call it with: +# $1: git repo +# $2: git cset +# $3: package's basename (eg. foobar-1.2.3) +# $4: output file +# And this environment: +# BR2_DL_DIR: path to Buildroot's download dir +# GIT : the git command to call + +repo="${1}" +cset="${2}" +basename="${3}" +output="${4}" + +repodir="${BR2_DL_DIR}/${basename}" + +if [ -n "$(${GIT} ls-remote "${repo}" "${cset}" 2>&1)" ]; then + printf "Doing shallow clone\n" + ${GIT} clone --depth 1 -b "${cset}" --bare "${repo}" "${repodir}" +else + printf "Doing full clone\n" + ${GIT} clone --bare "${repo}" "${repodir}" +fi + +pushd "${repodir}" +${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}" +gzip -c "${output}.tmp" >"${output}" +rm -f "${output}.tmp" +popd + +rm -rf "${repodir}" -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 04/12] pkg-infra: move the svn download helper to a script 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (2 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 03/12] pkg-infra: move the git download helper to a script Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 05/12] pkg-infra: move the cvs " Yann E. MORIN ` (7 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Maintaining the download helpers in the Makefile has proved to be a bit complex, so move it to a shell script. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Samuel Martin <s.martin49@gmail.com> --- package/pkg-download.mk | 9 +++------ support/download/svn | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 6 deletions(-) create mode 100755 support/download/svn diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 9603ea8..86c7bfe 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -9,7 +9,7 @@ # Download method commands WGET := $(call qstrip,$(BR2_WGET)) $(QUIET) -SVN := $(call qstrip,$(BR2_SVN)) +export SVN := $(call qstrip,$(BR2_SVN)) CVS := $(call qstrip,$(BR2_CVS)) BZR := $(call qstrip,$(BR2_BZR)) export GIT := $(call qstrip,$(BR2_GIT)) @@ -133,11 +133,8 @@ endef define DOWNLOAD_SVN test -e $(DL_DIR)/$($(PKG)_SOURCE) || \ - (pushd $(DL_DIR) > /dev/null && \ - $(SVN) export -r $($(PKG)_DL_VERSION) $($(PKG)_SITE) $($(PKG)_DL_DIR) && \ - $(TAR) czf $($(PKG)_SOURCE) $($(PKG)_BASE_NAME)/ && \ - rm -rf $($(PKG)_DL_DIR) && \ - popd > /dev/null) + $(EXTRA_ENV) support/download/svn $($(PKG)_SITE) $($(PKG)_DL_VERSION) \ + $($(PKG)_BASE_NAME) $(DL_DIR)/$($(PKG)_SOURCE) endef define SOURCE_CHECK_SVN diff --git a/support/download/svn b/support/download/svn new file mode 100755 index 0000000..c3ab32c --- /dev/null +++ b/support/download/svn @@ -0,0 +1,25 @@ +#!/bin/sh + +# We want to catch any command failure, and exit immediately +set -e + +# Download helper for svn +# Call it with: +# $1: svn repo +# $2: svn revision +# $3: package's basename (eg. foobar-1.2.3) +# $4: output file +# And this environment: +# SVN : the svn command to call +# BR2_DL_DIR: path to Buildroot's download dir + +repo="${1}" +rev="${2}" +basename="${3}" +output="${4}" + +pushd "${BR2_DL_DIR}" +${SVN} export -r "${rev}" "${repo}" "${basename}" +tar czf "${output}" "${basename}" +rm -rf "${basename}" +popd -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 05/12] pkg-infra: move the cvs download helper to a script 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (3 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 04/12] pkg-infra: move the svn " Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 06/12] pkg-infra: move the hg " Yann E. MORIN ` (6 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Maintaining the download helpers in the Makefile has proved to be a bit complex, so move it to a shell script. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Samuel Martin <s.martin49@gmail.com> --- package/pkg-download.mk | 11 ++++------- support/download/cvs | 27 +++++++++++++++++++++++++++ 2 files changed, 31 insertions(+), 7 deletions(-) create mode 100755 support/download/cvs diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 86c7bfe..b8d0d02 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -10,7 +10,7 @@ # Download method commands WGET := $(call qstrip,$(BR2_WGET)) $(QUIET) export SVN := $(call qstrip,$(BR2_SVN)) -CVS := $(call qstrip,$(BR2_CVS)) +export CVS := $(call qstrip,$(BR2_CVS)) BZR := $(call qstrip,$(BR2_BZR)) export GIT := $(call qstrip,$(BR2_GIT)) HG := $(call qstrip,$(BR2_HG)) $(QUIET) @@ -114,12 +114,9 @@ endef define DOWNLOAD_CVS test -e $(DL_DIR)/$($(PKG)_SOURCE) || \ - (pushd $(DL_DIR) > /dev/null && \ - $(CVS) -z3 -d:pserver:anonymous@$(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) \ - co -d $($(PKG)_BASE_NAME) -r :$($(PKG)_DL_VERSION) -P $($(PKG)_RAWNAME) && \ - $(TAR) czf $($(PKG)_SOURCE) $($(PKG)_BASE_NAME)/ && \ - rm -rf $($(PKG)_DL_DIR) && \ - popd > /dev/null) + $(EXTRA_ENV) support/download/cvs $(call stripurischeme,$(call qstrip,$($(PKG)_SITE))) \ + $($(PKG)_DL_VERSION) $($(PKG)_RAWNAME) \ + $($(PKG)_BASE_NAME) $(DL_DIR)/$($(PKG)_SOURCE) endef # Not all CVS servers support ls/rls, use login to see if we can connect diff --git a/support/download/cvs b/support/download/cvs new file mode 100755 index 0000000..06b8647 --- /dev/null +++ b/support/download/cvs @@ -0,0 +1,27 @@ +#!/bin/sh + +# We want to catch any command failure, and exit immediately +set -e + +# Download helper for cvs +# Call it with: +# $1: cvs repo +# $2: cvs revision +# $3: package's name (eg. foobar) +# $4: package's basename (eg. foobar-1.2.3) +# $5: output file +# And this environment: +# CVS : the cvs command to call +# BR2_DL_DIR: path to Buildroot's download dir + +repo="${1}" +rev="${2}" +rawname="${3}" +basename="${4}" +output="${5}" + +cd "${BR2_DL_DIR}" +${CVS} -z3 -d":pserver:anonymous@${repo}" \ + co -d "${basename}" -r ":${rev}" -P "${rawname}" +tar czf "${output}" "${basename}" +rm -rf "${basename}" -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 06/12] pkg-infra: move the hg download helper to a script 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (4 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 05/12] pkg-infra: move the cvs " Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 07/12] pkg-infra: move the wget " Yann E. MORIN ` (5 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Maintaining the download helpers in the Makefile has proved to be a bit complex, so move it to a shell script. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Samuel Martin <s.martin49@gmail.com> --- package/pkg-download.mk | 11 +++-------- support/download/hg | 25 +++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 8 deletions(-) create mode 100755 support/download/hg diff --git a/package/pkg-download.mk b/package/pkg-download.mk index b8d0d02..121a795 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -13,7 +13,7 @@ export SVN := $(call qstrip,$(BR2_SVN)) export CVS := $(call qstrip,$(BR2_CVS)) BZR := $(call qstrip,$(BR2_BZR)) export GIT := $(call qstrip,$(BR2_GIT)) -HG := $(call qstrip,$(BR2_HG)) $(QUIET) +export HG := $(call qstrip,$(BR2_HG)) $(QUIET) SCP := $(call qstrip,$(BR2_SCP)) $(QUIET) SSH := $(call qstrip,$(BR2_SSH)) $(QUIET) LOCALFILES := $(call qstrip,$(BR2_LOCALFILES)) @@ -161,13 +161,8 @@ endef define DOWNLOAD_HG test -e $(DL_DIR)/$($(PKG)_SOURCE) || \ - (pushd $(DL_DIR) > /dev/null && \ - rm -rf $($(PKG)_BASE_NAME) && \ - $(HG) clone --noupdate --rev $($(PKG)_DL_VERSION) $($(PKG)_SITE) $($(PKG)_BASE_NAME) && \ - $(HG) archive --repository $($(PKG)_BASE_NAME) --type tgz --prefix $($(PKG)_BASE_NAME)/ \ - --rev $($(PKG)_DL_VERSION) $(DL_DIR)/$($(PKG)_SOURCE) && \ - rm -rf $($(PKG)_DL_DIR) && \ - popd > /dev/null) + $(EXTRA_ENV) support/download/hg $($(PKG)_SITE) $($(PKG)_DL_VERSION) \ + $($(PKG)_BASE_NAME) $(DL_DIR)/$($(PKG)_SOURCE) endef # TODO: improve to check that the given PKG_DL_VERSION exists on the remote diff --git a/support/download/hg b/support/download/hg new file mode 100755 index 0000000..70b49cf --- /dev/null +++ b/support/download/hg @@ -0,0 +1,25 @@ +#!/bin/sh + +# We want to catch any command failure, and exit immediately +set -e + +# Download helper for hg +# Call it with: +# $1: hg repo +# $2: hg cset +# $3: package's basename (eg. foobar-1.2.3) +# $4: output file +# And this environment: +# HG : the hg command to call +# BR2_DL_DIR: path to Buildroot's download dir + +repo="${1}" +cset="${2}" +basename="${3}" +output="${4}" + +cd "${BR2_DL_DIR}" +${HG} clone --noupdate --rev "${cset}" "${repo}" "${basename}" +${HG} archive --repository "${basename}" --type tgz --prefix "${basename}" \ + --rev "${cset}" "${output}" +rm -rf "${basename}" -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 07/12] pkg-infra: move the wget download helper to a script 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (5 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 06/12] pkg-infra: move the hg " Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 08/12] pkg-infra: move the bzr " Yann E. MORIN ` (4 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Maintaining the download helpers in the Makefile has proved to be a bit complex, so move it to a shell script. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Reviewed-by: Samuel Martin <s.martin49@gmail.com> --- package/pkg-download.mk | 13 +++---------- support/download/wget | 21 +++++++++++++++++++++ 2 files changed, 24 insertions(+), 10 deletions(-) create mode 100755 support/download/wget diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 121a795..0c0ee74 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -8,7 +8,7 @@ ################################################################################ # Download method commands -WGET := $(call qstrip,$(BR2_WGET)) $(QUIET) +export WGET := $(call qstrip,$(BR2_WGET)) $(QUIET) export SVN := $(call qstrip,$(BR2_SVN)) export CVS := $(call qstrip,$(BR2_CVS)) BZR := $(call qstrip,$(BR2_BZR)) @@ -175,17 +175,10 @@ define SHOW_EXTERNAL_DEPS_HG echo $($(PKG)_SOURCE) endef -# Download a file using wget. Only download the file if it doesn't -# already exist in the download directory. If the download fails, -# remove the file (because wget -O creates a 0-byte file even if the -# download fails). To handle an interrupted download as well, download -# to a temporary file first. The temporary file will be overwritten -# the next time the download is tried. + define DOWNLOAD_WGET test -e $(DL_DIR)/$(2) || \ - ($(WGET) -O $(DL_DIR)/$(2).tmp '$(call qstrip,$(1))' && \ - mv $(DL_DIR)/$(2).tmp $(DL_DIR)/$(2)) || \ - (rm -f $(DL_DIR)/$(2).tmp ; exit 1) + $(EXTRA_ENV) support/download/wget '$(call qstrip,$(1))' $(DL_DIR)/$(2) endef define SOURCE_CHECK_WGET diff --git a/support/download/wget b/support/download/wget new file mode 100755 index 0000000..7865517 --- /dev/null +++ b/support/download/wget @@ -0,0 +1,21 @@ +#!/bin/sh + +# We want to catch any command failure, and exit immediately +set -e + +# Download helper for wget +# Call it with: +# $1: URL +# $2: output file +# And this environment: +# WGET : the wget command to call + +url="${1}" +output="${2}" + +if ${WGET} -O "${output}.tmp" "${url}"; then + mv "${output}.tmp" "${output}" +else + rm -f "${output}.tmp" + exit 1 +fi -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 08/12] pkg-infra: move the bzr download helper to a script 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (6 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 07/12] pkg-infra: move the wget " Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 09/12] pkg-infra: don't use DL_DIR as scratchpad for temporary downloads Yann E. MORIN ` (3 subsequent siblings) 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Maintaining the download helpers in the Makefile has proved to be a bit complex, so move it to a shell script. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> --- package/pkg-download.mk | 4 ++-- support/download/bzr | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+), 2 deletions(-) create mode 100755 support/download/bzr diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 0c0ee74..5b9cc3a 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -11,7 +11,7 @@ export WGET := $(call qstrip,$(BR2_WGET)) $(QUIET) export SVN := $(call qstrip,$(BR2_SVN)) export CVS := $(call qstrip,$(BR2_CVS)) -BZR := $(call qstrip,$(BR2_BZR)) +export BZR := $(call qstrip,$(BR2_BZR)) export GIT := $(call qstrip,$(BR2_GIT)) export HG := $(call qstrip,$(BR2_HG)) $(QUIET) SCP := $(call qstrip,$(BR2_SCP)) $(QUIET) @@ -101,7 +101,7 @@ endef define DOWNLOAD_BZR test -e $(DL_DIR)/$($(PKG)_SOURCE) || \ - $(BZR) export $(DL_DIR)/$($(PKG)_SOURCE) $($(PKG)_SITE) -r $($(PKG)_DL_VERSION) + $(EXTRA_ENV) support/download/bzr $($(PKG)_SITE) $($(PKG)_DL_VERSION) $(DL_DIR)/$($(PKG)_SOURCE) endef define SOURCE_CHECK_BZR diff --git a/support/download/bzr b/support/download/bzr new file mode 100755 index 0000000..68121f4 --- /dev/null +++ b/support/download/bzr @@ -0,0 +1,19 @@ +#!/bin/sh + +# We want to catch any command failure, and exit immediately +set -e + +# Download helper for bzr +# Call it with: +# $1: bzr repo +# $2: bzr revision +# $3: output file +# And this environment: +# BZR : the bzr command to call +# BR2_DL_DIR: path to Buildroot's download dir + +repo="${1}" +rev="${2}" +output="${3}" + +${BZR} export "${output}" "${repo}" -r "${rev}" -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 09/12] pkg-infra: don't use DL_DIR as scratchpad for temporary downloads 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (7 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 08/12] pkg-infra: move the bzr " Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 19:51 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 10/12] pkg-infra: add possiblity to check downloaded files against known hashes Yann E. MORIN ` (2 subsequent siblings) 11 siblings, 1 reply; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> DL_DIR can be a very precious place for some users: they use it to store all the downloaded archives to share across all their Buildroot (and maybe non-Buildroot) builds. We do not want to trash this location with our temporary downloads (e.g. git, Hg, svn, cvs repository clones/checkouts, or wget, bzr tep tarballs). Turns out that we already have some kind of scratchpad, the BUILD_DIR. Although it is not really a disposable location, that's the best we have so far. Also, we create the temporary tarballs with mktemp using the final tarrball, as template, since we want the temporary to be on the same filesystem as the final location, so the 'mv' is just a plain, atomic rename(2), and we are not left with a half-copied file as the final location. Using mktemp ensures all temp file names are unique, so it allows for parallel downloads from different build dirs at the same time, without cloberring each downloads. Note: we're using neither ${TMP} nor ${TMPDIR} since they are shared locations, sometime with little place (eg. tmpfs), and some of the repositories we clone/checkout can be very big. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> --- Changes v5 -> v6: - ensure we can do parallel downloads from multiple build-dirs (Thomas) Changes v4 -> v5 - explain why we create the temp files where we create them (Arnout) Changes v3 -> v4: - remove spurious bump in package/fis (Samuel) --- support/download/bzr | 24 +++++++++++++++++++++--- support/download/cvs | 25 +++++++++++++++++++------ support/download/git | 25 +++++++++++++++++++------ support/download/hg | 27 ++++++++++++++++++++------- support/download/svn | 26 +++++++++++++++++++------- support/download/wget | 19 +++++++++++++++---- 6 files changed, 113 insertions(+), 33 deletions(-) diff --git a/support/download/bzr b/support/download/bzr index 68121f4..4be36a0 100755 --- a/support/download/bzr +++ b/support/download/bzr @@ -9,11 +9,29 @@ set -e # $2: bzr revision # $3: output file # And this environment: -# BZR : the bzr command to call -# BR2_DL_DIR: path to Buildroot's download dir +# BZR : the bzr command to call +# BUILD_DIR: path to Buildroot's build dir repo="${1}" rev="${2}" output="${3}" -${BZR} export "${output}" "${repo}" -r "${rev}" +tmp_dl="$( mktemp "${BUILD_DIR}/.XXXXXX" )" + +# Remove leftovers from a previous failed run +rm -rf "${BUILD_DIR}/.foo.tmp" + +# Play tric-trac with temp files +# - first, we download to a trashable location (the build-dir) +# - the we move to a temp file in the final location, so it is +# on the same filesystem as the final file +# - finally, we atomically rename to the final file + +if ${BZR} export "${tmp_dl}" "${repo}" -r "${rev}"; then + tmp_output="$( mktemp "${output}.XXXXXX" )" + mv "${tmp_dl}" "${tmp_output}" + mv "${tmp_output}" "${output}" +else + rm -f "${tmp_dl}" + exit 1 +fi diff --git a/support/download/cvs b/support/download/cvs index 06b8647..fa917aa 100755 --- a/support/download/cvs +++ b/support/download/cvs @@ -11,8 +11,8 @@ set -e # $4: package's basename (eg. foobar-1.2.3) # $5: output file # And this environment: -# CVS : the cvs command to call -# BR2_DL_DIR: path to Buildroot's download dir +# CVS : the cvs command to call +# BUILD_DIR: path to Buildroot's build dir repo="${1}" rev="${2}" @@ -20,8 +20,21 @@ rawname="${3}" basename="${4}" output="${5}" -cd "${BR2_DL_DIR}" +repodir="${basename}.tmp-cvs-checkout" + +cd "${BUILD_DIR}" +# Remove leftovers from a previous failed run +rm -rf "${repodir}" + +# Play tric-trac with temp files +# - first, we download to a trashable location (the build-dir) +# - then we create a temporary tarball in the final location, so it is +# on the same filesystem as the final file +# - finally, we atomically rename to the final file + ${CVS} -z3 -d":pserver:anonymous@${repo}" \ - co -d "${basename}" -r ":${rev}" -P "${rawname}" -tar czf "${output}" "${basename}" -rm -rf "${basename}" + co -d "${repodir}" -r ":${rev}" -P "${rawname}" +temp_output="$( mktemp "${output}.XXXXXX" )" +tar czf "${temp_output}" "${repodir}" +mv "${temp_output}" "${output}" +rm -rf "${repodir}" diff --git a/support/download/git b/support/download/git index 96db3a9..7515f91 100755 --- a/support/download/git +++ b/support/download/git @@ -10,15 +10,27 @@ set -e # $3: package's basename (eg. foobar-1.2.3) # $4: output file # And this environment: -# BR2_DL_DIR: path to Buildroot's download dir -# GIT : the git command to call +# GIT : the git command to call +# BUILD_DIR: path to Buildroot's build dir repo="${1}" cset="${2}" basename="${3}" output="${4}" -repodir="${BR2_DL_DIR}/${basename}" +repodir="${basename}.tmp-git-checkout" +tmp_tar="$( mktemp "${BUILD_DIR}/.XXXXXX" )" + +# Play tric-trac with temp files +# - first, we download to a trashable location (the build-dir) +# - then we create the uncomporessed tarball in tht same trashable location +# - then we create a temporary compressed tarball in the final location, so +# it is on the same filesystem as the final file +# - finally, we atomically rename to the final file + +cd "${BUILD_DIR}" +# Remove leftovers from a previous failed run +rm -rf "${repodir}" if [ -n "$(${GIT} ls-remote "${repo}" "${cset}" 2>&1)" ]; then printf "Doing shallow clone\n" @@ -29,9 +41,10 @@ else fi pushd "${repodir}" -${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}" -gzip -c "${output}.tmp" >"${output}" -rm -f "${output}.tmp" +${GIT} archive --prefix="${basename}/" -o "${tmp_tar}" --format=tar "${cset}" +tmp_output="$( mktemp "${output}.XXXXXX" )" +gzip -c "${tmp_tar}" >"${tmp_output}" +mv "${tmp_output}" "${output}" popd rm -rf "${repodir}" diff --git a/support/download/hg b/support/download/hg index 70b49cf..ee4eab8 100755 --- a/support/download/hg +++ b/support/download/hg @@ -10,16 +10,29 @@ set -e # $3: package's basename (eg. foobar-1.2.3) # $4: output file # And this environment: -# HG : the hg command to call -# BR2_DL_DIR: path to Buildroot's download dir +# HG : the hg command to call +# BUILD_DIR: path to Buildroot's build dir repo="${1}" cset="${2}" basename="${3}" output="${4}" -cd "${BR2_DL_DIR}" -${HG} clone --noupdate --rev "${cset}" "${repo}" "${basename}" -${HG} archive --repository "${basename}" --type tgz --prefix "${basename}" \ - --rev "${cset}" "${output}" -rm -rf "${basename}" +repodir="${basename}.tmp-hg-checkout" + +cd "${BUILD_DIR}" +# Remove leftovers from a previous failed run +rm -rf "${repodir}" + +# Play tric-trac with temp files +# - first, we download to a trashable location (the build-dir) +# - then we create a temporary tarball in the final location, so it is +# on the same filesystem as the final file +# - finally, we atomically rename to the final file + +${HG} clone --noupdate --rev "${cset}" "${repo}" "${repodir}" +tmp_output="$( mktemp "${output}.XXXXXX" )" +${HG} archive --repository "${repodir}" --type tgz --prefix "${basename}" \ + --rev "${cset}" "${tmp_output}" +mv "${tmp_output}" "${output}" +rm -rf "${repodir}" diff --git a/support/download/svn b/support/download/svn index c3ab32c..846ab6b 100755 --- a/support/download/svn +++ b/support/download/svn @@ -10,16 +10,28 @@ set -e # $3: package's basename (eg. foobar-1.2.3) # $4: output file # And this environment: -# SVN : the svn command to call -# BR2_DL_DIR: path to Buildroot's download dir +# SVN : the svn command to call +# BUILD_DIR: path to Buildroot's build dir repo="${1}" rev="${2}" basename="${3}" output="${4}" -pushd "${BR2_DL_DIR}" -${SVN} export -r "${rev}" "${repo}" "${basename}" -tar czf "${output}" "${basename}" -rm -rf "${basename}" -popd +repodir="${basename}.tmp-svn-checkout" + +cd "${BUILD_DIR}" +# Remove leftovers from a previous failed run +rm -rf "${repodir}" + +# Play tric-trac with temp files +# - first, we download to a trashable location (the build-dir) +# - then we create a temporary tarball in the final location, so it is +# on the same filesystem as the final file +# - finally, we atomically rename to the final file + +${SVN} export -r "${rev}" "${repo}" "${repodir}" +tmp_output="$( mktemp "${output}.XXXXXX" )" +tar czf "${tmp_output}" "${repodir}" +mv "${tmp_output}" "${output}" +rm -rf "${repodir}" diff --git a/support/download/wget b/support/download/wget index 7865517..b010a55 100755 --- a/support/download/wget +++ b/support/download/wget @@ -8,14 +8,25 @@ set -e # $1: URL # $2: output file # And this environment: -# WGET : the wget command to call +# WGET : the wget command to call +# BUILD_DIR: path to Buildroot's build dir url="${1}" output="${2}" -if ${WGET} -O "${output}.tmp" "${url}"; then - mv "${output}.tmp" "${output}" +tmp_dl="$( mktemp "${BUILD_DIR}/.XXXXXX" )" + +# Play tric-trac with temp files +# - first, we download to a trashable location (the build-dir) +# - then we copy to a temporary tarball in the final location, so it is +# on the same filesystem as the final file +# - finally, we atomically rename to the final file + +if ${WGET} -O "${tmp_dl}" "${url}"; then + tmp_output="$( mktemp "${output}.XXXXXX" )" + mv "${tmp_dl}" "${tmp_output}" + mv "${tmp_output}" "${output}" else - rm -f "${output}.tmp" + rm -f "${tmp_dl}" exit 1 fi -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 09/12] pkg-infra: don't use DL_DIR as scratchpad for temporary downloads 2014-06-01 17:40 ` [Buildroot] [PATCH 09/12] pkg-infra: don't use DL_DIR as scratchpad for temporary downloads Yann E. MORIN @ 2014-06-01 19:51 ` Yann E. MORIN 0 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 19:51 UTC (permalink / raw) To: buildroot All, On 2014-06-01 19:40 +0200, Yann E. MORIN spake thusly: [--SNIP--] > diff --git a/support/download/bzr b/support/download/bzr > index 68121f4..4be36a0 100755 > --- a/support/download/bzr > +++ b/support/download/bzr > @@ -9,11 +9,29 @@ set -e > # $2: bzr revision > # $3: output file > # And this environment: > -# BZR : the bzr command to call > -# BR2_DL_DIR: path to Buildroot's download dir > +# BZR : the bzr command to call > +# BUILD_DIR: path to Buildroot's build dir > > repo="${1}" > rev="${2}" > output="${3}" > > -${BZR} export "${output}" "${repo}" -r "${rev}" > +tmp_dl="$( mktemp "${BUILD_DIR}/.XXXXXX" )" > + > +# Remove leftovers from a previous failed run > +rm -rf "${BUILD_DIR}/.foo.tmp" As noticed by Thomas on IRC: leftovers from a previous attempt. Fixed locally, waiting for more review before re-spinning. Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 10/12] pkg-infra: add possiblity to check downloaded files against known hashes 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (8 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 09/12] pkg-infra: don't use DL_DIR as scratchpad for temporary downloads Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 12/12] package/ca-certificates: add tarball's hashes Yann E. MORIN 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Some of the packages that Buildroot might build are sensitive packages, related to security: openssl, dropbear, ca-certificates... Some of those packages are downloaded over plain http, because there is no way to get them over a secure channel, such as https. In these dark times of pervasive surveillance, the potential for harm that a tampered-with package could generate, we may want to check the integrity of those sensitive packages. So, each package may now provide a list of hashes for all files that needs to be downloaded, and Buildroot will just fail if any downloaded file does not match its known hash, in which case it is removed. Hashes can be any of the md5, sha1 or sha2 variants, and will be checked even if the file was pre-downloaded. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Baruch Siach <baruch@tkos.co.il> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Gustavo Zacarias <gustavo@zacarias.com.ar> Reviewed-by: Samuel Martin <s.martin49@gmail.com> --- Changes v4 -> v5: - fix detection of comments and empty lines --- Note: this is not a bullet-proof solution, since Buildroot may itself be compromised. But since we do sign our releases, then we secure the list of hashes at the same time. Only random snapshots from the repository may be at risk of tampering, although this is highly doubtfull, given how git stores its data. --- package/pkg-download.mk | 20 ++++++++++-- support/download/check-hash | 76 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 93 insertions(+), 3 deletions(-) create mode 100755 support/download/check-hash diff --git a/package/pkg-download.mk b/package/pkg-download.mk index 5b9cc3a..055fd96 100644 --- a/package/pkg-download.mk +++ b/package/pkg-download.mk @@ -58,6 +58,17 @@ domainseparator=$(if $(1),$(1),/) # github(user,package,version): returns site of github repository github = https://github.com/$(1)/$(2)/archive/$(3) +# Helper for checking a tarball's checksum +# If the hash does not match, remove the incorrect file +# $(1): the path to the file with the hashes +# $(2): the full path to the file to check +define VERIFY_HASH + if ! support/download/check-hash $(1) $(2); then \ + rm -f $(2); \ + exit 1; \ + fi +endef + ################################################################################ # The DOWNLOAD_* helpers are in charge of getting a working copy # of the source repository for their corresponding SCM, @@ -147,7 +158,8 @@ endef # to prepend the path with a slash: scp://[user@]host:/absolutepath define DOWNLOAD_SCP test -e $(DL_DIR)/$(2) || \ - $(SCP) '$(call stripurischeme,$(call qstrip,$(1)))' $(DL_DIR)/$(2) + $(SCP) '$(call stripurischeme,$(call qstrip,$(1)))' $(DL_DIR)/$(2) && \ + $(call VERIFY_HASH,$(PKGDIR)/$($(PKG)_NAME).hash,$(DL_DIR)/$(2)) endef define SOURCE_CHECK_SCP @@ -178,7 +190,8 @@ endef define DOWNLOAD_WGET test -e $(DL_DIR)/$(2) || \ - $(EXTRA_ENV) support/download/wget '$(call qstrip,$(1))' $(DL_DIR)/$(2) + $(EXTRA_ENV) support/download/wget '$(call qstrip,$(1))' $(DL_DIR)/$(2) && \ + $(call VERIFY_HASH,$(PKGDIR)/$($(PKG)_NAME).hash,$(DL_DIR)/$(2)) endef define SOURCE_CHECK_WGET @@ -191,7 +204,8 @@ endef define DOWNLOAD_LOCALFILES test -e $(DL_DIR)/$(2) || \ - $(LOCALFILES) $(call stripurischeme,$(call qstrip,$(1))) $(DL_DIR) + $(LOCALFILES) $(call stripurischeme,$(call qstrip,$(1))) $(DL_DIR) && \ + $(call VERIFY_HASH,$(PKGDIR)/$($(PKG)_NAME).hash,$(DL_DIR)/$(2)) endef define SOURCE_CHECK_LOCALFILES diff --git a/support/download/check-hash b/support/download/check-hash new file mode 100755 index 0000000..d498752 --- /dev/null +++ b/support/download/check-hash @@ -0,0 +1,76 @@ +#!/bin/sh +set -e + +# Helper to check a file matches its known hash +# Call it with: +# $1: the full path to the file to check +# $2: the path of the file containing all the the expected hashes + +h_file="${1}" +file="${2}" + +# Does the hash-file exist? +if [ ! -f "${h_file}" ]; then + exit 0 +fi + +# Check one hash for a file +# $1: known hash +# $2: file (full path) +check_one_hash() { + _h="${1}" + _known="${2}" + _file="${3}" + + # Note: sha3 is not supported, since there is currently no implemetation + # (the NIST has yet to publish the parameters). + case "${_h}" in + md5|sha1) ;; + sha224|sha256|sha384|sha512) ;; + *) # Unknown hash, exit with error + printf "ERROR: unknown hash '%s' for '%s'\n" \ + "${_h}" "${_file##*/}" >&2 + exit 1 + ;; + esac + + # Do the hashes match? + _hash=$( ${_h}sum "${_file}" |cut -d ' ' -f 1 ) + if [ "${_hash}" = "${_known}" ]; then + printf "%s: OK (%s: %s)\n" "${_file##*/}" "${_h}" "${_hash}" + return 0 + fi + + printf "ERROR: %s has wrong %s hash:\n" "${_file##*/}" "${_h}" >&2 + printf "ERROR: expected: %s\n" "${_known}" >&2 + printf "ERROR: got : %s\n" "${_hash}" >&2 + printf "ERROR: Incomplete download, or MITM attack\n" >&2 + + exit 1 +} + +# Do we know one or more hashes for that file? +nb_checks=0 +while read t h f; do + case "${t}" in + ''|'#'*) + # Skip comments and empty lines + continue + ;; + *) + if [ "${f}" = "${file##*/}" ]; then + check_one_hash "${t}" "${h}" "${file}" + : $((nb_checks++)) + fi + ;; + esac +done <"${h_file}" + +if [ ${nb_checks} -eq 0 ]; then + if [ -n "${BR2_ENFORCE_CHECK_HASH}" ]; then + printf "ERROR: No hash found for %s\n" "${file}" >&2 + exit 1 + else + printf "WARNING: No hash found for %s\n" "${file}" >&2 + fi +fi -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (9 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 10/12] pkg-infra: add possiblity to check downloaded files against known hashes Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 2014-06-01 20:08 ` Samuel Martin 2014-06-01 17:40 ` [Buildroot] [PATCH 12/12] package/ca-certificates: add tarball's hashes Yann E. MORIN 11 siblings, 1 reply; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Although md5 is, for legacy reasons, a supported hash type, it is not documented on purpose, since it is now known to be weak. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Baruch Siach <baruch@tkos.co.il> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com> --- docs/manual/adding-packages-directory.txt | 67 +++++++++++++++++++++++++++++++ support/download/check-hash | 1 + 2 files changed, 68 insertions(+) diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt index 3cf3afa..0e2cdb3 100644 --- a/docs/manual/adding-packages-directory.txt +++ b/docs/manual/adding-packages-directory.txt @@ -347,3 +347,70 @@ different way, using different infrastructures: Further formatting details: see xref:writing-rules-mk[the writing rules]. + +The +.hash+ file +~~~~~~~~~~~~~~~~ +[[adding-packages-hash]] + +Optionally, you can add a third file, named +libfoo.hash+, that contains +the hashes of the downloaded files for the +libfoo+ package. + +The hashes stored in that file are used to validate the integrity of the +downloaded files. + +The format of this file is one line for each file for which to check the +hash, each line being space-separated, with these three fields: + +* the type of hash, one of: +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ +* the hash of the file: +** for +sha1+, 40 hexadecimal characters +** for +sha224+, 56 hexadecimal characters +** for +sha256+, 64 hexadecimal characters +** for +sha384+, 96 hexadecimal characters +** for +sha512+, 128 hexadecimal characters +* the name of the file, without any directory component + +Lines starting with a +#+ sign are considered comments, and ignored. Empty +lines are ignored. + +There can be more than one hash for a single file, each on its own line. In +this case, all hashes must match. + +Ideally, the hashes stored in this file should match the hashes published by +upstream, e.g. on their website, in the e-mail announcement... If upstream +provides more than one type of hash (say, +sha1+ and +sha512+), then it is +best to add all those hashes in the +.hash+ file. If upstream does not +provide any hash, then compute at least one yourself, and mention this in a +comment line above the hashes. + +*Note:* the number of spaces does not matter, so one can use spaces to +properly align the different fields. + +The example below defines a +sha1+ and a +sha256+ published by upstream for +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: + +---- +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 + +# No upstream hashes for the following: +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin +---- + +If the +.hash+ file is present, and it contains one or more hashes for a +downloaded file, the hash(es) computed by Buildroot (after download) must +match the hash(es) stored in the +.hash+ file. If one or more hashes do +not match, Buildroot considers this an error, deletes the downloaded file, +and aborts. + +If the +.hash+ file is present, but it does not contain a hash for a +downloaded file, no check is done for that file. If you set the +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and +there is no hash for a downloaded file, Buildroot considers this an +error, deletes the downloaded file, and aborts. + +If the +.hash+ file is missing, then no check is done at all. diff --git a/support/download/check-hash b/support/download/check-hash index d498752..ce50e5a 100755 --- a/support/download/check-hash +++ b/support/download/check-hash @@ -22,6 +22,7 @@ check_one_hash() { _known="${2}" _file="${3}" + # Note: md5 is supported, but undocumented on purpose. # Note: sha3 is not supported, since there is currently no implemetation # (the NIST has yet to publish the parameters). case "${_h}" in -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-06-01 17:40 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN @ 2014-06-01 20:08 ` Samuel Martin 2014-06-01 20:13 ` Yann E. MORIN 0 siblings, 1 reply; 25+ messages in thread From: Samuel Martin @ 2014-06-01 20:08 UTC (permalink / raw) To: buildroot Yann, all, On Sun, Jun 1, 2014 at 7:40 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote: > From: "Yann E. MORIN" <yann.morin.1998@free.fr> > > Although md5 is, for legacy reasons, a supported hash type, > it is not documented on purpose, since it is now known to > be weak. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Baruch Siach <baruch@tkos.co.il> > Cc: Arnout Vandecappelle <arnout@mind.be> > Cc: Samuel Martin <s.martin49@gmail.com> > Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> > Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com> > --- > docs/manual/adding-packages-directory.txt | 67 +++++++++++++++++++++++++++++++ > support/download/check-hash | 1 + > 2 files changed, 68 insertions(+) > > diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt > index 3cf3afa..0e2cdb3 100644 > --- a/docs/manual/adding-packages-directory.txt > +++ b/docs/manual/adding-packages-directory.txt > @@ -347,3 +347,70 @@ different way, using different infrastructures: > > Further formatting details: see xref:writing-rules-mk[the writing > rules]. > + > +The +.hash+ file > +~~~~~~~~~~~~~~~~ For a couple of weeks, we are now using the one-line title style. > +[[adding-packages-hash]] > + > +Optionally, you can add a third file, named +libfoo.hash+, that contains > +the hashes of the downloaded files for the +libfoo+ package. > + > +The hashes stored in that file are used to validate the integrity of the > +downloaded files. > + > +The format of this file is one line for each file for which to check the > +hash, each line being space-separated, with these three fields: > + > +* the type of hash, one of: > +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ > +* the hash of the file: > +** for +sha1+, 40 hexadecimal characters > +** for +sha224+, 56 hexadecimal characters > +** for +sha256+, 64 hexadecimal characters > +** for +sha384+, 96 hexadecimal characters > +** for +sha512+, 128 hexadecimal characters > +* the name of the file, without any directory component > + > +Lines starting with a +#+ sign are considered comments, and ignored. Empty > +lines are ignored. > + > +There can be more than one hash for a single file, each on its own line. In > +this case, all hashes must match. > + > +Ideally, the hashes stored in this file should match the hashes published by > +upstream, e.g. on their website, in the e-mail announcement... If upstream > +provides more than one type of hash (say, +sha1+ and +sha512+), then it is > +best to add all those hashes in the +.hash+ file. If upstream does not > +provide any hash, then compute at least one yourself, and mention this in a > +comment line above the hashes. > + > +*Note:* the number of spaces does not matter, so one can use spaces to > +properly align the different fields. > + > +The example below defines a +sha1+ and a +sha256+ published by upstream for > +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, > +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: > + > +---- > +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: > +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 > +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 > + > +# No upstream hashes for the following: > +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch > +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin > +---- > + > +If the +.hash+ file is present, and it contains one or more hashes for a > +downloaded file, the hash(es) computed by Buildroot (after download) must > +match the hash(es) stored in the +.hash+ file. If one or more hashes do > +not match, Buildroot considers this an error, deletes the downloaded file, > +and aborts. > + > +If the +.hash+ file is present, but it does not contain a hash for a > +downloaded file, no check is done for that file. If you set the > +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and > +there is no hash for a downloaded file, Buildroot considers this an > +error, deletes the downloaded file, and aborts. > + > +If the +.hash+ file is missing, then no check is done at all. > diff --git a/support/download/check-hash b/support/download/check-hash > index d498752..ce50e5a 100755 > --- a/support/download/check-hash > +++ b/support/download/check-hash > @@ -22,6 +22,7 @@ check_one_hash() { > _known="${2}" > _file="${3}" > > + # Note: md5 is supported, but undocumented on purpose. > # Note: sha3 is not supported, since there is currently no implemetation > # (the NIST has yet to publish the parameters). > case "${_h}" in > -- > 1.8.3.2 > Regards, -- Samuel ^ permalink raw reply [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-06-01 20:08 ` Samuel Martin @ 2014-06-01 20:13 ` Yann E. MORIN 0 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 20:13 UTC (permalink / raw) To: buildroot Samuel, All, On 2014-06-01 22:08 +0200, Samuel Martin spake thusly: > On Sun, Jun 1, 2014 at 7:40 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote: [--SNIP--] > > diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt > > index 3cf3afa..0e2cdb3 100644 > > --- a/docs/manual/adding-packages-directory.txt > > +++ b/docs/manual/adding-packages-directory.txt > > @@ -347,3 +347,70 @@ different way, using different infrastructures: > > > > Further formatting details: see xref:writing-rules-mk[the writing > > rules]. > > + > > +The +.hash+ file > > +~~~~~~~~~~~~~~~~ > > For a couple of weeks, we are now using the one-line title style. Oh yeah! Thanks! Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 12/12] package/ca-certificates: add tarball's hashes 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN ` (10 preceding siblings ...) 2014-06-01 17:40 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN @ 2014-06-01 17:40 ` Yann E. MORIN 11 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-01 17:40 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> ca-certificates contains sensitive security-related information, and we want to ensure the archive that we download has not been compromised. Add the sha1 and sha256 hashes from Debian's packaging. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Martin Bark <martin@barkynet.com> Reviewed-by: Samuel Martin <s.martin49@gmail.com> --- Changes v4 -> v5: - update hashes since we've bumped the version --- package/ca-certificates/ca-certificates.hash | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 package/ca-certificates/ca-certificates.hash diff --git a/package/ca-certificates/ca-certificates.hash b/package/ca-certificates/ca-certificates.hash new file mode 100644 index 0000000..bcd0723 --- /dev/null +++ b/package/ca-certificates/ca-certificates.hash @@ -0,0 +1,3 @@ +# hashes from: $(CA_CERTIFICATES_SITE)/ca-certificates_$(CA_CERTIFICATES_VERSION).dsc : +sha1 ad57a45f0422fafd78a2e8191e5204f2306cc91b ca-certificates_20140223.tar.xz +sha256 815b7cd97200b0d76450bb3e7d9b65997ac494ab6467b17369f65b2ef94bcb0c ca-certificates_20140223.tar.xz -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 0/12 v10] Some download-related changes (branch yem/check-downloads)
@ 2014-07-02 21:11 Yann E. MORIN
2014-07-02 21:11 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN
0 siblings, 1 reply; 25+ messages in thread
From: Yann E. MORIN @ 2014-07-02 21:11 UTC (permalink / raw)
To: buildroot
Hello All!
This series contains those download-related changes:
- Patches 1..8 : move download helpers to shell scripts, one
such script for each download helper
- Patch 9 : don't use DL_DIR as scratchpad for temporary VCS
checkouts, since DL_DIR is a precious location
- Patches 10..11: check hashes of downloaded files, to ensure the
files are genuine (with doc)
- Patch 12 : add hashes for ca-certificates
Hashes for more packages (eg. openssh, openssh, dropbear...) can be
added at a later step.
Changes v9 -> v10:
- use /bin/bash instead of /bin/sh, since we use bashisms (Peter)
- drop patches already applied (Peter)
Changes v8 -> v9:
- include change by Ryan about svn peg-revision
- rebase on top of master after the crazy $$ springling ;-)
Changes v7 -> v8:
- convert scp and localfiles (Thomas DS, Peter)
- typoes, expand MITM (Thomas DS)
- use a cleaner and saner code path to detect errors in helpers
Changes v6 -> v7:
- use single-line title in manual (Samuel)
- use leftover cruft in bzr helper (Thomas)
Changes v5 -> v6:
- ensure we can run multiple parallel downloads from different build
dirs with the same BR2_DL_DIR (Thomas)
- also convert bzr
- fix for stray failed downloads
Changes v4 -> v5:
- update ca-certificates hashes after version bump
- fix detection of comments and empty lines in .hash file
- rebase on top of master
Changes v3 -> v4:
- enhance the manual, typoes (Gustavo, Samuel, Thomas DS)
- remove spurious test bump on package/fis (Samuel)
Changes v2 -> v3:
- avoid partial downloads (Thomas DS)
- store hash-type in .hash file (Gustavo)
- typoes (Samuel, Gustavo)
- add possibility to treat missing hashes as an error
Changes v1 -> v2:
- don't do any behavioural change when switching to shell scripts,
just reproduce exactly what was in the Makefile (Luca, Arnout)
- support more than one hash algorithm (Arnout, Gustavo)
- typoes (Luca, Baruch)
Regards,
Yann E. MORIN.
The following changes since commit 2d4fd5d358966b9959a0ea9f93daa3a08c6068e7:
pkg-infra: also set PKGDIR for the download step (2014-07-02 17:16:14 +0200)
are available in the git repository at:
git://gitorious.org/buildroot/buildroot.git yem/check-downloads
for you to fetch changes up to 6c119c59e1be7918d55ec414cec18f32ed7844b6:
package/ca-certificates: add tarball's hashes (2014-07-02 23:04:19 +0200)
----------------------------------------------------------------
Yann E. MORIN (12):
pkg-infra: move the git download helper to a script
pkg-infra: move the svn download helper to a script
pkg-infra: move the cvs download helper to a script
pkg-infra: move the hg download helper to a script
pkg-infra: move the wget download helper to a script
pkg-infra: move the bzr download helper to a script
pkg-infra: move the scp download helper to a script
pkg-infra: move the cp (aka localfiles) download helper to a script
pkg-infra: don't use DL_DIR as scratchpad for temporary downloads
pkg-infra: add possiblity to check downloaded files against known hashes
manual: add documentation about packages' hashes
package/ca-certificates: add tarball's hashes
docs/manual/adding-packages-directory.txt | 66 +++++++++++++++++++++
package/ca-certificates/ca-certificates.hash | 3 +
package/pkg-download.mk | 89 ++++++++++++----------------
support/download/bzr | 38 ++++++++++++
support/download/check-hash | 77 ++++++++++++++++++++++++
support/download/cp | 26 ++++++++
support/download/cvs | 47 +++++++++++++++
support/download/git | 56 +++++++++++++++++
support/download/hg | 46 ++++++++++++++
support/download/scp | 28 +++++++++
support/download/svn | 44 ++++++++++++++
support/download/wget | 35 +++++++++++
12 files changed, 504 insertions(+), 51 deletions(-)
create mode 100644 package/ca-certificates/ca-certificates.hash
create mode 100755 support/download/bzr
create mode 100755 support/download/check-hash
create mode 100755 support/download/cp
create mode 100755 support/download/cvs
create mode 100755 support/download/git
create mode 100755 support/download/hg
create mode 100755 support/download/scp
create mode 100755 support/download/svn
create mode 100755 support/download/wget
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 25+ messages in thread* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-07-02 21:11 [Buildroot] [PATCH 0/12 v10] Some download-related changes (branch yem/check-downloads) Yann E. MORIN @ 2014-07-02 21:11 ` Yann E. MORIN 0 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-07-02 21:11 UTC (permalink / raw) To: buildroot Although md5 is, for legacy reasons, a supported hash type, it is not documented on purpose, since it is now known to be weak. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Baruch Siach <baruch@tkos.co.il> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com> --- Changes v6 -> v7: - use single-line title (Samuel) --- docs/manual/adding-packages-directory.txt | 66 +++++++++++++++++++++++++++++++ support/download/check-hash | 1 + 2 files changed, 67 insertions(+) diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt index 69b012c..c2f0a1a 100644 --- a/docs/manual/adding-packages-directory.txt +++ b/docs/manual/adding-packages-directory.txt @@ -357,3 +357,69 @@ different way, using different infrastructures: Further formatting details: see xref:writing-rules-mk[the writing rules]. + +[[adding-packages-hash]] +=== The +.hash+ file + +Optionally, you can add a third file, named +libfoo.hash+, that contains +the hashes of the downloaded files for the +libfoo+ package. + +The hashes stored in that file are used to validate the integrity of the +downloaded files. + +The format of this file is one line for each file for which to check the +hash, each line being space-separated, with these three fields: + +* the type of hash, one of: +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ +* the hash of the file: +** for +sha1+, 40 hexadecimal characters +** for +sha224+, 56 hexadecimal characters +** for +sha256+, 64 hexadecimal characters +** for +sha384+, 96 hexadecimal characters +** for +sha512+, 128 hexadecimal characters +* the name of the file, without any directory component + +Lines starting with a +#+ sign are considered comments, and ignored. Empty +lines are ignored. + +There can be more than one hash for a single file, each on its own line. In +this case, all hashes must match. + +Ideally, the hashes stored in this file should match the hashes published by +upstream, e.g. on their website, in the e-mail announcement... If upstream +provides more than one type of hash (say, +sha1+ and +sha512+), then it is +best to add all those hashes in the +.hash+ file. If upstream does not +provide any hash, then compute at least one yourself, and mention this in a +comment line above the hashes. + +*Note:* the number of spaces does not matter, so one can use spaces to +properly align the different fields. + +The example below defines a +sha1+ and a +sha256+ published by upstream for +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: + +---- +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 + +# No upstream hashes for the following: +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin +---- + +If the +.hash+ file is present, and it contains one or more hashes for a +downloaded file, the hash(es) computed by Buildroot (after download) must +match the hash(es) stored in the +.hash+ file. If one or more hashes do +not match, Buildroot considers this an error, deletes the downloaded file, +and aborts. + +If the +.hash+ file is present, but it does not contain a hash for a +downloaded file, no check is done for that file. If you set the +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and +there is no hash for a downloaded file, Buildroot considers this an +error, deletes the downloaded file, and aborts. + +If the +.hash+ file is missing, then no check is done at all. diff --git a/support/download/check-hash b/support/download/check-hash index 9ea7c41..f1e0c1b 100755 --- a/support/download/check-hash +++ b/support/download/check-hash @@ -22,6 +22,7 @@ check_one_hash() { _known="${2}" _file="${3}" + # Note: md5 is supported, but undocumented on purpose. # Note: sha3 is not supported, since there is currently no implementation # (the NIST has yet to publish the parameters). case "${_h}" in -- 1.9.1 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 0/12 v7] Some download-related changes (branch yem/check-downloads)
@ 2014-06-08 20:43 Yann E. MORIN
2014-06-08 20:43 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN
0 siblings, 1 reply; 25+ messages in thread
From: Yann E. MORIN @ 2014-06-08 20:43 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Hello All!
This series contains those download-related changes:
- Patches 1..2 : preparatory patches for upcoming helper scripts and
hash checks
- Patches 3..8 : move download helpers to shell scripts, one
such script for each download helper
- Patch 9 : don't use DL_DIR as scratchpad for temporary VCS
checkouts, since DL_DIR is a precious location
- Patches 10..11: check hashes of downloaded files, to ensure the
files are genuine (with doc)
- Patch 12 : add hashes for ca-certificates
Changes v6 -> v7:
- use single-line title in manual (Samuel)
- use leftover cruft in bzr helper (Thomas)
Changes v5 -> v6:
- ensure we can run multiple parallel downloads from different build
dirs with the same BR2_DL_DIR (Thomas)
- also convert bzr
- fix for stray failed downloads
Changes v4 -> v5:
- update ca-certificates hashes after version bump
- fix detection of comments and empty lines in .hash file
- rebase on top of master
Changes v3 -> v4:
- enhance the manual, typoes (Gustavo, Samuel, Thomas DS)
- remove spurious test bump on package/fis (Samuel)
Changes v2 -> v3:
- avoid partial downloads (Thomas DS)
- store hash-type in .hash file (Gustavo)
- typoes (Samuel, Gustavo)
- add possibility to treat missing hashes as an error
Changes v1 -> v2:
- don't do any behavioural change when switching to shell scripts,
just reproduce exactly what was in the Makefile (Luca, Arnout)
- support more than one hash algorithm (Arnout, Gustavo)
- typoes (Luca, Baruch)
Regards,
Yann E. MORIN.
The following changes since commit 1ff9045f4b62b53f55457f00a80d63e77acd758c:
sunxi-mali: use only -I${includedir} as pkg-config Cflags (2014-06-08 18:52:14 +0200)
are available in the git repository at:
git://gitorious.org/buildroot/buildroot.git yem/check-downloads
for you to fetch changes up to 305759abe954f35112f303fe909e97c1ca33bb63:
package/ca-certificates: add tarball's hashes (2014-06-08 22:39:47 +0200)
----------------------------------------------------------------
Yann E. MORIN (12):
Makefile: add BR2_DL_DIR to EXTRA_ENV
pkg-infra: also set PKGDIR for the download step
pkg-infra: move the git download helper to a script
pkg-infra: move the svn download helper to a script
pkg-infra: move the cvs download helper to a script
pkg-infra: move the hg download helper to a script
pkg-infra: move the wget download helper to a script
pkg-infra: move the bzr download helper to a script
pkg-infra: don't use DL_DIR as scratchpad for temporary downloads
pkg-infra: add possiblity to check downloaded files against known hashes
manual: add documentation about packages' hashes
package/ca-certificates: add tarball's hashes
docs/manual/adding-packages-directory.txt | 66 ++++++++++++++++++++++
package/Makefile.in | 1 +
package/ca-certificates/ca-certificates.hash | 3 +
package/pkg-download.mk | 83 ++++++++++++----------------
package/pkg-generic.mk | 1 +
support/download/bzr | 34 ++++++++++++
support/download/check-hash | 77 ++++++++++++++++++++++++++
support/download/cvs | 40 ++++++++++++++
support/download/git | 50 +++++++++++++++++
support/download/hg | 38 +++++++++++++
support/download/svn | 37 +++++++++++++
support/download/wget | 32 +++++++++++
12 files changed, 413 insertions(+), 49 deletions(-)
create mode 100644 package/ca-certificates/ca-certificates.hash
create mode 100755 support/download/bzr
create mode 100755 support/download/check-hash
create mode 100755 support/download/cvs
create mode 100755 support/download/git
create mode 100755 support/download/hg
create mode 100755 support/download/svn
create mode 100755 support/download/wget
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 25+ messages in thread* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-06-08 20:43 [Buildroot] [PATCH 0/12 v7] Some download-related changes (branch yem/check-downloads) Yann E. MORIN @ 2014-06-08 20:43 ` Yann E. MORIN 0 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-06-08 20:43 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Although md5 is, for legacy reasons, a supported hash type, it is not documented on purpose, since it is now known to be weak. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Baruch Siach <baruch@tkos.co.il> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com> --- Changes v6 -> v7: - use single-line title (Samuel) --- docs/manual/adding-packages-directory.txt | 66 +++++++++++++++++++++++++++++++ support/download/check-hash | 1 + 2 files changed, 67 insertions(+) diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt index 69b012c..c2f0a1a 100644 --- a/docs/manual/adding-packages-directory.txt +++ b/docs/manual/adding-packages-directory.txt @@ -357,3 +357,69 @@ different way, using different infrastructures: Further formatting details: see xref:writing-rules-mk[the writing rules]. + +[[adding-packages-hash]] +=== The +.hash+ file + +Optionally, you can add a third file, named +libfoo.hash+, that contains +the hashes of the downloaded files for the +libfoo+ package. + +The hashes stored in that file are used to validate the integrity of the +downloaded files. + +The format of this file is one line for each file for which to check the +hash, each line being space-separated, with these three fields: + +* the type of hash, one of: +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ +* the hash of the file: +** for +sha1+, 40 hexadecimal characters +** for +sha224+, 56 hexadecimal characters +** for +sha256+, 64 hexadecimal characters +** for +sha384+, 96 hexadecimal characters +** for +sha512+, 128 hexadecimal characters +* the name of the file, without any directory component + +Lines starting with a +#+ sign are considered comments, and ignored. Empty +lines are ignored. + +There can be more than one hash for a single file, each on its own line. In +this case, all hashes must match. + +Ideally, the hashes stored in this file should match the hashes published by +upstream, e.g. on their website, in the e-mail announcement... If upstream +provides more than one type of hash (say, +sha1+ and +sha512+), then it is +best to add all those hashes in the +.hash+ file. If upstream does not +provide any hash, then compute at least one yourself, and mention this in a +comment line above the hashes. + +*Note:* the number of spaces does not matter, so one can use spaces to +properly align the different fields. + +The example below defines a +sha1+ and a +sha256+ published by upstream for +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: + +---- +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 + +# No upstream hashes for the following: +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin +---- + +If the +.hash+ file is present, and it contains one or more hashes for a +downloaded file, the hash(es) computed by Buildroot (after download) must +match the hash(es) stored in the +.hash+ file. If one or more hashes do +not match, Buildroot considers this an error, deletes the downloaded file, +and aborts. + +If the +.hash+ file is present, but it does not contain a hash for a +downloaded file, no check is done for that file. If you set the +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and +there is no hash for a downloaded file, Buildroot considers this an +error, deletes the downloaded file, and aborts. + +If the +.hash+ file is missing, then no check is done at all. diff --git a/support/download/check-hash b/support/download/check-hash index d498752..ce50e5a 100755 --- a/support/download/check-hash +++ b/support/download/check-hash @@ -22,6 +22,7 @@ check_one_hash() { _known="${2}" _file="${3}" + # Note: md5 is supported, but undocumented on purpose. # Note: sha3 is not supported, since there is currently no implemetation # (the NIST has yet to publish the parameters). case "${_h}" in -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 0/12 v4] Some download-related changes (branch yem/check-downloads)
@ 2014-03-10 20:51 Yann E. MORIN
2014-03-10 20:51 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN
0 siblings, 1 reply; 25+ messages in thread
From: Yann E. MORIN @ 2014-03-10 20:51 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Hello All!
This series contains those download-related changes:
- Patches 1..2 : some cleanups and preparatory works in the
download infra,
- Patches 3..7 : move download helpers to shell scripts, one
such script for each download helper
- Patch 8 : don't use DL_DIR as scratchpad for temporary VCS
checkouts, since DL_DIR is a precious location
- Patches 9..11 : check hashes of downloaded files, to ensure the
files are genuine
- Patch 12 : add hashes for ca-certificates
Changes v3 -> v4:
- enhance the manual, typoes (Gustavo, Samuel, Thomas DS)
- remove spurious test bump on package/fis (Samuel)
Changes v2 -> v3:
- avoid partial downloads (Thomas DS)
- store hash-type in .hash file (Gustavo)
- typoes (Samuel, Gustavo)
- add possibility to treat missing hashes as an error
Changes v1 -> v2:
- don't do any behavioural change when switching to shell scripts,
just reproduce exactly what was in the Makefile (Luca, Arnout)
- support more than one hash algorithm (Arnout, Gustavo)
- typoes (Luca, Baruch)
Regards,
Yann E. MORIN.
The following changes since commit ca4e9a78732bcacddd0c91ab35ee39a60ff040aa:
jack2: fix missing alsa-lib feature dependency (2014-03-10 13:13:03 +0100)
are available in the git repository at:
git://gitorious.org/buildroot/buildroot.git yem/check-downloads
for you to fetch changes up to 985eb7379f1347835b44d685bbcef295fea730c5:
package/ca-certificates: add tarball's hashes (2014-03-10 18:15:03 +0100)
----------------------------------------------------------------
Yann E. MORIN (12):
Makefile: rename USER_HOOKS_EXTRA_ENV to EXTRA_ENV
Makefile: add BR2_DL_DIR to EXTRA_ENV
pkg-infra: move the git download helper to a script
pkg-infra: move the svn download helper to a script
pkg-infra: move the cvs download helper to a script
pkg-infra: move the hg download helper to a script
pkg-infra: move the wget download helper to a script
pkg-infra: don't use DL_DIR as scratchpad for temporary VCS checkouts
pkg-infra: also set PKGDIR for the download step
pkg-infra: add possiblity to check downloaded files against known hashes
manual: add documentation about packages' hashes
package/ca-certificates: add tarball's hashes
Makefile | 4 +-
docs/manual/adding-packages-directory.txt | 67 +++++++++++++++++++++++
package/Makefile.in | 5 +-
package/ca-certificates/ca-certificates.hash | 3 ++
package/pkg-download.mk | 79 +++++++++++-----------------
package/pkg-generic.mk | 3 +-
support/download/check-hash | 77 +++++++++++++++++++++++++++
support/download/cvs | 33 ++++++++++++
support/download/git | 41 +++++++++++++++
support/download/hg | 31 +++++++++++
support/download/svn | 30 +++++++++++
support/download/wget | 24 +++++++++
12 files changed, 345 insertions(+), 52 deletions(-)
create mode 100644 package/ca-certificates/ca-certificates.hash
create mode 100755 support/download/check-hash
create mode 100755 support/download/cvs
create mode 100755 support/download/git
create mode 100755 support/download/hg
create mode 100755 support/download/svn
create mode 100755 support/download/wget
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 25+ messages in thread* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-03-10 20:51 [Buildroot] [PATCH 0/12 v4] Some download-related changes (branch yem/check-downloads) Yann E. MORIN @ 2014-03-10 20:51 ` Yann E. MORIN 2014-03-11 10:37 ` Thomas De Schampheleire 0 siblings, 1 reply; 25+ messages in thread From: Yann E. MORIN @ 2014-03-10 20:51 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Although md5 is, for legacy resaons, a supported hash type, it is not documented on purpose, since it is now known to be weak. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Baruch Siach <baruch@tkos.co.il> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> --- docs/manual/adding-packages-directory.txt | 67 +++++++++++++++++++++++++++++++ support/download/check-hash | 1 + 2 files changed, 68 insertions(+) diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt index e56e59a..ba0c05f 100644 --- a/docs/manual/adding-packages-directory.txt +++ b/docs/manual/adding-packages-directory.txt @@ -346,3 +346,70 @@ different way, using different infrastructures: Further formatting details: see xref:writing-rules-mk[the writing rules]. + +The +.hash+ file +~~~~~~~~~~~~~~~~ +[[adding-packages-hash]] + +Optionally, you can add a third file, named +libfoo.hash+, that contains +the hashes of the downloaded files for the +libfoo+ package. + +The hashes stored in that file are used to validate the integrity of the +downloaded files. + +The format of this file is one line for each file for which to check the +hash, each line being space-separated, with these three fields: + +* the type of hash, one of: +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ +* the hash of the file: +** for +sha1+, 40 hexadecimal characters +** for +sha224+, 56 hexadecimal characters +** for +sha256+, 64 hexadecimal characters +** for +sha384+, 96 hexadecimal characters +** for +sha512+, 128 hexadecimal characters +* the name of the file, without any directory component + +Lines starting with a +#+ sign are considered comments, and ignored. Empty +lines are ignored. + +There can be more than one hash for a single file, each on its own line. In +this case, all hashes must match. + +Ideally, the hashes stored in this file should match the hashes published by +upstream, e.g. on their website, in the e-mail announcement... If upstream +provides more than one type of hash (say, +sha1+ and +sha512+), then it is +best to add all those hashes in the +.hash+ file. If upstream does not +provide any hash, then compute at least one yourself, and mention this in a +comment line above the hashes. + +*Note:* the number of spaces does not matter, so one can use spaces to +properly align the different fields. + +The example below defines a +sha1+ and a +sha256+ published by upstream for +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: + +---- +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 + +# No upstream hashes for the following: +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin +---- + +If the +.hash+ file is present, and it contains one or more hashes for a +downloaded file, the hash(es) computed by Buildroot (after download) must +match the hash(es) stored in the +.hash+ file. If one or more hashes do +not match, Buildroot considers this an error, deletes the downloaded file, +and aborts. + +If the +.hash+ file is present, but it does not contain a hash for a +downloaded file, no check is done for that file. If you set the +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and +there is no hash for a downloaded file, Buildroot considers this an +error, deletes the downloaded file, and aborts. + +If the +.hash+ file is missing, then no check is done at all. diff --git a/support/download/check-hash b/support/download/check-hash index 1db8b8c..90bfc4e 100755 --- a/support/download/check-hash +++ b/support/download/check-hash @@ -22,6 +22,7 @@ check_one_hash() { _known="${2}" _file="${3}" + # Note: md5 is supported, but undocumented on purpose. # Note: sha3 is not supported, since there is currently no implemetation # (the NIST has yet to publish the parameters). case "${_h}" in -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-03-10 20:51 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN @ 2014-03-11 10:37 ` Thomas De Schampheleire 0 siblings, 0 replies; 25+ messages in thread From: Thomas De Schampheleire @ 2014-03-11 10:37 UTC (permalink / raw) To: buildroot On Mon, Mar 10, 2014 at 9:51 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote: > From: "Yann E. MORIN" <yann.morin.1998@free.fr> > > Although md5 is, for legacy resaons, a supported hash type, > it is not documented on purpose, since it is now known to > be weak. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Baruch Siach <baruch@tkos.co.il> > Cc: Arnout Vandecappelle <arnout@mind.be> > Cc: Samuel Martin <s.martin49@gmail.com> > Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com> Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com> ^ permalink raw reply [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 0/12 v3] some download-related changes
@ 2014-03-05 21:47 Yann E. MORIN
2014-03-05 21:47 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN
0 siblings, 1 reply; 25+ messages in thread
From: Yann E. MORIN @ 2014-03-05 21:47 UTC (permalink / raw)
To: buildroot
From: "Yann E. MORIN" <yann.morin.1998@free.fr>
Hello All!
This series contains those download-related changes:
- Patches 1..2 : some cleanups and preparatory works in the
download infra,
- Patches 3..7 : move download helpers to shell scripts, one
such script for each download helper
- Patches 8..9 : don't use DL_DIR as scratchpad for temporary VCS
checkouts, since DL_DIR is a precious location
- Patches 10..12: check hashes of downloaded files, to ensure the
files are genuine
Samuel, I removed your reviewed-by tag on the patches that have changed
quite a bit since last round: patches 8, 10 & 11.
Changes v2 -> v3:
- avoid partial downloads (Thomas DS)
- store hash-type in .hash file (Gustavo)
- typoes (Samuel, Gustavo)
- add possibility to treat missing hashes as an error
Changes v1 -> v2:
- don't do any behavioural change when switching to shell scripts,
just reproduce exactly what was in the Makefile (Luca, Arnout)
- support more than one hash algorithm (Arnout, Gustavo)
- typoes (Luca, Baruch)
Regards,
Yann E. MORIN.
----------------------------------------------------------------
Yann E. MORIN (12):
Makefile: rename USER_HOOKS_EXTRA_ENV to EXTRA_ENV
Makefile: add BR2_DL_DIR to EXTRA_ENV
pkg-infra: move the git download helper to a script
pkg-infra: move the svn download helper to a script
pkg-infra: move the cvs download helper to a script
pkg-infra: move the hg download helper to a script
pkg-infra: move the wget download helper to a script
pkg-infra: don't use DL_DIR as scratchpad for temporary VCS checkouts
pkg-infra: also set PKGDIR for the download step
pkg-infra: add possiblity to check downloaded files against known hashes
manual: add documentation about packages' hashes
package/ca-certificates: add tarball's hashes
Makefile | 4 +-
docs/manual/adding-packages-directory.txt | 65 +++++++++++++++++++++++
package/Makefile.in | 5 +-
package/ca-certificates/ca-certificates.hash | 3 ++
package/fis/fis.mk | 2 +-
package/pkg-download.mk | 79 +++++++++++-----------------
package/pkg-generic.mk | 3 +-
support/download/check-hash | 77 +++++++++++++++++++++++++++
support/download/cvs | 33 ++++++++++++
support/download/git | 41 +++++++++++++++
support/download/hg | 31 +++++++++++
support/download/svn | 30 +++++++++++
support/download/wget | 24 +++++++++
13 files changed, 344 insertions(+), 53 deletions(-)
create mode 100644 package/ca-certificates/ca-certificates.hash
create mode 100755 support/download/check-hash
create mode 100755 support/download/cvs
create mode 100755 support/download/git
create mode 100755 support/download/hg
create mode 100755 support/download/svn
create mode 100755 support/download/wget
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
^ permalink raw reply [flat|nested] 25+ messages in thread* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-03-05 21:47 [Buildroot] [PATCH 0/12 v3] some download-related changes Yann E. MORIN @ 2014-03-05 21:47 ` Yann E. MORIN 2014-03-06 10:56 ` Samuel Martin 2014-03-06 17:14 ` Thomas De Schampheleire 0 siblings, 2 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-03-05 21:47 UTC (permalink / raw) To: buildroot From: "Yann E. MORIN" <yann.morin.1998@free.fr> Although md5 is, for legacy resaons, a supported hash type, it is not documented on purpose, since it is now known to be weak. Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> Cc: Baruch Siach <baruch@tkos.co.il> Cc: Arnout Vandecappelle <arnout@mind.be> Cc: Samuel Martin <s.martin49@gmail.com> --- docs/manual/adding-packages-directory.txt | 65 +++++++++++++++++++++++++++++++ support/download/check-hash | 1 + 2 files changed, 66 insertions(+) diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt index e56e59a..4609a7e 100644 --- a/docs/manual/adding-packages-directory.txt +++ b/docs/manual/adding-packages-directory.txt @@ -346,3 +346,68 @@ different way, using different infrastructures: Further formatting details: see xref:writing-rules-mk[the writing rules]. + +The +.hash+ file +~~~~~~~~~~~~~~~~ +[[adding-packages-hash]] + +Optionally, you can add a third file, named +libfoo.hash+, that contains +the hashes of the downloaded files for the +libfoo+ package. + +The hashes stored in that file are used to validate the integrity of the +downloaded files. + +The format for this file is one line for each file for which to check the +hash, each line being space-separated, with these three fields: + +* the type of hash, one of: +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ +* the hash of the file: +** for +sha1+, 40 hexa-decimal characters +** for +sha224+, 56 hexa-decimal characters +** for +sha256+, 64 hexa-decimal characters +** for +sha384+, 96 hexa-decimal characters +** for +sha512+, 128 hexa-decimal characters +* the name of the file, without any directory component + +Lines starting with a +#+ sign are considered comments, and ignored. Empty +lines are ignored. + +There can be more than one hash for a single file, each of its own line. In +this case, all hashes must match. + +Ideally, the hashes stored in this file should match the hashes published by +upstream, eg, on their website, in the email announcement... If no such info +is available, then compute them yourself, and state so in a comment line +above the hashes. + +*Note:* the number of spaces does not matter, so one can use spaces to +properly align the different fields. + +The example below defines a +sha1+ and a +sha256+ published by upstream for +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: + +---- +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 + +# No upstream hashes for the following: +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin +---- + +If the +.hash+ file is present, and there is one or more hashes for a +downloaded file, the hash(es) of the downloaded file must match the +hash(es) stored in the +.hash+ file. If one or more hashes do not match, +Buildroot considers this as an error, deletes the downloaded file, and +aborts. + +If the +.hash+ file is present, but there is no hash for a downloaded +file, no check is done for that downloaded file. If you set the +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and +there is no hash for a downloaded file, Buildroot considers this as an +error, deletes the downloaded file, and aborts. + +If the +.hash+ file is missing, then no check is done at all. diff --git a/support/download/check-hash b/support/download/check-hash index 1db8b8c..61678d4 100755 --- a/support/download/check-hash +++ b/support/download/check-hash @@ -22,6 +22,7 @@ check_one_hash() { _known="${2}" _file="${3}" + # Note: md5 is supported, but un-documented on-purpose. # Note: sha3 is not supported, since there is currently no implemetation # (the NIST has yet to publish the parameters). case "${_h}" in -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-03-05 21:47 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN @ 2014-03-06 10:56 ` Samuel Martin 2014-03-06 17:09 ` Yann E. MORIN 2014-03-06 17:14 ` Thomas De Schampheleire 1 sibling, 1 reply; 25+ messages in thread From: Samuel Martin @ 2014-03-06 10:56 UTC (permalink / raw) To: buildroot Yann, all, On Wed, Mar 5, 2014 at 10:47 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote: > From: "Yann E. MORIN" <yann.morin.1998@free.fr> > > Although md5 is, for legacy resaons, a supported hash type, > it is not documented on purpose, since it is now known to > be weak. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Baruch Siach <baruch@tkos.co.il> > Cc: Arnout Vandecappelle <arnout@mind.be> > Cc: Samuel Martin <s.martin49@gmail.com> > --- > docs/manual/adding-packages-directory.txt | 65 +++++++++++++++++++++++++++++++ > support/download/check-hash | 1 + > 2 files changed, 66 insertions(+) > > diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt > index e56e59a..4609a7e 100644 > --- a/docs/manual/adding-packages-directory.txt > +++ b/docs/manual/adding-packages-directory.txt > @@ -346,3 +346,68 @@ different way, using different infrastructures: > > Further formatting details: see xref:writing-rules-mk[the writing > rules]. > + > +The +.hash+ file > +~~~~~~~~~~~~~~~~ > +[[adding-packages-hash]] > + > +Optionally, you can add a third file, named +libfoo.hash+, that contains > +the hashes of the downloaded files for the +libfoo+ package. > + > +The hashes stored in that file are used to validate the integrity of the > +downloaded files. > + > +The format for this file is one line for each file for which to check the > +hash, each line being space-separated, with these three fields: > + > +* the type of hash, one of: > +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ > +* the hash of the file: > +** for +sha1+, 40 hexa-decimal characters > +** for +sha224+, 56 hexa-decimal characters > +** for +sha256+, 64 hexa-decimal characters > +** for +sha384+, 96 hexa-decimal characters > +** for +sha512+, 128 hexa-decimal characters > +* the name of the file, without any directory component > + > +Lines starting with a +#+ sign are considered comments, and ignored. Empty > +lines are ignored. > + > +There can be more than one hash for a single file, each of its own line. In > +this case, all hashes must match. Maybe a note explaining why it's better to provide more than 1 hash for a file could be added. > + > +Ideally, the hashes stored in this file should match the hashes published by > +upstream, eg, on their website, in the email announcement... If no such info > +is available, then compute them yourself, and state so in a comment line > +above the hashes. > + > +*Note:* the number of spaces does not matter, so one can use spaces to > +properly align the different fields. > + > +The example below defines a +sha1+ and a +sha256+ published by upstream for > +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, > +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: > + > +---- > +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: > +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 > +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 > + > +# No upstream hashes for the following: > +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch > +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin > +---- > + > +If the +.hash+ file is present, and there is one or more hashes for a > +downloaded file, the hash(es) of the downloaded file must match the > +hash(es) stored in the +.hash+ file. If one or more hashes do not match, > +Buildroot considers this as an error, deletes the downloaded file, and > +aborts. > + > +If the +.hash+ file is present, but there is no hash for a downloaded > +file, no check is done for that downloaded file. If you set the > +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and > +there is no hash for a downloaded file, Buildroot considers this as an > +error, deletes the downloaded file, and aborts. > + > +If the +.hash+ file is missing, then no check is done at all. > diff --git a/support/download/check-hash b/support/download/check-hash > index 1db8b8c..61678d4 100755 > --- a/support/download/check-hash > +++ b/support/download/check-hash > @@ -22,6 +22,7 @@ check_one_hash() { > _known="${2}" > _file="${3}" > > + # Note: md5 is supported, but un-documented on-purpose. > # Note: sha3 is not supported, since there is currently no implemetation > # (the NIST has yet to publish the parameters). > case "${_h}" in > -- > 1.8.3.2 > Regards, -- Samuel ^ permalink raw reply [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-03-06 10:56 ` Samuel Martin @ 2014-03-06 17:09 ` Yann E. MORIN 0 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-03-06 17:09 UTC (permalink / raw) To: buildroot Samuel, All, On 2014-03-06 11:56 +0100, Samuel Martin spake thusly: > On Wed, Mar 5, 2014 at 10:47 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote: [--SNIP--] > > diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt > > index e56e59a..4609a7e 100644 > > --- a/docs/manual/adding-packages-directory.txt > > +++ b/docs/manual/adding-packages-directory.txt > > @@ -346,3 +346,68 @@ different way, using different infrastructures: > > > > Further formatting details: see xref:writing-rules-mk[the writing > > rules]. > > + > > +The +.hash+ file > > +~~~~~~~~~~~~~~~~ > > +[[adding-packages-hash]] > > + > > +Optionally, you can add a third file, named +libfoo.hash+, that contains > > +the hashes of the downloaded files for the +libfoo+ package. > > + > > +The hashes stored in that file are used to validate the integrity of the > > +downloaded files. > > + > > +The format for this file is one line for each file for which to check the > > +hash, each line being space-separated, with these three fields: > > + > > +* the type of hash, one of: > > +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ > > +* the hash of the file: > > +** for +sha1+, 40 hexa-decimal characters > > +** for +sha224+, 56 hexa-decimal characters > > +** for +sha256+, 64 hexa-decimal characters > > +** for +sha384+, 96 hexa-decimal characters > > +** for +sha512+, 128 hexa-decimal characters > > +* the name of the file, without any directory component > > + > > +Lines starting with a +#+ sign are considered comments, and ignored. Empty > > +lines are ignored. > > + > > +There can be more than one hash for a single file, each of its own line. In > > +this case, all hashes must match. > > Maybe a note explaining why it's better to provide more than 1 hash > for a file could be added. As I said to Gustavo on IRC, I'd prefer we only document the format of the .hash file in the manual, not define any policy. Ie. I don't think it is sensible to say something like: For security considerations, adding more than one hash will ower the risk of collusions if more than one hash type is provided. However, we can say, and I will add, something like: If upstream provides more than one type of hash (say, sha1 and sha512), then it is best to add all those hashes in the .hash file. This is more policy-neutral. We have to keep in mind that this feature is a first-level stop-gap for security-conscious people, but in no way a security measure. Those security-conscious users are encouraged to check the downloaded files using a side-band channel (eg. manually checking signatures and so on...) Buildroot itself can't check signatures: if the user does not have a chain-of-trust, from his own key and up to the signer's key, there is no point in checking the signature in the first place. We can't expect all users to have such a chain-of-trust, even less that all have a PGP key. Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-03-05 21:47 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN 2014-03-06 10:56 ` Samuel Martin @ 2014-03-06 17:14 ` Thomas De Schampheleire 2014-03-06 17:37 ` Yann E. MORIN 1 sibling, 1 reply; 25+ messages in thread From: Thomas De Schampheleire @ 2014-03-06 17:14 UTC (permalink / raw) To: buildroot Hi Yann, On Wed, Mar 5, 2014 at 10:47 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote: > From: "Yann E. MORIN" <yann.morin.1998@free.fr> > > Although md5 is, for legacy resaons, a supported hash type, > it is not documented on purpose, since it is now known to > be weak. > > Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr> > Cc: Baruch Siach <baruch@tkos.co.il> > Cc: Arnout Vandecappelle <arnout@mind.be> > Cc: Samuel Martin <s.martin49@gmail.com> > --- > docs/manual/adding-packages-directory.txt | 65 +++++++++++++++++++++++++++++++ > support/download/check-hash | 1 + > 2 files changed, 66 insertions(+) > > diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt > index e56e59a..4609a7e 100644 > --- a/docs/manual/adding-packages-directory.txt > +++ b/docs/manual/adding-packages-directory.txt > @@ -346,3 +346,68 @@ different way, using different infrastructures: > > Further formatting details: see xref:writing-rules-mk[the writing > rules]. > + > +The +.hash+ file > +~~~~~~~~~~~~~~~~ > +[[adding-packages-hash]] > + > +Optionally, you can add a third file, named +libfoo.hash+, that contains > +the hashes of the downloaded files for the +libfoo+ package. > + > +The hashes stored in that file are used to validate the integrity of the > +downloaded files. > + > +The format for this file is one line for each file for which to check the I would say: The format _of_ this file > +hash, each line being space-separated, with these three fields: > + > +* the type of hash, one of: > +** +sha1+, +sha224+, +sha256+, +sha384+, +sha512+ > +* the hash of the file: > +** for +sha1+, 40 hexa-decimal characters > +** for +sha224+, 56 hexa-decimal characters > +** for +sha256+, 64 hexa-decimal characters > +** for +sha384+, 96 hexa-decimal characters > +** for +sha512+, 128 hexa-decimal characters hexadecimal is usually written in one word in English, for example: http://en.wikipedia.org/wiki/Hexadecimal > +* the name of the file, without any directory component > + > +Lines starting with a +#+ sign are considered comments, and ignored. Empty > +lines are ignored. > + > +There can be more than one hash for a single file, each of its own line. In I think you can say: ..., each on its own line ..., each on a line of its own but not 'each of its own line' > +this case, all hashes must match. > + > +Ideally, the hashes stored in this file should match the hashes published by > +upstream, eg, on their website, in the email announcement... If no such info 'eg' should be written with dots (e.g. , see http://en.wikipedia.org/wiki/Exempli_gratia#exempli_gratia ) e-mail (with dash) > +is available, then compute them yourself, and state so in a comment line > +above the hashes. I would write for this last sentence: If upstream does not provide hashes, then compute them yourself, and mention this in a comment line above the hashes. > + > +*Note:* the number of spaces does not matter, so one can use spaces to > +properly align the different fields. > + > +The example below defines a +sha1+ and a +sha256+ published by upstream for > +the main +libfoo-1.2.3.tar.bz2+ tarball, plus two locally-computed hashes, > +a +sha256+ for a downloaded patch, and a +sha1+ for a downloaded binary blob: > + > +---- > +# Hashes from: http://www.foosoftware.org/download/libfoo-1.2.3.tar.bz2.{sha1,sha256}: > +sha1 486fb55c3efa71148fe07895fd713ea3a5ae343a libfoo-1.2.3.tar.bz2 > +sha256 efc8103cc3bcb06bda6a781532d12701eb081ad83e8f90004b39ab81b65d4369 libfoo-1.2.3.tar.bz2 > + > +# No upstream hashes for the following: > +sha256 ff52101fb90bbfc3fe9475e425688c660f46216d7e751c4bbdb1dc85cdccacb9 libfoo-fix-blabla.patch > +sha1 2d608f3c318c6b7557d551a5a09314f03452f1a1 libfoo-data.bin > +---- > + > +If the +.hash+ file is present, and there is one or more hashes for a 'there are', but I would rather write: ..., and it contains one or more hashes for a downloaded file, ... > +downloaded file, the hash(es) of the downloaded file must match the > +hash(es) stored in the +.hash+ file. Maybe: ..., the hash(es) computed by Buildroot (after download) must match the hash(es) stored in the +.hash+ file. > If one or more hashes do not match, > +Buildroot considers this as an error, deletes the downloaded file, and considers this an error ('as' is technically allowed but not commonly added, in my experience). > +aborts. > + > +If the +.hash+ file is present, but there is no hash for a downloaded > +file, no check is done for that downloaded file. Similarly as above, I would write: "but it does not contain a hash for a downloaded file". Also I think the second occurrence of 'downloaded' is redundant: 'for that file'. > If you set the > +environment variable +BR2_ENFORCE_CHECK_HASH+ to a non-empty value, and > +there is no hash for a downloaded file, Buildroot considers this as an without 'as' > +error, deletes the downloaded file, and aborts. > + > +If the +.hash+ file is missing, then no check is done at all. > diff --git a/support/download/check-hash b/support/download/check-hash > index 1db8b8c..61678d4 100755 > --- a/support/download/check-hash > +++ b/support/download/check-hash > @@ -22,6 +22,7 @@ check_one_hash() { > _known="${2}" > _file="${3}" > > + # Note: md5 is supported, but un-documented on-purpose. undocumented (see http://en.wikipedia.org/wiki/Undocumented_feature) on purpose (see http://www.thefreedictionary.com/on+purpose) > # Note: sha3 is not supported, since there is currently no implemetation > # (the NIST has yet to publish the parameters). > case "${_h}" in > -- > 1.8.3.2 Best regards, Thomas ^ permalink raw reply [flat|nested] 25+ messages in thread
* [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes 2014-03-06 17:14 ` Thomas De Schampheleire @ 2014-03-06 17:37 ` Yann E. MORIN 0 siblings, 0 replies; 25+ messages in thread From: Yann E. MORIN @ 2014-03-06 17:37 UTC (permalink / raw) To: buildroot Thomas, All, On 2014-03-06 18:14 +0100, Thomas De Schampheleire spake thusly: > On Wed, Mar 5, 2014 at 10:47 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote: [--SNIP--] > > diff --git a/docs/manual/adding-packages-directory.txt b/docs/manual/adding-packages-directory.txt > > index e56e59a..4609a7e 100644 > > --- a/docs/manual/adding-packages-directory.txt > > +++ b/docs/manual/adding-packages-directory.txt [--SNIP--] > > +* the name of the file, without any directory component > > + > > +Lines starting with a +#+ sign are considered comments, and ignored. Empty > > +lines are ignored. > > + > > +There can be more than one hash for a single file, each of its own line. In > > I think you can say: > ..., each on its own line > ..., each on a line of its own > but not 'each of its own line' Funny how one reads what one thinks he has written, and not what was actually written. Of course, I meant 'on', not 'of'. > > +this case, all hashes must match. > > + > > +Ideally, the hashes stored in this file should match the hashes published by > > +upstream, eg, on their website, in the email announcement... If no such info > > 'eg' should be written with dots (e.g. , see > http://en.wikipedia.org/wiki/Exempli_gratia#exempli_gratia ) > e-mail (with dash) > > > +is available, then compute them yourself, and state so in a comment line > > +above the hashes. > > I would write for this last sentence: > If upstream does not provide hashes, [...] Furthermore, since there is no hash, it can be plural. I now wrote: If upstream does not provide any hash, [...] All applied. Thank you! Regards, Yann E. MORIN. -- .-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' ^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2014-07-02 21:11 UTC | newest] Thread overview: 25+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-06-01 17:40 [Buildroot] [PATCH 0/12 v6] Some download-related changes (branch yem/check-downloads) Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 01/12] Makefile: add BR2_DL_DIR to EXTRA_ENV Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 02/12] pkg-infra: also set PKGDIR for the download step Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 03/12] pkg-infra: move the git download helper to a script Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 04/12] pkg-infra: move the svn " Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 05/12] pkg-infra: move the cvs " Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 06/12] pkg-infra: move the hg " Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 07/12] pkg-infra: move the wget " Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 08/12] pkg-infra: move the bzr " Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 09/12] pkg-infra: don't use DL_DIR as scratchpad for temporary downloads Yann E. MORIN 2014-06-01 19:51 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 10/12] pkg-infra: add possiblity to check downloaded files against known hashes Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN 2014-06-01 20:08 ` Samuel Martin 2014-06-01 20:13 ` Yann E. MORIN 2014-06-01 17:40 ` [Buildroot] [PATCH 12/12] package/ca-certificates: add tarball's hashes Yann E. MORIN -- strict thread matches above, loose matches on Subject: below -- 2014-07-02 21:11 [Buildroot] [PATCH 0/12 v10] Some download-related changes (branch yem/check-downloads) Yann E. MORIN 2014-07-02 21:11 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN 2014-06-08 20:43 [Buildroot] [PATCH 0/12 v7] Some download-related changes (branch yem/check-downloads) Yann E. MORIN 2014-06-08 20:43 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN 2014-03-10 20:51 [Buildroot] [PATCH 0/12 v4] Some download-related changes (branch yem/check-downloads) Yann E. MORIN 2014-03-10 20:51 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN 2014-03-11 10:37 ` Thomas De Schampheleire 2014-03-05 21:47 [Buildroot] [PATCH 0/12 v3] some download-related changes Yann E. MORIN 2014-03-05 21:47 ` [Buildroot] [PATCH 11/12] manual: add documentation about packages' hashes Yann E. MORIN 2014-03-06 10:56 ` Samuel Martin 2014-03-06 17:09 ` Yann E. MORIN 2014-03-06 17:14 ` Thomas De Schampheleire 2014-03-06 17:37 ` Yann E. MORIN
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox