Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: "Jérôme Pouiller" <jezz@sysmic.org>
To: buildroot@busybox.net
Subject: [Buildroot] [PATCH 2/5] download/git: allow to create archives containing shallowed git repos
Date: Thu, 19 Nov 2015 11:36:03 +0100	[thread overview]
Message-ID: <1447929366-8972-3-git-send-email-jezz@sysmic.org> (raw)
In-Reply-To: <1447929366-8972-1-git-send-email-jezz@sysmic.org>

We introduce a new tarball format for projects managed with git. This
new format is a shallowed version of upstream repository.

This format is a little larger (20 to 50% in my tests) than standard
one and contains information about upstream git repository.

That simplify workflow with upstream. It is now possible to reconnect
a build directory with upstream repository using:

   git fetch --unshallow

If you want to also get remote tags and branch (as in a standard
clone), run:

   git config remote.origin.fetch refs/heads/*:refs/remotes/origin/*
   git fetch

For large repository, you can use a local repository as reference in
order to speedup unshallow process (read warnings about '--reference'
option in git-clone manual page):

   echo "repo_path/.git/objects" > .git/objects/info/alternates
   git fetch --unshallow

Note 1: some repository does not support unshallowing (especially
anonscm.debian.org)

Note 2: I have tested this patch with git 2.1 (from Debian Jessie).

Signed-off-by: J?r?me Pouiller <jezz@sysmic.org>
---
 support/download/git | 30 +++++++++++++++++++++++++++---
 1 file changed, 27 insertions(+), 3 deletions(-)

diff --git a/support/download/git b/support/download/git
index 0e6103b..6755f65 100755
--- a/support/download/git
+++ b/support/download/git
@@ -46,7 +46,31 @@ if [ ${git_done} -eq 0 ]; then
     ${GIT} clone ${verbose} --bare "${repo}" "${basename}"
 fi
 
-GIT_DIR="${basename}" \
-${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
+if [[ ${output} != *.git.tar.gz* ]]; then
+    GIT_DIR="${basename}" \
+    ${GIT} archive --prefix="${basename}/" -o "${output}.tmp" --format=tar "${cset}"
 
-gzip <"${output}.tmp" >"${output}"
+    gzip <"${output}.tmp" >"${output}"
+fi
+if [[ ${output} = *.git.tar.gz* ]]; then
+    GIT_DIR="${basename}" ${GIT} reset --soft ${cset}
+
+    # Git cannot shallow a repository on an arbitrary commit-id. It must be a
+    # symbolic reference of a remote repository (HEAD, tag or branch) (problem
+    # seems to be resolved in git 2.5).
+    # However, we want to be able to shallow on any commit-id.
+    # We didn't find any other way than use an intermediate repository in order
+    # to work around this limitation.
+    # (In add, this method make easier pruning of packed objects)
+    #
+    # Note 1: --depth is ignored in local clones. Git suggest to use file://
+    # in order to force copy of objects. We have to use an absolute path with
+    # these URIs
+    #
+    # Note 2: ${basename}.git will be removed by dl-wrapper (since it is in
+    # same directory than ${basename})
+    ${GIT} clone "file://$(readlink -f ${basename})" "${basename}.git" --bare --depth 1
+    GIT_DIR="${basename}.git" ${GIT} remote set-url origin "${repo}"
+
+    tar czf "${output}" "${basename}.git"
+fi
-- 
2.1.4

  parent reply	other threads:[~2015-11-19 10:36 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-19 10:36 [Buildroot] [PATCH 0/5] Introduce alternative archive format Jérôme Pouiller
2015-11-19 10:36 ` [Buildroot] [PATCH 1/5] pkg-download: do not test SITE_METHOD Jérôme Pouiller
2015-11-29 17:57   ` Yann E. MORIN
2015-12-18  9:08   ` Thomas Petazzoni
2015-11-19 10:36 ` Jérôme Pouiller [this message]
2015-11-19 10:36 ` [Buildroot] [PATCH 3/5] pkg-generic: allow to populate build directory from a git archive Jérôme Pouiller
2015-11-19 10:36 ` [Buildroot] [PATCH 4/5] pkg-generic: provide an option to use git archives Jérôme Pouiller
2015-11-19 10:36 ` [Buildroot] [PATCH 5/5] pkg-generic: tag sources if git is used Jérôme Pouiller
2015-11-19 12:02 ` [Buildroot] [PATCH 0/5] Introduce alternative archive format Thomas Petazzoni
2015-11-23  9:54   ` Jérôme Pouiller
2015-11-29 18:02   ` Yann E. MORIN
2015-11-30 12:32     ` Jérôme Pouiller
2015-11-29 21:05 ` Arnout Vandecappelle
2015-12-29 21:36 ` Yann E. MORIN

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1447929366-8972-3-git-send-email-jezz@sysmic.org \
    --to=jezz@sysmic.org \
    --cc=buildroot@busybox.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox