Buildroot Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Yann E. MORIN <yann.morin.1998@free.fr>
To: buildroot@busybox.net
Subject: [Buildroot] [v3 13/13] download: git: introduce cache feature
Date: Sun, 1 Apr 2018 20:13:49 +0200	[thread overview]
Message-ID: <20180401181349.GK2613@scaer> (raw)
In-Reply-To: <c64b0f87-ce14-5825-2404-d27cfd1d423c@mind.be>

Arnout, All,

On 2018-04-01 16:58 +0200, Arnout Vandecappelle spake thusly:
> On 01-04-18 14:57, Yann E. MORIN wrote:
> > On 2018-03-31 16:24 +0200, Maxime Hadjinlian spake thusly:
[--SNIP--]
> >> +# If the cache directory already exists, don't try to clone.
> >> +if [ ! -d "${git_cache}" ]; then
> >> +    # Try a shallow clone, since it is faster than a full clone - but that
> >> +    # only works if the versionis a ref (tag or branch). Before trying to do a
> >> +    # shallow clone we check if ${cset} is in the list provided by git
> >> +    # ls-remote. If not we fall back on a full clone.
> >> +    #
> >> +    # Messages for the type of clone used are provided to ease debugging in
> >> +    # case of problems
> >> +    git_done=0
> >> +    if [ -n "$(_git ls-remote "'${uri}'" "'${cset}'" 2>&1)" ]; then
> >> +        printf "Doing shallow clone\n"
> >> +        if _git clone ${verbose} "${@}" --depth 1 -b "'${cset}'" "'${uri}'" "'${git_cache}'"; then
> >> +            git_done=1
> >> +        else
> >> +            printf "Shallow clone failed, falling back to doing a full clone\n"
> >> +        fi
> >> +    fi
> >> +    if [ ${git_done} -eq 0 ]; then
> >> +        printf "Doing full clone\n"
> >> +        _git clone ${verbose} "${@}" "'${uri}'" "'${git_cache}'"
> 
>  Can't we get rid of all the code above, and instead initialize an empty repository:
> 
> _git init "'${git_cache}'"
> cd "'${git_cache}'"
> _git remote set-url origin "'${uri}'"
> 
>  then do a fetch, and make it support shallow as well:
> 
> if [ -n "$(_git ls-remote origin "'${cset}'" 2>&1)" ]; then
>   if _git fetch "${@}" --depth 1 origin "'${cset}'";
>     git_done=1
>   fi
> fi
> if [ ${git_done} -eq 0 ]; then
>  _git fetch origin -t
> fi
> 
>  Would this work?

As seen IRL, this would work with "extra tweaks left as an exercise to
the reader". ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

      reply	other threads:[~2018-04-01 18:13 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-03-31 14:23 [Buildroot] [v3 01/13] core/pkg-download: change all helpers to use common options Maxime Hadjinlian
2018-03-31 14:23 ` [Buildroot] [v3 02/13] download: put most of the infra in dl-wrapper Maxime Hadjinlian
2018-03-31 17:02   ` Maxime Hadjinlian
2018-03-31 14:23 ` [Buildroot] [v3 03/13] packages: use new $($PKG)_DL_DIR) variable Maxime Hadjinlian
2018-03-31 14:23 ` [Buildroot] [v3 04/13] arc/xtensa: store the eXtensa overlay in the per-package DL_DIR Maxime Hadjinlian
2018-03-31 14:23 ` [Buildroot] [v3 05/13] pkg-{download, generic}: use new $($(PKG)_DL_DIR) Maxime Hadjinlian
2018-04-01 18:20   ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 06/13] support/download: make sure the download folder is created Maxime Hadjinlian
2018-04-01 18:18   ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 07/13] pkg-generic: add a subdirectory to the DL_DIR Maxime Hadjinlian
2018-04-01 18:17   ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 08/13] pkg-download: support new subdir for mirrors Maxime Hadjinlian
2018-04-01 14:42   ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 09/13] pkg-generic: introduce _SAME_SOURCE_AS Maxime Hadjinlian
2018-04-01 14:26   ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 10/13] package: share downloaded files for big packages Maxime Hadjinlian
2018-04-01 14:18   ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 11/13] help/manual: update help about the new $(LIBFOO_DL_DIR) Maxime Hadjinlian
2018-04-01 14:15   ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 12/13] download: add flock call before dl-wrapper Maxime Hadjinlian
2018-04-01 14:09   ` Yann E. MORIN
2018-04-01 17:53     ` Yann E. MORIN
2018-03-31 14:24 ` [Buildroot] [v3 13/13] download: git: introduce cache feature Maxime Hadjinlian
2018-04-01 12:57   ` Yann E. MORIN
2018-04-01 14:58     ` Arnout Vandecappelle
2018-04-01 18:13       ` Yann E. MORIN [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180401181349.GK2613@scaer \
    --to=yann.morin.1998@free.fr \
    --cc=buildroot@busybox.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox