All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Paul Gortmaker" <paul.gortmaker@windriver.com>
To: Bruce Ashfield <bruce.ashfield@gmail.com>,
	Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: linux-yocto@lists.yoctoproject.org, bitbake-devel@lists.openembedded.org
Subject: [PATCH 02/21] bitbake: fetch2/git: allow limiting upstream fetch refs to a subset
Date: Fri,  2 Apr 2021 13:15:38 -0400	[thread overview]
Message-ID: <20210402171557.981599-3-paul.gortmaker@windriver.com> (raw)
In-Reply-To: <20210402171557.981599-1-paul.gortmaker@windriver.com>

If a clone in the download directory is not static, and was created with
single-branch to avoid additonal unwanted content, then the fetcher will
come along and spoil that effort by unconditonally getting refs/* from
the server and downloading everything available at the server.

To that end, allow a companion variable that can optionally be used to
limit the fetch to a subset of references that are in line with the
desired content for the recipe in question, and so that large unwanted
downloads aren't inadvertently triggered during an update.

If not specified, the existing "fetch everything" behaviour remains.

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
---
 bitbake/lib/bb/fetch2/git.py           | 3 ++-
 documentation/ref-manual/variables.rst | 8 ++++++++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index 22281e2cfb98..b54ec76d7174 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -362,7 +362,8 @@ class Git(FetchMethod):
               runfetchcmd("%s remote rm origin" % ud.basecmd, d, workdir=ud.clonedir)
 
             runfetchcmd("%s remote add --mirror=fetch origin %s" % (ud.basecmd, shlex.quote(repourl)), d, workdir=ud.clonedir)
-            fetch_cmd = "LANG=C %s fetch -f --progress %s refs/*:refs/*" % (ud.basecmd, shlex.quote(repourl))
+            fetchrefs = d.getVar('GITFETCHREFS_' + ud.names[0]) or d.getVar('GITFETCHREFS') or "refs/*:refs/*"
+            fetch_cmd = "LANG=C %s fetch -f --progress %s %s" % (ud.basecmd, shlex.quote(repourl), fetchrefs)
             if ud.proto.lower() != 'file':
                 bb.fetch2.check_network_access(d, fetch_cmd, ud.url)
             progresshandler = GitProgressHandler(d)
diff --git a/documentation/ref-manual/variables.rst b/documentation/ref-manual/variables.rst
index cd2b77604e87..55028987c08a 100644
--- a/documentation/ref-manual/variables.rst
+++ b/documentation/ref-manual/variables.rst
@@ -2661,6 +2661,14 @@ system and gives an overview of their function and contents.
       The directory in which a local copy of a Git repository is stored
       when it is cloned.
 
+   :term:`GITFETCHREFS`
+      The refs requested during a "git fetch" when updating a repo from
+      an external server into the local download area.  The default is
+      "refs/*:refs/*".  Used in conjunction with GITCLONEARGS to limit what
+      is cloned/fetched from a repo otherwise dense with branches.  Note
+      that an URL specific GITFETCHREFS_<name> will take precedence over
+      a recipe wide GITFETCHREFS setting, if both are present.
+
    :term:`GLIBC_GENERATE_LOCALES`
       Specifies the list of GLIBC locales to generate should you not wish
       to generate all LIBC locals, which can be time consuming.
-- 
2.25.1


  parent reply	other threads:[~2021-04-02 17:16 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-02 17:15 [PATCH RFC 00/21] Git repository sharing for kernel (and other) repos Paul Gortmaker
2021-04-02 17:15 ` [PATCH 01/21] bitbake: fetch2/git: allow override of clone args with GITCLONEARGS Paul Gortmaker
2021-04-02 17:15 ` Paul Gortmaker [this message]
2021-04-03  7:43   ` [PATCH 02/21] bitbake: fetch2/git: allow limiting upstream fetch refs to a subset Richard Purdie
2021-04-02 17:15 ` [PATCH 03/21] bitbake: fetch2/git: allow optional git download name overrride Paul Gortmaker
2021-04-02 17:15 ` [PATCH 04/21] bitbake: fetch2/git: allow specifying repos as static/unchanging Paul Gortmaker
2021-04-02 17:15 ` [PATCH 05/21] bitbake: fetch2/git: ensure static repos have at least one refs/heads Paul Gortmaker
2021-04-02 17:15 ` [PATCH 06/21] bitbake: fetch2/git: allow alt references within download dir Paul Gortmaker
2021-04-02 17:15 ` [PATCH 07/21] bitbake: fetch2/git: append new altref line if/when SRC_URI changed value Paul Gortmaker
2021-04-02 17:15 ` [PATCH 08/21] bitbake: fetch2/git: allow pack references within download dir Paul Gortmaker
2021-04-02 17:15 ` [PATCH 09/21] bitbake: fetch2/git: use constant names for packs in static repos Paul Gortmaker
2021-04-02 17:15 ` [PATCH 10/21] kernel: add basic boilerplate for fetch-only recipes Paul Gortmaker
2021-04-02 17:15 ` [PATCH 11/21] kernel: add a fetch-only recipe for mainline v5.10 source Paul Gortmaker
2021-04-02 20:13   ` Bruce Ashfield
2021-04-02 17:15 ` [PATCH 12/21] kernel: allow splitting mainline v5.10 source download in two Paul Gortmaker
2021-04-02 17:15 ` [PATCH 13/21] kernel: allow splitting mainline v5.10 source download in three Paul Gortmaker
2021-04-02 17:15 ` [PATCH 14/21] kernel: allow splitting mainline v5.10 source download in four Paul Gortmaker
2021-04-02 17:15 ` [PATCH 15/21] kernel: add recipe for linux-master (mainline latest) Paul Gortmaker
2021-04-02 20:16   ` Bruce Ashfield
2021-04-02 17:15 ` [PATCH 16/21] kernel: add stable fetch recipes for v5.4.x, v5.10.x and v5.12.x Paul Gortmaker
2021-04-02 17:15 ` [PATCH 17/21] kernel: add preempt-rt fetch recipes for v5.4.x, v5.10.x and 5.12.x Paul Gortmaker
2021-04-02 17:15 ` [PATCH 18/21] kernel: make v5.4.x Yocto recipes use shared source Paul Gortmaker
2021-04-02 17:15 ` [PATCH 19/21] kernel: make v5.10.x " Paul Gortmaker
2021-04-02 17:15 ` [PATCH 20/21] kernel: make linux-yocto-dev recipe " Paul Gortmaker
2021-04-02 17:15 ` [PATCH 21/21] kernel: disable (pre)mirror for linux-yocto and linux-yocto-dev Paul Gortmaker
2021-04-02 20:19   ` Bruce Ashfield
2021-04-02 22:14 ` [PATCH RFC 00/21] Git repository sharing for kernel (and other) repos Richard Purdie
2021-04-03  1:44   ` Paul Gortmaker
2021-04-03  8:33     ` Richard Purdie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210402171557.981599-3-paul.gortmaker@windriver.com \
    --to=paul.gortmaker@windriver.com \
    --cc=bitbake-devel@lists.openembedded.org \
    --cc=bruce.ashfield@gmail.com \
    --cc=linux-yocto@lists.yoctoproject.org \
    --cc=richard.purdie@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.