[PATCH 04/21] bitbake: fetch2/git: allow specifying repos as static/unchanging

["Paul Gortmaker" <paul.gortmaker@windriver.com>]

If we know a repository content has what we need, we can flag it as
static and then optimize accordingly for the future, by not trying to do
any fetch operations against it.  In a way, this can be thought of as
the special case of GITFETCHREFS = None.

There are multiple advantages to having this.  We can skip trying any
updates and speed things up for a project we know is EOL.  We can avoid
doing "ls-remote" operations against servers, easing use for
mirror/no-network users.  We can open the door to accepting tags in
SRC_URI more often as absolute references, instead of the less humanly
readable SHA.  Archives that change upstream in incompatible ways, or
have simply vanished, can exist as a snapshot in time after initial
creation via mirror tarball or manually.

Finally, we use the flexibility of git-config to store that static
information in the cloned repo itself, so we can tell at any time,
independent of the SRC_URI, that the repo has static content.

This will be used for reference purposes, but might also be used in the
download dir itself outside of BB recipes -- for example a script could
easily spot any static content and hence prioritize it as a possible
mirroring candidate.

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
---
 .../bitbake-user-manual-fetching.rst          |  5 ++++
 bitbake/lib/bb/fetch2/git.py                  | 27 +++++++++++++++++++
 2 files changed, 32 insertions(+)

diff --git a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
index 0f12eaa3a81f..94f4cf3363f6 100644
--- a/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
+++ b/bitbake/doc/bitbake-user-manual/bitbake-user-manual-fetching.rst
@@ -426,6 +426,11 @@ This fetcher supports the following parameters:
 -  *"rev":* The revision to use for the checkout. The default is
    "master".
 
+-  *"static":* The repository is a "one and done" - clone/untar and the
+   content is static and unchanging.  This allows optimizations like
+   skipping fetch/pull and doing "ls-remote" on servers.  Typically used
+   for repositories that will act as a commit library for other repos.
+
 -  *"tag":* Specifies a tag to use for the checkout. To correctly
    resolve tags, BitBake must access the network. For that reason, tags
    are often not used. As far as Git is concerned, the "tag" parameter
diff --git a/bitbake/lib/bb/fetch2/git.py b/bitbake/lib/bb/fetch2/git.py
index de698a3d9f24..7cba990c3be7 100644
--- a/bitbake/lib/bb/fetch2/git.py
+++ b/bitbake/lib/bb/fetch2/git.py
@@ -54,6 +54,11 @@ Supported SRC_URI options are:
    referring to commit which is valid in tag instead of branch.
    The default is "0", set nobranch=1 if needed.
 
+- static
+   Repository is of fixed/static content with no active/new development.
+   Once cloned, nothing will change, so optimize accordingly.  The default
+   is "0", set static=1 when appropriate.
+
 - usehead
    For local git:// urls to use the current branch HEAD as the revision for use with
    AUTOREV. Implies nobranch.
@@ -158,6 +163,8 @@ class Git(FetchMethod):
 
         ud.nobranch = ud.parm.get("nobranch","0") == "1"
 
+        ud.static = ud.parm.get("static","0") == "1"
+
         ud.dlname = ud.parm.get("dlname","")
 
         # usehead implies nobranch
@@ -167,6 +174,10 @@ class Git(FetchMethod):
                  raise bb.fetch2.ParameterError("The usehead option is only for use with local ('protocol=file') git repositories", ud.url)
             ud.nobranch = 1
 
+        if ud.static:
+            if ud.rebaseable:
+                 raise bb.fetch2.ParameterError("The rebaseable option is incompatible with the static option", ud.url)
+
         # bareclone implies nocheckout
         ud.bareclone = ud.parm.get("bareclone","0") == "1"
         if ud.bareclone:
@@ -311,6 +322,8 @@ class Git(FetchMethod):
     def clonedir_need_update(self, ud, d):
         if not os.path.exists(ud.clonedir):
             return True
+        if ud.static:
+            return False
         if ud.shallow and ud.write_shallow_tarballs and self.clonedir_need_shallow_revs(ud, d):
             return True
         for name in ud.names:
@@ -341,6 +354,17 @@ class Git(FetchMethod):
             return False
         return True
 
+    def get_git_config(self, ud, d, repo, cfgvar):
+        try:
+            output = runfetchcmd("%s config --get %s" % (ud.basecmd, cfgvar), d, workdir=repo)
+        except (bb.fetch2.FetchError,ValueError):
+            return ""
+        return output.split()[0]
+
+    def repo_is_static(self, ud, d, repo):
+        static = self.get_git_config(ud, d, repo, "bitbake.static")
+        return (static == "true")
+
     def download(self, ud, d):
         """Fetch url"""
 
@@ -368,6 +392,9 @@ class Git(FetchMethod):
             progresshandler = GitProgressHandler(d)
             runfetchcmd(clone_cmd, d, log=progresshandler)
 
+            if ud.static:
+                runfetchcmd("%s config --bool --add bitbake.static 1" % ud.basecmd, d, workdir=ud.clonedir)
+
         # Update the checkout if needed
         if self.clonedir_need_update(ud, d):
             output = runfetchcmd("%s remote" % ud.basecmd, d, quiet=True, workdir=ud.clonedir)
-- 
2.25.1