[master][PATCH 0/3] Implement support for shallow git mirror tarballs

[Christopher Larson <kergoth@gmail.com>]

From: Christopher Larson <chris_larson@mentor.com>

This adds support to the git fetcher for fetching and generating mirror
tarballs of shallow git repositories, with the specified revisions / depths.

This implements support for shallow mirror tarballs, not shallow clones.
Supporting shallow clones directly would be rather more problematic, as we'd
need to hardcode the depth between branch HEAD and the SRCREV, and that depth
would change as the branch is updated. Sadly, git's remote protocol is
extremely limited.

When BB_GIT_SHALLOW is enabled, we will always attempt to fetch a shallow
mirror tarball. If the shallow mirror tarball cannot be fetched, it will try
to fetch the full mirror tarball and use that.

Multiple variables exist to exert control over what revisions and refs are
kept at what are not.

BB_GIT_SHALLOW: enable/disable shallow support as a whole, boolean
BB_GIT_SHALLOW_DEPTH: specify commit depth for the included refs. defaults to 1 (the top commit alone). This can be set to 0 or the empty string.
BB_GIT_SHALLOW_DEPTH_<name>: specify commit depth for a specific named url/branch
BB_GIT_SHALLOW_REVS: specific revisions whose history should be removed, beyond that specified by refs+depth
BB_GIT_SHALLOW_TRIM_REFS: by default, unused refs/branches are removed. set this to 0 or the empty string to keep all refs around. this is required for linux-yocto kernel repositories, otherwise the branch checking done by the kernel scripts will fail
BB_GIT_SHALLOW_EXTRA_REFS: additional refs to keep beyond those referenced in SRC_URI, when TRIM_REFS is enabled
BB_GENERATE_SHALLOW_TARBALLS: explicitly control creation of shallow mirror tarballs. this defaults to enabled when BB_GENERATE_MIRROR_TARBALLS is enabled

Example usage:

    BB_GIT_SHALLOW ?= "1"
    BB_GIT_SHALLOW_DEPTH ?= "1"

    # Usage in a recipe with multiple named uris or multiple named branches
    BB_GIT_SHALLOW_DEPTH_doc = ""

    # Remove the upstream history from our kernel repo, keeping our own
    BB_GIT_SHALLOW_REVS_pn-linux-mel_mx6 = "v3.14"
    BB_GIT_SHALLOW_DEPTH_pn-linux-mel_mx6 = ""

    # Keep the branches in linux-yocto repositories
    BB_GIT_SHALLOW_TRIM_REFS_pn-linux-yocto = "0"


Please review the following changes for suitability for inclusion. If you have
any objections or suggestions for improvement, please respond to the patches. If
you agree with the changes, please provide your Acked-by.

The following changes since commit 309f5907a3661821e041ed14645b5d165007b058:

  bitbake: Switch to post release version (2016-04-29 07:41:34 +0100)

are available in the git repository at:

  https://github.com/kergoth/bitbake shallow-simplify
  https://github.com/kergoth/bitbake/tree/shallow-simplify

Christopher Larson (3):
  bb.fetch: simplify mirror tarball handling in try_mirror_url
  bb.fetch: support ud.mirrortarballs
  bb.fetch.git: add support for shallow mirror tarballs

 lib/bb/fetch2/__init__.py |  27 ++---
 lib/bb/fetch2/git.py      | 260 ++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 255 insertions(+), 32 deletions(-)

-- 
2.8.0