[BUG] git clone from bundle with --all does not fetch all refs

[BUG] git clone from bundle with --all does not fetch all refs

[Andrew Harmon]

# Problem with git bundle --all and git clone for air-gapped transfer to offline environments

## Description

When creating a bundle using `git bundle create --all`, all refs including `refs/remotes/origin/*` are included in the bundle. However, when cloning from this bundle using `git clone`, these remote refs are not automatically fetched, making many branches inaccessible.

## Steps to Reproduce

1. In a repository with multiple branches and remote tracking branches (e.g., after cloning from GitLab/GitHub)
2. Create a bundle: `git bundle create repo.bundle --all`
3. Verify bundle contents: `git bundle list-heads repo.bundle` (shows both `refs/heads/*` and `refs/remotes/origin/*`)
4. Clone from bundle: `git clone repo.bundle cloned-repo`
5. Check available branches: `cd cloned-repo && git branch -a`

## Expected Behavior

All refs included in the bundle (both `refs/heads/*` and `refs/remotes/origin/*`) should be accessible after cloning. Users should be able to see and checkout all branches that were in the original repository.

## Actual Behavior

Only refs under `refs/heads/*` in the bundle become remote tracking branches. Refs stored as `refs/remotes/origin/*` in the bundle are not fetched during clone, making these branches inaccessible without manual intervention.

## Workaround

After cloning, manually fetch the remote refs:

```bash
git fetch origin 'refs/remotes/origin/*:refs/remotes/origin/*'
```

## Impact

This breaks the expected workflow for distributing complete repository snapshots via bundles (e.g., for offline environments). Users expect `git bundle --all` followed by `git clone` to preserve all branches.

The `--all` flag documentation states it includes "all refs", but the cloning behavior does not match this expectation. This creates a surprising and unintuitive user experience when bundles are used for offline repository distribution.

## Environment

[System Info]
git version:
git version 2.45.2.windows.1
cpu: x86_64
built from commit: 91d03cb2e4fbf6ad961ace739b8a646868cb154d
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
uname: Windows 10.0 26100 
compiler info: gnuc: 14.1
libc info: no libc information available
$SHELL (typically, interactive shell): C:\Program Files\Git\usr\bin\bash.exe

## Suggested Fix

One of the following approaches could address this issue:

1. **Automatic fetch during clone**: `git clone` should automatically fetch all refs present in a bundle, including `refs/remotes/origin/*`, OR

2. **Bundle creation remapping**: `git bundle create --all` should convert `refs/remotes/origin/*` to `refs/heads/*` so they're properly restored during clone, OR

3. **Documentation improvement**: Document this behavior clearly in `git-bundle` and `git-clone` documentation with the workaround, including a note that `--all` does not guarantee all refs will be available after cloning without additional steps.

## Additional Context

This issue was discovered while preparing repository snapshots for developers working in offline/air-gapped environments. The workflow of `git bundle create --all` >> transfer >> `git clone` appears to be a complete solution but silently loses access to most branches.


*********************************
Andrew Harmon
aharmon@signalquest.com

SignalQuest - precision microsensors
10 Water Street, Lebanon, NH, 03766
(603)-448-6266
*********************************

Re: [BUG] git clone from bundle with --all does not fetch all refs

[Junio C Hamano]

Andrew Harmon <aharmon@signalquest.com> writes:

> # Problem with git bundle --all and git clone for air-gapped
> transfer to offline environments
>
> ## Description
>
> When creating a bundle using `git bundle create --all`, all refs
> including `refs/remotes/origin/*` are included in the
> bundle. However, when cloning from this bundle using `git clone`,
> these remote refs are not automatically fetched, making many
> branches inaccessible.
>
> ## Steps to Reproduce
>
> 1. In a repository with multiple branches and remote tracking branches (e.g., after cloning from GitLab/GitHub)
> 2. Create a bundle: `git bundle create repo.bundle --all`
> 3. Verify bundle contents: `git bundle list-heads repo.bundle` (shows both `refs/heads/*` and `refs/remotes/origin/*`)
> 4. Clone from bundle: `git clone repo.bundle cloned-repo`
> 5. Check available branches: `cd cloned-repo && git branch -a`

> ## Expected Behavior
>
> All refs included in the bundle (both `refs/heads/*` and `refs/remotes/origin/*`) should be accessible after cloning. Users should be able to see and checkout all branches that were in the original repository.

If I am not misreading the scenario presented, then this expectation
is wrong.

> ## Actual Behavior
>
> Only refs under `refs/heads/*` in the bundle become remote tracking branches. Refs stored as `refs/remotes/origin/*` in the bundle are not fetched during clone, making these branches inaccessible without manual intervention.

This is totally expected.  Think of cloning from a bundle is just
like cloning from the original remote (limited to the refs included
in the bundle, of course).  Local branches of the remote (i.e. the
ones that corresponds to refs/heads/* you saw in your bundle) become
your remote-tracking branches.  Their remote-tracking branches are
not even visible, unless you explicitly ask "clone" to.  Which means ...

> ## Workaround
>
> After cloning, manually fetch the remote refs:
>
> ```bash
> git fetch origin 'refs/remotes/origin/*:refs/remotes/origin/*'
> ```

... this is not even a workaround, but how you would ask for their
remote-tracking branches.

Or 

    $ git init && git fetch repo.bndl "refs/*:refs/*"

which is like doing a mirror clone ("git clone --mirror").

RE: [BUG] git clone from bundle with --all does not fetch all refs

[Andrew Harmon]

Hi Junio,

Thanks for the reply. I'm a bit out of my depth but tried to write a good bug report.

The experience of "cloning from bundle should be just like cloning from github" did not happen for me. Maybe I created the bundle wrong?

See the workflow, below. Is the problem that refs are put in the bundle at refs/remotes/origin/* instead of refs/heads/*?

user@machine MINGW64 ~
$ mkdir tmp

user@machine MINGW64 ~
$ cd tmp

user@machine MINGW64 ~/tmp
$ git clone git@GITLAB_HOST:external_sources/matrice_sq.git
Cloning into 'matrice_sq'...
remote: Enumerating objects: 1512, done.
remote: Counting objects: 100% (139/139), done.
remote: Compressing objects: 100% (137/137), done.
remote: Total 1512 (delta 79), reused 0 (delta 0), pack-reused 1373 (from 1)
Receiving objects: 100% (1512/1512), 537.88 KiB | 4.68 MiB/s, done.
Resolving deltas: 100% (1070/1070), done.

user@machine MINGW64 ~/tmp
$ cd matrice_sq

user@machine MINGW64 ~/tmp/matrice_sq (develop)
$ git branch -r
  origin/21-Add-benchmark-application
  origin/51-inlining-config-matrix-base
  origin/Add_readme
  origin/DspCortexM4
  origin/HEAD -> origin/develop
  origin/develop
  origin/master
  origin/wip-develop-no-inline
  origin/wip-expression-templates
  origin/wip-inlining-config
  origin/wip-kf-imu-gm-states
  origin/wip-nested-submatrix-tests
  origin/wip-sqinav-initial-version
  origin/wip/matrix-base-public-operators
  origin/wip_ins_resource_investigation

user@machine MINGW64 ~/tmp/matrice_sq (develop)
$ git bundle create ../matrice_sq.bundle --all
Enumerating objects: 1512, done.
Counting objects: 100% (1512/1512), done.
Delta compression using up to 16 threads
Compressing objects: 100% (435/435), done.
Writing objects: 100% (1512/1512), 537.83 KiB | 76.83 MiB/s, done.
Total 1512 (delta 1070), reused 1512 (delta 1070), pack-reused 0 (from 0)

user@machine MINGW64 ~/tmp/matrice_sq (develop)
$ git bundle verify ../matrice_sq.bundle
The bundle contains these 18 refs:
1b32e892e571c11c764e8d79e7970afe89378327 refs/heads/develop
06881c8a501f166b41d71c8a7a025daa5bf768ea refs/remotes/origin/21-Add-benchmark-application
b09e56be3d6ae35774544590c1dc8ae88b4d87f9 refs/remotes/origin/51-inlining-config-matrix-base
093d3bff0316e7281ba0b094f7d79e3d16793b15 refs/remotes/origin/Add_readme
241e5d6f6116f8bfae399c766c92d9b2013e61ab refs/remotes/origin/DspCortexM4
1b32e892e571c11c764e8d79e7970afe89378327 refs/remotes/origin/HEAD
1b32e892e571c11c764e8d79e7970afe89378327 refs/remotes/origin/develop
0c6d1fce26af62c104a0c0297b693d9c0f164bdc refs/remotes/origin/master
e9afacb6d77e888b29e9ece12863fb13bc8df3ad refs/remotes/origin/wip-develop-no-inline
47004f08fc09bf97f9a12e81726edc05f7390409 refs/remotes/origin/wip-expression-templates
8047b97041ce82d2e6b4e193332ddb1d8337fd84 refs/remotes/origin/wip-inlining-config
ed8238c5ab80ec3039a29c65dcec574fca6e7017 refs/remotes/origin/wip-kf-imu-gm-states
4c20baac85dfce2077f9e997f2c5ea4f25392b46 refs/remotes/origin/wip-nested-submatrix-tests
e909df72e6e52ece8b7fed9362578e1d4a89fb45 refs/remotes/origin/wip-sqinav-initial-version
9a8ab3d9bcb4a2c2384d0c545634d45e0daf60ed refs/remotes/origin/wip/matrix-base-public-operators
8720c64e04aeeef02f54a611665f9b0777e8572c refs/remotes/origin/wip_ins_resource_investigation
0c6d1fce26af62c104a0c0297b693d9c0f164bdc refs/tags/Matrice-SQ-v1.0.0
1b32e892e571c11c764e8d79e7970afe89378327 HEAD
The bundle records a complete history.
The bundle uses this hash algorithm: sha1
../matrice_sq.bundle is okay

user@machine MINGW64 ~/tmp/matrice_sq (develop)
$ cd ..

user@machine MINGW64 ~/tmp
$ git clone matrice_sq.bundle matrice_copy
Cloning into 'matrice_copy'...
Receiving objects: 100% (1512/1512), 537.83 KiB | 11.69 MiB/s, done.
Resolving deltas: 100% (1070/1070), done.

user@machine MINGW64 ~/tmp
$ cd matrice_copy

user@machine MINGW64 ~/tmp/matrice_copy (develop)
$ git branch -r
  origin/HEAD -> origin/develop
  origin/develop

>>>>> At this point, I would expect to see everything from origin that I normally get when cloning from gitlab, but I don't.



-----Original Message-----
From: Junio C Hamano <gitster@pobox.com> 
Sent: Tuesday, October 7, 2025 18:15
To: Andrew Harmon <aharmon@signalquest.com>
Cc: git@vger.kernel.org
Subject: Re: [BUG] git clone from bundle with --all does not fetch all refs

Andrew Harmon <aharmon@signalquest.com> writes:

> # Problem with git bundle --all and git clone for air-gapped transfer 
> to offline environments
>
> ## Description
>
> When creating a bundle using `git bundle create --all`, all refs 
> including `refs/remotes/origin/*` are included in the bundle. However, 
> when cloning from this bundle using `git clone`, these remote refs are 
> not automatically fetched, making many branches inaccessible.
>
> ## Steps to Reproduce
>
> 1. In a repository with multiple branches and remote tracking branches 
> (e.g., after cloning from GitLab/GitHub) 2. Create a bundle: `git 
> bundle create repo.bundle --all` 3. Verify bundle contents: `git 
> bundle list-heads repo.bundle` (shows both `refs/heads/*` and 
> `refs/remotes/origin/*`) 4. Clone from bundle: `git clone repo.bundle 
> cloned-repo` 5. Check available branches: `cd cloned-repo && git 
> branch -a`

> ## Expected Behavior
>
> All refs included in the bundle (both `refs/heads/*` and `refs/remotes/origin/*`) should be accessible after cloning. Users should be able to see and checkout all branches that were in the original repository.

If I am not misreading the scenario presented, then this expectation is wrong.

> ## Actual Behavior
>
> Only refs under `refs/heads/*` in the bundle become remote tracking branches. Refs stored as `refs/remotes/origin/*` in the bundle are not fetched during clone, making these branches inaccessible without manual intervention.

This is totally expected.  Think of cloning from a bundle is just like cloning from the original remote (limited to the refs included in the bundle, of course).  Local branches of the remote (i.e. the ones that corresponds to refs/heads/* you saw in your bundle) become your remote-tracking branches.  Their remote-tracking branches are not even visible, unless you explicitly ask "clone" to.  Which means ...

> ## Workaround
>
> After cloning, manually fetch the remote refs:
>
> ```bash
> git fetch origin 'refs/remotes/origin/*:refs/remotes/origin/*'
> ```

... this is not even a workaround, but how you would ask for their remote-tracking branches.

Or

    $ git init && git fetch repo.bndl "refs/*:refs/*"

which is like doing a mirror clone ("git clone --mirror").

Re: [BUG] git clone from bundle with --all does not fetch all refs

[Junio C Hamano]

Andrew Harmon <aharmon@signalquest.com> writes:

> The experience of "cloning from bundle should be just like cloning
> from github" did not happen for me. Maybe I created the bundle
> wrong?

I did not say "github", though ;-)

> See the workflow, below. Is the problem that refs are put in the
> bundle at refs/remotes/origin/* instead of refs/heads/*?

Everything looks as expected, including how you prepared a bundle
file.  Perhaps your expectation of what a bundle file is for is
different from what bundle files are designed for?  By that, I mean
that you may have a use case the designers of "git bundle" feature
never anticipated.

The primary and only use case "git bundle" was designed to cater to
was this.  The user has a repository on this machine that they want
to be cloned to another machine, but for whatever reason, it cannot
be done over the network by typing "git clone ..." on that other
machine against this machine.  So the user makes a bundle out of the
repository on this machine, copy that file on a USB stick, bring it
over there, and then the user says "git clone ..." against the
bundle file as if they are cloning from the original.

For that to work, refs/heads/master in the original repository is
stored as refs/heads/master in the bundle.  The remote-tracking
branches may by default not copied into the bundle, but you can by
instructing "git bundle" command.

And if you want to copy the remote-tracking branches via "git clone"
or "git fetch" over the network, you'd specifically ask for them,
as "clone" would by default prepare fetch refspecs for their local 
branches to be copied to your remote-tracking branches, and their
refs/tags/ copied to your refs/tags/. and nothing else.  As a bundle
file wants to imitate end-user experience of cloning or fetching
over the network from the original repository for sneaker-net
operation, the need for specifically asking is the same if you want
to grab (their) remote-tracking branches out of a bundle file.

Stepping back a bit, how would you make a more-or-less exact copy
of an existing repository over the network?  "git clone --mirror"
is probably the mechanism where their refs/heads/master becomes the
refs/heads/master in the resulting repository and the remote-tracking
branches they have in their refs/remotes/origin/* would become the
remote-tracking branches refs/remotes/origin/* in the resulting
repository.  So perhaps doing that against the bundle file would do
what you wanted to do?  If that is the case, then perhaps your use
case was covered by the original design of the "git bundle" feature
after all---to allow you to clone or fetch from the file as if you
are cloning or fetching from the original repository.

HTH.

RE: [BUG] git clone from bundle with --all does not fetch all refs

[Andrew Harmon]

> The primary and only use case "git bundle" was designed to cater to was this.
> The user has a repository on this machine that they want to be cloned to
> another machine, but for whatever reason, it cannot be done over the network
> by typing "git clone ..." on that other machine against this machine.  So the
> user makes a bundle out of the repository on this machine, copy that file on a
> USB stick, bring it over there, and then the user says "git clone ..." against
> the bundle file as if they are cloning from the original.

This is exactly my use case. I found that I also needed to do:
git fetch origin 'refs/remotes/origin/*:refs/remotes/origin/*'

As an end user, I found this very surprising. I was expecting to call:
git clone <repo-bundle>
and have this behave just like:
git clone <ssh-or-https-target>

Maybe we are talking past each other? Following the documentation, I didn't think that anything beyond "git clone <repo-bundle>" was required to unpack the bundle.


FILE: bundle-demo.sh

#!/bin/bash

set -e

# cleanup from last run
rm -rf matrice_sq*

echo ""
echo "Clone the repo via SSH"
git clone git@UBUBEAR:external_sources/matrice_sq.git matrice_sq

echo ""
echo "View available remote branches"
(cd matrice_sq && git branch -r)

echo ""
echo "Pack the bundle for offline distribution"
(cd matrice_sq && git bundle create ../matrice_sq.bundle --all)
(cd matrice_sq && git bundle verify ../matrice_sq.bundle)

echo ""
echo "Unpack the bundle on new machine"
git clone matrice_sq.bundle matrice_sq.new
(cd matrice_sq.new && git branch -r)

echo ""
echo "Manually fetch refs from refs/remotes/origin/*"
(cd matrice_sq.new && git fetch origin 'refs/remotes/origin/*:refs/remotes/origin/*')
(cd matrice_sq.new && git branch -r)



-----Original Message-----
From: Junio C Hamano <gitster@pobox.com> 
Sent: Wednesday, October 8, 2025 13:24
To: Andrew Harmon <aharmon@signalquest.com>
Cc: git@vger.kernel.org
Subject: Re: [BUG] git clone from bundle with --all does not fetch all refs

Andrew Harmon <aharmon@signalquest.com> writes:

> The experience of "cloning from bundle should be just like cloning 
> from github" did not happen for me. Maybe I created the bundle wrong?

I did not say "github", though ;-)

> See the workflow, below. Is the problem that refs are put in the 
> bundle at refs/remotes/origin/* instead of refs/heads/*?

Everything looks as expected, including how you prepared a bundle file.  Perhaps your expectation of what a bundle file is for is different from what bundle files are designed for?  By that, I mean that you may have a use case the designers of "git bundle" feature never anticipated.

The primary and only use case "git bundle" was designed to cater to was this.  The user has a repository on this machine that they want to be cloned to another machine, but for whatever reason, it cannot be done over the network by typing "git clone ..." on that other machine against this machine.  So the user makes a bundle out of the repository on this machine, copy that file on a USB stick, bring it over there, and then the user says "git clone ..." against the bundle file as if they are cloning from the original.

For that to work, refs/heads/master in the original repository is stored as refs/heads/master in the bundle.  The remote-tracking branches may by default not copied into the bundle, but you can by instructing "git bundle" command.

And if you want to copy the remote-tracking branches via "git clone"
or "git fetch" over the network, you'd specifically ask for them, as "clone" would by default prepare fetch refspecs for their local branches to be copied to your remote-tracking branches, and their refs/tags/ copied to your refs/tags/. and nothing else.  As a bundle file wants to imitate end-user experience of cloning or fetching over the network from the original repository for sneaker-net operation, the need for specifically asking is the same if you want to grab (their) remote-tracking branches out of a bundle file.

Stepping back a bit, how would you make a more-or-less exact copy of an existing repository over the network?  "git clone --mirror"
is probably the mechanism where their refs/heads/master becomes the refs/heads/master in the resulting repository and the remote-tracking branches they have in their refs/remotes/origin/* would become the remote-tracking branches refs/remotes/origin/* in the resulting repository.  So perhaps doing that against the bundle file would do what you wanted to do?  If that is the case, then perhaps your use case was covered by the original design of the "git bundle" feature after all---to allow you to clone or fetch from the file as if you are cloning or fetching from the original repository.

HTH.

Re: [BUG] git clone from bundle with --all does not fetch all refs

[Andreas Schwab]

On Okt 08 2025, Andrew Harmon wrote:

> As an end user, I found this very surprising. I was expecting to call:
> git clone <repo-bundle>
> and have this behave just like:
> git clone <ssh-or-https-target>

If you want that equivalency, then you need to create the bundle either
directly from <ssh-or-https-target>, or a mirror clone thereof.  By
using a regular clone as the source for the bundle you already have a
different type of repository than the one you see at the
<ssh-or-https-target>.

> echo ""
> echo "Clone the repo via SSH"
> git clone git@UBUBEAR:external_sources/matrice_sq.git matrice_sq
>
> echo ""
> echo "View available remote branches"
> (cd matrice_sq && git branch -r)

Compare the output of "git ls-remote matrice_sq" with the output of "git
ls-remote git@UBUBEAR:external_sources/matrice_sq.git"
to see the difference between the two repositories.

>
> echo ""
> echo "Pack the bundle for offline distribution"
> (cd matrice_sq && git bundle create ../matrice_sq.bundle --all)

The bundle picks up the refs from your local clone matice_sq, which is
different from the list of refs you'll see in the external source.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

RE: [BUG] git clone from bundle with --all does not fetch all refs

[Andrew Harmon]

> If you want that equivalency, then you need to create the bundle either
> directly from <ssh-or-https-target>, or a mirror clone thereof.  By using a
> regular clone as the source for the bundle you already have a different type
> of repository than the one you see at the <ssh-or-https-target>.

Thank you! This explanation really helps. Creating the bundle from a mirrored clone of the ssh target is exactly what I needed. This could be clearer in the docs. Several hours of time with Google, Claude, and ChatGPT all failed to point me at this simple (and now obvious) explanation.

I appreciate your help, and your dedication to the Git project.

Kind regards,
Andrew

-----Original Message-----
From: Andreas Schwab <schwab@linux-m68k.org> 
Sent: Wednesday, October 8, 2025 14:52
To: Andrew Harmon <aharmon@signalquest.com>
Cc: Junio C Hamano <gitster@pobox.com>; git@vger.kernel.org
Subject: Re: [BUG] git clone from bundle with --all does not fetch all refs

On Okt 08 2025, Andrew Harmon wrote:

> As an end user, I found this very surprising. I was expecting to call:
> git clone <repo-bundle>
> and have this behave just like:
> git clone <ssh-or-https-target>

If you want that equivalency, then you need to create the bundle either directly from <ssh-or-https-target>, or a mirror clone thereof.  By using a regular clone as the source for the bundle you already have a different type of repository than the one you see at the <ssh-or-https-target>.

> echo ""
> echo "Clone the repo via SSH"
> git clone git@UBUBEAR:external_sources/matrice_sq.git matrice_sq
>
> echo ""
> echo "View available remote branches"
> (cd matrice_sq && git branch -r)

Compare the output of "git ls-remote matrice_sq" with the output of "git ls-remote git@UBUBEAR:external_sources/matrice_sq.git"
to see the difference between the two repositories.

>
> echo ""
> echo "Pack the bundle for offline distribution"
> (cd matrice_sq && git bundle create ../matrice_sq.bundle --all)

The bundle picks up the refs from your local clone matice_sq, which is different from the list of refs you'll see in the external source.

--
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1 "And now for something completely different."