cloning a namespace downloads all the objects

cloning a namespace downloads all the objects

[Sitaram Chamarty]

Hi,

I have a repo on a server, which contains, as namespaces, the contents
of several different repos of varying sizes.  When I run a clone
command for the smallest of the namespaces (I have a script that
intercepts the clone and sets GIT_NAMESPACE appropriately), I get the
correct set of refs, but *all* the objects from *all* the namespaces.

And since no refs from the other namespaces have come down, a 'git gc
--prune=now', run immediately after, reduces the size of
".git/objects" to the size I would expect for just that small
namespace.

In effect, it is bringing down data that is not reachable and will be
wiped out on the next gc.

Is this expected?

-- 
Sitaram

Re: cloning a namespace downloads all the objects

[Junio C Hamano]

Sitaram Chamarty <sitaramc@gmail.com> writes:

> Hi,
>
> I have a repo on a server, which contains, as namespaces, the contents
> of several different repos of varying sizes.  When I run a clone
> command for the smallest of the namespaces (I have a script that
> intercepts the clone and sets GIT_NAMESPACE appropriately), I get the
> correct set of refs, but *all* the objects from *all* the namespaces.
>
> And since no refs from the other namespaces have come down, a 'git gc
> --prune=now', run immediately after, reduces the size of
> ".git/objects" to the size I would expect for just that small
> namespace.
>
> In effect, it is bringing down data that is not reachable and will be
> wiped out on the next gc.
>
> Is this expected?

I do not think so.

This was done with a series between a1bea2c (ref namespaces:
infrastructure, 2011-07-05) and bf7930c (ref namespaces: tests,
2011-07-21); Josh, care to comment on and to look into it?

Re: cloning a namespace downloads all the objects

[Jeff King]

On Fri, Nov 16, 2012 at 08:43:13AM -0800, Junio C Hamano wrote:

> > I have a repo on a server, which contains, as namespaces, the contents
> > of several different repos of varying sizes.  When I run a clone
> > command for the smallest of the namespaces (I have a script that
> > intercepts the clone and sets GIT_NAMESPACE appropriately), I get the
> > correct set of refs, but *all* the objects from *all* the namespaces.
> >
> > And since no refs from the other namespaces have come down, a 'git gc
> > --prune=now', run immediately after, reduces the size of
> > ".git/objects" to the size I would expect for just that small
> > namespace.
> >
> > In effect, it is bringing down data that is not reachable and will be
> > wiped out on the next gc.
> >
> > Is this expected?
> 
> I do not think so.
> 
> This was done with a series between a1bea2c (ref namespaces:
> infrastructure, 2011-07-05) and bf7930c (ref namespaces: tests,
> 2011-07-21); Josh, care to comment on and to look into it?

I'd guess that the "create_full_pack" logic in create_pack_file is to
blame. The client asked for everything we advertised, so we pass "--all"
to pack-objects rather than giving it the specific list of tips.

We'd have to either fix that logic, or teach the pack-objects subprocess
to respect GIT_NAMESPACE when processing --all.

-Peff