Submodules can't work recursively because Git implements policy?

Submodules can't work recursively because Git implements policy?

[Klas Lindberg]

On Mon, Apr 6, 2009 at 3:16 PM, Finn Arne Gangstad <finnag@pvv.org> wrote:

> git submodule update just does "git fetch" and hopes that the required
> commit appears. In practice this means that you (may) need to invent a
> tag or a branch for all the submodules, otherwise they are not
> fetchable.
>
> This bit us pretty hard when we tried to use submodules earlier, so we
> gave up. Maybe some day...

It "hopes" to find them? This is actually my other reason for bringing
the whole SHA key fetching thing up. From what I can see, it is not
possible to implement submodules sensibly without support for fetching
SHA keys. I.e. I want fetch, checkout and every other command to
recurse as needed in the presence of submodules. This limitation
forces me to implement a whole CM tool where none should be necessary.

It appears to me that the security concern (being able to hide commits
by making them unreachable from a named reference) is actually a
policy decision and not a technical one. On what grounds does Git
decide for me how to handle security concerns? It just seems more
important to be able to have recursive submodule behaviour than to
provide band aid for careless users.

Out of curiosity: Is it really possible to change the value of an
already pushed tag? Can you only do the hiding trick with branches?

BR / Klas

Re: Submodules can't work recursively because Git implements policy?

[Finn Arne Gangstad]

On Mon, Apr 06, 2009 at 03:42:31PM +0200, Klas Lindberg wrote:
> On Mon, Apr 6, 2009 at 3:16 PM, Finn Arne Gangstad <finnag@pvv.org> wrote:
> 
> > git submodule update just does "git fetch" and hopes that the required
> > commit appears. In practice this means that you (may) need to invent a
> > tag or a branch for all the submodules, otherwise they are not
> > fetchable.
> >
> > This bit us pretty hard when we tried to use submodules earlier, so we
> > gave up. Maybe some day...
> 
> It "hopes" to find them?

Perhaps "hopes" is a loaded word, "expects" at least. The code just
does the equivalent of "git fetch; git checkout <sha-1> || die .. "

>  This is actually my other reason for bringing
> the whole SHA key fetching thing up. From what I can see, it is not
> possible to implement submodules sensibly without support for fetching
> SHA keys. I.e. I want fetch, checkout and every other command to
> recurse as needed in the presence of submodules. This limitation
> forces me to implement a whole CM tool where none should be necessary.

Yes, I could not agree more.  You may also end up writing some really
complicated wrappers around git push to get things going (where do you
push, for example). We made some interesting "concept art" around this
last year at $dayjob, but decided to drop it.

> It appears to me that the security concern (being able to hide commits
> by making them unreachable from a named reference) is actually a
> policy decision and not a technical one. On what grounds does Git
> decide for me how to handle security concerns? It just seems more
> important to be able to have recursive submodule behaviour than to
> provide band aid for careless users.

Maybe the security concerns could be handled by adding some
functionality to (quickly) get rid of unwanted commits?

> Out of curiosity: Is it really possible to change the value of an
> already pushed tag? Can you only do the hiding trick with branches?

Yes, but if you modify a tag, you get additional complications. In
particular, no one will ever try to refetch the tag, so everyone who
has already fetched it will have a permanently broken tag.

- Finn Arne

Re: Submodules can't work recursively because Git implements policy?

[Klas Lindberg]

On Mon, Apr 6, 2009 at 3:56 PM, Finn Arne Gangstad <finnag@pvv.org> wrote:

> Yes, I could not agree more.  You may also end up writing some really
> complicated wrappers around git push to get things going (where do you
> push, for example). We made some interesting "concept art" around this
> last year at $dayjob, but decided to drop it.

I don't see how pushing could work at all without recursion.

> Maybe the security concerns could be handled by adding some
> functionality to (quickly) get rid of unwanted commits?

Why not simply allow users with write permissions to "pop" revisions
from the top of the history DAG in a way that actually really deletes
the them? Or at least moves those commits to a separate, locked down
DAG that cannot be read by people without write permissions?

But anyway: If I implement support for fetching SHA keys and full
recursive behaviour in the presence of submodules; would my patches
automatically be rejected because of the rationale for the current
behaviour?

/Klas

Re: Submodules can't work recursively because Git implements policy?

[Shawn O. Pearce]

Klas Lindberg <klas.lindberg@gmail.com> wrote:
> On Mon, Apr 6, 2009 at 3:56 PM, Finn Arne Gangstad <finnag@pvv.org> wrote:
> > Maybe the security concerns could be handled by adding some
> > functionality to (quickly) get rid of unwanted commits?
> 
> Why not simply allow users with write permissions to "pop" revisions
> from the top of the history DAG in a way that actually really deletes
> the them? Or at least moves those commits to a separate, locked down
> DAG that cannot be read by people without write permissions?

What, like a secret shadow repository that you move the objects into?

That could be very expensive in terms of disk IO if those objects
are in large packs.  You'd need to break the pack apart into the
"ok" and "sekret" parts.  Ick.
 
> But anyway: If I implement support for fetching SHA keys and full
> recursive behaviour in the presence of submodules; would my patches
> automatically be rejected because of the rationale for the current
> behaviour?

See my recent email (like ~10-15 minutes ago).  It will be rejected
due to the issue that unreachable objects are subjected to GC and
you'd easily see your repository delete that data on the next "git
gc" invocation.  Automatic data destruction is not something that
users come to git for.

-- 
Shawn.

Re: Submodules can't work recursively because Git implements policy?

[Klas Lindberg]

On Mon, Apr 6, 2009 at 4:51 PM, Shawn O. Pearce <spearce@spearce.org> wrote:

> What, like a secret shadow repository that you move the objects into?
>
> That could be very expensive in terms of disk IO if those objects
> are in large packs.  You'd need to break the pack apart into the
> "ok" and "sekret" parts.  Ick.

Well, ok. Just popping then?
Or adding the wrong publication to a forbidden fetch list?

>> But anyway: If I implement support for fetching SHA keys and full
>> recursive behaviour in the presence of submodules; would my patches
>> automatically be rejected because of the rationale for the current
>> behaviour?
>
> See my recent email (like ~10-15 minutes ago).  It will be rejected
> due to the issue that unreachable objects are subjected to GC and
> you'd easily see your repository delete that data on the next "git
> gc" invocation.  Automatic data destruction is not something that
> users come to git for.

Indeed not. But I don't want to suggest that named references
shouldn't be necessary. Just that it should be possible to fetch based
on the SHA key if that commit is (still) available on the remote end.

BR / Klas