Re: rebase parents, or tracking upstream but removing non-distributable bits

[Yann Dirson <ydirson@free.fr>]

On Thu, Dec 30, 2010 at 03:54:29PM -0200, Alexandre Oliva wrote:
> Given this problem statement, I started looking for solutions that
> didn't require modifying git.

This is a problem I have come to think a bit about already.  Although
I do not have a proper solution either, let's share those ideas.

> I first looked into rewriting history, removing the unwanted bits and
> replaying subsequent changes, but quickly discarded it, for it would
> make my local repository incompatible with upstream both ways: I
> wouldn't be able to pull from it; upstream wouldn't be able to pull to
> it; third parties would run into ugly situations trying to carry patches
> from either one to the other.
> 
> Now, it looks like I might be able to pull from upstream if I maintain
> manually a graft file that named each upstream commit as an additional
> parent of the corresponding local rebase commit that brought it into my
> rewritten tree.  Workable, maybe, but this wouldn't help third parties
> that used my public repository.

As a side note: I fear grafts won't scale very nicely performance-wise
if you graft every commit.

My use-case was similar to yours, about "how could Debian distribute a
git tree of the kernel ?", but my focus was on distributing
well-defined snapshots, and I ended up with the idea of grafting only
the set of successive revisions that get published - and not
necessarily have a mirror of all upstream revisions in the filtered
tree: such grafts do represent points in history where we would merge
from upstream, and without the grafts the filtered tree would look
like successive imports of well-defined revisions (notably makes it
realistic to adhere to the "all commits have been tested" paradigm).


> Besides, I'm concerned that pushing from the local repository (with the
> graft file) to the public repository would end up publishing the changes
> I'm not supposed to distribute, because they'd be taken as parents of
> the local commits.

Grafts cannot be pushed/pulled.  For this, as Jonathan suggests in
another reply, "git replace" looks like a better choice, esp. in that
by default the replacement commits don't get pulled: you need to
explicitely request fetching refs/replace/commits (note: you may need
to handle merges there when needed).  OTOH, those replacement commits
would pull the whole upstream history, so they cannot be part of the
filtered repository: it must come from an unfiltered kernel repo, that
may be a real problem if you cannot redistribute some upstream parts.

OTOH, a grafts file can be distributed out-of-band, and would only
pull the problematic contents when put in place, so it might indeed be
more adequate.


> Are there any other ways to support the desired features with git as-is?
> AFAICT, there isn't, so I've been thinking of how to introduce this.  I
> suppose the simplest way to accomplish this is to introduce the notion
> of a ???weak parent???: one that is taken into account for purposes of
> checking whether a commit is present in a branch being merged- or
> rebased into, but that is not transmitted over pushes, and that is not
> retained over purges, and not complained about when missing.

That sounds like heavy surgery with too many implications I can think
of.

When it comes to "modifying commit metadata", it rings "git notes" in
my ear - but then, you cannot add/change commit metadata from a note,
maybe that could be a better direction to dig into, so we can add
parents when a specific notes namespace is activated ?  But then we
would get back to the problem I mentionned for refs/replace/


> I'm under the impression that this could not just work, but also make
> rebasing in general (especially the hard case) far less problematic, for
> git would be able to relate a rebased commit with an original commit.

I suppose that by "hard case" you mean forking off a branch that gets
rebased later ?  Note that "git pull" seems to be able to cope with
this using reflogs already (although I did not test that feature).  A
less volatile place than reflogs could again using notes, without a
need for core changes.


> I'm not sure this would enable upstream to easily integrate
> my changes.  Or would it?

This problem suggests a more generic one: how to "merge back" most
changes from a branch while still not merging some specific changes ?
It would also help when a maint branch gets some short-term
workarounds that we don't want in the master branch, but the very idea
has a serious flaw: it implies that the "merge back" commit contains
also the commits we don't want (here, the "filtering commits").  So I
guess cherry-pick will be the way here.  Anyway, I doubt Linus would
like the idea of merging from such a filtered repo, sending patches is
probably prefered.

Best regards,
-- 
Yann