Partially private repository?

Partially private repository?

[Daed Lee]

Hi, I'm wondering if git can handle the following use. I have a
project that started as private experiment, but has morphed into
something I'd like to release publicly. I want to give others access
to the repository, but only to commits after a certain cutoff date.
Commits prior to that date have things like hardcoded file paths,
emails, etc. that I'd like to keep private.

I suppose the easiest thing to do would be to create a new repository,
add the project files to it, and make that public, however I'd like to
keep my private commit history along with the public commit history
going forward in a single repository if possible. Is there a way to do
this with git?

Appreciate any pointers in the right direction. Thanks.

Re: Partially private repository?

[Avery Pennarun]

On Fri, Jan 29, 2010 at 5:01 PM, Daed Lee <daed@thoughtsofcode.com> wrote:
> Hi, I'm wondering if git can handle the following use. I have a
> project that started as private experiment, but has morphed into
> something I'd like to release publicly. I want to give others access
> to the repository, but only to commits after a certain cutoff date.
> Commits prior to that date have things like hardcoded file paths,
> emails, etc. that I'd like to keep private.
>
> I suppose the easiest thing to do would be to create a new repository,
> add the project files to it, and make that public, however I'd like to
> keep my private commit history along with the public commit history
> going forward in a single repository if possible. Is there a way to do
> this with git?

You should probably split your history into two pieces: the "before"
and "after" parts.  To split out the "after" part, you could use
git-filter-branch
(http://www.kernel.org/pub/software/scm/git/docs/v1.6.0.6/git-filter-branch.html).
 Then, in your private copy of the repo, you could reattach the
"before" part of the history using git grafts.

Have fun,

Avery

Re: Partially private repository?

[Daed Lee]

On Fri, Jan 29, 2010 at 5:10 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
> On Fri, Jan 29, 2010 at 5:01 PM, Daed Lee <daed@thoughtsofcode.com> wrote:
>> Hi, I'm wondering if git can handle the following use. I have a
>> project that started as private experiment, but has morphed into
>> something I'd like to release publicly. I want to give others access
>> to the repository, but only to commits after a certain cutoff date.
>> Commits prior to that date have things like hardcoded file paths,
>> emails, etc. that I'd like to keep private.
>>
>> I suppose the easiest thing to do would be to create a new repository,
>> add the project files to it, and make that public, however I'd like to
>> keep my private commit history along with the public commit history
>> going forward in a single repository if possible. Is there a way to do
>> this with git?
>
> You should probably split your history into two pieces: the "before"
> and "after" parts.  To split out the "after" part, you could use
> git-filter-branch
> (http://www.kernel.org/pub/software/scm/git/docs/v1.6.0.6/git-filter-branch.html).
>  Then, in your private copy of the repo, you could reattach the
> "before" part of the history using git grafts.

Going forward, if I made changes to my private repository (containing
the "before" and "after" parts) and pushed to the public repository
(containing only the "after" part), would this only push the commits
in the "after" part? Essentially, I want to develop in my private
repository and see my "before" and "after" changes when I "git
log/show", but only push the "after" changes to the public repository.

I've also been looking into private branches. Could I do something
like keep my "before" changes on a private branch, and then do all
future development on a public branch?

Thanks for the pointer to git grafts, I'll have to read up on it further.

Re: Partially private repository?

[Johannes Sixt]

Daed Lee schrieb:
> On Fri, Jan 29, 2010 at 5:10 PM, Avery Pennarun <apenwarr@gmail.com> wrote:
>> On Fri, Jan 29, 2010 at 5:01 PM, Daed Lee <daed@thoughtsofcode.com> wrote:
>>> Hi, I'm wondering if git can handle the following use. I have a
>>> project that started as private experiment, but has morphed into
>>> something I'd like to release publicly. I want to give others access
>>> to the repository, but only to commits after a certain cutoff date.
>>> Commits prior to that date have things like hardcoded file paths,
>>> emails, etc. that I'd like to keep private.
>>>
>>> I suppose the easiest thing to do would be to create a new repository,
>>> add the project files to it, and make that public, however I'd like to
>>> keep my private commit history along with the public commit history
>>> going forward in a single repository if possible. Is there a way to do
>>> this with git?
>> You should probably split your history into two pieces: the "before"
>> and "after" parts.  To split out the "after" part, you could use
>> git-filter-branch
>> (http://www.kernel.org/pub/software/scm/git/docs/v1.6.0.6/git-filter-branch.html).

Sidenote: http://www.kernel.org/pub/software/scm/git/docs/git.html is the
most up-to-date version with pointers to older versions. The page for
filter-branch has an example how to prune history with grafts.

>>  Then, in your private copy of the repo, you could reattach the
>> "before" part of the history using git grafts.

That is, you use grafts twice: Once to split off the "after" part, again
to re-attach it locally.

> Going forward, if I made changes to my private repository (containing
> the "before" and "after" parts) and pushed to the public repository
> (containing only the "after" part), would this only push the commits
> in the "after" part? Essentially, I want to develop in my private
> repository and see my "before" and "after" changes when I "git
> log/show", but only push the "after" changes to the public repository.

The thing is that grafts are an old mechanism, and they do not always do
the right thing. In particular, when you push the "after" part that you
have spliced with the "before" part using a graft, then the objects in the
"before" part will be pushed as well. But since the remote repository does
not have the graft, these objects are inaccessible - *except* when offer
access via dumb protocols (rsync or dumb http).

There is a newer mechanism: 'git replace'. It does the right thing. That
is, after you have split the "before" and "after" parts (using a graft and
filter-branch), you splice them together using 'git replace'.

Here is a recipe that should do it (not tested, try it on a clone):

--- 8< ---
# interesting commits
first_after=first-commit-of-the-after-part  # all 40 digits!
before=$first_after^
git tag first-after $first_after

# split using a graft
echo $first_after > .git/info/grafts
git filter-branch --tag-name-filter=cat -- --all --not $before

# reattach
git replace first-after $first_after
--- 8< ---

We know that the old $first_after has the right parenthood that we need to
re-attach the split histories. We tag $first_after, and make sure the tag
gets rewritten. Then we can use the tag name in the 'git replace' call:
The tag name points to the rewritten (parentless) commit, and $first_after
is the old commit whose parent points to "before".

> I've also been looking into private branches. Could I do something
> like keep my "before" changes on a private branch, and then do all
> future development on a public branch?

You can, and you *must* do that, i.e., you must have a branch (or tag)
that points to the last commit on the "before" part. Otherwise, it is all
too easy to garbage-collect this part.

But you must make sure that you never merge a private branch into the
public history.

-- Hannes