Question on git-filter-branch

Question on git-filter-branch

[Bill Lear]

In my repo, I have sensitive files A, C, and a directory full of
sensitive stuff I want to remove forever from the git repository.

% ls
A B C D sensitive_stuff

% git --version
git version 1.5.3.5

% git filter-branch --index-filter 'git update-index --remove A' HEAD
Rewrite 5dd7d5f2d7d3a5f43c242188ac96294628267673 (7/7)
Ref 'refs/heads/master' was rewritten

These refs were rewritten:
% ls
B  C  D  sensitive_stuff

% git status
# On branch master
nothing to commit (working directory clean)

Ok, so I guess it has done what I wanted.  So, I try to remove the
next file that has sensitive information in it:

% git filter-branch --index-filter 'git update-index --remove C' HEAD
Namespace refs/original/ not empty

And there I am stuck: what do I do now?

I did also try this (on a fresh clone of the original repo):

% git filter-branch --index-filter 'git update-index --remove A C sensitive_stuff/*' HEAD
Rewrite 5dd7d5f2d7d3a5f43c242188ac96294628267673 (7/7)
Ref 'refs/heads/master' was rewritten

These refs were rewritten:
% ls
B  D  sensitive_stuff/
% ls sensitive_stuff
E  F  G

So, that appears to have removed A and C, but it did nothing to the
files in the subdirectory.  Is there a way to remove the files in the
subdirectory --- or, better, to simply remove the entire subdirectory?

I did try reading the man page for this, and the first example is
essentially what I want.  It says "you will get the rewritten history
saved in HEAD", but I'm not sure what I'm supposed to do next.


Bill

Re: Question on git-filter-branch

[Mike Hommey]

On Thu, Nov 01, 2007 at 06:55:37AM -0600, Bill Lear wrote:
> In my repo, I have sensitive files A, C, and a directory full of
> sensitive stuff I want to remove forever from the git repository.
> 
> % ls
> A B C D sensitive_stuff
> 
> % git --version
> git version 1.5.3.5
> 
> % git filter-branch --index-filter 'git update-index --remove A' HEAD
> Rewrite 5dd7d5f2d7d3a5f43c242188ac96294628267673 (7/7)
> Ref 'refs/heads/master' was rewritten
> 
> These refs were rewritten:
> % ls
> B  C  D  sensitive_stuff
> 
> % git status
> # On branch master
> nothing to commit (working directory clean)
> 
> Ok, so I guess it has done what I wanted.  So, I try to remove the
> next file that has sensitive information in it:
> 
> % git filter-branch --index-filter 'git update-index --remove C' HEAD
> Namespace refs/original/ not empty

Just remove .git/refs/original/

Mike

Re: Question on git-filter-branch

[Bill Lear]

On Thursday, November 1, 2007 at 13:58:45 (+0100) Mike Hommey writes:
>On Thu, Nov 01, 2007 at 06:55:37AM -0600, Bill Lear wrote:
>> In my repo, I have sensitive files A, C, and a directory full of
>> sensitive stuff I want to remove forever from the git repository.
>> 
>> % ls
>> A B C D sensitive_stuff
>> 
>> % git --version
>> git version 1.5.3.5
>> 
>> % git filter-branch --index-filter 'git update-index --remove A' HEAD
>> Rewrite 5dd7d5f2d7d3a5f43c242188ac96294628267673 (7/7)
>> Ref 'refs/heads/master' was rewritten
>> 
>> These refs were rewritten:
>> % ls
>> B  C  D  sensitive_stuff
>> 
>> % git status
>> # On branch master
>> nothing to commit (working directory clean)
>> 
>> Ok, so I guess it has done what I wanted.  So, I try to remove the
>> next file that has sensitive information in it:
>> 
>> % git filter-branch --index-filter 'git update-index --remove C' HEAD
>> Namespace refs/original/ not empty
>
>Just remove .git/refs/original/

Ok, thanks.  But, how do I remove the subdirectory?

% rm -rf .git/refs/original
% git filter-branch --index-filter 'git update-index --remove sensitive_stuff' HEAD
Rewrite 6711f6a50605918326f67ca0c3402eab9a4c8571 (8/8)
WARNING: Ref 'refs/heads/master' is unchanged

% ls sensitive_stuff
E  F  G
% git filter-branch --index-filter 'git update-index --remove sensitive_stuff/*' HEAD
Rewrite 6711f6a50605918326f67ca0c3402eab9a4c8571 (8/8)
WARNING: Ref 'refs/heads/master' is unchanged

% ls sensitive_stuff
E  F  G

% cd sensitive_stuff
% git filter-branch --index-filter 'git update-index --remove E F G' HEAD
fatal: Not a git repository: '.git'
You need to run this command from the toplevel of the working tree.


Bill

Re: Question on git-filter-branch

[Mike Hommey]

On Thu, Nov 01, 2007 at 07:19:05AM -0600, Bill Lear wrote:
> >Just remove .git/refs/original/
> 
> Ok, thanks.  But, how do I remove the subdirectory?
> 
> % rm -rf .git/refs/original
> % git filter-branch --index-filter 'git update-index --remove sensitive_stuff' HEAD
> Rewrite 6711f6a50605918326f67ca0c3402eab9a4c8571 (8/8)
> WARNING: Ref 'refs/heads/master' is unchanged

git-update-index --remove will remove files, not trees, so you need to
use something like
git-ls-files ensitive_stuff | xargs -d "\n" git-update-index --remove
instead.

Mike

Re: Question on git-filter-branch

[Bill Lear]

On Thursday, November 1, 2007 at 14:34:41 (+0100) Mike Hommey writes:
>On Thu, Nov 01, 2007 at 07:19:05AM -0600, Bill Lear wrote:
>> >Just remove .git/refs/original/
>> 
>> Ok, thanks.  But, how do I remove the subdirectory?
>> 
>> % rm -rf .git/refs/original
>> % git filter-branch --index-filter 'git update-index --remove sensitive_stuff' HEAD
>> Rewrite 6711f6a50605918326f67ca0c3402eab9a4c8571 (8/8)
>> WARNING: Ref 'refs/heads/master' is unchanged
>
>git-update-index --remove will remove files, not trees, so you need to
>use something like
>git-ls-files ensitive_stuff | xargs -d "\n" git-update-index --remove
>instead.

Oh yeah, now we're cookin' with gas:

% git filter-branch --index-filter 'git-ls-files sensitive_stuff | xargs -d "\n" git-update-index --remove' HEAD
Rewrite 6711f6a50605918326f67ca0c3402eab9a4c8571 (8/8)
Ref 'refs/heads/master' was rewritten

These refs were rewritten:
blake^master ~*/sensitive.clone ls
B  D

Excellent.

Thanks for the help.


Bill

Re: Question on git-filter-branch

[Alex Riesen]

Mike Hommey, Thu, Nov 01, 2007 14:34:41 +0100:
> On Thu, Nov 01, 2007 at 07:19:05AM -0600, Bill Lear wrote:
> > >Just remove .git/refs/original/
> > 
> > Ok, thanks.  But, how do I remove the subdirectory?
> > 
> > % rm -rf .git/refs/original
> > % git filter-branch --index-filter 'git update-index --remove sensitive_stuff' HEAD
> > Rewrite 6711f6a50605918326f67ca0c3402eab9a4c8571 (8/8)
> > WARNING: Ref 'refs/heads/master' is unchanged
> 
> git-update-index --remove will remove files, not trees, so you need to
> use something like
> git-ls-files ensitive_stuff | xargs -d "\n" git-update-index --remove
> instead.

git ls-files -z sensitive_stuff | git update-index --remove -z --stdin

Question on git-filter-branch

[Bill Lear]

I'm testing out git-filter-branch, as I would like to use it to remove
proprietary information from our repository.

I created a test repo with "sensitive information" in a file 'A', some
other "plain" information, more sensitive stuff in file 'D', a subdirectory
of sensitive information (some of this added on a branch 'branch_1',
some added on master):

% ls -F sensitive
A  B  C  D  sensitive_stuff/

I then cloned this repo and tried the filter:

% git clone sensitive sensitive.clone
% cd sensitive.clone
% git filter-branch --index-filter 'git update-index --remove A' HEAD
Rewrite 5dd7d5f2d7d3a5f43c242188ac96294628267673 (7/7)
Ref 'refs/heads/master' was rewritten

These refs were rewritten:
% ls
A  B  C  D  sensitive_stuff

Ok, so it doesn't list the refs, so I do git status:

% git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       new file:   A
#

So, it seems to have done something to A, but I don't know what to do
next.

The man page says: "Now, you will get the rewritten history saved in
the branch newbranch (your current branch is left untouched)."  Well,
I don't see any new branch created:

% git branch -a
* master
  origin/HEAD
  origin/branch_1
  origin/master

Then next part of the man page counsels that "To set a commit ...",
but I'm not sure if that is what I want to do (I think it is).
However, I'm not sure what the 'graft-id' refers to, or if I'm
supposed to type in the command as specified, especially since this is
followed by this caution: "if the parent string is empty - which
happens when we are dealing with the initial commit - add graftcommit
as a parent".  Here, I'm unsure what graftcommit is, most especially
since the use of 'graft' first appears as 'graft-id'...

Could someone help, please?


Bill