[BUG] git-log: tracking deleted file in a repository with multiple "initial commit" histories

[BUG] git-log: tracking deleted file in a repository with multiple "initial commit" histories

[Brian Norris]

Hi,

I'm not sure if this is a known behavior or a new bug report. I at least
couldn't find anyone mentioning this exact problem.

I'm using a git repository that has multiple "inital commits" (i.e., a
few different directory trees were imported via svn-to-git as different
branches) whose histories have been merged together to the single master
branch, and the file I want to track is both added and removed in only
one of those lineages. When I try to do:

  $ git log -- <file>

on the deleted file in the master branch, I get no history. But if I
checkout the particular sub-tree of the merge history, then git-log
returns the appropriate history.

For specifics, I'm looking at this repo:

  https://chromium.googlesource.com/chromiumos/platform2

and this file:

  init/iptables.conf

which is added in this commit:

  882271d255f4 Still more platform modules.

and deleted here:

  65a8de6f85b8 chromeos-init: Remove firewall upstart jobs from platform/init

and whose branch of history is merged in via the following merge commit:

  8f4314b70b78 Move 'src/platform/init' to 'src/platform2'

.

Test 1:

  $ git checkout 8f4314b70b78
  $ git log -- init/iptables.conf
  ## No output

Test 2:

  $ git checkout 8f4314b70b78^2
  $ git log -- init/iptables.conf
  ## See proper history

The behavior of Test 1 seems like a bug to me. Thoughts?

Regards,
Brian

Re: [BUG] git-log: tracking deleted file in a repository with multiple "initial commit" histories

[Jeff King]

On Tue, Feb 16, 2016 at 12:24:42PM -0800, Brian Norris wrote:

> I'm not sure if this is a known behavior or a new bug report. I at least
> couldn't find anyone mentioning this exact problem.
> 
> I'm using a git repository that has multiple "inital commits" (i.e., a
> few different directory trees were imported via svn-to-git as different
> branches) whose histories have been merged together to the single master
> branch, and the file I want to track is both added and removed in only
> one of those lineages. When I try to do:
> 
>   $ git log -- <file>
> 
> on the deleted file in the master branch, I get no history. But if I
> checkout the particular sub-tree of the merge history, then git-log
> returns the appropriate history.

See the section on History Simplification in git-log. But basically,
when you specify a pathspec, git does not traverse side branches that
had no effect on the given pathspec.

So imagine we are walking backwards through history and showing each
commit, and we hit a merge where the content at that path is at some
sha1 X. We see on one side of the merge that the parent was also at X,
and on the other it was at Y. We do not bother following the commits
down the second parent at all. Perhaps it touched the file and perhaps
it did not, but it does not matter, as the merge threw away anything it
did in favor of the first parent.

Yours is a special case where the final result is "deleted". So there is
a merge where one side had already deleted it, and the other side had
some content, but the merge chose the deletion.

If you want to see the full history, you can with "--full-history"
(there are some other simplification possibilities, but I don't think
any of them are interesting for your particular case).

If you want to find the merge in question, something like:

  git log --graph --full-history -m --oneline --raw -- init/iptables.conf

would work. In your case, it's probably:

  8f4314b Move 'src/platform/init' to 'src/platform2'

-Peff

Re: [BUG] git-log: tracking deleted file in a repository with multiple "initial commit" histories

[Brian Norris]

On Tue, Feb 16, 2016 at 03:45:57PM -0500, Jeff King wrote:
> See the section on History Simplification in git-log. But basically,
> when you specify a pathspec, git does not traverse side branches that
> had no effect on the given pathspec.

Thanks for the pointer. Is this done primarily for performance reasons,
or for UI simplicity (e.g., to avoid some kinds of double-counting)?
Seems like it generates unintuitive behaviors, but if it's helping block
other unintuitive behaviors, then maybe it can't be resolved easily.

FWIW, I quite often use git-log to look at the history of a deleted
file. Seems like a pretty big hole if the default behavior is going to
prune away the entire history of the file.

[...]

> If you want to see the full history, you can with "--full-history"
> (there are some other simplification possibilities, but I don't think
> any of them are interesting for your particular case).

--full-history gives me what I want (I'll admit, I didn't read through
all the other "History Simplification" documentation). Can I make this
the default somehow?

Brian

Re: [BUG] git-log: tracking deleted file in a repository with multiple "initial commit" histories

[Jeff King]

On Tue, Feb 16, 2016 at 01:24:29PM -0800, Brian Norris wrote:

> On Tue, Feb 16, 2016 at 03:45:57PM -0500, Jeff King wrote:
> > See the section on History Simplification in git-log. But basically,
> > when you specify a pathspec, git does not traverse side branches that
> > had no effect on the given pathspec.
> 
> Thanks for the pointer. Is this done primarily for performance reasons,
> or for UI simplicity (e.g., to avoid some kinds of double-counting)?
> Seems like it generates unintuitive behaviors, but if it's helping block
> other unintuitive behaviors, then maybe it can't be resolved easily.

Both, I think. Try looking at "--full-history" and you will see that it
has a lot of cruft that is probably not that interesting. But
simplifying further (e.g., with "--simplify-merges") doesn't tell much
of the whole story (or maybe it does; we do see the final deletion
there, which is the end state).

But it's been a long time since I've thought hard on simplification like
this. Most of these issues were resolved many years ago; you'd have to
dig in the list archives to get the full story.

> FWIW, I quite often use git-log to look at the history of a deleted
> file. Seems like a pretty big hole if the default behavior is going to
> prune away the entire history of the file.

It doesn't normally. What happened is that you had two parallel
histories, and the final state of the file could be explained by
following the simpler of the two histories (the one where it never
existed in the first place).

> > If you want to see the full history, you can with "--full-history"
> > (there are some other simplification possibilities, but I don't think
> > any of them are interesting for your particular case).
> 
> --full-history gives me what I want (I'll admit, I didn't read through
> all the other "History Simplification" documentation). Can I make this
> the default somehow?

No, but you can make an alias that includes it.

-Peff

Re: [BUG] git-log: tracking deleted file in a repository with multiple "initial commit" histories

[Brian Norris]

On Tue, Feb 16, 2016 at 05:29:42PM -0500, Jeff King wrote:
> On Tue, Feb 16, 2016 at 01:24:29PM -0800, Brian Norris wrote:
> 
> > On Tue, Feb 16, 2016 at 03:45:57PM -0500, Jeff King wrote:
> > > See the section on History Simplification in git-log. But basically,
> > > when you specify a pathspec, git does not traverse side branches that
> > > had no effect on the given pathspec.
> > 
> > Thanks for the pointer. Is this done primarily for performance reasons,
> > or for UI simplicity (e.g., to avoid some kinds of double-counting)?
> > Seems like it generates unintuitive behaviors, but if it's helping block
> > other unintuitive behaviors, then maybe it can't be resolved easily.
> 
> Both, I think. Try looking at "--full-history" and you will see that it
> has a lot of cruft that is probably not that interesting. But

I wasn't seeing the "cruft" at first, but now I see. It appears, BTW,
that 'git log --full-history -- <path>' gives vastly different commits
than 'git log --full-history --graph -- <path>'. (The latter has a lot
more "cruft" about unrelated merges.) That seems like a weird
inconsistency...

> simplifying further (e.g., with "--simplify-merges") doesn't tell much
> of the whole story (or maybe it does; we do see the final deletion
> there, which is the end state).

git log --full-history --simplify-merges -- init/iptables.conf
and
git log --full-history --simplify-merges --graph --oneline -- init/iptables.conf

give good results for me, showing every commit that actually modifies
the file, AFAICT.

> > FWIW, I quite often use git-log to look at the history of a deleted
> > file. Seems like a pretty big hole if the default behavior is going to
> > prune away the entire history of the file.
> 
> It doesn't normally.

That doesn't really change my statement.

> What happened is that you had two parallel
> histories, and the final state of the file could be explained by
> following the simpler of the two histories (the one where it never
> existed in the first place).

I (sort of) understand what happened. But I disagree that it's a good
default. It's obviously not what the user is asking for.

> > > If you want to see the full history, you can with "--full-history"
> > > (there are some other simplification possibilities, but I don't think
> > > any of them are interesting for your particular case).
> > 
> > --full-history gives me what I want (I'll admit, I didn't read through
> > all the other "History Simplification" documentation). Can I make this
> > the default somehow?
> 
> No, but you can make an alias that includes it.

Sure.

Thanks for the help!

Brian