GIT - breaking backward compatibility

GIT - breaking backward compatibility

[Junio C Hamano]

I raised the following issues in my previous messages but did
not hear many opinions [*1*].  I do not want to take it as a
blank check from the community to do whatever I please.  So here
is a recap.

 * Tools renaming plan calls for removal of the backward
   compatible command names (e.g. git-fsck-cache and
   git-update-cache) sometime in the future.  This is scheduled
   for 0.99.8 around beginning of October.  If somebody wants
   extended amnesty period, this can be pushed back but unless I
   hear otherwise...

 * After reviewing the current set of commands, the following do
   not seem to be useful anymore; Linus said he feels they can
   go, and nobody else objected:

   git-diff-helper git-diff-stages git-export git-rev-tree

   I'd like to remove them before 1.0, and planning to do it
   within the 0.99.8 timeframe unless I hear otherwise.

 * After Brian Gerst posted a patch to show 'modified' files in
   ls-files [*2*], there was a brief discussion to change the
   tagged output markings to make them more readable, but
   neither Cogito nor StGIT seems to use tagged output.  I am
   currently thinking about removing '-t' altogether.

   Again, unless I hear otherwise, I'd like to remove it within
   the 0.99.8 timeframe.


BTW, independent from any of these I'll be doing a 0.99.7a
soonish for "fixes only" on top of 0.99.7.


[Footnote]

*1* Well, Pasky indicated he does not like some of the terms in
the glossary in his recent Cogito release announcement, but that
was unfortunately after the fact.

*2* I haven't taken this patch not because I do not think
showing 'modified' file is a bad idea but because showing cache
dirty files as 'modified' did not feel right to me.  I think
doing what 'git-update-index --refresh' does without actually
refreshing the cache status bits would be the right way to go.

Re: GIT - breaking backward compatibility

[Brian Gerst]

Junio C Hamano wrote:
> I raised the following issues in my previous messages but did
> not hear many opinions [*1*].  I do not want to take it as a
> blank check from the community to do whatever I please.  So here
> is a recap.
> 
>  * Tools renaming plan calls for removal of the backward
>    compatible command names (e.g. git-fsck-cache and
>    git-update-cache) sometime in the future.  This is scheduled
>    for 0.99.8 around beginning of October.  If somebody wants
>    extended amnesty period, this can be pushed back but unless I
>    hear otherwise...
> 
>  * After reviewing the current set of commands, the following do
>    not seem to be useful anymore; Linus said he feels they can
>    go, and nobody else objected:
> 
>    git-diff-helper git-diff-stages git-export git-rev-tree
> 
>    I'd like to remove them before 1.0, and planning to do it
>    within the 0.99.8 timeframe unless I hear otherwise.
> 
>  * After Brian Gerst posted a patch to show 'modified' files in
>    ls-files [*2*], there was a brief discussion to change the
>    tagged output markings to make them more readable, but
>    neither Cogito nor StGIT seems to use tagged output.  I am
>    currently thinking about removing '-t' altogether.
> 
>    Again, unless I hear otherwise, I'd like to remove it within
>    the 0.99.8 timeframe.
> 
> 
> BTW, independent from any of these I'll be doing a 0.99.7a
> soonish for "fixes only" on top of 0.99.7.
> 
> 
> [Footnote]
> 
> *1* Well, Pasky indicated he does not like some of the terms in
> the glossary in his recent Cogito release announcement, but that
> was unfortunately after the fact.
> 
> *2* I haven't taken this patch not because I do not think
> showing 'modified' file is a bad idea but because showing cache
> dirty files as 'modified' did not feel right to me.  I think
> doing what 'git-update-index --refresh' does without actually
> refreshing the cache status bits would be the right way to go.

Essentially what I want to do is:

git-ls-files --others | xargs git-update-index --add --
git-ls-files --deleted | xargs git-update-index --remove --
git-ls-files --modified | xargs git-update-index --

This will completely resync the index and cache to the working tree 
state after applying a patch.  git-update-index --refresh only updates 
the stat info in the index.  It does _not_ write a new cache object if 
the file contents have actually changed.

Cogito would benefit from this too.  It currently uses git-diff-index 
and some ugly sed expressions in cg-commit to detect modified files.

If your objection is to calling the files modifed, then call it dirty or 
something else.

--
				Brian Gerst

Re: GIT - breaking backward compatibility

[Linus Torvalds]

On Tue, 20 Sep 2005, Brian Gerst wrote:
> 
> Essentially what I want to do is:
> 
> git-ls-files --others | xargs git-update-index --add --
> git-ls-files --deleted | xargs git-update-index --remove --
> git-ls-files --modified | xargs git-update-index --
> 
> This will completely resync the index and cache to the working tree 
> state after applying a patch.

It will also be extremely inefficient.

If you really have a _patch_, then "git-apply --index" is what you want to 
apply it with. It applies a patch _and_ updates the index as appropriate. 
It's how git-applymbox can apply hundreds of patches in short order.

		Linus

Re: GIT - breaking backward compatibility

[Brian Gerst]

Linus Torvalds wrote:
> 
> On Tue, 20 Sep 2005, Brian Gerst wrote:
> 
>>Essentially what I want to do is:
>>
>>git-ls-files --others | xargs git-update-index --add --
>>git-ls-files --deleted | xargs git-update-index --remove --
>>git-ls-files --modified | xargs git-update-index --
>>
>>This will completely resync the index and cache to the working tree 
>>state after applying a patch.
> 
> 
> It will also be extremely inefficient.
> 
> If you really have a _patch_, then "git-apply --index" is what you want to 
> apply it with. It applies a patch _and_ updates the index as appropriate. 
> It's how git-applymbox can apply hundreds of patches in short order.
> 
> 		Linus
> 

That would be great, if git-apply accepted fuzzy patches.  I am trying 
to apply the -mm series patches, which often are slightly out of date. 
Andrew doesn't rebase them until they won't apply at all.

--
				Brian Gerst

Re: GIT - breaking backward compatibility

[Linus Torvalds]

On Tue, 20 Sep 2005, Brian Gerst wrote:
> 
> That would be great, if git-apply accepted fuzzy patches.  I am trying 
> to apply the -mm series patches, which often are slightly out of date. 
> Andrew doesn't rebase them until they won't apply at all.

Patches welcome..

Actually, fuzzy patches themselves are pretty easy to do (yeah, the
"memcmp" needs to become something smarter - not a huge deal), but one big
issue is what the "priority" should be.

Should we prefer an exact match that is a hundred lines away from the line 
indicated, over a fuzzy patch that is right where we indicated? What if 
it's 50 lines and 10 lines? What if there's one that applies with fuzz 1 
that is further away from one that applies with fuzz 2? 

Personally I don't much like fuzzy patches. I think it's perfectly valid 
to say "try exact patch by default, and do that really fast", and then 
fall back on something slower for the fuzzy case.

In other words: I'd suggest you use git-apply --index by default. It fails
very gracefully: if will apply _all_ of a patch, or it won't apply
anything at all (that means that if the last of a hundred files will 
fail, git-apply will not have modified any of the first 99 either).

In other words, git-apply has _none_ of that traditional "patch" crap
behaviour. It does patch application _right_.

Of course it does. I wrote it.

		Linus

Re: GIT - breaking backward compatibility

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 06:41:32AM CEST, I got a letter
where Brian Gerst <bgerst@didntduck.org> told me that...
> Linus Torvalds wrote:
> >
> >On Tue, 20 Sep 2005, Brian Gerst wrote:
> >
> >>Essentially what I want to do is:
> >>
> >>git-ls-files --others | xargs git-update-index --add --
> >>git-ls-files --deleted | xargs git-update-index --remove --
> >>git-ls-files --modified | xargs git-update-index --
> >>
> >>This will completely resync the index and cache to the working tree 
> >>state after applying a patch.
> >
> >
> >It will also be extremely inefficient.
> >
> >If you really have a _patch_, then "git-apply --index" is what you want to 
> >apply it with. It applies a patch _and_ updates the index as appropriate. 
> >It's how git-applymbox can apply hundreds of patches in short order.
> >
> >		Linus
> >
> 
> That would be great, if git-apply accepted fuzzy patches.  I am trying 
> to apply the -mm series patches, which often are slightly out of date. 
> Andrew doesn't rebase them until they won't apply at all.

cg-patch will process fuzzy patches and update the cache properly. It
doesn't handle rename/copy patches yet, though.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: GIT - breaking backward compatibility

[Junio C Hamano]

Brian Gerst <bgerst@didntduck.org> writes:

> Junio C Hamano wrote:

[Long quotation snipped]

Brian, you did not have to quote the whole thing if you wanted
to respond to only one bullet point in my footnotes.

> Essentially what I want to do is:
>
> git-ls-files --others | xargs git-update-index --add --
> git-ls-files --deleted | xargs git-update-index --remove --
> git-ls-files --modified | xargs git-update-index --
>
> This will completely resync the index and cache to the working tree 
> state after applying a patch.  git-update-index --refresh only updates 
> the stat info in the index.  It does _not_ write a new cache object if 
> the file contents have actually changed.

First of all, what I meant to say with 'update-index --refresh'
was not the refreshing part itself, but the fact that says
'needs update' -- meaning it knows those paths have been truly
modified.

Forgetting 'use git-apply' comment from Linus for now,... if you
are going to do the above in separate steps, then the last one
is already available (but not as an option to 'git-ls-files'):

    git-diff-files -r --name-only -z | xargs -0 git-update-index --

> If your objection is to calling the files modifed, then call it dirty or 
> something else.

I was not talking about the name, but the semantics.

* The user may be interested in cache-dirty files, but I suspect
  that is a very limited audience.  I do not offhand think of a
  good reason to want to know which files are cache-dirty
  without wanting to know if they are really modified, except
  when debugging git itself.  If you really want to know that,
  you can always say `git-diff-files`, or if you want to be
  pickier, `git-diff-files --diff-filter=M --name-only`.

* Showing a list of *truly* 'modified' files, disregarding the
  false hits from cache-dirty but otherwise unmodified files,
  would be another useful thing.  But that is something
  `git-update-cache --refresh` already gives you.

* As a front-end, `git status` shows you list of modifications
  between HEAD and cache, and between cache and working tree.
  The latter is done with `git-diff-files` after running
  `git-update-cache --refresh`.  This probably gives the most
  useful information to the end user.

Having said all that, `git-ls-files`, especially with `-t` flag,
is a handy way to know the status of all files in the working
tree with a single command.  What it does not currently give us
that would be nicer to have as an addition is not cache-dirty
status, but *true* 'modified' flag.  Although this is something
available from `git-update-cache --refresh` as I said earlier,
it would still be nice to be able to get it out of a single
command invocation, together with files in other status.

So that is what I have on the proposed updates branch tonight.
Does it more-or-less do what you wanted?

Re: GIT - breaking backward compatibility

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 06:12:35AM CEST, I got a letter
where Brian Gerst <bgerst@didntduck.org> told me that...
> Cogito would benefit from this too.  It currently uses git-diff-index 
> and some ugly sed expressions in cg-commit to detect modified files.

Actually, I think most of it is unnecessary - it was around from very
early days, some of it from even before the time when we started
recording adds/removed in the cache. It just doesn't break anything so I
kept it around just to be sure, but I'll remove it soon - I want to
write some cg-commit regression tests first.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: GIT - breaking backward compatibility

[Sven Verdoolaege]

On Mon, Sep 19, 2005 at 11:45:35PM -0700, Junio C Hamano wrote:
> Sven Verdoolaege <skimo@kotnet.org> writes:
> 
> > What tool can be used as a replacement for git-diff-stages ?
> 
> If we removed it, you would probably end up scripting it by
> using output from git-ls-files --stage (or --unmerged), and
> comparing the mode and SHA1 for the stages, and if you want -p
> output then git-unpack-file and diff too.

OK, that was basically what I had planed to do, until I noticed
in your mail that there was a tool called git-diff-stages.
I just thought your removing it meant that it had become
a special case of some other tool.

> But the real question was how useful what git-diff-stages does
> is.  Comparing stage3 (or stage1) and working tree might be
> suseful, but comparing stage<n> and stage<m> is what the command
> is about.  Since those stages are present after/during an
> incomplete merge, probably the user or script is trying to
> figure out what happened by inspecting the trees being merged --
> if that is the case, instead of inspecting what the merge
> algorithm which did not complete used to apply its heuristics
> (which is essentially what is left in those stages), you or your
> script can run 'git-diff-tree' across trees involved in the
> merge directly.

But the merge heuristics usually do a pretty good job
so the user would have to perform fewer manual merge operations
if she starts off from the result of the failed merge.
(And if it turns out that the merge heuristics really didn't
work, she can still throw the results away.)

The reason I'm asking is that I used dirdiff to do a "difficult"
merge which required some manual intervention.
The way I used it now, was to load the merge base, the two
heads and the working directory (containing the result of the
failed merge) in dirdiff and then to merge the final
result into the working directory.

I still need to look at the merging stuff in more detail,
but I figured that it would actually be more interesting
if dirdiff could perform the merge on the index rather than
the working directory.
What do you think ?

skimo

Re: GIT - breaking backward compatibility

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 04:07:28AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
> I raised the following issues in my previous messages but did
> not hear many opinions [*1*].  I do not want to take it as a
> blank check from the community to do whatever I please.  So here
> is a recap.

When I'm active on the mailing list, you can be sure that I'd complain
if I didn't like it. ;-)

>  * Tools renaming plan calls for removal of the backward
>    compatible command names (e.g. git-fsck-cache and
>    git-update-cache) sometime in the future.  This is scheduled
>    for 0.99.8 around beginning of October.  If somebody wants
>    extended amnesty period, this can be pushed back but unless I
>    hear otherwise...

I think the start of October is fine for Cogito. Cogito users usually
upgrade both Cogito and GIT at once, it seems.

>  * After reviewing the current set of commands, the following do
>    not seem to be useful anymore; Linus said he feels they can
>    go, and nobody else objected:
> 
>    git-diff-helper git-diff-stages git-export git-rev-tree
> 
>    I'd like to remove them before 1.0, and planning to do it
>    within the 0.99.8 timeframe unless I hear otherwise.

Cogito does not use any of those.

>  * After Brian Gerst posted a patch to show 'modified' files in
>    ls-files [*2*], there was a brief discussion to change the
>    tagged output markings to make them more readable, but
>    neither Cogito nor StGIT seems to use tagged output.  I am
>    currently thinking about removing '-t' altogether.
> 
>    Again, unless I hear otherwise, I'd like to remove it within
>    the 0.99.8 timeframe.

Well, if it should be kept, it should certainly be in sync with
git-diff-* tag letters. But Cogito uses only the diff tag letters and
I'm not sure if tag letters for all files in repository would really be
useful for anything.

> [Footnote]
> 
> *1* Well, Pasky indicated he does not like some of the terms in
> the glossary in his recent Cogito release announcement, but that
> was unfortunately after the fact.

Yes. I would complain loudly during the discussion, but I unwisely
skipped it at first and then didn't read the list for a few weeks.
My fault, I have to make the best way out of the terminology mess I'm
in now.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: GIT - breaking backward compatibility

[Petr Baudis]

Dear diary, on Tue, Sep 20, 2005 at 04:07:28AM CEST, I got a letter
where Junio C Hamano <junkio@cox.net> told me that...
>  * Tools renaming plan calls for removal of the backward
>    compatible command names (e.g. git-fsck-cache and
>    git-update-cache) sometime in the future.  This is scheduled
>    for 0.99.8 around beginning of October.  If somebody wants
>    extended amnesty period, this can be pushed back but unless I
>    hear otherwise...

Actually, could we please keep the old git-ssh-* stuff for a bit
(perhaps a lot) longer? The other renames are fine because people
usually keep their git-core and porcelain versions in sync, but the
git-ssh-* stuff is about network interoperability and you are forcing
other people to upgrade their installations, which may be troublesome
for them in the case their distribution didn't package the new stuff yet
or whatever.  It's just two commands after all, so could we please have
them for at least another month or so?

Thanks,

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.

Re: GIT - breaking backward compatibility

[Junio C Hamano]

Petr Baudis <pasky@suse.cz> writes:

> Actually, could we please keep the old git-ssh-* stuff for a bit
> (perhaps a lot) longer?

Yeah, I think that's very sensible.  Thanks!

Updated the renames plan in the TODO document.