FFmpeg considering GIT

FFmpeg considering GIT

[Panagiotis Issaris]

Hi,

Some of the people of the FFmpeg project are looking at both GIT and Mercurial
as possible replacements for the current Subversion repository. They have some
questions regarding the possibility of doing certain things, which I prefer not
to answer as I am not sure my answer would be correct :) Which is why I am
posting here...

The questions are stated in this e-mail [1]. One of the things that are being
discussed is the following action on a publicly mirrored repository:
git branch -m master dead_end
git branch -m last_good master

I'd think this would fail as people could have pulled from the repository while
the "dead_end" commit was already available, right?

There are some other things the FFmpeg maintainer mentions, namely:
* He wants to be able to revert a commit in some way without "wiping" history.
That is without committing a patch which reverses the broken commit, as this
would pollute "git blame". The maintainer sees this as critical feature for
switching to git as it apparently can be doing using Subversion:
"in svn we can do this with svn cp from a specific
revission git and mercurial lack proper copy support"

* And finally, he noticed that when copying files, history is sometimes lost
(mentioned at the bottom of [1]).


Any answers are greatly appreciated, as I'd really like to see FFmpeg switch to
GIT.

With friendly regards,
Takis

[1]
http://article.gmane.org/gmane.comp.video.ffmpeg.devel/49673
[2]
http://article.gmane.org/gmane.comp.video.ffmpeg.devel/49656

Re: FFmpeg considering GIT

[Jakub Narebski]

Panagiotis Issaris wrote:

> Some of the people of the FFmpeg project are looking at both GIT and Mercurial
> as possible replacements for the current Subversion repository. They have some
> questions regarding the possibility of doing certain things, which I prefer not
> to answer as I am not sure my answer would be correct :) Which is why I am
> posting here...
> 
> The questions are stated in this e-mail [1]. One of the things that are being
> discussed is the following action on a publicly mirrored repository:
> git branch -m master dead_end
> git branch -m last_good master
> 
> I'd think this would fail as people could have pulled from the repository while
> the "dead_end" commit was already available, right?
> 
> There are some other things the FFmpeg maintainer mentions, namely:
> * He wants to be able to revert a commit in some way without "wiping" history.
> That is without committing a patch which reverses the broken commit, as this
> would pollute "git blame". The maintainer sees this as critical feature for
> switching to git as it apparently can be doing using Subversion:
> "in svn we can do this with svn cp from a specific
> revission git and mercurial lack proper copy support"

About removing a commit: assume that you have the following history

  A---B---C---D---E          <---  branch

Now you have noticed that commit C is wrong, and it should not be there.
One solution, which is used usually if the history was published, is to
revert a commit, resulting in the following history:

  A---B---C---D---E---C^-1   <--- branch

(which is what git-revert does).

Now if you didn't publish this history, or you don't care that you are
rewriting history, it is fairly easy to remove commit C (for example
using "git rebase --onto B D E" command), resulting in the following
history:

  A---B---C---D---E
       \
        \
         \----D'--E'          <--- branch

(which after pruning would result in A---B---D'--E' history).

The problem exists _only_ if somebody based his/her work on commit
C or its descendant, i.e. original D, E commits. He/she would have
to rebase his/her work on top of _changed_ (moved) commits D' and E'.

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: FFmpeg considering GIT

[Petr Baudis]

On Thu, May 03, 2007 at 01:48:26AM CEST, Jakub Narebski wrote:
> About removing a commit: assume that you have the following history
> The problem exists _only_ if somebody based his/her work on commit
> C or its descendant, i.e. original D, E commits. He/she would have
> to rebase his/her work on top of _changed_ (moved) commits D' and E'.

"_Only_"?

I think it's just totally unsustainable to do this history rewriting in
an "upstream" git repository. You will get horridly confused, then
frustrated and then just move from software development to beekeeping.

Imagine what will happen in gitk --all - you will see many commits
several times in a row because each is part of different subhistory for
a given head. Merging between branches will become totally impossible.
People keeping their clones (or even forking history) will be confused
and horrified. Bits of patches inbetween the original commit and the
revert moment will lose their meaning, the history won't be trustworthy
anymore at all.

In the end, using these practices git will end up useful roughly as a
faster but crippled SVN. So please don't ever just suggests how random
git commands and features with special usage might work without
carefully explaining the implications and why this is _not_ the way to
use git. If ffmpeg insists on having an X feature and it's not feasible
to make it work well with principles git is built on, ffmpeg will be
better off without git and staying with SVN, if anything to not make git
bad name between frustrated ffmpeg users and developers.

PS: Beekeeping _is_ kind of cool, really.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: FFmpeg considering GIT

[Jakub Narebski]

Petr Baudis wrote:
> On Thu, May 03, 2007 at 01:48:26AM CEST, Jakub Narebski wrote:

>> About removing a commit: assume that you have the following history
>> The problem exists _only_ if somebody based his/her work on commit
>> C or its descendant, i.e. original D, E commits. He/she would have
>> to rebase his/her work on top of _changed_ (moved) commits D' and E'.
> 
> "_Only_"?
> 
> I think it's just totally unsustainable to do this history rewriting in
> an "upstream" git repository. You will get horridly confused, then
> frustrated and then just move from software development to beekeeping.

Perhaps I should have said: "There always would be problems if somebody
based his/her work on commit C or its descendant..."

But there are some times when you can rewrite history without bad
consequences. 

You can without any problems rewrite _unpublished_ commits; if one for
example pushes to public repo once per day, or few times a week,
there is time to remove a commit, or amend a commit, or change commit
deeper in a history. Or even use StGIT to manage patches, and change
their sequence, add patch in the midle of patch series, split or join
patches, all that working on creating 'a perfect patch [series]'.

You can rewrite a branch which never would be published, like feature
branches in git.git repository (which are visible only via 'pu' -- proposed
updates branch, which is meant to have history rewritten). Or you can
announce that given branch might be rewritten, and not to base any work
on it (well, you can, but you always should rebase before sending).


Because there always are, and always will be problems if somebody would
base work on series including now removed commit, even if SCM need not
to rewrite history to remove a commit [*1*]. And with history rewriting
even more so, for example accidental inclusion of removed commit.

Besides I think it would be better to teach blame to ignore reversion
commits (for example based on first line of commit message) than to mess
with the history. Note also that git has more tools for forensic analysis
than git-blame; blame / annotate was added later because people are used
to it (and it is I think better than any other, because it can detect
moving and copying code blocks). The primary examining tools are history
browsing limited to specified pathspec, and pickaxe i.e. searching for
commits which changed given line.

Footnotes:
----------
 [1] Git began as content adressed filesystem, where each object is named
     by its contents (or rather cryptographics hash function of contents).
     This results in hash (object id) of commit identifying whole lineage
     of it, and makes signing specified commit (using signed tag)
     identifying / signing whole history.

-- 
Jakub Narebski
ShadeHawk on #git
Poland

[RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Johan Herland]

[-- Attachment #1: Type: text/plain, Size: 4036 bytes --]

On Friday 04 May 2007, Jakub Narebski wrote:
> Besides I think it would be better to teach blame to ignore reversion
> commits (for example based on first line of commit message) than to
> mess with the history.

I'm starting to see a pattern where people would like to tell git about 
more complicated relationships between commits, so that git can make 
more intelligent decisions when doing merge, blame, pickaxe, etc.

Adding these relationships as part of the commit message seems like a 
really stupid idea because git suddenly has to make sense of something 
it has never parsed before, thus making all future and former git 
commit messages a potential target for pattern (mis)matching by git. 
Also, we seem to forget that we already have the perfect place to put 
such information: The header fields preceding the commit message.

I therefore propose adding header field names to commit objects that 
illustrate the relationships people want to tell git about. Examples 
include:

1. "Reverts": Mark a commit as reverting another commit. This could be 
used by git-log to cancel out pairs of commits, resulting in a cleaner 
view of history. It can help blame/annotate. There are probably other 
tools that can benefit from this information also.

2. "Cherry-Pick": When cherry-picking a commit onto another branch, you 
should be able to tell git which commit you are cherry-picking 
(git-cherry-pick would of course do this automatically). This could 
enable git to make smarter decisions when merging the two branches: If 
the cherry-picked commit would cause a conflict with the original 
commit, git can either skip it (since it knows that one version of this 
patch is already present), or it can at least present the conflict to 
the user with some more context than what is available today. Not to 
mention how this information could be used by blame/annotate.

3. "Rebased-From": This one can be filled in automatically by 
git-rebase, but when I think about it, it may be too similar 
to "Cherry-Pick" to warrant a separate field.

4. "Rebased-To": When doing a rebase like the following:

   A---B---C---D---E       <--- branch

       (Hmm. C is broken. Rebase D and E onto B)

   A---B---C---D---E
        \
         \--D'--E'         <--- branch

   git-rebase could now add a dummy commit F* to E with "Rebased-To: 
{Commit ID of D'}", thus making:

   A---B---C---D---E---F*..
        \    ,............:  (yes, this is a poorly drawn meta-arrow)
         \   v
          \--D'--E'        <--- branch

   This would make it easier for git to do the Right Thing when someone 
following the old branch tries to pull after the rebase.

5. Heck, while we're at it, move "Signed-off-by" into the header fields, 
where git can make more use of it.

6. Finally, allow people to add custom header fields prefixed by "X-" 
(like in HTTP), and make it easy for them to extend git tools to use 
these custom fields in various ways. If some of them end up being 
really useful, we can import them into git (and lose the "X-" prefix).


Now, in order to let people specify these fields we probably want to 
make these fields names settable from the command line. It should also 
be possible to use a template when doing the commit message in an 
editor. Something like:
==========
Optional headers fields (fill in if applicable)
Cherry-Pick:   ________
Reverts:       ________
Signed-Off-By: ________

Your commit message goes here:
________________________________
==========

Of course, git would have to verify/sanitize these fields when input, so 
they probably need some type information associated with them.


Furthermore we might want to think about the possibility of allowing 
annotations to previous commits, in order to allow these fields to be 
set after the commit has happened, but that's a topic for a 
whole 'nother discussion.


Have fun!

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Alex Riesen]

On 5/4/07, Johan Herland <johan@herland.net> wrote:
> 1. "Reverts": Mark a commit as reverting another commit. This could be
> used by git-log to cancel out pairs of commits, resulting in a cleaner
> view of history. It can help blame/annotate. There are probably other
> tools that can benefit from this information also.
>
> 2. "Cherry-Pick": When cherry-picking a commit onto another branch, you
> should be able to tell git which commit you are cherry-picking
> (git-cherry-pick would of course do this automatically). This could
> enable git to make smarter decisions when merging the two branches: If
> the cherry-picked commit would cause a conflict with the original
> commit, git can either skip it (since it knows that one version of this
> patch is already present), or it can at least present the conflict to
> the user with some more context than what is available today. Not to
> mention how this information could be used by blame/annotate.

These are completely useless after the first "git gc --prune" or "git clone"
unless these tools taught to preserve the reverted or cherry-picked commits
(and all their history). And if you are about to teach them that, please notice
that as for now cloning and repacking does not even look at the
objects contents.
You'll absolutely kill their performance.

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Andy Parkins]

On Friday 2007 May 04, Alex Riesen wrote:
> On 5/4/07, Johan Herland <johan@herland.net> wrote:
> > 1. "Reverts": Mark a commit as reverting another commit. This could be

> These are completely useless after the first "git gc --prune" or "git

Agreed for the cherry pick (and the rebase), but the original of a revert 
won't be pruned - in fact it's almost certain that the original is a subset 
of the revert itself (otherwise the revert wouldn't have applied cleanly).

 * --- * --- X --- * --- !X --- * --- *

See?  X won't ever be pruned without !X having been pruned first.

It doesn't seem unreasonable to record in a machine readable manner that !X 
undid X.  It might be useful to someone one day.

As for custom headers - it's a great idea; here's the one that would be most 
useful:

 X-Git-SVN-ID: 9553f0bf-9b14-0410-a0b8-cfaf0461ba5b

That way git-svn wouldn't (necessarily) need to keep its .rev_db file, and it 
wouldn't need any special handling to allow the repository to be cloned.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Andrew Ruder]

On Fri, May 04, 2007 at 12:39:18PM +0100, Andy Parkins wrote:
> That way git-svn wouldn't (necessarily) need to keep its .rev_db file, and it 
> wouldn't need any special handling to allow the repository to be cloned.

Which, BTW, would be a great thing as on subversion repositories with
lots of revisions and lots of branches/tags, the disk space for all those
.rev_db files gets pretty bad.  i.e. du -sh .git/objects == 14M, du -sh
.git == 120M.

- Andy

-- 
Andrew Ruder <andy@aeruder.net>
http://www.aeruder.net

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Johan Herland]

On Friday 04 May 2007, Andy Parkins wrote:
> As for custom headers - it's a great idea; here's the one that would be
> most useful:
>
>  X-Git-SVN-ID: 9553f0bf-9b14-0410-a0b8-cfaf0461ba5b
>
> That way git-svn wouldn't (necessarily) need to keep its .rev_db file, and
> it wouldn't need any special handling to allow the repository to be cloned.

That's _exactly_ the kind of use of this I'd like to see. Great example. :)


Have fun!

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Johan Herland]

[-- Attachment #1: Type: text/plain, Size: 2513 bytes --]

On Friday 04 May 2007, Alex Riesen wrote:
> On 5/4/07, Johan Herland <johan@herland.net> wrote:
> > 1. "Reverts": Mark a commit as reverting another commit. This could be
> > used by git-log to cancel out pairs of commits, resulting in a cleaner
> > view of history. It can help blame/annotate. There are probably other
> > tools that can benefit from this information also.
> >
> > 2. "Cherry-Pick": When cherry-picking a commit onto another branch, you
> > should be able to tell git which commit you are cherry-picking
> > (git-cherry-pick would of course do this automatically). This could
> > enable git to make smarter decisions when merging the two branches: If
> > the cherry-picked commit would cause a conflict with the original
> > commit, git can either skip it (since it knows that one version of this
> > patch is already present), or it can at least present the conflict to
> > the user with some more context than what is available today. Not to
> > mention how this information could be used by blame/annotate.
>
> These are completely useless after the first "git gc --prune" or "git
> clone" unless these tools taught to preserve the reverted or cherry-picked
> commits (and all their history). And if you are about to teach them that,
> please notice that as for now cloning and repacking does not even look at
> the
> objects contents.
> You'll absolutely kill their performance.

Of course I don't want "git gc --prune" or "git clone" to follow these links, 
or know anything about them at all.

As for "Reverts", the commit pointed to should already be in your history, 
since you cannot revert something that hasn't already been applied at an 
earlier point in your history. In other words, the reverted commit will 
automatically be included in your "git gc --prune" or "git clone" regardless 
of the "Reverts" fields, since "Reverts" can only point to an ancestor.

As for "Cherry-Pick", it's a fairly weak relationship that shouldn't affect 
anything except to give a hint to merge, blame, and similar tools. 
If "Cherry-Pick" identifies an object not in your repo (because of "git 
gc --prune" or "git clone"), that is obviously equivalent to not having 
a "Cherry-Pick" field in the first place. "Cherry-Pick" is only useful when 
you have access to the original commit (pointed to by "Cherry-Pick"), but in 
that case I think it could be _really_ useful.


Have fun!

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Alex Riesen]

Johan Herland, Fri, May 04, 2007 13:53:10 +0200:
> As for "Reverts", the commit pointed to should already be in your history, 
> since you cannot revert something that hasn't already been applied at an 
> earlier point in your history. In other words, the reverted commit will 
> automatically be included in your "git gc --prune" or "git clone" regardless 
> of the "Reverts" fields, since "Reverts" can only point to an ancestor.

So it becomes useless after rebase

> As for "Cherry-Pick", it's a fairly weak relationship that shouldn't affect 
> anything except to give a hint to merge, blame, and similar tools. 

In which case, just put it in the message part of commit (in fact, it
was there for some time. And was mostly useless, and got dropped).

And how exactly do you think the tools _can_ use this hint?
Especially merge, which should be absolutely certain about what inputs
and hints gets.
And what use is it for blame? How do you prioritze the hint? Is it
more important than the history (which describes each and every line),
or less? If the hint is more important, than how (and how often) do
you tell the user that the hint was not found (because the commit is
long pruned) and the tool switched back to looking into history.

It's useless.

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Johan Herland]

[-- Attachment #1: Type: text/plain, Size: 4260 bytes --]

On Saturday 05 May 2007, Alex Riesen wrote:
> Johan Herland, Fri, May 04, 2007 13:53:10 +0200:
> > As for "Reverts", the commit pointed to should already be in your
> > history, since you cannot revert something that hasn't already been
> > applied at an earlier point in your history. In other words, the
> > reverted commit will automatically be included in your "git gc
> > --prune" or "git clone" regardless of the "Reverts" fields, since
> > "Reverts" can only point to an ancestor.
>
> So it becomes useless after rebase

Only if rebase also rebases the commit pointed to by "Reverts" (the 
reverted commit). And even in that case, it should be possible for 
rebase to detect the "Reverts" relationship and rewrite it properly, 
or - if people want to - skip both the reverted and the reverting 
commit in the rebase process.

> > As for "Cherry-Pick", it's a fairly weak relationship that
> > shouldn't affect anything except to give a hint to merge, blame,
> > and similar tools.
>
> In which case, just put it in the message part of commit (in fact, it
> was there for some time. And was mostly useless, and got dropped).

Ok. If merging branches which have had cherry-picks between them is such 
a rare occurrence that there is no point in adding hints for merge (to 
do better conflict resolution), blame (to see who _really_ wrote the 
piece of code that was cherry-picked by someone else), etc. then there 
is indeed no justification for the "Cherry-Pick" header field.

> And how exactly do you think the tools _can_ use this hint?
> Especially merge, which should be absolutely certain about what
> inputs and hints gets.

When merging two branches where one branch has a commit that is later 
reverted, and the other branch has cherry-picked the first/reverted 
commit, but not the second/reverting: With these hints, git can now ask 
the user a more intelligent question like "The following commit was 
reverted in one of the branches. Do you want to keep it or revert it?". 
The current alternative seems to be to auto-choose one or the other (in 
my testing, the reverting commit was dropped in the merge). Will git 
always make the correct decision? If git is always correct, then what I 
suggest is obviously useless.

> And what use is it for blame? How do you prioritze the hint? Is it
> more important than the history (which describes each and every
> line), or less? If the hint is more important, than how (and how
> often) do you tell the user that the hint was not found (because the
> commit is long pruned) and the tool switched back to looking into
> history.

Consider the following scenario:

====
$ mkdir test
$ cd test
$ git init
Initialized empty Git repository in .git/
$ git config user.name "User A"
$ cat >f <<\EOF
foo
bar
baz
EOF
$ git add f && git commit -m "User A: foo, bar, baz"
Created initial commit bb0203aabb4936d95dca30f946cb1d849df59f24
 1 files changed, 3 insertions(+), 0 deletions(-)
 create mode 100644 f
$ git config user.name "User B"
$ cat >f <<\EOF
foo
barf
baz
EOF
$ git commit -a -m "User B: bar -> barf"
Created commit 5ced0ccaba0bf4a982dc2cdd792a1a0e7b1883eb
 1 files changed, 1 insertions(+), 1 deletions(-)
$ git config user.name "User C"
$ git revert HEAD
Created commit 38da1083ae4677000f8bb70729f474f358c71a3e
 1 files changed, 1 insertions(+), 1 deletions(-)
====

At this point, what output do we _really_ want from "git blame f"?

Currently we get:
====
^bb0203a (User A 2007-05-05 12:25:44 +0200 1) foo
38da1083 (User C 2007-05-05 12:28:00 +0200 2) bar
^bb0203a (User A 2007-05-05 12:25:44 +0200 3) baz
====

Can you categorically say that there is no use for the following output? 
(even if you need to pass an option to "git blame" to get it):
====
^bb0203a (User A 2007-05-05 12:25:44 +0200 1) foo
^bb0203a (User A 2007-05-05 12:25:44 +0200 1) bar
^bb0203a (User A 2007-05-05 12:25:44 +0200 3) baz
====

> It's useless.

Maybe. At least some of the fields I proposed are probably useless. But 
I don't think we should throw away the core idea unless we can show 
that _all_ fields are useless.


Have fun!

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Alex Riesen]

Johan Herland, Sat, May 05, 2007 14:49:42 +0200:
> Can you categorically say that there is no use for the following output? 
> (even if you need to pass an option to "git blame" to get it):
> ====
> ^bb0203a (User A 2007-05-05 12:25:44 +0200 1) foo
> ^bb0203a (User A 2007-05-05 12:25:44 +0200 1) bar
> ^bb0203a (User A 2007-05-05 12:25:44 +0200 3) baz
> ====

Assuming a repo which has 50% of all commits - reverts (just because
someone could not be bothered to learn to use rebase, format-patch
and git-am before sending things upstream) I would use the exact
wording I used before. I'd say "it's dangerous" now. It hides the
mess this repo is.

> > It's useless.
> 
> Maybe. At least some of the fields I proposed are probably useless. But 
> I don't think we should throw away the core idea unless we can show 
> that _all_ fields are useless.

Just think of something you actually _can_ use. Implement it and try.
And than, if you are convinced it actually is useful, try it on your
friends. And after them, if you're still alive, try using it with old
git (like 1.4 from Debian and Ubuntu).

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Johan Herland]

[-- Attachment #1: Type: text/plain, Size: 4260 bytes --]

On Saturday 05 May 2007, Alex Riesen wrote:
> Johan Herland, Fri, May 04, 2007 13:53:10 +0200:
> > As for "Reverts", the commit pointed to should already be in your
> > history, since you cannot revert something that hasn't already been
> > applied at an earlier point in your history. In other words, the
> > reverted commit will automatically be included in your "git gc
> > --prune" or "git clone" regardless of the "Reverts" fields, since
> > "Reverts" can only point to an ancestor.
>
> So it becomes useless after rebase

Only if rebase also rebases the commit pointed to by "Reverts" (the 
reverted commit). And even in that case, it should be possible for 
rebase to detect the "Reverts" relationship and rewrite it properly, 
or - if people want to - skip both the reverted and the reverting 
commit in the rebase process.

> > As for "Cherry-Pick", it's a fairly weak relationship that
> > shouldn't affect anything except to give a hint to merge, blame,
> > and similar tools.
>
> In which case, just put it in the message part of commit (in fact, it
> was there for some time. And was mostly useless, and got dropped).

Ok. If merging branches which have had cherry-picks between them is such 
a rare occurrence that there is no point in adding hints for merge (to 
do better conflict resolution), blame (to see who _really_ wrote the 
piece of code that was cherry-picked by someone else), etc. then there 
is indeed no justification for the "Cherry-Pick" header field.

> And how exactly do you think the tools _can_ use this hint?
> Especially merge, which should be absolutely certain about what
> inputs and hints gets.

When merging two branches where one branch has a commit that is later 
reverted, and the other branch has cherry-picked the first/reverted 
commit, but not the second/reverting: With these hints, git can now ask 
the user a more intelligent question like "The following commit was 
reverted in one of the branches. Do you want to keep it or revert it?". 
The current alternative seems to be to auto-choose one or the other (in 
my testing, the reverting commit was dropped in the merge). Will git 
always make the correct decision? If git is always correct, then what I 
suggest is obviously useless.

> And what use is it for blame? How do you prioritze the hint? Is it
> more important than the history (which describes each and every
> line), or less? If the hint is more important, than how (and how
> often) do you tell the user that the hint was not found (because the
> commit is long pruned) and the tool switched back to looking into
> history.

Consider the following scenario:

----
$ mkdir test
$ cd test
$ git init
Initialized empty Git repository in .git/
$ git config user.name "User A"
$ cat >f <<\EOF
foo
bar
baz
EOF
$ git add f && git commit -m "User A: foo, bar, baz"
Created initial commit bb0203aabb4936d95dca30f946cb1d849df59f24
 1 files changed, 3 insertions(+), 0 deletions(-)
 create mode 100644 f
$ git config user.name "User B"
$ cat >f <<\EOF
foo
barf
baz
EOF
$ git commit -a -m "User B: bar -> barf"
Created commit 5ced0ccaba0bf4a982dc2cdd792a1a0e7b1883eb
 1 files changed, 1 insertions(+), 1 deletions(-)
$ git config user.name "User C"
$ git revert HEAD
Created commit 38da1083ae4677000f8bb70729f474f358c71a3e
 1 files changed, 1 insertions(+), 1 deletions(-)
----

At this point, what output do we _really_ want from "git blame f"?

Currently we get:
----
^bb0203a (User A 2007-05-05 12:25:44 +0200 1) foo
38da1083 (User C 2007-05-05 12:28:00 +0200 2) bar
^bb0203a (User A 2007-05-05 12:25:44 +0200 3) baz
----

Can you categorically say that there is no use for the following output? 
(even if you need to pass an option to "git blame" to get it):
----
^bb0203a (User A 2007-05-05 12:25:44 +0200 1) foo
^bb0203a (User A 2007-05-05 12:25:44 +0200 1) bar
^bb0203a (User A 2007-05-05 12:25:44 +0200 3) baz
----

> It's useless.

Maybe. At least some of the fields I proposed are probably useless. But 
I don't think we should throw away the core idea unless we can show 
that _all_ fields are useless.


Have fun!

...Johan
-- 
Johan Herland, <johan@herland.net>
www.herland.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Petr Baudis]

On Fri, May 04, 2007 at 09:21:29AM CEST, Johan Herland wrote:
> On Friday 04 May 2007, Jakub Narebski wrote:
> > Besides I think it would be better to teach blame to ignore reversion
> > commits (for example based on first line of commit message) than to
> > mess with the history.
> 
> I'm starting to see a pattern where people would like to tell git about 
> more complicated relationships between commits, so that git can make 
> more intelligent decisions when doing merge, blame, pickaxe, etc.
> 
> Adding these relationships as part of the commit message seems like a 
> really stupid idea because git suddenly has to make sense of something 
> it has never parsed before, thus making all future and former git 
> commit messages a potential target for pattern (mis)matching by git. 
> Also, we seem to forget that we already have the perfect place to put 
> such information: The header fields preceding the commit message.
> 
> I therefore propose adding header field names to commit objects that 
> illustrate the relationships people want to tell git about.

  So I've looked it up, and the Linus' writeup on this is at

	http://news.gmane.org/find-root.php?message_id=<Pine.LNX.4.64.0604250758000.3701@g5.osdl.org>

> 1. "Reverts": Mark a commit as reverting another commit. This could be 
> used by git-log to cancel out pairs of commits, resulting in a cleaner 
> view of history. It can help blame/annotate. There are probably other 
> tools that can benefit from this information also.

  Actually I think git-log is the one tool which shouldn't cancel it
out. The number of reverts likely won't be overwhelming and reverting is
actually pretty important event - it says "this has been tried and we
decided it's not the way", also can have social meanings etc. It is an
important piece of history. And people still want to actually see the
change and possibly revive it. BTW, imagine their confusion if the
history looks like

	1abcd5 Feature X
	37efab Release 2.3.1
	724b9c Revert feature X

and git log would cancel out 1abcd5 and 724b9c. Feature X is part of
2.3.1 but not in the log..?!

  The point is that the reverting/reverted commit pairs don't affect
your current content (except maybe in an highly abstract way), and this
is why pickaxe and blame should skip it (by default).

  The question wrt. Linus' criteria is if "it has enough of a meaning",
and I wonder about that too. I think it does, though.


  For the other suggested headers, it should be already mostly obvious
from Linus' writeup why they shouldn't qualify, though.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [RFC?] Telling git about more complex relationships between commits (Was: Re: FFmpeg considering GIT)

[Johan Herland]

[-- Attachment #1: Type: text/plain, Size: 4599 bytes --]

On Friday 04 May 2007, Petr Baudis wrote:
> On Fri, May 04, 2007 at 09:21:29AM CEST, Johan Herland wrote:
> > On Friday 04 May 2007, Jakub Narebski wrote:
> > > Besides I think it would be better to teach blame to ignore reversion
> > > commits (for example based on first line of commit message) than to
> > > mess with the history.
> >
> > I'm starting to see a pattern where people would like to tell git about
> > more complicated relationships between commits, so that git can make
> > more intelligent decisions when doing merge, blame, pickaxe, etc.
> >
> > Adding these relationships as part of the commit message seems like a
> > really stupid idea because git suddenly has to make sense of something
> > it has never parsed before, thus making all future and former git
> > commit messages a potential target for pattern (mis)matching by git.
> > Also, we seem to forget that we already have the perfect place to put
> > such information: The header fields preceding the commit message.
> >
> > I therefore propose adding header field names to commit objects that
> > illustrate the relationships people want to tell git about.
>
>   So I've looked it up, and the Linus' writeup on this is at
>
> 	http://news.gmane.org/find-root.php?message_id=<Pine.LNX.4.64.060425075800
>0.3701@g5.osdl.org>

Thanks a lot for the link. I hadn't seen that writeup.

For the record: I'm only interested in adding "machine-readable" headers in 
cases where _both_ of the following holds:
1. The header has a _clear_ and _unambiguous_ _meaning_.
2. git can use the header in a well-defined manner to make informed and better 
decisions on how to behave.

In Linus' writeup, he's correct in that "prior" is too loosely defined. 
However, if we can meet Linus' requirements for clearness and semantics, I 
actually think the core idea is very good.

> > 1. "Reverts": Mark a commit as reverting another commit. This could be
> > used by git-log to cancel out pairs of commits, resulting in a cleaner
> > view of history. It can help blame/annotate. There are probably other
> > tools that can benefit from this information also.
>
>   Actually I think git-log is the one tool which shouldn't cancel it
> out. The number of reverts likely won't be overwhelming and reverting is
> actually pretty important event - it says "this has been tried and we
> decided it's not the way", also can have social meanings etc. It is an
> important piece of history. And people still want to actually see the
> change and possibly revive it. BTW, imagine their confusion if the
> history looks like
>
> 	1abcd5 Feature X
> 	37efab Release 2.3.1
> 	724b9c Revert feature X
>
> and git log would cancel out 1abcd5 and 724b9c. Feature X is part of
> 2.3.1 but not in the log..?!
>
>   The point is that the reverting/reverted commit pairs don't affect
> your current content (except maybe in an highly abstract way), and this
> is why pickaxe and blame should skip it (by default).

Of course git-log shouldn't skip reverted commit pairs _by_default_. But if 
someone is interested in a cleaner view of history (e.g. when making a 
changelog or whatnot), a command-line option for turning on this behaviour 
might be useful. Or maybe we don't want git-log to be affected by "Reverts" 
at all. But if pickaxe and blame can make real use of this header, that's 
sufficient reason to add it, I think.

>   The question wrt. Linus' criteria is if "it has enough of a meaning",
> and I wonder about that too. I think it does, though.

As stated above, I don't want header fields unless they have clearly defined 
meaning and semantics. I doubt that all of my examples will fulfill these 
criteria, but some of them should, and that may be useful enough.

>   For the other suggested headers, it should be already mostly obvious
> from Linus' writeup why they shouldn't qualify, though.

I agree with Linus in that if we cannot define clear meaning and accompanying 
semantics, then adding a header is useless. I do, however, think that there 
are cases where we _can_ define the meaning and semantics, and in those 
cases, I do believe header fields to be a good idea.

As for "Cherry-Pick", it is of course not useful when the commit pointed to is 
not in the repo, but in the cases where it _is_, it might be very useful. 
It's a tradeoff, and we might end up deciding that "Cherry-Pick" is not worth 
it, but we should at least consider the possibility.


Have fun!

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Martin Langhoff]

On 5/2/07, Panagiotis Issaris <takis.issaris@uhasselt.be> wrote:
> The questions are stated in this e-mail [1]. One of the things that are being
> discussed is the following action on a publicly mirrored repository:
> git branch -m master dead_end
> git branch -m last_good master
>
> I'd think this would fail as people could have pulled from the repository while
> the "dead_end" commit was already available, right?

Yes - that's something you shouldn't do on a normal branch... but
that's a feature ;-) -- we call it re-winding a branch.

A good workaround if you expect to go down some dead_ends is to have
an experimental branch that you pre-announce that will be rewound
regularly. On the git repo, Junio does exactly that with "pu"
("proposed updates"), and several  feature-development branches have
been dropped or rewound at times.

Now, for your main dev and various maintenance branches, just do a
revert. If something made it into the main dev branch it means it's
not so experimental anymore and all the developers are building
further development on top. At that stage, the potential mistake has
made it "quite far" so you can't rewind it and pretend it didn't exist
;-)

So the good practice is to never rewind the long-term branches people
base their work on. Branches in your repo, and public branches clearly
marked as experimental, anything goes.

cheers,


martin

Re: FFmpeg considering GIT

[Uwe Kleine-König]

Hello,

Panagiotis Issaris wrote:
> There are some other things the FFmpeg maintainer mentions, namely:
> * He wants to be able to revert a commit in some way without "wiping" history.
> That is without committing a patch which reverses the broken commit, as this
> would pollute "git blame". The maintainer sees this as critical feature for
> switching to git as it apparently can be doing using Subversion:
> "in svn we can do this with svn cp from a specific
> revission git and mercurial lack proper copy support"
To add more context, Michael Niedermayer (=FFmpeg maintainer) wrote (in
[1]):

	let me explain a little bit why this is critically needed
	think of someone misstakely commiting the whole ffmpeg
	reindented or mistakely commiting a old ffmpeg version over the
	new or another total messup, these things do happen, and
	especially if they cannot be corrected and at the time where
	none of the developers is around

For me this sounds like:  I don't want people with commit access doing
this, and if they do, I want to be able to revert it.

If FFmpeg used a development scheme similar to the linux kernel, there
should be no need for revert:  The upstream maintainer only needs to pay
attention to the things he does directly (he probably does in any case)
and check the patches he applies and the trees he pulls.  As git gives a
diffstat on pull and he reviews patches before applying the problem is
maybe gone?

Commit access is simply different in a distributed environment, see
http://thread.gmane.org/gmane.comp.version-control.git/45849/focus=45956

Best regards
Uwe

> [1]
> http://article.gmane.org/gmane.comp.video.ffmpeg.devel/49673

-- 
Uwe Kleine-König

http://www.google.com/search?q=1+newton+in+kg*m+%2F+s%5E2

Re: FFmpeg considering GIT

[Petr Baudis]

  Hi,

On Thu, May 03, 2007 at 08:00:16PM CEST, Uwe Kleine-König wrote:
> Panagiotis Issaris wrote:
> > There are some other things the FFmpeg maintainer mentions, namely:
> > * He wants to be able to revert a commit in some way without "wiping" history.
> > That is without committing a patch which reverses the broken commit, as this
> > would pollute "git blame". The maintainer sees this as critical feature for
> > switching to git as it apparently can be doing using Subversion:
> > "in svn we can do this with svn cp from a specific
> > revission git and mercurial lack proper copy support"
> To add more context, Michael Niedermayer (=FFmpeg maintainer) wrote (in
> [1]):
> 
> 	let me explain a little bit why this is critically needed
> 	think of someone misstakely commiting the whole ffmpeg
> 	reindented or mistakely commiting a old ffmpeg version over the
> 	new or another total messup, these things do happen, and
> 	especially if they cannot be corrected and at the time where
> 	none of the developers is around
> 
> For me this sounds like:  I don't want people with commit access doing
> this, and if they do, I want to be able to revert it.
> 
> If FFmpeg used a development scheme similar to the linux kernel, there
> should be no need for revert:  The upstream maintainer only needs to pay
> attention to the things he does directly (he probably does in any case)
> and check the patches he applies and the trees he pulls.  As git gives a
> diffstat on pull and he reviews patches before applying the problem is
> maybe gone?
> 
> Commit access is simply different in a distributed environment, see
> http://thread.gmane.org/gmane.comp.version-control.git/45849/focus=45956

  I believe that the development scheme is largely independent on the
version control system, except that git makes the "both ways" equally
easy. But that doesn't mean that the "multiple people with commit
access" scheme is wrong or anything. It has important upsides as well -
there's no single human point of failure (_yes_, if the maintainer gets
stuck in hospital for two months you can fork and maintain own
repository, but then it's again just you and it is complicated socially
etc.), the load of the maintainer is significantly lowered, and in many
projects there is simply no "single maintainer" but a team of people
where decisions are made by consensus.

  Still, if this kind of bogus change checkins happens at any frequent
rate in the ffmpeg project, there is a serious problem somewhere. :-)
But I think the git way of alleviating this problem would be to have a
way to hint the pickaxe and blame tools to ignore changes in given
commits. So, you don't _cover up_ the messy things that happened during
the history, but avoid in getting in the way in your view. You can still
look it up (with git log or something) in case you'd need to (perhaps
the revert patch was a bit complicated because of conflicting with some
other changes, and a subtle bug was introduced; this would be thousand
times harder to track down if you would've rewritten the history).

  Would crafting up a patch to implement something like this help ffmpeg
people in their decision?

  Let's say you have .git/info/reverts with one "revert pair" (two
commit ids, one for the bogus change and one for reverting it) per line,
and the blame/pickaxe tools take it into account. The downside is that
this isn't preserved over clones and fetches. That's a pretty big one.

  Another way might be to have say a magic "Reverts: commitid" line at
the last paragraph of a commit message recognized by git. The downside
is that the body of commit message might have magic meaning for some
non-core plumbing; I'm not sure how big a downside that is. For adding
it to commit header it might be a little bit too non-core, I might meet
with Linus' ethernal fury, and I'm not sure how big of a compatibility
problem would it be.

  Ideas?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: FFmpeg considering GIT

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2190 bytes --]

On Thu, 3 May 2007, Petr Baudis wrote:

> On Thu, May 03, 2007 at 08:00:16PM CEST, Uwe Kleine-König wrote:
>
>  I believe that the development scheme is largely independent on the
> version control system, except that git makes the "both ways" equally
> easy. But that doesn't mean that the "multiple people with commit
> access" scheme is wrong or anything. It has important upsides as well -
> there's no single human point of failure (_yes_, if the maintainer gets
> stuck in hospital for two months you can fork and maintain own
> repository, but then it's again just you and it is complicated socially
> etc.), the load of the maintainer is significantly lowered, and in many
> projects there is simply no "single maintainer" but a team of people
> where decisions are made by consensus.
>
>  Still, if this kind of bogus change checkins happens at any frequent
> rate in the ffmpeg project, there is a serious problem somewhere. :-)
> But I think the git way of alleviating this problem would be to have a
> way to hint the pickaxe and blame tools to ignore changes in given
> commits. So, you don't _cover up_ the messy things that happened during
> the history, but avoid in getting in the way in your view. You can still
> look it up (with git log or something) in case you'd need to (perhaps
> the revert patch was a bit complicated because of conflicting with some
> other changes, and a subtle bug was introduced; this would be thousand
> times harder to track down if you would've rewritten the history).
>
>  Would crafting up a patch to implement something like this help ffmpeg
> people in their decision?

is this needed?

wouldn't you do something like

a--b--c--d

oops, b was really bad

rebase c b

a--b--c--d
     \
      c'--d'--e--f

and you just start tagging d', e, f as the releases, logicly changing 
things to

a--b--c'--d'--e--f
     \
      c--d  dead branch

the only thing that this costs is space. unless it's a 'mess up all the 
whitespace in the entire tree' type of thing (and if it was, whoever 
did the commit would see the _huge_ diffstat and probably catch it) it's 
not likely to be a significant amount of space in the overall history.

David Lang

Re: FFmpeg considering GIT

[Petr Baudis]

On Thu, May 03, 2007 at 10:05:34PM CEST, david@lang.hm wrote:
> On Thu, 3 May 2007, Petr Baudis wrote:
> > Would crafting up a patch to implement something like this help ffmpeg
> >people in their decision?
> 
> is this needed?
> 
> wouldn't you do something like
> 
> a--b--c--d
> 
> oops, b was really bad
> 
> rebase c b
..snip..

This is immensely problematic, but I think I've fully covered all my
reservations in the other mail in this thread; if anything there was
unclear or you disagree with something I said, please reply to it.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: FFmpeg considering GIT

[Michael Niedermayer]

Petr Baudis <pasky <at> suse.cz> writes:
[...]
> > 	let me explain a little bit why this is critically needed
> > 	think of someone misstakely commiting the whole ffmpeg
> > 	reindented or mistakely commiting a old ffmpeg version over the
> > 	new or another total messup, these things do happen, and
> > 	especially if they cannot be corrected and at the time where
> > 	none of the developers is around
> > 
[...]
>   Still, if this kind of bogus change checkins happens at any frequent
> rate in the ffmpeg project, there is a serious problem somewhere. 

well, my example above was exagerated, noone ever reindented the whole
ffmpeg or checked in a old version over HEAD. what did and does occasionally
happen is that people check in several things at once (like a 100k reindenton
mixed with various functional changes)
for these we currently copy the last good version of the affected files
over the current one with svn cp and then apply the changes in nicely
split manner. (possibly without the reindention if its uneeded ...)
Another thing that happens occasionally is that complete nonsense is checked
in like checking in the wrong file or some "private" debuging code

we never use the svn cp method to revert normal buggy code ...



> But I think the git way of alleviating this problem would be to have a
> way to hint the pickaxe and blame tools to ignore changes in given
> commits. So, you don't _cover up_ the messy things that happened during
> the history, but avoid in getting in the way in your view. You can still
> look it up (with git log or something) in case you'd need to (perhaps
> the revert patch was a bit complicated because of conflicting with some
> other changes, and a subtle bug was introduced; this would be thousand
> times harder to track down if you would've rewritten the history).
> 
>   Would crafting up a patch to implement something like this help ffmpeg
> people in their decision?

well if git blame and others could somehow be told to automatically ignore
nonsense changes and matching nonsense reverts that would be great
maybe by searching for some keyword in the revert message?

ignoring all or no reverts though would again be suboptimal as that would
also ignore some reverts due to normal buggy changes

actually i think ive found an almost working solution for replacing svn cp
(though i dont know if its safe on a public repo? or if theres some other
issue with it iam missing)

ascii > testfile
git add testfile ; git commit
Created initial commit c14755cd59af4b0e6c53fb3d4bf8fa7d5aad3f3d
 1 files changed, 23 insertions(+), 0 deletions(-)
 create mode 100644 testfile

vim testfile 
git add testfile ; git commit
Created commit 0fd74c0955ae4281ac17520eabefea639f635354
 1 files changed, 2 insertions(+), 2 deletions(-)

vim testfile 
git add testfile ; git commit
Created commit d1dce0e5a20603faa0e64b722d93e847f5b80845
 1 files changed, 23 insertions(+), 23 deletions(-)

git checkout 0fd74c0955ae4281ac17520eabefea639f635354
Note: moving to "0fd74c0955ae4281ac17520eabefea639f635354" which 
isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
  git checkout -b <new_branch_name>
HEAD is now at 0fd74c0... good change

cp testfile testbak
git add testbak ; git commit
Created commit 0e55c6f422005e64fd3b73595f0fe409148d725f
 1 files changed, 23 insertions(+), 0 deletions(-)
 create mode 100644 testbak

git checkout d1dce0e5a20603faa0e64b722d93e847f5b80845
HEAD is now at d1dce0e... bad change
$git rm testfile 
rm 'testfile'

git merge 0e55c6f422005e64fd3b73595f0fe409148d725f
 100% (1/1) done
Merge made by recursive.
 testbak |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)
 create mode 100644 testbak

git mv testbak testfile 
fatal: destination exists, source=testbak, destination=testfile
git rm testfile 
rm 'testfile'
git mv testbak testfile 

git commit
Created commit ca5bcbcadb9799b0a6eaa792fae322d511ecd55f
 2 files changed, 23 insertions(+), 46 deletions(-)
 delete mode 100644 testbak

git blame -C1 -C1 -M testfile
(this just shows ca5bcbca)

vim testfile (changing a single line)
git add testfile ; git commit
Created commit 7a0a828629935ce139177fc4623a0eb9916b78fd
 1 files changed, 1 insertions(+), 1 deletions(-)

git blame -C1 -C1 -M testfile | cut -d ' ' -f 1
^c14755c
^c14755c
^c14755c
^c14755c
^c14755c
0fd74c09
0fd74c09
^c14755c
^c14755c
^c14755c
^c14755c
^c14755c
^c14755c
^c14755c
^c14755c
^c14755c
7a0a8286
ca5bcbca
ca5bcbca
ca5bcbca
ca5bcbca
ca5bcbca
ca5bcbca

this is correct except the last 6 lines

[...]

Michael

Re: FFmpeg considering GIT

[Andy Parkins]

On Friday 2007 May 04, Michael Niedermayer wrote:

> well, my example above was exagerated, noone ever reindented the whole
> ffmpeg or checked in a old version over HEAD. what did and does
> occasionally happen is that people check in several things at once (like a
> 100k reindenton mixed with various functional changes)
> for these we currently copy the last good version of the affected files
> over the current one with svn cp and then apply the changes in nicely
> split manner. (possibly without the reindention if its uneeded ...)

I might be misunderstanding, but doesn't that leave the "bad" commit in the 
history?

 * -- * -- G -- B -- !B -- 1 -- 2 -- 3

B is the bad commit; !B would be the result of the svn cp from the previous 
known-good revision, "G"; then 1, 2, and 3 would be the correctly split 
version of "B".

Have I correctly understood?  If so - git would have no trouble at all 
emulating that.  !B would actually be easier to create because you could use 
git-revert to automatically create the inverse of B.  If you wanted to only 
revert a single file, well you could use

  git-checkout G-REVISION -- file

To pull only that file out of G, and then commit that back, before starting 
the tidy up.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: FFmpeg considering GIT

[Johannes Sixt]

Andy Parkins wrote:
> On Friday 2007 May 04, Michael Niedermayer wrote:
> > for these we currently copy the last good version of the affected files
> > over the current one with svn cp and then apply the changes in nicely
> > split manner. (possibly without the reindention if its uneeded ...)
> 
> I might be misunderstanding, but doesn't that leave the "bad" commit in the
> history?

In the history? Yes.
In the blame? No.

> 
>  * -- * -- G -- B -- !B -- 1 -- 2 -- 3
> 
> B is the bad commit; !B would be the result of the svn cp from the previous
> known-good revision, "G"; then 1, 2, and 3 would be the correctly split
> version of "B".

With svn cp you actually create this "blame" history:

* -- * -- G -- B
           \
             ----- G* -- 1 -- 2 -- 3

where G* is a new revision, but since it is otherwise identical to G, it
does not introduce new blame-able lines.

-- Hannes

Re: FFmpeg considering GIT

[Florian Weimer]

* Johannes Sixt:

> Andy Parkins wrote:
>> On Friday 2007 May 04, Michael Niedermayer wrote:
>> > for these we currently copy the last good version of the affected files
>> > over the current one with svn cp and then apply the changes in nicely
>> > split manner. (possibly without the reindention if its uneeded ...)
>> 
>> I might be misunderstanding, but doesn't that leave the "bad" commit in the
>> history?
>
> In the history? Yes.
> In the blame? No.
>
>> 
>>  * -- * -- G -- B -- !B -- 1 -- 2 -- 3
>> 
>> B is the bad commit; !B would be the result of the svn cp from the previous
>> known-good revision, "G"; then 1, 2, and 3 would be the correctly split
>> version of "B".
>
> With svn cp you actually create this "blame" history:
>
> * -- * -- G -- B
>            \
>              ----- G* -- 1 -- 2 -- 3
>
> where G* is a new revision, but since it is otherwise identical to G, it
> does not introduce new blame-able lines.


With GIT, you could create:


* -- * -- G --- B
           \     \
             ---- 1 -- 2 -- 3

Or perhaps :

* -- * -- G --- B
           \     \
             ---- G* -- 1 -- 2 -- 3

How do the history viewers handle this situation?

Re: FFmpeg considering GIT

[Nicolas Pitre]

On Fri, 4 May 2007, Michael Niedermayer wrote:

> well, my example above was exagerated, noone ever reindented the whole
> ffmpeg or checked in a old version over HEAD. what did and does occasionally
> happen is that people check in several things at once (like a 100k reindenton
> mixed with various functional changes)
> for these we currently copy the last good version of the affected files
> over the current one with svn cp and then apply the changes in nicely
> split manner. (possibly without the reindention if its uneeded ...)
> Another thing that happens occasionally is that complete nonsense is checked
> in like checking in the wrong file or some "private" debuging code
> 
> we never use the svn cp method to revert normal buggy code ...

A big difference between git and svn is that git allows you to commit 
your changes individually to your local repository before pushing them 
out to the world.  With svn you make your changes visible to the world 
as soon as you commit something, including the commit screwups.

With Git you always have the opportunity to look at your commits and 
test them all together before pushing which should make commit mistakes 
obvious before they leave your machine.  If a mistake happened in one of 
those commits you can ammend them, rebase them, etc. and only push when 
they're satisfactory, something that svn doesn't allow.

So I think that something that you got used to with svn simply has no 
serious need for with git.


Nicolas

Re: FFmpeg considering GIT

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 4182 bytes --]

On Fri, 4 May 2007 13:46:28 +0000 (UTC), Michael Niedermayer wrote:
> well, my example above was exagerated, noone ever reindented the whole
> ffmpeg or checked in a old version over HEAD. what did and does occasionally
> happen is that people check in several things at once (like a 100k reindenton
> mixed with various functional changes)

That sounds like an opportunity to educate your contributors a bit on
what good commits should look like. So I think this is more a social
issue than a technical issue, (but git has some technical means that
make it much easier to address the social issues).

Your description above makes an assumption that there is a single
central repository that multiple people push changes into, (which is
really the only way to organize a project with svn or cvs). And with
those systems all you get is a bit than you can flip on for whether
you trust someone to push changes into the repository or not. But git
is much more flexible than that.

The opposite extreme is to organize the project in a way similar to
the linux kernel---all contributors maintain their own repositories
and things get merged only when a maintainer reviews and pulls. With
this approach, garbage never lands in your own repository by
definition, (since you don't pull if it looks like garbage to you). So
that solves the problem, but this organization might seem too radical
a shift for your project.

Fortunately, git is flexible enough to do things in between as
well. For example, you can have a central repository where multiple
people push changes, and also have personal repositories. Git reduces
the cost of creating a new personal repository to basically zero, so
you can use these quite freely. They make a great place for new
contributors to publish changes where the more experienced maintainers
can review and educate the new contributors on mistakes like you
describe above.

So with this, you can let people play in their own repositories while
they're still learning the cultural aspects of what code should look
like. I've found that new contributors really like the freedom this
gives them, (there's no fear that they are going to break anything
this way, since they are relying on others to review and pull at
first). So the trust relationship can grow as you work together,
(which is how it should be).

And that whole relationship-building happens while you're both
benefiting from the support of the tool, (not like cvs or svn where
the new contributor is cut off from almost all help from the tool
until you flip the "absolute trust" bit).

> well if git blame and others could somehow be told to automatically ignore
> nonsense changes and matching nonsense reverts that would be great
> maybe by searching for some keyword in the revert message?

That sounds like a bad technical workaround for a problem that really
shouldn't exist. You should look for ways to create the history you'd
really like to have rather than trying to find a way to get the tool
to ignore the history that's actually there.

Sure, mistakes will happen. Just learn to live with that.

Oh, and I also think the emphasis on "blame" is due to a lack of other
more powerful history exploration features in other systems. For
example, the fact that "git log" can filter based on subsets of the
directory tree:

	git log -p -- some/directory

or by content of the patches themselves:

	git log -p -S'snippet of interest'
	[*]

is often just plain more powerful than blame is, and it makes it quite
trivial to skip past any noise, (since you get the complete history of
what you care about, not just information about the last time a line
in a file got touched).

For example, I often use git-log to find out what happened to code
that used to be in the file, but doesn't appear there anymore. That's
simple to do with git log, (sometimes even just plain "git log -p" and
searching with the pager), but it's something that something like cvs
or svn blame just fundametally cannot even help with.

-Carl

[*] I just noticed that -S isn't mentioned in the documentation for
git-log at all, (though, oddly enough, a 'git log -S' example is
provided in the git-blame documentation).

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Johan Herland]

[-- Attachment #1: Type: text/plain, Size: 480 bytes --]

On Friday 04 May 2007, Carl Worth wrote:
> [*] I just noticed that -S isn't mentioned in the documentation for
> git-log at all, (though, oddly enough, a 'git log -S' example is
> provided in the git-blame documentation).

It's also used in an example in the User Manual (Chapter 1. Git Quick 
Start -- Exploring history). I was also surprised that it wasn't 
mentioned in the git-log manual page.


...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: FFmpeg considering GIT

[Michael Niedermayer]

[-- Attachment #1: Type: text/plain, Size: 4193 bytes --]

Hi

On Fri, May 04, 2007 at 11:17:05AM -0700, Carl Worth wrote:
> On Fri, 4 May 2007 13:46:28 +0000 (UTC), Michael Niedermayer wrote:
> > well, my example above was exagerated, noone ever reindented the whole
> > ffmpeg or checked in a old version over HEAD. what did and does occasionally
> > happen is that people check in several things at once (like a 100k reindenton
> > mixed with various functional changes)
> 
> That sounds like an opportunity to educate your contributors a bit on
> what good commits should look like. 

we have a nice svn policy which explains that, also people wont receive
write access without having submitted a few clean patches first
so i dont know if more education would really help, the problems are IMHO
rather caused by a mix of lazyness, arrogance and plain oversight
but please dont missunderstand, these problems are not that common, its
rather once every few month


> So I think this is more a social
> issue than a technical issue, 

yes i think so too, the added push after commit wont stop a bad commit
as the developer already saw the change when running svn diff ...


> (but git has some technical means that
> make it much easier to address the social issues).
> 
> Your description above makes an assumption that there is a single
> central repository that multiple people push changes into, (which is
> really the only way to organize a project with svn or cvs). And with
> those systems all you get is a bit than you can flip on for whether
> you trust someone to push changes into the repository or not. But git
> is much more flexible than that.
> 
> The opposite extreme is to organize the project in a way similar to
> the linux kernel---all contributors maintain their own repositories
> and things get merged only when a maintainer reviews and pulls. With
> this approach, garbage never lands in your own repository by
> definition, (since you don't pull if it looks like garbage to you). So
> that solves the problem, but this organization might seem too radical
> a shift for your project.

yes, id like to switch ffmpeg to git or mercurial as that seems like a
good idea and many of our developers seem to want it, the question
about the organization is a different thing, not a single ffmpeg 
developer suggested to change the current "every developer has write access"
system, actually its even more than just that, almost every mplayer
developer has technically write access to ffmpeg and almost every ffmpeg
developer has technically write access to mplayer and this has never
caused a problem ...

also its kinda nice to review a patch and reply with "looks ok" and
someone else applies the patch locally, tests it extensively and
commits it, it reduces the work for reviewers ...


[...]

> 
> > well if git blame and others could somehow be told to automatically ignore
> > nonsense changes and matching nonsense reverts that would be great
> > maybe by searching for some keyword in the revert message?
> 
> That sounds like a bad technical workaround for a problem that really
> shouldn't exist. You should look for ways to create the history you'd
> really like to have rather than trying to find a way to get the tool
> to ignore the history that's actually there.
> 
> Sure, mistakes will happen. Just learn to live with that.

btw, that leads me to another minor issue, i think commit log
messages cannot be changed in git after they are public, while we
commonly did change them to improve them, the issue simply is that some
developers are not good at writing nice commit log messages, sometimes
due them being plain bad in english or bad at writing descriptive
log messages ...

also our docs team loves to correct spelling errors in the commit messages
not that i consider that of any importance :)


> 
> Oh, and I also think the emphasis on "blame" is due to a lack of other
> more powerful history exploration features in other systems. For

yes

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In a rich man's house there is no place to spit but his face.
-- Diogenes of Sinope

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Linus Torvalds]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2895 bytes --]



On Fri, 4 May 2007, Michael Niedermayer wrote:
> 
> we have a nice svn policy which explains that, also people wont receive
> write access without having submitted a few clean patches first
> so i dont know if more education would really help, the problems are IMHO
> rather caused by a mix of lazyness, arrogance and plain oversight
> but please dont missunderstand, these problems are not that common, its
> rather once every few month

[ I was away for a few days, so others probably answered already ... ]

With git, the right way to do thigns is to not ever give "write access" to 
the "standard" tree to developers, but to make each developer have their 
own tree, and then one or more developers are the ones that merge other 
peoples work. 

Since I'm the one who does the merging for the kernel, I've made damn sure 
that merging other peoples work is as easy as humanly possible, so that I 
can just sit there, sipping my foofy tropical drink, drunk as a skunk and 
enjoying every moment of seeing my peons work their little fingers to the 
bone, when I do a "git pull ..." and in two seconds I've downloaded their 
work and merged it, and I can take another sip of the Piña Colada. 

Burp.

And git also makes it really easy to see when somebody does something 
stupid. The one thing it always shows to the person doing the merging is 
the diffstat from the result, so if somebody re-indented the source base, 
the merger goes "Whaa", and assuming he's not too drunk to type, he should 
just send a sternly worded message to the developer who did the bad deed, 
and tell them that their work was unacceptable, and won't be pulled.

A simple "git reset --hard ORIG_HEAD" will undo the merge, so the 
person(s) who actually does the integration again doesn't actually have to 
work all that hard.

In other words, the proper sequence really should be to *not* let the 
horribly buggy commits into the standard version in the first place! Sure, 
individual developers will make mistakes, but the fact that they screwed 
up should in _no_ way mean that they can screw up the main repository. The 
whole point in being distributed is that developers can screw up in their 
own _private_ repositories and still have all the power of a proper SCM 
tool, but without actually getting to screw up the main repo.

(And yes, then very occasionally both the developer *and* the maintainer 
screws up, and something bad gets through, and yeah, then you need to 
revert, but the point I'm arguing is that with a fairly good flow of 
development, you don't have to worry about the more clueless people 
screwing up - they can still do development, and you can still pull from 
them, but *if* they screw up, you can tell them to clean up their mess 
*before* you actually put it into any standard tree, and the mess can be 
entirely their _local_ mistake and never visible anywhere else).

			Linus

Re: FFmpeg considering GIT

[Karl Hasselström]

On 2007-05-04 11:17:05 -0700, Carl Worth wrote:

> or by content of the patches themselves:
>
>       git log -p -S'snippet of interest'

Somewhat unrelated: how can I make gitk display these (and only these)
commits? git-log is not bad, but in 95% of cases I find gitk easier to
use.

I know that I can ask it to highlight commits that insert or remove
"snippet of interest", but frequently the highlighted commits are ten
out of ten thousand, and not that easy to find even when boldfaced.
What I want is to make it display only those commits.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Linus Torvalds]

On Sat, 5 May 2007, Karl Hasselstr?m wrote:
>
> On 2007-05-04 11:17:05 -0700, Carl Worth wrote: 
> > or by content of the patches themselves:
> >
> >       git log -p -S'snippet of interest'
> 
> Somewhat unrelated: how can I make gitk display these (and only these)
> commits? git-log is not bad, but in 95% of cases I find gitk easier to
> use.
> 
> I know that I can ask it to highlight commits that insert or remove
> "snippet of interest", but frequently the highlighted commits are ten
> out of ten thousand, and not that easy to find even when boldfaced.
> What I want is to make it display only those commits.

The "-S" thing doesn't really interact well with "gitk", because it 
doesn't rewrite the parent information (it is basically just a "hide 
commits that don't pass this criteria"). As such, gitk, which requires 
parent information to generate the graph, is not very amenable to using 
"-S" and such.

That said, you can apply this fairly trivial patch to "gitk" to make it 
parse the output of "git log" rather than "git rev-list", and that will 
actually get you working -S'xyz' parsing automatically. It's just that the 
commit history window will look like crap.

This patch may be worth applying regardless, since there is really no real 
reason to use "git rev-list". In fact, I really like the ability to say

	gitk --stat

and have the diffstat output visible in the commit window automatically ;)

We might want to teach people that "git rev-list" isn't really all that 
useful any more, at least with the fancy stuff (it's still useful for just 
generating a list of objects, and for doing things like

	git rev-list v2.6.21.. | wc -l

just to count commits).

Junio, Paul?

		Linus
---
 gitk |    9 +++++----
 1 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/gitk b/gitk
index b1c65d7..bec7bb9 100755
--- a/gitk
+++ b/gitk
@@ -33,8 +33,8 @@ proc start_rev_list {view} {
 	set order "--date-order"
     }
     if {[catch {
-	set fd [open [concat | git rev-list --header $order \
-			  --parents --boundary --default HEAD $args] r]
+	set fd [open [concat | git log -z --pretty=raw $order \
+			  --parents --boundary $args] r]
     } err]} {
 	puts stderr "Error executing git rev-list: $err"
 	exit 1
@@ -129,7 +129,8 @@ proc getcommitlines {fd view}  {
 	set ok 0
 	set listed 1
 	if {$j >= 0} {
-	    set ids [string range $cmit 0 [expr {$j - 1}]]
+	    # start with 'commit '
+	    set ids [string range $cmit 6 [expr {$j - 1}]]
 	    if {[string range $ids 0 0] == "-"} {
 		set listed 0
 		set ids [string range $ids 1 end]
@@ -147,7 +148,7 @@ proc getcommitlines {fd view}  {
 	    if {[string length $shortcmit] > 80} {
 		set shortcmit "[string range $shortcmit 0 80]..."
 	    }
-	    error_popup "Can't parse git rev-list output: {$shortcmit}"
+	    error_popup "Can't parse git git log output: {$shortcmit}"
 	    exit 1
 	}
 	set id [lindex $ids 0]

Re: FFmpeg considering GIT

[Linus Torvalds]

On Sat, 5 May 2007, Linus Torvalds wrote:
> 
> This patch may be worth applying regardless, since there is really no real 
> reason to use "git rev-list". In fact, I really like the ability to say
> 
> 	gitk --stat
> 
> and have the diffstat output visible in the commit window automatically ;)

Btw, testing this a bit more actually shows what I would consider a real 
buglet in "git log --boundary": the option would be honoured only if 
"left-right" was enabled.

This patch fixes "git log --boundary" to actually show the "-" in front of 
a commit name regardless of whether you _also_ asked for left-right.

(It also shows that my "gitk" patch was incorrectly getting the commit 
name from character 6 onward, even though it should have been 7, but I'll 
also try to make gitk understand the "<" and ">" markers, and make it 
possible to say

	gitk --left-right a...b

and have the commits colored appropriately. That would be cool, but it 
might need more tcl/tk knowledge than I actually possess).

		Linus

---
diff --git a/log-tree.c b/log-tree.c
index c679324..4bef909 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -244,10 +244,10 @@ void show_log(struct rev_info *opt, const char *sep)
 		      stdout);
 		if (opt->commit_format != CMIT_FMT_ONELINE)
 			fputs("commit ", stdout);
-		if (opt->left_right) {
-			if (commit->object.flags & BOUNDARY)
-				putchar('-');
-			else if (commit->object.flags & SYMMETRIC_LEFT)
+		if (commit->object.flags & BOUNDARY)
+			putchar('-');
+		else if (opt->left_right) {
+			if (commit->object.flags & SYMMETRIC_LEFT)
 				putchar('<');
 			else
 				putchar('>');

Re: FFmpeg considering GIT

[Linus Torvalds]

On Sat, 5 May 2007, Linus Torvalds wrote:
> 
> (It also shows that my "gitk" patch was incorrectly getting the commit 
> name from character 6 onward, even though it should have been 7, but I'll 
> also try to make gitk understand the "<" and ">" markers, and make it 
> possible to say
> 
> 	gitk --left-right a...b
> 
> and have the commits colored appropriately. That would be cool, but it 
> might need more tcl/tk knowledge than I actually possess).

Ok, that turned out to be the case.

Here's an updated patch to gitk, which at least *parses* the 
"--left-right" data properly, it just doesn't use it. But with the fix to 
"git log" I just posted, and this, you at least have the same capabilities 
gitk used to have, and it should be fairly easy for somebody who knows 
tcltk to squirrel away the "leftright" data per commit and use that to 
color the commit lines in the top-most pane.

I'm also sure the "if first character is one of '-'/'<'/'>'" test can be 
written more prettily, rather than have three if-statements on it. 

Finally, it realy _should_ check that the first 7 characters of the commit 
log (the ones it ignores by just asking for substring 7..) are actually 
the exact characters "commit ", but I'll blame my lack of comfort with the 
language again.

Somebody? Please? It really should be pretty cool. Do

	gitk --left-right commit^1...commit^2

for an appropriate 'commit' that is a merge, and the two sides getting 
merged should show up with different colors!

		Linus

----
 gitk |   18 ++++++++++++++----
 1 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/gitk b/gitk
index b1c65d7..0bf00ee 100755
--- a/gitk
+++ b/gitk
@@ -33,8 +33,8 @@ proc start_rev_list {view} {
 	set order "--date-order"
     }
     if {[catch {
-	set fd [open [concat | git rev-list --header $order \
-			  --parents --boundary --default HEAD $args] r]
+	set fd [open [concat | git log -z --pretty=raw $order \
+			  --parents --boundary $args] r]
     } err]} {
 	puts stderr "Error executing git rev-list: $err"
 	exit 1
@@ -127,13 +127,23 @@ proc getcommitlines {fd view}  {
 	set start [expr {$i + 1}]
 	set j [string first "\n" $cmit]
 	set ok 0
+	set leftright 0
 	set listed 1
 	if {$j >= 0} {
-	    set ids [string range $cmit 0 [expr {$j - 1}]]
+	    # start with 'commit '
+	    set ids [string range $cmit 7 [expr {$j - 1}]]
 	    if {[string range $ids 0 0] == "-"} {
 		set listed 0
 		set ids [string range $ids 1 end]
 	    }
+	    if {[string range $ids 0 0] == "<"} {
+		set leftright -1
+		set ids [string range $ids 1 end]
+	    }
+	    if {[string range $ids 0 0] == ">"} {
+		set leftright 1
+		set ids [string range $ids 1 end]
+	    }
 	    set ok 1
 	    foreach id $ids {
 		if {[string length $id] != 40} {
@@ -147,7 +157,7 @@ proc getcommitlines {fd view}  {
 	    if {[string length $shortcmit] > 80} {
 		set shortcmit "[string range $shortcmit 0 80]..."
 	    }
-	    error_popup "Can't parse git rev-list output: {$shortcmit}"
+	    error_popup "Can't parse git git log output: {$shortcmit}"
 	    exit 1
 	}
 	set id [lindex $ids 0]

Re: FFmpeg considering GIT

[Junio C Hamano]

Linus Torvalds <torvalds@linux-foundation.org> writes:

>> and have the commits colored appropriately. That would be cool, but it 
>> might need more tcl/tk knowledge than I actually possess).
>
> Ok, that turned out to be the case.
>
> Here's an updated patch to gitk, which at least *parses* the 
> "--left-right" data properly, it just doesn't use it.

This on top of yours makes it use it.

 gitk |   31 ++++++++++++++++++++++++++-----
 1 files changed, 26 insertions(+), 5 deletions(-)

diff --git a/gitk b/gitk
index 0bf00ee..a6e762d 100755
--- a/gitk
+++ b/gitk
@@ -74,7 +74,7 @@ proc getcommits {} {
 proc getcommitlines {fd view}  {
     global commitlisted nextupdate
     global leftover commfd
-    global displayorder commitidx commitrow commitdata
+    global displayorder commitidx commitrow commitdata commitside
     global parentlist childlist children curview hlview
     global vparentlist vchildlist vdisporder vcmitlisted
 
@@ -178,6 +178,7 @@ proc getcommitlines {fd view}  {
 	}
 	set commitdata($id) [string range $cmit [expr {$j + 1}] end]
 	set commitrow($view,$id) $commitidx($view)
+	set commitside($id) $leftright
 	incr commitidx($view)
 	if {$view == $curview} {
 	    lappend parentlist $olds
@@ -2986,7 +2987,7 @@ proc drawlines {id} {
 
 proc drawcmittext {id row col rmx} {
     global linespc canv canv2 canv3 canvy0 fgcolor
-    global commitlisted commitinfo rowidlist
+    global commitlisted commitinfo commitside rowidlist
     global rowtextx idpos idtags idheads idotherrefs
     global linehtag linentag linedtag
     global mainfont canvxmax boldrows boldnamerows fgcolor
@@ -2995,9 +2996,29 @@ proc drawcmittext {id row col rmx} {
     set x [xc $row $col]
     set y [yc $row]
     set orad [expr {$linespc / 3}]
-    set t [$canv create oval [expr {$x - $orad}] [expr {$y - $orad}] \
-	       [expr {$x + $orad - 1}] [expr {$y + $orad - 1}] \
-	       -fill $ofill -outline $fgcolor -width 1 -tags circle]
+
+    if {[info exists commitside($id)]} {
+	set leftright $commitside($id)
+    } else {
+	set leftright 0
+    }
+    if {$leftright == 0} {
+	set t [$canv create oval [expr {$x - $orad}] [expr {$y - $orad}] \
+		   [expr {$x + $orad - 1}] [expr {$y + $orad - 1}] \
+		   -fill $ofill -outline $fgcolor -width 1 -tags circle]
+    } elseif {$leftright < 0} {
+	set t [$canv create polygon \
+		   [expr {$x - $orad}] $y \
+		   [expr {$x + $orad - 1}] [expr {$y - $orad}] \
+		   [expr {$x + $orad - 1}] [expr {$y + $orad - 1}] \
+		   -fill $ofill -outline $fgcolor -width 1 -tags circle]
+    } else {
+	set t [$canv create polygon \
+		   [expr {$x + $orad - 1}] $y \
+		   [expr {$x - $orad}] [expr {$y - $orad}] \
+		   [expr {$x - $orad}] [expr {$y + $orad - 1}] \
+		   -fill $ofill -outline $fgcolor -width 1 -tags circle]
+    }
     $canv raise $t
     $canv bind $t <1> {selcanvline {} %x %y}
     set xt [xc $row [llength [lindex $rowidlist $row]]]

Re: FFmpeg considering GIT

[Paul Mackerras]

Linus Torvalds writes:

> Finally, it realy _should_ check that the first 7 characters of the commit 
> log (the ones it ignores by just asking for substring 7..) are actually 
> the exact characters "commit ", but I'll blame my lack of comfort with the 
> language again.

I have thought about rewriting it in a different language, but I
haven't found anything that really appeals.  I don't want to go to
C/GTK or C/Qt since that would make it hard to port to Windows and
MacOS AFAIK.  Python/Tk would be a possibility, but I have never
learnt python and I'm actually not all that comfortable with having to
do things the object-oriented way.

Any suggestions?

Tcl/Tk does come with a comprehensive set of man pages, usually
installed either in section n or sections 3tcl and 3tk.  So you can do
"man string" to find out how to do string manipulations, for instance.
The syntax is quite regular and is explained in the "Tcl" man page.

Paul.

Re: FFmpeg considering GIT

[Karl Hasselström]

On 2007-05-07 22:13:44 +1000, Paul Mackerras wrote:

> I have thought about rewriting it in a different language, but I
> haven't found anything that really appeals. I don't want to go to
> C/GTK or C/Qt since that would make it hard to port to Windows and
> MacOS AFAIK. Python/Tk would be a possibility, but I have never
> learnt python and I'm actually not all that comfortable with having
> to do things the object-oriented way.
>
> Any suggestions?

wxWidgets (http://www.wxwidgets.org/) is a cross-platform C++ library
that seems popular. There are bindings for lots of languages,
including Python (http://www.wxpython.org/).

  "wxWidgets lets developers create applications for Win32, Mac OS X,
  GTK+, X11, Motif, WinCE, and more using one codebase. It can be used
  from languages such as C++, Python, Perl, and C#/.NET."

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Johan Herland]

[-- Attachment #1: Type: text/plain, Size: 386 bytes --]

On Monday 07 May 2007, Paul Mackerras wrote:
> I don't want to go to C/Qt since that would make it hard to port to
> Windows and MacOS AFAIK.

Wrong. Qt is now GPL for Windows, Linux, Unix, and Mac OS X. See 
http://www.trolltech.com/products/qt/licenses/licensing/opensource for more 
info.


Have fun!

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Alex Riesen]

On 5/7/07, Paul Mackerras <paulus@samba.org> wrote:
> I have thought about rewriting it in a different language, but I
> haven't found anything that really appeals.  I don't want to go to
> C/GTK or C/Qt since that would make it hard to port to Windows and
> MacOS AFAIK.  Python/Tk would be a possibility, but I have never

C++/Qt4 is ported to Windows.

> Any suggestions?

It(Qt4) wasn't a suggestion though. I still consider Tcl/Tk more portable.

Re: FFmpeg considering GIT

[Marco Costalba]

On 5/7/07, Alex Riesen <raa.lkml@gmail.com> wrote:
> On 5/7/07, Paul Mackerras <paulus@samba.org> wrote:
> > I have thought about rewriting it in a different language, but I
> > haven't found anything that really appeals.  I don't want to go to
> > C/GTK or C/Qt since that would make it hard to port to Windows and
> > MacOS AFAIK.  Python/Tk would be a possibility, but I have never
>
> C++/Qt4 is ported to Windows.
>
> > Any suggestions?
>
> It(Qt4) wasn't a suggestion though. I still consider Tcl/Tk more portable.
> -

Qt4 is a very good designed and elegant API and it works perfectly
under Windows, see current porting of qgit under Windows:

              git://git.kernel.org/pub/scm/qgit/qgit4


Language to use is C++, not C (much more powerful IMHO)

One thing that you should note is that if you go Qt4/C++ you can
forget to have it all in one file.

I think you should consider the new gitk 'footprint' becuase if you
leave TCL, any windows library you use will force you to create a
multi file project.

 Marco

P.S: If you choose Qt/C++ (the best technically speaking ;-)  please
you could consider starting from an already laid out code base instead
of starting from scratch.
As example, hmmmm, I think there is one called 'qgit', if I remember
correctly. It's nice and very very very fast.

Re: FFmpeg considering GIT

[Paul Mackerras]

Marco Costalba writes:

> Language to use is C++, not C (much more powerful IMHO)

Sorry, C++ is not an option because I dislike it so much.  The main
reason for changing languages would be to enable people like Linus to
hack on it more easily, and I don't think C++ would achieve that.

> P.S: If you choose Qt/C++ (the best technically speaking ;-)  please
> you could consider starting from an already laid out code base instead
> of starting from scratch.
> As example, hmmmm, I think there is one called 'qgit', if I remember
> correctly. It's nice and very very very fast.

Yes, but isn't there already a talented hacker working on that? :)

Paul.

Re: FFmpeg considering GIT

[Marco Costalba]

On 5/9/07, Paul Mackerras <paulus@samba.org> wrote:
> Marco Costalba writes:
>
> > Language to use is C++, not C (much more powerful IMHO)
>
> Sorry, C++ is not an option because I dislike it so much.

Well, speaking about GUI applications, the 90% is in the graphic
library and only in small part in the language. With Qt we are at 95%

Anyhow does exist also python bindings for Qt.


>  The main
> reason for changing languages would be to enable people like Linus to
> hack on it more easily, and I don't think C++ would achieve that.
>

Poor Linus ;-)

I think the design of the application states the easiness of changes,
spaghetti code and bad designed functions are much worst then any ugly
language you can think about.

That's for substantial changes,  for one liners or for adding little
features encapsulation and modularity of the code are the magical
words here, and an object oriented language *could* help achieving
that.


> > P.S: If you choose Qt/C++ (the best technically speaking ;-)  please
> > you could consider starting from an already laid out code base instead
> > of starting from scratch.
> > As example, hmmmm, I think there is one called 'qgit', if I remember
> > correctly. It's nice and very very very fast.
>
> Yes, but isn't there already a talented hacker working on that? :)
>

Two is better then one :-)


  Marco

Re: FFmpeg considering GIT

[Robin Rosenberg]

onsdag 09 maj 2007 skrev Marco Costalba:
> On 5/9/07, Paul Mackerras <paulus@samba.org> wrote:
> > Marco Costalba writes:
> >
> > > Language to use is C++, not C (much more powerful IMHO)
> >
> > Sorry, C++ is not an option because I dislike it so much.
> 
> Well, speaking about GUI applications, the 90% is in the graphic
> library and only in small part in the language. With Qt we are at 95%
> 
> Anyhow does exist also python bindings for Qt.

Yes, there is package called PyQT (GPL) here:  http://www.riverbankcomputing.co.uk/pyqt/

There are bindings for most languages, even a Java binding.

-- robin

Re: FFmpeg considering GIT

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 881 bytes --]

On Wed, May 09, 2007 at 08:38:48 +0200, Marco Costalba wrote:
> On 5/9/07, Paul Mackerras <paulus@samba.org> wrote:
> >Marco Costalba writes:
> >
> >> Language to use is C++, not C (much more powerful IMHO)
> >
> >Sorry, C++ is not an option because I dislike it so much.
> 
> Well, speaking about GUI applications, the 90% is in the graphic
> library and only in small part in the language. With Qt we are at 95%
> 
> Anyhow does exist also python bindings for Qt.

Tried them, beed deeply disapointed. Qt always destroys all child objects
with the parent, which is OK in C++, but does not play well with
garbage-collection. And the python bindings (ruby ones seem to be better)
fail to check reference validity, so you can quite easily segfault the python
interpreter. Gtk plays much better with dynamic languages.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Fredrik Kuivinen]

On 5/9/07, Jan Hudec <bulb@ucw.cz> wrote:
> On Wed, May 09, 2007 at 08:38:48 +0200, Marco Costalba wrote:
> > On 5/9/07, Paul Mackerras <paulus@samba.org> wrote:
> > >Marco Costalba writes:
> > >
> > >> Language to use is C++, not C (much more powerful IMHO)
> > >
> > >Sorry, C++ is not an option because I dislike it so much.
> >
> > Well, speaking about GUI applications, the 90% is in the graphic
> > library and only in small part in the language. With Qt we are at 95%
> >
> > Anyhow does exist also python bindings for Qt.
>
> Tried them, beed deeply disapointed. Qt always destroys all child objects
> with the parent, which is OK in C++, but does not play well with
> garbage-collection. And the python bindings (ruby ones seem to be better)
> fail to check reference validity, so you can quite easily segfault the python
> interpreter. Gtk plays much better with dynamic languages.

I have used PyQt for some smaller projects (notably Hgct, a no longer developed
commit tool for git and Mercurial. See
http://repo.or.cz/w/hgct.git?a=tree). For me
PyQt has worked very well. The python interface to Qt is more or less a direct
translation of the C++ interface, so the excellent documentation troll
tech provides
for Qt can be used when developing with PyQt as well.

I have never seen the segfaulting you mention. Maybe my programs have been too
small to trigger that bug...

- Fredrik

Re: FFmpeg considering GIT

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

On Wed, May 09, 2007 at 23:09:25 +0200, Fredrik Kuivinen wrote:
> I have used PyQt for some smaller projects (notably Hgct, a no longer 
> developed
> commit tool for git and Mercurial. See
> http://repo.or.cz/w/hgct.git?a=tree). For me
> PyQt has worked very well. The python interface to Qt is more or less a 
> direct
> translation of the C++ interface, so the excellent documentation troll
> tech provides
> for Qt can be used when developing with PyQt as well.
> 
> I have never seen the segfaulting you mention. Maybe my programs have been 
> too
> small to trigger that bug...

It's not about size of the programs. It's about having to be careful not to
refer to widgets inside eg. dialog box from outside and close that dialog
box. That is having to be careful about something, that is normal in C++, but
what you normally expect python to handle for you. And such quirks of the
bindings are completely undocumented.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Marco Costalba]

On 5/9/07, Jan Hudec <bulb@ucw.cz> wrote:
> On Wed, May 09, 2007 at 23:09:25 +0200, Fredrik Kuivinen wrote:
> > I have used PyQt for some smaller projects (notably Hgct, a no longer
> > developed
> > commit tool for git and Mercurial. See
> > http://repo.or.cz/w/hgct.git?a=tree). For me
> > PyQt has worked very well. The python interface to Qt is more or less a
> > direct
> > translation of the C++ interface, so the excellent documentation troll
> > tech provides
> > for Qt can be used when developing with PyQt as well.
> >
> > I have never seen the segfaulting you mention. Maybe my programs have been
> > too
> > small to trigger that bug...
>
> It's not about size of the programs. It's about having to be careful not to
> refer to widgets inside eg. dialog box from outside and close that dialog
> box.

In Qt all the classes that ineriths from QObject are memory managed,
to be more clear
you can say that one class is "child" of another class (always
ineritherd from QObject) that becames the parent.

When you delete the parent, all his children are deleted too, this is
a (big) feature to avoid
missing free() calls for resources created with mallocs() , (well, in
C++ we say 'delete' for resources created by 'new' but the concept is
more or less the same).

Note that this property can be nested: create a main window, inside a
window there is a tab form, inside the tab there is a list view,
inside the list view there are items (lines of list view).

So *when* you delete the main window all this stuff is automatically
deleted by Qt. It is diffrent from a garbage collector because there
is no delay in releasing memory and all the thing is strict
deterministic.

So coming to your problem, if you need to refer to a widget inside a
dialog *after* the dialog has been deleted you can simply reparent to
NULL the widget before closing the dialog so to remove your object
from the delete list of the dialog.

Another option, in case your obect is not a graphical widget, is to
avoid declaring your object "child" of the dialog in first instance
setting his parent to NULL. This is clearly better because documents
'in code' also the real relationship between the dialog and your
object.

  Marco

Re: FFmpeg considering GIT

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1995 bytes --]

On Thu, May 10, 2007 at 13:20:02 +0200, Marco Costalba wrote:
> On 5/9/07, Jan Hudec <bulb@ucw.cz> wrote:
> >On Wed, May 09, 2007 at 23:09:25 +0200, Fredrik Kuivinen wrote:
> >> I have used PyQt for some smaller projects (notably Hgct, a no longer
> >> developed
> >> commit tool for git and Mercurial. See
> >> http://repo.or.cz/w/hgct.git?a=tree). For me
> >> PyQt has worked very well. The python interface to Qt is more or less a
> >> direct
> >> translation of the C++ interface, so the excellent documentation troll
> >> tech provides
> >> for Qt can be used when developing with PyQt as well.
> >>
> >> I have never seen the segfaulting you mention. Maybe my programs have 
> >been
> >> too
> >> small to trigger that bug...
> >
> >It's not about size of the programs. It's about having to be careful not to
> >refer to widgets inside eg. dialog box from outside and close that dialog
> >box.
> 
> In Qt all the classes that ineriths from QObject are memory managed,
> to be more clear
> you can say that one class is "child" of another class (always
> ineritherd from QObject) that becames the parent.
> 
> When you delete the parent, all his children are deleted too, this is
> a (big) feature to avoid
> missing free() calls for resources created with mallocs() , (well, in
> C++ we say 'delete' for resources created by 'new' but the concept is
> more or less the same).

I know well how it works. And while it is definitely a nice feature in C++
(though it can't beat well done reference-counting smart pointers as Gtkmm
has), it is a gross misfeature in any dynamic language.

And no, I am not objecting to existence of that system -- it's useful in C++.
What I say is, that the PyQt bindings are buggy because it completely
fails to make this feature compatible with python memory management - python
program should not be able to segfault the interpreter no matter how buggy
that program is.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1363 bytes --]

On Mon, May 07, 2007 at 22:13:44 +1000, Paul Mackerras wrote:
> Linus Torvalds writes:
> 
> > Finally, it realy _should_ check that the first 7 characters of the commit 
> > log (the ones it ignores by just asking for substring 7..) are actually 
> > the exact characters "commit ", but I'll blame my lack of comfort with the 
> > language again.
> 
> I have thought about rewriting it in a different language, but I
> haven't found anything that really appeals.  I don't want to go to
> C/GTK or C/Qt since that would make it hard to port to Windows and

C/Gtk would be perfectly portable. As would C++/Gtk, Python/Gtk and Perl/Gtk.
C++/Qt4 would be perfectly portable as well, so choose whichever you find
easier to work with. For C/C++ they are on par, for Python/Perl/Ruby I think
Gtk has better bindings.

> MacOS AFAIK.  Python/Tk would be a possibility, but I have never
> learnt python and I'm actually not all that comfortable with having to
> do things the object-oriented way.

I would actually recommend against Python/Tk, because (tried it) py2exe does
not work seem to work with it, so you couldn't wrap it to easy to install
binary for windows folks. I did not try Python/Gtk, but I expect you might
have better luck with it (it's the Tcl/Tk interpreter that causes problems).

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: FFmpeg considering GIT

[Gábor Farkas]

Jan Hudec wrote:
> On Mon, May 07, 2007 at 22:13:44 +1000, Paul Mackerras wrote:
>> I have thought about rewriting it in a different language, but I
>> haven't found anything that really appeals.  I don't want to go to
>> C/GTK or C/Qt since that would make it hard to port to Windows and
> 
> C/Gtk would be perfectly portable. As would C++/Gtk, Python/Gtk and Perl/Gtk.
> C++/Qt4 would be perfectly portable as well, so choose whichever you find
> easier to work with. For C/C++ they are on par, for Python/Perl/Ruby I think
> Gtk has better bindings.
> 
>> MacOS AFAIK.

GTK does not work natively on OSX (it only works using the X11 server ).
QT works fine.

gabor

Re: FFmpeg considering GIT

[Randal L. Schwartz]

>>>>> "Gábor" == Gábor Farkas <gabor@nekomancer.net> writes:

Gábor> GTK does not work natively on OSX (it only works using the X11 server ).

But "Tk" works fine. I don't fire up X11, except rarely.  And gitk works fine.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

Re: FFmpeg considering GIT

[Gábor Farkas]

Nguyen Thai Ngoc Duy wrote:
> On 5/7/07, Gábor Farkas <gabor@nekomancer.net> wrote:
>> >> MacOS AFAIK.
>>
>> GTK does not work natively on OSX (it only works using the X11 server ).
>> QT works fine.
>>
>> gabor
> 
> Recent GTK+ can work on OSX without X11:
> http://developer.imendio.com/projects/gtk-macosx
> 

i know about that project, but it does not seem to be finished yet.

for example, in 
http://developer.imendio.com/projects/gtk-macosx/build-instructions

they write:



NOTE: This is mainly meant for developers wanting to help out with GTK+ 
Mac OS X, not for users. The port is not yet finished or usable for 
mainstream use.



gabor

Re: FFmpeg considering GIT

[Junio C Hamano]

Paul Mackerras <paulus@samba.org> writes:

> Linus Torvalds writes:
>
>> Finally, it realy _should_ check that the first 7 characters of the commit 
>> log (the ones it ignores by just asking for substring 7..) are actually 
>> the exact characters "commit ", but I'll blame my lack of comfort with the 
>> language again.
>
> I have thought about rewriting it in a different language, but I
> haven't found anything that really appeals.  I don't want to go to
> C/GTK or C/Qt since that would make it hard to port to Windows and
> MacOS AFAIK.  Python/Tk would be a possibility, but I have never
> learnt python and I'm actually not all that comfortable with having to
> do things the object-oriented way.
>
> Any suggestions?

I've looked at Perl and Python Tk integration in the past; they
take slightly different approaches.  The Perl integration tries
to first libify Tk part to make it less dependent to the host
language, tcl, and then retargets it to a new host language,
Perl.  Compared to that, Python integration was shallower;
calling from Python to Tk and callback from Tk to Python were
done using Tcl as intermediary.  Which looked somewhat hacky but
at the same time cleaner.  From scriptability point of view,
both were much more pleasant to use than tcl.  You would have
object-orientation in the nature of data anyway (e.g. your
commitdata, commitlisted, commitidx and friends will not be
look-up tables keyed with commit object name, rather they will
become attributes to commit objects), so I would expect doing it
in Python+Tk would feel natural.

Re: FFmpeg considering GIT

[Shawn O. Pearce]

Paul Mackerras <paulus@samba.org> wrote:
> I have thought about rewriting it in a different language, but I
> haven't found anything that really appeals.  I don't want to go to
> C/GTK or C/Qt since that would make it hard to port to Windows and
> MacOS AFAIK.  Python/Tk would be a possibility, but I have never
> learnt python and I'm actually not all that comfortable with having to
> do things the object-oriented way.
> 
> Any suggestions?

Funny that you mention this.  Lately I have been hacking on git-gui,
trying to improve it and clean up some of the code.

I've thought about wxWindows but didn't really dig into it to see
how usuable it would be - primary reason is not everyone has it
installed on their system.  The same for GTK and Qt.  Actually I
don't even have GTK installed on my Mac but I did install Qt3
(took half a day!)  so I could build qgit at one point in time.

But almost everyone already has a wish installed.

I've thought about writing git-gui in C, but linking to the Tk
library for the "portable UI".  But not everyone has the Tcl/Tk
development headers and libraries installed, but they probably do
have the wish executable installed.

I want to limit the barrier to entry for git, and that means limiting
the barrier of entry for git-gui.  Keeping our requirements to a
minimum helps.

So I think I've settled on sticking to Tcl and its Tk extensions,
but making more use of newer Tcl constructs like namespaces.  If you
look at my `pu` branch of git-gui I have actually split the program
down into many files, and have started to organize the code in each
into different namespaces, depending on function.

-- 
Shawn.

Re: FFmpeg considering GIT

[Jeff King]

On Mon, May 07, 2007 at 10:13:44PM +1000, Paul Mackerras wrote:

> I have thought about rewriting it in a different language, but I
> haven't found anything that really appeals.  I don't want to go to
> C/GTK or C/Qt since that would make it hard to port to Windows and
> MacOS AFAIK.  Python/Tk would be a possibility, but I have never
> learnt python and I'm actually not all that comfortable with having to
> do things the object-oriented way.
> 
> Any suggestions?

I just ran across this today (it was just packaged for Debian):

  http://developer.imendio.com/projects/giggle

It seems to be a C/GTK repository browser (but also with a few git-gui
type features). It doesn't seem very far along (the viewer barfs on the
git.git repo, but shows some of my simpler repos), but it might worth
starting a dialogue with those guys.

-Peff

Re: FFmpeg considering GIT

[Karl Hasselström]

Thanks for the patch! I haven't had time to try it out yet, but ...

On 2007-05-05 10:26:12 -0700, Linus Torvalds wrote:

> -	    error_popup "Can't parse git rev-list output: {$shortcmit}"
> +	    error_popup "Can't parse git git log output: {$shortcmit}"

... this error message should probably lose one of its two "git "s.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Karl Hasselström]

On 2007-05-05 10:26:12 -0700, Linus Torvalds wrote:

> The "-S" thing doesn't really interact well with "gitk", because it
> doesn't rewrite the parent information (it is basically just a "hide
> commits that don't pass this criteria"). As such, gitk, which
> requires parent information to generate the graph, is not very
> amenable to using "-S" and such.
>
> That said, you can apply this fairly trivial patch to "gitk" to make
> it parse the output of "git log" rather than "git rev-list", and
> that will actually get you working -S'xyz' parsing automatically.
> It's just that the commit history window will look like crap.

OK, now I've tested it, and just as you said, it works (and is _very_
useful) but looks like crap. :-)

Is there any fundamental reason why

  gitk -- some/path/name

generates a nice, connected graph, while

  gitk -S'some string'

generates disconnected spaghetti? Or could the latter be made to use
the same parent-rewriting logic as the first?

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Linus Torvalds]

On Sun, 6 May 2007, Karl Hasselstr?m wrote:
> 
> OK, now I've tested it, and just as you said, it works (and is _very_
> useful) but looks like crap. :-)
> 
> Is there any fundamental reason why
> 
>   gitk -- some/path/name
> 
> generates a nice, connected graph, while
> 
>   gitk -S'some string'
> 
> generates disconnected spaghetti?

There is a reason, and it's fairly fundamental: the path limiting code is 
deeply embedded in the revision walking, and I've spent a fair amount of 
effort on making that work and efficient as hell (it's one of the few 
areas in git where I'm probably still the main author). Because it's 
literally what I do 90% of the time: for me, the path-limiting code is 
basically _the_ most important git feature, and I care very deeply.

In contrast, the "-S" thing is not actually part of the revision walking 
at all, and is a totally separate phase that is done when revisions are 
_shown_. I almost never use it myself, and it grew out of a totally 
separate effort by Junio. 

> Or could the latter be made to use the same parent-rewriting logic as 
> the first?

It would probably be possible to make the -S logic be another part of the 
"prune_fn()" logic in revision.c, and it might even simplify some of the 
logic, but I suspect it would actually suck really really badly from a 
performance standpoint.

Why? Because the prune_fn() logic is done when we generate the revision 
graph, which is generally something that a lot of the operations have to 
do up-front before they can do _anything_ else. Eg, any revision limiter 
(and that's a very common case) like "v2.6.21.." will cause the revision 
pruning to happen synchronously and early on.

And the path-limiting is *fast*. It's so incredibly fast that people don't 
really realize how fast it is. And it absolutely needs to be fast, because 
when you do something like "gitk v2.6.18.. drivers/" on the kernel you end 
up doing a _lot_ of tree comparisons. It's why I'm pretty sure nobody else 
can ever do what git does - it takes full advantage of how git can tell 
that a whole subdirectory hasn't changed without even recursing into it.

In contrast, "-S" is _slow_. It's a really really expensive operation. Git 
makes generating diffs faster than just about anything else, but it's 
still really expensive. This is a really unfair comparison, but:

	time git log drivers/net/ > /dev/null

	real    0m1.488s
	user    0m1.444s
	sys     0m0.040s

ie we can do the log pruning for the whole kernel git history on a 
subdirectory in less than two seconds. 

Try to compare it with

	time git log -Sdrivers/net/ > /dev/null

and I suspect you won't have the patience to wait for the end result.

And yeah, the operations are fundamentally very very different, and yes, 
the latter operation is really really expensive (which is why I said it's 
a really unfair comparison). But the point is that the expense comes from 
how git has been designed: seeing differences in the paths is cheap by 
design (it's how the data structures are laid out), but seeing differences 
in actual diffs means that we have to fully generate each diff for each 
revision!

A different approach to the underlying datastructures could change the 
equation. For example, if the fundamental data representation was the 
"diff" (rather than the "whole tree") maybe -S would be as fast as path 
limiting. But you'd *really* suck for other things.

To summarize a long story: the path limiting is simply more fundamental in 
git. Both by design, and then - obviously partly _due_ to that - by pure 
effort we've spent on it. It's something very deep and very important. In 
comparison, the -S thing is a cute extra feature, nothing really "deep".

		Linus

Re: FFmpeg considering GIT

[Marco Costalba]

On 5/5/07, Karl Hasselström <kha@treskal.com> wrote:
> On 2007-05-04 11:17:05 -0700, Carl Worth wrote:
>
> I know that I can ask it to highlight commits that insert or remove
> "snippet of interest", but frequently the highlighted commits are ten
> out of ten thousand, and not that easy to find even when boldfaced.
> What I want is to make it display only those commits.
>

Use qgit ;-)

Re: FFmpeg considering GIT

[Karl Hasselström]

On 2007-05-06 10:15:16 +0200, Marco Costalba wrote:

> Use qgit ;-)

Do I have to use any particular autoconf version? With

  kha@yoghurt:~/qgit> autoreconf --version
  autoreconf (GNU Autoconf) 2.60

I get a few warnings and errors when I run "autoreconf -i", and the
generated configure script has bugs that prevent it from finishing
successfully. (I'm trying to build qgit
4facdd5fc1731662ff9cdf096a576d40b938885c.)

Output follows:

kha@yoghurt:~/qgit> autoreconf -i
configure.ac: 9: `automake requires `AM_CONFIG_HEADER', not `AC_CONFIG_HEADER'
automake: configure.ac: installing `config/install-sh'
automake: configure.ac: installing `config/mkinstalldirs'
automake: configure.ac: installing `config/missing'
automake: configure.ac: installing `config/config.guess'
automake: configure.ac: installing `config/config.sub'
automake: Makefile.am: installing `./INSTALL'
automake: Makefile.am: required file `./NEWS' not found
automake: Makefile.am: required file `./AUTHORS' not found
configure.ac: 9: required file `./[config.h].in' not found
src/Makefile.am:30: invalid unused variable name: `nodist_qgit_SOURCES'
autoreconf: automake failed with exit status: 1

kha@yoghurt:~/qgit> ./configure --prefix=/usr/local/stow/qgit
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking whether make sets $(MAKE)... yes
./configure: line 1874: 1.8: command not found
checking for working aclocal-1.4... found
checking for working autoconf... found
checking for working automake-1.4... found
checking for working autoheader... found
checking for working makeinfo... found
checking for g++... g++
checking for C++ compiler default output file name... a.out
checking whether the C++ compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables... 
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking how to run the C++ preprocessor... g++ -E
checking for X... libraries , headers 
checking for gethostbyname... yes
checking for connect... yes
checking for remove... yes
checking for shmat... yes
checking for IceConnectionNumber in -lICE... yes
checking for Qt includes... /usr/share/qt3/include
checking Qt version... 3.3.6
checking for moc... /usr/share/qt3/bin/moc
checking for uic... /usr/share/qt3/bin/uic
checking for pthread_exit in -lpthread... yes
checking for XftFontOpen in -lXft... yes
checking for main in -lqt-mt... yes
configure: QT_CPPFLAGS = -I/usr/share/qt3/include -D_REENTRANT -DQT_THREAD_SUPPORT
configure: QT_LDFLAGS =  -L/usr/share/qt3/lib
configure: QT_LIBS = -lqt-mt -lpthread  -lXft  -lSM -lICE -lX11 
configure: creating ./config.status
config.status: creating Makefile
config.status: WARNING:  Makefile.in seems to ignore the --datarootdir setting
config.status: creating src/Makefile
config.status: WARNING:  src/Makefile.in seems to ignore the --datarootdir setting
config.status: creating config.h
kha@yoghurt:~/qgit> make
make[1]: Entering directory `/home/kha/qgit'
cd . && autoheader
make[1]: Leaving directory `/home/kha/qgit'
cd . \
          && CONFIG_FILES= CONFIG_HEADERS=[config.h] \
             /bin/bash ./config.status
config.status: error: cannot find input file: [config.h].in
make: *** [stamp-h] Error 1

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Marco Costalba]

On 5/6/07, Karl Hasselström <kha@treskal.com> wrote:
> On 2007-05-06 10:15:16 +0200, Marco Costalba wrote:
>
> > Use qgit ;-)
>
> Do I have to use any particular autoconf version? With
>

Sorry but I cannot reproduce the misbehaviour, my logs are below.

The only difference is that my autoreconf is 2.61 instead of 2.60

Pavel, some ideas?


In the mean time you can download the released tarball from
http://prdownloads.sourceforge.net/qgit/qgit-1.5.5.tar.bz2?download

You don't need autoreconf in that case and can go directly with

        configure/make/make install-strip



My log:

bash-3.1$ git clone git://git.kernel.org/pub/scm/qgit/qgit.git
Initialized empty Git repository in /home/marco/tmp/qgit/.git/
remote: Generating pack...
remote: Done counting 5723 objects.
remote: Deltifying 5723 objects.
remote:  100% (5723/5723) done
Indexing 5723 objects...
remote: Total 5723 (delta 4541), reused 5612 (delta 4461)
 100% (5723/5723) done
Resolving 4541 deltas...
 100% (4541/4541) done

bash-3.1$ cd qgit
bash-3.1$ autoreconf --version
autoreconf (GNU Autoconf) 2.61
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software.  You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

Written by David J. MacKenzie and Akim Demaille.
bash-3.1$ autoreconf -i
configure.ac: installing `config/install-sh'
configure.ac: installing `config/missing'
src/Makefile.am: installing `config/depcomp'
bash-3.1$ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for g++... g++
checking for C++ compiler default output file name... a.out
checking whether the C++ compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of g++... gcc3
checking for a BSD-compatible install... /usr/bin/install -c
checking for prefix by checking for qgit... /home/marco/bin/qgit
checking how to run the C++ preprocessor... g++ -E
checking for X... libraries , headers
checking for gethostbyname... yes
checking for connect... yes
checking for remove... yes
checking for shmat... yes
checking for IceConnectionNumber in -lICE... yes
checking for Qt includes... /usr/lib/qt3//include
checking Qt version... 3.3.8
checking for moc... /usr/lib/qt3//bin/moc
checking for uic... /usr/lib/qt3//bin/uic
checking for pthread_exit in -lpthread... yes
checking for XftFontOpen in -lXft... yes
checking for main in -lqt-mt... yes
configure: QT_CPPFLAGS = -I/usr/lib/qt3//include -D_REENTRANT
-DQT_THREAD_SUPPORT
configure: QT_LDFLAGS =  -L/usr/lib/qt3//lib
configure: QT_LIBS = -lqt-mt -lpthread  -lXft  -lSM -lICE -lX11
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating config.h
config.status: executing depfiles commands

bash-3.1$ make
make  all-recursive
make[1]: Entering directory `/home/marco/tmp/qgit'
Making all in src
make[2]: Entering directory `/home/marco/tmp/qgit/src'
/usr/lib/qt3//bin/uic -o commitbase.h commitbase.ui
/usr/lib/qt3//bin/uic -o consolebase.h consolebase.ui

-------------- cut ---------------------

if g++ -DHAVE_CONFIG_H -I. -I. -I..   -g -O2 -I/usr/lib/qt3//include
-D_REENTRANT -DQT_THREAD_SUPPORT  -g -O2 -Wall -Wno-non-virtual-dtor
-W -Wno-long-long -pedantic -frepo -MT settingsbase.uic.o -MD -MP -MF
".deps/settingsbase.uic.Tpo" -c -o settingsbase.uic.o
settingsbase.uic.cc; \
        then mv -f ".deps/settingsbase.uic.Tpo"
".deps/settingsbase.uic.Po"; else rm -f ".deps/settingsbase.uic.Tpo";
exit 1; fi
g++  -g -O2 -Wall -Wno-non-virtual-dtor -W -Wno-long-long -pedantic
-frepo  -L/usr/lib/qt3//lib -o qgit  annotate.o cache.o commitimpl.o
consoleimpl.o customactionimpl.o dataloader.o domain.o
exceptionmanager.o filecontent.o filelist.o fileview.o git.o
git_startup.o lanes.o listview.o mainimpl.o myprocess.o
namespace_def.o patchview.o qgit.o rangeselectimpl.o revdesc.o
revsview.o settingsimpl.o treeview.o   commitbase.moc.o
consolebase.moc.o customactionbase.moc.o filebase.moc.o helpbase.moc.o
mainbase.moc.o patchbase.moc.o rangeselectbase.moc.o revbase.moc.o
settingsbase.moc.o annotate.moc.o commitimpl.moc.o consoleimpl.moc.o
customactionimpl.moc.o dataloader.moc.o domain.moc.o filecontent.moc.o
filelist.moc.o fileview.moc.o git.moc.o listview.moc.o mainimpl.moc.o
myprocess.moc.o patchview.moc.o rangeselectimpl.moc.o revdesc.moc.o
revsview.moc.o settingsimpl.moc.o treeview.moc.o commitbase.uic.o
consolebase.uic.o customactionbase.uic.o filebase.uic.o helpbase.uic.o
mainbase.uic.o patchbase.uic.o rangeselectbase.uic.o revbase.uic.o
settingsbase.uic.o  -lqt-mt -lpthread  -lXft  -lSM -lICE -lX11
collect: recompiling patchview.cpp
collect: recompiling mainimpl.cpp
collect: recompiling git_startup.cpp
collect: recompiling git.cpp

-------------- cut ---------------------

collect: relinking
collect: recompiling mainimpl.cpp
collect: recompiling git_startup.cpp
collect: recompiling git.cpp
collect: recompiling annotate.cpp
collect: relinking
make[3]: Leaving directory `/home/marco/tmp/qgit/src'
make[2]: Leaving directory `/home/marco/tmp/qgit/src'
make[2]: Entering directory `/home/marco/tmp/qgit'
make[2]: Nothing to be done for `all-am'.
make[2]: Leaving directory `/home/marco/tmp/qgit'
make[1]: Leaving directory `/home/marco/tmp/qgit'

bash-3.1$ make install-strip
make  INSTALL_PROGRAM="/bin/sh /home/marco/tmp/qgit/config/install-sh -c -s" \
          install_sh_PROGRAM="/bin/sh
/home/marco/tmp/qgit/config/install-sh -c -s" INSTALL_STRIP_FLAG=-s \
          `test -z '' || \
            echo "INSTALL_PROGRAM_ENV=STRIPPROG=''"` install
make[1]: Entering directory `/home/marco/tmp/qgit'
Making install in src
make[2]: Entering directory `/home/marco/tmp/qgit/src'
make  install-am

-------------- cut ---------------------

make[2]: Leaving directory `/home/marco/tmp/qgit'
make[1]: Leaving directory `/home/marco/tmp/qgit'

bash-3.1$ qgit
Found GNU source-highlight 2.5
Saving cache. Please wait...
Compressing data...
Done.
bash-3.1$

Re: FFmpeg considering GIT

[Karl Hasselström]

On 2007-05-06 14:19:55 +0200, Marco Costalba wrote:

> In the mean time you can download the released tarball from
> http://prdownloads.sourceforge.net/qgit/qgit-1.5.5.tar.bz2?download
>
> You don't need autoreconf in that case and can go directly with
>
>        configure/make/make install-strip

Works like a charm. Thanks.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Marco Costalba]

On 5/6/07, Marco Costalba <mcostalba@gmail.com> wrote:
> On 5/6/07, Karl Hasselström <kha@treskal.com> wrote:
> > On 2007-05-06 10:15:16 +0200, Marco Costalba wrote:
> >
> > > Use qgit ;-)
> >
> > Do I have to use any particular autoconf version? With
> >
>

After googling a little bit....

Please apply this patch before to run 'autoreconf -i' on a *fresh
cloned* repository

diff --git a/configure.ac b/configure.ac
index e352eba..95f45d1 100644
--- a/configure.ac
+++ b/configure.ac
@@ -6,7 +6,7 @@ AC_INIT(qgit, 1.5.5)
 AC_CONFIG_AUX_DIR(config)
 AM_INIT_AUTOMAKE(foreign 1.8)
 AC_CONFIG_SRCDIR([src/annotate.cpp])
-AC_CONFIG_HEADER([config.h])
+AM_CONFIG_HEADER([config.h])

 # Checks for programs.
 AC_LANG(C++)


On my box it compiles the same and perhaps you can avoid the reported errors.

Please, let me know how it goes.

Marco

Re: FFmpeg considering GIT

[Karl Hasselström]

On 2007-05-06 14:33:59 +0200, Marco Costalba wrote:

> After googling a little bit....
>
> Please apply this patch before to run 'autoreconf -i' on a *fresh
> cloned* repository
>
> diff --git a/configure.ac b/configure.ac
> index e352eba..95f45d1 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -6,7 +6,7 @@ AC_INIT(qgit, 1.5.5)
> AC_CONFIG_AUX_DIR(config)
> AM_INIT_AUTOMAKE(foreign 1.8)
> AC_CONFIG_SRCDIR([src/annotate.cpp])
> -AC_CONFIG_HEADER([config.h])
> +AM_CONFIG_HEADER([config.h])
>
> # Checks for programs.
> AC_LANG(C++)
>
>
> On my box it compiles the same and perhaps you can avoid the
> reported errors. Please, let me know how it goes.

Nope, no luck. I get the same error messages from autoreconf:

kha@yoghurt:~/qgit> autoreconf -i
automake: configure.ac: installing `config/install-sh'
automake: configure.ac: installing `config/mkinstalldirs'
automake: configure.ac: installing `config/missing'
automake: configure.ac: installing `config/config.guess'
automake: configure.ac: installing `config/config.sub'
automake: Makefile.am: installing `./INSTALL'
automake: Makefile.am: required file `./NEWS' not found
automake: Makefile.am: required file `./AUTHORS' not found
configure.ac: 9: required file `./[config.h].in' not found
src/Makefile.am:30: invalid unused variable name: `nodist_qgit_SOURCES'
autoreconf: automake failed with exit status: 1

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Karl Hasselström]

On 2007-05-06 14:59:38 +0200, Karl Hasselström wrote:

> Nope, no luck. I get the same error messages from autoreconf:
>
> kha@yoghurt:~/qgit> autoreconf -i
> automake: configure.ac: installing `config/install-sh'
> automake: configure.ac: installing `config/mkinstalldirs'
> automake: configure.ac: installing `config/missing'
> automake: configure.ac: installing `config/config.guess'
> automake: configure.ac: installing `config/config.sub'
> automake: Makefile.am: installing `./INSTALL'
> automake: Makefile.am: required file `./NEWS' not found
> automake: Makefile.am: required file `./AUTHORS' not found
> configure.ac: 9: required file `./[config.h].in' not found
> src/Makefile.am:30: invalid unused variable name: `nodist_qgit_SOURCES'
> autoreconf: automake failed with exit status: 1

Well, actually some luck. This line was first in my previous output,
but doesn't appear with your patch:

configure.ac: 9: `automake requires `AM_CONFIG_HEADER', not `AC_CONFIG_HEADER'

But there seems to be more stuff to fix. :-(

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Re: FFmpeg considering GIT

[Pavel Roskin]

Hello!

On Sun, 2007-05-06 at 14:59 +0200, Karl Hasselström wrote:
> configure.ac: 9: required file `./[config.h].in' not found

Sorry for being late with the comments, but it looks like some very old
Automake doesn't understand that the argument to AC_CONFIG_HEADER or
AM_CONFIG_HEADER can be quoted.

Regarding AM_CONFIG_HEADER vs AC_CONFIG_HEADERS, the documentation for
Automake says:

AC_CONFIG_HEADERS
Automake will generate rules to rebuild these headers.  Older versions
of Automake required the use of AM_CONFIG_HEADER; this is no longer the
case today.

So I suggest that we keep AC_CONFIG_HEADERS.  Automake's NEWS file says
AM_CONFIG_HEADER is obsolete since version 1.7.

> src/Makefile.am:30: invalid unused variable name:
> `nodist_qgit_SOURCES'

That's another sign of an obsolete version of Automake.  "nodist_" was
introduced many years ago, back in the good old days when I had time to
track its progress.  NEWS says it was introduced in Automake 1.5!

Perhaps we should require version 1.7 by adding this to the top-level
Makefile.am:

AUTOMAKE_OPTIONS = 1.7

-- 
Regards,
Pavel Roskin

Re: FFmpeg considering GIT

[Brett Schwarz]

Sorry for the posting, my email reader sucks.

What is the real issue? Is it that there isn't enough people to maintain gitk? I've been hiding in the bushes, mostly because of time issues, but if there's a real need, I'd be willing to help. I'm a seasoned Tcl/Tk coder, and wouldn't have any problems helping out.

Also, I've been waiting for the git lib to get done. When this gets done, a lot of the procs in gitk can be re-written in 'C' as Tcl commands. This obviously gives the advantage of speed, but since it is written in 'C', the potential maintainership would be larger. The 'C' code would just be dyn loaded into the Tcl interpreter.

As Shawn mentions below, he started using namespaces for git-gui. I think gitk could benefit from that as well, along with a few other changes.


----- Original Message ----
From: Shawn O. Pearce <spearce@spearce.org>
To: Paul Mackerras <paulus@samba.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>; Karl Hasselstr?m <kha@treskal.com>; Junio C Hamano <junkio@cox.net>; Carl Worth <cworth@cworth.org>; Michael Niedermayer <michaelni@gmx.at>; Git Mailing List <git@vger.kernel.org>
Sent: Monday, May 7, 2007 7:03:38 PM
Subject: Re: FFmpeg considering GIT

Paul Mackerras <paulus@samba.org> wrote:
> I have thought about rewriting it in a different language, but I
> haven't found anything that really appeals.  I don't want to go to
> C/GTK or C/Qt since that would make it hard to port to Windows and
> MacOS AFAIK.  Python/Tk would be a possibility, but I have never
> learnt python and I'm actually not all that comfortable with having to
> do things the object-oriented way.
> 
> Any suggestions?

Funny that you mention this.  Lately I have been hacking on git-gui,
trying to improve it and clean up some of the code.

I've thought about wxWindows but didn't really dig into it to see
how usuable it would be - primary reason is not everyone has it
installed on their system.  The same for GTK and Qt.  Actually I
don't even have GTK installed on my Mac but I did install Qt3
(took half a day!)  so I could build qgit at one point in time.

But almost everyone already has a wish installed.

I've thought about writing git-gui in C, but linking to the Tk
library for the "portable UI".  But not everyone has the Tcl/Tk
development headers and libraries installed, but they probably do
have the wish executable installed.

I want to limit the barrier to entry for git, and that means limiting
the barrier of entry for git-gui.  Keeping our requirements to a
minimum helps.

So I think I've settled on sticking to Tcl and its Tk extensions,
but making more use of newer Tcl constructs like namespaces.  If you
look at my `pu` branch of git-gui I have actually split the program
down into many files, and have started to organize the code in each
into different namespaces, depending on function.

-- 
Shawn.
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html




__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: FFmpeg considering GIT

[Paul Mackerras]

Brett Schwarz writes:

> What is the real issue? Is it that there isn't enough people to

The real issue is that I would like, if possible, to make it easier
for people like Linus to hack on gitk and add cool features that I
wouldn't necessarily think of.

> maintain gitk? I've been hiding in the bushes, mostly because of
> time issues, but if there's a real need, I'd be willing to help. I'm
> a seasoned Tcl/Tk coder, and wouldn't have any problems helping
> out.

That could be very useful, thanks.

> As Shawn mentions below, he started using namespaces for git-gui. I
> think gitk could benefit from that as well, along with a few other
> changes.

Gitk ends up handling pretty significant amounts of data.  In
particular the per-commit data can get to gigabytes, and processing it
is pretty cpu-intensive.  I did try using namespaces for the
per-commit data but I found that the performance hit to be more than I
was willing to tolerate.

Paul.

Re: FFmpeg considering GIT

[Shawn O. Pearce]

Paul Mackerras <paulus@samba.org> wrote:
> Brett Schwarz writes:
> > As Shawn mentions below, he started using namespaces for git-gui. I
> > think gitk could benefit from that as well, along with a few other
> > changes.
> 
> Gitk ends up handling pretty significant amounts of data.  In
> particular the per-commit data can get to gigabytes, and processing it
> is pretty cpu-intensive.  I did try using namespaces for the
> per-commit data but I found that the performance hit to be more than I
> was willing to tolerate.

If that is the case then an obvious direction is to start using C
for the actual Git operations/datastore and Tcl/Tk for the basic
UI layout and event handlers.

If we go down that path for gitk then I may wind up doing the
same for git-gui.  Because gitk would require the tcl/tk heders
and libraries at that point, so also requiring them for git-gui
wouldn't be too unreasonable.

But fortunately git-gui doesn't have to deal with gigabytes
of data; I'm only really looking at the "dirty" stuff, or
at worst, the blame for an entire file.

-- 
Shawn.