gitweb wishlist

gitweb wishlist

[Petr Baudis]

  Hello,

  I would be very happy if you could extend the gitweb scripts a little.
Basically, what I need is to have ability to create a permanent link to
a given file in the repository, which stays same across revisions (as
long as the file stays with the given name, obviously).

  E.g. I would like to have something like

	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib/ciabot.pl

for file contrib/ciabot.pl in the latest Cogito tree, and

	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib

for the list of the contrib/ directory in the latest Cogito tree.

  I think I would prefer the link from the repository index to go not to
the log page, but some "summary" page, which would have some short
information about the repository (owner, description, list of branches
if gitweb supports that, list of tags, link to the latest tree and link
to the log).

  BTW, why are people using ';' in the URIs for separators instead of
good old '&'s? Hmm, actually it just occurred to me that it might be to
workaround entity string problems?

  Thanks,

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: gitweb wishlist

[YOSHIFUJI Hideaki / 吉藤英明]

In article <20050511012626.GL26384@pasky.ji.cz> (at Wed, 11 May 2005 03:26:26 +0200), Petr Baudis <pasky@ucw.cz> says:

>   E.g. I would like to have something like
> 
> 	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib/ciabot.pl
                                                     ~~~ why not /?

--yoshfuji

Re: gitweb wishlist

[Petr Baudis]

Dear diary, on Wed, May 11, 2005 at 03:49:07AM CEST, I got a letter
where "YOSHIFUJI Hideaki / ?$B5HF#1QL@" <yoshfuji@linux-ipv6.org> told me that...
> In article <20050511012626.GL26384@pasky.ji.cz> (at Wed, 11 May 2005 03:26:26 +0200), Petr Baudis <pasky@ucw.cz> says:
> 
> >   E.g. I would like to have something like
> > 
> > 	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib/ciabot.pl
>                                                      ~~~ why not /?

This was a simple cut'n'paste. I don't care either way personally, but I
think it's better to be conservative when generating links and escape
this. OTOH it should obviously also accept a URL containing a slash
instead.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: gitweb wishlist

[Kay Sievers]

On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:
>   Hello,
> 
>   I would be very happy if you could extend the gitweb scripts a little.
> Basically, what I need is to have ability to create a permanent link to
> a given file in the repository, which stays same across revisions (as
> long as the file stays with the given name, obviously).
> 
>   E.g. I would like to have something like
> 
> 	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib/ciabot.pl
> 
> for file contrib/ciabot.pl in the latest Cogito tree, and
> 
> 	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib
> 
> for the list of the contrib/ directory in the latest Cogito tree.

With the next round I will try to introduce a real browser through the
tree instead of the current damn simple tree-object list. It would be
nice if it can show the the current path and provide a "up to parent"
way. Along with that it should be easy to add the stuff you asked for.

>   I think I would prefer the link from the repository index to go not to
> the log page, but some "summary" page, which would have some short
> information about the repository (owner, description, list of branches
> if gitweb supports that, list of tags, link to the latest tree and link
> to the log).
> 
>   BTW, why are people using ';' in the URIs for separators instead of
> good old '&'s? Hmm, actually it just occurred to me that it might be to
> workaround entity string problems?

Yes, but it's "bad old" not "good". :)
  http://www.w3.org/TR/html4/appendix/notes.html#h-B.2.2

Thanks,
Kay

Re: gitweb wishlist

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 804 bytes --]

On Wed, 2005-05-11 03:26:26 +0200, Petr Baudis <pasky@ucw.cz> wrote:
>   I would be very happy if you could extend the gitweb scripts a little.
> Basically, what I need is to have ability to create a permanent link to
> a given file in the repository, which stays same across revisions (as
> long as the file stays with the given name, obviously).

I've got another one. I'd like to see st_mtime on the file lists to see
when a file was last touched...

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: gitweb wishlist

[Junio C Hamano]

* [Previous page] [Next page] would be nice in addition to last
  10, day, week, etc.

* Putting the commit headline and "X hour"s ago in a separate
  div or span next to each other, so that a long commit headline
  wraps properly and does not start the second line just under
  the "X hours ago" timestamp would be nicer (you can see what I
  mean easily by narrowing the browser window).

Re: gitweb wishlist

[Kay Sievers]

On Thu, 2005-05-12 at 13:07 -0700, Junio C Hamano wrote:
> * [Previous page] [Next page] would be nice in addition to last
>   10, day, week, etc.

That should be easy to do with the parameters we have now for the
git-rev-list. I will first finish the new browser through the
trees/files, then the project overview page and after that try the
pager,

> * Putting the commit headline and "X hour"s ago in a separate
>   div or span next to each other, so that a long commit headline
>   wraps properly and does not start the second line just under
>   the "X hours ago" timestamp would be nicer (you can see what I
>   mean easily by narrowing the browser window).

Block elements (div) are not allowed inside an a-tag in XHTML/Strict -
don't know how to do this, cause the whole headline should be a link
without the use of javascript. :)

Thanks,
Kay

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "KS" == Kay Sievers <kay.sievers@vrfy.org> writes:

>> * Putting the commit headline and "X hour"s ago in a separate
>> div or span next to each other, so that a long commit headline
>> wraps properly and does not start the second line just under
>> the "X hours ago" timestamp would be nicer (you can see what I
>> mean easily by narrowing the browser window).

KS> Block elements (div) are not allowed inside an a-tag in XHTML/Strict -
KS> don't know how to do this,...

Wouldn't splitting the commit headline and "X hours ago" into
two separate elements and wrap them individually inside an a-tag
each pointing at the same destination good enough then?

Re: gitweb wishlist

[Jonas Fonseca]

I don't know if this is intentional, but it looks like gitweb discards
everything after the first line starting with 'Signed-off-by:' on the
log page. This will in some cases remove valuable log information when
a second author or the committer adds additional comments to a commit:

        <log message by author>

        Signed-off-by: <author>

        <log message by committer>

        Signed-off-by: <committer>

The commit page gets it right, which is why I suspect it might just be a
way to trim the amount of text on the log page.

I also noticed that there is a 'faulty' signed-off-by line in commit
14ebb908e10f068dc1901d35f4b716bc69143d19 in case the above is
intentional. Dunno if that should be matched by relaxing the regexp a
little.

-- 
Jonas Fonseca

Re: gitweb wishlist

[Kay Sievers]

On Fri, 2005-05-13 at 14:06 +0200, Jonas Fonseca wrote:
> I don't know if this is intentional, but it looks like gitweb discards
> everything after the first line starting with 'Signed-off-by:' on the
> log page. This will in some cases remove valuable log information when
> a second author or the committer adds additional comments to a commit:
> 
>         <log message by author>
> 
>         Signed-off-by: <author>
> 
>         <log message by committer>
> 
>         Signed-off-by: <committer>
> 
> The commit page gets it right, which is why I suspect it might just be a
> way to trim the amount of text on the log page.
> 
> I also noticed that there is a 'faulty' signed-off-by line in commit
> 14ebb908e10f068dc1901d35f4b716bc69143d19 in case the above is
> intentional. Dunno if that should be matched by relaxing the regexp a
> little.

All that should work now.

Thanks,
Kay

Re: gitweb wishlist

[Kay Sievers]

On Wed, 2005-05-11 at 11:30 +0200, Jan-Benedict Glaw wrote:
> On Wed, 2005-05-11 03:26:26 +0200, Petr Baudis <pasky@ucw.cz> wrote:
> >   I would be very happy if you could extend the gitweb scripts a little.
> > Basically, what I need is to have ability to create a permanent link to
> > a given file in the repository, which stays same across revisions (as
> > long as the file stays with the given name, obviously).
> 
> I've got another one. I'd like to see st_mtime on the file lists to see
> when a file was last touched...

We don't have that information in the trees. The date is only available
in the commits and reading all commits or storing that information in a
index or a database is beyond the scope of that simple cgi.

Thanks,
Kay

Re: gitweb wishlist

[Kay Sievers]

On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:
>   I would be very happy if you could extend the gitweb scripts a little.
> Basically, what I need is to have ability to create a permanent link to
> a given file in the repository, which stays same across revisions (as
> long as the file stays with the given name, obviously).
> 
>   E.g. I would like to have something like
> 
> 	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib/ciabot.pl
> 
> for file contrib/ciabot.pl in the latest Cogito tree, and

http://www.kernel.org/git/gitweb.cgi?p=cogito/cogito.git;a=blob;f=contrib/ciabot.pl

> 	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib
> 
> for the list of the contrib/ directory in the latest Cogito tree.

http://www.kernel.org/git/gitweb.cgi?p=cogito/cogito.git;a=tree;f=contrib

>   I think I would prefer the link from the repository index to go not to
> the log page, but some "summary" page, which would have some short
> information about the repository (owner, description, list of branches
> if gitweb supports that, list of tags, link to the latest tree and link
> to the log).

Sounds reasonable, I will try that with the next round.

Thanks,
Kay

Re: gitweb wishlist

[Jonas Fonseca]

Kay Sievers <kay.sievers@vrfy.org> wrote Sat, May 14, 2005:
> On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:
> >   I would be very happy if you could extend the gitweb scripts a little.
> > Basically, what I need is to have ability to create a permanent link to
> > a given file in the repository, which stays same across revisions (as
> > long as the file stays with the given name, obviously).
> > 
> >   E.g. I would like to have something like
> > 
> > 	http://www.kernel.org/git/gitweb.cgi?p=cogito%2Fcogito.git;n=contrib/ciabot.pl
> > 
> > for file contrib/ciabot.pl in the latest Cogito tree, and
> 
> http://www.kernel.org/git/gitweb.cgi?p=cogito/cogito.git;a=blob;f=contrib/ciabot.pl

How about support for getting the 'raw' files and diffs without all the
HTML markup?

-- 
Jonas Fonseca

Re: gitweb wishlist

[Kay Sievers]

On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:

>   I think I would prefer the link from the repository index to go not to
> the log page, but some "summary" page, which would have some short
> information about the repository (owner, description, list of branches
> if gitweb supports that, list of tags, link to the latest tree and link
> to the log).

I did this now. The top-link shows now the repository listing with a
nice "last change" field. The default link points to an overview page
which also list the tags.
(The owner filed in that list is not correct until now, cause the
cron-job needs to be adapted.)

Thanks,
Kay

Re: gitweb wishlist

[Petr Baudis]

Dear diary, on Wed, May 18, 2005 at 04:55:51AM CEST, I got a letter
where Kay Sievers <kay.sievers@vrfy.org> told me that...
> On Wed, 2005-05-11 at 03:26 +0200, Petr Baudis wrote:
> 
> >   I think I would prefer the link from the repository index to go not to
> > the log page, but some "summary" page, which would have some short
> > information about the repository (owner, description, list of branches
> > if gitweb supports that, list of tags, link to the latest tree and link
> > to the log).
> 
> I did this now. The top-link shows now the repository listing with a
> nice "last change" field. The default link points to an overview page
> which also list the tags.
> (The owner filed in that list is not correct until now, cause the
> cron-job needs to be adapted.)

Thanks, this is exactly how I envisioned it. :-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor

Re: gitweb wishlist

[Linus Torvalds]

On Wed, 18 May 2005, Kay Sievers wrote:
> 
> I did this now. The top-link shows now the repository listing with a
> nice "last change" field. The default link points to an overview page
> which also list the tags.

In the summary page, could we get authorship information too? Right now it 
looks like

	recent commits

	15 minutes ago	[PATCH] Add tests for diff-tree
	31 minutes ago	diff-tree: use new base_name_compare() helper function
	34 minutes ago	Introduce "base_name_compare()" helper function 
	...

and wouldn't it be nice if it told you who had written these things, like

	recent commits

	15 minutes ago	Junio C Hamano	[PATCH] Add tests for diff-tree
	31 minutes ago	Linus Torvalds	diff-tree: use new base_name_compare() helper function
	34 minutes ago	Linus Torvalds	Introduce "base_name_compare()" helper function 
	...

(limit the name to the first 20 characters or something to make things 
line up).

If the lines get too long, you could changes "minutes" to "min", and maybe 
limit the name to the first 10 characters (or even first name, but there's 
a lot of David's around, while 10-12 characters tends to get enough of 
the last name to be reasonable unique).

		Linus

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:

LT> If the lines get too long, you could changes "minutes" to "min", and maybe 
LT> limit the name to the first 10 characters (or even first name, but there's 
LT> a lot of David's around, while 10-12 characters tends to get enough of 
LT> the last name to be reasonable unique).

Or abbreviate the first names.  L Torvalds, JC Hamano, etc.

Re: gitweb wishlist

[Linus Torvalds]

On Fri, 20 May 2005, Junio C Hamano wrote:
> 
> Or abbreviate the first names.  L Torvalds, JC Hamano, etc.

Technically, yes. Except I end up at least personally going by either
first names (or _possibly_ email addresses), not by last name. Maybe Linux
is fairly unique in that, but dammit, I'm not "Torvalds", I'm "Linus". And
like it or not, you're either Junio or junkio at least to me ;)

		Linus

Re: gitweb wishlist

[Kay Sievers]

On Fri, 2005-05-20 at 09:54 -0700, Linus Torvalds wrote:
> 
> On Wed, 18 May 2005, Kay Sievers wrote:
> > 
> > I did this now. The top-link shows now the repository listing with a
> > nice "last change" field. The default link points to an overview page
> > which also list the tags.
> 
> In the summary page, could we get authorship information too? Right now it 
> looks like
> 
> 	recent commits
> 
> 	15 minutes ago	[PATCH] Add tests for diff-tree
> 	31 minutes ago	diff-tree: use new base_name_compare() helper function
> 	34 minutes ago	Introduce "base_name_compare()" helper function 
> 	...
> 
> and wouldn't it be nice if it told you who had written these things, like
> 
> 	recent commits
> 
> 	15 minutes ago	Junio C Hamano	[PATCH] Add tests for diff-tree
> 	31 minutes ago	Linus Torvalds	diff-tree: use new base_name_compare() helper function
> 	34 minutes ago	Linus Torvalds	Introduce "base_name_compare()" helper function 
> 	...
> 
> (limit the name to the first 20 characters or something to make things 
> line up).

Something like that: :)
  http://www.kernel.org/git/?p=cogito/cogito.git;a=summary

Kay

Re: gitweb wishlist

[Linus Torvalds]

On Fri, 20 May 2005, Kay Sievers wrote:
> 
> Something like that: :)
>   http://www.kernel.org/git/?p=cogito/cogito.git;a=summary

Looking good.

I still think "minutes" is an awfully long word to waste vertical space
with, especially since it's also the only one that will have up to three
digits associated with it. It seems kind of silly to give more (or even
equal) space to the simplified time field than to the person who did
something.

(To see minutes, you need to use "git/git.git" instead of cogito.

So I still think you should change "minutes" to "min", which is the common 
abbreviation.

(That "ago" is also very repetitive, but removing it would seem to make it 
unreadable, so I guess it's needed).

				Linus

Re: gitweb wishlist

[Linus Torvalds]

On Fri, 20 May 2005, Linus Torvalds wrote:
>
> Looking good.

Oh, dang, while I'm at it, why not ask for the "commmitdiff" thing to have 
the commit message in it too, ie basically look like a prettified version 
of "git-diff-tree -v -M <cmitname>"

You already do the first line of it, so there's not much missing. Right
now there is no place where everything important from one commit is
"brought together" (ie no single place where you can see both what the
diff is all about, and what it actually does).

		Linus

Re: gitweb wishlist

[Kay Sievers]

On Fri, 2005-05-20 at 11:16 -0700, Linus Torvalds wrote:
> 
> On Fri, 20 May 2005, Kay Sievers wrote:
> > 
> > Something like that: :)
> >   http://www.kernel.org/git/?p=cogito/cogito.git;a=summary
> 
> Looking good.
> 
> I still think "minutes" is an awfully long word to waste vertical space
> with, especially since it's also the only one that will have up to three
> digits associated with it.

Changed to "min".

Kay

Re: gitweb wishlist

[Kay Sievers]

On Fri, 2005-05-20 at 11:28 -0700, Linus Torvalds wrote:
> 
> On Fri, 20 May 2005, Linus Torvalds wrote:
> >
> > Looking good.
> 
> Oh, dang, while I'm at it, why not ask for the "commmitdiff" thing to have 
> the commit message in it too, ie basically look like a prettified version 
> of "git-diff-tree -v -M <cmitname>"
> 
> You already do the first line of it, so there's not much missing. Right
> now there is no place where everything important from one commit is
> "brought together" (ie no single place where you can see both what the
> diff is all about, and what it actually does).

Somehting like this?:
  http://kernel.org/git/?p=git/git.git;a=commitdiff;h=de809dbbce497e0d107562615c1d85ff35b4e0c5

Kay

Re: gitweb wishlist

[Thomas Glanzmann]

Hello Kay,
I would like to see that I can klick on the file instead of the seperate
'blob' link in a directory view, becasue that is more intuitive and you can
already an klick on directories:

http://www.kernel.org/git/?p=git/git.git;a=tree;h=665a48af9e192ed84d2707c95d4c0d9c45eb45ad;hb=411746940f02f6fb90c4b6b97c6f07cee599c2e1

Thanks for this great tool!

Sincerely,
	Thomas

Re: gitweb wishlist

[Linus Torvalds]

On Fri, 20 May 2005, Kay Sievers wrote:
> 
> Somehting like this?:
>   http://kernel.org/git/?p=git/git.git;a=commitdiff;h=de809dbbce497e0d107562615c1d85ff35b4e0c5

Yes, except I don't think you should repeat the part of the commit message
that you used as a header (so in your example, since you already used "Fix
up previous commit" in the header, don't show it below).

Btw, any chance to see gitweb itself under gitweb on kernel.org?

		Linus

Re: gitweb wishlist

[Linus Torvalds]

On Fri, 20 May 2005, Kay Sievers wrote:
> 
> Somehting like this?:
>   http://kernel.org/git/?p=git/git.git;a=commitdiff;h=de809dbbce497e0d107562615c1d85ff35b4e0c5

Btw, at least for me, this looks much more interesting than the "commit" 
thing, and maybe it would make sense to make the summary links be to the 
"commitdiff" instead of the "commit"?

Or is it just so much more expensive to generate, that we want to not have
people go there normally? (hpa cc'd, since he may have some insight into
whether this is likely to be an issue or not? It's not like git-diff-tree
is that expensive, but it _does_ end up doing a "diff" against each
changed file, of course, modulo any caching of results).

		Linus

Re: gitweb wishlist

[H. Peter Anvin]

Linus Torvalds wrote:
> 
> On Fri, 20 May 2005, Kay Sievers wrote:
> 
>>Somehting like this?:
>>  http://kernel.org/git/?p=git/git.git;a=commitdiff;h=de809dbbce497e0d107562615c1d85ff35b4e0c5
> 
> 
> Btw, at least for me, this looks much more interesting than the "commit" 
> thing, and maybe it would make sense to make the summary links be to the 
> "commitdiff" instead of the "commit"?
> 
> Or is it just so much more expensive to generate, that we want to not have
> people go there normally? (hpa cc'd, since he may have some insight into
> whether this is likely to be an issue or not? It's not like git-diff-tree
> is that expensive, but it _does_ end up doing a "diff" against each
> changed file, of course, modulo any caching of results).
> 

What I ended up doing for the diff viewer on kernel.org is that every 
page that's generated gets stuffed in a cache (locklessly indexed by a 
SHA-1 of a canonicalized form of the query); the pages people actually 
see are then simply pulled from the cache.  This caching was a just 
enormous win.  In the case of the diff viewer, the header is generated 
each time, since I allow the user to select a custom style sheet (and 
don't want to cache versions for each style sheet), but that's a trivial 
detail.

	-hpa

Re: gitweb wishlist

[Linus Torvalds]

On Fri, 20 May 2005, H. Peter Anvin wrote:

> 
> What I ended up doing for the diff viewer on kernel.org is that every 
> page that's generated gets stuffed in a cache (locklessly indexed by a 
> SHA-1 of a canonicalized form of the query); the pages people actually 
> see are then simply pulled from the cache.  This caching was a just 
> enormous win.

Ok. That still leaves the bandwidth issue (the full diffs are bigger than 
the commit object), but usually the diffs in individual commits aren't 
_that_ large, so maybe it's a non-issue.

Oh, btw, I notice that you moved klibc over to git - care to share your
cvs->git script (I assume you scripted it ;)? That would seem to be an 
obvious addition to the core stuff..

		Linus

Re: gitweb wishlist

[H. Peter Anvin]

Linus Torvalds wrote:
> 
> Oh, btw, I notice that you moved klibc over to git - care to share your
> cvs->git script (I assume you scripted it ;)? That would seem to be an 
> obvious addition to the core stuff..
> 

Actually, Kay did the conversion... the scripts are clearly very 
cantankerous, because if *I* run them -- I tried -- they don't work! 
Since it's Kay's work, I'll leave them to him, but I would definitely 
love to move more of my CVS repos over to git, especially syslinux.

	-hpa

Re: gitweb wishlist

[Thomas Glanzmann]

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]

Hello,
I imported the mutt-cvs for the 1.5 branch into GIT using the following
script. But it is a hack. I also think that I will use something like
that to build a CVS->GIT vendortracking.

cvsps -x -z 10 -b HEAD -g -p ../../patches/

And using the attached script to import the patches in GIT. It works
quiet well.

See also msgid: 1115080139.21105.18.camel@localhost.localdomain there
are the scripts which he used to convert the CVS to GIT for HPA. My
scripts are based on his work.

	Thomas

[-- Attachment #2: cvsps-import.pl --]
[-- Type: text/plain, Size: 1810 bytes --]

#!/usr/bin/perl

use strict;
use warnings;
use File::Temp qw/ tempfile tempdir /;

# ---------------------
# PatchSet 1 
# Date: 2002/07/23 07:41:30
# Author: hpa
# Branch: HEAD
# Tag: (none) 
# Log:
# Initial revision
# 
# Members: 
# 	klibc.cvsroot/snprintf.c:INITIAL->1.1 
# 	klibc.cvsroot/vsnprintf.c:INITIAL->1.1 
# 	klibc.cvsroot/klibc/Makefile:INITIAL->1.1 
# 	klibc.cvsroot/klibc/snprintf.c:INITIAL->1.1 
# 	klibc.cvsroot/klibc/vsnprintf.c:INITIAL->1.1 
# 
# --- /dev/null	2005-04-30 18:00:24.840397008 +0200
# +++ klibc/klibc.cvsroot/snprintf.c	2005-05-02 19:57:42.879913000 +0200
# @@ -0,0 +1,19 @@
# +/*

my $patch = $ARGV[0];

my %committer = (
	brendan  => [ 'Brendan Cully',   'brendan@kublai.com' ],
	me       => [ 'Michael Elkins',  'me@sigpipe.org' ],
	roessler => [ 'Thomas Roessler', 'roessler@does-not-exist.org' ]
);

my @log = ();

$ENV{GIT_AUTHOR_EMAIL} = "";
$ENV{GIT_COMMITTER_EMAIL} = "";

open (my $fd, $patch);
while (my $line = <$fd>) {
	if ($line =~ m/^Date: (.*)/) {
		$ENV{GIT_AUTHOR_DATE} = $1;

	} elsif ($line =~ m/^Author: (.*)/) {
		if (defined($committer{$1})) {
			$ENV{GIT_COMMITTER_NAME}  = @{$committer{$1}}[0];
			$ENV{GIT_COMMITTER_EMAIL} = @{$committer{$1}}[1];
			$ENV{GIT_AUTHOR_NAME}         = @{$committer{$1}}[0];
			$ENV{GIT_AUTHOR_EMAIL}        = @{$committer{$1}}[1];
		} else {
			$ENV{GIT_COMMITTER_NAME} = $1;
			$ENV{GIT_AUTHOR_NAME}        = $1;
		}

	} elsif ($line =~ m/^Log:/) {
		while (my $line = <$fd>) {
			if ($line =~ m/^Members: $/) {
				pop(@log);
				last;
			} elsif ($line =~ /^From: (.+) <([^>]+@[^>]+)>$/) {
				$ENV{GIT_AUTHOR_NAME}  = $1;
				$ENV{GIT_AUTHOR_EMAIL} = $2;
			}
			push @log, $line;
		}
	}
}

close($fd);

my ($fh, $logfile) = tempfile(CLEANUP => 1);
print $fh @log;
system("git patch $patch < $logfile");
close($fh);

Re: gitweb wishlist

[Kay Sievers]

On Fri, 2005-05-20 at 12:22 -0700, Linus Torvalds wrote:
> 
> On Fri, 20 May 2005, Kay Sievers wrote:
> > 
> > Somehting like this?:
> >   http://kernel.org/git/?p=git/git.git;a=commitdiff;h=de809dbbce497e0d107562615c1d85ff35b4e0c5
> 
> Btw, at least for me, this looks much more interesting than the "commit" 
> thing, and maybe it would make sense to make the summary links be to the 
> "commitdiff" instead of the "commit"?

How about this:
  http://www.kernel.org/git/?p=git/git.git;a=summary

The default link is still the same, but you can use the link at the end.

Kay

Re: gitweb wishlist

[Kay Sievers]

[-- Attachment #1: Type: text/plain, Size: 1607 bytes --]

On Fri, 2005-05-20 at 13:50 -0700, H. Peter Anvin wrote:
> Linus Torvalds wrote:
> > 
> > Oh, btw, I notice that you moved klibc over to git - care to share your
> > cvs->git script (I assume you scripted it ;)? That would seem to be an 
> > obvious addition to the core stuff..
> > 
> 
> Actually, Kay did the conversion... the scripts are clearly very 
> cantankerous, because if *I* run them -- I tried -- they don't work! 
> Since it's Kay's work, I'll leave them to him, but I would definitely 
> love to move more of my CVS repos over to git, especially syslinux.

Here we go;

These scripts are just a quick hack, I just wanted to know how nice the
stupid cvs file history can be converted to git-committs.

It exports the CVS repo with the help of the nice cvsps to individual
patches. (Every patch contains something like a "ChangeSet" by searching
for file revisions with the same checkin-date)

Then the patches with the header are split into individual files for
committing it into git (similar to Linus' git-mbox-tools).

If we reach a CVS tag with a patch during sequential patching, the
script throws away the whole current working tree and checks the
revision out of CVS. This way we make sure, that the git-tag matches
tree CVS has tagged. (I've encountered two mismatches in the
"patch-chain" with the CVS revision-tag. These corrections are hardcoded
into the script. :)

For every CVS revision-tag a git-tag without any content except the name
is created.

And the klibc-repo was created with a patched git-commit to fake the
commit date with the author date. :)

Good luck with it,
Kay

[-- Attachment #2: export-to-git.sh --]
[-- Type: application/x-shellscript, Size: 2581 bytes --]

[-- Attachment #3: split-cvsps-patch.pl --]
[-- Type: application/x-perl, Size: 2386 bytes --]

Re: gitweb wishlist

[H. Peter Anvin]

Kay Sievers wrote:
> 
> And the klibc-repo was created with a patched git-commit to fake the
> commit date with the author date. :)
> 

In fact, I kind of wish we'd also made committer == author.

Since this whole thing is an import from another revision control 
system, one really wants that.  It's one of those very rare situations 
in which fudging the commit date is not only fully legitimate, but darn 
near required.

	-hpa

Re: gitweb wishlist

[Linus Torvalds]

On Sat, 21 May 2005, Kay Sievers wrote:
> 
> These scripts are just a quick hack, I just wanted to know how nice the
> stupid cvs file history can be converted to git-committs.

Ugh, indeed.

Is it a cvsps bug or what that causes you to have to re-order the patches?  
Or is it that you don't handle branches or something in CVS? The fact that
you also remove one of the tags "suppress ash-branch" in that same number
sequence that you had to fix up by re-ordering seems to imply that the
breakage has something to do with branching.

Does anybody have any suggestions for a nice and smallish CVS project that
has branches that I should look at?

		Linus

Re: gitweb wishlist

[Linus Torvalds]

[ Thomas added to cc, since he seems to have also worked on this ]

On Fri, 20 May 2005, H. Peter Anvin wrote:
> 
> Here is my "main" OSS CVS repository; look at the syslinux module.  It 
> has at least some minor branching.

Ok, "cvsps" output scares me. I wonder what

	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
	    memdisk/init32.asm:1.3=after, memdisk/Makefile:1.26=before. Treated as 'before'
	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
	    memdisk/init32.asm:1.3=after, memdisk/e820test.c:1.7=before. Treated as 'before'
	...

means..

Also, your syslinux repo is interesting and shows another thing: doing a

	cvsps -g -p separate

ends badly with

	Directing PatchSet 938 to file separate/938.patch
	cvs rdiff: failed to read diff file header /tmp/cvso8PswZ for mdiskchk.com,v: end of file
	system command returned non-zero exit status: 1: aborting

which doesn't look very promising and causes an empty diff for
mdiskck.com. Trying with --cvs-direct shows the reason:

	Index: syslinux/sample/mdiskchk.com
	===================================================================
	RCS file: 
	/home/torvalds/src/osscvs/cvsroot/syslinux/sample/mdiskchk.com,v
	retrieving revision 1.1
	retrieving revision 1.2
	diff -u -r1.1 -r1.2
	Binary files /tmp/cvsU6MGU0 and /tmp/cvsiskFVR differ

which shows that anything that bases itself of diffs (ie uses "-g" with
cvsps) is just doomed to failure, since there's no good way to handle
binary data. Both Kay's and Thomas' scripts try to do the "-g" thing, 
that's just not right.

So the cvs->git thing would need to be based on the actual objects, which 
obviously fits git quite well, but I was really hoping to have cvsps give 
some nice intermediate format..

So it looks like we should avoid the diff format, and instead use

	cvsps -p separate

and then just parse the "Members" thing and turning each of them either
into a "delete"  (for ->.*DEAD) or "cvs checkout -rxxx" (for ".*->xxx").

Handling branches by literally treating them as different heads in git 
sounds quite simple, and indeed it looks like the basic logic for cvs->git 
translation would be

	for-each-patch-from-cvsps
	do
		git-read-tree -m branchname-from-patch
		git-update-cache -f -u -q -a
		for-each-member-in-patch
		do
			if [ DEAD ]; then
				rm member
				git-update-cache --remove member
			else
				cvs co -rREV member
				git-update-cache --add member
			fi
			cat commit-message-from-patch | 
				git-commit-tree $(git-write-tree) -p branchname-from-patch > .git/revs/heads/branchname-from-patch
		done
	done

which looks like it should work, and handle binary files right.

There seems to be two questions:

 - what to do about branch creation (ie a branch name we haven't seen
   before): it looks like cvsps doesn't tell you what the _originating_
   branch was for a new branch (that may be my confusion - maybe you can't
   create branches off branches in CVS?)

   For syslinux, it looks like you can always base it on HEAD, or possibly 
   just the previous patch (which looks like it is always HEAD). The above 
   pseudo-script will actually do that automatically, simply by virtue of
   the "git-read-tree -m" at the top of the loop failing when the
   branchname doesn't exist yet.

 - whether to bother to create merge entries for when somebody tried to 
   merge a branch back or forth in CVS. 

   CVS fundamentally doesn't have the notion of such a thing, and cvsps 
   can't either. But we could try to guess, based on the commit message, 
   perhaps.

   NOTE! Such a "merge" would not have any real GIT merge functionality 
   what-so-ever. It would just introduce a second parent into the commit, 
   nothing more.

Bah. What crud.

		Linus

Re: gitweb wishlist

[Matthias Urlichs]

Hi, Kay Sievers wrote:

> Something like that: :)

Cool.

More feature requests:  ;-)
- Alternate white and almost-white backgrounds in the lists (all of them ;-)
  so that wide-screened people like me don't lose context when their eyes
  travel the long road from left to right edge of the screen. ;-)

- Merges currently don't have diff links. It'd be nice to have one for
  each parent.

- File diffs have the "diff" link on the *parent*, not on the child.
  That's counter-intuitive -- if I want to see what the Foo patch changes,
  I should be able to click on the "diff" link on _that_ line, not the one
  below it. Example:
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=history;h=9636273dae265b9354b861b373cd43cd76a6d0fe;f=MAINTAINERS

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de

cvs->git (was Re: gitweb wishlist)

[Matthias Urlichs]

Hi, Linus Torvalds wrote:

> Bah. What crud.

I have an old CVS->BK merge script lying around which does all of this
reasonably correctly. If somebody wants to git-ify it, be my guest
(I won't have time in the foreseeable futurre :-( ); it's in the
rsync://server.smurf.noris.de/sourcemgr.git/#main repository as
bin/b.cvs{,.pl}.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de

Re: gitweb wishlist

[Kay Sievers]

On Sat, 2005-05-21 at 09:29 +0200, Matthias Urlichs wrote:
> Hi, Kay Sievers wrote:
> 
> > Something like that: :)
> 
> Cool.
> 
> More feature requests:  ;-)
> - Alternate white and almost-white backgrounds in the lists (all of them ;-)
>   so that wide-screened people like me don't lose context when their eyes
>   travel the long road from left to right edge of the screen. ;-)

Done!

> - Merges currently don't have diff links. It'd be nice to have one for
>   each parent.

Done! But I'm not sure if that is really useful. It may create a very
very big diff. :)

> - File diffs have the "diff" link on the *parent*, not on the child.
>   That's counter-intuitive -- if I want to see what the Foo patch changes,
>   I should be able to click on the "diff" link on _that_ line, not the one
>   below it. Example:
> http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=history;h=9636273dae265b9354b861b373cd43cd76a6d0fe;f=MAINTAINERS

No, the "diff" link is a diff against the current commit not an
incremental one from revision to revision. Me may change that, I'm not
sure what's the best here.

Kay

Re: gitweb wishlist

[David Mansfield]

[-- Attachment #1: Type: text/plain, Size: 4239 bytes --]

Hi Linus,

Linus Torvalds wrote:
> [ Thomas added to cc, since he seems to have also worked on this ]
> 
> On Fri, 20 May 2005, H. Peter Anvin wrote:
> 
>>Here is my "main" OSS CVS repository; look at the syslinux module.  It 
>>has at least some minor branching.
> 
> 
> Ok, "cvsps" output scares me. I wonder what
> 
> 	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
> 	    memdisk/init32.asm:1.3=after, memdisk/Makefile:1.26=before. Treated as 'before'
> 	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
> 	    memdisk/init32.asm:1.3=after, memdisk/e820test.c:1.7=before. Treated as 'before'
> 	...
> 
> means..
> 

Ok.  I'll tell you.  It means that the committer uses bad practices in 
tagging ;-)  It generally means that force tag (cvs tag -F <file>) was 
used on a specific file.  Here's the scenario:

cvsps is trying to associate a tag to a specific commit.  But in the cvs 
world this is not always at all possible.  If, for example, a commit 
made and  all files are tagged.  Now some random file is modified and 
committed.  Then, a bug is found in a file from the previously tagged 
set, say the file 'memdisk/init32.asm'.  The bug is fixed, committed and 
the tag is MOVED for _just that file_ forward to the new version.  Now 
there is no commit that can be associated with the tag.  In this case, 
cvsps believes this to be a 'FUNKY' tag.  There is a more pathological 
case having to do with 'INVALID' tags...  It's enough to make a grown 
man cry.

> Also, your syslinux repo is interesting and shows another thing: doing a
> 
> 	cvsps -g -p separate
> 
> ends badly with
> 
> 	Directing PatchSet 938 to file separate/938.patch
> 	cvs rdiff: failed to read diff file header /tmp/cvso8PswZ for mdiskchk.com,v: end of file
> 	system command returned non-zero exit status: 1: aborting
> 
> which doesn't look very promising and causes an empty diff for
> mdiskck.com. Trying with --cvs-direct shows the reason:
> 
> 	Index: syslinux/sample/mdiskchk.com
> 	===================================================================
> 	RCS file: 
> 	/home/torvalds/src/osscvs/cvsroot/syslinux/sample/mdiskchk.com,v
> 	retrieving revision 1.1
> 	retrieving revision 1.2
> 	diff -u -r1.1 -r1.2
> 	Binary files /tmp/cvsU6MGU0 and /tmp/cvsiskFVR differ
> 
> which shows that anything that bases itself of diffs (ie uses "-g" with
> cvsps) is just doomed to failure, since there's no good way to handle
> binary data. Both Kay's and Thomas' scripts try to do the "-g" thing, 
> that's just not right.
> 

I accept patches ;-)  Honestly, handling binary data should be trivial I 
just haven't had the interest, and surprisingly noone else on the 
internet ever has.  The only binary file in the kernel appears to be the 
logo.gif, according to Ingo.

[ discussion on working around broken handling of binary files in cvsps]
> 
> There seems to be two questions:
> 
>  - what to do about branch creation (ie a branch name we haven't seen
>    before): it looks like cvsps doesn't tell you what the _originating_
>    branch was for a new branch (that may be my confusion - maybe you can't
>    create branches off branches in CVS?)
> 
>    For syslinux, it looks like you can always base it on HEAD, or possibly 
>    just the previous patch (which looks like it is always HEAD). The above 
>    pseudo-script will actually do that automatically, simply by virtue of
>    the "git-read-tree -m" at the top of the loop failing when the
>    branchname doesn't exist yet.
> 

See attached patch to cvsps.c which displays 'Ancestor branch' when this 
differs from Branch.

>  - whether to bother to create merge entries for when somebody tried to 
>    merge a branch back or forth in CVS. 
> 
>    CVS fundamentally doesn't have the notion of such a thing, and cvsps 
>    can't either. But we could try to guess, based on the commit message, 
>    perhaps.
> 
>    NOTE! Such a "merge" would not have any real GIT merge functionality 
>    what-so-ever. It would just introduce a second parent into the commit, 
>    nothing more.
> 
> Bah. What crud.
> 

Hey, a polished turd is only so shiny...  cvsps is a 99% solution [to 
the problem of extracting metatdata from cvs] only and cvs makes the 
other 1% impossible.

David

[-- Attachment #2: show-ancestor-branch.patch --]
[-- Type: text/x-patch, Size: 752 bytes --]

--- cvsps.c~	2003-04-11 10:06:01.000000000 -0400
+++ cvsps.c	2005-05-23 23:26:12.110231536 -0400
@@ -1402,6 +1402,16 @@
 	   tm->tm_hour, tm->tm_min, tm->tm_sec);
     printf("Author: %s\n", ps->author);
     printf("Branch: %s\n", ps->branch);
+    
+    /* check if ancestor was different branch */
+    if (!list_empty(&ps->members)) 
+    {
+	    PatchSetMember * psm = list_entry(ps->members.next, PatchSetMember, link);
+	    const char * abr = psm->pre_rev ? psm->pre_rev->branch : NULL;
+	    if (abr && strcmp(ps->branch, abr) != 0)
+		    printf("Ancestor branch: %s\n", abr);
+    }
+
     printf("Tag: %s %s\n", ps->tag ? ps->tag : "(none)", tag_flag_descr[ps->tag_flags]);
     printf("Log:\n%s\n", ps->descr);
     printf("Members: \n");

Re: gitweb wishlist

[H. Peter Anvin]

David Mansfield wrote:
> 
> Ok.  I'll tell you.  It means that the committer uses bad practices in 
> tagging ;-)  It generally means that force tag (cvs tag -F <file>) was 
> used on a specific file.  Here's the scenario:
> 
> cvsps is trying to associate a tag to a specific commit.  But in the cvs 
> world this is not always at all possible.  If, for example, a commit 
> made and  all files are tagged.  Now some random file is modified and 
> committed.  Then, a bug is found in a file from the previously tagged 
> set, say the file 'memdisk/init32.asm'.  The bug is fixed, committed and 
> the tag is MOVED for _just that file_ forward to the new version.  Now 
> there is no commit that can be associated with the tag.  In this case, 
> cvsps believes this to be a 'FUNKY' tag.  There is a more pathological 
> case having to do with 'INVALID' tags...  It's enough to make a grown 
> man cry.
> 

This is only pathological if the tag now represents a state that never 
actually existed in the history of the repository.  I don't believe 
there are any such cases in the syslinux repository; I could be wrong, 
but I am *highly* sceptical.

> 
> I accept patches ;-)  Honestly, handling binary data should be trivial I 
> just haven't had the interest, and surprisingly noone else on the 
> internet ever has.  The only binary file in the kernel appears to be the 
> logo.gif, according to Ingo.
> 
> [ discussion on working around broken handling of binary files in cvsps]
> 

Actually, as long as we can create the tree that exists between each 
changeset, we should be OK.

> 
> Hey, a polished turd is only so shiny...  cvsps is a 99% solution [to 
> the problem of extracting metatdata from cvs] only and cvs makes the 
> other 1% impossible.
> 

No sh*t...

	-hpa

Re: gitweb wishlist

[Linus Torvalds]

On Mon, 23 May 2005, David Mansfield wrote:
> > 
> > Bah. What crud.
> > 
> 
> Hey, a polished turd is only so shiny...  cvsps is a 99% solution [to 
> the problem of extracting metatdata from cvs] only and cvs makes the 
> other 1% impossible.

The "what crud" refers to cvs. cvsps seems to be a great way to make a
tool to migrate away from CVS (or if forced to use CVS, at least show it
in a sane manner). So don't take it the wrong way.

I've gotten side-tracked with purely git issues, and since I don't 
actually have any CVS archives, the cvs->git translation will be on the 
back-burner for a while, but your "Ancestor branch" patch seems to at 
least solve the problem that cvsps didn't show all the information that 
was there. So now I know how to do branches, even if I don't think I'd 
ever _really_ merge them back (which is as much info as CVS contains). 

They'd just be dangling references, ie you could get to them if you wanted 
to, for historical reasons, and they could be merged merged by hand, of 
course. Some day..

		Linus

Re: gitweb wishlist

[David Mansfield]

H. Peter Anvin wrote:
> David Mansfield wrote:
> 
>>
>> Ok.  I'll tell you.  It means that the committer uses bad practices in 
>> tagging ;-)  It generally means that force tag (cvs tag -F <file>) was 
>> used on a specific file.  Here's the scenario:
>>
>> cvsps is trying to associate a tag to a specific commit.  But in the 
>> cvs world this is not always at all possible.  If, for example, a 
>> commit made and  all files are tagged.  Now some random file is 
>> modified and committed.  Then, a bug is found in a file from the 
>> previously tagged set, say the file 'memdisk/init32.asm'.  The bug is 
>> fixed, committed and the tag is MOVED for _just that file_ forward to 
>> the new version.  Now there is no commit that can be associated with 
>> the tag.  In this case, cvsps believes this to be a 'FUNKY' tag.  
>> There is a more pathological case having to do with 'INVALID' tags...  
>> It's enough to make a grown man cry.
>>
> 
> This is only pathological if the tag now represents a state that never 
> actually existed in the history of the repository.  I don't believe 
> there are any such cases in the syslinux repository; I could be wrong, 
> but I am *highly* sceptical.
> 

I didn't mean that YOUR repository had more pathological stuff in it, 
just that SOME do.  'FUNKY' tags are not really that bad, it's just that 
there is not a single commit to assign them to (i.e. at no point were 
all of the objects in the repository at that state simultaneously), 
which makes the import of such a tag difficult into a more commit 
oriented system.

Another way to reach 'funky'ness is to modify a file, commit and tag, 
without having done a 'cvs update' first (and a colleague has done a 
commit since your last 'cvs update')

David

Re: gitweb wishlist

[Thomas Glanzmann]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

Hello,

> 	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
> 	    memdisk/init32.asm:1.3=after, memdisk/Makefile:1.26=before. Treated as 'before'
> 	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
> 	    memdisk/init32.asm:1.3=after, memdisk/e820test.c:1.7=before. Treated as 'before'
> 	...

actually I think this is the broken upstream version. It can't parse
dates right. Just look at the exported patches and see if them all from
1970. However the debian package has a patch in which solves it:

maybe you should try with the attached patch or with the version that
comes with debian sarge. I also reported this problem a while back to
the original author.

	Thomas

[-- Attachment #2: cvsps_2.0rc1-5.diff --]
[-- Type: text/plain, Size: 1785 bytes --]

--- cvsps-2.0rc1.orig/util.c
+++ cvsps-2.0rc1/util.c
@@ -13,6 +13,7 @@
 #include <time.h>
 #include <errno.h>
 #include <signal.h>
+#include <regex.h>
 #include <sys/stat.h>
 #include <sys/time.h>
 #include <sys/types.h>
@@ -140,24 +141,51 @@
     return *res;
 }
 
+static int get_int_substr(const char * str, const regmatch_t * p)
+{
+    char buff[256];
+    memcpy(buff, str + p->rm_so, p->rm_eo - p->rm_so);
+    buff[p->rm_eo - p->rm_so] = 0;
+    return atoi(buff);
+}
+
 void convert_date(time_t * t, const char * dte)
 {
-    /* HACK: this routine parses two formats,
-     * 1) 'cvslog' format YYYY/MM/DD HH:MM:SS
-     * 2) time_t formatted as %d
-     */
-       
-    if (strchr(dte, '/'))
+    static regex_t date_re;
+    static int init_re;
+
+#define MAX_MATCH 16
+    size_t nmatch = MAX_MATCH;
+    regmatch_t match[MAX_MATCH];
+
+    if (!init_re) 
+    {
+	if (regcomp(&date_re, "([0-9]{4})[-/]([0-9]{2})[-/]([0-9]{2}) ([0-9]{2}):([0-9]{2}):([0-9]{2})", REG_EXTENDED)) 
+	{
+	    fprintf(stderr, "FATAL: date regex compilation error\n");
+	    exit(1);
+	}
+	init_re = 1;
+    }
+    
+    if (regexec(&date_re, dte, nmatch, match, 0) == 0)
     {
+	regmatch_t * pm = match;
 	struct tm tm;
+
+	/* first regmatch_t is match location of entire re */
+	pm++;
 	
-	memset(&tm, 0, sizeof(tm));
-	sscanf(dte, "%d/%d/%d %d:%d:%d", 
-	       &tm.tm_year, &tm.tm_mon, &tm.tm_mday, 
-	       &tm.tm_hour, &tm.tm_min, &tm.tm_sec);
-	
+	tm.tm_year = get_int_substr(dte, pm++);
+	tm.tm_mon  = get_int_substr(dte, pm++);
+	tm.tm_mday = get_int_substr(dte, pm++);
+	tm.tm_hour = get_int_substr(dte, pm++);
+	tm.tm_min  = get_int_substr(dte, pm++);
+	tm.tm_sec  = get_int_substr(dte, pm++);
+
 	tm.tm_year -= 1900;
 	tm.tm_mon--;
+	tm.tm_isdst = 0;
 	
 	*t = mktime(&tm);
     }

Re: gitweb wishlist

[H. Peter Anvin]

David Mansfield wrote:
>>
>> This is only pathological if the tag now represents a state that never 
>> actually existed in the history of the repository.  I don't believe 
>> there are any such cases in the syslinux repository; I could be wrong, 
>> but I am *highly* sceptical.
> 
> I didn't mean that YOUR repository had more pathological stuff in it, 
> just that SOME do.  'FUNKY' tags are not really that bad, it's just that 
> there is not a single commit to assign them to (i.e. at no point were 
> all of the objects in the repository at that state simultaneously), 
> which makes the import of such a tag difficult into a more commit 
> oriented system.
> 
> Another way to reach 'funky'ness is to modify a file, commit and tag, 
> without having done a 'cvs update' first (and a colleague has done a 
> commit since your last 'cvs update')
> 

Not sure, sounds more likely.

Either which way, I guess there are two ways to deal with them in 'git'; 
either as standalone trees (tags pointing to tree objects), or probably 
more sensical, as impromptu branches if one can find a sane origin object.

	-hpa

Re: gitweb wishlist

[Linus Torvalds]

On Mon, 23 May 2005, Linus Torvalds wrote:
> 
> I've gotten side-tracked with purely git issues, and since I don't 
> actually have any CVS archives, the cvs->git translation will be on the 
> back-burner for a while, but your "Ancestor branch" patch seems to at 
> least solve the problem that cvsps didn't show all the information that 
> was there.

Naff.

I just checked in a "cvs2git.c" file in the "tools" project (which has my 
patch application stuff).

It's still buggy, and it's hacky as hell, but you can basically do 
something like this:

	cvsps | cvs2git > script

with the normal setup for "cvsps", and "cvs2git" needs one additional
stage, namely it wants to know the RCSDIR where to find the RCS files
(that should be basically "$CVSROOT/module").

That _script_ then creates a git archive. Very hacky. So after you've 
successfully created the conversion script, check it to see that it looks 
sane, and then do

	sh script

and the end result is a git'ified version of your CVS repo (and a 
corrupted working directory, btw, so look out. It _shouldn't_ corrupt 
your old CVS repo, though, so it should be ok).

It has the logic for branches, but it doesn't work, and I'm fed up enough
with CVS and RCS for the moment that I'm not going to work on it any more
tonight. I don't know what stupid bug I have (I've had about a million of
them on this silly program), but it's at a point where I think others
might find it interesting, and it's probably/hopefully some really
embarrassing typo or something and easily fixed.

It converted Peter's "syslinux" repository in a couple of minutes, 
resulting in 1038 commits (it _should_ have resulted in 1046 commits, 
that's the branch thing afaik) and most of it looks sane:

	diff-tree cfb715c827e19226a446d47c98a7460fd94633ff (from a809559323f1b370717e475dd252b24686f97727)
	Author: hpa <hpa>
	Date:   Thu May 19 22:30:50 2005 -0700
	    
	    gcc4 compilation fix
	
	
	diff-tree a809559323f1b370717e475dd252b24686f97727 (from 4d65331b50a7b5ce858bb55a58f37b17ebc26c72)
	Author: hpa <hpa>
	Date:   Sun May 8 22:47:03 2005 -0700
	    
	    New Multiboot module; increase command line limit to 1023
	
	
	diff-tree 4d65331b50a7b5ce858bb55a58f37b17ebc26c72 (from e88244753d528f695790adc96f0542d20dc33882)
	Author: hpa <hpa>
	Date:   Fri Apr 29 07:08:03 2005 -0700
	    
	    Don't clobber live registers, it's not nice
	
	
	diff-tree e88244753d528f695790adc96f0542d20dc33882 (from a49e189e35d208648a0d0b52ff652a5f3f8a707e)
	Author: hpa <hpa>
	Date:   Fri Apr 29 07:05:52 2005 -0700
	    
	    Use the correct register

	...
	...
	...

	diff-tree 350772d45425a85dae86ec721d6bd3fde5595d50 (from 47ee894e7821f50cb83ea14b08132337577b2a1e)
	Author: hpa <hpa>
	Date:   Sat Jan 31 13:24:35 1998 -0800
	    
	    Slightly less ugly Id tag.
	
	
	diff-tree 47ee894e7821f50cb83ea14b08132337577b2a1e (from a8b52f1c31055049b276d14c67436d06dd7757aa)
	Author: hpa <hpa>
	Date:   Sat Jan 31 13:22:38 1998 -0800
	    
	    Added Id tags.
	
	
	diff-tree b924672aadb2c3b7f3cac1aaf52fbb4a1ed86b8d (from root)
	Author: hpa <hpa>
	Date:   Sat Jan 31 13:16:05 1998 -0800
	    
	    Initial revision

And btw, it's definitely cvsps that does all the heavy lifting here. 
"cvs2git" itself is 255 lines of horrid crud, and should have been 
written in perl, except I only do C..

		Linus

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Linus Torvalds wrote:
> 
> It has the logic for branches, but it doesn't work, and I'm fed up enough
> with CVS and RCS for the moment that I'm not going to work on it any more
> tonight.

I'm back, and yes, it was a really stupid thing.

However, David, I need more help deciphering "cvsps" output..

Fixing the branch handling shows that cvsps does some really strange
things with the newly added "Ancestor grpah". Here's one example:

	---------------------
	PatchSet 372 
	Date: 2002/02/03 21:37:50
	Author: hpa
	Branch: syslinux-1_6x-1
	Ancestor branch: HEAD
	Tag: syslinux-1_67 
	Log:
	New mailing list information
	
	Members: 
	        syslinux.doc:1.48->1.48.2.1 
	
	---------------------
	PatchSet 373 
	Date: 2002/02/11 23:08:47
	Author: hpa
	Branch: HEAD
	Tag: (none) 
	Log:
	tftpd32 needs version 2.11 or later.
	
	Members: 
	        pxelinux.doc:1.28->1.29 
	
	---------------------
	PatchSet 374 
	Date: 2002/02/18 23:43:43
	Author: hpa
	Branch: syslinux-1_6x-1
	Ancestor branch: HEAD
	Tag: syslinux-1_6x-merge-2 
	Log:
	Actually make the -o option work properly.
	
	Members: 
	        syslinux.c:1.13->1.13.2.1 
	
	---------------------

note how both 372 _and_ 374 claim to have HEAD as their ancestor, and are 
on the "syslinux-1_6x-1" branch. What's up with that? Right now this 
causes my git archive to first create 372 as a branch off HEAD, and then 
overwrite that with 374, resulting in a dangling branch for 372 that 
_exists_, but it's not reachable any more, because the branch name that it 
used has been overwritten by the _new_ branch off HEAD.

Side note: cvs2git is pretty robust since it doesn't rely on patches
anywhere, so the head of the branch likely ends up being correct, if that
"syslinux.doc" file has been modified anywhere else in the branch. So this
_usually_ just results in (a) git-fsck-cache complaining about unreachable
commits and (b) possible history being hard to find.

Maybe this cvs2git behaviour is the right thing to do, and what really
happened was that the changes described by PatchSet 372 aren't really
available any more even in CVS, unless you go back by date or something 
like that.

However, I suspect it's a cvsps bug in the "ancestor branch" thing. I
could work around it by just saying "if I have already seen this branch,
I'll ignore the ancestor information".

So I'd like to know whether this is a cvsps issue or whether I actually
ended up doing the right thing and it really should be a dangling
branch-name that got re-used...

(And if it's a cvsps issue, I'd obviously prefer to get a cvsps patch 
instead of having a questionable workaround in cvs2git).

"Davi-Mansfieldobi, you're our only hope.."

		Linus

Re: gitweb wishlist

[David Mansfield]

Linus Torvalds wrote:
> 
> On Mon, 23 May 2005, Linus Torvalds wrote:
> 
>>I've gotten side-tracked with purely git issues, and since I don't 
>>actually have any CVS archives, the cvs->git translation will be on the 
>>back-burner for a while, but your "Ancestor branch" patch seems to at 
>>least solve the problem that cvsps didn't show all the information that 
>>was there.
> 
> 
> Naff.
> 
> I just checked in a "cvs2git.c" file in the "tools" project (which has my 
> patch application stuff).
> 
> It's still buggy, and it's hacky as hell, but you can basically do 
> something like this:
> 
> 	cvsps | cvs2git > script
> 
> with the normal setup for "cvsps", and "cvs2git" needs one additional
> stage, namely it wants to know the RCSDIR where to find the RCS files
> (that should be basically "$CVSROOT/module").
> 
> That _script_ then creates a git archive. Very hacky. So after you've 
> successfully created the conversion script, check it to see that it looks 
> sane, and then do
> 
> 	sh script
> 
> and the end result is a git'ified version of your CVS repo (and a 
> corrupted working directory, btw, so look out. It _shouldn't_ corrupt 
> your old CVS repo, though, so it should be ok).

I'll take a look.  One problem is that many folks use non-local cvs... 
not sure if that will be an issue.  I'll look to cleaning this up if 
necessary.

> 
> It has the logic for branches, but it doesn't work, and I'm fed up enough
> with CVS and RCS for the moment that I'm not going to work on it any more
> tonight. I don't know what stupid bug I have (I've had about a million of
> them on this silly program), but it's at a point where I think others
> might find it interesting, and it's probably/hopefully some really
> embarrassing typo or something and easily fixed.
> 

I actually found an issue with the 30-second ancestor branch patch I 
sent and I'm doing that one properly now.  Once that's done I can look 
at the branch capture logic in cvs2git and see if anything pops out.


> It converted Peter's "syslinux" repository in a couple of minutes, 
> resulting in 1038 commits (it _should_ have resulted in 1046 commits, 
> that's the branch thing afaik) and most of it looks sane:

Really cool.  What's 8 commits between friends?

David

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Linus Torvalds wrote:
> 
> Fixing the branch handling shows that cvsps does some really strange
> things with the newly added "Ancestor grpah". Here's one example:

Ahh, looking at cvsps source, I think I see what's going on. 

It's deciding the "previous branch" by looking at what the previous branch 
for the first individual file in the PatchSet was, which fails because in 
this case, PatchSet 372 was changing "syslinux.doc", and Patchset 374 was 
changing "syslinux.c", and thus the previous version of the individual 
_files_ were both in the HEAD branch.

So it does look like I should just ignore the "Ancestor branch" 
information if the new branch already existed.

Of course, some semantics will never be translatable when trying to treat 
CVS as a sane system (ie treating CVS as if it was changeset-based is 
always going to cause strange corner cases since it really is file-based), 
but that should most likely give the best approximation of what a 
conversion should do.

		Linus

Re: gitweb wishlist

[Thomas Glanzmann]

Hello Linus,
I tried cvs2git and have the following problem:

	---------------------
	PatchSet 26
	Date: 1998/06/20 03:53:44
	Author: roessler
	Branch: mutt-0-93
	Ancestor branch: HEAD
	Tag: (none)
	Log:
	documenting alias-path

	Members:
		doc/manual.sgml:1.2->1.2.2.1

	---------------------

And your script does that:

	export GIT_COMMITTER_NAME=roessler
	export GIT_COMMITTER_EMAIL=roessler
	export GIT_AUTHOR_NAME=roessler
	export GIT_AUTHOR_EMAIL=roessler
	export GIT_AUTHOR_DATE='1998/06/20 03:53:44'
	ln -sf refs/heads/'master' .git/HEAD
	git-read-tree -m HEAD
	git-checkout-cache -f -u -a
	mkdir -p doc
	co -p -r1.2.2.1 '/home/cip/adm/sithglan/work/mutt/cvsrepository/doc/Attic/manual.sgml,v' > 'doc/manual.sgml'
	git-update-cache --add -- 'doc/manual.sgml'
	tree=$(git-write-tree)
	cat > .cmitmsg <<EOFMSG
	documenting alias-path
	EOFMSG
	commit=$(cat .cmitmsg | git-commit-tree $tree -p HEAD)
	echo $commit > .git/HEAD

The problem might be that this is the first commit in the branch. But I thought
it should end up in refs/heads/mutt-0-93. The problem is that this ends
up a empty file and next time the script is working on it, it fails
because the branch is empty:

	+ export GIT_COMMITTER_NAME=roessler
	+ GIT_COMMITTER_NAME=roessler
	+ export GIT_COMMITTER_EMAIL=roessler
	+ GIT_COMMITTER_EMAIL=roessler
	+ export GIT_AUTHOR_NAME=roessler
	+ GIT_AUTHOR_NAME=roessler
	+ export GIT_AUTHOR_EMAIL=roessler
	+ GIT_AUTHOR_EMAIL=roessler
	+ export 'GIT_AUTHOR_DATE=1998/06/20 07:12:32'
	+ GIT_AUTHOR_DATE=1998/06/20 07:12:32
	+ ln -sf refs/heads/mutt-0-93 .git/HEAD
	+ git-read-tree -m HEAD
	usage: git-read-tree (<sha> | -m <sha1> [<sha2> <sha3>])
	+ git-checkout-cache -f -u -a
	+ co -p -r1.1.1.1.2.2 /home/cip/adm/sithglan/work/mutt/cvsrepository/handler.c,v
	/home/cip/adm/sithglan/work/mutt/cvsrepository/handler.c,v  -->  standard output
	revision 1.1.1.1.2.2
	+ git-update-cache --add -- handler.c
	++ git-write-tree
	+ tree=9e4d085838e4e62a8c4236a6713a7dd8d7b07b4e
	+ cat
	++ cat .cmitmsg
	++ git-commit-tree 9e4d085838e4e62a8c4236a6713a7dd8d7b07b4e -p HEAD
	usage: git-commit-tree <sha1> [-p <sha1>]* < changelog
	+ commit=
	+ echo

	Thomas

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Thomas Glanzmann wrote:
> 
> And your script does that:
> 
> 	export GIT_COMMITTER_NAME=roessler
> 	export GIT_COMMITTER_EMAIL=roessler
> 	export GIT_AUTHOR_NAME=roessler
> 	export GIT_AUTHOR_EMAIL=roessler
> 	export GIT_AUTHOR_DATE='1998/06/20 03:53:44'
> 	ln -sf refs/heads/'master' .git/HEAD
> 	git-read-tree -m HEAD
> 	git-checkout-cache -f -u -a
> 	mkdir -p doc
> 	co -p -r1.2.2.1 '/home/cip/adm/sithglan/work/mutt/cvsrepository/doc/Attic/manual.sgml,v' > 'doc/manual.sgml'
> 	git-update-cache --add -- 'doc/manual.sgml'
> 	tree=$(git-write-tree)
> 	cat > .cmitmsg <<EOFMSG
> 	documenting alias-path
> 	EOFMSG
> 	commit=$(cat .cmitmsg | git-commit-tree $tree -p HEAD)
> 	echo $commit > .git/HEAD
> 
> The problem might be that this is the first commit in the branch. But I thought
> it should end up in refs/heads/mutt-0-93.

Yes, you're using the cvs2git from yesterday, which didn't write the new
commit to the right branch. This is part of the branch fixing I've done.

Wait another few minutes and I'll commit my fix the problem with cvsps
branch handling (and I need to escape '$' in <<EOFMSG handling).

		Linus

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Linus Torvalds wrote:
> 
> Wait another few minutes and I'll commit my fix the problem with cvsps
> branch handling (and I need to escape '$' in <<EOFMSG handling).

Ok, committed. It takes a few minutes for the mirroring to pick it up, but 
you should soon see a commit that says

    cvs2git: escape <<EOF messages, and work around cvsps branch handling
    
    This escapes '$' characters in <<-handling, and gives preference to
    the new branch when cvsps incorrectly reports a commit as originating
    on an old branch.

and once you do, you should have something that works. 

Of course, I've still only tested it on syslinux, but it converts a 
syslinux CVS repo in 64 seconds for me, and now the result really _does_ 
look correct at least superficially. Ie I can see 1029 commits on HEAD, 
which is exactly what cvsps also reports.

And I see four different branches (HEAD is called "master" as per the 
normal naming):

	torvalds@ppc970:~/src/osscvs/syslinux> ll .git/refs/heads/
	total 16K
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:36 branch-1_xx
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:37 master
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:36 syslinux
	-rw-rw-r--  1 torvalds torvalds 41 May 24 09:36 syslinux-1_6x-1

and doing a 

	git-rev-tree branch-1_xx master syslinux syslinux-1_6x-1 | wc -l

reports 1046 total revisions (which also matches cvsps exactly).

So things look ok, but I haven't actually checked the _contents_ of the
tree, except to look that the pathces that "git-whatchanged -p" reports
look sane.

There's two remaining bad things:

 - name translation doesn't exist (so all of Peters changesets get 
   reported as author "hpa <hpa>")

 - the commit time will be the conversion time, not the original commit 
   time (but the _author_ time will be correct). I suspect that for a 
   conversion like this, we really should add support for GIT_COMMIT_DATE. 

   That would also make the archive conversion 100% reproducible, ie
   everybody should get the exact same objects (and thus the exact same
   SHA1 values) which is good.

I'll add the GIT_COMMIT_DATE thing, but the name translation is for 
somebody else.

		Linus

Re: gitweb wishlist

[David Mansfield]

Linus Torvalds wrote:
> 
> On Tue, 24 May 2005, Linus Torvalds wrote:
> 
>>It has the logic for branches, but it doesn't work, and I'm fed up enough
>>with CVS and RCS for the moment that I'm not going to work on it any more
>>tonight.
> 
> 
> I'm back, and yes, it was a really stupid thing.
> 
> However, David, I need more help deciphering "cvsps" output..
> 
> Fixing the branch handling shows that cvsps does some really strange
> things with the newly added "Ancestor grpah". Here's one example:
> 

Yes.  While not falling asleep last night I realized that the 
quick-and-dirty approach was bogus.  I need to track what the ancestor 
is as I'm building up the data structure, not while outputting it.  So 
I'm working on a correct version which puts ancestor_branch into the 
PatchSet structure itself.

It's completely done now except for that it segfaults instantly.

BTW where did you get the cvsroot for syslinux?  Could I get a copy 
somewhere?

David

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Linus Torvalds wrote:
> 
>  - the commit time will be the conversion time, not the original commit 
>    time (but the _author_ time will be correct). I suspect that for a 
>    conversion like this, we really should add support for GIT_COMMIT_DATE. 
> 
>    That would also make the archive conversion 100% reproducible, ie
>    everybody should get the exact same objects (and thus the exact same
>    SHA1 values) which is good.
> 
> I'll add the GIT_COMMIT_DATE thing, but the name translation is for 
> somebody else.

Done. I've also fixed the timezone to "+0000", so that it doesn't matter 
where you do the conversion, you should always get the same results 
(again, I just pushed that out, it might not have hit the public mirrors 
yet).

To get GIT_COMMITTER_DATE (note: COMMITTER, not COMMIT, to illogically 
match the name/email ones) you obviously also need a new git. So to have 
it all working right, you should have the top commits in git-tools and git 
be

    cvs2git: set timezone info to UTC, the way CVS does

and

    git-commit-tree: allow overriding of commit date

respectively.

And if this doesn't work for you, point me to the CVS archive that causes 
you trouble. 

		Linus

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, David Mansfield wrote:
> 
> Yes.  While not falling asleep last night I realized that the 
> quick-and-dirty approach was bogus.  I need to track what the ancestor 
> is as I'm building up the data structure, not while outputting it. 

Yes.

> It's completely done now except for that it segfaults instantly.

Very good. Are you going to also make a new release at some point, so that 
we don't have strange random patches floating around?

> BTW where did you get the cvsroot for syslinux?  Could I get a copy 
> somewhere?

Peter sent it in private email, I don't know how public that is (it
probably is perfectly public and he just didn't want to spam the mailing
list or run afoul of size limits, but I just don't know for sure, so..),
but I bet he'll happily send it to you too.

		Linus

Re: gitweb wishlist

[H. Peter Anvin]

Linus Torvalds wrote:
> 
> Peter sent it in private email, I don't know how public that is (it
> probably is perfectly public and he just didn't want to spam the mailing
> list or run afoul of size limits, but I just don't know for sure, so..),
> but I bet he'll happily send it to you too.
> 

Already sent... I haven't looked it over to make sure there isn't 
anything that shouldn't be in there yet, so if you need to distribute it 
please give me a warning so I can look it over first.

	-hpa

Re: gitweb wishlist

[Thomas Glanzmann]

Hello,

>  - name translation doesn't exist (so all of Peters changesets get 
>    reported as author "hpa <hpa>")

I pick that up.

>  - the commit time will be the conversion time, not the original commit 
>    time (but the _author_ time will be correct). I suspect that for a 
>    conversion like this, we really should add support for GIT_COMMIT_DATE. 

Thanks. I wanted to send patches or at least ask you this for ages, but
never did. :-)

	Thomas

Re: gitweb wishlist

[Thomas Glanzmann]

[-- Attachment #1: Type: text/plain, Size: 2400 bytes --]

Hello,

> And if this doesn't work for you, point me to the CVS archive that causes 
> you trouble.

you should try the mutt cvs repository[1].

I have the following issues all seem easy to fix:

	- PatchSet 1 depends on PatchSet 2 (but cvsps gets the ordering wrong;
	  should be easy fixable) (I just swichted the two before
	  running cvs2git)

	- Some Shell escapes (I didn't looked into them yet)

		(faui02new) [/var/tmp/sithglan/mutt-cvs] bash ~/work/cvsps/sane
		defaulting to local storage area
		Committing initial tree 7e68fd9a5104b61192a7da7357549d95b3a0620c
		Ignoring path .cvsignore
		...
		/home/cip/adm/sithglan/work/cvsps/sane: line 1: ...: command not found
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 4: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 5: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 2: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 3: syntax error: unexpected end of file
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 1: unexpected EOF while looking for matching `''
		/home/cip/adm/sithglan/work/cvsps/sane: command substitution: line 26: syntax error: unexpected end of file

But hey this looks really good: :-))))))

(faui02new) [/var/tmp/sithglan/mutt-cvs] git parent ~/work/mutt/git/mutt-cvs
(faui02new) [/var/tmp/sithglan/mutt-cvs] git parentdiff
(faui02new) [/var/tmp/sithglan/mutt-cvs]

I think I will run my 'import patch by patch script again' and check the
changesets against the cvs2git tree, but it looks fine for me.

	Thomas

[1] To make it reproducable for you:

I used the attached patch against cvsps-2.0rc1 which fixes date
covnersion problems and of course includes the ancestor thing.

rsync -r rsync://cvs.gnupg.org/mutt-cvs-rep mutt-cvs-rep

[-- Attachment #2: diff --]
[-- Type: text/plain, Size: 12426 bytes --]

diff --git a/cvs_direct.c b/cvs_direct.c
--- a/cvs_direct.c
+++ b/cvs_direct.c
@@ -126,7 +126,7 @@ CvsServerCtx * open_cvs_server(char * p_
 	send_string(ctx, "Root %s\n", ctx->root);
 
 	/* this is taken from 1.11.1p1 trace - but with Mbinary removed. we can't handle it (yet!) */
-	send_string(ctx, "Valid-responses ok error Valid-requests Checked-in New-entry Checksum Copy-file Updated Created Update-existing Merged Patched Rcs-diff Mode Mod-time Removed Remove-entry Set-static-directory Clear-static-directory Set-sticky Clear-sticky Template Set-checkin-prog Set-update-prog Notified Module-expansion Wrapper-rcsOption M E F MT\n", ctx->root);
+	send_string(ctx, "Valid-responses ok error Valid-requests Checked-in New-entry Checksum Copy-file Updated Created Update-existing Merged Patched Rcs-diff Mode Mod-time Removed Remove-entry Set-static-directory Clear-static-directory Set-sticky Clear-sticky Template Set-checkin-prog Set-update-prog Notified Module-expansion Wrapper-rcsOption M E F\n", ctx->root);
 
 	send_string(ctx, "valid-requests\n");
 
@@ -894,6 +894,7 @@ char * cvs_rlog_fgets(char * buff, int b
     }
     else if (strcmp(lbuff, "ok") == 0 ||strcmp(lbuff, "error") == 0)
     {
+	debug(DEBUG_TCP, "cvs_direct: rlog: got command completion");
 	return NULL;
     }
 
diff --git a/cvsps.1 b/cvsps.1
--- a/cvsps.1
+++ b/cvsps.1
@@ -3,7 +3,7 @@
 CVSps \- create patchset information from CVS
 .SH SYNOPSIS
 .B cvsps
-[-h] [-x] [-u] [-z <fuzz>] [-g] [-s <patchset>] [-a <author>] [-f <file>] [-d <date1> [-d <date2>]] [-l <text>] [-b <branch>] [-r <tag> [-r <tag>]] [-p <directory>] [-v] [-t] [--norc] [--summary-first] [--test-log <filename>] [--bkcvs] [--no-rlog] [--diff-opts <option string>] [--cvs-direct] [--debuglvl <bitmask>] [-Z <compression>] [--root <cvsroot>] [-q] [<repository>] 
+[\-h] [\-x] [\-u] [\-z <fuzz>] [\-g] [\-s <patchset>] [\-a <author>] [\-f <file>] [\-d <date1> [\-d <date2>]] [\-l <text>] [\-b <branch>] [\-r <tag> [\-r <tag>]] [\-p <directory>] [\-v] [\-t] [\-\-norc] [\-\-summary-first] [\-\-test\-log <filename>] [\-\-bkcvs] [\-\-no\-rlog] [\-\-diff\-opts <option string>] [\-\-cvs\-direct] [\-\-debuglvl <bitmask>] [\-Z <compression>] [\-\-root <cvsroot>] [\-q] [<repository>] 
 .SH DESCRIPTION
 CVSps is a program for generating 'patchset' information from a CVS
 repository.  A patchset in this case is defined as a set of changes made
@@ -29,7 +29,7 @@ set the timestamp fuzz factor for identi
 .B \-g
 generate diffs of the selected patch sets
 .TP
-.B \-s <patchset>[-[<patchset>]][,<patchset>...]
+.B \-s <patchset>[\-[<patchset>]][,<patchset>...]
 generate a diff for a given patchsets and patchset ranges
 .TP
 .B \-a <author>
@@ -38,7 +38,7 @@ restrict output to patchsets created by 
 .B \-f <file>
 restrict output to patchsets involving file
 .TP
-.B \-d <date1> -d <date2>
+.B \-d <date1> \-d <date2>
 if just one date specified, show
 revisions newer than date1.  If two dates specified,
 show revisions between two dates.
@@ -50,7 +50,7 @@ restrict output to patchsets matching re
 restrict output to patchsets affecting history of branch.
 If you want to restrict to the main branch, use a branch of 'HEAD'.
 .TP
-.B \-r <tag1> -r <tag2>
+.B \-r <tag1> \-r <tag2>
 if just one tag specified, show
 revisions since tag1. If two tags specified, show
 revisions between the two tags.
@@ -64,47 +64,47 @@ show very verbose parsing messages
 .B \-t
 show some brief memory usage statistics
 .TP
-.B \--norc
+.B \-\-norc
 when invoking cvs, ignore the .cvsrc file
 .TP
-.B \--summary-first
+.B \-\-summary\-first
 when multiple patchset diffs are being generated, put the patchset
 summary for all patchsets at the beginning of the output.
 .TP
-.B \--test-log <captured cvs log file>
+.B \-\-test\-log <captured cvs log file>
 for testing changes, you can capture cvs log output, then test against
 this captured file instead of hammering some poor CVS server
 .TP
-.B \--bkcvs
+.B \-\-bkcvs
 (see note below) for use in parsing the BK->CVS tree log formats only.  This enables
 some hacks which are not generally applicable.
 .TP
-.B \--no-rlog
+.B \-\-no\-rlog
 disable the use of rlog internally.  Note: rlog is
 required for stable PatchSet numbering.  Use with care.
 .TP
-.B \--diffs-opts <option string>
+.B \-\-diffs\-opts <option string>
 send a custom set of options to diff, for example to increase
 the number of context lines, or change the diff format.
 .TP
-.B \--cvs-direct (--no-cvs-direct)
-enable (disable) built-in cvs client code. This enables the 'pipelining' of multiple
+.B \-\-cvs\-direct (\-\-no-cvs\-direct)
+enable (disable) built\-in cvs client code. This enables the 'pipelining' of multiple
 requests over a single client, reducing the overhead of handshaking and
 authentication to one per PatchSet instead of one per file.
 .TP
-.B \--debuglvl <bitmask>
+.B \-\-debuglvl <bitmask>
 enable various debug output channels.
 .TP
 .B \-Z <compression>
 A value 1-9 which specifies amount of compression.  A value of 0 disables compression.
 .TP
-.B \--root <cvsroot>
+.B \-\-root <cvsroot>
 Override the setting of CVSROOT (overrides working dir. and environment)
 .TP
 .B \-q
 Be quiet about warnings.
 .TP
-.B \<repository>
+.B <repository>
 Operate on the specified repository (overrides working dir.)
 .SH "NOTE ON TAG HANDLING"
 Tags are fundamentally 'file at a time' in cvs, but like everything else,
@@ -159,17 +159,17 @@ directory in the path, and -p0 will be r
 diffs are generated in cvs-direct mode (see below), however, they will always
 be -p1 style patches.
 .SH "NOTE ON BKCVS"
-The --bkcvs option is a special operating mode that should only be used when parsing
+The \-\-bkcvs option is a special operating mode that should only be used when parsing
 the log files from the BK -> CVS exported linux kernel trees.  cvsps uses special
 semantics for recreating the BK ChangeSet metadata that has been embedded in the log
-files for those trees.  The --bkcvs option should only be specified when the cache
-file is being created or updated (i.e. initial run of cvsps, or when -u and -x options
+files for those trees.  The \-\-bkcvs option should only be specified when the cache
+file is being created or updated (i.e. initial run of cvsps, or when \-u and \-x options
 are used).
 .SH "NOTE ON CVS-DIRECT"
 As of version 2.0b6 cvsps has a partial implementation of the cvs client code built 
 in.  This reduces the RTT and/or handshaking overhead from one per patchset member
 to one per patchset.  This dramatically increases the speed of generating diffs
-over a slow link, and improves the consistency of operation.  Currently the --cvs-direct
+over a slow link, and improves the consistency of operation.  Currently the \-\-cvs-direct
 option turns on the use of this code, but it very well may be default by the time
 2.0 comes out.  The built-in cvs code attempts to be compatible with cvs, but may
 have problems, which should be reported.  It honors the CVS_RSH and CVS_SERVER 
@@ -179,7 +179,9 @@ CVSps parses an rc file at startup.  Thi
 The file should contain arguments, in the exact syntax as the command line, one per line.
 If an argument takes a parameter, the parameter should be on the same line as the argument.
 .SH "NOTE ON DATE FORMATS"
-Dates have formats.  Fixme.
+Dates must be in the format 'yyyy/mm/dd hh:mm:ss'; for example,
+.IP "" 4
+$ cvsps -d '2004/05/01 00:00:00' -d '2004/07/07 12:00:00'
 .SH "SEE ALSO"
 .BR cvs ( 1 ),
 .BR ci ( 1 ),
diff --git a/cvsps.c b/cvsps.c
--- a/cvsps.c
+++ b/cvsps.c
@@ -1402,6 +1402,16 @@ static void print_patch_set(PatchSet * p
 	   tm->tm_hour, tm->tm_min, tm->tm_sec);
     printf("Author: %s\n", ps->author);
     printf("Branch: %s\n", ps->branch);
+    
+    /* check if ancestor was different branch */
+    if (!list_empty(&ps->members)) 
+    {
+	    PatchSetMember * psm = list_entry(ps->members.next, PatchSetMember, link);
+	    const char * abr = psm->pre_rev ? psm->pre_rev->branch : NULL;
+	    if (abr && strcmp(ps->branch, abr) != 0)
+		    printf("Ancestor branch: %s\n", abr);
+    }
+
     printf("Tag: %s %s\n", ps->tag ? ps->tag : "(none)", tag_flag_descr[ps->tag_flags]);
     printf("Log:\n%s\n", ps->descr);
     printf("Members: \n");
@@ -1646,6 +1656,7 @@ static void do_cvs_diff(PatchSet * ps)
     const char * dopts;
     const char * utype;
     char use_rep_path[PATH_MAX];
+    char esc_use_rep_path[PATH_MAX];
 
     fflush(stdout);
     fflush(stderr);
@@ -1666,6 +1677,8 @@ static void do_cvs_diff(PatchSet * ps)
 	dtype = "rdiff";
 	utype = "co";
 	sprintf(use_rep_path, "%s/", repository_path);
+	/* the rep_path may contain characters that the shell will barf on */
+	escape_filename(esc_use_rep_path, PATH_MAX, use_rep_path);
     }
     else
     {
@@ -1673,6 +1686,7 @@ static void do_cvs_diff(PatchSet * ps)
 	dtype = "diff";
 	utype = "update";
 	use_rep_path[0] = 0;
+	esc_use_rep_path[0] = 0;
     }
 
     for (next = ps->members.next; next != &ps->members; next = next->next)
@@ -1740,7 +1754,7 @@ static void do_cvs_diff(PatchSet * ps)
 	    else
 	    {
 		snprintf(cmdbuff, PATH_MAX * 2, "cvs %s %s %s -p -r %s %s%s | diff %s %s /dev/null %s | sed -e '%s s|^\\([+-][+-][+-]\\) -|\\1 %s%s|g'",
-			 compress_arg, norc, utype, rev, use_rep_path, esc_file, dopts,
+			 compress_arg, norc, utype, rev, esc_use_rep_path, esc_file, dopts,
 			 cr?"":"-",cr?"-":"", cr?"2":"1",
 			 use_rep_path, psm->file->filename);
 	    }
@@ -1760,7 +1774,7 @@ static void do_cvs_diff(PatchSet * ps)
 
 		snprintf(cmdbuff, PATH_MAX * 2, "cvs %s %s %s %s -r %s -r %s %s%s",
 			 compress_arg, norc, dtype, dopts, psm->pre_rev->rev, psm->post_rev->rev, 
-			 use_rep_path, esc_file);
+			 esc_use_rep_path, esc_file);
 	    }
 	}
 
@@ -2113,7 +2127,7 @@ static void resolve_global_symbols()
 	    Tag * tag = list_entry(next, Tag, global_link);
 	    CvsFileRevision * rev = tag->rev;
 
-	    if (!rev->present)
+	    if (!rev->present || !rev->post_psm)
 	    {
 		struct list_head *tmp = next->prev;
 		debug(DEBUG_APPERROR, "revision %s of file %s is tagged but not present",
diff --git a/cvsps.h b/cvsps.h
--- a/cvsps.h
+++ b/cvsps.h
@@ -11,6 +11,10 @@
 typedef struct _CvsServerCtx CvsServerCtx;
 #endif
 
+#ifndef PATH_MAX
+#define PATH_MAX 4096
+#endif
+
 extern struct hash_table * file_hash;
 extern const char * tag_flag_descr[];
 extern CvsServerCtx * cvs_direct_ctx;
diff --git a/util.c b/util.c
--- a/util.c
+++ b/util.c
@@ -13,6 +13,7 @@
 #include <time.h>
 #include <errno.h>
 #include <signal.h>
+#include <regex.h>
 #include <sys/stat.h>
 #include <sys/time.h>
 #include <sys/types.h>
@@ -140,24 +141,51 @@ char *get_string(char const *str)
     return *res;
 }
 
+static int get_int_substr(const char * str, const regmatch_t * p)
+{
+    char buff[256];
+    memcpy(buff, str + p->rm_so, p->rm_eo - p->rm_so);
+    buff[p->rm_eo - p->rm_so] = 0;
+    return atoi(buff);
+}
+
 void convert_date(time_t * t, const char * dte)
 {
-    /* HACK: this routine parses two formats,
-     * 1) 'cvslog' format YYYY/MM/DD HH:MM:SS
-     * 2) time_t formatted as %d
-     */
-       
-    if (strchr(dte, '/'))
+    static regex_t date_re;
+    static int init_re;
+
+#define MAX_MATCH 16
+    size_t nmatch = MAX_MATCH;
+    regmatch_t match[MAX_MATCH];
+
+    if (!init_re) 
+    {
+	if (regcomp(&date_re, "([0-9]{4})[-/]([0-9]{2})[-/]([0-9]{2}) ([0-9]{2}):([0-9]{2}):([0-9]{2})", REG_EXTENDED)) 
+	{
+	    fprintf(stderr, "FATAL: date regex compilation error\n");
+	    exit(1);
+	}
+	init_re = 1;
+    }
+    
+    if (regexec(&date_re, dte, nmatch, match, 0) == 0)
     {
+	regmatch_t * pm = match;
 	struct tm tm;
+
+	/* first regmatch_t is match location of entire re */
+	pm++;
 	
-	memset(&tm, 0, sizeof(tm));
-	sscanf(dte, "%d/%d/%d %d:%d:%d", 
-	       &tm.tm_year, &tm.tm_mon, &tm.tm_mday, 
-	       &tm.tm_hour, &tm.tm_min, &tm.tm_sec);
-	
+	tm.tm_year = get_int_substr(dte, pm++);
+	tm.tm_mon  = get_int_substr(dte, pm++);
+	tm.tm_mday = get_int_substr(dte, pm++);
+	tm.tm_hour = get_int_substr(dte, pm++);
+	tm.tm_min  = get_int_substr(dte, pm++);
+	tm.tm_sec  = get_int_substr(dte, pm++);
+
 	tm.tm_year -= 1900;
 	tm.tm_mon--;
+	tm.tm_isdst = 0;
 	
 	*t = mktime(&tm);
     }
diff --git a/util.h b/util.h
--- a/util.h
+++ b/util.h
@@ -6,6 +6,10 @@
 #ifndef UTIL_H
 #define UTIL_H
 
+#ifndef PATH_MAX
+#define PATH_MAX 4096
+#endif
+
 #define CVSPS_PREFIX ".cvsps"
 
 char *xstrdup(char const *);

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Thomas Glanzmann wrote:
>
> Hello,
> 
> >  - name translation doesn't exist (so all of Peters changesets get 
> >    reported as author "hpa <hpa>")
> 
> I pick that up.

Note that one advantage of the unconverted output is that while it's
unreadable and not very helpful, it _is_ the raw output from CVS. Again,
that means that everybody will convert the same CVS archive into exactly
the same git tree, and that means (among other things) that you can then
immediately merge between the trees.

Perhaps more interestingly, it should also mean that you can _continue_ to 
use CVS, then re-convert it at a later date, and I think you should be 
able to merge with somebody who has been using git in the meantime.

In contrast, if you have fancy name translation, the converted tree will 
obviously depend on your translation rules.

I dunno. Maybe the advantages of having nice names outweigh the 
disadvantage of possibly generating incompatible trees.

			Linus

Re: gitweb wishlist

[Thomas Glanzmann]

Hello,

> Note that one advantage of the unconverted output is that while it's
> unreadable and not very helpful, it _is_ the raw output from CVS. Again,
> that means that everybody will convert the same CVS archive into exactly
> the same git tree, and that means (among other things) that you can then
> immediately merge between the trees.

I see your point. But it depends on the usage scenario. For me for
example I would like to vendortrack a few CVS repositories. And I use it
only to maintain a few patches (branches and the merging facilities of
git come handy in here), not in a distributed environment were I need
that much reproducability. So having the option to use them is fine for
me and when I need reproducability than I simple don't. When I think
about this scenario ... it comes in my mind that it maybe would be
helpful to have a helper applications like git-merge-base which looks at
the treeids and not the commit ids to find the merge base, or is that
just bullshit?

	Thomas

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:

LT> Perhaps more interestingly, it should also mean that you can _continue_ to 
LT> use CVS, then re-convert it at a later date, and I think you should be 
LT> able to merge with somebody who has been using git in the meantime.

LT> In contrast, if you have fancy name translation, the converted tree will 
LT> obviously depend on your translation rules.

LT> I dunno. Maybe the advantages of having nice names outweigh the 
LT> disadvantage of possibly generating incompatible trees.

LT> 			Linus

Porcelain layers should be capable of mapping author/committer
names taken out of the commit object, just like they already
convert the human unreadable unixtime value into something human
readable.  I'd vote for keeping the original value taken from
CVS for this particular "conversion" application.

What _all_ Porcelain layer implementation would benefit from is
if we had a common output format routine that is similar to the
spirit of show_date() function.  Have format_commit_fancy()
function that takes a commit object and have it do the mapping.
Then everybody can use it for their own Porcelain.

The diff-tree header generation can use it when (and only when)
it is operating under a new flag (--map-author-names), to
prettyprint the author names.  I'd also suggest to have a flag
to reduce the prettyprinting it does in the current output (like
omitting committer information) to make its output be usable for
reproducing the commit history exactly.  diff-tree with recent
enhancement you did (I am talking about single commit output and
--stdin, not my diffcore stuff) has become quite useful tool for
this kind of thing.

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Thomas Glanzmann wrote:
> 
> I have the following issues all seem easy to fix:
> 
> 	- PatchSet 1 depends on PatchSet 2 (but cvsps gets the ordering wrong;
> 	  should be easy fixable) (I just swichted the two before
> 	  running cvs2git)

Ok, this seems to be a cvsps bug, and I'll treat it as such. David, any 
ideas? It seems to be because of how cvsps sorts things by date, which is 
obviously bogus.

The cvs2git thing wouldn't normally even _care_ (ie would happily re-order
the thing), but for the fact that it causes problems with branches that
are used before they are created in this case.

cvsps really should do some kind of topo-sort. Probably doesn't need a lot
(ie it probably doesn't even need to be topological, but the "order"  
should be based on trivial dependencies first, and time second. For
example, once David does the per-commit branch handling, I suspect enough
of an ordering to keep git happy falls out of that).

> 	- Some Shell escapes (I didn't looked into them yet)

Ok, I'll check it out. I didn't figure out what characters are 
shell-expanded by "<<EOF" handling, and only did '$' because that showed 
up in the syslinux archives.

> (faui02new) [/var/tmp/sithglan/mutt-cvs] git parent ~/work/mutt/git/mutt-cvs
> (faui02new) [/var/tmp/sithglan/mutt-cvs] git parentdiff
> (faui02new) [/var/tmp/sithglan/mutt-cvs]
> 
> I think I will run my 'import patch by patch script again' and check the
> changesets against the cvs2git tree, but it looks fine for me.

In theory, they should give the exact same results, no? At least if there 
are no binary objects. Of course, you'd have to update your import script 
to do the times the same way.

		Linus

Re: gitweb wishlist

[David Mansfield]

Thomas Glanzmann wrote:
> Hello,
> 
> 
>>And if this doesn't work for you, point me to the CVS archive that causes 
>>you trouble.
> 
> 
> you should try the mutt cvs repository[1].

Sounds good.  I'll give it a try.  I'm testing the branch ancestor 
logic, which seems to be working better now.  The version I sent to the 
list yesterday was pretty bogus for some cases, as well as reporting the 
ancestor multiple times for any give branch.

> 
> I have the following issues all seem easy to fix:
> 
> 	- PatchSet 1 depends on PatchSet 2 (but cvsps gets the ordering wrong;
> 	  should be easy fixable) (I just swichted the two before
> 	  running cvs2git)
> 

There is something strange about 'cvs import' I believe which causes 
various bizarre things to happen to the first cvsps patchset.  I haven't 
looked at mutt cvs yet, but this could be the cause.  If you see a lot 
of version numbers 1.1.1.1 then this is indeed the problem.

> I used the attached patch against cvsps-2.0rc1 which fixes date
> covnersion problems and of course includes the ancestor thing.

I'll look at taking these patches upstream.  The 'MT' fix is already in 
my cvs of cvsps, and the rest looks pretty good.

Do you know where I can get attribution information for these changes? 
Are they all from you? (I'm not familiar with debian at all)

David

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "JCH" == Junio C Hamano <junkio@cox.net> writes:

JCH> What _all_ Porcelain layer implementation would benefit from is
JCH> if we had a common output format routine that is similar to the
JCH> spirit of show_date() function.  Have format_commit_fancy()
JCH> function that takes a commit object and have it do the mapping.

Here is a small script called "whodunnit.sh", and its output can
be cleaned up if we had a git-format-commit command that used
format_commit_fancy(), that massages author/committer names (and
probably some other prettyprinting), instead of plain old
"git-cat-file commit".

#!/bin/sh
git-rev-list ${1:-HEAD} |
while read commit
do
	git-cat-file commit $commit |
	sed -ne '/^author \([^>]*>\).*/{s//\1/p;q;}'
done | sort | uniq -c | sort -n

The lines it currently spits out looks like this:

      6 ...
      8 Linus Torvalds <torvalds@ppc970.osdl.org.(none)>
        ...
    155 ...
    229 Linus Torvalds <torvalds@ppc970.osdl.org>

My suggestion for Thomas is not to volunteer changing
cvsps-to-git to munge names at conversion time, but instead to
volunteer doing the format_commit_fancy() on the core-ish side.
It would read from $GIT_DIR/author-names which would be a plain
text file that is a sequence of:

    "bogus name" TAB "good name" LF

where you would put ".(none)" version to "bogus" side and the
corrected one on the "good" side.

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Thomas Glanzmann wrote:
> 
> [1] To make it reproducable for you:
> 
> I used the attached patch against cvsps-2.0rc1 which fixes date
> covnersion problems and of course includes the ancestor thing.
> 
> rsync -r rsync://cvs.gnupg.org/mutt-cvs-rep mutt-cvs-rep

Ok, that's a lot bigger and slower than syslinux. It seems to be importing 
about 9.5 changesets per second, and there's 3757 patchsets, so it looks 
like about 6 minutes.

Oh, done.

And yes, there's a few problems. It seems to be the fault of a frowning
"smiley" - the '\' followed by newline in this:

	[unstable] Re-add in-reply-to.  This time with a suitable default. #-\

and one back-tick.

Will fix. This will take another six minutes of testing ;)

		Linus

Re: gitweb wishlist

[David Mansfield]

Linus Torvalds wrote:
> 
> On Tue, 24 May 2005, Linus Torvalds wrote:
> 
>>Fixing the branch handling shows that cvsps does some really strange
>>things with the newly added "Ancestor grpah". Here's one example:
> 
> 
> Ahh, looking at cvsps source, I think I see what's going on. 
> 
> It's deciding the "previous branch" by looking at what the previous branch 
> for the first individual file in the PatchSet was, which fails because in 
> this case, PatchSet 372 was changing "syslinux.doc", and Patchset 374 was 
> changing "syslinux.c", and thus the previous version of the individual 
> _files_ were both in the HEAD branch.
> 
> So it does look like I should just ignore the "Ancestor branch" 
> information if the new branch already existed.
> 

I now consider all files in a commit, and all commits in a branch to 
determine the ancestor, and only report it in the first commit on the 
branch.

Strangely, you have to look at (potentially) all commits on a branch to 
find the 'true' ancestor branch.

The problem is for branch-off-branch branches where the first commit on 
the new branch modifies only files never modified on the branch-off-HEAD 
branch.  This is because cvs only REALLY creates the branch when the 
first commit is made (for that file) on the branch.  Before that, it is 
just a 'potential' branch...

But I have code now which (seems to) works, but needs a bit more checking.

> Of course, some semantics will never be translatable when trying to treat 
> CVS as a sane system (ie treating CVS as if it was changeset-based is 
> always going to cause strange corner cases since it really is file-based), 
> but that should most likely give the best approximation of what a 
> conversion should do.
> 

Yes.

David

Re: gitweb wishlist

[David Mansfield]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

Linus Torvalds wrote:
> 
> On Tue, 24 May 2005, Linus Torvalds wrote:
> 
>>Fixing the branch handling shows that cvsps does some really strange
>>things with the newly added "Ancestor grpah". Here's one example:
> 
> 
> Ahh, looking at cvsps source, I think I see what's going on. 
> 
> It's deciding the "previous branch" by looking at what the previous branch 
> for the first individual file in the PatchSet was, which fails because in 
> this case, PatchSet 372 was changing "syslinux.doc", and Patchset 374 was 
> changing "syslinux.c", and thus the previous version of the individual 
> _files_ were both in the HEAD branch.
> 
> So it does look like I should just ignore the "Ancestor branch" 
> information if the new branch already existed.
> 

I've attached what I just committed.  The previous 'show ancestor' patch 
needs to be reversed and this applied.  It works for me on a half-dozen 
repos including syslinux.

You no longer should need to work around multiple reporting of the 
ancestor for a given branch, though it couldn't hurt.

I'm going to finish getting some of Thomas's patches in and make an 
actual release so people won't have to scour the lists.

David

[-- Attachment #2: show-branch-ancestry-2.patch --]
[-- Type: text/x-patch, Size: 8668 bytes --]

---------------------
PatchSet 176 
Date: 2005/05/24 19:57:37
Author: david
Branch: HEAD
Tag: (none) 
Log:
show branch ancestry

Members: 
	cvsps.c:4.99->4.100 
	cvsps_types.h:4.9->4.10 

Index: cvsps/cvsps.c
diff -u cvsps/cvsps.c:4.99 cvsps/cvsps.c:4.100
--- cvsps/cvsps.c:4.99	Wed Jan 26 14:46:41 2005
+++ cvsps/cvsps.c	Tue May 24 15:57:37 2005
@@ -26,7 +26,7 @@
 #include "cap.h"
 #include "cvs_direct.h"
 
-RCSID("$Id: cvsps.c,v 4.99 2005/01/26 19:46:41 david Exp $");
+RCSID("$Id: cvsps.c,v 4.100 2005/05/24 19:57:37 david Exp $");
 
 #define CVS_LOG_BOUNDARY "----------------------------\n"
 #define CVS_FILE_BOUNDARY "=============================================================================\n"
@@ -75,6 +75,7 @@
 static int do_write_cache;
 static int statistics;
 static const char * test_log_file;
+static struct hash_table * branch_heads;
 
 /* settable via options */
 static int timestamp_fuzz_factor = 300;
@@ -101,6 +102,7 @@
 static int cvs_direct;
 static int compress;
 static char compress_arg[8];
+static int track_branch_ancestry;
 
 static void check_norc(int, char *[]);
 static int parse_args(int, char *[]);
@@ -112,7 +114,7 @@
 static void assign_pre_revision(PatchSetMember *, CvsFileRevision * rev);
 static void check_print_patch_set(PatchSet *);
 static void print_patch_set(PatchSet *);
-static void set_ps_id(const void *, const VISIT, const int);
+static void walk_all_ps(const void *, const VISIT, const int);
 static void show_ps_tree_node(const void *, const VISIT, const int);
 static int compare_patch_sets_bk(const void *, const void *);
 static int compare_patch_sets(const void *, const void *);
@@ -131,6 +133,7 @@
 static int check_rev_funk(PatchSet *, CvsFileRevision *);
 static CvsFileRevision * rev_follow_branch(CvsFileRevision *, const char *);
 static int before_tag(CvsFileRevision * rev, const char * tag);
+static void determine_branch_ancestor(PatchSet * ps, PatchSet * head_ps);
 
 int main(int argc, char *argv[])
 {
@@ -164,6 +167,7 @@
 
     file_hash = create_hash_table(1023);
     global_symbols = create_hash_table(111);
+    branch_heads = create_hash_table(1023);
 
     /* this parses some of the CVS/ files, and initializes
      * the repository_path and other variables 
@@ -197,7 +201,7 @@
     }
 
     ps_counter = 0;
-    twalk(ps_tree_bytime, set_ps_id);
+    twalk(ps_tree_bytime, walk_all_ps);
 
     resolve_global_symbols();
 
@@ -536,7 +540,7 @@
     debug(DEBUG_APPERROR, "             [--test-log <captured cvs log file>] [--bkcvs]");
     debug(DEBUG_APPERROR, "             [--no-rlog] [--diff-opts <option string>] [--cvs-direct]");
     debug(DEBUG_APPERROR, "             [--debuglvl <bitmask>] [-Z <compression>] [--root <cvsroot>]");
-    debug(DEBUG_APPERROR, "             [<repository>] [-q]");
+    debug(DEBUG_APPERROR, "             [-q] [-A] [<repository>]");
     debug(DEBUG_APPERROR, "");
     debug(DEBUG_APPERROR, "Where:");
     debug(DEBUG_APPERROR, "  -h display this informative message");
@@ -569,6 +573,7 @@
     debug(DEBUG_APPERROR, "  -Z <compression> A value 1-9 which specifies amount of compression");
     debug(DEBUG_APPERROR, "  --root <cvsroot> specify cvsroot.  overrides env. and working directory");
     debug(DEBUG_APPERROR, "  -q be quiet about warnings");
+    debug(DEBUG_APPERROR, "  -A track and report branch ancestry");
     debug(DEBUG_APPERROR, "  <repository> apply cvsps to repository.  overrides working directory");
     debug(DEBUG_APPERROR, "\ncvsps version %s\n", VERSION);
 
@@ -867,6 +872,13 @@
 	    continue;
 	}
 
+	if (strcmp(argv[i], "-A") == 0)
+	{
+	    track_branch_ancestry = 1;
+	    i++;
+	    continue;
+	}
+
 	if (argv[i][0] == '-')
 	    return usage("invalid argument", argv[i]);
 	
@@ -1398,6 +1410,8 @@
 	   tm->tm_hour, tm->tm_min, tm->tm_sec);
     printf("Author: %s\n", ps->author);
     printf("Branch: %s\n", ps->branch);
+    if (ps->ancestor_branch)
+	printf("Ancestor branch: %s\n", ps->ancestor_branch);
     printf("Tag: %s %s\n", ps->tag ? ps->tag : "(none)", tag_flag_descr[ps->tag_flags]);
     printf("Log:\n%s\n", ps->descr);
     printf("Members: \n");
@@ -1425,7 +1439,10 @@
     printf("\n");
 }
 
-static void set_ps_id(const void * nodep, const VISIT which, const int depth)
+/* walk all the patchsets to assign monotonic psid, 
+ * and to establish  branch ancestry
+ */
+static void walk_all_ps(const void * nodep, const VISIT which, const int depth)
 {
     PatchSet * ps;
 
@@ -1442,6 +1459,18 @@
 	{
 	    ps_counter++;
 	    ps->psid = ps_counter;
+
+	    if (track_branch_ancestry && strcmp(ps->branch, "HEAD") != 0)
+	    {
+		PatchSet * head_ps = (PatchSet*)get_hash_object(branch_heads, ps->branch);
+		if (!head_ps) 
+		{
+		    head_ps = ps;
+		    put_hash_object(branch_heads, ps->branch, head_ps);
+		}
+
+		determine_branch_ancestor(ps, head_ps);
+	    }
 	}
 	else
 	{
@@ -1912,6 +1941,7 @@
 	ps->tag_flags = 0;
 	ps->branch_add = 0;
 	ps->funk_factor = 0;
+	ps->ancestor_branch = NULL;
     }
 
     return ps;
@@ -2235,21 +2265,25 @@
     return 0;
 }
 
-/*
- * When importing vendor sources, (apparently people do this)
- * the code is added on a 'vendor' branch, which, for some reason
- * doesn't use the magic-branch-tag format.  Try to detect that now
- */
-static int is_vendor_branch(const char * rev)
+static int count_dots(const char * p)
 {
     int dots = 0;
-    const char *p = rev;
 
     while (*p)
 	if (*p++ == '.')
 	    dots++;
 
-    return !(dots&1);
+    return dots;
+}
+
+/*
+ * When importing vendor sources, (apparently people do this)
+ * the code is added on a 'vendor' branch, which, for some reason
+ * doesn't use the magic-branch-tag format.  Try to detect that now
+ */
+static int is_vendor_branch(const char * rev)
+{
+    return !(count_dots(rev)&1);
 }
 
 void patch_set_add_member(PatchSet * ps, PatchSetMember * psm)
@@ -2395,5 +2429,69 @@
 	    break;
 	}
 	i++;
+    }
+}
+
+static void determine_branch_ancestor(PatchSet * ps, PatchSet * head_ps)
+{
+    struct list_head * next;
+    CvsFileRevision * rev;
+
+    /* PatchSet 1 has no ancestor */
+    if (ps->psid == 1)
+	return;
+
+    /* HEAD branch patchsets have no ancestry, but callers should know that */
+    if (strcmp(ps->branch, "HEAD") == 0)
+    {
+	debug(DEBUG_APPMSG1, "WARNING: no branch ancestry for HEAD");
+	return;
+    }
+
+    for (next = ps->members.next; next != &ps->members; next = next->next) 
+    {
+	PatchSetMember * psm = list_entry(next, PatchSetMember, link);
+	rev = psm->pre_rev;
+	int d1, d2;
+
+	/* the reason this is at all complicated has to do with a 
+	 * branch off of a branch.  it is possible (and indeed 
+	 * likely) that some file would not have been modified 
+	 * from the initial branch point to the branch-off-branch 
+	 * point, and therefore the branch-off-branch point is 
+	 * really branch-off-HEAD for that specific member (file).  
+	 * in that case, rev->branch will say HEAD but we want 
+	 * to know the symbolic name of the first branch
+	 * so we continue to look member after member until we find
+	 * the 'deepest' branching.  deepest can actually be determined
+	 * by considering the revision currently indicated by 
+	 * ps->ancestor_branch (by symbolic lookup) and rev->rev. the 
+	 * one with more dots wins
+	 *
+	 * also, the first commit in which a branch-off-branch is 
+	 * mentioned may ONLY modify files never committed since
+	 * original branch-off-HEAD was created, so we have to keep
+	 * checking, ps after ps to be sure to get the deepest ancestor
+	 *
+	 * note: rev is the pre-commit revision, not the post-commit
+	 */
+	if (!head_ps->ancestor_branch)
+	    d1 = 0;
+	else if (strcmp(ps->branch, rev->branch) == 0)
+	    continue;
+	else if (strcmp(head_ps->ancestor_branch, "HEAD") == 0)
+	    d1 = 1;
+	else {
+	    /* branch_rev may not exist if the file was added on this branch for example */
+	    const char * branch_rev = (char *)get_hash_object(rev->file->branches_sym, head_ps->ancestor_branch);
+	    d1 = branch_rev ? count_dots(branch_rev) : 1;
+	}
+
+	d2 = count_dots(rev->rev);
+	
+	if (d2 > d1)
+	    head_ps->ancestor_branch = rev->branch;
+
+ 	//printf("-----> %d ancestry %s %s %s\n", ps->psid, ps->branch, head_ps->ancestor_branch, rev->file->filename);
     }
 }
Index: cvsps/cvsps_types.h
diff -u cvsps/cvsps_types.h:4.9 cvsps/cvsps_types.h:4.10
--- cvsps/cvsps_types.h:4.9	Mon Mar 31 18:06:18 2003
+++ cvsps/cvsps_types.h	Tue May 24 15:57:37 2005
@@ -110,6 +110,7 @@
     char *tag;
     int tag_flags;
     char *branch;
+    char *ancestor_branch;
     struct list_head members;
     /*
      * A 'branch add' patch set is a bogus patch set created automatically

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Linus Torvalds wrote:
> 
> Will fix. This will take another six minutes of testing ;)

Almost eight minutes. Still, the final average was 8 changesets per
second, which sounds pretty damn good to me, actually.

Anyway, I've checked in the fix for the quoting, and I now get the right 
number of revisions, ie

	git-rev-tree $(ls .git/refs/heads/) | wc -l

returns the same "3757" that cvsps reports. 

However, "git-fsck-cache --unreachable" reports 102 unreachable blobs,
which worries me. It's really blobs only, which is strange: it implies
that we did the "git-update-cache" but not a "git-write-tree" (or that the
git-write-tree failed for some reason, but that sounds even stranger,
since we did successfully do all the commits)

The only way I can see the unreachable blobs happening is if one of he
ChangeSet entries in cvsps mentions the _same_ pathname twice for a single
ChangeSet. David, is that possible?

Exactly because it's only blobs, it really does smell like a cvsps issue. 
My scripts always use "git-update-cache --add -- filename", so it never 
creates any blobs _except_ when it adds them to the index (and thus 
write-tree should always pick them up, unless we update the index again 
before the next write-tree happens).

			Linus

Re: gitweb wishlist

[David Mansfield]

David Mansfield wrote:
> Linus Torvalds wrote:
> 
>>On Tue, 24 May 2005, Linus Torvalds wrote:
>>
>>
>>>Fixing the branch handling shows that cvsps does some really strange
>>>things with the newly added "Ancestor grpah". Here's one example:
>>
>>
>>Ahh, looking at cvsps source, I think I see what's going on. 
>>
>>It's deciding the "previous branch" by looking at what the previous branch 
>>for the first individual file in the PatchSet was, which fails because in 
>>this case, PatchSet 372 was changing "syslinux.doc", and Patchset 374 was 
>>changing "syslinux.c", and thus the previous version of the individual 
>>_files_ were both in the HEAD branch.
>>
>>So it does look like I should just ignore the "Ancestor branch" 
>>information if the new branch already existed.
>>
> 
> 
> I've attached what I just committed.  The previous 'show ancestor' patch 
> needs to be reversed and this applied.  It works for me on a half-dozen 
> repos including syslinux.
> 

Oops.  I forgot to metion I made the tracking of branch ancestry an 
option because it potentially increases the cpu time a fair margin 
(though here it seemed trivial).  You need to pass '-A' as an additional 
argument.

David

Re: gitweb wishlist

[Thomas Glanzmann]

Hello,

> There is something strange about 'cvs import' I believe which causes 
> various bizarre things to happen to the first cvsps patchset.  I haven't 
> looked at mutt cvs yet, but this could be the cause.  If you see a lot 
> of version numbers 1.1.1.1 then this is indeed the problem.

yes, that is happening. But it should be fairly easy to fix that.
Because the second one says INITIAL->1.1 and the first says 1.1->1.1.1.1
a lot.

> I'll look at taking these patches upstream.  The 'MT' fix is already in 
> my cvs of cvsps, and the rest looks pretty good.

Good. :-)

> Do you know where I can get attribution information for these changes? 
> Are they all from you? (I'm not familiar with debian at all)

none of them is from me, they're all from Debian. Here are a few URLs
how to get the attribution:

http://packages.qa.debian.org/c/cvsps.html
	-> You can get the source files from there: DSC are
	metainformation, ORIG is your upstream version and DIFF are the
	debian changes against your upstream version as patch. There is
	also a Changelog in the diff for the package. You should find in
	there everything you need.

http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=cvsps
	-> The bug tracking system of debian could also be of help.


	Thomas

Re: gitweb wishlist

[David Mansfield]

Linus Torvalds wrote:
> 
> On Tue, 24 May 2005, Linus Torvalds wrote:
> 
>>Will fix. This will take another six minutes of testing ;)
> 
> 
> Almost eight minutes. Still, the final average was 8 changesets per
> second, which sounds pretty damn good to me, actually.
> 
> Anyway, I've checked in the fix for the quoting, and I now get the right 
> number of revisions, ie
> 
> 	git-rev-tree $(ls .git/refs/heads/) | wc -l
> 
> returns the same "3757" that cvsps reports. 
> 
> However, "git-fsck-cache --unreachable" reports 102 unreachable blobs,
> which worries me. It's really blobs only, which is strange: it implies
> that we did the "git-update-cache" but not a "git-write-tree" (or that the
> git-write-tree failed for some reason, but that sounds even stranger,
> since we did successfully do all the commits)
> 
> The only way I can see the unreachable blobs happening is if one of he
> ChangeSet entries in cvsps mentions the _same_ pathname twice for a single
> ChangeSet. David, is that possible?
>

Sounds possible.  Unfortunately, the 'uniqueness' of a commit actually 
doesn't exist.  It's all smoke-and-mirrors.  In order to disallow this 
(which I think need to do) I'd need to use some commit member 
information, and add some heuristic: if this file is already in the 
commit, then this MUST be a different commit.  Unfortunately, it's 
possible that the 'member' already in the commit is the wrong one and 
this is the right one, which just sounds horribly ugly to me.

I'll think on it.

David

Re: gitweb wishlist

[Martin Langhoff]

On 5/24/05, David Mansfield <david@cobite.com> wrote:
> > means..
> >
> 
> Ok.  I'll tell you.  It means that the committer uses bad practices in
> tagging ;-)  It generally means that force tag (cvs tag -F <file>) was
> used on a specific file.  Here's the scenario:

Projects that branch on release (and maintain a long-lived stable
branch following the release) often use a floating MERGED branch to
keep track of what bugfixes have been merged back into HEAD. This
practice, broken as it is, is the recommended approach AFAIK.

It would be a good thing to be able to tell cvsps to ignore certain
tags (by name or by regex).


martin

Re: gitweb wishlist

[Thomas Glanzmann]

Hello,

> Almost eight minutes. Still, the final average was 8 changesets per
> second, which sounds pretty damn good to me, actually.

yes, it is. ;-)

> Anyway, I've checked in the fix for the quoting, and I now get the right 
> number of revisions, ie

> 	git-rev-tree $(ls .git/refs/heads/) | wc -l

> returns the same "3757" that cvsps reports. 

Nice! :-)

btw:

For the mutt tree there are a few 'empty commits' eg were the
parent tree is the same as the current. This is because git ignores
.cvsignore and they commited some .cvsignore files without any other
deltas. I don't know if you want to handle this. Just a note.

> However, "git-fsck-cache --unreachable" reports 102 unreachable blobs,
> which worries me. It's really blobs only, which is strange: it implies
> that we did the "git-update-cache" but not a "git-write-tree" (or that the
> git-write-tree failed for some reason, but that sounds even stranger,
> since we did successfully do all the commits)

> The only way I can see the unreachable blobs happening is if one of the
> ChangeSet entries in cvsps mentions the _same_ pathname twice for a single
> ChangeSet. David, is that possible?

Yes, it is, I had that problem before. For example tlr commtis the
changelog seperate with '# changlog' or so log message and cvsps thinks
because of the 'time fuzz which defaults to a way to high value' that
three changelog commits are all one delta. And that it adds three
entries. And what annoys me most in the wrong direction. So if you would
apply them as patches they don't apply because of the wrong ordering.

Reference:
		PatchSet 3005
		Date: 2002/12/07 19:19:42
		Author: roessler
		Branch: HEAD
		Tag: (none)
		Log:
		# changelog commit

		Members:
			ChangeLog:3.7->3.8
			ChangeLog:3.6->3.7
			ChangeLog:3.5->3.6

Just call cvsps with -z "20" for the mutt repository also -z 1 should
work because the timestamps of one 'commit' are all set to the same
value.

	Thomas

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Linus Torvalds wrote:
> 
> Exactly because it's only blobs, it really does smell like a cvsps issue. 
> My scripts always use "git-update-cache --add -- filename", so it never 
> creates any blobs _except_ when it adds them to the index (and thus 
> write-tree should always pick them up, unless we update the index again 
> before the next write-tree happens).

Looking at the contents of these files, all but one of them are changelog 
files, which would be consistent with this theory - if gitps ends up 
"smushing together" two separate commits (and mutt seems to have the bad 
habit of having just a simple "# changelog commit" as the commit message, 
so it would likely trigger the "same commit message" logic), you'd get 
exactly this.

The one non-changelog file looks like some kind of message translation
thing:

	# This file was prepared by (in alphabetical order):
	#
	#   Alexey Vyskubov (alexey@pepper.spb.ru)
	#   Andrew W. Nosenko (awn@bcs.zp.ua)
	#   Michael Sobolev (mss@transas.com)
	#   Vsevolod Volkov (vvv@mutt.org.ua)
	#
	# To contact translators, please use mutt-ru mailing list:
	#   http://woe.spb.ru/mailman/listinfo/mutt-ru
	#
	msgid ""
	msgstr ""
	"Project-Id-Version: mutt-1.4i\n"
	"POT-Creation-Date: 2002-05-02 01:08+0200\n"
	"PO-Revision-Date: 2002-05-03 22:53+0300\n"

	...

	#: alias.c:280
	#, c-format
	msgid "[%s = %s] Accept?"
	msgstr "[%s = %s] ðÒÉÎÑÔØ?"

	...

and it looks like it is "po/ru.po". Indeed, that's a big clue:

	---------------------
	PatchSet 2869 
	Date: 2002/05/13 21:17:48
	Author: roessler
	Branch: mutt-1-4-stable
	Tag: (none) 
	Log:
	From: Vsevolod Volkov <vvv@mutt.org.ua>
	
	update
	
	Members: 
	        po/ru.po:1.129.2.5->1.129.2.6 
	        po/ru.po:1.129.2.4->1.129.2.5 
	
	---------------------

and I thus rest my case. cvs2git is doing the right thing, and this is 
something that needs to be fixed in cvsps in case anybody cares.

		Linus

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, David Mansfield wrote:
> 
> Sounds possible.  Unfortunately, the 'uniqueness' of a commit actually 
> doesn't exist.  It's all smoke-and-mirrors.  In order to disallow this 
> (which I think need to do) I'd need to use some commit member 
> information, and add some heuristic: if this file is already in the 
> commit, then this MUST be a different commit.  Unfortunately, it's 
> possible that the 'member' already in the commit is the wrong one and 
> this is the right one, which just sounds horribly ugly to me.
> 
> I'll think on it.

I think it's a fundamentally hard problem to fix, but it may be that the 
fix is to give hints about command line options and in particular the time 
fuzz thing to try.

So maybe just _detection_ logic in cvsps, along with a warning like

	"time fuzz is 600 seconds, and the time difference between the two
	 commits of this file was 431 seconds. You may want to try a lower
	 -z argument"

or something.

It might also be possible to try to sort all the names by date of commit
first, and see if they "bunch up" into groups of low fuzz with much bigger 
fuzz in between groups..

		Linus

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Thomas Glanzmann wrote:
> 
> Just call cvsps with -z "20" for the mutt repository also -z 1 should
> work because the timestamps of one 'commit' are all set to the same
> value.

Ahh, the mutt people really use something else for development, and this 
is just an export into CVS (like the Linux bkcvs tree)? Or do they just 
have fast machines and no networking? Or are there good versions of CVS 
around that re-use the same time across one whole commit?

		Linus

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Thomas Glanzmann wrote:
> 
> btw:
> 
> For the mutt tree there are a few 'empty commits' eg were the
> parent tree is the same as the current. This is because git ignores
> .cvsignore and they commited some .cvsignore files without any other
> deltas. I don't know if you want to handle this. Just a note.

I don't like source repositories with dot-files, and I thought it was a
good idea to disallow them, but on the other hand I'd like it even less if
some CVS-weenie goes and says "I can't convert my project to git without
potentially losing information".

So in the name of furthering humanity through allowing people to migrate
away from CVS, I'm considering making the git dot-file check be more
specific to "." ".." and ".git". After all, project-specific rules might
have their own porcelain-related ignore-files that cause dot-files to
never appear..

Hmm.

		Linus

Re: gitweb wishlist

[H. Peter Anvin]

Linus Torvalds wrote:
> 
> I don't like source repositories with dot-files, and I thought it was a
> good idea to disallow them, but on the other hand I'd like it even less if
> some CVS-weenie goes and says "I can't convert my project to git without
> potentially losing information".
> 
> So in the name of furthering humanity through allowing people to migrate
> away from CVS, I'm considering making the git dot-file check be more
> specific to "." ".." and ".git". After all, project-specific rules might
> have their own porcelain-related ignore-files that cause dot-files to
> never appear..
> 

There is another good reason to allow dot files: since git handles 
sparse trees well, and only needs metadata at the top level, it's pretty 
ideal for keeping track of people's account profiles across accounts, 
and that's mostly dot files.

I was considering using CVS for that at one point, but the way CVS 
distributes its metadata and recurses makes that insane.  For git, it 
would be trivial.

	-hpa

Re: gitweb wishlist

[Thomas Glanzmann]

Hello,

> So in the name of furthering humanity through allowing people to migrate
> away from CVS, I'm considering making the git dot-file check be more
> specific to "." ".." and ".git". After all, project-specific rules might
> have their own porcelain-related ignore-files that cause dot-files to
> never appear..

Allowing dot files is a good thing, I think. But there is another issue
I want to hear your comment on. My git frontend uses regular expressions
from .git/ignore to filter orphan files I know about when calling 'git
status' or 'git orhpan'. However, I thought to add this file to
versioning. Because I think it belongs there. However I have to think
that through.

So if there would be a .git/etc or something were I can put files which
stay under revision control and maybe add some options to the frontend
or git core to don't diff against them per default. But this sounds just
ugly. Maybe I should just retrieve the ignore list when doing
push/pulls. But I dislike this idea to. However I have to think a bit
longer over this.

Comments, anyone?

	Thomas

Re: gitweb wishlist

[Kay Sievers]

On Tue, May 24, 2005 at 02:13:26PM -0700, Linus Torvalds wrote:
> 
> 
> On Tue, 24 May 2005, Thomas Glanzmann wrote:
> > 
> > btw:
> > 
> > For the mutt tree there are a few 'empty commits' eg were the
> > parent tree is the same as the current. This is because git ignores
> > .cvsignore and they commited some .cvsignore files without any other
> > deltas. I don't know if you want to handle this. Just a note.
> 
> I don't like source repositories with dot-files, and I thought it was a
> good idea to disallow them, but on the other hand I'd like it even less if
> some CVS-weenie goes and says "I can't convert my project to git without
> potentially losing information".
> 
> So in the name of furthering humanity through allowing people to migrate
> away from CVS, I'm considering making the git dot-file check be more
> specific to "." ".." and ".git". After all, project-specific rules might
> have their own porcelain-related ignore-files that cause dot-files to
> never appear..

What about allowing to put some file inside of .git/ under revision-control
too? Wouldn't it be nice to have something like an "ignore" file or other
repository meta-data managed by git itself.

Kay

Re: gitweb wishlist

[Thomas Glanzmann]

[-- Attachment #1: Type: text/plain, Size: 1759 bytes --]

Hello,

> I was considering using CVS for that at one point, but the way CVS 
> distributes its metadata and recurses makes that insane.  For git, it 
> would be trivial.

I have a shared environment, and handle it this way:

	- I edit all files on my machine on university or at least put
	  it there per scp when I am done modifing them local.

	- I have file called .env in my home which contains a list with
	  all files I want to 'distribute'.

	- There is a script called envup, which updates all files which
	  are not up2date. It uses perl because perl is everywhere I
	  have an account on. After that I call a perl preprocessor to
	  generate ssh config files (there are floating so much ssh
	  versions out there) and one to adopt my .screenrc (for the
	  machine local screen) to my needs.

my .bash_profile does a lot of testing to add paths and sets aliases
based on the environment of the machine and not based on host/domain
names. Files with lot of deltas like my .fvwm2rc or .bash_profile are
all version controled using rcs on a per file basis of course. This
works very well for me on Linux, Solaris, OpenBSD, FreeBSD, NetBSD, OSF,
AIX, MacOSX, name it ...

Oh and to bootstrap my environment on a 'new' machine:

tar cfz env.tgz `cat .env`, move it over, unpack and re login.

A while ago I also distributed binaries (bash, rar, fvwm) via envup for
my environment, but I adopted now to something that works everywhere,
and if it doesn't I do that manual. So envup only transfers stuff which
is new. (I have some accounts on low bandwidth sites). One known side
effect. If envup doesn't find md5sum binary it just transfers all files.

Oh and a friend and colleague has a similliar script only that he pushes
his environment out.

	Thomas

[-- Attachment #2: envup --]
[-- Type: text/plain, Size: 1293 bytes --]

#!/usr/bin/perl -w

use strict;
use integer;

my $command  = '';

my %local    = ();
my %remote   = ();

my @sums     = ();
my @transfer = ();

if ( -f '/home/cip/adm/sithglan/bin/envup.server'
  || -f "$ENV{HOME}/bin/envup.server" ) {
	die "NEVER EVER RUN THIS IN THE CIP POOL: YOU SUCK :-)";
	exit; # way to secure\x17redundant
}


# [remote] ------------------------------------------
@sums = `ssh 131.188.30.105 bin/envup.server`;

foreach $_ (@sums) {
	chomp;
	if (/(^[a-f0-9]{32})[ ]{2}([^\s]+)$/) {
		$remote{$2} = $1;
		# print "$1 -> $2\n";
	} else {
		print "remote: can't match: <$_>\n";
	}
}

# [local] ------------------------------------------

$command = 'md5sum ' . join(' ', keys(%remote));

@sums = qx($command 2> /dev/null);

foreach $_ (@sums) {
	chomp;
	if (/(^[a-f0-9]{32})[ ]{2}([^\s]+)$/) {
		$local{$2} = $1;
		#print "$1 -> $2\n";

	} else {
		print "local can't match: <$_>\n";
	}
}

foreach my $file (keys(%remote)) {
	if (exists($local{$file})) {
		if ($remote{$file} ne $local{$file}) {
			push(@transfer, $file);
		}

	} else {
		push(@transfer, $file);
	}
}

if (@transfer) {
        $command = "ssh 131.188.30.105 -- tar cf - " . join(' ', @transfer) . " | tar xf -";
        system($command);
}


[-- Attachment #3: envup.server --]
[-- Type: text/plain, Size: 53 bytes --]

#!/bin/bash

md5sum $(find `cat $HOME/.env` -type f)

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Kay Sievers wrote:
> 
> What about allowing to put some file inside of .git/ under revision-control
> too? Wouldn't it be nice to have something like an "ignore" file or other
> repository meta-data managed by git itself.

Put them into ".git-ignore" if so..

		Linus

Re: gitweb wishlist

[Thomas Glanzmann]

Hello,

> Ahh, the mutt people really use something else for development, and this 
> is just an export into CVS (like the Linux bkcvs tree)? Or do they just 
> have fast machines and no networking? Or are there good versions of CVS 
> around that re-use the same time across one whole commit?

I did one sampling and though it would be representative which it isn't.
What I don't understand why noone ever fixed this? cvs has its own rcs
implementation anyway to speed things up, hasn't it?

	Thomas

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Thomas Glanzmann wrote:
> 
> I did one sampling and though it would be representative which it isn't.
> What I don't understand why noone ever fixed this? cvs has its own rcs
> implementation anyway to speed things up, hasn't it?

CVS has so many warts, that people can't even be bothered to fix things
like this. It's file-based, and that's that.

Using "-z 1" with cvsps doesn't seem to work well for me, but "-z 5" seems
ok, and together with the new git that allows .cvsignore, it doesn't
generate any warnings, nor any unreachable blobs. It does generate 300 new
changesets, and how many of those are required, I dunno. Clearly 102 of
them were, to disambiguate those changelog things.

Maybe "-z 10" would have generated a better thing with fewer changes yet
still unique changesets without dup files.

That's a bit irritating, that there's this magic tweakable that generates
different trees. Oh, well.

Anyway, what worries me more is that cvsps might have re-ordered other 
changesets than just the first two. It probably doesn't _matter_, but 
still...

		Linus

Re: gitweb wishlist

[David Mansfield]

>Linus Saith,
> 
> Anyway, what worries me more is that cvsps might have re-ordered other 
> changesets than just the first two. It probably doesn't _matter_, but 
> still...
> 

I think I have an idea to prevent this incorrect ordering of patchsets. 
  We'll see in the morning how it works out.

David

Re: gitweb wishlist

[Edgar Toernig]

Linus Torvalds wrote:
> > 	- Some Shell escapes (I didn't looked into them yet)
> 
> Ok, I'll check it out. I didn't figure out what characters are 
> shell-expanded by "<<EOF" handling, and only did '$' because that showed 
> up in the syslinux archives.

Nothing is expanded when you quote the EOF-word:

cat <<"EOF"
`foo` $PATH
bar \
baz
EOF

gives:

`foo` $PATH
bar \
baz

Ciao, ET.

Re: gitweb wishlist

[Linus Torvalds]

On Wed, 25 May 2005, Edgar Toernig wrote:
> 
> Nothing is expanded when you quote the EOF-word:

Oh, wow.

It's even documented in the bash man-page, now that I understand what to 
look for.

Anyway, that doc also tells me that I'm quoting the right characters, so 
now it's not worth fixing any more.

		Linus

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:

LT> On Wed, 25 May 2005, Edgar Toernig wrote:
>> 
>> Nothing is expanded when you quote the EOF-word:

LT> Oh, wow.

Test scripts in your t/ uses <<\EOF all over the place.

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Junio C Hamano wrote:
> 
> Test scripts in your t/ uses <<\EOF all over the place.

Yeah, yeah, rub it in. I suck at shell.

The sad part is that I don't even have an excuse like "I use perl all the 
time". No, shell is my _forte_ when it comes to scripting.

		Linus "crawl under a rock" Torvalds

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:

LT> The sad part is that I don't even have an excuse like "I use
LT> perl all the time". No, shell is my _forte_ when it comes to
LT> scripting.

You are allowed to suck at scripting when you are so good at C
;-).

By the by, are you still having problem with "whatchanged -s"?
I see a few of my patches still not in your tree, and I do not
care too much about the last one (rename/copy similarity
estimator update) after you made it clear that -M and -C are of
lower priority, I would like to know if the "diff-tree -s" fix I
sent you was missing the point, and if so how.  Also the test
scripts to verify your fix to the "checkout-cache --prefix
pointing at some directory via symlink" along with your fix is
something I feel ready to go.

Hoping it would be easy to look for a message by message-ID,
here they are.  The first one is the proposed "diff-tree -s"
fix.

Subject: Re: [PATCH 3/3] Diff overhaul, adding the other half...
Message-ID: <7vr7fxuh8b.fsf@assigned-by-dhcp.cox.net>

Subject: Squelch compiler warning
Message-ID: <7vis18v4ea.fsf_-_@assigned-by-dhcp.cox.net>

Subject: [PATCH] Allow symlinks in the leading path in checkout-cache --prefix=
Message-ID: <7vacmlvwfk.fsf_-_@assigned-by-dhcp.cox.net>

Subject: [PATCH] Update rename/copy similarity estimator.
Message-ID: <7vbr70v3tf.fsf@assigned-by-dhcp.cox.net>

Re: gitweb wishlist

[Junio C Hamano]

I was browsing www.kernel.org/git and noticed that it shows
only files that exist at the tip.  How do I get history of a
file that does not exist anymore at the tip?

For example, diff-helper.c history is (quite correctly)
truncated somewhere close to where diff-tree-helper.c was
renamed to it.  From the commit log, humans can easily tell that
it used to be called diff-tree-helper.c.  I could not find an
easy way to see the history of diff-tree-helper.c file.

On probably a bit different topic (but I do not know who is
updating the copy on www.kernel.org, sorry).  Could somebody
update http://www.kernel.org/pub/software/scm/git/docs/ to
rename git-diff-tree-helper to git-diff-helper please?
git.git/Documentation/git.txt has been corrected quite some
time ago [*1*] and I do not know how the updates are propagated
to the web version of the documentation; is it a manual process?

[Footnotes]

*1* The following command tells me it was done about 10 days ago.

    git-rev-list 0 |
    git-diff-tree -S'diff-tree-helper' --stdin -v -p Documentation/git.txt

Re: gitweb wishlist

[Linus Torvalds]

On Tue, 24 May 2005, Junio C Hamano wrote:
>
> I was browsing www.kernel.org/git and noticed that it shows
> only files that exist at the tip.  How do I get history of a
> file that does not exist anymore at the tip?

The only sane interface I can think of is to expose the subdirectory 
history and then pick from that. Otherwise you'd have to actually type in 
the name, which is a bit against the notion of a graphical browsing 
interface.

		Linus

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:

LT> The only sane interface I can think of is to expose the subdirectory 
LT> history and then pick from that. Otherwise you'd have to actually type in 
LT> the name, which is a bit against the notion of a graphical browsing 
LT> interface.

Knowing to type "merge-tree.c" you need to be an old timer ;-).

Since I asked that question I found out that each commit has a
link to the diff and the tree, so if I know when merge-tree.c
disappeared, I can go backwards from there.

I think what is useful, from software archaeologist point of
view, would be to give a way to the web users to use pickaxe.
Type piece of code in the textbox and have the CGI run "rev-log
| diff-tree -S'<that piece of the code>'".

Re: gitweb wishlist

[Kay Sievers]

On Tue, May 24, 2005 at 07:23:39PM -0700, Junio C Hamano wrote:
> I was browsing www.kernel.org/git and noticed that it shows
> only files that exist at the tip.  How do I get history of a
> file that does not exist anymore at the tip?
> 
> For example, diff-helper.c history is (quite correctly)
> truncated somewhere close to where diff-tree-helper.c was
> renamed to it.  From the commit log, humans can easily tell that
> it used to be called diff-tree-helper.c.  I could not find an
> easy way to see the history of diff-tree-helper.c file.

I will add a search function when the git binaries on kernel.org get an
update. Currently it's cogito-0.10.

> On probably a bit different topic (but I do not know who is
> updating the copy on www.kernel.org, sorry).  Could somebody
> update http://www.kernel.org/pub/software/scm/git/docs/ to
> rename git-diff-tree-helper to git-diff-helper please?
> git.git/Documentation/git.txt has been corrected quite some
> time ago [*1*] and I do not know how the updates are propagated
> to the web version of the documentation; is it a manual process?

David Greaves can write to that directory. David?

Kay

Re: gitweb wishlist

[David Greaves]

>>On probably a bit different topic (but I do not know who is
>>updating the copy on www.kernel.org, sorry).  Could somebody
>>update http://www.kernel.org/pub/software/scm/git/docs/ to
>>rename git-diff-tree-helper to git-diff-helper please?
>>git.git/Documentation/git.txt has been corrected quite some
>>time ago [*1*] and I do not know how the updates are propagated
>>to the web version of the documentation; is it a manual process?
>>    
>>
>
>David Greaves can write to that directory. David?
>  
>
Yep.
It is a manual rsync process.

I last ran it on the 22nd as soon as Linus committed my latest doc patches.

It's my bad though - I didn't update index.html which should be a copy
of git.html

 http://www.kernel.org/pub/software/scm/git/docs/git.html
is correct.

Anyway, done now, give it time to replicate.

David

Re: gitweb wishlist

[Kay Sievers]

On Tue, May 24, 2005 at 07:23:39PM -0700, Junio C Hamano wrote:
> I was browsing www.kernel.org/git and noticed that it shows
> only files that exist at the tip.  How do I get history of a
> file that does not exist anymore at the tip?
> 
> For example, diff-helper.c history is (quite correctly)
> truncated somewhere close to where diff-tree-helper.c was
> renamed to it.  From the commit log, humans can easily tell that
> it used to be called diff-tree-helper.c.  I could not find an
> easy way to see the history of diff-tree-helper.c file.

If git-diff-tree is given the -M:
  git-rev-list HEAD | git-diff-tree -r -M --stdin diff-helper.c

and it would print:
  99665af5c0be0fe4319b39183e84917993153576 (from 13ab4462d2aefb252d7c916bd537151856b7c967)
  :100644 100644 51bb658be4f73c00016b4ecb82f09d30941998a4 51bb658be4f73c00016b4ecb82f09d30941998a4 R10000 diff-tree-helper.c diff-helper.c

instead of:
  99665af5c0be0fe4319b39183e84917993153576 (from 13ab4462d2aefb252d7c916bd537151856b7c967)
  :000000 100644 0000000000000000000000000000000000000000 51bb658be4f73c00016b4ecb82f09d30941998a4 N      diff-helper.c

gitweb could follow the old filename and show the whole history. :)

Thanks,
Kay

Re: gitweb wishlist

[Kay Sievers]

On Wed, May 25, 2005 at 02:35:44PM +0200, Kay Sievers wrote:
> On Tue, May 24, 2005 at 07:23:39PM -0700, Junio C Hamano wrote:
> > I was browsing www.kernel.org/git and noticed that it shows
> > only files that exist at the tip.  How do I get history of a
> > file that does not exist anymore at the tip?
> > 
> > For example, diff-helper.c history is (quite correctly)
> > truncated somewhere close to where diff-tree-helper.c was
> > renamed to it.  From the commit log, humans can easily tell that
> > it used to be called diff-tree-helper.c.  I could not find an
> > easy way to see the history of diff-tree-helper.c file.
> 
> If git-diff-tree is given the -M:
>   git-rev-list HEAD | git-diff-tree -r -M --stdin diff-helper.c
> 
> and it would print:
>   99665af5c0be0fe4319b39183e84917993153576 (from 13ab4462d2aefb252d7c916bd537151856b7c967)
>   :100644 100644 51bb658be4f73c00016b4ecb82f09d30941998a4 51bb658be4f73c00016b4ecb82f09d30941998a4 R10000 diff-tree-helper.c diff-helper.c
                                                                                                      ^^^^^
Btw:
Can we add a leading '0' to have 3-digit values every time? It's pretty
stupid to get the '100' out ot that field instead of '10'. :)

Kay

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "KS" == Kay Sievers <kay.sievers@vrfy.org> writes:

KS> gitweb could follow the old filename and show the whole history. :)

Yes, it would work for diff-helper.c, but not for merge-tree.c,
which does not have any equivalent in the tip of the tree.

Re: gitweb wishlist

[Junio C Hamano]

Stupidly, I use "%d" without precision to output score, and
worse yet the score is not scaled (meaning it would change when
definition of MAX_SCORE changes, exposing internal scale that
rename/copy uses).

For the sake of parsability, I think I should make that number
always 3 digit and per-cent; so 86% hit would be "C086" and the
bogus R10000 you pointed out would become "R100".

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "DG" == David Greaves <david@dgreaves.com> writes:

DG> Anyway, done now, give it time to replicate.

Thanks.  Would something like the following (1) easy to arrange
and (2) make your life easier?

    - make index.html symlink to git.html
    - have cron job to build and install those pages at
      www.kernel.org

Re: gitweb wishlist

[David Greaves]

Junio C Hamano wrote:

>>>>>>"DG" == David Greaves <david@dgreaves.com> writes:
>>>>>>            
>>>>>>
>
>DG> Anyway, done now, give it time to replicate.
>
>Thanks.  Would something like the following (1) easy to arrange
>and (2) make your life easier?
>
>    - make index.html symlink to git.html
>    - have cron job to build and install those pages at
>      www.kernel.org
>  
>
Hmm - sounds complicated ;)



-- 

Re: gitweb wishlist

[David Mansfield]

Thomas Glanzmann wrote:
> Hello,
> 
> 
>>	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
>>	    memdisk/init32.asm:1.3=after, memdisk/Makefile:1.26=before. Treated as 'before'
>>	WARNING: Invalid PatchSet 775, Tag syslinux-2_12-pre7:
>>	    memdisk/init32.asm:1.3=after, memdisk/e820test.c:1.7=before. Treated as 'before'
>>	...
> 
> 
> actually I think this is the broken upstream version. It can't parse
> dates right. Just look at the exported patches and see if them all from
> 1970. However the debian package has a patch in which solves it:
> 
> maybe you should try with the attached patch or with the version that
> comes with debian sarge. I also reported this problem a while back to
> the original author.
> 

I was about to apply this and I already had in it my cvs tree!  Funny 
how these things go.  I must have gotten it before, applied it and never 
released a new version.  Funny that this one hase the tm.tm_isdst = 0 
that is missing from the version I applied (and fixes an important bug).

Anyway, I'm about to release a new version cumulative with all this, a 
fixed ancestor version, correct ordering for those pesky import commits, 
and a couple other annoying fixes.

BTW: the above warnings are actually legit in this case.

David

Re: gitweb wishlist

[Junio C Hamano]

>>>>> "KS" == Kay Sievers <kay.sievers@vrfy.org> writes:

KS> On Thu, 2005-05-12 at 13:07 -0700, Junio C Hamano wrote:
>> * [Previous page] [Next page] would be nice in addition to last
>> 10, day, week, etc.

KS> That should be easy to do with the parameters we have now for the
KS> git-rev-list. I will first finish the new browser through the
KS> trees/files, then the project overview page and after that try the
KS> pager,

Is this still on your list of things to do?