* Re: [PATCH 5/8] Add a config option for remotes to specify a foreign vcs
From: Jeff King @ 2009-08-11 23:02 UTC (permalink / raw)
To: Nanako Shiraishi
Cc: Johannes Schindelin, Bert Wesarg, Junio C Hamano, Daniel Barkalow,
git, Brian Gernhardt
In-Reply-To: <20090812075914.6117@nanako3.lavabit.com>
On Wed, Aug 12, 2009 at 07:59:14AM +0900, Nanako Shiraishi wrote:
> Then how about using a prefix that has been invalid, "vcs::$string",
> for example?
I have no problem with that, and I think it makes it even more visually
obvious what is going. For example:
svn::http://server/path/to/repo
makes the "svn" prefix jump out a bit more.
-Peff
^ permalink raw reply
* Re: sharing git work while downstream from svn?
From: Avery Pennarun @ 2009-08-11 23:03 UTC (permalink / raw)
To: tfogal; +Cc: git
In-Reply-To: <auto-000020209577@sci.utah.edu>
On Tue, Aug 11, 2009 at 10:55 PM, tom fogal<tfogal@alumni.unh.edu> wrote:
> This gets to be a mess when trunk changes: I'll rebase + potentially
> fix some conflicts. Other developers with some of the experimental
> patches will svn update, and get similar conflicts. These might differ
> in subtle ways, and now exchanging patches gets more difficult.
We use git-svn at Versabanq, and our process is very simple: never use
rebase, period.
Instead, do all your work in a branch *other* than the git-svn main
branch. When you're ready to merge your stuff into svn, do:
git checkout git-svn
git svn rebase
git checkout myworkingbranch
git merge git-svn
# and resolve any conflicts
git checkout git-svn
git merge --no-ff myworkingbranch # the --no-ff is very important here!
git svn dcommit
This basically results in a *single* commit getting sent to svn,
rather than the batch of all the git commits you've been working on.
Most svn users don't care about this, because they lose all that
granularity whenever they merge a branch anyhow. Meanwhile, all your
git users never have to worry about rebasing *anything* and can use
the normal git merge/pull stuff.
Have fun,
Avery
^ permalink raw reply
* Re: sharing git work while downstream from svn?
From: tom fogal @ 2009-08-11 23:14 UTC (permalink / raw)
To: Avery Pennarun; +Cc: git
In-Reply-To: <32541b130908111603v1e3f6c42peac792caf7097e0d@mail.gmail.com>
Avery Pennarun <apenwarr@gmail.com> writes:
> On Tue, Aug 11, 2009 at 10:55 PM, tom fogal<tfogal@alumni.unh.edu> wrote:
> > This gets to be a mess when trunk changes: I'll rebase + potentially
> > fix some conflicts. Other developers with some of the experimental
> > patches will svn update, and get similar conflicts. These might differ
> > in subtle ways, and now exchanging patches gets more difficult.
>
> Instead, do all your work in a branch *other* than the git-svn main
> branch. When you're ready to merge your stuff into svn, do:
[snip]
> This basically results in a *single* commit getting sent to svn,
> rather than the batch of all the git commits you've been working
> on. Most svn users don't care about this, because they lose all that
> granularity whenever they merge a branch anyhow.
... but I, as a git user forced to live in an svn world, *do* value all
of that history. When I find a bug a month later, I want git-bisect
to be useful. Further, when I'm reviewing sets of changes in a search
for some particular change, I want to be able to skip over large sets
of patches simply by looking at the first line of a commit log. If I
squash all that history down, I have to wade into the patch itself.
-tom
^ permalink raw reply
* Re: block-sha1: improve code on large-register-set machines
From: Brandon Casey @ 2009-08-11 23:13 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Nicolas Pitre, Git Mailing List, Junio C Hamano
In-Reply-To: <alpine.LFD.2.01.0908111553460.28882@localhost.localdomain>
Linus Torvalds wrote:
>
> On Tue, 11 Aug 2009, Brandon Casey wrote:
>> In that case, why not change the interface of blk_SHA1Block() so that its
>> second argument is const unsigned char* and get rid of __d and the { } ?
>
> Because on big-endian, or on architectures like x86 that have an efficient
> byte swap, that would be horrible.
Sorry, I missed Nicolas's first message where he said his SHA_SRC macro was
for arm only.
I started at your reply to him which only has the snippet which says
"...this provides the exact same performance as the ntohl() based version
except that this now cope with unaligned buffers too".
My mistake.
-brandon
^ permalink raw reply
* Re: block-sha1: improve code on large-register-set machines
From: Linus Torvalds @ 2009-08-11 23:14 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Git Mailing List, Junio C Hamano
In-Reply-To: <alpine.LFD.2.01.0908111550470.28882@localhost.localdomain>
On Tue, 11 Aug 2009, Linus Torvalds wrote:
>
>
> On Tue, 11 Aug 2009, Nicolas Pitre wrote:
> >
> > Well... gcc is really strange in this case (and similar other ones) with
> > ARM compilation. A good indicator of the quality of the code is the
> > size of the stack frame. When using the "+m" then gcc creates a 816
> > byte stack frame, the generated binary grows by approx 3000 bytes, and
> > performances is almost halved (7.600s). Looking at the assembly result
> > I just can't figure out all the crazy moves taking place. Even the
> > version with no barrier what so ever produces better assembly with a
> > stack frame of 560 bytes.
>
> Ok, that's just crazy. That function has a required stack size of exactly
> 64 bytes, and anything more than that is just spilling. And if you end up
> with a stack frame of 560 bytes, that means that gcc is doing some _crazy_
> spilling.
Btw, what I think happens is:
- gcc turns all those array accesses into pseudo's
So now the 'array[16]' is seen as just another 16 variables rather than
an array.
- gcc then turns it into SSA, where each assignment basically creates a
new variable. So the 16 array variables (and 5 regular variables) are
now expanded to 80 SSA asignments (one array assignment per SHA1 round)
plus an additional 2 assignments to the "regular" variables per round
(B and E are changed each round). So in SSA form, you actually end up
having 240 pseudo's associated with the actual variables. Plus all
the temporaries of course.
- the thing then goes crazy and tries to generate great code from that
internal SSA model. And since there are never more than ~25 things
_live_ at any particular point, it works fine with lots of registers,
but on small-register machines gcc just goes crazy and has to spill.
And it doesn't spill 'array[x]' entries - it spills the _pseudo's_ it
has created - hundreds of them.
- End result: if the spill code doesn't share slots, it's going to create
a totally unholy mess of crap.
That's what the whole 'volatile unsigned int *' game tried to avoid. But
it really sounds like it's not working too well for you. And the _big_
memory barrier ends up helping just because with that in place, you end up
being almost entirely unable to schedule _anything_ between the different
SHA rounds, so you end up with only six or seven variables "live" in
between those barriers, and the stupid register allocator/spill logic
doesn't break down too badly.
The thing is, if you do full memory barriers, then you're probably better
off making both the loads and the stores be "volatile". That should have
similar effects.
The downside with that is that it really limits the loads. So (like the
full memory barrier) it's a big hammer approach. But it probably generates
better code for you, because it avoids the mental breakdown of gcc
spilling its pseudo's.
Linus
^ permalink raw reply
* Re: sharing git work while downstream from svn?
From: Avery Pennarun @ 2009-08-11 23:17 UTC (permalink / raw)
To: tom fogal; +Cc: git
In-Reply-To: <auto-000020209671@sci.utah.edu>
On Tue, Aug 11, 2009 at 11:14 PM, tom fogal<tfogal@alumni.unh.edu> wrote:
> Avery Pennarun <apenwarr@gmail.com> writes:
>> On Tue, Aug 11, 2009 at 10:55 PM, tom fogal<tfogal@alumni.unh.edu> wrote:
>> > This gets to be a mess when trunk changes: I'll rebase + potentially
>> > fix some conflicts. Other developers with some of the experimental
>> > patches will svn update, and get similar conflicts. These might differ
>> > in subtle ways, and now exchanging patches gets more difficult.
>>
>> Instead, do all your work in a branch *other* than the git-svn main
>> branch. When you're ready to merge your stuff into svn, do:
> [snip]
>> This basically results in a *single* commit getting sent to svn,
>> rather than the batch of all the git commits you've been working
>> on. Most svn users don't care about this, because they lose all that
>> granularity whenever they merge a branch anyhow.
>
> ... but I, as a git user forced to live in an svn world, *do* value all
> of that history. When I find a bug a month later, I want git-bisect
> to be useful. Further, when I'm reviewing sets of changes in a search
> for some particular change, I want to be able to skip over large sets
> of patches simply by looking at the first line of a commit log. If I
> squash all that history down, I have to wade into the patch itself.
That's the great thing! Your git history never gets lost with this
technique, since you *never* rewind any branches. The detailed
history just never makes it into svn, which has no way of representing
it anyhow.
As a bonus, the fact that your git history becomes more and more
detailed vs. the svn history slowly makes your case for switching
everybody else to git that much stronger :)
Have fun,
Avery
P.S. Shameless plug: I wrote the chapter about git-svn in O'Reilly's
"Version Control With Git," in which I described this technique in
more detail. (I *don't* make any money on commissions on that book,
as I'm not the primary author.)
^ permalink raw reply
* Re: [RFC/PATCH 5/6] Let transport_helper_init() decide if a remote helper program can be used
From: Daniel Barkalow @ 2009-08-11 23:28 UTC (permalink / raw)
To: Johan Herland; +Cc: git, gitster, benji, Johannes.Schindelin
In-Reply-To: <1249985426-13726-6-git-send-email-johan@herland.net>
On Tue, 11 Aug 2009, Johan Herland wrote:
> @@ -293,6 +301,16 @@ int transport_helper_init(struct transport *transport)
> data->name = xstrndup(transport->url, eom - transport->url);
> }
>
> + strbuf_addf(&buf, "remote-%s", data->name);
> + cmd = strbuf_detach(&buf, NULL);
> + if (!is_git_command_or_alias(cmd)) {
> + warning("Could not find remote helper command \"git %s\". Assuming native remote.", cmd);
Won't this give you a warning for:
url = master.kernel.org:something
of 'Could not find (...) "git remote-master.kernel.org" (...)'? That
would be certain to upset some people. I think we can assume that people's
scheme parts of their URLs that are actually URLs don't contain '.', and
that people with:
url = master:something
will append their domains if the warning gets annoying.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: [PATCH] Change mentions of "git programs" to "git commands"
From: Ori Avtalion @ 2009-08-11 23:29 UTC (permalink / raw)
To: Nanako Shiraishi; +Cc: Junio C Hamano, git
In-Reply-To: <20090812075946.6117@nanako3.lavabit.com>
On 08/12/2009 01:59 AM, Nanako Shiraishi wrote:
>> -git-mailsplit - Simple UNIX mbox splitter program
>> +git-mailsplit - Simple UNIX mbox splitter
>>
>> SYNOPSIS
>> --------
>
> Sorry, but I don't understand why it is necessary.
>
It's another case where a command is called a "program" when, to the
user, it's simply a command such as "git mailsplit". Having the word
"command" in a command description is redundant, so I just dropped the word.
And here's another command with a similar description that I missed before:
"git-merge-one-file - The standard helper *program* to use with
git-merge-index"
^ permalink raw reply
* Re: block-sha1: improve code on large-register-set machines
From: Artur Skawina @ 2009-08-11 23:45 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Nicolas Pitre, Git Mailing List, Junio C Hamano
In-Reply-To: <alpine.LFD.2.01.0908111550470.28882@localhost.localdomain>
Linus Torvalds wrote:
>
> One thing that strikes me is that I've been just testing with gcc-4.4, and
> BenH (who did some tests on PPC where SHA1 is just _trivial_ because it
> all fits in the normal register space) noticed that older versions of gcc
> that he tested did much worse on this.
>
> I think Artur also posted (x86) numbers with older gcc versions doing
> worse. Maybe you're seeing some of that?
FWIW, this is how it looks here. On 32-bit x86 gcc4.4 makes a large
difference[1], but your code does fairly well w/ most gccs, relatively
to all the other C implementations.
artur
P4: [linusv is the recent one w/ the volatile stores]
### sha1bench-gcc295: GCCVER 2.95.4 20030502 (prerelease)
rfc3174 0.9438 64.67
linus 0.9081 67.21
linusv 0.4155 146.9
linusp4 0.8761 69.66
linusas 0.9619 63.45
linusas2 1.025 59.52
mozilla 1.314 46.46
mozillaas 1.132 53.92
### sha1bench-gcc31: GCCVER 3.2 2002-07-26 (prerelease)
rfc3174 0.8582 71.12
linus 0.7943 76.84
linusv 0.5667 107.7
linusp4 0.7224 84.48
linusas 0.7127 85.64
linusas2 0.5109 119.5
mozilla 1.251 48.79
mozillaas 1.239 49.27
### sha1bench-gcc32: GCCVER 3.2.3
rfc3174 0.9062 67.35
linus 0.5555 109.9
linusv 0.3647 167.4
linusp4 0.5337 114.4
linusas 0.7126 85.66
linusas2 0.5089 119.9
mozilla 1.138 53.64
mozillaas 1.075 56.78
### sha1bench-gcc33: GCCVER 3.3.6
rfc3174 0.9029 67.6
linus 0.6059 100.7
linusv 0.3734 163.4
linusp4 0.6695 91.16
linusas 0.7832 77.93
linusas2 0.571 106.9
mozilla 1.083 56.36
mozillaas 1.078 56.62
### sha1bench-gcc34: GCCVER 3.4.6 20060121 (prerelease)
rfc3174 0.9277 65.79
linus 0.6583 92.71
linusv 0.6096 100.1
linusp4 0.7326 83.31
linusas 0.7383 82.67
linusas2 0.6264 97.44
mozilla 1.398 43.67
mozillaas 1.392 43.84
### sha1bench-gcc40: GCCVER 4.0.4 20061113 (prerelease)
rfc3174 0.9889 61.72
linus 0.7508 81.29
linusv 0.7752 78.73
linusp4 0.6548 93.21
linusas 0.4904 124.5
linusas2 0.6378 95.7
mozilla 1.528 39.93
mozillaas 1.596 38.24
### sha1bench-gcc41: GCCVER 4.1.3 20080704 (prerelease)
rfc3174 0.9798 62.29
linus 0.6993 87.28
linusv 0.767 79.57
linusp4 0.6785 89.95
linusas 0.6555 93.11
linusas2 0.691 88.32
mozilla 1.594 38.3
mozillaas 1.566 38.98
### sha1bench-gcc42: GCCVER 4.2.5 20090330 (prerelease)
rfc3174 1.138 53.63
linus 0.7772 78.53
linusv 0.6138 99.43
linusp4 0.7018 86.97
linusas 0.8164 74.76
linusas2 0.7038 86.73
mozilla 1.697 35.97
mozillaas 1.491 40.94
### sha1bench-gcc43: GCCVER 4.3.5 20090810 (prerelease)
rfc3174 1.148 53.15
linus 0.7085 86.14
linusv 0.5474 111.5
linusp4 0.5399 113
linusas 0.7522 81.14
linusas2 0.5341 114.3
mozilla 1.723 35.43
mozillaas 1.502 40.64
### sha1bench-gcc44: GCCVER 4.4.2 20090809 (prerelease)
rfc3174 1.451 42.06
linus 0.5871 104
linusv 0.3713 164.4
linusp4 0.4367 139.8
linusas 0.4083 149.5
linusas2 0.4372 139.6
mozilla 1.104 55.27
mozillaas 1.314 46.44
And on Atom:
### sha1bench-gcc295: GCCVER 2.95.4 20030502 (prerelease)
rfc3174 1.905 32.04
linus 1.089 56.06
linusv 0.8134 75.04
linusp4 1.086 56.19
linusas 1.009 60.52
linusas2 1.255 48.63
mozilla 2.663 22.92
### sha1bench-gcc31: GCCVER 3.2 2002-07-26 (prerelease)
rfc3174 2.141 28.51
linus 1.022 59.75
linusv 0.8323 73.34
linusp4 1.061 57.54
linusas 0.9889 61.72
linusas2 0.9204 66.32
mozilla 2.573 23.72
### sha1bench-gcc32: GCCVER 3.2.3
rfc3174 2.155 28.32
linus 0.9031 67.58
linusv 0.7849 77.76
linusp4 0.847 72.06
linusas 0.9888 61.73
linusas2 0.912 66.93
mozilla 2.485 24.56
### sha1bench-gcc33: GCCVER 3.3.6
rfc3174 2.178 28.02
linus 0.9489 64.32
linusv 0.8642 70.63
linusp4 0.8784 69.48
linusas 1.017 60.03
linusas2 0.906 67.37
mozilla 2.541 24.02
### sha1bench-gcc34: GCCVER 3.4.6 20060121 (prerelease)
rfc3174 2.157 28.3
linus 0.9481 64.37
linusv 0.8383 72.8
linusp4 0.965 63.25
linusas 0.9852 61.95
linusas2 0.9809 62.22
mozilla 3.143 19.42
### sha1bench-gcc40: GCCVER 4.0.4 20061113 (prerelease)
rfc3174 2.088 29.24
linus 0.9706 62.89
linusv 0.928 65.77
linusp4 1.003 60.85
linusas 0.9478 64.4
linusas2 0.9475 64.42
mozilla 2.742 22.26
### sha1bench-gcc41: GCCVER 4.1.3 20080704 (prerelease)
rfc3174 2.047 29.81
linus 0.9778 62.42
linusv 1.051 58.06
linusp4 1.062 57.46
linusas 1.052 58.01
linusas2 1.069 57.12
mozilla 2.6 23.47
### sha1bench-gcc42: GCCVER 4.2.5 20090330 (prerelease)
rfc3174 2.025 30.14
linus 0.9622 63.43
linusv 0.7984 76.44
linusp4 0.8967 68.07
linusas 1.018 59.94
linusas2 0.9048 67.46
mozilla 2.748 22.21
### sha1bench-gcc43: GCCVER 4.3.5 20090810 (prerelease)
rfc3174 2.043 29.88
linus 0.9436 64.69
linusv 0.8532 71.54
linusp4 0.8531 71.54
linusas 1.04 58.71
linusas2 0.8495 71.85
mozilla 2.678 22.79
### sha1bench-gcc44: GCCVER 4.4.2 20090809 (prerelease)
rfc3174 2.119 28.8
linus 0.9132 66.84
linusv 0.8632 70.71
linusp4 0.9842 62.02
linusas 1.027 59.45
linusas2 0.9844 62
mozilla 2.214 27.57
^ permalink raw reply
* Re: [PATCH 5/8] Add a config option for remotes to specify a foreign vcs
From: Johannes Schindelin @ 2009-08-11 23:53 UTC (permalink / raw)
To: Jeff King
Cc: Bert Wesarg, Junio C Hamano, Daniel Barkalow, git,
Brian Gernhardt
In-Reply-To: <20090811214849.GA3868@sigill.intra.peff.net>
Hi,
On Tue, 11 Aug 2009, Jeff King wrote:
> On Tue, Aug 11, 2009 at 06:20:54PM +0200, Johannes Schindelin wrote:
>
> > > o [user@]host.xz:/path/to/repo.git/
> >
> > That is a valid concern. I'd suggest to disallow @ and . in the protocol
> > name (maybe even everything non-alnum). For shortcuts that really read
> > like "svn", I think it is not too much asked for to require adjusting the
> > URL (by prefixing ssh:// and adjusting the path).
> >
> > If there is really anybody who uses Git via ssh to access a server called
> > "svn", we could just as well have a little fun with them, don't you agree?
>
> It is not actually that unreasonable. I have remotes which point to:
>
> vcs:git/foo.git
That is still not "svn".
> where "vcs" is a shorthand for vcs.gtisc.gatech.edu, defined in my
> .ssh/config (it's a machine which hosts several different version
> control systems). So I could see somebody having something like "svn" if
> they were, e.g., hosting a git-to-svn conversion on their company's svn
> server.
If _I_ were to judge whether to make it convenient for computer-savvy
people like you who would have _no_ problem diagnosing the problem (_if_
they have the problem, having edited .ssh/config themselves!), who would
curse briefly, and then go on fixing the problem, or in the alternative
make it convenient for people who do not know their way around .ssh/config
as well as you (and who happen to make up the _vast_ majority of Git users
by now [*1*]), and who would really prefer to have an easy way to clone
"foreign" repositories, I have _no_ problem deciding which way to go.
So I'm a bastard. Big news. But I'm a pragmatic one.
Ciao,
Dscho
Footnote [*1*]: Maybe I was helping to increase the number of Git users
who do not know .ssh/config with my work on Git for Windows, but that was
not my original intention. Maybe I should start to feel better about
it...
^ permalink raw reply
* Re: [PATCH 5/8] Add a config option for remotes to specify a foreign vcs
From: Johannes Schindelin @ 2009-08-12 0:14 UTC (permalink / raw)
To: Jeff King
Cc: Nanako Shiraishi, Bert Wesarg, Junio C Hamano, Daniel Barkalow,
git, Brian Gernhardt
In-Reply-To: <20090811230233.GA25642@coredump.intra.peff.net>
Hi,
On Tue, 11 Aug 2009, Jeff King wrote:
> On Wed, Aug 12, 2009 at 07:59:14AM +0900, Nanako Shiraishi wrote:
>
> > Then how about using a prefix that has been invalid, "vcs::$string",
> > for example?
>
> I have no problem with that, and I think it makes it even more visually
> obvious what is going. For example:
>
> svn::http://server/path/to/repo
>
> makes the "svn" prefix jump out a bit more.
... and http:://repo.or.cz/r/git.git ? Thanks. But no, thanks.
Ciao,
Dscho
^ permalink raw reply
* [RFCv3 0/4] CVS remote helper
From: Johan Herland @ 2009-08-12 0:13 UTC (permalink / raw)
To: git; +Cc: Johan Herland, barkalow, gitster, Johannes.Schindelin
Hi,
Another iteration of the patch series implementing a CVS remote helper, as
promised a couple of days ago. Changes from the previous iteration:
- Rebased on top of Daniel Barkalow's latest foreign VCS helpers work
(aka. 'db/vcs-helper' (early part)). This means that all the "generic"
foreign-scm patches that were part of the previous iteration are no
longer needed, and only the CVS remote helper patches remain.
Also, this series applies cleanly to current 'pu'.
- Replaced the "git-vcs-cvs" naming with "git-remote-cvs" throughout the code
and documentation.
- Minor updates to the git-remote-cvs implementation in order to more closely
follow the current remote helper API.
- Split up the patch series into somewhat smaller patches to (hopefully) be
allowed onto git@vger.kernel.org.
Have fun! :)
...Johan
Johan Herland (4):
Basic build infrastructure for Python scripts
Add Python support library for CVS remote helper
Third draft of CVS remote helper program
Add simple selftests of git-remote-cvs functionality
Documentation/git-remote-cvs.txt | 85 ++++
Makefile | 46 ++
configure.ac | 3 +
git-remote-cvs.py | 697 ++++++++++++++++++++++++++++
git_remote_cvs/.gitignore | 2 +
git_remote_cvs/Makefile | 27 ++
git_remote_cvs/changeset.py | 114 +++++
git_remote_cvs/commit_states.py | 52 +++
git_remote_cvs/cvs.py | 884 ++++++++++++++++++++++++++++++++++++
git_remote_cvs/cvs_revision_map.py | 362 +++++++++++++++
git_remote_cvs/cvs_symbol_cache.py | 283 ++++++++++++
git_remote_cvs/git.py | 586 ++++++++++++++++++++++++
git_remote_cvs/setup.py | 12 +
git_remote_cvs/util.py | 147 ++++++
t/t9800-remote-cvs-basic.sh | 524 +++++++++++++++++++++
t/t9801-remote-cvs-fetch.sh | 291 ++++++++++++
t/test-lib.sh | 1 +
17 files changed, 4116 insertions(+), 0 deletions(-)
create mode 100644 Documentation/git-remote-cvs.txt
create mode 100755 git-remote-cvs.py
create mode 100644 git_remote_cvs/.gitignore
create mode 100644 git_remote_cvs/Makefile
create mode 100644 git_remote_cvs/__init__.py
create mode 100644 git_remote_cvs/changeset.py
create mode 100644 git_remote_cvs/commit_states.py
create mode 100644 git_remote_cvs/cvs.py
create mode 100644 git_remote_cvs/cvs_revision_map.py
create mode 100644 git_remote_cvs/cvs_symbol_cache.py
create mode 100644 git_remote_cvs/git.py
create mode 100644 git_remote_cvs/setup.py
create mode 100644 git_remote_cvs/util.py
create mode 100755 t/t9800-remote-cvs-basic.sh
create mode 100755 t/t9801-remote-cvs-fetch.sh
^ permalink raw reply
* [RFCv3 1/4] Basic build infrastructure for Python scripts
From: Johan Herland @ 2009-08-12 0:13 UTC (permalink / raw)
To: git; +Cc: Johan Herland, barkalow, gitster, Johannes.Schindelin
In-Reply-To: <1250036031-32272-1-git-send-email-johan@herland.net>
This patch adds basic boilerplate support (based on corresponding Perl
sections) for enabling the building and installation Python scripts.
There are currently no Python scripts being built, and when Python
scripts are added in future patches, their building and installation
can be disabled by defining NO_PYTHON.
Signed-off-by: Johan Herland <johan@herland.net>
---
Makefile | 13 +++++++++++++
configure.ac | 3 +++
t/test-lib.sh | 1 +
3 files changed, 17 insertions(+), 0 deletions(-)
diff --git a/Makefile b/Makefile
index 189aee5..969cef5 100644
--- a/Makefile
+++ b/Makefile
@@ -166,6 +166,8 @@ all::
#
# Define NO_PERL if you do not want Perl scripts or libraries at all.
#
+# Define NO_PYTHON if you do not want Python scripts or libraries at all.
+#
# Define NO_TCLTK if you do not want Tcl/Tk GUI.
#
# The TCL_PATH variable governs the location of the Tcl interpreter
@@ -311,6 +313,7 @@ LIB_H =
LIB_OBJS =
PROGRAMS =
SCRIPT_PERL =
+SCRIPT_PYTHON =
SCRIPT_SH =
TEST_PROGRAMS =
@@ -349,6 +352,7 @@ SCRIPT_PERL += git-svn.perl
SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) \
$(patsubst %.perl,%,$(SCRIPT_PERL)) \
+ $(patsubst %.py,%,$(SCRIPT_PYTHON)) \
git-instaweb
# Empty...
@@ -404,8 +408,12 @@ endif
ifndef PERL_PATH
PERL_PATH = /usr/bin/perl
endif
+ifndef PYTHON_PATH
+ PYTHON_PATH = /usr/bin/python
+endif
export PERL_PATH
+export PYTHON_PATH
LIB_FILE=libgit.a
XDIFF_LIB=xdiff/lib.a
@@ -1259,6 +1267,10 @@ ifeq ($(PERL_PATH),)
NO_PERL=NoThanks
endif
+ifeq ($(PYTHON_PATH),)
+NO_PYTHON=NoThanks
+endif
+
QUIET_SUBDIR0 = +$(MAKE) -C # space to separate -C and subdir
QUIET_SUBDIR1 =
@@ -1306,6 +1318,7 @@ prefix_SQ = $(subst ','\'',$(prefix))
SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
+PYTHON_PATH_SQ = $(subst ','\'',$(PYTHON_PATH))
TCLTK_PATH_SQ = $(subst ','\'',$(TCLTK_PATH))
LIBS = $(GITLIBS) $(EXTLIBS)
diff --git a/configure.ac b/configure.ac
index 3f1922d..3749e5c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -241,6 +241,9 @@ GIT_ARG_SET_PATH(shell)
# Define PERL_PATH to provide path to Perl.
GIT_ARG_SET_PATH(perl)
#
+# Define PYTHON_PATH to provide path to Python.
+GIT_ARG_SET_PATH(python)
+#
# Define ZLIB_PATH to provide path to zlib.
GIT_ARG_SET_PATH(zlib)
#
diff --git a/t/test-lib.sh b/t/test-lib.sh
index a5b8d03..01ea386 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -714,6 +714,7 @@ case $(uname -s) in
esac
test -z "$NO_PERL" && test_set_prereq PERL
+test -z "$NO_PYTHON" && test_set_prereq PYTHON
# test whether the filesystem supports symbolic links
ln -s x y 2>/dev/null && test -h y 2>/dev/null && test_set_prereq SYMLINKS
--
1.6.4.rc3.138.ga6b98.dirty
^ permalink raw reply related
* [RFCv3 2/4] Add Python support library for CVS remote helper
From: Johan Herland @ 2009-08-12 0:13 UTC (permalink / raw)
To: git; +Cc: Johan Herland, barkalow, gitster, Johannes.Schindelin
In-Reply-To: <1250036031-32272-1-git-send-email-johan@herland.net>
This patch introduces a Python package called "git_remote_cvs" containing
the building blocks of the CVS remote helper. The CVS remote helper itself
is NOT part of this patch.
The patch includes the necessary Makefile additions to build and install
the git_remote_cvs Python package along with the rest of Git.
Signed-off-by: Johan Herland <johan@herland.net>
---
Makefile | 9 +
git_remote_cvs/.gitignore | 2 +
git_remote_cvs/Makefile | 27 ++
git_remote_cvs/changeset.py | 114 +++++
git_remote_cvs/commit_states.py | 52 +++
git_remote_cvs/cvs.py | 884 ++++++++++++++++++++++++++++++++++++
git_remote_cvs/cvs_revision_map.py | 362 +++++++++++++++
git_remote_cvs/cvs_symbol_cache.py | 283 ++++++++++++
git_remote_cvs/git.py | 586 ++++++++++++++++++++++++
git_remote_cvs/setup.py | 12 +
git_remote_cvs/util.py | 147 ++++++
11 files changed, 2478 insertions(+), 0 deletions(-)
create mode 100644 git_remote_cvs/.gitignore
create mode 100644 git_remote_cvs/Makefile
create mode 100644 git_remote_cvs/__init__.py
create mode 100644 git_remote_cvs/changeset.py
create mode 100644 git_remote_cvs/commit_states.py
create mode 100644 git_remote_cvs/cvs.py
create mode 100644 git_remote_cvs/cvs_revision_map.py
create mode 100644 git_remote_cvs/cvs_symbol_cache.py
create mode 100644 git_remote_cvs/git.py
create mode 100644 git_remote_cvs/setup.py
create mode 100644 git_remote_cvs/util.py
diff --git a/Makefile b/Makefile
index 969cef5..bb5cea2 100644
--- a/Makefile
+++ b/Makefile
@@ -1350,6 +1350,9 @@ endif
ifndef NO_PERL
$(QUIET_SUBDIR0)perl $(QUIET_SUBDIR1) PERL_PATH='$(PERL_PATH_SQ)' prefix='$(prefix_SQ)' all
endif
+ifndef NO_PYTHON
+ $(QUIET_SUBDIR0)git_remote_cvs $(QUIET_SUBDIR1) PYTHON_PATH='$(PYTHON_PATH_SQ)' prefix='$(prefix_SQ)' all
+endif
$(QUIET_SUBDIR0)templates $(QUIET_SUBDIR1)
please_set_SHELL_PATH_to_a_more_modern_shell:
@@ -1701,6 +1704,9 @@ install: all
ifndef NO_PERL
$(MAKE) -C perl prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install
endif
+ifndef NO_PYTHON
+ $(MAKE) -C git_remote_cvs prefix='$(prefix_SQ)' DESTDIR='$(DESTDIR_SQ)' install
+endif
ifndef NO_TCLTK
$(MAKE) -C gitk-git install
$(MAKE) -C git-gui gitexecdir='$(gitexec_instdir_SQ)' install
@@ -1821,6 +1827,9 @@ ifndef NO_PERL
$(RM) gitweb/gitweb.cgi
$(MAKE) -C perl clean
endif
+ifndef NO_PYTHON
+ $(MAKE) -C git_remote_cvs clean
+endif
$(MAKE) -C templates/ clean
$(MAKE) -C t/ clean
ifndef NO_TCLTK
diff --git a/git_remote_cvs/.gitignore b/git_remote_cvs/.gitignore
new file mode 100644
index 0000000..2247d5f
--- /dev/null
+++ b/git_remote_cvs/.gitignore
@@ -0,0 +1,2 @@
+/build
+/dist
diff --git a/git_remote_cvs/Makefile b/git_remote_cvs/Makefile
new file mode 100644
index 0000000..8dbf3fa
--- /dev/null
+++ b/git_remote_cvs/Makefile
@@ -0,0 +1,27 @@
+#
+# Makefile for the git_remote_cvs python support modules
+#
+pysetupfile:=setup.py
+
+ifndef PYTHON_PATH
+ PYTHON_PATH = /usr/bin/python
+endif
+ifndef prefix
+ prefix = $(HOME)
+endif
+ifndef V
+ QUIET = @
+ QUIETSETUP = --quiet
+endif
+
+PYLIBDIR=`$(PYTHON_PATH) -c "import sys; print 'lib/python%i.%i/site-packages' % sys.version_info[:2]"`
+
+all: $(pysetupfile)
+ $(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) build
+install: $(pysetupfile)
+ $(PYTHON_PATH) $(pysetupfile) install --prefix $(prefix)
+instlibdir: $(pysetupfile)
+ @echo "$(prefix)/$(PYLIBDIR)"
+clean:
+ $(QUIET)$(PYTHON_PATH) $(pysetupfile) $(QUIETSETUP) clean -a
+ $(RM) *.pyo *.pyc
diff --git a/git_remote_cvs/__init__.py b/git_remote_cvs/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/git_remote_cvs/changeset.py b/git_remote_cvs/changeset.py
new file mode 100644
index 0000000..27c4129
--- /dev/null
+++ b/git_remote_cvs/changeset.py
@@ -0,0 +1,114 @@
+#!/usr/bin/env python
+
+"""Functionality for collecting individual CVS revisions into "changesets"
+
+A changeset is a collection of CvsRev objects that belong together in the same
+"commit". This is a somewhat artificial construct on top of CVS, which only
+stores changes at the per-file level. Normally, CVS users create several CVS
+revisions simultaneously by applying the "cvs commit" command to several files
+with related changes. This module tries to reconstruct this notion of related
+revisions.
+"""
+
+from util import *
+
+
+class Changeset (object):
+ """Encapsulate a single changeset/commit"""
+
+ __slots__ = ('revs', 'date', 'author', 'message')
+
+ # The maximum time between the changeset's date, and the date of a rev
+ # to included in that changeset.
+ MaxSecondsBetweenRevs = 8 * 60 * 60 # 8 hours
+
+ @classmethod
+ def from_rev (cls, rev):
+ """Return a Changeset based on the given CvsRev object"""
+ c = cls(rev.date, rev.author, rev.message)
+ result = c.add(rev)
+ assert result
+ return c
+
+ def __init__ (self, date, author, message):
+ self.revs = {} # dict: path -> CvsRev object
+ self.date = date # CvsDate object
+ self.author = author
+ self.message = message # Lines of commit message
+
+ def __str__ (self):
+ msg = self.message[0] # First line only
+ if len(msg) > 25: msg = msg[:22] + "..." # Max 25 chars long
+ return "<Changeset @(%s) by %s (%s) updating %i files>" % (
+ self.date, self.author, msg, len(self.revs))
+
+ def __iter__ (self):
+ return self.revs.itervalues()
+
+ def __getitem__ (self, path):
+ return self.revs[path]
+
+ def within_time_window (self, rev):
+ """Return True iff the rev is within the time window of self"""
+ return abs(rev.date.diff(self.date)) <= \
+ self.MaxSecondsBetweenRevs
+
+ def add (self, rev):
+ """Add the given CvsRev to this Changeset
+
+ The addition will only succeed if the following holds:
+ - rev.author == self.author
+ - rev.message == self.message
+ - rev.path is not in self.revs
+ - rev.date is within MaxSecondsBetweenRevs of self.date
+ If the addition succeeds, True is returned; otherwise False.
+ """
+ if rev.author != self.author: return False
+ if rev.message != self.message: return False
+ if rev.path in self.revs: return False
+ if not self.within_time_window(rev): return False
+
+ self.revs[rev.path] = rev
+ return True
+
+
+def build_changesets_from_revs (cvs_revs):
+ """Organize CvsRev objects into a chronological list of Changesets"""
+
+ # Construct chronological list of CvsRev objects
+ chron_revs = []
+ for path, d in cvs_revs.iteritems():
+ i = 0 # Current index into chronRevs
+ for revnum, cvsrev in sorted(d.iteritems()):
+ while i < len(chron_revs) \
+ and cvsrev.date > chron_revs[i].date:
+ i += 1
+ # Insert cvsRev at position i in chronRevs
+ chron_revs.insert(i, cvsrev)
+ i += 1
+
+ changesets = [] # Chronological list of Changeset objects
+ while len(chron_revs): # There are still revs to be added to Changesets
+ # Create Changeset based on the first rev in chronRevs
+ changeset = Changeset.from_rev(chron_revs.pop(0))
+ # Keep adding revs chronologically until MaxSecondsBetweenRevs
+ rejects = [] # Revs that cannot be added to this changeset
+ while len(chron_revs):
+ rev = chron_revs.pop(0)
+ reject = False
+ # First, if we have one of rev's parents in rejects, we
+ # must also reject rev
+ for r in rejects:
+ if r.path == rev.path:
+ reject = True
+ break
+ # Next, add rev to changeset, reject if add fails
+ if not reject: reject = not changeset.add(rev)
+ if reject:
+ rejects.append(rev)
+ # stop trying when rev is too far in the future
+ if not changeset.within_time_window(rev): break
+ chron_revs = rejects + chron_revs # Reconstruct remaining revs
+ changesets.append(changeset)
+
+ return changesets
diff --git a/git_remote_cvs/commit_states.py b/git_remote_cvs/commit_states.py
new file mode 100644
index 0000000..f4b769b
--- /dev/null
+++ b/git_remote_cvs/commit_states.py
@@ -0,0 +1,52 @@
+#!/usr/bin/env python
+
+"""Functionality for relating Git commits to corresponding CvsState objects"""
+
+from util import *
+from cvs import CvsState
+
+
+class CommitStates (object):
+ """Provide a mapping from Git commits to CvsState objects
+
+ Behaves like a dictionary of Git commit names -> CvsState mappings.
+
+ Every Git commit converted from CVS has a corresponding CvsState, which
+ describes exactly which CVS revisions are present in a checkout of that
+ commit.
+
+ This class provides the interface to map from a Git commit to its
+ corresponding CvsState. The mapping uses GitNotes as a persistent
+ storage backend.
+ """
+
+ def __init__ (self, notes):
+ self.notes = notes
+ self._cache = {} # commitname -> CvsState
+
+ def _load (self, commit):
+ if commit is None: return None
+ if commit in self._cache: return self._cache[commit]
+ notedata = self.notes.get(commit)
+ if notedata is None: # Given commit has no associated note
+ return None
+ state = CvsState()
+ state.load_data(notedata)
+ self._cache[commit] = state
+ return state
+
+ def add (self, commit, state, gfi):
+ assert commit not in self._cache
+ self._cache[commit] = state
+ self.notes.import_note(commit, str(state), gfi)
+
+ def __getitem__ (self, commit):
+ state = self._load(commit)
+ if state is None:
+ raise KeyError, "Unknown commit '%s'" % (commit)
+ return state
+
+ def get (self, commit, default = None):
+ state = self._load(commit)
+ if state is None: return default
+ return state
diff --git a/git_remote_cvs/cvs.py b/git_remote_cvs/cvs.py
new file mode 100644
index 0000000..cc2e13f
--- /dev/null
+++ b/git_remote_cvs/cvs.py
@@ -0,0 +1,884 @@
+#!/usr/bin/env python
+
+"""Functionality for interacting with CVS repositories
+
+This module provides classes for interrogating a CVS repository via a CVS
+working directory (aka. checkout), or via direct queries using the "cvs rlog"
+command.
+
+Also, classes for encapsulating fundamental CVS concepts (like CVS numbers)
+are provided.
+"""
+
+import sys, time, shutil
+from calendar import timegm
+
+from util import *
+
+
+class CvsNum (object):
+ """Encapsulate a single CVS revision/branch number
+
+ Provides functionality for common operations on CVS numbers.
+
+ A CVS number consists of a list of components separated by dots ('.'),
+ where each component is a decimal number. Inspecting the components
+ from left to right, the odd-numbered (1st, 3rd, 5th, etc.) components
+ represent branches in the CVS history tree, while the even-numbered
+ (2nd, 4th, 6th, etc.) components represent revisions on the branch
+ specified in the previous position. Thus "1.2" denotes the second
+ revision on the first branch (aka. trunk), while "1.2.4.6" denotes the
+ sixth revision of the fourth branch started from revision "1.2".
+
+ Therefore, in general, a CVS number with an even number of components
+ denotes a revision (we call this a "revision number"), while an odd
+ number of components denotes a branch (called a "branch number").
+
+ There are a few complicating peculiarities: If there is an even number
+ of components, and the second-last component is 0, the number is not
+ a revision number, but is rather equivalent to the branch number we get
+ by removing the 0-component. I.e. "1.2.0.4" is equivalent to "1.2.4".
+
+ A branch number (except the trunk: "1") always has a "branch point"
+ revision, i.e. the revision from which the branch was started. This
+ revision is found by removing the last component of the branch number.
+ For example the branch point of "1.2.4" is "1.2".
+
+ Conversely, all revision numbers belong to a corresponding branch,
+ whose branch number is found by removing the last component. Examples:
+ The "1.2.4.6" revision belong to the "1.2.4" branch, while the "1.2"
+ revision belongs to the "1" branch (the "trunk").
+
+ From this we can programatically determine the ancestry of any revision
+ number, by decrementing the last revision component until it equals 1,
+ and then trim off the last two components to get to the branch point,
+ and repeat the process from there until we reach the initial revision
+ (typically "1.1"). For example, recursively enumerating the parent
+ revisions of "1.2.4.6" yields the following revisions:
+ "1.2.4.5", "1.2.4.4", "1.2.4.3", "1.2.4.2", "1.2.4.1", "1.2", "1.1"
+ """
+
+ __slots__ = ('l',)
+
+ @staticmethod
+ def decompose (cvsnum):
+ """Split the given CVS number into a list of integer components
+
+ Branch numbers are normalized to the odd-numbered components
+ form (i.e. removing the second last '0' component)
+
+ Examples:
+ '1.2.4.8' -> [1, 2, 4, 8]
+ '1.2.3' -> [1, 2, 3]
+ '1.2.0.5' -> [1, 2, 5]
+ """
+ if cvsnum: r = map(int, cvsnum.split('.'))
+ else: r = []
+ if len(r) >= 2 and r[-2] == 0: del r[-2]
+ return tuple(r)
+
+ @staticmethod
+ def compose (l):
+ """Join the given list of integer components into a CVS number
+
+ E.g.: (1, 2, 4, 8) -> '1.2.4.8'
+ """
+ return ".".join(map(str, l))
+
+ @classmethod
+ def from_components (cls, args):
+ return cls(cls.compose(args))
+
+ def __init__ (self, cvsnum):
+ self.l = self.decompose(str(cvsnum))
+
+ def __repr__ (self):
+ return self.compose(self.l)
+
+ def __str__ (self):
+ return repr(self)
+
+ def __hash__ (self):
+ return hash(repr(self))
+
+ def __len__ (self):
+ return len(self.l)
+
+ def __cmp__ (self, other):
+ try: return cmp(self.l, other.l)
+ except AttributeError: return 1
+
+ def __getitem__ (self, key):
+ return self.l[key]
+
+ def is_rev (self):
+ """Return True iff this number is a CVS revision number"""
+ return len(self.l) % 2 == 0 \
+ and len(self.l) >= 2 \
+ and self.l[-2] != 0
+
+ def is_branch (self):
+ """Return True iff this number is a CVS branch number"""
+ return len(self.l) % 2 != 0 \
+ or (len(self.l) >= 2 and self.l[-2] == 0)
+
+ def enumerate (self):
+ """Return a list of integer components in this CVS number"""
+ return list(self.l)
+
+ def branch (self):
+ """Return the branch on which the given number lives.
+
+ Revisions: chop the last component to find the branch, e.g.:
+ 1.2.4.6 -> 1.2.4
+ 1.1 -> 1
+ Branches: unchanged
+ """
+ if self.is_rev():
+ return self.from_components(self.l[:-1])
+ return self
+
+ def parent (self):
+ """Return the parent/previous revision number to this number.
+
+ For revisions, this is the previous revision, e.g.:
+ 1.2.4.6 -> 1.2.4.5
+ 1.2.4.1 -> 1.2
+ 1.1 -> None
+ 2.1 -> None
+ For branches, this is the branch point, e.g.:
+ 1.2.4 -> 1.2
+ 1 -> None
+ 2 -> None
+ """
+ if len(self.l) < 2: return None
+ elif len(self.l) % 2: # branch number
+ return self.from_components(self.l[:-1])
+ else: # revision number
+ assert self.l[-1] > 0
+ result = self.enumerate()
+ result[-1] -= 1 # Decrement final component
+ if result[-1] == 0: # We're at the start of the branch
+ del result[-2:] # Make into branch point
+ if not result: return None
+ return self.from_components(result)
+
+ def follows (self, other):
+ """Return True iff self historically follows the given rev
+
+ This iterates through the parents of self, and returns True iff
+ any of them equals the given rev. Otherwise, False is returned.
+ """
+ assert other.is_rev()
+ cur = self
+ while cur:
+ if cur == other: return True
+ cur = cur.parent()
+ return False
+
+ def on_branch (self, branch):
+ """Return True iff this rev is on the given branch.
+
+ The revs considered to be "on" a branch X also includes the
+ branch point of branch X.
+ """
+ return branch == self.branch() or branch.parent() == self
+
+ @classmethod
+ def disjoint (self, a, b):
+ """Return True iff the CVS numbers are historically disjoint
+
+ Two CVS numbers are disjoint if they do not share the same
+ historical line back to the initial revision. In other words:
+ the two numbers are disjoint if the history (i.e. set of parent
+ revisions all the way back to the intial (1.1) revision) of
+ neither number is a superset of the other's history.
+ See test_disjoint() for practical examples:
+ """
+ if a.is_branch(): a = self.from_components(a.l + (1,))
+ if b.is_branch(): b = self.from_components(b.l + (1,))
+ if len(a.l) > len(b.l): a, b = b, a # a is now shortest
+ pairs = zip(a.l, b.l)
+ for pa, pb in pairs[:-1]:
+ if pa != pb: return True
+ if len(a) == len(b): return False
+ common_len = len(a)
+ if a.l[common_len - 1] <= b.l[common_len - 1]: return False
+ return True
+
+ @classmethod
+ def test_disjoint (cls):
+ tests = [
+ ("1.2", "1.1", False),
+ ("1.2", "1.2", False),
+ ("1.2", "1.3", False),
+ ("1.2", "1.1.2", True),
+ ("1.2", "1.1.2.3", True),
+ ("1.2.4", "1.1", False),
+ ("1.2.4", "1.2", False),
+ ("1.2.4", "1.3", True),
+ ("1.2.4", "1.2.2", True),
+ ("1.2.4", "1.2.4", False),
+ ("1.2.4", "1.2.6", True),
+ ("1.2.4", "1.2.2.4", True),
+ ("1.2.4", "1.2.4.4", False),
+ ("1.2.4", "1.2.6.4", True),
+ ("1.2.4.6", "1.1", False),
+ ("1.2.4.6", "1.2", False),
+ ("1.2.4.6", "1.3", True),
+ ("1.2.4.6", "1.2.2", True),
+ ("1.2.4.6", "1.2.2.1", True),
+ ("1.2.4.6", "1.2.4", False),
+ ("1.2.4.6", "1.2.4.5", False),
+ ("1.2.4.6", "1.2.4.6", False),
+ ("1.2.4.6", "1.2.4.7", False),
+ ("1.2.4.6.8.10", "1.2.4.5", False),
+ ("1.2.4.6.8.10", "1.2.4.6", False),
+ ("1.2.4.6.8.10", "1.2.4.7", True),
+ ]
+ for a, b, result in tests:
+ a, b = map(cls, (a, b))
+ assert cls.disjoint(a, b) == result, \
+ "disjoint(%s, %s) is not %s" % (a, b, result)
+ assert cls.disjoint(b, a) == result, \
+ "disjoint(%s, %s) is not %s" % (a, b, result)
+
+ @classmethod
+ def test (cls):
+ assert cls("1.2.4") == cls("1.2.0.4")
+ assert cls("1.2.4").is_branch()
+ assert cls("1.2").is_rev()
+ assert cls("1").is_branch()
+ assert cls("1.2.4.6").is_rev()
+ assert cls("1.2.4.6").enumerate() == [1, 2, 4, 6]
+ assert cls.from_components([1, 2, 4, 6]) == cls("1.2.4.6")
+ assert str(cls.from_components([1, 2, 4, 6])) == "1.2.4.6"
+ assert len(cls("1.2.4.6")) == 4
+ assert cls("1.2.4.6").branch() == cls("1.2.4")
+ assert cls("1.2.4").branch() == cls("1.2.4")
+ assert cls("1.1").branch() == cls("1")
+ assert cls("1").branch() == cls("1")
+ assert cls("1.2.4.6").parent() == cls("1.2.4.5")
+ assert cls("1.2.4.1").parent() == cls("1.2")
+ assert cls("1.2").parent() == cls("1.1")
+ assert cls("1.1").parent() == None
+ assert cls("2.1").parent() == None
+ assert cls("1.2.4").parent() == cls("1.2")
+ assert cls("1").parent() == None
+ assert cls("2").parent() == None
+ assert cls("1.2.4.6").follows(cls("1.1"))
+ assert cls("1.2.4.6").follows(cls("1.2"))
+ assert cls("1.2.4.6").follows(cls("1.2.4.1"))
+ assert cls("1.2.4.6").follows(cls("1.2.4.2"))
+ assert cls("1.2.4.6").follows(cls("1.2.4.3"))
+ assert cls("1.2.4.6").follows(cls("1.2.4.4"))
+ assert cls("1.2.4.6").follows(cls("1.2.4.5"))
+ assert cls("1.2.4.6").follows(cls("1.2.4.6"))
+ assert not cls("1.2.4.6").follows(cls("1.2.4.7"))
+ assert not cls("1.2.4.6").follows(cls("1.3"))
+ assert not cls("1.1").follows(cls("1.2.4.6"))
+
+ cls.test_disjoint()
+
+
+class CvsState (object):
+ """Encapsulate a historical state in CVS (a set of paths and nums)
+
+ This class is a container of CVS pathnames and associated CvsNum
+ objects.
+
+ No communication with a CVS working directory or repository is done in
+ this class, hence only basic sanity checks are performed:
+ - A path may only appear once in a CvsState.
+ - When adding a path:num pair, path may not already exist in self
+ - When replacing a path:num pair, path must already exist in self
+ - When removing a path:num pair, both path and num must be given
+
+ IMPORTANT: Objects of this class are hash()able (to support being used
+ as keys in a dict), but they are also mutable. It is therefore up to
+ the caller to make sure that the object is not changed after being
+ stored in a data structure indexed by its hash value.
+ """
+
+ __slots__ = ('revs', '_hash')
+
+ def __init__ (self):
+ self.revs = {} # dict: path -> CvsNum object
+ self._hash = None
+
+ def __iter__ (self):
+ return self.revs.iteritems()
+
+ def __getitem__ (self, path):
+ return self.revs[path]
+
+ def __cmp__ (self, other):
+ return cmp(self.revs, other.revs)
+
+ def __str__ (self):
+ return "".join(["%s:%s\n" % (p, n) for p, n in sorted(self)])
+
+ def __hash__ (self):
+ if self._hash is None:
+ self._hash = hash(str(self))
+ return self._hash
+
+ def paths (self):
+ return self.revs.iterkeys()
+
+ def get (self, path, default = None):
+ return self.revs.get(path, default)
+
+ def add (self, path, revnum):
+ assert path not in self.revs
+ self._hash = None
+ self.revs[path] = revnum
+
+ def replace (self, path, revnum):
+ assert path in self.revs
+ self._hash = None
+ self.revs[path] = revnum
+
+ def remove (self, path, revnum):
+ assert path in self.revs and self.revs[path] == revnum
+ self._hash = None
+ del self.revs[path]
+
+ def copy (self):
+ """Create and return a copy of this object"""
+ ret = CvsState()
+ ret.revs = self.revs.copy()
+ ret._hash = self._hash
+ return ret
+
+ def load_data (self, note_data):
+ """Load note data as formatted by self.__str__()"""
+ for line in note_data.split("\n"):
+ line = line.strip()
+ if not line: continue
+ path, num = line.rsplit(':', 1)
+ self.add(path, CvsNum(num))
+ self._hash = hash(note_data)
+
+ def print_members (self, f = sys.stdout, prefix = ""):
+ for path, num in sorted(self):
+ print >>f, "%s%s:%s" % (prefix, path, num)
+
+ @file_reader_method(missing_ok = True)
+ def load (self, f):
+ """Load CVS state from the given file name/object"""
+ if not f: return
+ self.load_data(f.read())
+
+ @file_writer_method
+ def save (self, f):
+ """Save CVS state to the given file name/object"""
+ assert f
+ print >>f, str(self),
+
+
+class CvsDate (object):
+ """Encapsulate a timestamp, as reported by CVS
+
+ The internal representation of a timestamp is two integers, the first
+ representing the timestamp as #seconds since epoch (UTC), and the
+ second representing the timezone as #minutes offset from UTC.
+ E.g.: "2007-09-05 17:26:28 -0200" is converted to (1189013188, -120)
+ """
+
+ __slots__ = ('ts', 'tz')
+
+ def __init__ (self, date_str = None, in_utc = False):
+ """Convert CVS date string into a CvsDate object
+
+ A CVS timestamp string has one of the following forms:
+ - "YYYY-MM-DD hh:mm:ss SZZZZ"
+ - "YYYY/MM/DD hh:mm:ss" (with timezone assumed to be UTC)
+ The in_utc parameter determines whether the timestamp part of
+ the given string (the "YYYY-MM-DD hh:mm:ss" part) is given in
+ local time or UTC (normally CVS dates are given in local time.
+ If given in local time, the timezone offset is subtracted from
+ the timestamp in order to make the time in UTC format.
+ """
+ if date_str is None:
+ self.ts, self.tz = 0, 0
+ return
+ if date_str == "now":
+ self.ts, self.tz = time.time(), 0
+ return
+ date_str = date_str.strip()
+ # Set up self.ts and self.tz
+ if date_str.count(" ") == 2:
+ # Assume format "YYYY-MM-DD hh:mm:ss SZZZZ"
+ t, z = date_str.rsplit(" ", 1)
+ # Convert timestamp to #secs since epoch (UTC)
+ self.ts = timegm(time.strptime(t, "%Y-%m-%d %H:%M:%S"))
+ # Convert timezone into #mins offset from UTC
+ self.tz = int(z[1:3]) * 60 + int(z[3:5])
+ # Incorporate timezone sign
+ if z[0] == '-': self.tz *= -1
+ else:
+ assert date_str.count(" ") == 1
+ # Assume format "YYYY/MM/DD hh:mm:ss"
+ self.ts = timegm(time.strptime(
+ date_str, "%Y/%m/%d %H:%M:%S"))
+ self.tz = 0
+ # Adjust timestamp if not already in UTC
+ if not in_utc: self.ts -= self.tz * 60
+
+ def tz_str (self):
+ """Return timezone part of self in string format"""
+ sign = '+'
+ if self.tz < 0: sign = '-'
+ (hours, minutes) = divmod(abs(self.tz), 60)
+ return "%s%02d%02d" % (sign, hours, minutes)
+
+ def __str__ (self):
+ """Reconstruct date string from members"""
+ s = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(self.ts))
+ return "%s %s" % (s, self.tz_str())
+
+ def __repr__ (self):
+ return "CvsDate('%s')" % (str(self))
+
+ def __hash__ (self):
+ return hash((self.ts, self.tz))
+
+ def __nonzero__ (self):
+ return bool(self.ts or self.tz)
+
+ def __cmp__ (self, other):
+ return cmp(self.ts, other.ts) or cmp(self.tz, other.tz)
+
+ def __eq__ (self, other):
+ return self.ts == other.ts and self.tz == other.tz
+
+ def diff (self, other):
+ """Return difference between self and other in #seconds
+
+ Invariant: self == other.add(self.diff(other))
+ """
+ return self.ts - other.ts
+
+ @classmethod
+ def test (cls):
+ a = cls("2009-05-10 14:34:56 +0200")
+ b = cls("2009/05/10 12:34:56")
+ assert a
+ assert b
+ assert str(a) == "2009-05-10 12:34:56 +0200", str(a)
+ assert str(b) == "2009-05-10 12:34:56 +0000", str(b)
+ assert a != b
+ assert a.diff(b) == 0
+ c = cls("2009-05-10 16:34:56 +0200")
+ assert c
+ assert str(c) == "2009-05-10 14:34:56 +0200", str(c)
+ assert c != a
+ assert c.diff(a) == 2 * 60 * 60
+ assert a.diff(c) == -2 * 60 * 60
+
+
+class CvsRev (object):
+ """Encapsulate metadata on a CVS revision"""
+ __slots__ = ('path', 'num', 'date', 'author', 'deleted', 'message')
+
+ def __init__ (self, path, num):
+ self.path = path
+ self.num = num
+ self.date = None # CvsDate object
+ self.author = ""
+ self.deleted = None # True or False
+ self.message = [] # Lines of commit message
+
+ def __str__ (self):
+ return "<%s:%s on %s by %s%s>" % (
+ self.path, self.num, self.date, self.author,
+ self.deleted and ", deleted" or "")
+
+ def __cmp__ (self, other):
+ return cmp(self.path, other.path) or cmp(self.num, other.num)
+
+
+class CvsWorkDir (object):
+ """Encapsulate a CVS working directory
+
+ This class auto-creates a CVS workdir/checkout in the directory given
+ to the constructor, and provides various methods for interacting with
+ this workdir.
+ """
+
+ def __init__ (self, workdir, cvs_repo):
+ """Create a new CvsWorkDir
+
+ The cvs_repo argument must be a (cvs_root, cvs_module) tuple
+ """
+ self.d = workdir
+ self.cvs_root, self.cvs_module = cvs_repo
+ parent_dir = os.path.dirname(self.d)
+ if not os.path.isdir(parent_dir): os.makedirs(parent_dir)
+ self._valid = None
+
+ def makepath(self, *args):
+ """Create path relative to working directory"""
+ return os.path.join(self.d, *args)
+
+ def valid (self):
+ """Return True iff this workdir is present and valid"""
+ if self._valid is not None: return True
+ try:
+ f = open(self.makepath("CVS", "Root"), 'r')
+ assert f.read().strip() == self.cvs_root
+ f.close()
+ f = open(self.makepath("CVS", "Repository"), 'r')
+ assert f.read().strip() == self.cvs_module
+ f.close()
+ self._valid = True
+ except:
+ self._valid = False
+ return self._valid
+
+ def remove (self):
+ """Remove this checkout"""
+ shutil.rmtree(self.d, True)
+ assert not os.path.exists(self.d)
+ self._valid = False
+
+ def checkout (self, revision = "HEAD"):
+ """Create a checkout of the given revision"""
+ self.remove()
+ parent_dir, co_dir = os.path.split(self.d)
+ args = ["cvs", "-f", "-Q", "-d", self.cvs_root, "checkout"]
+ if str(revision) != "HEAD": args.extend(["-r", str(revision)])
+ args.extend(["-d", co_dir, self.cvs_module])
+ exit_code, output, errors = run_command(args, cwd = parent_dir)
+ if exit_code:
+ die("Failed to checkout CVS working directory")
+ assert not errors
+ assert not output, "output = '%s'" % (output)
+ self._valid = None
+ assert self.valid()
+
+ def update (self, revision = "HEAD", paths = []):
+ """Update the given paths to the given revision"""
+ if not self.valid(): self.checkout()
+ args = ["cvs", "-f", "-Q", "update", "-kk"]
+ if str(revision) == "HEAD": args.append("-A")
+ else: args.extend(["-r", str(revision)])
+ args.extend(paths)
+ exit_code, output, errors = run_command(args, cwd = self.d)
+ if exit_code:
+ die("Failed to checkout CVS working directory")
+ assert not errors
+ assert not output, "output = '%s'" % (output)
+
+ def get_revision_data (self, path, revision):
+ """Return the contents of the given CVS path:revision"""
+ if not self.valid(): self.checkout()
+ args = ["cvs", "-f", "-Q", "update", "-p", "-kk"]
+ if str(revision) == "HEAD": args.append("-A")
+ else: args.extend(["-r", str(revision)])
+ args.append(path)
+ exit_code, output, errors = run_command(args, cwd = self.d)
+ if exit_code:
+ die("Failed to checkout CVS working directory")
+ assert not errors
+ return output
+
+ def get_modeinfo (self, paths = []):
+ """Return mode information for the given paths.
+
+ Returns a dict of path -> mode number mappings. If paths are
+ not specified, mode information for all files in the current
+ checkout will be returned. No checkout/update will be done.
+ """
+ result = {}
+ if paths:
+ for path in paths:
+ fullpath = os.path.join(self.d, path)
+ mode = 644
+ if os.access(fullpath, os.X_OK):
+ mode = 755
+ assert path not in result
+ result[path] = mode
+ else: # Return mode information for all paths
+ for dirpath, dirnames, filenames in os.walk(self.d):
+ # Don't descend into CVS subdirs
+ try: dirnames.remove('CVS')
+ except ValueError: pass
+
+ assert dirpath.startswith(self.d)
+ directory = dirpath[len(self.d):].lstrip("/")
+
+ for fname in filenames:
+ path = os.path.join(directory, fname)
+ fullpath = os.path.join(dirpath, fname)
+ mode = 644
+ if os.access(fullpath, os.X_OK):
+ mode = 755
+ assert path not in result
+ result[path] = mode
+ return result
+
+ @classmethod
+ def parse_entries (cls, entries, prefix, directory = ""):
+ """Recursively parse CVS/Entries files
+
+ Return a dict of CVS paths found by parsing the CVS/Entries
+ files rooted at the given directory.
+
+ See http://ximbiot.com/cvs/manual/feature/cvs_2.html#SEC19 for
+ information on the format of the CVS/Entries file.
+ """
+ fname = os.path.join(prefix, directory, "CVS", "Entries")
+ subdirs = []
+ f = open(fname, 'r')
+ for line in f:
+ line = line.strip()
+ if line == "D": continue # There are no subdirectories
+ t, path, revnum, date, options, tag = line.split("/")
+ if t == "D":
+ subdirs.append(path)
+ continue
+ assert line.startswith("/")
+ path = os.path.join(directory, path)
+ revnum = CvsNum(revnum)
+ assert path not in entries
+ entries[path] = (revnum, date, options, tag)
+ f.close()
+
+ for d in subdirs:
+ d = os.path.join(directory, d)
+ cls.parse_entries(entries, prefix, d)
+
+ def get_state (self):
+ """Return CvsState reflecting current state of this checkout
+
+ Note that the resulting CvsState will never contain any
+ deleted/dead files. Other CvsStates to be compared to the one
+ returned from here should remove deleted/dead entries first.
+ """
+ assert self.valid()
+ entries = {}
+ result = CvsState()
+ self.parse_entries(entries, self.d)
+ for path, info in entries.iteritems():
+ result.add(path, info[0])
+ return result
+
+
+class CvsLogParser (object):
+ """Encapsulate the execution of a "cvs rlog" command"""
+
+ def __init__ (self, cvs_repo):
+ """Create a new CvsLogParser
+
+ The cvs_repo argument must be a (cvs_root, cvs_module) tuple
+ """
+ self.cvs_root, self.cvs_module = cvs_repo
+
+ def cleanup_path (self, cvs_path):
+ """Utility method for parsing CVS paths from CVS log"""
+ cvsprefix = "/".join((self.cvs_root[self.cvs_root.index("/"):],
+ self.cvs_module))
+ assert cvs_path.startswith(cvsprefix)
+ assert cvs_path.endswith(",v")
+ # Drop cvsprefix and ,v-extension
+ cvs_path = cvs_path[len(cvsprefix):-2]
+ # Split the remaining path into components
+ components = filter(None, cvs_path.strip().split('/'))
+ # Remove 'Attic' from CVS paths
+ if len(components) >= 2 and components[-2] == "Attic":
+ del components[-2]
+ # Reconstruct resulting "cleaned" path
+ return "/".join(components)
+
+ def __call__ (self, line):
+ """Line parser; this method is invoked for each line in the log
+
+ Must be reimplemented by subclass
+ """
+ pass
+
+ def finish (self):
+ """This method is invoked after the last line has been parsed
+
+ May be reimplemented by subclass
+ """
+ pass
+
+ def run (self, paths = [], no_symbols = False, revisions = None):
+ args = ["cvs", "-f", "-q", "-d", self.cvs_root, "rlog"]
+ if no_symbols: args.append("-N")
+ if revisions: args.append("-r%s" % (revisions))
+ if paths:
+ for p in paths:
+ args.append("%s/%s" % (self.cvs_module, p))
+ else:
+ args.append(self.cvs_module)
+
+ proc = start_command(args)
+ proc.stdin.close()
+ while True:
+ for line in proc.stdout:
+ self(line.rstrip()) # Call subclass line parser
+ if proc.poll() is not None:
+ break
+
+ assert proc.stdout.read() == ""
+ self.finish() # Notify subclass that parsing is finished
+ exit_code = proc.returncode
+ if exit_code:
+ error("'%s' returned exit code %i, and errors:\n" \
+ "---\n%s---", " ".join(args), exit_code,
+ proc.stderr.read())
+ return exit_code
+
+
+class CvsRevLister (CvsLogParser):
+ """Extract CvsRev objects (with revision metadata) from a CVS log"""
+
+ def __init__ (self, cvs_repo, show_progress = False):
+ super(CvsRevLister, self).__init__(cvs_repo)
+ self.cur_file = None # current CVS file being processed
+ self.cur_file_numrevs = 0 # #revs in current CVS file
+ self.cur_rev = None # Current CvsRev under construction
+
+ self.progress = None
+ if show_progress:
+ self.progress = ProgressIndicator("\t", sys.stderr)
+
+ # Store found revs in a two-level dict structure:
+ # filename -> revnum -> CvsRev
+ self.revs = {}
+
+ # Possible states:
+ # - BeforeRevs - waiting for "total revisions:"
+ # - BetweenRevs - waiting for "----------------------------"
+ # - ReadingRev - reading CVS revision details
+ self.state = 'BeforeRevs'
+
+ def __call__ (self, line):
+ if self.state == 'BeforeRevs':
+ if line.startswith("RCS file: "):
+ self.cur_file = self.cleanup_path(line[10:])
+ assert self.cur_file not in self.revs
+ self.revs[self.cur_file] = {}
+ elif line.startswith("total revisions: "):
+ assert self.cur_file
+ totalrevs, selectedrevs = line.split(";")
+ self.cur_file_numrevs = \
+ int(selectedrevs.split(":")[1].strip())
+ self.state = 'BetweenRevs'
+ elif self.state == 'BetweenRevs':
+ if line == "----------------------------" or \
+ line == "======================================" \
+ "=======================================":
+ if self.cur_rev:
+ # Finished current revision
+ f = self.revs[self.cur_file]
+ assert self.cur_rev.num not in f
+ f[self.cur_rev.num] = self.cur_rev
+ self.cur_rev = None
+ if self.progress: self.progress()
+ if line == "----------------------------":
+ self.state = 'ReadingRev'
+ else:
+ # Finalize current CVS file
+ assert len(self.revs[self.cur_file]) \
+ == self.cur_file_numrevs
+ self.cur_file = None
+ self.state = 'BeforeRevs'
+ elif self.cur_rev:
+ # Currently in the middle of a revision.
+
+ if line.startswith("branches: %s" \
+ % (self.cur_rev.num)) \
+ and line.endswith(";"):
+ return # Skip 'branches:' lines
+ # This line is part of the commit message.
+ self.cur_rev.message.append(line)
+ elif self.state == 'ReadingRev':
+ if line.startswith("revision "):
+ self.cur_rev = CvsRev(
+ self.cur_file, CvsNum(line.split()[1]))
+ else:
+ date, author, state, dummy = line.split(";", 3)
+ assert date.startswith("date: ")
+ self.cur_rev.date = CvsDate(date[6:])
+ assert author.strip().startswith("author: ")
+ self.cur_rev.author = author.strip()[8:]
+ assert state.strip().startswith("state: ")
+ state = state.strip()[7:]
+ self.cur_rev.deleted = state == "dead"
+ self.state = 'BetweenRevs'
+
+ def finish (self):
+ assert self.state == 'BeforeRevs'
+ if self.progress:
+ self.progress.finish("Parsed %i revs in %i files" % (
+ self.progress.n, len(self.revs)))
+
+
+def fetch_revs (path, from_rev, to_rev, symbol, cvs_repo):
+ """Fetch CvsRev objects for each rev in <path:from_rev, path:symbol]
+
+ Return a dict of CvsRev objects (revnum -> CvsRev), where each CvsRev
+ encapsulates a CVS revision in the range from path:from_rev to
+ path:symbol (inclusive). If symbol currently refers to from_rev (i.e.
+ nothing has happened since the last import), the returned dict will
+ have exactly one entry (from_rev). If there is no valid revision range
+ between from_rev and symbol, the returned dict will be empty.
+ Situations in which an empty dict is returned, include:
+ - symbol is no longer defined on this path
+ - symbol refers to a revision that is disjoint from from_rev
+
+ from_rev may be None, meaning that all revisions from the initial
+ version of path up to the revision currently referenced by symbol
+ should be fetched.
+
+ If the revision currently referenced by symbol is disjoint from
+ from_rev, the returned dict will be empty.
+
+ Note that there is lots of unexpected behaviour in the handling of the
+ 'cvs rlog -r' parameter: Say you have a branch, called 'my_branch',
+ that points to branch number 1.1.2 of a file. Say there are 3 revisions
+ on this branch: 1.1.2.1 -> 1.1.2.3. (in additions to the branch point
+ 1.1). Now, observe the following 'cvs rlog' executions:
+ - cvs rlog -r0:my_branch ... returns 1.1, 1.1.2.1, 1.1.2.2, 1.1.2.3
+ - cvs rlog -r1.1:my_branch ... returns the same revs
+ - cvs rlog -rmy_branch ... returns 1.1.2.1, 1.1.2.2, 1.1.2.3
+ - cvs rlog -rmy_branch: ... returns the same revs
+ - cvs rlog -r:my_branch ... returns the same revs
+ - cvs rlog -r::my_branch ... returns the same revs
+ - cvs rlog -r1.1.2.1: ... returns the same revs
+ Here is where it gets really weird:
+ - cvs rlog -r1.1.2.1:my_branch ... returns 1.1.2.1 only
+ - cvs rlog -r1.1.2.2:my_branch ... returns 1.1.2.1, 1.1.2.2
+ - cvs rlog -r1.1.2.3:my_branch ... returns 1.1.2.1, 1.1.2.2, 1.1.2.3
+
+ In other words the 'cvs rlog -rfrom_rev:symbol' scheme that we normally
+ use will not work in the case where from_rev is _on_ the branch pointed
+ at by the symbol.
+
+ Therefore, we need an extra parameter, to_rev, which we can use to:
+ 1. Detect when this situation is present.
+ 2. Work around by using 'cvs rlog -rfrom_ref:to_rev' instead.
+ """
+
+ if from_rev is None: # Initial import
+ from_rev = "0" # cvs rlog -r0:X fetches from initial revision
+ elif to_rev and to_rev.branch() == from_rev.branch():
+ symbol = to_rev # Use to_rev instead of given symbol
+
+ # Run 'cvs rlog' on range [from_rev, symbol] and parse CvsRev objects
+ parser = CvsRevLister(cvs_repo)
+ parser.run((path,), True, "%s:%s" % (from_rev, symbol))
+ assert len(parser.revs) == 1
+ assert path in parser.revs
+ return parser.revs[path]
+
+
+if __name__ == '__main__':
+ # Run selftests
+ CvsNum.test()
+ CvsDate.test()
diff --git a/git_remote_cvs/cvs_revision_map.py b/git_remote_cvs/cvs_revision_map.py
new file mode 100644
index 0000000..7d7810f
--- /dev/null
+++ b/git_remote_cvs/cvs_revision_map.py
@@ -0,0 +1,362 @@
+#!/usr/bin/env python
+
+"""Functionality for mapping CVS revisions to associated metainformation"""
+
+from util import *
+from cvs import CvsNum, CvsDate
+from git import GitFICommit, GitFastImport, GitObjectFetcher
+
+
+class CvsPathInfo (object):
+ """Information on a single CVS path"""
+ __slots__ = ('revs', 'mode')
+
+ def __init__ (self, mode = None):
+ self.revs = {}
+ self.mode = mode
+
+class CvsRevInfo (object):
+ """Information on a single CVS revision"""
+ __slots__ = ('blob', 'commits')
+
+ def __init__ (self, blob):
+ self.blob = blob
+ self.commits = []
+
+class CvsRevisionMap (object):
+ """Encapsulate the mapping of CVS revisions to associated metainfo
+
+ This container maps CVS revisions (a combination of a CVS path and a
+ CVS revision number) into Git blob names, Git commit names, and CVS
+ path information. Git object (blob/commit) names are either 40-char
+ SHA1 strings, or "git fast-import" mark numbers if the form ":<num>".
+
+ The data structure is organized as follows:
+ - A CvsRevisionMap instance knows about a set of CVS paths.
+ For each CVS path, the following is known:
+ - The mode (permission bits) of that CVS path (644 or 755)
+ - The CVS revision numbers that exist for that CVS path.
+ For each revision number, the following is known:
+ - Exactly 1 blob name; the Git blob containing the contents of the
+ revision (the contents of the CVS path at that CVS revision).
+ - One or more commit names; the Git commits which encapsulate a
+ CVS state in which this CVS revision
+
+ To store this data structure persistently, this class uses a Git ref
+ that points to a tree structure containing the above information. When
+ changes to the structure are made, this class will produce
+ git-fast-import commands to update that tree structure accordingly.
+
+ IMPORTANT: No changes to the CvsRevisionMap will be stored unless
+ .commit() is called with a valid GitFastImport instance
+
+ NOTES: Mark numbers are only transient references bound to the current
+ "git fast-import" process (assumed to be running alongside this
+ process). Therefore, when the "git fast-import" process ends, it must
+ write out a mark number -> SHA1 mapping (see the "--export-marks"
+ argument to "git fast-import"). Subsequently, this mapping must be
+ parsed, and the mark numbers in this CvsRevisionMap must be resolved
+ into persistent SHA1 names. In order to quickly find all the unresolved
+ mark number entries in the data structure, and index mapping mark
+ numbers to revisions is kept in a separate file in the tree structure.
+ See the loadMarksFile() method for more details.
+ """
+
+ MarkNumIndexFile = "CVS/marks" # invalid CVS path name
+
+ def __init__ (self, git_ref, obj_fetcher):
+ self.git_ref = git_ref
+ self.obj_fetcher = obj_fetcher
+
+ # The following data structure is a cache of the persistent
+ # data structure found at self.git_ref.
+ # It is structured as follows:
+ # - self.d is a dict, mapping CVS paths to CvsPathInfo objects.
+ # CvsPathInfo object have two fields: revs, mode:
+ # - mode is either 644 (non-executable) or 755 (executable).
+ # - revs is a dict, mapping CVS revision numbers (CvsNum
+ # instances) to CvsRevInfo objects. CvsRevInfo objects have
+ # two fields: blob, commits:
+ # - blob is the name of the Git blob object holding the
+ # contents of that revision.
+ # - commits is a collection of zero or more Git commit
+ # names where the commit contains this revision.
+ self.d = {} # dict: path -> CvsPathInfo
+ self.mods = set() # (path, revnum) pairs for all modified revs
+ self.marknum_idx = {} # dict: mark num -> [(path, revnum), ...]
+ self._load_marknum_idx()
+
+ def __del__ (self):
+ if self.mods:
+ error("Missing call to self.commit().")
+ error("%i revision changes are lost!", len(self.mods))
+
+ def __nonzero__ (self):
+ """Return True iff any information is currently stored here"""
+ return bool(self.d)
+
+ # Private methods:
+
+ def _add_to_marknum_idx(self, marknum, path, revnum):
+ """Add the given marknum -> (path, revnum) association"""
+ entry = self.marknum_idx.setdefault(marknum, [])
+ entry.append((path, revnum))
+
+ def _load_marknum_idx(self):
+ """Load contents of MarkNumIndexFile into self.marknum_idx"""
+ blobspec = "%s:%s" % (self.git_ref, self.MarkNumIndexFile)
+ try: f = self.obj_fetcher.open_obj(blobspec)
+ except KeyError: return # missing object; nothing to do
+
+ for line in f:
+ # Format of line is "<marknum> <path>:<revnum>"
+ mark, rest = line.strip().split(' ', 1)
+ path, rev = rest.rsplit(':', 1)
+ self._add_to_marknum_idx(int(mark), path, CvsNum(rev))
+ f.close()
+
+ def _save_marknum_idx(self):
+ """Prepare data for storage into MarkNumIndexFile
+
+ The returned string contains the current contents of
+ self.marknum_idx, formatted to be stored verbatim in
+ self.MarkNumIndexFile.
+ """
+ lines = []
+ for marknum, revs in sorted(self.marknum_idx.iteritems()):
+ for path, revnum in revs:
+ # Format of line is "<marknum> <path>:<revnum>"
+ line = "%i %s:%s\n" % (marknum, path, revnum)
+ lines.append(line)
+ return "".join(lines)
+
+ def _save_rev(self, path, revnum):
+ """Return blob data for storing the given revision persistently
+
+ Generate the blob contents that will reconstitute the same
+ revision entry when read back in with _fetch_path().
+ """
+ lines = []
+ rev_info = self.d[path].revs[revnum]
+ lines.append("blob %s\n" % (rev_info.blob))
+ for commit in rev_info.commits:
+ lines.append("commit %s\n" % (commit))
+ return "".join(lines)
+
+ @staticmethod
+ def _valid_objname (objname):
+ """Return the argument as a SHA1 (string) or mark num (int)"""
+ # Blob names are either 40-char SHA1 strings, or mark numbers
+ if isinstance(objname, int) or len(objname) != 40: # mark num
+ return int(objname)
+ return objname
+
+ def _load (self, path, mode, data):
+ """GitObjectFetcher.walk_tree() callback"""
+ assert mode in (644, 755)
+ cvsPath, revnum = os.path.split(path)
+ revnum = CvsNum(revnum)
+ if cvsPath in self.d:
+ assert mode == self.d[cvsPath].mode
+ else:
+ self.d[cvsPath] = CvsPathInfo(mode)
+ assert revnum not in self.d[cvsPath].revs
+ rev_info = None
+ for line in data.split("\n"):
+ if not line: continue
+ t, objname = line.split()
+ objname = self._valid_objname(objname)
+ if t == "blob":
+ assert rev_info is None
+ rev_info = CvsRevInfo(objname)
+ elif t == "commit":
+ assert rev_info is not None
+ rev_info.commits.append(objname)
+ else:
+ assert False, "Unknown type '%s'" % (t)
+ assert rev_info.commits # each rev is in at least one commit
+ self.d[cvsPath].revs[revnum] = rev_info
+
+ def _fetch_path (self, path):
+ """If the given path exists, create a path entry in self.d"""
+ tree_spec = "%s:%s" % (self.git_ref, path)
+ self.obj_fetcher.walk_tree(tree_spec, self._load, path) # *** Don't load entire tree at once?
+
+ # Public methods:
+
+ def has_path (self, path):
+ """Return True iff the given path is present"""
+ if path not in self.d: self._fetch_path(path)
+ return path in self.d
+
+ def has_rev (self, path, revnum):
+ """Return True iff the given path:revnum is present"""
+ if path not in self.d: self._fetch_path(path)
+ return path in self.d and revnum in self.d[path].revs
+
+ def get_mode (self, path):
+ """Return mode bits for the given path"""
+ if path not in self.d: self._fetch_path(path)
+ return self.d[path].mode
+
+ def get_blob (self, path, revnum):
+ """Return the blob name for the given revision"""
+ if path not in self.d: self._fetch_path(path)
+ return self.d[path].revs[revnum].blob
+
+ def get_commits (self, path, revnum):
+ """Return the commit names containing the given revision"""
+ if path not in self.d: self._fetch_path(path)
+ return self.d[path].revs[revnum].commits
+
+ def has_unresolved_marks (self):
+ """Return True iff there are mark nums in the data structure"""
+ return self.marknum_idx
+
+ # Public methods that change/add information:
+
+ def add_path (self, path, mode):
+ """Add the given path and associated mode bits to this map"""
+ if path not in self.d: self._fetch_path(path)
+ if path in self.d:
+ if self.d[path].mode:
+ assert mode == self.d[path].mode, \
+ "The mode of %s has changed from %s " \
+ "to %s since the last import. This " \
+ "is not supported." % (
+ path, self.d[path].mode, mode)
+ else:
+ self.d[path].mode = mode
+ else:
+ self.d[path] = CvsPathInfo(mode)
+ # Do not add to self.mods yet, as we expect revisions to be
+ # added before commit() is called
+
+ def add_blob (self, path, revnum, blobname):
+ """Add the given path:revnum -> blobname association"""
+ assert blobname
+ if path not in self.d: self._fetch_path(path)
+ blobname = self._valid_objname(blobname)
+ if isinstance(blobname, int): # mark num
+ self._add_to_marknum_idx(blobname, path, revnum)
+ entry = self.d.setdefault(path, CvsPathInfo())
+ assert revnum not in entry.revs
+ entry.revs[revnum] = CvsRevInfo(blobname)
+ self.mods.add((path, revnum))
+
+ def add_commit (self, path, revnum, commitname):
+ """Add the given path:revnum -> commitname association"""
+ if path not in self.d: self._fetch_path(path)
+ commitname = self._valid_objname(commitname)
+ if isinstance(commitname, int): # mark num
+ self._add_to_marknum_idx(commitname, path, revnum)
+ assert revnum in self.d[path].revs
+ assert commitname not in self.d[path].revs[revnum].commits
+ self.d[path].revs[revnum].commits.append(commitname)
+ self.mods.add((path, revnum))
+
+ @file_reader_method(missing_ok = True)
+ def load_marks_file (self, f):
+ """Load mark -> SHA1 mappings from git-fast-import marks file
+
+ Replace all mark numbers with proper SHA1 names in this data
+ structure (using self.marknum_idx to find them quickly).
+ """
+ if not f: return 0
+ marks = {}
+ last_mark = 0
+ for line in f:
+ (mark, sha1) = line.strip().split()
+ assert mark.startswith(":")
+ mark = int(mark[1:])
+ assert mark not in marks
+ marks[mark] = sha1
+ if mark > last_mark: last_mark = mark
+ for marknum, revs in self.marknum_idx.iteritems():
+ sha1 = marks[marknum]
+ for path, revnum in revs:
+ if path not in self.d: self._fetch_path(path)
+ rev_info = self.d[path].revs[revnum]
+ if rev_info.blob == marknum: # replace blobname
+ rev_info.blob = sha1
+ else: # replace commitname
+ assert marknum in rev_info.commits
+ i = rev_info.commits.index(marknum)
+ rev_info.commits[i] = sha1
+ assert marknum not in rev_info.commits
+ self.mods.add((path, revnum))
+ self.marknum_idx = {} # resolved all transient mark numbers
+ return last_mark
+
+ def sync_modeinfo_from_cvs (self, cvs):
+ """Update with mode information from current CVS checkout
+
+ This method will retrieve mode information on all paths in the
+ current CVS checkout, and update this data structure
+ correspondingly. In the case where mode information is already
+ present for a given CVS path, this method will verify that such
+ information is correct.
+ """
+ for path, mode in cvs.get_modeinfo().iteritems():
+ self.add_path(path, mode)
+
+ def commit_map (self, gfi, author, message):
+ """Produce git-fast-import commands for storing changes
+
+ The given GitFastImport object is used to produce a commit
+ making the changes done to this data structure persistent.
+ """
+ now = CvsDate("now")
+ commitdata = GitFICommit(
+ author[0], author[1], now.ts, now.tz_str(), message)
+
+ # Add updated MarkNumIndexFile to commit
+ mark = gfi.blob(self._save_marknum_idx())
+ commitdata.modify(644, mark, self.MarkNumIndexFile)
+
+ for path, revnum in self.mods:
+ mark = gfi.blob(self._save_rev(path, revnum))
+ mode = self.d[path].mode
+ assert mode in (644, 755)
+ commitdata.modify(mode, mark, "%s/%s" % (path, revnum))
+
+ gfi.commit(self.git_ref, commitdata)
+ self.mods = set()
+
+
+class CvsStateMap (object):
+ """Map CvsState object to the commit names which produces that state"""
+ def __init__ (self, cvs_rev_map):
+ self.cvs_rev_map = cvs_rev_map
+
+ def get_commits (self, state):
+ """Map the given CvsState to commits that contain this state
+
+ Return all commits where the given state is a subset of the
+ state produced by that commit.
+
+ Returns a set of commit names. The set may be empty.
+ """
+ candidate_commits = None
+ for path, revnum in state:
+ commits = self.cvs_rev_map.get_commits(path, revnum)
+ if candidate_commits is None:
+ candidate_commits = set(commits)
+ else:
+ candidate_commits.intersection_update(commits)
+ return candidate_commits
+
+ def get_exact_commit (self, state, commit_map):
+ """Map the given CvsState to the commit with this exact state
+
+ The given commit_map must be a CommitStates object.
+
+ Return the only commit (if any) that produces the exact given
+ CvsState.
+
+ Returns a commit name, or None if no matching commit is found.
+ """
+ commits = self.get_commits(state)
+ for c in commits:
+ if state == commit_map.get(c): return c
+ return None
diff --git a/git_remote_cvs/cvs_symbol_cache.py b/git_remote_cvs/cvs_symbol_cache.py
new file mode 100644
index 0000000..a8ec18a
--- /dev/null
+++ b/git_remote_cvs/cvs_symbol_cache.py
@@ -0,0 +1,283 @@
+#!/usr/bin/env python
+
+"""Functionality for interacting with the local CVS symbol cache"""
+
+import os
+
+from util import *
+from cvs import *
+
+
+class CvsSymbolStateLister (CvsLogParser):
+ """Extract current CvsStates for all CVS symbols from a CVS log"""
+
+ def __init__ (self, cvs_repo, show_progress = False):
+ super(CvsSymbolStateLister, self).__init__(cvs_repo)
+ self.symbols = {} # CVS symbol name -> CvsState object
+
+ self.cur_file = None # current CVS file being processed
+ self.cur_file_numrevs = 0 # #revs in current CVS file
+ self.cur_revnum = None # current revision number
+ self.rev2syms = {} # CvsNum -> [CVS symbol names]
+ self.cur_revs = {} # CvsNum -> True/False (deleted)
+ self.head_num = None # CvsNum of the HEAD rev or branch
+
+ # Possible states:
+ # - BeforeSymbols - waiting for "symbolic names:"
+ # - WithinSymbols - reading CVS symbol names
+ # - BeforeRevs - waiting for "total revisions:"
+ # - BetweenRevs - waiting for "----------------------------"
+ # - ReadingRev - reading CVS revision details
+ self.state = 'BeforeSymbols'
+
+ self.progress = None
+ if show_progress:
+ self.progress = ProgressIndicator("\t", sys.stderr)
+
+ def finalize_symbol_states (self):
+ """Adjust CvsStates in self.symbols based on revision data
+
+ Based on the information found in self.rev2syms and
+ self.cur_revs, remove deleted revisions and turn branch numbers
+ into corresponding revisions in the CvsStates found in
+ self.symbols.
+ """
+ # Create a mapping from branch numbers to the last existing
+ # revision number on those branches
+ branch2lastrev = {} # branch number -> revision number
+ for revnum in self.cur_revs.iterkeys():
+ branchnum = revnum.branch()
+ if (branchnum not in branch2lastrev) or \
+ (revnum > branch2lastrev[branchnum]):
+ branch2lastrev[branchnum] = revnum
+
+ for cvsnum, symbols in self.rev2syms.iteritems():
+ if cvsnum.is_branch():
+ # Turn into corresponding revision number
+ revnum = branch2lastrev.get(
+ cvsnum, cvsnum.parent())
+ for s in symbols:
+ state = self.symbols[s]
+ assert state[self.cur_file] == cvsnum
+ state.replace(self.cur_file, revnum)
+ cvsnum = revnum
+ assert cvsnum.is_rev()
+ assert cvsnum in self.cur_revs
+ if self.cur_revs[cvsnum]: # cvsnum is a deleted rev
+ # Remove from CvsStates
+ for s in symbols:
+ state = self.symbols[s]
+ state.remove(self.cur_file, cvsnum)
+
+ self.rev2syms = {}
+ self.cur_revs= {}
+ self.cur_file = None
+
+ def __call__ (self, line):
+ if self.state == 'BeforeSymbols':
+ if line.startswith("RCS file: "):
+ self.cur_file = self.cleanup_path(line[10:])
+ if self.progress:
+ self.progress("%5i symbols found - " \
+ "Parsing CVS file #%i: %s " \
+ % (
+ len(self.symbols),
+ self.progress.n,
+ self.cur_file,
+ ))
+ if line.startswith("head: "):
+ self.head_num = CvsNum(line[6:])
+ if line.startswith("branch: "):
+ self.head_num = CvsNum(line[8:])
+ elif line == "symbolic names:":
+ assert self.head_num
+ s = self.symbols.setdefault("HEAD", CvsState())
+ s.add(self.cur_file, self.head_num)
+ r = self.rev2syms.setdefault(self.head_num, [])
+ r.append("HEAD")
+ self.head_num = None
+ self.state = 'WithinSymbols'
+ elif self.state == 'WithinSymbols':
+ if line.startswith("\t"):
+ symbol, cvsnum = line.split(":", 1)
+ symbol = symbol.strip()
+ cvsnum = CvsNum(cvsnum)
+ s = self.symbols.setdefault(symbol, CvsState())
+ s.add(self.cur_file, cvsnum)
+ r = self.rev2syms.setdefault(cvsnum, [])
+ r.append(symbol)
+ else:
+ self.state = 'BeforeRevs'
+ elif self.state == 'BeforeRevs':
+ if line.startswith("total revisions: "):
+ assert self.cur_file
+ totalrevs, selectedrevs = line.split(";")
+ self.cur_file_numrevs = \
+ int(selectedrevs.split(":")[1].strip())
+ self.state = 'BetweenRevs'
+ elif self.state == 'BetweenRevs':
+ if line == "----------------------------" or \
+ line == "======================================" \
+ "=======================================":
+ if self.cur_revnum:
+ assert self.cur_revnum in self.cur_revs
+ self.cur_revnum = None
+ if line == "----------------------------":
+ self.state = 'ReadingRev'
+ else:
+ # Finalize current CVS file
+ assert len(self.cur_revs) \
+ == self.cur_file_numrevs
+ self.finalize_symbol_states()
+ self.state = 'BeforeSymbols'
+ elif self.state == 'ReadingRev':
+ if line.startswith("revision "):
+ self.cur_revnum = CvsNum(line.split()[1])
+ else:
+ date, author, state, dummy = line.split(";", 3)
+ assert date.startswith("date: ")
+ assert author.strip().startswith("author: ")
+ assert state.strip().startswith("state: ")
+ state = state.strip()[7:]
+ assert self.cur_revnum not in self.cur_revs
+ deleted = state == "dead"
+ self.cur_revs[self.cur_revnum] = deleted
+ self.state = 'BetweenRevs'
+
+ def finish (self):
+ assert self.state == 'BeforeSymbols'
+ if self.progress:
+ self.progress.finish("Parsed %i symbols in %i files" \
+ % (len(self.symbols), self.progress.n))
+
+
+class CvsSymbolCache (object):
+ """Local cache of the current CvsState of CVS symbols
+
+ Behaves like a dictionary of CVS symbol -> CvsState mappings.
+ """
+
+ def __init__ (self, symbols_dir):
+ self.symbols_dir = symbols_dir
+ if not os.path.isdir(self.symbols_dir):
+ os.makedirs(self.symbols_dir)
+
+ def __len__ (self):
+ return len(os.listdir(self.symbols_dir))
+
+ def __iter__ (self):
+ for filename in os.listdir(self.symbols_dir):
+ yield filename
+
+ def items (self):
+ for filename in self:
+ yield (filename, self[filename])
+
+ def __contains__ (self, symbol):
+ """Return True if the given symbol is present in this cache"""
+ return os.access(os.path.join(self.symbols_dir, symbol),
+ os.F_OK | os.R_OK)
+
+ def __getitem__ (self, symbol):
+ """Return the cached CvsState of the given CVS symbol"""
+ try:
+ f = open(os.path.join(self.symbols_dir, symbol), 'r')
+ except IOError:
+ raise KeyError, "'%s'" % (symbol)
+ ret = CvsState()
+ ret.load(f)
+ f.close()
+ return ret
+
+ def __setitem__ (self, symbol, cvs_state):
+ """Store the given CVS symbol and CvsState into the cache"""
+ cvs_state.save(os.path.join(self.symbols_dir, symbol))
+
+ def __delitem__ (self, symbol):
+ """Remove the the given CVS symbol from the cache"""
+ os.remove(os.path.join(self.symbols_dir, symbol))
+
+ def get (self, symbol, default = None):
+ try:
+ return self[symbol]
+ except KeyError:
+ return default
+
+ def clear (self):
+ """Remove all entries from the symbol cache"""
+ for filename in os.listdir(self.symbols_dir):
+ os.remove(os.path.join(self.symbols_dir, filename))
+
+ def sync_symbol (self, symbol, cvs, progress):
+ """Synchronize the given CVS symbol with the CVS server
+
+ The given CVS workdir is used for the synchronization.
+ The retrieved CvsState is also returned
+ """
+ progress("Retrieving state of CVS symbol '%s'..." % (symbol))
+ cvs.update(symbol)
+ state = cvs.get_state()
+
+ progress("Saving state of '%s' to symbol cache..." % (symbol))
+ self[symbol] = state
+
+ def sync_all_symbols (self, cvs_repo, progress, symbol_filter = None):
+ """Synchronize this symbol cache with the CVS server
+
+ This may be very expensive if the CVS repository is large, or
+ has many symbols. After this method returns, the symbol cache
+ will be in sync with the current state on the server.
+
+ This method returns a dict with the keys 'unchanged',
+ 'changed', 'added', and 'deleted', where each map to a list of
+ CVS symbols. Each CVS symbol appears in exactly one of these
+ lists.
+
+ If symbol_filter is given, it specifies functions that takes
+ one parameter - a CVS symbol name - and returns True if that
+ symbol should be synchronized, and False if that symbol should
+ be skipped. Otherwise all CVS symbols are synchronized.
+ """
+ if symbol_filter is None: symbol_filter = lambda symbol: True
+
+ # 1. Run cvs rlog to fetch current CvsState for all CVS symbols
+ progress("Retrieving current state of all CVS symbols from " \
+ "CVS server...", lf = True)
+ parser = CvsSymbolStateLister(cvs_repo, True)
+ retcode = parser.run()
+ if retcode:
+ raise EnvironmentError, (retcode,
+ "cvs rlog returned exit code %i" % (retcode))
+
+ # 2. Update symbol cache with new states from the CVS server
+ progress("Updating symbol cache with current CVS state...")
+ results = {}
+ result_keys = ("unchanged", "changed", "added", "deleted")
+ for k in result_keys: results[k] = []
+ # Classify existing symbols as unchanged, changed, or deleted
+ for symbol in filter(symbol_filter, self):
+ if symbol not in parser.symbols: # deleted
+ results["deleted"].append(symbol)
+ del self[symbol]
+ elif self[symbol] != parser.symbols[symbol]: # changed
+ results["changed"].append(symbol)
+ self[symbol] = parser.symbols[symbol]
+ else: # unchanged
+ results["unchanged"].append(symbol)
+ progress()
+ # Add symbols that are not in self
+ for symbol, state in parser.symbols.iteritems():
+ if not symbol_filter(symbol):
+ debug("Skipping CVS symbol '%s'...", symbol)
+ elif symbol in self:
+ assert state == self[symbol]
+ else: # added
+ results["added"].append(symbol)
+ self[symbol] = state
+ progress()
+
+ progress("Synchronized local symbol cache (%s)" % (
+ ", ".join(["%i %s" % (len(results[k]), k) \
+ for k in result_keys])), True)
+
+ return results
diff --git a/git_remote_cvs/git.py b/git_remote_cvs/git.py
new file mode 100644
index 0000000..0696962
--- /dev/null
+++ b/git_remote_cvs/git.py
@@ -0,0 +1,586 @@
+#!/usr/bin/env python
+
+"""Functionality for interacting with Git repositories
+
+This module provides classes for interfacing with a Git repository.
+"""
+
+import os, re
+from binascii import hexlify
+from cStringIO import StringIO
+
+from util import *
+
+def get_git_dir ():
+ """Return the path to the GIT_DIR for this repo"""
+ args = ("git", "rev-parse", "--git-dir")
+ exit_code, output, errors = run_command(args)
+ if exit_code:
+ die("Failed to retrieve git dir")
+ assert not errors
+ return output.strip()
+
+def parse_git_config ():
+ """Return a dict containing the parsed version of 'git config -l'"""
+ exit_code, output, errors = run_command(("git", "config", "-z", "-l"))
+ if exit_code:
+ die("Failed to retrieve git configuration")
+ assert not errors
+ return dict([e.split('\n', 1) for e in output.split("\0") if e])
+
+def git_config_bool (value):
+ """Convert the given git config string value to either True or False
+
+ Raise ValueError if the given string was not recognized as a boolean
+ value.
+ """
+ norm_value = str(value).strip().lower()
+ if norm_value in ("true", "1", "yes", "on", ""): return True
+ if norm_value in ("false", "0", "no", "off", "none"): return False
+ raise ValueError, "Failed to parse '%s' into a boolean value" % (value)
+
+def valid_git_ref (ref_name):
+ """Return True iff the given ref name is a valid git ref name"""
+ # The following is a reimplementation of the git check-ref-format
+ # command. The rules were derived from the git check-ref-format(1)
+ # manual page. This code should be replaced by a call to
+ # check_ref_format() in the git library, when such is available.
+ if ref_name.endswith('/') or \
+ ref_name.startswith('.') or \
+ ref_name.count('/.') or \
+ ref_name.count('..') or \
+ ref_name.endswith('.lock'):
+ return False
+ for c in ref_name:
+ if ord(c) < 0x20 or ord(c) == 0x7f or c in " ~^:?*[":
+ return False
+ return True
+
+
+class GitObjectFetcher (object):
+ """Provide parsed access to 'git cat-file --batch'"""
+ def __init__ (self):
+ """Initiate a 'git cat-file --batch' session"""
+ self.queue = [] # list of object names to be submitted
+ self.in_transit = None # object name currently in transit
+
+ # 'git cat-file --batch' produces binary output which is likely
+ # to be corrupted by the default "rU"-mode pipe opened by
+ # start_command. (Mode == "rU" does universal new-line
+ # conversion, which mangles carriage returns.) Therefore, we
+ # open an explicitly binary-safe pipe for transferring the
+ # output from 'git cat-file --batch'.
+ pipe_r_fd, pipe_w_fd = os.pipe()
+ pipe_r = os.fdopen(pipe_r_fd, "rb")
+ pipe_w = os.fdopen(pipe_w_fd, "wb")
+ self.proc = start_command(
+ ("git", "cat-file", "--batch"), stdout = pipe_w)
+ self.f = pipe_r
+
+ def __del__ (self):
+ assert not self.queue
+ assert self.in_transit is None
+ self.proc.stdin.close()
+ assert self.proc.wait() == 0 # zero exit code
+ assert self.f.read() == "" # no remaining output
+
+ def _submit_next_object (self):
+ """Submit queue items to the 'git cat-file --batch' process
+
+ If there are items in the queue, and there is currently no item
+ currently in 'transit', then pop the first item off the queue,
+ and submit it.
+ """
+ if self.queue and self.in_transit is None:
+ self.in_transit = self.queue.pop(0)
+ print >>self.proc.stdin, self.in_transit[0]
+
+ def push (self, obj, callback):
+ """Push the given object name onto the queue
+
+ The given callback function will at some point in the future
+ be called exactly once with the following arguments:
+ - self (this GitObjectFetcher instance)
+ - obj (the object name provided to push())
+ - sha1 (the SHA1 of the object, if 'None' obj is missing)
+ - t (the type of the object (tag/commit/tree/blob))
+ - size (the size of the object in bytes)
+ - data (the object contents)
+ """
+ self.queue.append((obj, callback))
+ self._submit_next_object() # (re)start queue processing
+
+ def process_next_entry (self):
+ """Read the next entry off the queue and invoke callback"""
+ obj, cb = self.in_transit
+ self.in_transit = None
+ header = self.f.readline()
+ if header == "%s missing\n" % (obj):
+ cb(self, obj, None, None, None, None)
+ return
+ sha1, t, size = header.split(" ")
+ assert len(sha1) == 40
+ assert t in ("tag", "commit", "tree", "blob")
+ assert size.endswith("\n")
+ size = int(size.strip())
+ data = self.f.read(size)
+ assert self.f.read(1) == "\n"
+ cb(self, obj, sha1, t, size, data)
+ self._submit_next_object()
+
+ def process (self):
+ """Process the current queue until empty"""
+ while self.in_transit is not None:
+ self.process_next_entry()
+
+ # High-level convenience methods:
+
+ def get_sha1 (self, objspec):
+ """Return the SHA1 of the object specified by 'objspec'
+
+ Return None if 'objspec' does not specify an existing object.
+ """
+ class ObjHandler (object):
+ def __init__ (self, parser):
+ self.parser = parser
+ self.sha1 = None
+
+ def __call__ (self, parser, obj, sha1, t, size, data):
+ assert parser == self.parser
+ self.sha1 = sha1
+
+ handler = ObjHandler(self)
+ self.push(objspec, handler)
+ self.process()
+ return handler.sha1
+
+ def open_obj (self, objspec):
+ """Return a file object wrapping the contents of a named object
+
+ The caller is responsible for calling .close() on the returned
+ file object.
+
+ Raise KeyError if 'objspec' does not exist in the repo.
+ """
+ class ObjHandler (object):
+ def __init__ (self, parser):
+ self.parser = parser
+ self.contents = StringIO()
+ self.err = None
+
+ def __call__ (self, parser, obj, sha1, t, size, data):
+ assert parser == self.parser
+ if not sha1: # missing object
+ self.err = "Missing object '%s'" % obj
+ return
+ assert size == len(data)
+ self.contents.write(data)
+
+ handler = ObjHandler(self)
+ self.push(objspec, handler)
+ self.process()
+ if handler.err: raise KeyError, handler.err
+ handler.contents.seek(0)
+ return handler.contents
+
+ def walk_tree (self, tree_objspec, callback, prefix = ""):
+ """Recursively walk the given Git tree object
+
+ Recursively walks all subtrees of the given tree object, and
+ invokes the given callback passing three arguments:
+ (path, mode, data) with the path, permission bits, and contents
+ of all the blobs found in the entire tree structure.
+ """
+
+ class ObjHandler (object):
+ """Helper class for walking a git tree structure"""
+ def __init__ (self, parser, cb, path, mode = None):
+ self.parser = parser
+ self.cb = cb
+ self.path = path
+ self.mode = mode
+ self.err = None
+
+ def parse_tree (self, treedata):
+ """Parse tree object data, yield tree entries
+
+ Each tree entry is a 3-tuple (mode, sha1, path)
+
+ self.path is prepended to all paths yielded
+ from this method.
+ """
+ while treedata:
+ mode = int(treedata[:6], 10)
+ # Turn 100xxx into xxx
+ if mode > 100000: mode -= 100000
+ assert treedata[6] == " "
+ i = treedata.find("\0", 7)
+ assert i > 0
+ path = treedata[7:i]
+ sha1 = hexlify(treedata[i + 1: i + 21])
+ yield (mode, sha1, self.path + path)
+ treedata = treedata[i + 21:]
+
+ def __call__ (self, parser, obj, sha1, t, size, data):
+ assert parser == self.parser
+ if not sha1: # missing object
+ self.err = "Missing object '%s'" % obj
+ return
+ assert size == len(data)
+ if t == "tree":
+ if self.path: self.path += "/"
+ # recurse into all blobs and subtrees
+ for m, s, p in self.parse_tree(data):
+ parser.push(s, self.__class__(
+ self.parser, self.cb,
+ p, m))
+ elif t == "blob":
+ self.cb(self.path, self.mode, data)
+ else:
+ raise ValueError, "Unknown object " \
+ "type '%s'" % (t)
+
+ self.push(tree_objspec, ObjHandler(self, callback, prefix))
+ self.process()
+
+
+class GitRefMap (object):
+ """Map Git ref names to the Git object names they currently point to
+
+ Behaves like a dictionary of Git ref names -> Git object names.
+ """
+
+ def __init__ (self, obj_fetcher):
+ self.obj_fetcher = obj_fetcher
+ self._cache = {} # dict: refname -> objname
+
+ def _load (self, ref):
+ if ref not in self._cache:
+ self._cache[ref] = self.obj_fetcher.get_sha1(ref)
+ return self._cache[ref]
+
+ def __contains__ (self, refname):
+ """Return True if the given refname is present in this cache"""
+ return bool(self._load(refname))
+
+ def __getitem__ (self, refname):
+ """Return the CvsState of the given refname"""
+ commit = self._load(refname)
+ if commit is None:
+ raise KeyError, "Unknown ref '%s'" % (refname)
+ return commit
+
+ def get (self, refname, default = None):
+ commit = self._load(refname)
+ if commit is None: return default
+ return commit
+
+
+class GitFICommit (object):
+ """Encapsulate the data in a Git fast-import commit command"""
+
+ SHA1RE = re.compile(r'^[0-9a-f]{40}$')
+
+ @classmethod
+ def parse_mode (cls, mode):
+ assert mode in (644, 755, 100644, 100755, 120000)
+ return "%i" % (mode)
+
+ @classmethod
+ def parse_objname (cls, objname):
+ if isinstance(objname, int): # object name is a mark number
+ assert objname > 0
+ return ":%i" % (objname)
+
+ # No existence check is done, only checks for valid format
+ assert cls.SHA1RE.match(objname) # object name is valid SHA1
+ return objname
+
+ @classmethod
+ def quote_path (cls, path):
+ path = path.replace("\\", "\\\\")
+ path = path.replace("\n", "\\n")
+ path = path.replace('"', '\\"')
+ return '"%s"' % (path)
+
+ @classmethod
+ def parse_path (cls, path):
+ assert not isinstance(path, int) # cannot be a mark number
+
+ # These checks verify the rules on the fast-import man page
+ assert not path.count("//")
+ assert not path.endswith("/")
+ assert not path.startswith("/")
+ assert not path.count("/./")
+ assert not path.count("/../")
+ assert not path.endswith("/.")
+ assert not path.endswith("/..")
+ assert not path.startswith("./")
+ assert not path.startswith("../")
+
+ if path.count('"') + path.count('\n') + path.count('\\'):
+ return cls.quote_path(path)
+ return path
+
+ @classmethod
+ def test (cls):
+ def expect_fail (method, data):
+ try: method(data)
+ except AssertionError: return
+ raise AssertionError, "Failed test for invalid data " \
+ "'%s(%s)'" % (method.__name__, repr(data))
+
+ # Test parse_mode()
+ assert cls.parse_mode(644) == "644"
+ assert cls.parse_mode(755) == "755"
+ assert cls.parse_mode(100644) == "100644"
+ assert cls.parse_mode(100755) == "100755"
+ assert cls.parse_mode(120000) == "120000"
+ expect_fail(cls.parse_mode, 0)
+ expect_fail(cls.parse_mode, 123)
+ expect_fail(cls.parse_mode, 600)
+ expect_fail(cls.parse_mode, "644")
+ expect_fail(cls.parse_mode, "abc")
+
+ # Test parse_objname()
+ assert cls.parse_objname(1) == ":1"
+ expect_fail(cls.parse_objname, 0)
+ expect_fail(cls.parse_objname, -1)
+ assert cls.parse_objname("0123456789" * 4) == "0123456789" * 4
+ assert cls.parse_objname("2468abcdef" * 4) == "2468abcdef" * 4
+ expect_fail(cls.parse_objname, "abcdefghij" * 4)
+
+ # Test parse_path()
+ assert cls.parse_path("foo/bar") == "foo/bar"
+ assert cls.parse_path(1) == ":1"
+ assert cls.parse_path("path/with\n and \" in it") \
+ == '"path/with\\n and \\" in it"'
+ expect_fail(cls.parse_path, 0)
+ expect_fail(cls.parse_path, -1)
+ expect_fail(cls.parse_path, "foo//bar")
+ expect_fail(cls.parse_path, "foo/bar/")
+ expect_fail(cls.parse_path, "/foo/bar")
+ expect_fail(cls.parse_path, "foo/./bar")
+ expect_fail(cls.parse_path, "foo/../bar")
+ expect_fail(cls.parse_path, "foo/bar/.")
+ expect_fail(cls.parse_path, "foo/bar/..")
+ expect_fail(cls.parse_path, "./foo/bar")
+ expect_fail(cls.parse_path, "../foo/bar")
+
+ def __init__ (self, name, email, timestamp, timezone, message):
+ self.name = name
+ self.email = email
+ self.timestamp = timestamp
+ self.timezone = timezone
+ self.message = message
+ self.pathops = [] # List of path operations in this commit
+
+ def modify (self, mode, blobname, path):
+ self.pathops.append(("M",
+ self.parse_mode(mode),
+ self.parse_objname(blobname),
+ self.parse_path(path)))
+
+ def delete (self, path):
+ self.pathops.append(("D", self.parse_path(path)))
+
+ def copy (self, path, newpath):
+ self.pathops.append(("C",
+ self.parse_path(path),
+ self.parse_path(newpath)))
+
+ def rename (self, path, newpath):
+ self.pathops.append(("R",
+ self.parse_path(path),
+ self.parse_path(newpath)))
+
+ def note (self, blobname, commit):
+ self.pathops.append(("N",
+ self.parse_objname(blobname),
+ self.parse_objname(commit)))
+
+ def deleteall (self):
+ self.pathops.append("deleteall")
+
+
+class GitFastImport (object):
+ """Encapsulate communication with git fast-import"""
+
+ def __init__ (self, f, obj_fetcher, last_mark = 0):
+ self.f = f # File object where fast-import stream is written
+ self.obj_fetcher = obj_fetcher # GitObjectFetcher instance
+ self.next_mark = last_mark + 1 # Next mark number
+ self.refs = set() # keep track of the refnames we've seen
+
+ def comment (self, s):
+ """Write the given comment in the fast-import stream"""
+ assert "\n" not in s, "Malformed comment: '%s'" % (s)
+ self.f.write("# %s\n" % (s))
+
+ def commit (self, ref, commitdata):
+ """Make a commit on the given ref, with the given GitFICommit
+
+ Return the mark number identifying this commit.
+ """
+ self.f.write("""\
+commit %(ref)s
+mark :%(mark)i
+committer %(name)s <%(email)s> %(timestamp)i %(timezone)s
+data %(msgLength)i
+%(msg)s
+"""% {
+ 'ref': ref,
+ 'mark': self.next_mark,
+ 'name': commitdata.name,
+ 'email': commitdata.email,
+ 'timestamp': commitdata.timestamp,
+ 'timezone': commitdata.timezone,
+ 'msgLength': len(commitdata.message),
+ 'msg': commitdata.message,
+})
+
+ if ref not in self.refs:
+ self.refs.add(ref)
+ parent = ref + "^0"
+ if self.obj_fetcher.get_sha1(parent):
+ self.f.write("from %s\n" % (parent))
+
+ for op in commitdata.pathops:
+ self.f.write(" ".join(op))
+ self.f.write("\n")
+ self.f.write("\n")
+ retval = self.next_mark
+ self.next_mark += 1
+ return retval
+
+ def blob (self, data):
+ """Import the given blob
+
+ Return the mark number identifying this blob.
+ """
+ self.f.write("blob\nmark :%i\ndata %i\n%s\n" \
+ % (self.next_mark, len(data), data))
+ retval = self.next_mark
+ self.next_mark += 1
+ return retval
+
+ def reset (self, ref, objname):
+ self.f.write("reset %s\nfrom %s\n\n" % \
+ (ref, GitFICommit.parse_objname(objname)))
+ if ref not in self.refs:
+ self.refs.add(ref)
+
+
+class GitNotes (object):
+ """Encapsulate access to Git notes
+
+ Behaves like a dictionary of object name (SHA1) -> Git note mappings.
+ """
+ def __init__ (self, notes_ref, obj_fetcher):
+ self.notes_ref = notes_ref
+ self.obj_fetcher = obj_fetcher # used to get objects from repo
+ self.imports = [] # list: (objname, note data blob name) tuples
+
+ def __del__ (self):
+ if self.imports:
+ error("Missing call to self.commit_notes().")
+ error("%i notes are not committed!", len(self.imports))
+
+ def _load (self, objname):
+ try:
+ f = self.obj_fetcher.open_obj(
+ "%s:%s" % (self.notes_ref, objname))
+ ret = f.read()
+ f.close()
+ except KeyError:
+ ret = None
+ return ret
+
+ def __getitem__ (self, objname):
+ """Return the note contents associated with the given object
+
+ Raise KeyError if given object has no associated note.
+ """
+ blobdata = self._load(objname)
+ if blobdata is None:
+ raise KeyError, "Object '%s' has no note" % (objname)
+ return blobdata
+
+ def get (self, objname, default = None):
+ """Return the note contents associated with the given object
+
+ Return given default if given object has no associated note.
+ """
+ blobdata = self._load(objname)
+ if blobdata is None: return default
+ return blobdata
+
+ def import_note (self, objname, data, gfi):
+ """Tell git fast-import to store data as a note for objname
+
+ This method uses the given GitFastImport object to create a
+ blob containing the given note data. Also an entry mapping the
+ given object name to the created blob is stored until
+ commit_notes() is called.
+
+ Note that this method only works if later followed with a call
+ to commit_notes() (which produces the note commit that refers
+ to the blob produced here).
+ """
+ if not data.endswith("\n"): data += "\n"
+ gfi.comment("Importing note for object %s" % (objname))
+ mark = gfi.blob(data)
+ self.imports.append((objname, mark))
+
+ def commit_notes (self, gfi, author, message):
+ """Produce a git fast-import note commit for the imported notes
+
+ This method uses the given GitFastImport object to create a
+ commit on the notes ref, introducing the notes previously
+ submitted to import_note().
+ """
+ if not self.imports: return # Nothing to do
+
+ from cvs import CvsDate
+ now = CvsDate("now")
+ commitdata = GitFICommit(
+ author[0], author[1], now.ts, now.tz_str(), message)
+
+ for objname, blobname in self.imports:
+ assert isinstance(objname, int) and objname > 0
+ assert isinstance(blobname, int) and blobname > 0
+ commitdata.note(blobname, objname)
+
+ gfi.commit(self.notes_ref, commitdata)
+ self.imports = []
+
+
+class GitCachedNotes (GitNotes):
+ """Encapsulate access to Git notes (cached version)
+
+ Only use this class if no caching is done at a higher level.
+
+ Behaves like a dictionary of object name (SHA1) -> Git note mappings.
+ """
+ def __init__ (self, notes_ref, obj_fetcher):
+ GitNotes.__init__(self, notes_ref, obj_fetcher)
+ self._cache = {} # cache: object name -> note data
+
+ def __del__ (self):
+ GitNotes.__del__(self)
+
+ def _load (self, objname):
+ if objname not in self._cache:
+ self._cache[objname] = GitNotes._load(self, objname)
+ return self._cache[objname]
+
+ def import_note (self, objname, data, gfi):
+ if not data.endswith("\n"): data += "\n"
+ assert objname not in self._cache
+ self._cache[objname] = data
+ GitNotes.import_note(self, objname, data, gfi)
+
+
+if __name__ == '__main__':
+ # Run selftests
+ GitFICommit.test()
diff --git a/git_remote_cvs/setup.py b/git_remote_cvs/setup.py
new file mode 100644
index 0000000..64c9209
--- /dev/null
+++ b/git_remote_cvs/setup.py
@@ -0,0 +1,12 @@
+from distutils.core import setup
+setup(
+ name = 'git_remote_cvs',
+ version = '0.1.0',
+ description = 'Git remote helper program for CVS repositories',
+ license = 'GPLv2',
+ author = 'The Git Community',
+ author_email = 'git@vger.kernel.org',
+ url = 'http://www.git-scm.com/',
+ package_dir = {'git_remote_cvs': ''},
+ packages = ['git_remote_cvs'],
+)
diff --git a/git_remote_cvs/util.py b/git_remote_cvs/util.py
new file mode 100644
index 0000000..6877454
--- /dev/null
+++ b/git_remote_cvs/util.py
@@ -0,0 +1,147 @@
+#!/usr/bin/env python
+
+"""Misc. useful functionality used by the rest of this package
+
+This module provides common functionality used by the other modules in this
+package.
+"""
+
+import sys, os, subprocess
+
+
+# Whether or not to show debug messages
+Debug = False
+
+def debug (msg, *args):
+ if Debug: print >>sys.stderr, msg % args
+
+def error (msg, *args):
+ print >>sys.stderr, "ERROR:", msg % args
+
+def die (msg, *args):
+ error(msg, *args)
+ sys.exit(1)
+
+
+class ProgressIndicator (object):
+ """Simple progress indicator
+
+ Displayed as a spinning character by default, but can be customized by
+ passing custom messages that will override the spinning character.
+ """
+
+ States = ("|", "/", "-", "\\")
+
+ def __init__ (self, prefix = "", f = sys.stdout):
+ self.n = 0 # Simple progress counter
+ self.f = f # Progress is written to this file object
+ self.prev_len = 0 # Length of previous msg (to be overwritten)
+ self.prefix = prefix
+ self.prefix_lens = []
+
+ def pushprefix (self, prefix):
+ self.prefix_lens.append(len(self.prefix))
+ self.prefix += prefix
+
+ def popprefix (self):
+ prev_len = self.prefix_lens.pop()
+ self.prefix = self.prefix[:prev_len]
+
+ def __call__ (self, msg = None, lf = False):
+ if msg is None: msg = self.States[self.n % len(self.States)]
+ msg = self.prefix + msg
+ print >>self.f, "\r%-*s" % (self.prev_len, msg),
+ self.prev_len = len(msg.expandtabs())
+ if lf:
+ print >>self.f
+ self.prev_len = 0
+ self.n += 1
+
+ def finish (self, msg = "done", noprefix = False):
+ if noprefix: self.prefix = ""
+ self(msg, True)
+
+
+def start_command (args, cwd = None, shell = False, add_env = None,
+ stdin = subprocess.PIPE,
+ stdout = subprocess.PIPE,
+ stderr = subprocess.PIPE):
+ env = None
+ if add_env is not None:
+ env = os.environ.copy()
+ env.update(add_env)
+ return subprocess.Popen(args, # run command
+ bufsize = 1, # line buffered
+ stdin = stdin, # write to process
+ stdout = stdout, # read from process
+ stderr = stderr,
+ cwd = cwd,
+ shell = shell,
+ env = env,
+ universal_newlines = True,
+ )
+
+def run_command (args, cwd = None, shell = False, add_env = None,
+ flag_error = True):
+ process = start_command(args, cwd, shell, add_env)
+ (output, errors) = process.communicate()
+ exit_code = process.returncode
+ if flag_error and errors:
+ error("'%s' returned errors:\n---\n%s---",
+ " ".join(args), errors)
+ if flag_error and exit_code:
+ error("'%s' returned exit code %i", " ".join(args), exit_code)
+ return (exit_code, output, errors)
+
+
+def file_reader_method (missing_ok = False):
+ """Decorator for simplifying reading of files.
+
+ If missing_ok is True, a failure to open a file for reading will not
+ raise the usual IOError, but instead the wrapped method will be called
+ with f == None. The method must in this case properly handle f == None.
+ """
+ def wrap (method):
+ """Teach given method to handle both filenames and file objects
+
+ The given method must take a file object as its second argument
+ (the first argument being 'self', of course). This decorator
+ will take a filename given as the second argument and promote
+ it to a file object.
+ """
+ def wrapped_method (self, filename, *args, **kwargs):
+ if isinstance(filename, file): f = filename # No-op
+ else:
+ try:
+ f = open(filename, 'r')
+ except IOError:
+ if missing_ok: f = None
+ else: raise
+ try:
+ return method(self, f, *args, **kwargs)
+ finally:
+ if not isinstance(filename, file) and f:
+ f.close()
+ return wrapped_method
+ return wrap
+
+def file_writer_method (method):
+ """Enable the given method to handle both filenames and file objects
+
+ The given method must take a file object as its second argument (the
+ first argument being 'self', of course). This decorator will take a
+ filename given as the second argument and promote it to a file object.
+ """
+ def new_method (self, filename, *args, **kwargs):
+ if isinstance(filename, file): f = filename # Nothing to do
+ else:
+ # Make sure the containing directory exists
+ parent_dir = os.path.dirname(filename)
+ if not os.path.isdir(parent_dir):
+ os.makedirs(parent_dir)
+ f = open(filename, 'w')
+ try:
+ return method(self, f, *args, **kwargs)
+ finally:
+ if not isinstance(filename, file): f.close()
+ return new_method
--
1.6.4.rc3.138.ga6b98.dirty
^ permalink raw reply related
* [RFCv3 3/4] Third draft of CVS remote helper program
From: Johan Herland @ 2009-08-12 0:13 UTC (permalink / raw)
To: git; +Cc: Johan Herland, barkalow, gitster, Johannes.Schindelin
In-Reply-To: <1250036031-32272-1-git-send-email-johan@herland.net>
Implements the import of objects from a local or remote CVS repository.
This helper program uses the "git_remote_cvs" Python package introduced
earlier, and provides a working draft implementation of the remote helper
API, as described in Documentation/git-remote-helpers.txt. Further details
about this specific helper are described in the new
Documentation/git-remote-cvs.txt.
This patch has been improved by the following contributions:
- Daniel Barkalow: Updates reflecting changes in remote helper API
Signed-off-by: Johan Herland <johan@herland.net>
---
Documentation/git-remote-cvs.txt | 85 +++++
Makefile | 24 ++
git-remote-cvs.py | 697 ++++++++++++++++++++++++++++++++++++++
3 files changed, 806 insertions(+), 0 deletions(-)
create mode 100644 Documentation/git-remote-cvs.txt
create mode 100755 git-remote-cvs.py
diff --git a/Documentation/git-remote-cvs.txt b/Documentation/git-remote-cvs.txt
new file mode 100644
index 0000000..783d542
--- /dev/null
+++ b/Documentation/git-remote-cvs.txt
@@ -0,0 +1,85 @@
+git-remote-cvs(1)
+==============
+
+NAME
+----
+git-remote-cvs - Helper program for interoperation with CVS repositories
+
+SYNOPSIS
+--------
+'git remote-cvs' <remote>
+
+DESCRIPTION
+-----------
+
+Please see the linkgit:git-remote-helpers[1] documentation for general
+information about remote helper programs.
+
+CONFIGURATION
+-------------
+
+remote.*.cvsRoot::
+ The URL of the CVS repository (as found in a `CVSROOT` variable, or
+ in a `CVS/Root` file).
+ Example: "`:pserver:user@server/var/cvs/cvsroot`".
+
+remote.*.cvsModule::
+ The path of the CVS module (as found in a `CVS/Repository` file)
+ within the CVS repository specified in `remote.*.cvsRoot`.
+ Example: "`foo/bar`"
+
+remote.*.cachedSymbolsOnly::
+ When 'true', a cache of CVS symbols is used instead of querying the
+ CVS server for all existing symbols (potentially expensive). In this
+ mode, git-remote-cvs will not discover new CVS symbols unless you add
+ them explicitly with the "`addsymbol <symbol>`" command (on
+ git-remote-cvs's stdin), or request an explicit symbol cache update
+ from the CVS server with the "`syncsymbols`" command (on
+ git-remote-cvs's stdin). When 'false' (the default), the CVS server
+ will be queried whenever a list of CVS symbols is required.
+
+remote.*.usernameMap::
+ The path (absolute, or relative to the repository (NOT the worktree))
+ to the file that contains the mapping from CVS usernames to the
+ corresponding full names and email addresses, as used by Git in the
+ Author and Committer fields of commit objects. When this config
+ variable is set, CVS usernames will be resolved against this file.
+ If no match is found in the file, or if this config variable is unset,
+ or if the variable points to a non-existing file, the original CVS
+ username will be used as the Author/Committer name, and the
+ corresponding email address will be set to "`<username>@example.com`".
++
+The format of the usernameMap file is one entry per line, where each line is
+of the form "`username: Full Name <email@address>`".
+Example: `johndoe: John Doe <johndoe@example.com>`
+Blank lines and lines starting with '#' are ignored.
+
+COMMANDS
+--------
+
+In addition to the commands that constitute the git-remote-helpers API, the
+following extra commands are supported for managing the local symbol cache when
+the `remote.*.cachedSymbolsOnly` config variable is true. The following
+commands can be given on the standard input of git-remote-cvs:
+
+'addsymbol'::
+ Takes one CVS symbol name as argument. The given CVS symbol is
+ fetched from the CVS server and stored into the local CVS symbol
+ cache. If `remote.*.cachedSymbolsOnly` is enabled, this can be used
+ to introduce a new CVS symbol to the CVS helper application.
+
+'syncsymbols'::
+ All CVS symbols that are available from the given remote are
+ fetched from the CVS server and stored into the local CVS symbol
+ cache. This is equivalent to disabling `remote.{asterisk}.cachedSymbolsOnly`,
+ running the "list" command, and then finally re-enabling the
+ `remote.*.cachedSymbolsOnly` config variable. I.e. this command can
+ be used to manually synchronize the CVS symbols available to the
+ CVS helper application.
+
+'verify'::
+ Takes one CVS symbol name as argument. Verifies that the CVS symbol
+ has been successfully imported be checking out the CVS symbol from
+ the CVS server, and comparing the CVS working tree against the Git
+ tree object identified by `refs/cvs/<remote>/<symbol>`. This can be
+ used to verify the correctness of a preceding 'import' command.
diff --git a/Makefile b/Makefile
index bb5cea2..b2af678 100644
--- a/Makefile
+++ b/Makefile
@@ -350,6 +350,8 @@ SCRIPT_PERL += git-relink.perl
SCRIPT_PERL += git-send-email.perl
SCRIPT_PERL += git-svn.perl
+SCRIPT_PYTHON += git-remote-cvs.py
+
SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) \
$(patsubst %.perl,%,$(SCRIPT_PERL)) \
$(patsubst %.py,%,$(SCRIPT_PYTHON)) \
@@ -1474,6 +1476,28 @@ $(patsubst %.perl,%,$(SCRIPT_PERL)) git-instaweb: % : unimplemented.sh
mv $@+ $@
endif # NO_PERL
+ifndef NO_PYTHON
+$(patsubst %.py,%,$(SCRIPT_PYTHON)): % : %.py
+ $(QUIET_GEN)$(RM) $@ $@+ && \
+ INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C git_remote_cvs -s --no-print-directory instlibdir` && \
+ sed -e '1{' \
+ -e ' s|#!.*python|#!$(PYTHON_PATH_SQ)|' \
+ -e '}' \
+ -e 's|^import sys.*|&; sys.path.insert(0, "@@INSTLIBDIR@@")|' \
+ -e 's|@@INSTLIBDIR@@|'"$$INSTLIBDIR"'|g' \
+ $@.py >$@+ && \
+ chmod +x $@+ && \
+ mv $@+ $@
+else # NO_PYTHON
+$(patsubst %.py,%,$(SCRIPT_PYTHON)): % : unimplemented.sh
+ $(QUIET_GEN)$(RM) $@ $@+ && \
+ sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
+ -e 's|@@REASON@@|NO_PYTHON=$(NO_PYTHON)|g' \
+ unimplemented.sh >$@+ && \
+ chmod +x $@+ && \
+ mv $@+ $@
+endif # NO_PYTHON
+
configure: configure.ac
$(QUIET_GEN)$(RM) $@ $<+ && \
sed -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
diff --git a/git-remote-cvs.py b/git-remote-cvs.py
new file mode 100755
index 0000000..1720d4c
--- /dev/null
+++ b/git-remote-cvs.py
@@ -0,0 +1,697 @@
+#!/usr/bin/env python
+
+"""Usage: git-remote-cvs <remote> [<url>]
+
+Git remote helper for interacting with CVS repositories
+
+See git-remote-helpers documentation for details on external interface, usage,
+etc. See git-remote-cvs documentation for specific configuration details of
+this remote helper.
+"""
+
+# PRINCIPLES:
+# -----------
+# - Importing same symbol twice (with no CVS changes in between) should yield
+# the exact same Git state (and the second import should have no commits).
+# - Importing several CVS symbols pointing to the same state should yield
+# corresponding refs pointing to the _same_ commit in Git.
+# - Importing a CVS symbol which has received only "regular commits" since
+# last import should yield a fast-forward straight line of commits.
+
+# TODO / KNOWN PROBLEMS:
+# ----------------------
+# - Remove cachedSymbolsOnly config variable for now?
+# - Author map handling; mapping CVS usernames to Git full name + email address
+# - Handle files that have been created AND deleted since the last import
+# - How to handle CVS tags vs. CVS branches. Turn CVS tags into Git tags?
+# - Better CVS branch handling: When a branch as a super/subset of files/revs
+# compared to another branch, find a way to base one branch on the other
+# instead of creating parallel lines of development with roughly the same
+# commits.
+# - Profiling, optimizations...
+
+import sys, string, os
+
+from git_remote_cvs.util import *
+from git_remote_cvs.cvs import *
+from git_remote_cvs.git import *
+from git_remote_cvs.cvs_symbol_cache import CvsSymbolCache
+from git_remote_cvs.commit_states import CommitStates
+from git_remote_cvs.cvs_revision_map import CvsRevisionMap, CvsStateMap
+from git_remote_cvs.changeset import build_changesets_from_revs
+
+class Config (object):
+ # Author name/email tuple for commits created by this tool
+ Author = ("git remote-cvs", "invalid@example.com")
+
+ # Git remote name
+ Remote = None
+
+ # CVS symbols are imported into this refs namespace/directory
+ RefSpace = None
+
+ # Git notes ref, the refname pointing to our git notes
+ NotesRef = None
+
+ # CVS repository identifier, a 2-tuple (cvs_root, cvs_module), where
+ # cvs_root is the CVS server/repository URL (as found in $CVSROOT, or
+ # in a CVS/Root file), and cvs_module is the path to a CVS module
+ # relative to the CVS repository (as found in a CVS/Repository file)
+ CvsRepo = (None, None)
+
+ # Path to the git-remote-cvs cache/work directory
+ # (normally "info/cvs/$remote" within $GIT_DIR)
+ WorkDir = None
+
+ # If False, the list of CVS symbols will always be retrieved from the
+ # CVS server using 'cvs rlog'. If True, only the cached symbols within
+ # the "symbols" subdirectory of WorkDir are consulted.
+ CachedSymbolsOnly = False
+
+ @classmethod
+ def init (cls, remote):
+ """Fetch configurations parameters for the given remote"""
+ git_config = parse_git_config()
+ assert git_config["remote.%s.vcs" % (remote)] == "cvs"
+
+ cls.Author = (
+ git_config["user.name"], git_config["user.email"])
+ cls.Remote = remote
+ cls.RefSpace = "refs/cvs/%s/" % (remote)
+ cls.NotesRef = "refs/notes/cvs/%s" % (remote)
+ cls.CvsRepo = (
+ git_config["remote.%s.cvsroot" % (remote)],
+ git_config["remote.%s.cvsmodule" % (remote)])
+ cls.WorkDir = os.path.join(get_git_dir(), "info/cvs", remote)
+ cls.CachedSymbolsOnly = git_config_bool(git_config.get(
+ "remote.%s.cachedsymbolsonly" % (remote), "false"))
+
+def work_path (*args):
+ """Return the given path appended to git-remote-cvs's cache/work dir"""
+ return os.path.join(Config.WorkDir, *args)
+
+def cvs_to_refname (cvsname):
+ """Return the git ref name for the given CVS symbolic name"""
+ if cvsname.startswith(Config.RefSpace): # Already converted
+ return cvsname
+ return Config.RefSpace + cvsname
+
+def ref_to_cvsname (refname):
+ """Return the CVS symbolic name for the given git ref name"""
+ if refname.startswith(Config.RefSpace):
+ return refname[len(Config.RefSpace):]
+ return refname
+
+def valid_cvs_symbol (symbol):
+ """Return True iff the given CVS symbol can be imported into Git"""
+ return valid_git_ref(cvs_to_refname(symbol))
+
+def die_usage (msg, *args):
+ # Use this file's docstring as a usage string
+ print >>sys.stderr, __doc__
+ die(msg, *args)
+
+def import_cvs_revs (symbol, prev_state, cur_state, progress):
+ """Import the CVS revisions involved in importing the given CVS symbol
+
+ This method will determine the CVS revisions involved in moving from
+ the given prev_state to the given cur_state. This includes looking at
+ revision metadata in CVS, and importing needed blobs from CVS.
+
+ The revision metadata is returned as a 2-level dict of CvsRev objects:
+ mapping path -> revnum -> CvsRev object.
+ """
+
+ # Calculate the revisions involved in moving from prev_state to
+ # cur_state, and fetch CvsRev objects for these revisions.
+ progress.pushprefix("Importing CVS revisions: ")
+ paths = set(prev_state.paths()).union(cur_state.paths())
+ num_fetched_revs = 0 # Number of CvsRev objects involved
+ num_imported_blobs = 0 # Number of blobs actually imported
+ cvs_revs = {} # path -> revnum -> CvsRev
+ for i, path in enumerate(sorted(paths)):
+ progress.pushprefix("(%i/%i) %s: " % (i + 1, len(paths), path))
+ progress("")
+ prev_rev = prev_state.get(path)
+ cur_rev = cur_state.get(path)
+ if prev_rev and cur_rev and prev_rev == cur_rev:
+ # No changes since last import
+ progress.popprefix()
+ continue
+
+ # Fetch CvsRev objects for range [path:prev_rev, path:symbol]
+ path_revs = fetch_revs(path, prev_rev, cur_rev, symbol,
+ Config.CvsRepo)
+ if not path_revs:
+ # Failed to find revs between prev_rev and symbol
+ if cur_rev:
+ assert not cur_rev.follows(prev_rev)
+ # The CVS symbol has been moved/reset since the
+ # last import in such a way that we cannot
+ # deduce the history between the last import
+ # and the current import.
+ # FIXME: Can we can work around this?
+ die("CVS symbol %s has been moved/reset from" \
+ " %s:%s to %s:%s since the last" \
+ " import. This is not supported",
+ symbol, path, prev_rev, path, cur_rev)
+ else:
+ # CVS symbol has been removed from this path.
+ # We cannot conclusively determine the history
+ # of this path following prev_rev.
+ # FIXME: Can we can work around this?
+ die("CVS symbol %s has been removed from %s" \
+ " since the last import. This is not" \
+ " supported", symbol, path)
+
+ # OK. We've got the revs in range [prev_rev, symbol]
+
+ # Verify/determine cur_rev
+ real_cur_rev = max(path_revs.keys())
+ if cur_rev: assert cur_rev == real_cur_rev
+ else: cur_rev = real_cur_rev
+
+ # No need to re-import prev_rev if already imported
+ if prev_rev:
+ assert cur_rev.follows(prev_rev)
+ assert prev_rev in path_revs
+ del path_revs[prev_rev]
+
+ assert path_revs # There should be more revs than just prev_rev
+
+ # Sanity checks:
+ # All revs from prev_rev to cur_rev are about to be imported
+ check_rev = cur_rev
+ while check_rev and check_rev != prev_rev:
+ assert check_rev in path_revs
+ check_rev = check_rev.parent()
+ # All previous revs have already been imported
+ check_rev = prev_rev
+ while check_rev:
+ assert Globals.CvsRevisionMap.has_rev(path, check_rev)
+ check_rev = check_rev.parent()
+
+ # Import CVS revisions as Git blobs
+ j = 0
+ for num, rev in sorted(path_revs.iteritems(), reverse = True):
+ j += 1
+ progress("(%i/%i) %s" % (j, len(path_revs), num))
+ assert num == rev.num
+
+ # Skip if already imported
+ if Globals.CvsRevisionMap.has_rev(rev.path, rev.num):
+ continue
+ # ...or if rev is a deletion
+ elif rev.deleted:
+ continue
+
+ # Import blob for reals
+ data = Globals.CvsWorkDir.get_revision_data(
+ rev.path, rev.num)
+ Globals.GitFastImport.comment(
+ "Importing CVS revision %s:%s" % (
+ rev.path, rev.num))
+ mark = Globals.GitFastImport.blob(data)
+ Globals.CvsRevisionMap.add_blob(
+ rev.path, rev.num, mark)
+ num_imported_blobs += 1
+
+ # Add path_revs to the overall structure of revs to be imported
+ assert path not in cvs_revs
+ cvs_revs[path] = path_revs
+ num_fetched_revs += len(path_revs)
+
+ progress.popprefix()
+
+ progress.popprefix()
+ progress("Imported %i blobs (reused %i existing blobs)" % (
+ num_imported_blobs, num_fetched_revs - num_imported_blobs),
+ True)
+
+ return cvs_revs
+
+def advance_state (state, changeset):
+ """Advance the given state by applying the given changeset"""
+ # Verify that the given changeset "fits" on top of the given state
+ for rev in changeset:
+ prev_num = rev.num.parent()
+ state_num = state.get(rev.path)
+ if prev_num is None and state_num is None:
+ # 'rev' is the first revision of this path being added
+ state.add(rev.path, rev.num)
+ elif prev_num and state_num and prev_num == state_num:
+ if rev.deleted: # rev deletes path from state
+ state.remove(rev.path, prev_num)
+ else: # rev follows state's revision of this path
+ state.replace(rev.path, rev.num)
+ else:
+ error("Cannot apply changeset with %s:%s on top of " \
+ "CVS state with %s:%s.",
+ rev.path, changeset[rev.path].num,
+ rev.path, state.get(rev.path))
+ error(" changeset: %s", changeset)
+ error(" CVS state: \n---\n%s---", state)
+ die("Failed to apply changeset. Aborting.")
+
+def revert_state (state, changeset):
+ """Revert the given state to _before_ the given changeset is applied
+
+ This is the reverse of the above advance_state() function.
+ """
+ for rev in changeset:
+ prev_num = rev.num.parent()
+ state_num = state.get(rev.path)
+ if state_num is None: # revert deletion of file
+ assert rev.deleted
+ state.add(rev.path, prev_num)
+ else:
+ assert state_num == rev.num
+ if prev_num is None: # revert addition of file
+ state.remove(rev.path, rev.num)
+ else: # regular revert to previous version
+ state.replace(rev.path, prev_num)
+
+def import_changesets (ref, changesets, from_state, to_state, progress):
+ """Apply the given list of Changeset objects to the given ref
+
+ Also verify that the changesets bring us from the given from_state to
+ the given to_state.
+ """
+ state = from_state
+ for i, c in enumerate(changesets):
+ advance_state(state, c)
+ progress("(%i/%i) Committing %s" % (i + 1, len(changesets), c))
+ # Make a git commit from changeset c
+ commitdata = GitFICommit(
+ c.author, # TODO: author_map handling
+ c.author + "@example.com", # TODO: author_map handling
+ c.date.ts,
+ c.date.tz_str(),
+ "".join(["%s\n" % (line) for line in c.message]),
+ )
+
+ for rev in c:
+ p, n = rev.path, rev.num
+ if rev.deleted:
+ commitdata.delete(p)
+ continue
+ blobname = Globals.CvsRevisionMap.get_blob(p, n)
+ mode = Globals.CvsRevisionMap.get_mode(p)
+ if mode is None: # Must retrieve mode from CVS checkout
+ debug("Retrieving mode info for '%s'" % (p))
+ Globals.CvsWorkDir.update(n, [p])
+ mode = Globals.CvsWorkDir.get_modeinfo([p])[p]
+ Globals.CvsRevisionMap.add_path(p, mode)
+ commitdata.modify(mode, blobname, p)
+
+ commitname = Globals.GitFastImport.commit(ref, commitdata)
+ Globals.CommitStates.add(
+ commitname, state, Globals.GitFastImport)
+ for path, revnum in state:
+ Globals.CvsRevisionMap.add_commit(
+ path, revnum, commitname)
+ assert commitname in Globals.CvsStateMap.get_commits(state)
+
+ assert state == to_state
+ return len(changesets)
+
+def import_cvs_symbol (cvs_symbol, progress):
+ """Import the given CVS symbol from CVS to Git
+
+ Return False if nothing was imported, True otherwise.
+ """
+ progress.pushprefix("%s: " % (cvs_symbol))
+
+ git_ref = cvs_to_refname(cvs_symbol)
+
+ # Verify that we are asked to import valid git ref names
+ if not valid_git_ref(git_ref):
+ progress("Invalid git ref '%s'. Skipping." % (git_ref), True)
+ progress.popprefix()
+ return False
+
+ # Retrieve previously imported CVS state
+ progress("Loading previously imported state of %s..." % (git_ref))
+ prev_commit = Globals.GitRefMap.get(git_ref)
+ prev_state = Globals.CommitStates.get(prev_commit, CvsState())
+
+ # Retrieve current CVS state of symbol
+ # Also: At some point we will need mode information for all CVS paths
+ # (stored in CvsRevisionMap). This information can be added for each
+ # path on demand (using CvsWorkDir.get_modeinfo()), but doing so may
+ # be an expensive process. It is much cheaper to load mode information
+ # for as many paths as possible in a _single_ operation. We do this
+ # below, by calling CvsRevisionMap.sync_modeinfo_from_cvs() in
+ # appropriate places
+ if Config.CachedSymbolsOnly:
+ progress("Synchronizing local CVS symbol cache for symbol...")
+ # The symbol cache is likely not up-to-date. Synchronize the
+ # given CVS symbol explicitly, to make sure we get the version
+ # current with the CVS server.
+ Globals.CvsSymbolCache.sync_symbol(
+ cvs_symbol, Globals.CvsWorkDir, progress)
+
+ # The above method updates the CVS workdir to the current CVS
+ # version. Hence, now is a convenient time to preload mode info
+ # from the currently checked-out CVS files. There may be more
+ # files for which we'll need mode information, but we'll deal
+ # with those when needed.
+ progress("Updating path mode info from current CVS checkout.")
+ Globals.CvsRevisionMap.sync_modeinfo_from_cvs(
+ Globals.CvsWorkDir)
+ elif not Globals.CvsRevisionMap: # There is no info for any paths, yet
+ # Pure optimization: We didn't get to preload all the mode info
+ # above. Normally, the only alternative is load mode info for
+ # each path on-demand. However, if our CvsRevisionMap is
+ # currently empty, that's probably going to be very expensive.
+ # Therefore, in this case, do an explicit CVS update here, and
+ # preload mode info for all paths.
+ progress("Updating CVS checkout to sync path mode info.")
+ Globals.CvsWorkDir.update(cvs_symbol)
+ Globals.CvsRevisionMap.sync_modeinfo_from_cvs(
+ Globals.CvsWorkDir)
+
+ progress("Loading current CVS state...")
+ try: cur_state = Globals.CvsSymbolCache[cvs_symbol]
+ except KeyError:
+ progress("Couldn't find symbol '%s'. Skipping." % (cvs_symbol),
+ True)
+ progress.popprefix()
+ return False
+
+ # Optimization: Check if the previous import of this symbol is still
+ # up-to-date. If so, there's nothing more to be done.
+ progress("Checking if we're already up-to-date...")
+ if cur_state == prev_state:
+ progress("Already up-to-date. Skipping.", True)
+ progress.popprefix()
+ return False
+
+ progress("Fetching CVS revisions...")
+ cvs_revs = import_cvs_revs(cvs_symbol, prev_state, cur_state, progress)
+
+ # Organize CvsRevs into a chronological list of changesets
+ progress("Organizing revisions into changesets...")
+ changesets = build_changesets_from_revs(cvs_revs)
+
+ # When importing a new branch, try to optimize branch start point,
+ # instead of importing entire branch from scratch
+ if prev_commit is None:
+ progress("Finding startpoint for new symbol...")
+ i = len(changesets)
+ state = cur_state.copy()
+ for c in reversed(changesets):
+ commit = Globals.CvsStateMap.get_exact_commit(
+ state, Globals.CommitStates)
+ if commit is not None:
+ # We have found a commit that exactly matches the state after commit #i (changesets[i - 1])
+ Globals.GitFastImport.reset(git_ref, commit)
+ changesets = changesets[i:]
+ break
+ revert_state(state, c)
+ i -= 1
+
+ num_changesets = len(changesets)
+ num_applied = 0
+
+
+ # Apply changesets, bringing git_ref from prev_state to cur_state
+ if num_changesets:
+ progress("Importing changesets...")
+ num_applied = import_changesets(git_ref, changesets, prev_state,
+ cur_state, progress)
+
+ progress("Imported %i changesets (reused %i existing changesets)" % (
+ num_applied, num_changesets - num_applied), True)
+ progress.popprefix()
+ return True
+
+def do_import (*args):
+ """Do the 'import' command; import refs from a remote"""
+ if not args: die_usage("'import' takes at least one parameter: ref...")
+
+ progress = ProgressIndicator(" ", sys.stderr)
+
+ cvs_symbols = map(ref_to_cvsname, args)
+ empty_import = True
+
+ for symbol in cvs_symbols:
+ if import_cvs_symbol(symbol, progress):
+ empty_import = False
+
+ if empty_import:
+ progress.finish("Everything up-to-date", True)
+ return 0
+
+ progress.finish("Finished importing %i CVS symbols to Git" % (
+ len(cvs_symbols)), True)
+ return 0
+
+def do_list (*args):
+ """Do the 'list' command; list refs available from a CVS remote"""
+ if args: die_usage("'list' takes no parameters")
+
+ progress = ProgressIndicator(" ", sys.stderr)
+
+ if Config.CachedSymbolsOnly:
+ progress("Listing symbols in local symbol cache...", True)
+ for symbol in sorted(Globals.CvsSymbolCache):
+ print cvs_to_refname(symbol)
+ progress.finish()
+ print # terminate output with blank line
+ return 0
+
+ # Synchronize local symbol cache with CVS server
+ progress("Synchronizing local symbol cache with CVS server...")
+ Globals.CvsSymbolCache.sync_all_symbols(Config.CvsRepo, progress,
+ valid_cvs_symbol)
+
+ # Load current states of Git refs
+ progress("Loading current state of Git refs...")
+ changed, unchanged = 0, 0
+ for cvs_symbol, cvs_state in sorted(Globals.CvsSymbolCache.items()):
+ git_ref = cvs_to_refname(cvs_symbol)
+ progress("\tChecking if Git ref is up-to-date: %s" % (git_ref))
+ git_commit = Globals.GitRefMap.get(git_ref)
+ git_state = Globals.CommitStates.get(git_commit)
+ attrs = ""
+ if git_state and git_state == cvs_state:
+ attrs = " unchanged"
+ unchanged += 1
+ else:
+ git_commit = "?"
+ changed += 1
+ print "%s %s%s" % (git_commit, git_ref, attrs)
+
+ progress.finish("Found %i CVS symbols (%i changed, %i unchanged)" % (
+ changed + unchanged, changed, unchanged))
+ print # terminate with blank line
+ return 0
+
+def do_capabilities (*args):
+ """Do the 'capabilities' command; report supported features"""
+ if args: die_usage("'capabilities' takes no parameters")
+ print "import"
+ print "marks %s" % (work_path("marks"))
+# print "export"
+# print "export-branch"
+# print "export-merges"
+ print # terminate with blank line
+ return 0
+
+def do_addsymbol (*args):
+ """Do the 'addsymbol' command; add given CVS symbol to local cache"""
+ if len(args) != 1: die_usage("'addsymbol' takes one parameter: symbol")
+ symbol = args[0]
+
+ progress = ProgressIndicator(" ", sys.stderr)
+ if valid_cvs_symbol(symbol):
+ Globals.CvsSymbolCache.sync_symbol(
+ symbol, Globals.CvsWorkDir, progress)
+ progress.finish("Added '%s' to CVS symbol cache" % (symbol),
+ True)
+ else:
+ error("Skipping CVS symbol '%s'; it is not a valid git ref",
+ symbol)
+
+ print # terminate with blank line
+ return 0
+
+def do_syncsymbols (*args):
+ """Do the 'syncsymbols' command; sync all symbols with CVS server"""
+ if args: die_usage("'syncsymbols' takes no parameters")
+ progress = ProgressIndicator(" ", sys.stderr)
+ Globals.CvsSymbolCache.sync_all_symbols(Config.CvsRepo, progress,
+ valid_cvs_symbol)
+ progress.finish()
+ print # terminate with blank line
+ return 0
+
+def do_verify (*args):
+ """Do the 'verify' command; Compare CVS checkout and Git tree"""
+ if len(args) != 1: die_usage("'verify' takes one parameter: symbol")
+ symbol = args[0]
+ gitref = cvs_to_refname(symbol)
+
+ progress = ProgressIndicator(" ", sys.stderr)
+ assert valid_cvs_symbol(symbol)
+
+ progress("Checking out '%s' from CVS..." % (symbol))
+ Globals.CvsWorkDir.update(symbol)
+
+ add_env = {"GIT_INDEX_FILE": os.path.abspath(work_path("temp_index"))}
+ progress("Creating Git index from tree object @ '%s'..." % (gitref))
+ cmd = ("git", "read-tree", gitref)
+ assert run_command(cmd, add_env = add_env)[0] == 0
+
+ progress("Comparing CVS checkout to Git index...", True)
+ cmd = ("git", "--work-tree=%s" % (os.path.abspath(work_path("cvs"))),
+ "ls-files",
+ "--exclude=CVS", "--deleted", "--modified", "--others", "-t")
+ exit_code, output, errors = run_command(cmd, add_env = add_env)
+ assert exit_code == 0 and not errors
+
+ if output:
+ progress.finish("Failed verification of '%s'" % (symbol), True)
+ error("The '%s' command returned:\n---\n%s---", " ".join(cmd),
+ output)
+ else:
+ progress.finish("Successfully verified '%s'" % (symbol), True)
+
+ print # terminate with blank line
+ return exit_code
+
+def not_implemented (*args):
+ die_usage("Command not implemented")
+
+Commands = {
+ "capabilities": do_capabilities,
+ "list": do_list,
+ # Special handling of 'import' in main()
+ # "import": do_import,
+ "export": not_implemented,
+ # Custom commands
+ "addsymbol": do_addsymbol,
+ "syncsymbols": do_syncsymbols,
+ "verify": do_verify,
+}
+
+class Globals (object):
+ """Global variables are placed here at the start of main()"""
+ pass
+
+def main (*args):
+ debug("Invoked '%s'", " ".join(args))
+
+ ### Initialization of subsystems
+
+ # Read config for the given remote
+ assert len(args) >= 2
+ Config.init(args[1])
+
+ # Local CVS symbol cache (CVS symbol -> CVS state mapping)
+ Globals.CvsSymbolCache = CvsSymbolCache(work_path("symbols"))
+
+ # Local CVS checkout
+ Globals.CvsWorkDir = CvsWorkDir(work_path("cvs"), Config.CvsRepo)
+
+ # Interface to 'git cat-file --batch'
+ Globals.GitObjectFetcher = GitObjectFetcher()
+
+ # Interface to Git object notes
+ Globals.GitNotes = GitNotes(Config.NotesRef, Globals.GitObjectFetcher)
+
+ # Mapping from Git commit objects to CVS states
+ Globals.CommitStates = CommitStates(Globals.GitNotes)
+
+ # Mapping from Git ref names to Git object names
+ Globals.GitRefMap = GitRefMap(Globals.GitObjectFetcher)
+
+ # Mapping from CVS revision to Git blob and commit objects
+ Globals.CvsRevisionMap = CvsRevisionMap(
+ cvs_to_refname("_metadata"), Globals.GitObjectFetcher)
+ last_mark = 0
+ if Globals.CvsRevisionMap.has_unresolved_marks():
+ # Update with marks from last import
+ last_mark = Globals.CvsRevisionMap.load_marks_file(
+ work_path("marks"))
+ else:
+ # Truncate marks file. We cannot automatically do this after
+ # .load_marks_file() above, since we cannot yet guarantee that
+ # we will be able to save the revision map persistently. (That
+ # can only happen if we are given one or more import commands
+ # below.) We can only truncate this file when we know there are
+ # no unresolved marks in the revision map.
+ open(work_path("marks"), "w").close()
+
+ # Mapping from CVS states to commit objects that contain said state
+ Globals.CvsStateMap = CvsStateMap(Globals.CvsRevisionMap)
+
+ ### Main program loop
+
+ import_refs = [] # accumulate import commands here
+ # cannot use "for line in sys.stdin" for buffering (?) reasons
+ line = sys.stdin.readline()
+ while (line):
+ cmdline = line.strip().split()
+ if not cmdline: break # blank line means we're about to quit
+
+ debug("Got command '%s'", " ".join(cmdline))
+ cmd = cmdline.pop(0)
+
+ if cmd == "import":
+ import_refs.extend(cmdline)
+ else:
+ if cmd not in Commands:
+ die_usage("Unknown command '%s'", cmd)
+ if Commands[cmd](*cmdline):
+ die("Command '%s' failed", line.strip())
+ sys.stdout.flush()
+ line = sys.stdin.readline()
+
+ ret = 0
+ if import_refs: # trigger import processing after last import command
+ # Init producer of output in the git-fast-import format
+ Globals.GitFastImport = GitFastImport(
+ sys.stdout, Globals.GitObjectFetcher, last_mark)
+
+ # Perform import of given refs
+ ret = do_import(*import_refs)
+
+ ### Notes on persistent storage of subsystems' data structures:
+ #
+ # Because the "import" command has been called, we here _know_
+ # that there is a fast-import process running in parallel.
+ # (This is NOT the case when there are no "import" commands).
+ # We can therefore now (and only now) safely commit the extra
+ # information that we store in the Git repo.
+ # In other words, the data structures that we commit to
+ # persistent storage with the following calls will NOT be
+ # committed if there are no "import" commands. The data
+ # structures must handle this in one of two ways:
+ # - In the no-"import" scenario, there is simply nothing to
+ # commit, so it can safely be skipped.
+ # - Any information that should have been committed in the
+ # no-"import" scenario can be reconstructed repeatedly in
+ # subsequent executions of this program, until the next
+ # invocation of an "import" command provides an opportunity
+ # to commit the data structure to persistent storage.
+
+ # Write out commit notes (mapping git commits to CvsStates)
+ # The following call would be a no-op in the no-"import" case
+ Globals.GitNotes.commit_notes(
+ Globals.GitFastImport, Config.Author,
+ 'Annotate commits imported by "git remote-cvs"\n')
+
+ # Save CVS revision metadata
+ # This data structure can handle the no-"import" case as long
+ # as the marks file from the last fast-import run is still
+ # present upon the next execution of this program.
+ Globals.CvsRevisionMap.commit_map(
+ Globals.GitFastImport, Config.Author,
+ 'Updated metadata used by "git remote-cvs"\n')
+
+ return ret
+
+if __name__ == '__main__':
+ sys.exit(main(*sys.argv))
--
1.6.4.rc3.138.ga6b98.dirty
^ permalink raw reply related
* [RFCv3 4/4] Add simple selftests of git-remote-cvs functionality
From: Johan Herland @ 2009-08-12 0:13 UTC (permalink / raw)
To: git; +Cc: Johan Herland, barkalow, gitster, Johannes.Schindelin
In-Reply-To: <1250036031-32272-1-git-send-email-johan@herland.net>
Add two new selftests:
- t9800-remote-cvs-basic: Test the git-remote-cvs implementation of the
remote helper API, by verifying the expected output of the git-remote-cvs
program when invoking remote helper API commands while doing some simple
CVS operations.
- t9801-remote-cvs-fetch: A more high-level test of the fetch/import-side
of the git-remote-cvs implementation, by verifying the expected repository
state after doing "git fetch" from a CVS remote where a variety of CVS
operations are being performed.
Signed-off-by: Johan Herland <johan@herland.net>
---
t/t9800-remote-cvs-basic.sh | 524 +++++++++++++++++++++++++++++++++++++++++++
t/t9801-remote-cvs-fetch.sh | 291 ++++++++++++++++++++++++
2 files changed, 815 insertions(+), 0 deletions(-)
create mode 100755 t/t9800-remote-cvs-basic.sh
create mode 100755 t/t9801-remote-cvs-fetch.sh
diff --git a/t/t9800-remote-cvs-basic.sh b/t/t9800-remote-cvs-basic.sh
new file mode 100755
index 0000000..6ddec17
--- /dev/null
+++ b/t/t9800-remote-cvs-basic.sh
@@ -0,0 +1,524 @@
+#!/bin/sh
+
+test_description='git remote-cvs basic tests'
+. ./test-lib.sh
+
+if ! test_have_prereq PYTHON; then
+ say 'skipping CVS foreign-vcs helper tests, python not available'
+ test_done
+fi
+
+CVS_EXEC=cvs
+CVS_OPTS="-f -q"
+CVS="$CVS_EXEC $CVS_OPTS"
+
+CVSROOT=$(pwd)/cvsroot
+export CVSROOT
+unset CVS_SERVER
+
+CVSMODULE=cvsmodule
+GITREMOTE=cvsremote
+
+if ! type $CVS_EXEC >/dev/null 2>&1
+then
+ say 'skipping remote-cvs tests, $CVS_EXEC not found'
+ test_done
+fi
+
+test_expect_success 'setup cvsroot' '$CVS init'
+
+test_expect_success '#1: setup a cvs module' '
+
+ mkdir "$CVSROOT/$CVSMODULE" &&
+ $CVS co -d module-cvs $CVSMODULE &&
+ (
+ cd module-cvs &&
+ cat <<EOF >o_fortuna &&
+O Fortuna
+velut luna
+statu variabilis,
+
+semper crescis
+aut decrescis;
+vita detestabilis
+
+nunc obdurat
+et tunc curat
+ludo mentis aciem,
+
+egestatem,
+potestatem
+dissolvit ut glaciem.
+EOF
+ $CVS add o_fortuna &&
+ cat <<EOF >message &&
+add "O Fortuna" lyrics
+
+These public domain lyrics make an excellent sample text.
+EOF
+ $CVS commit -f -F message o_fortuna
+ )
+'
+
+test_expect_success 'set up CVS repo as a foreign remote' '
+
+ git config "user.name" "Test User"
+ git config "user.email" "test@example.com"
+ git config "remote.$GITREMOTE.vcs" cvs
+ git config "remote.$GITREMOTE.cvsRoot" "$CVSROOT"
+ git config "remote.$GITREMOTE.cvsModule" "$CVSMODULE"
+ git config "remote.$GITREMOTE.fetch" \
+ "+refs/cvs/$GITREMOTE/*:refs/remotes/$GITREMOTE/*"
+
+'
+
+test_expect_success '#1: git-remote-cvs "capabilities" command' '
+
+ echo "capabilities" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+import
+marks .git/info/cvs/$GITREMOTE/marks
+
+EOF
+ test_cmp expect actual
+
+'
+
+test_expect_success '#1: git-remote-cvs "list" command' '
+
+ echo "list" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+? refs/cvs/$GITREMOTE/HEAD
+
+EOF
+ test_cmp expect actual
+
+'
+
+test_expect_success '#1: git-remote-cvs "import" command' '
+
+ echo "import refs/cvs/$GITREMOTE/HEAD" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+# Importing CVS revision o_fortuna:1.1
+blob
+mark :1
+data 180
+O Fortuna
+velut luna
+statu variabilis,
+
+semper crescis
+aut decrescis;
+vita detestabilis
+
+nunc obdurat
+et tunc curat
+ludo mentis aciem,
+
+egestatem,
+potestatem
+dissolvit ut glaciem.
+
+commit refs/cvs/$GITREMOTE/HEAD
+mark :2
+data 82
+add "O Fortuna" lyrics
+
+These public domain lyrics make an excellent sample text.
+
+M 644 :1 o_fortuna
+
+# Importing note for object 2
+blob
+mark :3
+data 14
+o_fortuna:1.1
+
+commit refs/notes/cvs/$GITREMOTE
+mark :4
+data 46
+Annotate commits imported by "git remote-cvs"
+
+N :3 :2
+
+blob
+mark :5
+data 32
+1 o_fortuna:1.1
+2 o_fortuna:1.1
+
+blob
+mark :6
+data 16
+blob 1
+commit 2
+
+commit refs/cvs/$GITREMOTE/_metadata
+mark :7
+data 42
+Updated metadata used by "git remote-cvs"
+
+M 644 :5 CVS/marks
+M 644 :6 o_fortuna/1.1
+
+EOF
+ grep -v "^committer " actual > actual.filtered &&
+ test_cmp expect actual.filtered
+
+'
+
+test_expect_success '#1: Passing git-remote-cvs output to git-fast-import' '
+
+ git fast-import --quiet \
+ --export-marks=".git/info/cvs/$GITREMOTE/marks" \
+ < actual &&
+ git gc
+
+'
+
+test_expect_success '#1: Verifying correctness of import' '
+
+ echo "verify HEAD" | git remote-cvs "$GITREMOTE"
+
+'
+
+test_expect_success '#2: update cvs module' '
+
+ (
+ cd module-cvs &&
+ cat <<EOF >o_fortuna &&
+O Fortune,
+like the moon
+you are changeable,
+
+ever waxing
+and waning;
+hateful life
+
+first oppresses
+and then soothes
+as fancy takes it;
+
+poverty
+and power
+it melts them like ice.
+EOF
+ cat <<EOF >message &&
+translate to English
+
+My Latin is terrible.
+EOF
+ $CVS commit -f -F message o_fortuna
+ )
+'
+
+test_expect_success '#2: git-remote-cvs "capabilities" command' '
+
+ echo "capabilities" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+import
+marks .git/info/cvs/$GITREMOTE/marks
+
+EOF
+ test_cmp expect actual
+
+'
+
+test_expect_success '#2: git-remote-cvs "list" command' '
+
+ echo "list" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+? refs/cvs/$GITREMOTE/HEAD
+
+EOF
+ test_cmp expect actual
+
+'
+
+test_expect_success '#2: git-remote-cvs "import" command' '
+
+ echo "import refs/cvs/$GITREMOTE/HEAD" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+# Importing CVS revision o_fortuna:1.2
+blob
+mark :8
+data 179
+O Fortune,
+like the moon
+you are changeable,
+
+ever waxing
+and waning;
+hateful life
+
+first oppresses
+and then soothes
+as fancy takes it;
+
+poverty
+and power
+it melts them like ice.
+
+commit refs/cvs/$GITREMOTE/HEAD
+mark :9
+data 44
+translate to English
+
+My Latin is terrible.
+
+from refs/cvs/$GITREMOTE/HEAD^0
+M 644 :8 o_fortuna
+
+# Importing note for object 9
+blob
+mark :10
+data 14
+o_fortuna:1.2
+
+commit refs/notes/cvs/$GITREMOTE
+mark :11
+data 46
+Annotate commits imported by "git remote-cvs"
+
+from refs/notes/cvs/$GITREMOTE^0
+N :10 :9
+
+blob
+mark :12
+data 32
+8 o_fortuna:1.2
+9 o_fortuna:1.2
+
+blob
+mark :13
+data 94
+
+blob
+mark :14
+data 16
+blob 8
+commit 9
+
+commit refs/cvs/$GITREMOTE/_metadata
+mark :15
+data 42
+Updated metadata used by "git remote-cvs"
+
+from refs/cvs/$GITREMOTE/_metadata^0
+M 644 :12 CVS/marks
+M 644 :13 o_fortuna/1.1
+M 644 :14 o_fortuna/1.2
+
+EOF
+ grep -v -e "^committer " -e "\b[0-9a-f]\{40\}\b" actual > actual.filtered &&
+ test_cmp expect actual.filtered
+
+'
+
+test_expect_success '#2: Passing git-remote-cvs output to git-fast-import' '
+
+ git fast-import --quiet \
+ --import-marks=".git/info/cvs/$GITREMOTE/marks" \
+ --export-marks=".git/info/cvs/$GITREMOTE/marks" \
+ < actual &&
+ git gc
+
+'
+
+test_expect_success '#2: Verifying correctness of import' '
+
+ echo "verify HEAD" | git remote-cvs "$GITREMOTE"
+
+'
+
+test_expect_success '#3: update cvs module' '
+
+ (
+ cd module-cvs &&
+ echo 1 >tick &&
+ $CVS add tick &&
+ $CVS commit -f -m 1 tick
+ )
+
+'
+
+test_expect_success '#3: git-remote-cvs "capabilities" command' '
+
+ echo "capabilities" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+import
+marks .git/info/cvs/$GITREMOTE/marks
+
+EOF
+ test_cmp expect actual
+
+'
+
+test_expect_success '#3: git-remote-cvs "list" command' '
+
+ echo "list" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+? refs/cvs/$GITREMOTE/HEAD
+
+EOF
+ test_cmp expect actual
+
+'
+
+test_expect_success '#3: git-remote-cvs "import" command' '
+
+ echo "import refs/cvs/$GITREMOTE/HEAD" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+# Importing CVS revision tick:1.1
+blob
+mark :16
+data 2
+1
+
+commit refs/cvs/$GITREMOTE/HEAD
+mark :17
+data 2
+1
+
+from refs/cvs/$GITREMOTE/HEAD^0
+M 644 :16 tick
+
+# Importing note for object 17
+blob
+mark :18
+data 23
+o_fortuna:1.2
+tick:1.1
+
+commit refs/notes/cvs/$GITREMOTE
+mark :19
+data 46
+Annotate commits imported by "git remote-cvs"
+
+from refs/notes/cvs/$GITREMOTE^0
+N :18 :17
+
+blob
+mark :20
+data 41
+16 tick:1.1
+17 tick:1.1
+17 o_fortuna:1.2
+
+blob
+mark :21
+data 104
+commit 17
+
+blob
+mark :22
+data 18
+blob 16
+commit 17
+
+commit refs/cvs/$GITREMOTE/_metadata
+mark :23
+data 42
+Updated metadata used by "git remote-cvs"
+
+from refs/cvs/$GITREMOTE/_metadata^0
+M 644 :20 CVS/marks
+M 644 :21 o_fortuna/1.2
+M 644 :22 tick/1.1
+
+EOF
+ grep -v -e "^committer " -e "\b[0-9a-f]\{40\}\b" actual > actual.filtered &&
+ test_cmp expect actual.filtered
+
+'
+
+test_expect_success '#3: Passing git-remote-cvs output to git-fast-import' '
+
+ git fast-import --quiet \
+ --import-marks=".git/info/cvs/$GITREMOTE/marks" \
+ --export-marks=".git/info/cvs/$GITREMOTE/marks" \
+ < actual &&
+ git gc
+
+'
+
+test_expect_success '#3: Verifying correctness of import' '
+
+ echo "verify HEAD" | git remote-cvs "$GITREMOTE"
+
+'
+
+test_expect_success '#4: git-remote-cvs "capabilities" command' '
+
+ echo "capabilities" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+import
+marks .git/info/cvs/$GITREMOTE/marks
+
+EOF
+ test_cmp expect actual
+
+'
+
+commit=$(git rev-parse "refs/cvs/$GITREMOTE/HEAD")
+
+test_expect_success '#4: git-remote-cvs "list" command' '
+
+ echo "list" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+$commit refs/cvs/$GITREMOTE/HEAD unchanged
+
+EOF
+ test_cmp expect actual
+
+'
+
+test_expect_success '#4: git-remote-cvs "import" command' '
+
+ echo "import refs/cvs/$GITREMOTE/HEAD" | git remote-cvs "$GITREMOTE" > actual &&
+ cat <<EOF >expect &&
+blob
+mark :24
+data 0
+
+blob
+mark :25
+data 142
+
+blob
+mark :26
+data 94
+
+commit refs/cvs/$GITREMOTE/_metadata
+mark :27
+data 42
+Updated metadata used by "git remote-cvs"
+
+from refs/cvs/$GITREMOTE/_metadata^0
+M 644 :24 CVS/marks
+M 644 :25 o_fortuna/1.2
+M 644 :26 tick/1.1
+
+EOF
+ grep -v -e "^committer " -e "\b[0-9a-f]\{40\}\b" actual > actual.filtered &&
+ test_cmp expect actual.filtered
+
+'
+
+test_expect_success '#4: Passing git-remote-cvs output to git-fast-import' '
+
+ git fast-import --quiet \
+ --import-marks=".git/info/cvs/$GITREMOTE/marks" \
+ --export-marks=".git/info/cvs/$GITREMOTE/marks" \
+ < actual &&
+ git gc
+
+'
+
+test_expect_success '#4: Verifying correctness of import' '
+
+ echo "verify HEAD" | git remote-cvs "$GITREMOTE"
+
+'
+
+test_done
diff --git a/t/t9801-remote-cvs-fetch.sh b/t/t9801-remote-cvs-fetch.sh
new file mode 100755
index 0000000..93d44a7
--- /dev/null
+++ b/t/t9801-remote-cvs-fetch.sh
@@ -0,0 +1,291 @@
+#!/bin/sh
+
+test_description='git remote-cvs basic tests'
+. ./test-lib.sh
+
+if ! test_have_prereq PYTHON; then
+ say 'skipping CVS foreign-vcs helper tests, python not available'
+ test_done
+fi
+
+CVS_EXEC=cvs
+CVS_OPTS="-f -q"
+CVS="$CVS_EXEC $CVS_OPTS"
+
+CVSROOT=$(pwd)/cvsroot
+export CVSROOT
+unset CVS_SERVER
+
+CVSMODULE=cvsmodule
+GITREMOTE=cvsremote
+
+if ! type $CVS_EXEC >/dev/null 2>&1
+then
+ say 'skipping remote-cvs tests, $CVS_EXEC not found'
+ test_done
+fi
+
+verify () {
+ git log --reverse --format="--- %T%n%s%n%n%b" "$GITREMOTE/$1" >actual &&
+ test_cmp "expect.$1" actual &&
+ echo "verify $1" | git remote-cvs "$GITREMOTE"
+}
+
+test_expect_success 'setup CVS repo' '$CVS init'
+
+test_expect_success 'create CVS module with initial commit' '
+
+ mkdir "$CVSROOT/$CVSMODULE" &&
+ $CVS co -d module-cvs $CVSMODULE &&
+ (
+ cd module-cvs &&
+ cat <<EOF >o_fortuna &&
+O Fortuna
+velut luna
+statu variabilis,
+
+semper crescis
+aut decrescis;
+vita detestabilis
+
+nunc obdurat
+et tunc curat
+ludo mentis aciem,
+
+egestatem,
+potestatem
+dissolvit ut glaciem.
+EOF
+ $CVS add o_fortuna &&
+ cat <<EOF >message &&
+add "O Fortuna" lyrics
+
+These public domain lyrics make an excellent sample text.
+EOF
+ $CVS commit -f -F message o_fortuna
+ )
+'
+
+test_expect_success 'set up CVS repo/module as a foreign remote' '
+
+ git config "user.name" "Test User"
+ git config "user.email" "test@example.com"
+ git config "remote.$GITREMOTE.vcs" cvs
+ git config "remote.$GITREMOTE.cvsRoot" "$CVSROOT"
+ git config "remote.$GITREMOTE.cvsModule" "$CVSMODULE"
+ git config "remote.$GITREMOTE.fetch" \
+ "+refs/cvs/$GITREMOTE/*:refs/remotes/$GITREMOTE/*"
+
+'
+
+test_expect_success 'initial fetch from CVS remote' '
+
+ cat <<EOF >expect.HEAD &&
+--- 0e06d780dedab23e683c686fb041daa9a84c936c
+add "O Fortuna" lyrics
+
+These public domain lyrics make an excellent sample text.
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD
+
+'
+
+test_expect_success 'CVS commit' '
+
+ (
+ cd module-cvs &&
+ cat <<EOF >o_fortuna &&
+O Fortune,
+like the moon
+you are changeable,
+
+ever waxing
+and waning;
+hateful life
+
+first oppresses
+and then soothes
+as fancy takes it;
+
+poverty
+and power
+it melts them like ice.
+EOF
+ cat <<EOF >message &&
+translate to English
+
+My Latin is terrible.
+EOF
+ $CVS commit -f -F message o_fortuna
+ ) &&
+ cat <<EOF >>expect.HEAD &&
+--- daa87269a5e00388135ad9542dc16ab6754466e5
+translate to English
+
+My Latin is terrible.
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD
+
+'
+
+test_expect_success 'CVS commit with new file' '
+
+ (
+ cd module-cvs &&
+ echo 1 >tick &&
+ $CVS add tick &&
+ $CVS commit -f -m 1 tick
+ ) &&
+ cat <<EOF >>expect.HEAD &&
+--- 486935b4fccecea9b64cbed3a797ebbcbe2b7461
+1
+
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD
+
+'
+
+test_expect_success 'fetch without CVS changes' '
+
+ git fetch "$GITREMOTE" &&
+ verify HEAD
+
+'
+
+test_expect_success 'add 2 CVS commits' '
+
+ (
+ cd module-cvs &&
+ echo 2 >tick &&
+ $CVS commit -f -m 2 tick &&
+ echo 3 >tick &&
+ $CVS commit -f -m 3 tick
+ ) &&
+ cat <<EOF >>expect.HEAD &&
+--- 83437ab3e57bf0a42915de5310e3419792b5a36f
+2
+
+
+--- 60fc50406a82dc6bd32dc6e5f7bd23e4c3cdf7ef
+3
+
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD
+
+'
+
+test_expect_success 'CVS commit with removed file' '
+
+ (
+ cd module-cvs &&
+ $CVS remove -f tick &&
+ $CVS commit -f -m "remove file" tick
+ ) &&
+ cat <<EOF >>expect.HEAD &&
+--- daa87269a5e00388135ad9542dc16ab6754466e5
+remove file
+
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD
+
+'
+
+test_expect_success 'CVS commit with several new files' '
+
+ (
+ cd module-cvs &&
+ echo spam >spam &&
+ echo sausage >sausage &&
+ echo eggs >eggs &&
+ $CVS add spam sausage eggs &&
+ $CVS commit -f -m "spam, sausage, and eggs" spam sausage eggs
+ ) &&
+ cat <<EOF >>expect.HEAD &&
+--- 3190dfce44a6d5e9916b4870dbf8f37d1ca4ddaf
+spam, sausage, and eggs
+
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD
+
+'
+
+test_expect_success 'new CVS branch' '
+
+ (
+ cd module-cvs &&
+ $CVS tag -b foo
+ ) &&
+ cp expect.HEAD expect.foo &&
+ git fetch "$GITREMOTE" &&
+ verify HEAD &&
+ verify foo
+
+'
+
+test_expect_success 'CVS commit on branch' '
+
+ (
+ cd module-cvs &&
+ $CVS up -r foo &&
+ echo "spam spam spam" >spam &&
+ $CVS commit -f -m "commit on branch foo" spam
+ ) &&
+ cat <<EOF >>expect.foo &&
+--- 1aba123e5c83898ce3a8b976cc6064d60246aef4
+commit on branch foo
+
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD &&
+ verify foo
+
+'
+
+test_expect_success 'create CVS tag' '
+
+ (
+ cd module-cvs &&
+ $CVS tag bar
+ ) &&
+ cp expect.foo expect.bar &&
+ git fetch "$GITREMOTE" &&
+ verify HEAD &&
+ verify foo &&
+ verify bar
+
+'
+
+test_expect_success 'another CVS commit on branch' '
+
+ (
+ cd module-cvs &&
+ echo "spam spam spam spam spam spam" >> spam &&
+ $CVS commit -f -m "another commit on branch foo" spam
+ ) &&
+ cat <<EOF >>expect.foo &&
+--- 15a2635e76e8e5a5a8746021643de317452f2340
+another commit on branch foo
+
+
+EOF
+ git fetch "$GITREMOTE" &&
+ verify HEAD &&
+ verify foo &&
+ verify bar
+
+'
+
+test_done
--
1.6.4.rc3.138.ga6b98.dirty
^ permalink raw reply related
* Re: [PATCH] Fix typos in git-remote.txt and git-symbolic-ref.txt
From: Wesley J. Landaker @ 2009-08-12 0:26 UTC (permalink / raw)
To: Thomas Rast; +Cc: Štěpán Němec, git
In-Reply-To: <200908110909.33904.trast@student.ethz.ch>
On Tuesday 11 August 2009 01:09:32 Thomas Rast wrote:
> So until we know for certain that they're not needed, I'd rather not
> actively remove them. Admittedly, there are lots of instances of `--
> in the docs too, so making sure which way is "right" and then
> replacing _all_ of them would probably be a nice cleanup.
I'm not fully sure of the current situation, but if it, for instance, turns
out that "--" gives an em-dash and "\--" gives two hyphens in a row, we'd only
want to change the "--"'s to "\--" for things like command-line options, where
an em-dash is totally inappropriate. If it's a sentence that is actually using
an em-dash for another purpose, it is probably intentional and desirable to
actually get an em-dash.
Obviously we want consistency, but--as this sentence shows--there is a
difference between an em-dash and a command-line option like --version.
^ permalink raw reply
* [PATCH/RFC] Add diff tests for trailing-space and now newline
From: Thell Fowler @ 2009-08-12 0:47 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <alpine.DEB.1.00.0908052239180.8306@pacific.mpi-cbg.de>
- Test each diff whitespace ignore option on trailing-space at eof
Signed-off-by: Thell Fowler <git@tbfowler.name>
---
Johannes Schindelin (Johannes.Schindelin@gmx.de) wrote on Aug 5, 2009:
>On Tue, 4 Aug 2009, Thell Fowler wrote:
>
>> mkdir test_ws_eof
>> cd test_ws_eof
>> git init
>> echo -n "Test" > test.txt
>> git add .
>> git commit -m'test'
>> git symbolic-ref HEAD refs/heads/with_space
>> rm .git/index
>> git clean -f
>> echo -n "Test ">test.txt
>> git add .
>> git commit -m'test'
>> # Ignoring all whitespace there shouldn't be a diff.
>> git diff -w master -- test.txt
>> # Ignoring space at eol there shouldn't be a diff
>> git diff --ignore-space-at-eol master -- test.txt
>> # Ignoring with -b might have a case for a diff showing.
>> git diff -b master -- test.txt
>
>If you turn that into a patch to, say, t/t4015-diff-whitespace.sh (adding
>a test_expect_failure for a known bug), it is much easier to convince
>developers to work on the issue.
Is this more along the right line?
Thell
t/t4015-diff-whitespace.sh | 23 +++++++++++++++++++++++
1 files changed, 23 insertions(+), 0 deletions(-)
diff --git a/t/t4015-diff-whitespace.sh b/t/t4015-diff-whitespace.sh
index 6d13da3..fddbf20 100755
--- a/t/t4015-diff-whitespace.sh
+++ b/t/t4015-diff-whitespace.sh
@@ -395,4 +395,27 @@ test_expect_success 'combined diff with autocrlf conversion' '
'
+test_expect_failure 'diff -w on trailing-space with no newline' '
+
+ git reset --hard &&
+ printf "foo " >x &&
+ git commit -m "trailing-space @ eof test" x &&
+ printf "foo" >x &&
+ git commit -m "trailing-space @ eof test" x &&
+ test_must_pass git diff -w HEAD^ -- x | grep "foo "
+
+'
+
+test_expect_failure 'diff -b on trailing-space with no newline' '
+
+ test_must_pass git diff -b HEAD^ -- x | grep "foo "
+
+'
+
+test_expect_failure 'diff --ignore-space-at-eol on trailing-space with no newline' '
+
+ test_must_pass git diff -ignore-space-at-eol HEAD^ -- x | grep "foo "
+
+'
+
test_done
--
1.6.4.240.g4cd31
^ permalink raw reply related
* [PATCH 1/2] Throw IllegalStateException if DirCacheEntry has not been fully initialized.
From: Grzegorz Kossakowski @ 2009-08-12 0:56 UTC (permalink / raw)
To: git; +Cc: Grzegorz Kossakowski
From: Grzegorz Kossakowski <grek@google.com>
When mode's object type of entry equals to Constants.OBJ_BAD it's a sign
of bad (uninitialized) state of an entry and not of problems with merging.
Signed-off-by: Grzegorz Kossakowski <grek@google.com>
---
.../org/spearce/jgit/dircache/DirCacheTree.java | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheTree.java b/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheTree.java
index 9d5af70..79e95cb 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheTree.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/dircache/DirCacheTree.java
@@ -377,7 +377,8 @@ private int computeSize(final DirCacheEntry[] cache, int cIdx,
final FileMode mode = e.getFileMode();
if (mode.getObjectType() == Constants.OBJ_BAD)
- throw new UnmergedPathException(e);
+ throw new IllegalStateException("Entry \"" + e.getPathString()
+ + "\" has incorrect mode set up.");
size += mode.copyToLength();
size += ep.length - pathOffset;
--
1.6.3.3
^ permalink raw reply related
* [PATCH 2/2] Add PathSuffixFilter class which is TreeFilter.
From: Grzegorz Kossakowski @ 2009-08-12 0:56 UTC (permalink / raw)
To: git; +Cc: Grzegorz Kossakowski
In-Reply-To: <1250038581-31241-1-git-send-email-grek@tuffmail.com>
From: Grzegorz Kossakowski <grek@google.com>
Added a simple TreeFilter that filters all entries that have path with
given suffix. This filter is always recursive.
Signed-off-by: Grzegorz Kossakowski <grek@google.com>
---
.../treewalk/filter/PathSuffixFilterTestCase.java | 110 ++++++++++++++++++++
.../src/org/spearce/jgit/treewalk/TreeWalk.java | 33 ++++++
.../jgit/treewalk/filter/PathSuffixFilter.java | 74 +++++++++++++
3 files changed, 217 insertions(+), 0 deletions(-)
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java
diff --git a/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java b/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
new file mode 100644
index 0000000..6042119
--- /dev/null
+++ b/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
@@ -0,0 +1,110 @@
+// Copyright (C) 2009 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package org.spearce.jgit.treewalk.filter;
+
+import java.io.IOException;
+import java.util.LinkedList;
+import java.util.List;
+
+import org.spearce.jgit.dircache.DirCache;
+import org.spearce.jgit.dircache.DirCacheBuilder;
+import org.spearce.jgit.dircache.DirCacheEntry;
+import org.spearce.jgit.lib.FileMode;
+import org.spearce.jgit.lib.ObjectId;
+import org.spearce.jgit.lib.ObjectWriter;
+import org.spearce.jgit.lib.RepositoryTestCase;
+import org.spearce.jgit.treewalk.TreeWalk;
+
+public class PathSuffixFilterTestCase extends RepositoryTestCase {
+
+ public void testNonRecursiveFiltering() throws IOException {
+ final ObjectWriter ow = new ObjectWriter(db);
+ final ObjectId aSth = ow.writeBlob("a.sth".getBytes());
+ final ObjectId aTxt = ow.writeBlob("a.txt".getBytes());
+ final DirCache dc = DirCache.read(db);
+ final DirCacheBuilder builder = dc.builder();
+ final DirCacheEntry aSthEntry = new DirCacheEntry("a.sth");
+ aSthEntry.setFileMode(FileMode.REGULAR_FILE);
+ aSthEntry.setObjectId(aSth);
+ final DirCacheEntry aTxtEntry = new DirCacheEntry("a.txt");
+ aTxtEntry.setFileMode(FileMode.REGULAR_FILE);
+ aTxtEntry.setObjectId(aTxt);
+ builder.add(aSthEntry);
+ builder.add(aTxtEntry);
+ builder.finish();
+ final ObjectId treeId = dc.writeTree(ow);
+
+
+ final TreeWalk tw = new TreeWalk(db);
+ tw.setFilter(PathSuffixFilter.create(".txt"));
+ tw.addTree(treeId);
+
+ List<String> paths = new LinkedList<String>();
+ while (tw.next()) {
+ paths.add(tw.getPathString());
+ }
+
+ List<String> expected = new LinkedList<String>();
+ expected.add("a.txt");
+
+ assertEquals(expected, paths);
+ }
+
+ public void testRecursiveFiltering() throws IOException {
+ final ObjectWriter ow = new ObjectWriter(db);
+ final ObjectId aSth = ow.writeBlob("a.sth".getBytes());
+ final ObjectId aTxt = ow.writeBlob("a.txt".getBytes());
+ final ObjectId bSth = ow.writeBlob("b.sth".getBytes());
+ final ObjectId bTxt = ow.writeBlob("b.txt".getBytes());
+ final DirCache dc = DirCache.read(db);
+ final DirCacheBuilder builder = dc.builder();
+ final DirCacheEntry aSthEntry = new DirCacheEntry("a.sth");
+ aSthEntry.setFileMode(FileMode.REGULAR_FILE);
+ aSthEntry.setObjectId(aSth);
+ final DirCacheEntry aTxtEntry = new DirCacheEntry("a.txt");
+ aTxtEntry.setFileMode(FileMode.REGULAR_FILE);
+ aTxtEntry.setObjectId(aTxt);
+ builder.add(aSthEntry);
+ builder.add(aTxtEntry);
+ final DirCacheEntry bSthEntry = new DirCacheEntry("sub/b.sth");
+ bSthEntry.setFileMode(FileMode.REGULAR_FILE);
+ bSthEntry.setObjectId(bSth);
+ final DirCacheEntry bTxtEntry = new DirCacheEntry("sub/b.txt");
+ bTxtEntry.setFileMode(FileMode.REGULAR_FILE);
+ bTxtEntry.setObjectId(bTxt);
+ builder.add(bSthEntry);
+ builder.add(bTxtEntry);
+ builder.finish();
+ final ObjectId treeId = dc.writeTree(ow);
+
+
+ final TreeWalk tw = new TreeWalk(db);
+ tw.setRecursive(true);
+ tw.setFilter(PathSuffixFilter.create(".txt"));
+ tw.addTree(treeId);
+
+ List<String> paths = new LinkedList<String>();
+ while (tw.next()) {
+ paths.add(tw.getPathString());
+ }
+
+ List<String> expected = new LinkedList<String>();
+ expected.add("a.txt");
+ expected.add("sub/b.txt");
+
+ assertEquals(expected, paths);
+ }
+
+}
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
index 5705936..ad19093 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
@@ -739,6 +739,39 @@ public int isPathPrefix(final byte[] p, final int pLen) {
}
/**
+ * Test if the supplied path matches (being suffix of) the current entry's
+ * path.
+ * <p>
+ * This method tests that the supplied path is exactly equal to the current
+ * entry, or is relative to one of entry's parent directories. It is faster
+ * to use this method then to use {@link #getPathString()} to first create
+ * a String object, then test <code>endsWith</code> or some other type of
+ * string match function.
+ *
+ * @param p
+ * path buffer to test. Callers should ensure the path does not
+ * start with '/' prior to invocation.
+ * @param pLen
+ * number of bytes from <code>buf</code> to test.
+ * @return true if p is suffix of the current path;
+ * false if otherwise
+ *
+ */
+ public boolean isPathSuffix(final byte[] p, final int pLen) {
+ final AbstractTreeIterator t = currentHead;
+ final byte[] c = t.path;
+ final int cLen = t.pathLen;
+ int ci;
+
+ for (ci = 1; ci < cLen && ci < pLen; ci++) {
+ if (c[cLen-ci] != p[pLen-ci])
+ return false;
+ }
+
+ return true;
+ }
+
+ /**
* Get the current subtree depth of this walker.
*
* @return the current subtree depth of this walker.
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java
new file mode 100644
index 0000000..b5b2461
--- /dev/null
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java
@@ -0,0 +1,74 @@
+// Copyright (C) 2009 The Android Open Source Project
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package org.spearce.jgit.treewalk.filter;
+
+import java.io.IOException;
+
+import org.spearce.jgit.errors.IncorrectObjectTypeException;
+import org.spearce.jgit.errors.MissingObjectException;
+import org.spearce.jgit.lib.Constants;
+import org.spearce.jgit.treewalk.TreeWalk;
+
+/**
+ * Includes tree entries only if they match the configured path.
+ */
+public class PathSuffixFilter extends TreeFilter {
+
+ /**
+ * Create a new tree filter for a user supplied path.
+ * <p>
+ * Path strings use '/' to delimit directories on all platforms.
+ *
+ * @param path
+ * the path (suffix) to filter on. Must not be the empty string.
+ * @return a new filter for the requested path.
+ * @throws IllegalArgumentException
+ * the path supplied was the empty string.
+ */
+ public static PathSuffixFilter create(String path) {
+ if (path.length() == 0)
+ throw new IllegalArgumentException("Empty path not permitted.");
+ return new PathSuffixFilter(path);
+ }
+
+ final String pathStr;
+ final byte[] pathRaw;
+
+ private PathSuffixFilter(final String s) {
+ pathStr = s;
+ pathRaw = Constants.encode(pathStr);
+ }
+
+ @Override
+ public TreeFilter clone() {
+ return this;
+ }
+
+ @Override
+ public boolean include(TreeWalk walker) throws MissingObjectException,
+ IncorrectObjectTypeException, IOException {
+ if (walker.isSubtree())
+ return true;
+ else
+ return walker.isPathSuffix(pathRaw, pathRaw.length);
+
+ }
+
+ @Override
+ public boolean shouldBeRecursive() {
+ return true;
+ }
+
+}
--
1.6.3.3
^ permalink raw reply related
* [JGIT PATCH 1/1] Fix for Repository.stripWorkDir when using partial paths
From: Adam W. Hawks @ 2009-08-12 0:48 UTC (permalink / raw)
To: git
>From ef993e633cdcb1dddda5e71db1b62306df7ce83f Mon Sep 17 00:00:00 2001
Date: Tue, 11 Aug 2009 20:02:56 -0400
When you call stripWorkDir with a relative path
you can get a string out of bounds error.
This change fixes that problem by using the absolute paths
of the file instead of its relative name.
Signed-off-by: Adam W. Hawks <awhawks@writeme.com>
---
.../src/org/spearce/jgit/lib/Repository.java | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/Repository.java b/org.spearce.jgit/src/org/spearce/jgit/lib/Repository.java
index 468cf4c..a68817b 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/Repository.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/Repository.java
@@ -1036,7 +1036,7 @@ public static boolean isValidRefName(final String refName) {
* @return normalized repository relative path
*/
public static String stripWorkDir(File wd, File f) {
- String relName = f.getPath().substring(wd.getPath().length() + 1);
+ String relName = f.getAbsolutePath().substring(wd.getPath().length() + 1);
relName = relName.replace(File.separatorChar, '/');
return relName;
}
--
1.6.0.2
^ permalink raw reply related
* Re: [PATCH 1/2] Throw IllegalStateException if DirCacheEntry has not been fully initialized.
From: Johannes Schindelin @ 2009-08-12 1:04 UTC (permalink / raw)
To: Grzegorz Kossakowski; +Cc: git, Grzegorz Kossakowski
In-Reply-To: <1250038581-31241-1-git-send-email-grek@tuffmail.com>
Hi,
On Tue, 11 Aug 2009, Grzegorz Kossakowski wrote:
> From: Grzegorz Kossakowski <grek@google.com>
>
> When mode's object type of entry equals to Constants.OBJ_BAD it's a sign
> of bad (uninitialized) state of an entry and not of problems with merging.
>
> Signed-off-by: Grzegorz Kossakowski <grek@google.com>
> ---
> .../org/spearce/jgit/dircache/DirCacheTree.java | 3 ++-
That's great, but next time, please prefix your mail subjects with [JGIT
PATCH] so that people who don't speak Java know that this is not for them.
Thanks!
Dscho
^ permalink raw reply
* Re: [PATCH 2/2] Add PathSuffixFilter class which is TreeFilter.
From: Grzegorz Kossakowski @ 2009-08-12 0:58 UTC (permalink / raw)
To: git
In-Reply-To: <1250038581-31241-2-git-send-email-grek@tuffmail.com>
Grzegorz Kossakowski wrote:
> From: Grzegorz Kossakowski <grek@google.com>
>
> Added a simple TreeFilter that filters all entries that have path with
> given suffix. This filter is always recursive.
>
> Signed-off-by: Grzegorz Kossakowski <grek@google.com>
> ---
> .../treewalk/filter/PathSuffixFilterTestCase.java | 110 ++++++++++++++++++++
> .../src/org/spearce/jgit/treewalk/TreeWalk.java | 33 ++++++
> .../jgit/treewalk/filter/PathSuffixFilter.java | 74 +++++++++++++
> 3 files changed, 217 insertions(+), 0 deletions(-)
> create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
> create mode 100644 org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java
>
> diff --git a/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java b/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
> new file mode 100644
> index 0000000..6042119
> --- /dev/null
> +++ b/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
> @@ -0,0 +1,110 @@
> +// Copyright (C) 2009 The Android Open Source Project
> +//
> +// Licensed under the Apache License, Version 2.0 (the "License");
> +// you may not use this file except in compliance with the License.
> +// You may obtain a copy of the License at
> +//
> +// http://www.apache.org/licenses/LICENSE-2.0
> +//
> +// Unless required by applicable law or agreed to in writing, software
> +// distributed under the License is distributed on an "AS IS" BASIS,
> +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +// See the License for the specific language governing permissions and
> +// limitations under the License.
> +
>
Oups. Wrong headers.
I'll resend these patches.
--
Grzegorz Kossakowski
^ permalink raw reply
* Re: [PATCH 1/2] Throw IllegalStateException if DirCacheEntry has not been fully initialized.
From: Grzegorz Kossakowski @ 2009-08-12 1:15 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, Grzegorz Kossakowski
In-Reply-To: <alpine.DEB.1.00.0908120303240.8306@pacific.mpi-cbg.de>
Johannes Schindelin wrote:
> Hi,
>
> On Tue, 11 Aug 2009, Grzegorz Kossakowski wrote:
>
>
>> From: Grzegorz Kossakowski <grek@google.com>
>>
>> When mode's object type of entry equals to Constants.OBJ_BAD it's a sign
>> of bad (uninitialized) state of an entry and not of problems with merging.
>>
>> Signed-off-by: Grzegorz Kossakowski <grek@google.com>
>> ---
>> .../org/spearce/jgit/dircache/DirCacheTree.java | 3 ++-
>>
>
> That's great, but next time, please prefix your mail subjects with [JGIT
> PATCH] so that people who don't speak Java know that this is not for them.
>
Sorry for that. I'll remember about it next time.
--
Grzegorz Kossakowski
^ permalink raw reply
* [JGIT PATCH] Add PathSuffixFilter class which is TreeFilter.
From: Grzegorz Kossakowski @ 2009-08-12 1:16 UTC (permalink / raw)
To: git; +Cc: Grzegorz Kossakowski
From: Grzegorz Kossakowski <grek@google.com>
Added a simple TreeFilter that filters all entries that have path with
given suffix. This filter is always recursive.
Signed-off-by: Grzegorz Kossakowski <grek@google.com>
---
.../treewalk/filter/PathSuffixFilterTestCase.java | 133 ++++++++++++++++++++
.../src/org/spearce/jgit/treewalk/TreeWalk.java | 32 +++++
.../jgit/treewalk/filter/PathSuffixFilter.java | 97 ++++++++++++++
3 files changed, 262 insertions(+), 0 deletions(-)
create mode 100644 org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
create mode 100644 org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java
diff --git a/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java b/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
new file mode 100644
index 0000000..5240ed9
--- /dev/null
+++ b/org.spearce.jgit.test/tst/org/spearce/jgit/treewalk/filter/PathSuffixFilterTestCase.java
@@ -0,0 +1,133 @@
+/*
+ * 2009 Copyright Google, Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.treewalk.filter;
+
+import java.io.IOException;
+import java.util.LinkedList;
+import java.util.List;
+
+import org.spearce.jgit.dircache.DirCache;
+import org.spearce.jgit.dircache.DirCacheBuilder;
+import org.spearce.jgit.dircache.DirCacheEntry;
+import org.spearce.jgit.lib.FileMode;
+import org.spearce.jgit.lib.ObjectId;
+import org.spearce.jgit.lib.ObjectWriter;
+import org.spearce.jgit.lib.RepositoryTestCase;
+import org.spearce.jgit.treewalk.TreeWalk;
+
+public class PathSuffixFilterTestCase extends RepositoryTestCase {
+
+ public void testNonRecursiveFiltering() throws IOException {
+ final ObjectWriter ow = new ObjectWriter(db);
+ final ObjectId aSth = ow.writeBlob("a.sth".getBytes());
+ final ObjectId aTxt = ow.writeBlob("a.txt".getBytes());
+ final DirCache dc = DirCache.read(db);
+ final DirCacheBuilder builder = dc.builder();
+ final DirCacheEntry aSthEntry = new DirCacheEntry("a.sth");
+ aSthEntry.setFileMode(FileMode.REGULAR_FILE);
+ aSthEntry.setObjectId(aSth);
+ final DirCacheEntry aTxtEntry = new DirCacheEntry("a.txt");
+ aTxtEntry.setFileMode(FileMode.REGULAR_FILE);
+ aTxtEntry.setObjectId(aTxt);
+ builder.add(aSthEntry);
+ builder.add(aTxtEntry);
+ builder.finish();
+ final ObjectId treeId = dc.writeTree(ow);
+
+
+ final TreeWalk tw = new TreeWalk(db);
+ tw.setFilter(PathSuffixFilter.create(".txt"));
+ tw.addTree(treeId);
+
+ List<String> paths = new LinkedList<String>();
+ while (tw.next()) {
+ paths.add(tw.getPathString());
+ }
+
+ List<String> expected = new LinkedList<String>();
+ expected.add("a.txt");
+
+ assertEquals(expected, paths);
+ }
+
+ public void testRecursiveFiltering() throws IOException {
+ final ObjectWriter ow = new ObjectWriter(db);
+ final ObjectId aSth = ow.writeBlob("a.sth".getBytes());
+ final ObjectId aTxt = ow.writeBlob("a.txt".getBytes());
+ final ObjectId bSth = ow.writeBlob("b.sth".getBytes());
+ final ObjectId bTxt = ow.writeBlob("b.txt".getBytes());
+ final DirCache dc = DirCache.read(db);
+ final DirCacheBuilder builder = dc.builder();
+ final DirCacheEntry aSthEntry = new DirCacheEntry("a.sth");
+ aSthEntry.setFileMode(FileMode.REGULAR_FILE);
+ aSthEntry.setObjectId(aSth);
+ final DirCacheEntry aTxtEntry = new DirCacheEntry("a.txt");
+ aTxtEntry.setFileMode(FileMode.REGULAR_FILE);
+ aTxtEntry.setObjectId(aTxt);
+ builder.add(aSthEntry);
+ builder.add(aTxtEntry);
+ final DirCacheEntry bSthEntry = new DirCacheEntry("sub/b.sth");
+ bSthEntry.setFileMode(FileMode.REGULAR_FILE);
+ bSthEntry.setObjectId(bSth);
+ final DirCacheEntry bTxtEntry = new DirCacheEntry("sub/b.txt");
+ bTxtEntry.setFileMode(FileMode.REGULAR_FILE);
+ bTxtEntry.setObjectId(bTxt);
+ builder.add(bSthEntry);
+ builder.add(bTxtEntry);
+ builder.finish();
+ final ObjectId treeId = dc.writeTree(ow);
+
+
+ final TreeWalk tw = new TreeWalk(db);
+ tw.setRecursive(true);
+ tw.setFilter(PathSuffixFilter.create(".txt"));
+ tw.addTree(treeId);
+
+ List<String> paths = new LinkedList<String>();
+ while (tw.next()) {
+ paths.add(tw.getPathString());
+ }
+
+ List<String> expected = new LinkedList<String>();
+ expected.add("a.txt");
+ expected.add("sub/b.txt");
+
+ assertEquals(expected, paths);
+ }
+
+}
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
index 5705936..16088b4 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/TreeWalk.java
@@ -739,6 +739,38 @@ public int isPathPrefix(final byte[] p, final int pLen) {
}
/**
+ * Test if the supplied path matches (being suffix of) the current entry's
+ * path.
+ * <p>
+ * This method tests that the supplied path is exactly equal to the current
+ * entry, or is relative to one of entry's parent directories. It is faster
+ * to use this method then to use {@link #getPathString()} to first create
+ * a String object, then test <code>endsWith</code> or some other type of
+ * string match function.
+ *
+ * @param p
+ * path buffer to test.
+ * @param pLen
+ * number of bytes from <code>buf</code> to test.
+ * @return true if p is suffix of the current path;
+ * false if otherwise
+ *
+ */
+ public boolean isPathSuffix(final byte[] p, final int pLen) {
+ final AbstractTreeIterator t = currentHead;
+ final byte[] c = t.path;
+ final int cLen = t.pathLen;
+ int ci;
+
+ for (ci = 1; ci < cLen && ci < pLen; ci++) {
+ if (c[cLen-ci] != p[pLen-ci])
+ return false;
+ }
+
+ return true;
+ }
+
+ /**
* Get the current subtree depth of this walker.
*
* @return the current subtree depth of this walker.
diff --git a/org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java b/org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java
new file mode 100644
index 0000000..8777e7d
--- /dev/null
+++ b/org.spearce.jgit/src/org/spearce/jgit/treewalk/filter/PathSuffixFilter.java
@@ -0,0 +1,97 @@
+/*
+ * 2009 Copyright Google, Inc.
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials provided
+ * with the distribution.
+ *
+ * - Neither the name of the Git Development Community nor the
+ * names of its contributors may be used to endorse or promote
+ * products derived from this software without specific prior
+ * written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+ * CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+ * ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+package org.spearce.jgit.treewalk.filter;
+
+import java.io.IOException;
+
+import org.spearce.jgit.errors.IncorrectObjectTypeException;
+import org.spearce.jgit.errors.MissingObjectException;
+import org.spearce.jgit.lib.Constants;
+import org.spearce.jgit.treewalk.TreeWalk;
+
+/**
+ * Includes tree entries only if they match the configured path.
+ */
+public class PathSuffixFilter extends TreeFilter {
+
+ /**
+ * Create a new tree filter for a user supplied path.
+ * <p>
+ * Path strings use '/' to delimit directories on all platforms.
+ *
+ * @param path
+ * the path (suffix) to filter on. Must not be the empty string.
+ * @return a new filter for the requested path.
+ * @throws IllegalArgumentException
+ * the path supplied was the empty string.
+ */
+ public static PathSuffixFilter create(String path) {
+ if (path.length() == 0)
+ throw new IllegalArgumentException("Empty path not permitted.");
+ return new PathSuffixFilter(path);
+ }
+
+ final String pathStr;
+ final byte[] pathRaw;
+
+ private PathSuffixFilter(final String s) {
+ pathStr = s;
+ pathRaw = Constants.encode(pathStr);
+ }
+
+ @Override
+ public TreeFilter clone() {
+ return this;
+ }
+
+ @Override
+ public boolean include(TreeWalk walker) throws MissingObjectException,
+ IncorrectObjectTypeException, IOException {
+ if (walker.isSubtree())
+ return true;
+ else
+ return walker.isPathSuffix(pathRaw, pathRaw.length);
+
+ }
+
+ @Override
+ public boolean shouldBeRecursive() {
+ return true;
+ }
+
+}
--
1.6.3.3
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox