* Re: git 0.99.9: Subversion importer breaks RPM generation (rpmbuild bug)
From: H. Peter Anvin @ 2005-10-31 19:36 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7v4q6xfpqg.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
>
> I do not know much about how things are done in the RPM world,
> but is there a concept of "the upstream" vs "packaging
> maintainer" there? IOW, are the majority of RPM binary packages
> done by the upstream maintainer?
>
It does both ways. I think Chris Wright has been doing the formal
maintenance of RPM for Fedora.
Latency is an issue, though, especially for kernel.org.
-hpa
^ permalink raw reply
* Re: git push sends more objects than it needs to
From: Linus Torvalds @ 2005-10-31 19:36 UTC (permalink / raw)
To: Luck, Tony; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0510311035310.27915@g5.osdl.org>
On Mon, 31 Oct 2005, Linus Torvalds wrote:
> >
> > Now the "unpack" on kernel.org did the right thing and noticed
> > that over 9000 of the objects were already in the packfile. But
> > I wonder if it couldn't have been smarter and not sent them?
>
> It should have been smarter, but I suspect you got caught by the fact that
> kernel.org by default has git-0.99.8f on it, which has the old
> pre-multi_ack code to figure out what the common commit was.
Oh, actually, I take that back.
This is not the multi-ack code at all, I suspect.
The problem is totally different: you copied the new pack to your
master.kernel.org repository, but you never updated any branches there.
So you had the objects, but git had no way of knowing. The pack generation
doesn't look at what _objects_ you have, it looks at what _refs_ you have.
And your refs were all to the old state.
So git actually did everything right (you can never rely on objects: you
may have a partial object list due to some earlier incomplete pull/push).
This is actually easy enough to fix up in one of several ways:
a) The "don't do that then" approach:
Don't go behind git's back and add objects on your own, and expect git
to realize what you did. ;^p
b) The "live with it" approach:
You copied the pack by hand, and that will keep git-unpack-objects
from duplicating the objects, but you'll still waste time and network
when trying pushing the objects (just once, though).
Ie this is what happened this time: nothing really lost, and the end
result is fine. Now you know why it happened, and you're fine.
c) The "I'm smarter than git" approach:
When you copy my objects, copy my reference to the top-of-tree too
(and rename it). NOTE! Now you need to be really careful, and you need
to make sure you copy _all_ the objects, because if you screw this up,
your repo will be missing objects that you claim are there, and it
will be all your fault.
I really don't advocate this approach at all. It's certainly doable,
but it's also the only approach where you can really screw up.
d) Just let git do it for you.
Copy the pack-files, or add my object directory as an "alternates" for
your object directory, do the "git prune-objects", and then _locally_
on master.kernel.org just do something like
git fetch ..linus-directory.. master:linus
which will still create the unnecessary pack-file and unpack it into
nothingness (since you have the objects in the pack-file you copied by
hand), but at least it won't eat any network bandwidth, and it will do
the right thing if it turns out that I've pushed something after doing
the pack-file, and fetch those individual objects in _addition_ to the
pack-file you snarfed by hand. It will also obviously update a ref in
your tree (the "linus" branch), so now when you send stuff later, it
will know all about the objects you already have.
e) Re-create the tree entirely
Blow away your tree on master entirely, just re-create it locally with
"git clone -l -s" from my tree (which will do all the "alternates"
object files for you), and then populate the result with a simple push
from your home tree.
f) any number of variations on a theme. IOW, there are endless ways to do
this.
Hmm?
Linus
^ permalink raw reply
* Re: git 0.99.9: Subversion importer breaks RPM generation (rpmbuild bug)
From: Junio C Hamano @ 2005-10-31 19:31 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: git
In-Reply-To: <43663EEA.5050102@zytor.com>
"H. Peter Anvin" <hpa@zytor.com> writes:
> Chris Wright wrote:
>> It's fine for FC3. Certain irony that git now effectively requires
>> subversion. I'm all for splitting these out, but have no time until
>> later in the week. BTW, mind pushing the tag?
>
> The git-core RPM definitiely needs to be split. Doubly ironic that it's
> called "core".
BTW, did that "require 5.008" change make the resulting package
RPM happy on RH-EL4?
I agree that it is an extremely good thing to split the binary
packages into separate ones so that the system administrators
can pick and choose only the bits that are needed. Here is a
strawman:
git-tla-import, git-cvs-import, git-svn-import, ...::
Importers, one per foreign SCMs.
git-docs::
Generated documentation from Documentation hierarchy.
git-core::
All the rest, plus man pages. We could separate out
commit walkers if we wanted to, but I do not think that
is necessary.
Having said that, I consider this purely binary packaging issue.
I.e. I do not think you are advocating for splitting the source
tree.
I do not know much about how things are done in the RPM world,
but is there a concept of "the upstream" vs "packaging
maintainer" there? IOW, are the majority of RPM binary packages
done by the upstream maintainer?
I am currently generating i386 RPMs and i386 debs myself but I
am not particularly proud of the current setup. I do not have
an RPM based machine that I can install the result myself to
test (which is what started this thread). Since I am not a
Debian developer (and I do not particularly wish to become one
myself), the debs I generate will not be official anyway.
Personally I'd be happier if I can just lose rpm and deb targets
from the "upstream" Makefile (git-core.spec file and debian/
subdirectory as well while we are at it), ask "packaging
maintainers" to pull from kernel.org/ tree and do RPMs and Debs
outside.
On the other hand, having the basic support for packagers in the
upstream might be easier for port maintainers. I honestly do
not know.
One thing we could do without breaking much of the current
arrangement is to have a team of people to help porting for
major packaging formats (RPMs and Debs mostly but I know we have
OpenBSD and Darwin people here too), and ask them to feed me the
updates to rpm/deb/whatever target in the Makefile as needed.
Especially before a major release I could ask them to test
things out and generate binary packages, perhaps taken out of
the tip of the master branch, or even another "for-porters"
branch for this purpose.
^ permalink raw reply
* Re: git push sends more objects than it needs to
From: Junio C Hamano @ 2005-10-31 18:49 UTC (permalink / raw)
To: Luck, Tony; +Cc: git
In-Reply-To: <20051031182355.GA7368@agluck-lia64.sc.intel.com>
"Luck, Tony" <tony.luck@intel.com> writes:
> Now the "unpack" on kernel.org did the right thing and noticed
> that over 9000 of the objects were already in the packfile. But
> I wonder if it couldn't have been smarter and not sent them?
>
> Or am I just subverting the whole paradigm by hand-copying
> packfiles around?
The fix for this problem is in 0.99.9 (credits to Johannes).
Unfortunately, 0.99.8f is still in /usr/bin/ on kernel.org.
^ permalink raw reply
* Re: git push sends more objects than it needs to
From: Linus Torvalds @ 2005-10-31 18:44 UTC (permalink / raw)
To: Luck, Tony; +Cc: git
In-Reply-To: <20051031182355.GA7368@agluck-lia64.sc.intel.com>
On Mon, 31 Oct 2005, Luck, Tony wrote:
>
> Finally, after making sure that release and test
> branches still built with all the new stuff I did
> a "git push" to update my kernel repo. I saw this:
...
> Packing 10785 objects
> Unpacking 10785 objects
>
> Now the "unpack" on kernel.org did the right thing and noticed
> that over 9000 of the objects were already in the packfile. But
> I wonder if it couldn't have been smarter and not sent them?
It should have been smarter, but I suspect you got caught by the fact that
kernel.org by default has git-0.99.8f on it, which has the old
pre-multi_ack code to figure out what the common commit was.
The pack-file contains over ten times as many objects, so you definitely
didn't get all of them - but because there has been a lot of merges
lately, and the common commit finder algorithm wasn't all that careful,
you ended up getting many more objects than you really needed.
Remember: the pack generation is not "exact" - it will often generate a
few extra objects for any non-trivial case (for example, it fundamentally
happens if there has been reverts: it won't realize that you had the older
version of a file already). It just so happens that the old algorithm had
some cases where it would decide on totally the wrong common commit, and
re-send a _lot: more objects than it needs.
I'm not sure multi-ack fixes it entirely either, but I think it makes it a
lot less likely (but even with multi-ack, the "file revert" case still
happens, so you should always expect that can get a _couple_ of
unnecessary objects).
Now, even the old stupid algorithm got the _easy_ cases obviously right,
so people might have incorrectly gotten the idea that it was careful and
exact, just because quite often it ends up being that in practice.
Linus
^ permalink raw reply
* Re: Another problem with not existing directories
From: Junio C Hamano @ 2005-10-31 18:43 UTC (permalink / raw)
To: Marcel Holtmann; +Cc: git
In-Reply-To: <1130780772.5848.32.camel@blade>
Marcel Holtmann <marcel@holtmann.org> writes:
>> Are you sure you just don't have an old version of git on the other end?
>
> this might be possible, but then it is an old git on kernel.org. I never
> installed any git by myself on it.
I just tried this myself; master.kernel.org seems to have
0.99.8f in /usr/bin/, which explains this symptom.
^ permalink raw reply
* git push sends more objects than it needs to
From: Luck, Tony @ 2005-10-31 18:23 UTC (permalink / raw)
To: git
I just pulled from Linus' Linux repository ... and my
wrapper script alerted me to the fact that he had just
re-packed.
So, I logged into kernel.org and linked the new packfiles
across to my repo there, and did a "git prune-packed" to
clean away all the unneeded unpacked objects.
Back on my local box I merged all the new stuff that
had been pulled into my "release" and "test" branches.
(just a fast-forward for the release branch).
Next I used "scp" to copy Linus' new pack files to my
local objects/pack ... and ran git prune packed here.
Finally, after making sure that release and test
branches still built with all the new stuff I did
a "git push" to update my kernel repo. I saw this:
updating 'refs/heads/test'
from 0711c558ff3cf4d1e498a9686f4d8ce48aa8d79f
to 2e488d3f8f299eb0ed826055bc30003e2c27d909
updating 'refs/heads/release'
from 8a212ab6b8a4ccc6f3c3d1beba5f92655c576404
to 581c1b14394aee60aff46ea67d05483261ed6527
Packing 10785 objects
Unpacking 10785 objects
100% (10785/10785) done
refs/heads/test: 0711c558ff3cf4d1e498a9686f4d8ce48aa8d79f -> 2e488d3f8f299eb0ed826055bc30003e2c27d909
refs/heads/release: 8a212ab6b8a4ccc6f3c3d1beba5f92655c576404 -> 581c1b14394aee60aff46ea67d05483261ed6527
Now the "unpack" on kernel.org did the right thing and noticed
that over 9000 of the objects were already in the packfile. But
I wonder if it couldn't have been smarter and not sent them?
Or am I just subverting the whole paradigm by hand-copying
packfiles around?
-Tony
^ permalink raw reply
* Re: git versus CVS (versus bk)
From: wa1ter @ 2005-10-31 18:18 UTC (permalink / raw)
To: git
In-Reply-To: <Pine.LNX.4.64.0510310804400.27915@g5.osdl.org>
On Mon, 31 Oct 2005, Linus Torvalds wrote:
> ...CVS inevitably leads to "politics" - having a
> central place means that you have people who fight for control over it,
> even when they'd otherwise _want_ to cooperate...
Ahh -- the lightbulb just lit up. Using CVS is just like being married.
No wonder you hate it...
^ permalink raw reply
* Re: Another problem with not existing directories
From: Marcel Holtmann @ 2005-10-31 17:46 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0510310929361.27915@g5.osdl.org>
Hi Linus,
> > I think there exists another problem with git-unpack-objects if the
> > sub-directory in "objects" doesn't exists.
>
> Are you sure you just don't have an old version of git on the other end?
this might be possible, but then it is an old git on kernel.org. I never
installed any git by myself on it.
Regards
Marcel
^ permalink raw reply
* Re: Another problem with not existing directories
From: Linus Torvalds @ 2005-10-31 17:35 UTC (permalink / raw)
To: Marcel Holtmann; +Cc: git
In-Reply-To: <1130779234.5848.15.camel@blade>
On Mon, 31 Oct 2005, Marcel Holtmann wrote:
>
> I think there exists another problem with git-unpack-objects if the
> sub-directory in "objects" doesn't exists.
Are you sure you just don't have an old version of git on the other end?
As far as I know, git-unpack-objects will use the bog-standard
"write_sha1_file()" which already knows how to create missing
subdirectories.
This _sounds_ like the other end has two different versions of git, and
when you "push" to it, it executes an old version, but then you have a
newer version somewhere that created the sparsely-populated object
directory in the first place.
This happens (for example) if you have installed an older version of git
through an rpm (say, the one that comes from Fedora Core: 0.99.8) and you
have that in /usr/bin, but then you have a newer version that you've built
and installed yourself in your ~/bin/ directory.
If you have your PATH being set in your .bashrc, it won't trigger with a
non-interactive login, so you'll use the /usr/bin/ version when pushing,
but then when you log in interactively to a shell, you'll use the ~/bin
one (or the other way around, I'm easily confused by the bash config file
rules ;)
Linus
^ permalink raw reply
* Another problem with not existing directories
From: Marcel Holtmann @ 2005-10-31 17:20 UTC (permalink / raw)
To: git
Hi guys,
I think there exists another problem with git-unpack-objects if the
sub-directory in "objects" doesn't exists.
# cg-push master
updating 'refs/heads/master'
from 581c1b14394aee60aff46ea67d05483261ed6527
to 197c190b0b00689074c03fc3ebcfa114dbed1d73
Packing 37 objects
Unpacking 37 objects
unable to write sha1 filename ./objects/dc/59cd72edc7bc6169c2c9c073c86483f9318e7c: No such file or directoryfatal: failed to write object
fatal: git-unpack-objects exited with error code 128
In this case the directory "objects/dc" doesn't exists. Creating it
fixes the problem, but I think they should be created on demand.
Regards
Marcel
^ permalink raw reply
* Re: git 0.99.9: Subversion importer breaks RPM generation (rpmbuild bug)
From: H. Peter Anvin @ 2005-10-31 16:27 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Chris Wright, Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0510310819290.27915@g5.osdl.org>
Linus Torvalds wrote:
>
> Now, I'm not certain how happy RPM would be with having one source RPM
> generate multiple binary RPM's, so we might have problems with some stupid
> RPM rules, but I think we should really do this. There's always going to
> be some extra feature that not everybody needs, but that it would be silly
> to have its own project for.
>
RPM is more than happy to do this. It's a standard feature of RPM. The
current RPM, however, is structured in a way that makes it somewhat
painful, as it depends a little too much on wildcards.
-hpa
^ permalink raw reply
* Re: git 0.99.9: Subversion importer breaks RPM generation (rpmbuild bug)
From: Linus Torvalds @ 2005-10-31 16:25 UTC (permalink / raw)
To: H. Peter Anvin; +Cc: Chris Wright, Junio C Hamano, git
In-Reply-To: <43663EEA.5050102@zytor.com>
On Mon, 31 Oct 2005, H. Peter Anvin wrote:
>
> The git-core RPM definitiely needs to be split. Doubly ironic that it's
> called "core".
I don't think it's necessarily ironic. It's actually a good thing.
I think that what we want to have is one _project_ (called "git"), which
can generate multiple RPM's ("git-core", "git-svnimport", "git-docs",
whatever).
So I think it was good that we called the RPM "git-core". We've just not
yet done the obvious thing to create a few _other_ RPM's.
Now, I'm not certain how happy RPM would be with having one source RPM
generate multiple binary RPM's, so we might have problems with some stupid
RPM rules, but I think we should really do this. There's always going to
be some extra feature that not everybody needs, but that it would be silly
to have its own project for.
Having one bigger project means that it's much easier to maintain, and
there's less administrative overhead (good maintainers are really hard to
find: you should realize how lucky we are to have Junio). Trying to split
up the source code into independent projects at this level would just be
much more pain than it's worth. But clearly we want to split up the RPM's.
Linus
^ permalink raw reply
* Re: git versus CVS (versus bk)
From: Linus Torvalds @ 2005-10-31 16:18 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: walt, git
In-Reply-To: <Pine.LNX.4.63.0510311111340.2916@wbgn013.biozentrum.uni-wuerzburg.de>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 3180 bytes --]
On Mon, 31 Oct 2005, Johannes Schindelin wrote:
>
> How about adding the whole explanation as
>
> git/Documentation/howto/tell-why-cvs-sucks.txt
>
> (maybe with some more polite name)?
Hey, if somebody else does it, that's fine.
I'm personally _so_ biased against CVS that I'm not neutral. I really hate
the thing. I'd much rather use tar-balls and patches than CVS: I think
"quilt" ends up being much nicer in many ways than CVS can be.
So feel free to take my explanations and write something up. I just don't
want to do it, because I fear I might be unfair to CVS (well.. I'm
personally 120% convinced I'm not, but still, there's a lot of people who
actually _use_ it, so..).
> Also, I´d like to add that CVS branching/merging is no good:
>
> <tryingtoputonalbertsshoes>
>
> Sometimes a developer gets an idea, or the need, to implement a certain
> feature to a piece of free software. Now, this idea might seem good, but
> it might take a while to
>
> - implement it,
> - flesh the bugs out, and
> - maybe realize the idea was not all that good.
[ details deleted ]
Yes. I consider this to be part of the centralized/distributed issue, but
it's worth talking about. A distributed system automatically implies that
you can have "throw-away branches" for testing, and you can combine two or
more of these test-branches without ever disturbing (or even _notifying_)
the "real development" branch.
In a centralized system, any branches will always be in that central
repository, so you can't do throw-away stuff without affecting everybody
else. Maybe the actual _code_ won't be in the "real development" branch,
but you're writing and affecting the same repository where the development
happens.
Again, you could replicate the whole repo, and do things there, but then
you can't ever merge again, which in effect makes the small branch a big
fork. So in that sense the CVS/SVN mentality basically encourages forking
and discourages working together. This is more of the same thing that I
alluded to when saying that CVS inevitably leads to "politics" - having a
central place means that you have people who fight for control over it,
even when they'd otherwise _want_ to cooperate.
With git (or with BK, or any truly decentralized model), you just make
your own repo, do your development there, and you never need to ask for
permissions from the central repo people. If the development works out,
you just ask people to merge back. And if it doesn't, you don't even have
to tell people what a total failure you were.
So with the distributed model, you don't have to publicly humiliate
yourself when you do something stupid. Similarly, you don't have to
publicly prove that you're "good enough" before you can play in the
sandbox. In other words, there's no need for politics and egos.
(Now, obviously, when you've actually done the work, if people recognize
that you're a mental giant, they'll be more likely to merge with you, and
you'll generally have an easier time. So I'm not saying that a distributed
model takes away the need for showing how good you are, but it doesn't
make that a central pre-conception).
Linus
^ permalink raw reply
* Re: git objects directory
From: Johannes Schindelin @ 2005-10-31 16:08 UTC (permalink / raw)
To: Lexington Luthor; +Cc: git
In-Reply-To: <dk5dgq$isv$1@sea.gmane.org>
Hi,
On Mon, 31 Oct 2005, Lexington Luthor wrote:
> Can someone please explain why git puts its objects in 256
> sub-directories of objects instead of directly in the objects directory
> itself?
Some filesystems have problems with huge directories. And originally the
plan was to have all objects as files (now the older ones live in packs).
Hth,
Dscho
^ permalink raw reply
* Re: git objects directory
From: Nikolai Weibull @ 2005-10-31 16:08 UTC (permalink / raw)
To: git
In-Reply-To: <dk5dgq$isv$1@sea.gmane.org>
Lexington Luthor wrote:
> Can someone please explain why git puts its objects in 256
> sub-directories of objects instead of directly in the objects directory
> itself? I am trying to learn git (actually cogito), but I still don't
> understand why this structure is used.
Because otherwise there would be a lot of objects (files) in the
objects subdirectory, which would cause a lot of problems with many
filesystems.
Anyway, you best leave that directory be, but I understand your desire
to understand :-).
nikolai
--
Nikolai Weibull: now available free of charge at http://bitwi.se/!
Born in Chicago, IL USA; currently residing in Gothenburg, Sweden.
main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
^ permalink raw reply
* Re: git 0.99.9: Subversion importer breaks RPM generation (rpmbuild bug)
From: H. Peter Anvin @ 2005-10-31 15:57 UTC (permalink / raw)
To: Chris Wright; +Cc: Junio C Hamano, git
In-Reply-To: <20051031064105.GV8041@shell0.pdx.osdl.net>
Chris Wright wrote:
>
> It's fine for FC3. Certain irony that git now effectively requires
> subversion. I'm all for splitting these out, but have no time until
> later in the week. BTW, mind pushing the tag?
>
The git-core RPM definitiely needs to be split. Doubly ironic that it's
called "core".
-hpa
^ permalink raw reply
* git objects directory
From: Lexington Luthor @ 2005-10-31 15:33 UTC (permalink / raw)
To: git
Hi,
Can someone please explain why git puts its objects in 256
sub-directories of objects instead of directly in the objects directory
itself? I am trying to learn git (actually cogito), but I still don't
understand why this structure is used.
What is the point of having the first byte of the hash as the directory
name?
-- LL
^ permalink raw reply
* Re: git versus CVS (versus bk)
From: wa1ter @ 2005-10-31 13:00 UTC (permalink / raw)
To: git
In-Reply-To: <Pine.LNX.4.64.0510301811390.27915@g5.osdl.org>
On Sun, 30 Oct 2005, Linus Torvalds wrote:
> On Sun, 30 Oct 2005, walt wrote:
> > Could someone explain to me the shortcomings of CVS which prompted
> > the development of bk (and then git) -- in a way that a non-developer
> > like me can understand?
> It's really not very easy to explain...
<explanation snipped>
But you explained very well, thank you! And thanks to the others who
responded -- all very helpful. I'm off to read the link that Martin
supplied -- looks gossipy enough to keep me awake :o)
^ permalink raw reply
* Re: git versus CVS (versus bk)
From: Johannes Schindelin @ 2005-10-31 10:24 UTC (permalink / raw)
To: Linus Torvalds; +Cc: walt, git
In-Reply-To: <Pine.LNX.4.64.0510301811390.27915@g5.osdl.org>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1568 bytes --]
Hi,
On Sun, 30 Oct 2005, Linus Torvalds wrote:
> It's really not very easy to explain why CVS sucks. After all, sometimes
> people who have used it for decades have a hard time understanding the
> suckiness.
>
> [...]
How about adding the whole explanation as
git/Documentation/howto/tell-why-cvs-sucks.txt
(maybe with some more polite name)?
Also, I´d like to add that CVS branching/merging is no good:
<tryingtoputonalbertsshoes>
Sometimes a developer gets an idea, or the need, to implement a certain
feature to a piece of free software. Now, this idea might seem good, but
it might take a while to
- implement it,
- flesh the bugs out, and
- maybe realize the idea was not all that good.
All the while, the project is prospering, and you have to keep up-to-date.
With CVS, you would do "cvs update" every once in a while, and clean up
the merge conflicts. In effect, you would track the history of the
upstream project.
Often, however, you would like to track *your* changes, too. This is not
possible in CVS. You just can´t track two different histories in the same
working directory.
Now, if you are working on two or more different ideas, which you want to
test separately *and* together, you need to merge your local branches
every once in a while. If it weren´t for "every once in a while", but
"once", you still could do it in CVS. If you want to merge several times
(keeping the separate development branches), you can´t.
</tryingtoputonalbertsshoesfailingmiserably>
Ciao,
Dscho
^ permalink raw reply
* [PATCH] Several fixes to import mono's svn tree
From: Yaacov Akiba Slama @ 2005-10-31 10:09 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 159 bytes --]
I successfully imported the mono svn tree (in fact a local mirror of it)
which has more that 50000 revisions with the included git-svnimport.
Thanks,
--yas
[-- Attachment #2: 0001-Several-fixes-to-import-mono-s-svn-tree.txt --]
[-- Type: text/plain, Size: 9067 bytes --]
(The mono tree can be found svn://svn.myrealbox.com/source
and a web interface at http://svn.myrealbox.com/viewcvs/)
1) Fix the memory leak (using svn pools).
2) Don't output errors if the following paths appear in the logs :
/, /branches, /tags, taking into account the opt_b and opt_t.
3) Unify the handling of "A" (add) and "R" (replace) actions.
4) Handle better the difference between directory and file using check_path
5) When a branch is using files from several branches, add them as parents of
the commit.
6) Handle the case when the only operation in a revision is to add a tag
- that is - to create a directory called /tags/thetag.
7) Other small fixes.
Signed-off-by: Yaacov Akiba Slama <ya@slamail.org>
---
git-svnimport.perl | 147 ++++++++++++++++++++++++++++++++++++----------------
1 files changed, 101 insertions(+), 46 deletions(-)
applies-to: 8c8ace55eb085e930dcc9482f7cf66f9399374b4
850208bc0981001fb62a9af1dfa14b2fb2aed05c
diff --git a/git-svnimport.perl b/git-svnimport.perl
index 45b6a19..e97f470 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -112,7 +112,9 @@ sub file {
DIR => File::Spec->tmpdir(), UNLINK => 1);
print "... $rev $path ...\n" if $opt_v;
- eval { $self->{'svn'}->get_file($path,$rev,$fh); };
+ my $pool = SVN::Pool->new();
+ eval { $self->{'svn'}->get_file($path,$rev,$fh,$pool); };
+ $pool->clear;
if($@) {
return undef if $@ =~ /Attempted to get checksum/;
die $@;
@@ -258,10 +260,17 @@ EOM
open BRANCHES,">>", "$git_dir/svn2git";
-sub get_file($$$) {
- my($rev,$branch,$path) = @_;
+sub node_kind($$$) {
+ my ($branch, $path, $revision) = @_;
+ my $pool=SVN::Pool->new;
+ my $kind = $svn->{'svn'}->check_path(revert_split_path($branch,$path),$revision,$pool);
+ $pool->clear;
+ return $kind;
+}
+
+sub revert_split_path($$) {
+ my($branch,$path) = @_;
- # revert split_path(), below
my $svnpath;
$path = "" if $path eq "/"; # this should not happen, but ...
if($branch eq "/") {
@@ -272,6 +281,14 @@ sub get_file($$$) {
$svnpath = "$branch_name/$branch/$path";
}
+ return $svnpath
+}
+
+sub get_file($$$) {
+ my($rev,$branch,$path) = @_;
+
+ my $svnpath = revert_split_path($branch,$path);
+
# now get it
my $name;
if($opt_d) {
@@ -319,28 +336,57 @@ sub split_path($$) {
} elsif($path =~ s#^/\Q$branch_name\E/([^/]+)/?##) {
$branch = $1;
} else {
- print STDERR "$rev: Unrecognized path: $path\n";
+ my %no_error = (
+ "/" => 1,
+ "/$tag_name" => 1,
+ "/$branch_name" => 1
+ );
+ print STDERR "$rev: Unrecognized path: $path\n" unless (defined $no_error{$path});
return ()
}
$path = "/" if $path eq "";
return ($branch,$path);
}
-sub copy_subdir($$$$$$) {
+sub branch_rev($$) {
+
+ my ($srcbranch,$uptorev) = @_;
+
+ my $bbranches = $branches{$srcbranch};
+ my @revs = reverse sort { ($a eq 'LAST' ? 0 : $a) <=> ($b eq 'LAST' ? 0 : $b) } keys %$bbranches;
+ my $therev;
+ foreach my $arev(@revs) {
+ next if ($arev eq 'LAST');
+ if ($arev <= $uptorev) {
+ $therev = $arev;
+ last;
+ }
+ }
+ return $therev;
+}
+
+sub copy_path($$$$$$$$) {
# Somebody copied a whole subdirectory.
# We need to find the index entries from the old version which the
# SVN log entry points to, and add them to the new place.
- my($newrev,$newbranch,$path,$oldpath,$rev,$new) = @_;
- my($branch,$srcpath) = split_path($rev,$oldpath);
+ my($newrev,$newbranch,$path,$oldpath,$rev,$node_kind,$new,$parents) = @_;
- my $gitrev = $branches{$branch}{$rev};
+ my($srcbranch,$srcpath) = split_path($rev,$oldpath);
+ my $therev = branch_rev($srcbranch, $rev);
+ my $gitrev = $branches{$srcbranch}{$therev};
unless($gitrev) {
print STDERR "$newrev:$newbranch: could not find $oldpath \@ $rev\n";
return;
}
- print "$newrev:$newbranch:$path: copying from $branch:$srcpath @ $rev\n" if $opt_v;
- $srcpath =~ s#/*$#/#;
+ if ($srcbranch ne $newbranch) {
+ push(@$parents, $branches{$srcbranch}{'LAST'});
+ }
+ print "$newrev:$newbranch:$path: copying from $srcbranch:$srcpath @ $rev\n" if $opt_v;
+ if ($node_kind eq $SVN::Node::dir) {
+ $srcpath =~ s#/*$#/#;
+ }
+
open my $f,"-|","git-ls-tree","-r","-z",$gitrev,$srcpath;
local $/ = "\0";
while(<$f>) {
@@ -348,9 +394,12 @@ sub copy_subdir($$$$$$) {
my($m,$p) = split(/\t/,$_,2);
my($mode,$type,$sha1) = split(/ /,$m);
next if $type ne "blob";
- $p = substr($p,length($srcpath)-1);
- print "... found $path$p ...\n" if $opt_v;
- push(@$new,[$mode,$sha1,$path.$p]);
+ if ($node_kind eq $SVN::Node::dir) {
+ $p = $path . substr($p,length($srcpath)-1);
+ } else {
+ $p = $path;
+ }
+ push(@$new,[$mode,$sha1,$p]);
}
close($f) or
print STDERR "$newrev:$newbranch: could not list files in $oldpath \@ $rev\n";
@@ -359,7 +408,7 @@ sub copy_subdir($$$$$$) {
sub commit {
my($branch, $changed_paths, $revision, $author, $date, $message) = @_;
my($author_name,$author_email,$dest);
- my(@old,@new);
+ my(@old,@new,@parents);
if (not defined $author) {
$author_name = $author_email = "unknown";
@@ -446,6 +495,8 @@ sub commit {
$last_rev = $rev;
}
+ push (@parents, $rev) if defined $rev;
+
my $cid;
if($tag and not %$changed_paths) {
$cid = $rev;
@@ -454,39 +505,31 @@ sub commit {
foreach my $path(@paths) {
my $action = $changed_paths->{$path};
- if ($action->[0] eq "A") {
- my $f = get_file($revision,$branch,$path);
- if($f) {
- push(@new,$f) if $f;
- } elsif($action->[1]) {
- copy_subdir($revision,$branch,$path,$action->[1],$action->[2],\@new);
- } else {
- my $opath = $action->[3];
- print STDERR "$revision: $branch: could not fetch '$opath'\n";
+ if ($action->[0] eq "R") {
+ # refer to a file/tree in an earlier commit
+ push(@old,$path); # remove any old stuff
+ }
+ if(($action->[0] eq "A") || ($action->[0] eq "R")) {
+ my $node_kind = node_kind($branch,$path,$revision);
+ if($action->[1]) {
+ copy_path($revision,$branch,$path,$action->[1],$action->[2],$node_kind,\@new,\@parents);
+ } elsif ($node_kind eq $SVN::Node::file) {
+ my $f = get_file($revision,$branch,$path);
+ if ($f) {
+ push(@new,$f) if $f;
+ } else {
+ my $opath = $action->[3];
+ print STDERR "$revision: $branch: could not fetch '$opath'\n";
+ }
}
} elsif ($action->[0] eq "D") {
push(@old,$path);
} elsif ($action->[0] eq "M") {
- my $f = get_file($revision,$branch,$path);
- push(@new,$f) if $f;
- } elsif ($action->[0] eq "R") {
- # refer to a file/tree in an earlier commit
- push(@old,$path); # remove any old stuff
-
- # ... and add any new stuff
- my($b,$srcpath) = split_path($revision,$action->[1]);
- $srcpath =~ s#/*$#/#;
- open my $F,"-|","git-ls-tree","-r","-z", $branches{$b}{$action->[2]}, $srcpath;
- local $/ = "\0";
- while(<$F>) {
- chomp;
- my($m,$p) = split(/\t/,$_,2);
- my($mode,$type,$sha1) = split(/ /,$m);
- next if $type ne "blob";
- $p = substr($p,length($srcpath)-1);
- push(@new,[$mode,$sha1,$path.$p]);
+ my $node_kind = node_kind($branch,$path,$revision);
+ if ($node_kind eq $SVN::Node::file) {
+ my $f = get_file($revision,$branch,$path);
+ push(@new,$f) if $f;
}
- close($F);
} else {
die "$revision: unknown action '".$action->[0]."' for $path\n";
}
@@ -554,7 +597,6 @@ sub commit {
$pw->close();
my @par = ();
- @par = ("-p",$rev) if defined $rev;
# loose detection of merges
# based on the commit msg
@@ -564,11 +606,17 @@ sub commit {
if ($mparent eq 'HEAD') { $mparent = $opt_o };
if ( -e "$git_dir/refs/heads/$mparent") {
$mparent = get_headref($mparent, $git_dir);
- push @par, '-p', $mparent;
+ push (@parents, $mparent);
print OUT "Merge parent branch: $mparent\n" if $opt_v;
}
}
}
+ my %seen_parents = ();
+ my @unique_parents = grep { ! $seen_parents{$_} ++ } @parents;
+ foreach my $bparent (@unique_parents) {
+ push @par, '-p', $bparent;
+ print OUT "Merge parent branch: $bparent\n" if $opt_v;
+ }
exec("env",
"GIT_AUTHOR_NAME=$author_name",
@@ -600,6 +648,10 @@ sub commit {
die "Error running git-commit-tree: $?\n" if $?;
}
+ if (not defined $cid) {
+ $cid = $branches{"/"}{"LAST"};
+ }
+
if(not defined $dest) {
print "... no known parent\n" if $opt_v;
} elsif(not $tag) {
@@ -616,6 +668,7 @@ sub commit {
# the tag was 'complex', i.e. did not refer to a "real" revision
$dest =~ tr/_/\./ if $opt_u;
+ $branch = $dest;
my $pid = open2($in, $out, 'git-mktag');
print $out ("object $cid\n".
@@ -674,7 +727,9 @@ sub commit_all {
}
while(++$current_rev <= $svn->{'maxrev'}) {
- $svn->{'svn'}->get_log("/",$current_rev,$current_rev,$current_rev,1,1,\&_commit_all,"");
+ my $pool=SVN::Pool->new;
+ $svn->{'svn'}->get_log("/",$current_rev,$current_rev,1,1,1,\&_commit_all,$pool);
+ $pool->clear;
commit_all();
if($opt_l and not --$opt_l) {
print STDERR "Stopping, because there is a memory leak (in the SVN library).\n";
---
@@GIT_VERSION@@
^ permalink raw reply related
* Re: [PATCH] Add support for renaming multiple items at once, and for the destination to be a directory.
From: Peter Eriksen @ 2005-10-31 7:53 UTC (permalink / raw)
To: git
In-Reply-To: <11307435312942-git-send-email-ryan@michonline.com>
On Mon, Oct 31, 2005 at 02:25:31AM -0500, Ryan Anderson wrote:
> Signed-off-by: Ryan Anderson <ryan@michonline.com>
>
> ---
>
> > *1* It strikes me that git rename *could* be friendlier by emulating
> > how "mv" treats the paths parameters (current implementation insists
> > two parameters $src and $dst). What do you think, Ryan?
>
> How does this look?
See commit 1114b26e8f2d06912d855c631e51a4ee8a06c4e2 which adds git-mv.
"It supersedes git-rename by adding functionality to move multiple
files, directories or symlinks into another directory. It also
provides according documentation."
Perhaps you didn't see this commit?
Regards,
Peter
^ permalink raw reply
* [PATCH] Add support for renaming multiple items at once, and for the destination to be a directory.
From: Ryan Anderson @ 2005-10-31 7:25 UTC (permalink / raw)
To: Junio C Hamano, git; +Cc: Ryan Anderson
In-Reply-To: <7vzmp04uot.fsf@assigned-by-dhcp.cox.net>
Signed-off-by: Ryan Anderson <ryan@michonline.com>
---
> *1* It strikes me that git rename *could* be friendlier by emulating
> how "mv" treats the paths parameters (current implementation insists
> two parameters $src and $dst). What do you think, Ryan?
How does this look?
Documentation/git-rename.txt | 1 +
git-rename.perl | 69 +++++++++++++++++++++++++++---------------
2 files changed, 46 insertions(+), 24 deletions(-)
applies-to: 0146495a383d5af298e9c9ef33e1c5d506c63afc
172761431e16af8650f973fc16fae6e74935b422
diff --git a/Documentation/git-rename.txt b/Documentation/git-rename.txt
index 583cb03..17ca558 100644
--- a/Documentation/git-rename.txt
+++ b/Documentation/git-rename.txt
@@ -9,6 +9,7 @@ git-rename - Script used to rename a fil
SYNOPSIS
--------
'git-rename' <source> <destination>
+'git-rename' <source> [source ... ] <destination directory>
DESCRIPTION
-----------
diff --git a/git-rename.perl b/git-rename.perl
index 3b1127b..51dec91 100755
--- a/git-rename.perl
+++ b/git-rename.perl
@@ -8,6 +8,9 @@
use warnings;
use strict;
+require 5.008;
+
+use File::Basename qw(basename);
sub usage($);
@@ -19,40 +22,58 @@ unless ( -d $GIT_DIR && -d $GIT_DIR . "/
usage("Git repository not found.");
}
-usage("") if scalar @ARGV != 2;
+usage("") if scalar @ARGV < 2;
-my ($src,$dst) = @ARGV;
+my $dst = pop @ARGV;
+my @src = @ARGV;
-unless (-f $src || -l $src || -d $src) {
- usage("git rename: bad source '$src'");
+foreach my $src (@src) {
+ unless (-f $src || -l $src || -d $src) {
+ usage("git rename: bad source '$src'");
+ }
}
-if (-e $dst) {
- usage("git rename: destinations '$dst' already exists");
+if (-e $dst && !-d $dst) {
+ usage("git rename: destination '$dst' already exists");
}
-my (@allfiles,@srcfiles,@dstfiles);
-
-$/ = "\0";
-open(F,"-|","git-ls-files","-z")
- or die "Failed to open pipe from git-ls-files: " . $!;
+# Append a "/" if one doesn't exist on the end of $dst, and $dst is
+# a directory.
+$dst .= "/" if -d $dst && $dst !~ m#/$#;
+
+foreach my $src (@src) {
+ my (@allfiles,@srcfiles,@dstfiles);
+ my $actualdst = $dst;
+
+ $/ = "\0";
+ open(F,"-|","git-ls-files","-z")
+ or die "Failed to open pipe from git-ls-files: " . $!;
+
+ @allfiles = map { chomp; $_; } <F>;
+ close(F);
+
+ if (-d $src) {
+ # Remove a trailing / if any
+ $src =~ s#/$##;
+ my $tsrc = basename $src;
+ $actualdst = $dst . $tsrc;
+ }
+
+ my $safesrc = quotemeta($src);
+ @srcfiles = grep /^$safesrc/, @allfiles;
+ @dstfiles = @srcfiles;
+ s#^$safesrc(/|$)#$actualdst$1# for @dstfiles;
-@allfiles = map { chomp; $_; } <F>;
-close(F);
-my $safesrc = quotemeta($src);
-@srcfiles = grep /^$safesrc/, @allfiles;
-@dstfiles = @srcfiles;
-s#^$safesrc(/|$)#$dst$1# for @dstfiles;
+ rename($src,$actualdst)
+ or die "rename($src,$actualdst) failed: $!";
-rename($src,$dst)
- or die "rename failed: $!";
+ my $rc = system("git-update-index","--add","--",@dstfiles);
+ die "git-update-index failed to add new name (related to $src and $dst) with code $?\n" if $rc;
-my $rc = system("git-update-index","--add","--",@dstfiles);
-die "git-update-index failed to add new name with code $?\n" if $rc;
-
-$rc = system("git-update-index","--remove","--",@srcfiles);
-die "git-update-index failed to remove old name with code $?\n" if $rc;
+ $rc = system("git-update-index","--remove","--",@srcfiles);
+ die "git-update-index failed to remove old name (related to $src) with code $?\n" if $rc;
+}
sub usage($) {
---
0.99.9.GIT
^ permalink raw reply related
* Re: git 0.99.9: Subversion importer breaks RPM generation (rpmbuild bug)
From: Junio C Hamano @ 2005-10-31 6:48 UTC (permalink / raw)
To: Chris Wright; +Cc: H. Peter Anvin, git
In-Reply-To: <20051031064105.GV8041@shell0.pdx.osdl.net>
Chris Wright <chrisw@osdl.org> writes:
> It's fine for FC3. Certain irony that git now effectively requires
> subversion.
Haha.
> BTW, mind pushing the tag?
Done; thanks for noticing.
^ permalink raw reply
* Re: git 0.99.9: Subversion importer breaks RPM generation (rpmbuild bug)
From: Chris Wright @ 2005-10-31 6:41 UTC (permalink / raw)
To: Junio C Hamano; +Cc: H. Peter Anvin, git
In-Reply-To: <7vy84ajl4c.fsf@assigned-by-dhcp.cox.net>
* Junio C Hamano (junkio@cox.net) wrote:
> "H. Peter Anvin" <hpa@zytor.com> writes:
>
> > git-svnimport.perl:require v5.8.0; # for shell-safe open("-|",LIST)
> >
> > ... which RPM thinks means that you need a Perl module called v5.8.0
> > which doesn't, of course, exist. This is arguably an rpmbuild bug, but
> > it nevertheless breaks at the moment.
>
> I took Ryan's suggestion and pushed 0.99.9a out. Does it make
> RHEL4 happy?
It's fine for FC3. Certain irony that git now effectively requires
subversion. I'm all for splitting these out, but have no time until
later in the week. BTW, mind pushing the tag?
thanks,
-chris
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox