* Re: [PATCH] cvsimport: rewritten in Perl
From: Nicolas Pitre @ 2005-06-30 17:22 UTC (permalink / raw)
To: Matthias Urlichs; +Cc: Sven Verdoolaege, git
In-Reply-To: <20050630163000.GT10850@kiste.smurf.noris.de>
On Thu, 30 Jun 2005, Matthias Urlichs wrote:
> Duh. Will post an incremental patch shortly.
Until Linus merges it I'd suggest that you post the updated full patch
instead.
Nicolas
^ permalink raw reply
* verify_pack.c vs verify-pack.c
From: Nicolas Pitre @ 2005-06-30 17:41 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
This is confusing. Isn't possible to rename git-verify-pack to, say,
git-fsck-pack instead?
Nicolas
^ permalink raw reply
* "git-send-pack"
From: Linus Torvalds @ 2005-06-30 17:54 UTC (permalink / raw)
To: Git Mailing List; +Cc: Daniel Barkalow, Junio C Hamano, ftpadmin
Ok,
I'm happy to say that the first cut of my new packed-object-sending thing
seems to work. I have successfully sent updates both locally and over ssh,
and it seems to work fine, although it has some limitations.
The syntax is very simple indeed:
git-send-pack destination
will go to the destination (which can be either a local directory or a
remote ssh one, with the remote destination format currently being _only_
the "machine:path" format), and it will go through all the refs in the
remote destination, compare them with the local ones, and create a pack
that updates from one to the other.
If the pack/unpack sequence is successful, it then updates the refs at the
other end, and is done.
My quick tests were very successful, in the sense that it even performed
really well. But I only tested some small updates.
Anyway, what are the limitations? Here's a few obvious ones:
- the code actually contains support for limiting the refs to be updated
on the remote end, but I don't actually pass the arguments to the
remote git-receive-pack binary yet, so this is currently not
functional. Call me lazy.
- the thing currently refuses to create new refs. Again, this is mainly
just me being lazy: it should be easy to add support for creating a new
branch, it just requires some care to make sure that we take the old
branches into account when generating the pack-file so that we don't
send too many objects over.
- I really hate how "ssh" apparently cannot be told to have alternate
paths. For example, on master.kernel.org, I don't control the setup, so
I can't install my own git binaries anywhere except in my ~/bin
directory, but I also cannot get ssh to accept that that is a valid
path. This one really bums me out, and I think it's an ssh deficiency.
You apparently have to compile in the paths at compile-time into sshd,
and PermitUserEnvironment is disabled by default (not that it even
seems to work for the PATH environment, but that may have been my
testing that didn't re-start sshd).
That just sucks.
- It doesn't update the working directory at the other end. This is fine
for what it's intended for (pushing to a central "raw" git archives),
so this could be considered a feature, but it's worth pointing out.
Only a "pull" will update your working directory, and this pack sending
really is meant to be used in a kind of "push to central archive" way.
- this is also (at least once we've tested it a lot more and added the
code to allow it to create new refs on the remote side) meant to be a
good way to mirror things out, since clearly rsync isn't scaling.
However, I don't know what the rules for acceptable mirroring
approaches are, and it's entirely possible (nay, probable) that an ssh
connection from the "master" ain't it. It would be good to know what
(of any) would be acceptable solutions..
Anyway, please do give it a test. I think I'll use this to sync up to
kernel.org, except I _really_ would want to solve that ssh issue some
other way than hardcoding the /home/torvalds/bin/ path in my local
copies.. If somebody knows a good solution, pls holler.
Linus
^ permalink raw reply
* Re: [PATCH] cvsimport-in-Perl: Limit the number of arguments to git-update-cache
From: Matthias Urlichs @ 2005-06-30 18:02 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0506301314280.1667@localhost.localdomain>
[-- Attachment #1: Type: text/plain, Size: 630 bytes --]
Hi,
Nicolas Pitre:
> That example should be:
>
> write( "| xargs git-update-cache --add --", @new)
>
> of course.
>
Actually, 'local $\ = "\0";' and 'xargs -0', which is roughly the point
where doing it all in Perl starts being more readable. ;-)
--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
- -
The only secure computer is one that's unplugged, locked in a safe,
and buried 20 feet under the ground in a secret location... and I'm
not even too sure about that one.
-- Dennis Huges, FBI.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: "git-send-pack"
From: A Large Angry SCM @ 2005-06-30 18:24 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List
In-Reply-To: <Pine.LNX.4.58.0506301025510.14331@ppc970.osdl.org>
Have you tried something like the following?
ssh torvalds@master.kernel.org \
'/bin/sh -c "export PATH=/tmp/foo:$PATH ; env"'
Linus Torvalds wrote:
>
...
>
> Anyway, please do give it a test. I think I'll use this to sync up to
> kernel.org, except I _really_ would want to solve that ssh issue some
> other way than hardcoding the /home/torvalds/bin/ path in my local
> copies.. If somebody knows a good solution, pls holler.
^ permalink raw reply
* Re: "git-send-pack"
From: A Large Angry SCM @ 2005-06-30 18:27 UTC (permalink / raw)
To: gitzilla; +Cc: Linus Torvalds, Git Mailing List
In-Reply-To: <42C438CA.3040507@gmail.com>
Damn! That should have been:
ssh torvalds@master.kernel.org \
'/bin/sh -c "export PATH=~/tmp/foo:$PATH ; env"'
A Large Angry SCM wrote:
> Have you tried something like the following?
>
> ssh torvalds@master.kernel.org \
> '/bin/sh -c "export PATH=/tmp/foo:$PATH ; env"'
>
> Linus Torvalds wrote:
>>
> ...
> >
>> Anyway, please do give it a test. I think I'll use this to sync up to
>> kernel.org, except I _really_ would want to solve that ssh issue some
>> other way than hardcoding the /home/torvalds/bin/ path in my local
>> copies.. If somebody knows a good solution, pls holler.
>
^ permalink raw reply
* Re: "git-send-pack"
From: Jan Harkes @ 2005-06-30 18:45 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List
In-Reply-To: <Pine.LNX.4.58.0506301025510.14331@ppc970.osdl.org>
On Thu, Jun 30, 2005 at 10:54:48AM -0700, Linus Torvalds wrote:
> Anyway, please do give it a test. I think I'll use this to sync up to
> kernel.org, except I _really_ would want to solve that ssh issue some
> other way than hardcoding the /home/torvalds/bin/ path in my local
> copies.. If somebody knows a good solution, pls holler.
I've got a couple of 'export FOO=bar' lines in ~/.bashrc on the
"remote-side" and it looks like they are set correctly when
I do something like "ssh remote.host env".
Jan
^ permalink raw reply
* Re: [PATCH] cvsimport: rewritten in Perl
From: Stephen C. Tweedie @ 2005-06-30 18:48 UTC (permalink / raw)
To: Matthias Urlichs; +Cc: Stephen Tweedie, git
In-Reply-To: <pan.2005.06.28.19.23.08.307486@smurf.noris.de>
Hi,
On Tue, 2005-06-28 at 20:23, Matthias Urlichs wrote:
> That was cvs2git's shell script. Which, by the way, is slow as hell.
I discovered why yesterday --- it's this patch:
commit deb153a75ae1f5eca628a38b911474a69edd242d
...
[PATCH] cvs2git and file permissions
git-cvs2git: propagate mode information
Let cvs checkout in a temporary directory rather than
using the pipe option to avoid loss of mode information.
This has a very unfortunate side effect --- when CVS checks out into a
new directory, it then waits for the time to advance to the next second
so that it can guarantee that future writes to the checked-out file
always change the timestamp from what it stores in CVS/Entries.
And when you're doing repeated checkouts, that means you get one per
second, max. And so the git cvs import was taking one ... second ...
per ... revision ... per ... file, instead of the 8 complete changesets
or better I was getting before the above change.
Reverting the change back to checkout-via-pipe fixed the performance
problem for me, but obviously we're back to losing the mode information.
--Stephen
^ permalink raw reply
* Re: "git-send-pack"
From: Mike Taht @ 2005-06-30 19:01 UTC (permalink / raw)
To: Linus Torvalds
Cc: Git Mailing List, Daniel Barkalow, Junio C Hamano, ftpadmin
In-Reply-To: <Pine.LNX.4.58.0506301025510.14331@ppc970.osdl.org>
> However, I don't know what the rules for acceptable mirroring
> approaches are, and it's entirely possible (nay, probable) that an ssh
> connection from the "master" ain't it. It would be good to know what
> (of any) would be acceptable solutions..
Flute, perhaps
http://www.atm.tut.fi/mad/
or fcast
http://www.inrialpes.fr/planete/people/roca/mcl/mcl.html
> -
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: "git-send-pack"
From: Linus Torvalds @ 2005-06-30 19:04 UTC (permalink / raw)
To: A Large Angry SCM; +Cc: Git Mailing List
In-Reply-To: <42C438CA.3040507@gmail.com>
On Thu, 30 Jun 2005, A Large Angry SCM wrote:
>
> Have you tried something like the following?
>
> ssh torvalds@master.kernel.org \
> '/bin/sh -c "export PATH=/tmp/foo:$PATH ; env"'
The point is that the user does not call "ssh" itself, but git-send-pack
does it automatically.
And that means that git-send-pack will always do the same thing, for any
host it is given. If one host needs a special PATH, that's an effing pain.
However, Kees Cook points out that it's driver error: I set up my PATH in
.bash_profile, and if I just do it in .bashrc instead it all works.
Danke,
Linus
^ permalink raw reply
* Re: [PATCH] cvsimport: rewritten in Perl
From: Sven Verdoolaege @ 2005-06-30 19:38 UTC (permalink / raw)
To: Matthias Urlichs; +Cc: git
In-Reply-To: <20050630161043.GR10850@kiste.smurf.noris.de>
On Thu, Jun 30, 2005 at 06:10:43PM +0200, Matthias Urlichs wrote:
> I'll find it. However, you don't actually need to re-import your
> existing CVS->GIT trees; as long as the dates and the branch names
> match, my script will continue where the other left off.
That seems to work, once I figured out I had to pass in the "-o master"
option (as cvs2git didn't create an "origin" branch).
If you don't, you get a rather cryptic message:
usage: git-read-tree (<sha> | -m [-u] <sha1> [<sha2> [<sha3>]])
read-tree failed: 256
You probably want to check whether the origin branch actually
exists.
Apparently you also need to pass the '-x' option to cvsps.
Otherwise, it won't look at anything new.
It would also be nice if the user could pass extra options
to cvsps (notably '-z').
skimo
^ permalink raw reply
* Re: "git-send-pack"
From: Linus Torvalds @ 2005-06-30 19:44 UTC (permalink / raw)
To: Git Mailing List; +Cc: Daniel Barkalow, Junio C Hamano, ftpadmin
In-Reply-To: <Pine.LNX.4.58.0506301025510.14331@ppc970.osdl.org>
On Thu, 30 Jun 2005, Linus Torvalds wrote:
>
> Anyway, please do give it a test. I think I'll use this to sync up to
> kernel.org
In fact, the most recent push was gone with a
git-send-pack master.kernel.org:/pub/scm/linux/kernel/git/torvalds/git.git
so if the new commit ("Do ref matching on the sender side rather than on
receiver") shows up after the mirrors have caught up, then this thing is
officially in production use..
Linus
^ permalink raw reply
* Re: "git-send-pack"
From: Linus Torvalds @ 2005-06-30 19:42 UTC (permalink / raw)
To: Mike Taht; +Cc: Git Mailing List, Daniel Barkalow, Junio C Hamano, ftpadmin
In-Reply-To: <42C4419C.7000309@timesys.com>
On Thu, 30 Jun 2005, Mike Taht wrote:
>
> > However, I don't know what the rules for acceptable mirroring
> > approaches are, and it's entirely possible (nay, probable) that an ssh
> > connection from the "master" ain't it. It would be good to know what
> > (of any) would be acceptable solutions..
>
> Flute, perhaps
>
> http://www.atm.tut.fi/mad/
Well, I was hoping for something that has git knowledge, since there are
issues like updating objects in the right order.
So "git-send-pack" is nice in many ways: it allows you to update any
number of branches (in particular, it allows you to update just a _subset_
of the branches, which is nice if you have a shared central repository,
and some people have write permissions to some branches but not to
others), but it also allows for efficient unpacking on the receiver side
in a way no "general-purpose" mirror program can really match.
However, that requires the receiver to run a git-aware unpacker (in this
case git-receive-pack). I'm hoping that would be acceptable, I'm just
wondering what kind of safety concerns I'd need to make sure of in order
to make people comfortable running a special receiver program.
So the current approach is very flexible: if the pusher has ssh access, he
can do it. Safe, secure, and no new security issues. And since the only
programs the receiver has to be able to run is two git programs
(git-receive-pack will run git-unpack-objects), maybe it would be ok to
even have "git-receive-pack" as the shell for the receiver side, so that
you don't actually give the mirrorer any shell access at all. But it's
still "push-based" in the sense that it's kernel.org that is doing the
pushing, and that may simply not be acceptable.
Linus
^ permalink raw reply
* Re: "git-send-pack"
From: Daniel Barkalow @ 2005-06-30 19:49 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List, Junio C Hamano, ftpadmin
In-Reply-To: <Pine.LNX.4.58.0506301025510.14331@ppc970.osdl.org>
On Thu, 30 Jun 2005, Linus Torvalds wrote:
> Anyway, what are the limitations? Here's a few obvious ones:
>
> - I really hate how "ssh" apparently cannot be told to have alternate
> paths. For example, on master.kernel.org, I don't control the setup, so
> I can't install my own git binaries anywhere except in my ~/bin
> directory, but I also cannot get ssh to accept that that is a valid
> path. This one really bums me out, and I think it's an ssh deficiency.
>
> You apparently have to compile in the paths at compile-time into sshd,
> and PermitUserEnvironment is disabled by default (not that it even
> seems to work for the PATH environment, but that may have been my
> testing that didn't re-start sshd).
>
> That just sucks.
The easiest thing might be to have a centrally-installed wrapper script
that could run programs installed in your home directory. E.g., if
"git" had a "source ~/.git-env" at the beginning, and your ~/.git-env
fixed your PATH, then "git receive-pack ARGS" should work, for a generic
centrally installed git and special stuff in your home directory.
> - It doesn't update the working directory at the other end. This is fine
> for what it's intended for (pushing to a central "raw" git archives),
> so this could be considered a feature, but it's worth pointing out.
> Only a "pull" will update your working directory, and this pack sending
> really is meant to be used in a kind of "push to central archive" way.
I thought only "resolve" (as part of "fetch") updated your working
directory, so this is completely consistant.
> - this is also (at least once we've tested it a lot more and added the
> code to allow it to create new refs on the remote side) meant to be a
> good way to mirror things out, since clearly rsync isn't scaling.
>
> However, I don't know what the rules for acceptable mirroring
> approaches are, and it's entirely possible (nay, probable) that an ssh
> connection from the "master" ain't it. It would be good to know what
> (of any) would be acceptable solutions..
The right solution probably involves getting each pack file you push to
the mirrors as well as to the master. They'll probably update no less
frequently than you push, and they should go through a series of states
which matches the master, so it's not necessary to have anything smart on
master sending them, and they only have to unpack the files they get (and
update the refs afterward). That should make the cross-system trust
requirements relatively minimal; the mirror can fetch things from master,
and neither side has to allow the other to specify a command line.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: "git-send-pack"
From: Linus Torvalds @ 2005-06-30 20:12 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: Git Mailing List, Junio C Hamano, ftpadmin
In-Reply-To: <Pine.LNX.4.21.0506301403300.30848-100000@iabervon.org>
On Thu, 30 Jun 2005, Daniel Barkalow wrote:
>
> The right solution probably involves getting each pack file you push to
> the mirrors as well as to the master. They'll probably update no less
> frequently than you push, and they should go through a series of states
> which matches the master, so it's not necessary to have anything smart on
> master sending them, and they only have to unpack the files they get (and
> update the refs afterward).
Hmm, yes. That would work, together with just fetching the heads.
It won't _really_ solve the problem, since the pushed pack objects will
grow at a proportional rate to the current objects - it's just a constant
factor (admittedly a potentially fairly _big_ constant factor)
improvement both in size and in number of files.
So the mirroring ends up getting slowly slower and slower as the number of
pack files go up. In contrast, a git-aware thing can be basically
constant-time, and mirroring expense ends up being relative to the size of
the change rather than the size of the repository.
But mirroring just pack-files might solve the problem for the forseeable
future, so..
"git-receive-pack" would need to take a flag to tell it to instead of
unpacking just check the object instead (ie call "git-unpack-object" with
the "-n" flag - it will check that everything looks ok, including the
embedded protecting SHA1 hash), and write it out to the filesystem (as it
comes in) and then rename it to the right place.
Linus
^ permalink raw reply
* Re: "git-send-pack"
From: H. Peter Anvin @ 2005-06-30 20:23 UTC (permalink / raw)
To: Linus Torvalds
Cc: Daniel Barkalow, Git Mailing List, Junio C Hamano, ftpadmin
In-Reply-To: <Pine.LNX.4.58.0506301302410.14331@ppc970.osdl.org>
Linus Torvalds wrote:
>
> It won't _really_ solve the problem, since the pushed pack objects will
> grow at a proportional rate to the current objects - it's just a constant
> factor (admittedly a potentially fairly _big_ constant factor)
> improvement both in size and in number of files.
>
If I've understood this correctly, it's not a constant factor
improvement in the number of files (in the size, yes); it's changing it
from O(t*c) to O(t) where t is number of trees and c is number of
changesets. That's key.
The problem we're having (on kernel.org) right now is that there isn't a
hierarchial time stamp in Unix, so we have to compare on a file-by-file
level. rsync is quite good at discovering an invariant beginning of a
file, but when it comes to a mass of files it has to compare the stamps
on each and every one, each time. It will only descend into a single
file, however, if that file has had its timestamp changed.
For the purposes of rsync, storing the objects in a single append-only
file would be a very efficient method, since the rsync algorithm will
quickly discover an invariant head and only transmit the tail. It's not
ideal, and having something git-aware would be better, but I think it's
really would be nice to have something which also plays well with rsync.
There is a *lot* of infrastructure in rsync which is actually hard to
replicate with another tool (including the server architecture); in many
ways it would be easier to convince the rsync developers to create a
plugin architecture and re-use all that code rather than developing an
equivalent tool from scratch.
-hpa
^ permalink raw reply
* Re: how about this as a basis for git-annotate?
From: Junio C Hamano @ 2005-06-30 20:20 UTC (permalink / raw)
To: lode leroy; +Cc: git
In-Reply-To: <BAY22-F40C3079B41F242AEB407E1FFE30@phx.gbl>
>>>>> "ll" == lode leroy <lode_leroy@hotmail.com> writes:
ll> Still the idea should work.
Multiple parents? Renames? Total rewrites?
To save some time for you to think, here is an excerpt from my
sloooooow one describing how it works. The final line-by-line
blame assignment is done exactly the same way as what you did in
your outline, reading from "diff -u0"; but there are other
things you need to worry about.
You will find at http://members.cox.net/junkio/blame.perl the
full implementation. This is what I have been using to generate
the http://members.cox.net/junkio/Summary.txt file.
I do not need "how about this as a basis" at this point. Basis
is already there and I have verified that it produces correct
results. What I would like to see is for somebody to come up
with an implementation that produces the identical results in
1/20th of the time ;-). That solution might probably involve
not calling diff-tree but instead doing things inside diff-tree.
I donno.
------------
How does this work, and what do we do about merges?
The algorithm considers that the first parent is our main line of
development and treats it somewhat special than other parents. So we
pass on the blame to the first parent if a line has not changed from
it. For lines that have changed from the first parent, we must have
either inherited that change from some other parent, or it could have
been merge conflict resolution edit we did on our own.
The following picture illustrates how we pass on and assign blames.
In the sample, the original O was forked into A and B and then merged
into M. Line 1, 2, and 4 did not change. Line 3 and 5 are changed in
A, and Line 5 and 6 are changed in B. M made its own decision to
resolve merge conflicts at Line 5 to something different from A and B:
A: 1 2 T 4 T 6
/ \
O: 1 2 3 4 5 6 M: 1 2 T 4 M S
\ /
B: 1 2 3 4 S S
In the following picture, each line is annotated with a blame letter.
A lowercase blame (e.g. "a" for "1") means that commit or its ancestor
is the guilty party but we do not know which particular ancestor is
responsible for the change yet. An uppercase blame means that we know
that commit is the guilty party.
First we look at M (the HEAD) and initialize Git::Blame->{LINE} like
this:
M: 1 2 T 4 M S
m m m m m m
That is, we know all lines are results of modification made by some
ancestor of M, so we assign lowercase 'm' to all of them.
Then we examine our first parent A. Throughout the algorithm, we are
always only interested in the lines we are the suspect, but this being
the initial round, we are the suspect for all of them. We notice that
1 2 T 4 are the same as the parent A, so we pass the blame for these
four lines to A. M and S are different from A, so we leave them as
they are (note that we do not immediately take the blame for them):
M: 1 2 T 4 M S
a a a a m m
Next we go on to examine parent B. Again, we are only interested in
the lines we are still the suspect (i.e. M and S). We notice S is
something we inherited from B, so we pass the blame on to it, like
this:
M: 1 2 T 4 M S
a a a a m b
Once we exhausted the parents, we look at the results and take
responsibility for the remaining ones that we are still the suspect:
M: 1 2 T 4 M S
a a a a M b
We are done with M. And we know commits A and B need to be examined
further, so we do them recursively. When we look at A, we again only
look at the lines that A is the suspect:
A: 1 2 T 4 T 6
a a a a M b
Among 1 2 T 4, comparing against its parent O, we notice 1 2 4 are
the same so pass the blame for those lines to O:
A: 1 2 T 4 T 6
o o a o M b
A is a non-merge commit; we have already exhausted the parents and
take responsibility for the remaining ones that A is the suspect:
A: 1 2 T 4 T 6
o o A o M b
We go on like this and the final result would become:
O: 1 2 3 4 5 6
O O A O M B
^ permalink raw reply
* Re: verify_pack.c vs verify-pack.c
From: Junio C Hamano @ 2005-06-30 20:29 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0506301336460.1667@localhost.localdomain>
>>>>> "NP" == Nicolas Pitre <nico@cam.org> writes:
NP> This is confusing. Isn't possible to rename git-verify-pack to, say,
NP> git-fsck-pack instead?
Yes it is possible (I am not attached to the name). The
question is what to name it.
git-fsck-cache (which is not about cache, whose name somehow
implies it is trying to see consistency of the index file, but
in reality which is about the object database --- it should be
renamed to git-fsck-objects) does the same verification when the
pack is placed under objects/pack/, and the way I envision
git-verify/check/fsck-pack to be used is to run it _before_ you
place a pack in your object database; it is like running sha1sum
on a tarball before extracting it.
Also I would like to add some "info" output options to the
git-verify/check/fsck-pack program that:
- lists all the contained SHA1 objects;
- with their type and size;
- with their delta depth (if deltified);
- with the delta-data size for each delta step.
to help optimizing the git-pack-objects.
BTW, is there a filesystem that would house the source to the
core GIT that do not like names that differ only in underscore
and dash?
^ permalink raw reply
* Re: verify_pack.c vs verify-pack.c
From: Nicolas Pitre @ 2005-06-30 20:35 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vy88r1tcp.fsf@assigned-by-dhcp.cox.net>
On Thu, 30 Jun 2005, Junio C Hamano wrote:
> >>>>> "NP" == Nicolas Pitre <nico@cam.org> writes:
>
> NP> This is confusing. Isn't possible to rename git-verify-pack to, say,
> NP> git-fsck-pack instead?
>
> Yes it is possible (I am not attached to the name). The
> question is what to name it.
... or you could rename verify_pack.c to pack-check.c or whatever.
> BTW, is there a filesystem that would house the source to the
> core GIT that do not like names that differ only in underscore
> and dash?
Probably not. It is just weird in a ls output that's all. Certainly
not a big issue.
Nicolas
^ permalink raw reply
* Re: "git-send-pack"
From: Junio C Hamano @ 2005-06-30 20:38 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Daniel Barkalow, Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.58.0506301242470.14331@ppc970.osdl.org>
>>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:
LT> In fact, the most recent push was gone with a
LT> git-send-pack master.kernel.org:/pub/scm/linux/kernel/git/torvalds/git.git
Congrats for a job well done.
Now is there anything for us poor mortals who would want to have
a "pull" support? Logging in via ssh and run send-pack on the
other end is workable but not so pretty ;-).
^ permalink raw reply
* Re: "git-send-pack"
From: Linus Torvalds @ 2005-06-30 20:52 UTC (permalink / raw)
To: H. Peter Anvin
Cc: Daniel Barkalow, Git Mailing List, Junio C Hamano, ftpadmin
In-Reply-To: <42C454B2.6090307@zytor.com>
On Thu, 30 Jun 2005, H. Peter Anvin wrote:
>
> If I've understood this correctly, it's not a constant factor
> improvement in the number of files (in the size, yes); it's changing it
> from O(t*c) to O(t) where t is number of trees and c is number of
> changesets. That's key.
No, it _is_ a constant factor even in number of files, if you just keep
the pack objects around without re-packing them.
Basically, you'd get one new pack-file every time I push. That's better
than getting <n> "raw object" files (where <n> can be anything from just a
couple to several thousand, depending on whether I had pulled things), but
it's still just a constant factor on both number of files and size of
files.
Now, you could re-pack the objects every once in a while: it would force a
whole new "epoch", of course and then the mirrorers would have to fetch
the whole repacked file, but that might be fine. Especially if you stop
re-packing after you've hit a certain size (say, a couple of megs), and
then start on the next pack.
> For the purposes of rsync, storing the objects in a single append-only
> file would be a very efficient method, since the rsync algorithm will
> quickly discover an invariant head and only transmit the tail.
Actually, it won't be "quick" - it will have to read the whole file and do
it's hash window thing.
You _could_ append the pack-files into one single "superpack" file (since
you can figure out where the pack boundaries are), but it would be
extremely big after a while, and rsync would spend all its time doing over
the hash window. You'd definitely be better off with re-packing.
Linus
^ permalink raw reply
* Re: "git-send-pack"
From: Daniel Barkalow @ 2005-06-30 20:49 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List, Junio C Hamano, ftpadmin
In-Reply-To: <Pine.LNX.4.58.0506301302410.14331@ppc970.osdl.org>
On Thu, 30 Jun 2005, Linus Torvalds wrote:
> On Thu, 30 Jun 2005, Daniel Barkalow wrote:
> >
> > The right solution probably involves getting each pack file you push to
> > the mirrors as well as to the master. They'll probably update no less
> > frequently than you push, and they should go through a series of states
> > which matches the master, so it's not necessary to have anything smart on
> > master sending them, and they only have to unpack the files they get (and
> > update the refs afterward).
>
> Hmm, yes. That would work, together with just fetching the heads.
>
> It won't _really_ solve the problem, since the pushed pack objects will
> grow at a proportional rate to the current objects - it's just a constant
> factor (admittedly a potentially fairly _big_ constant factor)
> improvement both in size and in number of files.
>
> So the mirroring ends up getting slowly slower and slower as the number of
> pack files go up. In contrast, a git-aware thing can be basically
> constant-time, and mirroring expense ends up being relative to the size of
> the change rather than the size of the repository.
>
> But mirroring just pack-files might solve the problem for the forseeable
> future, so..
Whenever it gets slow, you could replace all the old packs with a single
new pack containing all the old objects; and master could repack whenever
it has a lot of pack files. That's pretty close to O(n) in change size.
Alternatively, having a reverse-ordered list of pack files would mean that
mirrors could just go through that list until they found one they already
had, and stop there, which would really be O(n).
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: [PATCH] cvsimport: rewritten in Perl
From: Matthias Urlichs @ 2005-06-30 21:00 UTC (permalink / raw)
To: Sven Verdoolaege; +Cc: git
In-Reply-To: <20050630193825.GA17345@pc117b.liacs.nl>
[-- Attachment #1: Type: text/plain, Size: 1372 bytes --]
Hi,
Sven Verdoolaege:
> On Thu, Jun 30, 2005 at 06:10:43PM +0200, Matthias Urlichs wrote:
> > I'll find it. However, you don't actually need to re-import your
> > existing CVS->GIT trees; as long as the dates and the branch names
> > match, my script will continue where the other left off.
>
> That seems to work, once I figured out I had to pass in the "-o master"
> option (as cvs2git didn't create an "origin" branch).
>
Yes -- that's intentional, as the "master" branch is the one you're
going to add your own work to once the improt is finished. Using
"master" as CVS HEAD would mean that incremental imports no longer work.
> If you don't, you get a rather cryptic message:
> usage: git-read-tree (<sha> | -m [-u] <sha1> [<sha2> [<sha3>]])
> read-tree failed: 256
>
> You probably want to check whether the origin branch actually
> exists.
>
I'll add that, thanks.
> Apparently you also need to pass the '-x' option to cvsps.
> Otherwise, it won't look at anything new.
>
Ditto.
> It would also be nice if the user could pass extra options
> to cvsps (notably '-z').
>
Ditto. ;-)
--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
- -
Don't let grass grow on the path of friendship.
-- Blackfoot Indian
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: "git-send-pack"
From: Linus Torvalds @ 2005-06-30 21:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Daniel Barkalow, git
In-Reply-To: <7vll4r1sxz.fsf@assigned-by-dhcp.cox.net>
On Thu, 30 Jun 2005, Junio C Hamano wrote:
>
> Now is there anything for us poor mortals who would want to have
> a "pull" support? Logging in via ssh and run send-pack on the
> other end is workable but not so pretty ;-).
I'm thinking about it. You can't actually do send-pack from the other end,
since send-pack needs to know what the base is, and the base you have may
not even exist in the remote.
So a "git-pull-pack" will follow the objects on the other side until it
hits one we have, and _then_ it can send a nice pack. It's not hard per
se, and some of the problems are actually simpler than git-send-pack, but
it needs more communication (and in order to be efficient you want to not
ping-pong a "do-you-have-it" query every time around).
I also want to make sure that the biggest burden is on the pull side, not
the push side. I have a plan, though.
Linus
^ permalink raw reply
* Re: "git-send-pack"
From: Daniel Barkalow @ 2005-06-30 21:05 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Linus Torvalds, git
In-Reply-To: <7vll4r1sxz.fsf@assigned-by-dhcp.cox.net>
On Thu, 30 Jun 2005, Junio C Hamano wrote:
> >>>>> "LT" == Linus Torvalds <torvalds@osdl.org> writes:
>
> LT> In fact, the most recent push was gone with a
>
> LT> git-send-pack master.kernel.org:/pub/scm/linux/kernel/git/torvalds/git.git
>
> Congrats for a job well done.
>
> Now is there anything for us poor mortals who would want to have
> a "pull" support? Logging in via ssh and run send-pack on the
> other end is workable but not so pretty ;-).
I suspect that I'll be able to merge send-pack/receive-pack with
ssh-push/ssh-pull this evening, and then it'll have the feature of not
caring too much which side your command line is on.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox