* Re: [PATCH] write_sha1_file(): Perform Z_FULL_FLUSH between header and data
From: Junio C Hamano @ 2006-03-08 11:04 UTC (permalink / raw)
To: Sergey Vlasov; +Cc: git
In-Reply-To: <20060308134519.78ea313d.vsu@altlinux.ru>
Sergey Vlasov <vsu@altlinux.ru> writes:
> However, a straight reuse still will not be possible, because
> sha1write_compressed() uses deflateInit(&stream, Z_DEFAULT_COMPRESSION),
> which writes zlib headers around the deflate stream, and the zlib footer
> contains adler32 checksum. So, as a minimum, you will need to
> decompress the object data, calculate its adler32 checksum and write the
> zlib header yourself.
Hmph. Thanks for helping, but it sounds like my original plan
was not useful at all. Probably inflating would be still
cheaper than inflating and then deflating, but it would not be
as cool as a straight copy. Sigh...
^ permalink raw reply
* [PATCH] write_sha1_file(): Perform Z_FULL_FLUSH between header and data
From: Sergey Vlasov @ 2006-03-08 10:45 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Nicolas Pitre, Linus Torvalds
In-Reply-To: <7vzmk1izpa.fsf_-_@assigned-by-dhcp.cox.net>
Data after Z_FULL_FLUSH will be compressed independently of the
header, and could therefore be reused without recompressing when
creating a pack.
---
This passes "make test" and unpacking of the whole git repo with
git-fsck-objects afterwards.
However, a straight reuse still will not be possible, because
sha1write_compressed() uses deflateInit(&stream, Z_DEFAULT_COMPRESSION),
which writes zlib headers around the deflate stream, and the zlib footer
contains adler32 checksum. So, as a minimum, you will need to
decompress the object data, calculate its adler32 checksum and write the
zlib header yourself.
sha1_file.c | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
8b12d9a58e87a4c5b5a2a7b20d06fe29a5afb903
diff --git a/sha1_file.c b/sha1_file.c
index a80d849..34d4da4 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1399,7 +1399,8 @@ int write_sha1_file(void *buf, unsigned
/* Set it up */
memset(&stream, 0, sizeof(stream));
deflateInit(&stream, Z_BEST_COMPRESSION);
- size = deflateBound(&stream, len+hdrlen);
+ /* Additional 6 bytes for the Z_FULL_FLUSH marker */
+ size = deflateBound(&stream, hdrlen) + 6 + deflateBound(&stream, len);
compressed = xmalloc(size);
/* Compress it */
@@ -1412,6 +1413,10 @@ int write_sha1_file(void *buf, unsigned
while (deflate(&stream, 0) == Z_OK)
/* nothing */;
+ /* Flush before data */
+ while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
+ /* nothing */;
+
/* Then the data itself.. */
stream.next_in = buf;
stream.avail_in = len;
--
1.2.GIT
^ permalink raw reply related
* Re: Pulling tags from git.git
From: Andreas Ericsson @ 2006-03-08 10:13 UTC (permalink / raw)
To: gitzilla; +Cc: Junio C Hamano, git
In-Reply-To: <440E5E40.7090700@gmail.com>
A Large Angry SCM wrote:
>
> Why is a "pull" bothering with tags? A "fetch" yes, but not a pull.
>
A pull is a fetch + merge. I said pull because what little I know of
Linus' workflow is the the emails he gets from susbsystem maintainers
are called "pull requests".
>>
>> Tags not meant to be distributed are unannotated, and unannotated tags
>> are kept out of published repos which are always stored at a central
>> server. Everybody synchronize to those central repos, so nobody pulls
>> from each other. Perhaps this is how the kernel devs work too, but if
>> it ever changes the update hook will no longer be able to safeguard
>> from it and the, in my eyes, temporary tags will be distributed in a
>> criss-crossing mesh so no-one will ever know where it came from or who
>> created it or why. I.e. a Bad Thing.
>
>
> The distinction here is not annotated tags or temporary tags but _local_
> tags. _Your_ workflow conventions treat unannotated tags as local tags
> but declaring that unannotated tags can not be pushed is imposing _your_
> conventions on other groups. Just as branch names, themselves, can be
> meaningful, so can tag names.
>
Yes, that's why I said it's better to discourage than to disallow. The
default update-hook is disabled by default and there are comments
aplenty to make it possible even for the most die-hard point-and-click
monkey to be able to comment out the disallowing of unannotated tags.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: [RFH] zlib gurus out there?
From: Johannes Schindelin @ 2006-03-08 9:46 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0603071753370.32577@g5.osdl.org>
Hi,
On Tue, 7 Mar 2006, Linus Torvalds wrote:
> On Tue, 7 Mar 2006, Junio C Hamano wrote:
> > >
> > > No, I don't think that's good. You're only doing a partial deflate, you
> > > can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
> > > buffer, and you don't.
>
> Actually, I misread what you were trying to do, and thought this was the
> inflate phase, not the deflate.
I don't think it matters if it is inflate or deflate. ZLib keeps an
internal state depending on the data. That is the whole reason why the
packing is so good: it uses the redundancy in the data already seen to
construct a codebook. (And that's also the reason why you can't start to
deflate in the middle.)
Ciao,
Dscho
^ permalink raw reply
* Some more cvsimport
From: Rajkumar S @ 2006-03-08 8:49 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 677 bytes --]
Hi all,
Thanks every one for replying to my previous mails and to Smurf for his helpful replies
and patch.
The cvs project I am trying to track has 2 branchs. One head and another for releng_1. The
upstream cvs tree gets updated on both branchs. I also make modifications to both. When I
normally do cvsimport, the releng_1 tree gets updated automatically, but if I have checked
out the releng_1 branch in my local git repository when cvsimport happens the files do not
get updated. I can fix this by checking out the master branch just before cvsimport and
then switching back.
I am attaching a test script to simulate the problem and the fix is commented out.
raj
[-- Attachment #2: git_branch_init.sh --]
[-- Type: application/x-shellscript, Size: 2297 bytes --]
^ permalink raw reply
* Re: Pulling tags from git.git
From: A Large Angry SCM @ 2006-03-08 4:32 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Junio C Hamano, git
In-Reply-To: <440DA82D.3060909@op5.se>
Andreas Ericsson wrote:
> A Large Angry SCM wrote:
>> Andreas Ericsson wrote:
>>
>>> Junio C Hamano wrote:
>>>
>>>> Andreas Ericsson <ae@op5.se> writes:
>>>>
>>>>
>>>>> With the git or git+ssh protocol, tags will be autofollowed
>>>>> when you do a pull (only signed tags, I think). The
>>>>> auto-following is done by detecting tags that are fetched,
>>>>
>>>>
>>>>
>>>> Ah, you are correct. We do not follow lightweight tags; I am
>>>> not sure if we should.
>>>>
>>>
>>> I'm fairly sure we shouldn't. The default update-hook prevents them
>>> (if enabled), and I can't for the life of me think of why anyone
>>> would want to distribute such tags.
>>>
>>> OTOH, preventing unannotated tags from being pushed seems like a
>>> better way than to not have the ability to auto-follow those same
>>> tags. After all, it's better to discourage than to disallow.
>>>
>>
>> Before you do this, please explain why unannotated tags are not
>> useful, and so should not be allowed to be pushed.
>
>
> Imagine Linus, getting his "please pull" emails and doing so only to
> find dozens of temporary tags fetched by the pull. Junio's patch (if I
> read it correctly) unconditionally fetches *ALL* tags reachable from the
> top of the commit-chain, which means there is no longer any way to keep
> temporary tags in a repo from which someone else will pull.
Why is a "pull" bothering with tags? A "fetch" yes, but not a pull.
> I for one riddle my repos with temporary tags whenever I'm trying
> something I'm not so sure of, or find an interesting bug or a design
> decision I'm not 100% sure of. Perhaps I should rather do this with
> branches, but imo branches are for doing work, whereas tags just mark a
> spot in the development so I easily can find them with gitk or some such.
>
> I may be biased by the way we do things at work. In our workflow, all
> tags meant to be distributed have a short note in them which explains
> the rationale of the tag. For example, new versions have a very brief
> changelog that sales-people get on email (a blessing, that, since we
> devs no longer have to update feature-lists and such).
>
> Tags not meant to be distributed are unannotated, and unannotated tags
> are kept out of published repos which are always stored at a central
> server. Everybody synchronize to those central repos, so nobody pulls
> from each other. Perhaps this is how the kernel devs work too, but if it
> ever changes the update hook will no longer be able to safeguard from it
> and the, in my eyes, temporary tags will be distributed in a
> criss-crossing mesh so no-one will ever know where it came from or who
> created it or why. I.e. a Bad Thing.
The distinction here is not annotated tags or temporary tags but _local_
tags. _Your_ workflow conventions treat unannotated tags as local tags
but declaring that unannotated tags can not be pushed is imposing _your_
conventions on other groups. Just as branch names, themselves, can be
meaningful, so can tag names.
^ permalink raw reply
* Re: Problems with using git
From: Greg KH @ 2006-03-08 2:16 UTC (permalink / raw)
To: Mark Wooding; +Cc: git
In-Reply-To: <20060304222557.GA24628@kroah.com>
On Sat, Mar 04, 2006 at 02:25:57PM -0800, Greg KH wrote:
> On Sat, Mar 04, 2006 at 10:56:09AM +0000, Mark Wooding wrote:
> > Greg KH <greg@kroah.com> wrote:
> >
> > > The latest development tree, and the latest public betas contain
> > > 1.1.3. If you think this should be newer, I can easily go poke the
> > > proper people...
> >
> > Given that there's a security issue which got fixed in 1.1.5, I think
> > this is really a bit poor.
> >
> > I notice, by contrast, that Debian had managed to repackage and release
> > a new GIT the day after Junio fixed the bug in the first place. That
> > was more than a month ago.
>
> Fair enough, I'll go poke the proper people now...
They jumped and the cogito and git should be updated to the latest
version in the next update.
thanks,
greg k-h
^ permalink raw reply
* Re: [RFH] zlib gurus out there?
From: Linus Torvalds @ 2006-03-08 2:00 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vslptivbg.fsf@assigned-by-dhcp.cox.net>
On Tue, 7 Mar 2006, Junio C Hamano wrote:
> >
> > No, I don't think that's good. You're only doing a partial deflate, you
> > can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
> > buffer, and you don't.
Actually, I misread what you were trying to do, and thought this was the
inflate phase, not the deflate. Now that I understand what you want,
> So, in short there is no way to create:
>
> hdr part deflated.
> flush.
> data part deflated independently.
>
> and have the current sha1_read_file() not to notice that flush,
Actually, try the patch you already tried, except you'll need to add a
deflateEnd(&stream);
deflateInit(&stream, Z_BEST_COMPRESSION);
.. set up output parameters again ..
and you need to change the initial
size = deflateBound(&stream, len+hdrlen);
to
size = deflateBound(&stream, len) + deflateBound(&stream, hdrlen);
and then you might be ok.
That said, I'm not sure I agree with what you're trying to do.
Linus
^ permalink raw reply
* [PATCH] contrib/git-svn: fix UUID reading w/pre-1.2 svn; fetch args
From: Eric Wong @ 2006-03-08 1:57 UTC (permalink / raw)
To: Yann Dirson; +Cc: GIT list
In-Reply-To: <20060308014207.GA31137@localdomain>
Junio: please don't apply this patch to git.git just yet. It seems fine
to me, but I haven't tested it heavily yet (Yann can help me, I hope :)
I hardly slept the past few days and I may have broken something badly
(it pasts all the tests, though).
---
As a side effect, this should also work better for 'init' off
directories that are no longer in the latest revision of the
repository.
Fix 'fetch' args (<rev>=<commit> options) on brand-new heads
Signed-off-by: Eric Wong <normalperson@yhbt.net>
---
contrib/git-svn/git-svn.perl | 26 ++++++++++++++++++--------
1 files changed, 18 insertions(+), 8 deletions(-)
9f59596bde5bdd68d1a0a116f7383df74966de44
diff --git a/contrib/git-svn/git-svn.perl b/contrib/git-svn/git-svn.perl
index c575883..b8d2b3e 100755
--- a/contrib/git-svn/git-svn.perl
+++ b/contrib/git-svn/git-svn.perl
@@ -162,7 +162,8 @@ sub rebuild {
croak "SVN repository location required: $url\n";
}
$SVN_URL ||= $url;
- $SVN_UUID ||= setup_git_svn();
+ $SVN_UUID ||= $uuid;
+ setup_git_svn();
$latest = $rev;
}
assert_revision_eq_or_unknown($rev, $c);
@@ -226,10 +227,12 @@ sub fetch {
push @svn_co,'--ignore-externals' unless $_no_ignore_ext;
sys(@svn_co, $SVN_URL, $SVN_WC);
chdir $SVN_WC or croak $!;
+ read_uuid();
$last_commit = git_commit($base, @parents);
assert_svn_wc_clean($base->{revision}, $last_commit);
} else {
chdir $SVN_WC or croak $!;
+ read_uuid();
$last_commit = file_to_s("$REV_DIR/$base->{revision}");
}
my @svn_up = qw(svn up);
@@ -275,7 +278,9 @@ sub commit {
fetch();
chdir $SVN_WC or croak $!;
- my $svn_current_rev = svn_info('.')->{'Last Changed Rev'};
+ my $info = svn_info('.');
+ read_uuid($info);
+ my $svn_current_rev = $info->{'Last Changed Rev'};
foreach my $c (@revs) {
my $mods = svn_checkout_tree($svn_current_rev, $c);
if (scalar @$mods == 0) {
@@ -314,6 +319,14 @@ sub show_ignore {
########################### utility functions #########################
+sub read_uuid {
+ return if $SVN_UUID;
+ my $info = shift || svn_info('.');
+ $SVN_UUID = $info->{'Repository UUID'} or
+ croak "Repository UUID unreadable\n";
+ s_to_file($SVN_UUID,"$GIT_DIR/$GIT_SVN/info/uuid");
+}
+
sub setup_git_svn {
defined $SVN_URL or croak "SVN repository location required\n";
unless (-d $GIT_DIR) {
@@ -323,14 +336,10 @@ sub setup_git_svn {
mkpath(["$GIT_DIR/$GIT_SVN/info"]);
mkpath([$REV_DIR]);
s_to_file($SVN_URL,"$GIT_DIR/$GIT_SVN/info/url");
- $SVN_UUID = svn_info($SVN_URL)->{'Repository UUID'} or
- croak "Repository UUID unreadable\n";
- s_to_file($SVN_UUID,"$GIT_DIR/$GIT_SVN/info/uuid");
open my $fd, '>>', "$GIT_DIR/$GIT_SVN/info/exclude" or croak $!;
print $fd '.svn',"\n";
close $fd or croak $!;
- return $SVN_UUID;
}
sub assert_svn_wc_clean {
@@ -860,7 +869,6 @@ sub git_commit {
my ($log_msg, @parents) = @_;
assert_revision_unknown($log_msg->{revision});
my $out_fh = IO::File->new_tmpfile or croak $!;
- $SVN_UUID ||= svn_info('.')->{'Repository UUID'};
map_tree_joins() if (@_branch_from && !%tree_map);
@@ -922,7 +930,9 @@ sub git_commit {
}
my @update_ref = ('git-update-ref',"refs/remotes/$GIT_SVN",$commit);
if (my $primary_parent = shift @exec_parents) {
- push @update_ref, $primary_parent;
+ if (!system('git-rev-parse',"refs/remotes/$GIT_SVN")){
+ push @update_ref, $primary_parent;
+ }
}
sys(@update_ref);
sys('git-update-ref',"$GIT_SVN/revs/$log_msg->{revision}",$commit);
--
1.2.4.g198d
^ permalink raw reply related
* Re: git-svn, tree moves, and --no-stop-on-copy
From: Eric Wong @ 2006-03-08 1:42 UTC (permalink / raw)
To: Yann Dirson; +Cc: GIT list
In-Reply-To: <20060307220837.GB27397@nowhere.earth>
Yann Dirson <ydirson@altern.org> wrote:
> It looks that the --no-stop-on-copy flag has been dropped by error
> during the "options cleanup" commit a couple of days ago. This
> trivial patch appears at first sight to address the problem:
Thanks for the patch, but on second thought...
I'm tempted to drop it as an option... IIRC, the only reason
--no-stop-on-copy exists in git-svn is in case ancient versions of svn
did not support --stop-on-copy. I haven't bothered looking deeply into
SVN history to see if it was always supported or not.
> Before I find out in the doc about --no-stop-on-copy, I did a coupld
> of experimentation. Among them, was using a peg-revision in the URL
> passed to "git-svn init":
>
> $ GIT_SVN_ID=git-oldsvn git-svn init https://svn.sourceforge.net/svnroot/ufoai/trunk/src@165
>
> That succeeds, but then "git-svn fetch" will fail with:
>
> svn: REPORT request failed on '/svnroot/ufoai/!svn/bc/190/trunk/src@165'
> svn: '/svnroot/ufoai/!svn/bc/190/trunk/src@165' path not found
> 256 at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 783
> man::svn_log_raw('https://svn.sourceforge.net/svnroot/ufoai/trunk/src@165', '-r0:HEAD', '--stop-on-copy') called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 219
> main::fetch() called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 81
If you want full repository history for reorganized repositories,
easiest way is to pay the price for full repository and all of its
history.
git-svn init https://svn.sourceforge.net/svnroot/ufoai
git-svn fetch
# this puts all your branches and tags into one single big git tree.
However, the following should always work: (after the following patch,
GIT_SVN_ID=git-oldsvn git-svn init \
https://svn.sourceforge.net/svnroot/ufoai/trunk
GIT_SVN_ID=git-oldsvn git-svn fetch -r1:165
GIT_SVN_ID=git-newsvn git-svn init
https://svn.sourceforge.net/svnroot/ufoai/ufoai/trunk
GIT_SVN_ID=git-newsvn git-svn fetch \
166=`git-rev-parse refs/remotes/git-oldsvn`
Unfortunately, it does not, at least with svn 1.2.3... I have a patch
coming that should fix things for 1.1.1 (and give better 1.1.x support
in general). I'm not sure, but it feels like something is screwed up
with svn 1.2.3dfsg1-3:
This works: svn log -r1 https://svn.sourceforge.net/svnroot/ufoai/trunk
This doesn't: svn co -r1 https://svn.sourceforge.net/svnroot/ufoai/trunk
But this: svn co -r1 https://svn.sourceforge.net/svnroot/ufoai
will create the following structure:
ufoai/{trunk,branches,tags}
I'm quite puzzled about it, as I swear I've seen it work on a different
project recently (of course I cannot remember which :<)
> Maybe git-svn could also be guarded against peg-revisions on init
> command-line, since that appears to confuse it quite a bit :)
Possibly, but having '@' in URLs is valid in some cases outside of
peg-revisions.
> Additionally, it may be worth poiting out in the doc at least one
> valid use of the --no-stop-on-copy flag that is friendly to the user's
> sanity: when the svn repository has undergone a reorg, such that the
> URL passed to "init" indeed moved - at least, when/if it is made to
> work :)
In the face of repository reorgs, git-svn is happiest tracking partial
history. Or tracking the entire repository from the root.
Hopefully I've been reasonably coherent, having insomnia lately.
--
Eric Wong
^ permalink raw reply
* Re: [RFH] zlib gurus out there?
From: Junio C Hamano @ 2006-03-08 1:22 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0603071658300.32577@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> writes:
> On Tue, 7 Mar 2006, Junio C Hamano wrote:
>>
>> However, I am stuck with the first step, which is to do a full
>> flush after the header. An obvious change to the code quoted
>> above writes out a corrupt object:
>>
>> /* First header.. */
>> stream.next_in = hdr;
>> stream.avail_in = hdrlen;
>> - while (deflate(&stream, 0) == Z_OK)
>> + while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
>> /* nothing */;
>
> No, I don't think that's good. You're only doing a partial deflate, you
> can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
> buffer, and you don't.
So, in short there is no way to create:
hdr part deflated.
flush.
data part deflated independently.
and have the current sha1_read_file() not to notice that flush,
while I can inspect the deflated stream to find the "flush", and
copy only the defalted data part into a pack? Bummer... I was
really shooting for full backward compatibility.
^ permalink raw reply
* Re: [RFH] zlib gurus out there?
From: Linus Torvalds @ 2006-03-08 0:59 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Nicolas Pitre
In-Reply-To: <7vzmk1izpa.fsf_-_@assigned-by-dhcp.cox.net>
On Tue, 7 Mar 2006, Junio C Hamano wrote:
>
> However, I am stuck with the first step, which is to do a full
> flush after the header. An obvious change to the code quoted
> above writes out a corrupt object:
>
> /* First header.. */
> stream.next_in = hdr;
> stream.avail_in = hdrlen;
> - while (deflate(&stream, 0) == Z_OK)
> + while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
> /* nothing */;
No, I don't think that's good. You're only doing a partial deflate, you
can't ask for a Z_FULL_FLUSH. That only works if you give it the whole
buffer, and you don't.
iirc, wtf, wdik, and ianal.
Linus
^ permalink raw reply
* [PATCH] Update http-push functionality
From: Nick Hengeveld @ 2006-03-08 0:13 UTC (permalink / raw)
To: git
This brings http-push functionality more in line with the ssh/git version,
by borrowing bits from send-pack and rev-list to process refspecs and
revision history in more standard ways. Also, the status of remote objects
is determined using PROPFIND requests for the object directory rather than
HEAD requests for each object - while it may be less efficient for small
numbers of objects, this approach is able to get the status of all remote
loose objects in a maximum of 256 requests.
---
I've tested this by running "git push --all --tags remote" into a newly
initialized repo, and by rewinding one or multiple overlapping heads and
pushing combinations of them.
This also lays a bit of groundwork for updating remote info/refs.
Getting my head around the underlying object/refspec/revision stuff made
it hurt, and I've got to say that you guys rule.
Makefile | 2
http-push.c | 1060 ++++++++++++++++++++++++++++++++++++++++-------------------
2 files changed, 727 insertions(+), 335 deletions(-)
9d2903e7bf846eeefd3f108207d7c212139ff97c
diff --git a/Makefile b/Makefile
index b6d8804..1c555da 100644
--- a/Makefile
+++ b/Makefile
@@ -526,7 +526,7 @@ git-http-fetch$X: fetch.o http.o http-fe
$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(LIBS) $(CURL_LIBCURL)
-git-http-push$X: http.o http-push.o $(LIB_FILE)
+git-http-push$X: revision.o http.o http-push.o $(LIB_FILE)
$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) \
$(LIBS) $(CURL_LIBCURL) $(EXPAT_LIBEXPAT)
diff --git a/http-push.c b/http-push.c
index fe92560..970cbfe 100644
--- a/http-push.c
+++ b/http-push.c
@@ -5,6 +5,8 @@
#include "tag.h"
#include "blob.h"
#include "http.h"
+#include "refs.h"
+#include "revision.h"
#include <expat.h>
@@ -42,14 +44,24 @@ enum XML_Status {
#define DAV_ACTIVELOCK_OWNER ".prop.lockdiscovery.activelock.owner.href"
#define DAV_ACTIVELOCK_TIMEOUT ".prop.lockdiscovery.activelock.timeout"
#define DAV_ACTIVELOCK_TOKEN ".prop.lockdiscovery.activelock.locktoken.href"
+#define DAV_PROPFIND_RESP ".multistatus.response"
+#define DAV_PROPFIND_NAME ".multistatus.response.href"
+#define DAV_PROPFIND_COLLECTION ".multistatus.response.propstat.prop.resourcetype.collection"
/* DAV request body templates */
-#define PROPFIND_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:prop xmlns:R=\"%s\">\n<D:supportedlock/>\n</D:prop>\n</D:propfind>"
+#define PROPFIND_SUPPORTEDLOCK_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:prop xmlns:R=\"%s\">\n<D:supportedlock/>\n</D:prop>\n</D:propfind>"
+#define PROPFIND_ALL_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:propfind xmlns:D=\"DAV:\">\n<D:allprop/>\n</D:propfind>"
#define LOCK_REQUEST "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n<D:lockinfo xmlns:D=\"DAV:\">\n<D:lockscope><D:exclusive/></D:lockscope>\n<D:locktype><D:write/></D:locktype>\n<D:owner>\n<D:href>mailto:%s</D:href>\n</D:owner>\n</D:lockinfo>"
#define LOCK_TIME 600
#define LOCK_REFRESH 30
+/* bits #0-4 in revision.h */
+
+#define LOCAL (1u << 5)
+#define REMOTE (1u << 6)
+#define PUSHING (1u << 7)
+
static int pushing = 0;
static int aborted = 0;
static char remote_dir_exists[256];
@@ -61,17 +73,19 @@ static int push_verbosely = 0;
static int push_all = 0;
static int force_all = 0;
+static struct object_list *objects = NULL;
+
struct repo
{
char *url;
+ int path_len;
struct packed_git *packs;
};
static struct repo *remote = NULL;
+static struct remote_lock *remote_locks = NULL;
enum transfer_state {
- NEED_CHECK,
- RUN_HEAD,
NEED_PUSH,
RUN_MKCOL,
RUN_PUT,
@@ -82,10 +96,10 @@ enum transfer_state {
struct transfer_request
{
- unsigned char sha1[20];
+ struct object *obj;
char *url;
char *dest;
- struct active_lock *lock;
+ struct remote_lock *lock;
struct curl_slist *headers;
struct buffer buffer;
char filename[PATH_MAX];
@@ -114,14 +128,23 @@ struct xml_ctx
void *userData;
};
-struct active_lock
+struct remote_lock
{
char *url;
char *owner;
char *token;
time_t start_time;
long timeout;
+ int active;
int refreshing;
+ struct remote_lock *next;
+};
+
+struct remote_dentry
+{
+ char *base;
+ char *name;
+ int is_dir;
};
static void finish_request(struct transfer_request *request);
@@ -134,42 +157,9 @@ static void process_response(void *callb
finish_request(request);
}
-static void start_check(struct transfer_request *request)
-{
- char *hex = sha1_to_hex(request->sha1);
- struct active_request_slot *slot;
- char *posn;
-
- request->url = xmalloc(strlen(remote->url) + 55);
- strcpy(request->url, remote->url);
- posn = request->url + strlen(remote->url);
- strcpy(posn, "objects/");
- posn += 8;
- memcpy(posn, hex, 2);
- posn += 2;
- *(posn++) = '/';
- strcpy(posn, hex + 2);
-
- slot = get_active_slot();
- slot->callback_func = process_response;
- slot->callback_data = request;
- curl_easy_setopt(slot->curl, CURLOPT_ERRORBUFFER, request->errorstr);
- curl_easy_setopt(slot->curl, CURLOPT_URL, request->url);
- curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 1);
-
- if (start_active_slot(slot)) {
- request->slot = slot;
- request->state = RUN_HEAD;
- } else {
- request->state = ABORTED;
- free(request->url);
- request->url = NULL;
- }
-}
-
static void start_mkcol(struct transfer_request *request)
{
- char *hex = sha1_to_hex(request->sha1);
+ char *hex = sha1_to_hex(request->obj->sha1);
struct active_request_slot *slot;
char *posn;
@@ -203,7 +193,7 @@ static void start_mkcol(struct transfer_
static void start_put(struct transfer_request *request)
{
- char *hex = sha1_to_hex(request->sha1);
+ char *hex = sha1_to_hex(request->obj->sha1);
struct active_request_slot *slot;
char *posn;
char type[20];
@@ -214,7 +204,7 @@ static void start_put(struct transfer_re
ssize_t size;
z_stream stream;
- unpacked = read_sha1_file(request->sha1, type, &len);
+ unpacked = read_sha1_file(request->obj->sha1, type, &len);
hdrlen = sprintf(hdr, "%s %lu", type, len) + 1;
/* Set it up */
@@ -309,64 +299,90 @@ static void start_move(struct transfer_r
}
}
-static int refresh_lock(struct active_lock *lock)
+static int refresh_lock(struct remote_lock *check_lock)
{
struct active_request_slot *slot;
char *if_header;
char timeout_header[25];
struct curl_slist *dav_headers = NULL;
- int rc = 0;
+ struct remote_lock *lock;
+ int time_remaining;
+ time_t current_time;
- lock->refreshing = 1;
+ /* Refresh all active locks if they're close to expiring */
+ for (lock = remote_locks; lock; lock = lock->next) {
+ if (!lock->active)
+ continue;
- if_header = xmalloc(strlen(lock->token) + 25);
- sprintf(if_header, "If: (<opaquelocktoken:%s>)", lock->token);
- sprintf(timeout_header, "Timeout: Second-%ld", lock->timeout);
- dav_headers = curl_slist_append(dav_headers, if_header);
- dav_headers = curl_slist_append(dav_headers, timeout_header);
+ current_time = time(NULL);
+ time_remaining = lock->start_time + lock->timeout
+ - current_time;
+ if (time_remaining > LOCK_REFRESH)
+ continue;
- slot = get_active_slot();
- curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
- curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null);
- curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url);
- curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK);
- curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+ lock->refreshing = 1;
- if (start_active_slot(slot)) {
- run_active_slot(slot);
- if (slot->curl_result != CURLE_OK) {
- fprintf(stderr, "Got HTTP error %ld\n", slot->http_code);
- } else {
- lock->start_time = time(NULL);
- rc = 1;
+ if_header = xmalloc(strlen(lock->token) + 25);
+ sprintf(if_header, "If: (<opaquelocktoken:%s>)", lock->token);
+ sprintf(timeout_header, "Timeout: Second-%ld", lock->timeout);
+ dav_headers = curl_slist_append(dav_headers, if_header);
+ dav_headers = curl_slist_append(dav_headers, timeout_header);
+
+ slot = get_active_slot();
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_null);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, lock->url);
+ curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (slot->curl_result != CURLE_OK) {
+ fprintf(stderr, "Got HTTP error %ld\n", slot->http_code);
+ lock->active = 0;
+ } else {
+ lock->active = 1;
+ lock->start_time = time(NULL);
+ }
}
+
+ lock->refreshing = 0;
+ curl_slist_free_all(dav_headers);
+ free(if_header);
}
- lock->refreshing = 0;
- curl_slist_free_all(dav_headers);
- free(if_header);
+ if (check_lock)
+ return check_lock->active;
+ else
+ return 0;
+}
- return rc;
+static void release_request(struct transfer_request *request)
+{
+ struct transfer_request *entry = request_queue_head;
+
+ if (request == request_queue_head) {
+ request_queue_head = request->next;
+ } else {
+ while (entry->next != NULL && entry->next != request)
+ entry = entry->next;
+ if (entry->next == request)
+ entry->next = entry->next->next;
+ }
+
+ if (request->url != NULL)
+ free(request->url);
+ free(request);
}
static void finish_request(struct transfer_request *request)
{
- time_t current_time = time(NULL);
- int time_remaining;
-
request->curl_result = request->slot->curl_result;
request->http_code = request->slot->http_code;
request->slot = NULL;
- /* Refresh the lock if it is close to timing out */
- time_remaining = request->lock->start_time + request->lock->timeout
- - current_time;
- if (time_remaining < LOCK_REFRESH && !request->lock->refreshing) {
- if (!refresh_lock(request->lock)) {
- fprintf(stderr, "Unable to refresh remote lock\n");
- aborted = 1;
- }
- }
+ /* Keep locks active */
+ refresh_lock(request->lock);
if (request->headers != NULL)
curl_slist_free_all(request->headers);
@@ -377,27 +393,14 @@ static void finish_request(struct transf
request->url = NULL;
}
- if (request->state == RUN_HEAD) {
- if (request->http_code == 404) {
- request->state = NEED_PUSH;
- } else if (request->curl_result == CURLE_OK) {
- remote_dir_exists[request->sha1[0]] = 1;
- request->state = COMPLETE;
- } else {
- fprintf(stderr, "HEAD %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
- request->curl_result, request->http_code);
- request->state = ABORTED;
- aborted = 1;
- }
- } else if (request->state == RUN_MKCOL) {
+ if (request->state == RUN_MKCOL) {
if (request->curl_result == CURLE_OK ||
request->http_code == 405) {
- remote_dir_exists[request->sha1[0]] = 1;
+ remote_dir_exists[request->obj->sha1[0]] = 1;
start_put(request);
} else {
fprintf(stderr, "MKCOL %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
+ sha1_to_hex(request->obj->sha1),
request->curl_result, request->http_code);
request->state = ABORTED;
aborted = 1;
@@ -407,21 +410,21 @@ static void finish_request(struct transf
start_move(request);
} else {
fprintf(stderr, "PUT %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
+ sha1_to_hex(request->obj->sha1),
request->curl_result, request->http_code);
request->state = ABORTED;
aborted = 1;
}
} else if (request->state == RUN_MOVE) {
if (request->curl_result == CURLE_OK) {
- if (push_verbosely)
- fprintf(stderr,
- "sent %s\n",
- sha1_to_hex(request->sha1));
+ fprintf(stderr, " sent %s\n",
+ sha1_to_hex(request->obj->sha1));
request->state = COMPLETE;
+ request->obj->flags |= REMOTE;
+ release_request(request);
} else {
fprintf(stderr, "MOVE %s failed, aborting (%d/%ld)\n",
- sha1_to_hex(request->sha1),
+ sha1_to_hex(request->obj->sha1),
request->curl_result, request->http_code);
request->state = ABORTED;
aborted = 1;
@@ -429,24 +432,6 @@ static void finish_request(struct transf
}
}
-static void release_request(struct transfer_request *request)
-{
- struct transfer_request *entry = request_queue_head;
-
- if (request == request_queue_head) {
- request_queue_head = request->next;
- } else {
- while (entry->next != NULL && entry->next != request)
- entry = entry->next;
- if (entry->next == request)
- entry->next = entry->next->next;
- }
-
- if (request->url != NULL)
- free(request->url);
- free(request);
-}
-
void fill_active_slots(void)
{
struct transfer_request *request = request_queue_head;
@@ -457,14 +442,12 @@ void fill_active_slots(void)
return;
while (active_requests < max_requests && request != NULL) {
- if (!pushing && request->state == NEED_CHECK) {
- start_check(request);
- curl_multi_perform(curlm, &num_transfers);
- } else if (pushing && request->state == NEED_PUSH) {
- if (remote_dir_exists[request->sha1[0]])
+ if (pushing && request->state == NEED_PUSH) {
+ if (remote_dir_exists[request->obj->sha1[0]] == 1) {
start_put(request);
- else
+ } else {
start_mkcol(request);
+ }
curl_multi_perform(curlm, &num_transfers);
}
request = request->next;
@@ -479,26 +462,34 @@ void fill_active_slots(void)
}
}
-static void add_request(unsigned char *sha1, struct active_lock *lock)
+static void get_remote_object_list(unsigned char parent);
+
+static void add_request(struct object *obj, struct remote_lock *lock)
{
struct transfer_request *request = request_queue_head;
struct packed_git *target;
-
- while (request != NULL && memcmp(request->sha1, sha1, 20))
- request = request->next;
- if (request != NULL)
- return;
- target = find_sha1_pack(sha1, remote->packs);
- if (target)
+ /*
+ * Don't push the object if it's known to exist on the remote
+ * or is already in the request queue
+ */
+ if (remote_dir_exists[obj->sha1[0]] == -1)
+ get_remote_object_list(obj->sha1[0]);
+ if (obj->flags & (REMOTE | PUSHING))
return;
+ target = find_sha1_pack(obj->sha1, remote->packs);
+ if (target) {
+ obj->flags |= REMOTE;
+ return;
+ }
+ obj->flags |= PUSHING;
request = xmalloc(sizeof(*request));
- memcpy(request->sha1, sha1, 20);
+ request->obj = obj;
request->url = NULL;
request->lock = lock;
request->headers = NULL;
- request->state = NEED_CHECK;
+ request->state = NEED_PUSH;
request->next = request_queue_head;
request_queue_head = request;
@@ -698,14 +689,13 @@ static char *quote_ref_url(const char *b
int len, baselen, ch;
baselen = strlen(base);
- len = baselen + 12; /* "refs/heads/" + NUL */
+ len = baselen + 1;
for (cp = ref; (ch = *cp) != 0; cp++, len++)
if (needs_quote(ch))
len += 2; /* extra two hex plus replacement % */
qref = xmalloc(len);
memcpy(qref, base, baselen);
- memcpy(qref + baselen, "refs/heads/", 11);
- for (cp = ref, dp = qref + baselen + 11; (ch = *cp) != 0; cp++) {
+ for (cp = ref, dp = qref + baselen; (ch = *cp) != 0; cp++) {
if (needs_quote(ch)) {
*dp++ = '%';
*dp++ = hex((ch >> 4) & 0xF);
@@ -751,6 +741,27 @@ int fetch_ref(char *ref, unsigned char *
return 0;
}
+static void one_remote_object(const char *hex)
+{
+ unsigned char sha1[20];
+ struct object *obj;
+
+ if (get_sha1_hex(hex, sha1) != 0)
+ return;
+
+ obj = lookup_object(sha1);
+ if (!obj)
+ obj = parse_object(sha1);
+
+ /* Ignore remote objects that don't exist locally */
+ if (!obj)
+ return;
+
+ obj->flags |= REMOTE;
+ if (!object_list_contains(objects, obj))
+ add_object(obj, &objects, NULL, "");
+}
+
static void handle_lockprop_ctx(struct xml_ctx *ctx, int tag_closed)
{
int *lock_flags = (int *)ctx->userData;
@@ -772,7 +783,7 @@ static void handle_lockprop_ctx(struct x
static void handle_new_lock_ctx(struct xml_ctx *ctx, int tag_closed)
{
- struct active_lock *lock = (struct active_lock *)ctx->userData;
+ struct remote_lock *lock = (struct remote_lock *)ctx->userData;
if (tag_closed && ctx->cdata) {
if (!strcmp(ctx->name, DAV_ACTIVELOCK_OWNER)) {
@@ -791,6 +802,57 @@ static void handle_new_lock_ctx(struct x
}
}
+static void one_remote_ref(char *refname);
+static void crawl_remote_refs(char *path);
+
+static void handle_crawl_ref_ctx(struct xml_ctx *ctx, int tag_closed)
+{
+ struct remote_dentry *dentry = (struct remote_dentry *)ctx->userData;
+
+
+ if (tag_closed) {
+ if (!strcmp(ctx->name, DAV_PROPFIND_RESP) && dentry->name) {
+ if (dentry->is_dir) {
+ if (strcmp(dentry->name, dentry->base)) {
+ crawl_remote_refs(dentry->name);
+ }
+ } else {
+ one_remote_ref(dentry->name);
+ }
+ } else if (!strcmp(ctx->name, DAV_PROPFIND_NAME) && ctx->cdata) {
+ dentry->name = xmalloc(strlen(ctx->cdata) -
+ remote->path_len + 1);
+ strcpy(dentry->name,
+ ctx->cdata + remote->path_len);
+ } else if (!strcmp(ctx->name, DAV_PROPFIND_COLLECTION)) {
+ dentry->is_dir = 1;
+ }
+ } else if (!strcmp(ctx->name, DAV_PROPFIND_RESP)) {
+ dentry->name = NULL;
+ dentry->is_dir = 0;
+ }
+}
+
+static void handle_remote_object_list_ctx(struct xml_ctx *ctx, int tag_closed)
+{
+ char *path;
+ char *obj_hex;
+
+ if (tag_closed) {
+ if (!strcmp(ctx->name, DAV_PROPFIND_NAME) && ctx->cdata) {
+ path = ctx->cdata + remote->path_len;
+ if (strlen(path) != 50)
+ return;
+ path += 9;
+ obj_hex = xmalloc(strlen(path));
+ strncpy(obj_hex, path, 2);
+ strcpy(obj_hex + 2, path + 3);
+ one_remote_object(obj_hex);
+ free(obj_hex);
+ }
+ }
+}
+
static void
xml_start_tag(void *userData, const char *name, const char **atts)
{
@@ -848,7 +910,7 @@ xml_cdata(void *userData, const XML_Char
strncpy(ctx->cdata, s, len);
}
-static struct active_lock *lock_remote(char *file, long timeout)
+static struct remote_lock *lock_remote(char *path, long timeout)
{
struct active_request_slot *slot;
struct buffer out_buffer;
@@ -858,14 +920,26 @@ static struct active_lock *lock_remote(c
char *url;
char *ep;
char timeout_header[25];
- struct active_lock *new_lock = NULL;
+ struct remote_lock *lock = remote_locks;
XML_Parser parser = XML_ParserCreate(NULL);
enum XML_Status result;
struct curl_slist *dav_headers = NULL;
struct xml_ctx ctx;
- url = xmalloc(strlen(remote->url) + strlen(file) + 1);
- sprintf(url, "%s%s", remote->url, file);
+ url = xmalloc(strlen(remote->url) + strlen(path) + 1);
+ sprintf(url, "%s%s", remote->url, path);
+
+ /* Make sure the url is not already locked */
+ while (lock && strcmp(lock->url, url)) {
+ lock = lock->next;
+ }
+ if (lock) {
+ free(url);
+ if (refresh_lock(lock))
+ return lock;
+ else
+ return NULL;
+ }
/* Make sure leading directories exist for the remote ref */
ep = strchr(url + strlen(remote->url) + 11, '/');
@@ -921,11 +995,11 @@ static struct active_lock *lock_remote(c
curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_LOCK);
curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
- new_lock = xcalloc(1, sizeof(*new_lock));
- new_lock->owner = NULL;
- new_lock->token = NULL;
- new_lock->timeout = -1;
- new_lock->refreshing = 0;
+ lock = xcalloc(1, sizeof(*lock));
+ lock->owner = NULL;
+ lock->token = NULL;
+ lock->timeout = -1;
+ lock->refreshing = 0;
if (start_active_slot(slot)) {
run_active_slot(slot);
@@ -934,7 +1008,7 @@ static struct active_lock *lock_remote(c
ctx.len = 0;
ctx.cdata = NULL;
ctx.userFunc = handle_new_lock_ctx;
- ctx.userData = new_lock;
+ ctx.userData = lock;
XML_SetUserData(parser, &ctx);
XML_SetElementHandler(parser, xml_start_tag,
xml_end_tag);
@@ -946,7 +1020,7 @@ static struct active_lock *lock_remote(c
fprintf(stderr, "XML error: %s\n",
XML_ErrorString(
XML_GetErrorCode(parser)));
- new_lock->timeout = -1;
+ lock->timeout = -1;
}
}
} else {
@@ -957,23 +1031,26 @@ static struct active_lock *lock_remote(c
free(out_data);
free(in_data);
- if (new_lock->token == NULL || new_lock->timeout <= 0) {
- if (new_lock->token != NULL)
- free(new_lock->token);
- if (new_lock->owner != NULL)
- free(new_lock->owner);
+ if (lock->token == NULL || lock->timeout <= 0) {
+ if (lock->token != NULL)
+ free(lock->token);
+ if (lock->owner != NULL)
+ free(lock->owner);
free(url);
- free(new_lock);
- new_lock = NULL;
+ free(lock);
+ lock = NULL;
} else {
- new_lock->url = url;
- new_lock->start_time = time(NULL);
+ lock->url = url;
+ lock->active = 1;
+ lock->start_time = time(NULL);
+ lock->next = remote_locks;
+ remote_locks = lock;
}
- return new_lock;
+ return lock;
}
-static int unlock_remote(struct active_lock *lock)
+static int unlock_remote(struct remote_lock *lock)
{
struct active_request_slot *slot;
char *lock_token_header;
@@ -1005,15 +1082,171 @@ static int unlock_remote(struct active_l
curl_slist_free_all(dav_headers);
free(lock_token_header);
- if (lock->owner != NULL)
- free(lock->owner);
- free(lock->url);
- free(lock->token);
- free(lock);
+ lock->active = 0;
return rc;
}
+static void crawl_remote_refs(char *path)
+{
+ char *url;
+ struct active_request_slot *slot;
+ struct buffer in_buffer;
+ struct buffer out_buffer;
+ char *in_data;
+ char *out_data;
+ XML_Parser parser = XML_ParserCreate(NULL);
+ enum XML_Status result;
+ struct curl_slist *dav_headers = NULL;
+ struct xml_ctx ctx;
+ struct remote_dentry dentry;
+
+ fprintf(stderr, " %s\n", path);
+
+ dentry.base = path;
+ dentry.name = NULL;
+ dentry.is_dir = 0;
+
+ url = xmalloc(strlen(remote->url) + strlen(path) + 1);
+ sprintf(url, "%s%s", remote->url, path);
+
+ out_buffer.size = strlen(PROPFIND_ALL_REQUEST);
+ out_data = xmalloc(out_buffer.size + 1);
+ snprintf(out_data, out_buffer.size + 1, PROPFIND_ALL_REQUEST);
+ out_buffer.posn = 0;
+ out_buffer.buffer = out_data;
+
+ in_buffer.size = 4096;
+ in_data = xmalloc(in_buffer.size);
+ in_buffer.posn = 0;
+ in_buffer.buffer = in_data;
+
+ dav_headers = curl_slist_append(dav_headers, "Depth: 1");
+ dav_headers = curl_slist_append(dav_headers, "Content-Type: text/xml");
+
+ slot = get_active_slot();
+ curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.size);
+ curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+ curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1);
+ curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (slot->curl_result == CURLE_OK) {
+ ctx.name = xcalloc(10, 1);
+ ctx.len = 0;
+ ctx.cdata = NULL;
+ ctx.userFunc = handle_crawl_ref_ctx;
+ ctx.userData = &dentry;
+ XML_SetUserData(parser, &ctx);
+ XML_SetElementHandler(parser, xml_start_tag,
+ xml_end_tag);
+ XML_SetCharacterDataHandler(parser, xml_cdata);
+ result = XML_Parse(parser, in_buffer.buffer,
+ in_buffer.posn, 1);
+ free(ctx.name);
+
+ if (result != XML_STATUS_OK) {
+ fprintf(stderr, "XML error: %s\n",
+ XML_ErrorString(
+ XML_GetErrorCode(parser)));
+ }
+ }
+ } else {
+ fprintf(stderr, "Unable to start request\n");
+ }
+
+ free(url);
+ free(out_data);
+ free(in_buffer.buffer);
+ curl_slist_free_all(dav_headers);
+}
+
+static void get_remote_object_list(unsigned char parent)
+{
+ char *url;
+ struct active_request_slot *slot;
+ struct buffer in_buffer;
+ struct buffer out_buffer;
+ char *in_data;
+ char *out_data;
+ XML_Parser parser = XML_ParserCreate(NULL);
+ enum XML_Status result;
+ struct curl_slist *dav_headers = NULL;
+ struct xml_ctx ctx;
+ char path[] = "/objects/XX/";
+ static const char hex[] = "0123456789abcdef";
+ unsigned int val = parent;
+
+ path[9] = hex[val >> 4];
+ path[10] = hex[val & 0xf];
+ url = xmalloc(strlen(remote->url) + strlen(path) + 1);
+ sprintf(url, "%s%s", remote->url, path);
+
+ out_buffer.size = strlen(PROPFIND_ALL_REQUEST);
+ out_data = xmalloc(out_buffer.size + 1);
+ snprintf(out_data, out_buffer.size + 1, PROPFIND_ALL_REQUEST);
+ out_buffer.posn = 0;
+ out_buffer.buffer = out_data;
+
+ in_buffer.size = 4096;
+ in_data = xmalloc(in_buffer.size);
+ in_buffer.posn = 0;
+ in_buffer.buffer = in_data;
+
+ dav_headers = curl_slist_append(dav_headers, "Depth: 1");
+ dav_headers = curl_slist_append(dav_headers, "Content-Type: text/xml");
+
+ slot = get_active_slot();
+ curl_easy_setopt(slot->curl, CURLOPT_INFILE, &out_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_INFILESIZE, out_buffer.size);
+ curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, fread_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_FILE, &in_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+ curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 1);
+ curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, DAV_PROPFIND);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, dav_headers);
+
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (slot->curl_result == CURLE_OK) {
+ remote_dir_exists[parent] = 1;
+ ctx.name = xcalloc(10, 1);
+ ctx.len = 0;
+ ctx.cdata = NULL;
+ ctx.userFunc = handle_remote_object_list_ctx;
+ XML_SetUserData(parser, &ctx);
+ XML_SetElementHandler(parser, xml_start_tag,
+ xml_end_tag);
+ XML_SetCharacterDataHandler(parser, xml_cdata);
+ result = XML_Parse(parser, in_buffer.buffer,
+ in_buffer.posn, 1);
+ free(ctx.name);
+
+ if (result != XML_STATUS_OK) {
+ fprintf(stderr, "XML error: %s\n",
+ XML_ErrorString(
+ XML_GetErrorCode(parser)));
+ }
+ } else {
+ remote_dir_exists[parent] = 0;
+ }
+ } else {
+ fprintf(stderr, "Unable to start request\n");
+ }
+
+ free(url);
+ free(out_data);
+ free(in_buffer.buffer);
+ curl_slist_free_all(dav_headers);
+}
+
static int locking_available(void)
{
struct active_request_slot *slot;
@@ -1027,9 +1260,12 @@ static int locking_available(void)
struct xml_ctx ctx;
int lock_flags = 0;
- out_buffer.size = strlen(PROPFIND_REQUEST) + strlen(remote->url) - 2;
+ out_buffer.size =
+ strlen(PROPFIND_SUPPORTEDLOCK_REQUEST) +
+ strlen(remote->url) - 2;
out_data = xmalloc(out_buffer.size + 1);
- snprintf(out_data, out_buffer.size + 1, PROPFIND_REQUEST, remote->url);
+ snprintf(out_data, out_buffer.size + 1,
+ PROPFIND_SUPPORTEDLOCK_REQUEST, remote->url);
out_buffer.posn = 0;
out_buffer.buffer = out_data;
@@ -1085,85 +1321,99 @@ static int locking_available(void)
return lock_flags;
}
-static int is_ancestor(unsigned char *sha1, struct commit *commit)
+static struct object_list **process_blob(struct blob *blob,
+ struct object_list **p,
+ struct name_path *path,
+ const char *name)
{
- struct commit_list *parents;
+ struct object *obj = &blob->object;
- if (parse_commit(commit))
- return 0;
- parents = commit->parents;
- for (; parents; parents = parents->next) {
- if (!memcmp(sha1, parents->item->object.sha1, 20)) {
- return 1;
- } else if (parents->item->object.type == commit_type) {
- if (is_ancestor(
- sha1,
- (struct commit *)&parents->item->object
- ))
- return 1;
- }
+ obj->flags |= LOCAL;
+
+ if (obj->flags & (UNINTERESTING | SEEN))
+ return p;
+
+ obj->flags |= SEEN;
+ return add_object(obj, p, path, name);
+}
+
+static struct object_list **process_tree(struct tree *tree,
+ struct object_list **p,
+ struct name_path *path,
+ const char *name)
+{
+ struct object *obj = &tree->object;
+ struct tree_entry_list *entry;
+ struct name_path me;
+
+ obj->flags |= LOCAL;
+
+ if (obj->flags & (UNINTERESTING | SEEN))
+ return p;
+ if (parse_tree(tree) < 0)
+ die("bad tree object %s", sha1_to_hex(obj->sha1));
+
+ obj->flags |= SEEN;
+ p = add_object(obj, p, NULL, name);
+ me.up = path;
+ me.elem = name;
+ me.elem_len = strlen(name);
+ entry = tree->entries;
+ tree->entries = NULL;
+ while (entry) {
+ struct tree_entry_list *next = entry->next;
+ if (entry->directory)
+ p = process_tree(entry->item.tree, p, &me, entry->name);
+ else
+ p = process_blob(entry->item.blob, p, &me, entry->name);
+ free(entry);
+ entry = next;
}
- return 0;
+ return p;
}
-static void get_delta(unsigned char *sha1, struct object *obj,
- struct active_lock *lock)
+static void get_delta(struct rev_info *revs, struct remote_lock *lock)
{
struct commit *commit;
- struct commit_list *parents;
- struct tree *tree;
- struct tree_entry_list *entry;
+ struct object_list **p = &objects, *pending;
- if (sha1 && !memcmp(sha1, obj->sha1, 20))
- return;
+ while ((commit = get_revision(revs)) != NULL) {
+ p = process_tree(commit->tree, p, NULL, "");
+ commit->object.flags |= LOCAL;
+ if (!(commit->object.flags & UNINTERESTING))
+ add_request(&commit->object, lock);
+ }
- if (aborted)
- return;
+ for (pending = revs->pending_objects; pending; pending = pending->next) {
+ struct object *obj = pending->item;
+ const char *name = pending->name;
- if (obj->type == commit_type) {
- if (push_verbosely)
- fprintf(stderr, "walk %s\n", sha1_to_hex(obj->sha1));
- add_request(obj->sha1, lock);
- commit = (struct commit *)obj;
- if (parse_commit(commit)) {
- fprintf(stderr, "Error parsing commit %s\n",
- sha1_to_hex(obj->sha1));
- aborted = 1;
- return;
+ if (obj->flags & (UNINTERESTING | SEEN))
+ continue;
+ if (obj->type == tag_type) {
+ obj->flags |= SEEN;
+ p = add_object(obj, p, NULL, name);
+ continue;
}
- parents = commit->parents;
- for (; parents; parents = parents->next)
- if (sha1 == NULL ||
- memcmp(sha1, parents->item->object.sha1, 20))
- get_delta(sha1, &parents->item->object,
- lock);
- get_delta(sha1, &commit->tree->object, lock);
- } else if (obj->type == tree_type) {
- if (push_verbosely)
- fprintf(stderr, "walk %s\n", sha1_to_hex(obj->sha1));
- add_request(obj->sha1, lock);
- tree = (struct tree *)obj;
- if (parse_tree(tree)) {
- fprintf(stderr, "Error parsing tree %s\n",
- sha1_to_hex(obj->sha1));
- aborted = 1;
- return;
+ if (obj->type == tree_type) {
+ p = process_tree((struct tree *)obj, p, NULL, name);
+ continue;
}
- entry = tree->entries;
- tree->entries = NULL;
- while (entry) {
- struct tree_entry_list *next = entry->next;
- get_delta(sha1, entry->item.any, lock);
- free(entry->name);
- free(entry);
- entry = next;
+ if (obj->type == blob_type) {
+ p = process_blob((struct blob *)obj, p, NULL, name);
+ continue;
}
- } else if (obj->type == blob_type || obj->type == tag_type) {
- add_request(obj->sha1, lock);
+ die("unknown pending object %s (%s)", sha1_to_hex(obj->sha1), name);
+ }
+
+ while (objects) {
+ if (!(objects->item->flags & UNINTERESTING))
+ add_request(objects->item, lock);
+ objects = objects->next;
}
}
-static int update_remote(unsigned char *sha1, struct active_lock *lock)
+static int update_remote(unsigned char *sha1, struct remote_lock *lock)
{
struct active_request_slot *slot;
char *out_data;
@@ -1218,22 +1468,146 @@ static int update_remote(unsigned char *
return 1;
}
+static struct ref *local_refs, **local_tail;
+static struct ref *remote_refs, **remote_tail;
+
+static int one_local_ref(const char *refname, const unsigned char *sha1)
+{
+ struct ref *ref;
+ int len = strlen(refname) + 1;
+ ref = xcalloc(1, sizeof(*ref) + len);
+ memcpy(ref->new_sha1, sha1, 20);
+ memcpy(ref->name, refname, len);
+ *local_tail = ref;
+ local_tail = &ref->next;
+ return 0;
+}
+
+static void one_remote_ref(char *refname)
+{
+ struct ref *ref;
+ unsigned char remote_sha1[20];
+
+ if (fetch_ref(refname, remote_sha1) != 0) {
+ fprintf(stderr,
+ "Unable to fetch ref %s from %s\n",
+ refname, remote->url);
+ return;
+ }
+
+ int len = strlen(refname) + 1;
+ ref = xcalloc(1, sizeof(*ref) + len);
+ memcpy(ref->old_sha1, remote_sha1, 20);
+ memcpy(ref->name, refname, len);
+ *remote_tail = ref;
+ remote_tail = &ref->next;
+}
+
+static void get_local_heads(void)
+{
+ local_tail = &local_refs;
+ for_each_ref(one_local_ref);
+}
+
+static void get_dav_remote_heads(void)
+{
+ remote_tail = &remote_refs;
+ crawl_remote_refs("refs/");
+}
+
+static int is_zero_sha1(const unsigned char *sha1)
+{
+ int i;
+
+ for (i = 0; i < 20; i++) {
+ if (*sha1++)
+ return 0;
+ }
+ return 1;
+}
+
+static void unmark_and_free(struct commit_list *list, unsigned int mark)
+{
+ while (list) {
+ struct commit_list *temp = list;
+ temp->item->object.flags &= ~mark;
+ list = temp->next;
+ free(temp);
+ }
+}
+
+static int ref_newer(const unsigned char *new_sha1,
+ const unsigned char *old_sha1)
+{
+ struct object *o;
+ struct commit *old, *new;
+ struct commit_list *list, *used;
+ int found = 0;
+
+ /* Both new and old must be commit-ish and new is descendant of
+ * old. Otherwise we require --force.
+ */
+ o = deref_tag(parse_object(old_sha1), NULL, 0);
+ if (!o || o->type != commit_type)
+ return 0;
+ old = (struct commit *) o;
+
+ o = deref_tag(parse_object(new_sha1), NULL, 0);
+ if (!o || o->type != commit_type)
+ return 0;
+ new = (struct commit *) o;
+
+ if (parse_commit(new) < 0)
+ return 0;
+
+ used = list = NULL;
+ commit_list_insert(new, &list);
+ while (list) {
+ new = pop_most_recent_commit(&list, TMP_MARK);
+ commit_list_insert(new, &used);
+ if (new == old) {
+ found = 1;
+ break;
+ }
+ }
+ unmark_and_free(list, TMP_MARK);
+ unmark_and_free(used, TMP_MARK);
+ return found;
+}
+
+static void mark_edge_parents_uninteresting(struct commit *commit)
+{
+ struct commit_list *parents;
+
+ for (parents = commit->parents; parents; parents = parents->next) {
+ struct commit *parent = parents->item;
+ if (!(parent->object.flags & UNINTERESTING))
+ continue;
+ mark_tree_uninteresting(parent->tree);
+ }
+}
+
+static void mark_edges_uninteresting(struct commit_list *list)
+{
+ for ( ; list; list = list->next) {
+ struct commit *commit = list->item;
+
+ if (commit->object.flags & UNINTERESTING) {
+ mark_tree_uninteresting(commit->tree);
+ continue;
+ }
+ mark_edge_parents_uninteresting(commit);
+ }
+}
+
int main(int argc, char **argv)
{
struct transfer_request *request;
struct transfer_request *next_request;
int nr_refspec = 0;
char **refspec = NULL;
- int do_remote_update;
- int new_branch;
- int force_this;
- char *local_ref;
- unsigned char local_sha1[20];
- struct object *local_object = NULL;
- char *remote_ref = NULL;
- unsigned char remote_sha1[20];
- struct active_lock *remote_lock;
- char *remote_path = NULL;
+ struct remote_lock *ref_lock;
+ struct rev_info revs;
int rc = 0;
int i;
@@ -1242,6 +1616,7 @@ int main(int argc, char **argv)
remote = xmalloc(sizeof(*remote));
remote->url = NULL;
+ remote->path_len = 0;
remote->packs = NULL;
argv++;
@@ -1249,7 +1624,7 @@ int main(int argc, char **argv)
char *arg = *argv;
if (*arg == '-') {
- if (!strcmp(arg, "--complete")) {
+ if (!strcmp(arg, "--all")) {
push_all = 1;
continue;
}
@@ -1265,6 +1640,12 @@ int main(int argc, char **argv)
}
if (!remote->url) {
remote->url = arg;
+ char *path = strstr(arg, "//");
+ if (path) {
+ path = index(path+2, '/');
+ if (path)
+ remote->path_len = strlen(path);
+ }
continue;
}
refspec = argv;
@@ -1275,7 +1656,7 @@ int main(int argc, char **argv)
if (!remote->url)
usage(http_push_usage);
- memset(remote_dir_exists, 0, 256);
+ memset(remote_dir_exists, -1, 256);
http_init();
@@ -1293,92 +1674,113 @@ int main(int argc, char **argv)
goto cleanup;
}
- /* Process each refspec */
- for (i = 0; i < nr_refspec; i++) {
- char *ep;
- force_this = 0;
- do_remote_update = 0;
- new_branch = 0;
- local_ref = refspec[i];
- if (*local_ref == '+') {
- force_this = 1;
- local_ref++;
- }
- ep = strchr(local_ref, ':');
- if (ep) {
- remote_ref = ep + 1;
- *ep = 0;
+ /* Get a list of all local and remote heads to validate refspecs */
+ get_local_heads();
+ fprintf(stderr, "Fetching remote heads...\n");
+ get_dav_remote_heads();
+
+ /* match them up */
+ if (!remote_tail)
+ remote_tail = &remote_refs;
+ if (match_refs(local_refs, remote_refs, &remote_tail,
+ nr_refspec, refspec, push_all))
+ return -1;
+ if (!remote_refs) {
+ fprintf(stderr, "No refs in common and none specified; doing nothing.\n");
+ return 0;
+ }
+
+ int ret = 0;
+ int new_refs = 0;
+ struct ref *ref;
+ for (ref = remote_refs; ref; ref = ref->next) {
+ char old_hex[60], *new_hex;
+ if (!ref->peer_ref)
+ continue;
+ if (!memcmp(ref->old_sha1, ref->peer_ref->new_sha1, 20)) {
+ if (push_verbosely || 1)
+ fprintf(stderr, "'%s': up-to-date\n", ref->name);
+ continue;
}
- else
- remote_ref = local_ref;
+
+ if (!force_all &&
+ !is_zero_sha1(ref->old_sha1) &&
+ !ref->force) {
+ if (!has_sha1_file(ref->old_sha1) ||
+ !ref_newer(ref->peer_ref->new_sha1,
+ ref->old_sha1)) {
+ /* We do not have the remote ref, or
+ * we know that the remote ref is not
+ * an ancestor of what we are trying to
+ * push. Either way this can be losing
+ * commits at the remote end and likely
+ * we were not up to date to begin with.
+ */
+ error("remote '%s' is not a strict "
+ "subset of local ref '%s'. "
+ "maybe you are not up-to-date and "
+ "need to pull first?",
+ ref->name,
+ ref->peer_ref->name);
+ ret = -2;
+ continue;
+ }
+ }
+ memcpy(ref->new_sha1, ref->peer_ref->new_sha1, 20);
+ if (is_zero_sha1(ref->new_sha1)) {
+ error("cannot happen anymore");
+ ret = -3;
+ continue;
+ }
+ new_refs++;
+ strcpy(old_hex, sha1_to_hex(ref->old_sha1));
+ new_hex = sha1_to_hex(ref->new_sha1);
+
+ fprintf(stderr, "updating '%s'", ref->name);
+ if (strcmp(ref->name, ref->peer_ref->name))
+ fprintf(stderr, " using '%s'", ref->peer_ref->name);
+ fprintf(stderr, "\n from %s\n to %s\n", old_hex, new_hex);
+
/* Lock remote branch ref */
- if (remote_path)
- free(remote_path);
- remote_path = xmalloc(strlen(remote_ref) + 12);
- sprintf(remote_path, "refs/heads/%s", remote_ref);
- remote_lock = lock_remote(remote_path, LOCK_TIME);
- if (remote_lock == NULL) {
+ ref_lock = lock_remote(ref->name, LOCK_TIME);
+ if (ref_lock == NULL) {
fprintf(stderr, "Unable to lock remote branch %s\n",
- remote_ref);
+ ref->name);
rc = 1;
continue;
}
- /* Resolve local and remote refs */
- if (fetch_ref(remote_ref, remote_sha1) != 0) {
- fprintf(stderr,
- "Remote branch %s does not exist on %s\n",
- remote_ref, remote->url);
- new_branch = 1;
- }
- if (get_sha1(local_ref, local_sha1) != 0) {
- fprintf(stderr, "Error resolving local branch %s\n",
- local_ref);
- rc = 1;
- goto unlock;
- }
-
- /* Find relationship between local and remote */
- local_object = parse_object(local_sha1);
- if (!local_object) {
- fprintf(stderr, "Unable to parse local object %s\n",
- sha1_to_hex(local_sha1));
- rc = 1;
- goto unlock;
- } else if (new_branch) {
- do_remote_update = 1;
- } else {
- if (!memcmp(local_sha1, remote_sha1, 20)) {
- fprintf(stderr,
- "* %s: same as branch '%s' of %s\n",
- local_ref, remote_ref, remote->url);
- } else if (is_ancestor(remote_sha1,
- (struct commit *)local_object)) {
- fprintf(stderr,
- "Remote %s will fast-forward to local %s\n",
- remote_ref, local_ref);
- do_remote_update = 1;
- } else if (force_all || force_this) {
- fprintf(stderr,
- "* %s on %s does not fast forward to local branch '%s', overwriting\n",
- remote_ref, remote->url, local_ref);
- do_remote_update = 1;
- } else {
- fprintf(stderr,
- "* %s on %s does not fast forward to local branch '%s'\n",
- remote_ref, remote->url, local_ref);
- rc = 1;
- goto unlock;
- }
+ /* Set up revision info for this refspec */
+ const char *commit_argv[3];
+ int commit_argc = 2;
+ char *new_sha1_hex = strdup(sha1_to_hex(ref->new_sha1));
+ char *old_sha1_hex = NULL;
+ commit_argv[1] = new_sha1_hex;
+ if (!push_all && !is_zero_sha1(ref->old_sha1)) {
+ old_sha1_hex = xmalloc(42);
+ sprintf(old_sha1_hex, "^%s",
+ sha1_to_hex(ref->old_sha1));
+ commit_argv[2] = old_sha1_hex;
+ commit_argc++;
+ }
+ revs.commits = NULL;
+ setup_revisions(commit_argc, commit_argv, &revs, NULL);
+ revs.tag_objects = 1;
+ revs.tree_objects = 1;
+ revs.blob_objects = 1;
+ free(new_sha1_hex);
+ if (old_sha1_hex) {
+ free(old_sha1_hex);
+ commit_argv[1] = NULL;
}
- /* Generate and check list of required objects */
+ /* Generate a list of objects that need to be pushed */
pushing = 0;
- if (do_remote_update || push_all)
- fetch_indices();
- get_delta(push_all ? NULL : remote_sha1,
- local_object, remote_lock);
+ prepare_revision_walk(&revs);
+ mark_edges_uninteresting(revs.commits);
+ fetch_indices();
+ get_delta(&revs, ref_lock);
finish_all_active_slots();
/* Push missing objects to remote, this would be a
@@ -1388,25 +1790,15 @@ int main(int argc, char **argv)
finish_all_active_slots();
/* Update the remote branch if all went well */
- if (do_remote_update) {
- if (!aborted && update_remote(local_sha1,
- remote_lock)) {
- fprintf(stderr, "%s remote branch %s\n",
- new_branch ? "Created" : "Updated",
- remote_ref);
- } else {
- fprintf(stderr,
- "Unable to %s remote branch %s\n",
- new_branch ? "create" : "update",
- remote_ref);
- rc = 1;
- goto unlock;
- }
+ if (aborted || !update_remote(ref->new_sha1, ref_lock)) {
+ rc = 1;
+ goto unlock;
}
unlock:
- unlock_remote(remote_lock);
- free(remote_path);
+ if (!rc)
+ fprintf(stderr, " done\n");
+ unlock_remote(ref_lock);
}
cleanup:
--
1.2.4.gb477-dirty
^ permalink raw reply related
* [RFH] zlib gurus out there?
From: Junio C Hamano @ 2006-03-07 23:48 UTC (permalink / raw)
To: git; +Cc: Nicolas Pitre, Linus Torvalds
In-Reply-To: <Pine.LNX.4.64.0602250012230.31162@localhost.localdomain>
I've been staring at reusing existing data while packing, and
this occurred to me...
During packing, suppose that we chose to store an object in
base form, undeltified. And also suppose we have that object
loose in .git/objects/??/ directory. We already have it in
deflated form, but with its own header. I started wondering if
we can somehow reuse this.
A short object format brush-up lesson is in order here.
* An undeltified object in a pack is represented like this:
(1) the header is a dense variable size binary data, that
encodes type and inflated length;
(2) deflated data immediately follows the header.
* On the other hand, a loose object is represented like this:
(1) the header looks like sprintf("%s %lu%c", type, len, 0);
(2) concatenate the data to the header;
(3) SHA1 checksum of the above becomes the object name.
(4) deflate the header and data using the same z_stream, in two
steps, like this (sha1_file.c::write_sha1_file):
/* Compress it */
stream.next_out = compressed;
stream.avail_out = size;
/* First header.. */
stream.next_in = hdr;
stream.avail_in = hdrlen;
while (deflate(&stream, 0) == Z_OK)
/* nothing */;
/* Then the data itself.. */
stream.next_in = buf;
stream.avail_in = len;
while (deflate(&stream, Z_FINISH) == Z_OK)
/* nothing */;
deflateEnd(&stream);
size = stream.total_out;
So I thought... if we cause a full flush after the header part,
I can find the flush boundaries from a loose object file and
copy the rest into a packfile I am generating, after placing the
binary encoded header. If this works, we do not have to inflate
loose object to read it and deflate it to store that in the
pack. We will get a better packing as well, since we deflate
loose objects with Z_BEST_COMPRESSION, while packs are done with
Z_DEFAULT_COMPRESSION. While pack-objects read from a loose
object, if we can detect that there is no full flush after the
header, we would do the traditional inflate-deflate cycle, so
this would be backward compatible.
However, I am stuck with the first step, which is to do a full
flush after the header. An obvious change to the code quoted
above writes out a corrupt object:
/* First header.. */
stream.next_in = hdr;
stream.avail_in = hdrlen;
- while (deflate(&stream, 0) == Z_OK)
+ while (deflate(&stream, Z_FULL_FLUSH) == Z_OK)
/* nothing */;
git-fsck-objects complains that sha1 does not match. It appears
that the sha1_file.c::unpack_sha1_rest() somehow barfs upon
seeing the full flush, but I haven't dug into it yet.
Would anybody with more experience with zlib want to help?
^ permalink raw reply
* git-svn, tree moves, and --no-stop-on-copy
From: Yann Dirson @ 2006-03-07 22:08 UTC (permalink / raw)
To: Eric Wong; +Cc: GIT list
It looks that the --no-stop-on-copy flag has been dropped by error
during the "options cleanup" commit a couple of days ago. This
trivial patch appears at first sight to address the problem:
diff --git a/contrib/git-svn/git-svn.perl b/contrib/git-svn/git-svn.perl
index 3c860e4..91a0d3b 100755
--- a/contrib/git-svn/git-svn.perl
+++ b/contrib/git-svn/git-svn.perl
@@ -36,7 +36,9 @@ my %fc_opts = ( 'no-ignore-externals' =>
'authors-file|A=s' => \$_authors );
my %cmd = (
fetch => [ \&fetch, "Download new revisions from SVN",
- { 'revision|r=s' => \$_revision, %fc_opts } ],
+ { 'revision|r=s' => \$_revision,
+ 'no-stop-on-copy' => \$_no_stop_copy,
+ %fc_opts } ],
init => [ \&init, "Initialize and fetch (import)", { } ],
commit => [ \&commit, "Commit git revisions to SVN",
{ 'stdin|' => \$_stdin,
But then, "git-svn fetch" fails when trying to deal with the renaming:
$ git-svn.perl init https://svn.sourceforge.net/svnroot/ufoai/ufoai/trunk/src
$ git-svn.perl fetch --no-stop-on-copy
[...]
Updated to revision 163.
r163 = 68da9a01b2d5a2372262ada33a401e03aef41e97
svn: REPORT request failed on '/svnroot/ufoai/!svn/vcc/default'
svn: Cannot replace a directory from within
256 at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 812
main::sys('svn', 'up', '--ignore-externals', '-r166') called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 243
main::fetch() called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 83
The impacted revision in svn is:
------------------------------------------------------------------------
r166 | ydirson | 2006-03-02 01:36:55 +0100 (Thu, 02 Mar 2006) | 1 line
Changed paths:
D /trunk
A /ufoai/trunk (from /trunk:165)
Adding an ufoai toplevel module
------------------------------------------------------------------------
Obviously "svn update" does not like it at all.
"svn switch --relocate" does not seem to be of any help. Switching
manually .git/git-svn/tree/ to the new repository location does not
help either, since I must obviously update to r166 in that case, and
then a further "git-svn fetch" fails because it does not find
.git/git-svn/revs/166 aleady imported.
Any idea as to how to get the work done ?
I could surely import in another branch not using --no-stop-on-copy,
and then use a graft, but if we could make it work in a more
streamlined manner, it would surely be useful.
Before I find out in the doc about --no-stop-on-copy, I did a coupld
of experimentation. Among them, was using a peg-revision in the URL
passed to "git-svn init":
$ GIT_SVN_ID=git-oldsvn git-svn init https://svn.sourceforge.net/svnroot/ufoai/trunk/src@165
That succeeds, but then "git-svn fetch" will fail with:
svn: REPORT request failed on '/svnroot/ufoai/!svn/bc/190/trunk/src@165'
svn: '/svnroot/ufoai/!svn/bc/190/trunk/src@165' path not found
256 at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 783
main::svn_log_raw('https://svn.sourceforge.net/svnroot/ufoai/trunk/src@165', '-r0:HEAD', '--stop-on-copy') called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 219
main::fetch() called at /export/work/yann/git/git/contrib/git-svn/git-svn.perl line 81
Maybe git-svn could also be guarded against peg-revisions on init
command-line, since that appears to confuse it quite a bit :)
Additionally, it may be worth poiting out in the doc at least one
valid use of the --no-stop-on-copy flag that is friendly to the user's
sanity: when the svn repository has undergone a reorg, such that the
URL passed to "init" indeed moved - at least, when/if it is made to
work :)
--
Yann Dirson <ydirson@altern.org> |
Debian-related: <dirson@debian.org> | Support Debian GNU/Linux:
| Freedom, Power, Stability, Gratis
http://ydirson.free.fr/ | Check <http://www.debian.org/>
^ permalink raw reply related
* Re: What's in git.git
From: Francis Daly @ 2006-03-07 22:23 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
> I'd like asciidoc tweaks in "next" by Francis Daly tested by
> people who have access to different vintages of docbook-xsl by
> trying to build manpages. Look for displayed examples, such as
> the one in git-branch documentation. I've tried it with v1.68
> and getting far better results than before, and Francis says
> v1.69 works fine with or without the change. IOW this is a
> workaround for a problem in v1.68.
For completeness / comparison:
I have docbook-xsl v1.68 (strictly, 1.68.1-0.1 as packaged in debian
stable) and docbook-xsl v1.69 (1.69.1 freshly downloaded with the
manpages/ChangeLog listing the last change on 2005-08-11).
I have asciidoc 7.1.1 (local install) and xmlto version 0.0.18 as packaged
in debian stable.
Also:
$ xsltproc -V
Using libxml 20616, libxslt 10112 and libexslt 810
xsltproc was compiled against libxml 20616, libxslt 10112 and libexslt 810
libxslt 10112 was compiled against libxml 20616
libexslt 810 was compiled against libxml 20616
Of that lot, I believe only the docbook-xsl version actually matters
for this test, but I'm happy to learn otherwise.
I run
asciidoc -b docbook -d manpage -f asciidoc.conf git-branch.txt
xmlto man git-branch.xml
asciidoc -b xhtml11 -d manpage -f asciidoc.conf git-branch.txt
I vary the docbook-xsl version being used and I vary the asciidoc.conf
file to include or exclude the "literallayout" change.
The four html files produced differ only in the "Last updated" time.
The two man pages produced with v1.69 are identical.
The two man pages produced with v1.68 differ in the expected and intended
ways, replacing ".IP" with a ".nf/.fi" pair in two places.
And the differences between 1.68 and 1.69 with the old configuration, and
1.68 and 1.69 with the new configuration, are also confined to that area.
Repeating the tests on git-clone.txt (which also has a multiline example)
gets the same result.
And doing the test on git-mv.txt (with no multiline example) shows
that the asciidoc.conf change has no effect on pages like that. No real
surprise there.
So I'm happy that the (fixed) change improves something and breaks
nothing, at least for those versions of the tools.
f
--
Francis Daly francis@daoine.org
^ permalink raw reply
* Re: [PATCH] gitk : fix missing lines when displaying diff.
From: Paul Mackerras @ 2006-03-07 22:17 UTC (permalink / raw)
To: Robert Fitzsimons; +Cc: Jacob Kroon, git
In-Reply-To: <20060307204648.GA13620@localhost>
Robert Fitzsimons writes:
> Lines which are added (removed) and begin with +++ (---) are not shown
> correctly when displaying in the diff panel.
>
> The +++ and --- lines are part of the diff header, with the +++
> indicating the end of the header.
Thanks for pointing out the easy fix. I had been going to do
something much more complicated. :)
However, your patch didn't apply due to the whitespace being munged.
I committed an equivalent fix, but I took the opportunity to change
the [regexp] to a couple of [string compare]s as well.
Paul.
^ permalink raw reply
* [PATCH] gitk : fix missing lines when displaying diff.
From: Robert Fitzsimons @ 2006-03-07 20:46 UTC (permalink / raw)
To: Jacob Kroon; +Cc: paulus, git
In-Reply-To: <440DE262.7020007@gmail.com>
Lines which are added (removed) and begin with +++ (---) are not shown
correctly when displaying in the diff panel.
The +++ and --- lines are part of the diff header, with the +++
indicating the end of the header.
Signed-off-by: Robert Fitzsimons <robfitz@273k.net>
---
gitk | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
d0da9f04cf565df64a954e1bfd4c5c0913b83322
diff --git a/gitk b/gitk
index f4c6624..5e3c34c 100755
--- a/gitk
+++ b/gitk
@@ -2595,8 +2595,10 @@ proc getblobdiffline {bdf ids} {
set pad [string range "----------------------------------------" 1 $l]
$ctext insert end "$pad $header $pad\n" filesep
set diffinhdr 1
- } elseif {[regexp {^(---|\+\+\+)} $line]} {
- set diffinhdr 0
+ } elseif {$diffinhdr == 1 && [regexp {^(---|\+\+\+) } $line]} {
+ if {[regexp {^\+\+\+ } $line]} {
+ set diffinhdr 0
+ }
} elseif {[regexp {^@@ -([0-9]+),([0-9]+) \+([0-9]+),([0-9]+) @@(.*)} \
$line match f1l f1c f2l f2c rest]} {
$ctext insert end "$line\n" hunksep
--
1.2.3.g20d2
^ permalink raw reply related
* Re: [PATCH] Allow git-repack to optionally run git-prune-packed.
From: Alex Riesen @ 2006-03-07 21:29 UTC (permalink / raw)
To: Martin Atukunda; +Cc: git
In-Reply-To: <11417445722524-git-send-email-matlads@dsmagic.com>
Martin Atukunda, Tue, Mar 07, 2006 16:16:12 +0100:
> +-p::
> + Run `git-prune-packed` after packing, see
> + gitlink:git-prune-packed[1]
> +
Maybe just make "-d" work? I.e. "git repack -a -d" repacks and prunes
everything, and "git repack -d" prunes just what was packed
incrementally.
Something like this:
diff --git a/git-repack.sh b/git-repack.sh
index 3d6fec1..be6c7ab 100755
--- a/git-repack.sh
+++ b/git-repack.sh
@@ -74,6 +74,8 @@ then
esac
done
)
+ else
+ git-prune-packed
fi
fi
^ permalink raw reply related
* [PATCH] ls-files: add --abbrev[=<n>] option
From: Eric Wong @ 2006-03-07 19:59 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <20060307195826.GA26870@localdomain>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
---
Documentation/git-ls-files.txt | 7 ++++++-
ls-files.c | 19 +++++++++++++++++--
2 files changed, 23 insertions(+), 3 deletions(-)
1aa70bab7a4af7b79db3c86b9f88884c1dfad6ee
diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt
index e813f84..68d32a8 100644
--- a/Documentation/git-ls-files.txt
+++ b/Documentation/git-ls-files.txt
@@ -16,7 +16,7 @@ SYNOPSIS
[-X <file>|--exclude-from=<file>]
[--exclude-per-directory=<file>]
[--error-unmatch]
- [--full-name] [--] [<file>]\*
+ [--full-name] [--abbrev] [--] [<file>]\*
DESCRIPTION
-----------
@@ -98,6 +98,11 @@ OPTIONS
option forces paths to be output relative to the project
top directory.
+--abbrev[=<n>]::
+ Instead of showing the full 40-byte hexadecimal object
+ lines, show only handful hexdigits prefix.
+ Non default number of digits can be specified with --abbrev=<n>.
+
--::
Do not interpret any more arguments as options.
diff --git a/ls-files.c b/ls-files.c
index df25c8c..585f6a7 100644
--- a/ls-files.c
+++ b/ls-files.c
@@ -11,6 +11,7 @@
#include "cache.h"
#include "quote.h"
+static int abbrev = 0;
static int show_deleted = 0;
static int show_cached = 0;
static int show_others = 0;
@@ -488,7 +489,8 @@ static void show_ce_entry(const char *ta
printf("%s%06o %s %d\t",
tag,
ntohl(ce->ce_mode),
- sha1_to_hex(ce->sha1),
+ abbrev ? find_unique_abbrev(ce->sha1,abbrev)
+ : sha1_to_hex(ce->sha1),
ce_stage(ce));
write_name_quoted("", 0, ce->name + offset,
line_terminator, stdout);
@@ -629,7 +631,8 @@ static void verify_pathspec(void)
static const char ls_files_usage[] =
"git-ls-files [-z] [-t] [-v] (--[cached|deleted|others|stage|unmerged|killed|modified])* "
"[ --ignored ] [--exclude=<pattern>] [--exclude-from=<file>] "
- "[ --exclude-per-directory=<filename> ] [--full-name] [--] [<file>]*";
+ "[ --exclude-per-directory=<filename> ] [--full-name] [--abbrev] "
+ "[--] [<file>]*";
int main(int argc, const char **argv)
{
@@ -736,6 +739,18 @@ int main(int argc, const char **argv)
error_unmatch = 1;
continue;
}
+ if (!strncmp(arg, "--abbrev=", 9)) {
+ abbrev = strtoul(arg+9, NULL, 10);
+ if (abbrev && abbrev < MINIMUM_ABBREV)
+ abbrev = MINIMUM_ABBREV;
+ else if (abbrev > 40)
+ abbrev = 40;
+ continue;
+ }
+ if (!strcmp(arg, "--abbrev")) {
+ abbrev = DEFAULT_ABBREV;
+ continue;
+ }
if (*arg == '-')
usage(ls_files_usage);
break;
--
1.2.4.gc279
^ permalink raw reply related
* Re: [PATCH] ls-tree: add --abbrev[=<n>] option
From: Eric Wong @ 2006-03-07 19:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vzmk2jfzr.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> wrote:
> Output from ls-tree and ls-files -s are primarily for script
> consumption. diff-raw output which already has abbrev is also
> primarily for scripts, but we have an excuse there that human
> eyes can spot differences easier in abbreviated form and two
> values on the same row being different is the only information
> we are conveying in "git whatchanged" default output, and not
> abbreviating the object names makes pathnames pushed to far
> right of the display which will be chopped by "less -S".
>
> So I am not quite sure if I should take this, and a subsequent
> one you would probably send for ls-files ;-).
I've never used ls-files with -s, but it's a simple patch on its way :>
> Care to point out a use case I might have missed that this is
> useful? I've never done ls-tree to be read by _me_ (not by
> script) unless I am debugging ls-tree itself.
I'm a weirdo and I use vim as a repository browser:
(ref: http://www.gelato.unsw.edu.au/archives/git/0511/13088.html ,
I've since renamed my 'git-show' script to 'git-cat-any').
My usual buffer width is 80 columns. So when I want to see a file from
a certain tree that's not checked out: I'll run git-ls-tree -r <tree>
and output it to my vim buffer, find the corresponding sha1 for the
file, and hit my ',G' macro with my cursor over the blob sha1 to display
its contents. Abbreviating the sha1 is very useful for seeing longer
file paths on an 80 column buffer.
--
Eric Wong
^ permalink raw reply
* gitk : Lines not showing up in diff
From: Jacob Kroon @ 2006-03-07 19:43 UTC (permalink / raw)
To: paulus; +Cc: git
I "think" this is a bug in gitk:
If I have a textfile in which I remove or add a line that looks like
"-----------...", it won't show up in the diff gitk produces. "cg-log"
shows the line correctly being added or deleted.
I'm using gitk 1.2.4 (From Fedora, gitk-1.2.4-1.fc4)
//Jacob
^ permalink raw reply
* Re: git-status too verbose?
From: Carl Worth @ 2006-03-07 18:26 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Andreas Ericsson, Junio C Hamano, Eric Jaffe, git
In-Reply-To: <Pine.LNX.4.64.0603071020530.3573@g5.osdl.org>
[-- Attachment #1: Type: text/plain, Size: 732 bytes --]
On Tue, 7 Mar 2006 10:22:28 -0800 (PST), Linus Torvalds wrote:
>
> What I want to know is "what is committed", and "what is not".
Agreed. Me too.
> That table makes it really really hard to see what you are committing, if
> you have a hundred files changed that are _not_ being committed. The
> actual committed information will be interspersed in the files you're not
> interested in, and vice versa.
The current sorting and grouping should not be changed.
I just want to be able to know if I'm looking at a section of
"committed" vs. "not committed" files when the headers have scrolled
off.
I think it's really just that one bit of information needed. Maybe a
'*' before the word 'modified' (or 'M' or whatever)?
-Carl
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: git-status too verbose?
From: Linus Torvalds @ 2006-03-07 18:22 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Junio C Hamano, Eric Jaffe, Carl Worth, git
In-Reply-To: <440D503E.8090007@op5.se>
On Tue, 7 Mar 2006, Andreas Ericsson wrote:
> >
> > I agree that it would be useful if we had a tool that showed the
> > two status that matter for each file, grouped together on one
> > line, e.g.
> >
> > HEAD->index index->files
> > ------------------------------------------------
> > hello.c unmodified modified
> > world.c modified unmodified
> > frotz.c new unmodified
> > ...
> > garbage.c~ ??? n/a
> >
> > for the current index file and the current HEAD commit.
> >
>
> Could we have 'same' or some such instead of 'unmodified'? It's a bit close to
> 'modified' for the eye to find it quickly.
I really _really_ hate that table anyway.
What I want to know is "what is committed", and "what is not".
That table makes it really really hard to see what you are committing, if
you have a hundred files changed that are _not_ being committed. The
actual committed information will be interspersed in the files you're not
interested in, and vice versa.
The current commit message is a million times superior, even if it might
not be as _pretty_.
Linus
^ permalink raw reply
* Re: Pulling tags from git.git
From: Junio C Hamano @ 2006-03-07 18:10 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: gitzilla, git
In-Reply-To: <440DA82D.3060909@op5.se>
Andreas Ericsson <ae@op5.se> writes:
> Imagine Linus, getting his "please pull" emails and doing so only to
> find dozens of temporary tags fetched by the pull. Junio's patch (if I
> read it correctly) unconditionally fetches *ALL* tags reachable from
> the top of the commit-chain, which means there is no longer any way to
> keep temporary tags in a repo from which someone else will pull.
I thought we made fetch made by such a promiscous pull not to
follow tags, so that wouldn't be a problem. Tag following is
only to happen when you track other's branches. That is:
$ git pull git://git.kernel.org/pub/scm/git/git.git master
would store my "master" branch tip only in your .git/FETCH_HEAD and
you merge it immediately, without following my tags, while:
$ git pull git://git.kernel.org/pub/scm/git/git.git master:origin
(which is what you get by "git pull" without arguments) would
follow my tags, because you are storing the branch head into
your local branch "origin".
> I for one riddle my repos with temporary tags whenever I'm trying
> something I'm not so sure of, or find an interesting bug or a design
> decision I'm not 100% sure of. Perhaps I should rather do this with
> branches, but imo branches are for doing work, whereas tags just mark
> a spot in the development so I easily can find them with gitk or some
> such.
I also have many throwaway unannotated tags. Whenever I have a
WIP that I want to split up or reorder, I tag the tip of that
topic branch with "git tag anchor-blah" tag, rewind the tip to
the commit before the one I want to redo, and then do this
repeatedly:
$ git diff -R anchor-blah >P.diff
$ ... edit P.diff to keep the part I want to apply first
$ git apply --index P.diff
$ ... maybe edit a bit further
$ git commit
until there is no difference between the rewound-and-redone tip
and anchor-blah other than whatever clean-ups I do during the
above cycle. Propagating such throw-away tags is not very
useful. And I think it is reasonable to say that throw-away
tags tend to be unannotated.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox