* Re: [RFH] git-svn documentation
From: Seth Falcon @ 2006-10-18 17:22 UTC (permalink / raw)
To: git
In-Reply-To: <20061016183101.GL27128@hand.yhbt.net>
Eric Wong <normalperson@yhbt.net> writes:
> Anybody willing to supply patches for better documentation? I'll be
> quite busy with other projects the next two weeks, asciidoc is quite
> slow for me; but I'd like to have better docs for git-svn in 1.4.3.
I'm starting to use git-svn more and would be willing to attempt to
contribute some additional docs. What I have in mind is a howto use
git-svn sort of document (unless that isn't seen as useful)
... unfortunately, as I use git-svn more I'm encountering some
questions which will need resolution before I can have much confidence
in my howto :-\
Will post some howto and questions separately.
+ seth
^ permalink raw reply
* Re: VCS comparison table
From: Jeff Licquia @ 2006-10-18 18:03 UTC (permalink / raw)
To: Jakub Narebski; +Cc: bazaar-ng, git
In-Reply-To: <200610171555.56778.jnareb@gmail.com>
On Tue, 2006-10-17 at 15:55 +0200, Jakub Narebski wrote:
> Matthieu Moy wrote:
> > This took time to come in bzr, but that's the bisect plugin:
> >
> > http://bazaar-vcs.org/PluginRegistry
>
> Hmmm... I winder which SCM had it first.
You did. The plugin is largely based on my experiences with the git
version, and explicitly gives credit in the comments.
^ permalink raw reply
* [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Petr Baudis @ 2006-10-18 18:52 UTC (permalink / raw)
To: Linus Torvalds
Cc: Jeff King, Jakub Narebski, Aaron Bentley, Andreas Ericsson,
bazaar-ng, git
In-Reply-To: <Pine.LNX.4.64.0610180739230.3962@g5.osdl.org>
Dear diary, on Wed, Oct 18, 2006 at 04:52:25PM CEST, I got a letter
where Linus Torvalds <torvalds@osdl.org> said that...
> In other words, to get such a pack, we'd _literally_ just do something
> like
>
> git-rev-list --objects-edge origin.. |
> git-pack-objects --stdout |
> uuencode
>
> and that would be it. You'd still need to add a "diffstat" to the thing,
> and tell the other end what the current HEAD is (so that it knows what
> it's supposed to fast-forward to), but it _literally_ is that simple.
>
> "plug-in architecture" my ass. "I recognize this - it's UNIX!".
Took me exactly an hour from mkdir cogito-bundle to cg-push to
kernel.org. :-)
cogito-bundle is an example on how to create third-party addons or
plugins adding own commands to Cogito and using Cogito's infrastructure.
It's not _that_ easy currently since you have to replicate large part of
the build infrastructure locally; that could be fixed by installing some
"library makefiles" and asciidoc toolkit to /usr/share or something, if
there would be a real demand for such an addon API. cg-help and the cg
wrapper will pick up the newly installed commands automagically. The
only thing missing is updating cogito(7) to list the addon commands,
which would take a bit more work.
Though it's an example, it's actually supposed to be useful, by doing
exactly what is outlined above - l - it lets you exchange commits over
mail by so-called "bundles", similar to e.g. Bazaar bundles - basically,
it is like push or fetch, but over email, and the commit ids are
preserved when transferred in bundles (if you just send patches, the
commit ids will end up different).
The provided cg-bundle and cg-unbundle commands are rather crude and
don't support many things - they don't actually include a diff, only a
diffstat, etc. The uuencoded bundle is inlined in the mail, which I
suspect isn't very useful; perhaps it would be more practical to just
attach it binarily. Feel free to send patches (or bundles ;).
An example bundle is available at
http://pasky.or.cz/~pasky/cp/example-bundle.txt
as generated by
cogito.master$ cg-bundle -r v0.18 -m"Subject is this" \
-m"And some body now..." --stdout
and cogito-bundle is available at
git://git.kernel.org/pub/scm/cogito/cogito-bundle.git/
(gitweb http://kernel.org/git/?p=cogito/cogito-bundle.git)
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj
$/=unpack('H*',$_);$_=`echo 16dio\U$k"SK$/SM$n\EsN0p[lN*1
lK[d2%Sa2/d0$^Ixp"|dc`;s/\W//g;$_=pack('H*',/((..)*)$/)
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Petr Baudis @ 2006-10-18 18:59 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <20061018185225.GU20017@pasky.or.cz>
Dear diary, on Wed, Oct 18, 2006 at 08:52:25PM CEST, I got a letter
where Petr Baudis <pasky@suse.cz> said that...
> Dear diary, on Wed, Oct 18, 2006 at 04:52:25PM CEST, I got a letter
> where Linus Torvalds <torvalds@osdl.org> said that...
> > In other words, to get such a pack, we'd _literally_ just do something
> > like
> >
> > git-rev-list --objects-edge origin.. |
> > git-pack-objects --stdout |
> > uuencode
> >
> > and that would be it. You'd still need to add a "diffstat" to the thing,
> > and tell the other end what the current HEAD is (so that it knows what
> > it's supposed to fast-forward to), but it _literally_ is that simple.
> >
> > "plug-in architecture" my ass. "I recognize this - it's UNIX!".
>
> Took me exactly an hour from mkdir cogito-bundle to cg-push to
> kernel.org. :-)
By the way, originally I just wanted to index and save the pack, but
when trying to feed it to git-index-pack, I kept getting
fatal: packfile '.git/objects/pack/pack-b2ab684daebea5b9c5a6492fa732e0d2e1799c8e.pack' has unresolved deltas
while feeding it to git-unpack-objects works fine. Any idea what's wrong?
(BTW, I got the id by sha1summing the pack file; is there an existing
way to name a pack properly if I have it lying around, unnamed? sha1sum
seems to be specific to a fairly new GNU coreutils version.)
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj
$/=unpack('H*',$_);$_=`echo 16dio\U$k"SK$/SM$n\EsN0p[lN*1
lK[d2%Sa2/d0$^Ixp"|dc`;s/\W//g;$_=pack('H*',/((..)*)$/)
^ permalink raw reply
* Re: [BUG] git-log shows first parent and repeated last for octopus merge
From: Junio C Hamano @ 2006-10-18 19:02 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.63.0610181551150.14200@wbgn013.biozentrum.uni-wuerzburg.de>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi,
>
> On Wed, 18 Oct 2006, Jakub Narebski wrote:
>
>> When trying to find how many merges and how many octopus merges (merges with
>> more than two parents) are in git.git repository I have encountered the
>> following strange output of git-log:
>>
>> 1000:jnareb@roke:~/git> git log --parents --full-history --max-count=1 \
>> 211232bae64bcc60bbf5d1b5e5b2344c22ed767e -- a//b
>> commit 211232bae64bcc60bbf5d1b5e5b2344c22ed767e <last parent repeated>
>> Merge: d0d0d0b... d0d0d0b... d0d0d0b... d0d0d0b... d0d0d0b...
>> [...]
>
> This happens because a//b rewrites the history, i.e. the parents are
> edited. IMHO it makes no sense at all to show the parents in such a case,
> since they are bogus.
If the command line did not have --full-history, I would agree
with you. The caller explicitly told us not to remove side
branches that do not end up modifying the paths, and also told
us, with --parent, to show parenthood information after removing
intermediate commits that do not change the tree shape (with
respect to the specified paths).
So it is showing what the user asked -- the user may not have
understood what he was asking, but that is a separate problem, I
would think.
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Junio C Hamano @ 2006-10-18 19:04 UTC (permalink / raw)
To: git
In-Reply-To: <20061018185907.GV20017@pasky.or.cz>
Petr Baudis <pasky@suse.cz> writes:
> Dear diary, on Wed, Oct 18, 2006 at 08:52:25PM CEST, I got a letter
> where Petr Baudis <pasky@suse.cz> said that...
>> Dear diary, on Wed, Oct 18, 2006 at 04:52:25PM CEST, I got a letter
>> where Linus Torvalds <torvalds@osdl.org> said that...
>> > In other words, to get such a pack, we'd _literally_ just do something
>> > like
>> >
>> > git-rev-list --objects-edge origin.. |
>> > git-pack-objects --stdout |
>> > uuencode
>> >
>> > and that would be it. You'd still need to add a "diffstat" to the thing,
>> > and tell the other end what the current HEAD is (so that it knows what
>> > it's supposed to fast-forward to), but it _literally_ is that simple.
>> >
>> > "plug-in architecture" my ass. "I recognize this - it's UNIX!".
>>
>> Took me exactly an hour from mkdir cogito-bundle to cg-push to
>> kernel.org. :-)
>
> By the way, originally I just wanted to index and save the pack, but
> when trying to feed it to git-index-pack, I kept getting
>
> fatal: packfile '.git/objects/pack/pack-b2ab684daebea5b9c5a6492fa732e0d2e1799c8e.pack' has unresolved deltas
>
> while feeding it to git-unpack-objects works fine. Any idea what's wrong?
Yes. You told the pipeline, with --objects-edge, to create a
thin pack. By definition that is _not_ indexable.
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Nicolas Pitre @ 2006-10-18 19:09 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20061018185907.GV20017@pasky.or.cz>
On Wed, 18 Oct 2006, Petr Baudis wrote:
> By the way, originally I just wanted to index and save the pack, but
> when trying to feed it to git-index-pack, I kept getting
>
> fatal: packfile '.git/objects/pack/pack-b2ab684daebea5b9c5a6492fa732e0d2e1799c8e.pack' has unresolved deltas
>
> while feeding it to git-unpack-objects works fine. Any idea what's wrong?
Did you really manage to miss the "heads-up: git-index-pack in "next" is
broken" thread?
The fix:
diff --git a/index-pack.c b/index-pack.c
index fffddd2..56c590e 100644
--- a/index-pack.c
+++ b/index-pack.c
@@ -23,6 +23,12 @@ union delta_base {
unsigned long offset;
};
+/*
+ * Even if sizeof(union delta_base) == 24 on 64-bit archs, we really want
+ * to memcmp() only the first 20 bytes.
+ */
+#define UNION_BASE_SZ 20
+
struct delta_entry
{
struct object_entry *obj;
@@ -211,7 +217,7 @@ static int find_delta(const union delta_
struct delta_entry *delta = &deltas[next];
int cmp;
- cmp = memcmp(base, &delta->base, sizeof(*base));
+ cmp = memcmp(base, &delta->base, UNION_BASE_SZ);
if (!cmp)
return next;
if (cmp < 0) {
@@ -232,9 +238,9 @@ static int find_delta_childs(const union
if (first < 0)
return -1;
- while (first > 0 && !memcmp(&deltas[first - 1].base, base, sizeof(*base)))
+ while (first > 0 && !memcmp(&deltas[first - 1].base, base, UNION_BASE_SZ))
--first;
- while (last < end && !memcmp(&deltas[last + 1].base, base, sizeof(*base)))
+ while (last < end && !memcmp(&deltas[last + 1].base, base, UNION_BASE_SZ))
++last;
*first_index = first;
*last_index = last;
@@ -312,7 +318,7 @@ static int compare_delta_entry(const voi
{
const struct delta_entry *delta_a = a;
const struct delta_entry *delta_b = b;
- return memcmp(&delta_a->base, &delta_b->base, sizeof(union delta_base));
+ return memcmp(&delta_a->base, &delta_b->base, UNION_BASE_SZ);
}
static void parse_pack_objects(void)
Nicolas
^ permalink raw reply related
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Nicolas Pitre @ 2006-10-18 19:13 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vy7rd1m4q.fsf@assigned-by-dhcp.cox.net>
On Wed, 18 Oct 2006, Junio C Hamano wrote:
> Petr Baudis <pasky@suse.cz> writes:
>
> > By the way, originally I just wanted to index and save the pack, but
> > when trying to feed it to git-index-pack, I kept getting
> >
> > fatal: packfile '.git/objects/pack/pack-b2ab684daebea5b9c5a6492fa732e0d2e1799c8e.pack' has unresolved deltas
> >
> > while feeding it to git-unpack-objects works fine. Any idea what's wrong?
>
> Yes. You told the pipeline, with --objects-edge, to create a
> thin pack. By definition that is _not_ indexable.
Ah true. I missed the "thin" pack.
Any idea why we should still prevent this? It is not like it was a
technical limitation.
Nicolas
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Shawn Pearce @ 2006-10-18 19:18 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0610181510510.1971@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Wed, 18 Oct 2006, Junio C Hamano wrote:
>
> > Petr Baudis <pasky@suse.cz> writes:
> >
> > > By the way, originally I just wanted to index and save the pack, but
> > > when trying to feed it to git-index-pack, I kept getting
> > >
> > > fatal: packfile '.git/objects/pack/pack-b2ab684daebea5b9c5a6492fa732e0d2e1799c8e.pack' has unresolved deltas
> > >
> > > while feeding it to git-unpack-objects works fine. Any idea what's wrong?
> >
> > Yes. You told the pipeline, with --objects-edge, to create a
> > thin pack. By definition that is _not_ indexable.
>
> Ah true. I missed the "thin" pack.
>
> Any idea why we should still prevent this? It is not like it was a
> technical limitation.
It still is in sha1-file.c; or at least the last time I looked at
that code. The base is always resolved from the same pack/index
as the delta. If you fix sha1-file.c sure, I don't see why you
can't allow indexing thin packs.
--
Shawn.
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Junio C Hamano @ 2006-10-18 19:33 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0610181510510.1971@xanadu.home>
Nicolas Pitre <nico@cam.org> writes:
> Ah true. I missed the "thin" pack.
>
> Any idea why we should still prevent this? It is not like it was a
> technical limitation.
It is a technical limitation. We have never assumed that the
virtual address space is big enough to hold more than one whole
pack mmapped at the same time.
Lifting this needs the piecemeal mmap() change somebody was
talking about.
I might bite the bullet and do that myself but I've been hoping
to get an appliable patch from somewhere else ;-).
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Nicolas Pitre @ 2006-10-18 19:33 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Junio C Hamano, git
In-Reply-To: <20061018191834.GA18829@spearce.org>
On Wed, 18 Oct 2006, Shawn Pearce wrote:
> Nicolas Pitre <nico@cam.org> wrote:
> > On Wed, 18 Oct 2006, Junio C Hamano wrote:
> >
> > > Petr Baudis <pasky@suse.cz> writes:
> > >
> > > > By the way, originally I just wanted to index and save the pack, but
> > > > when trying to feed it to git-index-pack, I kept getting
> > > >
> > > > fatal: packfile '.git/objects/pack/pack-b2ab684daebea5b9c5a6492fa732e0d2e1799c8e.pack' has unresolved deltas
> > > >
> > > > while feeding it to git-unpack-objects works fine. Any idea what's wrong?
> > >
> > > Yes. You told the pipeline, with --objects-edge, to create a
> > > thin pack. By definition that is _not_ indexable.
> >
> > Ah true. I missed the "thin" pack.
> >
> > Any idea why we should still prevent this? It is not like it was a
> > technical limitation.
>
> It still is in sha1-file.c; or at least the last time I looked at
> that code. The base is always resolved from the same pack/index
> as the delta.
Yep. I mean this doesn't have to be like that fundamentally.
> If you fix sha1-file.c sure, I don't see why you
> can't allow indexing thin packs.
If there are advantages to do so then maybe. That would be for another
day though, as I've been burned a bit with packs recently.
Nicolas
^ permalink raw reply
* [PATCH] reduce delta head inflated size
From: Nicolas Pitre @ 2006-10-18 19:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
Supposing that both the base and result sizes were both full size 64-bit
values, their encoding would occupy only 9.2 bytes each. Therefore
inflating 64 bytes is way overkill. Limit it to 20 bytes instead which
should be plenty enough for a couple years to come.
Signed-off-by: Nicolas Pitre <nico@cam.org>
---
diff --git a/sha1_file.c b/sha1_file.c
index 25c0bf9..e89d24c 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -943,7 +943,7 @@ static int packed_delta_info(struct pack
if (sizep) {
const unsigned char *data;
- unsigned char delta_head[64];
+ unsigned char delta_head[20];
unsigned long result_size;
z_stream stream;
int st;
^ permalink raw reply related
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Sean @ 2006-10-18 19:57 UTC (permalink / raw)
To: Petr Baudis
Cc: Linus Torvalds, Jeff King, Jakub Narebski, Aaron Bentley,
Andreas Ericsson, bazaar-ng, git
In-Reply-To: <20061018185225.GU20017@pasky.or.cz>
On Wed, 18 Oct 2006 20:52:25 +0200
Petr Baudis <pasky@suse.cz> wrote:
> Took me exactly an hour from mkdir cogito-bundle to cg-push to
> kernel.org. :-)
Nicely done :-).
> cogito-bundle is an example on how to create third-party addons or
> plugins adding own commands to Cogito and using Cogito's infrastructure.
> It's not _that_ easy currently since you have to replicate large part of
> the build infrastructure locally; that could be fixed by installing some
> "library makefiles" and asciidoc toolkit to /usr/share or something, if
> there would be a real demand for such an addon API. cg-help and the cg
> wrapper will pick up the newly installed commands automagically. The
> only thing missing is updating cogito(7) to list the addon commands,
> which would take a bit more work.
Couldn't these just as easily have been written as git-bundle and
git-unbundle without needing any plugins or other cogito infrastructure?
> Though it's an example, it's actually supposed to be useful, by doing
> exactly what is outlined above - l - it lets you exchange commits over
> mail by so-called "bundles", similar to e.g. Bazaar bundles - basically,
> it is like push or fetch, but over email, and the commit ids are
> preserved when transferred in bundles (if you just send patches, the
> commit ids will end up different).
Not sure if it would be useful, but it shouldn't be too hard to have
same commit ids regenerated at receiving end with git patches.
> The provided cg-bundle and cg-unbundle commands are rather crude and
> don't support many things - they don't actually include a diff, only a
> diffstat, etc. The uuencoded bundle is inlined in the mail, which I
> suspect isn't very useful; perhaps it would be more practical to just
> attach it binarily. Feel free to send patches (or bundles ;).
Think you're right about making it an attachment instead.
Sean
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Linus Torvalds @ 2006-10-18 20:08 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20061018185907.GV20017@pasky.or.cz>
On Wed, 18 Oct 2006, Petr Baudis wrote:
>
> By the way, originally I just wanted to index and save the pack, but
> when trying to feed it to git-index-pack, I kept getting
>
> fatal: packfile '.git/objects/pack/pack-b2ab684daebea5b9c5a6492fa732e0d2e1799c8e.pack' has unresolved deltas
>
> while feeding it to git-unpack-objects works fine. Any idea what's wrong?
Since you created a "thin" pack (that's what the "--objects-edge" means),
the pack actually contains deltas to objects that are _not_ in the pack.
In other words, it's not a valid stand-alone pack, it's only a valid thin
pack, useful to transfer data to the other end (and the other end had
better have the objects that the deltas are against already).
As a result, index-file refuses to index it: it cannot be used as a
stand-alone pack, it's _only_ useful as a transfer medium.
So don't even _try_ to use it as a standalone pack-file. It won't work.
(If you want somethign that actually works as a stand-alone pack-file,
change the "--objects-edge" flag to just "--objects" - that makes the
pack-file self-sufficient, and doesn't try to delta against "edge"
objects).
> (BTW, I got the id by sha1summing the pack file; is there an existing
> way to name a pack properly if I have it lying around, unnamed? sha1sum
> seems to be specific to a fairly new GNU coreutils version.)
A properly named _standalone_ pack gets named not by its actual contents,
but by the SHA1-sum of the sorted list of objects it contains. That's so
that a pack-file will be named the same thing regardless of how the
contents are actually packed.
A thin pack cannot be named that way at all, for the same reason you
cannot index it: it has a set of objects it enumerates (so you could name
it by them), but it _also_ has a set of objects outside of it that it
depends on.
That said, even a thin pack internally has a SHA1 checksum of its
contents: the last 20 bytes should be the SHA1-sum of all preceding bytes.
So if you just want _some_ kind of name, you can use the last 20 bytes of
a pack, which is just its internal integrity-checksum (but that is
_different_ from the "pack-xxxxxx.idx"/"pack-xxxxxx.pack" naming).
Linus
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Petr Baudis @ 2006-10-18 20:46 UTC (permalink / raw)
To: Sean
Cc: Linus Torvalds, Jeff King, Jakub Narebski, Aaron Bentley,
Andreas Ericsson, bazaar-ng, git
In-Reply-To: <20061018155704.b94b441d.seanlkml@sympatico.ca>
Dear diary, on Wed, Oct 18, 2006 at 09:57:04PM CEST, I got a letter
where Sean <seanlkml@sympatico.ca> said that...
> Couldn't these just as easily have been written as git-bundle and
> git-unbundle without needing any plugins or other cogito infrastructure?
They could be written, but certainly not "just as easily". I'm more used
to coding Cogito, I find it much more convenient than hacking git's
shell scripts (those two may be interconnected ;), and there's plenty of
infrastructure in Cogito missing in Git - Cogito has more flexible
arguments parsing, documentation bundled with code, I could just
cut'n'paste the code to handle -m arguments and message editor (and most
of it is libified anyway) so I got that basically for free, and I think
Cogito beats Git hands down in code readability.
> Not sure if it would be useful, but it shouldn't be too hard to have
> same commit ids regenerated at receiving end with git patches.
It would be of course technically possible, yes. But somewhat more work,
this is just a quick hack.
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
#!/bin/perl -sp0777i<X+d*lMLa^*lN%0]dsXx++lMlN/dsM0<j]dsj
$/=unpack('H*',$_);$_=`echo 16dio\U$k"SK$/SM$n\EsN0p[lN*1
lK[d2%Sa2/d0$^Ixp"|dc`;s/\W//g;$_=pack('H*',/((..)*)$/)
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Shawn Pearce @ 2006-10-18 20:46 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0610181525410.1971@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> If there are advantages to do so then maybe. That would be for another
> day though, as I've been burned a bit with packs recently.
I guess its my turn then to work in the mmap window code, huh? :-)
--
Shawn.
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Shawn Pearce @ 2006-10-18 20:47 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nicolas Pitre, git
In-Reply-To: <7vr6x51ks3.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano <junkio@cox.net> wrote:
> Nicolas Pitre <nico@cam.org> writes:
>
> > Ah true. I missed the "thin" pack.
> >
> > Any idea why we should still prevent this? It is not like it was a
> > technical limitation.
>
> It is a technical limitation. We have never assumed that the
> virtual address space is big enough to hold more than one whole
> pack mmapped at the same time.
Even though its not big enough for some larger packs on a 32
bit system.
> Lifting this needs the piecemeal mmap() change somebody was
> talking about.
>
> I might bite the bullet and do that myself but I've been hoping
> to get an appliable patch from somewhere else ;-).
I might be able to do it this weekend. I'll try to spend some time
on it. You'll either see a patch series, or you won't. ;-)
--
Shawn.
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Sean @ 2006-10-18 20:53 UTC (permalink / raw)
To: Petr Baudis
Cc: Linus Torvalds, Jeff King, Jakub Narebski, Aaron Bentley,
Andreas Ericsson, bazaar-ng, git
In-Reply-To: <20061018204618.GW20017@pasky.or.cz>
On Wed, 18 Oct 2006 22:46:18 +0200
Petr Baudis <pasky@suse.cz> wrote:
> They could be written, but certainly not "just as easily". I'm more used
> to coding Cogito, I find it much more convenient than hacking git's
> shell scripts (those two may be interconnected ;), and there's plenty of
> infrastructure in Cogito missing in Git - Cogito has more flexible
> arguments parsing, documentation bundled with code, I could just
> cut'n'paste the code to handle -m arguments and message editor (and most
> of it is libified anyway) so I got that basically for free, and I think
> Cogito beats Git hands down in code readability.
Hmmm, if I get some time over the weekend i'll take a look at porting
them to Git. But maybe some of the items you mentioned above deserve
to become part of Git proper? It would definitely be nice to see
something like what you just did put into the hands of more users than
just those using Cogito, and its unfortunate that the current state
of Git code kept you from going that route.
> It would be of course technically possible, yes. But somewhat more work,
> this is just a quick hack.
No doubt, there would be some slightly thorny issues to deal with. It
might even end up too fragile to be worthwhile.
Sean
^ permalink raw reply
* Re: VCS comparison table
From: Charles Duffy @ 2006-10-18 21:04 UTC (permalink / raw)
To: bazaar-ng; +Cc: git
In-Reply-To: <BAYC1-PASMTP01369CD694D75CB61ACCC7AE0E0@CEZ.ICE>
Sean wrote:
> Hmm.. It's pretty easy to test out Git ideas too. People do it all
> the time, and without plugins. Junio maintains several such trees
> for instance. Dunno.. I just think plugs _sounds_ good to developers
> without much real benefit to users over regular ole source code.
Example time!
There's a plugin for Bzr which adds support for Cygwin-compatible
symlink support on Windows. (IIRC, this involves monkey-patching some of
the Python standard library bits).
Now, this is something which is *proposed* as a feature to be merged
into upstream bzr, and it may happen at some point. That said, when I
have a Windows-using coworker who wants to check out a repository that
has symlinks in it (with his win32-native, no-cygwin-required bzr
upstream binary), I don't need to tell him to go download and build bzr
from a third party; instead, I just need to tell him to run a single
command to check out the plugin in question into the bzr plugins folder.
From an end-user convenience perspective, it's a pretty significant win.
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Linus Torvalds @ 2006-10-18 21:17 UTC (permalink / raw)
To: Shawn Pearce; +Cc: Nicolas Pitre, Junio C Hamano, git
In-Reply-To: <20061018204626.GA19194@spearce.org>
On Wed, 18 Oct 2006, Shawn Pearce wrote:
>
> I guess its my turn then to work in the mmap window code, huh? :-)
There are bigger reasons to _never_ allow packs to contain deltas to
outside of themselves:
- there's no point.
If you have many small packs, you're doing something wrong. The whole
_point_ of packs is to put things into the same file, so that you can
avoid the filesystem overhead. And once packs are big and few, the
advantage of having deltas to outside the pack is basically zero.
- it's a bad design.
Self-sufficient packs means that a pack is a "safe" thing. When the
index says that it contains an object, then it damn well contains it.
In contrast, if you had packs that only contained a delta, and the pack
needed some _other_ pack (or loose object) to actually generate that
object, then it's not safe any more. You could end up with a situation
where you get two packs from two different sources, and they contain
deltas to _each_other_, and you have no way of actually generating the
object itself any more.
(Or you end up having to have rules to figure out when you have a loop,
and stop looking just in the packed files, and start looking for loose
objects instead)
In other words, it has potentially _serious_ downsides.
So DAMMIT! Stop looking to make the data structures worse. The fact is,
the git data structures are FINE. They are well-designed. They work well.
There's no _point_ in changing them, especially since changing them seems
to be all about making things less reliable for dubious gain.
One of the advantages of git is that you can explain things with object
relationships, and that the file format is stable as _hell_. Thats a GOOD
thing. Please realize that if you want to change the file formats, you'd
have a hell of a better reason for it that "just because I can".
Please. Really.
So next time somebody suggests a new pack-format, ask yourself:
- does it save disk-space by 50% or more?
- does it drop memory usage by 50% or more?
- does it improve performance by 50% of more?
- does it make something possible that really fundamentally isn't
possible right now?
And if the answer to those questions is "no", then JUST DON'T DO IT.
It really needs to be _damn_ spectacular to be worthy of a new format.
Really. We've had a few of those, so it clearly does happen:
- The "compress _after_ SHA1". The original object format was just
broken, and the SHA1 name depended on how things compressed. I fixed
it. It needed fixing. We couldn't have done a lot of the things we did
without switching compression and SHA1-hashing around.
- the pack-file in the first place: this saved orders of magnitude both
in diskspace _and_ performance. Not "10%". More like "factors of 100".
THAT was worthy of a major format change.
- the "make loose object contents look the same as packed objects". This
was not just a cleanup, it allows us to create pack-files much faster.
That said, we're still defaulting to the legacy format, and maybe it
wasn't really worth it.
My personal suspicion is that we'll want to have a 64-bit index file some
day, and THAT is worthy of a format change. That day is not now, btw. It's
probably not even very close. Even the mozilla repo that was pushing the
limit was only doing so until it was optimized better, and now it's
apparently nowhere _near_ that limit.
But even then, we might well want to update _just_ the index file format.
Because in an SCM, stability and trustworthiness is more important than
just about _anything_ else.
Linus
^ permalink raw reply
* Re: VCS comparison table
From: Jeff King @ 2006-10-18 21:20 UTC (permalink / raw)
To: Linus Torvalds; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0610180739230.3962@g5.osdl.org>
On Wed, Oct 18, 2006 at 07:52:25AM -0700, Linus Torvalds wrote:
> git send origin..
>
> and that "origin" is what the other end is expected to already have.
>
> Of course, if you send an unconnected bundle (ie you give an origin that
> the other end _doesn't_ have), you're screwed.
OK, that was how I was envisioning it, as well, but I was concerned
about the "screwed" part. But I'm not sure how often that would be an
issue in practice (after all, patches require some matchup of the base,
though not as strict as SHA1s).
Thanks for the explanation.
-Peff
^ permalink raw reply
* Re: heads-up: git-index-pack in "next" is broken
From: Davide Libenzi @ 2006-10-18 21:21 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Nicolas Pitre, Sergey Vlasov, Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0610180938540.3962@g5.osdl.org>
On Wed, 18 Oct 2006, Linus Torvalds wrote:
> On Wed, 18 Oct 2006, Davide Libenzi wrote:
> >
> > Speaking in general, seen at the hash function level, of course an interface
> > should not give different result for different word sizes or word endianess.
> > Considering the diff algorithm as interface, as I said, the output was
> > unaffected by the 64 bits word size. It was just very slow.
>
> Well, even the output may actually be affected, in the case of _real_ hash
> collisions (as opposed to just the hash _list_ collision that XDL_HASHLONG
> caused).
>
> So I actually think it would be better to have "uint32_t" as the hash
> value - because that would mean that all diffs (or, in the case of the
> block-algorithm, the deltas) are guaranteed to give the same results
> regardless of architecture.
>
> Right now, we actually generate a 64-bit hash value (BUT: for short lines,
> it's likely only _interesting_ in the low bits, so the high bits tend to
> have a very high likelihood of being zero). So hash collisions are
> different: on a 32-bit architecture, two lines may have the same hash,
> while on a 64-bit one, they are different.
>
> And together with some of the limiters we have (eg XDL_MAX_EQLIMIT) hash
> collisions can sometimes affect the output.
>
> Admittedly, in _practice_ this is really unlikely to affect anything
> (you'd get a valid diff in either case, they'd just possibly be subtly
> different, and the input data must be _really_ strange to even see that
> case), but I do think that the hash algorithm can matter.
>
> NOTE! I'm not talking about XDL_HASHLONG(), I'm talking about the
> xdl_hash_record() hash, which returns differently-sized hash results on
> 32-bit and 64-bit. And there are cases where we _only_ compare the hashes,
> and don't actually double-check the contents.
The hash value (hence the hash bucket index) simply directs you to the
bucket where a real record-compare loop is performed. Collisions here
means only performance loss (you don't spread buckets enough), nothing
more. So the different behaviour does not apply to the diff algo.
But, it would apply if the knowledge of the hash indexing would be
"exported", for example inside an external file. Think about a trivial DB
file where you store hash buckets on file an you want to lookup records
based on the store hash layout. In that case, of course, if the hash
function that generated the DB bucket layout is different from the one
that you use to get the bucket index at lookup time, you're in trouble.
IOW if the hash function result is not "exported" is some way, it doesn't
really matter if it's an 'unsigned long' or a 'uint32_t'. In the same way
you cannot export binary structures using 'int' or 'long', and expect to
be compatible over different architectures.
- Davide
^ permalink raw reply
* Re: VCS comparison table
From: Sean @ 2006-10-18 21:29 UTC (permalink / raw)
To: Charles Duffy; +Cc: git, bazaar-ng
In-Reply-To: <eh64tk$rug$2@sea.gmane.org>
On Wed, 18 Oct 2006 16:04:52 -0500
Charles Duffy <cduffy@spamcop.net> wrote:
> Example time!
>
> There's a plugin for Bzr which adds support for Cygwin-compatible
> symlink support on Windows. (IIRC, this involves monkey-patching some of
> the Python standard library bits).
>
> Now, this is something which is *proposed* as a feature to be merged
> into upstream bzr, and it may happen at some point. That said, when I
> have a Windows-using coworker who wants to check out a repository that
> has symlinks in it (with his win32-native, no-cygwin-required bzr
> upstream binary), I don't need to tell him to go download and build bzr
> from a third party; instead, I just need to tell him to run a single
> command to check out the plugin in question into the bzr plugins folder.
>
> From an end-user convenience perspective, it's a pretty significant win.
You'll need a better example than that. Git has supported a version
of Cygwin-compatible symlink support on Windows for quite some time.
And no plugins were needed.
Sean
^ permalink raw reply
* Re: [ANNOUNCE] Example Cogito Addon - cogito-bundle
From: Shawn Pearce @ 2006-10-18 21:32 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Nicolas Pitre, Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0610181358200.3962@g5.osdl.org>
Linus Torvalds <torvalds@osdl.org> wrote:
> On Wed, 18 Oct 2006, Shawn Pearce wrote:
> >
> > I guess its my turn then to work in the mmap window code, huh? :-)
>
> There are bigger reasons to _never_ allow packs to contain deltas to
> outside of themselves:
>
> - there's no point.
> - it's a bad design.
That and all of the other reasons you cited in your message are
why I haven't finished trying to use some sort of dictionary based
compression for packing objects.
On the other hand we've already seen how packs >1.5 GiB in size
(certainly well within the 4 GiB limitation in the current index
file format) cannot be repacked by git-repack-objects on a 32
bit address space as the entire pack file is mmap'd on one shot.
After the kernel space of ~1 GiB and the pack file at ~1.5 GiB
there's very little address space left for the application code.
My comment that you quoted was about mmap'ing the pack files in
large chunks (around 64-128 MiB at a time, but configurable from
.git/config) rather than as an entire massive mapping. It had
absolutely nothing to do about changing the pack file format, the
index format, or any other on disk format. Although it would add
a new pair of configuration options to .git/config. Is that change
too radical? :-)
With such a change the Git and Linux kernel repositories would both
still mmap in one chunk but much larger projects like Mozilla or
very large pack files coming out of git-fastimport would actually
be usable on 32 bit architectures without running into address space
limitations so quickly. Git would also be slightly more usable for
some people who have a lot of very uncompressable data stored in Git.
Unless of course you are actively working on a fix for the Linux
kernel so that we can actually have all 4 GiB of virtual address
space available for the userspace git-repack-objects process.
Or have some sort of secret plan to upgrade everyone who uses Git
to 64 bit processors which support 64 bit address spaces...
--
Shawn.
^ permalink raw reply
* Re: VCS comparison table
From: Shawn Pearce @ 2006-10-18 21:37 UTC (permalink / raw)
To: Sean; +Cc: Charles Duffy, git, bazaar-ng
In-Reply-To: <20061018172945.c0c58c38.seanlkml@sympatico.ca>
Sean <seanlkml@sympatico.ca> wrote:
> On Wed, 18 Oct 2006 16:04:52 -0500
> Charles Duffy <cduffy@spamcop.net> wrote:
>
> > Example time!
> >
> > There's a plugin for Bzr which adds support for Cygwin-compatible
> > symlink support on Windows. (IIRC, this involves monkey-patching some of
> > the Python standard library bits).
> >
> > Now, this is something which is *proposed* as a feature to be merged
> > into upstream bzr, and it may happen at some point. That said, when I
> > have a Windows-using coworker who wants to check out a repository that
> > has symlinks in it (with his win32-native, no-cygwin-required bzr
> > upstream binary), I don't need to tell him to go download and build bzr
> > from a third party; instead, I just need to tell him to run a single
> > command to check out the plugin in question into the bzr plugins folder.
> >
> > From an end-user convenience perspective, it's a pretty significant win.
>
> You'll need a better example than that. Git has supported a version
> of Cygwin-compatible symlink support on Windows for quite some time.
> And no plugins were needed.
Actually I think the only part of that example that was really
interesting was that Bzr runs natively on Windows and that Bzr's
native method of extending the tool with additional features doesn't
require Cygwin.
Today Git doesn't run natively on Windows. It runs slowly through
Cygwin, thanks to lots of various overheads in different places.
And due to the crappy disk drive in my Windows box. :-)
Today Git is typically extended (at least initially in prototyping
mode) through Perl, Python, TCL or Bourne shell scripts. Although
the first three are available natively on Windows the last requires
Cygwin... and we've had some issues with ActiveState Perl on Windows
in the past too.
--
Shawn.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox