* Re: [PATCH] Documented --no-checkout option in git-svn
From: public_vi @ 2008-10-31 22:36 UTC (permalink / raw)
To: Sam Vilain; +Cc: git, Deskin Miller
In-Reply-To: <1225390505.19891.41.camel@maia.lan>
OK, agree with all changes. This is my the first patch.
/* not including the patch text as it gets broken by Iceweasel. */
Sam Vilain wrote:
> I think the wording is close enough; here's a version which looks good
> to me and fixes the asciidoc differences.
>
^ permalink raw reply
* Re: libgit2 - a true git library
From: Jakub Narebski @ 2008-10-31 23:56 UTC (permalink / raw)
To: Shawn O. Pearce
Cc: Junio C Hamano, Nicolas Pitre, Pierre Habouzit, git, Scott Chacon
In-Reply-To: <20081031234115.GD14786@spearce.org>
"Shawn O. Pearce" <spearce@spearce.org> writes:
> Junio C Hamano <gitster@pobox.com> wrote:
> > Although no license asks this, my wish is that if somebody built on top of
> > what I wrote to make the world a better place, I'd like the same access to
> > that additional code so that I too can enjoy the improved world. Because
> > almost all of my code in git.git are under GPLv2, in reality I do not have
> > any access to your software as long as you do not distribute your
> > additional code that made the world a better place, which is a bit sad.
>
> IMHO, its a flaw of the GPL. GitHub anyone? Heck, even Google uses
> a lot of GPL'd software internally (yes, we have Linux desktops and
> servers) but not all of the software we distribute internally goes
> external, so not all of our patches are published. *sigh*
>
> I've actually stayed awake at night sometimes wondering what the
> world would be like if the GPL virual clause forced the source code
> for a website to be opened, or forced you to publish your code
> even if you never distribute binaries beyond "you" (where "you"
> is some mega corp in many countries with many employees).
There is such license, and it is called AGPLv3, Affero GPL[1].
And of course Google prohibits (or did prohibit) using it for projects
hosted at Google Code... wonder why... ;-)
[1] http://en.wikipedia.org/wiki/AGPL
--
Jakub Narebski
Poland
ShadeHawk on #git
^ permalink raw reply
* Re: getting list of objects for packing
From: Brandon Casey @ 2008-11-01 0:00 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Nicolas Pitre, Git Mailing List
In-Reply-To: <fAUegZ3bxPo8HquZjUM9syW-giYefuAzBtb1XXHQ-TwbmvJvMZvmDA@cipher.nrlssc.navy.mil>
Brandon Casey wrote:
> Junio C Hamano wrote:
>> Brandon Casey <casey@nrlssc.navy.mil> writes:
>>
>>> Nicolas Pitre wrote:
>>>> On Fri, 31 Oct 2008, Brandon Casey wrote:
>>>>> -The sed statement is stripping off anything after the sha1. Any way to
>>>>> get rev-list to print out just the sha1 so that sed is not necessary?
>>>> If you strip the data after the SHA1 when pipping into pack-objects then
>>>> you'll have horrible delta compression results. The path names after
>>>> each SHA1 is used to sort objects when trying to find best matches for
>>>> delta compression. So you should preserve those and feed it back
>>>> especially with those packs that you still want delta compression for.
>>> Ah, I'll have to rethink my script then. Thanks!
>> Yeah, but wasn't the purpose of your whole exercise to list objects that
>> do not delta nor compress well with each other, in which case the delta
>> compression order (aka name hash) would not matter, no?
>
> The script I wrote actually starts up two pack-objects instances and I was
> writing the objects I wanted to pack _normally_ to one, and the ones that I
> did not want compressed/deltafied to the other (which was started with
> --no-reuse-object --window=0 --depth=0 --compression=0).
So, my script created two pack files: one packed normally, and one packed without
compression or delta. I removed my original packs, and put these two new ones in
my pack directory and ran 'git fsck --full' and it completed successfully. There
are no loose objects in the repo.
I added a .keep file for each pack.
Since my script removed the extra info from rev-parse's output, I removed the
.keep file from the appropriate pack and ran 'git gc --aggressive'.
The 1.7GB pack that had the .keep file removed has been replaced with a +3GB
pack file. The other pack file which still has the .keep file is 2.3GB.
In another repo with 3 packs marked .keep, and one 388KB pack with ~300
objects in it, and 3 loose dangling objects, the 388KB pack was replaced with
a 3.5GB pack.
It appears that the entire repository is being packed into the new pack file
even though there are existing pack files with .keep files.
If I compare the output from 'git verify-pack -v' I can see that many of the
objects in the packs marked with a .keep file are indeed in the new pack file.
But not all of them.
-brandon
^ permalink raw reply
* Re: libgit2 - a true git library
From: Pierre Habouzit @ 2008-11-01 0:02 UTC (permalink / raw)
To: Junio C Hamano
Cc: Shawn O. Pearce, Pieter de Bie, Nicolas Pitre, git, Scott Chacon
In-Reply-To: <7v63n872bs.fsf@gitster.siamese.dyndns.org>
[-- Attachment #1: Type: text/plain, Size: 1992 bytes --]
On Fri, Oct 31, 2008 at 11:49:11PM +0000, Junio C Hamano wrote:
> "Shawn O. Pearce" <spearce@spearce.org> writes:
>
> > You are the end-user of the library, so it needs to suit you. Ok,
> > you aren't the only end-user, but you and other developers like
> > you... :-)
>
> I will be the end-user of the library because if we want libgit2 to be
> anywhere close to successful, you should be able to port C-git to it.
>
> I understand that the apidocs/ is a very early work-in-progress, but
> still, it bothers me that it is unclear to me what lifetime rules are in
> effect on the in-core objects. For example, in C-git, commit objects are
> not just parsed but are modified in place as history is traversed
> (e.g. their flags smudged and their parents simplified). You have "flags"
> field in commit, which implies to me that the design shares this same
> "modified by processing in-place" assumption. It is great for processing
> efficiency as long as you are a "run once and let exit(3) clean-up" type
> of program, but is quite problematic otherwise. commit.flags that C-git
> uses for traversal marker purposes, together with "who are parents and
> children of this commit", should probably be kept inside traversal module,
> if you want to make this truly reusable.
I don't think it's impossible to have something efficient without this
kind of hacks. You just need to dissociate the objects from their
annotations, though use some kind of allocator that allow numbering of
the objects, and use that number as a lookup in an array of annotations.
It will require pool allocators for the annotations, but that should
work fine and efficientely.
> By the way, I hate git_result_t. That should be "int", the most natural
> integral type on the platform.
I concur.
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: libgit2 - a true git library
From: Shawn O. Pearce @ 2008-11-01 0:13 UTC (permalink / raw)
To: Junio C Hamano
Cc: Pieter de Bie, Pierre Habouzit, Nicolas Pitre, git, Scott Chacon
In-Reply-To: <7v63n872bs.fsf@gitster.siamese.dyndns.org>
Junio C Hamano <gitster@pobox.com> wrote:
> I understand that the apidocs/ is a very early work-in-progress, but
> still, it bothers me that it is unclear to me what lifetime rules are in
> effect on the in-core objects.
Yes, this needs a lot more documentation.
> For example, in C-git, commit objects are
> not just parsed but are modified in place as history is traversed
> (e.g. their flags smudged and their parents simplified). You have "flags"
> field in commit, which implies to me that the design shares this same
> "modified by processing in-place" assumption.
Yup. I was assuming the same model, we modify in-place.
> It is great for processing
> efficiency as long as you are a "run once and let exit(3) clean-up" type
> of program, but is quite problematic otherwise. commit.flags that C-git
> uses for traversal marker purposes, together with "who are parents and
> children of this commit", should probably be kept inside traversal module,
> if you want to make this truly reusable.
Its not efficient to keep this data inside of the "traversal module"
instance (aka what I called git_revp_t). You really want it inside
of the commit itself (aka git_commit_t).
My thought here is that git_commit_t's are scoped within a given
git_revp_t that was used when they were parsed. That is:
git_revp_t *pool_a = git_revp_alloc(db, NULL);
git_revp_t *pool_b = git_revp_alloc(db, NULL);
git_oid_t id;
git_commit_t *commit_a, *commit_b, *commit_c;
git_oid_mkstr(&id, "3c223b36af9cace4f802a855fbb588b1dccf0648");
commit_a = git_commit_parse(pool_a, &id);
commit_b = git_commit_parse(pool_b, &id);
commit_c = git_commit_parse(pool_a, &id);
if (commit_a == commit_b)
die("the world just exploded");
else
printf("this was correct behavior\n");
if (commit_a == commit_c)
printf("this was correct behavior\n");
else
die("the hash table is broken");
To completely different git_revp_t's on the same database yeild
different commit pointers, but successive calls to parse the same
commit in the same pool yield the same pointer.
Certain operations on the pool can cause it to alter its state
in a way that cannot be reversed (e.g. rewrite parents). In such
cases the caller should free the pool and alloc a new one in order
to issue new traversals against the same object database, but with
the original (or differently rewritten) parent information.
> By the way, I hate git_result_t. That should be "int", the most natural
> integral type on the platform.
Yea, I'm torn on git_result_t myself. Some library APIs use their
own result type, but as a typedef off int.
I'm tempted to stick with int for the result type, but I don't
want readers to confuse our result type of 0 == success, <0 ==
failure with some case where we return a signed integral value as
a result of a computation.
I'm also debating the error handling. Do we return the error
code as the return value from the function, or do we stick it into
some sort of thread-global like classic "errno", or do we ask the
application to pass in a structure to us?
E.g.:
Return code:
git_result_t r = git_foo_bar(...);
if (r < 0)
die("foo_bar failed: %s", git_strerr(r));
Use an errno:
if (git_foo_bar(...))
die("foo_bar failed: %s", git_strerr(git_errno));
Use a caller allocated struct:
git_error_t err;
if (git_foo_bar(..., &err))
die("foo_bar failed: %s", git_strerr(&err));
I'm slightly leaning towards the result code approach, as it means
we don't have to mess around with thread local variables.
We don't get to pass back anything more complex than an int (possibly
losing context about parameter values and/or on-disk state we want
to report on), but we also don't have to deal with thread-locals
or some messy "always pass a thread context parameter".
--
Shawn.
^ permalink raw reply
* Re: [PATCH] Use find instead of perl in t5000 to get file modification time
From: Johannes Schindelin @ 2008-11-01 0:23 UTC (permalink / raw)
To: Alex Riesen
Cc: Sam Vilain, Git Mailing List, Junio C Hamano, Jeff King,
René Scharfe
In-Reply-To: <20081031233736.GB21997@steel.home>
Hi,
On Sat, 1 Nov 2008, Alex Riesen wrote:
> Johannes Schindelin, Fri, Oct 31, 2008 23:14:59 +0100:
>
> > On Fri, 31 Oct 2008, Alex Riesen wrote:
> >
> > > ActiveState Perl on Windows is portable? To another windows, maybe.
> >
> > /me wonders why you could not use the Perl that ships with Git for
> > Windows, at least for the purposes of Git.
>
> Corporate policy and very-very incompetent IT service. Besides being
> illegal to install actually usable software, there are some of internal
> programs which are quite sensitive to perl (any other perl) appearing
> anywhere on disk, not to mention PATH (the build system, for one). Same
> for cygwin revisions.
>
> I tried using other perl and more modern cygwin, and it can be made to
> work, but it is cumbersome and boring. And one more thing to reinstall
> after IT wipes your computer because you've got a new disk or they
> decided you need winxp instead of w2k, or something else... So I'm just
> trying reduce effort I put into what I'm not supposed to do at work
> anyway.
Well, if you install Git for Windows (as opposed to cygwin), it is minimum
hassle, and Perl is delivered right with it.
Ciao,
Dscho
P.S.: some guys at the GSoC mentor summit convinced me in at least trying
to fix _their_ problems on msysGit, so chances are good I'll fix issues
you would encounter in the same run.
^ permalink raw reply
* Re: [PATHv2 7/8] gitweb: embed snapshot format parameter in PATH_INFO
From: Jakub Narebski @ 2008-11-01 0:18 UTC (permalink / raw)
To: Giuseppe Bilotta; +Cc: git, Petr Baudis, Junio C Hamano
In-Reply-To: <1224426270-27755-2-git-send-email-giuseppe.bilotta@gmail.com>
On Sun, 19 Oct 2008, Giuseppe Bilotta wrote:
I'm sorry for the delay.
> When PATH_INFO is active, get rid of the sf CGI parameter by embedding
> the snapshot format information in the PATH_INFO URL, in the form of an
> appropriate extension.
The question is: should we use format suffix (e.g. 'tar.gz'),
or format name (e.g. 'tgz')? The latter is later easier to parse,
see comments to first patch in better snapshot support for path_info.
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index e9e9e60..5fd5a1f 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -795,6 +795,7 @@ sub href (%) {
> # - action
> # - hash_parent or hash_parent_base:/file_parent
> # - hash or hash_base:/file_name
> + # - the snapshot_format as an appropriate suffix
>
> # When the script is the root DirectoryIndex for the domain,
> # $href here would be something like http://gitweb.example.com/
Good.
> @@ -806,6 +807,10 @@ sub href (%) {
> $href .= "/".esc_url($params{'project'}) if defined $params{'project'};
> delete $params{'project'};
>
> + # since we destructively absorb parameters, we keep this
> + # boolean that remembers if we're handling a snapshot
> + my $is_snapshot = $params{'action'} eq 'snapshot';
> +
Side note: we destructively absorb parameters, because parameters
which are not absorbed are then used to generate query string part
of URL.
Deleting parameter but remembering the fact that it was used is one
(but not only) solution.
> # Summary just uses the project path URL, any other action is
> # added to the URL
> if (defined $params{'action'}) {
> @@ -845,6 +850,22 @@ sub href (%) {
> $href .= esc_url($params{'hash'});
> delete $params{'hash'};
> }
> +
> + # If the action was a snapshot, we can absorb the
> + # snapshot_format parameter too
> + if ($is_snapshot) {
> + my $fmt = $params{'snapshot_format'};
> + # snapshot_format should always be defined when href()
> + # is called, but just in case some code forgets, we
> + # fall back to the default
> + if (!$fmt) {
> + my @snapshot_fmts = gitweb_check_feature('snapshot');
> + @snapshot_fmts = filter_snapshot_fmts(@snapshot_fmts);
> + $fmt = $snapshot_fmts[0];
> + }
I anderstand that the above code is improved with new patch?
> + $href .= $known_snapshot_formats{$fmt}{'suffix'};
Again: should we use snapshot prefix, or snapshot name, which means here
do we use $known_snapshot_formats{$fmt}{'suffix'}; or just $fmt; ?
> + delete $params{'snapshot_format'};
> + }
> }
>
> # now encode the parameters explicitly
> --
> 1.5.6.5
>
>
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: libgit2 - a true git library
From: Bruno Santos @ 2008-10-31 23:18 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20081031170704.GU14786@spearce.org>
Shawn O. Pearce wrote:
> During the GitTogether we were kicking around the idea of a ground-up
> implementation of a Git library. This may be easier than trying
> to grind down git.git into a library, as we aren't tied to any
> of the current global state baggage or the current die() based
> error handling.
>
> I've started an _extremely_ rough draft. The code compiles into a
> libgit.a but it doesn't even implement what it describes in the API,
> let alone a working Git implementation. Really what I'm trying to
> incite here is some discussion on what the API looks like.
>
> API Docs:
> http://www.spearce.org/projects/scm/libgit2/apidocs/html/modules.html
>
> Source Code Clone URL:
> http://www.spearce.org/projects/scm/libgit2/libgit2.git
>
We should take the opportunity a make it more portable. Instead of using
the posix api directly we should warp it in "git_" APIs. And be carefull
with certain APIs like fork or fork+exec and instead provided a more
generic solution: for fork one that would use the best solution in the
given platform, either by forking or threading; and for fork+exec a
generic create_process/run_command.
Here's an example, for the 'read' API, on how we can simply do this
without worries for the posix crowd:
ssize_t git_read(git_fildes_t fildes, void* buf, size_t bufsize);
Were git_fildes_t would be an int for posix and an HANDLE for win32.
For the posix case git_read can be simply inlined and we get zero overhead:
static inline ssize_t git_read(git_fildes_t fildes, void *buf,
size_t bufsize)
{
return read(fildes, buf, bufsize);
}
And for the win32 case it would be much more easier to implement the
equivalent, something like:
ssize_t git_read(git_fildes_t fildes, void *buf, size_t bufsize)
{
DWORD rd;
if (!ReadFile(fildes, buf, bufsize, &rd, NULL)) {
//translate win32 error to errno
return -1;
}
return rd;
}
Of course, there is also the issue of using the c runtime on win32, but
that problem can be easily solved outside git, provided that we don't
use a 'fileno' like API.
Bruno Santos
^ permalink raw reply
* Re: libgit2 - a true git library
From: Shawn O. Pearce @ 2008-11-01 0:19 UTC (permalink / raw)
To: Pierre Habouzit
Cc: Junio C Hamano, Pieter de Bie, Nicolas Pitre, git, Scott Chacon
In-Reply-To: <20081101000213.GB29036@artemis.corp>
Pierre Habouzit <madcoder@debian.org> wrote:
> On Fri, Oct 31, 2008 at 11:49:11PM +0000, Junio C Hamano wrote:
> >
> > I understand that the apidocs/ is a very early work-in-progress, but
> > still, it bothers me that it is unclear to me what lifetime rules are in
> > effect on the in-core objects. For example, in C-git, commit objects are
> > not just parsed but are modified in place as history is traversed
> > (e.g. their flags smudged and their parents simplified). You have "flags"
> > field in commit, which implies to me that the design shares this same
> > "modified by processing in-place" assumption.
>
> I don't think it's impossible to have something efficient without this
> kind of hacks. You just need to dissociate the objects from their
> annotations, though use some kind of allocator that allow numbering of
> the objects, and use that number as a lookup in an array of annotations.
> It will require pool allocators for the annotations, but that should
> work fine and efficientely.
Interesting approach. I don't know why I didn't think of that one.
You'll still need to be able to toss parts of the git graph though.
If you just pin everything in memory under a single global object
table you'll run server processes out of memory as they chug through
large numbers of repositories.
> > By the way, I hate git_result_t. That should be "int", the most natural
> > integral type on the platform.
>
> I concur.
int it is then.
--
Shawn.
^ permalink raw reply
* Re: [PATCH] Documentation: add a planning document for the next CLI revamp
From: Johannes Schindelin @ 2008-11-01 0:37 UTC (permalink / raw)
To: Pierre Habouzit
Cc: Mike Hommey, Shawn O. Pearce, Nicolas Pitre, Sam Vilain, git
In-Reply-To: <20081030150135.GG24098@artemis.corp>
Hi,
On Thu, 30 Oct 2008, Pierre Habouzit wrote:
> On Thu, Oct 30, 2008 at 02:59:28PM +0000, Mike Hommey wrote:
> > On Thu, Oct 30, 2008 at 07:52:53AM -0700, Shawn O. Pearce <spearce@spearce.org> wrote:
> > > +1 to Nico's NAK.
> > >
> > > Although I was at the GitTogether I don't remember this change to
> > > checkout being discussed. I must have been asleep reading email
> > > or something. I am _NOT_ in favor of this change; I think the
> > > current behavior of "git checkout origin/master" is correct and as
> > > sane as we can make it.
> >
> > Except he was talking about 'git checkout branch', not 'git checkout
> > origin/branch'. And I would be fine with 'git checkout branch' doing
> > what 'git checkout -b branch $remote/branch' does if $remote is unique
> > (i.e. there is no other 'branch' branch in any other remote) and the
> > 'branch' branch doesn't already exist.
>
> Seconded.
>
> Having git-checkout $foo being a shorthand for git checkout -b $foo
> origin/$foo when origin/$foo exists and $foo doesn't is definitely handy.
Have you guys actually checked out what
git checkout -t origin/foo
does?
Ciao,
Dscho
^ permalink raw reply
* Re: libgit2 - a true git library
From: david @ 2008-11-01 0:41 UTC (permalink / raw)
To: Shawn O. Pearce
Cc: Junio C Hamano, Nicolas Pitre, Pierre Habouzit, git, Scott Chacon
In-Reply-To: <20081031234115.GD14786@spearce.org>
On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
> Junio C Hamano <gitster@pobox.com> wrote:
>>
>>
>> I.e. use the supplied custom function to do proprietary magic, such as
>> reading the object lazily from elsewhere over the network. And we will
>> never get that magic bit back.
>
> As a maintainer I'd never accept such a patch. I'd ask for the
> code under read_object_custom, or toss the patch on the floor.
> But that doesn't stop them from distributing the patched sources
> like above, keeping the fun bits in the closed source portion of
> the executable they distribute.
>
> Maybe I just think too highly of the other guy, but I'd hope that
> anyone patching libgit2 like above would try to avoid it, because
> they'd face merge issues in the future.
the issue that I see is that libgit2 will be (on most systems) a shared
library.
what's to stop someone from taking the libgit2 code, adding the magic
proprietary piece, and selling a new libgit2 library binary 'just replace
your existing shared library with this new one and all your git related
programs gain this feature'
they would only face merge issues if they need to keep up to date with
you, and git makes it pretty easy to maintain a fork if you only have to
do one-way merging (rere)
David Lang
^ permalink raw reply
* Re: libgit2 - a true git library
From: Pierre Habouzit @ 2008-11-01 1:02 UTC (permalink / raw)
To: Shawn O. Pearce
Cc: Junio C Hamano, Pieter de Bie, Nicolas Pitre, git, Scott Chacon
In-Reply-To: <20081101001926.GF14786@spearce.org>
[-- Attachment #1: Type: text/plain, Size: 5028 bytes --]
On Sat, Nov 01, 2008 at 12:19:26AM +0000, Shawn O. Pearce wrote:
> Pierre Habouzit <madcoder@debian.org> wrote:
> > On Fri, Oct 31, 2008 at 11:49:11PM +0000, Junio C Hamano wrote:
> > >
> > > I understand that the apidocs/ is a very early work-in-progress, but
> > > still, it bothers me that it is unclear to me what lifetime rules are in
> > > effect on the in-core objects. For example, in C-git, commit objects are
> > > not just parsed but are modified in place as history is traversed
> > > (e.g. their flags smudged and their parents simplified). You have "flags"
> > > field in commit, which implies to me that the design shares this same
> > > "modified by processing in-place" assumption.
> >
> > I don't think it's impossible to have something efficient without this
> > kind of hacks. You just need to dissociate the objects from their
> > annotations, though use some kind of allocator that allow numbering of
> > the objects, and use that number as a lookup in an array of annotations.
> > It will require pool allocators for the annotations, but that should
> > work fine and efficientely.
>
> Interesting approach. I don't know why I didn't think of that one.
>
> You'll still need to be able to toss parts of the git graph though.
> If you just pin everything in memory under a single global object
> table you'll run server processes out of memory as they chug through
> large numbers of repositories.
Sure, but for that you just need to reinject the numbers into some kind
of free list (hint a bitmap) to reuse old slots. Of course this is some
kind of take-once never-release approach _BUT_ one can do better and
"defrag" this at times.
E.g. for a server if we take your idea, there are some times we probably
*know* nobody has kept a reference to one of the pointers and we can
reorganize some pointers around to free chunks of data not allocated.
An escape way is to use mmap + madvise for those pools, the former to
allocate the memory and the latter to drop large unused ranges when
needed (remapping with MAP_FIXED is also supposed to work but fails on
Macos it seems). Even win32 has what it takes to do so (just skim
through the code of jemalloc, that is used for mozilla, it's quite
portable on POSIX + Windows).
I was thinking that one should create and register pools of annotations
as such, define the size of the annotation (IOW the size of cell), and
let deal with that. I imagine the stuff as some kind of allocator that
would allocate the first time e.g. 4k of objects, and if you need more
8k, and if need more 16k and so on exponentially. Mapping an integer to
a cell in this kind of ropes is _really_ efficient: you need to know the
first bit set (__builtin_clz is of help with gcc, it can be emulated
with proper asm for many non-gcc platforms also) to give you the number
of the rope component that you intend to address, you clear that bit,
the resulting number is the index in that rope component of the cell
you're interested in. On most machines such an operation would be a few
cycles, which should be fairly little wrt the operation you would do
during a traversal.
Maintaining the bitmap is important, so that we can give back large
chunks of physical memory back to the system, and many malloc
implementations would likely use such kind of implementations, we would
just do our (which is arguably heavy) but gives us control on the cell
number, which is just what we need.
Of _course_ when we have a better / more natural place to put
annotations we need, let's do it ! But the kind of thing I just imagined
is more generic and can help to do arbitrary complex stuff during a
traversal.
We may want to explore a storage that is aware of the objects type also,
as I expect the liveness of objects to depend on the objects type a lot,
and some traversal to not care about some kind of objects at all, which
when combined with lazy allocation in the annotations pools, could end
up with reduced memory footprint. We could e.g. use two bits of the
"object handler" to store the type and dispatch into 3 (commit, tree,
blob) ropes instead of a big one.
The object and annotations store _is_ what makes git efficient, and is
the very service the library will have to support well. We _will_ have
to put some clever code in there and we cannot sacrifice any kind of
performance, this will be the tight loop every time. I don't think
there will be anything else in git that is so central for performance.
On the other hand, it's probably ok for the first versions of the
library to have some things in it that don't perform well in long lived
processes (repacking e.g., if we ever want to do that in the library, as
it's slow and that forking a git-repack away should not be a too
expensive cost anyway).
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: libgit2 - a true git library
From: Shawn O. Pearce @ 2008-11-01 1:00 UTC (permalink / raw)
To: david; +Cc: git
In-Reply-To: <alpine.DEB.1.10.0810311738100.5851@asgard.lang.hm>
david@lang.hm wrote:
> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
>> Junio C Hamano <gitster@pobox.com> wrote:
>>>
>>> I.e. use the supplied custom function to do proprietary magic, such as
>>> reading the object lazily from elsewhere over the network. And we will
>>> never get that magic bit back.
>>
>> Maybe I just think too highly of the other guy, but I'd hope that
>> anyone patching libgit2 like above would try to avoid it, because
>> they'd face merge issues in the future.
>
> the issue that I see is that libgit2 will be (on most systems) a shared
> library.
>
> what's to stop someone from taking the libgit2 code, adding the magic
> proprietary piece, and selling a new libgit2 library binary 'just replace
> your existing shared library with this new one and all your git related
> programs gain this feature'
True. The only thing that prevents that is the normal GPL. The
LGPL and GPL+"gcc exception" allow this sort of mean behavior.
I doubt there's enough of a market for that; replacing a library
is something of a pain and if the feature really is interesting or
useful someone will write a clean-room re-implementation and submit
patches to do the same thing.
> they would only face merge issues if they need to keep up to date with
> you, and git makes it pretty easy to maintain a fork if you only have to
> do one-way merging (rere)
In other words, we're too good for our own good. ;-)
--
Shawn.
^ permalink raw reply
* Re: libgit2 - a true git library
From: david @ 2008-11-01 1:04 UTC (permalink / raw)
To: Shawn O. Pearce; +Cc: git
In-Reply-To: <20081101010011.GG14786@spearce.org>
On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
> david@lang.hm wrote:
>> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
>>> Junio C Hamano <gitster@pobox.com> wrote:
>>>>
>>>> I.e. use the supplied custom function to do proprietary magic, such as
>>>> reading the object lazily from elsewhere over the network. And we will
>>>> never get that magic bit back.
>>>
>>> Maybe I just think too highly of the other guy, but I'd hope that
>>> anyone patching libgit2 like above would try to avoid it, because
>>> they'd face merge issues in the future.
>>
>> the issue that I see is that libgit2 will be (on most systems) a shared
>> library.
>>
>> what's to stop someone from taking the libgit2 code, adding the magic
>> proprietary piece, and selling a new libgit2 library binary 'just replace
>> your existing shared library with this new one and all your git related
>> programs gain this feature'
>
> True. The only thing that prevents that is the normal GPL. The
> LGPL and GPL+"gcc exception" allow this sort of mean behavior.
> I doubt there's enough of a market for that; replacing a library
> is something of a pain and if the feature really is interesting or
> useful someone will write a clean-room re-implementation and submit
> patches to do the same thing.
how would the LGPL of GPL+gcc extention allow this? if they modify the
code in the library and then distribute the modified library wouldn't they
be required to distribute the changes to that library?
they could use the LGPL or GPL+exception library with their propriatary
program, but I don't see how they could get away with modifying the
library.
David Lang
^ permalink raw reply
* Re: libgit2 - a true git library
From: Pierre Habouzit @ 2008-11-01 1:06 UTC (permalink / raw)
To: david; +Cc: Shawn O. Pearce, Junio C Hamano, Nicolas Pitre, git, Scott Chacon
In-Reply-To: <alpine.DEB.1.10.0810311738100.5851@asgard.lang.hm>
[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]
On Sat, Nov 01, 2008 at 12:41:22AM +0000, david@lang.hm wrote:
> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
>
> >Junio C Hamano <gitster@pobox.com> wrote:
> >>
> >>
> >>I.e. use the supplied custom function to do proprietary magic, such as
> >>reading the object lazily from elsewhere over the network. And we will
> >>never get that magic bit back.
> >
> >As a maintainer I'd never accept such a patch. I'd ask for the
> >code under read_object_custom, or toss the patch on the floor.
> >But that doesn't stop them from distributing the patched sources
> >like above, keeping the fun bits in the closed source portion of
> >the executable they distribute.
> >
> >Maybe I just think too highly of the other guy, but I'd hope that
> >anyone patching libgit2 like above would try to avoid it, because
> >they'd face merge issues in the future.
>
> the issue that I see is that libgit2 will be (on most systems) a shared
> library.
>
> what's to stop someone from taking the libgit2 code, adding the magic
> proprietary piece, and selling a new libgit2 library binary 'just replace
> your existing shared library with this new one and all your git related
> programs gain this feature'
Its license. GPL even with GCC exception would not allow you to do that.
Though they could propose a fork of the library patched, with the patch
distributed. The downside would be that their code would not be binary
compatible with the "true" libgit2, so they would probably have to
change the name to avoid namespace clashes, or overwrite the "real"
library.
But yes, it's theoretically feasible. I'm not sure it would be worth the
hassle, and if they respect the license (if they don't they can already
do that with the current git anyway) then the fact that someone would
want to do something like that would be known fact, probably not
avoided, but known.
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: libgit2 - a true git library
From: Pierre Habouzit @ 2008-11-01 1:08 UTC (permalink / raw)
To: david; +Cc: Shawn O. Pearce, git
In-Reply-To: <alpine.DEB.1.10.0810311802360.5851@asgard.lang.hm>
[-- Attachment #1: Type: text/plain, Size: 2225 bytes --]
On Sat, Nov 01, 2008 at 01:04:50AM +0000, david@lang.hm wrote:
> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
>
> >david@lang.hm wrote:
> >>On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
> >>>Junio C Hamano <gitster@pobox.com> wrote:
> >>>>
> >>>>I.e. use the supplied custom function to do proprietary magic, such
> >>>>as
> >>>>reading the object lazily from elsewhere over the network. And we
> >>>>will
> >>>>never get that magic bit back.
> >>>
> >>>Maybe I just think too highly of the other guy, but I'd hope that
> >>>anyone patching libgit2 like above would try to avoid it, because
> >>>they'd face merge issues in the future.
> >>
> >>the issue that I see is that libgit2 will be (on most systems) a shared
> >>library.
> >>
> >>what's to stop someone from taking the libgit2 code, adding the magic
> >>proprietary piece, and selling a new libgit2 library binary 'just
> >>replace
> >>your existing shared library with this new one and all your git related
> >>programs gain this feature'
> >
> >True. The only thing that prevents that is the normal GPL. The
> >LGPL and GPL+"gcc exception" allow this sort of mean behavior.
> >I doubt there's enough of a market for that; replacing a library
> >is something of a pain and if the feature really is interesting or
> >useful someone will write a clean-room re-implementation and submit
> >patches to do the same thing.
>
> how would the LGPL of GPL+gcc extention allow this? if they modify the
> code in the library and then distribute the modified library wouldn't
> they be required to distribute the changes to that library?
See junio's example. It's rather easy to add hooks into the library to
implement a feature outside of it. It's even possible to do it while
preserving the ABI fully IMHO (by being a strict superset of it).
The patch would be so trivial, that I see no reason why they wouldn't
provide it. Though the real implementation of the feature that would be
delegated through it would be in their closed source stuff.
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: libgit2 - a true git library
From: Nicolas Pitre @ 2008-11-01 1:15 UTC (permalink / raw)
To: Shawn O. Pearce
Cc: Junio C Hamano, Pieter de Bie, Pierre Habouzit, git, Scott Chacon
In-Reply-To: <20081101001300.GE14786@spearce.org>
On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
> I'm tempted to stick with int for the result type, but I don't
> want readers to confuse our result type of 0 == success, <0 ==
> failure with some case where we return a signed integral value as
> a result of a computation.
Does this actually happen ?
> I'm also debating the error handling. Do we return the error
> code as the return value from the function, or do we stick it into
> some sort of thread-global like classic "errno", or do we ask the
> application to pass in a structure to us?
Passing a structure pointer for errors is adding overhead to the API in
all cases, please don't do that.
Both the negative code and errno style are lightweight in the common "no
error" case. The errno style is probably more handy for those functions
returning a pointer which should be NULL in the error case.
Nicolas
^ permalink raw reply
* Re: libgit2 - a true git library
From: Shawn O. Pearce @ 2008-11-01 1:19 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Junio C Hamano, Pieter de Bie, Pierre Habouzit, git, Scott Chacon
In-Reply-To: <alpine.LFD.2.00.0810312106310.13034@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
>
> > I'm tempted to stick with int for the result type, but I don't
> > want readers to confuse our result type of 0 == success, <0 ==
> > failure with some case where we return a signed integral value as
> > a result of a computation.
>
> Does this actually happen ?
Sometimes, but yea, its not that often. I think we've already
settled that it shall be "int".
I'm updating some stuff like that (and dropping _t) as I write
this email.
> > I'm also debating the error handling. Do we return the error
> > code as the return value from the function, or do we stick it into
> > some sort of thread-global like classic "errno", or do we ask the
> > application to pass in a structure to us?
>
> Passing a structure pointer for errors is adding overhead to the API in
> all cases, please don't do that.
Agreed. Not going there.
> Both the negative code and errno style are lightweight in the common "no
> error" case. The errno style is probably more handy for those functions
> returning a pointer which should be NULL in the error case.
I'm sticking with return a negative code for now, to the extent
that some functions which return a pointer but also have many
common failure modes (e.g. git_odb_open) use an output parameter
as their first arg, so the error code can be returned as the result
of the function.
--
Shawn.
^ permalink raw reply
* Re: libgit2 - a true git library
From: Nicolas Pitre @ 2008-11-01 1:33 UTC (permalink / raw)
To: Pierre Habouzit; +Cc: david, Shawn O. Pearce, git
In-Reply-To: <20081101010824.GE29036@artemis.corp>
On Sat, 1 Nov 2008, Pierre Habouzit wrote:
> See junio's example. It's rather easy to add hooks into the library to
> implement a feature outside of it. It's even possible to do it while
> preserving the ABI fully IMHO (by being a strict superset of it).
>
> The patch would be so trivial, that I see no reason why they wouldn't
> provide it. Though the real implementation of the feature that would be
> delegated through it would be in their closed source stuff.
But at that point it's a matter of public perception. It would clearly
be against the spirit of the license even though the license itself
couldn't prevent such tortuous practices. Those people doing such
things would clearly be identified as bad guys and get bad press, and
libgit contributors could even attempt law suits based on the derived
work angle. That might be just enough to prevent such things to happen.
OTOH they would certainly come out clean if the license was BSD since
the spirit of that license explicitly allows closing up the whole
library and adding extra features.
Nicolas
^ permalink raw reply
* Re: libgit2 - a true git library
From: david @ 2008-11-01 1:36 UTC (permalink / raw)
To: Pierre Habouzit
Cc: Shawn O. Pearce, Junio C Hamano, Nicolas Pitre, git, Scott Chacon
In-Reply-To: <20081101010620.GD29036@artemis.corp>
On Sat, 1 Nov 2008, Pierre Habouzit wrote:
> On Sat, Nov 01, 2008 at 12:41:22AM +0000, david@lang.hm wrote:
>> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
>>
>>> Junio C Hamano <gitster@pobox.com> wrote:
>>>>
>>>>
>>>> I.e. use the supplied custom function to do proprietary magic, such as
>>>> reading the object lazily from elsewhere over the network. And we will
>>>> never get that magic bit back.
>>>
>>> As a maintainer I'd never accept such a patch. I'd ask for the
>>> code under read_object_custom, or toss the patch on the floor.
>>> But that doesn't stop them from distributing the patched sources
>>> like above, keeping the fun bits in the closed source portion of
>>> the executable they distribute.
>>>
>>> Maybe I just think too highly of the other guy, but I'd hope that
>>> anyone patching libgit2 like above would try to avoid it, because
>>> they'd face merge issues in the future.
>>
>> the issue that I see is that libgit2 will be (on most systems) a shared
>> library.
>>
>> what's to stop someone from taking the libgit2 code, adding the magic
>> proprietary piece, and selling a new libgit2 library binary 'just replace
>> your existing shared library with this new one and all your git related
>> programs gain this feature'
>
> Its license. GPL even with GCC exception would not allow you to do that.
> Though they could propose a fork of the library patched, with the patch
> distributed. The downside would be that their code would not be binary
> compatible with the "true" libgit2, so they would probably have to
> change the name to avoid namespace clashes, or overwrite the "real"
> library.
the proposal had been for the library to be BSD, not GPL. and I don't see
any reason why such a thing couldn't be done with a BSE library.
David Lang
> But yes, it's theoretically feasible. I'm not sure it would be worth the
> hassle, and if they respect the license (if they don't they can already
> do that with the current git anyway) then the fact that someone would
> want to do something like that would be known fact, probably not
> avoided, but known.
>
>
^ permalink raw reply
* Re: libgit2 - a true git library
From: Pierre Habouzit @ 2008-11-01 1:38 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: david, Shawn O. Pearce, git
In-Reply-To: <alpine.LFD.2.00.0810312121000.13034@xanadu.home>
[-- Attachment #1: Type: text/plain, Size: 592 bytes --]
On Sat, Nov 01, 2008 at 01:33:51AM +0000, Nicolas Pitre wrote:
> OTOH they would certainly come out clean if the license was BSD since
> the spirit of that license explicitly allows closing up the whole
> library and adding extra features.
I think it's pretty clear most contributor from git.git would be driven
away if the license isn't GPLish. So I was basing my answer on a
supposition it would be.
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply
* Re: libgit2 - a true git library
From: Shawn O. Pearce @ 2008-11-01 1:43 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Pierre Habouzit, david, git
In-Reply-To: <alpine.LFD.2.00.0810312121000.13034@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Sat, 1 Nov 2008, Pierre Habouzit wrote:
>
> > See junio's example. It's rather easy to add hooks into the library to
> > implement a feature outside of it. It's even possible to do it while
> > preserving the ABI fully IMHO (by being a strict superset of it).
> >
> > The patch would be so trivial, that I see no reason why they wouldn't
> > provide it. Though the real implementation of the feature that would be
> > delegated through it would be in their closed source stuff.
>
> But at that point it's a matter of public perception. It would clearly
> be against the spirit of the license even though the license itself
> couldn't prevent such tortuous practices. Those people doing such
> things would clearly be identified as bad guys and get bad press, and
> libgit contributors could even attempt law suits based on the derived
> work angle. That might be just enough to prevent such things to happen.
Yes.
> OTOH they would certainly come out clean if the license was BSD since
> the spirit of that license explicitly allows closing up the whole
> library and adding extra features.
My take on the consensus for the license part of the discussion is
that libgit2 should be under the "GPL gcc library" license.
BTW, I can't actually find a copy of that license; the only thing
I can locate in the GCC SVN tree is a copy of the LGPL.
--
Shawn.
^ permalink raw reply
* Re: libgit2 - a true git library
From: Nicolas Pitre @ 2008-11-01 1:45 UTC (permalink / raw)
To: Shawn O. Pearce
Cc: Junio C Hamano, Pieter de Bie, Pierre Habouzit, git, Scott Chacon
In-Reply-To: <20081101011910.GH14786@spearce.org>
On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
> > Both the negative code and errno style are lightweight in the common "no
> > error" case. The errno style is probably more handy for those functions
> > returning a pointer which should be NULL in the error case.
>
> I'm sticking with return a negative code for now, to the extent
> that some functions which return a pointer but also have many
> common failure modes (e.g. git_odb_open) use an output parameter
> as their first arg, so the error code can be returned as the result
> of the function.
Actually, the pointer-returning functions can encode error cases into a
"negative" pointer. See include/linux/err.h for example.
void *ptr = git_alloc_foo(...);
if (IS_ERR(ptr))
die("git_alloc_foo failed: %s", git_strerr(PTR_ERR(ptr)));
Nicolas
^ permalink raw reply
* Re: libgit2 - a true git library
From: Nicolas Pitre @ 2008-11-01 1:49 UTC (permalink / raw)
To: Pierre Habouzit; +Cc: david, Shawn O. Pearce, git
In-Reply-To: <20081101013815.GF29036@artemis.corp>
On Sat, 1 Nov 2008, Pierre Habouzit wrote:
> On Sat, Nov 01, 2008 at 01:33:51AM +0000, Nicolas Pitre wrote:
> > OTOH they would certainly come out clean if the license was BSD since
> > the spirit of that license explicitly allows closing up the whole
> > library and adding extra features.
>
> I think it's pretty clear most contributor from git.git would be driven
> away if the license isn't GPLish. So I was basing my answer on a
> supposition it would be.
Sure. My point is the difference in guiltiness. Even if a LGPL license
doesn't prevent all abuses, it at least allows for public denunciations
at the very least.
Nicolas
^ permalink raw reply
* Re: libgit2 - a true git library
From: Shawn O. Pearce @ 2008-11-01 1:52 UTC (permalink / raw)
To: Nicolas Pitre
Cc: Junio C Hamano, Pieter de Bie, Pierre Habouzit, git, Scott Chacon
In-Reply-To: <alpine.LFD.2.00.0810312135190.13034@xanadu.home>
Nicolas Pitre <nico@cam.org> wrote:
> On Fri, 31 Oct 2008, Shawn O. Pearce wrote:
>
> > > Both the negative code and errno style are lightweight in the common "no
> > > error" case. The errno style is probably more handy for those functions
> > > returning a pointer which should be NULL in the error case.
> >
> > I'm sticking with return a negative code for now, to the extent
> > that some functions which return a pointer but also have many
> > common failure modes (e.g. git_odb_open) use an output parameter
> > as their first arg, so the error code can be returned as the result
> > of the function.
>
> Actually, the pointer-returning functions can encode error cases into a
> "negative" pointer. See include/linux/err.h for example.
>
> void *ptr = git_alloc_foo(...);
> if (IS_ERR(ptr))
> die("git_alloc_foo failed: %s", git_strerr(PTR_ERR(ptr)));
Oh, good point. We could also stagger the errors so they are
always odd, and never return an odd-alignment pointer from a
successful function. Thus IS_ERR can be written as:
#define IS_ERR(ptr) (((intptr_t)(ptr)) & 1)
which is quite cheap, and given the (probably required anyway)
aligned allocation policy means we still have 2^31 possible
error codes available.
--
Shawn.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox