* Re: git clone over http
From: Josh England @ 2007-09-04 16:54 UTC (permalink / raw)
To: Robin Rosenberg; +Cc: git
In-Reply-To: <200709021123.04218.robin.rosenberg.lists@dewire.com>
On Sun, 2007-09-02 at 11:23 +0200, Robin Rosenberg wrote:
> git clone (1.5.3) with http is somewhat unreliable. I've noticed if
> can actually give
> me different versions of a branch each time I run it, eventually yielding
> the one I'm expecting. And now this:
I don't see this behavior, but I do see other wierdness cloning via
http. Anytime I clone a repo (any repo) via http from behind the
firewall/proxy at work I never get any branches. 'git branch' only ever
shows 'master'. I can't checkout and work on any other branch. I can
work around the problem by cloning on a box that is more open, but was
wondering if anyone knows how to fix this.
-JE
^ permalink raw reply
* Re: Git's database structure
From: Andreas Ericsson @ 2007-09-04 16:51 UTC (permalink / raw)
To: Jon Smirl; +Cc: Git Mailing List
In-Reply-To: <9e4733910709040947ia32bda4i6e30efb2d7848308@mail.gmail.com>
Jon Smirl wrote:
> On 9/4/07, Andreas Ericsson <ae@op5.se> wrote:
>> Jon Smirl wrote:
>>> Another way of looking at the problem,
>>>
>>> Let's build a full-text index for git. You put a string into the index
>>> and it returns the SHAs of all the file nodes that contain the string.
>>> How do I recover the path names of these SHAs?
>>>
>> I wouldn't know, but presumably any table can have more than one column.
>>
>> Is this a problem you face with git so often that it requires a complete
>> re-design of its very core?
>
> That's the whole point. We need to discuss the impact of merging a
> field (path names) with an index (tree nodes) has on future things we
> may want to do with the data stored in git.
>
Yes, but as nobody seems to know what those future things are, it feels
rather pointless speculating about adding support to git for them. git
is a tool. It's a great one at that, because it was built to solve a
particular problem, which it does an amazing job at.
Other SCM's which had the potential to become amazingly good tools too
drowned somewhere between prototype and product in a sea of intellectual
masturbation, which had little to do with solving real-world problems.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: Git's database structure
From: Jon Smirl @ 2007-09-04 16:47 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Git Mailing List
In-Reply-To: <46DD887D.3090508@op5.se>
On 9/4/07, Andreas Ericsson <ae@op5.se> wrote:
> Jon Smirl wrote:
> > Another way of looking at the problem,
> >
> > Let's build a full-text index for git. You put a string into the index
> > and it returns the SHAs of all the file nodes that contain the string.
> > How do I recover the path names of these SHAs?
> >
>
> I wouldn't know, but presumably any table can have more than one column.
>
> Is this a problem you face with git so often that it requires a complete
> re-design of its very core?
That's the whole point. We need to discuss the impact of merging a
field (path names) with an index (tree nodes) has on future things we
may want to do with the data stored in git.
Databases don't usually blend fields/indexes without also duplicating
the field in the table. You need all the fields in the table so that
it is possible to create indexes on other fields.
>
> --
> Andreas Ericsson andreas.ericsson@op5.se
> OP5 AB www.op5.se
> Tel: +46 8-230225 Fax: +46 8-230231
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: [PATCH] Add a new lstat and fstat implementation based on Win32 API
From: Marius Storm-Olsen @ 2007-09-04 16:32 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Johannes Sixt, Johannes Sixt, Git Mailing List
In-Reply-To: <Pine.LNX.4.64.0709041547240.28586@racer.site>
[-- Attachment #1: Type: text/plain, Size: 1079 bytes --]
Johannes Schindelin wrote:
> On Tue, 4 Sep 2007, Marius Storm-Olsen wrote:
>> Johannes Schindelin wrote:
>>> Bad news. I do not know if it was the newest version I tried,
>>> but I could no longer fetch... said something about some bad
>>> file.
>> Then you're missing this patch:
>> http://repo.or.cz/w/git/mingw/4msysgit.git?a=commitdiff;h=f15974add93bdfa92775c77c00e7c65aefd42127
>>
>> I guess the quickest way is to manually apply this patch and
>> recompile. (or add the '#undef fstat', and have git_fstat just
>> 'return fstat(fd, buf)')
>>
>> The problem is that without this patch fstat(0, buf) would fail
>> with bad filedescriptor instead of returning the st_mode = S_IFIFO.
>
> I guessed as much, but could not fetch the patch, since fetch was
> broken ;-)
Heh, yeah, that's happened to me too before. One neat thing with this
gitweb thingy though, is that you can actually grab the patch, like this
http://repo.or.cz/w/git/mingw/4msysgit.git?a=commitdiff_plain;h=f15974add93bdfa92775c77c00e7c65aefd42127
No excuse! ;-)
--
.marius
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 187 bytes --]
^ permalink raw reply
* Re: Git's database structure
From: Andreas Ericsson @ 2007-09-04 16:31 UTC (permalink / raw)
To: Jon Smirl; +Cc: Git Mailing List
In-Reply-To: <9e4733910709040928n6535e49esaf713b2c63ba0831@mail.gmail.com>
Jon Smirl wrote:
> Another way of looking at the problem,
>
> Let's build a full-text index for git. You put a string into the index
> and it returns the SHAs of all the file nodes that contain the string.
> How do I recover the path names of these SHAs?
>
I wouldn't know, but presumably any table can have more than one column.
Is this a problem you face with git so often that it requires a complete
re-design of its very core?
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: Git's database structure
From: Andreas Ericsson @ 2007-09-04 16:29 UTC (permalink / raw)
To: Jon Smirl; +Cc: Git Mailing List
In-Reply-To: <9e4733910709040919u3d252b91s2785ed4d20086c88@mail.gmail.com>
Jon Smirl wrote:
> On 9/4/07, Andreas Ericsson <ae@op5.se> wrote:
>> Jon Smirl wrote:
>>> Let's back up a little bit from "Caclulating tree node". What are the
>>> elements of git's data structures?
>>>
>>> Right now we have an index structure (tree nodes) integrated in to a
>>> base table. Integrating indexing into the data is not normally done in
>>> a database. Doing a normalization analysis like this may expose flaws
>>> in the way the data is structured. Of course we may also decide to
>>> leave everything the way it is.
>>>
>>> What about the special status of a rename? In the current model we
>>> effectively have three tables.
>>>
>>> commit - a set of all SHAs in the commit, previous commit, comment, author, etc
>>> blob - a file, permissions, etc.
>>> file names - name, SHA
>> commit - SHA1 of its parent(s) and its root-tree, along with
>> author info and a free-form field
>> blob - content addressable by *multiple trees*
>> file names - List of path-names inside a tree object.
>>
>>
>> To draw some sort of relationship model here, you'd have
>>
>> commit 1<->M roottree
>> tree M<->M tree
>> tree M<->M blob
>
> By introducing tree nodes you have blended a specific indexing scheme
> into the data. There are many other ways the path names could be
> indexed hash tables, binary trees, etc.
>
> This problem exists in files systems. Since the path names have been
> encoded into the directory structures there is no way to query
> something like "all files created yesterday" from a file system
> without building another mapping table or a brute force search. I keep
> using Google as an example, Google is indexing hierarchical URLs but
> they do not use a hierarchical index to do it.
>
Pathnames are by far the most common search-/delimiting criteria for
git though, so I fail to see why this is a problem for you.
> Databases keep the knowledge of how things are indexed out of the
> data. A data structure analysis of git should remove the blended index
> and start from the set theory.
>
Why? This is the core of the problem, really. You haven't specified a
single, real-life reason *why* it should be any other way than it
already is. It sounds a bit to me as if you've been to a really
inspiring seminar about "how database-like things *should* be done"
and then decided to go berserk on your favourite database-like thing,
which is git.
Code and benchmarks or bust. In the meantime, I'll settle for a recount
of what problems you're having with the current layout, or what gains
you're hoping to achieve with the new one. As it's the 3rd time I'm
asking, this'll be the last.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: Git's database structure
From: Jon Smirl @ 2007-09-04 16:28 UTC (permalink / raw)
To: Git Mailing List
In-Reply-To: <9e4733910709040823k731f0ffchba1f93bdb4a8373d@mail.gmail.com>
Another way of looking at the problem,
Let's build a full-text index for git. You put a string into the index
and it returns the SHAs of all the file nodes that contain the string.
How do I recover the path names of these SHAs?
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: [PATCH] Add post-merge hook.
From: Josh England @ 2007-09-04 16:25 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7v7inc7hao.fsf@gitster.siamese.dyndns.org>
Sorry for late response --- *long* weekend. :)
On Thu, 2007-08-30 at 16:07 -0700, Junio C Hamano wrote:
> > This adds a post-merge hook that will run after `git pull` operations
> > if enabled. The hook is passed no arguments and cannot affect the
> > outcome of a merge.
> >
> > Signed-off-by: Josh England <jjengla@sandia.gov>
>
> Thanks for your patch.
>
> Two questions.
>
> * Do you want to run the post-merge hook even for a squash
> merge?
Yes. I'd like to run it at any time that the working tree might be
updated.
> * After resolving a conflicted merge, you would conclude it
> with "git commit"; don't you want to have the same hook apply
> at the commit time, or is that what you expect the user to
> deal with with post-commit hook?
I wouldn't want the post-merge hook running there because (in my case)
it needs to do something slightly different than the post-commit hook,
and would seem counter-intuitive to me. The pre-commit and post-commit
hooks are already there and handle that scenario nicely.
> And two requests and a half.
>
> - We would want a new test in the test suite for this, to make
> sure that later changes by others would not break this new
> feature you would depend upon.
Can do. You want me to resubmit the original patch along with tests or
submit the tests as a new patch?
> - You described _what_ the patch and the new feature do in the
> log message and in the documentation.
>
> You need to also explain _why_ somebody might want to have
> such a hook in his repository.
>
> . In the documentation, you would want to make the reader
> realize "aha, this is something that would help me" with an
> example.
>
> . In the log message, you would want to make sure other
> people understand why this new feature was added, what the
> design consideration were at the point of the feature was
> added.
I may have to come up with a use-case that is more mainstream than mine.
I'm personally using it to update permissions/ownership in the working
tree based on a git-controlled file (created by pre-commit). I'll post
the script when its fully fleshed out. Should I just put "If you're
doing something crazy like me, this hook may be useful." :)
> - We are deep in feature-freeze for 1.5.3; I'd appreciate a
> resend for any patch that is not a bugfix / documentation
> update after 1.5.3 final.
Sure thing. It looks like the 1.5.4 cycled just started. I'll resend
with the additions you requested.
-JE
^ permalink raw reply
* Re: Calculating tree nodes
From: Daniel Hulme @ 2007-09-04 16:20 UTC (permalink / raw)
To: Jon Smirl; +Cc: Shawn O. Pearce, Git Mailing List
In-Reply-To: <9e4733910709032026s7f94eed9h25d5165840cc38d2@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2786 bytes --]
On Mon, Sep 03, 2007 at 11:26:30PM -0400, Jon Smirl wrote:
> This is something that has always bugged me about file systems. File
> systems force hierarchical naming due to their directory structure.
> There is no reason they have to work that way. Google is an example of
> a giant file system that works just fine without hierarchical
> directories. The full path should be just another attribute on the
> file. If you want a hierarchical index into the file system you can
> generate it by walking the files or using triggers. But you could also
> delete the hierarchical directory and replace it with something else
> like a full text index. Directories would become a computationally
> generated cache, not a critical part of the file system. But this is a
> git list so I shouldn't go too far off into file system design.
Am I the only one who thinks that this idea of moving filenames from
tree objects into blobs does the *opposite* of what you're trying to
achieve?
It seems, though I could be completely misinterpreting what you're
saying, that you want to be able to get rid of directories and replace
them with some other index into your files: maybe a full-text index,
maybe a spatial index for geographic data, maybe something else
entirely. As things stand, you could do that by editing the core to
introduce a new object type 'fulltext' whose contents maybe look like
aardvark <sha1 of a blob> <sha1 of another blob>
abacus <sha1>
...
zebra <yet another sha1> <maybe the same sha1 I mentioned before>
or even something hierarchical, with each index mapping from the first
letter of the index term to the sha1 of another index, which in term
maps second letters, and so on. Whatever. The point is, it works
parallel to tree. You could have the blobs referenced by your fulltext
object also be referenced by a tree object. If you really don't like
directory trees, you can dispense with tree objects in your repo
entirely. Either way you have a mapping from keys to blobs.
Then you could have your commits and tags include sha1's of fulltext
objects rather than (or as well as) tree objects, and you get your wish.
OTOH, imagine if you move filenames into the blobs. Now, no matter what
other index types you introduce, they'll always be secondary to the
traditional, path-and-filename method of finding files. Crucially, you
can't introduce new blobs into the repo without giving them filenames.
As you said in your other thread,
> Integrating indexing into the data is not normally done in a database.
But isn't this exactly what integrating filenames into blobs would do?
--
There is no such thing as a small specification change.
http://surreal.istic.org/ Forcing the lines through the snow.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: Git's database structure
From: Jon Smirl @ 2007-09-04 16:19 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Git Mailing List
In-Reply-To: <46DD7FE4.1060908@op5.se>
On 9/4/07, Andreas Ericsson <ae@op5.se> wrote:
> Jon Smirl wrote:
> > Let's back up a little bit from "Caclulating tree node". What are the
> > elements of git's data structures?
> >
> > Right now we have an index structure (tree nodes) integrated in to a
> > base table. Integrating indexing into the data is not normally done in
> > a database. Doing a normalization analysis like this may expose flaws
> > in the way the data is structured. Of course we may also decide to
> > leave everything the way it is.
> >
> > What about the special status of a rename? In the current model we
> > effectively have three tables.
> >
> > commit - a set of all SHAs in the commit, previous commit, comment, author, etc
>
> > blob - a file, permissions, etc.
> > file names - name, SHA
>
> commit - SHA1 of its parent(s) and its root-tree, along with
> author info and a free-form field
> blob - content addressable by *multiple trees*
> file names - List of path-names inside a tree object.
>
>
> To draw some sort of relationship model here, you'd have
>
> commit 1<->M roottree
> tree M<->M tree
> tree M<->M blob
By introducing tree nodes you have blended a specific indexing scheme
into the data. There are many other ways the path names could be
indexed hash tables, binary trees, etc.
This problem exists in files systems. Since the path names have been
encoded into the directory structures there is no way to query
something like "all files created yesterday" from a file system
without building another mapping table or a brute force search. I keep
using Google as an example, Google is indexing hierarchical URLs but
they do not use a hierarchical index to do it.
Databases keep the knowledge of how things are indexed out of the
data. A data structure analysis of git should remove the blended index
and start from the set theory.
> Assuming SHA1 never collides (collisions rule out any form of storage,
> so we might as well hope it never happens), that leaves us with this:
>
> Each root tree can only ever belong to a single commit, unless you
> intentionally force git to make completely empty commits. git
> won't complain about this, so long as you don't make two in the
> same second, because it relies more heavily on the DAG than on
> developer sanity.
>
> Each root tree can point to multiple sub-trees. The sub-trees can be
> linked to any number of root-trees.
>
> Blobs can be linked to any number of tree objects, or even multiple
> times to the same tree object. This wouldn't be possible if the
> blob objects had their own pathnames stored inside them, so to speak.
>
> >
> > The file name table is encoded as an index and it has been
> > intermingled with the commit table.
> >
> > Looking at this from a set theory angle brings up the question, do we
> > really have three tables and file names are an independent variable
> > from the blobs, or should file names be an attribute of the blob?
> >
>
> File names are not independant variables. They belong inside the
> table created for them, which is the tree objects.
>
> > How this gets structured in the db is an independent question about
> > how renames get detected on a commit. The current scheme for detecting
> > renames by comparing diffs is working fine. The question is, once we
> > detect a rename how should it be stored?
> >
>
> Do you realize that you're contradicting yourself in two upon each
> other following sentences here?
>
> Detecting renames after the fashion works fine. Not storing them
> is part of the "detect them by comparing diffs".
>
> > Ignoring the performance impacts and looking at the problem from the
> > set theory view point, should:
> > the pathnames be in their own table with a row for each alias
> > the pathnames be stored as an attribute of the blob
> >
> > Both of these are the same information, we're just looking at how
> > things are normalized.
> >
>
> Except that
>
> git init
> echo foo > a
> cp -a a b
> git add .
> git commit -m testing
> git count-objects
>
> yields 3 objects at the moment; A commit-object, a tree object and *one*
> blob object. With your scheme the 2 blob objects would differ, and there
> would be 4 of them. If you propose to ignore the path-name you have
> effectively broken support for having two identical files with different
> names in the same directory.
>
> Now, can you please tell me what gains you're hoping to see with this
> new layout of yours?
>
> --
> Andreas Ericsson andreas.ericsson@op5.se
> OP5 AB www.op5.se
> Tel: +46 8-230225 Fax: +46 8-230231
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: [PATCH] Rework strbuf API and semantics.
From: Pierre Habouzit @ 2007-09-04 16:18 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git
In-Reply-To: <Pine.LNX.4.64.0709041642350.28586@racer.site>
[-- Attachment #1: Type: text/plain, Size: 1150 bytes --]
On Tue, Sep 04, 2007 at 03:44:30PM +0000, Johannes Schindelin wrote:
> Hi,
>
> On Tue, 4 Sep 2007, Pierre Habouzit wrote:
>
> > +void strbuf_grow(struct strbuf *sb, size_t extra) {
> > + if (sb->len + extra + STRBUF_GROW_STEP < sb->len)
> > + die("you want to use way to much memory");
> > +
> > + sb->alloc = ((sb->len + extra) + STRBUF_GROW_STEP) & ~(STRBUF_GROW_STEP - 1);
> > + sb->buf = xrealloc(sb->buf, sb->alloc);
> > +}
>
> Why not use ALLOC_GROW()? Seems to me more efficient than growing by 1kB
> blocks all the time, for big strings as for short strings.
ooooh, now I'm guilty of not knowing all git APIs very well yet :) Indeed,
this should just be:
void strbuf_grow(struct strbuf *sb, size_t extra) {
if (sb->len + extra + 1 < sb->len)
die("you want to use way to much memory");
ALLOC_GROW(sb->buf, sb->len + extra + 1, sb->alloc);
}
This is definitely better on so many levels !
--
·O· Pierre Habouzit
··O madcoder@debian.org
OOO http://www.madism.org
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: Git's database structure
From: Andreas Ericsson @ 2007-09-04 16:10 UTC (permalink / raw)
To: Mike Hommey; +Cc: Git Mailing List
In-Reply-To: <20070904160726.GA17509@glandium.org>
Mike Hommey wrote:
> On Tue, Sep 04, 2007 at 05:55:16PM +0200, Andreas Ericsson <ae@op5.se> wrote:
>> Each root tree can only ever belong to a single commit, unless you
>> intentionally force git to make completely empty commits. git
>> won't complain about this, so long as you don't make two in the
>> same second, because it relies more heavily on the DAG than on
>> developer sanity.
>
> Actually, you don't need to be insane to have multiple commits pointing
> at the same root tree. It is actually very easy:
> - git clone
> - do some stuff on your master branch and commit
> - send your changes upstream
> - upstream applies as is
> - git pull
>
> You now have everything merged, and the last commit on your master branch,
> while being a different commit object due to its parenting, has the same
> root tree as the tip of the remote branch.
>
That explains why it felt so awkward writing that sentence. :)
Thanks for correcting me. Even so, one more M<->M relation-ship
certainly speaks for rather than against the current model.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: Git's database structure
From: Mike Hommey @ 2007-09-04 16:07 UTC (permalink / raw)
To: Andreas Ericsson; +Cc: Git Mailing List
In-Reply-To: <46DD7FE4.1060908@op5.se>
On Tue, Sep 04, 2007 at 05:55:16PM +0200, Andreas Ericsson <ae@op5.se> wrote:
> Each root tree can only ever belong to a single commit, unless you
> intentionally force git to make completely empty commits. git
> won't complain about this, so long as you don't make two in the
> same second, because it relies more heavily on the DAG than on
> developer sanity.
Actually, you don't need to be insane to have multiple commits pointing
at the same root tree. It is actually very easy:
- git clone
- do some stuff on your master branch and commit
- send your changes upstream
- upstream applies as is
- git pull
You now have everything merged, and the last commit on your master branch,
while being a different commit object due to its parenting, has the same
root tree as the tip of the remote branch.
Mike
^ permalink raw reply
* Re: git-commit: if run with <file> arguments, include files removed through git rm
From: Junio C Hamano @ 2007-09-04 16:00 UTC (permalink / raw)
To: Gerrit Pape; +Cc: git
In-Reply-To: <20070904144304.5920.qmail@42e302a26078dc.315fe32.mid.smarden.org>
Gerrit Pape <pape@smarden.org> writes:
> This patch lets it additionally use git ls-tree to look for the files in
> the HEAD tree, but I guess there's a smarter way to fix this.
You raised a good issue, but this also needs other parts of the
system to be adjusted.
> commit_only=`git ls-files --error-unmatch -- "$@"` || exit
> + commit_only="$commit_only "`git ls-tree -r --name-only HEAD -- "$@"` || exit
The arguments to git-commit are *NOT* "<file> arguments". They
are file patterns and 'ls-files --error-unmatch -- "$@"' is
there to allow you to say something like:
$ git commit 'p*/*.c'
Alas, ls-tree does not grok globbing yet. Try this in git.git
repository (surrounding quotes are essential):
$ git ls-files 'p*/*.c'
$ git ls-tree -r HEAD 'p*/*.c'
Currently we have two semantics of "pathspec", and unifying
these semantics is one of the items with somewhat higher
priority on my TODO list for the 1.5.4 cycle (I started looking
at diff-tree and log last night):
* ls-files, diff-files, diff-index and grep understand both
"leading directory prefix" and "glob pattern"; you can say:
$ git ls-files -- arm/ 'p*/*.c'
$ git grep int -- arm/ 'p*/*.c'
and they do what you would expect them to do.
* diff-tree, log family and ls-tree understand only "leading
directory prefix" and not "glob pattern"; you cannot say:
$ git log -- 'p*/*.c'
So your patch is a definite improvement for normal ("non
pattern" but "exact pathname") case, but needs the updates to
pathspec semantics to be the correct fix.
^ permalink raw reply
* Re: Git's database structure
From: Andreas Ericsson @ 2007-09-04 15:55 UTC (permalink / raw)
To: Jon Smirl; +Cc: Git Mailing List
In-Reply-To: <9e4733910709040823k731f0ffchba1f93bdb4a8373d@mail.gmail.com>
Jon Smirl wrote:
> Let's back up a little bit from "Caclulating tree node". What are the
> elements of git's data structures?
>
> Right now we have an index structure (tree nodes) integrated in to a
> base table. Integrating indexing into the data is not normally done in
> a database. Doing a normalization analysis like this may expose flaws
> in the way the data is structured. Of course we may also decide to
> leave everything the way it is.
>
> What about the special status of a rename? In the current model we
> effectively have three tables.
>
> commit - a set of all SHAs in the commit, previous commit, comment, author, etc
> blob - a file, permissions, etc.
> file names - name, SHA
commit - SHA1 of its parent(s) and its root-tree, along with
author info and a free-form field
blob - content addressable by *multiple trees*
file names - List of path-names inside a tree object.
To draw some sort of relationship model here, you'd have
commit 1<->M roottree
tree M<->M tree
tree M<->M blob
Assuming SHA1 never collides (collisions rule out any form of storage,
so we might as well hope it never happens), that leaves us with this:
Each root tree can only ever belong to a single commit, unless you
intentionally force git to make completely empty commits. git
won't complain about this, so long as you don't make two in the
same second, because it relies more heavily on the DAG than on
developer sanity.
Each root tree can point to multiple sub-trees. The sub-trees can be
linked to any number of root-trees.
Blobs can be linked to any number of tree objects, or even multiple
times to the same tree object. This wouldn't be possible if the
blob objects had their own pathnames stored inside them, so to speak.
>
> The file name table is encoded as an index and it has been
> intermingled with the commit table.
>
> Looking at this from a set theory angle brings up the question, do we
> really have three tables and file names are an independent variable
> from the blobs, or should file names be an attribute of the blob?
>
File names are not independant variables. They belong inside the
table created for them, which is the tree objects.
> How this gets structured in the db is an independent question about
> how renames get detected on a commit. The current scheme for detecting
> renames by comparing diffs is working fine. The question is, once we
> detect a rename how should it be stored?
>
Do you realize that you're contradicting yourself in two upon each
other following sentences here?
Detecting renames after the fashion works fine. Not storing them
is part of the "detect them by comparing diffs".
> Ignoring the performance impacts and looking at the problem from the
> set theory view point, should:
> the pathnames be in their own table with a row for each alias
> the pathnames be stored as an attribute of the blob
>
> Both of these are the same information, we're just looking at how
> things are normalized.
>
Except that
git init
echo foo > a
cp -a a b
git add .
git commit -m testing
git count-objects
yields 3 objects at the moment; A commit-object, a tree object and *one*
blob object. With your scheme the 2 blob objects would differ, and there
would be 4 of them. If you propose to ignore the path-name you have
effectively broken support for having two identical files with different
names in the same directory.
Now, can you please tell me what gains you're hoping to see with this
new layout of yours?
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: [PATCH] Rework strbuf API and semantics.
From: Johannes Schindelin @ 2007-09-04 15:44 UTC (permalink / raw)
To: Pierre Habouzit; +Cc: git
In-Reply-To: <11889144741644-git-send-email-madcoder@debian.org>
Hi,
On Tue, 4 Sep 2007, Pierre Habouzit wrote:
> +void strbuf_grow(struct strbuf *sb, size_t extra) {
> + if (sb->len + extra + STRBUF_GROW_STEP < sb->len)
> + die("you want to use way to much memory");
> +
> + sb->alloc = ((sb->len + extra) + STRBUF_GROW_STEP) & ~(STRBUF_GROW_STEP - 1);
> + sb->buf = xrealloc(sb->buf, sb->alloc);
> +}
Why not use ALLOC_GROW()? Seems to me more efficient than growing by 1kB
blocks all the time, for big strings as for short strings.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Functions for updating refs.
From: Johannes Schindelin @ 2007-09-04 15:33 UTC (permalink / raw)
To: Johannes Sixt; +Cc: Carlos Rica, git, Junio C Hamano
In-Reply-To: <46DD7506.3090503@eudaptics.com>
Hi,
On Tue, 4 Sep 2007, Johannes Sixt wrote:
> Johannes Schindelin schrieb:
>
> > Well, my idea was to let _error() print the message, and just die().
>
> How do you avoid that die() prints an error, too?
Good point. Colour me convinced.
Ciao,
Dscho
^ permalink raw reply
* Re: publishing a forked^W cloned directory with ancestry
From: martin f krafft @ 2007-09-04 14:17 UTC (permalink / raw)
To: git discussion list, mdadm development team, J. Bruce Fields,
Bart Trojanowski
In-Reply-To: <20070830194947.GB10808@fieldses.org>
[-- Attachment #1: Type: text/plain, Size: 2508 bytes --]
also sprach J. Bruce Fields <bfields@fieldses.org> [2007.08.30.2149 +0200]:
> Maybe the one extra "git remote add ...; git remote fetch" isn't
> such a big deal?
[...]
> Sure, each of those could add a "linus" branch that tracked
> upstream, so I could still get some idea what Linus's tree was
> even if I didn't happen to already have it. But then I'd end up
> with 4 different slightly-out-of-date pointers to the head of
> linus's repo in each of those trees, which would end up being just
> be a bunch of cruft that I'd have to ignore whenever I looked at
> them.
Thanks to you and everyone else who replied! I spent some time off
computers and still found myself thinking about this as I was hiking
around woods and mountains and in the end, it's perfectly obvious:
I am trying to make it easy for third parties to contribute to my
git-managed project and I (somewhat rightfully) assumed that
I needed to shield my poor contributors from git's complex
tentacles. In doing so, I find myself going backwards and turning
git into something svn-like, thus losing much of its power.
The solution is clear: maintain only my Debian branch on
git.debian.org and expect those cloning it to add the upstream
remote themselves in true distributed manners. A concise file in the
project root with instructions is a nice add-on.
also sprach Bart Trojanowski <bart@jukie.net> [2007.08.30.2227 +0200]:
> Now when you 'git push debian' it will populate the 'upstream' and
> 'master' branches properly.
>
> When someone clones your repo, they will get origin/master (your
> branch) and origin/upstream (the official mdadm branch).
This is the solution I had stuck in my head for the longest time
(after you proposed it), but I eventually discard it for one simple
reason: the danger that someone pushes to the upstream branch and
thus diverges it from the real upstream is just too high and would
result in a pretty nasty mess, as far as I can tell.
> Did I understand the problem correctly?
I think you did. And thanks for that. I hope my reasoning above also
makes sense.
--
martin; (greetings from the heart of the sun.)
\____ echo mailto: !#^."<*>"|tr "<*> mailto:" net@madduck
"the surest way to corrupt a youth is to instruct him to hold in
higher esteem those who think alike than those who think
differently."
-- friedrich nietzsche
spamtraps: madduck.bogus@madduck.net
[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: [PATCH] Functions for updating refs.
From: Karl Hasselström @ 2007-09-04 15:29 UTC (permalink / raw)
To: Johannes Sixt; +Cc: Johannes Schindelin, Carlos Rica, git, Junio C Hamano
In-Reply-To: <46DD7506.3090503@eudaptics.com>
On 2007-09-04 17:08:54 +0200, Johannes Sixt wrote:
> Johannes Schindelin schrieb:
>
> > Well, my idea was to let _error() print the message, and just die().
>
> How do you avoid that die() prints an error, too?
Make a generic function that takes a function pointer argument, then
call it with die or error as appropriate.
--
Karl Hasselström, kha@treskal.com
www.treskal.com/kalle
^ permalink raw reply
* Git's database structure
From: Jon Smirl @ 2007-09-04 15:23 UTC (permalink / raw)
To: Git Mailing List
Let's back up a little bit from "Caclulating tree node". What are the
elements of git's data structures?
Right now we have an index structure (tree nodes) integrated in to a
base table. Integrating indexing into the data is not normally done in
a database. Doing a normalization analysis like this may expose flaws
in the way the data is structured. Of course we may also decide to
leave everything the way it is.
What about the special status of a rename? In the current model we
effectively have three tables.
commit - a set of all SHAs in the commit, previous commit, comment, author, etc
blob - a file, permissions, etc.
file names - name, SHA
The file name table is encoded as an index and it has been
intermingled with the commit table.
Looking at this from a set theory angle brings up the question, do we
really have three tables and file names are an independent variable
from the blobs, or should file names be an attribute of the blob?
How this gets structured in the db is an independent question about
how renames get detected on a commit. The current scheme for detecting
renames by comparing diffs is working fine. The question is, once we
detect a rename how should it be stored?
Ignoring the performance impacts and looking at the problem from the
set theory view point, should:
the pathnames be in their own table with a row for each alias
the pathnames be stored as an attribute of the blob
Both of these are the same information, we're just looking at how
things are normalized.
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
* Re: git-svn and a nested branches folder
From: Russ Brown @ 2007-09-04 15:21 UTC (permalink / raw)
To: git
In-Reply-To: <86r6leeaq7.fsf@lola.quinscape.zz>
David Kastrup wrote:
> Russ Brown <pickscrape@gmail.com> writes:
>
>> David Kastrup wrote:
>>> Russ Brown <pickscrape@gmail.com> writes:
>>>
>>>> I'm having some trouble with using git-svn to fetch a repository, and I
>>>> think it's because the repository doesn't store branches as a flat list
>>>> directly under the 'branches' directory.
>>>>
>>>> Basically, we have a structure like this:
>>>>
>>>> |
>>>> +-trunk
>>>> +-tags
>>>> +-branches
>>>> + category-a
>>>> + branch-a
>>>> + branch-b
>>>> + category-b
>>>> + branch-c
>>>> + branch-d
>>>>
>>>> etc. category-a and category-b are simple directories created using svn
>>>> mkdir. The branches are created using svn cp.
>>>>
>>>> It helps us to organise the branches better, but the rationale is
>>>> besides the point. The problem is that git-svn seems to want to
>>>> treat category-a and category-b as branches, which isn't right at
>>>> all. As a result, git-svn seems to skip most (if not all) revisions
>>>> that occur in these directories and creates a lot of entries in
>>>> unhandled.log.
>>> So what did you specify in your .git/config file regarding the svn
>>> structure?
>> I specified the 'branches' directory, but that's because earlier in
>> the life of the repo we did just do the flat branch layout, but
>> decided to make it more structured once that got unwieldy.
>
> Cough, cough. _What_ did you specify in your .git/config file
> regarding the svn structure? Please quote the section.
>
Erm, sorry.
[svn-remote "svn"]
url = svn://svn.<name>.com
fetch = trunk:refs/remotes/trunk
branches = branches/*:refs/remotes/*
tags = tags/*:refs/remotes/tags/*
(URL changed in case it annoys my employers)
I didn't write this by hand: it was generated by git-svn init.
>> Is it possible to specify more than one folder for the branches
>> option?
>
> It is possible to adapt the config section to the actual layout. If
> not otherwise, by starting with
> git svn init
> with a clean slate, editing the config file, and only then actually
> fetching stuff.
>
> However, git-svn will not magically start guessing that you changed
> your structure around. You have to edit the configuration
> appropriately.
>
That's why I suggested that a method involving detecting branches based
on whether the directory is a copy of trunk or another branch might
'magically' work in all scenarios. I've used a similar branch 'scanning'
technique before for a different reason. But I realise there may be
technical reasons as to why that might not be possible.
--
Russ
^ permalink raw reply
* Re: Calculating tree nodes
From: Andreas Ericsson @ 2007-09-04 15:14 UTC (permalink / raw)
To: Jon Smirl
Cc: Johannes Schindelin, Martin Langhoff, Shawn O. Pearce,
Git Mailing List
In-Reply-To: <9e4733910709040731s2695ab14kb9750923fcac007@mail.gmail.com>
Jon Smirl wrote:
> On 9/4/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>> Hi,
>>
>> On Tue, 4 Sep 2007, Jon Smirl wrote:
>>
>>> In my scheme the path info is moved into the file object nodes and the
>>> SHA list is in the commit node.
>> And how should this "SHA list" be any different from a single tree object,
>> except that you now merge it with the commit object?
>>
>> Really, go back to the mail Martin mentioned. Having all objects in one
>> list kills performance.
>
> You are ignoring the fact the in this scheme temp indexes can be
> created as needed. These temp indexes could look just like tree nodes.
>
What's the cost associated with creating such an index?
What are the estimated savings in size?
What other performance penalties will git take due to this?
While I'm certainly interested in improving git (well, discussing its
improvements anyway - I write too shabby code to participate fully),
I fail to see how this scheme of storing things could help doing that,
seeing as it's already the fastest SCM out there and does an insanely
good job at storing large repositories packed up tight.
What you're proposing would effectively shave off ~4kb or so of stored
data and save a *possible* disk access (note that there will most
likely be none, as most users pack large repos, and for small ones it
shouldn't matter much either way). The way you propose to do so
involves fundamental and backwards-incompatible alterations to the
very core of the git plumbing.
Yet you haven't shown a single estimate of how much diskspace would
be saved, or what other gains are to be had, let alone any code to
implement even a prototype of this new storage scheme.
Note that I'm currently in a state of mind where I'm feeling inclined
to whoop and cheer at every small suggestion that sounds even the
slightest bit of fun. No, I'm not doing drugs, but I just met the
possible love of my life, got a big fat raise and my stocks just
went ballistic. Someone less fortunate than I am right now is probably
too bored to even argue against it.
> I'm saying that it may be a mistake to be recording the indexes (aka
> file names) as part of the commit when they really aren't.
But they are, since they're part of the snapshot.
> The
> essential part of the commit is the SHA1 list. The path names belong
> to the file objects and should be stored there.
>
Sorry, but it'll take code and benchmarks to convince me this is a
good idea. Not that it matters much what I think, but I've got an
inkling Junio will need the same before he's swayed to your point
of view.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply
* Re: [PATCH] Add a new lstat and fstat implementation based on Win32 API
From: David Kastrup @ 2007-09-04 15:05 UTC (permalink / raw)
To: git
In-Reply-To: <Pine.LNX.4.64.0709041547240.28586@racer.site>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> On Tue, 4 Sep 2007, Marius Storm-Olsen wrote:
>
>> The problem is that without this patch fstat(0, buf) would fail
>> with bad filedescriptor instead of returning the st_mode = S_IFIFO.
>
> I guessed as much, but could not fetch the patch, since fetch was broken
> ;-)
>
> For some utterly strange reason, "git fetch" accessed "git-fetch" in
> the cwd, not in /bin/... Funny.
Regardless of what you write in your PATH variable, Windows will
_always_ search for binaries first in your current work directory. It
is not really funny.
--
David Kastrup
^ permalink raw reply
* Re: [PATCH] Functions for updating refs.
From: Johannes Sixt @ 2007-09-04 15:08 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Carlos Rica, git, Junio C Hamano
In-Reply-To: <Pine.LNX.4.64.0709041557590.28586@racer.site>
Johannes Schindelin schrieb:
> Hi,
>
> On Tue, 4 Sep 2007, Johannes Sixt wrote:
>
>> Johannes Schindelin schrieb:
>>> On Tue, 4 Sep 2007, Carlos Rica wrote:
>>>> +int update_ref_or_die(const char *action, const char *refname,
>>>> + const unsigned char *sha1,
>>>> + const unsigned char *oldval, int flags)
>>> Should this not be "void"? And should it not use update_ref_or_error()?
>> It should not use *_error() directly because then it would print two error
>> messages in a row.
>
> Well, my idea was to let _error() print the message, and just die().
How do you avoid that die() prints an error, too?
-- Hannes
^ permalink raw reply
* Re: Calculating tree nodes
From: Johannes Schindelin @ 2007-09-04 15:05 UTC (permalink / raw)
To: Jon Smirl; +Cc: Martin Langhoff, Shawn O. Pearce, Git Mailing List
In-Reply-To: <9e4733910709040731s2695ab14kb9750923fcac007@mail.gmail.com>
Hi,
On Tue, 4 Sep 2007, Jon Smirl wrote:
> On 9/4/07, Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>
> > On Tue, 4 Sep 2007, Jon Smirl wrote:
> >
> > > In my scheme the path info is moved into the file object nodes and
> > > the SHA list is in the commit node.
> >
> > And how should this "SHA list" be any different from a single tree
> > object, except that you now merge it with the commit object?
> >
> > Really, go back to the mail Martin mentioned. Having all objects in
> > one list kills performance.
>
> You are ignoring the fact the in this scheme temp indexes can be
> created as needed. These temp indexes could look just like tree nodes.
>
> I'm saying that it may be a mistake to be recording the indexes (aka
> file names) as part of the commit when they really aren't.
You still have not acknowledged that you want to do (in essence) _exactly_
the same. Your "temp indexes" awfully remind me of tree objects.
And that is no wonder, since you _need_ something like that, however you
want to avoid it. Just admit it that you found -- again -- that flat
directory structure does not work, and you therefore need some smaller
structures. We just happen to call them "tree objects", and for ease of
concept we wrap directories in them.
Hth,
Dscho
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox