* Re: Yet another base64 patch
From: David A. Wheeler @ 2005-04-17 16:29 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: Paul Jackson, git
In-Reply-To: <Pine.LNX.4.21.0504171018410.30848-100000@iabervon.org>
I wrote:
>>>It's a trade-off, I know.
Paul Jackson replied:
>>So where do you recommend we make that trade-off?
Daniel Barkalow wrote:
> So why do we have to be consistant? It seems like we need a standard
> format for these reasons:
>
> - We use rsync to interact with remote repositories, and rsync won't
> understand if they aren't organized the same way. But I'm working on
> having everything go through git-specific code, which could understand
> different layouts.
>
> - Everything that shares a local repository needs to understand the
> format of that repository. But the filesystem constraints on the local
> repository will be the same regardless of who is looking, so they'd all
> expect the same format anyway.
>
> So my idea is, once we're using git-smart transfer code (which can verify
> objects, etc.), add support for different implementations of
> sha1_file_name suitable for different filesystems, and vary based either
> on a compile-time option or on a setting stored in the objects
> directory.
I think that's the perfect answer: make it a setting stored
in the objects directory (presumably set during
initialization of the directory), and handled automagically
by the tools. I recommend handling them NOT be a compile-time option,
so that the same set of tools works everywhere automatically
(who wants to recompile tools just to work on a different file layout?).
> The only thing that matters is that repositories on
> non-special web servers have a standard format, because they'll be serving
> objects by URL, not by sha1.
If the "layout info" is stored in a standard location for a
given repository, then the rest doesn't matter. The library would just
download that, then know how to find the rest.
--- David A. Wheeler
^ permalink raw reply
* Re: using git directory cache code in darcs?
From: Linus Torvalds @ 2005-04-17 16:24 UTC (permalink / raw)
To: David Roundy; +Cc: git, darcs-devel
In-Reply-To: <20050417121712.GA22772@abridgegame.org>
On Sun, 17 Apr 2005, David Roundy wrote:
>
> That's all right. Darcs would only access the cached data through a
> git-caching layer, and we've already got an abstraction layer over the
> pristine cache. As long as the git layer can quickly retrieve the contents
> of a given file, we should be fine.
Yes.
In fact, one of my hopes was that other SCM's could just use the git
plumbing. But then I'd really suggest that you use "git" itself, not any
"libgit". Ie you take _all_ the plumbing as real programs, and instead of
trying to link against individual routines, you'd _script_ it.
In other words, "git" would be an independent cache of the real SCM,
and/or the "old history" (ie an SCM that uses git could decide that the
git stuff is fine for archival, and really use git as the base: and then
the SCM could entirely concentrate on _only_ the "interesting" parts, ie
the actual merging etc).
That was really what I always personally saw "git" as, just the plumbing
beneath the surface. For example, something like arch, which is based on
"patches and tar-balls" (I think darcs is similar in that respect), could
use git as a _hell_ of a better "history of tar-balls".
The thing is, unless you take the git object database approach, using
_just_ the index part doesn't really mean all that much. Sure, you could
just keep the "current objects" in the object database, but quite
frankly, there would probably not be a whole lot of point to that. You'd
waste so much time pruning and synchronizing with your "real" database
that I suspect you'd be better off not using it.
(Or you could prune nightly or something, I guess).
Linus
^ permalink raw reply
* Re: Storing permissions
From: David A. Wheeler @ 2005-04-17 16:21 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Paul Jackson, Morten Welinder, mj, git
In-Reply-To: <Pine.LNX.4.58.0504170857580.7211@ppc970.osdl.org>
Linus Torvalds wrote:
>
> On Sun, 17 Apr 2005, David A. Wheeler wrote:
>
>>There's a minor reason to write out ALL the perm bit data, but
>>only care about a few bits coming back in: Some people use
>>SCM systems as a generalized backup system
>
> Yes. I was actually thinking about having system config files in a git
> repository when I started it, since I noticed how nicely it would do
> exactly that.
>
> However, since the mode bits also end up being part of the name of the
> tree object (ie they are most certainly part of the hash), it's really
> basically impossible to only care about one bit but writing out many bits:
> it's the same issue of having multiple "identical" blocks with different
> names.
...
> One solution is to tell git with a command line flag and/or config file
> entry that "for this repo, I want you to honor all bits". That should be
> easy enough to add at some point, and then you really get what you want.
Yes, I thought of that too. And I agree, that should do the job.
My real concern is I'm looking at the early design of the
storage format so that it's POSSIBLE to extend git in obvious ways.
As long as it's possible later, then that's a great thing.
...
> Also, I made a design decision that git only cares about non-dotfiles. Git
> literally never sees or looks at _anything_ that starts with a ".". I
> think that's absolutely the right thing to do for an SCM (if you hide your
> files, I really don't think you should expect the SCM to see it), but it's
> obviously not the right thing for a backup thing.
Again, a command line flag or config file entry could change that
in the future, if desired. So this is a decision that could be
changed later... the best kind of decision :-).
--- David A. Wheeler
^ permalink raw reply
* Re: Parsing code in revision.h
From: Petr Baudis @ 2005-04-17 16:09 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504171120400.30848-100000@iabervon.org>
Dear diary, on Sun, Apr 17, 2005 at 05:24:20PM CEST, I got a letter
where Daniel Barkalow <barkalow@iabervon.org> told me that...
> This adds support to revision.h for parsing commit records (but not going
> any further than parsing a single record). Something like this is needed
> by anything that uses revision.h, but older programs open-code it.
>
> Signed-Off-By: Daniel Barkalow <barkalow@iabervon.org>
Could you please convert the current users (rev-tree.c and fsck-cache.c)
to use this in the same patch?
> Index: revision.h
> ===================================================================
> --- 45f926575d2c44072bfcf2317dbf3f0fbb513a4e/revision.h (mode:100644 sha1:28d0de3261a61f68e4e0948a25a416a515cd2e83)
> +++ 37a0b01b85c2999243674d48bfc71cdba0e5518e/revision.h (mode:100644 sha1:523bde6e14e18bb0ecbded8f83ad4df93fc467ab)
> @@ -24,6 +24,7 @@
> unsigned int flags;
> unsigned char sha1[20];
> unsigned long date;
> + unsigned char tree[20];
> struct parent *parent;
> };
>
> @@ -111,4 +112,29 @@
> }
> }
>
> +static int parse_commit_object(struct revision *rev)
> +{
> + if (!(rev->flags & SEEN)) {
> + void *buffer, *bufptr;
> + unsigned long size;
> + char type[20];
> + unsigned char parent[20];
> +
> + rev->flags |= SEEN;
> + buffer = bufptr = read_sha1_file(rev->sha1, type, &size);
> + if (!buffer || strcmp(type, "commit"))
> + return -1;
> + get_sha1_hex(bufptr + 5, rev->tree);
> + bufptr += 46; /* "tree " + "hex sha1" + "\n" */
> + while (!memcmp(bufptr, "parent ", 7) &&
> + !get_sha1_hex(bufptr+7, parent)) {
> + add_relationship(rev, parent);
> + bufptr += 48; /* "parent " + "hex sha1" + "\n" */
> + }
> + //rev->date = parse_commit_date(bufptr);
I don't like this.
> + free(buffer);
> + }
> + return 0;
> +}
> +
> #endif /* REVISION_H */
BTW, I think that in longer term having this stuffed in revision.h is a
bad idea, we should have revision.c. I will accept patches putting the
stuff to revision.h for now, though (unless it gets outrageous).
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Re: Storing permissions
From: Linus Torvalds @ 2005-04-17 16:10 UTC (permalink / raw)
To: David A. Wheeler; +Cc: Paul Jackson, Morten Welinder, mj, git
In-Reply-To: <42620092.9040402@dwheeler.com>
On Sun, 17 Apr 2005, David A. Wheeler wrote:
>
> There's a minor reason to write out ALL the perm bit data, but
> only care about a few bits coming back in: Some people use
> SCM systems as a generalized backup system
Yes. I was actually thinking about having system config files in a git
repository when I started it, since I noticed how nicely it would do
exactly that.
However, since the mode bits also end up being part of the name of the
tree object (ie they are most certainly part of the hash), it's really
basically impossible to only care about one bit but writing out many bits:
it's the same issue of having multiple "identical" blocks with different
names.
It's ok if it happens occasionally (it _will_ happen at the point of a
tree conversion to the new format, for example), but it's not ok if it
happens all the time - which it would, since some people have umask 002
(and individual groups) and others have umask 022 (and shared groups), and
I can imagine that some anal people have umask 0077 ("I don't want to play
with others").
The trees would constantly bounce between a million different combinations
(since _some_ files would be checked out with the "other" mode).
At least if you always honor umask or always totally ignore umask, you get
a nice repetable thing. We tried the "always ignore" umask thing, and the
problem with that is that while _git_ ended up always doing a "fchmod()"
to reset the whole permission mask, anybody who created files any other
way and then checked them in would end up using umask.
One solution is to tell git with a command line flag and/or config file
entry that "for this repo, I want you to honor all bits". That should be
easy enough to add at some point, and then you really get what you want.
That said, git won't be really good at doing system backup. I actually
_do_ save a full 32-bit of "mode" (hey, you could have "immutable" bits
etc set), but anybody who does anything fancy at all with mtime would be
screwed, for example.
Also, right now we don't actually save any other type of file than
regular/directory, so you'd have to come up with a good save-format for
symlinks (easy, I guess - just make a "link" blob) and device nodes (that
one probably should be saved in the "cache_entry" itself, possibly
encoded where the sha1 hash normally is).
Also, I made a design decision that git only cares about non-dotfiles. Git
literally never sees or looks at _anything_ that starts with a ".". I
think that's absolutely the right thing to do for an SCM (if you hide your
files, I really don't think you should expect the SCM to see it), but it's
obviously not the right thing for a backup thing.
(It _might_ be the right thing for a system config file, though, eg
tracking something like "/etc" with git might be ok, modulo the other
issues).
Linus
^ permalink raw reply
* Re: Re-done kernel archive - real one?
From: Russell King @ 2005-04-17 16:05 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List, Peter Anvin
In-Reply-To: <Pine.LNX.4.58.0504161543590.7211@ppc970.osdl.org>
On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:
> So I re-created the dang thing (hey, it takes just a few minutes), and
> pushed it out, and there's now an archive on kernel.org in my public
> "personal" directory called "linux-2.6.git". I'll continue the tradition
> of naming git-archive directories as "*.git", since that really ends up
> being the ".git" directory for the checked-out thing.
BTW, there appears to be "errors" in the history committed thus far.
I'm not sure where this came from though. Some of them could be
UTF8 vs ASCII issues, but there's a number which seem to have extra
random crap in them ("^M)" and lots of blank lines).
One thing which definitely needs to be considered is - what character
encoding are the comments to be stored as? Currently, it's whatever
the committers character encoding is, which will be completely random.
For instance, dwmw2 will definitely be using UTF, whereas I'll definitely
be using ISO-8859-1 (UTF is far too much of a pain in the ass to use,
unless _all_ your systems are running UTF, which mine don't yet.)
ID: 75f86bac962b7609b0f3c21d25e10647ff8ed280
[PATCH] intel8x0: AC'97 audio patch for Intel ESB2
This patch adds the Intel ESB2 DID's to the intel8x0.c file for AC'97 audio
support.
Signed-off-by: <A0>Jason Gaston <Jason.d.gaston@intel.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
ID: baab5c52135dfa2a02c5dcc8422fc5d048acc682
[PATCH] ppc32: fix compilation error in arch/ppc/kernel/time.c
make defconfig give the following error on ppc (gcc-4):
arch/ppc/kernel/time.c:92: error: static declaration of <E2><80><98>time_offset<E2><80><99>
follows non-static declaration
include/linux/timex.h:236: error: previous declaration of <E2><80><98>time_offset<E2><80><99>
was here
The following patch solves it (time_offset is declared in timer.c).
Signed-Off-By: Benoit Boissinot <benoit.boissinot@ens-lyon.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
ID: fa357627062fa6f4727638bc4f302b23f5a3acb8
[PATCH] vmscan: pageout(): remove unneeded test
^M)
We only call pageout() for dirty pages, so this test is redundant.
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
ID: e5fc9026c6a372a67e652c6ba1648e5f8e543600
[PATCH] arm: fix SIGBUS handling
^M)
From: Russell King <rmk+lkml@arm.linux.org.uk>
ARM wasn't raising a SIGBUS with a siginfo structure. Fix
__do_user_fault() to allow us to use it for SIGBUS conditions, and arrange
for the sigbus path to use this.
We need to prevent the siginfo code being called if we do not have a user
space context to call it, so consolidate the "user_mode()" tests.
Thanks to Ian Campbell who spotted this oversight.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
--
Russell King
^ permalink raw reply
* Re: Add merge-base
From: Petr Baudis @ 2005-04-17 16:01 UTC (permalink / raw)
To: Daniel Barkalow; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504171124340.30848-100000@iabervon.org>
Dear diary, on Sun, Apr 17, 2005 at 05:27:13PM CEST, I got a letter
where Daniel Barkalow <barkalow@iabervon.org> told me that...
> merge-base finds one of the best common ancestors of a pair of commits. In
> particular, it finds one of the ones which is fewest commits away from the
> further of the heads.
What does it return when I have
A -- C
\/ \
/\ /
B -- D
? >:)
I assume just either A or B, randomly?
I think it would be best if it could list all the "first-class" matches
(both A and B in this case), each on a separate line; this way the
overlay tools could choose an algorithm to evaluate those further as
they see fit - e.g. sort them by time (you might aid that by listing the
commit time in front of them), then take the first n and try to diff
them all and take the one with least changes (as suggested by Linus).
And if someone doesn't care, he just does | head -n 1 | cut -f 2.
> Index: merge-base.c
> ===================================================================
> --- /dev/null (tree:37a0b01b85c2999243674d48bfc71cdba0e5518e)
> +++ d662b707e11391f6cfe597fd4d0bf9c41d34d01a/merge-base.c (mode:100644 sha1:0f85e7d9e9a896d1142a54170ddf1159f11f9cdd)
> @@ -0,0 +1,108 @@
> +#include <stdlib.h>
> +#include "cache.h"
> +#include "revision.h"
> +
> +struct revision *common_ancestor(struct revision *rev1, struct revision *rev2)
> +{
> + struct parent *parent;
> +
> + struct parent *rev1list = malloc(sizeof(struct parent));
> + struct parent *rev2list = malloc(sizeof(struct parent));
Did I overlook anything or you could have just a single revlist?
> +
I smell trailing whitespaces!
> + struct parent *posn, *temp;
> +
> + rev1list->parent = rev1;
> + rev1list->next = NULL;
> +
> + rev2list->parent = rev2;
> + rev2list->next = NULL;
> +
> + while (rev1list || rev2list) {
> + posn = rev1list;
> + rev1list = NULL;
> + while (posn) {
> + parse_commit_object(posn->parent);
> + if (posn->parent->flags & 0x0001) {
> + /*
> + printf("1 already seen %s %x\n",
> + sha1_to_hex(posn->parent->sha1),
> + posn->parent->flags);
> + */
> + // do nothing
Mostly for consistency, I'd prefer you to use /* */ comments in general.
I think a terrified squeak at stderr in this situation (possibly
suggesting fsck-cache) might be appropriate.
> + } else if (posn->parent->flags & 0x0002) {
> + // XXXX free lists
Hmm, so, why not free the lists?
> + return posn->parent;
> + } else {
> + /*
> + printf("1 based on %s\n",
> + sha1_to_hex(posn->parent->sha1));
> + */
> + posn->parent->flags |= 0x0001;
> +
> + parent = posn->parent->parent;
> + while (parent) {
> + temp = malloc(sizeof(struct parent));
> + temp->next = rev1list;
> + temp->parent = parent->parent;
> + rev1list = temp;
> + parent = parent->next;
> + }
> + }
> + posn = posn->next;
> + }
> + posn = rev2list;
> + rev2list = NULL;
> + while (posn) {
> + parse_commit_object(posn->parent);
> + if (posn->parent->flags & 0x0002) {
> + /*
> + printf("2 already seen %s\n",
> + sha1_to_hex(posn->parent->sha1));
> + */
> + // do nothing
> + } else if (posn->parent->flags & 0x0001) {
> + // XXXX free lists
> + return posn->parent;
> + } else {
> + /*
> + printf("2 based on %s\n",
> + sha1_to_hex(posn->parent->sha1));
> + */
> + posn->parent->flags |= 0x0002;
> +
> + parent = posn->parent->parent;
> + while (parent) {
> + temp = malloc(sizeof(struct parent));
> + temp->next = rev2list;
> + temp->parent = parent->parent;
> + rev2list = temp;
> + parent = parent->next;
> + }
> + }
> + posn = posn->next;
> + }
Symmetrical notes apply to this half. Actually, they are too similar.
What about factoring them to a common function?
> + }
> + return NULL;
> +}
> +
> +int main(int argc, char **argv)
> +{
> + struct revision *rev1, *rev2, *ret;
> + unsigned char rev1key[20], rev2key[20];
A newline here please.
> + if (argc != 3 ||
> + get_sha1_hex(argv[1], rev1key) ||
> + get_sha1_hex(argv[2], rev2key)) {
> + usage("mergebase <commit-id> <commit-id>");
> + }
> + rev1 = lookup_rev(rev1key);
> + rev2 = lookup_rev(rev2key);
> + ret = common_ancestor(rev1, rev2);
> + if (ret) {
> + printf("%s\n", sha1_to_hex(ret->sha1));
> + return 0;
> + } else {
> + printf("Sorry.\n");
> + return 1;
Please stay silent if you don't have anything useful to say.
> + }
> +
> +}
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Re: Parseable commit header
From: Linus Torvalds @ 2005-04-17 15:57 UTC (permalink / raw)
To: Stefan-W. Hahn; +Cc: git
In-Reply-To: <20050417062236.GA3261@scotty.home>
On Sun, 17 Apr 2005, Stefan-W. Hahn wrote:
>
> after playing a while with git-pasky it is a crap to interpret the date of
> commit logs. Though it was a good idea to put the date in a parseable format
> (seconds since), but the format of the commit itself is not good parseable.
Actually, it is. The commit stuff removes all special characters from the
strings, so '<' and '>' around the email do indeed act as delimiters, and
cannot exist anywhere else.
Linus
^ permalink raw reply
* [patch] Fix off by one error in show-diff.c:sq_expand
From: Brad Roberts @ 2005-04-17 15:53 UTC (permalink / raw)
To: git
tree b2eb41fec5cb7a3b83b53f1eb59e59785e57cf5d
parent 1c45567134945bccefc1b5fbacc2862cd8f346c4
author Brad Roberts <braddr@puremagic.com> 1113752911 -0700
committer Brad Roberts,,, <braddr@gameboy2> 1113752911 -0700
Fix off by one error in sq_expand, forgot space for the null.
Signed-off-by: Brad Roberts <braddr@puremagic.com>
show-diff.c | 2 +-
1 files changed, 1 insertion(+), 1 deletion(-)
Index: show-diff.c
===================================================================
--- f9e06a309f63ac6858d019b51f2172283378d2ef/show-diff.c (mode:100644 sha1:d85d79b97a59342390bd34da09049dd58d56900f)
+++ b2eb41fec5cb7a3b83b53f1eb59e59785e57cf5d/show-diff.c (mode:100644 sha1:462f88a67cb3a100bbf307e567eebbcf911292ce)
@@ -36,7 +36,7 @@
if (buf_size < cnt) {
free(buf);
buf_size = cnt;
- buf = malloc(cnt);
+ buf = malloc(cnt+1);
}
cp = buf;
^ permalink raw reply
* [5/5] Add commit-id to version
From: Daniel Barkalow @ 2005-04-17 15:37 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504171108060.30848-100000@iabervon.org>
For people who run intermediate versions of git, it is useful to know
exactly which post-release version you've installed. This adds the
commit-id to the version info, so you can tell exactly, provided you make
sure to commit before installing.
Signed-Off-By: Daniel Barkalow <barkalow@iabervon.org>
Index: Makefile
===================================================================
--- 08f7700831e056ad710af69f91e3a8a705b6b2b1/Makefile (mode:100644 sha1:a60fa46404c0487158d232bd021e4798bc8df8de)
+++ 6467ed39f19b48563ff25782ebe2c6f951b0af3c/Makefile (mode:100644 sha1:0e84e3cd12f836602b420c197e08fabefe975493)
@@ -50,7 +50,7 @@
@echo Generating gitversion.sh...
@rm -f $@
@echo "#!/bin/sh" > $@
- @echo "echo \"$(shell cat $(VERSION))\"" >> $@
+ @echo "echo \"$(shell cat $(VERSION)) $(shell commit-id)\"" >> $@
@chmod +x $@
clean:
^ permalink raw reply
* [4/5] Add option for hardlinkable cache of extracted blobs
From: Daniel Barkalow @ 2005-04-17 15:35 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504171108060.30848-100000@iabervon.org>
This adds an option (compile time, defined in the Makefile) to have a
cache of extracted blobs so that different working directories can
hardlink against them instead of creating new files for every
checkout. You should only use this if you're sure the programs you use
break links on modification and you care about storing many large working
directories with few changes at the same time.
Signed-Off-By: Daniel Barkalow <barkalow@iabervon.org>
Index: Makefile
===================================================================
--- 157b46ce1d82b3579e2e1258927b0d9bdbc033ab/Makefile (mode:100644 sha1:940ef8578cf469354002cd8feaec25d907015267)
+++ 08f7700831e056ad710af69f91e3a8a705b6b2b1/Makefile (mode:100644 sha1:a60fa46404c0487158d232bd021e4798bc8df8de)
@@ -2,6 +2,9 @@
# 1461501637330902918203684832716283019655932542976 hashes do not give you
# enough guarantees about no collisions between objects ever hapenning.
#
+# -DUSE_HARDLINK_CACHE if you want a cache of files to be hardlinked
+# to for unmodified checked out files.
+#
# -DNSEC if you want git to care about sub-second file mtimes and ctimes.
# Note that you need some new glibc (at least >2.2.4) for this, and it will
# BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely randomly
Index: checkout-cache.c
===================================================================
--- 157b46ce1d82b3579e2e1258927b0d9bdbc033ab/checkout-cache.c (mode:100644 sha1:5d3028df0a45329e45fff2006719c9267adeb946)
+++ 08f7700831e056ad710af69f91e3a8a705b6b2b1/checkout-cache.c (mode:100644 sha1:338588259e17dd235fdc7db759d770004a760e15)
@@ -34,6 +34,10 @@
*/
#include "cache.h"
+#ifdef USE_HARDLINK_CACHE
+#define HARDLINK_CACHE ".git/blobs"
+#endif /* USE_HARDLINK_CACHE */
+
static int force = 0, quiet = 0;
static void create_directories(const char *path)
@@ -67,6 +71,80 @@
return fd;
}
+#ifdef HARDLINK_CACHE
+
+/*
+ * NOTE! This returns a statically allocated buffer, so you have to be
+ * careful about using it. Do a "strdup()" if you need to save the
+ * filename.
+ */
+char *sha1_blob_cache_file_name(const unsigned char *sha1)
+{
+ int i;
+ static char *name, *base;
+
+ if (!base) {
+ char *sha1_file_directory = HARDLINK_CACHE;
+ int len = strlen(sha1_file_directory);
+ base = malloc(len + 60);
+ memcpy(base, sha1_file_directory, len);
+ memset(base+len, 0, 60);
+ base[len] = '/';
+ base[len+3] = '/';
+ name = base + len + 1;
+ }
+ for (i = 0; i < 20; i++) {
+ static char hex[] = "0123456789abcdef";
+ unsigned int val = sha1[i];
+ char *pos = name + i*2 + (i > 0);
+ *pos++ = hex[val >> 4];
+ *pos = hex[val & 0xf];
+ }
+ return base;
+}
+
+static int write_entry(struct cache_entry *ce)
+{
+ int fd;
+ void *new;
+ unsigned long size;
+ long wrote;
+ char type[20];
+ char *cache_name;
+ struct stat st;
+
+ cache_name = sha1_blob_cache_file_name(ce->sha1);
+
+ if (stat(cache_name, &st)) {
+ new = read_sha1_file(ce->sha1, type, &size);
+ if (!new || strcmp(type, "blob")) {
+ return error("checkout-cache: unable to read sha1 file of %s (%s)",
+ ce->name, sha1_to_hex(ce->sha1));
+ }
+ fd = create_file(cache_name, ntohl(ce->ce_mode));
+ if (fd < 0) {
+ free(new);
+ return error("checkout-cache: unable to create %s (%s)",
+ ce->name, strerror(errno));
+ }
+ wrote = write(fd, new, size);
+ close(fd);
+ free(new);
+ if (wrote != size)
+ return error("checkout-cache: unable to write %s",
+ ce->name);
+ }
+ if (link(cache_name, ce->name)) {
+ if (errno == ENOENT) {
+ create_directories(ce->name);
+ link(cache_name, ce->name);
+ }
+ }
+ return 0;
+}
+
+#else
+
static int write_entry(struct cache_entry *ce)
{
int fd;
@@ -94,6 +172,8 @@
return 0;
}
+#endif
+
static int checkout_entry(struct cache_entry *ce)
{
struct stat st;
^ permalink raw reply
* [3/5] Add http-pull
From: Daniel Barkalow @ 2005-04-17 15:31 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504171108060.30848-100000@iabervon.org>
http-pull is a program that downloads from a (normal) HTTP server a commit
and all of the tree and blob objects it refers to (but not other commits,
etc.). Options could be used to make it download a larger or different
selection of objects. It depends on libcurl, which I forgot to mention in
the README again.
Signed-Off-By: Daniel Barkalow <barkalow@iabervon.org>
Index: Makefile
===================================================================
--- d662b707e11391f6cfe597fd4d0bf9c41d34d01a/Makefile (mode:100644 sha1:b2ce7c5b63fffca59653b980d98379909f893d44)
+++ 157b46ce1d82b3579e2e1258927b0d9bdbc033ab/Makefile (mode:100644 sha1:940ef8578cf469354002cd8feaec25d907015267)
@@ -14,7 +14,7 @@
PROG= update-cache show-diff init-db write-tree read-tree commit-tree \
cat-file fsck-cache checkout-cache diff-tree rev-tree show-files \
- check-files ls-tree merge-base
+ check-files ls-tree http-pull merge-base
SCRIPT= parent-id tree-id git gitXnormid.sh gitadd.sh gitaddremote.sh \
gitcommit.sh gitdiff-do gitdiff.sh gitlog.sh gitls.sh gitlsobj.sh \
@@ -35,6 +35,7 @@
LIBS= -lssl -lz
+http-pull: LIBS += -lcurl
$(PROG):%: %.o $(COMMON)
$(CC) $(CFLAGS) -o $@ $^ $(LIBS)
Index: http-pull.c
===================================================================
--- /dev/null (tree:d662b707e11391f6cfe597fd4d0bf9c41d34d01a)
+++ 157b46ce1d82b3579e2e1258927b0d9bdbc033ab/http-pull.c (mode:100644 sha1:106ca31239e6afe6784e7c592234406f5c149e44)
@@ -0,0 +1,126 @@
+#include <fcntl.h>
+#include <unistd.h>
+#include <string.h>
+#include <stdlib.h>
+#include "cache.h"
+#include "revision.h"
+#include <errno.h>
+#include <stdio.h>
+
+#include <curl/curl.h>
+#include <curl/easy.h>
+
+static CURL *curl;
+
+static char *base;
+
+static int fetch(unsigned char *sha1)
+{
+ char *hex = sha1_to_hex(sha1);
+ char *filename = sha1_file_name(sha1);
+
+ char *url;
+ char *posn;
+ FILE *local;
+ struct stat st;
+
+ if (!stat(filename, &st)) {
+ return 0;
+ }
+
+ local = fopen(filename, "w");
+
+ if (!local) {
+ fprintf(stderr, "Couldn't open %s\n", filename);
+ return -1;
+ }
+
+ curl_easy_setopt(curl, CURLOPT_FILE, local);
+
+ url = malloc(strlen(base) + 50);
+ strcpy(url, base);
+ posn = url + strlen(base);
+ strcpy(posn, "objects/");
+ posn += 8;
+ memcpy(posn, hex, 2);
+ posn += 2;
+ *(posn++) = '/';
+ strcpy(posn, hex + 2);
+
+ curl_easy_setopt(curl, CURLOPT_URL, url);
+
+ curl_easy_perform(curl);
+
+ fclose(local);
+
+ return 0;
+}
+
+static int process_tree(unsigned char *sha1)
+{
+ void *buffer;
+ unsigned long size;
+ char type[20];
+
+ buffer = read_sha1_file(sha1, type, &size);
+ if (!buffer)
+ return -1;
+ if (strcmp(type, "tree"))
+ return -1;
+ while (size) {
+ int len = strlen(buffer) + 1;
+ unsigned char *sha1 = buffer + len;
+ unsigned int mode;
+ int retval;
+
+ if (size < len + 20 || sscanf(buffer, "%o", &mode) != 1)
+ return -1;
+
+ buffer = sha1 + 20;
+ size -= len + 20;
+
+ retval = fetch(sha1);
+ if (retval)
+ return -1;
+
+ if (S_ISDIR(mode)) {
+ retval = process_tree(sha1);
+ if (retval)
+ return -1;
+ }
+ }
+ return 0;
+}
+
+static int process_commit(unsigned char *sha1)
+{
+ struct revision *rev = lookup_rev(sha1);
+ if (parse_commit_object(rev))
+ return -1;
+
+ fetch(rev->tree);
+ process_tree(rev->tree);
+ return 0;
+}
+
+int main(int argc, char **argv)
+{
+ char *commit_id = argv[1];
+ char *url = argv[2];
+
+ unsigned char sha1[20];
+
+ get_sha1_hex(commit_id, sha1);
+
+ curl_global_init(CURL_GLOBAL_ALL);
+
+ curl = curl_easy_init();
+
+ base = url;
+
+ fetch(sha1);
+ process_commit(sha1);
+
+ curl_global_cleanup();
+ return 0;
+}
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Ingo Molnar @ 2005-04-17 15:28 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Petr Baudis, Simon Fowler, David Lang, git
In-Reply-To: <20050417145232.GA5289@elte.hu>
* Ingo Molnar <mingo@elte.hu> wrote:
> The compromise relies on you having reviewed something harmless, while
> in reality what happened within the DB was far less harmless. And the
> DB remains self-consistent: neither fsck, nor others importing your
> tree will be able to detect the compromise. This attack can only be
> detected when you apply the patch, after that point all the
> information (except Malice's message in your inbox) is gone.
in fact, this attack cannot even be proven to be malicious, purely via
the email from Malice: it could be incredible bad luck that caused that
good-looking patch to be mistakenly matching a dangerous object.
In fact this could happen even today, _accidentally_. (but i'm willing
to bet that hell will be freezing over first, and i'll have some really
good odds ;) There's probably a much higher likelyhood of Linus' tree
getting corrupted in some old fashioned way and introducing a security
hole by accident)
Ingo
^ permalink raw reply
* [2/5] Add merge-base
From: Daniel Barkalow @ 2005-04-17 15:27 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504171108060.30848-100000@iabervon.org>
merge-base finds one of the best common ancestors of a pair of commits. In
particular, it finds one of the ones which is fewest commits away from the
further of the heads.
Signed-Off-By: Daniel Barkalow <barkalow@iabervon.org>
Index: Makefile
===================================================================
--- 37a0b01b85c2999243674d48bfc71cdba0e5518e/Makefile (mode:100644 sha1:346e3850de026485802e41e16a1180be2df85e4a)
+++ d662b707e11391f6cfe597fd4d0bf9c41d34d01a/Makefile (mode:100644 sha1:b2ce7c5b63fffca59653b980d98379909f893d44)
@@ -14,7 +14,7 @@
PROG= update-cache show-diff init-db write-tree read-tree commit-tree \
cat-file fsck-cache checkout-cache diff-tree rev-tree show-files \
- check-files ls-tree
+ check-files ls-tree merge-base
SCRIPT= parent-id tree-id git gitXnormid.sh gitadd.sh gitaddremote.sh \
gitcommit.sh gitdiff-do gitdiff.sh gitlog.sh gitls.sh gitlsobj.sh \
Index: merge-base.c
===================================================================
--- /dev/null (tree:37a0b01b85c2999243674d48bfc71cdba0e5518e)
+++ d662b707e11391f6cfe597fd4d0bf9c41d34d01a/merge-base.c (mode:100644 sha1:0f85e7d9e9a896d1142a54170ddf1159f11f9cdd)
@@ -0,0 +1,108 @@
+#include <stdlib.h>
+#include "cache.h"
+#include "revision.h"
+
+struct revision *common_ancestor(struct revision *rev1, struct revision *rev2)
+{
+ struct parent *parent;
+
+ struct parent *rev1list = malloc(sizeof(struct parent));
+ struct parent *rev2list = malloc(sizeof(struct parent));
+
+ struct parent *posn, *temp;
+
+ rev1list->parent = rev1;
+ rev1list->next = NULL;
+
+ rev2list->parent = rev2;
+ rev2list->next = NULL;
+
+ while (rev1list || rev2list) {
+ posn = rev1list;
+ rev1list = NULL;
+ while (posn) {
+ parse_commit_object(posn->parent);
+ if (posn->parent->flags & 0x0001) {
+ /*
+ printf("1 already seen %s %x\n",
+ sha1_to_hex(posn->parent->sha1),
+ posn->parent->flags);
+ */
+ // do nothing
+ } else if (posn->parent->flags & 0x0002) {
+ // XXXX free lists
+ return posn->parent;
+ } else {
+ /*
+ printf("1 based on %s\n",
+ sha1_to_hex(posn->parent->sha1));
+ */
+ posn->parent->flags |= 0x0001;
+
+ parent = posn->parent->parent;
+ while (parent) {
+ temp = malloc(sizeof(struct parent));
+ temp->next = rev1list;
+ temp->parent = parent->parent;
+ rev1list = temp;
+ parent = parent->next;
+ }
+ }
+ posn = posn->next;
+ }
+ posn = rev2list;
+ rev2list = NULL;
+ while (posn) {
+ parse_commit_object(posn->parent);
+ if (posn->parent->flags & 0x0002) {
+ /*
+ printf("2 already seen %s\n",
+ sha1_to_hex(posn->parent->sha1));
+ */
+ // do nothing
+ } else if (posn->parent->flags & 0x0001) {
+ // XXXX free lists
+ return posn->parent;
+ } else {
+ /*
+ printf("2 based on %s\n",
+ sha1_to_hex(posn->parent->sha1));
+ */
+ posn->parent->flags |= 0x0002;
+
+ parent = posn->parent->parent;
+ while (parent) {
+ temp = malloc(sizeof(struct parent));
+ temp->next = rev2list;
+ temp->parent = parent->parent;
+ rev2list = temp;
+ parent = parent->next;
+ }
+ }
+ posn = posn->next;
+ }
+ }
+ return NULL;
+}
+
+int main(int argc, char **argv)
+{
+ struct revision *rev1, *rev2, *ret;
+ unsigned char rev1key[20], rev2key[20];
+ if (argc != 3 ||
+ get_sha1_hex(argv[1], rev1key) ||
+ get_sha1_hex(argv[2], rev2key)) {
+ usage("mergebase <commit-id> <commit-id>");
+ }
+ rev1 = lookup_rev(rev1key);
+ rev2 = lookup_rev(rev2key);
+ ret = common_ancestor(rev1, rev2);
+ if (ret) {
+ printf("%s\n", sha1_to_hex(ret->sha1));
+ return 0;
+ } else {
+ printf("Sorry.\n");
+ return 1;
+ }
+
+}
^ permalink raw reply
* Re: Re-done kernel archive - real one?
From: Russell King @ 2005-04-17 15:24 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Git Mailing List, Peter Anvin
In-Reply-To: <Pine.LNX.4.58.0504161543590.7211@ppc970.osdl.org>
On Sat, Apr 16, 2005 at 04:01:45PM -0700, Linus Torvalds wrote:
> So I re-created the dang thing (hey, it takes just a few minutes), and
> pushed it out, and there's now an archive on kernel.org in my public
> "personal" directory called "linux-2.6.git". I'll continue the tradition
> of naming git-archive directories as "*.git", since that really ends up
> being the ".git" directory for the checked-out thing.
We need to work out how we're going to manage to get our git changes to
you. At the moment, I've very little idea how to do that. Ideas?
At the bottom is the script itself. There's probably some aspects of
it which aren't nice, maybe Petr can advise on this (and maybe increase
the functionality of the git shell script to fill in where necessary.)
However, I've made a start to generate the necessary emails. How about
this format?
I'm not keen on the tree, parent, author and committer objects appearing
in this - they appear to clutter it up. What're your thoughts?
I'd rather not have the FQDN of the machine where the commit happened
appearing in the logs. (I've 'xxxx'd it out for the time being, because
I'd rather not have yet more email-address-like objects get into spammers
databases with which to hammer my 512kbps DSL line.)
Linus,
Please incorporate the latest ARM changes.
This will update the following files:
arm/kernel/process.c | 15 +++++++++++----
arm/kernel/traps.c | 8 ++------
arm/lib/changebit.S | 11 ++---------
arm/lib/clearbit.S | 13 ++-----------
arm/lib/setbit.S | 11 ++---------
arm/lib/testchangebit.S | 15 ++-------------
arm/lib/testclearbit.S | 15 ++-------------
arm/lib/testsetbit.S | 15 ++-------------
arm/mach-footbridge/dc21285-timer.c | 4 ++--
arm/mach-sa1100/h3600.c | 2 +-
asm-arm/ptrace.h | 5 +----
asm-arm/system.h | 3 +++
12 files changed, 32 insertions(+), 85 deletions(-)
through these ChangeSets:
tree 7c4d75539c29ef7a9dde81acf84a072649f4f394
parent d5922e9c35d21f0b6b82d1fd8b1444cfce57ca34
author Russell King <rmk@xxxx.arm.linux.org.uk> 1113749462 +0100
committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113749462 +0100
[PATCH] ARM: bitops
Convert ARM bitop assembly to a macro. All bitops follow the same
format, so it's silly duplicating the code when only one or two
instructions are different.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
tree fc10d3ffa6062cda10a10cb8262d8df238aea4fb
parent 5d9a545981893629c8f95e2b8b50d15d18c6ddbc
author Russell King <rmk@xxxx.arm.linux.org.uk> 1113749436 +0100
committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113749436 +0100
[PATCH] ARM: showregs
Fix show_regs() to provide a backtrace. Provide a new __show_regs()
function which implements the common subset of show_regs() and die().
Add prototypes to asm-arm/system.h
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
tree 5591fced9a2b5f84c6772dcbe2eb4b24e29161fc
parent 488faba31f59c5960aabbb2a5877a0f2923937a3
author Russell King <rmk@xxxx.arm.linux.org.uk> 1113748846 +0100
committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113748846 +0100
[PATCH] ARM: h3600_irda_set_speed arguments
h3600_irda_set_speed() had the wrong type for the "speed" argument.
Fix this.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
tree 2493491da6e446e48d5443f0a549a10ed3d35b62
parent e7905b2f22eb5d5308c9122b9c06c2d02473dd4f
author Russell King <rmk@xxxx.arm.linux.org.uk> 1113748615 +0100
committer Russell King <rmk@xxxx.arm.linux.org.uk> 1113748615 +0100
[PATCH] ARM: footbridge rtc init
The footbridge ISA RTC was being initialised before we had setup the
kernel timer. This caused a divide by zero error when the current
time of day is set. Resolve this by initialising the RTC after
the kernel timer has been initialised.
Signed-off-by: Russell King <rmk@arm.linux.org.uk>
---
#!/bin/sh
prev=$(cat .git/heads/origin)
to=$(cat .git/HEAD)
who=Linus
what=ARM
cat << EOT
${who},
Please incorporate the latest ${what} changes.
This will update the following files:
EOT
git diff $prev $to | diffstat -p1
cat << EOT
through these ChangeSets:
EOT
this=$to
while [ "$this" != "$prev" ]; do
cat-file commit $this | sed 's,.*,\t&,'
this=$(cat-file commit $this | grep ^parent | cut -d ' ' -f 2)
done
--
Russell King
^ permalink raw reply
* [1/5] Parsing code in revision.h
From: Daniel Barkalow @ 2005-04-17 15:24 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <Pine.LNX.4.21.0504171108060.30848-100000@iabervon.org>
This adds support to revision.h for parsing commit records (but not going
any further than parsing a single record). Something like this is needed
by anything that uses revision.h, but older programs open-code it.
Signed-Off-By: Daniel Barkalow <barkalow@iabervon.org>
Index: revision.h
===================================================================
--- 45f926575d2c44072bfcf2317dbf3f0fbb513a4e/revision.h (mode:100644 sha1:28d0de3261a61f68e4e0948a25a416a515cd2e83)
+++ 37a0b01b85c2999243674d48bfc71cdba0e5518e/revision.h (mode:100644 sha1:523bde6e14e18bb0ecbded8f83ad4df93fc467ab)
@@ -24,6 +24,7 @@
unsigned int flags;
unsigned char sha1[20];
unsigned long date;
+ unsigned char tree[20];
struct parent *parent;
};
@@ -111,4 +112,29 @@
}
}
+static int parse_commit_object(struct revision *rev)
+{
+ if (!(rev->flags & SEEN)) {
+ void *buffer, *bufptr;
+ unsigned long size;
+ char type[20];
+ unsigned char parent[20];
+
+ rev->flags |= SEEN;
+ buffer = bufptr = read_sha1_file(rev->sha1, type, &size);
+ if (!buffer || strcmp(type, "commit"))
+ return -1;
+ get_sha1_hex(bufptr + 5, rev->tree);
+ bufptr += 46; /* "tree " + "hex sha1" + "\n" */
+ while (!memcmp(bufptr, "parent ", 7) &&
+ !get_sha1_hex(bufptr+7, parent)) {
+ add_relationship(rev, parent);
+ bufptr += 48; /* "parent " + "hex sha1" + "\n" */
+ }
+ //rev->date = parse_commit_date(bufptr);
+ free(buffer);
+ }
+ return 0;
+}
+
#endif /* REVISION_H */
^ permalink raw reply
* [0/5] Patch set for various things
From: Daniel Barkalow @ 2005-04-17 15:20 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20050417144947.GG1487@pasky.ji.cz>
Here are a bunch of patches which I made first against linus, that I've
rebased against pasky because they're mostly more version-control-like.
1: Add a parsing function to revision.h
2: Add merge-base
3: Add http-pull
4: Add option to make a hardlinkable cache of extracted options
5: Add commit id to version info
This also served as a test of cleaning up a patch series with git; I took
my current working directory, diffed it against its common ancestor with
pasky (no longer current), split the patch into logical pieces, and
applied them in sequence against the current pasky, committing after each
one. This gives a history as if I'd actually written the code just like I
would have had I known what I was doing in advance and done it very
quickly this morning. I think this should work in the future as a way to
avoid having the global revision control keeping developers' local
mistakes while keeping history the way the mainline saw the development.
A thought for future work: it would be nice if I could identify commits
that were used in creating a commit, but which should not be tracked down
unless you were unfortunate enough to have been exposed to them (in which
case you'd like know to deal with them).
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Ingo Molnar @ 2005-04-17 15:18 UTC (permalink / raw)
To: Brad Roberts; +Cc: Linus Torvalds, Petr Baudis, Simon Fowler, David Lang, git
In-Reply-To: <Pine.LNX.4.44.0504170804130.2625-100000@bellevue.puremagic.com>
* Brad Roberts <braddr@puremagic.com> wrote:
> While I agree that a hash collision is bad and certainly worth
> preventing during new object creation, for it to actually implant a
> trojan in a build successfully it'd have to meet even more criteria
> than you've layed out. It'd have to...
> - provide all the public symbols the shadowed object provided so that it
> would still build and link successfully
that's not a problem. Most modules dont provide public symbols.
Especially not drivers. Generally it's the modules that _dont_ have any
global impact that get reviewed less stringently - an attacker would
thus choose them for psychological reasons anyway.
> - be shadowing an object that's part of an active tree
>
> Shadowing an object that's not part of the working tree means
> something on another branch or obsoleted some time in the past is
> still db corruption, but not nearly as big an issue from a trojan
> standpoint.
it's not DB corruption, it's a feature of GIT: it's a content _cache_,
new and old alike. Nothing in GIT says that old objects in the
repository (which are still very much part of history) cannot be revived
in newer trees. (in fact it regularly happens - e.g. if a fix is undone
manually.)
Ingo
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Brad Roberts @ 2005-04-17 15:08 UTC (permalink / raw)
To: Ingo Molnar; +Cc: Linus Torvalds, Petr Baudis, Simon Fowler, David Lang, git
In-Reply-To: <20050417145232.GA5289@elte.hu>
On Sun, 17 Apr 2005, Ingo Molnar wrote:
> Date: Sun, 17 Apr 2005 16:52:32 +0200
> From: Ingo Molnar <mingo@elte.hu>
> To: Linus Torvalds <torvalds@osdl.org>
> Cc: Petr Baudis <pasky@ucw.cz>, Simon Fowler <simon@himi.org>,
> David Lang <david.lang@digitalinsight.com>, git@vger.kernel.org
> Subject: Re: Re: Merge with git-pasky II.
>
>
> * Linus Torvalds <torvalds@osdl.org> wrote:
>
> > Almost all attacks on sha1 will depend on _replacing_ a file with a
> > bogus new one. So guys, instead of using sha256 or going overboard,
> > just make sure that when you synchronize, you NEVER import a file you
> > already have.
>
> With tens of thousands (or hundreds of thousands) of objects expected in
> the repository sooner or later, there's quite a selection to pick from.
> Once you apply the patch, instead of the expected new file that you
> reviewed and found safe, the attacker has the other object included in
> the official kernel.
>
> A dangerous object can be anything: e.g. a debugging hack that allows
> arbitrary kernel-space writes. Or a known-insecure module (which since
> then got fixed, but the buggy code still exists in the DB). The module
> is in a single file and is self-installing (e.g. it has __init code to
> register itself as some driver.)
While I agree that a hash collision is bad and certainly worth preventing
during new object creation, for it to actually implant a trojan in a build
successfully it'd have to meet even more criteria than you've layed out.
It'd have to...
- be shadowing an object that's part of an active tree
- provide all the public symbols the shadowed object provided so that it
would still build and link successfully
Shadowing an object that's not part of the working tree means something on
another branch or obsoleted some time in the past is still db corruption,
but not nearly as big an issue from a trojan standpoint.
Later,
Brad
^ permalink raw reply
* Re: Re: Merge with git-pasky II.
From: Ingo Molnar @ 2005-04-17 14:52 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Petr Baudis, Simon Fowler, David Lang, git
In-Reply-To: <Pine.LNX.4.58.0504160913180.7211@ppc970.osdl.org>
* Linus Torvalds <torvalds@osdl.org> wrote:
> Almost all attacks on sha1 will depend on _replacing_ a file with a
> bogus new one. So guys, instead of using sha256 or going overboard,
> just make sure that when you synchronize, you NEVER import a file you
> already have.
here is a bit complex, but still practical attack that doesnt rely on
replacement and which can only be detected if we check the sha1
uniqueness assumptions.
If you can generate a duplicate sha1 key for an arbitrary 'target' file,
and Malice sends you a GIT-generated patch that introduces a new file
(which doesnt exist in the current tree) which you review (in the email)
and which looks safe to apply & harmless. Maybe the patch has a bit
weird formatting and some weird comments (which in reality Malice used
to generate the proper sha1 key) but otherwise the patch is for some
seldom used arcane driver that no-one used for quite some time and
no-one really cares about, so you are happy to apply the patch.
The compromise occurs when you apply the patch: the seemingly harmless
patch has an sha1 key that Malice manufacured to match that of an
already existing, 'dangerous' object in your database.
With tens of thousands (or hundreds of thousands) of objects expected in
the repository sooner or later, there's quite a selection to pick from.
Once you apply the patch, instead of the expected new file that you
reviewed and found safe, the attacker has the other object included in
the official kernel.
A dangerous object can be anything: e.g. a debugging hack that allows
arbitrary kernel-space writes. Or a known-insecure module (which since
then got fixed, but the buggy code still exists in the DB). The module
is in a single file and is self-installing (e.g. it has __init code to
register itself as some driver.)
Malice might even previously plant a dangerous object as some 'firmware
module' in another arcane driver, which doesnt get compiled by default,
but still shows up in the DB. Or Malice might plant a dangerous object
via an innocent-looking documentation file. (which contains some sample
code and is called sample.txt)
this type of 'false sharing attack' can only be prevented if an object
is only 'shared' with another object if it has been memcmp-ed with the
object in the repository. I.e. if we trust the sharing decision! Once
the attack has occured it cannot be detected automatically: only people
will notice it. (why did that weird unrelated module show up in that old
driver?)
The compromise relies on you having reviewed something harmless, while
in reality what happened within the DB was far less harmless. And the DB
remains self-consistent: neither fsck, nor others importing your tree
will be able to detect the compromise. This attack can only be detected
when you apply the patch, after that point all the information (except
Malice's message in your inbox) is gone.
so unless we actively check for collisions, once an sha1 key can be
generated at will on near-arbitrary input, it's not a secure system
anymore. We might be lucky and safe, but we wont be secure.
Ingo
^ permalink raw reply
* Re: Storing permissions
From: Daniel Barkalow @ 2005-04-17 14:51 UTC (permalink / raw)
To: David A. Wheeler; +Cc: Linus Torvalds, Paul Jackson, Morten Welinder, mj, git
In-Reply-To: <42620092.9040402@dwheeler.com>
On Sun, 17 Apr 2005, David A. Wheeler wrote:
> There's a minor reason to write out ALL the perm bit data, but
> only care about a few bits coming back in: Some people use
> SCM systems as a generalized backup system, so you can back up
> your system to an arbitrary known state in the past
> (e.g., "Change my /etc files to the state I was at
> just before I installed that &*#@ program!").
> For more on this, see:
> http://www.onlamp.com/pub/a/onlamp/2005/01/06/svn_homedir.html
>
> If you store all the bits, then you CAN restore things
> more exactly the way they were. This is imperfect, since
> it doesn't cover more exotic permission
> values from SELinux, xattrs, whatever. For some, that's enough.
I think this should be possible with a different tag than "tree". All the
bits aren't sufficient, anyway; the unincluded values include the user and
group, which are likely to matter for some things in /etc. But there's no
reason that the core can't support both a system-local complete
representation of the dentry and a user-relative representation of a
source distribution with different tags.
For that matter, it could accept "dir" objects in commits as well, and use
version-control-type logic on history while refusing to do non-sensical
things with them.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: Yet another base64 patch
From: Daniel Barkalow @ 2005-04-17 14:30 UTC (permalink / raw)
To: Paul Jackson; +Cc: dwheeler, git
In-Reply-To: <20050416210513.1ba26967.pj@sgi.com>
On Sat, 16 Apr 2005, Paul Jackson wrote:
> David wrote:
> > It's a trade-off, I know.
>
> So where do you recommend we make that trade-off?
So why do we have to be consistant? It seems like we need a standard
format for these reasons:
- We use rsync to interact with remote repositories, and rsync won't
understand if they aren't organized the same way. But I'm working on
having everything go through git-specific code, which could understand
different layouts.
- Everything that shares a local repository needs to understand the
format of that repository. But the filesystem constraints on the local
repository will be the same regardless of who is looking, so they'd all
expect the same format anyway.
So my idea is, once we're using git-smart transfer code (which can verify
objects, etc.), add support for different implementations of
sha1_file_name suitable for different filesystems, and vary based either
on a compile-time option or on a setting stored in the objects
directory. The only thing that matters is that repositories on
non-special web servers have a standard format, because they'll be serving
objects by URL, not by sha1.
-Daniel
*This .sig left intentionally blank*
^ permalink raw reply
* Re: Add lsremote command.
From: Petr Baudis @ 2005-04-17 14:03 UTC (permalink / raw)
To: Steven Cole; +Cc: git
In-Reply-To: <200504162336.51937.elenstev@mesatop.com>
Dear diary, on Sun, Apr 17, 2005 at 07:36:51AM CEST, I got a letter
where Steven Cole <elenstev@mesatop.com> told me that...
> This is a fairly trivial addition, but if users are adding remote repositories
> with git addremote, then those users should be able to list out the remote
> list without having to know the details of where the remotes file is kept.
Could you please send your patches inline? (Either in the body or with
correct content-disposition header.)
You got the return values other way around and you are missing a
copyright notice at the top; you should also mention that you take no
parameters.
Please use -s instead of -e, since it is more appropriate in this case.
Also, you should report the "no remotes" message to stderr. And always
exit when you found that .git/remotes exists, not only if cat succeeds.
Kind regards,
--
Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
^ permalink raw reply
* Re: Merge with git-pasky II.
From: David Woodhouse @ 2005-04-17 13:14 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Linus Torvalds, Junio C Hamano, Petr Baudis, git
In-Reply-To: <Pine.LNX.4.58.0504161733110.31775@wgmdd8.biozentrum.uni-wuerzburg.de>
On Sat, 2005-04-16 at 17:33 +0200, Johannes Schindelin wrote:
> > But if it can be done cheaply enough at a later date even though we end
> > up repeating ourselves, and if it can be done _well_ enough that we
> > shouldn't have just asked the user in the first place, then yes, OK I
> > agree.
>
> The repetition could be helped by using a cache.
Perhaps. Since neither such a cache nor even the commit comments are
strictly part of the git data, they probably shouldn't be included in
the sha1 hash of the commit object. However, I don't see a fundamental
reason why we couldn't store them in the same file but omit them from
the hash calculations. That also allows us to retrospectively edit
commit comments without completely changing the entire subsequent
history.
Or is that a little too heretical a suggestion?
--
dwmw2
^ permalink raw reply
* Re: fork optional branch point normazilation
From: Brad Roberts @ 2005-04-17 12:40 UTC (permalink / raw)
To: Petr Baudis; +Cc: git
In-Reply-To: <20050417122534.GD1487@pasky.ji.cz>
> >
> > Index: gitfork.sh
> > ===================================================================
> > --- 51b1bddbbc05e50d5bbf1f9662e503c2e85d5e96/gitfork.sh (mode:100755 sha1:e5692ea9bdbc39b028fe1e1205381da632541bab)
> > +++ c291316b28eff4042c80850cd93445345a606835/gitfork.sh (mode:100755 sha1:386148ae9a99739d06a09742ff4157d0f7e4e223)
> > @@ -37,6 +37,7 @@
> > [ -e "$destdir" ] && die "$destdir already exists"
> >
> > [ "$head" ] || head=$(commit-id)
> > +head=$(gitXnormid.sh -c $head)
> >
> > git lntree "$destdir"
> > echo $head >.git/heads/$name
>
> commit-id always returns the normalized commit ID.
>
> --
> Petr "Pasky" Baudis
> Stuff: http://pasky.or.cz/
> C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
>
This feels better to me. Diffed against my previus commit. The problem
was that commit-id wasn't called if a branch point was specified nor was
that value checked for validity.
Index: gitfork.sh
===================================================================
--- c9ccaa172ccab8e56f2fe621ee24896bfddacf26/gitfork.sh (mode:100755 sha1:386148ae9a99739d06a09742ff4157d0f7e4e223)
+++ f9e06a309f63ac6858d019b51f2172283378d2ef/gitfork.sh (mode:100755 sha1:dbb508b8431368fc95cc9516eada52f5bf0f8bc1)
@@ -16,7 +16,7 @@
name=$1
destdir=$2
-head=$3
+head=$(gitXnormid.sh -c $3)
die () {
echo gitfork.sh: $@ >&2
@@ -36,9 +36,6 @@
[ -e "$destdir" ] && die "$destdir already exists"
-[ "$head" ] || head=$(commit-id)
-head=$(gitXnormid.sh -c $head)
-
git lntree "$destdir"
echo $head >.git/heads/$name
ln -s heads/$name "$destdir/.git/HEAD"
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox