Git development
 help / color / mirror / Atom feed
* Re: MinGW port updated to 1.5.2
From: Aaron Gray @ 2007-05-26 21:36 UTC (permalink / raw)
  To: hanwen, git
In-Reply-To: <4658A01E.5060209@xs4all.nl>

> Aaron Gray escreveu:
>>>> GIT_EXTERNAL_DIFF works now.
>>>
>>> http://lilypond.org/git/binaries/mingw/
>>>
>>> uploading as I speak. Untested.
>>
>> This installs to "C:\Program Files\Git" and is not availiable on the
>> path. What is going on here please ?
>
> this uses the same buildscripts as before, so if it worked, it should
> still work. The path should point to $PF/Git/usr/bin/ ; you might need to
> log out and log in again for the path to visible.

Still not working properly.

>From CMD.EXE, neither 'git clone' and 'git-clone' are accessable.

>From MSYS '/usr/share/git-core/templates/' is not accessable.

    $ git clone git://git.kernel.org/pub/scm/git/git.git
    Initialized empty Git repository in C:/MSYS/home/Aaron Gray/git/.git/
    warning: templates not found /usr/share/git-core/templates/

Another issue is git-clone is no atomic if it fails it leaves an open file 
that can only be deleted by logging off and on again.

Aaron

^ permalink raw reply

* Re: MinGW port updated to 1.5.2
From: Han-Wen Nienhuys @ 2007-05-26 21:01 UTC (permalink / raw)
  To: git; +Cc: git
In-Reply-To: <001d01c79fd7$3d7f5ad0$0200a8c0@AMD2500>

Aaron Gray escreveu:
>>> GIT_EXTERNAL_DIFF works now.
>>
>> http://lilypond.org/git/binaries/mingw/
>>
>> uploading as I speak. Untested.
> 
> This installs to "C:\Program Files\Git" and is not availiable on the
> path. What is going on here please ?

this uses the same buildscripts as before, so if it worked, it should 
still work. The path should point to $PF/Git/usr/bin/ ; you might need to 
log out and log in again for the path to visible. 

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

^ permalink raw reply

* Re: MinGW port updated to 1.5.2
From: Aaron Gray @ 2007-05-26 20:48 UTC (permalink / raw)
  To: hanwen, git
In-Reply-To: <4658922B.1010807@xs4all.nl>

> Johannes Sixt escreveu:
>> This minute I've pushed the MinGW port, which is now at 1.5.2 to:
>>
>> gitweb:     http://repo.or.cz/w/git/mingw.git
>> pull/clone: git://repo.or.cz/git/mingw.git
>>
>> In addition to the 1.5.2 features, the progress indicator is shown in
>> more places (most notably when "Counting objects"), and
>> GIT_EXTERNAL_DIFF works now.
>
> http://lilypond.org/git/binaries/mingw/
>
> uploading as I speak. Untested.

This installs to "C:\Program Files\Git" and is not availiable on the path. 
What is going on here please ?

Thanks,

Aaron

^ permalink raw reply

* Re: [RFH] QGit: how to cram a patch in a crowded screen
From: Marco Costalba @ 2007-05-26 20:44 UTC (permalink / raw)
  To: Andy Parkins; +Cc: git, Pavel Roskin
In-Reply-To: <200705261034.53723.andyparkins@gmail.com>

On 5/26/07, Andy Parkins <andyparkins@gmail.com> wrote:
>
> For example, the log view widget would show:
>
> <Header>
> <Log Message>
> <Patch>
>
> All visually distinct to improve searching by eye (perhaps including
> clear separators between files patched).  Then the file list could
> include a "<header>" psuedo-file that would jump back to the top of the
> viewer.
>

This seems really gitk like. Not that I don' t like it, but _if_ it's
possible I would prefer something a little bit more original.

>
> Here's another option, keeping the patch tab, but putting the tab widget
> in the log view window.  That way the list would be visible and you
> would just switch between the log and the patch.
>

Yes. More or less along the lines of an handful of patches I've just
pushed to git://git.kernel.org/pub/scm/qgit/qgit4.git

Now user can toggle between revision commit message and patch content
with the menu entry 'View->toggle message/diff', or with shortcut 'm'
or also with mouse clicking on top right links 'Diff->' and 'Log->'.

There is also a check box in 'Edit->settings' called 'Show always
revision message as first' that, if set, let automatically to switch
to revision message when browsing on a new revision.

With this, browsing on a repo reading both log message and patch it's
a matter of 2 clicks or 2 shortcuts per revision, one for changing
revision (and read the log message) and another to view the patch
content.

This is a good speedup from before when we needed 3 commands, one for
selecting the rev, one for switching to patch tab and viewing the diff
and one for switching back to main list.


Comments? like/dislike?

Thanks
Marco

^ permalink raw reply

* Re: MinGW port updated to 1.5.2
From: Han-Wen Nienhuys @ 2007-05-26 20:01 UTC (permalink / raw)
  To: git; +Cc: git
In-Reply-To: <46569DA5.4F0EBF9A@eudaptics.com>

Johannes Sixt escreveu:
> This minute I've pushed the MinGW port, which is now at 1.5.2 to:
> 
> gitweb:     http://repo.or.cz/w/git/mingw.git
> pull/clone: git://repo.or.cz/git/mingw.git
> 
> In addition to the 1.5.2 features, the progress indicator is shown in
> more places (most notably when "Counting objects"), and
> GIT_EXTERNAL_DIFF works now.

http://lilypond.org/git/binaries/mingw/

uploading as I speak. Untested.


-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

^ permalink raw reply

* Re: Pulling from refs/remotes/ ?
From: Han-Wen Nienhuys @ 2007-05-26 19:47 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git
In-Reply-To: <7vwsyxsit8.fsf@assigned-by-dhcp.cox.net>

Junio C Hamano escreveu:
> And the reason it is "insane" is the same as why we currently do
> not interpret "git pull $repo origin/master" as a request to
> pull from remotes/origin/master.  It really is none of your
> business what kind of remote branches the other end is
> interacting with.

fair enough; my remaining gripe is that the error message is broken.
My suggestion is to have  

  git fetch . XXX

(and by extension: pull) print something like

  Will not fetch from self. 
  Do you mean "git merge XXX" or "git update-ref XXX" ? 

It may be an artificial restriction when compared to "git pull $dir XXX" 
but I think it helps the user better than "Fetch failure".

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

^ permalink raw reply

* Re: GIT on MinGW problem
From: Han-Wen Nienhuys @ 2007-05-26 19:42 UTC (permalink / raw)
  To: Johannes Schindelin; +Cc: Johannes Sixt, git
In-Reply-To: <Pine.LNX.4.64.0705251113280.4648@racer.site>

Johannes Schindelin escreveu:
> Hi,
> 
> On Fri, 25 May 2007, Johannes Sixt wrote:
> 
>> * I personally think that the files should go into
>>
>> 	$PROGRAMFILES/Git/{bin,share,lib}
>> instead of
>> 	$PROGRAMFILES/Git/usr/{bin,share,lib}
> 
> Agree. It is trivial, but it will help others. It might also be a good 
> idea to have a shortcut in "$PF/Git/Git Gui.lnk" to the git gui (once it 
> is working, that is).
> 
>> * git-gui and gitk don't work out of the box because they have the path
>> to wish hardcoded. They can't be started from CMD at all. I have written
>> wrappers gitk.cmd and git-gui.cmd with these 2 lines:
>>
>> @echo off
>> start wish84 D:/MSYS/1.0/git/bin/gitk %*
>>
>> But as you can see, the path is still hard-coded (but it is good enough
>> for me for the moment).
> 
> I'd also like to see bash, perl and wish bundled with the install (Windows 

Where is the info on the wish and bash port to Mingw?


-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

^ permalink raw reply

* Re: GIT on MinGW problem
From: Han-Wen Nienhuys @ 2007-05-26 19:41 UTC (permalink / raw)
  To: git
In-Reply-To: <4656A304.AF39A0B6@eudaptics.com>

Johannes Sixt escreveu:
>>
>> http://lilypond.org/git/binaries/mingw/git-1.5.1-2.mingw.exe
>>
>> it should also set $PATH.
> 
> I gave this some more testing and it turns out to be a well working
> toolset. Thank you very much!
> 
> There were still some issues remaining. These are the ones that should
> be fixable easily:
> 
> * git version reports just:
> 
> 	git version -dirty
> 
> Since git-gui parses the output of git version, but does not expect it
> to be of this format, and fails with an error message that it cannot
> parse the version.

My biggest problem is that the makefiles of git are an unmitigated
disaster, and there seems to be little interest in solving this
problem. For example, my suggestion to introduce autoconf was met with
derision.  Most of the effort was patching out makefile parts that
made my life harder. I may have patched the version part out as well.

In this, part of the pain is that Git tries to guess the version number
by itself in a complicated way.  It would be easiest if I could just 
specify the version number externally. In that case I can sync the installer
version number (1.5.1-2 in this case) and the version that git reports.


> * git without an correct git subcommand should list 20 or so commands,
> but it doesn't. The list is just empty.


there was a problem in generate cmd list,  (I have sort in /bin/ ). I
recommend to add

  set -u -v   

to all shell scripts so this doesn't go unnoticed.

> * I personally think that the files should go into
> 
> 	$PROGRAMFILES/Git/{bin,share,lib}
> instead of
> 	$PROGRAMFILES/Git/usr/{bin,share,lib}
> 
> The more difficult to solve problems are:

I understand, but it makes my life a lot more difficult.

> * git-gui and gitk don't work out of the box because they have the path
> to wish hardcoded. They can't be started from CMD at all. I have written
> wrappers gitk.cmd and git-gui.cmd with these 2 lines:
> 
> @echo off
> start wish84 D:/MSYS/1.0/git/bin/gitk %*
> 
> But as you can see, the path is still hard-coded (but it is good enough
> for me for the moment).

The only solution is to x-compile wish and include it as well.  I need several 
strong drinks to start trying this.  Is there a MinGW wish port?

> * perl scripts like git-remote contain a hard-coded path to the
> installation directory and don't work for this reason.

I actually commented out most perl stuff because the Makefile is just
too spaghetti-ish. I seem to have forgotten commenting out git-remote.

I thought the policy was to abandon Perl scripts for git commands?

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

^ permalink raw reply

* Re: [PATCH] user-manual: fixed typo in example
From: J. Bruce Fields @ 2007-05-26 19:35 UTC (permalink / raw)
  To: Steffen Prohaska; +Cc: git
In-Reply-To: <1180206987131-git-send-email-prohaska@zib.de>

On Sat, May 26, 2007 at 09:16:27PM +0200, Steffen Prohaska wrote:
>  -------------------------------------------------
> -$ gitk ($ git show-ref --heads ) --not  $( git show-ref --tags )
> +$ gitk $( git show-ref --heads ) --not  $( git show-ref --tags )
>  -------------------------------------------------

Oops; thanks!--b.

^ permalink raw reply

* [PATCH v3] Prevent megablobs from gunking up git packs
From: Dana How @ 2007-05-26 19:16 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Git Mailing List, danahow


Extremely large blobs distort general-purpose git packfiles.
These megablobs can be either stored in separate "kept" packfiles,
or left as loose objects.  Here we add some features to help
either approach.

This patch implements the following:
1. git pack-objects accepts --max-blob-size=N,  with the effect that
   only loose blobs less than N KB are written to the packfiles(s).
   If an already packed blob violates this limit (perhaps these are
   fast-import packs or max-blob-size was reduced),  it _is_ passed
   through if from a local pack and no loose copy exists.
2. git repack inspects repack.maxblobsize .  If set,  its
   value is passed to git pack-objects on the command line.
   --max-blob-size=N is also accepted by git repack.
3. No other git pack-objects caller uses this feature or sees any change.

During pack *creation* this minimizes including & deltifying megablobs.

During pack *use* this feature helps performance by keeping metadata
in a single smaller packfile,  and possibly reducing the number of index
files that must be read.  Megablobs could be separately packed,  or
left as loose objects.

Documentation has been updated and operation with pack-object's
--stdout is prevented.  This patch is based on "next".

Signed-off-by: Dana L. How <danahow@gmail.com>
---
 Documentation/config.txt           |    6 ++++++
 Documentation/git-pack-objects.txt |    5 +++++
 Documentation/git-repack.txt       |    9 +++++++++
 builtin-pack-objects.c             |   33 ++++++++++++++++++++++++++++-----
 cache.h                            |    1 +
 git-repack.sh                      |    9 ++++++++-
 sha1_file.c                        |    2 +-
 7 files changed, 58 insertions(+), 7 deletions(-)

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 179cb17..4a14f05 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -599,6 +599,12 @@ remotes.<group>::
 	The list of remotes which are fetched by "git remote update
 	<group>".  See gitlink:git-remote[1].
 
+repack.maxblobsize::
+	Prevent gitlink:git-repack[1] from newly packing blobs larger than
+	the specified number in kB,  unless overridden by --max-blob-size=N switch.
+	Affected blobs will still be repacked if from a local pack and no loose
+	copy exists.  Defaults to zero which means no maximum size is in effect.
+
 repack.usedeltabaseoffset::
 	Allow gitlink:git-repack[1] to create packs that uses
 	delta-base offset.  Defaults to false.
diff --git a/Documentation/git-pack-objects.txt b/Documentation/git-pack-objects.txt
index cfe127a..9b2e33d 100644
--- a/Documentation/git-pack-objects.txt
+++ b/Documentation/git-pack-objects.txt
@@ -85,6 +85,11 @@ base-name::
 	times to get to the necessary object.
 	The default value for --window is 10 and --depth is 50.
 
+--max-blob-size=<n>::
+	Maximum size of newly packed blobs, expressed in kB.
+	The default is unlimited.  Affected blobs will still be repacked
+	if from a local pack and no loose copy exists.
+
 --max-pack-size=<n>::
 	Maximum size of each output packfile, expressed in MiB.
 	If specified,  multiple packfiles may be created.
diff --git a/Documentation/git-repack.txt b/Documentation/git-repack.txt
index 2847c9b..b9d47e1 100644
--- a/Documentation/git-repack.txt
+++ b/Documentation/git-repack.txt
@@ -65,6 +65,11 @@ OPTIONS
 	to be applied that many times to get to the necessary object.
 	The default value for --window is 10 and --depth is 50.
 
+--max-blob-size=<n>::
+	Maximum size of newly packed blobs, expressed in kB.
+	The default is unlimited.  Affected blobs will still be repacked
+	if from a local pack and no loose copy exists.
+
 --max-pack-size=<n>::
 	Maximum size of each output packfile, expressed in MiB.
 	If specified,  multiple packfiles may be created.
@@ -84,6 +89,10 @@ be able to read (this includes repositories from which packs can
 be copied out over http or rsync, and people who obtained packs
 that way can try to use older git with it).
 
+The configuration variable `repack.MaxBlobSize` provides the
+default for the --max-blob-size option if set.  The latter
+takes precedence.
+
 
 Author
 ------
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 19b0aa1..59be849 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -17,7 +17,7 @@
 
 static const char pack_usage[] = "\
 git-pack-objects [{ -q | --progress | --all-progress }] [--max-pack-size=N] \n\
-	[--local] [--incremental] [--window=N] [--depth=N] \n\
+	[--local] [--incremental] [--window=N] [--depth=N] [--max-blob-size=N]\n\
 	[--no-reuse-delta] [--no-reuse-object] [--delta-base-offset] \n\
 	[--non-empty] [--revs [--unpacked | --all]*] [--reflog] \n\
 	[--stdout | base-name] [<ref-list | <object-list]";
@@ -75,6 +75,7 @@ static int num_preferred_base;
 static struct progress progress_state;
 static int pack_compression_level = Z_DEFAULT_COMPRESSION;
 static int pack_compression_seen;
+static uint32_t max_blob_size;
 
 /*
  * The object names in objects array are hashed with this hashtable,
@@ -371,8 +372,6 @@ static unsigned long write_object(struct sha1file *f,
 				pack_size_limit - write_offset : 0;
 				/* no if no delta */
 	int usable_delta =	!entry->delta ? 0 :
-				/* yes if unlimited packfile */
-				!pack_size_limit ? 1 :
 				/* no if base written to previous pack */
 				entry->delta->offset == (off_t)-1 ? 0 :
 				/* otherwise double-check written to this
@@ -408,7 +407,7 @@ static unsigned long write_object(struct sha1file *f,
 		buf = read_sha1_file(entry->sha1, &type, &size);
 		if (!buf)
 			die("unable to read %s", sha1_to_hex(entry->sha1));
-		if (size != entry->size)
+		if (size != entry->size && type == obj_type)
 			die("object %s size inconsistency (%lu vs %lu)",
 			    sha1_to_hex(entry->sha1), size, entry->size);
 		if (usable_delta) {
@@ -564,6 +563,17 @@ static off_t write_one(struct sha1file *f,
 			return 0;
 	}
 
+	/* refuse to include as many megablobs as possible */
+	if (max_blob_size && e->size >= max_blob_size) {
+		struct stat st;
+		/* skip if unpacked, remotely packed, or loose anywhere */
+		if (!e->in_pack || !e->in_pack->pack_local || find_sha1_file(e->sha1, &st)) {
+			e->offset = (off_t)-1;	/* might drop reused delta base if mbs less */
+			written++;
+			return offset;
+		}
+	}
+
 	e->offset = offset;
 	size = write_object(f, e, offset);
 	if (!size) {
@@ -1422,13 +1432,16 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
 
 	/* Now some size filtering heuristics. */
 	trg_size = trg_entry->size;
+	src_size = src_entry->size;
+	/* prevent use if could be later dropped from packfile */
+	if (max_blob_size && (trg_size >= max_blob_size || src_size >= max_blob_size))
+		return 0;
 	max_size = trg_size/2 - 20;
 	max_size = max_size * (max_depth - src_entry->depth) / max_depth;
 	if (max_size == 0)
 		return 0;
 	if (trg_entry->delta && trg_entry->delta_size <= max_size)
 		max_size = trg_entry->delta_size-1;
-	src_size = src_entry->size;
 	sizediff = src_size < trg_size ? trg_size - src_size : 0;
 	if (sizediff >= max_size)
 		return 0;
@@ -1735,6 +1748,13 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 			incremental = 1;
 			continue;
 		}
+		if (!prefixcmp(arg, "--max-blob-size=")) {
+			char *end;
+			max_blob_size = strtoul(arg+16, &end, 0) * 1024;
+			if (!arg[16] || *end)
+				usage(pack_usage);
+			continue;
+		}
 		if (!prefixcmp(arg, "--compression=")) {
 			char *end;
 			int level = strtoul(arg+14, &end, 0);
@@ -1855,6 +1875,9 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 	if (!pack_to_stdout && thin)
 		die("--thin cannot be used to build an indexable pack.");
 
+	if (pack_to_stdout && max_blob_size)
+		die("--max-blob-size cannot be used to build a pack for transfer.");
+
 	prepare_packed_git();
 
 	if (progress)
diff --git a/cache.h b/cache.h
index 4994d03..424b321 100644
--- a/cache.h
+++ b/cache.h
@@ -356,6 +356,7 @@ extern int move_temp_to_file(const char *tmpfile, const char *filename);
 
 extern int has_sha1_pack(const unsigned char *sha1, const char **ignore);
 extern int has_sha1_file(const unsigned char *sha1);
+extern char *find_sha1_file(const unsigned char *sha1, struct stat *st);
 extern void *map_sha1_file(const unsigned char *sha1, unsigned long *);
 
 extern int has_pack_file(const unsigned char *sha1);
diff --git a/git-repack.sh b/git-repack.sh
index 4ea6e5b..6b4e1af 100755
--- a/git-repack.sh
+++ b/git-repack.sh
@@ -8,7 +8,7 @@ SUBDIRECTORY_OK='Yes'
 . git-sh-setup
 
 no_update_info= all_into_one= remove_redundant=
-local= quiet= no_reuse= extra=
+local= quiet= no_reuse= extra= max_blob_size=
 while case "$#" in 0) break ;; esac
 do
 	case "$1" in
@@ -18,6 +18,7 @@ do
 	-q)	quiet=-q ;;
 	-f)	no_reuse=--no-reuse-object ;;
 	-l)	local=--local ;;
+	--max-blob-size=*) extra="$extra $1" max_blob_size=t ;;
 	--max-pack-size=*) extra="$extra $1" ;;
 	--window=*) extra="$extra $1" ;;
 	--depth=*) extra="$extra $1" ;;
@@ -35,6 +36,12 @@ true)
 	extra="$extra --delta-base-offset" ;;
 esac
 
+# handle blob limiting
+if [ -z "$max_blob_size" ]; then
+	mbs="`git config --int repack.maxblobsize`"
+	[ -n "$mbs" ] && extra="$extra --max-blob-size=$mbs"
+fi
+
 PACKDIR="$GIT_OBJECT_DIRECTORY/pack"
 PACKTMP="$GIT_OBJECT_DIRECTORY/.tmp-$$-pack"
 rm -f "$PACKTMP"-*
diff --git a/sha1_file.c b/sha1_file.c
index e4c3288..17e9dbf 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -387,7 +387,7 @@ void prepare_alt_odb(void)
 	read_info_alternates(get_object_directory(), 0);
 }
 
-static char *find_sha1_file(const unsigned char *sha1, struct stat *st)
+char *find_sha1_file(const unsigned char *sha1, struct stat *st)
 {
 	char *name = sha1_file_name(sha1);
 	struct alternate_object_database *alt;
-- 
1.5.2.764.g7ae34

^ permalink raw reply related

* [PATCH] user-manual: fixed typo in example
From: Steffen Prohaska @ 2007-05-26 19:16 UTC (permalink / raw)
  To: git; +Cc: Steffen Prohaska


Signed-off-by: Steffen Prohaska <prohaska@zib.de>
---
 Documentation/user-manual.txt |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Documentation/user-manual.txt b/Documentation/user-manual.txt
index 52247aa..4fabb8e 100644
--- a/Documentation/user-manual.txt
+++ b/Documentation/user-manual.txt
@@ -872,7 +872,7 @@ Obviously, endless variations are possible; for example, to see all
 commits reachable from some head but not from any tag in the repository:
 
 -------------------------------------------------
-$ gitk ($ git show-ref --heads ) --not  $( git show-ref --tags )
+$ gitk $( git show-ref --heads ) --not  $( git show-ref --tags )
 -------------------------------------------------
 
 (See gitlink:git-rev-parse[1] for explanations of commit-selecting
-- 
1.5.1.2

^ permalink raw reply related

* Re: [PATCH] Fix mishandling of $Id$ expanded in the repository copy in convert.c
From: Junio C Hamano @ 2007-05-26 19:05 UTC (permalink / raw)
  To: Andy Parkins; +Cc: git
In-Reply-To: <200705261012.50530.andyparkins@gmail.com>

Andy Parkins <andyparkins@gmail.com> writes:

> I had to jump through quite a few hoops to get the expanded $Id$ into a 
> repository (originally it was because I used an older version of git in 
> one place, and a newer one in another).
>
> I'll see what I can do to make a test case though.

Wouldn't it be sufficient to:

 (1) prepare a file with "$Id$", use ident in .gitattributes,
     check it in and commit;

 (2) remove it from the working tree, check it out with
     "checkout -f";

 (3) temorarily move away .gitattributes, modify the file, and
     check it in;

 (4) move .gitattributes back into its place, and commit.

^ permalink raw reply

* GitWeb: Adding fork visualization based on objects/info/alternates
From: Marius Storm-Olsen @ 2007-05-26 18:31 UTC (permalink / raw)
  To: Git Mailing List

Hi,

In an effort to tidy up the project list, and given that the
current fork support in GitWeb doesn't exactly fit my
structure (where forks are in subdirs of parent repo, and has
to be first level down; alas no seek unlike when not showing
forks), I set out to add fork support based on
    objects/info/alternates
of each repo. The result can be seen here:
    http://chaos.troll.no/~marius/git/forks_collapsed.png
    http://chaos.troll.no/~marius/git/forks_expanded.png

I doubt you'd want it in the mainline, as you can't turn the
feature off; it's just like another column which you can sort
by. If you do not sort by the 'fork' column, the forks will
still be colored with the first level color, but not ordered
under it's parent repo.

Just wanted to feed back what I did so others can get
inspiration, tweak it, and add it to their own gitwebs.

It cleans up the project list real nicely though.

Later!

--
.marius

--------------------------------------------------------------

>From a3659e21447181717b8b9edec319fe78f5643bdd Mon Sep 17 00:00:00 2001
From: Marius Storm-Olsen <marius@devil.troll.no>
Date: Sat, 26 May 2007 18:25:41 +0200
Subject: [PATCH] Adding fork visualization based on objects/info/alternates.
  1) All projects which have alternates are colorized
  2) When sorting based on forks, the forks are shown in
     a tree-like structure, which is collapsed by default.
     The user can expand each node to see the fork projects.
---
 gitweb/closed.png        |  Bin 0 -> 496 bytes
 gitweb/gitweb.css        |   23 ++++++++
 gitweb/gitweb.perl       |  137 +++++++++++++++++++++++++++++++++++----------
 gitweb/gitweb_scripts.js |   55 ++++++++++++++++++
 gitweb/open.png          |  Bin 0 -> 497 bytes
 5 files changed, 184 insertions(+), 31 deletions(-)
 create mode 100644 gitweb/closed.png
 create mode 100644 gitweb/gitweb_scripts.js
 create mode 100644 gitweb/open.png

diff --git a/gitweb/closed.png b/gitweb/closed.png
new file mode 100644
index 0000000000000000000000000000000000000000..ae630014d068e541f2b4d6b24617a4d6c896bd62
GIT binary patch
literal 496
zcmV<M0T2F(P)<h;3K|Lk000e1NJLTq000gE000gM1^@s6A4o0H00006VoOIv0RI60
z0RN!9r;`8x010qNS#tmY3ljhU3ljkVnw%H_000McNliru*995~FB_u49)$n^0ewkC
zK~#9!y^_yL8(|d1&!n>$0-=kVw~$cm3%Km_NZqH<g<E|G%~;xkwuTxKIwpgOAeeL#
zu&vPPBF2yih0dREa_@IP7eb%}T1D`z?&fz6oO6MP$aO1TskFR$Z!iphn(=sScHg>p
zH?VEn>Gis1FzB0$iyvk<9GacVwzu-)`OANCmg6{2eZTeF%H;q6ni(R3LZN_8r+u`!
zwN?Jw3%mu;vMl@Lq(cBOBSMnIP^m(aBp@a@j)Q(bXt?ff`P=z-e-7sJ`DdQ@Whx>_
zRSK0VL~(>9Nf5^|q9}q=DYVuI!(V>wu=aX1x~ki=Sr!9;7=dODB7$ZOCIT~qnUQ4~
zqCXKvqf1swDFCnmKtw<@0&4~lftWyidovS)AZWCYI~$Y9v_3C2mwRH*6SamIxium}
zw|jP4t$tp+o?f>W))3JwMqtFufoAilvA_56uY?6qYheJ$vJ3|YpBnGqeRwmOOuhd;
mfMr?stMxU%SX^4gW9SCp%l`C>LXp`30000<MNUMnLSTX$t<ll|

literal 0
HcmV?d00001

diff --git a/gitweb/gitweb.css b/gitweb/gitweb.css
index 9f0822f..ca9a77a 100644
--- a/gitweb/gitweb.css
+++ b/gitweb/gitweb.css
@@ -214,6 +214,29 @@ th {
 	text-align: left;
 }

+.folder {
+	background: url(closed.png) no-repeat;
+	float: left;
+	height: 14px;
+	width: 10px;
+	padding-right: 3px;
+}
+
+tr.fork, /* All forks higher than 10 will use this one */
+tr.fork1 { background-color: #ffffc0; }
+tr.fork2 { background-color: #ffffa0; }
+tr.fork3 { background-color: #ffff80; }
+tr.fork4 { background-color: #ffff60; }
+tr.fork5 { background-color: #ffff40; }
+tr.fork6 { background-color: #ffff20; }
+tr.fork7 { background-color: #ffff00; }
+tr.fork8 { background-color: #ffc000; }
+tr.fork9 { background-color: #ffa000; }
+
+tr.alternates {
+	color: #a0a0a0;
+}
+
 tr.light:hover {
 	background-color: #edece6;
 }
diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index c3921cb..65984dc 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -72,7 +72,7 @@ our $logo_label = "git homepage";
 our $projects_list = "++GITWEB_LIST++";

 # default order of projects list
-# valid values are none, project, descr, owner, and age
+# valid values are none, fork, project, descr, owner, and age
 our $default_projects_order = "project";

 # show repository only if this file exists
@@ -1282,6 +1282,16 @@ sub git_get_project_owner {
 	return $owner;
 }

+sub git_get_project_alternates_list {
+	my $path = shift;
+
+	open my $fd, "$projectroot/$path/objects/info/alternates" or return;
+	my @git_project_alternates_list = map { chomp; $_ } <$fd>;
+	close $fd;
+
+	return wantarray ? @git_project_alternates_list : \@git_project_alternates_list;
+}
+
 sub git_get_last_activity {
 	my ($path) = @_;
 	my $fd;
@@ -1909,8 +1919,9 @@ EOF
 		print qq(<link rel="shortcut icon" href="$favicon" type="image/png"/>\n);
 	}

+	print "<script src=\"gitweb_scripts.js\" type=\"text/javascript\"></script>\n";
 	print "</head>\n" .
-	      "<body>\n";
+	      "<body onload=\"collapseAllRows();\">\n";

 	if (-f $site_header) {
 		open (my $fd, $site_header);
@@ -2928,6 +2939,7 @@ sub git_project_list_body {
 	my ($check_forks) = gitweb_check_feature('forks');

 	my @projects;
+	my %forkProjects = ();
 	foreach my $pr (@$projlist) {
 		my (@aa) = git_get_last_activity($pr->{'path'});
 		unless (@aa) {
@@ -2942,6 +2954,12 @@ sub git_project_list_body {
 		if (!defined $pr->{'owner'}) {
 			$pr->{'owner'} = get_file_owner("$projectroot/$pr->{'path'}") || "";
 		}
+		my @altPaths = map { $_ =~ s=\/objects$==g; $_ }
+				   git_get_project_alternates_list($pr->{'path'});
+		if (!defined $pr->{'alternates'}) {
+			$pr->{'alternates'} = join(":",  @altPaths);
+		}
+
 		if ($check_forks) {
 			my $pname = $pr->{'path'};
 			if (($pname =~ s/\.git$//) &&
@@ -2954,6 +2972,12 @@ sub git_project_list_body {
 			}
 		}
 		push @projects, $pr;
+
+		if ($#altPaths + 1) {
+                    foreach my $altPath (@altPaths) {
+		        $forkProjects{$altPath}{"$projectroot/$pr->{'path'}"} = $pr;
+		    }
+		}
 	}

 	$order ||= $default_projects_order;
@@ -2966,6 +2990,15 @@ sub git_project_list_body {
 		if ($check_forks) {
 			print "<th></th>\n";
 		}
+		if ($order eq "fork") {
+			@projects = sort {$a->{'path'} cmp $b->{'path'}} @projects;
+			print "<th>+</th>\n";
+		} else {
+			print "<th>" .
+			      $cgi->a({-href => href(project=>undef, order=>'fork'),
+			               -class => "header"}, "+") .
+			      "</th>\n";
+		}
 		if ($order eq "project") {
 			@projects = sort {$a->{'path'} cmp $b->{'path'}} @projects;
 			print "<th>Project</th>\n";
@@ -3006,38 +3039,21 @@ sub git_project_list_body {
 		      "</tr>\n";
 	}
 	my $alternate = 1;
+	my $isForkOrder = ($order eq "fork");
 	for (my $i = $from; $i <= $to; $i++) {
 		my $pr = $projects[$i];
-		if ($alternate) {
-			print "<tr class=\"dark\">\n";
-		} else {
-			print "<tr class=\"light\">\n";
+		my $isFork = ($pr->{'alternates'} ne "");
+		my $rowColor = "light";
+		$rowColor = "dark" if ($alternate);
+		$rowColor = "fork" if ($isFork);
+
+		# If we're in fork ordering and this is a fork, we will not output this node
+		# since it has/will be outputted as a fork (child node) of another project.
+		if (!($isForkOrder && $isFork)) {
+			git_project_body($pr, $isForkOrder, $rowColor, "$i", $check_forks, \%forkProjects, "", 0);
 		}
+
 		$alternate ^= 1;
-		if ($check_forks) {
-			print "<td>";
-			if ($pr->{'forks'}) {
-				print "<!-- $pr->{'forks'} -->\n";
-				print $cgi->a({-href => href(project=>$pr->{'path'}, action=>"forks")}, "+");
-			}
-			print "</td>\n";
-		}
-		print "<td>" . $cgi->a({-href => href(project=>$pr->{'path'}, action=>"summary"),
-		                        -class => "list"}, esc_html($pr->{'path'})) . "</td>\n" .
-		      "<td>" . $cgi->a({-href => href(project=>$pr->{'path'}, action=>"summary"),
-		                        -class => "list", -title => $pr->{'descr_long'}},
-		                        esc_html($pr->{'descr'})) . "</td>\n" .
-		      "<td><i>" . chop_str($pr->{'owner'}, 15) . "</i></td>\n";
-		print "<td class=\"". age_class($pr->{'age'}) . "\">" .
-		      (defined $pr->{'age_string'} ? $pr->{'age_string'} : "No commits") . "</td>\n" .
-		      "<td class=\"link\">" .
-		      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"summary")}, "summary")   . " | " .
-		      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"shortlog")}, "shortlog") . " | " .
-		      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"log")}, "log") . " | " .
-		      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"tree")}, "tree") .
-		      ($pr->{'forks'} ? " | " . $cgi->a({-href => href(project=>$pr->{'path'}, action=>"forks")}, "forks") : '') .
-		      "</td>\n" .
-		      "</tr>\n";
 	}
 	if (defined $extra) {
 		print "<tr>\n";
@@ -3050,6 +3066,58 @@ sub git_project_list_body {
 	print "</table>\n";
 }

+sub git_project_body {
+	my ($pr, $isForkOrder, $rowColor, $parentNode, $check_forks, $forkProjectsRef, $indent, $indentLevel) = @_;
+
+	# Get the forks keys for this project
+	my $forkHash = $forkProjectsRef->{"$projectroot/$pr->{'path'}"};
+	my @forkKeys = sort keys (%$forkHash);
+	my $forkLevel = "";
+	$forkLevel = "$indentLevel" if ($indentLevel && $indentLevel < 10);
+	print "<tr class=\"$rowColor$forkLevel\" id=\"$parentNode\">\n";
+
+	if ($check_forks) {
+		print "<td>";
+		if ($pr->{'forks'}) {
+			print "<!-- $pr->{'forks'} -->\n";
+			print $cgi->a({-href => href(project=>$pr->{'path'}, action=>"forks")}, "+");
+		}
+		print "</td>\n";
+	}
+
+	# Add a folder anchor if this project has forks and we're in fork order
+	my $forkToggle = ">";
+	if ($#forkKeys + 1 && $isForkOrder) {
+	    $forkToggle = " onclick=\"toggleRows(this)\" class=\"folder\">";
+	}
+	print "<td><div class=\"tier\"><a href=\"#\"$forkToggle</a></div></td>";
+
+	print "<td>$indent" . $cgi->a({-href => href(project=>$pr->{'path'}, action=>"summary"),
+	                        -class => "list"}, esc_html($pr->{'path'})) . "</td>\n" .
+	      "<td>" . $cgi->a({-href => href(project=>$pr->{'path'}, action=>"summary"),
+	                        -class => "list", -title => $pr->{'descr_long'}},
+	                        esc_html($pr->{'descr'})) . "</td>\n" .
+	      "<td><i>" . chop_str($pr->{'owner'}, 15) . "</i></td>\n";
+	print "<td class=\"". age_class($pr->{'age'}) . "\">" .
+	      (defined $pr->{'age_string'} ? $pr->{'age_string'} : "No commits") . "</td>\n" .
+	      "<td class=\"link\">" .
+	      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"summary")}, "summary")   . " | " .
+	      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"shortlog")}, "shortlog") . " | " .
+	      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"log")}, "log") . " | " .
+	      $cgi->a({-href => href(project=>$pr->{'path'}, action=>"tree")}, "tree") .
+	      ($pr->{'forks'} ? " | " . $cgi->a({-href => href(project=>$pr->{'path'}, action=>"forks")}, "forks") : '') .
+	      "</td>\n" .
+	      "</tr>\n";
+
+	return if (!$isForkOrder || !($#forkKeys + 1));
+	my $siblingCount = 0;
+	for my $key (@forkKeys) {
+		my $forkProject = ${$forkHash}{$key};
+		$siblingCount++;
+		git_project_body($forkProject, $isForkOrder, "fork", $parentNode."-$siblingCount", $check_forks, $forkProjectsRef, $indent."&nbsp;&nbsp;", $indentLevel+1);
+	}
+}
+
 sub git_shortlog_body {
 	# uses global variable $project
 	my ($commitlist, $from, $to, $refs, $extra) = @_;
@@ -3315,7 +3383,7 @@ sub git_search_grep_body {

 sub git_project_list {
 	my $order = $cgi->param('o');
-	if (defined $order && $order !~ m/none|project|descr|owner|age/) {
+	if (defined $order && $order !~ m/none|fork|project|descr|owner|age/) {
 		die_error(undef, "Unknown order parameter");
 	}

@@ -3419,6 +3487,13 @@ sub git_summary {
 		print "<tr><td>$url_tag</td><td>$git_url</td></tr>\n";
 		$url_tag = "";
 	}
+	my $alternates_tag = "ALT";
+	my @alternates_list = git_get_project_alternates_list($project);
+	foreach my $alternate (@alternates_list) {
+		next unless $alternate;
+		print "<tr class=\"alternates\"><td>$alternates_tag</td><td>$alternate</td></tr>\n";
+		$alternates_tag = "";
+	}
 	print "</table>\n";

 	if (-s "$projectroot/$project/README.html") {
diff --git a/gitweb/gitweb_scripts.js b/gitweb/gitweb_scripts.js
new file mode 100644
index 0000000..79f43dc
--- /dev/null
+++ b/gitweb/gitweb_scripts.js
@@ -0,0 +1,55 @@
+// Javascript based on the SSTree project
+// http://sstree.tigris.org/
+
+function toggleRows(elm) {
+    var rows = document.getElementsByTagName("tr");
+    elm.style.backgroundImage = "url(closed.png)";
+    var newDisplay = "none";
+    var thisID = elm.parentNode.parentNode.parentNode.id + "-";
+    // Are we expanding or contracting? If the first child is hidden, we expand
+    for (var i = 0; i < rows.length; i++) {
+        var r = rows[i];
+        if (matchStart(r.id, thisID, true)) {
+            if (r.style.display == "none") {
+                if (document.all) newDisplay = "block";     // IE4+ specific code
+                else newDisplay = "table-row";              // Netscape and Mozilla
+                elm.style.backgroundImage = "url(open.png)";
+            }
+            break;
+        }
+    }
+
+    // When expanding, only expand one level.  Collapse all desendants.
+    var matchDirectChildrenOnly = (newDisplay != "none");
+
+    for (var j = 0; j < rows.length; j++) {
+        var s = rows[j];
+        if (matchStart(s.id, thisID, matchDirectChildrenOnly)) {
+            s.style.display = newDisplay;
+            var cell = s.getElementsByTagName("td")[0];
+            var tier = cell.getElementsByTagName("div")[0];
+            var folder = tier.getElementsByTagName("a")[0];
+            if (folder.getAttribute("onclick") != null) {
+                folder.style.backgroundImage = "url(closed.png)";
+            }
+        }
+    }
+}
+
+function matchStart(target, pattern, matchDirectChildrenOnly) {
+    var pos = target.indexOf(pattern);
+    if (pos != 0) return false;
+    if (!matchDirectChildrenOnly) return true;
+    if (target.slice(pos + pattern.length, target.length).indexOf("-") >= 0) return false;
+    return true;
+}
+
+function collapseAllRows() {
+    var rows = document.getElementsByTagName("tr");
+    for (var j = 0; j < rows.length; j++) {
+        var r = rows[j];
+        if (r.id.indexOf("-") >= 0) {
+            r.style.display = "none";
+        }
+    }
+}
diff --git a/gitweb/open.png b/gitweb/open.png
new file mode 100644
index 0000000000000000000000000000000000000000..ea1f3a0d4affc87d570d36a960479efd07105c9c
GIT binary patch
literal 497
zcmV<N0S^9&P)<h;3K|Lk000e1NJLTq000gE000gM1^@s6A4o0H0005FNkl<Zc-pOx
zziU%b7>1v7?hmhtH#axwVB=!*Z)l)Y7mHI}>ehm!d(q7yItA=v(a_>Q(CVZ@aj-%_
z+HC3^2uX49c8vF)4n60@!A8>9Mev1h_+Flm_x&6IA%rW0kgn_auImLtAf=RU5L}M5
z)~7)bgn+QtYR)789zI(CytH)h27nY204e2|K}w0|c|=h}r_*UKFW;Yc0J`07Z}HCJ
z3M0@)V~s^f!C4!m<KX)~QB<T}f3TV)$9)GtDb)`P`P%ilxvJJ0V+}?dj5UD3^E~qT
zJcpfw_RANo27oiJGZ+kx7H%!9SZmRz8fz^hJHi;vCEur1Dzmm$UrmzZ{<%Y?RKFNU
zwMyk$bzD~)4I{|qa(w;vrQK?6HU9j~gx<Y<-y9ByWLZX*r6{FPN)d)38ynA>ldLnj
zD5ZKcGo`!ba=Bux#TbL@xeNyV_V(7R$CK>2M1Z}$59?W$lBT~W2m+pOZZ@VlXWBSX
zCr4r7N-d7#DvU^yAN#L&-#ne>{z?QmJm@sjpDD%|c6N3f7ctI7rc$aG6$|sm*zfxv
nKRw}J6vxGzv$I!k^S|{A;Yhe4$I~??00000NkvXXu0mjf*^uU~

literal 0
HcmV?d00001

--
1.5.2.58.g98ee

^ permalink raw reply related

* Re: [PATCH] Don't ignore write failure from git-diff, git-log, etc.
From: Junio C Hamano @ 2007-05-26 17:27 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Jim Meyering, git
In-Reply-To: <alpine.LFD.0.98.0705260910220.26602@woody.linux-foundation.org>

Linus Torvalds <torvalds@linux-foundation.org> writes:

> Also, PLEASE don't do this:
>
>> +		if (0 <= fcntl(fileno (stdout), F_GETFD)
>
> That's totally unreadable to any normal human.
>
> You don't say "if zero is smaller or equal to X". You say "if X is larger 
> than or equal to zero". Stop messing with peoples minds, dammit!
>
> Anybody who thinks that code like this causes fewer errors is just fooling 
> himself. It causes *more* bugs, because people have a harder time reading 
> it.
>
> Maybe you and Junio have taught yourself bad manners, but you're a tiny 
> tiny part of humanity or the development community. Junio can do it just 
> because while he's just a single person, he's a big part of the git coding 
> base, but anybody else who does it should just be shot.

Whew, that is a blast from the past.

cf. http://thread.gmane.org/gmane.comp.version-control.git/3903/focus=3906

 (1) Maybe Jim was just being nice, trying to make the code look
     like surrounding code;

 (2) Maybe Jim and the person I learned the style from worked
     together for a long time and they picked it up from the
     same source;

 (3) Maybe I am not alone, and it is not native language -
     mother tongue issue as some suspected in the quoted thread.

In any case, I think my recent code have much less "textual
order should reflect actual order" convention than before,
because I have been forcing myself to say aloud "if X is larger"
or "if X is smaller" before writing my comparisons, in order to
match the "peoples minds" expectation you mentioned above.

This initially slowed me down and made my head hurt quite a bit,
and sometimes it still does.

Once you learn to _visualize_ the ordering relationship in "X op
Y" by relying on "op" being always < or <=, you will get the
"number line" pop in your head whenever you see a comparision
expression, without even having to think about it, and you "see"
X and Y on the number line:

        ... -2        -1         0         1         2  ...  
    ---------+---------+---------+---------+---------+---------
    true:                        0   <=  fcntl(...)


        ... -2        -1         0         1         2  ...  
    ---------+---------+---------+---------+---------+---------
    false:    (0 <= fcntl(...))

What the comparison is doing comes naturally to you, without
even having to translate it back to human language "X is larger
(or smaller) than this constant".  The ordering is right there,
in front of your eyes, before you vocalize it.

In a sense, just like it is hard to go back from git to CVS (or
it is hard to go back to not knowing the power of the index), it
is very hard to go back once you learn to do this.

^ permalink raw reply

* Re: [PATCH 1/3] Lazily open pack index files on demand
From: Shawn O. Pearce @ 2007-05-26 17:30 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Dana How
In-Reply-To: <7vabvsm1h8.fsf@assigned-by-dhcp.cox.net>

Junio C Hamano <junkio@cox.net> wrote:
> "Shawn O. Pearce" <spearce@spearce.org> writes:
> 
> >  This conflicts (in a subtle way) with Dana How's
> >  "sha1_file.c:rearrange_packed_git() should consider packs' object
> >  sizes" patch as we now have num_objects = 0 for any indexes we
> >  have not opened.  In the case of Dana's patch this would cause
> >  those packfiles to have very high ranks, possibly sorting much
> >  later than they should have.
> 
> I am keeping that rearrange stuff on hold, partly because I am
> moderately hesitant to do the fp, which feels overkill at that
> low level of code.

Yea, I've actually been having similiar thoughts.
 
> Also, I am hoping that we can discard that the object density
> criteria altogether by making the default repack behaviour
> friendlier to the pathological cases, e.g. by emitting huge
> blobs at the end of the packstream, potentially pushing it out
> to later parts of split packs by themselves and automatically
> marking them with the .keep flag.  Until that kind of
> improvements materialize, people with pathological cases could
> (1) handcraft a pack that contains only megablob, (2) place that
> on central alternate, (3) touch it with artificially old
> timestamp, which hopefully is a good enough workaround.

Right, I was having the same idea.  If we have pack-objects just
shuffle the really big stuff to the end of its object list they
will naturally fall into the end of the packfile, and the split
out packfiles.  Then if we do the mtime flipping you suggested
earlier right before we exit pack-objects the larger blob packs
will automatically sort behind the smaller commit/tree packs.
No fp needed.

My patch was exactly because I did what you say above; I handcrafted
a pack that contains only large-ish blobs, placed them into a
central repo, and connected it by alternates.  Because of the
local flag logic it is automatically behind my commit/tree pack.
But I also rarely (if ever) have to access that megablob packfile.
Yet the .idx was still being opened.  On Cygwin/Windows that penalty
is high enough to have almost doubled the running time of a simple
"git show".

-- 
Shawn.

^ permalink raw reply

* Re: [PATCH 1/3] Lazily open pack index files on demand
From: Dana How @ 2007-05-26 17:31 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: Shawn O. Pearce, git, danahow
In-Reply-To: <7vabvsm1h8.fsf@assigned-by-dhcp.cox.net>

On 5/26/07, Junio C Hamano <junkio@cox.net> wrote:
> "Shawn O. Pearce" <spearce@spearce.org> writes:
> >  This conflicts (in a subtle way) with Dana How's
> >  "sha1_file.c:rearrange_packed_git() should consider packs' object
> >  sizes" patch as we now have num_objects = 0 for any indexes we
> >  have not opened.  In the case of Dana's patch this would cause
> >  those packfiles to have very high ranks, possibly sorting much
> >  later than they should have.
> I am keeping that rearrange stuff on hold, partly because I am
> moderately hesitant to do the fp, which feels overkill at that
> low level of code.
Oh,  I thought the fp might cause a gag reflex -- I had to add -lm.
Unfortunately,  when trying to automatically detect and grade outliers,
which is what I was trying to do,  (datum - mean) / std_dev is hard to beat,
and I needed sqrt for std_dev -- all other fp could be easily written out.

> Also, I am hoping that we can discard that the object density
> criteria altogether by making the default repack behaviour
> friendlier to the pathological cases, e.g. by emitting huge
> blobs at the end of the packstream, potentially pushing it out
> to later parts of split packs by themselves and automatically
> marking them with the .keep flag.  Until that kind of
> improvements materialize, people with pathological cases could
> (1) handcraft a pack that contains only megablob, (2) place that
> on central alternate, (3) touch it with artificially old
> timestamp, which hopefully is a good enough workaround.
I think we should do what we can to make the timestamp as
meaningful as possible,  which is why I submitted that stamping patch.

I think there are two interesting strategies compatible
with maximally-informative timestamps:

(1) git-repack -a -d repacks everything on each call.  You would need:
(1a) Rewrite builtin-pack-objects.c so only the object_ix hash
       accesses the "objects" array directly, everything else
       goes through a pointer table.
(1b) Sort the new pointer table by object type,  in order
       tag -> commit -> tree -> nice blob -> naughty blob.
      The sort is stable so the order within each group is unchanged.
(1c) Do not deltify naughty blobs.  Naughty blobs are those
      blobs marked "nodelta" or very large blobs.
(1d) Write out objects in new pointer table order.  Splitting
       will cause metadata to be in first pack,  naughty blobs
       tend to be in the last pack.
(1e) When done writing all packs,  swap their timestamps
      so current timestamp sorting will look at naughty blobs last.

(2) git-repack -a -d runs in two passes and maintains .keep files:
(2a) Add a new flag --types=[gctb]+ to pack-objects to be supplied
      by git-repack.  This means only taGs/Commits/Trees/Blobs
      are to be passed,  all others dropped.
(2b) Put a new loop around the core of git-repack.  In the first iteration,
      pack with --types=b, then with --types=gct in the second.
      Thus metadata will have more recent timestamp.
(2c) If packs are split, also swap timestamps like in (1e),
       within each iteration.
(2d) If an iteration produces split packs, mark all but the last
      in the sequence with a .keep file automatically.  The
      .keep files contain the string "repack".
(2e) Add new option to repack: -A.  If specified,  the first
      thing repack does is remove any keep file containing "repack".
(2f) The existing response of repack to keep files -- do not repack them --
     is retained to ensure on each -a/not -A repack,  we only
     repack the tail of each set of packs: metadata, data.
     The metadata set will probably only ever contain one pack
     and will always be repacked.

I've (badly) implemented (1b) and confirmed it had no impact
on linux-2.6 repo.  I've also implemented (2a), (2b), (2d), and (2f),
but not fully measured them.  I'd like to finish this work,  but
"megapacks" are very time-consuming to manipulate,  and
with the loose megablob approach they are not as useful for me.

Finally,  some people might want more esoteric repacking
strategies than what I've listed above.  We could add a
--packed flag to pack-objects to help them.  This means that
git pack-objects --packed --unpacked=<pack1> --unpacked=<pack2>
would only repack pack1 and pack2 and would not absorb
any loose blobs.  This would allow you to maintain any number
of packfile classes you want and maintain them yourself.
Each would be indicated by something different in a .keep file.
(To newly absorb loose blobs in a class,  you would do
 cat object-list | git-pack-objects --incremental
from some object-list you built following your rules).
These strategies would be too special-purpose to be in git,
but adding --packed is a small and useful change.

Shawn:  When I first saw the index-loading code,  my first
thought was that all the index tables should be
merged (easy since sorted) so callers only need to do one search.
With indices loaded lazily,  either you can't merge,  or you
merge sequentially, raising merge cost from (total entries) to
almost (index files) * (total entries).  What do you think about
merging the SHA-1 tables,  and how would/should it interact with
lazy index file loading?

BTW,  if it's not apparent,  I think my object density patch
should be dropped.  It has served its purpose as a thought experiment.

Thanks,
-- 
Dana L. How  danahow@gmail.com  +1 650 804 5991 cell

^ permalink raw reply

* Make the pack-refs interfaces usable from outside
From: Linus Torvalds @ 2007-05-26 16:25 UTC (permalink / raw)
  To: Junio C Hamano, Git Mailing List


This just basically creates a "pack_refs()" function that could be used by 
anybody. You pass it in the flags you want as a bitmask (PACK_REFS_ALL and 
PACK_REFS_PRUNE), and it will do all the heavy lifting.

Of course, it's still static, and it's all in the builtin-pack-refs.c 
file, so it's not actually visible to the outside, but the next step would 
be to just move it all to a library file (probably refs.c) and expose it.

Then we could easily make "git gc" do this too.

While I did it, I also made it check the return value of the fflush and 
fsync stage, to make sure that we don't overwrite the old packed-refs file 
with something that got truncated due to write errors!

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
---

It all looks very obvious, and it passed the trivial tests I did, but 
somebody should probably double-check.

Hmm?

		Linus

---
 builtin-pack-refs.c |   66 +++++++++++++++++++++++++++++---------------------
 1 files changed, 38 insertions(+), 28 deletions(-)

diff --git a/builtin-pack-refs.c b/builtin-pack-refs.c
index d080e30..1952950 100644
--- a/builtin-pack-refs.c
+++ b/builtin-pack-refs.c
@@ -12,9 +12,11 @@ struct ref_to_prune {
 	char name[FLEX_ARRAY];
 };
 
+#define PACK_REFS_PRUNE	0x0001
+#define PACK_REFS_ALL	0x0002
+
 struct pack_refs_cb_data {
-	int prune;
-	int all;
+	unsigned int flags;
 	struct ref_to_prune *ref_to_prune;
 	FILE *refs_file;
 };
@@ -39,7 +41,7 @@ static int handle_one_ref(const char *path, const unsigned char *sha1,
 	is_tag_ref = !prefixcmp(path, "refs/tags/");
 
 	/* ALWAYS pack refs that were already packed or are tags */
-	if (!cb->all && !is_tag_ref && !(flags & REF_ISPACKED))
+	if (!(cb->flags & PACK_REFS_ALL) && !is_tag_ref && !(flags & REF_ISPACKED))
 		return 0;
 
 	fprintf(cb->refs_file, "%s %s\n", sha1_to_hex(sha1), path);
@@ -53,7 +55,7 @@ static int handle_one_ref(const char *path, const unsigned char *sha1,
 		}
 	}
 
-	if (cb->prune && !do_not_prune(flags)) {
+	if ((cb->flags & PACK_REFS_PRUNE) && !do_not_prune(flags)) {
 		int namelen = strlen(path) + 1;
 		struct ref_to_prune *n = xcalloc(1, sizeof(*n) + namelen);
 		hashcpy(n->sha1, sha1);
@@ -85,26 +87,51 @@ static void prune_refs(struct ref_to_prune *r)
 
 static struct lock_file packed;
 
-int cmd_pack_refs(int argc, const char **argv, const char *prefix)
+static int pack_refs(unsigned int flags)
 {
-	int fd, i;
+	int fd;
 	struct pack_refs_cb_data cbdata;
 
 	memset(&cbdata, 0, sizeof(cbdata));
+	cbdata.flags = flags;
+
+	fd = hold_lock_file_for_update(&packed, git_path("packed-refs"), 1);
+	cbdata.refs_file = fdopen(fd, "w");
+	if (!cbdata.refs_file)
+		die("unable to create ref-pack file structure (%s)",
+		    strerror(errno));
+
+	/* perhaps other traits later as well */
+	fprintf(cbdata.refs_file, "# pack-refs with: peeled \n");
+
+	for_each_ref(handle_one_ref, &cbdata);
+	if (fflush(cbdata.refs_file) || fsync(fd) || fclose(cbdata.refs_file))
+		die("failed to write ref-pack file (%s)", strerror(errno));
+	if (commit_lock_file(&packed) < 0)
+		die("unable to overwrite old ref-pack file (%s)", strerror(errno));
+	if (cbdata.flags & PACK_REFS_PRUNE)
+		prune_refs(cbdata.ref_to_prune);
+	return 0;
+}
 
-	cbdata.prune = 1;
+int cmd_pack_refs(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	unsigned int flags;
+
+	flags = PACK_REFS_PRUNE;
 	for (i = 1; i < argc; i++) {
 		const char *arg = argv[i];
 		if (!strcmp(arg, "--prune")) {
-			cbdata.prune = 1; /* now the default */
+			flags |= PACK_REFS_PRUNE; /* now the default */
 			continue;
 		}
 		if (!strcmp(arg, "--no-prune")) {
-			cbdata.prune = 0;
+			flags &= ~PACK_REFS_PRUNE;
 			continue;
 		}
 		if (!strcmp(arg, "--all")) {
-			cbdata.all = 1;
+			flags |= PACK_REFS_ALL;
 			continue;
 		}
 		/* perhaps other parameters later... */
@@ -113,22 +140,5 @@ int cmd_pack_refs(int argc, const char **argv, const char *prefix)
 	if (i != argc)
 		usage(builtin_pack_refs_usage);
 
-	fd = hold_lock_file_for_update(&packed, git_path("packed-refs"), 1);
-	cbdata.refs_file = fdopen(fd, "w");
-	if (!cbdata.refs_file)
-		die("unable to create ref-pack file structure (%s)",
-		    strerror(errno));
-
-	/* perhaps other traits later as well */
-	fprintf(cbdata.refs_file, "# pack-refs with: peeled \n");
-
-	for_each_ref(handle_one_ref, &cbdata);
-	fflush(cbdata.refs_file);
-	fsync(fd);
-	fclose(cbdata.refs_file);
-	if (commit_lock_file(&packed) < 0)
-		die("unable to overwrite old ref-pack file (%s)", strerror(errno));
-	if (cbdata.prune)
-		prune_refs(cbdata.ref_to_prune);
-	return 0;
+	return pack_refs(flags);
 }

^ permalink raw reply related

* Re: [PATCH] Don't ignore write failure from git-diff, git-log, etc.
From: Linus Torvalds @ 2007-05-26 16:18 UTC (permalink / raw)
  To: Jim Meyering; +Cc: git
In-Reply-To: <87bqg724gp.fsf@rho.meyering.net>



On Sat, 26 May 2007, Jim Meyering wrote:
>
> Each git command should report such a failure.
> Some already do, but with the patch below, they all do, and we
> won't have to rely on code in each command's implementation to
> perform the right incantation.

The patch is wrong.

Some write errors are expected and GOOD.

For example, EPIPE should not be reported. It's normal. The user got 
bored. It might be hidden by the SIGPIPE killing us, but regardless, 
reporting it for the normal log/diff thing is just not correct. EPIPE 
isn't an error, it's a "ok, nobody is listening any more".

Also, PLEASE don't do this:

> +		if (0 <= fcntl(fileno (stdout), F_GETFD)

That's totally unreadable to any normal human.

You don't say "if zero is smaller or equal to X". You say "if X is larger 
than or equal to zero". Stop messing with peoples minds, dammit!

Anybody who thinks that code like this causes fewer errors is just fooling 
himself. It causes *more* bugs, because people have a harder time reading 
it.

Maybe you and Junio have taught yourself bad manners, but you're a tiny 
tiny part of humanity or the development community. Junio can do it just 
because while he's just a single person, he's a big part of the git coding 
base, but anybody else who does it should just be shot.

			Linus

^ permalink raw reply

* Re: Problems importing SVN repo via git-svnimport
From: Frank Lichtenheld @ 2007-05-26 15:34 UTC (permalink / raw)
  To: Art Haas; +Cc: git
In-Reply-To: <20070526151046.GA2239@artsapartment.org>

On Sat, May 26, 2007 at 10:10:46AM -0500, Art Haas wrote:
> On Sat, May 26, 2007 at 03:06:43PM +0200, Frank Lichtenheld wrote:
> > On Sat, May 26, 2007 at 07:55:53AM -0500, Art Haas wrote:
> > > I'm happy to dive into the perl code in svnimport and make changes to
> > > help debug this problem, if there is actually a problem and not user
> > > error on my part. Any help from 'svnimport' experts will be greatly
> > > appreciated. The public repo listed above has the same contents as my
> > > local repo, so feel free to poke around it to see how things are
> > > structured.
> > 
> > You might want to try git-svn instead. It should support everything you
> > can do with git-svnimport and much more. Plus it is actively maintained
> > which I wouldn't say about git-svnimport.
> 
> Hi.
> 
> The git-svn docs say that it serves as a bi-directionaly gateway

It can be happily used for one-time imports, too. You might want to
use some options like the authors file and perhaps --no-metadata.
Can't say much about the rest of your plans since I'm no git-svn
expert. I have only ever used it for one-time imports so far.

> between git and svn, but that the main repo is a Subversion repository.
> My goal with the PythonCAD repo is to convert it so that the main
> repository is in git, and then I'm going to try to use git-svn to
> push my changes into the now-secondary Subversion repo. This way, I can
> make a distributed git repo available to developers wanting to
> contribute to PythonCAD, plus keep the developers currently using
> Subversion from having to make any changes if they either don't want to
> or are unable to (i.e. windows developers/users).
> 
> Is git-svnimport maintenence on the wane?

Search the archive of the last month for posts on git-svnimport...
IIRC there was one typo fix and one thread about a problem with it
that ended in the question wether it should be removed.

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

^ permalink raw reply

* Re: [PATCH] improve delta long block matching with big files
From: Martin Koegler @ 2007-05-26 15:19 UTC (permalink / raw)
  To: Nicolas Pitre; +Cc: Junio C Hamano, git

Nicolas Pitre wrote:
> Martin Koegler noted that create_delta() performs a new hash lookup
> after every block copy encoding which are currently limited to 64KB.
> 
> In case of larger identical blocks, the next hash lookup would normally
> point to the next 64KB block in the reference buffer and multiple block
> copy operations will be consecutively encoded.
> 
> It is however possible that the reference buffer be sparsely indexed if
> hash buckets have been trimmed down in create_delta_index() when hashing
> of the reference buffer isn't well balanced.  In that case the hash
> lookup following a block copy might fail to match anything and the fact
> that the reference buffer still matches beyond the previous 64KB block
> will be missed.
> 
> Let's rework the code so that buffer comparison isn't bounded to 64KB
> anymore.  The match size should be as large as possible up front and
> only then should multiple block copy be encoded to cover it all.
> Also, fewer hash lookups will be performed in the end.
> 
> According to Martin, this patch should reduce his 92MB pack down to 75MB
> with the dataset he has.
> 
> Tests performed on the Linux kernel repo show a slightly smaller pack and
> a slightly faster repack.
>
Acked-by: Martin Koegler <mkoegler@auto.tuwien.ac.at>
> Signed-off-by: Nicolas Pitre <nico@cam.org>
---

The patch results in a 75 MB pack file for my repository and is
faster:

Total 6452 (delta 4581), reused 1522 (delta 0)
10073.11user 5200.33system 4:14:36elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1371504760minor)pagefaults 0swaps

mfg Martin Kögler

^ permalink raw reply

* Re: Problems importing SVN repo via git-svnimport
From: Art Haas @ 2007-05-26 15:10 UTC (permalink / raw)
  To: Frank Lichtenheld; +Cc: git
In-Reply-To: <20070526130643.GA2674@planck.djpig.de>

On Sat, May 26, 2007 at 03:06:43PM +0200, Frank Lichtenheld wrote:
> On Sat, May 26, 2007 at 07:55:53AM -0500, Art Haas wrote:
> > I'm happy to dive into the perl code in svnimport and make changes to
> > help debug this problem, if there is actually a problem and not user
> > error on my part. Any help from 'svnimport' experts will be greatly
> > appreciated. The public repo listed above has the same contents as my
> > local repo, so feel free to poke around it to see how things are
> > structured.
> 
> You might want to try git-svn instead. It should support everything you
> can do with git-svnimport and much more. Plus it is actively maintained
> which I wouldn't say about git-svnimport.

Hi.

The git-svn docs say that it serves as a bi-directionaly gateway
between git and svn, but that the main repo is a Subversion repository.
My goal with the PythonCAD repo is to convert it so that the main
repository is in git, and then I'm going to try to use git-svn to
push my changes into the now-secondary Subversion repo. This way, I can
make a distributed git repo available to developers wanting to
contribute to PythonCAD, plus keep the developers currently using
Subversion from having to make any changes if they either don't want to
or are unable to (i.e. windows developers/users).

Is git-svnimport maintenence on the wane?

Art Haas

Offtopic P.S.: Anyone know of a git mirror of the GCC repo, or do people
use git-svn to access that code via git?
-- 
Man once surrendering his reason, has no remaining guard against absurdities
the most monstrous, and like a ship without rudder, is the sport of every wind.

-Thomas Jefferson to James Smith, 1822

^ permalink raw reply

* Re: [PATCH] Add git-submodule command
From: Lars Hjemli @ 2007-05-26 14:48 UTC (permalink / raw)
  To: Simon Hausmann; +Cc: git
In-Reply-To: <200705261638.00302.simon@lst.de>

On 5/26/07, Simon Hausmann <simon@lst.de> wrote:
> On Saturday 26 May 2007 15:56:40 Lars Hjemli wrote:
> > +             test -e "$path"
> > +             die "A file already exist at path '$path'"
>
> Isn't there a && missing at the end of the test line? :)
>

Hmph. There's always something slipping through, thanks for noticing.
I will make the testscript now....

-- 
larsh

^ permalink raw reply

* Re: [PATCH] Add git-submodule command
From: Simon Hausmann @ 2007-05-26 14:37 UTC (permalink / raw)
  To: Lars Hjemli; +Cc: git
In-Reply-To: <11801878003599-git-send-email-hjemli@gmail.com>

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

On Saturday 26 May 2007 15:56:40 Lars Hjemli wrote:
[...]
> +modules_init()
> +{
> +	git ls-files --stage -- "$@" | grep -e '^160000 ' |
> +	while read mode sha1 stage path
> +	do
> +		# Skip submodule paths that already contain a .git directory.
> +		# This will also trigger if $path is a symlink to a git
> +		# repository
> +		test -d "$path"/.git && continue
> +
> +		# If there already is a directory at the submodule path,
> +		# expect it to be empty (since that is the default checkout
> +		# action) and try to remove it.
> +		# Note: if $path is a symlink to a directory the test will
> +		# succeed but the rmdir will fail. We might want to fix this.
> +		if test -d "$path"
> +		then
> +			rmdir "$path" 2>/dev/null ||
> +			die "Directory '$path' exist, but is neither empty "
> +			    "nor a git repository"
> +		fi
> +
> +		test -e "$path"
> +		die "A file already exist at path '$path'"

Isn't there a && missing at the end of the test line? :)


Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply

* [PATCH] Add git-submodule command
From: Lars Hjemli @ 2007-05-26 13:56 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git, Johannes Schindelin, Linus Torvalds
In-Reply-To: <7v1wh4ped4.fsf@assigned-by-dhcp.cox.net>

This command can be used to initialize, update and inspect submodules. It
uses a .gitmodules file, readable by git-config, in the top level directory
of the 'superproject' to specify a mapping between submodule paths and
repository url.

Example .gitmodules layout:

[module "git"]
	url = git://git.kernel.org/pub/scm/git/git.git

With this entry in .gitmodules (and a commit reference in the index entry for
the path "git"), the command 'git submodule init' will clone the repository
at kernel.org into the directory "git".

Known issues
============
There is currently no way to override the url found in the .gitmodules file,
except by manually creating the subproject repository. The place to fix this
in the script has a rather long comment about a possible plan.

Funny paths will be quoted in the output from git-ls-files, but git-submodule
does not attempt to unquote (or even detect the presence of) such paths.

Signed-off-by: Lars Hjemli <hjemli@gmail.com>
---

So here is another attempt on initial submodule porcelain, still trying to
keep it simple. A proper test-script hasn't materialized yet, sorry about
that.

 .gitignore                      |    1 +
 Documentation/cmd-list.perl     |    1 +
 Documentation/git-submodule.txt |   65 +++++++++++++
 Makefile                        |    2 +-
 git-submodule.sh                |  195 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 263 insertions(+), 1 deletions(-)
 create mode 100644 Documentation/git-submodule.txt
 create mode 100755 git-submodule.sh

diff --git a/.gitignore b/.gitignore
index 4dc0c39..8fc4923 100644
--- a/.gitignore
+++ b/.gitignore
@@ -126,6 +126,7 @@ git-ssh-push
 git-ssh-upload
 git-status
 git-stripspace
+git-submodule
 git-svn
 git-svnimport
 git-symbolic-ref
diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
index 443802a..51dc6b4 100755
--- a/Documentation/cmd-list.perl
+++ b/Documentation/cmd-list.perl
@@ -180,6 +180,7 @@ git-ssh-fetch                           synchingrepositories
 git-ssh-upload                          synchingrepositories
 git-status                              mainporcelain
 git-stripspace                          purehelpers
+git-submodule                           mainporcelain
 git-svn                                 foreignscminterface
 git-svnimport                           foreignscminterface
 git-symbolic-ref                        plumbingmanipulators
diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt
new file mode 100644
index 0000000..02932a7
--- /dev/null
+++ b/Documentation/git-submodule.txt
@@ -0,0 +1,65 @@
+git-submodule(1)
+================
+
+NAME
+----
+git-submodule - Initialize, update or inspect submodules
+
+
+SYNOPSIS
+--------
+'git-submodule' [--quiet] [--cached] [status|init|update] [--] [<path>...]
+
+
+COMMANDS
+--------
+status::
+	Show the status of the submodules. This will print the SHA-1 of the
+	currently checked out commit for each submodule, along with the
+	submodule path and the output of gitlink:git-describe[1] for the
+	SHA-1. Each SHA-1 will be prefixed with "-" if the submodule is not
+	initialized and "+" if the currently checked out submodule commit
+	does not match the SHA-1 found in the index of the containing
+	repository. This command is the default command for git-submodule.
+
+init::
+	Initialize the submodules, i.e. clone the git repositories specified
+	in the .gitmodules file and checkout the submodule commits specified
+	in the index of the containing repository. This will make the
+	submodules HEAD be detached.
+
+update::
+	Update the initialized submodules, i.e. checkout the submodule commits
+	specified in the index of the containing repository. This will make
+	the submodules HEAD be detached.
+
+
+OPTIONS
+-------
+-q, --quiet::
+	Only print error messages.
+
+--cached::
+	Display the SHA-1 stored in the index, not the SHA-1 of the currently
+	checked out submodule commit. This option is only valid for the
+	status command.
+
+<path>::
+	Path to submodule(s). When specified this will restrict the command
+	to only operate on the submodules found at the specified paths.
+
+FILES
+-----
+When cloning submodules, a .gitmodules file in the top-level directory
+of the containing repository is used to find the url of each submodule.
+This file should be formatted in the same way as $GIR_DIR/config. The key
+to each submodule url is "module.$path.url".
+
+
+AUTHOR
+------
+Written by Lars Hjemli <hjemli@gmail.com>
+
+GIT
+---
+Part of the gitlink:git[7] suite
diff --git a/Makefile b/Makefile
index 29243c6..5cf2169 100644
--- a/Makefile
+++ b/Makefile
@@ -209,7 +209,7 @@ SCRIPT_SH = \
 	git-applymbox.sh git-applypatch.sh git-am.sh \
 	git-merge.sh git-merge-stupid.sh git-merge-octopus.sh \
 	git-merge-resolve.sh git-merge-ours.sh \
-	git-lost-found.sh git-quiltimport.sh
+	git-lost-found.sh git-quiltimport.sh git-submodule.sh
 
 SCRIPT_PERL = \
 	git-add--interactive.perl \
diff --git a/git-submodule.sh b/git-submodule.sh
new file mode 100755
index 0000000..1601bca
--- /dev/null
+++ b/git-submodule.sh
@@ -0,0 +1,195 @@
+#!/bin/sh
+#
+# git-submodules.sh: init, update or list git submodules
+#
+# Copyright (c) 2007 Lars Hjemli
+
+USAGE='[--quiet] [--cached] [status|init|update] [--] [<path>...]'
+. git-sh-setup
+require_work_tree
+
+init=
+update=
+status=
+quiet=
+cached=
+
+#
+# print stuff on stdout unless -q was specified
+#
+say()
+{
+	if test -z "$quiet"
+	then
+		echo "$@"
+	fi
+}
+
+#
+# Run clone + checkout on missing submodules
+#
+# $@ = requested paths (default to all)
+#
+modules_init()
+{
+	git ls-files --stage -- "$@" | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		# Skip submodule paths that already contain a .git directory.
+		# This will also trigger if $path is a symlink to a git
+		# repository
+		test -d "$path"/.git && continue
+
+		# If there already is a directory at the submodule path,
+		# expect it to be empty (since that is the default checkout
+		# action) and try to remove it.
+		# Note: if $path is a symlink to a directory the test will
+		# succeed but the rmdir will fail. We might want to fix this.
+		if test -d "$path"
+		then
+			rmdir "$path" 2>/dev/null ||
+			die "Directory '$path' exist, but is neither empty "
+			    "nor a git repository"
+		fi
+
+		test -e "$path"
+		die "A file already exist at path '$path'"
+
+		url=$(GIT_CONFIG=.gitmodules git-config module."$path".url)
+		test -z "$url" &&
+		die "No url found for submodule '$path' in .gitmodules"
+
+		# MAYBE FIXME: this would be the place to check GIT_CONFIG
+		# for a preferred url for this submodule, possibly like this:
+		#
+		# modname=$(GIT_CONFIG=.gitmodules git-config module."$path".name)
+		# alturl=$(git-config module."$modname".url)
+		#
+		# This would let the versioned .gitmodules file use the submodule
+		# path as key, while the unversioned GIT_CONFIG would use the
+		# logical modulename (if present) as key. But this would need
+		# another fallback mechanism if the module wasn't named.
+
+		git-clone -n "$url" "$path" ||
+		die "Clone of submodule '$path' failed"
+
+		(unset GIT_DIR && cd "$path" && git-checkout -q "$sha1") ||
+		die "Checkout of submodule '$path' failed"
+
+		say "Submodule '$path' initialized"
+	done
+}
+
+#
+# Checkout correct revision of each initialized submodule
+#
+# $@ = requested paths (default to all)
+#
+modules_update()
+{
+	git ls-files --stage -- "$@" | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path"/.git
+		then
+			# Only mention uninitialized submodules when its
+			# path have been specified
+			test "$#" != "0" &&
+			say "Submodule '$path' not initialized"
+			continue;
+		fi
+		subsha1=$(unset GIT_DIR && cd "$path" &&
+			git-rev-parse --verify HEAD) ||
+		die "Unable to find current revision of submodule '$path'"
+
+		if test "$subsha1" != "$sha1"
+		then
+			(unset GIT_DIR && cd "$path" && git-fetch &&
+				git-checkout -q "$sha1") ||
+			die "Unable to checkout '$sha1' in submodule '$path'"
+
+			say "Submodule '$path': checked out '$sha1'"
+		fi
+	done
+}
+
+#
+# List all registered submodules, prefixed with:
+#  - submodule not initialized
+#  + different revision checked out
+#
+# If --cached was specified the revision in the index will be printed
+# instead of the currently checked out revision.
+#
+# $@ = requested paths (default to all)
+#
+modules_list()
+{
+	git ls-files --stage -- "$@" | grep -e '^160000 ' |
+	while read mode sha1 stage path
+	do
+		if ! test -d "$path"/.git
+		then
+			say "-$sha1 $path"
+			continue;
+		fi
+		revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+		if git diff-files --quiet -- "$path"
+		then
+			say " $sha1 $path ($revname)"
+		else
+			if test -z "$cached"
+			then
+				sha1=$(unset GIT_DIR && cd "$path" && git-rev-parse --verify HEAD)
+				revname=$(unset GIT_DIR && cd "$path" && git-describe $sha1)
+			fi
+			say "+$sha1 $path ($revname)"
+		fi
+	done
+}
+
+while case "$#" in 0) break ;; esac
+do
+	case "$1" in
+	init)
+		init=1
+		;;
+	update)
+		update=1
+		;;
+	status)
+		status=1
+		;;
+	-q|--quiet)
+		quiet=1
+		;;
+	--cached)
+		cached=1
+		;;
+	--)
+		break
+		;;
+	-*)
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+	shift
+done
+
+case "$init,$update,$status,$cached" in
+1,,,)
+	modules_init "$@"
+	;;
+,1,,)
+	modules_update "$@"
+	;;
+,,*,*)
+	modules_list "$@"
+	;;
+*)
+	usage
+	;;
+esac
-- 
1.5.2.74.ga303-dirty

^ permalink raw reply related

* Re: Problems importing SVN repo via git-svnimport
From: Frank Lichtenheld @ 2007-05-26 13:06 UTC (permalink / raw)
  To: Art Haas; +Cc: git
In-Reply-To: <20070526125553.GC10324@artsapartment.org>

On Sat, May 26, 2007 at 07:55:53AM -0500, Art Haas wrote:
> I'm happy to dive into the perl code in svnimport and make changes to
> help debug this problem, if there is actually a problem and not user
> error on my part. Any help from 'svnimport' experts will be greatly
> appreciated. The public repo listed above has the same contents as my
> local repo, so feel free to poke around it to see how things are
> structured.

You might want to try git-svn instead. It should support everything you
can do with git-svnimport and much more. Plus it is actively maintained
which I wouldn't say about git-svnimport.

Gruesse,
-- 
Frank Lichtenheld <frank@lichtenheld.de>
www: http://www.djpig.de/

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox