* Re: [PATCH] Add git-submodule command
From: Johannes Schindelin @ 2007-05-25 18:41 UTC (permalink / raw)
To: Lars Hjemli, Junio C Hamano; +Cc: git
In-Reply-To: <11801165433267-git-send-email-hjemli@gmail.com>
Hi,
On Fri, 25 May 2007, Lars Hjemli wrote:
> Btw: testing this quickly becomes tedious, so I'll try to make a proper
> testscript later tonight.
Very good.
> +'git-submodule' [--init | --update | --cached] [--quiet] [--] [<path>...]
I did not realize this earlier, but we seem to have more and more programs
where actions are specified without "--", i.e. "git-svn fetch", or
"git-bundle create".
I actually like that, to separate actions from options. Hmm?
> +-i, --init::
> + Initialize the specified submodules, i.e. clone the git repository
> + specified in .gitmodules and checkout the sha1 specified in the
> + index.
How about "Initialize the submodules...", and then another sentence "If
you do not want to initialize all submodules, you can specify the subset
to initialize"?
> +-u, --update::
> + Update the specified submodules, i.e. checkout the sha1 specified
> + in the index
The full stop is missing here. And again, I would add another sentence
"Submodules which have not been initialized are not touched by this
operation."
> +FILES
> +-----
> +When cloning submodules, a .gitmodules file in the top-level directory
> +of the containing work-tree is examined for the url of each submodule.
> +The url is the value of the key module.$path.url.
IIRC Junio talked about a name for overriding. But I think it would be
even better to to override by mapping the URLs from .gitmodules to the
locally-wanted URLs.
Junio?
> +When updating submodules, the same .gitmodules file is examined for a key
> +named 'module.$path.branch'. If found, and if the named branch is currently
> +at the same revision as the commit-id in the containing repositories index,
> +the specified branch will be checked out in the submodule. If not found, or
> +if the branch isn't currently positioned at the wanted revision, a checkout
> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
A very good description, and I think this is the only method to checkout
the submodule which makes sense. (Just maybe default the value of
module.<path>.branch to "master"?)
> +++ b/git-submodule.sh
> @@ -0,0 +1,178 @@
> +#!/bin/sh
> +#
> +# git-submodules.sh: init, update or list git submodules
> +#
> +# Copyright (c) 2007 Lars Hjemli
> +
> +USAGE='[--init | --update | --cached] [--quiet] [--] [<path>...]'
> +. git-sh-setup
> +require_work_tree
Maybe
test -f "$GIT_DIR"/.gitmodules || die "Not a superproject"
Hmm?
> + rmdir "$path" 2>/dev/null ||
Just out of curiousity: is rmdir portable? I always used "rm -r"...
> +case "$init,$update,$cached" in
> +1,,)
> + modules_init $@
> + ;;
:-)
Now I run out of comments...
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Enhance unpack-objects for extracting large objects
From: Junio C Hamano @ 2007-05-25 19:22 UTC (permalink / raw)
To: Dana How; +Cc: Git Mailing List
In-Reply-To: <46569C37.5000201@gmail.com>
Dana How <danahow@gmail.com> writes:
> Nicolas Pitre wrote:
>> I wouldn't mind a _separate_ tool that would load a pack index,
>> determine object sizes from it, and then extract big objects to write
>> them as loose objects ...
>
> Below we add two new options to git-unpack-objects:
>
> --min-blob-size=<n>:: Unpacking is only done for objects
> larger than or equal to n kB (uncompressed size by Junio).
Elsewhere you wanted to use --max-* and that was counted in megs;
isn't using kilo here and meg there inconsistent?
> --force:: Loose objects will be created even if they
> already exist in the repository packed. This is an option
> I've wanted before for other reasons.
... but if they already exist in the repository as loose
objects, do not replace it.
Usually we do not overwrite existing loose objects and it is one
of the security measure --- if you have an object already, that
cannot be touched by somebody who maliciously creats a hash
colliding loose object and tries to inject it into your
repository via unpack-objects. It's good that you kept this
behaviour intact.
> This passes the tests in "t" but has not yet been used on my large repos.
> Based on "next" but should apply to "master" as well.
>
> Signed-off-by: Dana L. How <danahow@gmail.com>
> ---
> Documentation/git-unpack-objects.txt | 17 +++++++++++++----
> builtin-unpack-objects.c | 20 ++++++++++++++++++--
> cache.h | 2 ++
> sha1_file.c | 11 +++++++++--
> 4 files changed, 42 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/git-unpack-objects.txt b/Documentation/git-unpack-objects.txt
> index ff6184b..4513d8d 100644
> --- a/Documentation/git-unpack-objects.txt
> +++ b/Documentation/git-unpack-objects.txt
> ...
> @@ -17,9 +17,10 @@ Read a packed archive (.pack) from the standard input, expanding
> the objects contained within and writing them into the repository in
> "loose" (one object per file) format.
>
> -Objects that already exist in the repository will *not* be unpacked
> -from the pack-file. Therefore, nothing will be unpacked if you use
> -this command on a pack-file that exists within the target repository.
> +By default, objects that already exist in the repository will *not*
> +be unpacked from the pack-file. Therefore, nothing will be unpacked
> +if you use this command on a pack-file that exists within the target
> +repository, unless you specify --force.
I would want to add:
If an object already exists unpacked in the repository,
it will not be replaced with the copy from the pack,
with or without `--force`.
> diff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c
> index a6ff62f..a42bf0d 100644
> --- a/builtin-unpack-objects.c
> +++ b/builtin-unpack-objects.c
> @@ -10,13 +10,16 @@
> #include "progress.h"
>
> static int dry_run, quiet, recover, has_errors;
> -static const char unpack_usage[] = "git-unpack-objects [-n] [-q] [-r] < pack-file";
> +static const char unpack_usage[] =
> +"git-unpack-objects [-n] [-q] [-r] [--force] [--min-blob-size=N] < pack-file";
Maybe we would want to call it '-f' for consistency. Another
possibility is the other way around, giving others a longer
synonyms, like --quiet, but this command is plumbing and I do
not think long options matters that much, so my preference is to
do '-f' not '--force'.
> @@ -131,7 +134,9 @@ static void added_object(unsigned nr, enum object_type type,
> static void write_object(unsigned nr, enum object_type type,
> void *buf, unsigned long size)
> {
> - if (write_sha1_file(buf, size, typename(type), obj_list[nr].sha1) < 0)
> + int force2 = size < min_blob_size ? -1 : force;
> + if (write_sha1_file_maybe(buf, size, typename(type),
> + force2, obj_list[nr].sha1) < 0)
> die("failed to write object");
> added_object(nr, type, buf, size);
> }
Without --min-blob-size option, min_blob_size is initialized to
0u and force2 always gets the value of force. With the option,
blobs smaller than the threshold gets -1 and otherwise the value
of force.
"write_sha1_file_maybe()" can take 0, 1, or -1 as its fourth
parameter. The reader is left puzzled what the distinction
among these three and decides to read on to figure it out before
complaining too much about the code, but no matter what it does,
doesn't the above logic already feel wrong?
* You already have the size here, so if min_blob_size is set
and the size is larger, you do not even have to call
write_sha1_file() at all.
* If you do so, write_sha1_file_maybe()'s additional parameter
can be "skip the check to see if we have one packed".
> diff --git a/cache.h b/cache.h
> index ec85d93..d0c3030 100644
> --- a/cache.h
> +++ b/cache.h
> @@ -343,6 +343,8 @@ extern int sha1_object_info(const unsigned char *, unsigned long *);
> extern void * read_sha1_file(const unsigned char *sha1, enum object_type *type, unsigned long *size);
> extern int hash_sha1_file(const void *buf, unsigned long len, const char *type, unsigned char *sha1);
> extern int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *return_sha1);
> +extern int write_sha1_file_maybe(void *buf, unsigned long len, const char *type,
> + int ignore, unsigned char *return_sha1);
> extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned char *);
>
> extern int check_sha1_signature(const unsigned char *sha1, void *buf, unsigned long size, const char *type);
... and it says "ignore". The reader is still puzzled and reads on...
> diff --git a/sha1_file.c b/sha1_file.c
> index 12d2ef2..68b8db8 100644
> --- a/sha1_file.c
> +++ b/sha1_file.c
> @@ -1979,7 +1979,8 @@ int hash_sha1_file(const void *buf, unsigned long len, const char *type,
> return 0;
> }
>
> -int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *returnsha1)
> +int write_sha1_file_maybe(void *buf, unsigned long len, const char *type,
> + int ignore, unsigned char *returnsha1)
> {
> int size, ret;
> unsigned char *compressed;
> @@ -1997,7 +1998,7 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha
> filename = sha1_file_name(sha1);
> if (returnsha1)
> hashcpy(returnsha1, sha1);
> - if (has_sha1_file(sha1))
> + if (ignore < 0 || !ignore && has_sha1_file(sha1))
> return 0;
> fd = open(filename, O_RDONLY);
> if (fd >= 0) {
So "ignore" means:
negative: never write it out, even if it does not exist.
zero: do not write it out if it is available (in pack,
or loose, either local or alternate), do
write it out otherwise; it is the same
as the current behaviour of write_sha1_file().
positive: always write it out.
That does not sound like "ignore".
My suggestion would be:
> static void write_object(unsigned nr, enum object_type type,
> void *buf, unsigned long size)
> {
if (!min_blob_size || size < min_blob_size) {
if (write_sha1_file_maybe(buf, size, typename(type),
force, obj_list[nr].sha1) < 0)
die("failed to write object");
}
}
added_object(nr, type, buf, size);
> }
And then.
int write_sha1_file_maybe(void *buf, unsigned long len, const char *type,
int make_loose, unsigned char *returnsha1)
{
...
> filename = sha1_file_name(sha1);
> if (returnsha1)
> hashcpy(returnsha1, sha1);
if (!make_loose && has_sha1_file(sha1))
> return 0;
> fd = open(filename, O_RDONLY);
> if (fd >= 0) {
^ permalink raw reply
* Re: [PATCH] Add git-submodule command
From: Junio C Hamano @ 2007-05-25 19:31 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Lars Hjemli, git
In-Reply-To: <Pine.LNX.4.64.0705251924280.4648@racer.site>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> On Fri, 25 May 2007, Lars Hjemli wrote:
>
>> Btw: testing this quickly becomes tedious, so I'll try to make a proper
>> testscript later tonight.
>
> Very good.
>
>> +'git-submodule' [--init | --update | --cached] [--quiet] [--] [<path>...]
>
> I did not realize this earlier, but we seem to have more and more programs
> where actions are specified without "--", i.e. "git-svn fetch", or
> "git-bundle create".
>
> I actually like that, to separate actions from options. Hmm?
I think it is a sensible thing to do for this kind of "wrapper
of different functionalities related to one area".
>> +-i, --init::
>> + Initialize the specified submodules, i.e. clone the git repository
>> + specified in .gitmodules and checkout the sha1 specified in the
>> + index.
>
> How about "Initialize the submodules...", and then another sentence "If
> you do not want to initialize all submodules, you can specify the subset
> to initialize"?
>> +FILES
>> +-----
>> +When cloning submodules, a .gitmodules file in the top-level directory
>> +of the containing work-tree is examined for the url of each submodule.
>> +The url is the value of the key module.$path.url.
>
> IIRC Junio talked about a name for overriding. But I think it would be
> even better to to override by mapping the URLs from .gitmodules to the
> locally-wanted URLs.
>
> Junio?
I really do not want that (mis)conception that .gitmodules
specify the default and .git/config the override. I really
think we should use the .git/config as _the_ only authority to
get URL, but keyed with the three-level scheme, with URL in
.gitmodules used _solely_ as a hint when setting up the URL in
the .git/config file.
cf. $gmane/47502, 47548, 47621
>> +When updating submodules, the same .gitmodules file is examined for a key
>> +named 'module.$path.branch'. If found, and if the named branch is currently
>> +at the same revision as the commit-id in the containing repositories index,
>> +the specified branch will be checked out in the submodule. If not found, or
>> +if the branch isn't currently positioned at the wanted revision, a checkout
>> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
>
> A very good description, and I think this is the only method to checkout
> the submodule which makes sense. (Just maybe default the value of
> module.<path>.branch to "master"?)
I suspect leaving the HEAD always detached if the superproject
tree names a concrete commit object name would be less confusing
and consistent. When the name of the commit object in the
superproject tree and/or index is 0{40}, it would be a good
extension to use "whatever commit that happens to be at the tip
of this branch" taken from the .gitmodules file.
^ permalink raw reply
* Re: [PATCH] Enhance unpack-objects for extracting large objects
From: Dana How @ 2007-05-25 19:49 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Git Mailing List, danahow
In-Reply-To: <7vsl9kr9mz.fsf@assigned-by-dhcp.cox.net>
On 5/25/07, Junio C Hamano <junkio@cox.net> wrote:
> Dana How <danahow@gmail.com> writes:
> > Nicolas Pitre wrote:
> >> I wouldn't mind a _separate_ tool that would load a pack index,
> >> determine object sizes from it, and then extract big objects to write
> >> them as loose objects ...
> >
> > Below we add two new options to git-unpack-objects:
> >
> > --min-blob-size=<n>:: Unpacking is only done for objects
> > larger than or equal to n kB (uncompressed size by Junio).
>
> Elsewhere you wanted to use --max-* and that was counted in megs;
> isn't using kilo here and meg there inconsistent?
For git-repack --max-pack-size=N I used MB to be consistent
with git-fast-import. I think it makes sense to use MB everywhere
we talk about packfile size.
For the old degunking patch's -max-blob-size=N , and this patch's
-min-blob-size=N , I was using KB to describe blob size.
It looked like I needed finer granularity, at least for experiments.
I think if we always use MB for packfile sizes, and KB
for blob sizes, we should be OK.
> > --force:: Loose objects will be created even if they
> > already exist in the repository packed. This is an option
> > I've wanted before for other reasons.
>
> ... but if they already exist in the repository as loose
> objects, do not replace it.
>
> Usually we do not overwrite existing loose objects and it is one
> of the security measure --- if you have an object already, that
> cannot be touched by somebody who maliciously creats a hash
> colliding loose object and tries to inject it into your
> repository via unpack-objects. It's good that you kept this
> behaviour intact.
I agree.
I'll add the "... but" part to the corrected patch's commit msg.
> > -Objects that already exist in the repository will *not* be unpacked
> > -from the pack-file. Therefore, nothing will be unpacked if you use
> > -this command on a pack-file that exists within the target repository.
> > +By default, objects that already exist in the repository will *not*
> > +be unpacked from the pack-file. Therefore, nothing will be unpacked
> > +if you use this command on a pack-file that exists within the target
> > +repository, unless you specify --force.
> I would want to add:
> If an object already exists unpacked in the repository,
> it will not be replaced with the copy from the pack,
> with or without `--force`.
OK, that's clearer.
> > -static const char unpack_usage[] = "git-unpack-objects [-n] [-q] [-r] < pack-file";
> > +static const char unpack_usage[] =
> > +"git-unpack-objects [-n] [-q] [-r] [--force] [--min-blob-size=N] < pack-file";
>
> Maybe we would want to call it '-f' for consistency. Another
> possibility is the other way around, giving others a longer
> synonyms, like --quiet, but this command is plumbing and I do
> not think long options matters that much, so my preference is to
> do '-f' not '--force'.
I picked the longer one only because I didn't view it as frequently used.
But it will be more used than min-blob-size. I'll change it to -f.
> * You already have the size here, so if min_blob_size is set
> and the size is larger, you do not even have to call
> write_sha1_file() at all.
The way I read the code, it looks like unpack-objects needs
the last argument always to be initialized with the SHA-1 computed
from the object contents. Therefore I always need to call
write_sha1_file(), even if I don't want it to write anything.
> So "ignore" means:
> negative: never write it out, even if it does not exist.
> zero: do not write it out if it is available (in pack,
> or loose, either local or alternate), do
> write it out otherwise; it is the same
> as the current behaviour of write_sha1_file().
> positive: always write it out.
> That does not sound like "ignore".
I agree "ignore" is confusing;
I will change it to "when", which is more consistent with the
_maybe prefix on the function name.
> My suggestion would be:
I like this suggestion, but since I need to call the function
to get the SHA-1, I don't think I can follow it.
I'll send you an updated patch in a moment.
Thanks,
--
Dana L. How danahow@gmail.com +1 650 804 5991 cell
^ permalink raw reply
* Re: [PATCH] Enhance unpack-objects for extracting large objects
From: Nicolas Pitre @ 2007-05-25 19:52 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Dana How, Git Mailing List
In-Reply-To: <7vsl9kr9mz.fsf@assigned-by-dhcp.cox.net>
On Fri, 25 May 2007, Junio C Hamano wrote:
> Maybe we would want to call it '-f' for consistency. Another
> possibility is the other way around, giving others a longer
> synonyms, like --quiet, but this command is plumbing and I do
> not think long options matters that much, so my preference is to
> do '-f' not '--force'.
OTOH, I like to have long options for weird or obscur parameters. Their
action is less likely to be presumed by casual inspection of a script
using them. I don't feel strongly about it either ways though.
> > @@ -131,7 +134,9 @@ static void added_object(unsigned nr, enum object_type type,
> > static void write_object(unsigned nr, enum object_type type,
> > void *buf, unsigned long size)
> > {
> > - if (write_sha1_file(buf, size, typename(type), obj_list[nr].sha1) < 0)
> > + int force2 = size < min_blob_size ? -1 : force;
> > + if (write_sha1_file_maybe(buf, size, typename(type),
> > + force2, obj_list[nr].sha1) < 0)
> > die("failed to write object");
> > added_object(nr, type, buf, size);
> > }
>
> Without --min-blob-size option, min_blob_size is initialized to
> 0u and force2 always gets the value of force. With the option,
> blobs smaller than the threshold gets -1 and otherwise the value
> of force.
>
> "write_sha1_file_maybe()" can take 0, 1, or -1 as its fourth
> parameter. The reader is left puzzled what the distinction
> among these three and decides to read on to figure it out before
> complaining too much about the code, but no matter what it does,
> doesn't the above logic already feel wrong?
>
> * You already have the size here, so if min_blob_size is set
> and the size is larger, you do not even have to call
> write_sha1_file() at all.
You still do to get the object's SHA1.
Nicolas
^ permalink raw reply
* Re: [PATCH] Add git-submodule command
From: Linus Torvalds @ 2007-05-25 19:57 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Lars Hjemli, Junio C Hamano, git
In-Reply-To: <Pine.LNX.4.64.0705251924280.4648@racer.site>
On Fri, 25 May 2007, Johannes Schindelin wrote:
>
> I did not realize this earlier, but we seem to have more and more programs
> where actions are specified without "--", i.e. "git-svn fetch", or
> "git-bundle create".
Hey, don't forget "git bisect", the granddaddy of them all.
> I actually like that, to separate actions from options. Hmm?
I agree. If something effectively always takes a separate command, it's
not an option, it's a subcommand.
Linus
^ permalink raw reply
* Re: [PATCH] Enhance unpack-objects for extracting large objects
From: Junio C Hamano @ 2007-05-25 19:59 UTC (permalink / raw)
To: Dana How; +Cc: Nicolas Pitre, Git Mailing List
In-Reply-To: <56b7f5510705251249u74b754f1y4f8cafd5f5c35f19@mail.gmail.com>
"Dana How" <danahow@gmail.com> writes:
>> * You already have the size here, so if min_blob_size is set
>> and the size is larger, you do not even have to call
>> write_sha1_file() at all.
> The way I read the code, it looks like unpack-objects needs
> the last argument always to be initialized with the SHA-1 computed
> from the object contents. Therefore I always need to call
> write_sha1_file(), even if I don't want it to write anything.
Ah, that is what I missed.
There is a separate function to only hash, named (surprisingly)
"hash_sha1_file(). Maybe you can teach the caller's "don't
write it out" codepath to call it.
^ permalink raw reply
* Re: [PATCH] Enhance unpack-objects for extracting large objects
From: Nicolas Pitre @ 2007-05-25 20:05 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Dana How, Git Mailing List
In-Reply-To: <7viragr7xb.fsf@assigned-by-dhcp.cox.net>
On Fri, 25 May 2007, Junio C Hamano wrote:
> "Dana How" <danahow@gmail.com> writes:
>
> >> * You already have the size here, so if min_blob_size is set
> >> and the size is larger, you do not even have to call
> >> write_sha1_file() at all.
> > The way I read the code, it looks like unpack-objects needs
> > the last argument always to be initialized with the SHA-1 computed
> > from the object contents. Therefore I always need to call
> > write_sha1_file(), even if I don't want it to write anything.
>
> Ah, that is what I missed.
>
> There is a separate function to only hash, named (surprisingly)
> "hash_sha1_file(). Maybe you can teach the caller's "don't
> write it out" codepath to call it.
That would be clearer indeed.
Nicolas
^ permalink raw reply
* Re: [PATCH] Add git-submodule command
From: Johannes Schindelin @ 2007-05-25 20:28 UTC (permalink / raw)
To: Linus Torvalds; +Cc: Lars Hjemli, Junio C Hamano, git
In-Reply-To: <alpine.LFD.0.98.0705251256250.26602@woody.linux-foundation.org>
Hi,
On Fri, 25 May 2007, Linus Torvalds wrote:
> On Fri, 25 May 2007, Johannes Schindelin wrote:
> >
> > I did not realize this earlier, but we seem to have more and more programs
> > where actions are specified without "--", i.e. "git-svn fetch", or
> > "git-bundle create".
>
> Hey, don't forget "git bisect", the granddaddy of them all.
Oh, sorry! How could I? Respect the elders. ;-)
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Add git-submodule command
From: Lars Hjemli @ 2007-05-25 20:29 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Schindelin, git, Linus Torvalds
In-Reply-To: <7vodk8r97s.fsf@assigned-by-dhcp.cox.net>
On 5/25/07, Junio C Hamano <junkio@cox.net> wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Fri, 25 May 2007, Lars Hjemli wrote:
> >
> >> Btw: testing this quickly becomes tedious, so I'll try to make a proper
> >> testscript later tonight.
> >
> > Very good.
> >
> >> +'git-submodule' [--init | --update | --cached] [--quiet] [--] [<path>...]
> >
> > I did not realize this earlier, but we seem to have more and more programs
> > where actions are specified without "--", i.e. "git-svn fetch", or
> > "git-bundle create".
> >
> > I actually like that, to separate actions from options. Hmm?
>
> I think it is a sensible thing to do for this kind of "wrapper
> of different functionalities related to one area".
Ok, there seems to be general agreement on this, so how about
something like this instead:
git-submodule [--quiet] [--cached] [init | update] [--] [<path>...]
or would you rather have
git-submodule [--quiet] [--cached] <cmd> [<path>...]
with cmd being one of init, update, status?
> >> +FILES
> >> +-----
> >> +When cloning submodules, a .gitmodules file in the top-level directory
> >> +of the containing work-tree is examined for the url of each submodule.
> >> +The url is the value of the key module.$path.url.
> >
> > IIRC Junio talked about a name for overriding. But I think it would be
> > even better to to override by mapping the URLs from .gitmodules to the
> > locally-wanted URLs.
> >
> > Junio?
>
> I really do not want that (mis)conception that .gitmodules
> specify the default and .git/config the override. I really
> think we should use the .git/config as _the_ only authority to
> get URL, but keyed with the three-level scheme, with URL in
> .gitmodules used _solely_ as a hint when setting up the URL in
> the .git/config file.
>
> cf. $gmane/47502, 47548, 47621
>
I've read these articles, but I think much of the concerns about
trusting the url supplied by upstream goes away when the submodule
clone/checkout isn't an integrated part of the superproject
clone/checkout. Besides, if you trust your upstream enough to clone
their repository (the superproject), why wouldn't you trust the data
(.gitmodules) in that very repository?
Still, a way to locally override the suggested url is probably needed.
This can be easily achieved by either yours or Linus' suggestion
about 'url rewriting'. And if clone/checkout doesn't touch the
submodules at all, the downstream user can look at/override the
.gitmodules file in his local tree before deciding to do the submodule
clone or checkout. I don't think there is any need for an interactive
tool here.
Another possibility is simply doing the submodule clone/checkout by
hand (i.e. do 'git clone preferred-url path', don't do 'git submodule
init path').
> >> +When updating submodules, the same .gitmodules file is examined for a key
> >> +named 'module.$path.branch'. If found, and if the named branch is currently
> >> +at the same revision as the commit-id in the containing repositories index,
> >> +the specified branch will be checked out in the submodule. If not found, or
> >> +if the branch isn't currently positioned at the wanted revision, a checkout
> >> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
> >
> > A very good description, and I think this is the only method to checkout
> > the submodule which makes sense. (Just maybe default the value of
> > module.<path>.branch to "master"?)
>
> I suspect leaving the HEAD always detached if the superproject
> tree names a concrete commit object name would be less confusing
> and consistent.
And easier to explain :)
>When the name of the commit object in the
> superproject tree and/or index is 0{40}, it would be a good
> extension to use "whatever commit that happens to be at the tip
> of this branch" taken from the .gitmodules file.
>
I really can't imagine what kind of superproject would have such a
setup. Why would this be needed?
--
larsh
^ permalink raw reply
* Re: [PATCH] Add git-submodule command
From: Johannes Schindelin @ 2007-05-25 20:30 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Lars Hjemli, git
In-Reply-To: <7vodk8r97s.fsf@assigned-by-dhcp.cox.net>
Hi,
On Fri, 25 May 2007, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > On Fri, 25 May 2007, Lars Hjemli wrote:
> >
> >> +FILES
> >> +-----
> >> +When cloning submodules, a .gitmodules file in the top-level directory
> >> +of the containing work-tree is examined for the url of each submodule.
> >> +The url is the value of the key module.$path.url.
> >
> > IIRC Junio talked about a name for overriding. But I think it would be
> > even better to to override by mapping the URLs from .gitmodules to the
> > locally-wanted URLs.
> >
> > Junio?
>
> I really do not want that (mis)conception that .gitmodules
> specify the default and .git/config the override. I really
> think we should use the .git/config as _the_ only authority to
> get URL, but keyed with the three-level scheme, with URL in
> .gitmodules used _solely_ as a hint when setting up the URL in
> the .git/config file.
>
> cf. $gmane/47502, 47548, 47621
Yes, I skipped over the first two, and concentrated on the third. If
.gitmodules are used to initialize what is in .git/config, I agree.
However, could we have that as a separate commit, to keep things simple
enough so that I can understand them?
> >> +When updating submodules, the same .gitmodules file is examined for a key
> >> +named 'module.$path.branch'. If found, and if the named branch is currently
> >> +at the same revision as the commit-id in the containing repositories index,
> >> +the specified branch will be checked out in the submodule. If not found, or
> >> +if the branch isn't currently positioned at the wanted revision, a checkout
> >> +of the wanted sha1 will happen in the submodule, leaving its HEAD detached.
> >
> > A very good description, and I think this is the only method to checkout
> > the submodule which makes sense. (Just maybe default the value of
> > module.<path>.branch to "master"?)
>
> I suspect leaving the HEAD always detached if the superproject
> tree names a concrete commit object name would be less confusing
> and consistent. When the name of the commit object in the
> superproject tree and/or index is 0{40}, it would be a good
> extension to use "whatever commit that happens to be at the tip
> of this branch" taken from the .gitmodules file.
Yes, that's a much better idea.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH 07/16] git-read-tree: take --submodules option
From: Jan Hudec @ 2007-05-25 20:15 UTC (permalink / raw)
To: Martin Waitz; +Cc: Junio C Hamano, skimo, Alex Riesen, git
In-Reply-To: <20070524154833.GL5412@admingilde.org>
[-- Attachment #1: Type: text/plain, Size: 5640 bytes --]
On Thu, May 24, 2007 at 17:48:33 +0200, Martin Waitz wrote:
> On Tue, May 22, 2007 at 09:37:06PM +0200, Jan Hudec wrote:
> > > We don't have to move the entire subproject.git into the superproject,
> > > but we need to have all _referenced_ objects in the .git dir of the
> > > superproject.
> > >
> > > There are several possibilities to do so:
> > >
> > > * move the entire .git dir
> > > * move .git/objects
> > > * explicitly copy all referenced objects
> >
> > I believe we really need entire .git dir. When the superporject checks out
> > revision which does not reference that subproject, we still need to preserve
> > not only the objects of subproject, but also the refs and config.
>
> but all the other refs do not belong to the superproject.
> For those who are working on the subproject there are of course a lot
> of refs which they have to work with, but that can be dealt with
> outside of the superproject scope. The subproject is still a normal
> Git repository, after all.
> That is, you can have remote entries, branches and what not.
> But all that is not interesting in the superproject scope.
>
> So I thing moving the entire subproject.git into the superproject.git is too
> much. The superproject is only interested in the objects and in one
> ref -- the one stored inside its tree.
No, the way I mean it the subproject and superproject don't share a single
bit. The subproject.git dir is subdirectory of superproject.git, but has no
thing in common with it.
> > > I have some experimental code to configure a per-subproject directory
> > > in the superproject/.git as alternate object store for the submodule
> > > to make the last two solutions possible. Perhaps I should dig it out again
> > > and adapt it to current git.
Ah. My bad. Didn't notice this. I do NOT want to share any objects between
subproject and superproject. At least not unless the user explicitely asks
for that, which might make sense if the subproject was carved out of the
superproject.
> > > If there is a 1:1 relationship between subproject and object store then
> > > even efficient fsck and repack/prune are possible for the submodule without
> > > loosing objects.
> > > But such a 1:1 relationship is bad when you move subprojects to another
> > > location (or include the same subproject several times in different
> > > locations of the tree).
> > > Perhaps the user should be able to choose which one he wants.
> >
> > That's why there should be the extra level of indirection using .gitmodules.
> > It should map the directory name to the object store name, so you can
> > relocate the subproject.
> >
> > Including the same project several times is indeed interesting. Maybe the
> > subprojects should be "light checkouts" (I believe something like this was
> > already discussed on the list sometime). Those would be .git dirs, that would
> > only have HEAD and pointer to another .git dir with everything else.
>
> Well, even if they might share a lot of objects they might be included
> for completely different reasons and so might need to work with
> different communities (remote entries, branches, etc.).
>
> So sharing objects makes sense, sharing the rest of .git is not
> neccessary.
No, I didn't mean the subproject and superproject would share anything.
The case I talk about is if project foo has subdirs A and B and they both
contain (different revisions of) the same subproject. The .gitmodules
definition is:
[submodule "A"]
name=bar
[submodule "B"]
name=bar
In such case A/.git and B/.git can't both be symlinks to
.git/subprojects/bar.git, because they have different HEAD, but everything
else should be defined by .git/subprojects/bar.git
> > > I think it will be _very_ common to store super and subprojects in
> > > related locations. First to be independent from third-party servers
> > > while working on the superproject.
> > > Second (and I think more important) because many times there will
> > > be superproject related adaptations in the subproject. Yes they
> > > are independent, and exactly for that reason the subproject upstream
> > > maintainers may not take every change which is needed to satisfy the
> > > superproject. We _now_ see that in all Linux distributions already.
> > > So when you use superprojects to integrate several independent projects,
> > > then the superproject maintainer/administrator should really keep a
> > > clone of all subprojects handy on his site.
> >
> > Yes, repositories with distribution-specific patches will add a large class
> > of cases requiring multiple sources support.
>
> You don't really need multiple sources for it.
> The subproject contains both upstream and local changes, but I think
Upstream + local is not the interesting case. Multiple upstreams is.
> it makes sense to keep the entire object store local (the same way
> to keep all the entire history local even if you only want to add to it
> in a normal repository). Those people who work on the subproject and
> communicate with its upstream developers of course need remote entries
> and have to synchronize the subproject with upstream. But that is
> not related to the superproject at all.
>
> So yes, you have different sources but you don't need extra support
> in the subproject implementation for it.
Well, if the subproject is not auto-fetched, there's no need for extra
support. But if there is auto-fetch, it should be aware of possibility to
have multiple upstreams.
--
Jan 'Bulb' Hudec <bulb@ucw.cz>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: [PATCH 07/16] git-read-tree: take --submodules option
From: Jan Hudec @ 2007-05-25 20:35 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Martin Waitz, skimo, Alex Riesen, git
In-Reply-To: <7viraixeme.fsf@assigned-by-dhcp.cox.net>
[-- Attachment #1: Type: text/plain, Size: 5170 bytes --]
On Thu, May 24, 2007 at 11:26:01 -0700, Junio C Hamano wrote:
> Martin Waitz <tali@admingilde.org> writes:
> > On Mon, May 21, 2007 at 06:59:38PM +0200, Jan Hudec wrote:
> [...]
>
> I was hoping that we can start from an initial cut that supports
> only a superproject that had its subprojects in their places
> from its initial commit, and did not have to worry about this
> from day one, and deal with this kind of "more advanced" stuff
> incrementally. Unfortunately it's more fun to talk about more
> advanced stuff than starting with small but solid stuff.
>
> And we would need to make sure whatever we do as the "small but
> solid" initial round can later support more advanced
> arrangements later, so we would need to think about the issues
> now anyway to a certain degree.
>
> How about doing something like this, instead?
It's almost exactly what I had in mind, except much better described, so
I definitely support this. It seems that it can work.
> (1) superproject .gitmodules (in-tree) and .git/config (local
> repository) use the three-level naming in $gmane/47567.
> Namely, (1a) .gitmodules says which subdirectory has a
> checkout of what project, and names the project in
> logical/abstract terms, not with a URL (e.g. "kernel26");
> (1b) .gitmodules also associates a set of suggested URLs
> for each of the logical/abstract project name; (1c)
> .git/config records which project are of interest.
>
> (2) In superproject .git/, we would have a bare repository for
> each project used by the superproject.
>
> .git/subproject/kernel26/{objects,refs,...}
>
> This is created by making a bare clone from the upstream
> URL, decided by the user with the help from suggested URL
> described in the superproject .gitmodules.
>
> The idea is to use this repository as a long-term
> subproject state across branch switching.
>
> (3) When we need to check out a revision of superproject whose
> .gitmodules has "kernel-src/ -> kernel26", and when we
> haven't done so (perhaps we are doing an initial checkout,
> perhaps we are switching from a different revision of the
> superproject that did not have "kernel26" project at
> kernel-src/ directory), we rm -f kernel-src/ and then
> "git-clone -l -s" from the repository we keep in (2) to
> populate kernel-src/ directory.
If the "clone" could also share the refs, config and everything except HEAD,
it would make it completely (or almost so) transparent to the user. Making it
non-transparent will work well enough though and should not require any new
changes.
The problem is, that than the HEAD could get out of sync with the refs
(because they are linked from other repo), so it would have to remember both
the commit name and the symbolic ref. I would behave as symref only if the
commit name in it and the target are the same and as hard ref otherwise.
> (4) Before performing the above step (3), we need to make sure
> we are not losing anything in kernel-src/ if exists. Three
> cases plus one:
>
> (4a) The path kernel-src/ in the old checkout was not a
> subproject (either it did not exist, it was a blob, or it
> was a directory with files that are tracked as part of the
> superproject). The usual "don't lose local modification"
> rule we use try to carry local changes forward across
> branch switching, but in this case we shouldn't do so.
>
> (4b) It has the same logical/abstract project checked out;
> the commit recorded in the superproject tree may or may not
> be the same as what its HEAD points at. In this case we do
> not have to worry about swapping the git repository at
> kernel-src/ directory, although we would need to check out
> the correct revision, and worry about what to do with any
> local modification (I think the usual "don't lose local
> modification but carry them forward" rule would be Ok in
> this case).
>
> (4c) It has a different project checked out; we need to be
> careful to keep local changes, and also we need to make
> sure the local changes in this subproject repository are
> pushed back to (2). It could be that automated "git push"
> after making sure everything is committed is sufficient and
> have the user handle failure cases.
>
> (4d) This applies not just "before step (3)", but in cases
> where we need to replace a checked out subproject directory
> with something else (e.g. blob or directory that belong to
> the superproject, or noneness). We would need to make sure
> no local change is lost, and the repository is synched up
> with (2).
>
> I think an arrangement like this would solve "symlink is a bitch
> for MinGW" problem Johannes Sixt brought up today with Sven's
> RFC as well.
It would also solve (rare) case when for some reason the same subproject
should be checked out twice (different revisions).
--
Jan 'Bulb' Hudec <bulb@ucw.cz>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: git-svn and SVK mirror between two repositories
From: Steven Grimm @ 2007-05-25 20:50 UTC (permalink / raw)
To: Vinubalaji Gopal; +Cc: git
In-Reply-To: <7d8fb81e0705241004u1c52fa7aub42d3793d4bfeaa7@mail.gmail.com>
Vinubalaji Gopal wrote:
> Hi all,
> I have been trying hard to find if it is possible to mirror (or
> clone in git-svn terms) two svn repositories and update changes from
> one to other or do even more complex operations with these two
> repositories.
I don't know how svk would change things, but I did something similar
and wrote it up here:
http://thread.gmane.org/gmane.comp.version-control.git/45060
-Steve
^ permalink raw reply
* Re: [PATCH] Add git-submodule command
From: Junio C Hamano @ 2007-05-25 20:52 UTC (permalink / raw)
To: Lars Hjemli; +Cc: Junio C Hamano, Johannes Schindelin, git, Linus Torvalds
In-Reply-To: <8c5c35580705251329u33ac1462m9db35cac0c37e3a9@mail.gmail.com>
"Lars Hjemli" <hjemli@gmail.com> writes:
> On 5/25/07, Junio C Hamano <junkio@cox.net> wrote:
> ...
>> I really do not want that (mis)conception that .gitmodules
>> specify the default and .git/config the override. I really
>> think we should use the .git/config as _the_ only authority to
>> get URL, but keyed with the three-level scheme, with URL in
>> .gitmodules used _solely_ as a hint when setting up the URL in
>> the .git/config file.
>>
>> cf. $gmane/47502, 47548, 47621
>>
>
> I've read these articles, but I think much of the concerns about
> trusting the url supplied by upstream goes away when the submodule
> clone/checkout isn't an integrated part of the superproject
> clone/checkout. Besides, if you trust your upstream enough to clone
> their repository (the superproject), why wouldn't you trust the data
> (.gitmodules) in that very repository?
It's not about trusting. You would need to support the mapping
for network connectivity reasons, and you would also need to
notice and reconfirm when the suggested URL in .gitmodules
changes (perhaps because the upstream relocated from sf.net to
repo.or.cz ;-), you would need something like what I described
in order to keep track of user preference for each submodule in
.git/config anyway. If that "mapping" ends up to be ident
mapping for most people, that is fine. At least by always doing
the three-level mapping we would not have any special case in
the code, and this is not the performance critical part of the
system.
I think the response to the case when upstream repository
relocates from the ".gitmodule for default, .git/config for
override" camp would be "you asked to override in .git/config,
so it is your job to notice the change in .gitmodules and adjust
your override URL". That is a serious mistake in usability
point of view. Repository relocation would (hopefully) seldom
happen, but when it does happen, things either would break
(which is easier to diagnose and manually fix up), or things
clone fine but we reach a wrong repository (which is harder to
notice, as "fetch" may succeed -- it just would not fetch the
right commit). Being able to notice when upstream repository
relocates and to ask for confirmation when that happens would
eliminate a lot of confusion from that.
> Another possibility is simply doing the submodule clone/checkout by
> hand (i.e. do 'git clone preferred-url path', don't do 'git submodule
> init path').
But that is what this patch is trying to help the users, isn't
it? It reduces the attractiveness of this new tool greatly if
you give up there.
>>When the name of the commit object in the
>> superproject tree and/or index is 0{40}, it would be a good
>> extension to use "whatever commit that happens to be at the tip
>> of this branch" taken from the .gitmodules file.
>
> I really can't imagine what kind of superproject would have such a
> setup. Why would this be needed?
"We would work with any working version of Linux 2.6 kernel"
would be a sensible thing to say, I would think.
It's purely optional, and as you seem to agree always detaching
HEAD is easier to explain, you do not need "module.$path.branch"
at all. I just mentioned 0{40} as a possible use case for that
configuration variable.
^ permalink raw reply
* Re: [PATCH 07/16] git-read-tree: take --submodules option
From: Junio C Hamano @ 2007-05-25 21:05 UTC (permalink / raw)
To: Jan Hudec; +Cc: Martin Waitz, skimo, Alex Riesen, git
In-Reply-To: <20070525203505.GB4493@efreet.light.src>
Jan Hudec <bulb@ucw.cz> writes:
> It would also solve (rare) case when for some reason the same subproject
> should be checked out twice (different revisions).
I have a feeling that you are equating two subprojects that
happens to have the same upstream URL the same, and if that is
the case I think that is a mistake.
If you were doing an efficient cgi script that renders history
of git managed projects, binding git as its subproject, and that
system can be built with either 'maint' (i.e. 1.5.2 series) or
'master' (i.e. ultrastable WIP towards 1.5.3), even though they
both might come from git://git.kernel.org/pub/scm/git/git.git/,
I think they should be registered as two separate logical
subprojects.
The .gitmodules file might have:
[module "git-maint"]
path = git-stale/
url = git://git.kernel.org/pub/scm/git/git.git/
;; branch = maint
[module "git-master"]
path = git-stable/
url = git://git.kernel.org/pub/scm/git/git.git/
;; branch = master
and two paths (git-stale/ and git-stable/) in the superproject
tree would have commit object names from the named branches.
^ permalink raw reply
* [PATCH 1/3] run-command: optionally clear git environment
From: skimo @ 2007-05-25 21:07 UTC (permalink / raw)
To: git, Junio C Hamano; +Cc: Martin Waitz, Alex Riesen
From: Sven Verdoolaege <skimo@kotnet.org>
Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
run-command.c | 6 ++++++
run-command.h | 1 +
2 files changed, 7 insertions(+), 0 deletions(-)
diff --git a/run-command.c b/run-command.c
index 7e779d3..5c47f45 100644
--- a/run-command.c
+++ b/run-command.c
@@ -2,6 +2,10 @@
#include "run-command.h"
#include "exec_cmd.h"
+static const char* git_env_list[] = { ALTERNATE_DB_ENVIRONMENT, DB_ENVIRONMENT,
+ CONFIG_ENVIRONMENT, GIT_DIR_ENVIRONMENT,
+ GRAFT_ENVIRONMENT, INDEX_ENVIRONMENT, NULL };
+
static inline void close_pair(int fd[2])
{
close(fd[0]);
@@ -153,6 +157,8 @@ static void prepare_run_command_v_opt(struct child_process *cmd,
cmd->no_stdin = opt & RUN_COMMAND_NO_STDIN ? 1 : 0;
cmd->git_cmd = opt & RUN_GIT_CMD ? 1 : 0;
cmd->stdout_to_stderr = opt & RUN_COMMAND_STDOUT_TO_STDERR ? 1 : 0;
+ if (opt & RUN_COMMAND_CLEAR_GIT_ENV)
+ cmd->env = git_env_list;
}
int run_command_v_opt(const char **argv, int opt)
diff --git a/run-command.h b/run-command.h
index 7958eb1..a5374cc 100644
--- a/run-command.h
+++ b/run-command.h
@@ -33,6 +33,7 @@ int run_command(struct child_process *);
#define RUN_COMMAND_NO_STDIN 1
#define RUN_GIT_CMD 2 /*If this is to be git sub-command */
#define RUN_COMMAND_STDOUT_TO_STDERR 4
+#define RUN_COMMAND_CLEAR_GIT_ENV (1 << 3)
int run_command_v_opt(const char **argv, int opt);
int run_command_v_opt_cd(const char **argv, int opt, const char *dir);
--
1.5.2.838.gbeec
^ permalink raw reply related
* [PATCH 3/3] test for simple submodule checkout support
From: skimo @ 2007-05-25 21:07 UTC (permalink / raw)
To: git, Junio C Hamano; +Cc: Martin Waitz, Alex Riesen
In-Reply-To: <1180127233729-git-send-email-skimo@liacs.nl>
From: Martin Waitz <tali@admingilde.org>
Signed-off-by: Martin Waitz <tali@admingilde.org>
Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
t/t3041-subprojects-checkout.sh | 39 +++++++++++++++++++++++++++++++++++++++
1 files changed, 39 insertions(+), 0 deletions(-)
create mode 100755 t/t3041-subprojects-checkout.sh
diff --git a/t/t3041-subprojects-checkout.sh b/t/t3041-subprojects-checkout.sh
new file mode 100755
index 0000000..4b3cea9
--- /dev/null
+++ b/t/t3041-subprojects-checkout.sh
@@ -0,0 +1,39 @@
+#!/bin/sh
+
+test_description='submodule checkout'
+. ./test-lib.sh
+
+test_expect_success 'submodule creation' \
+ '(mkdir A && cd A &&
+ git init &&
+ echo 1 > a &&
+ git add a &&
+ git commit -m "create submodule" || exit $? )'
+
+test_expect_success 'Super module creation' \
+ 'git add A &&
+ git commit -m "supermodule creation" &&
+ git branch one'
+
+test_expect_success 'submodule change' \
+ '(cd A &&
+ echo 2 > a &&
+ git add a &&
+ git commit -m "create submodule" || exit $? )'
+
+test_expect_success 'supermodule change' \
+ 'git add A &&
+ git commit -m "supermodule creation"'
+
+test_expect_success 'supermodule switching branch' \
+ 'git checkout one &&
+ echo 1 > expected &&
+ git diff expected A/a'
+
+test_expect_success 'supermodule reset' \
+ 'git reset --hard master &&
+ echo 2 > expected &&
+ git diff expected A/a'
+
+
+test_done
--
1.5.2.838.gbeec
^ permalink raw reply related
* [PATCH 2/3] entry.c: checkout available submodules
From: skimo @ 2007-05-25 21:07 UTC (permalink / raw)
To: git, Junio C Hamano; +Cc: Martin Waitz, Alex Riesen
In-Reply-To: <1180127233729-git-send-email-skimo@liacs.nl>
From: Sven Verdoolaege <skimo@kotnet.org>
That is, checkout any submodule that has a valid HEAD in it.
Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
---
Makefile | 5 +++--
entry.c | 30 ++++++++++++++++++++++++++++--
submodules.c | 8 ++++++++
submodules.h | 6 ++++++
4 files changed, 45 insertions(+), 4 deletions(-)
create mode 100644 submodules.c
create mode 100644 submodules.h
diff --git a/Makefile b/Makefile
index c79a6da..6d24048 100644
--- a/Makefile
+++ b/Makefile
@@ -297,7 +297,7 @@ LIB_H = \
run-command.h strbuf.h tag.h tree.h git-compat-util.h revision.h \
tree-walk.h log-tree.h dir.h path-list.h unpack-trees.h builtin.h \
utf8.h reflog-walk.h patch-ids.h attr.h decorate.h progress.h \
- mailmap.h remote.h
+ mailmap.h remote.h submodules.h
DIFF_OBJS = \
diff.o diff-lib.o diffcore-break.o diffcore-order.o \
@@ -319,7 +319,8 @@ LIB_OBJS = \
write_or_die.o trace.o list-objects.o grep.o match-trees.o \
alloc.o merge-file.o path-list.o help.o unpack-trees.o $(DIFF_OBJS) \
color.o wt-status.o archive-zip.o archive-tar.o shallow.o utf8.o \
- convert.o attr.o decorate.o progress.o mailmap.o symlinks.o remote.o
+ convert.o attr.o decorate.o progress.o mailmap.o symlinks.o remote.o \
+ submodules.o
BUILTIN_OBJS = \
builtin-add.o \
diff --git a/entry.c b/entry.c
index ae64764..97f95c6 100644
--- a/entry.c
+++ b/entry.c
@@ -1,5 +1,7 @@
#include "cache.h"
#include "blob.h"
+#include "run-command.h"
+#include "submodules.h"
static void create_directories(const char *path, const struct checkout *state)
{
@@ -75,6 +77,31 @@ static void *read_blob_entry(struct cache_entry *ce, const char *path, unsigned
return NULL;
}
+static int checkout_submodule(struct cache_entry *ce, const char *path, const struct checkout *state)
+{
+ const char *args[10];
+ int argc;
+ int err;
+
+ if (!is_checkedout_submodule(ce->name))
+ return 0;
+
+ argc = 0;
+ args[argc++] = "checkout";
+ if (state->force)
+ args[argc++] = "-f";
+ args[argc++] = sha1_to_hex(ce->sha1);
+ args[argc] = NULL;
+
+ err = run_command_v_opt_cd(args, RUN_GIT_CMD|RUN_COMMAND_CLEAR_GIT_ENV,
+ path);
+
+ if (err)
+ return error("failed to run git-checkout in submodule '%s'", path);
+
+ return 0;
+}
+
static int write_entry(struct cache_entry *ce, char *path, const struct checkout *state, int to_tempfile)
{
int fd;
@@ -193,9 +220,8 @@ int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *t
*/
unlink(path);
if (S_ISDIR(st.st_mode)) {
- /* If it is a gitlink, leave it alone! */
if (S_ISGITLINK(ntohl(ce->ce_mode)))
- return 0;
+ return checkout_submodule(ce, path, state);
if (!state->force)
return error("%s is a directory", path);
remove_subtree(path);
diff --git a/submodules.c b/submodules.c
new file mode 100644
index 0000000..5baf90a
--- /dev/null
+++ b/submodules.c
@@ -0,0 +1,8 @@
+#include "cache.h"
+#include "refs.h"
+
+int is_checkedout_submodule(const char *path)
+{
+ unsigned char sha1[20];
+ return resolve_gitlink_ref(path, "HEAD", sha1) == 0;
+}
diff --git a/submodules.h b/submodules.h
new file mode 100644
index 0000000..099c4c3
--- /dev/null
+++ b/submodules.h
@@ -0,0 +1,6 @@
+#ifndef SUBMODULES_H
+#define SUBMODULES_H
+
+int is_checkedout_submodule(const char *path);
+
+#endif
--
1.5.2.838.gbeec
^ permalink raw reply related
* Re: [PATCH 07/16] git-read-tree: take --submodules option
From: Steven Grimm @ 2007-05-25 21:16 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jan Hudec, Martin Waitz, skimo, Alex Riesen, git
In-Reply-To: <7vwsywpqaj.fsf@assigned-by-dhcp.cox.net>
Junio C Hamano wrote:
> If you were doing an efficient cgi script that renders history
> of git managed projects, binding git as its subproject, and that
> system can be built with either 'maint' (i.e. 1.5.2 series) or
> 'master' (i.e. ultrastable WIP towards 1.5.3), even though they
> both might come from git://git.kernel.org/pub/scm/git/git.git/,
> I think they should be registered as two separate logical
> subprojects.
>
I agree strongly with this, and it's another good reason that we have to
be able to use something other than the URL as the key to look up a
subproject's repository location. If you use the URL it is impossible to
differentiate the two subprojects in this case.
-Steve
^ permalink raw reply
* Bug: git-rebase goofs up \n in commit messages
From: Szekeres Istvan @ 2007-05-25 21:11 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 1043 bytes --]
Hello,
while playing with git I found the following bug: if a commit message
contains "\n" (as a string, not as a character), git-rebase changes this
string into a literal newline character.
This is how to reproduce it:
mkdir tmp
cd tmp
git init-db
echo xx > xx.txt
git add xx.txt
git commit -m foo
echo xx >> xx.txt
git add xx.txt
git commit -m foo
git branch other 'HEAD^'
git checkout other
echo yy > yy.txt
git add yy.txt
git commit -m 'foo \\n bar'
git log 'HEAD^..' [1]
git rebase master
git log 'HEAD^..' [2]
The output of [1] is the following (correctly):
commit 694daa542b83dc1bbd6c070630f73c9a111f6e40
Author: Istvan Szekeres <szekeres@iii.hu>
Date: Fri May 25 23:09:32 2007 +0200
foo \n bar
The output of [2] is the following (wrong!):
commit 68ba80d2927d4e21c7a1d1d758f9023dbe063bde
Author: Istvan Szekeres <szekeres@iii.hu>
Date: Fri May 25 22:58:28 2007 +0200
foo
bar
....
I think this is a bug.
Best regards,
Istvan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 252 bytes --]
^ permalink raw reply
* Re: [PATCH 2/3] entry.c: checkout available submodules
From: Martin Waitz @ 2007-05-25 21:31 UTC (permalink / raw)
To: skimo; +Cc: git, Junio C Hamano, Alex Riesen
In-Reply-To: <1180127233893-git-send-email-skimo@liacs.nl>
[-- Attachment #1: Type: text/plain, Size: 828 bytes --]
hoi :)
On Fri, May 25, 2007 at 11:07:12PM +0200, skimo@liacs.nl wrote:
> create mode 100644 submodules.c
> create mode 100644 submodules.h
I think the list tends to prefer subproject over submodule.
> @@ -193,9 +220,8 @@ int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *t
> */
> unlink(path);
> if (S_ISDIR(st.st_mode)) {
> - /* If it is a gitlink, leave it alone! */
> if (S_ISGITLINK(ntohl(ce->ce_mode)))
> - return 0;
> + return checkout_submodule(ce, path, state);
> if (!state->force)
> return error("%s is a directory", path);
> remove_subtree(path);
I think the call to checkout_submodule should be moved to write_entry,
to keep it in line with the other mode types.
Aside from that I really like it :-)
--
Martin Waitz
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: GIT on MinGW problem
From: Aaron Gray @ 2007-05-25 21:37 UTC (permalink / raw)
To: git, Johannes Sixt
In-Reply-To: <4656A304.AF39A0B6@eudaptics.com>
> Han-Wen Nienhuys wrote:
>>
>> Aaron Gray escreveu:
>> > Hello,
>> >
>> > I have installed the git-1.5.1-1.mingw.exe from
>> > http://lilypond.org/git/binaries/mingw/.
>> >
>> > On typing 'git' I get a message box saying :-
>> >
>> > The procedure entry point libiconv could not be located in the
>> > dynamic link library libiconv-2.dll.
>> >
>> > I cannot seem to find libiconv-2.dll anywhere either.
>>
>> This should be fixed in
>>
>> http://lilypond.org/git/binaries/mingw/git-1.5.1-2.mingw.exe
>>
>> it should also set $PATH.
>
> I gave this some more testing and it turns out to be a well working
> toolset. Thank you very much!
>
> There were still some issues remaining. These are the ones that should
> be fixable easily:
>
> * git version reports just:
>
> git version -dirty
>
> Since git-gui parses the output of git version, but does not expect it
> to be of this format, and fails with an error message that it cannot
> parse the version.
>
> * git without an correct git subcommand should list 20 or so commands,
> but it doesn't. The list is just empty.
>
> * I personally think that the files should go into
>
> $PROGRAMFILES/Git/{bin,share,lib}
> instead of
> $PROGRAMFILES/Git/usr/{bin,share,lib}
>
> The more difficult to solve problems are:
>
> * git-gui and gitk don't work out of the box because they have the path
> to wish hardcoded. They can't be started from CMD at all. I have written
> wrappers gitk.cmd and git-gui.cmd with these 2 lines:
>
> @echo off
> start wish84 D:/MSYS/1.0/git/bin/gitk %*
>
> But as you can see, the path is still hard-coded (but it is good enough
> for me for the moment).
>
> * perl scripts like git-remote contain a hard-coded path to the
> installation directory and don't work for this reason.
Are git init and git clone working for you ?
Aaron
^ permalink raw reply
* [PATCH v2] Enhance unpack-objects for live repo and large objects
From: Dana How @ 2007-05-25 21:40 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Git Mailing List, danahow
Nicolas Pitre wrote:
> I wouldn't mind a _separate_ tool that would load a pack index,
> determine object sizes from it, and then extract big objects to write
> them as loose objects ...
Add two new options to git-unpack-objects:
--force:: Loose objects will be created even if they
already exist in the repository packed.
--min-blob-size=<n>:: Unpacking is only done for objects
larger than or equal to n kB (uncompressed size).
Passes the tests in "t" and tested on big objects.
Based on "next" but should apply to "master" as well.
Signed-off-by: Dana L. How <danahow@gmail.com>
---
Documentation/git-unpack-objects.txt | 23 +++++++++++++++++++----
builtin-unpack-objects.c | 29 +++++++++++++++++++++++++++--
cache.h | 2 ++
sha1_file.c | 16 ++++++++++++----
4 files changed, 60 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-unpack-objects.txt b/Documentation/git-unpack-objects.txt
index ff6184b..3df2641 100644
--- a/Documentation/git-unpack-objects.txt
+++ b/Documentation/git-unpack-objects.txt
@@ -8,7 +8,7 @@ git-unpack-objects - Unpack objects from a packed archive
SYNOPSIS
--------
-'git-unpack-objects' [-n] [-q] [-r] <pack-file
+'git-unpack-objects' [-n] [-q] [-r] [-f] [--min-blob-size=N] <pack-file
DESCRIPTION
@@ -17,9 +17,12 @@ Read a packed archive (.pack) from the standard input, expanding
the objects contained within and writing them into the repository in
"loose" (one object per file) format.
-Objects that already exist in the repository will *not* be unpacked
-from the pack-file. Therefore, nothing will be unpacked if you use
-this command on a pack-file that exists within the target repository.
+By default, objects that already exist in the repository will *not*
+be unpacked from the pack-file. Therefore, nothing will be unpacked
+if you use this command on a pack-file that exists within the target
+repository, unless you specify -f. If an object already exists
+unpacked in the repository, it will not be replaced with the copy
+from the pack, with or without -f.
Please see the `git-repack` documentation for options to generate
new packs and replace existing ones.
@@ -40,6 +43,18 @@ OPTIONS
and make the best effort to recover as many objects as
possible.
+-f::
+ Allow loose objects to be created in the same repository that
+ contains the packfile.
+
+--min-blob-size=<n>::
+ Smallest loose object to create, expressed in kB.
+ Blobs smaller than this will not be unpacked. Default is 0.
+ If you specify this option with a deltified source packfile,
+ the source packfile should reside in the current repository
+ so delta bases too small to unpack are still accessible, and
+ therefore -f will be needed for anything to be written.
+
Author
------
diff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c
index a6ff62f..b8ee7b5 100644
--- a/builtin-unpack-objects.c
+++ b/builtin-unpack-objects.c
@@ -10,13 +10,16 @@
#include "progress.h"
static int dry_run, quiet, recover, has_errors;
-static const char unpack_usage[] = "git-unpack-objects [-n] [-q] [-r] < pack-file";
+static const char unpack_usage[] =
+"git-unpack-objects [-n] [-q] [-r] [-f] [--min-blob-size=N] < pack-file";
/* We always read in 4kB chunks. */
static unsigned char buffer[4096];
static unsigned int offset, len;
static off_t consumed_bytes;
static SHA_CTX ctx;
+static int force = 0;
+uint32_t min_blob_size;
/*
* Make sure at least "min" bytes are available in the buffer, and
@@ -131,7 +134,18 @@ static void added_object(unsigned nr, enum object_type type,
static void write_object(unsigned nr, enum object_type type,
void *buf, unsigned long size)
{
- if (write_sha1_file(buf, size, typename(type), obj_list[nr].sha1) < 0)
+ /*
+ * We never need to write it when it's too small.
+ * Otherwise, without -f, we write it only when
+ * it does not exist in the repository in any form.
+ * Finally, with -f, we write it only when it does
+ * not exist in the local repository as a loose object.
+ * In all cases we fill in obj_list[nr].sha1 .
+ */
+ if (size < min_blob_size)
+ hash_sha1_file(buf, size, typename(type), obj_list[nr].sha1);
+ else if (write_sha1_file_maybe(buf, size, typename(type),
+ force, obj_list[nr].sha1) < 0)
die("failed to write object");
added_object(nr, type, buf, size);
}
@@ -361,6 +375,17 @@ int cmd_unpack_objects(int argc, const char **argv, const char *prefix)
recover = 1;
continue;
}
+ if (!strcmp(arg, "-f")) {
+ force = 1;
+ continue;
+ }
+ if (!prefixcmp(arg, "--min-blob-size=")) {
+ char *end;
+ min_blob_size = strtoul(arg+16, &end, 0) * 1024;
+ if (!arg[16] || *end)
+ usage(unpack_usage);
+ continue;
+ }
if (!prefixcmp(arg, "--pack_header=")) {
struct pack_header *hdr;
char *c;
diff --git a/cache.h b/cache.h
index ec85d93..4994d03 100644
--- a/cache.h
+++ b/cache.h
@@ -343,6 +343,8 @@ extern int sha1_object_info(const unsigned char *, unsigned long *);
extern void * read_sha1_file(const unsigned char *sha1, enum object_type *type, unsigned long *size);
extern int hash_sha1_file(const void *buf, unsigned long len, const char *type, unsigned char *sha1);
extern int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *return_sha1);
+extern int write_sha1_file_maybe(void *buf, unsigned long len, const char *type,
+ int dup_ok, unsigned char *return_sha1);
extern int pretend_sha1_file(void *, unsigned long, enum object_type, unsigned char *);
extern int check_sha1_signature(const unsigned char *sha1, void *buf, unsigned long size, const char *type);
diff --git a/sha1_file.c b/sha1_file.c
index 12d2ef2..e4c3288 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1979,7 +1979,8 @@ int hash_sha1_file(const void *buf, unsigned long len, const char *type,
return 0;
}
-int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned char *returnsha1)
+int write_sha1_file_maybe(void *buf, unsigned long len, const char *type,
+ int dup_ok, unsigned char *returnsha1)
{
int size, ret;
unsigned char *compressed;
@@ -1990,14 +1991,15 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha
char hdr[32];
int fd, hdrlen;
- /* Normally if we have it in the pack then we do not bother writing
- * it out into .git/objects/??/?{38} file.
+ /* Normally if in a pack (or any where else) then we do not write
+ * it out into .git/objects/??/?{38} file, but with dup_ok != 0
+ * we only avoid over-writing a loose blob in the local repo.
*/
write_sha1_file_prepare(buf, len, type, sha1, hdr, &hdrlen);
filename = sha1_file_name(sha1);
if (returnsha1)
hashcpy(returnsha1, sha1);
- if (has_sha1_file(sha1))
+ if (!dup_ok && has_sha1_file(sha1))
return 0;
fd = open(filename, O_RDONLY);
if (fd >= 0) {
@@ -2062,6 +2064,12 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha
return move_temp_to_file(tmpfile, filename);
}
+int write_sha1_file(void *buf, unsigned long len, const char *type,
+ unsigned char *returnsha1)
+{
+ return write_sha1_file_maybe(buf, len, type, 0, returnsha1);
+}
+
/*
* We need to unpack and recompress the object for writing
* it out to a different file.
--
1.5.2.762.gd8c6-dirty
^ permalink raw reply related
* Re: [PATCH 2/3] entry.c: checkout available submodules
From: Sven Verdoolaege @ 2007-05-25 21:42 UTC (permalink / raw)
To: Martin Waitz; +Cc: git, Junio C Hamano, Alex Riesen
In-Reply-To: <20070525213103.GA8361@admingilde.org>
On Fri, May 25, 2007 at 11:31:03PM +0200, Martin Waitz wrote:
> I think the list tends to prefer subproject over submodule.
Does it? It seems that everyone writing code is use submodule
instead of subproject. Either way, I don't really care.
> > @@ -193,9 +220,8 @@ int checkout_entry(struct cache_entry *ce, const struct checkout *state, char *t
> > */
> > unlink(path);
> > if (S_ISDIR(st.st_mode)) {
> > - /* If it is a gitlink, leave it alone! */
> > if (S_ISGITLINK(ntohl(ce->ce_mode)))
> > - return 0;
> > + return checkout_submodule(ce, path, state);
> > if (!state->force)
> > return error("%s is a directory", path);
> > remove_subtree(path);
>
> I think the call to checkout_submodule should be moved to write_entry,
> to keep it in line with the other mode types.
Well, like your patch, this only deals with cases where the submodule
is already available. In write_entry you could potentially clone
submodules based on some criteria, but I'm not doing this just yet
since some people apparently prefer to get these things in pieces.
Also, it seems that some people would like this to be a step
that is separated from git-checkout (see Lars' patch).
skimo
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox