question about git-submodule

question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 398 bytes --]

hi,

even after reading the manpage, i'm not sure about what git submodules
are for :)

i mean, first i thought that after configuring a submodule under libfoo,
then a git pull will update libfoo's repo under libfoo, too. or
something like that. but in fact even after reasing the source, i'm not
sure about what is git submodule update is for :s

sorry for the lame question :)

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Sven Verdoolaege]

On Fri, Jul 13, 2007 at 11:46:30PM +0200, VMiklos wrote:
> i mean, first i thought that after configuring a submodule under libfoo,
> then a git pull will update libfoo's repo under libfoo, too. or
> something like that.

Unless I've missed something, it doesn't.
Some of use would like git to do just that (at least Alex Riesen,
Martin Waitz and myself have sent in patches to that effect),
but we haven't been able to convince Junio yet.

skimo

Re: question about git-submodule

[Alex Riesen]

Sven Verdoolaege, Sun, Jul 15, 2007 10:39:59 +0200:
> On Fri, Jul 13, 2007 at 11:46:30PM +0200, VMiklos wrote:
> > i mean, first i thought that after configuring a submodule under libfoo,
> > then a git pull will update libfoo's repo under libfoo, too. or
> > something like that.
> 
> Unless I've missed something, it doesn't.
> Some of use would like git to do just that (at least Alex Riesen,
> Martin Waitz and myself have sent in patches to that effect),
> but we haven't been able to convince Junio yet.

Count me out. Junio convinced me instead and having tried the
subprojects I find it really convenient: I can choose when and what
should be updated and I can see what _can_ be updated, iff I decide
to. Subprojects defined in such a loosely way are more flexible then
having git-pull fetch subprojects by default. Sometimes I even want be
_sure_ the subprojects are completely untouched (I have some critical
parts in them).

Re: question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 471 bytes --]

Hello,

Na Sun, Jul 15, 2007 at 12:47:12PM +0200, Alex Riesen <raa.lkml@gmail.com> pisal(a):
> to. Subprojects defined in such a loosely way are more flexible then
> having git-pull fetch subprojects by default. Sometimes I even want be
> _sure_ the subprojects are completely untouched (I have some critical
> parts in them).

Okay, but where can you overwrite that default? It would be nice to have
a config variable for this or something like that.

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Alex Riesen]

VMiklos, Sun, Jul 15, 2007 12:54:50 +0200:
> Na Sun, Jul 15, 2007 at 12:47:12PM +0200, Alex Riesen <raa.lkml@gmail.com> pisal(a):
> > to. Subprojects defined in such a loosely way are more flexible then
> > having git-pull fetch subprojects by default. Sometimes I even want be
> > _sure_ the subprojects are completely untouched (I have some critical
> > parts in them).
> 
> Okay, but where can you overwrite that default? It would be nice to have
> a config variable for this or something like that.

It is not a default in a sense where it control the behaviour of a git
command. git pull just does not do anything to subprojects and you use
a separate command (git-submodule update, I believe) to update them.

That said, I never used the git-submodule, nor did I have an urge to.
As I said, I find it convenient enough as it is with git-add and
git-diff, checking out the subproject as needed. Have to be careful
about "git commit -a" (which I almost never use anyway) and haven't
had conflicts yet.

Re: question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 662 bytes --]

Hello,

Na Sun, Jul 15, 2007 at 03:50:57PM +0200, Alex Riesen <raa.lkml@gmail.com> pisal(a):
> It is not a default in a sense where it control the behaviour of a git
> command. git pull just does not do anything to subprojects and you use
> a separate command (git-submodule update, I believe) to update them.
> 
> That said, I never used the git-submodule, nor did I have an urge to.

that's what i asked originally :) i believed that one is supposed to use
git-submodule update to pull each submodule, but it does not seems to
do so. my question is still about if it is possible to pull each
submodule using a single command

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Sven Verdoolaege]

On Sun, Jul 15, 2007 at 03:54:53PM +0200, VMiklos wrote:
> that's what i asked originally :) i believed that one is supposed to use
> git-submodule update to pull each submodule, but it does not seems to
> do so. my question is still about if it is possible to pull each
> submodule using a single command

Maybe you should explain in a bit more detail what you are trying
to do.

skimo

Re: question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

Hello,

Na Sun, Jul 15, 2007 at 04:02:44PM +0200, Sven Verdoolaege <skimo@kotnet.org> pisal(a):
> On Sun, Jul 15, 2007 at 03:54:53PM +0200, VMiklos wrote:
> > that's what i asked originally :) i believed that one is supposed to use
> > git-submodule update to pull each submodule, but it does not seems to
> > do so. my question is still about if it is possible to pull each
> > submodule using a single command
> 
> Maybe you should explain in a bit more detail what you are trying
> to do.

ok :) so i have a repo and i add a submodule in the libfoo dir. when i
add it, git will clone the repo, that's fine. it would be nice to have a
command (i thought git submodule update would do it for me) which would
pull in the repo _and_ would pull in the libfoo dir, too

currently if you have 10 submodules, you need to updated each of them
manually, and having a feature to all of them at once would be nice (of
course it's fine if this is not the default when using git pull in the
"main" repo)

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Sven Verdoolaege]

On Sun, Jul 15, 2007 at 04:26:24PM +0200, VMiklos wrote:
> currently if you have 10 submodules, you need to updated each of them
> manually, and having a feature to all of them at once would be nice (of
> course it's fine if this is not the default when using git pull in the
> "main" repo)

Isn't that what "git submodule update" does ?

skimo

Re: question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

Hello,

Na Sun, Jul 15, 2007 at 04:48:35PM +0200, Sven Verdoolaege <skimo@kotnet.org> pisal(a):
> Isn't that what "git submodule update" does ?

i can't get it to work, but here is a log:

http://frugalware.org/~vmiklos/logs/git-submodule.log

at the end of it, git submodule update does not pull anything, while
there are changes both in the main and in the libfoo repo, too

have i missed something?

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Sven Verdoolaege]

On Sun, Jul 15, 2007 at 05:05:40PM +0200, VMiklos wrote:
> Hello,
> 
> Na Sun, Jul 15, 2007 at 04:48:35PM +0200, Sven Verdoolaege <skimo@kotnet.org> pisal(a):
> > Isn't that what "git submodule update" does ?
> 
> i can't get it to work, but here is a log:
> 

> vmiklos@vmobile:~/git/test$ ls
> vmiklos@vmobile:~/git/test$ mkdir server
> vmiklos@vmobile:~/git/test$ cd server
> vmiklos@vmobile:~/git/test/server$ mkdir main
> vmiklos@vmobile:~/git/test/server$ cd main
> vmiklos@vmobile:~/git/test/server/main$ git init
> Initialized empty Git repository in .git/
> vmiklos@vmobile:~/git/test/server/main$ echo "foo" > main.c
> vmiklos@vmobile:~/git/test/server/main$ git add main.c
> vmiklos@vmobile:~/git/test/server/main$ git commit -m "import main"
> Created initial commit 1ceae8f: import main
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 main.c
> vmiklos@vmobile:~/git/test/server/main$ cd ..
> vmiklos@vmobile:~/git/test/server$ mkdir libfoo
> vmiklos@vmobile:~/git/test/server$ cd libfoo
> vmiklos@vmobile:~/git/test/server/libfoo$ git init
> Initialized empty Git repository in .git/
> vmiklos@vmobile:~/git/test/server/libfoo$ echo "foo" > libfoo.c
> vmiklos@vmobile:~/git/test/server/libfoo$ git add libfoo.c
> vmiklos@vmobile:~/git/test/server/libfoo$ git commit -m "import libfoo"
> Created initial commit 5979d36: import libfoo
>  1 files changed, 1 insertions(+), 0 deletions(-)
>  create mode 100644 libfoo.c
> vmiklos@vmobile:~/git/test/server/libfoo$ cd ../..
> vmiklos@vmobile:~/git/test$ mkdir client
> vmiklos@vmobile:~/git/test$ cd client
> vmiklos@vmobile:~/git/test/client$ git clone ../server/main
> Initialized empty Git repository in /home/vmiklos/git/test/client/main/.git/
> remote: Generating pack...
> Done counting 3 objects.
> Deltifying 3 objects...
>  100% (3/3) done
> Total 3 remote: (delta 0), reused 0 (delta 0)
> Indexing 3 objects...
>  100% (3/3) done
> 
> vmiklos@vmobile:~/git/test/client$ cd main
> vmiklos@vmobile:~/git/test/client/main$ ls
> main.c
> vmiklos@vmobile:~/git/test/client/main$ git submodule add ../../server/libfoo/ libfoo

Here you add a submodule in the copy of the superproject in client.

> Initialized empty Git repository in /home/vmiklos/git/test/client/main/libfoo/.git/
> remote: Generating pack...
> remote: Done counting 3 objects.
> remote: Deltifying 3 objects...
>  100% (3/3) done
> Total 3 (delta 0), reused 0 (delta 0)
> Indexing 3 objects...
>  100% (3/3) done
> 
> vmiklos@vmobile:~/git/test/client/main$ git submodule init
> Submodule 'libfoo' (/home/vmiklos/git/test/server/libfoo/.git) registered for path 'libfoo'
> vmiklos@vmobile:~/git/test/client/main$ git commit -a -m "registered libfoo submodule"
> Created commit 2c65310: registered libfoo submodule
>  2 files changed, 4 insertions(+), 0 deletions(-)
>  create mode 100644 .gitmodules
>  create mode 160000 libfoo
> vmiklos@vmobile:~/git/test/client/main$ cd ../../server/main
> vmiklos@vmobile:~/git/test/server/main$ echo bar >> main.c
> vmiklos@vmobile:~/git/test/server/main$ git add git commit -m "bar in main"
> vmiklos@vmobile:~/git/test/server/main$ git add main.c
> vmiklos@vmobile:~/git/test/server/main$ git commit -m "bar in main"
> Created commit 1c26399: bar in main
>  1 files changed, 1 insertions(+), 0 deletions(-)
> vmiklos@vmobile:~/git/test/server/main$ cd ../libfoo
> vmiklos@vmobile:~/git/test/server/libfoo$ echo bar >> libfoo.c
> vmiklos@vmobile:~/git/test/server/libfoo$ git add libfoo.c
> vmiklos@vmobile:~/git/test/server/libfoo$ git commit -m "bar in libfoo"
> Created commit 07ceadb: bar in libfoo
>  1 files changed, 1 insertions(+), 0 deletions(-)
> vmiklos@vmobile:~/git/test/server/libfoo$ cd ../../client/main
> vmiklos@vmobile:~/git/test/client/main$ git submodule update
> 
> at the end of it, git submodule update does not pull anything, while
> there are changes both in the main and in the libfoo repo, too

The last time you told the superproject about any changes
in the subprojects was when you added the subproject (in client),
so as far as the superproject is concerned nothing happened
in any submodule.  The submodules don't even exist in server.

skimo

Re: question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 2264 bytes --]

Hello,

Na Sun, Jul 15, 2007 at 05:21:01PM +0200, Sven Verdoolaege <skimo@kotnet.org> pisal(a):
> > vmiklos@vmobile:~/git/test/client/main$ git submodule add ../../server/libfoo/ libfoo
> 
> Here you add a submodule in the copy of the superproject in client.

correct

> > vmiklos@vmobile:~/git/test/client/main$ git submodule update
> > 
> > at the end of it, git submodule update does not pull anything, while
> > there are changes both in the main and in the libfoo repo, too
> 
> The last time you told the superproject about any changes
> in the subprojects was when you added the subproject (in client),
> so as far as the superproject is concerned nothing happened
> in any submodule.  The submodules don't even exist in server.

okay, let me continue the example, then:

vmiklos@vmobile:~/git/test/client2$ git clone ../client/main
Initialized empty Git repository in /home/vmiklos/git/test/client2/main/.git/
remote: Generating pack...
remote: Done counting 11 objects.
remote: Deltifying 11 objects...
remote:  100% (11/11) done
Indexing 11 objects...
 100% (11/11) done
Resolving 2 deltas...
 100% (2/2) done
remote: Total 11 (delta 2), reused 3 (delta 0)
vmiklos@vmobile:~/git/test/client2$ ls
main/
vmiklos@vmobile:~/git/test/client2$ cd ../client/main/
vmiklos@vmobile:~/git/test/client/main$ git pull
Already up-to-date.
vmiklos@vmobile:~/git/test/client/main$ cd libfoo
vmiklos@vmobile:~/git/test/client/main/libfoo$ git pull
remote: Generating pack...
remote: Done counting 5 objects.
Result has 3 objects.
remote: Deltifying 3 objects...
 100% (3/3) done
Total 3 (delta 0), reused 0 (delta remote: 0)
Unpacking 3 objects...
 100% (3/3) done
* refs/remotes/origin/master: fast forward to branch 'master' of /home/vmiklos/git/test/server/libfoo/
  old..new: 5979d36..07ceadb
Updating 5979d36..07ceadb
Fast forward
 libfoo.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)
vmiklos@vmobile:~/git/test/client/main/libfoo$ cd ../../../client2/main
vmiklos@vmobile:~/git/test/client2/main$ git submodule update

so the submodule exists now on the server and there would be something,
to update in the submodule, but git submodule update seem not to do
anything

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Sven Verdoolaege]

On Sun, Jul 15, 2007 at 05:40:28PM +0200, VMiklos wrote:
> vmiklos@vmobile:~/git/test/client2/main$ git submodule update

You should do a git submodule init first.

skimo

Re: question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

Hello,

Na Sun, Jul 15, 2007 at 11:45:30PM +0200, Sven Verdoolaege <skimo@kotnet.org> pisal(a):
> You should do a git submodule init first.

nice, now it works :)

then my question would be if the following workflow is possible or not:

1) you told me how to do the following:

a) somebody updates libfoo

b) a dev of main pulls libfoo, commits something like:

diff --git a/libfoo b/libfoo
index 07ceadb..459b61f 160000
--- a/libfoo
+++ b/libfoo
@@ -1 +1 @@
-Subproject commit 07ceadb17ca73636b0d96cb91688baecb1763d9b
+Subproject commit 459b61fa4bba78a6e6542b8fcc0fd8f320eafa8c

c) users have to "git pull; git submodule update" and they'll be happy

2) is it possible to do this without step b)?

i mean adding something like: "Subproject commit HEAD" or "Subproject
commit mybranch"

thanks,
- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Sven Verdoolaege]

On Mon, Jul 16, 2007 at 12:29:48AM +0200, VMiklos wrote:
> 2) is it possible to do this without step b)?
> 
> i mean adding something like: "Subproject commit HEAD" or "Subproject
> commit mybranch"

No.  You have to specify the specific revision of the subproject
that is included in any revision of the superproject.

Imagine someone making an incompatible change to a subproject.
You do not want to get this change in your copy of the superproject
unless the superproject has been adapted to this change.

skimo

Re: question about git-submodule

[VMiklos]

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

Hello,

Na Mon, Jul 16, 2007 at 12:39:55PM +0200, Sven Verdoolaege <skimo@kotnet.org> pisal(a):
> No.  You have to specify the specific revision of the subproject
> that is included in any revision of the superproject.
> 
> Imagine someone making an incompatible change to a subproject.
> You do not want to get this change in your copy of the superproject
> unless the superproject has been adapted to this change.

i don't say that the ability to stick to a given commit is bad, but imho
there are scenarios where the ability to stick to a branch is better

ok, i know, probably i should send a patch if i want this, then it can
be discussed if this is useful or not

anyway thanks for all your answers, now i at least know that what i
wanted is currently not supported :)

- VMiklos

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: question about git-submodule

[Sven Verdoolaege]

On Sun, Jul 15, 2007 at 12:47:12PM +0200, Alex Riesen wrote:
> Count me out. Junio convinced me instead and having tried the
> subprojects I find it really convenient: I can choose when and what
> should be updated and I can see what _can_ be updated, iff I decide
> to. Subprojects defined in such a loosely way are more flexible then
> having git-pull fetch subprojects by default.

I agree that fetching should probably be left as a separate operation,
but if you have all the data, then I find it very inconvenient that
every time you switch to a different commit you have to update
all the subprojects separately too.
Did you change your mind about this part too?

> Sometimes I even want be
> _sure_ the subprojects are completely untouched (I have some critical
> parts in them).

The update in the superproject would fail if the subproject is dirty
(just as with files.)

skimo

Re: question about git-submodule

[Alex Riesen]

Sven Verdoolaege, Sun, Jul 15, 2007 13:51:48 +0200:
> On Sun, Jul 15, 2007 at 12:47:12PM +0200, Alex Riesen wrote:
> > Count me out. Junio convinced me instead and having tried the
> > subprojects I find it really convenient: I can choose when and what
> > should be updated and I can see what _can_ be updated, iff I decide
> > to. Subprojects defined in such a loosely way are more flexible then
> > having git-pull fetch subprojects by default.
> 
> I agree that fetching should probably be left as a separate operation,
> but if you have all the data, then I find it very inconvenient that
> every time you switch to a different commit you have to update
> all the subprojects separately too.

I found I do _not_ need to do it every time I switch to a different
commit.

> Did you change your mind about this part too?

Yep. I have less to transfer (and in fact, the subproject I mentioned
is kind of heavy and has a lot of binary stuff in it).

> > Sometimes I even want be
> > _sure_ the subprojects are completely untouched (I have some critical
> > parts in them).
> 
> The update in the superproject would fail if the subproject is dirty
> (just as with files.)

Haven't noticed this yet. Merge ignores subprojects. What do you mean?

Re: question about git-submodule

[Sven Verdoolaege]

On Sun, Jul 15, 2007 at 03:42:12PM +0200, Alex Riesen wrote:
> Sven Verdoolaege, Sun, Jul 15, 2007 13:51:48 +0200:
> > I agree that fetching should probably be left as a separate operation,
> > but if you have all the data, then I find it very inconvenient that
> > every time you switch to a different commit you have to update
> > all the subprojects separately too.
> 
> I found I do _not_ need to do it every time I switch to a different
> commit.

Ok.  Then I guess we have different needs and I should only
do it if some config option is set if I ever try to resend
the patch.

> > The update in the superproject would fail if the subproject is dirty
> > (just as with files.)
> 
> Haven't noticed this yet. Merge ignores subprojects. What do you mean?
> 

I mean that if such a feature would be implemented, then it wouldn't
go ahead with the update if some of the subprojects couldn't be updated.

skimo

Re: question about git-submodule

[Eran Tromer]

On 2007-07-15 06:47, Alex Riesen wrote:
> Sven Verdoolaege, Sun, Jul 15, 2007 10:39:59 +0200:
>> On Fri, Jul 13, 2007 at 11:46:30PM +0200, VMiklos wrote:
>>> i mean, first i thought that after configuring a submodule under libfoo,
>>> then a git pull will update libfoo's repo under libfoo, too. or
>>> something like that.
>> Unless I've missed something, it doesn't.
>> Some of use would like git to do just that (at least Alex Riesen,
>> Martin Waitz and myself have sent in patches to that effect),
>> but we haven't been able to convince Junio yet.
> 
> Count me out. Junio convinced me instead and having tried the
> subprojects I find it really convenient: I can choose when and what
> should be updated and I can see what _can_ be updated, iff I decide
> to. Subprojects defined in such a loosely way are more flexible then
> having git-pull fetch subprojects by default. Sometimes I even want be
> _sure_ the subprojects are completely untouched (I have some critical
> parts in them).

All of the above also applies to normal files, not just submodules. So
by this logic, maybe "git pull" and "git checkout" should by default
update just the index, and require a manual "git file update" to update
the files in the working tree?

Leaving the working tree (including submodules) in an inconsistent state
right after a pull or checkout is a very strange thing to do. It means,
for example, that even simple switching between branches doesn't work in
a superproject. Consider this example (tested on next aac97aacbccd):

--------------------------------------------------
Create the subproject repo:

$ mkdir sub-to-be
$ cd sub-to-be
$ git init
$ echo AAA > foo
$ git add foo
$ git commit -m sub
$ cd ..

Create the superproject repo and clone subproject into it:

$ mkdir super
$ cd super
$ git init
$ git submodule add ../sub-to-be sub
$ git commit -m super

Update the submodule:

$ cd sub
$ echo BBB > foo
$ git add foo
$ git commit -m sub
$ cd ..

Update the supermodule:

$ git add sub
$ git commit -m super

Try to switch between versions, and fail miserably:

$ git checkout -b old HEAD^
M       sub
Switched to a new branch "old"
$ git checkout master
fatal: Entry 'sub' not uptodate. Cannot merge.
--------------------------------------------------

BTW, is there any easy way to create a submodule directly inside the
supermodule's working tree, instead of cloning it as in the above example?

  Eran

Re: question about git-submodule

[Sven Verdoolaege]

On Fri, Aug 03, 2007 at 07:01:38PM -0400, Eran Tromer wrote:
> $ mkdir super
> $ cd super
> $ git init
> $ git submodule add ../sub-to-be sub

You probably don't want to do this as this URL
probably won't work for anyone who wants to clone
your superproject.

> BTW, is there any easy way to create a submodule directly inside the
> supermodule's working tree, instead of cloning it as in the above example?

Do you want to use the submodule porcelain or just the plumbing?
If you simply want to add a submodule locally, "git add" will do that for
you just fine.  It's only when you want to make it easier for people who
clone your superproject to get updates of the submodules that you
should set up the information required by "git submodule" in .gitmodules
(see end of module_add in git-submodule.sh).

Note that the "url" can be set directly in your git config
(without specifying it in .gitmodules), but then anyone who clones
from you will have to modify his config "manually" too.
For some reason, the "path" element is required, even if it is equal
to the submodule name.

skimo

Re: question about git-submodule

[Eran Tromer]

On 2007-08-05 10:59, Sven Verdoolaege wrote:
>> BTW, is there any easy way to create a submodule directly inside the
>> supermodule's working tree, instead of cloning it as in the above example?
> 
> Do you want to use the submodule porcelain or just the plumbing?
> If you simply want to add a submodule locally, "git add" will do that for
> you just fine.  It's only when you want to make it easier for people who
> clone your superproject to get updates of the submodules that you
> should set up the information required by "git submodule" in .gitmodules
> (see end of module_add in git-submodule.sh).

If you just do "git add sub" (where sub is an existing repo subdir) then
any subsequent "git submodule update" predictably complains:
  No submodule mapping found in .gitmodules for path 'sub'

And if you wire up .gitmodules manually, then "git submodule update"
chokes on the "git-fetch" in modules_update().

So I suggest extending "git submodule add" to handle existing
subdirectories by simply using them as-is instead of creating a new
clone. And "git submodule update" shouldn't try fetching when the commit
is already present locally.

Speaking of which, currently "git submodule" clones/fetches *all*
branches on the remote repository, which seems quite excessive given
that the supermodule's content references just one submodule commit. By
default, "git submodule add" should fetch only the needed head, and "git
submodule update" should fetch only the commit sha1 (if needed).

  Eran