[0/4] What's not in 1.5.2 (overview)

[0/4] What's not in 1.5.2 (overview)

[Junio C Hamano]

Upcoming release 1.5.2 is nearing completion, so let's look at
the issues that are specifically excluded from it.

Before listing topics there is one thing.

*  sp/cvsexport (Thu May 10 01:06:36 2007 +0200) 1 commit
 + Optimized cvsexportcommit: calling 'cvs status' once instead of
   once per touched file.

This should really be 1.5.2.  Real-world users of cvsexport, if
you were burned by this, please NAK as soon as possible.  I'll
merge it to 1.5.2 unless I hear from anybody in a day or two.

The messages that follow this are:

  [1/4] Easy-to-decide ones; what's been cooking in next.
  [2/4] The ones to be cooked in next after 1.5.2
  [3/4] Things we do not have code for yet.
  [4/4] Leftover bits

[1/4] What's not in 1.5.2 (have been cooking in next)

[Junio C Hamano]

Here are the first batch that will be in 'master' after 1.5.2
happens.

They all have been cooking in 'next', or were not compelling
enough feature enhancements to be included after -rc1.

*  jb/statcolor (Sat May 5 16:48:54 2007 -0400) 1 commit
 + Add colour support in rebase and merge tree diff stats output.

*  tt/gc (Wed May 9 15:48:39 2007 -0400) 1 commit
 + Add --aggressive option to 'git gc'

*  np/pack (Wed May 9 14:42:42 2007 -0400) 3 commits
 + deprecate the new loose object header format
 + make "repack -f" imply "pack-objects --no-reuse-object"
 + allow for undeltified objects not to be reused.

*  sv/checkout (Wed May 9 12:33:20 2007 +0200) 1 commit
 + git-update-ref: add --no-deref option for overwriting/detaching
   ref

*  mst/connect (Wed May 16 20:09:41 2007 +0300) 1 commit
 + connect: display connection progress

*  dh/pack (Wed May 9 13:56:50 2007 -0700) 3 commits
 + Custom compression levels for objects and packs
 + make "repack -f" imply "pack-objects --no-reuse-object"
 + allow for undeltified objects not to be reused

[2/4] What's not in 1.5.2 (will cook in next)

[Junio C Hamano]

These have been cooking in 'pu'; will be in 'next' after 1.5.2
happens.

*  db/remote (Tue May 15 22:50:19 2007 -0400) 5 commits
 . Update local tracking refs when pushing
 . Add handlers for fetch-side configuration of remotes.
 . Move refspec parser from connect.c and cache.h to remote.{c,h}
 . Move remote parsing into a library file out of builtin-push.
 + git-update-ref: add --no-deref option for overwriting/detaching
   ref

This was rebased on to Sven's change to lock_any_ref_for_update();

*  dh/repack (Sun May 13 12:47:09 2007 -0700) 9 commits
 . git-repack --max-pack-size: add option parsing to enable feature
 . git-repack --max-pack-size: split packs as asked by
   write_{object,one}()
 . git-repack --max-pack-size: write_{object,one}() respect pack
   limit
 . git-repack --max-pack-size: new file statics and code
   restructuring
 . Alter sha1close() 3rd argument to request flush only
 + Custom compression levels for objects and packs
 + deprecate the new loose object header format
 + make "repack -f" imply "pack-objects --no-reuse-object"
 + allow for undeltified objects not to be reused

This follows the "custom compression levels" series from the
same author.

[3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

Here are random things that I'd like to see happen during 1.5.3
cycle.

 * git-clone should become a very thin wrapper around
   init/remote/fetch/checkout.

   - it may be necessary to teach git-remote about --bare
     layout.

   - by getting rid of 'fetch' logic from git-clone as much as
     possible, it would hopefully become easier to freeze what
     git-fetch needs to do and help rewriting the latter in C.

   - update native protocol with an extension to carry which
     branch HEAD (or any symref) points at, instead of guessing.
     We already know this information when clone is done over
     HTTP, but we are discarding that information in git-clone
     for the sake of implementation simplicity and having it
     also guess.

 * more superproject support in Porcelain-ish.

   - superproject support in Porcelain-ish is mostly about
     checking things out.  One possible scenario:

	$ git clone --recursive $url_of_superproject

     may instruct "clone that, and also clone any projects that
     the superproject uses, and check everything out".  I would
     imagine that lThis would be carried out in the following
     steps.

     (1) The usual "git-clone" dance, determining the directory
         name to create, running mkdir and init-db, setting up
         tracking refspec specification in the config, and
         running the initial fetch.

     (2) Because the command was not given '-n' ("do not
         checkout"), HEAD is checked out.  git-checkout notices
         that there are commits bound to some directories in its
         tree.

     (3) git-checkout finds there is .gitmodules file in the
         tree (and the checked-out working file), which
         describes these subprojects.  It looks at the config
         and notices that it does not yet know about them
         (obviously this is true, as this is the first checkout
         after clone, but I am trying to outline how checkout
         after a merge should work in the general case).

	 It determines where to fetch that subproject from,
	 perhaps it uses the default URL described in
	 .gitmodules file to, while asking the user for
	 confirmation and giving the user a chance to override
	 it.  And it records something in the config -- now that
	 project is known to this repository.

     (4) git-checkout then calls git-clone recursively in that
         subdirectory for the subproject (which may further
         contain subprojects of its own, but that would
         naturally work).

   - Another scenario, after "git clone" of the superproject
     _without_ the above "--recursive" behaviour.  There needs
     to be a way to later clone and checkout a named subproject
     (and that subproject alone).  Perhaps

	$ git checkout --populate-subproject subdir

     would notice that subdir corresponds to a subproject that
     is "not known" to this repository.

	 I am assuming .gitmodules from the upstream describes it,
	 but the default is not to recurse into any subproject,
	 which would leave the subdir _without_ its own .git
	 directory.  A subproject becomes "known" to your repository
	 when you tell git that you care about it (e.g. "clone
	 --recursive" above), and that decision would be recorded
	 somewhere in .git/config of the superproject.  .gitmodules
	 would give the default URL and probably the branch to
	 follow.  The URL definitely needs to be overridable by
	 per-repository configuration as network reachability would
	 be different for everybody; I am not sure about the need to
	 make it overridable which branch the subproject should
	 follow, but my gut feeling is that we do not have to.

     Then it does the same as (3) and (4) in "clone --reference"
     above (in fact, I am envisioning that that scenario would
     call "git checkout --populate-subproject" in those steps).

   - We might want to have a way to tell it to stop tracking an
     already "known" subproject, which means that the section in
     .git/config that would track the status of the subproject
     (e.g. "is it known", "what's the URL", etc.) needs another
     variable that says "are we still interested in it?".

   - There may be obvious "frills" we would want to eventually
     have, such as defaulting the repository to find the
     subproject that corresponds to subdir/ to $URL/subdir (iow
     if http://repo.or.cz/cgit.git binds git.git at its src/
     directory, we would expect that the repository to house
     that subproject is found at http://repo.or/cz/cgit.git/src
     by default), but during the first round, I think it is
     better to leave things explicit.  So, for the above
     example, instead of defaulting the $URL/$subdir location,
     always require .gitmodules to say where to fetch the
     subproject from.

   - "git diff", "git status" and friends might want to learn
     recursive behaviour.  These should be much easier than any
     of the above.

   Note: I am not trying to dictate the overall superproject
   Porcelain design should follow the above literally, but just
   throwing out a strawman.  As I do not expect to be a heavy
   user of superproject support myself yet, other people who
   have thought about the problem longer (much longer) than I
   have will certainly have better ideas.


 * Perhaps add 'tree' entries in the index.  This may make the
   current cache-tree extension unnecessary, and I suspect it
   will simplify various paths that deal with D/F conflicts in
   the current codebase.

   I suspect this might need 1.6, as it is a one-way backward
   incompatible change for the 'index', but 'index' is local so
   it might not be such a big deal.  In the worst case, when the
   users find "git checkout" from 1.5.2 does not work in a
   repository checked out with such an updated index format, we
   could ask them to "rm -f .git/index && git checkout HEAD".

 * make merge-recursive and read-tree -u more robust when D/F
   conflict is involved.

   I think that the use of current_{file,directory}_set is
   misguided and it should just ask the index if there are
   conflicts.  unpack-trees.c::check_updates() logic probably is
   the right place to deal with what to do with the working tree
   when the merge result contains D/F conflicts (i.e.
   merge-recursive wants to create file~branchname instead of
   not touching the working tree).

Re: [3/4] What's not in 1.5.2 (new topics)

[Andy Parkins]

On Wednesday 2007, May 16, Junio C Hamano wrote:

>      (3) git-checkout finds there is .gitmodules file in the
>          tree (and the checked-out working file), which
>          describes these subprojects.  It looks at the config
>          and notices that it does not yet know about them
>          (obviously this is true, as this is the first checkout
>          after clone, but I am trying to outline how checkout
>          after a merge should work in the general case).
>
> 	 It determines where to fetch that subproject from,
> 	 perhaps it uses the default URL described in
> 	 .gitmodules file to, while asking the user for
> 	 confirmation and giving the user a chance to override
> 	 it.  And it records something in the config -- now that
> 	 project is known to this repository.

I've been thinking about this .gitmodules thing and have a concern.  
Aren't we falling into the svn:externals trap?

The svn:externals property is analagous to our .gitmodules file.  svn 
properties were basically just version controlled out-of-tree meta data 
(making them annoying to work with - in-tree is better).  

svn "submodule" support was done by writing something like

  subproject svn://host/blah/blah

In the svn:externals property attached to the directory that 
the "subproject" directory was in.  To translate:

  svn propset svn:externals "subproject svn://blah/blah/blah" .

  git clone git://blah/blah/blah subproject
  git add subproject

The hole that this sort of thing gets you in to is that the 
svn:externals property is version controlled.  Time passes since you 
added the external; in that time the URL becomes invalid.  No problem, 
you simply change the svn:externals property.  KABOOM.  Now any 
historical checkout fails because it checks out the svn:externals 
property from that checkout and tries to use the wrong URL.

Our in-tree .gitmodules will have the same problem.  I recognise that 
you've mitigated that with some "confirm with the user, store in the 
config" hand waving; but that is just hiding the problem: the submodule 
URL is not something that should be version controlled; it is an 
all-of-history property; when it changes for revision N it changes for 
revision N-1, N-2, N-3, etc.  Storing it in .gitmodules implies that 
it's value in the past has meaning - it doesn't.

You mentioned yourself that that problem is not confined to the temporal 
accuracy of .gitmodules, there is spatial accuracy too - there is no 
guarantee that user A wants to use the same submodule URL as user B.  
Fast forward to when we've got submodule support; let's say you start 
using it for git-gui (for example).  Somehow (let's leave the "how" 
till later) I've gotten a working git tree with a git-gui checked out.  
I go to my laptop and clone that repository (note: NOT the upstream 
repository).  When git-clone hits the git-gui submodule it should not 
go looking for the upstream git-gui, I will want it to clone my local 
git-gui submodule.  i.e. in-tree .gitmodules URL for git-gui will be 
wrong.

I hope the above shows that in-tree .gitmodules is wrong; it can only 
ever be a hint, and in a great number of cases it will be an incorrect 
hint.

I know it's so enticing to store it in-tree; it would be great because 
the normal repository object transfer mechanism would get the URL of 
the submodule to the receiver with no changes to current 
infrastructure.  I say: tough luck - we need another mechanism.  The 
submodule URL is a per-repository setting, not a per-project setting.  
When fetching, some out-of-band mechanism for telling the other side 
what URL _this_ repository thinks the submodule is at needs to be 
supplied.  I don't know what space there is in the git protocol for 
putting that information, but I suspect that that is where it needs to 
go.

As an alternative to that, the supermodule could be given the ability to 
proxy for the submodule during clone.  It knows where the submodule is 
stored from it's point of view; is there scope for doing a 
virtual-server-like system were the supermodule git-daemon just changes 
to the submodule repository (in the case it is local) and thereby gives 
the downstream git access to the submodule without it even needing a 
URL.

>  * Perhaps add 'tree' entries in the index.  This may make the
>    current cache-tree extension unnecessary, and I suspect it
>    will simplify various paths that deal with D/F conflicts in
>    the current codebase.
>
>    I suspect this might need 1.6, as it is a one-way backward
>    incompatible change for the 'index', but 'index' is local so
>    it might not be such a big deal.  In the worst case, when the
>    users find "git checkout" from 1.5.2 does not work in a
>    repository checked out with such an updated index format, we
>    could ask them to "rm -f .git/index && git checkout HEAD".

I don't think even that would be necessary.  Assuming that the new index 
format is a superset of the old index format the only way that tree 
entries would get in the index would be by using git-1.6.  Almost by 
definition then, if they are in there your git is up-to-date enough to 
use them.  (modulo me not really understanding what you mean)



Andy

-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

Andy Parkins <andyparkins@gmail.com> writes:

> Our in-tree .gitmodules will have the same problem.  I recognise that 
> you've mitigated that with some "confirm with the user, store in the 
> config" hand waving; but that is just hiding the problem: the submodule 
> URL is not something that should be version controlled; it is an 
> all-of-history property; when it changes for revision N it changes for 
> revision N-1, N-2, N-3, etc.  Storing it in .gitmodules implies that 
> it's value in the past has meaning - it doesn't.

I think that depends _WHY_ the URL recorded .gitmodules are
updated.  It would perfectly be reasonable for release #1 of an
appliance project to bind linux 2.4 tree at kernel/ subdirectory
while release #2 source to have 2.6 one; they come from two
different repository URLs.  When you seek the superproject back
to release #1, you would still want to fetch from 2.4 upstream
if you are updating.

If the URL is changed only because the logically same project
was relocated to different hosting service, then what you say is
true.

What I was "handwaving" (or "envisioning") was to have something
like this in .gitmodules:

	[subproject "kernel/"]
        	URL = git://git.kernel.org/pub/linux-2.4.git

(or 2.6, depending on the revision of the superproject) and per
repository configuration would maps this with these two entries:

	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
        	URL = http://www.kernel.org/pub/linux-2.4.git

	[subproject "git://git.kernel.org/pub/linux-2.6.git"]
        	URL = http://www.kernel.org/pub/linux-2.6.git

The intent is 

	(1) "kernel/" directory is found to be a gitlink in the
            tree/index; .gitmodules is consulted to find the
            "URL", which is just a handle and the initial hint

	(2) That "initial hint" is used to look up the
            subproject entry from the configuration, to find the
            "real" URL that is used by this repository

> You mentioned yourself that that problem is not confined to the temporal 
> accuracy of .gitmodules, there is spatial accuracy too - there is no 
> guarantee that user A wants to use the same submodule URL as user B.  

which hopefully is already answered by the above handwaving ;-).

The case of "relocated to different hosting site" would also be
solved by having more than one entries in the configuration
file.  If a project that used to be hosted at git.or.cz has
migrated to git.sf.net, its .gitmodules file from an earlier
revision would have URL pointing at git.repo.cz and newer ones
would point at git.sf.net.  If you started following that
project before the migration, you would have:

	[subproject "git://git.or.cz/sub.git"]
        	URL = git://git.or.cz/sub.git

in your .git/config.  After the repository migrates to
git.sf.net, you would update that existing entry and also add
another entry, so that .git/config would have these two entries:

	[subproject "git://git.or.cz/sub.git"]
        	URL = git://git.sf.net/sub.git

	[subproject "git://git.sf.net/sub.git"]
        	URL = git://git.sf.net/sub.git

Re: [3/4] What's not in 1.5.2 (new topics)

[Andy Parkins]

On Thursday 2007 May 17, Junio C Hamano wrote:

> I think that depends _WHY_ the URL recorded .gitmodules are
> updated.  It would perfectly be reasonable for release #1 of an
> appliance project to bind linux 2.4 tree at kernel/ subdirectory
> while release #2 source to have 2.6 one; they come from two
> different repository URLs.  When you seek the superproject back
> to release #1, you would still want to fetch from 2.4 upstream
> if you are updating.

That's a very good point; I hadn't considered that there was a case for 
recording a change.

> What I was "handwaving" (or "envisioning") was to have something
> like this in .gitmodules:

Sorry, "handwaving" was a bit rude - I certainly didn't mean that you weren't 
supplying sufficient detail for the circumstance; I meant that in the sense 
of the tricky bits being papered over with user overrides and questions about 
which URL to /really/ use.  The fact that you even felt it necessary to 
mention those overrides signals, I think, that something is wrong.

> 	[subproject "kernel/"]
>         	URL = git://git.kernel.org/pub/linux-2.4.git
>
> (or 2.6, depending on the revision of the superproject) and per
> repository configuration would maps this with these two entries:

So now - running with your example - I'm in a project with a 2.6 URL 
in .gitmodules and config; now I check out a past revision.  .gitmodules is 
updated to show the URL at that time (2.4) - what happens to config, which 
must have higher precedence?  Am I meant to update that myself?  So, as I hop 
around between branches you expect that I will be updating the config file 
for each checkout?

> 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
>         	URL = http://www.kernel.org/pub/linux-2.4.git
>
> 	[subproject "git://git.kernel.org/pub/linux-2.6.git"]
>         	URL = http://www.kernel.org/pub/linux-2.6.git

Now this part I love.  _That_ is a proper solution.  To me though, these are a 
completely different category from the [subproject] above.  I think that 
should be highlighted with a different section name like "[urlmap]".
     
> The intent is
>
> 	(1) "kernel/" directory is found to be a gitlink in the
>             tree/index; .gitmodules is consulted to find the
>             "URL", which is just a handle and the initial hint

In which case that [subproject "kernel/"] section is not needed (I think it 
would be better to simply say "URL not found for submodule kernel/" or 
something if there is no .gitmodules rather than supplying that override).

> 	(2) That "initial hint" is used to look up the
>             subproject entry from the configuration, to find the
>             "real" URL that is used by this repository

Yes.  Excellent; the "hint" now becomes a lookup key into the url mappings.

> which hopefully is already answered by the above handwaving ;-).

Absolutely.  I'm very impressed.  It solves both the temporal and spatial 
changes problem because one can remap every URL that was ever used in the 
history of the .gitmodules file if one wanted.

> in your .git/config.  After the repository migrates to
> git.sf.net, you would update that existing entry and also add
> another entry, so that .git/config would have these two entries:
>
> 	[subproject "git://git.or.cz/sub.git"]
>         	URL = git://git.sf.net/sub.git
>
> 	[subproject "git://git.sf.net/sub.git"]
>         	URL = git://git.sf.net/sub.git

I don't suppose the second one is needed; wouldn't the default be $key = $url, 
when no override is found?

This also raises the point that these mappings would probably be 
order-dependent; because it may be that I want to do:

  [subproject "git://git.or.cz/sub.git"]
    URL = git://git.sf.net/sub.git

  [subproject "git://git.sf.net/sub.git"]
    URL = /home/andyp/git/mycopyofsub.git

In conclusion: I think that's a first class solution to the problem (and 
probably what you had in mind all along, and me screaming around wasn't 
helpful :-)).



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [3/4] What's not in 1.5.2 (new topics)

[Alex Riesen]

Junio C Hamano, Thu, May 17, 2007 07:21:40 +0200:
> What I was "handwaving" (or "envisioning") was to have something
> like this in .gitmodules:
> 
> 	[subproject "kernel/"]
>         	URL = git://git.kernel.org/pub/linux-2.4.git

So, assuming .gitmodules is versioned (afaics, it is), it would mean
that after a some unlucky git-pull, where someone changed the upstream
.gitmodules ("linux-2.4" for whatever reason is changed to just
"linux"). And suddenly all such local configuration is useless:

> (or 2.6, depending on the revision of the superproject) and per
> repository configuration would maps this with these two entries:
> 
> 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
>         	URL = http://www.kernel.org/pub/linux-2.4.git
>
> 	[subproject "git://git.kernel.org/pub/linux-2.6.git"]

isn't there a typo somewhere around "2.6"?

>         	URL = http://www.kernel.org/pub/linux-2.6.git

because there is no URL to map from.

why can't I just have _repo_ configuration:

 	[subproject "kernel/"]
         	URL = http://www.kernel.org/pub/linux-2.6.git
?
It can be first-time cloned from the upstream, but it stays after
people change it to suit their systems. They can depend on it not to
be broken by upstream.

> The intent is 
> 
> 	(1) "kernel/" directory is found to be a gitlink in the
>             tree/index; .gitmodules is consulted to find the
>             "URL", which is just a handle and the initial hint
> 
> 	(2) That "initial hint" is used to look up the
>             subproject entry from the configuration, to find the
>             "real" URL that is used by this repository

It is quite long-living to be just initial hint. And will be redundant
after the hint loses all meaning (after some time it _will_ happen,
sites do move around), and is just a strange looking mapping key.

Can I suggest a part of repo configuration to be clonable? So that
there is a something in .git/config.dist, which is _cloned_ with
git-clone. The obviuos thing to put there would be subproject
configuration, and maybe there will be something else in the future
(I'd think of description, which is a separate file now, and as for
now, the only way to get this description is to use gitweb or ssh).
git-ls-remote could be made to show this "remote-accessible"
configuration, in case someone have to update/compare local copy of
this config.

Re: [3/4] What's not in 1.5.2 (new topics)

[Petr Baudis]

On Thu, May 17, 2007 at 01:02:25PM CEST, Alex Riesen wrote:
> why can't I just have _repo_ configuration:
> 
>  	[subproject "kernel/"]
>          	URL = http://www.kernel.org/pub/linux-2.6.git
> ?
> It can be first-time cloned from the upstream, but it stays after
> people change it to suit their systems. They can depend on it not to
> be broken by upstream.

Because kernel/ can get removed, moved around, or point at entirely
*different* projects over time and branches - kernel/ can switch from
linux-2.4 to linux-2.6, libc/ can switch between glibc and uClibc, ...

> Can I suggest a part of repo configuration to be clonable? So that
> there is a something in .git/config.dist, which is _cloned_ with
> git-clone. The obviuos thing to put there would be subproject
> configuration, and maybe there will be something else in the future
> (I'd think of description, which is a separate file now, and as for
> now, the only way to get this description is to use gitweb or ssh).
> git-ls-remote could be made to show this "remote-accessible"
> configuration, in case someone have to update/compare local copy of
> this config.

This is troublesome because then you will also need a way to update the
configuration in the future, otherwise you will run into some
embarassing situations, and since we don't even support any motds while
fetching, when something *needs* to be changed you don't even have a
good way to tell your users. (Actually, I've been thinking about adding
motd support to the fetchers. :-)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [3/4] What's not in 1.5.2 (new topics)

[Jeff King]

On Thu, May 17, 2007 at 02:46:22PM +0200, Petr Baudis wrote:

> > why can't I just have _repo_ configuration:
> > 
> >  	[subproject "kernel/"]
> >          	URL = http://www.kernel.org/pub/linux-2.6.git
> > ?
> > It can be first-time cloned from the upstream, but it stays after
> > people change it to suit their systems. They can depend on it not to
> > be broken by upstream.
> 
> Because kernel/ can get removed, moved around, or point at entirely
> *different* projects over time and branches - kernel/ can switch from
> linux-2.4 to linux-2.6, libc/ can switch between glibc and uClibc, ...

I think we clearly need a 2-level system: a tracked pointer to the repo,
with an optional local override.

However, I don't quite like Junio's idea of using the URL as a key,
since it is intended to change. IOW, if I am overriding your URL via
.git/config, if you change your URL then my config is now broken.

Instead, why not:
  1. url location is supplied in configuration as
     [subproject "kernel/"]
       url = git://git.kernel.org/pub/linux-2.4.git
  2. .gitmodules is simply read as a lower-priority version of
     configuration

One advantage of this approach is that it's totally general; instead of
.gitmodules, we could in fact be talking about .gitconfig, a mechanism
for projects to contain tracked configuration that can be overridden by
individual repos. For some projects, I imagine some of the commit
encoding config options might make sense.

Thoughts?

-Peff

Re: [3/4] What's not in 1.5.2 (new topics)

[Petr Baudis]

On Thu, May 17, 2007 at 03:46:49PM CEST, Jeff King wrote:
> On Thu, May 17, 2007 at 02:46:22PM +0200, Petr Baudis wrote:
> 
> > > why can't I just have _repo_ configuration:
> > > 
> > >  	[subproject "kernel/"]
> > >          	URL = http://www.kernel.org/pub/linux-2.6.git
> > > ?
> > > It can be first-time cloned from the upstream, but it stays after
> > > people change it to suit their systems. They can depend on it not to
> > > be broken by upstream.
> > 
> > Because kernel/ can get removed, moved around, or point at entirely
> > *different* projects over time and branches - kernel/ can switch from
> > linux-2.4 to linux-2.6, libc/ can switch between glibc and uClibc, ...
> 
> I think we clearly need a 2-level system: a tracked pointer to the repo,
> with an optional local override.
> 
> However, I don't quite like Junio's idea of using the URL as a key,
> since it is intended to change. IOW, if I am overriding your URL via
> .git/config, if you change your URL then my config is now broken.
> 
> Instead, why not:
>   1. url location is supplied in configuration as
>      [subproject "kernel/"]
>        url = git://git.kernel.org/pub/linux-2.4.git
>   2. .gitmodules is simply read as a lower-priority version of
>      configuration

But, did you read what you actually quoted? Because I can only repeat my
argument in the face of (1), and you didn't seem to dispute any part of
it at all.

"kernel/" has _no_ meaning. Only a (treeid,"kernel/") pair has meaning,
nothing less - a particular tree contains a submodule in given subtree.
Different trees can have different submodules in different subtrees.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [3/4] What's not in 1.5.2 (new topics)

[Jeff King]

On Thu, May 17, 2007 at 06:10:02PM +0200, Petr Baudis wrote:

> But, did you read what you actually quoted? Because I can only repeat my
> argument in the face of (1), and you didn't seem to dispute any part of
> it at all.

You said:
> Because kernel/ can get removed, moved around, or point at entirely
> *different* projects over time and branches - kernel/ can switch from
> linux-2.4 to linux-2.6, libc/ can switch between glibc and uClibc, ...

which I took to mean that we must be able to track changes to the URL
which is pointed to by the kernel/ submodule, and therefore this
configuration must be in a tracked file.  Which is _precisely_ what I
advocated: it goes in a .gitmodules (or .gitconfig) file in the tracked
directory. This is counter to what Alex says, which is that one should
simply pull the config down during clone time and never change it.

However, I think we _must_ have an override mechanism, since I don't
necessarily use the same URLs that you do. I propose that such overrides
should go into the local repo config. The only difference between what I
have proposed and what Junio mentioned is that I would base the config
override key on the directory name, not the URL. This means that if
upstream changes their pointer to the URL, yours will change with it
_unless you have an override_. With Junio's, their change of URL will
override your change (since the key will no longer match your config).

How do you propose to handle overrides?

> "kernel/" has _no_ meaning. Only a (treeid,"kernel/") pair has meaning,
> nothing less - a particular tree contains a submodule in given subtree.
> Different trees can have different submodules in different subtrees.

Right. In my proposal (unlike Alex's), it _is_ tied to the tree, since
that tree has a particular .gitmodules. But I also think you should be
able to override the submodule URL for kernel/ _for all time_ if you
want.

-Peff

Re: [3/4] What's not in 1.5.2 (new topics)

[Petr Baudis]

On Thu, May 17, 2007 at 06:25:42PM CEST, Jeff King wrote:
> However, I think we _must_ have an override mechanism, since I don't
> necessarily use the same URLs that you do. I propose that such overrides
> should go into the local repo config. The only difference between what I
> have proposed and what Junio mentioned is that I would base the config
> override key on the directory name, not the URL. This means that if
> upstream changes their pointer to the URL, yours will change with it
> _unless you have an override_. With Junio's, their change of URL will
> override your change (since the key will no longer match your config).
> 
> How do you propose to handle overrides?

I think Junio's URL keying works fine. Their change of URL will override
your change, but that is bad thing only when the old upstream's URL
changed, but the upstream stays the same. Then either the problem is
clearly visible or it will result only in somewhat suboptimal behaviour.

OTOH, if the _upstream_ changed and your override scheme is at work, you
won't notice at all and simply will continue to use the same old
upstream.

> > "kernel/" has _no_ meaning. Only a (treeid,"kernel/") pair has meaning,
> > nothing less - a particular tree contains a submodule in given subtree.
> > Different trees can have different submodules in different subtrees.
> 
> Right. In my proposal (unlike Alex's), it _is_ tied to the tree, since
> that tree has a particular .gitmodules. But I also think you should be
> able to override the submodule URL for kernel/ _for all time_ if you
> want.

But again - "kernel/" means nothing, only "kernel/ in tree X". kernel/
might point to linux-2.4 in older trees, linux-2.6 in newer trees, -mm
in the experimental branch and freebsd tree in the weirdo branch. Such
an override is _never_ going to work in the general situation, only when
"kernel/" always in all commits on all branches points to the same
single project. (You can work that around by at least making the setting
branch-specific, but that still doens't take into account the history,
and then newly created branches won't have the override you want, etc.)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [3/4] What's not in 1.5.2 (new topics)

[Jeff King]

On Thu, May 17, 2007 at 07:30:05PM +0200, Petr Baudis wrote:

> I think Junio's URL keying works fine. Their change of URL will override
> your change, but that is bad thing only when the old upstream's URL
> changed, but the upstream stays the same. Then either the problem is
> clearly visible or it will result only in somewhat suboptimal behaviour.
> 
> OTOH, if the _upstream_ changed and your override scheme is at work, you
> won't notice at all and simply will continue to use the same old
> upstream.

Right. But I think it makes more sense for the error condition to go the
other way (that is, your override might get stale, but it will always be
an _override_). You'll notice eventually anyway when upstream moves to a
commit sha1 that you don't have in your submodule repo (or if they never
do, that means your override tree actually _is_ valid, and didn't need
to be changed anyway; this would be the case if upstream moved to a
different mirror, but you were already using an alternate source
anyway).

> But again - "kernel/" means nothing, only "kernel/ in tree X". kernel/
> might point to linux-2.4 in older trees, linux-2.6 in newer trees, -mm
> in the experimental branch and freebsd tree in the weirdo branch. Such
> an override is _never_ going to work in the general situation, only when
> "kernel/" always in all commits on all branches points to the same
> single project. (You can work that around by at least making the setting
> branch-specific, but that still doens't take into account the history,
> and then newly created branches won't have the override you want, etc.)

My point is that we _already_ have a mechanism that unambiguously points
to the linked commit, and it's _not_ the URL; it's the commit sha1
embedded in the tree.  Everything else is just a hint about where we
might find that commit.

-Peff

Re: [3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Instead, why not:
>   1. url location is supplied in configuration as
>      [subproject "kernel/"]
>        url = git://git.kernel.org/pub/linux-2.4.git
>   2. .gitmodules is simply read as a lower-priority version of
>      configuration

That does not support seeking back and forth between appliance
release #1 and release #2 which wants to say they want to bind
two different things at the same kernel/ path, does it?

Re: [3/4] What's not in 1.5.2 (new topics)

[Jeff King]

On Thu, May 17, 2007 at 11:49:12AM -0700, Junio C Hamano wrote:

> > Instead, why not:
> >   1. url location is supplied in configuration as
> >      [subproject "kernel/"]
> >        url = git://git.kernel.org/pub/linux-2.4.git
> >   2. .gitmodules is simply read as a lower-priority version of
> >      configuration
> 
> That does not support seeking back and forth between appliance
> release #1 and release #2 which wants to say they want to bind
> two different things at the same kernel/ path, does it?

I had a vague notion that the subproject could hold _both_ of them,
since it's really the commits you're flipping between. But obviously
that has quite complex semantics, so now it's me doing the handwaving.

What is the planned behavior when doing such a switch? I.e., if I have
kernel/ bound to linux-2.6, and I do a 'git-checkout' back in time to a
version that wants linux-2.4 bound at kernel/, what will happen? Blowing
away my subproject repo doesn't seem right.

-Peff

Re: [3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

Alex Riesen <raa.lkml@gmail.com> writes:

> Junio C Hamano, Thu, May 17, 2007 07:21:40 +0200:
>> What I was "handwaving" (or "envisioning") was to have something
>> like this in .gitmodules:
>> 
>> 	[subproject "kernel/"]
>>         	URL = git://git.kernel.org/pub/linux-2.4.git
>
> So, assuming .gitmodules is versioned (afaics, it is), it would mean
> that after a some unlucky git-pull, where someone changed the upstream
> .gitmodules ("linux-2.4" for whatever reason is changed to just
> "linux"). And suddenly all such local configuration is useless:

See below.

>> (or 2.6, depending on the revision of the superproject) and per
>> repository configuration would maps this with these two entries:
>> 
>> 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
>>         	URL = http://www.kernel.org/pub/linux-2.4.git
>>
>> 	[subproject "git://git.kernel.org/pub/linux-2.6.git"]
>
> isn't there a typo somewhere around "2.6"?
>
>>         	URL = http://www.kernel.org/pub/linux-2.6.git
>
> because there is no URL to map from.

The basic idea is that you keep mappings for all the URLs that
appear in versions of .gitmodules in the history you are
interested in checking out.  If the upstream switches from 2.4
based one to 2.6 based one, .gitmodules would contain a new URL,
which is not yet known to your configuration.  Then either the
UI would ask, with the default hint in the .gitmodules you just
pulled, refuse and have you manually add it to your config after
confirming, or just take the default (iow "trust the upstream").

So, no, it is not a reason to drop older mappings when your tip
was updated by a pull.  It should still be possible to checkout
older version that depend on the older 2.4 based subproject.

> why can't I just have _repo_ configuration:
>
>  	[subproject "kernel/"]
>          	URL = http://www.kernel.org/pub/linux-2.6.git
> ?
> It can be first-time cloned from the upstream, but it stays after
> people change it to suit their systems. They can depend on it not to
> be broken by upstream.

But that is a wrong thing to do when you are forking from the
release #1 of the appliance project, which wanted to have 2.4
based on at that path.

Re: [3/4] What's not in 1.5.2 (new topics)

[Nicolas Pitre]

On Wed, 16 May 2007, Junio C Hamano wrote:

> Andy Parkins <andyparkins@gmail.com> writes:
> 
> > Our in-tree .gitmodules will have the same problem.  I recognise that 
> > you've mitigated that with some "confirm with the user, store in the 
> > config" hand waving; but that is just hiding the problem: the submodule 
> > URL is not something that should be version controlled; it is an 
> > all-of-history property; when it changes for revision N it changes for 
> > revision N-1, N-2, N-3, etc.  Storing it in .gitmodules implies that 
> > it's value in the past has meaning - it doesn't.
> 
> I think that depends _WHY_ the URL recorded .gitmodules are
> updated.  It would perfectly be reasonable for release #1 of an
> appliance project to bind linux 2.4 tree at kernel/ subdirectory
> while release #2 source to have 2.6 one; they come from two
> different repository URLs.  When you seek the superproject back
> to release #1, you would still want to fetch from 2.4 upstream
> if you are updating.

I don't know if the above example should make sense.  In practice that 
would mean you'll have to _replace_ the repo within the submodule 
directory which is quite different from merely checking out a different 
version of the same repository.


Nicolas

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

> What I was "handwaving" (or "envisioning") was to have something
> like this in .gitmodules:
> 
> 	[subproject "kernel/"]
>         	URL = git://git.kernel.org/pub/linux-2.4.git
> 
> (or 2.6, depending on the revision of the superproject) and per
> repository configuration would maps this with these two entries:
> 
> 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
>         	URL = http://www.kernel.org/pub/linux-2.4.git
> 
> 	[subproject "git://git.kernel.org/pub/linux-2.6.git"]
>         	URL = http://www.kernel.org/pub/linux-2.6.git
> 
> The intent is 
> 
> 	(1) "kernel/" directory is found to be a gitlink in the
>             tree/index; .gitmodules is consulted to find the
>             "URL", which is just a handle and the initial hint
> 
> 	(2) That "initial hint" is used to look up the
>             subproject entry from the configuration, to find the
>             "real" URL that is used by this repository

I'm reading up on submodules, two questions on this:

1. I understand the usefulness of the hint for public repositories, (the user might
need help discovering where to get submodules) but for private ones would this
create a hassle: I start with a subproject in ~/subprojecttest and if that gets
put in the URL hint, I have to maintain a map for ~/subprojecttest in my
.git/config forever even after I move it to ~/subprojectproduction, just to make
old releases build?
Do you think it might make sense to support a mode where .gitmodules
is empty, and URLs come from the config directly?

2. Suppose .gitmodules in upstream tree points at subproject repo at kernel.org,
and I clone from there - my repo will point at kernel.org by default?
But now, I'd like everyone who clones from *my* repo to get
pointed at *my* server by default (e.g. for mirroring),
but would not changing .gitmodules create a commit so my
head will now differ from upstream  - so it won't be signed properly etc...
Did I misunderstand something?

-- 
MST

Re: [3/4] What's not in 1.5.2 (new topics)

[Josef Weidendorfer]

On Thursday 17 May 2007, Michael S. Tsirkin wrote:
> > What I was "handwaving" (or "envisioning") was to have something
> > like this in .gitmodules:
> > 
> > 	[subproject "kernel/"]
> >         	URL = git://git.kernel.org/pub/linux-2.4.git
> > 
> > (or 2.6, depending on the revision of the superproject) and per
> > repository configuration would maps this with these two entries:
> > 
> > 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
> >         	URL = http://www.kernel.org/pub/linux-2.4.git
> > 
> > 	[subproject "git://git.kernel.org/pub/linux-2.6.git"]
> >         	URL = http://www.kernel.org/pub/linux-2.6.git
> > 
> > The intent is 
> > 
> > 	(1) "kernel/" directory is found to be a gitlink in the
> >             tree/index; .gitmodules is consulted to find the
> >             "URL", which is just a handle and the initial hint
> > 
> > 	(2) That "initial hint" is used to look up the
> >             subproject entry from the configuration, to find the
> >             "real" URL that is used by this repository
> 
> I'm reading up on submodules, two questions on this:
> 
> 1. I understand the usefulness of the hint for public repositories, (the user might
> need help discovering where to get submodules) but for private ones would this
> create a hassle: I start with a subproject in ~/subprojecttest and if that gets
> put in the URL hint, I have to maintain a map for ~/subprojecttest in my
> .git/config forever even after I move it to ~/subprojectproduction, just to make
> old releases build?

Yes, AFAICS that was the original idea; but that is no problem as we will need an
override scheme.

However, I think the usage of "url"/"url hint" as the 1st level subproject identifier
really is badly misleading and confusing for users; it would be better for this
identifier to not look like a URL at all. But by naming it "url" in .gitmodules, the
user is tempted to put an URL at this place.

IMHO it is by far better to simply talk about the "subproject name/identifier" which is
valid in the subproject namespace of the superproject.

And why not use the .gitattributes for the ".gitmodules" needs? 
With linux 2.4 as subproject in "top/kernel/", there could be a "top/.gitattributes"
with 

 kernel subproject=linux24

We could have a default rule that in the absense of the attribute, we default to the
path of the submodule, ie. to

 kernel subproject=top/kernel

In .git/config, there needs to be a config entry like

	[subproject "linux24"]
		URL = http://www.kernel.org/pub/linux-2.4.git

Again, we could have a default URL in the absence of this config entry which is
relative to the URL of the superproject, and which allows for the superproject
repository to act as proxy.

As relative path I would propose $SUPERURL/subproject/$SUBPROJECTNAME, ie. if
the superproject is at git://git.kernel.org/pub/super.git, the above subproject
would default to the URL git://git.kernel.org/pub/super.git/subproject/linux24
which could be a symlink on the server.

To support different subproject repositories linked in at the
same path of a superproject, Nicolas noted that we would have to replace
the subproject repository at top/kernel/.git (taking my example above)
whenever we cross the subproject change boundary in a checkout (e.g. from
linux24 to linux26). The natural thing here would be to have
subproject repositories at a seperate place, like inside of the superproject
repository such as at ".git/subproject/linux24", which works well with my
default interpretation of relative subproject paths above. At checkout,
the correct repository would be bound by a symlink:

 top/kernel/.git -> .git/subproject/linux24

Instead of a symlink, a magically working linkage mechanisms would be better
(the .git/gitlink proposal).

> 2. Suppose .gitmodules in upstream tree points at subproject repo at kernel.org,
> and I clone from there - my repo will point at kernel.org by default?
> But now, I'd like everyone who clones from *my* repo to get
> pointed at *my* server by default (e.g. for mirroring),
> but would not changing .gitmodules create a commit so my
> head will now differ from upstream  - so it won't be signed properly etc...
> Did I misunderstand something?

No, that is correct. Supporting a relative URL specification as proposed above
should solve this issue.

Josef

> 

Re: [3/4] What's not in 1.5.2 (new topics)

[Steven Grimm]

It seems like a lot of the friction here is because people are trying to 
devise a single mechanism that will handle two distinct cases:

1. The location of a subproject's changed (the "public repository 
relocated to a different host" problem). This is not temporally 
sensitive -- if you check out an old version of the superproject, you 
need to look in the new location for the subproject. A local override 
for the subproject's location will likely still be perfectly valid.

2. The superproject no longer wants to use the same subproject; it wants 
to replace it with something else at the same point in the tree (the 
"version 2 of superproject uses the 2.6 kernel as opposed to the 2.4 
kernel"). This is temporally sensitive -- if you check out an old 
version of the superproject, you want to use the old location for the 
subproject too. A single local override will most likely not be valid 
for both versions.

I think these are fundamentally different operations and it's the desire 
to fold them into one mechanism that's leading to a lot of the 
discussion here. Would we simplify things by not conflating them?

For example -- and yes, this is partially a rehash of other people's 
ideas -- instead of mapping a subproject path directly to revision@URL, 
instead map it to revision@symbolic name. The symbolic name is then 
separately mapped to a URL, and it's that symbolic name that can be 
locally overridden. The mappings of symbolic names to URLs is 
unversioned; the mapping of subprojects to revision@symbolic is 
versioned. Local overrides happen at the symbolic->URL mapping.

So you'd have something like

version 1: kernel-src/ -> kernel24
version 2: kernel-src/ -> kernel26
unversioned:
    kernel24 -> git://whatever/2.4
    kernel26 -> git://whatever/2.6

And then locally, the override is:

    kernel24 -> git://myhost/2.4

When version 2 gets pulled down, you start off using the upstream's URL, 
which you know because you pulled down the new copy of the unversioned 
symbolic->URL map. Maybe git-pull gives you a warning like, "I see you 
have some overrides, so you might want to know about this new symbolic 
name too." With an appropriate option it might even stop before doing 
anything with the new symbolic name to give you a chance to override.

Maybe that has some problems I'm not seeing, but it seems like adding 
one more layer of indirection which has different versioning semantics 
would make this a more tractable problem.

-Steve

Re: [3/4] What's not in 1.5.2 (new topics)

[Petr Baudis]

On Fri, May 18, 2007 at 02:32:53AM CEST, Steven Grimm wrote:
> For example -- and yes, this is partially a rehash of other people's 
> ideas -- instead of mapping a subproject path directly to revision@URL, 
> instead map it to revision@symbolic name. The symbolic name is then 
> separately mapped to a URL, and it's that symbolic name that can be 
> locally overridden. The mappings of symbolic names to URLs is 
> unversioned; the mapping of subprojects to revision@symbolic is 
> versioned. Local overrides happen at the symbolic->URL mapping.
> 
> So you'd have something like
> 
> version 1: kernel-src/ -> kernel24
> version 2: kernel-src/ -> kernel26
> unversioned:
>    kernel24 -> git://whatever/2.4
>    kernel26 -> git://whatever/2.6
> 
> And then locally, the override is:
> 
>    kernel24 -> git://myhost/2.4

Yes, this would be nice; in one of my first mails in this thread I
devoted a non-trivially large writeup to this, then proceeded to remove
it since this has a serious problem.

Actually, Git already has a nice mechanism to handle these unversionaed
pointers - tags. Just make refs/tags/subproject/kernel24 containing the
URL to fetch. It's even easily overridable locally (and not easily
overridable remotely...).

The problem is ugly too, though - suddenly, you have created a SINGLE
UNIVERSE-WIDE NAMESPACE INSIDE A DISTRIBUTED VCS. And that's not going
to work well. I think I don't have to elaborate too much - the
aforementioned FreeBSD people will have different ideas about kernels
than you, _you_ will have different idea about kernels in few tens of
years than now, then if you need to merge or probably even fetch, you
will get into big trouble.

Notice that we don't have any such namespace right now (except the
D(SHA1) namespace, which is however possible only because it's so huge
_and_ the names are assigned automagically in a way that virtually
guarantees uniqueness across the whole universe) - tags come closest,
but there is nothing that fundamentally breaks when a clash happens
inside the namespace - it's just UI thing. But subproject names are
etched to the history - once you name it, you just can't get rid of it
forever.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [3/4] What's not in 1.5.2 (new topics)

[Josef Weidendorfer]

On Friday 18 May 2007, Petr Baudis wrote:
> The problem is ugly too, though - suddenly, you have created a SINGLE
> UNIVERSE-WIDE NAMESPACE INSIDE A DISTRIBUTED VCS. And that's not going
> to work well.

Actually, tags are already such a namespace. If you want to merge
two projects which have the same tag names, of course you still
preserve the different tag objects, but only one will appear in the
refs/tags namespace.

So what is the best identifier for a subprobject? It is one that
probably never clashes with any subproject identifier of another
superproject. At least, it should not clash between any superprojects
which ever could be a candidate for merging the two into one.

Junios proposal using an URL as identifier actually is quite good in
this regard, similar to JAVA package names.

However, I wonder whether the possible merge of two superprojects
into one is a real issue. When they use the same subprojects identifiers,
there is a workaround: instead of merging, make one superproject the
subproject of the other.

Josef

Re: [3/4] What's not in 1.5.2 (new topics)

[Torgil Svensson]

On 5/18/07, Josef Weidendorfer <Josef.Weidendorfer@gmx.de> wrote:
> On Friday 18 May 2007, Petr Baudis wrote:
> So what is the best identifier for a subprobject? It is one that
> probably never clashes with any subproject identifier of another
> superproject. At least, it should not clash between any superprojects
> which ever could be a candidate for merging the two into one.

Put all root-commit SHA1 in a file, call it something like "project
object" and take the SHA1 of that object the identifier. The
SHA1-route has been successful so far as distributed "keys".

//Torgil

Re: [3/4] What's not in 1.5.2 (new topics)

[Jakub Narebski]

[Cc: Petr Baudis <pasky@suse.cz>, Josef Weidendorfer
<Josef.Weidendorfer@gmx.de>, "Michael S. Tsirkin" <mst@dev.mellanox.co.il>,
Junio C Hamano <junkio@cox.net>, Andy Parkins <andyparkins@gmail.com>,
Nicolas Pitre <nico@cam.org>, git@vger.kernel.org]

Petr Baudis wrote:
> On Fri, May 18, 2007 at 02:32:53AM CEST, Steven Grimm wrote:

>> For example -- and yes, this is partially a rehash of other people's 
>> ideas -- instead of mapping a subproject path directly to revision@URL, 
>> instead map it to revision@symbolic name. The symbolic name is then 
>> separately mapped to a URL, and it's that symbolic name that can be 
>> locally overridden. The mappings of symbolic names to URLs is 
>> unversioned; the mapping of subprojects to revision@symbolic is 
>> versioned. Local overrides happen at the symbolic->URL mapping.
>> 
>> So you'd have something like
>> 
>> version 1: kernel-src/ -> kernel24
>> version 2: kernel-src/ -> kernel26
>> unversioned:
>>    kernel24 -> git://whatever/2.4
>>    kernel26 -> git://whatever/2.6
>> 
>> And then locally, the override is:
>> 
>>    kernel24 -> git://myhost/2.4
> 
> Yes, this would be nice; in one of my first mails in this thread I
> devoted a non-trivially large writeup to this, then proceeded to remove
> it since this has a serious problem.
> 
> Actually, Git already has a nice mechanism to handle these unversionaed
> pointers - tags. Just make refs/tags/subproject/kernel24 containing the
> URL to fetch. It's even easily overridable locally (and not easily
> overridable remotely...).
> 
> The problem is ugly too, though - suddenly, you have created a SINGLE
> UNIVERSE-WIDE NAMESPACE INSIDE A DISTRIBUTED VCS. And that's not going
> to work well. I think I don't have to elaborate too much - the
> aforementioned FreeBSD people will have different ideas about kernels
> than you, _you_ will have different idea about kernels in few tens of
> years than now, then if you need to merge or probably even fetch, you
> will get into big trouble.
> 
> Notice that we don't have any such namespace right now (except the
> D(SHA1) namespace, which is however possible only because it's so huge
> _and_ the names are assigned automagically in a way that virtually
> guarantees uniqueness across the whole universe) - tags come closest,
> but there is nothing that fundamentally breaks when a clash happens
> inside the namespace - it's just UI thing. But subproject names are
> etched to the history - once you name it, you just can't get rid of it
> forever.

There is a bit ugly solution for this: instead of using symbolic name
in versioned .gitmodules for a subproject (for a repo), use subproject
identifier (inode), and put it in the tag object (or config) together with
the URL.  Git would then search all the subproject / submodule info for
a given inode.  You could have more than one inode / identifier name for
a subproject repo; this would avoid the "independently created" issue
with using inodes / file-ids in distributed SCM.  One would have to
ensure however that different subprojects get assigned different inodes.

This is yet another level of indirection, and needs searching all the
subprojects info; but I don't think that there would be that many
subprojects used.


Besides we make use of one such global namespace: version tags (although
they _could_ have different names, e.g. v0.6.0 and gitgui-0.6.0).

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: [3/4] What's not in 1.5.2 (new topics)

[Petr Baudis]

On Fri, May 18, 2007 at 02:00:07PM CEST, Jakub Narebski wrote:
> There is a bit ugly solution for this: instead of using symbolic name
> in versioned .gitmodules for a subproject (for a repo), use subproject
> identifier (inode), and put it in the tag object (or config) together with
> the URL.  Git would then search all the subproject / submodule info for
> a given inode.  You could have more than one inode / identifier name for
> a subproject repo; this would avoid the "independently created" issue
> with using inodes / file-ids in distributed SCM.  One would have to
> ensure however that different subprojects get assigned different inodes.

Well, then it doesn't make any difference, no? You just renamed the
problem but it stays the same - to ensure uniqueness even across
repositories.

Ok, you can declare now that you will just think out a UUID for the
subproject, but aside of not fitting well with the whole git philosophy,
then you don't need the indirection again, just use the UUID as the tag
name.

I have the feeling that I'm missing something basic in your proposal...

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [3/4] What's not in 1.5.2 (new topics)

[Jakub Narebski]

On Fri, 18 May 2007, Petr Baudis wrote:
> On Fri, May 18, 2007 at 02:00:07PM CEST, Jakub Narebski wrote:

>> There is a bit ugly solution for this: instead of using symbolic name
>> in versioned .gitmodules for a subproject (for a repo), use subproject
>> identifier (inode), and put it in the tag object (or config) together with
>> the URL.  Git would then search all the subproject / submodule info for
>> a given inode.  You could have more than one inode / identifier name for
>> a subproject repo; this would avoid the "independently created" issue
>> with using inodes / file-ids in distributed SCM.  One would have to
>> ensure however that different subprojects get assigned different inodes.
> 
> Well, then it doesn't make any difference, no? You just renamed the
> problem but it stays the same - to ensure uniqueness even across
> repositories.
> 
> Ok, you can declare now that you will just think out a UUID for the
> subproject, but aside of not fitting well with the whole git philosophy,
> then you don't need the indirection again, just use the UUID as the tag
> name.
> 
> I have the feeling that I'm missing something basic in your proposal...

I was thinking about _automatic_ UUID, generated by git. For example it
could be sha1 of first subproject commit which appeared in supermodule.
It is easy to check if two UUID correspond to the same repository:
check if both objects are present, or perhaps that one is reachable from
the other, or that they have common parent. This kind of UUID is not
that different from (global) SHA1 of object.

So the idea is to have versioned, i.e. in-tree mapping from directory
names to repositories via some kind of identifier: Junio idea of using
URL of repository, with possibility of overriding it in repo config,
the idea of using tag name, and having URL for repo in tag contents,
and my idea of tag name of tag containing UUID. To find the URL you
would search all the repo-tags for UUID, or for existence of commit
with given sha1.

But I haven't thought this idea through, so it migh be utter rubbish.


The porcelain part of subproject / submodule support is not that
easy, to cover for moving subproject "mountpoint", project changing URL,
conflict of project names and different naming of the same project etc.
-- 
Jakub Narebski
Poland

Re: [3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> [Cc: Petr Baudis <pasky@suse.cz>, Josef Weidendorfer
> <Josef.Weidendorfer@gmx.de>, "Michael S. Tsirkin" <mst@dev.mellanox.co.il>,
> Junio C Hamano <junkio@cox.net>, Andy Parkins <andyparkins@gmail.com>,
> Nicolas Pitre <nico@cam.org>, git@vger.kernel.org]

Offtopic.  Why do you do this, and what benefit are you or
anybody in the above list, which is in body part of the message,
getting?

Re: [3/4] What's not in 1.5.2 (new topics)

[Julian Phillips]

On Fri, 18 May 2007, Junio C Hamano wrote:

> Jakub Narebski <jnareb@gmail.com> writes:
>
>> [Cc: Petr Baudis <pasky@suse.cz>, Josef Weidendorfer
>> <Josef.Weidendorfer@gmx.de>, "Michael S. Tsirkin" <mst@dev.mellanox.co.il>,
>> Junio C Hamano <junkio@cox.net>, Andy Parkins <andyparkins@gmail.com>,
>> Nicolas Pitre <nico@cam.org>, git@vger.kernel.org]
>
> Offtopic.  Why do you do this, and what benefit are you or
> anybody in the above list, which is in body part of the message,
> getting?

It looks like he is posting through gmane using a news reader ... so the 
list post comes from gmane while the CCs go out directly (I assume).

-- 
Julian

  ---
Green's Law of Debate:
 	Anything is possible if you don't know what you're talking about.

Re: [3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

Julian Phillips <julian@quantumfyre.co.uk> writes:

> On Fri, 18 May 2007, Junio C Hamano wrote:
>
>> Jakub Narebski <jnareb@gmail.com> writes:
>>
>>> [Cc: Petr Baudis <pasky@suse.cz>, Josef Weidendorfer
>>> <Josef.Weidendorfer@gmx.de>, "Michael S. Tsirkin" <mst@dev.mellanox.co.il>,
>>> Junio C Hamano <junkio@cox.net>, Andy Parkins <andyparkins@gmail.com>,
>>> Nicolas Pitre <nico@cam.org>, git@vger.kernel.org]
>>
>> Offtopic.  Why do you do this, and what benefit are you or
>> anybody in the above list, which is in body part of the message,
>> getting?
>
> It looks like he is posting through gmane using a news reader ... so
> the list post comes from gmane while the CCs go out directly (I
> assume).

Ah, I see.  The names listed on that in-body CC: do appear on
the To: in the copy of the message that came via e-mail.  If
that is how gmane operates then there is nothing Jakub to do to
improve it, I guess...

Thanks for the clarification.

Re: [3/4] What's not in 1.5.2 (new topics)

[Petr Baudis]

On Fri, May 18, 2007 at 08:45:16PM CEST, Junio C Hamano wrote:
> Julian Phillips <julian@quantumfyre.co.uk> writes:
> 
> > On Fri, 18 May 2007, Junio C Hamano wrote:
> >
> >> Jakub Narebski <jnareb@gmail.com> writes:
> >>
> >>> [Cc: Petr Baudis <pasky@suse.cz>, Josef Weidendorfer
> >>> <Josef.Weidendorfer@gmx.de>, "Michael S. Tsirkin" <mst@dev.mellanox.co.il>,
> >>> Junio C Hamano <junkio@cox.net>, Andy Parkins <andyparkins@gmail.com>,
> >>> Nicolas Pitre <nico@cam.org>, git@vger.kernel.org]
> >>
> >> Offtopic.  Why do you do this, and what benefit are you or
> >> anybody in the above list, which is in body part of the message,
> >> getting?
> >
> > It looks like he is posting through gmane using a news reader ... so
> > the list post comes from gmane while the CCs go out directly (I
> > assume).
> 
> Ah, I see.  The names listed on that in-body CC: do appear on
> the To: in the copy of the message that came via e-mail.  If
> that is how gmane operates then there is nothing Jakub to do to
> improve it, I guess...
> 
> Thanks for the clarification.

Actually, Jakub, if it can be turned off, could you, please?

Not getting cc'd on replies is slightly annoying. But this is highly
confusing - suddenly I must take care _not_ to reply to the private
copies, and we actually do have a parallel subthread of replies to your
mail not cc'd to the mailing list. :-(

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

News reader woes (was: Re: [3/4] What's not in 1.5.2 (new topics))

[Jakub Narebski]

On Sun, 20 May 2007, Petr Baudis <pasky@suse.cz> wrote:
> On Fri, May 18, 2007 at 08:45:16PM CEST, Junio C Hamano wrote:
>> Julian Phillips <julian@quantumfyre.co.uk> writes:
>>> On Fri, 18 May 2007, Junio C Hamano wrote:
>>>> Jakub Narebski <jnareb@gmail.com> writes:
>>>>
>>>>> [Cc: Petr Baudis <pasky@suse.cz>, Josef Weidendorfer
>>>>> <Josef.Weidendorfer@gmx.de>, "Michael S. Tsirkin" <mst@dev.mellanox.co.il>,
>>>>> Junio C Hamano <junkio@cox.net>, Andy Parkins <andyparkins@gmail.com>,
>>>>> Nicolas Pitre <nico@cam.org>, git@vger.kernel.org]
>>>>
>>>> Offtopic.  Why do you do this, and what benefit are you or
>>>> anybody in the above list, which is in body part of the message,
>>>> getting?
>>>
>>> It looks like he is posting through gmane using a news reader ... so
>>> the list post comes from gmane while the CCs go out directly (I
>>> assume).
>> 
>> Ah, I see.  The names listed on that in-body CC: do appear on
>> the To: in the copy of the message that came via e-mail.  If
>> that is how gmane operates then there is nothing Jakub to do to
>> improve it, I guess...
>> 
>> Thanks for the clarification.
> 
> Actually, Jakub, if it can be turned off, could you, please?
> 
> Not getting cc'd on replies is slightly annoying. But this is highly
> confusing - suddenly I must take care _not_ to reply to the private
> copies, and we actually do have a parallel subthread of replies to your
> mail not cc'd to the mailing list. :-(

The problem, and mentioned above trying to overcome it, lies in
complicated interaction between the GMane news to email gateway,
vger mail filtering (spam protection rules) and the _news_ reader
I use, namely KNode 0.10.2 from KDE 3.5.3.

I read git mailing list via GMane NNTP (Usenet, news) interface:
  nntp://gmane.comp.version-control.git
as I respond only rarely. I start new threads using email, and
if I get reply via email I try to reply also from mail client, not
via news. However, when replying to message which I read only via
NNTP interface, replying in news client, I have option of sending
reply only to gmane.comp.version-control.git which means sending
email only to git mailing list and breaking Cc: list. 

Another option is to copy Cc: list manually to To: field (no Cc:
in this version of KNode), and have gmane.comp.version-control.git
in the Group: field.  The fact that there is no Cc: to set is the
problem of KNode.  The problem with GMane and vger interaction lies
in the fact that I _cannot_ put git@vger.kernel.org in the To: list,
as somehow vger rejects mails sent via GMane (it does not rejects
news messages send via GMane).

I could also copy whole message to mail client, but this would break
In-Reply-To: and references: headers, thus breaking threading.


Do I understand correctly that you prefer 1st option, namely replying
only to newsgroup / git mailing list?


Note that changing news client is as hard as changing email client:
you would like to migrate configuration, news state (read/unread
articles) and sent/drafts folders to new news client. Not that easy.

-- 
Jakub Narebski
Poland

Re: [3/4] What's not in 1.5.2 (new topics)

[Andy Parkins]

On Friday 2007 May 18, Josef Weidendorfer wrote:

> However, I think the usage of "url"/"url hint" as the 1st level subproject
> identifier really is badly misleading and confusing for users; it would be
> better for this identifier to not look like a URL at all. But by naming it
> "url" in .gitmodules, the user is tempted to put an URL at this place.

Bear in mind that what you're suggesting is no different in implementation 
from what Junio is suggesting but with one difference: in Junio's option 
the "identifier" will act as a default URL if no override is found.

Yours:

.gitmodules:
  kernel mykernelsubprojectid
.git/config
  [subproject "mykernelsubprojectid"]
     url = git://host/blah/blah.git

Junio's:

.gitmodules:
  kernel git://oldhost/blah/blah.git
.git/config
  [subproject "git://oldhost/blah/blah.git"]
     url = git://host/blah/blah.git

There is no difference between these two in terms of implementation.  Both 
assign a key to the "kernel" submodule then use that key to look up an 
override.  The advantage of Junio's suggestion is that when an override is 
not needed the key itself is used and therefore it Just Works (tm) with no 
change to the .git/config necessary.

> And why not use the .gitattributes for the ".gitmodules" needs?

I can't think of a reason why not; I think that's a separate question though.

> Again, we could have a default URL in the absence of this config entry
> which is relative to the URL of the superproject, and which allows for the
> superproject repository to act as proxy.

This is why Junio's option of URL=Key is better.  You are relying on the 
default being correct in order for a simple clone to work.  The relative path 
scheme you propose as a default, while logical, doesn't match anything that 
anyone does right now.  Look at any server that hosts multiple projects; they 
are stored flat not deep:

 project1/
 project2/
 project3/

One advantage of submodule support is that multiple supermodules can contain 
the same submodule, so you really can't force a hierarchical representation 
on the world just to make the default URL correct.

 project2/
  project1/
 project3/
  project1/

Oops.

> As relative path I would propose $SUPERURL/subproject/$SUBPROJECTNAME, ie.
> if the superproject is at git://git.kernel.org/pub/super.git, the above
> subproject would default to the URL
> git://git.kernel.org/pub/super.git/subproject/linux24 which could be a
> symlink on the server.

I'm really uncomfortable with the idea of relying on directory structure 
passed the root repository path; from the
 git://git.kernel.org/pub/super.git/
point onwards; we don't have any right to expect that this is a real directory 
tree.  As an example; svn URLs don't match up with what's on disk:

 svn://svnhost/pub/repo/trunk/src
                       ^^^^^^^^^^

On disk there is no such directory as /trunk/src under the repository 
directory.  In the same way, even technically what you suggest would work, 
the part of the URL under git://git.kernel.org/pub/super.git/ is git's own 
namespace - it's not the users to mess with.  E.g. if I had a subproject 
called "refs" you'd be in trouble.

> To support different subproject repositories linked in at the
> same path of a superproject, Nicolas noted that we would have to replace
> the subproject repository at top/kernel/.git (taking my example above)
> whenever we cross the subproject change boundary in a checkout (e.g. from
> linux24 to linux26). The natural thing here would be to have
> subproject repositories at a seperate place, like inside of the
> superproject repository such as at ".git/subproject/linux24", which works
> well with my default interpretation of relative subproject paths above. At
> checkout, the correct repository would be bound by a symlink:

Your objection to the url=key scheme was lack of simplicity - to me the above 
is significantly more complex and is relying far too much on the submodule 
being on the same server as the supermodule.  Big mistake.  A typical use of 
submodules would be to integrate someone else's project, not your own, nor 
indeed your own checkout of that project.  Why should I have to keep my own 
copies of, say, kernel2.4 and kernel2.6 when there are perfectly acceptable 
URLs to the real repositories?

> > 2. Suppose .gitmodules in upstream tree points at subproject repo at
> > kernel.org, and I clone from there - my repo will point at kernel.org by
> > default? But now, I'd like everyone who clones from *my* repo to get
> > pointed at *my* server by default (e.g. for mirroring),
> > but would not changing .gitmodules create a commit so my
> > head will now differ from upstream  - so it won't be signed properly
> > etc... Did I misunderstand something?
>
> No, that is correct. Supporting a relative URL specification as proposed
> above should solve this issue.

I think that's the wrong solution.  A change of source URL for a submodule 
from what upstream uses to your own server is a _fork_ from upstream, 
therefore you would fork your own branch in your supermodule and 
alter .gitmodules to point at your server.  Everybody is happy, and the fork 
is recorded.

The override system is only there for the local repository (which always takes 
precedence) not for the server provider to hide detail from those checking 
the repo out.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [3/4] What's not in 1.5.2 (new topics)

[Josef Weidendorfer]

On Friday 18 May 2007, Andy Parkins wrote:
> Bear in mind that what you're suggesting is no different in implementation 
> >from what Junio is suggesting but with one difference: in Junio's option 
> the "identifier" will act as a default URL if no override is found.

Yes; actually, its exactly the same aside from the name used in .gitmodules
for it, as I proposed a default URL which is derived from the suproject identifier
if no config entry is found.

> > Again, we could have a default URL in the absence of this config entry
> > which is relative to the URL of the superproject, and which allows for the
> > superproject repository to act as proxy.
> 
> This is why Junio's option of URL=Key is better.

It all depends on how we construct the default URL out of the subproject
identifier. Options:
(1) do not try to construct a default URL at all. Error out without a config
(2) use a configurable rewriting scheme like s/(.*)/git://host/\1/
(3) automatically detect a senseful rewriting scheme

Let's start with (1). We can invent convenient default schemes later on.


Josef

Re: [3/4] What's not in 1.5.2 (new topics)

[Andy Parkins]

On Friday 2007 May 18, Josef Weidendorfer wrote:

> It all depends on how we construct the default URL out of the subproject
> identifier. Options:
> (1) do not try to construct a default URL at all. Error out without a
> config (2) use a configurable rewriting scheme like s/(.*)/git://host/\1/
> (3) automatically detect a senseful rewriting scheme
>
> Let's start with (1). We can invent convenient default schemes later on.

All good; except let's start with 

 (1) if no config, try using the key itself - error out if that fails

Then everybody is happy - if you want to use your system where the key is not 
a URL, then don't - you'll get the error you want.  If the user chose to use 
a URL then magic will happen.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

> Quoting Andy Parkins <andyparkins@gmail.com>:
> Subject: Re: [3/4] What's not in 1.5.2 (new topics)
> 
> On Friday 2007 May 18, Josef Weidendorfer wrote:
> 
> > It all depends on how we construct the default URL out of the subproject
> > identifier. Options:
> > (1) do not try to construct a default URL at all. Error out without a
> > config (2) use a configurable rewriting scheme like s/(.*)/git://host/\1/
> > (3) automatically detect a senseful rewriting scheme
> >
> > Let's start with (1). We can invent convenient default schemes later on.
> 
> All good; except let's start with 
> 
>  (1) if no config, try using the key itself - error out if that fails
> 
> Then everybody is happy - if you want to use your system where the key is not 
> a URL, then don't - you'll get the error you want.  If the user chose to use 
> a URL then magic will happen.

I don't want an error. No one wants an error.

I want to be able to clone a super project, a subproject,
and use my copy of both instead of the original - including
cloning my copy, pulls between such clones, being able to verify
that they are identical.

What I *don't* want is a situation where the fact that original repository
resides in north america necessarily means that everyone who looks at *my* clone
of it will do a round trip to north america too.

And this means that URLs must be out of tree, but does *not* mean that
git-daemon should not serve them for user's convenience.

How about an ability for git-daemon to get commands with git-config?

-- 
MST

Re: [3/4] What's not in 1.5.2 (new topics)

[Josef Weidendorfer]

On Friday 18 May 2007, Michael S. Tsirkin wrote:
> > Quoting Andy Parkins <andyparkins@gmail.com>:
> > Subject: Re: [3/4] What's not in 1.5.2 (new topics)
> > 
> > On Friday 2007 May 18, Josef Weidendorfer wrote:
> > 
> > > It all depends on how we construct the default URL out of the subproject
> > > identifier. Options:
> > > (1) do not try to construct a default URL at all. Error out without a
> > > config (2) use a configurable rewriting scheme like s/(.*)/git://host/\1/
> > > (3) automatically detect a senseful rewriting scheme
> > >
> > > Let's start with (1). We can invent convenient default schemes later on.
> > 
> > All good; except let's start with 
> > 
> >  (1) if no config, try using the key itself - error out if that fails
> > 
> > Then everybody is happy - if you want to use your system where the key is not 
> > a URL, then don't - you'll get the error you want.  If the user chose to use 
> > a URL then magic will happen.
> 
> I don't want an error. No one wants an error.

Heh.
Of course, a git-clone should come up with a URL in the local config
such that subproject clones can happen without any an error.

The error would potentially happen after cloning if no config entry
for the URL can be found, e.g. when you decided at clone time to not
fetch any subprojects, but do that later.

> I want to be able to clone a super project, a subproject,
> and use my copy of both instead of the original - including
> cloning my copy, pulls between such clones, being able to verify
> that they are identical.

What about using user-global git configuration for this *before*
git-clone of the superprojects happens?

Lets say you have a clone of the linux-2.6 repository at ~/gitrepo/linux26,
and you want to clone a superproject which includes linux-2.6 as subproject,
using "linux26" (or perhaps "git://git.kernel.org/pub/linux-2.6.git") as
subproject identifier.
Before cloning this superproject, you should be able to set up in ~/.gitconfig

 [project "linux26"]
   localurl = ~/gitrepo/linux26

and the clone of the superproject should be able to use this repository as
subproject repository.

> What I *don't* want is a situation where the fact that original repository
> resides in north america necessarily means that everyone who looks at *my* clone
> of it will do a round trip to north america too.

Someone which clones from you probably does not have access to "~/gitrepo/linux26",
so you have to provide a public visible URL either way, like

 [project "linux26"]
   localurl = ~/gitrepo/linux26
   url = git://myhost/mylinux26.git

and the "url" should be configured at the remote side at clone time.

Josef

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

> > What I *don't* want is a situation where the fact that original repository
> > resides in north america necessarily means that everyone who looks at *my* clone
> > of it will do a round trip to north america too.
> 
> Someone which clones from you probably does not have access to "~/gitrepo/linux26",
> so you have to provide a public visible URL either way, like
> 
>  [project "linux26"]
>    localurl = ~/gitrepo/linux26
>    url = git://myhost/mylinux26.git
> 
> and the "url" should be configured at the remote side at clone time.

Right. In other words, urls for each project are unversioned file, and git-daemon
needs to be taught to serve them up (I think it already serves head names, which are
unversioned too). Actually, can't something like what we do for remotes head
names work here as well?

-- 
MST

Re: [3/4] What's not in 1.5.2 (new topics)

[Aidan Van Dyk]

Michael S. Tsirkin wrote:

>> Quoting Andy Parkins <andyparkins@gmail.com>:
>> Subject: Re: [3/4] What's not in 1.5.2 (new topics)
>> 
>> On Friday 2007 May 18, Josef Weidendorfer wrote:
>> 
>> > It all depends on how we construct the default URL out of the
>> > subproject identifier. Options:
>> > (1) do not try to construct a default URL at all. Error out without a
>> > config (2) use a configurable rewriting scheme like
>> > s/(.*)/git://host/\1/ (3) automatically detect a senseful rewriting
>> > scheme
>> >
>> > Let's start with (1). We can invent convenient default schemes later
>> > on.
>> 
>> All good; except let's start with
>> 
>>  (1) if no config, try using the key itself - error out if that fails
>> 
>> Then everybody is happy - if you want to use your system where the key is
>> not
>> a URL, then don't - you'll get the error you want.  If the user chose to
>> use a URL then magic will happen.
> 
> I don't want an error. No one wants an error.
> 
> I want to be able to clone a super project, a subproject,
> and use my copy of both instead of the original - including
> cloning my copy, pulls between such clones, being able to verify
> that they are identical.
> 
> What I *don't* want is a situation where the fact that original repository
> resides in north america necessarily means that everyone who looks at *my*
> clone of it will do a round trip to north america too.

Again - if *I* create a project, and decide to use a particular key for a
subproject, then good.  If *you* create a project and decide to use a
particular key for a subproject, then good. 

If you clone *my* superproject, you get *my* choice of key.  If I clone
*your* superproject, I get *your* choice of key.

The fact that I can choose a URL as my key is in no way influencing the fact
that you can choose to *not* use a URL for your key.

And if if you want to "copy" my project, but "change" the key for the
subproject, that's something you can too to!  In GIT, that's a branch.

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

> Again - if *I* create a project, and decide to use a particular key for a
> subproject, then good.  If *you* create a project and decide to use a
> particular key for a subproject, then good. 
> 
> If you clone *my* superproject, you get *my* choice of key.  If I clone
> *your* superproject, I get *your* choice of key.

Absolutely. What I object to is using this key in clients
as a hint for where to get the objects.
This *must* be overridable by me, without creating commits.

-- 
MST

Re: [3/4] What's not in 1.5.2 (new topics)

[Sven Verdoolaege]

On Fri, May 18, 2007 at 02:08:04PM +0300, Michael S. Tsirkin wrote:
> How about an ability for git-daemon to get commands with git-config?

You mean something like dump-config ?

skimo

Re: [3/4] What's not in 1.5.2 (new topics)

[Jakub Narebski]

Sven Verdoolaege wrote:

> On Fri, May 18, 2007 at 02:08:04PM +0300, Michael S. Tsirkin wrote:
>> How about an ability for git-daemon to get commands with git-config?
> 
> You mean something like dump-config ?

Why not use tags, e.g. refs/tags/config tag to blob, or lightweight
tag to blob?

-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Re: [3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

Josef Weidendorfer <Josef.Weidendorfer@gmx.de> writes:

> On Friday 18 May 2007, Andy Parkins wrote:
>> Bear in mind that what you're suggesting is no different in implementation 
>> >from what Junio is suggesting but with one difference: in Junio's option 
>> the "identifier" will act as a default URL if no override is found.
>
> Yes; actually, its exactly the same aside from the name used in .gitmodules
> for it, as I proposed a default URL which is derived from the suproject identifier
> if no config entry is found.
>
>> > Again, we could have a default URL in the absence of this config entry
>> > which is relative to the URL of the superproject, and which allows for the
>> > superproject repository to act as proxy.
>> 
>> This is why Junio's option of URL=Key is better.

There is one thing that three-level thing Steven Grimm suggested
solves cleaner that my strawman would not.

If the superproject is about building a live-cd that hosts two
different systems, one BSD and the other Linux, you would have:

	kernel-bsd/ subproject pointing at http://some.bsd.org/
        kernel-linux/ pointing at http://www.kernel.org/kernel/

Now suppose kernel.org people forgot to renew the domain
registration and BSD people are quick to react, takes over the
nice "kernel.org" domain, and now the latter URL becomes a site
about the BSD kernel---what happens? ;-) URL=Key scheme uses the
URL as the key so newer kernel-bsd/ subproject may be pointed at
by http://www.kernel.org/kernel/ in the superproject after such
a transition.  Linux kernel wouldn't cease to be served, so
in your .git/config you will a map to tell git to fetch from the
new location http://www.linux-kernel.org/kernel, but what key
would you use?  Using http://www.kernel.org/kernel/ would not be
correct as it would work only for older parts of the history.
Newer part of the history would want to map the same URL (=Key)
to http://www.kernel.org/kernel which is now hosting the BSD
kernel.

In short, you would need to be able to express the "intent" in
the .gitmodules and say "I am talking about the Linux kernel
with this entry", and if the same URL used for that purpose is
ever retargetted to house something different that is also used
in your superproject, URL=Key scheme is screwed.

Also I would be lying if I said I do not like the three-level
thing --- that was one of the options I considered before the
URL=Key thing.  Steven's three-level thing does not have the
above problem (but there is one thing you need to be careful
about, which I'll mention at the end of this discussion).

> It all depends on how we construct the default URL out of the subproject
> identifier. Options:
> (1) do not try to construct a default URL at all. Error out without a config
> (2) use a configurable rewriting scheme like s/(.*)/git://host/\1/
> (3) automatically detect a senseful rewriting scheme
>
> Let's start with (1). We can invent convenient default schemes later on.

My preference is to stay at (1), and even if we are to do the
later steps, do it _always_ with confirmation from the user.

Fetching from a new URL (not just "different from what is
defined in .gitmodules") is a major deal from security point of
view (you should not fetch from stranger you do not trust).

There is one thing that I sense was misunderstood by some people
about my original strawman.  Entries in .git/config are _not_
used as an override.  They _are_ the only thing that are used.

Let's forget the above "BSD takes over kernel.org" example,
which was a tongue-in-cheek, and go back to the original
appliance release #1 and #2 uses kernel 2.4 and 2.6 example.

When you see this in .gitmodule:

 	[subproject "kernel/"]
         	URL = git://git.kernel.org/pub/linux-2.4.git
 
you can be in three states:

 (1) You haven't known about this subproject's URL.

 (2) You have already known about this subproject, and you
     earlier decided not to clone it nor check it out (i.e. in
     your working tree, kernel/ subdirectory is left empty).
     By default, you do not want to be bothered by this
     subproject.

 (3) You have known about this subproject, and you earlier
     decided that you are interested in it.  You have a
     repository and working tree that represents the subproject
     checked out in your kernel/ subdirectory.  By default, you
     want to keep track of this subproject.

Obviously in the initial-clone case you can only be in state
(1).  After the initial clone, if the upstream changed the
kernel/ binding to point at 2.6 kernel tree, you are also in the
same state (1).

In these cases, since the upstream clearly states that they are
now talking about a project unknown to you so far, or talking
about a different project (it could be just a new location for
the same thing, the case Steven's three-level thing can help you
to differenciate with this), I DO NOT want git to automatically
say "Ok, that's the new location" and blindly start following.

An unattended pull (actually, the checkout step after a pull)
SHOULD error out.

An interactive case should give an easy way for the user to
express his preference: (a) I do not care about this subproject,
(b) I do want to have this cloned and checked out, and the
suggested URL in .gitmodules would work fine for me, or (c) I do
want to have this, but I want to use this other URL because I
have a local mirror already.

I wrote in my original strawman to have these two entries in the
.git/config file:

 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
         	URL = http://www.kernel.org/pub/linux-2.4.git
 
The example mapped the URL=Key to a different URL, which gave a
false impression that I was only talking about override, but
that was my fault.  My intention was to use the _presense_ of
subproject.$URL section in .git/config as a way to detect state
(1), so when the user says "I do want to follow this subproject
and the .gitmodules URL is Ok" (iow, choice (b) above), you
would have an identical "mapping" there:

 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
         	URL = git://git.kernel.org/pub/linux-2.4.git

That's not an "override".  Lack of these two lines does not mean
you will blindly follow kernel/ subproject using the URL
recorded in .gitmodules file; it means you haven't decided what
to do about the kernel/ subproject yet.

If the user wants to say "I am not interested" (iow, choice (a)
above), we would not have URL section, but explicit variable to
say "I am not interested" there, like this:

 	[subproject "git://git.kernel.org/pub/linux-2.4.git"]
         	ignored

Then the presense of this section tells us that we are not in
state (1) about this subproject.

The above can easily be rewritten to use Steven's three-level
scheme, which I tend to think would work better.  But if we were
to do that, I think the .git/config section should give you a
way to differentiate not just known/unknown subprojects but also
a way to differentiate known/unknown URLs.

IOW, .gitmodules in three-level scheme might say:

	[subproject "kernel/"]
        	name = linux-2.6
                URL = git://git.kernel.org/pub/linux-2.6.git

and your .git/config would say:

	[subproject "linux-2.6"]
                URL = http://www.kernel.org/pub/linux-2.6.git
                seen = git://git.kernel.org/pub/linux-2.6.git

so that when the upstream .gitmodules changed the suggested URL
to "git://git.or.cz/pub/linux-2.6.git", the UI can say:

	The project uses "linux-2.6" project at kernel/
	subdirectory as subproject, we already know that you are
	interested in tracking it, that you have been tracking
	it with http://www.kernel.org/pub/linux-2.6.git URL.
	The upstream suggests a new URL you haven't seen, which is
	"git://git.or.cz/pub/linux-2.6.git".  Do you want to
	adjust the URL to follow this subproject?

The user may say yes and tell git to use http:// instead, in
which case the section in .git/config would become:

	[subproject "linux-2.6"]
                URL = http://git.or.cz/pub/linux-2.6.git
                seen = git://git.kernel.org/pub/linux-2.6.git
		seen = git://git.or.cz/pub/linux-2.6.git

After that, if the upstream wags the entry .gitmodules back to
point at git.kernel.org/, you already know about that repository
and the UI does not have to ask you what to do about it.  That's
made possible by using URL=Key in my strawman, but needs to be
done by this extra 'seen' multi-value variables in the three-level
scheme.

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

> Fetching from a new URL (not just "different from what is
> defined in .gitmodules") is a major deal from security point of
> view (you should not fetch from stranger you do not trust).

I'm sorry, I'm confused. I thought the "URL" in .gitmodules
is just a unique project key/name? So how come you are now
speaking about fetching from it?

-- 
MST

Re: [3/4] What's not in 1.5.2 (new topics)

[Junio C Hamano]

"Michael S. Tsirkin" <mst@dev.mellanox.co.il> writes:

>> Fetching from a new URL (not just "different from what is
>> defined in .gitmodules") is a major deal from security point of
>> view (you should not fetch from stranger you do not trust).
>
> I'm sorry, I'm confused. I thought the "URL" in .gitmodules
> is just a unique project key/name? So how come you are now
> speaking about fetching from it?

Sorry for confusing you.  The point was by default that we
should not blindly follow URL given from upstream -- the
statement you quoted is one justification why my strawman uses
the URL in .gitmodules as a mere hint and look-up key.

Having said that, I'd ask not to take minor details in the
strawman too literally and seriously.  I am 100% sure that we
would be in a serious trouble if what we end up doing matches
literally what my handwaving strawman suggested.  The strawman
was thrown out to the open primarily so that (smarter and more
beautiful) people who thought the issues longer and harder to
express their opinions easier by having something to compare
their unique ideas against, nothing more.

I am slightly more than 50% sure that we would not want to tie
subproject fetch/clone into superproject fetch/clone, and _if_
we would tie it to anything, it would be to the checkout, but
that is only my gut feeling.  Maybe we end up not tying
subproject fetch/clone to anything that happens in the
superproject; we may even do it in a completely different way
than the strawman said it _might_ work.  That's perfectly fine.

The expectation from me sending out that handwaving strawman was
to help encouraging others to present their ideas, with
justifications.  And having something to compare against, even
if it is just a handwaving strawman, is often much easier when
presenting your ideas and showing which part of your design is
important.  You can say something like "the strawman fails in
this scenario, which is important in real life for such and such
reasons, and my design handles it this way" -- and everybody
can discuss if it is an important design consideration, and what
the best design to solve that problem if it is.

So don't take that strawman, especially the details in it, too
seriously, but take it as what it was: a firestarter.

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

...

> > As relative path I would propose $SUPERURL/subproject/$SUBPROJECTNAME, ie.
> > if the superproject is at git://git.kernel.org/pub/super.git, the above
> > subproject would default to the URL
> > git://git.kernel.org/pub/super.git/subproject/linux24 which could be a
> > symlink on the server.
> 
> I'm really uncomfortable with the idea of relying on directory structure 
> passed the root repository path; from the
>  git://git.kernel.org/pub/super.git/
> point onwards; we don't have any right to expect that this is a real directory 
> tree.  As an example; svn URLs don't match up with what's on disk:
> 
>  svn://svnhost/pub/repo/trunk/src
>                        ^^^^^^^^^^
> 
> On disk there is no such directory as /trunk/src under the repository 
> directory.  In the same way, even technically what you suggest would work, 
> the part of the URL under git://git.kernel.org/pub/super.git/ is git's own 
> namespace - it's not the users to mess with.  E.g. if I had a subproject 
> called "refs" you'd be in trouble.

Oh, that's easily solvable: just stick a 'subprojects' directory in there.
That is, the default URL to find a subproject would be:

1. For non-bare repo foo/.git/, subproject bar will live in foo/bar/.git
   or foo/bar.git.
2. For a bare repo foo.git/, subproject bar will live in
   foo.git/subprojects/bar.git.

> > > 2. Suppose .gitmodules in upstream tree points at subproject repo at
> > > kernel.org, and I clone from there - my repo will point at kernel.org by
> > > default? But now, I'd like everyone who clones from *my* repo to get
> > > pointed at *my* server by default (e.g. for mirroring),
> > > but would not changing .gitmodules create a commit so my
> > > head will now differ from upstream  - so it won't be signed properly
> > > etc... Did I misunderstand something?
> >
> > No, that is correct. Supporting a relative URL specification as proposed
> > above should solve this issue.
> 
> I think that's the wrong solution.  A change of source URL for a submodule 
> from what upstream uses to your own server is a _fork_ from upstream, 
> therefore you would fork your own branch in your supermodule and 
> alter .gitmodules to point at your server.  Everybody is happy, and the fork 
> is recorded.

Why should I record it? If the content is the same, the commit name should
be the same, it shouldn't matter where did the content came from.

I wouldn't be happy: I have just cloned both project and superproject,
but to re-publish the superproject using my clone of subproject, I have
to create a new commit, which would have a different hash from the origin.
So how do people know they can trust my tree?
And what happens when the original super-project pulls from me -
it seems that his .gitmodules will now point to my server?

> The override system is only there for the local repository (which always takes 
> precedence) not for the server provider to hide detail from those checking 
> the repo out.

I really like it that currently, in git, there is no difference between a public
and local repository.  If the override system is only for the local repository,
we create a difference here - doesn't this break the distributed nature of git?

Take offline work as an example:

So I have have cloned the supermodule and the submodule to my laptop -
it's enough to edit .git/config and I can use the history locally - that's good.
But now I try to clone the local tree - and a clone will try to go out
to the URL which I cloned - bad.

-- 
MST

Re: [3/4] What's not in 1.5.2 (new topics)

[Andy Parkins]

On Friday 2007 May 18, Michael S. Tsirkin wrote:

> > I think that's the wrong solution.  A change of source URL for a
> > submodule from what upstream uses to your own server is a _fork_ from
> > upstream, therefore you would fork your own branch in your supermodule
> > and alter .gitmodules to point at your server.  Everybody is happy, and
> > the fork is recorded.
>
> Why should I record it? If the content is the same, the commit name should
> be the same, it shouldn't matter where did the content came from.

Because you have changed something that the upstream repository supplied with 
no way of detecting it.  It's the same as if upstream supplied 
important_login_function.c and then you clone it; if your clone had a way of 
changing important_login_function.c to add a backdoor and passing that to 
people who clone from you without changing the commit hash that would be bad.

Submodules is the same; upstream might say
 kernel git://git.kernel.org/kernel-2.6.git
Then you clone it and use the override system to override that to
 kernel git://git.dodgykernel.org/backdoors.git
without having to change the repository.

The server should not be allowed to override the url that the client sees.  
Only the client should make that decision.

> I wouldn't be happy: I have just cloned both project and superproject,
> but to re-publish the superproject using my clone of subproject, I have
> to create a new commit, which would have a different hash from the origin.
> So how do people know they can trust my tree?

That problem exists regardless of the method of changing URL - in your method 
though the change is entirely unrecorded because you've changed something 
that upstream supplied in an out-of-band manner.

> And what happens when the original super-project pulls from me -
> it seems that his .gitmodules will now point to my server?

Now that one is a good defence.  Okay; I accept that changing .gitmodules 
won't work.  However, I don't accept that the server should be allowed to 
supply overrides to the client.  Another method is needed.

> > The override system is only there for the local repository (which always
> > takes precedence) not for the server provider to hide detail from those
> > checking the repo out.
>
> I really like it that currently, in git, there is no difference between a
> public and local repository.  If the override system is only for the local

Of course there is a difference.  .git/config is different; .git/refs is 
different; .git/info/exclude is different; etc.  These are all per-repository 
settings - and there is no way for a server to force it's version of those 
files on a client.

> So I have have cloned the supermodule and the submodule to my laptop -
> it's enough to edit .git/config and I can use the history locally - that's
> good. But now I try to clone the local tree - and a clone will try to go
> out to the URL which I cloned - bad.

Yep.  That is the problem.  In the end the only practical solution might be to 
allow the server to supply part of the .git/config (which is essentially what 
your suggestion would do); but I think that that is a big step to take and 
has potential to be abused.



Andy
-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [3/4] What's not in 1.5.2 (new topics)

[Johannes Sixt]

Andy Parkins wrote:
> On Friday 2007 May 18, Michael S. Tsirkin wrote:
> > I wouldn't be happy: I have just cloned both project and superproject,
> > but to re-publish the superproject using my clone of subproject, I have
> > to create a new commit, which would have a different hash from the origin.
> > So how do people know they can trust my tree?
> 
> That problem exists regardless of the method of changing URL - in your method
> though the change is entirely unrecorded because you've changed something
> that upstream supplied in an out-of-band manner.

And it doesn't matter: Once you trust the superproject with its
.gitmodules (versioned or not), the trust is based on the SHA1 of the
gitlink entry. Where the so named subproject commit came from is
secondary.

-- Hannes

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

> The server should not be allowed to override the url that the client sees.  
> Only the client should make that decision.

Why is that? Content is what is important.  URLs are only a convenience measure
to help clients find the content.  The link must have a commit hash, so git can
*verify* that the content is correct. Where it comes from must be irrelevant.

So if someone looks at my tree, and does not know where to get the content, he
might want my hint on this.


-- 
MST

Re: [3/4] What's not in 1.5.2 (new topics)

[Andy Parkins]

On Friday 2007 May 18, Michael S. Tsirkin wrote:

> Why is that? Content is what is important.  URLs are only a convenience
> measure to help clients find the content.  The link must have a commit
> hash, so git can *verify* that the content is correct. Where it comes from
> must be irrelevant.
>
> So if someone looks at my tree, and does not know where to get the content,
> he might want my hint on this.

True.  Hannes also pointed out that the trust comes from the hash contained in 
the gitlink that is in tree, there is no need to assign trust to the URL.

I withdraw my objection.


Andy

-- 
Dr Andy Parkins, M Eng (hons), MIET
andyparkins@gmail.com

Re: [3/4] What's not in 1.5.2 (new topics)

[Steven Grimm]

Andy Parkins wrote:
> Bear in mind that what you're suggesting is no different in implementation 
> from what Junio is suggesting but with one difference: in Junio's option 
> the "identifier" will act as a default URL if no override is found.
>   

I don't like using the URL as the key for one simple reason: while it 
technically doesn't conflate the two cases of "I want to use a different 
code base for this subproject starting in version X of the superproject" 
and "I want to use the same code base I've been using all along, but it 
has moved" (in that you can, as you point out, simply map the old URL to 
a new one independent of the project's history) it does encourage people 
to conflate the two in their minds.

Relatively few users will look at an identifier that is a valid URL and 
think of it as anything but a URL, especially if, in the absence of any 
overrides, the software (from the user's perspective) treats it as a 
URL. The override capability is almost certain to remain obscure since 
you won't need to use it in the normal case. Therefore, when the 
submodule's home gets moved to a different host, the first thing a lot 
of people are going to think to do is not to leave the submodule's 
identifier (the original URL) alone and create a mapping config entry, 
but rather to change the submodule to use a brand-new identifier that 
happens to be the same as the new URL. At which point you're right back 
to the original problem of checking out an old version of the 
superproject and having it point to a now-nonexistent subproject.

That's why I suggested making the identifiers look nothing like URLs, 
though of course to the extent they're arbitrary strings, one could use 
a URL if one chose to. I don't object to the *capability* of using a URL 
as an identifier in a three-level scheme like I described -- it would be 
silly to forbid -- but I think it would be a dangerous convention to 
establish because it will eventually encourage people to shoot 
themselves in the foot for lack of knowing what's actually going on.

-Steve

Re: [3/4] What's not in 1.5.2 (new topics)

[Josef Weidendorfer]

On Saturday 19 May 2007, Steven Grimm wrote:
> Andy Parkins wrote:
> > Bear in mind that what you're suggesting is no different in implementation 
> > from what Junio is suggesting but with one difference: in Junio's option 
> > the "identifier" will act as a default URL if no override is found.
> >   
> 
> I don't like using the URL as the key for one simple reason:
> ...

Another argument against naming the key for subprojects "URL" in
config/.gitmodules:
It can happen quite easily that a superprojects includes 2 subprojects
which really are only different branches of the same project, e.g.
GCC 4.1 and GCC 4.2 branch, e.g. to do regression testing with different
compiler versions.
But these two subprojects would be cloned from exactly the same URL.
So you artificially have to change one of the two URLs for this to
work, already at the start of your subproject.

The same example shows that the SHA1 of a projects root commit can not
work as a subproject key.

Josef

Re: [3/4] What's not in 1.5.2 (new topics)

[Petr Baudis]

On Fri, May 18, 2007 at 05:51:14PM CEST, Josef Weidendorfer wrote:
> On Friday 18 May 2007, Michael S. Tsirkin wrote:
> > > Subproject identifiers appear in versioned .gitmodule files, so
> > > they are fixed with the history of the project. You can not change
> > > the names without rewriting history.
> > 
> > Actually, I think this means that moving the
> > directory where the subproject resides will involve
> > editing .gitmodule. I sthat true?
> 
> Yes.
> This was a design decision by Linus. The alternative way
> would have been a separate gitlink object, which includes the
> commit SHA1 of the subproject _and_ a subproject identifier,
> thus increasing the number of objects.
> 
> The current way is far simpler, but you have to edit the .gitmodule
> file when moving subprojects around. The argument by Linus was
> that this inconvenience should be acceptable as moving subprojects
> around should only happen very few times in the lifetime of a
> project, and involves heavy rearranging either way, such that editing
> .gitmodules info is the smaller issue.

Furthermore, git mv can trivially take care of this anyway...

> > This would be annoying. 
> > 
> > Can't a project name be a git attribute for the gitlink object?
> 
> Of course. The name can be put into the .gitattributes file instead of
> a separate .gitmodules file.
> 
> > This way I can move the object and it keeps the project name.
> 
> No.
> Git attributes do not magically move around when you move files or
> directories. You always have to change the .gitattribute file too, if
> you want to move attributes with files. Of course, this is not needed
> if you use glob patterns in .gitattributes which also fits for moved
> files. 

Actually, git-mv might take care of this too? ;-) (Would it be
considered a Bad Thing, or should I whip up a patch?)

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

Re: [3/4] What's not in 1.5.2 (new topics)

[Michael S. Tsirkin]

> > The current way is far simpler, but you have to edit the .gitmodule
> > file when moving subprojects around. The argument by Linus was
> > that this inconvenience should be acceptable as moving subprojects
> > around should only happen very few times in the lifetime of a
> > project, and involves heavy rearranging either way, such that editing
> > .gitmodules info is the smaller issue.
> 
> Furthermore, git mv can trivially take care of this anyway...

Good idea.

-- 
MST

[4/4] What's not in 1.5.2 (other bits and pieces)

[Junio C Hamano]

Here are the leftover pieces I have in todo:TODO file that I did
not mention in the earlier messages.  I left them out from the
third message of this series, as (1) some are more of "trivial
fix" category, and (2) others are heavier and would probably not
fit for a single release cycle such as 1.5.3.

The easier ones
---------------

* parse-remote.sh has POSIXLY incorrect shell construct.

Message-ID: <20070505080313.GA12170@gondor.apana.org.au>

* Use 'git diff' not 'git diff-tree' in merge and rebase

From: James Bowes <jbowes@dangerouslyinc.com>
Message-ID: <1178398134288-git-send-email-jbowes@dangerouslyinc.com>

* gitk --left-right

From: Linus Torvalds <torvalds@linux-foundation.org>
Message-ID: <alpine.LFD.0.98.0705051524300.17381@woody.linux-foundation.org>
From: Junio C Hamano <junkio@cox.net>
Message-ID: <7vabwifl23.fsf@assigned-by-dhcp.cox.net>

* Handling pushing into non-bare repository more gracefully.

When git-push is done to a non-bare repository and updates the
branch that is currently checked out, we currently do not do
anything special.

From: Linus Torvalds <torvalds@linux-foundation.org>
Message-ID: <Pine.LNX.4.64.0704160931550.5473@woody.linux-foundation.org>

* git-daemon bug?

From: Franck Bui-Huu <vagabon.xyz@gmail.com>
Message-ID: <450EABD0.1040102@innova-card.com>

Repeated requests against git-daemon makes it stuck under --syslog

[jc: does not reproduce easily for me; has anybody seen it?]

* AsciiDoc 8 would break our documentation.

From: Stefan Richter <stefanr@s5r6.in-berlin.de>
Message-ID: <4523EC14.6070806@s5r6.in-berlin.de>

AsciiDoc 8 does not grok documents written for AsciiDoc 7 out of
the box.

[jc: limbo?]

* Delegate gitweb part to somebody else.

* Use gitattributes for more things.

 - 'precious' files that are not tracked but not
   build-products.  Currently people seem to put them in
   .gitignore, but that is not quite right, as .gitignore is
   meant for ignoring things that can be lost (build products,
   editor backup files).  "git clean -x" and "git checkout" to
   another branch that has a file where the current branch has a
   directory could lose such 'precious' files.

 - Customized "diff -p" markers per path (Johannes, on #git
   2007-04-30).

   I think it makes sense to give an extra parameter to xdiff
   machinery to affect how "diff -p" markers are constructed (as
   opposed to teach xdiff machinery to read gitattributes -- the
   code does not have path information at that level).  The
   simplest interface would be to pass a regexp and have the
   existing code always look for that regexp backwards.  A more
   complex one would involve a callback function, but I do not
   know if that kind of complexity is worth it.

 - Others???

* upload-pack support for start fetching from any valid point on
  the history, not just published refs. (Erik W. Biederman
  <m164jc9ekx.fsf@ebiederm.dsl.xmission.com>)

* Give --stdin to git-log, similar to git-rev-list

From: "Marco Costalba" <mcostalba@gmail.com>
Message-ID: <e5bfff550705110413q28aef3d8k3aeb0d342eeb2016@mail.gmail.com>

* Update the lockfile protocol so that closing and renaming are
  done inside lockfile commit time.  Some filesystems do not
  like an open file renamed and then closed.  Come up with a
  patch and pass Alex for an Ack.


Probably not so important ones
------------------------------

* daemon --strict-symlink.

* Maybe grok PGP signed text/plain in applymbox as well.

* Mbx (not mbox) support for git-mailsplit.

* git-proxy should be spawned with sh -c 'command' $1 $2.

[jc: should it? -- deciding if it should may not be "trivial",
but if it turns out to be the right thing to do, the change
itself is trivial.]

* Maybe a true git-proxy command that reads the first request
  pkt-line, and redirects the request to its real destination.

* test scripts for the relative directory path stuff.


The heavier ones
----------------

* Use blame machinery to track a single file (not path) in a finer
  grained way.

From: Linus Torvalds <torvalds@linux-foundation.org>
Message-ID: <alpine.LFD.0.98.0704201554550.9964@woody.linux-foundation.org>

[jc: I have a fixed-up one parked in 'pu' and also outlined what
other things I think are needed in my response:

    Message-ID: <7vwt06wqv8.fsf@assigned-by-dhcp.cox.net>
]

* "git fetch" should be able to use foreign SCM import backends
  such as svnimport and cvsimport.

* git-clone fails .git/refs/foo (Yann Dirson <ydirson@altern.org>)
  <20060610225040.GA7766@nowhere.earth>