From mboxrd@z Thu Jan 1 00:00:00 1970 From: Nick Townsend Subject: Re: [PATCH] submodule recursion in git-archive Date: Mon, 02 Dec 2013 16:00:50 -0800 Message-ID: <3651F1C2-741E-4170-9468-0EF07F120CB9@mac.com> References: <2E636B58-47EB-4712-93CA-39E8D1BA3DB9@mac.com> <5294BB97.7010707@web.de> <9AB10474-6DEF-4FFD-B6B3-ED2AB21424AC@mac.com> Mime-Version: 1.0 (Mac OS X Mail 7.0 \(1822\)) Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: =?windows-1252?Q?Ren=E9_Scharfe?= , Jens Lehmann , git@vger.kernel.org, Jeff King To: Junio C Hamano X-From: git-owner@vger.kernel.org Tue Dec 03 01:01:54 2013 Return-path: Envelope-to: gcvg-git-2@plane.gmane.org Received: from vger.kernel.org ([209.132.180.67]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VndR2-0006ij-6g for gcvg-git-2@plane.gmane.org; Tue, 03 Dec 2013 01:01:52 +0100 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751624Ab3LCABs convert rfc822-to-quoted-printable (ORCPT ); Mon, 2 Dec 2013 19:01:48 -0500 Received: from nk11p03mm-asmtpout002.mac.com ([17.158.232.237]:37434 "EHLO nk11p03mm-asmtp002.mac.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751615Ab3LCAB2 convert rfc822-to-8bit (ORCPT ); Mon, 2 Dec 2013 19:01:28 -0500 Received: from [172.16.1.4] (99-120-78-245.lightspeed.sntcca.sbcglobal.net [99.120.78.245]) by nk11p03mm-asmtp002.mac.com (Oracle Communications Messaging Server 7u4-27.08(7.0.4.27.7) 64bit (built Aug 22 2013)) with ESMTPSA id <0MX7000O3EQ8QU50@nk11p03mm-asmtp002.mac.com> for git@vger.kernel.org; Tue, 03 Dec 2013 00:01:21 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.11.72,1.0.14,0.0.0000 definitions=2013-12-02_04:2013-12-02,2013-12-02,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1308280000 definitions=main-1312020192 In-reply-to: X-Mailer: Apple Mail (2.1822) Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Archived-At: On 27 Nov 2013, at 11:43, Junio C Hamano wrote: > Nick Townsend writes: >=20 >> On 26 Nov 2013, at 14:18, Junio C Hamano wrote: >>=20 >>> Even if the code is run inside a repository with a working tree, >>> when producing a tarball out of an ancient commit that had a >>> submodule not at its current location, --recurse-submodules option >>> should do the right thing, so asking for working tree location of >>> that submodule to find its repository is wrong, I think. It may >>> happen to find one if the archived revision is close enough to what >>> is currently checked out, but that may not necessarily be the case. >>>=20 >>> At that point when the code discovers an S_ISGITLINK entry, it >>> should have both a pathname to the submodule relative to the >>> toplevel and the commit object name bound to that submodule >>> location. What it should do, when it does not find the repository >>> at the given path (maybe because there is no working tree, or the >>> sudmodule directory has moved over time) is roughly: >>>=20 >>> - Read from .gitmodules at the top-level from the tree it is >>> creating the tarball out of; >>>=20 >>> - Find "submodule.$name.path" entry that records that path to the >>> submodule; and then >>>=20 >>> - Using that $name, find the stashed-away location of the submodule >>> repository in $GIT_DIR/modules/$name. >>>=20 >>> or something like that. >>>=20 >>> This is a related tangent, but when used in a repository that peopl= e >>> often use as their remote, the repository discovery may have to >>> interact with the relative URL. People often ship .gitmodules with >>>=20 >>> [submodule "bar"] >>> URL =3D ../bar.git >>> path =3D barDir >>>=20 >>> for a top-level project "foo" that can be cloned thusly: >>>=20 >>> git clone git://site.xz/foo.git >>>=20 >>> and host bar.git to be clonable with >>>=20 >>> git clone git://site.xz/bar.git barDir/ >>>=20 >>> inside the working tree of the foo project. In such a case, when >>> "archive --recurse-submodules" is running, it would find the >>> repository for the "bar" submodule at "../bar.git", I would think. >>>=20 >>> So this part needs a bit more thought, I am afraid. >>=20 >> I see that there is a lot of potential complexity around setting up = a submodule: >=20 > No question about it. >=20 >> * The .gitmodules file can be dirty (easy to flag, but should we >> allow archive to proceed?) >=20 > As we are discussing "archive", which takes a tree object from the > top-level project that is recorded in the object database, the > information _about_ the submodule in question should come from the > given tree being archived. There is no reason for the .gitmodules > file that happens to be sitting in the working tree of the top-level > project to be involved in the decision, so its dirtyness should not > matter, I think. If the tree being archived has a submodule whose > name is "kernel" at path "linux/" (relative to the top-level > project), its repository should be at .git/modules/kernel in the > layout recent git-submodule prepares, and we should find that > path-and-name mapping from .gitmodules recorded in that tree object > we are archiving. The version that happens to be checked out to the > working tree may have moved the submodule to a new path "linux-3.0/" > and "linux-3.0/.git" may have "gitdir: .git/modules/kernel" in it, > but when archiving a tree that has the submodule at "linux/", it > would not help---we would not know to look at "linux-3.0/.git" to > learn that information anyway because .gitmodules in the working > tree would say that the submodule at path "linux-3.0/" is with name > "kernel", and would not tell us anything about "linux/". >=20 >> * Users can mess with settings both prior to git submodule init >> and before git submodule update. >=20 > I think this is irrelevant for exactly the same reason as above. >=20 > What makes this tricker, however, is how to deal with an old-style > repository, where the submodule repositories are embedded in the > working tree that happens to be checked out. In that case, we may > have to read .gitmodules from two places, i.e. >=20 > (1) We are archiving a tree with a submodule at "linux/"; >=20 > (2) We read .gitmodules from that tree and learn that the submodule > has name "kernel"; >=20 > (3) There is no ".git/modules/kernel" because the repository uses > the old layout (if the user never was interested in this > submodule, .git/modules/kernel may also be missing, and we > should tell these two cases apart by checking .git/config to > see if a corresponding entry for the "kernel" submodule exists > there); >=20 > (4) In a repository that uses the old layout, there must be the > repository somewhere embedded in the current working tree (this > inability to remove is why we use the new layout these days). > We can learn where it is by looking at .gitmodules in the > working tree---map the name "kernel" we learned earlier, and > map it to the current path ("linux-3.0/" if you have been > following this example so far). >=20 > And in that fallback context, I would say that reading from a dirty > (or "messed with by the user") .gitmodules is the right thing to > do. Perhaps the user may be in the process of moving the submodule > in his working tree with >=20 > $ mv linux-3.0 linux-3.2 > $ git config -f .gitmodules submodule.kernel.path linux-3.2 >=20 > but hasn't committed the change yet. >=20 >> For those reasons I deliberately decided not to reproduce the >> above logic all by myself. >=20 > As I already hinted, I agree that the "how to find the location of > submodule repository, given a particular tree in the top-level > project the submodule belongs to and the path to the submodule in > question" deserves a separate thread to discuss with area experts. As per my email to Heiko on this thread, I=92m happy to start such=20 a discussion - I=92ll use your notes as a starting point. I=92m much mo= re comfortable using a wiki for this - is this common or should I start a new mail thr= ead with RFC in the title or similar? I did complete my work on my version of git-archive (for internal use) = and added some regression tests for current behaviour. Also the add_submodule_odb patch should IMHO be = incorporated anyway. I=92ll resubmit those two for consideration in a new thread. Kind Regards Nick Townsend