[PATCH 0/2] submodule add + autocrlf + safecrlf

[PATCH 0/2] submodule add + autocrlf + safecrlf

[Brad King]

Hi Folks,

When 'git submodule add' uses 'git config' to create a
'.gitmodules' file it gets LF newlines that the subsequent
'git add --force .gitmodules' rejects if autocrlf and
safecrlf are both enabled.  This series adds a test and
proposes a fix that simply uses '-c core.safecrlf=false'
to disable safecrlf when adding '.gitmodules'.

I'm not excited by allowing a LF file in work tree that
has clearly been configured to prefer CRLF, but avoiding
that for .gitmodules is probably a separate issue.

-Brad

Brad King (2):
  submodule: Demonstrate failure to add with auto/safecrlf
  submodule: Tolerate auto/safecrlf when adding .gitmodules

 git-submodule.sh           |    2 +-
 t/t7400-submodule-basic.sh |   13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

-- 
1.7.10

[PATCH 1/2] submodule: Demonstrate failure to add with auto/safecrlf

[Brad King]

If 'core.autocrlf' and 'core.safecrlf' are both enabled then
'git submodule add' fails with an error such as

 fatal: LF would be replaced by CRLF in .gitmodules
 Failed to register submodule 'submod'

because it generates a '.gitmodules' file with LF newlines that are
rejected by 'git add' under this configuration.  Demonstrate this known
breakage with a new test in t7400-submodule-basic covering the case.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 t/t7400-submodule-basic.sh |   13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 81827e6..5eaeb04 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -43,6 +43,7 @@ test_expect_success 'setup - hide init subdirectory' '
 
 test_expect_success 'setup - repository to add submodules to' '
 	git init addtest &&
+	git init addtest-crlf &&
 	git init addtest-ignore
 '
 
@@ -98,6 +99,18 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
+test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
+	(
+		cd addtest-crlf &&
+		git config core.autocrlf true &&
+		git config core.safecrlf true &&
+		git submodule add "$submodurl" submod &&
+		echo ".gitmodules" >expect &&
+		git ls-files -- .gitmodules >actual &&
+		test_cmp expect actual
+	)
+'
+
 test_expect_success 'submodule add to .gitignored path fails' '
 	(
 		cd addtest-ignore &&
-- 
1.7.10

[PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

[Brad King]

Temporarily disable 'core.safecrlf' to add '.gitmodules' so that
'git add' does not reject the LF newlines we write to the file
even if both 'core.autocrlf' and 'core.safecrlf' are enabled.
This fixes known breakage tested in t7400-submodule-basic.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 git-submodule.sh           |    2 +-
 t/t7400-submodule-basic.sh |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-submodule.sh b/git-submodule.sh
index 5c61ae2..ed9a54a 100755
--- a/git-submodule.sh
+++ b/git-submodule.sh
@@ -303,7 +303,7 @@ Use -f if you really want to add it." >&2
 
 	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
 	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
-	git add --force .gitmodules ||
+	git -c core.safecrlf=false add --force .gitmodules ||
 	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
 }
 
diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 5eaeb04..9a4da9b 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -99,7 +99,7 @@ test_expect_success 'submodule add' '
 	test_cmp empty untracked
 '
 
-test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
+test_expect_success 'submodule add with core.autocrlf and core.safecrlf' '
 	(
 		cd addtest-crlf &&
 		git config core.autocrlf true &&
-- 
1.7.10

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

[Jens Lehmann]

Am 20.06.2012 16:43, schrieb Brad King:
> Temporarily disable 'core.safecrlf' to add '.gitmodules' so that
> 'git add' does not reject the LF newlines we write to the file
> even if both 'core.autocrlf' and 'core.safecrlf' are enabled.
> This fixes known breakage tested in t7400-submodule-basic.

Hmm, I have no objections against the intention of the patch. But
as I understand it this message will reoccur when the user e.g.
edits the .gitmodules file later with any editor who just writes
lfs and adds it.

I don't know terribly much about crlf support but maybe flagging
the .gitmodules file in .gitattributes would be a better solution
here? Opinions?

> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  git-submodule.sh           |    2 +-
>  t/t7400-submodule-basic.sh |    2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/git-submodule.sh b/git-submodule.sh
> index 5c61ae2..ed9a54a 100755
> --- a/git-submodule.sh
> +++ b/git-submodule.sh
> @@ -303,7 +303,7 @@ Use -f if you really want to add it." >&2
>  
>  	git config -f .gitmodules submodule."$sm_path".path "$sm_path" &&
>  	git config -f .gitmodules submodule."$sm_path".url "$repo" &&
> -	git add --force .gitmodules ||
> +	git -c core.safecrlf=false add --force .gitmodules ||
>  	die "$(eval_gettext "Failed to register submodule '\$sm_path'")"
>  }
>  
> diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
> index 5eaeb04..9a4da9b 100755
> --- a/t/t7400-submodule-basic.sh
> +++ b/t/t7400-submodule-basic.sh
> @@ -99,7 +99,7 @@ test_expect_success 'submodule add' '
>  	test_cmp empty untracked
>  '
>  
> -test_expect_failure 'submodule add with core.autocrlf and core.safecrlf' '
> +test_expect_success 'submodule add with core.autocrlf and core.safecrlf' '
>  	(
>  		cd addtest-crlf &&
>  		git config core.autocrlf true &&
> 

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

[Brad King]

On 06/20/2012 01:52 PM, Jens Lehmann wrote:
> Am 20.06.2012 16:43, schrieb Brad King:
>> Temporarily disable 'core.safecrlf' to add '.gitmodules' so that
>> 'git add' does not reject the LF newlines we write to the file
>> even if both 'core.autocrlf' and 'core.safecrlf' are enabled.
>> This fixes known breakage tested in t7400-submodule-basic.
> 
> Hmm, I have no objections against the intention of the patch. But
> as I understand it this message will reoccur when the user e.g.
> edits the .gitmodules file later with any editor who just writes
> lfs and adds it.
> 
> I don't know terribly much about crlf support but maybe flagging
> the .gitmodules file in .gitattributes would be a better solution
> here? Opinions?

Once a user edits the file with an outside tool it is his/her
responsibility to add .gitattributes for the file.  In the reported
case Git is creating the file and already knows the crlf mode when
creating it.

I think Junio's proposal to teach "git config" to respect crlf
configuration is a more general solution.

-Brad

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

[Jens Lehmann]

Am 20.06.2012 20:06, schrieb Brad King:
> On 06/20/2012 01:52 PM, Jens Lehmann wrote:
>> Am 20.06.2012 16:43, schrieb Brad King:
>>> Temporarily disable 'core.safecrlf' to add '.gitmodules' so that
>>> 'git add' does not reject the LF newlines we write to the file
>>> even if both 'core.autocrlf' and 'core.safecrlf' are enabled.
>>> This fixes known breakage tested in t7400-submodule-basic.
>>
>> Hmm, I have no objections against the intention of the patch. But
>> as I understand it this message will reoccur when the user e.g.
>> edits the .gitmodules file later with any editor who just writes
>> lfs and adds it.
>>
>> I don't know terribly much about crlf support but maybe flagging
>> the .gitmodules file in .gitattributes would be a better solution
>> here? Opinions?
> 
> Once a user edits the file with an outside tool it is his/her
> responsibility to add .gitattributes for the file.  In the reported
> case Git is creating the file and already knows the crlf mode when
> creating it.
> 
> I think Junio's proposal to teach "git config" to respect crlf
> configuration is a more general solution.

Yep.

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

[Jeff King]

On Wed, Jun 20, 2012 at 02:06:43PM -0400, Brad King wrote:

> > Hmm, I have no objections against the intention of the patch. But
> > as I understand it this message will reoccur when the user e.g.
> > edits the .gitmodules file later with any editor who just writes
> > lfs and adds it.
> > 
> > I don't know terribly much about crlf support but maybe flagging
> > the .gitmodules file in .gitattributes would be a better solution
> > here? Opinions?
> 
> Once a user edits the file with an outside tool it is his/her
> responsibility to add .gitattributes for the file.  In the reported
> case Git is creating the file and already knows the crlf mode when
> creating it.
> 
> I think Junio's proposal to teach "git config" to respect crlf
> configuration is a more general solution.

I don't think so. It should not be an issue at all that .gitmodules has
CRLF or even mixed line endings; the config parser knows that its input
is a text file and ignores the CRs already.

The only problem is when adding the file, because git is not aware that
.gitmodules is, by definition, a text file.  The point of safecrlf was
to prevent accidents with files which are really binary, or for which
mixed line endings must be preserved; .gitmodules is not such a file. We
know that, but we never tell git.

We can paper over the problem by normalizing line endings when config
writes it out, but that does not cover the case of somebody editing it
manually and introducing mixed line endings. Nor does it help if
somebody checks in a .gitmodules file with CRLF, which we then checkout
on a LF-based system. Or if we do a merge between two versions with
different line endings.

The only sane thing is to have a canonical in-repo representation.
Fortunately we already have the infrastructure for that, and in theory
it should be as easy as adding ".gitmodules text" to our built-in
gitattributes (you could even do "eol=lf", but I don't see a reason not
to respect the native line endings in the working tree, given that git
can handle the CRLFs just fine).

I say "in theory" there because I am not sure whether specifying a file
as definitely text via attributes will actually suppress the safecrlf
check or not. IMHO, it should, since safecrlf is really about preventing
false positives via autocrlf or text=auto.

I don't see any reason for each individual repo to have to add these
attributes manually. This is a git-specific file, and the format is
dictated by git. We know that it's a text file, so why not help out the
user? We should possibly do the same thing for .gitattributes and
.gitignore.

-Peff

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> This is a git-specific file, and the format is
> dictated by git. We know that it's a text file, so why not help out the
> user? We should possibly do the same thing for .gitattributes and
> .gitignore.

OK.

Re: [PATCH 2/2] submodule: Tolerate auto/safecrlf when adding .gitmodules

[Jens Lehmann]

Am 20.06.2012 21:11, schrieb Jeff King:
> The only sane thing is to have a canonical in-repo representation.
> Fortunately we already have the infrastructure for that, and in theory
> it should be as easy as adding ".gitmodules text" to our built-in
> gitattributes (you could even do "eol=lf", but I don't see a reason not
> to respect the native line endings in the working tree, given that git
> can handle the CRLFs just fine).
> 
> I say "in theory" there because I am not sure whether specifying a file
> as definitely text via attributes will actually suppress the safecrlf
> check or not. IMHO, it should, since safecrlf is really about preventing
> false positives via autocrlf or text=auto.

A quick test shows that unfortunately theory differs from practice here.
Adding ".gitmodules text" to the built-in gitattributes lets the test
Brad wrote still fail. You have to use ".gitmodules eol=lf" to make it
pass. I stopped digging deeper at this point.

> I don't see any reason for each individual repo to have to add these
> attributes manually. This is a git-specific file, and the format is
> dictated by git. We know that it's a text file, so why not help out the
> user? We should possibly do the same thing for .gitattributes and
> .gitignore.

I really like this approach. (And in the long run would like to see a
ini-file aware merge driver being used for the .gitmodules file too,
which would just merge submodules added in different branches instead
of producing a conflict)

Re: [PATCH 0/2] submodule add + autocrlf + safecrlf

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> When 'git submodule add' uses 'git config' to create a
> '.gitmodules' file it gets LF newlines that the subsequent
> 'git add --force .gitmodules' rejects if autocrlf and
> safecrlf are both enabled.  This series adds a test and
> proposes a fix that simply uses '-c core.safecrlf=false'
> to disable safecrlf when adding '.gitmodules'.
>
> I'm not excited by allowing a LF file in work tree that
> has clearly been configured to prefer CRLF, but avoiding
> that for .gitmodules is probably a separate issue.

I have a suspicion that "git config" should be taught about this
kind of thing instead.

Shoudn't your .git/config file that is outside the revision control
also end with CRLF if your platform and project prefer CRLF over LF?

Re: [PATCH 0/2] submodule add + autocrlf + safecrlf

[Brad King]

On 06/20/2012 01:49 PM, Junio C Hamano wrote:
> I have a suspicion that "git config" should be taught about this
> kind of thing instead.
> 
> Shoudn't your .git/config file that is outside the revision control
> also end with CRLF if your platform and project prefer CRLF over LF?

That would be reasonable, but is beyond the scope I'm willing to
tackle myself.

I don't actually have a project like this so I have no strong
opinion on this issue.  I discovered the problem by accident
and have already worked around it in the obscure case it matters
for me.

Perhaps only the first patch in the series is worth inclusion.
It can become the beginning of a series if someone wants to address
handling crlf in config files.  Note that in the case I discovered
this the crlf configuration was in ~/.gitconfig so the project
knew nothing about it and had no .gitattributes.

-Brad

Re: [PATCH 0/2] submodule add + autocrlf + safecrlf

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> On 06/20/2012 01:49 PM, Junio C Hamano wrote:
>> I have a suspicion that "git config" should be taught about this
>> kind of thing instead.
>> 
>> Shoudn't your .git/config file that is outside the revision control
>> also end with CRLF if your platform and project prefer CRLF over LF?
>
> That would be reasonable, but is beyond the scope I'm willing to
> tackle myself.
>
> I don't actually have a project like this so I have no strong
> opinion on this issue.

If that is the case, my preference would be to wait until that "git
config" change happens (provided that it is a reasonably way
forward---it may turn out to be a bad idea) and do nothing to
clutter "git submodule" at all.  In the meantime, if there are real
projects that are hurt by this, we can tell them to explicitly
specify that .gitmodules is a LF text file in their .gitattributes.