The different EOL behavior between libgit2-based software and official Git

The different EOL behavior between libgit2-based software and official Git

[Yue Lin Ho]

Hi:

^_^

I did some test on the EOL behavior between official git and libgit2-based
software(TortoiseGit).
Then, I got that they have different EOL behavior.

The blob stored in repository is a text file with mixed EOLs.
Even set core.autocrlf = true, official git checkout the file as it is(means
still *mixed EOLs* there).
But, libgit2 checkout it with *All CRLF EOLs*.

 * The steps:
   * set core.autocrlf = false
   * add file with mixed EOLs
   * set core.autocrlf = true
   * delete that file in the working tree
   * checkout that file
   * examine the EOL

If you are interested in this, you might take a look at my testing
repository on GitHub.
(https://github.com/YueLinHo/TestAutoCrlf)

Thank you.

Yue Lin Ho



--
View this message in context: http://git.661346.n2.nabble.com/The-different-EOL-behavior-between-libgit2-based-software-and-official-Git-tp7613670.html
Sent from the git mailing list archive at Nabble.com.

Re: The different EOL behavior between libgit2-based software and official Git

[Torsten Bögershausen]

On 06/19/2014 04:59 AM, Yue Lin Ho wrote:
> Hi:
>
> ^_^
>
> I did some test on the EOL behavior between official git and libgit2-based
> software(TortoiseGit).
> Then, I got that they have different EOL behavior.
>
> The blob stored in repository is a text file with mixed EOLs.
> Even set core.autocrlf = true, official git checkout the file as it is(means
> still *mixed EOLs* there).
> But, libgit2 checkout it with *All CRLF EOLs*.
>
>   * The steps:
>     * set core.autocrlf = false
>     * add file with mixed EOLs
>     * set core.autocrlf = true
>     * delete that file in the working tree
>     * checkout that file
>     * examine the EOL
>
> If you are interested in this, you might take a look at my testing
> repository on GitHub.
> (https://github.com/YueLinHo/TestAutoCrlf)
>
> Thank you.
>
> Yue Lin Ho

(I send a similar mail to msysgit, I'm not sure if this came trough)

Sorry being late, I don't think there is something wrong with Git.
The core.autocrlf is the "old" crlf handling, which has been in Git for 
a long time.

If you exactly know what you are doing, know exactly
which tools are doing what, convince everybody who pulls or pushes to 
that repo to use
the same local config then it may be useful.

In short: I would strongly recommend to use gitattributes, please see 
below.


tb@msygit ~/temp
$ git clone https://github.com/YueLinHo/TestAutoCrlf.git
Cloning into 'TestAutoCrlf'...
[snip]
$ cd TestAutoCrlf/

tb@msygit ~/temp/TestAutoCrlf (master)
$ ls
CRLF.txt  LF.txt  MIX-more_CRLF.txt  MIX-more_LF.txt  Readme.md
###### Check how the file looks like:
$ od -c MIX-more_LF.txt
0000000   L   i   n   e       1  \n   l   i   n   e       (   2   ) \r
0000020  \n   l   i   n   e       3   .  \n   t   h   i   s i   s
0000040       l   i   n   e       4  \n   l       i       n       e
0000060   N   o   .       5  \n   L   i   n   e       N   u   m b e
0000100   r       6  \n
0000104
####### The file has one CRLF, the rest is LF, exactly how it had been
####### commited. So this is what we expect. Or do I miss something ?



####### Tell Git that MIX-more_LF.txt is a text file:
$ echo MIX-more_LF.txt text >.gitattributes

####### Verify that MIX-more_LF.txt is text, all other files
####### are "binary" (or "-text" in Git language)
$ git check-attr text *
CRLF.txt: text: unspecified
LF.txt: text: unspecified
MIX-more_CRLF.txt: text: unspecified
MIX-more_LF.txt: text: set
Readme.md: text: unspecified

####### Now ask Git to normalize the line endings in the working tree
$ rm  MIX-more_LF.txt

tb@msygit ~/temp/TestAutoCrlf (master)
$ git checkout   MIX-more_LF.txt

########## Check what we got:
tb@msygit ~/temp/TestAutoCrlf (master)
$ od -c MIX-more_LF.txt
0000000   L   i   n   e       1  \r  \n   l   i   n   e       ( 2 )
0000020  \r  \n   l   i   n   e       3   .  \r  \n   t   h   i   s
0000040   i   s       l   i   n   e       4  \r  \n   l i       n
0000060       e       N   o   .       5  \r  \n   L   i   n e N
0000100   u   m   b   e   r       6  \r  \n
0000111
######### (This is under Windows, under Linux I would expect only LF)
######### See core.eol for for information

##########
########## Now we need to normalize the file in the repo,
########## All line endings should be LF, otherwise Git
########## things the file is modified:
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.

Changes not staged for commit:
   (use "git add <file>..." to update what will be committed)
   (use "git checkout -- <file>..." to discard changes in working 
directory)

         modified:   MIX-more_LF.txt

Untracked files:
   (use "git add <file>..." to include in what will be committed)

         .gitattributes

no changes added to commit (use "git add" and/or "git commit -a")


###############
############### Do the normalization:
tb@msygit ~/temp/TestAutoCrlf (master)
$ git add MIX-more_LF.txt .gitattributes

tb@msygit ~/temp/TestAutoCrlf (master)
$ git commit -m  "MIX-more_LF.txt is text"
[master 200d874] MIX-more_LF.txt is text
  Committer: unknown <tb@msysgit>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:

     git config --global user.name "Your Name"
     git config --global user.email you@example.com

After doing this, you may fix the identity used for this commit with:

     git commit --amend --reset-author

  2 files changed, 2 insertions(+), 1 deletion(-)
  create mode 100644 .gitattributes

tb@msygit ~/temp/TestAutoCrlf (master)
$

######## From now on, MIX-more_LF.txt is treated as text by Git,
######## and get CRLF under Windows.
######## I think there is nothing wrong with Git here.
######## If libgit2 does something different, we need to ask
######## the libgit2 project, which is independent from Git

Re: The different EOL behavior between libgit2-based software and official Git

[Yue Lin Ho]

Hi Torsten Bögershausen:

Since you mail to msysGit first, I reply there.
(https://groups.google.com/forum/#!topic/msysgit/EDD3RipNeBQ)

Thank you again. ^_^

Yue Lin Ho



--
View this message in context: http://git.661346.n2.nabble.com/The-different-EOL-behavior-between-libgit2-based-software-and-official-Git-tp7613670p7613681.html
Sent from the git mailing list archive at Nabble.com.

Re: The different EOL behavior between libgit2-based software and official Git

[Torsten Bögershausen]

Hm,
I feeled puzzled here.
Even if I wouldn't recommend to use core.autocrlf, and prefer to use .gitattributes,
the CRLF conversion should work under Git, but it doensn't seem to do so.

Clone this repo:
origin  https://github.com/YueLinHo/TestAutoCrlf.git
Try to see if LF or CRLF can be converted into CRLF,
when core.autocrlf is true.


Neither msysgit nor Git under Linux produces CRLF (?)

Git under Mac OS produces the CRLF:
both Git 2.0.0  and the latest msygit code base (7e872d24a9bd03),
compiled under Mac OS

What do I miss ?

git --version
git version 2.0.0
tb@Linux:~/EOL_Test/TestAutoCrlf$ t=MIX-more_LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t  && od -c  $t
0000000   L   i   n   e       1  \n   l   i   n   e       (   2   )  \r
0000020  \n   l   i   n   e       3   .  \n   t   h   i   s       i   s
0000040       l   i   n   e       4  \n   l       i       n       e    
0000060   N   o   .       5  \n   L   i   n   e       N   u   m   b   e
0000100   r       6  \n

=============================================
$ git --version
git version 1.9.2.msysgit.0.1206.g7e872d2

tb@msgit ~/EOL_test/TestAutoCrlf (master)
$  t=MIX-more_LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t  && od -c  $t
0000000   L   i   n   e       1  \n   l   i   n   e       (   2   )  \r
0000020  \n   l   i   n   e       3   .  \n   t   h   i   s       i   s
0000040       l   i   n   e       4  \n   l       i       n       e
0000060   N   o   .       5  \n   L   i   n   e       N   u   m   b   e
0000100   r       6  \n

=============================================
tb@mac:~/EOL_Test/TestAutoCrlf> git --version
git version 2.0.0.622.g9478935
tb@mac:~/EOL_Test/TestAutoCrlf> t=MIX-more_LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t  && od -c  $t
0000000    L   i   n   e       1  \r  \n   l   i   n   e       (   2   )
0000020   \r  \n   l   i   n   e       3   .  \r  \n   t   h   i   s    
0000040    i   s       l   i   n   e       4  \r  \n   l       i       n
0000060        e       N   o   .       5  \r  \n   L   i   n   e       N
0000100    u   m   b   e   r       6  \r  \n                            

==============================================
tb@mac:~/EOL_Test/TestAutoCrlf> t=MIX-more_LF.txt  &&  rm -f $t &&  ~/projects/git/tb.msygit/git -c core.eol=CRLF checkout $t  && od -c  $t
0000000    L   i   n   e       1  \r  \n   l   i   n   e       (   2   )
0000020   \r  \n   l   i   n   e       3   .  \r  \n   t   h   i   s    
0000040    i   s       l   i   n   e       4  \r  \n   l       i       n
0000060        e       N   o   .       5  \r  \n   L   i   n   e       N
0000100    u   m   b   e   r       6  \r  \n                            

Re: The different EOL behavior between libgit2-based software and official Git

[Yue Lin Ho]

[-- Attachment #1: Type: text/plain, Size: 3942 bytes --]

Hi:


2014-06-20 14:56 GMT+08:00 Torsten Bögershausen <tboegi@web.de>:

> Hm,
> I feeled puzzled here.
> Even if I wouldn't recommend to use core.autocrlf, and prefer to use
> .gitattributes,
> the CRLF conversion should work under Git, but it doensn't seem to do so.
>
> Clone this repo:
> origin  https://github.com/YueLinHo/TestAutoCrlf.git
> Try to see if LF or CRLF can be converted into CRLF,
> when core.autocrlf is true.
>
>
> Neither msysgit nor Git under Linux produces CRLF (?)
>
> Git under Mac OS produces the CRLF:
> both Git 2.0.0  and the latest msygit code base (7e872d24a9bd03),
> compiled under Mac OS
>
> What do I miss ?
>
> git --version
> git version 2.0.0
> tb@Linux:~/EOL_Test/TestAutoCrlf$ t=MIX-more_LF.txt  &&  rm -f $t &&  git
> -c core.eol=CRLF checkout $t  && od -c  $t
> 0000000   L   i   n   e       1  \n   l   i   n   e       (   2   )  \r
> 0000020  \n   l   i   n   e       3   .  \n   t   h   i   s       i   s
> 0000040       l   i   n   e       4  \n   l       i       n       e
> 0000060   N   o   .       5  \n   L   i   n   e       N   u   m   b   e
> 0000100   r       6  \n
>
> =============================================
> $ git --version
> git version 1.9.2.msysgit.0.1206.g7e872d2
>
> tb@msgit ~/EOL_test/TestAutoCrlf (master)
> $  t=MIX-more_LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t
>  && od -c  $t
> 0000000   L   i   n   e       1  \n   l   i   n   e       (   2   )  \r
> 0000020  \n   l   i   n   e       3   .  \n   t   h   i   s       i   s
> 0000040       l   i   n   e       4  \n   l       i       n       e
> 0000060   N   o   .       5  \n   L   i   n   e       N   u   m   b   e
> 0000100   r       6  \n
>
> =============================================
> tb@mac:~/EOL_Test/TestAutoCrlf> git --version
> git version 2.0.0.622.g9478935
> tb@mac:~/EOL_Test/TestAutoCrlf> t=MIX-more_LF.txt  &&  rm -f $t &&  git
> -c core.eol=CRLF checkout $t  && od -c  $t
> 0000000    L   i   n   e       1  \r  \n   l   i   n   e       (   2   )
> 0000020   \r  \n   l   i   n   e       3   .  \r  \n   t   h   i   s
> 0000040    i   s       l   i   n   e       4  \r  \n   l       i       n
> 0000060        e       N   o   .       5  \r  \n   L   i   n   e       N
> 0000100    u   m   b   e   r       6  \r  \n
>
> ==============================================
> tb@mac:~/EOL_Test/TestAutoCrlf> t=MIX-more_LF.txt  &&  rm -f $t &&
>  ~/projects/git/tb.msygit/git -c core.eol=CRLF checkout $t  && od -c  $t
> 0000000    L   i   n   e       1  \r  \n   l   i   n   e       (   2   )
> 0000020   \r  \n   l   i   n   e       3   .  \r  \n   t   h   i   s
> 0000040    i   s       l   i   n   e       4  \r  \n   l       i       n
> 0000060        e       N   o   .       5  \r  \n   L   i   n   e       N
> 0000100    u   m   b   e   r       6  \r  \n
>
>
>
>
​Wow!

P.S.
libgit2 just has a PR that try to be identical with official git.
See https://github.com/libgit2/libgit2/pull/2432

Yue Lin Ho

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[-- Attachment #2: Type: text/html, Size: 6187 bytes --]

Re: [msysGit] Re: The different EOL behavior between libgit2-based software and official Git

[Torsten Bögershausen]

> ​Wow!
> 
> P.S.
> libgit2 just has a PR that try to be identical with official git.
> See https://github.com/libgit2/libgit2/pull/2432
> 
> Yue Lin Ho 
> 

I am not sure how much problems Git/libgit2 have with files contains mixed LF-CRLF,
as I have the same problem with the LF.txt

The handling, according to my understandig, is:
When core.eol is CRLF (or native under Windows) and core.autocrlf is true, and a file
is checked out:
  If a file has CRLF in one line in the repo, nothing is changed.
  If a file has LF in one line in the repo, LF is converted into CRLF in the workspace.

But here at my systems this doesn't seem to work as expected either for LF.txt:

tb@mac:~/EOL_Test/TestAutoCrlf>  t=LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t  && od -c  $t
0000000    L   i   n   e       1  \r  \n   l   i   n   e       (   2   )
0000020   \r  \n   l   i   n   e       3   .  \r  \n   t   h   i   s    
0000040    i   s       l   i   n   e       4  \r  \n   l       i       n
0000060        e       N   o   .       5  \r  \n   L   i   n   e       N
0000100    u   m   b   e   r       6  \r  \n  \r  \n                    
==================
tb@Linux:~/EOL_Test/TestAutoCrlf$ t=LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t  && od -c  $t
0000000   L   i   n   e       1  \n   l   i   n   e       (   2   )  \n
0000020   l   i   n   e       3   .  \n   t   h   i   s       i   s    
0000040   l   i   n   e       4  \n   l       i       n       e       N
0000060   o   .       5  \n   L   i   n   e       N   u   m   b   e   r
0000100       6  \n  \n

Re: Re: The different EOL behavior between libgit2-based software and official Git

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

tb@Linux:~/EOL_Test/TestAutoCrlf$ t=LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t  && od -c  $t
0000000   L   i   n   e       1  \n   l   i   n   e       (   2   )  \n
0000020   l   i   n   e       3   .  \n   t   h   i   s       i   s    
0000040   l   i   n   e       4  \n   l       i       n       e       N
0000060   o   .       5  \n   L   i   n   e       N   u   m   b   e   r
0000100       6  \n  \n

In Documentation/config.txt, we find:

    core.eol::
            Sets the line ending type to use in the working directory for
            files that have the `text` property set.  Alternatives are ...

Does that file $t in your practice "have the `text` property set"?

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [msysGit] Re: The different EOL behavior between libgit2-based software and official Git

[Torsten Bögershausen]

On 2014-06-20 18.33, Junio C Hamano wrote:
> Torsten Bögershausen <tboegi@web.de> writes:
> 
> tb@Linux:~/EOL_Test/TestAutoCrlf$ t=LF.txt  &&  rm -f $t &&  git -c core.eol=CRLF checkout $t  && od -c  $t
> 0000000   L   i   n   e       1  \n   l   i   n   e       (   2   )  \n
> 0000020   l   i   n   e       3   .  \n   t   h   i   s       i   s    
> 0000040   l   i   n   e       4  \n   l       i       n       e       N
> 0000060   o   .       5  \n   L   i   n   e       N   u   m   b   e   r
> 0000100       6  \n  \n
> 
> In Documentation/config.txt, we find:
> 
>     core.eol::
>             Sets the line ending type to use in the working directory for
>             files that have the `text` property set.  Alternatives are ...
> 
> Does that file $t in your practice "have the `text` property set"?
> 
No, it hadn't, under my Linux box.
(And I had a .gittatributes file on the Mac OS box, which I forgot about.
I am really sorry for the confusion and saying that Git behaves different under Linux and Mac OS).

You are pointing into the right direction:
Files with mixed LF CRLF in the repo are not changed by Git, when the checkout out
or checked in, unless the .gitattributes say that the file is text.

And libgit2 should do the same.


However, I was confused by this
https://www.kernel.org/pub/software/scm/git/docs/git-config.html
(My comments inline with ##)
-----------------------
core.autocrlf

    Setting this variable to "true" is almost the same as setting the text attribute to "auto" on all files except that text files are not guaranteed to be normalized: files that contain CRLF in the repository will not be touched. 
## And is this line still valid:
Use this setting if you want to have CRLF line endings in your working directory even though the repository does not have normalized line endings. 

## In 2010 the "the new safer autocrlf handling" was introduced,
## and it looks as if commits fd6cce9e and c4805393 are involved here.
## When the file in the repo has only LF, it will have CRLF in the work tree
## When the file in the repo has mixed LF and CRLF, it will not be changed in the work tree
## and will have mixed line endings in the work tree
## When the file in the repo has only CRLF, it will not be changed in the work tree
## and will have CRLF in the work tree as well as in the repo.

##Should this line simply be dropped in the documentation ?

Re: Re: The different EOL behavior between libgit2-based software and official Git

[Junio C Hamano]

Torsten Bögershausen <tboegi@web.de> writes:

> No, it hadn't, under my Linux box.
> (And I had a .gittatributes file on the Mac OS box, which I forgot about.

OK, that explains the difference; thanks for double-checking.

> Files with mixed LF CRLF in the repo are not changed by Git, when the checkout out
> or checked in, unless the .gitattributes say that the file is text.

Hmph, I would have thought, after reading the "Does anybody do that
crazy stuff" comment in convert.c, that we refrain from attempting
to convert a file with a mixed mess, even if the file is marked as
text, because it is unclear what it means to have both LF and CRLF
in a file that is text.  Is "A\rB\nC\r\n" a line terminated with a
CRLF that happens to have a lone CR and then LF in between?

You may be looking at a bug in a corner case that is so irrelevant
that nobody has even noticed, let alone attempted to fix.

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "msysGit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: The different EOL behavior between libgit2-based software and official Git

[Yue Lin Ho]

Wow!

P.S.
A note:
libgit2 just has a PR that try to be identical with official git 2.0.0.
See https://github.com/libgit2/libgit2/pull/2432



--
View this message in context: http://git.661346.n2.nabble.com/The-different-EOL-behavior-between-libgit2-based-software-and-official-Git-tp7613670p7613801.html
Sent from the git mailing list archive at Nabble.com.