Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Dmitry Bykov]

Hello,

Recently I installed 1.9.5 git version and faced the problem that one
of the files in my cloned repository with a name ICON~714.PNG is
marked as deleted even repository was freshly cloned. Trying to do
anything with that file resulted in constant "Invalid Path" errors.
Reverting back to 1.9.4. fixed that problem.

Thanks,
Dmitry

Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Stefan Beller]

On Wed, Jan 7, 2015 at 3:26 PM, Dmitry Bykov <pvrt74@gmail.com> wrote:
> Hello,
>
> Recently I installed 1.9.5 git version and faced the problem that one
> of the files in my cloned repository with a name ICON~714.PNG is
> marked as deleted even repository was freshly cloned. Trying to do
> anything with that file resulted in constant "Invalid Path" errors.
> Reverting back to 1.9.4. fixed that problem.
>
> Thanks,
> Dmitry

Git had a security issue with filenames which look similar to the .git
repository.
Please see the announcement at
http://article.gmane.org/gmane.linux.kernel/1853266
(That also updated 1.9.4 -> 1.9.5)

I'm not sure if I can advice though.

Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Junio C Hamano]

Dscho, this sounds to me like the additional "8.3 ambiguity
protection" (which is only in Git for Windows) in action. Any
thoughts?

On Wed, Jan 7, 2015 at 3:26 PM, Dmitry Bykov <pvrt74@gmail.com> wrote:
> Hello,
>
> Recently I installed 1.9.5 git version and faced the problem that one
> of the files in my cloned repository with a name ICON~714.PNG is
> marked as deleted even repository was freshly cloned. Trying to do
> anything with that file resulted in constant "Invalid Path" errors.
> Reverting back to 1.9.4. fixed that problem.
>
> Thanks,
> Dmitry
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Johannes Schindelin]

Hi Junio,

On Wed, 7 Jan 2015, Junio C Hamano wrote:

> Dscho, this sounds to me like the additional "8.3 ambiguity
> protection" (which is only in Git for Windows) in action. Any
> thoughts?

First thought: the Git for Windows mailing list should be Cc:ed (I was
traveling yesterday and somebody else might have been able to address
Dmitry's problem).

Second thought below:

> On Wed, Jan 7, 2015 at 3:26 PM, Dmitry Bykov <pvrt74@gmail.com> wrote:
> >
> > Recently I installed 1.9.5 git version and faced the problem that one
> > of the files in my cloned repository with a name ICON~714.PNG is
> > marked as deleted even repository was freshly cloned. Trying to do
> > anything with that file resulted in constant "Invalid Path" errors.
> > Reverting back to 1.9.4. fixed that problem.

ICON~714.PNG is a valid short name for a long name (such as
'icon.background.png') because it fits the shortening scheme (8.3 format,
the base name ends in ~<n>). As this can clash with a validly shortened
long name, Git for Windows refuses to check out such paths by default.

If you want the old -- unsafe -- behavior back, just set your
core.protectNTFS to false (this means that you agree that the incurred
problems are your own responsibility and cannot be blamed on anybody else
;-))

Ciao,
Johannes

Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Jeff King]

On Thu, Jan 08, 2015 at 11:06:18AM +0100, Johannes Schindelin wrote:

> ICON~714.PNG is a valid short name for a long name (such as
> 'icon.background.png') because it fits the shortening scheme (8.3 format,
> the base name ends in ~<n>). As this can clash with a validly shortened
> long name, Git for Windows refuses to check out such paths by default.
> 
> If you want the old -- unsafe -- behavior back, just set your
> core.protectNTFS to false (this means that you agree that the incurred
> problems are your own responsibility and cannot be blamed on anybody else
> ;-))

I wonder if it is worth having a "git-only" mode for core.protectNTFS.
Turning it off entirely would make him susceptible to GIT~1 attacks.

-Peff

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Git for Windows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Johannes Schindelin]

Hi Peff,

On Thu, 8 Jan 2015, Jeff King wrote:

> On Thu, Jan 08, 2015 at 11:06:18AM +0100, Johannes Schindelin wrote:
> 
> > ICON~714.PNG is a valid short name for a long name (such as
> > 'icon.background.png') because it fits the shortening scheme (8.3 format,
> > the base name ends in ~<n>). As this can clash with a validly shortened
> > long name, Git for Windows refuses to check out such paths by default.
> > 
> > If you want the old -- unsafe -- behavior back, just set your
> > core.protectNTFS to false (this means that you agree that the incurred
> > problems are your own responsibility and cannot be blamed on anybody else
> > ;-))
> 
> I wonder if it is worth having a "git-only" mode for core.protectNTFS.
> Turning it off entirely would make him susceptible to GIT~1 attacks.

That is a good idea!

Ciao,
Dscho

Re: Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Torsten Bögershausen]

On 01/08/2015 11:28 AM, Jeff King wrote:
> On Thu, Jan 08, 2015 at 11:06:18AM +0100, Johannes Schindelin wrote:
>
>> ICON~714.PNG is a valid short name for a long name (such as
>> 'icon.background.png') because it fits the shortening scheme (8.3 format,
>> the base name ends in ~<n>). As this can clash with a validly shortened
>> long name, Git for Windows refuses to check out such paths by default.
>>
>> If you want the old -- unsafe -- behavior back, just set your
>> core.protectNTFS to false (this means that you agree that the incurred
>> problems are your own responsibility and cannot be blamed on anybody else
>> ;-))
> I wonder if it is worth having a "git-only" mode for core.protectNTFS.
> Turning it off entirely would make him susceptible to GIT~1 attacks.
>
> -Peff
>
There is something more then just the "tilde" protection going on, it seems:


torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$  ~/projects/git/msysgit/git ls-files
ICON~714.PNG

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$  ~/projects/git/msysgit/git status
On branch master
nothing to commit, working directory clean

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$  ~/projects/git/tb/git status
On branch master
nothing to commit, working directory clean

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$  ~/projects/git/msysgit/git --version
git version 1.9.2.msysgit.0.2273.gc47d6ec

torstenbogershausen@TORBOGWM ~/projects/test_tilde (master)
$  ~/projects/git/tb/git --version
git version 2.2.1.268.g1e6f5b2

Neither latest msygit nor latest git.git reports a problem with a single 
file
within a short path.
It may be, that the '~' is not accepted in a long path or there are 
"problems"
when a directory is filled with many files, but that is speculation.

Dimitry,
is there a way to make the problem reproducible for other people ?
Either a public demo-repo, or a step-by-step receipe  ?

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Git for Windows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [msysGit] Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Johannes Schindelin]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 461 bytes --]

Hi Torsten,

On Thu, 8 Jan 2015, Torsten Bögershausen wrote:

> There is something more then just the "tilde" protection going on, [...]

Indeed. What is going on is that you build Git yourself, from git.git,
while Dmitry obviously used Git for Windows -- which carries a couple of
patches on top of upstream git.git.

In this particular case, the tilde protection was introduced in
https://github.com/msysgit/git/commit/2e2a2d12.

Ciao,
Johannes

Re: Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Torsten Bögershausen]

On 01/08/2015 04:58 PM, Johannes Schindelin wrote:
> Hi Torsten,
> 
> On Thu, 8 Jan 2015, Torsten Bögershausen wrote:
> 
>> There is something more then just the "tilde" protection going on, [...]
> 
> Indeed. What is going on is that you build Git yourself, from git.git,
> while Dmitry obviously used Git for Windows -- which carries a couple of
> patches on top of upstream git.git.
> 
> In this particular case, the tilde protection was introduced in
> https://github.com/msysgit/git/commit/2e2a2d12.
> 
> Ciao,
> Johannes
> 
My bad, I was building the msysgit master branch, which didn't have the 2e2a2d12.

However, I am wondering if 2e2a2d12 is really needed that much ?
In the same spirit that we do not prevent the checkout of "FILE" when "file" is present,
why do we prevent people from using "~" in file names ?
When a file is visible under different file names, I assume that the user chooses only
one representation to run "git add" on.
But that is another discussion.

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Git for Windows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Thu, Jan 08, 2015 at 11:06:18AM +0100, Johannes Schindelin wrote:
>
>> ICON~714.PNG is a valid short name for a long name (such as
>> 'icon.background.png') because it fits the shortening scheme (8.3 format,
>> the base name ends in ~<n>). As this can clash with a validly shortened
>> long name, Git for Windows refuses to check out such paths by default.
>> 
>> If you want the old -- unsafe -- behavior back, just set your
>> core.protectNTFS to false (this means that you agree that the incurred
>> problems are your own responsibility and cannot be blamed on anybody else
>> ;-))
>
> I wonder if it is worth having a "git-only" mode for core.protectNTFS.
> Turning it off entirely would make him susceptible to GIT~1 attacks.

Yes, I think having distinctions would make sense, but I am not sure
if "git-only" vs "every questionable included" should be the
classification.

To classify various funniness of paths by the extent of damage they
may cause for Windows:

 - ".Git" and "GIT~1" would be the worst level; 

 - "CON.c" and the like would be the next bad level; and

 - "ICON~714.PNG" would be in "to be avoided in an ideal world but
   as long as the project is comfortable with having it in, Git is
   in no position to complain" level.

that is just my knee-jerk reaction.

And if we were to go with just two level, I'd throw "CON.c" into the
same level as ".Git"; they both are to break the host that checks
out such paths.  The former may break Git on the host, and the
latter may break something that is not Git on the host, but they are
not that different in that the end user on that host is harmed.

"8.3 ambiguity" does not directly harm individual hosts, but the
reason to flag is primarily because such a path may make the
project's contents unreliable (e.g. "icon1234234.png" may alias
"ICON~714.PNG" on somebody's machine, causing confusion).  The same
reason should make us flag a tree object as suspect if it contains
two paths that are the same case-insensitively (e.g. a tree that
represents the "net/netfilter" part of the Linux kernel source would
be flagged because it contains both "xt_tcpmss.c" and "xt_TCPMSS.c").

Let's call the former Harmful and the latter Questionable.  "CON.c"
is Harmful on Windows, while "net/netfilter" is Questionable on
Windows.  Orthogonal to this, we earlier discussed what to do in
fsck and receive.fsckObjects.  I think we want to be express
different shades of grays between the two extremes:

 - A repository of a mono-culture project would want to flag
   Harmfuls and Questionables that apply to the project's intended
   platform as errors.  By definition, a mono-culture project would
   not care about paths that may only be Harmful on others'
   platforms.

 - A repository for a cross-platform project would want to flag
   paths that may be Harmful or Questionable on some platform, and
   does not care if these paths are Harmful or Questionable on the
   platform that happens to host the repository.  By definition, a
   cross-platform project wants to make sure everybody involved will
   not get hurt.

Re: Probably a bug with "~" symbol in filenames on Windows 7 x64 in git 1.9.5

[Jeff King]

On Thu, Jan 08, 2015 at 11:08:47AM -0800, Junio C Hamano wrote:

> > I wonder if it is worth having a "git-only" mode for core.protectNTFS.
> > Turning it off entirely would make him susceptible to GIT~1 attacks.
> 
> Yes, I think having distinctions would make sense, but I am not sure
> if "git-only" vs "every questionable included" should be the
> classification.

Yeah, I agree with everything you said below (I should have said
"security-relevant" instead of "git-only" originally). I also agree that
there may be more than two classifications, but I do not know enough
about the severity of the bad effects of each type to judge. I'd leave
that to the folks on each platform.

-Peff

-- 
-- 
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.

You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en

--- 
You received this message because you are subscribed to the Google Groups "Git for Windows" group.
To unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.