git-fast-import doc problem and git-fast-export does not quote filenames correctly

git-fast-import doc problem and git-fast-export does not quote filenames correctly

[Michael Haggerty]

According to the git-fast-import manpage:

> A `<path>` string must use UNIX-style directory separators (forward
> slash `/`), may contain any byte other than `LF`, and must not
> start with double quote (`"`).
> 
> If an `LF` or double quote must be encoded into `<path>` shell-style
> quoting should be used, e.g. `"path/with\n and \" in it"`.
> 
> The value of `<path>` must be in canonical form. That is it must not:
> 
> * contain an empty directory component (e.g. `foo//bar` is invalid),
> * end with a directory separator (e.g. `foo/` is invalid),
> * start with a directory separator (e.g. `/foo` is invalid),
> * contain the special component `.` or `..` (e.g. `foo/./bar` and
>   `foo/../bar` are invalid).
> 
> It is recommended that `<path>` always be encoded using UTF-8.

The first problem is that the doc seems to allow NUL bytes (which I
suspect are not really allowed) or the backslash '\' character (which,
if allowed, would logically have to be escaped, too).

The second problem is that "git fast-export" does not do even the
specified quoting:

$ git init
Initialized empty Git repository in /home/mhagger/tmp/gitfoo/.git/
$ touch '"path with
 and " in it"'
$ git add -u
$ git commit -am 'Madness'
[master (root-commit) 2472bdb] Madness
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 "\"path with\n and \" in it\""
$ git fast-export --all
blob
mark :1
data 0

reset refs/heads/master
commit refs/heads/master
mark :2
author Michael Haggerty <mhagger@michael.localdomain> 1283580642 +0200
committer Michael Haggerty <mhagger@michael.localdomain> 1283580642 +0200
data 8
Madness
M 100644 :1 "path with
 and " in it"

reset refs/heads/master
from :2

Michael

Re: git-fast-import doc problem and git-fast-export does not quote filenames correctly

[Sverre Rabbelier]

On Sat, Sep 4, 2010 at 01:17, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> The first problem is that the doc seems to allow NUL bytes (which I
> suspect are not really allowed)

Fast import keeps reading until it finds a '\n' and stores data in a
strbuf, so I don't see why 'NUL' would not be allowed.

> or the backslash '\' character (which,
> if allowed, would logically have to be escaped, too).

Backslashes are allowed, and it seems they need to be escaped as
usual. Perhaps Shawn can confirm that this should just be updated in
the docs.

> The second problem is that "git fast-export" does not do even the
> specified quoting:

That's a bug with fast-export then :)

-- 
Cheers,

Sverre Rabbelier

Re: git-fast-import doc problem and git-fast-export does not quote filenames correctly

[Michael Haggerty]

Sverre Rabbelier wrote:
> On Sat, Sep 4, 2010 at 01:17, Michael Haggerty <mhagger@alum.mit.edu> wrote:
>> The first problem is that the doc seems to allow NUL bytes (which I
>> suspect are not really allowed)
> 
> Fast import keeps reading until it finds a '\n' and stores data in a
> strbuf, so I don't see why 'NUL' would not be allowed.

I guess I just found it hard to believe that git, with all of its
scripts, would really be able to handle NUL characters in filenames.
Moreover, even if git allows it, I think NUL is prohibited in posix
filenames, so it would be pretty pointless.

In the end, I asked this question to help figure out what filenames
"cvs2git" should allow through to git.  A NUL character could only get
into a CVS filename if the user writes some strange code to transform
the filename, so I guess I don't mind letting it through to git if there
is some chance that git can handle it.

> [...] Backslashes are allowed, and it seems they need to be escaped as
> usual. Perhaps Shawn can confirm that this should just be updated in
> the docs.

Thanks for the info!  Assuming that Shawn doesn't contradict you, I will
let any characters through and quote characters as you suggest.

Michael