Problem pushing to a Novell share

Problem pushing to a Novell share

[Rüdiger Kessel]

Hi

a push to an .git-repository on a Novell network-share using msysgit
fails with the following error message:

error: unable to create temporary sha1 filename : File exists

The problem has been found in git version 1.7.9

The problem is caused by a non-existing sub-directory in the file path
which is not automatically created on Novell shares.

A quick fix is to improve the create_tmpfile() function in sha1_file.c:

static int create_tmpfile(char *buffer, size_t bufsiz, const char *filename)
{
    static struct stat sb;
    int fd, dirlen = directory_size(filename);

    if (dirlen + 20 > bufsiz) {
        errno = ENAMETOOLONG;
        return -1;
    }

    memcpy(buffer, filename, dirlen);
    buffer[dirlen-1] = 0;
    if (stat(buffer, &sb) != 0 && errno == ENOENT) {
        if (mkdir(buffer, 0777) || adjust_shared_perm(buffer))
            return -1;
    }

    memcpy(buffer, filename, dirlen);
    strcpy(buffer + dirlen, "tmp_obj_XXXXXX");
    fd = git_mkstemp_mode(buffer, 0444);
    return fd;
}

The function will create one missing directory level if it does not
exist prior to creating the tmp-file.

This method seems to work, but it works only if one directory level is
missing which might not be the case in general. A better solution
would be to create the whole directory tree if needed.

The best solution would be if MINGW would handle this issue.

I recommend including this patch into GIT since it does no harm on
posix systems but it improves compatibility on others.

Rüdiger

Re: Problem pushing to a Novell share

[Junio C Hamano]

Rüdiger Kessel <ruediger.kessel@gmail.com> writes:

> A quick fix is to improve the create_tmpfile() function in sha1_file.c:
>
> static int create_tmpfile(char *buffer, size_t bufsiz, const char *filename)
> {
>     static struct stat sb;
>     int fd, dirlen = directory_size(filename);
>
>     if (dirlen + 20 > bufsiz) {
>         errno = ENAMETOOLONG;
>         return -1;
>     }
>
>     memcpy(buffer, filename, dirlen);
>     buffer[dirlen-1] = 0;
>     if (stat(buffer, &sb) != 0 && errno == ENOENT) {
>         if (mkdir(buffer, 0777) || adjust_shared_perm(buffer))
>             return -1;
>     }
>
>     memcpy(buffer, filename, dirlen);
>     strcpy(buffer + dirlen, "tmp_obj_XXXXXX");
>     fd = git_mkstemp_mode(buffer, 0444);
>     return fd;
> }
>
> The function will create one missing directory level if it does not
> exist prior to creating the tmp-file.

Please remove your modification above, and follow the code we have.  We
try git_mkstemp_mode() first, because it is more common than not having
the leading directory and we do not want to waste an extra stat() every
tie we come here.  Only when the first attempt fails by returning error
(i.e. fd < 0 and errno == ENOENT), we try mkdir() and then try again.

If you see "missing directory", it would be because on your system the
code somehow does not notice the failure from the first attempt.  Why?

The breakage you are seeing could come from either:

 - git_mkstemp_mode(), more specifically, the underlying mkstemp(), not
   returning negative when it fails for some reason; or

 - when the above fails, it fails to set errno to ENOENT.

and *that* is what needs to be fixed.

I think pessimizing create_tmpfile() like you did is not a fix; it is
sweeping the real problem under the rug.

Re: Problem pushing to a Novell share

[Junio C Hamano]

Rüdiger Kessel <ruediger.kessel@gmail.com> writes:

> Trying git_mkstemp_mode() first means trying more 16000 times to create a
> random file before realizing that something might be wrong.

Well, git_mkstemps_mode() does have such a 16k loop, and it tries to
create a unique, unused file this way:

		fd = open(pattern, O_CREAT | O_EXCL | O_RDWR, mode);
		if (fd > 0)
			return fd;
		/*
		 * Fatal error (EPERM, ENOSPC etc).
		 * It doesn't make sense to loop.
		 */
		if (errno != EEXIST)
			break;

If you do not have a directory D and try to create D/tmp_random here,
shouldn't you get an error that is _NOT_ EEXIST and trigger this break?

Re: Problem pushing to a Novell share

[Junio C Hamano]

[jc: just forwarding to the list --- the msysgit list might have been a
better place to forward to, but I know msysgit folks read this list too,
so...]

Rüdiger Kessel <ruediger.kessel@gmail.com> writes:

I have confimed that it is not a mingw problem. Direct calls to
CreateFile() in kernel32.dll show the same behaviour.

On a Novel share I get Error 80 "File exists" if a directory does not
exist, while I get Error 3 "Path not found" on a local drive

2012/2/3 Rüdiger Kessel <ruediger.kessel@gmail.com>:
 Well, it is a Novell bug, but what can we do?

 I studied it in detail and open() sets errno=EEXIST if the directory
 is missing on a Novell share causing git_mkstemp_mode looping 16k
 times and never creates the dir.

 The correct errno would be ENOENT, but that not what the systems is sending.

 So it is either using stat on the directory before trying to create
 the tmp-file or GIT cannot be used with Novell shares. What is it
 going to be?

 Protocolling the mkdir error handling I saw that git creates quite a
 number of new directories this way. Therefore my pessimistic approach
 might be too costly at the end.