"git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

"git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,
when I call "git clone ~adglth0/work/scripts whatever" as user
everything works as expected. As soon as I call this command as root it
shreds ufs:

        (s092942) [~] git version
        git version 1.5.3.GIT
        (s092942) [~] git clone ~adglth0/work/scripts nonroot
        Initialized empty Git repository in /export/home/adglth0/nonroot/.git/
        remote: Generating pack...
        Done counting 220 objects.
        Deltifying 220 objects...
        2% (5/220) doneIndexing 220 objects...
        100% (220/220) done20) donedonedone
        Total 220 (delta 114), reused 181remote:  (delta 88)
        100% (220/220) done
        Resolving 114 deltas...
        100% (114/114) done
        (s092942) [~] cd nonroot
        ./nonroot
        (s092942) [~/nonroot] git status
        # On branch master
        nothing to commit (working directory clean)
        (s092942) [~/nonroot] fg
        sudo ${SHELL} --rcfile ${HOME}/.bash_profile    (wd: ~)
        (s092942) [~] git clone ~adglth0/work/scripts root
        DO *NOT* RUN GIT AS ROOT clone /export/home/adglth0/work/scripts root
        (s092942) [~] unalias git
        (s092942) [~] git clone ~adglth0/work/scripts root
        Initialized empty Git repository in /export/home/adglth0/root/.git/
        remote: Generating pack...
        remote: Done counting 220 objects.
        Deltifying 220 objects...
        3% (7/220) doneIndexing 220 objects...
        100% (220/220) done20) done0) done
        Total 220 (delta 114), reused 181 (delta 88)
        100% (220/220) done
        Resolving 114 deltas...
        100% (114/114) done
        (s092942) [~] cd root
        (s092942) [~/root] git status
        # On branch master
        # Changed but not updated:
        #   (use "git add/rm <file>..." to update what will be committed)
        #
        #       deleted:    eclogic/acct_per_host.pl
        #       deleted:    eclogic/check_3ware
        #       deleted:    eclogic/check_ipmi.pl
        #       deleted:    eclogic/check_load.pl
        #       deleted:    eclogic/create_nagios_config.pl
        #       deleted:    eclogic/mailrelay/config-2.6.20
        #       deleted:    eclogic/mailrelay/interfaces
        #       deleted:    eclogic/on_all_hosts.pl
        #       deleted:    lifebook/kernel-config
        #       deleted:    s017241/config-2.6.20
        #       deleted:    s017241/firewall
        #       deleted:    s017241/siemens.conf
        #       deleted:    siemens/aliasupdate-tg
        #       deleted:    siemens/check_raid.sh
        #       deleted:    siemens/create_nagios_config.pl
        #       deleted:    siemens/crontab_client
        #       deleted:    siemens/deploy
        #       deleted:    siemens/generate_password.pl.gpg
        #       deleted:    siemens/mail_verrechnung.pl
        #       deleted:    siemens/oracle_beispiel_client.pl
        #       deleted:    siemens/sar_deploy.pl
        #       deleted:    siemens/t3_check_via_telnet.pl
        #       deleted:    thinkpad/config-2.6.20
        #       deleted:    thinkpad/fglrx.patch
        #       deleted:    thinkpad/network-please
        #       deleted:    thinkpad/xorg/build
        #       deleted:    webfarm/config-2.6.20
        #
        no changes added to commit (use "git add" and/or "git commit -a")
        (s092942) [~/root] git checkout -f HEAD
        (s092942) [~/root] git status
        # On branch master
        # Changed but not updated:
        #   (use "git add/rm <file>..." to update what will be committed)
        #
        #       deleted:    eclogic/acct_per_host.pl
        #       deleted:    eclogic/check_3ware
        #       deleted:    eclogic/check_ipmi.pl
        #       deleted:    eclogic/check_load.pl
        #       deleted:    eclogic/create_nagios_config.pl
        #       deleted:    eclogic/mailrelay/config-2.6.20
        #       deleted:    eclogic/mailrelay/interfaces
        #       deleted:    eclogic/sms.pl
        #       deleted:    lifebook/network-please
        #       deleted:    s017241/config-2.6.20
        #       deleted:    s017241/firewall
        #       deleted:    s017241/xorg.conf
        #       deleted:    siemens/aliasupdate-tg
        #       deleted:    siemens/check_raid.sh
        #       deleted:    siemens/create_nagios_config.pl
        #       deleted:    siemens/crontab_client
        #       deleted:    siemens/deploy
        #       deleted:    siemens/generate_password.pl.gpg
        #       deleted:    siemens/mail_verrechnung.pl
        #       deleted:    siemens/oracle_beispiel_client.pl
        #       deleted:    siemens/sar_deploy.pl
        #       deleted:    siemens/tryserver.sh
        #       deleted:    thinkpad/config-2.6.20
        #       deleted:    thinkpad/fglrx.patch
        #       deleted:    thinkpad/network-please
        #       deleted:    thinkpad/xorg/cleanup
        #       deleted:    webfarm/unsplit.pl
        #
        no changes added to commit (use "git add" and/or "git commit -a")
        (s092942) [~/root] ls
        eclogic/   leo*       lifebook/  s017241/   siemens/   thinkpad/  webfarm/
        (s092942) [~/root] ls siemens/
        t3_check_via_telnet.pl*
        (s092942) [~/root] cd ..
        (s092942) [~] rm -rf root/
        rm: Unable to remove directory root/: File exists
        (s092942) [~] find root/
        root/
        (s092942) [~] ls -al root/
        total 6
        drwxr-xr-x  56 root     root         512 Jul 16 11:48 ./
        drwxr-xr-x  17 adglth0  root        1536 Jul 16 11:48 ../
        (s092942) [~] df -h .
        Filesystem             size   used  avail capacity  Mounted on
        /dev/dsk/c0t0d0s7       75G   9.0G    65G    13%    /export/home
        (s092942) [~] ls -ali root/
        total 6
        13657 drwxr-xr-x  56 root     root         512 Jul 16 11:48 ./
        5829 drwxr-xr-x  17 adglth0  root        1536 Jul 16 11:48 ../
        (s092942) [~] gfind /export/home -xdev -inum 13657
        /export/home/adglth0/root
        (s092942) [~]

A few notes and observations I did. Linux _does_ not allow userland to
create hardlinks for directories which makes totally sense. Solaris does
allow the _root_ user to allow hardlinks for directories. There is even
a command that makes that possible:

        (faui04a) [/var/tmp] mkdir test
        (faui04a) [/var/tmp] /usr/xpg4/bin/link test bla
        (faui04a) [/var/tmp] ls -ali
        total 13
        1639 drwxrwxrwt    4 root     sys          9216 Jul 16 11:13 ./
        4 drwxr-xr-x   48 root     sys          1024 Jan  1  2007 ../
        238939 drwx--x--x    3 snalwuer icipguru      512 Jul  5 19:08 1071895580/
        238945 drwxr-xr-x    3 root     root          512 Jul 16 11:11 bla/
        238945 drwxr-xr-x    3 root     root          512 Jul 16 11:11 test/
        (faui04a) [/var/tmp] find . -inum 238945

Whatever. My first impression was that the filesystem of the solaris is
broken. So I fscked it. And guess what it corrected the error. But after
the reboot I tried it again and you see the result above. So my guess is
that git calls the link system call in a way Solaris thinks it should
itself shoot in the head. This also happens with _remote_ clones as
root:

        (s092942) [~] git clone adglth0@157.163.208.106:/home/adglth0/work/scripts second_test_as_root
        Initialized empty Git repository in /export/home/adglth0/second_test_as_root/.git/
        The authenticity of host '157.163.208.106 (157.163.208.106)' can't be established.
        RSA key fingerprint is 72:27:be:82:f7:71:e5:23:a6:b6:57:a9:11:dd:26:26.
        Are you sure you want to continue connecting (yes/no)? yes
        Warning: Permanently added '157.163.208.106' (RSA) to the list of known hosts.
        remote: Generating pack...
        remote: Done counting 220 objects.
        remote: Deltifying 220 objects...
        remote:  100% (220/220) done
        Indexing 220 objects...
        remote: Total 220 (delta 115), reused 37 (delta 9)
        100% (220/220) done
        Resolving 115 deltas...
        100% (115/115) done
        (s092942) [~] cd second_test_as_root/
        ./second_test_as_root/
        (s092942) [~/second_test_as_root] git status
        # On branch master
        # Changed but not updated:
        #   (use "git add/rm <file>..." to update what will be committed)
        #
        #       deleted:    eclogic/acct_per_host.pl
        #       deleted:    eclogic/check_3ware
        #       deleted:    eclogic/check_ipmi.pl
        #       deleted:    eclogic/check_load.pl
        #       deleted:    eclogic/create_nagios_config.pl
        #       deleted:    eclogic/mailrelay/config-2.6.20
        #       deleted:    eclogic/mailrelay/interfaces
        #       deleted:    eclogic/on_all_hosts.pl
        #       deleted:    lifebook/kernel-config
        #       deleted:    s017241/config-2.6.20
        #       deleted:    s017241/firewall
        #       deleted:    s017241/siemens.conf
        #       deleted:    siemens/aliasupdate-tg
        #       deleted:    siemens/check_raid.sh
        #       deleted:    siemens/create_nagios_config.pl
        #       deleted:    siemens/crontab_client
        #       deleted:    siemens/deploy
        #       deleted:    siemens/generate_password.pl.gpg
        #       deleted:    siemens/mail_verrechnung.pl
        #       deleted:    siemens/oracle_beispiel_client.pl
        #       deleted:    siemens/sar_deploy.pl
        #       deleted:    siemens/t3_check_via_telnet.pl
        #       deleted:    thinkpad/config-2.6.20
        #       deleted:    thinkpad/fglrx.patch
        #       deleted:    thinkpad/network-please
        #       deleted:    thinkpad/xorg/build
        #       deleted:    webfarm/config-2.6.20
        #
        no changes added to commit (use "git add" and/or "git commit -a")
        (s092942) [~/second_test_as_root] uname -a
        SunOS s092942 5.10 Generic_118833-17 sun4u sparc SUNW,Sun-Blade-100

So has anyone a idea what is wrong. I have shareded a second machine in
a complete different domain. With a complete different git version. Same
thing. So it is not the hardware and not the specific git version and
not the Solaris version:

        (faui04a) [/var/tmp] git clone ~sithglan/work/sc
        screen/                       scripts/                      scripts_ccbank_what_the_hell/
        (faui04a) [/var/tmp] git clone ~sithglan/work/repositories/
        private/ public/
        (faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/
        blastwave.git/ bs.git/        easix.git/     mutt.git/      nzb.git/       ra.git/        watchdog.git/
        (faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/easix.git/
        DO *NOT* RUN GIT AS ROOT clone /home/cip/adm/sithglan/work/repositories/public/easix.git/
        (faui04a) [/var/tmp] unalias git
        (faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/easix.git/
        Initialized empty Git repository in /var/tmp/easix/.git/
        remote: Generating pack...
        remote: Done counting 317 objects.
        remote: Deltifying 317 objects...
        remote: emote: 317/317) done: ) done17) done
        Indexing 317 objects...
        remote: Total 317 (delta 182), reused 278 (delta 157)
        100% (317/317) done
        Resolving 182 deltas...
        100% (182/182) done

        (faui04a) [/var/tmp] cd easix/
        ./easix/
        (faui04a) [/var/tmp/easix] git status
        # On branch master
        # Changed but not updated:
        #   (use "git add/rm <file>..." to update what will be committed)
        #
        #       deleted:    profiles/icpc
        #       deleted:    profiles/sithglan
        #       deleted:    sources/boot.msg
        #       deleted:    sources/busybox-config
        #       deleted:    sources/config
        #       deleted:    sources/easix.logo.16
        #       deleted:    sources/easixdm
        #       deleted:    sources/f2
        #       deleted:    sources/home-faumachine.tar.gz
        #       deleted:    sources/home-icpc.tar.gz
        #       deleted:    sources/home-simigern.tar.gz
        #       deleted:    sources/home-sithglan.tar.gz
        #       deleted:    sources/home-sitowert.tar.gz
        #       deleted:    sources/linuxrc
        #       deleted:    sources/rc.local
        #
        no changes added to commit (use "git add" and/or "git commit -a")
        (faui04a) [/var/tmp/easix] cd ..
        (faui04a) [/var/tmp] rm -rf easix
        rm: Unable to remove directory easix: File exists
        (faui04a) [/var/tmp] ls -ali easix/
        total 10
        238947 drwxr-xr-x   17 root     root          512 Jul 16 12:01 ./
        1639 drwxrwxrwt    5 root     sys          9216 Jul 16 12:01 ../
        (faui04a) [/var/tmp] git version
        git version 1.5.2
        (faui04a) [/var/tmp] uname -a
        SunOS faui04a 5.10 Generic_118833-24 sun4u sparc SUNW,Sun-Fire-280R


[ I am off the net for 30 mintues ... eating ]

                Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,
I narrowed it down. It is in "git checkout" if I copy the ".git" dir via
"cp -a" and type in "git checkout" it breaks. And it _only_ breaks for
subdirectories:

        (faui04a) [/var/tmp/shit] git checkout
        D       profiles/icpc
        D       profiles/sithglan
        D       sources/boot.msg
        D       sources/busybox-config
        D       sources/config
        D       sources/easix.logo.16
        D       sources/easixdm
        D       sources/f2
        D       sources/home-faumachine.tar.gz
        D       sources/home-icpc.tar.gz
        D       sources/home-simigern.tar.gz
        D       sources/home-sithglan.tar.gz
        D       sources/home-sitowert.tar.gz
        D       sources/linuxrc
        D       sources/rc.local

and I also see that _one_ file in each subdirectory is checked out but the
others are not.

        Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[David Kastrup]

Thomas Glanzmann <thomas@glanzmann.de> writes:

> Hello,
> I narrowed it down. It is in "git checkout" if I copy the ".git" dir via
> "cp -a" and type in "git checkout" it breaks. And it _only_ breaks for
> subdirectories:
>
>         (faui04a) [/var/tmp/shit] git checkout
>         D       profiles/icpc
>         D       profiles/sithglan
>         D       sources/boot.msg
>         D       sources/busybox-config
>         D       sources/config
>         D       sources/easix.logo.16
>         D       sources/easixdm
>         D       sources/f2
>         D       sources/home-faumachine.tar.gz
>         D       sources/home-icpc.tar.gz
>         D       sources/home-simigern.tar.gz
>         D       sources/home-sithglan.tar.gz
>         D       sources/home-sitowert.tar.gz
>         D       sources/linuxrc
>         D       sources/rc.local
>
> and I also see that _one_ file in each subdirectory is checked out but the
> others are not.

Ok, stupidly trying one of our most frequently encountered problems:

Does it help to say
export CDPATH=""
in your root shell?

-- 
David Kastrup

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello David,

> Does it help to say export CDPATH="" in your root shell?

no. The problem is triggered by "git-checkout-index -a -f" so my CDPATH has
absolutly _nothing_ todo with it:

        (faui04a) [/var/tmp] mkdir -p test-4/.git
        (faui04a) [/var/tmp] cd test-4
        (faui04a) [/var/tmp/test-4] gcp -a ~/work/repositories/public/easix.git/* .git/
        (faui04a) [/var/tmp/test-4] git-read-tree HEAD
        (faui04a) [/var/tmp/test-4] git-checkout-index -a -f
        (faui04a) [/var/tmp/test-4] git status
        fatal: /usr/bin/git-status cannot be used without a working tree.
        (faui04a) [/var/tmp/test-4] rm .git/config
        (faui04a) [/var/tmp/test-4] git status
        # On branch master
        # Changed but not updated:
        #   (use "git add/rm <file>..." to update what will be committed)
        #
        #       deleted:    profiles/icpc
        #       deleted:    profiles/sithglan
        #       deleted:    sources/boot.msg
        #       deleted:    sources/busybox-config
        #       deleted:    sources/config
        #       deleted:    sources/easix.logo.16
        #       deleted:    sources/easixdm
        #       deleted:    sources/f2
        #       deleted:    sources/home-faumachine.tar.gz
        #       deleted:    sources/home-icpc.tar.gz
        #       deleted:    sources/home-simigern.tar.gz
        #       deleted:    sources/home-sithglan.tar.gz
        #       deleted:    sources/home-sitowert.tar.gz
        #       deleted:    sources/linuxrc
        #       deleted:    sources/rc.local
        #
        no changes added to commit (use "git add" and/or "git commit -a")
        (faui04a) [/var/tmp/test-4] cd ..
        (faui04a) [/var/tmp] rm -rf test-4
        rm: Unable to remove directory test-4: File exists
        (faui04a) [/var/tmp]

                Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[David Kastrup]

Thomas Glanzmann <thomas@glanzmann.de> writes:

> Hello,
> I narrowed it down. It is in "git checkout" if I copy the ".git" dir via
> "cp -a" and type in "git checkout" it breaks. And it _only_ breaks for
> subdirectories:

Another guess: cp -a copies files preserving all permissions and
ownerships (which works only as root).  Nominally, the files now don't
belong to you, and some scripts might take you by the word even if
they _could_ just overwrite things if they actually tried.  Try
making

chown -R root.root .git

on your tree and see whether this makes git more comfortable.

-- 
David Kastrup

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello David,
as I mentioned and showed in my original e-mail:

        git checkout /path/to/repo

triggers the problem also. So no, "cp -a" has nothing todo with it
because root is able to read _all_ files no matter who the files are
owned by and what the permissions of these files are[1]. This happens on
a _local_ fileystem. And what worries me is that git does something that
breaks the _hard link counter_ of a directory.

        Thomas

[1] Notes
        Of course that doesn't apply to all filesystems. For example AFS
        and NFS. But as I said before it happens on a _UFS_ filesystem.

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[David Kastrup]

Thomas Glanzmann <thomas@glanzmann.de> writes:

> Hello David,
> as I mentioned and showed in my original e-mail:
>
>         git checkout /path/to/repo
>
> triggers the problem also. So no, "cp -a" has nothing todo with it
> because root is able to read _all_ files no matter who the files are
> owned by and what the permissions of these files are[1]. This happens on
> a _local_ fileystem. And what worries me is that git does something that
> breaks the _hard link counter_ of a directory.

Hm?  The hard link counter is "broken" by creating or not creating
subdirectories, since their ".." is a hard link.

-- 
David Kastrup

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello David,

> Hm?  The hard link counter is "broken" by creating or not creating
> subdirectories, since their ".." is a hard link.

exactly. The question is: Is it a Solaris bug or is it something that is
supposed a user is able to do (it doesn't make sense for me)? I posted
this problem to comp.unix.solaris and also contacted the UFS Maintainer
of Solaris (it is not the first UFS bug I original found). If I don't
receive feedback, I am going to open a call with Sun. Whatever is going
on (hopefully it isn't PEBKAC - but I don't think so) and this time it
isn't broken hardware either (like the last time I reported a serious
git bug) because it happens on two machines.

        Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,


> exactly. The question is: Is it a Solaris bug or is it something that is
> supposed a user is able to do (it doesn't make sense for me)? I posted
> this problem to comp.unix.solaris and also contacted the UFS Maintainer
> of Solaris (it is not the first UFS bug I original found). If I don't
> receive feedback, I am going to open a call with Sun. Whatever is going
> on (hopefully it isn't PEBKAC - but I don't think so) and this time it
> isn't broken hardware either (like the last time I reported a serious
> git bug) because it happens on two machines.

below I quote the answer of the UFS maintainer:

Thomas,

some notes on the unlink(directory) part:

In the case of calling unlink(2) on a directory as a privilidged user, things
like ".." (and the link count in the parent) would not be cleaned up.

The man page for unlink(1M) and unlink(2) is very clear about using rmdir(1/2) to
remove a directory.  It's also very clear that this does exactly what
it's supposed to do: Remove a link to a file/directory.  Cleaning up
the ".." entry (which would clean up the link to the parent) could
be considered "extra".

First the standards issues:
    The link(BA_OS) and unlink(BA_OS) descriptions in SVID3 both specify that
    a process with appropriate privileges is allowed to operate on a directory.
    We have claimed to conform to SVID3 since Solaris 2.0 and have not announced
    that we ever plan to EOL SVID3 conformance.

    UFS does support link(2)/unlink(2) with appropriate privilidges of
    directories while ZFS does not.

    The change that would provide ``Cleaning up the ".." entry ...'' would
    violate both SVID3 and POSIX and SUS requirements.  (The unlink(path)
    system call is supposed to remove the directory entry named by path; not
    the directory entry named by path and an arbitrary number of other
    directory entries.)

Second some history:
    Back before there were mkdir() and rmdir() system calls, applications
    wanting to create a directory invoked the setuid root mkdir utility
    which performed a series of mknod() and link() operations to create the
    directory and create dot and dot-dot entries in the new directory.
    Similarly, applications wanting to remove a directory invoked the
    setuid root rmdir uility which performed a series of unlink()
    operations to destroy the directory if it was empty.

    Not quite so far back in ancient history, there is also the point
    that before symlinks were available it was common practice to make
    hard links to directories.  Privileged applications can still do this
    today (as mandated by SVID) using the link() and unlink() system calls
    and the link and unlink commands.

UFS and ZFS are actually both correct, the standards wording permits either way kind of:

<snip>
If path1 names a directory, link() shall fail unless the process has
appropriate privileges and the implementation supports using link() on directories.

The path argument shall not name a directory unless the process has
appropriate privileges and the implementation supports using unlink() on directories.
<snip end>

This seems to implicitely allow an escape route; we can declare that no process
has sufficient privileges to link(2)/unlink(2) directories or like ZFS did,
have the underlaying implementation not supporting it at all.

The proper way is for the application to use the rmdir(2) system call,
it's there since ages!

my bottom line: GIT should not call unlink on a directory.

        Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello again,
it is definitively "git-checkout-index -a -f":

truss output is at http://wwwcip.informatik.uni-erlangen.de/~sithglan/git_checkout_index_a_f.truss.gz

        Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,
might that be related to the problem:


        lstat64("profiles/icpc", 0xFFBFF350)            Err#2 ENOENT
=>       mkdir("profiles", 0777)                         = 0
        stat64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
        0xFFBFF088) = 0
        open64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
        O_RDONLY) = 4
        mmap64(0x00000000, 284, PROT_READ, MAP_PRIVATE, 4, 0) = 0xFF230000
        close(4)                                        = 0
        munmap(0xFF230000, 284)                         = 0
        open64("profiles/icpc", O_WRONLY|O_CREAT|O_EXCL, 0666) = 4
        open64("profiles/.gitattributes", O_RDONLY)     Err#2 ENOENT
        write(4, " #   I C P C   P r o f i".., 420)     = 420
        close(4)                                        = 0
        lstat64("profiles/sithglan", 0xFFBFF350)        Err#2 ENOENT
=>      mkdir("profiles", 0777)                         Err#17 EEXIST
=>      unlink("profiles")                              = 0
=>      mkdir("profiles", 0777)                         = 0

I think it is. Damn it. What seems to hapen here is that git does:

        - create a subdirectory
        - puts a file in
        - deletes a subdirectory (by call unlink - that would normally fail,
          but with solaris as root it does not fail)

                => here comes the dangling hard link counter

        - created the directory again
        - puts the file in

That is why I only see one file in each subdirectory (the one that got
checkedout last). So the fix for git should be straight forward. But I still
think that Solaris is obviously broken. Because if you ask me it should not be
possible to unlink a directory that has files in it?!

                Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[David Kastrup]

Thomas Glanzmann <thomas@glanzmann.de> writes:

> Hello,
> might that be related to the problem:
>
>
>         lstat64("profiles/icpc", 0xFFBFF350)            Err#2 ENOENT
> =>       mkdir("profiles", 0777)                         = 0
>         stat64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
>         0xFFBFF088) = 0
>         open64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
>         O_RDONLY) = 4
>         mmap64(0x00000000, 284, PROT_READ, MAP_PRIVATE, 4, 0) = 0xFF230000
>         close(4)                                        = 0
>         munmap(0xFF230000, 284)                         = 0
>         open64("profiles/icpc", O_WRONLY|O_CREAT|O_EXCL, 0666) = 4
>         open64("profiles/.gitattributes", O_RDONLY)     Err#2 ENOENT
>         write(4, " #   I C P C   P r o f i".., 420)     = 420
>         close(4)                                        = 0
>         lstat64("profiles/sithglan", 0xFFBFF350)        Err#2 ENOENT
> =>      mkdir("profiles", 0777)                         Err#17 EEXIST
> =>      unlink("profiles")                              = 0
> =>      mkdir("profiles", 0777)                         = 0
>
> I think it is. Damn it. What seems to hapen here is that git does:
>
>         - create a subdirectory
>         - puts a file in
>         - deletes a subdirectory (by call unlink - that would normally fail,
>           but with solaris as root it does not fail)
>
>                 => here comes the dangling hard link counter
>
>         - created the directory again
>         - puts the file in
>
> That is why I only see one file in each subdirectory (the one that got
> checkedout last). So the fix for git should be straight forward. But I still
> think that Solaris is obviously broken. Because if you ask me it should not be
> possible to unlink a directory that has files in it?!

<URL:http://www.opengroup.org/onlinepubs/000095399/functions/unlink.html>

    The path argument shall not name a directory unless the process has
    appropriate privileges and the implementation supports using unlink()
    on directories.

So Solaris has the right to do this.

    APPLICATION USAGE

        Applications should use rmdir() to remove a directory.

    RATIONALE

        Unlinking a directory is restricted to the superuser in many
        historical implementations for reasons given in link() (see
        also rename()).


In short: git should not call remove, ever.  It may succeed, and is a
badly low-level call.  If something is known to be a directory, then
it needs to be removed using rmdir, and if it is a nondirectory, with
unlink.

Hm, browsing through Posix indicates that unlink is probably the same
as remove.  Pity.  I thought that just "remove" was the potential
evildoer.

-- 
David Kastrup

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Linus Torvalds]

On Mon, 16 Jul 2007, Thomas Glanzmann wrote:
>
> I think it is. Damn it. What seems to hapen here is that git does:
> 
>         - create a subdirectory
>         - puts a file in
>         - deletes a subdirectory (by call unlink - that would normally fail,
>           but with solaris as root it does not fail)

Solaris is apparently a buggy piece of crap.

I'd suggest making a bug-report. ENOTEMPTY is not an optional error. Posix 
says "Shall fail".

			Linus

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Linus Torvalds]

On Mon, 16 Jul 2007, Linus Torvalds wrote:
>
> Solaris is apparently a buggy piece of crap.
> 
> I'd suggest making a bug-report. ENOTEMPTY is not an optional error. Posix 
> says "Shall fail".

Oh, it was apparetly "unlink()" that corrupts filesystems. Which isn't 
against Posix, it's just stupid.

The "unlink directories" is so old-fashioned that it's not even funny. The 
whole reason "rmdir()" was created was to *not* have that behaviour in the 
OS. 

So I'll have to correct my statement. Please remove the "buggy" from 
"buggy piece of crap". The rest stands.

		Linus

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello Linus,

> I'd suggest making a bug-report. ENOTEMPTY is not an optional error.
> Posix says "Shall fail".  The "unlink directories" is so old-fashioned
> that it's not even funny. The whole reason "rmdir()" was created was
> to *not* have that behaviour in the OS. 

I asked the UFS maintainer to reconsider to fix this.

        Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,

> I asked the UFS maintainer to reconsider to fix this.

the bug is filed.

        Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,

> > I asked the UFS maintainer to reconsider to fix this.

> the bug is filed.

http://bugs.opensolaris.org/view_bug.do?bug_id=6581318

	Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Linus Torvalds]

On Wed, 18 Jul 2007, Thomas Glanzmann wrote:
>
> > the bug is filed.
> 
> http://bugs.opensolaris.org/view_bug.do?bug_id=6581318

I don't think the recipient "got it", since it already has a comment about 
"just use rmdir".

Can you please tell them that that isn't the point?

		Linus

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,

> I don't think the recipient "got it", since it already has a comment about 
> "just use rmdir".  Can you please tell them that that isn't the point?

I already did. They got the point. I have confidence that they fix it.
But there is no ETA. And the fix will not be backported.

	Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello,

entry.c line 17:

        static void create_directories(const char *path, const struct checkout *state)
        {
        ...
                        if (mkdir(buf, 0777)) {
=>                              if (errno == EEXIST) {
                                        struct stat st;
=>                                      if (len > state->base_dir_len && state->force && !unlink(buf) && !mkdir(buf, 0777))
                                                continue;
                                        if (!stat(buf, &st) && S_ISDIR(st.st_mode))
                                                continue; /* ok */
                                }
                                die("cannot create directory at %s", buf);
                        }
        ...

I think here is the problem. I don't understand what the code should do
actually. Or why the directory is deleted and re-created (maybe something todo
with permissions)?

        Thomas

Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)

[Johannes Sixt]

Thomas Glanzmann wrote:
>         static void create_directories(const char *path, const struct checkout *state)
>         {
>         ...
>                         if (mkdir(buf, 0777)) {
> =>                              if (errno == EEXIST) {
>                                         struct stat st;
> =>                                      if (len > state->base_dir_len && state->force && !unlink(buf) && !mkdir(buf, 0777))
>                                                 continue;
>                                         if (!stat(buf, &st) && S_ISDIR(st.st_mode))
>                                                 continue; /* ok */
>                                 }
>                                 die("cannot create directory at %s", buf);
>                         }
>         ...
> 
> I think here is the problem. I don't understand what the code should do
> actually. Or why the directory is deleted and re-created (maybe something todo
> with permissions)?

It tries to remove a *file* that is in the way and create the directory
in its place. But since your unlink() behaves incorrectly (it is
supposed to *fail* for directories), the logic does not quite work as
expected - it mistakes the directory for a file.

Try swapping the two inner-most if-blocks.

-- Hannes

Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)

[Brian Downing]

On Mon, Jul 16, 2007 at 04:35:10PM +0200, Johannes Sixt wrote:
> It tries to remove a *file* that is in the way and create the directory
> in its place. But since your unlink() behaves incorrectly (it is
> supposed to *fail* for directories), the logic does not quite work as
> expected - it mistakes the directory for a file.

http://www.opengroup.org/onlinepubs/007908799/xsh/unlink.html

| The path argument must not name a directory unless the process has
| appropriate privileges and the implementation supports using unlink() on
| directories.

The above seems to imply that implementations are free to do something
with directories passed to unlink, and that it is the responsibility
if the user not to do that unless they know what is going to happen.

-bcd

Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)

[Brian Downing]

On Mon, Jul 16, 2007 at 10:45:59AM -0500, Brian Downing wrote:
> http://www.opengroup.org/onlinepubs/007908799/xsh/unlink.html
> 
> | The path argument must not name a directory unless the process has
> | appropriate privileges and the implementation supports using unlink() on
> | directories.
> 
> The above seems to imply that implementations are free to do something
> with directories passed to unlink, and that it is the responsibility
> if the user not to do that unless they know what is going to happen.

Also, the HISTORY section of the FreeBSD manpages states:

| The unlink system call traditionally allows the super-user to unlink
| directories which can damage the filesystem integrity. This
| implementation no longer permits it.

Great.

-bcd

Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)

[David Kastrup]

bdowning@lavos.net (Brian Downing) writes:

> On Mon, Jul 16, 2007 at 04:35:10PM +0200, Johannes Sixt wrote:
>> It tries to remove a *file* that is in the way and create the directory
>> in its place. But since your unlink() behaves incorrectly (it is
>> supposed to *fail* for directories), the logic does not quite work as
>> expected - it mistakes the directory for a file.
>
> http://www.opengroup.org/onlinepubs/007908799/xsh/unlink.html
>
> | The path argument must not name a directory unless the process has
> | appropriate privileges and the implementation supports using unlink() on
> | directories.

Isn't it funny?  The problem with the git code is that the path
argument must not name a directory _if_ the process has appropriate
privileges and the implementation supports using unlink() on
directories.

Failure is not an option, it is a requirement.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)

[Thomas Glanzmann]

Hello Johannes,

> Try swapping the two inner-most if-blocks.

I did and it works as expected:

        (faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/easix.git test-8
        Initialized empty Git repository in /var/tmp/test-8/.git/
        remote: Generating pack...
        remote: Done counting 317 objects.
        remote: Deltifying 317 objects...
        remote:  100% (317/317remote: ) done
        Indexing 317 objects...
        remote: Total 317 (delta 182), reused 278 (delta 157)
        100% (317/317) done
        Resolving 182 deltas...
        100% (182/182) done

        (faui04a) [/var/tmp] cd test-8
        ./test-8
        (faui04a) [/var/tmp/test-8] git status
        # On branch master
        nothing to commit (working directory clean)
        (faui04a) [/var/tmp/test-8] cd ..
        (faui04a) [/var/tmp] rm -rf test-8

... I have a filesystem to check. :-)

        Thomas