* "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
@ 2007-07-16 10:08 Thomas Glanzmann
2007-07-16 10:43 ` Thomas Glanzmann
` (3 more replies)
0 siblings, 4 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 10:08 UTC (permalink / raw)
To: GIT; +Cc: Linus Torvalds, Junio C Hamano
Hello,
when I call "git clone ~adglth0/work/scripts whatever" as user
everything works as expected. As soon as I call this command as root it
shreds ufs:
(s092942) [~] git version
git version 1.5.3.GIT
(s092942) [~] git clone ~adglth0/work/scripts nonroot
Initialized empty Git repository in /export/home/adglth0/nonroot/.git/
remote: Generating pack...
Done counting 220 objects.
Deltifying 220 objects...
2% (5/220) doneIndexing 220 objects...
100% (220/220) done20) donedonedone
Total 220 (delta 114), reused 181remote: (delta 88)
100% (220/220) done
Resolving 114 deltas...
100% (114/114) done
(s092942) [~] cd nonroot
./nonroot
(s092942) [~/nonroot] git status
# On branch master
nothing to commit (working directory clean)
(s092942) [~/nonroot] fg
sudo ${SHELL} --rcfile ${HOME}/.bash_profile (wd: ~)
(s092942) [~] git clone ~adglth0/work/scripts root
DO *NOT* RUN GIT AS ROOT clone /export/home/adglth0/work/scripts root
(s092942) [~] unalias git
(s092942) [~] git clone ~adglth0/work/scripts root
Initialized empty Git repository in /export/home/adglth0/root/.git/
remote: Generating pack...
remote: Done counting 220 objects.
Deltifying 220 objects...
3% (7/220) doneIndexing 220 objects...
100% (220/220) done20) done0) done
Total 220 (delta 114), reused 181 (delta 88)
100% (220/220) done
Resolving 114 deltas...
100% (114/114) done
(s092942) [~] cd root
(s092942) [~/root] git status
# On branch master
# Changed but not updated:
# (use "git add/rm <file>..." to update what will be committed)
#
# deleted: eclogic/acct_per_host.pl
# deleted: eclogic/check_3ware
# deleted: eclogic/check_ipmi.pl
# deleted: eclogic/check_load.pl
# deleted: eclogic/create_nagios_config.pl
# deleted: eclogic/mailrelay/config-2.6.20
# deleted: eclogic/mailrelay/interfaces
# deleted: eclogic/on_all_hosts.pl
# deleted: lifebook/kernel-config
# deleted: s017241/config-2.6.20
# deleted: s017241/firewall
# deleted: s017241/siemens.conf
# deleted: siemens/aliasupdate-tg
# deleted: siemens/check_raid.sh
# deleted: siemens/create_nagios_config.pl
# deleted: siemens/crontab_client
# deleted: siemens/deploy
# deleted: siemens/generate_password.pl.gpg
# deleted: siemens/mail_verrechnung.pl
# deleted: siemens/oracle_beispiel_client.pl
# deleted: siemens/sar_deploy.pl
# deleted: siemens/t3_check_via_telnet.pl
# deleted: thinkpad/config-2.6.20
# deleted: thinkpad/fglrx.patch
# deleted: thinkpad/network-please
# deleted: thinkpad/xorg/build
# deleted: webfarm/config-2.6.20
#
no changes added to commit (use "git add" and/or "git commit -a")
(s092942) [~/root] git checkout -f HEAD
(s092942) [~/root] git status
# On branch master
# Changed but not updated:
# (use "git add/rm <file>..." to update what will be committed)
#
# deleted: eclogic/acct_per_host.pl
# deleted: eclogic/check_3ware
# deleted: eclogic/check_ipmi.pl
# deleted: eclogic/check_load.pl
# deleted: eclogic/create_nagios_config.pl
# deleted: eclogic/mailrelay/config-2.6.20
# deleted: eclogic/mailrelay/interfaces
# deleted: eclogic/sms.pl
# deleted: lifebook/network-please
# deleted: s017241/config-2.6.20
# deleted: s017241/firewall
# deleted: s017241/xorg.conf
# deleted: siemens/aliasupdate-tg
# deleted: siemens/check_raid.sh
# deleted: siemens/create_nagios_config.pl
# deleted: siemens/crontab_client
# deleted: siemens/deploy
# deleted: siemens/generate_password.pl.gpg
# deleted: siemens/mail_verrechnung.pl
# deleted: siemens/oracle_beispiel_client.pl
# deleted: siemens/sar_deploy.pl
# deleted: siemens/tryserver.sh
# deleted: thinkpad/config-2.6.20
# deleted: thinkpad/fglrx.patch
# deleted: thinkpad/network-please
# deleted: thinkpad/xorg/cleanup
# deleted: webfarm/unsplit.pl
#
no changes added to commit (use "git add" and/or "git commit -a")
(s092942) [~/root] ls
eclogic/ leo* lifebook/ s017241/ siemens/ thinkpad/ webfarm/
(s092942) [~/root] ls siemens/
t3_check_via_telnet.pl*
(s092942) [~/root] cd ..
(s092942) [~] rm -rf root/
rm: Unable to remove directory root/: File exists
(s092942) [~] find root/
root/
(s092942) [~] ls -al root/
total 6
drwxr-xr-x 56 root root 512 Jul 16 11:48 ./
drwxr-xr-x 17 adglth0 root 1536 Jul 16 11:48 ../
(s092942) [~] df -h .
Filesystem size used avail capacity Mounted on
/dev/dsk/c0t0d0s7 75G 9.0G 65G 13% /export/home
(s092942) [~] ls -ali root/
total 6
13657 drwxr-xr-x 56 root root 512 Jul 16 11:48 ./
5829 drwxr-xr-x 17 adglth0 root 1536 Jul 16 11:48 ../
(s092942) [~] gfind /export/home -xdev -inum 13657
/export/home/adglth0/root
(s092942) [~]
A few notes and observations I did. Linux _does_ not allow userland to
create hardlinks for directories which makes totally sense. Solaris does
allow the _root_ user to allow hardlinks for directories. There is even
a command that makes that possible:
(faui04a) [/var/tmp] mkdir test
(faui04a) [/var/tmp] /usr/xpg4/bin/link test bla
(faui04a) [/var/tmp] ls -ali
total 13
1639 drwxrwxrwt 4 root sys 9216 Jul 16 11:13 ./
4 drwxr-xr-x 48 root sys 1024 Jan 1 2007 ../
238939 drwx--x--x 3 snalwuer icipguru 512 Jul 5 19:08 1071895580/
238945 drwxr-xr-x 3 root root 512 Jul 16 11:11 bla/
238945 drwxr-xr-x 3 root root 512 Jul 16 11:11 test/
(faui04a) [/var/tmp] find . -inum 238945
Whatever. My first impression was that the filesystem of the solaris is
broken. So I fscked it. And guess what it corrected the error. But after
the reboot I tried it again and you see the result above. So my guess is
that git calls the link system call in a way Solaris thinks it should
itself shoot in the head. This also happens with _remote_ clones as
root:
(s092942) [~] git clone adglth0@157.163.208.106:/home/adglth0/work/scripts second_test_as_root
Initialized empty Git repository in /export/home/adglth0/second_test_as_root/.git/
The authenticity of host '157.163.208.106 (157.163.208.106)' can't be established.
RSA key fingerprint is 72:27:be:82:f7:71:e5:23:a6:b6:57:a9:11:dd:26:26.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '157.163.208.106' (RSA) to the list of known hosts.
remote: Generating pack...
remote: Done counting 220 objects.
remote: Deltifying 220 objects...
remote: 100% (220/220) done
Indexing 220 objects...
remote: Total 220 (delta 115), reused 37 (delta 9)
100% (220/220) done
Resolving 115 deltas...
100% (115/115) done
(s092942) [~] cd second_test_as_root/
./second_test_as_root/
(s092942) [~/second_test_as_root] git status
# On branch master
# Changed but not updated:
# (use "git add/rm <file>..." to update what will be committed)
#
# deleted: eclogic/acct_per_host.pl
# deleted: eclogic/check_3ware
# deleted: eclogic/check_ipmi.pl
# deleted: eclogic/check_load.pl
# deleted: eclogic/create_nagios_config.pl
# deleted: eclogic/mailrelay/config-2.6.20
# deleted: eclogic/mailrelay/interfaces
# deleted: eclogic/on_all_hosts.pl
# deleted: lifebook/kernel-config
# deleted: s017241/config-2.6.20
# deleted: s017241/firewall
# deleted: s017241/siemens.conf
# deleted: siemens/aliasupdate-tg
# deleted: siemens/check_raid.sh
# deleted: siemens/create_nagios_config.pl
# deleted: siemens/crontab_client
# deleted: siemens/deploy
# deleted: siemens/generate_password.pl.gpg
# deleted: siemens/mail_verrechnung.pl
# deleted: siemens/oracle_beispiel_client.pl
# deleted: siemens/sar_deploy.pl
# deleted: siemens/t3_check_via_telnet.pl
# deleted: thinkpad/config-2.6.20
# deleted: thinkpad/fglrx.patch
# deleted: thinkpad/network-please
# deleted: thinkpad/xorg/build
# deleted: webfarm/config-2.6.20
#
no changes added to commit (use "git add" and/or "git commit -a")
(s092942) [~/second_test_as_root] uname -a
SunOS s092942 5.10 Generic_118833-17 sun4u sparc SUNW,Sun-Blade-100
So has anyone a idea what is wrong. I have shareded a second machine in
a complete different domain. With a complete different git version. Same
thing. So it is not the hardware and not the specific git version and
not the Solaris version:
(faui04a) [/var/tmp] git clone ~sithglan/work/sc
screen/ scripts/ scripts_ccbank_what_the_hell/
(faui04a) [/var/tmp] git clone ~sithglan/work/repositories/
private/ public/
(faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/
blastwave.git/ bs.git/ easix.git/ mutt.git/ nzb.git/ ra.git/ watchdog.git/
(faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/easix.git/
DO *NOT* RUN GIT AS ROOT clone /home/cip/adm/sithglan/work/repositories/public/easix.git/
(faui04a) [/var/tmp] unalias git
(faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/easix.git/
Initialized empty Git repository in /var/tmp/easix/.git/
remote: Generating pack...
remote: Done counting 317 objects.
remote: Deltifying 317 objects...
remote: emote: 317/317) done: ) done17) done
Indexing 317 objects...
remote: Total 317 (delta 182), reused 278 (delta 157)
100% (317/317) done
Resolving 182 deltas...
100% (182/182) done
(faui04a) [/var/tmp] cd easix/
./easix/
(faui04a) [/var/tmp/easix] git status
# On branch master
# Changed but not updated:
# (use "git add/rm <file>..." to update what will be committed)
#
# deleted: profiles/icpc
# deleted: profiles/sithglan
# deleted: sources/boot.msg
# deleted: sources/busybox-config
# deleted: sources/config
# deleted: sources/easix.logo.16
# deleted: sources/easixdm
# deleted: sources/f2
# deleted: sources/home-faumachine.tar.gz
# deleted: sources/home-icpc.tar.gz
# deleted: sources/home-simigern.tar.gz
# deleted: sources/home-sithglan.tar.gz
# deleted: sources/home-sitowert.tar.gz
# deleted: sources/linuxrc
# deleted: sources/rc.local
#
no changes added to commit (use "git add" and/or "git commit -a")
(faui04a) [/var/tmp/easix] cd ..
(faui04a) [/var/tmp] rm -rf easix
rm: Unable to remove directory easix: File exists
(faui04a) [/var/tmp] ls -ali easix/
total 10
238947 drwxr-xr-x 17 root root 512 Jul 16 12:01 ./
1639 drwxrwxrwt 5 root sys 9216 Jul 16 12:01 ../
(faui04a) [/var/tmp] git version
git version 1.5.2
(faui04a) [/var/tmp] uname -a
SunOS faui04a 5.10 Generic_118833-24 sun4u sparc SUNW,Sun-Fire-280R
[ I am off the net for 30 mintues ... eating ]
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:08 "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris) Thomas Glanzmann
@ 2007-07-16 10:43 ` Thomas Glanzmann
2007-07-16 10:48 ` David Kastrup
2007-07-16 10:51 ` David Kastrup
2007-07-16 12:46 ` Thomas Glanzmann
` (2 subsequent siblings)
3 siblings, 2 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 10:43 UTC (permalink / raw)
To: GIT; +Cc: Linus Torvalds, Junio C Hamano
Hello,
I narrowed it down. It is in "git checkout" if I copy the ".git" dir via
"cp -a" and type in "git checkout" it breaks. And it _only_ breaks for
subdirectories:
(faui04a) [/var/tmp/shit] git checkout
D profiles/icpc
D profiles/sithglan
D sources/boot.msg
D sources/busybox-config
D sources/config
D sources/easix.logo.16
D sources/easixdm
D sources/f2
D sources/home-faumachine.tar.gz
D sources/home-icpc.tar.gz
D sources/home-simigern.tar.gz
D sources/home-sithglan.tar.gz
D sources/home-sitowert.tar.gz
D sources/linuxrc
D sources/rc.local
and I also see that _one_ file in each subdirectory is checked out but the
others are not.
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:43 ` Thomas Glanzmann
@ 2007-07-16 10:48 ` David Kastrup
2007-07-16 12:35 ` Thomas Glanzmann
2007-07-16 10:51 ` David Kastrup
1 sibling, 1 reply; 25+ messages in thread
From: David Kastrup @ 2007-07-16 10:48 UTC (permalink / raw)
To: git
Thomas Glanzmann <thomas@glanzmann.de> writes:
> Hello,
> I narrowed it down. It is in "git checkout" if I copy the ".git" dir via
> "cp -a" and type in "git checkout" it breaks. And it _only_ breaks for
> subdirectories:
>
> (faui04a) [/var/tmp/shit] git checkout
> D profiles/icpc
> D profiles/sithglan
> D sources/boot.msg
> D sources/busybox-config
> D sources/config
> D sources/easix.logo.16
> D sources/easixdm
> D sources/f2
> D sources/home-faumachine.tar.gz
> D sources/home-icpc.tar.gz
> D sources/home-simigern.tar.gz
> D sources/home-sithglan.tar.gz
> D sources/home-sitowert.tar.gz
> D sources/linuxrc
> D sources/rc.local
>
> and I also see that _one_ file in each subdirectory is checked out but the
> others are not.
Ok, stupidly trying one of our most frequently encountered problems:
Does it help to say
export CDPATH=""
in your root shell?
--
David Kastrup
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:43 ` Thomas Glanzmann
2007-07-16 10:48 ` David Kastrup
@ 2007-07-16 10:51 ` David Kastrup
2007-07-16 12:39 ` Thomas Glanzmann
1 sibling, 1 reply; 25+ messages in thread
From: David Kastrup @ 2007-07-16 10:51 UTC (permalink / raw)
To: git
Thomas Glanzmann <thomas@glanzmann.de> writes:
> Hello,
> I narrowed it down. It is in "git checkout" if I copy the ".git" dir via
> "cp -a" and type in "git checkout" it breaks. And it _only_ breaks for
> subdirectories:
Another guess: cp -a copies files preserving all permissions and
ownerships (which works only as root). Nominally, the files now don't
belong to you, and some scripts might take you by the word even if
they _could_ just overwrite things if they actually tried. Try
making
chown -R root.root .git
on your tree and see whether this makes git more comfortable.
--
David Kastrup
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:48 ` David Kastrup
@ 2007-07-16 12:35 ` Thomas Glanzmann
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 12:35 UTC (permalink / raw)
To: David Kastrup; +Cc: git, Linus Torvalds, Junio C Hamano
Hello David,
> Does it help to say export CDPATH="" in your root shell?
no. The problem is triggered by "git-checkout-index -a -f" so my CDPATH has
absolutly _nothing_ todo with it:
(faui04a) [/var/tmp] mkdir -p test-4/.git
(faui04a) [/var/tmp] cd test-4
(faui04a) [/var/tmp/test-4] gcp -a ~/work/repositories/public/easix.git/* .git/
(faui04a) [/var/tmp/test-4] git-read-tree HEAD
(faui04a) [/var/tmp/test-4] git-checkout-index -a -f
(faui04a) [/var/tmp/test-4] git status
fatal: /usr/bin/git-status cannot be used without a working tree.
(faui04a) [/var/tmp/test-4] rm .git/config
(faui04a) [/var/tmp/test-4] git status
# On branch master
# Changed but not updated:
# (use "git add/rm <file>..." to update what will be committed)
#
# deleted: profiles/icpc
# deleted: profiles/sithglan
# deleted: sources/boot.msg
# deleted: sources/busybox-config
# deleted: sources/config
# deleted: sources/easix.logo.16
# deleted: sources/easixdm
# deleted: sources/f2
# deleted: sources/home-faumachine.tar.gz
# deleted: sources/home-icpc.tar.gz
# deleted: sources/home-simigern.tar.gz
# deleted: sources/home-sithglan.tar.gz
# deleted: sources/home-sitowert.tar.gz
# deleted: sources/linuxrc
# deleted: sources/rc.local
#
no changes added to commit (use "git add" and/or "git commit -a")
(faui04a) [/var/tmp/test-4] cd ..
(faui04a) [/var/tmp] rm -rf test-4
rm: Unable to remove directory test-4: File exists
(faui04a) [/var/tmp]
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:51 ` David Kastrup
@ 2007-07-16 12:39 ` Thomas Glanzmann
2007-07-16 12:58 ` David Kastrup
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 12:39 UTC (permalink / raw)
To: David Kastrup; +Cc: git
Hello David,
as I mentioned and showed in my original e-mail:
git checkout /path/to/repo
triggers the problem also. So no, "cp -a" has nothing todo with it
because root is able to read _all_ files no matter who the files are
owned by and what the permissions of these files are[1]. This happens on
a _local_ fileystem. And what worries me is that git does something that
breaks the _hard link counter_ of a directory.
Thomas
[1] Notes
Of course that doesn't apply to all filesystems. For example AFS
and NFS. But as I said before it happens on a _UFS_ filesystem.
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:08 "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris) Thomas Glanzmann
2007-07-16 10:43 ` Thomas Glanzmann
@ 2007-07-16 12:46 ` Thomas Glanzmann
2007-07-16 13:36 ` Thomas Glanzmann
2007-07-16 13:45 ` Thomas Glanzmann
3 siblings, 0 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 12:46 UTC (permalink / raw)
To: GIT; +Cc: Linus Torvalds, Junio C Hamano
Hello again,
it is definitively "git-checkout-index -a -f":
truss output is at http://wwwcip.informatik.uni-erlangen.de/~sithglan/git_checkout_index_a_f.truss.gz
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 12:39 ` Thomas Glanzmann
@ 2007-07-16 12:58 ` David Kastrup
2007-07-16 13:15 ` Thomas Glanzmann
0 siblings, 1 reply; 25+ messages in thread
From: David Kastrup @ 2007-07-16 12:58 UTC (permalink / raw)
To: git
Thomas Glanzmann <thomas@glanzmann.de> writes:
> Hello David,
> as I mentioned and showed in my original e-mail:
>
> git checkout /path/to/repo
>
> triggers the problem also. So no, "cp -a" has nothing todo with it
> because root is able to read _all_ files no matter who the files are
> owned by and what the permissions of these files are[1]. This happens on
> a _local_ fileystem. And what worries me is that git does something that
> breaks the _hard link counter_ of a directory.
Hm? The hard link counter is "broken" by creating or not creating
subdirectories, since their ".." is a hard link.
--
David Kastrup
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 12:58 ` David Kastrup
@ 2007-07-16 13:15 ` Thomas Glanzmann
2007-07-16 16:40 ` Thomas Glanzmann
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 13:15 UTC (permalink / raw)
To: David Kastrup; +Cc: GIT
Hello David,
> Hm? The hard link counter is "broken" by creating or not creating
> subdirectories, since their ".." is a hard link.
exactly. The question is: Is it a Solaris bug or is it something that is
supposed a user is able to do (it doesn't make sense for me)? I posted
this problem to comp.unix.solaris and also contacted the UFS Maintainer
of Solaris (it is not the first UFS bug I original found). If I don't
receive feedback, I am going to open a call with Sun. Whatever is going
on (hopefully it isn't PEBKAC - but I don't think so) and this time it
isn't broken hardware either (like the last time I reported a serious
git bug) because it happens on two machines.
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:08 "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris) Thomas Glanzmann
2007-07-16 10:43 ` Thomas Glanzmann
2007-07-16 12:46 ` Thomas Glanzmann
@ 2007-07-16 13:36 ` Thomas Glanzmann
2007-07-16 16:29 ` David Kastrup
2007-07-16 17:03 ` Linus Torvalds
2007-07-16 13:45 ` Thomas Glanzmann
3 siblings, 2 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 13:36 UTC (permalink / raw)
To: GIT; +Cc: Linus Torvalds, Junio C Hamano
Hello,
might that be related to the problem:
lstat64("profiles/icpc", 0xFFBFF350) Err#2 ENOENT
=> mkdir("profiles", 0777) = 0
stat64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
0xFFBFF088) = 0
open64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
O_RDONLY) = 4
mmap64(0x00000000, 284, PROT_READ, MAP_PRIVATE, 4, 0) = 0xFF230000
close(4) = 0
munmap(0xFF230000, 284) = 0
open64("profiles/icpc", O_WRONLY|O_CREAT|O_EXCL, 0666) = 4
open64("profiles/.gitattributes", O_RDONLY) Err#2 ENOENT
write(4, " # I C P C P r o f i".., 420) = 420
close(4) = 0
lstat64("profiles/sithglan", 0xFFBFF350) Err#2 ENOENT
=> mkdir("profiles", 0777) Err#17 EEXIST
=> unlink("profiles") = 0
=> mkdir("profiles", 0777) = 0
I think it is. Damn it. What seems to hapen here is that git does:
- create a subdirectory
- puts a file in
- deletes a subdirectory (by call unlink - that would normally fail,
but with solaris as root it does not fail)
=> here comes the dangling hard link counter
- created the directory again
- puts the file in
That is why I only see one file in each subdirectory (the one that got
checkedout last). So the fix for git should be straight forward. But I still
think that Solaris is obviously broken. Because if you ask me it should not be
possible to unlink a directory that has files in it?!
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 10:08 "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris) Thomas Glanzmann
` (2 preceding siblings ...)
2007-07-16 13:36 ` Thomas Glanzmann
@ 2007-07-16 13:45 ` Thomas Glanzmann
2007-07-16 14:35 ` "git clone" executed as root on solaris 10 shreds UFS (it ispossible " Johannes Sixt
3 siblings, 1 reply; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 13:45 UTC (permalink / raw)
To: GIT; +Cc: Linus Torvalds, Junio C Hamano
Hello,
entry.c line 17:
static void create_directories(const char *path, const struct checkout *state)
{
...
if (mkdir(buf, 0777)) {
=> if (errno == EEXIST) {
struct stat st;
=> if (len > state->base_dir_len && state->force && !unlink(buf) && !mkdir(buf, 0777))
continue;
if (!stat(buf, &st) && S_ISDIR(st.st_mode))
continue; /* ok */
}
die("cannot create directory at %s", buf);
}
...
I think here is the problem. I don't understand what the code should do
actually. Or why the directory is deleted and re-created (maybe something todo
with permissions)?
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)
2007-07-16 13:45 ` Thomas Glanzmann
@ 2007-07-16 14:35 ` Johannes Sixt
2007-07-16 15:45 ` Brian Downing
2007-07-16 17:12 ` Thomas Glanzmann
0 siblings, 2 replies; 25+ messages in thread
From: Johannes Sixt @ 2007-07-16 14:35 UTC (permalink / raw)
To: git; +Cc: Linus Torvalds, Junio C Hamano
Thomas Glanzmann wrote:
> static void create_directories(const char *path, const struct checkout *state)
> {
> ...
> if (mkdir(buf, 0777)) {
> => if (errno == EEXIST) {
> struct stat st;
> => if (len > state->base_dir_len && state->force && !unlink(buf) && !mkdir(buf, 0777))
> continue;
> if (!stat(buf, &st) && S_ISDIR(st.st_mode))
> continue; /* ok */
> }
> die("cannot create directory at %s", buf);
> }
> ...
>
> I think here is the problem. I don't understand what the code should do
> actually. Or why the directory is deleted and re-created (maybe something todo
> with permissions)?
It tries to remove a *file* that is in the way and create the directory
in its place. But since your unlink() behaves incorrectly (it is
supposed to *fail* for directories), the logic does not quite work as
expected - it mistakes the directory for a file.
Try swapping the two inner-most if-blocks.
-- Hannes
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)
2007-07-16 14:35 ` "git clone" executed as root on solaris 10 shreds UFS (it ispossible " Johannes Sixt
@ 2007-07-16 15:45 ` Brian Downing
2007-07-16 15:50 ` Brian Downing
2007-07-16 19:57 ` David Kastrup
2007-07-16 17:12 ` Thomas Glanzmann
1 sibling, 2 replies; 25+ messages in thread
From: Brian Downing @ 2007-07-16 15:45 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git, Linus Torvalds, Junio C Hamano
On Mon, Jul 16, 2007 at 04:35:10PM +0200, Johannes Sixt wrote:
> It tries to remove a *file* that is in the way and create the directory
> in its place. But since your unlink() behaves incorrectly (it is
> supposed to *fail* for directories), the logic does not quite work as
> expected - it mistakes the directory for a file.
http://www.opengroup.org/onlinepubs/007908799/xsh/unlink.html
| The path argument must not name a directory unless the process has
| appropriate privileges and the implementation supports using unlink() on
| directories.
The above seems to imply that implementations are free to do something
with directories passed to unlink, and that it is the responsibility
if the user not to do that unless they know what is going to happen.
-bcd
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)
2007-07-16 15:45 ` Brian Downing
@ 2007-07-16 15:50 ` Brian Downing
2007-07-16 19:57 ` David Kastrup
1 sibling, 0 replies; 25+ messages in thread
From: Brian Downing @ 2007-07-16 15:50 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git, Linus Torvalds, Junio C Hamano
On Mon, Jul 16, 2007 at 10:45:59AM -0500, Brian Downing wrote:
> http://www.opengroup.org/onlinepubs/007908799/xsh/unlink.html
>
> | The path argument must not name a directory unless the process has
> | appropriate privileges and the implementation supports using unlink() on
> | directories.
>
> The above seems to imply that implementations are free to do something
> with directories passed to unlink, and that it is the responsibility
> if the user not to do that unless they know what is going to happen.
Also, the HISTORY section of the FreeBSD manpages states:
| The unlink system call traditionally allows the super-user to unlink
| directories which can damage the filesystem integrity. This
| implementation no longer permits it.
Great.
-bcd
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 13:36 ` Thomas Glanzmann
@ 2007-07-16 16:29 ` David Kastrup
2007-07-16 17:03 ` Linus Torvalds
1 sibling, 0 replies; 25+ messages in thread
From: David Kastrup @ 2007-07-16 16:29 UTC (permalink / raw)
To: git
Thomas Glanzmann <thomas@glanzmann.de> writes:
> Hello,
> might that be related to the problem:
>
>
> lstat64("profiles/icpc", 0xFFBFF350) Err#2 ENOENT
> => mkdir("profiles", 0777) = 0
> stat64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
> 0xFFBFF088) = 0
> open64(".git/objects/66/6197b02f46c92f0273f16ac77d34d76b28f4f0",
> O_RDONLY) = 4
> mmap64(0x00000000, 284, PROT_READ, MAP_PRIVATE, 4, 0) = 0xFF230000
> close(4) = 0
> munmap(0xFF230000, 284) = 0
> open64("profiles/icpc", O_WRONLY|O_CREAT|O_EXCL, 0666) = 4
> open64("profiles/.gitattributes", O_RDONLY) Err#2 ENOENT
> write(4, " # I C P C P r o f i".., 420) = 420
> close(4) = 0
> lstat64("profiles/sithglan", 0xFFBFF350) Err#2 ENOENT
> => mkdir("profiles", 0777) Err#17 EEXIST
> => unlink("profiles") = 0
> => mkdir("profiles", 0777) = 0
>
> I think it is. Damn it. What seems to hapen here is that git does:
>
> - create a subdirectory
> - puts a file in
> - deletes a subdirectory (by call unlink - that would normally fail,
> but with solaris as root it does not fail)
>
> => here comes the dangling hard link counter
>
> - created the directory again
> - puts the file in
>
> That is why I only see one file in each subdirectory (the one that got
> checkedout last). So the fix for git should be straight forward. But I still
> think that Solaris is obviously broken. Because if you ask me it should not be
> possible to unlink a directory that has files in it?!
<URL:http://www.opengroup.org/onlinepubs/000095399/functions/unlink.html>
The path argument shall not name a directory unless the process has
appropriate privileges and the implementation supports using unlink()
on directories.
So Solaris has the right to do this.
APPLICATION USAGE
Applications should use rmdir() to remove a directory.
RATIONALE
Unlinking a directory is restricted to the superuser in many
historical implementations for reasons given in link() (see
also rename()).
In short: git should not call remove, ever. It may succeed, and is a
badly low-level call. If something is known to be a directory, then
it needs to be removed using rmdir, and if it is a nondirectory, with
unlink.
Hm, browsing through Posix indicates that unlink is probably the same
as remove. Pity. I thought that just "remove" was the potential
evildoer.
--
David Kastrup
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 13:15 ` Thomas Glanzmann
@ 2007-07-16 16:40 ` Thomas Glanzmann
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 16:40 UTC (permalink / raw)
To: GIT
Hello,
> exactly. The question is: Is it a Solaris bug or is it something that is
> supposed a user is able to do (it doesn't make sense for me)? I posted
> this problem to comp.unix.solaris and also contacted the UFS Maintainer
> of Solaris (it is not the first UFS bug I original found). If I don't
> receive feedback, I am going to open a call with Sun. Whatever is going
> on (hopefully it isn't PEBKAC - but I don't think so) and this time it
> isn't broken hardware either (like the last time I reported a serious
> git bug) because it happens on two machines.
below I quote the answer of the UFS maintainer:
Thomas,
some notes on the unlink(directory) part:
In the case of calling unlink(2) on a directory as a privilidged user, things
like ".." (and the link count in the parent) would not be cleaned up.
The man page for unlink(1M) and unlink(2) is very clear about using rmdir(1/2) to
remove a directory. It's also very clear that this does exactly what
it's supposed to do: Remove a link to a file/directory. Cleaning up
the ".." entry (which would clean up the link to the parent) could
be considered "extra".
First the standards issues:
The link(BA_OS) and unlink(BA_OS) descriptions in SVID3 both specify that
a process with appropriate privileges is allowed to operate on a directory.
We have claimed to conform to SVID3 since Solaris 2.0 and have not announced
that we ever plan to EOL SVID3 conformance.
UFS does support link(2)/unlink(2) with appropriate privilidges of
directories while ZFS does not.
The change that would provide ``Cleaning up the ".." entry ...'' would
violate both SVID3 and POSIX and SUS requirements. (The unlink(path)
system call is supposed to remove the directory entry named by path; not
the directory entry named by path and an arbitrary number of other
directory entries.)
Second some history:
Back before there were mkdir() and rmdir() system calls, applications
wanting to create a directory invoked the setuid root mkdir utility
which performed a series of mknod() and link() operations to create the
directory and create dot and dot-dot entries in the new directory.
Similarly, applications wanting to remove a directory invoked the
setuid root rmdir uility which performed a series of unlink()
operations to destroy the directory if it was empty.
Not quite so far back in ancient history, there is also the point
that before symlinks were available it was common practice to make
hard links to directories. Privileged applications can still do this
today (as mandated by SVID) using the link() and unlink() system calls
and the link and unlink commands.
UFS and ZFS are actually both correct, the standards wording permits either way kind of:
<snip>
If path1 names a directory, link() shall fail unless the process has
appropriate privileges and the implementation supports using link() on directories.
The path argument shall not name a directory unless the process has
appropriate privileges and the implementation supports using unlink() on directories.
<snip end>
This seems to implicitely allow an escape route; we can declare that no process
has sufficient privileges to link(2)/unlink(2) directories or like ZFS did,
have the underlaying implementation not supporting it at all.
The proper way is for the application to use the rmdir(2) system call,
it's there since ages!
my bottom line: GIT should not call unlink on a directory.
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 13:36 ` Thomas Glanzmann
2007-07-16 16:29 ` David Kastrup
@ 2007-07-16 17:03 ` Linus Torvalds
2007-07-16 17:07 ` Linus Torvalds
1 sibling, 1 reply; 25+ messages in thread
From: Linus Torvalds @ 2007-07-16 17:03 UTC (permalink / raw)
To: Thomas Glanzmann; +Cc: GIT, Junio C Hamano
On Mon, 16 Jul 2007, Thomas Glanzmann wrote:
>
> I think it is. Damn it. What seems to hapen here is that git does:
>
> - create a subdirectory
> - puts a file in
> - deletes a subdirectory (by call unlink - that would normally fail,
> but with solaris as root it does not fail)
Solaris is apparently a buggy piece of crap.
I'd suggest making a bug-report. ENOTEMPTY is not an optional error. Posix
says "Shall fail".
Linus
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 17:03 ` Linus Torvalds
@ 2007-07-16 17:07 ` Linus Torvalds
2007-07-16 17:17 ` Thomas Glanzmann
0 siblings, 1 reply; 25+ messages in thread
From: Linus Torvalds @ 2007-07-16 17:07 UTC (permalink / raw)
To: Thomas Glanzmann; +Cc: GIT, Junio C Hamano
On Mon, 16 Jul 2007, Linus Torvalds wrote:
>
> Solaris is apparently a buggy piece of crap.
>
> I'd suggest making a bug-report. ENOTEMPTY is not an optional error. Posix
> says "Shall fail".
Oh, it was apparetly "unlink()" that corrupts filesystems. Which isn't
against Posix, it's just stupid.
The "unlink directories" is so old-fashioned that it's not even funny. The
whole reason "rmdir()" was created was to *not* have that behaviour in the
OS.
So I'll have to correct my statement. Please remove the "buggy" from
"buggy piece of crap". The rest stands.
Linus
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)
2007-07-16 14:35 ` "git clone" executed as root on solaris 10 shreds UFS (it ispossible " Johannes Sixt
2007-07-16 15:45 ` Brian Downing
@ 2007-07-16 17:12 ` Thomas Glanzmann
1 sibling, 0 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 17:12 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git, Linus Torvalds, Junio C Hamano
Hello Johannes,
> Try swapping the two inner-most if-blocks.
I did and it works as expected:
(faui04a) [/var/tmp] git clone ~sithglan/work/repositories/public/easix.git test-8
Initialized empty Git repository in /var/tmp/test-8/.git/
remote: Generating pack...
remote: Done counting 317 objects.
remote: Deltifying 317 objects...
remote: 100% (317/317remote: ) done
Indexing 317 objects...
remote: Total 317 (delta 182), reused 278 (delta 157)
100% (317/317) done
Resolving 182 deltas...
100% (182/182) done
(faui04a) [/var/tmp] cd test-8
./test-8
(faui04a) [/var/tmp/test-8] git status
# On branch master
nothing to commit (working directory clean)
(faui04a) [/var/tmp/test-8] cd ..
(faui04a) [/var/tmp] rm -rf test-8
... I have a filesystem to check. :-)
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 17:07 ` Linus Torvalds
@ 2007-07-16 17:17 ` Thomas Glanzmann
2007-07-16 18:09 ` Thomas Glanzmann
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 17:17 UTC (permalink / raw)
To: Linus Torvalds; +Cc: GIT
Hello Linus,
> I'd suggest making a bug-report. ENOTEMPTY is not an optional error.
> Posix says "Shall fail". The "unlink directories" is so old-fashioned
> that it's not even funny. The whole reason "rmdir()" was created was
> to *not* have that behaviour in the OS.
I asked the UFS maintainer to reconsider to fix this.
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 17:17 ` Thomas Glanzmann
@ 2007-07-16 18:09 ` Thomas Glanzmann
2007-07-18 8:50 ` Thomas Glanzmann
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-16 18:09 UTC (permalink / raw)
To: Linus Torvalds; +Cc: GIT
Hello,
> I asked the UFS maintainer to reconsider to fix this.
the bug is filed.
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it ispossible to create hardlinks for directories as root under solaris)
2007-07-16 15:45 ` Brian Downing
2007-07-16 15:50 ` Brian Downing
@ 2007-07-16 19:57 ` David Kastrup
1 sibling, 0 replies; 25+ messages in thread
From: David Kastrup @ 2007-07-16 19:57 UTC (permalink / raw)
To: git
bdowning@lavos.net (Brian Downing) writes:
> On Mon, Jul 16, 2007 at 04:35:10PM +0200, Johannes Sixt wrote:
>> It tries to remove a *file* that is in the way and create the directory
>> in its place. But since your unlink() behaves incorrectly (it is
>> supposed to *fail* for directories), the logic does not quite work as
>> expected - it mistakes the directory for a file.
>
> http://www.opengroup.org/onlinepubs/007908799/xsh/unlink.html
>
> | The path argument must not name a directory unless the process has
> | appropriate privileges and the implementation supports using unlink() on
> | directories.
Isn't it funny? The problem with the git code is that the path
argument must not name a directory _if_ the process has appropriate
privileges and the implementation supports using unlink() on
directories.
Failure is not an option, it is a requirement.
--
David Kastrup, Kriemhildstr. 15, 44793 Bochum
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-16 18:09 ` Thomas Glanzmann
@ 2007-07-18 8:50 ` Thomas Glanzmann
2007-07-18 15:45 ` Linus Torvalds
0 siblings, 1 reply; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-18 8:50 UTC (permalink / raw)
To: Linus Torvalds; +Cc: GIT
Hello,
> > I asked the UFS maintainer to reconsider to fix this.
> the bug is filed.
http://bugs.opensolaris.org/view_bug.do?bug_id=6581318
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-18 8:50 ` Thomas Glanzmann
@ 2007-07-18 15:45 ` Linus Torvalds
2007-07-18 18:15 ` Thomas Glanzmann
0 siblings, 1 reply; 25+ messages in thread
From: Linus Torvalds @ 2007-07-18 15:45 UTC (permalink / raw)
To: Thomas Glanzmann; +Cc: GIT
On Wed, 18 Jul 2007, Thomas Glanzmann wrote:
>
> > the bug is filed.
>
> http://bugs.opensolaris.org/view_bug.do?bug_id=6581318
I don't think the recipient "got it", since it already has a comment about
"just use rmdir".
Can you please tell them that that isn't the point?
Linus
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris)
2007-07-18 15:45 ` Linus Torvalds
@ 2007-07-18 18:15 ` Thomas Glanzmann
0 siblings, 0 replies; 25+ messages in thread
From: Thomas Glanzmann @ 2007-07-18 18:15 UTC (permalink / raw)
To: Linus Torvalds; +Cc: GIT
Hello,
> I don't think the recipient "got it", since it already has a comment about
> "just use rmdir". Can you please tell them that that isn't the point?
I already did. They got the point. I have confidence that they fix it.
But there is no ETA. And the fix will not be backported.
Thomas
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2007-07-18 18:15 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-07-16 10:08 "git clone" executed as root on solaris 10 shreds UFS (it is possible to create hardlinks for directories as root under solaris) Thomas Glanzmann
2007-07-16 10:43 ` Thomas Glanzmann
2007-07-16 10:48 ` David Kastrup
2007-07-16 12:35 ` Thomas Glanzmann
2007-07-16 10:51 ` David Kastrup
2007-07-16 12:39 ` Thomas Glanzmann
2007-07-16 12:58 ` David Kastrup
2007-07-16 13:15 ` Thomas Glanzmann
2007-07-16 16:40 ` Thomas Glanzmann
2007-07-16 12:46 ` Thomas Glanzmann
2007-07-16 13:36 ` Thomas Glanzmann
2007-07-16 16:29 ` David Kastrup
2007-07-16 17:03 ` Linus Torvalds
2007-07-16 17:07 ` Linus Torvalds
2007-07-16 17:17 ` Thomas Glanzmann
2007-07-16 18:09 ` Thomas Glanzmann
2007-07-18 8:50 ` Thomas Glanzmann
2007-07-18 15:45 ` Linus Torvalds
2007-07-18 18:15 ` Thomas Glanzmann
2007-07-16 13:45 ` Thomas Glanzmann
2007-07-16 14:35 ` "git clone" executed as root on solaris 10 shreds UFS (it ispossible " Johannes Sixt
2007-07-16 15:45 ` Brian Downing
2007-07-16 15:50 ` Brian Downing
2007-07-16 19:57 ` David Kastrup
2007-07-16 17:12 ` Thomas Glanzmann
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).