Re: stat very inefficient

Peter Chubb
2004-07-29T07:27:10Z

>>>>> "viro" == viro <viro@parcelfarce.linux.theplanet.co.uk> writes:

>> On Wed, 28 Jul 2004 17:08:37 -0700 Chris Wedgwood <cw@f00f.org>
>> wrote:
>>
>> > Just How bad is it for you? I just tested stat on my crapbox and
>> for > a short path 1M stats takes 0.5s and for a longer path (30
>> bytes or > so) 2.8s.
>>
>> Run "time find . -type f" on the kernel tree, both before and after
>> removing the third unnecessary copy.

viro> ... with hot cache, otherwise IO time will dominate. I don't
viro> disagree with you, but in all realistic cases I can think of
viro> it's going to be noise (e.g. this find over kernel tree is
viro> almost certainly followed by xargs grep, etc.).

With hot cache the system time is really small.

On a 2GHz Pentium 4, Compare
find .-type f -mtime -2000 >/dev/null
with
find . -type f -mtime -2000
in a freshly checked out 2.8 kernel tree.

(the -mtime test is to force a stat, otherwise, as Ulrich says, almost
no stat system calls will take place)

to xterm >/dev/null | xargs grep foo
sys 0.34 0.103 0.35
user 0.29 0.08 0.104
real 18.551 0.204 220.25

Using strace reveals that around 60% of the system time in the
redirected to /dev/null case is lstat64 --- 41465 calls, 1.5usec per
call. Where it's in a pipe that uses the files, the time is swamped
by the time to process the files, and the time spent in write() ---
lstat64 drops to around 16% of the time in find.

The nice thing about the current three-copy implementation is that
it's simple and obviously correct. Personally, I don't think that the
increased complexity of arhcitecture-specific callbacks, etc., is
worth the small performance gain.

--
Dr Peter Chubb http://www.gelato.unsw.edu.au peterc AT gelato.unsw.edu.au
The technical we do immediately, the political takes *forever*