Re: git status: small difference between stating whole repository and small subdirectory

[Jeff King <peff@peff.net>]

On Thu, Feb 16, 2012 at 02:37:47PM +0100, Piotr Krukowiecki wrote:

> >> $ time git status  -- .
> >> real    0m2.503s
> >> user    0m0.160s
> >> sys     0m0.096s
> >>
> >> $ time git status
> >> real    0m9.663s
> >> user    0m0.232s
> >> sys     0m0.556s
> >
> > Did you drop caches here, too?
> 
> Yes I did - with cache the status takes something like 0.1-0.3s on whole repo.

OK, then that makes sense. It's pretty much just I/O on the filesystem
and on the object db.

You can break status down a little more to see which is which. Try "git
update-index --refresh" to see just how expensive the lstat and index
handling is.

And then try "git diff-index HEAD" for an idea of how expensive it is to
just read the objects and compare to the index.

> > Not really. You're showing an I/O problem, and repacking is git's way of
> > reducing I/O.
> 
> So if I understand correctly, the reason is because git must compare
> workspace files with packed objects - and the problem is
> reading/seeking/searching in the packs?

Mostly reading (we keep a sorted index and access the packfiles via
mmap, so we only touch the pages we need). But you're also paying to
lstat() the directory tree, too. And you're paying to load (probably)
the whole index into memory, although it's relatively compact compared
to the actual file data.

> Is there a way to make packs better? I think most operations are on
> workdir files - so maybe it'd be possible to tell gc/repack/whatever
> to optimize access to files which I currently have in workdir?

It already does optimize for that case. If you can make it even better,
I'm sure people would be happy to see the numbers.

Mostly I think it is just the case that disk I/O is slow, and the
operation you're asking for has to do a certain amount of it. What kind
of disk/filesystem are you pulling off of?

It's not a fuse filesystem by any chance, is it? I have a repo on an
encfs-mounted filesystem, and the lstat times are absolutely horrific.

-Peff