Re: [PATCH v3] Threaded grep

[Linus Torvalds <torvalds@linux-foundation.org>]

On Tue, 19 Jan 2010, Fredrik Kuivinen wrote:
> > 
> >  - git grep --no-ext-grep function
> > 
> > 	real	0m0.241s
> > 	user	0m1.436s
> > 	sys	0m0.216s
> 
> I haven't seen this overhead during my tests. But I'm _guessing_ that
> the problem is that the mutex grep_lock is held while the result is
> written to stdout. So when we are writing, no significant amount of
> work can be done by any thread. Here is a patch to fix this (applies
> on to of v3).

No, I get basically the same timings with this patch:

	real	0m0.239s
	user	0m1.372s
	sys	0m0.280s

(that _may_ be a slight real improvement, but it's more likely that the 
changing in user/sys time is just noise).

But I do think that you're right that there's a difference in my timings, 
and I am comparing to the one that uses strstr(). Your patches didn't have 
a "disable threads" option that I could see, so I just compared against my 
old numbers. 

So I guess a better example is to use a "complex" grep pattern like 
'function.*a' which also gets a lot of hits, and the threaded case does 
look much better in comparison. Pre-threaded:

	real	0m0.921s
	user	0m0.728s
	sys	0m0.184s

so it's less than 2x the CPU time, and a 3.85x real-time improvement. And 
that "less than 2x the CPU time" is the factor I'd expect from HT.

It's also worth noting that at least _some_ versions of glibc would 
actually use different versions of subroutines for the threaded vs 
non-threaded case, ie they'd avoid locking overhead when they know they 
aren't running with threads. So things like stdio actually got slower when 
you did any pthreads things, because suddenly glibc knew that it needed to 
do locking around the functions.

			Linus