Re: Making diff(1) of linux kernels faster

Marcelo Tosatti
2001-10-17T18:06:03Z

On Wed, 17 Oct 2001, Linus Torvalds wrote:

>
> On Wed, 17 Oct 2001, Paul Gortmaker wrote:
> >
> > Oh, and prereading the dirs of both trees (vs. just one and letting
> > normal execution read in the 2nd) seems to offer better improvements.
> > (Steady stream of requests results in better merging perhaps?)
>
> That doesn't make much sense, but I'll take your word for it. Does this
> behaviour show up on 2.4.x too? It sounds like a performance buglet in the
> kernel or some infrastructure, really.
>
> The one problem with pre-reading is that it will now artificially touch
> the data twice, and when running on 2.4.x it will activate the pages.
> That's going to be exactly what _I_ want it to do on my machine, but
> others are likely to be less happy about it.
>
> Btw, why use "slurp()" and actually doing the memory allocations etc, only
> to throw it away again? It would be better to either really keep the
> allocation around (which would also fix the touch-twice issue but would
> cause much more changes to 'diff'), or to just read into the same buffer
> over and over again..
>
> And I've for a long time thought about adding a "readahead()" system call.
> There are just too many uses for it, it has come up in many different
> areas..

There is a paper on USENIX 2001 which does implement directory readahead
and it shows huge improvements for some workload.

I'll dig it down and see if I can find that.