Re: [pagevec] resize pagevec to O(lg(NR_CPUS))

[Nick Piggin <nickpiggin@yahoo.com.au>]

William Lee Irwin III wrote:

> On Fri, Sep 10, 2004 at 02:49:15PM -0300, Marcelo Tosatti wrote:
> 
>>Oops, right. wli's patch is borked for NUMA. Clamping it at 64 should
>>do fine.
> 
> 
> No, it DTRT. Batching does not directly compensate for increases in
> arrival rates, rather most directly compensates for increases to lock
> transfer times, which do indeed increase on systems with large numbers
> of cpus.
> 

Generally though I think you could expect the lru lock to be most
often taken by the scanner by node local CPUs. Even on the big
systems. We'll see.

> 
> On Fri, Sep 10, 2004 at 02:56:11PM +1000, Nick Piggin wrote:
> 
>>>Secondly is that you'll might really start putting pressure on small L1
>>>caches (eg. Itanium 2) if you bite off too much in one go. If you blow
>>>it, you'll have to pull all the pages into cache again as you process
>>>the pagevec.
> 
> 
> On Fri, Sep 10, 2004 at 02:49:15PM -0300, Marcelo Tosatti wrote:
> 
>>Whats the L1 cache size of Itanium2? Each page is huge compared to the pagevec
>>structure (you need a 64 item pagevec array on 64-bits to occupy the space of 
>>one 4KB page). So I think you wont blow up the cache even with a really big 
>>pagevec.
> 
> 
> A 511 item pagevec is 4KB on 64-bit machines.
> 

Sure. And when you fill it with pages, they'll use up 32KB of dcache
by using a single 64B line per page. Now that you've blown the cache,
when you go to move those pages to another list, you'll have to pull
them out of L2 again one at a time.

OK, so a 511 item pagevec is pretty unlikely. How about a 64 item one
with 128 byte cachelines, and you're touching two cachelines per
page operation? That's 16K.