Re: page fault scalability patch V11 [0/7]: overview

Christoph Lameter
2004-11-22T17:44:02Z

On Sat, 20 Nov 2004, William Lee Irwin III wrote:
> I'm not particularly "stuck on" the per-cpu business, it was merely the
> most obvious method of splitting the RSS counter without catastrophes
> elsewhere. Robin Holt's 2.4 performance studies actually show that
> splitting the counter is not even essential.

There is no problem moving back to the atomic approach that is if it is
okay to also make anon_rss atomic. But its a pretty significant
performance hit (comparison with some old data from V4 of patch which
makes this data a bit suspect since the test environment is likely
slightly different. I should really test this again. Note that the old
performance test was only run 3 times instead of 10):

atomic vs. sloppy rss performance 64G allocation:

sloppy rss:

Gb Rep Threads User System Wall flt/cpu/s fault/wsec
16 10 1 1.818s 131.556s 133.038s 78618.592 78615.672
16 10 2 1.736s 121.167s 65.026s 85317.098 160656.362
16 10 4 1.835s 120.444s 36.002s 85751.810 291074.998
16 10 8 1.820s 131.068s 25.049s 78906.310 411304.895
16 10 16 3.275s 194.971s 22.019s 52892.356 472497.962
16 10 32 13.006s 496.628s 27.044s 20575.038 381999.865

atomic:

Gb Rep Threads User System Wall flt/cpu/s fault/wsec
16 3 1 0.610s 61.557s 62.016s 50600.438 50599.822
16 3 2 0.640s 83.116s 43.016s 37557.847 72869.978
16 3 4 0.621s 73.897s 26.023s 42214.002 119908.246
16 3 8 0.596s 86.587s 14.098s 36081.229 209962.059
16 3 16 0.646s 69.601s 7.000s 44780.269 448823.690
16 3 32 0.903s 185.609s 8.085s 16866.018 355301.694

Lets go for the approach to move rss into the thread structure but
keep the rss in the mm structure as is (need to take page_table_lock
for update) to consolidate the values. This allows to keep most
of the code as is and the rss in the task struct is only used if
we are not holding page_table_lock.

Maybe we can then find some way to regularly update the rss in the mm
structure to avoid the loop over the tasklist in proc.