[pagevec] resize pagevec to O(lg(NR_CPUS))

[William Lee Irwin III <wli@holomorphy.com>]

William Lee Irwin III <wli@holomorphy.com> wrote:
>> Reducing arrival rates by an Omega(NR_CPUS) factor would probably help,
>> though that may blow the stack on e.g. larger Altixen. Perhaps
>> O(lg(NR_CPUS)), e.g. NR_CPUS > 1 ? 4*lg(NR_CPUS) : 4 etc., will suffice,
>> though we may have debates about how to evaluate lg(n) at compile-time...
>> Would be nice if calls to sufficiently simple __attribute__((pure))
>> functions with constant args were considered constant expressions by gcc.

On Thu, Sep 09, 2004 at 04:22:45PM -0700, Andrew Morton wrote:
> Yes, that sort of thing.
> It wouldn't be surprising if increasing the pagevec up to 64 slots on big
> ia64 SMP provided a useful increase in some fs-intensive workloads.
> One needs to watch stack consumption though.

Okay, Marcelo, looks like we need to do cache alignment work with a
variable-size pagevec.

In order to attempt to compensate for arrival rates to zone->lru_lock
increasing as O(num_cpus_online()), this patch resizes the pagevec to
O(lg(NR_CPUS)) for lock amortization that adjusts better to the size of
the system. Compiletested on ia64.


Index: mm4-2.6.9-rc1/include/linux/pagevec.h
===================================================================
--- mm4-2.6.9-rc1.orig/include/linux/pagevec.h	2004-08-24 00:03:39.000000000 -0700
+++ mm4-2.6.9-rc1/include/linux/pagevec.h	2004-09-09 16:58:19.978150158 -0700
@@ -4,8 +4,27 @@
  * In many places it is efficient to batch an operation up against multiple
  * pages.  A pagevec is a multipage container which is used for that.
  */
+#include <linux/config.h>
+#include <linux/threads.h>
 
-#define PAGEVEC_SIZE	16
+#define __PAGEVEC_SIZE_0(n, k)						\
+	((k) * !!((unsigned long)(n) > (1ULL << (!(k) ? 0 : (k) - 1))	\
+			&& ((unsigned long)(n) <= (1ULL << (k)))))
+#define __PAGEVEC_SIZE_1(n, k)						\
+	(__PAGEVEC_SIZE_0(n, 2*(k)+1) + __PAGEVEC_SIZE_0(n, 2*(k)))
+#define __PAGEVEC_SIZE_2(n, k)						\
+	(__PAGEVEC_SIZE_1(n, 2*(k)+1) + __PAGEVEC_SIZE_1(n, 2*(k)))
+#define __PAGEVEC_SIZE_3(n, k)						\
+	(__PAGEVEC_SIZE_2(n, 2*(k)+1) + __PAGEVEC_SIZE_2(n, 2*(k)))
+#define __PAGEVEC_SIZE_4(n, k)						\
+	(__PAGEVEC_SIZE_3(n, 2*(k)+1) + __PAGEVEC_SIZE_3(n, 2*(k)))
+#define __PAGEVEC_SIZE_5(n, k)						\
+	(__PAGEVEC_SIZE_4(n, 2*(k)+1) + __PAGEVEC_SIZE_4(n, 2*(k)))
+#define __PAGEVEC_SIZE_6(n, k)						\
+	(__PAGEVEC_SIZE_5(n, 2*(k)+1) + __PAGEVEC_SIZE_5(n, 2*(k)))
+#define __PAGEVEC_SIZE(n)						\
+	(BITS_PER_LONG == 32 ? __PAGEVEC_SIZE_5(n, 0) : __PAGEVEC_SIZE_6(n, 0))
+#define PAGEVEC_SIZE		(NR_CPUS > 1 ? 4*__PAGEVEC_SIZE(NR_CPUS) : 4)
 
 struct page;
 struct address_space;