[PATCH v4 0/2] mm/compaction: allow more aggressive proactive compaction

[PATCH v4 0/2] mm/compaction: allow more aggressive proactive compaction

[Michal Clapinski]

Our goal is to keep memory usage of a VM low on the host. For that
reason, we use free page reporting which by default reports free pages
of order 9 and larger to the host to be freed. The feature works well
only if the memory in the guest is not fragmented below pages of order
9. Proactive compaction can be reused to achieve defragmentation after
some parameter tweaking.

When the fragmentation score (lower is better) gets larger than the
high watermark, proactive compaction kicks in. Compaction stops when
the score goes below the low watermark (or no progress is made and
backoff kicks in). Let's define the difference between high and low
watermarks as leeway. Before these changes, the minimum possible value
for low watermark was 5 and the leeway was hardcoded to 10 (so minimum
possible value for high watermark was 15).

To test this, I created a VM with 19GB of memory and free page
reporting enabled. The VM was ~idle. I meassured the memory usage from
inside the guest (/proc/meminfo) and from the host (provided by the
hypervisor).
Before:
https://drive.google.com/file/d/1Xw23lRry_PgEH3f6QRnSGvoHh2u9UHyI/view?usp=sharing
After:
https://drive.google.com/file/d/1wMhpIzepx6t44F70yCPA50n1S5V2AT-a/view?usp=sharing

v4:
- Replace the leeway tunable with an equation based on proactiveness.
- Add graphs to prove it works.
v3: Remove gerrit ids from commit msgs.
v2: Change commit msgs and document the new sysctl.

Michal Clapinski (2):
  mm/compaction: remove low watermark cap for proactive compaction
  mm/compaction: reduce the difference between low and high watermarks

 Documentation/admin-guide/sysctl/vm.rst |  6 ++++++
 mm/compaction.c                         | 12 ++++--------
 2 files changed, 10 insertions(+), 8 deletions(-)

-- 
2.49.0.504.g3bcea36a83-goog

[PATCH v4 1/2] mm/compaction: remove low watermark cap for proactive compaction

[Michal Clapinski]

Previously a min cap of 5 has been set in the commit introducing
proactive compaction. This was to make sure users don't hurt themselves
by setting the proactiveness to 100 and making their system
unresponsive. But the compaction mechanism has a backoff mechanism that
will sleep for 30s if no progress is made, so I don't see a significant
risk here. My system (19GB of memory) has been perfectly fine with both
watermarks hardcoded to 0.

Signed-off-by: Michal Clapinski <mclapinski@google.com>
---
 mm/compaction.c | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index a3203d97123ea..4ff6b6e1db2da 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -2251,12 +2251,7 @@ static unsigned int fragmentation_score_wmark(bool low)
 {
 	unsigned int wmark_low;
 
-	/*
-	 * Cap the low watermark to avoid excessive compaction
-	 * activity in case a user sets the proactiveness tunable
-	 * close to 100 (maximum).
-	 */
-	wmark_low = max(100U - sysctl_compaction_proactiveness, 5U);
+	wmark_low = 100U - sysctl_compaction_proactiveness;
 	return low ? wmark_low : min(wmark_low + 10, 100U);
 }
 
-- 
2.49.0.504.g3bcea36a83-goog

[PATCH v4 2/2] mm/compaction: reduce the difference between low and high watermarks

[Michal Clapinski]

Reduce the diff between low and high watermarks when compaction
proactiveness is set to high. This allows users who set the
proactiveness really high to have more stable fragmentation score over
time.

Signed-off-by: Michal Clapinski <mclapinski@google.com>
---
 Documentation/admin-guide/sysctl/vm.rst | 6 ++++++
 mm/compaction.c                         | 5 +++--
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst
index f48eaa98d22d2..d716ff1f37b57 100644
--- a/Documentation/admin-guide/sysctl/vm.rst
+++ b/Documentation/admin-guide/sysctl/vm.rst
@@ -130,6 +130,12 @@ to latency spikes in unsuspecting applications. The kernel employs
 various heuristics to avoid wasting CPU cycles if it detects that
 proactive compaction is not being effective.
 
+Setting the value above 80 will, in addition to lowering the acceptable level
+of fragmentation, make the compaction code more sensitive to increases in
+fragmentation, i.e. compaction will trigger more often, but reduce
+fragmentation by a smaller amount.
+This makes the fragmentation level more stable over time.
+
 Be careful when setting it to extreme values like 100, as that may
 cause excessive background compaction activity.
 
diff --git a/mm/compaction.c b/mm/compaction.c
index 4ff6b6e1db2da..f29a09def4515 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -2249,10 +2249,11 @@ static unsigned int fragmentation_score_node(pg_data_t *pgdat)
 
 static unsigned int fragmentation_score_wmark(bool low)
 {
-	unsigned int wmark_low;
+	unsigned int wmark_low, leeway;
 
 	wmark_low = 100U - sysctl_compaction_proactiveness;
-	return low ? wmark_low : min(wmark_low + 10, 100U);
+	leeway = min(10U, wmark_low / 2);
+	return low ? wmark_low : min(wmark_low + leeway, 100U);
 }
 
 static bool should_proactive_compact_node(pg_data_t *pgdat)
-- 
2.49.0.504.g3bcea36a83-goog