[RFC PATCH 0/4] mm/zswap: make shrinker writeback work with iocost

[Alexandre Ghiti <alex@ghiti.fr>]

We want to enable the zswap shrinker on our fleet, which runs iocost,
but with it on, a large production workload is slower than with it
off. Two reasons:

1. zswap writeback is REQ_SWAP, so the IO controllers issue it as root
   and charge it to the cgroup afterwards, as debt. The workload pays it
   back: its swapin and page cache reads wait, and iocost eventually
   stalls its allocations, which in turn stalls the application.
   Patches 1-3 let zswap writeback be throttled instead, by marking it
   REQ_BACKGROUND.

2. A throttled write sleeps in its submitter, which for the shrinker is
   whoever entered reclaim, often an application thread in a page fault.
   Patch 4 moves that writeback to a per-lruvec kworker.

Large production workload, 10 hosts per arm, CPU matched, one 24h run
per comparison. Change with this series:

                     vs shrinker off    vs stock shrinker
  p50                    -0.7%               -5.8%
  p99                    -0.2%               -6.5%
  memory PSI full        -52%                -94%
  zswap pool             -97%                +12%

The benchmarks below run with iocost enabled, in a cgroup whose
memory.max (the "cap") is smaller than their working set, so they are
under constant memcg reclaim.

UCacheBench, 12G cap, 5 runs:

                     shrinker off    stock shrinker    this series
  ops/s                  469k             280k             634k
  errors                  0%               44%              0%
  p99.9                 5.8 ms           16.6 ms          4.3 ms

MySQL, sysbench OLTP, 256M cap, 5 runs per host:

                     shrinker off    stock shrinker    this series
  TPS, host 1            OOM               413              609
  TPS, host 2            OOM               399              666
  p99, host 1            OOM             334 ms            90 ms
  p99, host 2            OOM             273 ms            44 ms

FeedSim, 16G cap:

                     shrinker off    stock shrinker    this series
  QPS                    4.7              30.2             32.4
  p95                  1306 ms           495 ms           494 ms

kbuild, defconfig -j4, 600M cap, 3 runs per host:

                     shrinker off    stock shrinker    this series
  build time, host 1    784 s             788 s            778 s
  build time, host 2    691 s             696 s            680 s

Alexandre Ghiti (4):
  block: do not issue background swap bios as root
  mm/zswap: turn shrink_memcg_cb()'s argument into a flags word
  mm/zswap: allow writeback to be throttled by the cgroup IO controllers
  mm/zswap: move reclaim-driven writeback to a kworker

 block/blk-cgroup.h       |   8 ++-
 include/linux/swap_ops.h |   1 +
 include/linux/zswap.h    |   4 ++
 mm/page_io.c             |   7 +-
 mm/zswap.c               | 148 ++++++++++++++++++++++++++++++++++-----
 5 files changed, 146 insertions(+), 22 deletions(-)


base-commit: fe2ec83746e501645709761605c2464a44fd2929
-- 
2.53.0-Meta