netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy v4-RESEND
@ 2015-12-08 15:30 Johannes Weiner
  2015-12-08 15:30 ` [PATCH 01/14] mm: memcontrol: export root_mem_cgroup Johannes Weiner
                   ` (15 more replies)
  0 siblings, 16 replies; 33+ messages in thread
From: Johannes Weiner @ 2015-12-08 15:30 UTC (permalink / raw)
  To: Andrew Morton; +Cc: linux-mm, netdev, cgroups, linux-kernel

Hi Andrew,

there was some build breakage in CONFIG_ combinations I hadn't tested
in the last revision, so here is a fixed-up resend with minimal CC
list. The only difference to the previous version is a section in
memcontrol.h, but it accumulates throughout the series and would have
been a pain to resolve on your end. So here goes. This also includes
the review tags that Dave and Vlad had sent out in the meantime.

Difference to the original v4:

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 9a19590..189f04d 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -702,14 +702,14 @@ static inline void mem_cgroup_wb_stats(struct bdi_writeback *wb,
 
 #endif	/* CONFIG_CGROUP_WRITEBACK */
 
-#ifdef CONFIG_INET
 struct sock;
-extern struct static_key_false memcg_sockets_enabled_key;
-#define mem_cgroup_sockets_enabled static_branch_unlikely(&memcg_sockets_enabled_key)
 void sock_update_memcg(struct sock *sk);
 void sock_release_memcg(struct sock *sk);
 bool mem_cgroup_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages);
 void mem_cgroup_uncharge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages);
+#if defined(CONFIG_MEMCG) && defined(CONFIG_INET)
+extern struct static_key_false memcg_sockets_enabled_key;
+#define mem_cgroup_sockets_enabled static_branch_unlikely(&memcg_sockets_enabled_key)
 static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg)
 {
 #ifdef CONFIG_MEMCG_KMEM
@@ -724,7 +724,11 @@ static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg)
 }
 #else
 #define mem_cgroup_sockets_enabled 0
-#endif /* CONFIG_INET */
+static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg)
+{
+	return false;
+}
+#endif
 
 #ifdef CONFIG_MEMCG_KMEM
 extern struct static_key_false memcg_kmem_enabled_key;
diff --git a/include/linux/vmpressure.h b/include/linux/vmpressure.h
index a77b142..3347cc3 100644
--- a/include/linux/vmpressure.h
+++ b/include/linux/vmpressure.h
@@ -43,7 +43,7 @@ extern int vmpressure_register_event(struct mem_cgroup *memcg,
 extern void vmpressure_unregister_event(struct mem_cgroup *memcg,
 					struct eventfd_ctx *eventfd);
 #else
-static inline void vmpressure(gfp_t gfp, struct mem_cgroup *memcg,
+static inline void vmpressure(gfp_t gfp, struct mem_cgroup *memcg, bool tree,
 			      unsigned long scanned, unsigned long reclaimed) {}
 static inline void vmpressure_prio(gfp_t gfp, struct mem_cgroup *memcg,
 				   int prio) {}


 Documentation/kernel-parameters.txt |   4 +
 include/linux/memcontrol.h          |  75 ++++++---
 include/linux/vmpressure.h          |   7 +-
 include/net/sock.h                  | 149 ++---------------
 include/net/tcp.h                   |   5 +-
 include/net/tcp_memcontrol.h        |   1 -
 mm/backing-dev.c                    |   2 +-
 mm/memcontrol.c                     | 302 ++++++++++++++++++++++------------
 mm/vmpressure.c                     |  78 ++++++---
 mm/vmscan.c                         |  10 +-
 net/core/sock.c                     |  78 ++-------
 net/ipv4/tcp.c                      |   3 +-
 net/ipv4/tcp_ipv4.c                 |   9 +-
 net/ipv4/tcp_memcontrol.c           |  82 ++++-----
 net/ipv4/tcp_output.c               |   7 +-
 net/ipv6/tcp_ipv6.c                 |   3 -
 16 files changed, 391 insertions(+), 424 deletions(-)

^ permalink raw reply related	[flat|nested] 33+ messages in thread
* [PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy
@ 2015-11-12 23:41 Johannes Weiner
  2015-11-12 23:41 ` [PATCH 01/14] mm: memcontrol: export root_mem_cgroup Johannes Weiner
  0 siblings, 1 reply; 33+ messages in thread
From: Johannes Weiner @ 2015-11-12 23:41 UTC (permalink / raw)
  To: David Miller, Andrew Morton
  Cc: Vladimir Davydov, Tejun Heo, Michal Hocko, netdev, linux-mm,
	cgroups, linux-kernel, kernel-team

Hi,

this is version 3 of the patches to add socket memory accounting to
the unified hierarchy memory controller. Changes since v2 include:

- Fixed an underflow bug in the mem+swap counter that came through the
  design of the per-cpu charge cache. To fix that, the unused mem+swap
  counter is now fully patched out on unified hierarchy. Double whammy.

- Restored the counting jump label such that the networking callbacks
  get patched out again when the last memory-controlled cgroup goes
  away. The code was already there, so we might as well keep it.

- Broke down the massive tcp_memcontrol rewrite patch into smaller
  logical pieces to (hopefully) make it easier to review and verify.

---

Socket buffer memory can make up a significant share of a workload's
memory footprint that can be directly linked to userspace activity,
and so it needs to be part of the memory controller to provide proper
resource isolation/containment.

Historically, socket buffers were accounted in a separate counter,
without any pressure equalization between anonymous memory, page
cache, and the socket buffers. When the socket buffer pool was
exhausted, buffer allocations would fail hard and cause network
performance to tank, regardless of whether there was still memory
available to the group or not. Likewise, struggling anonymous or cache
workingsets could not dip into an idle socket memory pool. Because of
this, the feature was not usable for many real life applications.

To not repeat this mistake, the new memory controller will account all
types of memory pages it is tracking on behalf of a cgroup in a single
pool. Upon pressure, the VM reclaims and shrinks and puts pressure on
whatever memory consumer in that pool is within its reach.

For socket memory, pressure feedback is provided through vmpressure
events. When the VM has trouble freeing memory, the network code is
instructed to stop growing the cgroup's transmit windows.

This series begins with a rework of the existing tcp memory controller
that simplifies and cleans up the code while allowing us to have only
one set of networking hooks for both memory controller versions. The
original behavior of the existing tcp controller should be preserved.

It then adds socket accounting to the v2 memory controller, including
the use of the per-cpu charge cache and async memory.high enforcement
from socket memory charges.

Lastly, vmpressure is hooked up to the socket code so that it stops
growing transmit windows when the VM has trouble reclaiming memory.

 include/linux/memcontrol.h   |  71 ++++++----
 include/net/sock.h           | 149 ++------------------
 include/net/tcp.h            |   5 +-
 include/net/tcp_memcontrol.h |   1 -
 mm/backing-dev.c             |   2 +-
 mm/memcontrol.c              | 303 +++++++++++++++++++++++++++--------------
 mm/vmpressure.c              |  25 +++-
 mm/vmscan.c                  |  31 +++--
 net/core/sock.c              |  78 +++--------
 net/ipv4/tcp.c               |   3 +-
 net/ipv4/tcp_ipv4.c          |   9 +-
 net/ipv4/tcp_memcontrol.c    |  85 ++++--------
 net/ipv4/tcp_output.c        |   7 +-
 net/ipv6/tcp_ipv6.c          |   3 -
 14 files changed, 353 insertions(+), 419 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 33+ messages in thread
end of thread, other threads:[~2016-01-22  3:25 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-12-08 15:30 [PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy v4-RESEND Johannes Weiner
2015-12-08 15:30 ` [PATCH 01/14] mm: memcontrol: export root_mem_cgroup Johannes Weiner
2015-12-08 15:30 ` [PATCH 02/14] net: tcp_memcontrol: properly detect ancestor socket pressure Johannes Weiner
2015-12-08 15:30 ` [PATCH 03/14] net: tcp_memcontrol: remove bogus hierarchy pressure propagation Johannes Weiner
2015-12-08 15:30 ` [PATCH 04/14] net: tcp_memcontrol: protect all tcp_memcontrol calls by jump-label Johannes Weiner
2015-12-08 15:30 ` [PATCH 05/14] net: tcp_memcontrol: remove dead per-memcg count of allocated sockets Johannes Weiner
2015-12-08 15:30 ` [PATCH 06/14] net: tcp_memcontrol: simplify the per-memcg limit access Johannes Weiner
2015-12-08 15:30 ` [PATCH 07/14] net: tcp_memcontrol: sanitize tcp memory accounting callbacks Johannes Weiner
2015-12-08 15:30 ` [PATCH 08/14] net: tcp_memcontrol: simplify linkage between socket and page counter Johannes Weiner
2015-12-08 15:30 ` [PATCH 09/14] mm: memcontrol: generalize the socket accounting jump label Johannes Weiner
2015-12-08 15:30 ` [PATCH 10/14] mm: memcontrol: do not account memory+swap on unified hierarchy Johannes Weiner
2015-12-08 15:30 ` [PATCH 11/14] mm: memcontrol: move socket code for unified hierarchy accounting Johannes Weiner
2015-12-08 15:30 ` [PATCH 12/14] mm: memcontrol: account socket memory in unified hierarchy memory controller Johannes Weiner
2015-12-15 19:50   ` Michal Hocko
2015-12-08 15:30 ` [PATCH 13/14] mm: memcontrol: hook up vmpressure to socket pressure Johannes Weiner
2015-12-08 15:30 ` [PATCH 14/14] mm: memcontrol: switch to the updated jump-label API Johannes Weiner
2015-12-08 16:28   ` David Miller
2015-12-08 16:28 ` [PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy v4-RESEND David Miller
     [not found] ` <1449588624-9220-1-git-send-email-hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2015-12-09 16:31   ` Arnd Bergmann
2015-12-09 16:32     ` [PATCH] mm: memcontrol: only manage socket pressure for CONFIG_INET Arnd Bergmann
2015-12-09 18:58       ` Johannes Weiner
     [not found]         ` <20151209185858.GA2342-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2015-12-09 22:28           ` Andrew Morton
2015-12-09 23:05             ` Johannes Weiner
2015-12-09 23:13               ` Andrew Morton
2016-01-22  3:25                 ` Masanari Iida
2015-12-09 16:32     ` [PATCH] mm: memcontrol: MEMCG no longer works with SLOB Arnd Bergmann
2015-12-09 20:01       ` Johannes Weiner
2015-12-09 21:03         ` Arnd Bergmann
     [not found]         ` <20151209200107.GA17409-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2015-12-10 11:24           ` Vladimir Davydov
2015-12-09 18:17     ` [PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy v4-RESEND Johannes Weiner
  -- strict thread matches above, loose matches on Subject: below --
2015-11-12 23:41 [PATCH 00/14] mm: memcontrol: account socket memory in unified hierarchy Johannes Weiner
2015-11-12 23:41 ` [PATCH 01/14] mm: memcontrol: export root_mem_cgroup Johannes Weiner
     [not found]   ` <1447371693-25143-2-git-send-email-hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>
2015-11-13 15:59     ` David Miller
2015-11-14 12:17   ` Vladimir Davydov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).