[PATCH] memcg: net: track network throttling due to memcg memory pressure

[PATCH] memcg: net: track network throttling due to memcg memory pressure

[Shakeel Butt]

The kernel can throttle network sockets if the memory cgroup associated
with the corresponding socket is under memory pressure. The throttling
actions include clamping the transmit window, failing to expand receive
or send buffers, aggressively prune out-of-order receive queue, FIN
deferred to a retransmitted packet and more. Let's add memcg metric to
indicate track such throttling actions.

At the moment memcg memory pressure is defined through vmpressure and in
future it may be defined using PSI or we may add more flexible way for
the users to define memory pressure, maybe through ebpf. However the
potential throttling actions will remain the same, so this newly
introduced metric will continue to track throttling actions irrespective
of how memcg memory pressure is defined.

Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
---
 Documentation/admin-guide/cgroup-v2.rst | 4 ++++
 include/linux/memcontrol.h              | 1 +
 include/net/sock.h                      | 6 +++++-
 kernel/cgroup/cgroup.c                  | 1 +
 mm/memcontrol.c                         | 3 +++
 5 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 0e6c67ac585a..057ee95e43ef 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -1515,6 +1515,10 @@ The following nested keys are defined.
           oom_group_kill
                 The number of times a group OOM has occurred.
 
+          socks_throttled
+                The number of times network sockets associated with
+                this cgroup are throttled.
+
   memory.events.local
 	Similar to memory.events but the fields in the file are local
 	to the cgroup i.e. not hierarchical. The file modified event
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 7ed15f858dc4..1434759eb111 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -52,6 +52,7 @@ enum memcg_memory_event {
 	MEMCG_SWAP_HIGH,
 	MEMCG_SWAP_MAX,
 	MEMCG_SWAP_FAIL,
+	MEMCG_SOCKS_THROTTLED,
 	MEMCG_NR_MEMORY_EVENTS,
 };
 
diff --git a/include/net/sock.h b/include/net/sock.h
index 60bcb13f045c..5a025eed08d1 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -2635,8 +2635,12 @@ static inline bool mem_cgroup_sk_under_memory_pressure(const struct sock *sk)
 #endif /* CONFIG_MEMCG_V1 */
 
 	do {
-		if (time_before64(get_jiffies_64(), mem_cgroup_get_socket_pressure(memcg)))
+		if (time_before64(get_jiffies_64(),
+				  mem_cgroup_get_socket_pressure(memcg))) {
+			memcg_memory_event(mem_cgroup_from_sk(sk),
+					   MEMCG_SOCKS_THROTTLED);
 			return true;
+		}
 	} while ((memcg = parent_mem_cgroup(memcg)));
 
 	return false;
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index fdee387f0d6b..8df671c59987 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -4704,6 +4704,7 @@ void cgroup_file_notify(struct cgroup_file *cfile)
 	}
 	spin_unlock_irqrestore(&cgroup_file_kn_lock, flags);
 }
+EXPORT_SYMBOL_GPL(cgroup_file_notify);
 
 /**
  * cgroup_file_show - show or hide a hidden cgroup file
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 3ae5cbcaed75..26b5fab19094 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -81,6 +81,7 @@ struct cgroup_subsys memory_cgrp_subsys __read_mostly;
 EXPORT_SYMBOL(memory_cgrp_subsys);
 
 struct mem_cgroup *root_mem_cgroup __read_mostly;
+EXPORT_SYMBOL(root_mem_cgroup);
 
 /* Active memory cgroup to use from an interrupt context */
 DEFINE_PER_CPU(struct mem_cgroup *, int_active_memcg);
@@ -4463,6 +4464,8 @@ static void __memory_events_show(struct seq_file *m, atomic_long_t *events)
 		   atomic_long_read(&events[MEMCG_OOM_KILL]));
 	seq_printf(m, "oom_group_kill %lu\n",
 		   atomic_long_read(&events[MEMCG_OOM_GROUP_KILL]));
+	seq_printf(m, "socks_throttled %lu\n",
+		   atomic_long_read(&events[MEMCG_SOCKS_THROTTLED]));
 }
 
 static int memory_events_show(struct seq_file *m, void *v)
-- 
2.47.3

Re: [PATCH] memcg: net: track network throttling due to memcg memory pressure

[Roman Gushchin]

Shakeel Butt <shakeel.butt@linux.dev> writes:

> The kernel can throttle network sockets if the memory cgroup associated
> with the corresponding socket is under memory pressure. The throttling
> actions include clamping the transmit window, failing to expand receive
> or send buffers, aggressively prune out-of-order receive queue, FIN
> deferred to a retransmitted packet and more. Let's add memcg metric to
> indicate track such throttling actions.
>
> At the moment memcg memory pressure is defined through vmpressure and in
> future it may be defined using PSI or we may add more flexible way for
> the users to define memory pressure, maybe through ebpf. However the
> potential throttling actions will remain the same, so this newly
> introduced metric will continue to track throttling actions irrespective
> of how memcg memory pressure is defined.
>
> Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> ---
>  Documentation/admin-guide/cgroup-v2.rst | 4 ++++
>  include/linux/memcontrol.h              | 1 +
>  include/net/sock.h                      | 6 +++++-
>  kernel/cgroup/cgroup.c                  | 1 +
>  mm/memcontrol.c                         | 3 +++
>  5 files changed, 14 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> index 0e6c67ac585a..057ee95e43ef 100644
> --- a/Documentation/admin-guide/cgroup-v2.rst
> +++ b/Documentation/admin-guide/cgroup-v2.rst
> @@ -1515,6 +1515,10 @@ The following nested keys are defined.
>            oom_group_kill
>                  The number of times a group OOM has occurred.
>  
> +          socks_throttled
> +                The number of times network sockets associated with
> +                this cgroup are throttled.

I'd prefer sockets_throttled or sock_throttled. And same for the
constant name.

Otherwise,
Acked-by: Roman Gushchin <roman.gushchin@linux.dev>

Thanks!

Re: [PATCH] memcg: net: track network throttling due to memcg memory pressure

[Kuniyuki Iwashima]

On Wed, Oct 15, 2025 at 6:40 PM Roman Gushchin <roman.gushchin@linux.dev> wrote:
>
> Shakeel Butt <shakeel.butt@linux.dev> writes:
>
> > The kernel can throttle network sockets if the memory cgroup associated
> > with the corresponding socket is under memory pressure. The throttling
> > actions include clamping the transmit window, failing to expand receive
> > or send buffers, aggressively prune out-of-order receive queue, FIN
> > deferred to a retransmitted packet and more. Let's add memcg metric to
> > indicate track such throttling actions.
> >
> > At the moment memcg memory pressure is defined through vmpressure and in
> > future it may be defined using PSI or we may add more flexible way for
> > the users to define memory pressure, maybe through ebpf. However the
> > potential throttling actions will remain the same, so this newly
> > introduced metric will continue to track throttling actions irrespective
> > of how memcg memory pressure is defined.
> >
> > Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> > ---
> >  Documentation/admin-guide/cgroup-v2.rst | 4 ++++
> >  include/linux/memcontrol.h              | 1 +
> >  include/net/sock.h                      | 6 +++++-
> >  kernel/cgroup/cgroup.c                  | 1 +
> >  mm/memcontrol.c                         | 3 +++
> >  5 files changed, 14 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> > index 0e6c67ac585a..057ee95e43ef 100644
> > --- a/Documentation/admin-guide/cgroup-v2.rst
> > +++ b/Documentation/admin-guide/cgroup-v2.rst
> > @@ -1515,6 +1515,10 @@ The following nested keys are defined.
> >            oom_group_kill
> >                  The number of times a group OOM has occurred.
> >
> > +          socks_throttled
> > +                The number of times network sockets associated with
> > +                this cgroup are throttled.
>
> I'd prefer sockets_throttled or sock_throttled. And same for the
> constant name.
>
> Otherwise,
> Acked-by: Roman Gushchin <roman.gushchin@linux.dev>

+1 for sock_ like "sock" in memory.stat and its MEMCG_SOCK.

Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>

Thanks!

Re: [PATCH] memcg: net: track network throttling due to memcg memory pressure

[Daniel Sedlak]

On 10/16/25 3:31 AM, Shakeel Butt wrote:
> The kernel can throttle network sockets if the memory cgroup associated
> with the corresponding socket is under memory pressure. The throttling
> actions include clamping the transmit window, failing to expand receive
> or send buffers, aggressively prune out-of-order receive queue, FIN
> deferred to a retransmitted packet and more. Let's add memcg metric to
> indicate track such throttling actions.
> 
> At the moment memcg memory pressure is defined through vmpressure and in
> future it may be defined using PSI or we may add more flexible way for
> the users to define memory pressure, maybe through ebpf. However the
> potential throttling actions will remain the same, so this newly
> introduced metric will continue to track throttling actions irrespective
> of how memcg memory pressure is defined.
> 
> Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>

Reviewed-by: Daniel Sedlak <daniel.sedlak@cdn77.com>

I am curious how the future work will unfold. If you need help with 
future developments I can help you, we have hundreds of servers where 
this throttling is happening.

Thanks!
Daniel

Re: [PATCH] memcg: net: track network throttling due to memcg memory pressure

[Shakeel Butt]

On Wed, Oct 15, 2025 at 10:46:54PM -0700, Kuniyuki Iwashima wrote:
> On Wed, Oct 15, 2025 at 6:40 PM Roman Gushchin <roman.gushchin@linux.dev> wrote:
> >
> > Shakeel Butt <shakeel.butt@linux.dev> writes:
> >
> > > The kernel can throttle network sockets if the memory cgroup associated
> > > with the corresponding socket is under memory pressure. The throttling
> > > actions include clamping the transmit window, failing to expand receive
> > > or send buffers, aggressively prune out-of-order receive queue, FIN
> > > deferred to a retransmitted packet and more. Let's add memcg metric to
> > > indicate track such throttling actions.
> > >
> > > At the moment memcg memory pressure is defined through vmpressure and in
> > > future it may be defined using PSI or we may add more flexible way for
> > > the users to define memory pressure, maybe through ebpf. However the
> > > potential throttling actions will remain the same, so this newly
> > > introduced metric will continue to track throttling actions irrespective
> > > of how memcg memory pressure is defined.
> > >
> > > Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> > > ---
> > >  Documentation/admin-guide/cgroup-v2.rst | 4 ++++
> > >  include/linux/memcontrol.h              | 1 +
> > >  include/net/sock.h                      | 6 +++++-
> > >  kernel/cgroup/cgroup.c                  | 1 +
> > >  mm/memcontrol.c                         | 3 +++
> > >  5 files changed, 14 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> > > index 0e6c67ac585a..057ee95e43ef 100644
> > > --- a/Documentation/admin-guide/cgroup-v2.rst
> > > +++ b/Documentation/admin-guide/cgroup-v2.rst
> > > @@ -1515,6 +1515,10 @@ The following nested keys are defined.
> > >            oom_group_kill
> > >                  The number of times a group OOM has occurred.
> > >
> > > +          socks_throttled
> > > +                The number of times network sockets associated with
> > > +                this cgroup are throttled.
> >
> > I'd prefer sockets_throttled or sock_throttled. And same for the
> > constant name.
> >
> > Otherwise,
> > Acked-by: Roman Gushchin <roman.gushchin@linux.dev>
> 
> +1 for sock_ like "sock" in memory.stat and its MEMCG_SOCK.
> 
> Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
> 

Thanks Roman and Kuniyuki, will change the name to sock_throttled in v2.

Re: [PATCH] memcg: net: track network throttling due to memcg memory pressure

[Shakeel Butt]

On Thu, Oct 16, 2025 at 12:42:19PM +0200, Daniel Sedlak wrote:
> On 10/16/25 3:31 AM, Shakeel Butt wrote:
> > The kernel can throttle network sockets if the memory cgroup associated
> > with the corresponding socket is under memory pressure. The throttling
> > actions include clamping the transmit window, failing to expand receive
> > or send buffers, aggressively prune out-of-order receive queue, FIN
> > deferred to a retransmitted packet and more. Let's add memcg metric to
> > indicate track such throttling actions.
> > 
> > At the moment memcg memory pressure is defined through vmpressure and in
> > future it may be defined using PSI or we may add more flexible way for
> > the users to define memory pressure, maybe through ebpf. However the
> > potential throttling actions will remain the same, so this newly
> > introduced metric will continue to track throttling actions irrespective
> > of how memcg memory pressure is defined.
> > 
> > Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> 
> Reviewed-by: Daniel Sedlak <daniel.sedlak@cdn77.com>

Thanks.

> 
> I am curious how the future work will unfold. If you need help with future
> developments I can help you, we have hundreds of servers where this
> throttling is happening.

I think first thing I would like to know if this patch is a good start
for your use-case of observability and debugging. What else do you need
for sufficient support for your use-case? I imagine that would be
tracepoints to extract more information on the source of the throttling.
If you don't mind, can you take a stab at that? In the long run, we want
more flexible definition of memcg memory pressure. Let us know of any
requirements you have for that. Thanks again for continuosly pushing
this conversation.

Re: [PATCH] memcg: net: track network throttling due to memcg memory pressure

[Daniel Sedlak]

On 10/16/25 6:02 PM, Shakeel Butt wrote:
> On Thu, Oct 16, 2025 at 12:42:19PM +0200, Daniel Sedlak wrote:
>> On 10/16/25 3:31 AM, Shakeel Butt wrote:
>> I am curious how the future work will unfold. If you need help with future
>> developments I can help you, we have hundreds of servers where this
>> throttling is happening.
> 
> I think first thing I would like to know if this patch is a good start
> for your use-case of observability and debugging.What else do you need
> for sufficient support for your use-case?

Yes, it is a good start, we can now hook this easily into our monitoring 
system and detect affected servers more easily.

> I imagine that would be
> tracepoints to extract more information on the source of the throttling.
> If you don't mind, can you take a stab at that?

We have some tracepoints that we have used for debugging this. We would 
like to upstream them, if that makes sense to you?

Thanks!
Daniel

Re: [PATCH] memcg: net: track network throttling due to memcg memory pressure

[Shakeel Butt]

On Fri, Oct 17, 2025 at 04:15:18PM +0200, Daniel Sedlak wrote:
> On 10/16/25 6:02 PM, Shakeel Butt wrote:
> > On Thu, Oct 16, 2025 at 12:42:19PM +0200, Daniel Sedlak wrote:
> > > On 10/16/25 3:31 AM, Shakeel Butt wrote:
> > > I am curious how the future work will unfold. If you need help with future
> > > developments I can help you, we have hundreds of servers where this
> > > throttling is happening.
> > 
> > I think first thing I would like to know if this patch is a good start
> > for your use-case of observability and debugging.What else do you need
> > for sufficient support for your use-case?
> 
> Yes, it is a good start, we can now hook this easily into our monitoring
> system and detect affected servers more easily.
> 
> > I imagine that would be
> > tracepoints to extract more information on the source of the throttling.
> > If you don't mind, can you take a stab at that?
> 
> We have some tracepoints that we have used for debugging this. We would like
> to upstream them, if that makes sense to you?

Yes please, send them out.