RFC limit sk_mem_quantum to 8192

RFC limit sk_mem_quantum to 8192

[Flavio Leitner]

Hi,

The page size can be 64k on ppc64, so SK_MEM_QUANTUM increases to
that value as well.

net/ipv4/tcp.c:
...
        sysctl_tcp_wmem[0] = SK_MEM_QUANTUM;
        sysctl_tcp_wmem[1] = 16*1024;
        sysctl_tcp_wmem[2] = max(64*1024, max_share);

        sysctl_tcp_rmem[0] = SK_MEM_QUANTUM;
        sysctl_tcp_rmem[1] = 87380;
        sysctl_tcp_rmem[2] = max(87380, max_share);

and:
include/net/sock.h:
#define SK_MEM_QUANTUM ((int)PAGE_SIZE)

ppc64 config:
# CONFIG_PPC_4K_PAGES is not set
# CONFIG_PPC_16K_PAGES is not set
CONFIG_PPC_64K_PAGES=y
# CONFIG_PPC_256K_PAGES is not set

It seems too much for a minimum reserved memory. Also, the
other values are meaningless in this case because default
is only 16k and the maximum is limited to 64k.

This will require a modification in the
Documentation/networking/ip-sysctl.txt as well which states
that default minimum is 1 page.

Also, sk_mem_schedule() and friends will have to consider
that SK_MEM_QUANTUM might not be PAGE_SIZE anymore.

Well, the patch below illustrates what I am talking.
thanks,
fbl

diff --git a/include/net/sock.h b/include/net/sock.h
index 5d84de4..d52fa2d 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -954,7 +954,12 @@ static inline struct inode *SOCK_INODE(struct socket *socket)
 extern int __sk_mem_schedule(struct sock *sk, int size, int kind);
 extern void __sk_mem_reclaim(struct sock *sk);
 
+#if PAGE_SIZE < 8192
 #define SK_MEM_QUANTUM ((int)PAGE_SIZE)
+#else
+#define SK_MEM_QUANTUM ((int)8192)
+#endif
+
 #define SK_MEM_QUANTUM_SHIFT ilog2(SK_MEM_QUANTUM)
 #define SK_MEM_SEND	0
 #define SK_MEM_RECV	1

Re: RFC limit sk_mem_quantum to 8192

[Eric Dumazet]

On Tue, 2013-05-21 at 21:45 -0300, Flavio Leitner wrote:
> Hi,
> 
> The page size can be 64k on ppc64, so SK_MEM_QUANTUM increases to
> that value as well.
> 
> net/ipv4/tcp.c:
> ...
>         sysctl_tcp_wmem[0] = SK_MEM_QUANTUM;
>         sysctl_tcp_wmem[1] = 16*1024;
>         sysctl_tcp_wmem[2] = max(64*1024, max_share);
> 
>         sysctl_tcp_rmem[0] = SK_MEM_QUANTUM;
>         sysctl_tcp_rmem[1] = 87380;
>         sysctl_tcp_rmem[2] = max(87380, max_share);
> 
> and:
> include/net/sock.h:
> #define SK_MEM_QUANTUM ((int)PAGE_SIZE)
> 
> ppc64 config:
> # CONFIG_PPC_4K_PAGES is not set
> # CONFIG_PPC_16K_PAGES is not set
> CONFIG_PPC_64K_PAGES=y
> # CONFIG_PPC_256K_PAGES is not set
> 
> It seems too much for a minimum reserved memory. Also, the
> other values are meaningless in this case because default
> is only 16k and the maximum is limited to 64k.
> 
> This will require a modification in the
> Documentation/networking/ip-sysctl.txt as well which states
> that default minimum is 1 page.
> 
> Also, sk_mem_schedule() and friends will have to consider
> that SK_MEM_QUANTUM might not be PAGE_SIZE anymore.
> 
> Well, the patch below illustrates what I am talking.
> thanks,
> fbl
> 
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 5d84de4..d52fa2d 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -954,7 +954,12 @@ static inline struct inode *SOCK_INODE(struct socket *socket)
>  extern int __sk_mem_schedule(struct sock *sk, int size, int kind);
>  extern void __sk_mem_reclaim(struct sock *sk);
>  
> +#if PAGE_SIZE < 8192
>  #define SK_MEM_QUANTUM ((int)PAGE_SIZE)
> +#else
> +#define SK_MEM_QUANTUM ((int)8192)
> +#endif
> +
>  #define SK_MEM_QUANTUM_SHIFT ilog2(SK_MEM_QUANTUM)
>  #define SK_MEM_SEND	0
>  #define SK_MEM_RECV	1
> 

What particular problem do you want to solve ?

Wouldn't be easier to chose 4096 on all arches ?

Are you sure a network driver doesn't provide skb using a full page ?

Re: RFC limit sk_mem_quantum to 8192

[Flavio Leitner]

On Tue, May 21, 2013 at 06:09:22PM -0700, Eric Dumazet wrote:
> On Tue, 2013-05-21 at 21:45 -0300, Flavio Leitner wrote:
> > diff --git a/include/net/sock.h b/include/net/sock.h
> > index 5d84de4..d52fa2d 100644
> > --- a/include/net/sock.h
> > +++ b/include/net/sock.h
> > @@ -954,7 +954,12 @@ static inline struct inode *SOCK_INODE(struct socket *socket)
> >  extern int __sk_mem_schedule(struct sock *sk, int size, int kind);
> >  extern void __sk_mem_reclaim(struct sock *sk);
> >  
> > +#if PAGE_SIZE < 8192
> >  #define SK_MEM_QUANTUM ((int)PAGE_SIZE)
> > +#else
> > +#define SK_MEM_QUANTUM ((int)8192)
> > +#endif
> > +
> >  #define SK_MEM_QUANTUM_SHIFT ilog2(SK_MEM_QUANTUM)
> >  #define SK_MEM_SEND	0
> >  #define SK_MEM_RECV	1
> > 
> 
> What particular problem do you want to solve ?

So far there is no other problem besides the weird tcp_wmem.

> Wouldn't be easier to chose 4096 on all arches ?

Not sure what you're referring to. That config comes from a
distro kernel, so it's not under my control. If it is about
the upper limit for sk_mem_quantum, 4k seems enough to me as well.
 
> Are you sure a network driver doesn't provide skb using a full page ?

You lost me. You're saying that today we consider a page size
a minimum and so if we reduce that, the skb wouldn't fit in the
min sk memory?

Thanks,
-- 
fbl

Re: RFC limit sk_mem_quantum to 8192

[Eric Dumazet]

On Tue, 2013-05-21 at 22:58 -0300, Flavio Leitner wrote:
> On Tue, May 21, 2013 at 06:09:22PM -0700, Eric Dumazet wrote:
>  
> > Are you sure a network driver doesn't provide skb using a full page ?
> 
> You lost me. You're saying that today we consider a page size
> a minimum and so if we reduce that, the skb wouldn't fit in the
> min sk memory?

SK_MEM_QUANTUM is also used in UDP stack, thats why I am asking.

Re: RFC limit sk_mem_quantum to 8192

[Flavio Leitner]

On Tue, May 21, 2013 at 07:21:01PM -0700, Eric Dumazet wrote:
> On Tue, 2013-05-21 at 22:58 -0300, Flavio Leitner wrote:
> > On Tue, May 21, 2013 at 06:09:22PM -0700, Eric Dumazet wrote:
> >  
> > > Are you sure a network driver doesn't provide skb using a full page ?
> > 
> > You lost me. You're saying that today we consider a page size
> > a minimum and so if we reduce that, the skb wouldn't fit in the
> > min sk memory?
> 
> SK_MEM_QUANTUM is also used in UDP stack, thats why I am asking.

Yeah, it is. SCTP too, but for the protocol cases, the most
complex one appears to be TCP, and it doesn't seem to be a problem
to replace the minimum with something not page sized.

For the drivers, it seems to have an indirect assumption that
page size bytes is a minimum acceptable, so changing this minimum
could cause a performance issue.

Well, this define is quite old, so I am not sure if there are
other historical reasons to keep it page size. However, if the
idea of fixing SK_MEM_QUANTUM to 4k seems reasonable, I am
willing to spend more time digging into this.

Thanks!
-- 
fbl