Re: [PATCH iproute2] lib/libnetlink: ensure a minimum of 32KB for the buffer used in rtnl_recvmsg()

[Stephen Hemminger <stephen@networkplumber.org>]

On Mon, 29 May 2023 15:29:51 +0300
Gal Pressman <gal@nvidia.com> wrote:

> On 13/02/2019 4:04, David Ahern wrote:
> > On 2/12/19 6:58 PM, Eric Dumazet wrote:  
> >> In the past, we tried to increase the buffer size up to 32 KB in order
> >> to reduce number of syscalls per dump.
> >>
> >> Commit 2d34851cd341 ("lib/libnetlink: re malloc buff if size is not enough")
> >> brought the size back to 4KB because the kernel can not know the application
> >> is ready to receive bigger requests.
> >>
> >> See kernel commits 9063e21fb026 ("netlink: autosize skb lengthes") and
> >> d35c99ff77ec ("netlink: do not enter direct reclaim from netlink_dump()")
> >> for more details.
> >>
> >> Fixes: 2d34851cd341 ("lib/libnetlink: re malloc buff if size is not enough")
> >> Signed-off-by: Eric Dumazet <edumazet@google.com>
> >> Cc: Hangbin Liu <liuhangbin@gmail.com>
> >> Cc: Phil Sutter <phil@nwl.cc>
> >> ---
> >>  lib/libnetlink.c | 2 ++
> >>  1 file changed, 2 insertions(+)
> >>
> >> diff --git a/lib/libnetlink.c b/lib/libnetlink.c
> >> index 1892a02ab5d0d73776c9882ffc77edcd2c663d01..0d48a3d43cf03065dacbd419578ab10af56431a4 100644
> >> --- a/lib/libnetlink.c
> >> +++ b/lib/libnetlink.c
> >> @@ -718,6 +718,8 @@ static int rtnl_recvmsg(int fd, struct msghdr *msg, char **answer)
> >>  	if (len < 0)
> >>  		return len;
> >>  
> >> +	if (len < 32768)
> >> +		len = 32768;
> >>  	buf = malloc(len);
> >>  	if (!buf) {
> >>  		fprintf(stderr, "malloc error: not enough buffer\n");
> >>  
> > 
> > I believe that negates the whole point of 2d34851cd341 - which I have no
> > problem with. 2 recvmsg calls per message is overkill.
> > 
> > Do we know of any single message sizes > 32k? 2d34851cd341 cites
> > increasing VF's but at some point there is a limit. If not, the whole
> > PEEK thing should go away and we just malloc 32k (or 64k) buffers for
> > each recvmsg.
> >   
> 
> Hey,
> 
> Sorry for reviving this old thread, but I see this topic was already
> discussed here :).
> I have a system where the large number of VFs result in a message
> greater than 32k, which makes a simple 'ip link' command return an error.
> 
> Should we change the kernel's 'max_recvmsg_len' to 64k? Any other (more
> robust) ideas to resolve this issue?

No matter what the size, someone will always have too many VF's to fit
in the response. There is no way to get a stable solution without doing
some API changes.

It is possible to dump millions of routes, so it is not directly a netlink
issue more that the current API is slamming all the VF's as info blocks
under a single message response.

That would mean replacing IFLA_VFINFO_LIST with a separate query