Re: [patch] socket.7: Document SO_INCOMING_CPU

[Eric Dumazet <eric.dumazet@gmail.com>]

On Wed, 2017-04-19 at 15:20 +0200, Michael Kerrisk (man-pages) wrote:
> Ping Eric!
> 
> Would you have a chance to review the proposed text below, please.
> 
> Thanks,
> 

Hi Michael

Sorry for the delay.

Note that setting the option is not supported if SO_REUSEPORT is used.

Socket will be selected from an array, either by a hash or BPF program
that has no access to this information.

Thanks !


> Michael
> 
> On 02/19/2017 09:55 PM, Michael Kerrisk (man-pages) wrote:
> > [CC += Eric, so that he might review]
> > 
> > Hello Francois,
> > 
> > On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
> >> This socket option is undocumented. Applies on the latest version
> >> (man-pages-4.09-511).
> >>
> >> diff --git a/man7/socket.7 b/man7/socket.7
> >> index 3efd7a5d8..1a3ffa253 100644
> >> --- a/man7/socket.7
> >> +++ b/man7/socket.7
> >> @@ -490,6 +490,26 @@ flag on a socket
> >>  operation.
> >>  Expects an integer boolean flag.
> >>  .TP
> >> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
> >> Linux 4.4)"
> >> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
> >> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
> >> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
> >> +.sp
> >> +.in +4n
> >> +.nf
> >> +int cpu = 1;
> >> +socklen_t len = sizeof(cpu);
> >> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
> >> +.fi
> >> +.in
> >> +.sp
> >> +The typical use case is one listener per RX queue, as the associated listener
> >> +should only accept flows handled in softirq by the same cpu.  This provides
> >> +optimal NUMA behavior and keep cpu caches hot.
> >> +.TP
> >>  .B SO_KEEPALIVE
> >>  Enable sending of keep-alive messages on connection-oriented sockets.
> >>  Expects an integer boolean flag.
> > 
> > Thank you! Patch applied.
> > 
> > I have tried to enhance the description somewhat. I'm not sure whether
> > what I've written is quite correct (or whether it should be further
> > extended). Eric, could you please take a look at the following, and let 
> > me know if anything needs fixing:
> > 
> >        SO_INCOMING_CPU  (gettable  since Linux 3.19, settable since Linux
> >        4.4)
> >               Sets or gets the CPU affinity  of  a  socket.   Expects  an
> >               integer flag.
> > 
> >                   int cpu = 1;
> >                   socklen_t len = sizeof(cpu);
> >                   setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
> > 
> >               Because  all  of the packets for a single stream (i.e., all
> >               packets for the same 4-tuple) arrive on the single RX queue
> >               that  is  associated with a particular CPU, the typical use
> >               case is to employ one listening process per RX queue,  with
> >               the  incoming  flow being handled by a listener on the same
> >               CPU that is handling the RX queue.  This  provides  optimal
> >               NUMA behavior and keeps CPU caches hot.
> > 
> > Cheers,
> > 
> > Michael
> > 
> 
>