Re: Multi-process on the same host

[Stephen Hemminger <stephen-OTpzqLSitTUnbdJkjeBofR2eb7JE58TQ@public.gmane.org>]

On Fri, 4 Oct 2013 13:47:02 +0200
Walter de Donato <walter.dedonato-7EJwN6hzp+g@public.gmane.org> wrote:

> Hello,
> 
> I've been using DPDK for a while and now I encountered the following issue:
> when I try to run two primary processes on the same host (with --no-shconf
> option enabled) respectively sending packets on one port and receiving them
> on a different port (the two ports are directly connected with a CAT-6
> cable), I get this error on the receiving process:
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x00000000004158a0 in rte_eth_rx_burst (port_id=0 '\000', queue_id=0,
> rx_pkts=0x7ffff5baa8f0, nb_pkts=128) at
> /home/devel/dpdk/build/include/rte_ethdev.h:1658
> 1658            return (*dev->rx_pkt_burst)(dev->data->rx_queues[queue_id],
> rx_pkts, nb_pkts);
> 
> To give some more details:
> - the options given to the two processes:
>   ./receiver -c 0x3 -n 2 -m 200 --no-shconf -- -p 0x1
>   ./sender -c 0xc -n 2 -m 200 --no-shconf -- -p 0x2
>   where the -p option is the binary mask to select the ports to enable.
> - the network card is a dualport Intel X540:
>   port 0: Intel Corporation Ethernet Controller 10 Gigabit X540-AT2 (rev 01)
>   port 1: Intel Corporation Ethernet Controller 10 Gigabit X540-AT2 (rev 01)
> - this is the hugeadm --pool-list output:
>       Size  Minimum  Current  Maximum  Default
>   1073741824        2        2        2        *
> 
> My first question is: should it be possible to let separate primary
> processes coexist if they use different resources (cores, ports, memory
> pools)?
> 
> A second question is: there is any other workaround to let this scenario
> work without merging the two processes into two lcores of the same process?
> 
> Thanks in advance,
> -Walter

The problem is that huge TLB filesystem is a shared resource.
Because of that the memory pools of the two processes overlap, and memory pools
are used for packet buffers, malloc, etc.

You might be able to use no-huge, but then other things would probably break.