From mboxrd@z Thu Jan  1 00:00:00 1970
From: Zoltan Kiss <zoltan.kiss@linaro.org>
Subject: Re: Calling rte_eth_rx_burst() multiple times
Date: Thu, 15 Oct 2015 11:23:08 +0100
Message-ID: <561F7E8C.1080508@linaro.org>
References: <561F64BA.2000502@ndsl.kaist.edu>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
To: Younghwan Go <yhwan@ndsl.kaist.edu>, dev@dpdk.org
Return-path: <dev-bounces@dpdk.org>
Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com
 [209.85.212.178]) by dpdk.org (Postfix) with ESMTP id 743C88E90
 for <dev@dpdk.org>; Thu, 15 Oct 2015 12:23:08 +0200 (CEST)
Received: by wijp11 with SMTP id p11so22140530wij.0
 for <dev@dpdk.org>; Thu, 15 Oct 2015 03:23:08 -0700 (PDT)
In-Reply-To: <561F64BA.2000502@ndsl.kaist.edu>
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>



On 15/10/15 09:32, Younghwan Go wrote:
> Hi,
>
> I'm pretty new to playing with DPDK. I was trying to see if I can always
> receive MAX_BURST packets by calling rte_eth_rx_burst() multiple times
> on same <port, queue> pair (code shown below). I'm using DPDK-2.1.0 on 2
> dual-port Intel 82599ES 10Gbps NICs with Ubuntu 14.04.3 (kernel
> 3.13.0-63-generic).
>
> Since packet processing is slower (~10 Gbps) than pure RX speed (~40
> Gbps), I assumed rte_eth_rx_burst() would always receive some number of
> packets, eventually filling up MAX_BURST. But for multi-core case (4
> CPUs, 4 ports), rte_eth_rx_burst() starts to always return 0 after some
> time, causing all cores to be blocked forever. Analyzing the DPDK code
> (drivers/net/ixgbe/ixgbe_rxtx.c), I'm seeing that inside
> ixgbe_rx_scan_hw_ring() function, "rxdp->wb.upper.status.error" always
> returns 0 (where is this value set by the way?).

I think it is set by the hardware.

>
> I didn't see this problem for single-core case, in which it returned
> MAX_BURST packets at every rte_eth_rx_burst() call. Also, if I break out
> of while loop when I receive 0, I keep receiving packets in next <port,
> queue> pairs. Does anyone know why this block might happen? Or am I not
> allowed to call rte_eth_rx_burst() multiple times on same <port, queue>
> pair if I get 0? Any help will be great! Thank you!

Although not mentioned in the documentation itself, as far as I know 
rte_eth_rx_burst() is not thread-safe. If you look in to receive 
functions, there are no locking anywhere. You should call it on separate 
queues from different threads, and configure e.g RSS to distribute the 
traffic by the hardware.

>
> ------------------------------------------------------------------------
> int cnt = MAX_BURST; // MAX_BURST = 32
> int off = 0;
> do {
>      ret = rte_eth_rx_burst(port_id, queue_id, &m_table[off], cnt);
>      if (ret == 0) {
>          // don't break out but continue
>      } else if (ret > 0) {
>          off += ret;
>          cnt -= ret;
>      }
> } while (cnt > 0);
> ------------------------------------------------------------------------
>
> Best,
> Younghwan