Re: rte_eth_tx_queue_setup() failing (error -22) when setting up tx queues on a VMXNET3 port...

[Stephen Hemminger <stephen@networkplumber.org>]

On Wed, 25 Nov 2015 10:32:33 +0000
"Montorsi, Francesco" <fmontorsi@empirix.com> wrote:

> Hi all,
> I have a server running VMWare ESXi 5.5.0 and VMWare vCenter 5.5.0. On such server I created a VM with two VMXNET3 NIC cards (one for management, the other one should be used with DPDK to enable fast-RX of packets coming from other VMs / bare-metal NICs).
> 
> Inside the VM I have successfully created 2MB hugepages, loaded igb_uio driver and binded the 2nd VMXNET3 NIC to igb_uio driver:
> 
> ----------------------------------------------------------------------------------------------------------------------
> $ ./dpdk_nic_bind.py --status
> 
> Network devices using DPDK-compatible driver
> ============================================
> 0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=vmxnet3
> 
> Network devices using kernel driver
> ===================================
> 0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3 unused=igb_uio *Active*
> 
> Other network devices
> =====================
> <none>
> ----------------------------------------------------------------------------------------------------------------------
> 
> However when I start my DPDK-application I get the following error during TX queue initialization:
> 
> ERR rte_eth_tx_queue_setup: err=-22, port=0: Unknown error -22
> 
> This is the code that I'm using:
> 
>               ...
>                rte_eth_dev_info_get(m_portid, m_dev_info);
> 
> 
> 	// proceed with configuration
> 
> 	struct rte_eth_conf port_conf;
> 	memset(&port_conf, 0, sizeof(port_conf));
> 
> 	port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
> 		/** The multi-queue packet distribution mode to be used, e.g. RSS.; this is important to use multiple RX queues per port ID */
> 	port_conf.rxmode.max_rx_pkt_len = ETHER_MAX_LEN;
> 	port_conf.rxmode.hw_strip_crc   = 1; /**< enable CRC stripping by hardware */
> 	port_conf.rxmode.jumbo_frame    = 1; /**< Jumbo Frame Support enabled */
> 	//port_conf.rxmode.enable_lro     = 1; /**< Enable LRO */		// NOT SUPPORTED ON TESTED HW
> 	port_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP;
> 	port_conf.txmode.mq_mode = ETH_MQ_TX_NONE;
> 
> 	m_num_queues = MIN(64, m_dev_info->max_rx_queues);
> 	m_num_queues = MIN(128, m_num_queues);
> 	ret = rte_eth_dev_configure(m_portid, m_num_queues, 1 /* number of tx queues */, &port_conf);
> 	if (ret < 0)
> 	{
> 		HMLogError("HwEmulDPDKPort::init() rte_eth_dev_configure: err=%d, port=%u: %s", ret, m_portid, rte_strerror(ret));
> 		return false;
> 	}
> 
> 	// init one TX queue: even if we never ever TX packets, at least 1 queue is needed!
> --->	ret = rte_eth_tx_queue_setup(m_portid, 0 /* queue ID */, 64 /* num descriptors */, rte_eth_dev_socket_id(m_portid), NULL);
> 	if (ret < 0)
> 	{
> 		// retry with just 1 descriptor
> 
> --->		ret = rte_eth_tx_queue_setup(m_portid, 0 /* queue ID */, 1 /* num descriptors */, rte_eth_dev_socket_id(m_portid), NULL);
> 		if (ret < 0)
> 		{
> 			HMLogError("HwEmulDPDKPort::init() rte_eth_tx_queue_setup: err=%d, port=%u: %s", ret, m_portid, rte_strerror(ret));
> 			return false;
> 		}
> 	}
> 
> 
> Basically since I want to only receive packets (no TX) I'm not really interested in TX queues / num of TX descriptors... However I have troubles decrypting this -22 error code... any hint?

First advice, DPDK is open source, therefore you have the source use it.

Error codes match Linux/Unix errno's. You can use strerror(-ret) to find the value
or look in errno.h. Hint -22 == EINVAL

If you read the source, you will see that it there are log messages enabled if you
configure with LIBRTE_ETHDEV_DEBUG enabled and rebuild DPDK. And there are log messages
for VMXNETE3 controlled by LIBRTE_VMXNET3_DEBUG_INIT