ixgbe network card has dev_info.max_rx_queues == 0

ixgbe network card has dev_info.max_rx_queues == 0

[Sergey Mironov]

Hi. I have face a strange error on one of my network cards. Call to
rte_eth_dev_configure returns with error code -22. Increaing the
verbosity level shows the following:


PMD: rte_eth_dev_configure: ethdev port_id=2 nb_rx_queues=3 > 0
EAL: Error - exiting with code: 1

here is the snippet of code which returns the error


./lib/librte_ether/rte_ethdev.c : 513

    (*dev->dev_ops->dev_infos_get)(dev, &dev_info);
    if (nb_rx_q > dev_info.max_rx_queues) {
        PMD_DEBUG_TRACE("ethdev port_id=%d nb_rx_queues=%d > %d\n",
                port_id, nb_rx_q, dev_info.max_rx_queues);
        return (-EINVAL);
    }

What does this error means (what is rx queues of an adapter?) What may
cause such a problem? I am using dpdk 1.5.1r1.

Thanks in advance,
Sergey

Re: ixgbe network card has dev_info.max_rx_queues == 0

[Alex Markuze]

RX and TX Are short hand for Receive and Transmit Queues.
These Queues Store the in/egress packets.

Just looking at the info you've sent it tells you that max_rx_queues
for this dev is 0 (Clearly something is wrong here) so the nb_rx_q
which is 3 is an Invalid Value -EINVAL == -22.

On Thu, Aug 21, 2014 at 3:26 PM, Sergey Mironov <grrwlf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi. I have face a strange error on one of my network cards. Call to
> rte_eth_dev_configure returns with error code -22. Increaing the
> verbosity level shows the following:
>
>
> PMD: rte_eth_dev_configure: ethdev port_id=2 nb_rx_queues=3 > 0
> EAL: Error - exiting with code: 1
>
> here is the snippet of code which returns the error
>
>
> ./lib/librte_ether/rte_ethdev.c : 513
>
>     (*dev->dev_ops->dev_infos_get)(dev, &dev_info);
>     if (nb_rx_q > dev_info.max_rx_queues) {
>         PMD_DEBUG_TRACE("ethdev port_id=%d nb_rx_queues=%d > %d\n",
>                 port_id, nb_rx_q, dev_info.max_rx_queues);
>         return (-EINVAL);
>     }
>
> What does this error means (what is rx queues of an adapter?) What may
> cause such a problem? I am using dpdk 1.5.1r1.
>
> Thanks in advance,
> Sergey

Re: ixgbe network card has dev_info.max_rx_queues == 0

[Sergey Mironov]

Thanks, I understand that. I see in the ixgbe's code, that
dev_info.max_rx_queues is initialized from the hw->mac.max_rx_queues
value

    dev_info->max_rx_queues = (uint16_t)hw->mac.max_rx_queues;
    dev_info->max_tx_queues = (uint16_t)hw->mac.max_tx_queues;

but I can't figure out how this hw->mac is produced. Is it mapped to
the memory region controlled by the network adapter directly?



2014-08-21 16:52 GMT+04:00 Alex Markuze <alex-I7wstvAgZDk@public.gmane.org>:
> RX and TX Are short hand for Receive and Transmit Queues.
> These Queues Store the in/egress packets.
>
> Just looking at the info you've sent it tells you that max_rx_queues
> for this dev is 0 (Clearly something is wrong here) so the nb_rx_q
> which is 3 is an Invalid Value -EINVAL == -22.
>
> On Thu, Aug 21, 2014 at 3:26 PM, Sergey Mironov <grrwlf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> Hi. I have face a strange error on one of my network cards. Call to
>> rte_eth_dev_configure returns with error code -22. Increaing the
>> verbosity level shows the following:
>>
>>
>> PMD: rte_eth_dev_configure: ethdev port_id=2 nb_rx_queues=3 > 0
>> EAL: Error - exiting with code: 1
>>
>> here is the snippet of code which returns the error
>>
>>
>> ./lib/librte_ether/rte_ethdev.c : 513
>>
>>     (*dev->dev_ops->dev_infos_get)(dev, &dev_info);
>>     if (nb_rx_q > dev_info.max_rx_queues) {
>>         PMD_DEBUG_TRACE("ethdev port_id=%d nb_rx_queues=%d > %d\n",
>>                 port_id, nb_rx_q, dev_info.max_rx_queues);
>>         return (-EINVAL);
>>     }
>>
>> What does this error means (what is rx queues of an adapter?) What may
>> cause such a problem? I am using dpdk 1.5.1r1.
>>
>> Thanks in advance,
>> Sergey

Fwd: ixgbe network card has dev_info.max_rx_queues == 0

[Sergey Mironov]

Forgot to CC to all

---------- Forwarded message ----------
From: Sergey Mironov <grrwlf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Date: 2014-08-25 13:40 GMT+04:00
Subject: Re: [dpdk-dev] ixgbe network card has dev_info.max_rx_queues == 0
To: Alex Markuze <alex-I7wstvAgZDk@public.gmane.org>


Hi. I've found more data on this issue. My machine (CB-8970) is
equipped with two ixgbe and six e1000 ports + one more e1000 port as
management interface.
I ask the DPDK to allocate 3 TX queues and 3 RX queues for each of 8
working ports, 6 * 8 = 48 queues (see the configuration code snippet
at the end of the letter)

As a result, I see the message I posted in my first letter:

  PMD: rte_eth_dev_configure: ethdev port_id=2 nb_rx_queues=3 > 0
  EAL: Error - exiting with code: 1

At the same time, if I configure only two ixgbe ports by filtering out
others with --use-device, everything works fine. It gives me the idea,
that my system has a limited resource 'number of queues' and all the
cards share it. Am I correct? Is it possible to view the total amount
of queues available for my system?

Regards,
Sergey


PS configuration code


    for (portid = 0; portid < nb_ports; portid++) {
        ret = rte_eth_dev_configure((uint8_t) portid,  // portid
                                    nb_lcores, //  nb_lcores == 3
                                    nb_lcores, // nb_lcores == 3
                                    &port_conf);
        if (ret < 0) {
            rte_exit(EXIT_FAILURE, "Cannot configure device: "
                     "err=%d, port=%u\n",
                     ret, portid);
        }

        ...
    }




2014-08-23 20:15 GMT+04:00 Alex Markuze <alex-I7wstvAgZDk@public.gmane.org>:
> I can onlt gues. But it is common to use memory mapped IO that is. The cpu
> can access some part of the nics memory via the pci bus just like it would
> acess its own RAM.
> So its possible these region addresses are stored in a BAR. Base address
> region.
>
> On Aug 22, 2014 10:12 AM, "Sergey Mironov" <grrwlf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>
>> Thanks, I understand that. I see in the ixgbe's code, that
>> dev_info.max_rx_queues is initialized from the hw->mac.max_rx_queues
>> value
>>
>>     dev_info->max_rx_queues = (uint16_t)hw->mac.max_rx_queues;
>>     dev_info->max_tx_queues = (uint16_t)hw->mac.max_tx_queues;
>>
>> but I can't figure out how this hw->mac is produced. Is it mapped to
>> the memory region controlled by the network adapter directly?
>>
>>
>>
>> 2014-08-21 16:52 GMT+04:00 Alex Markuze <alex-I7wstvAgZDk@public.gmane.org>:
>> > RX and TX Are short hand for Receive and Transmit Queues.
>> > These Queues Store the in/egress packets.
>> >
>> > Just looking at the info you've sent it tells you that max_rx_queues
>> > for this dev is 0 (Clearly something is wrong here) so the nb_rx_q
>> > which is 3 is an Invalid Value -EINVAL == -22.
>> >
>> > On Thu, Aug 21, 2014 at 3:26 PM, Sergey Mironov <grrwlf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>> > wrote:
>> >> Hi. I have face a strange error on one of my network cards. Call to
>> >> rte_eth_dev_configure returns with error code -22. Increaing the
>> >> verbosity level shows the following:
>> >>
>> >>
>> >> PMD: rte_eth_dev_configure: ethdev port_id=2 nb_rx_queues=3 > 0
>> >> EAL: Error - exiting with code: 1
>> >>
>> >> here is the snippet of code which returns the error
>> >>
>> >>
>> >> ./lib/librte_ether/rte_ethdev.c : 513
>> >>
>> >>     (*dev->dev_ops->dev_infos_get)(dev, &dev_info);
>> >>     if (nb_rx_q > dev_info.max_rx_queues) {
>> >>         PMD_DEBUG_TRACE("ethdev port_id=%d nb_rx_queues=%d > %d\n",
>> >>                 port_id, nb_rx_q, dev_info.max_rx_queues);
>> >>         return (-EINVAL);
>> >>     }
>> >>
>> >> What does this error means (what is rx queues of an adapter?) What may
>> >> cause such a problem? I am using dpdk 1.5.1r1.
>> >>
>> >> Thanks in advance,
>> >> Sergey

Re: Fwd: ixgbe network card has dev_info.max_rx_queues == 0

[Stephen Hemminger]

On Mon, 25 Aug 2014 13:41:34 +0400
Sergey Mironov <grrwlf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> Hi. I've found more data on this issue. My machine (CB-8970) is
> equipped with two ixgbe and six e1000 ports + one more e1000 port as
> management interface.
> I ask the DPDK to allocate 3 TX queues and 3 RX queues for each of 8
> working ports, 6 * 8 = 48 queues (see the configuration code snippet
> at the end of the letter)

You need to look at dev_info and not allocate more queues than the
hardware supports. Many E1000 devices only support a single queue.

Re: Fwd: ixgbe network card has dev_info.max_rx_queues == 0

[Sergey Mironov]

Yep, I understand now. The manual [1] says that intel 827574
controller (controlled by e1000 driver) has only 2 ports per port.
Thanks for the help.
Sergey

[1] - http://www.intel.com/content/www/us/en/ethernet-controllers/i211-ethernet-controller-datasheet.html

PS
I actually have another machine SCB-8973 equipped with i211
controller, where DPDK returns number of queues - zero. Probably, it
is a hardware bug, but I'll better start another thread for this
issue.


2014-08-26 4:14 GMT+04:00 Stephen Hemminger <stephen-OTpzqLSitTUnbdJkjeBofR2eb7JE58TQ@public.gmane.org>:
> On Mon, 25 Aug 2014 13:41:34 +0400
> Sergey Mironov <grrwlf-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
>> Hi. I've found more data on this issue. My machine (CB-8970) is
>> equipped with two ixgbe and six e1000 ports + one more e1000 port as
>> management interface.
>> I ask the DPDK to allocate 3 TX queues and 3 RX queues for each of 8
>> working ports, 6 * 8 = 48 queues (see the configuration code snippet
>> at the end of the letter)
>
> You need to look at dev_info and not allocate more queues than the
> hardware supports. Many E1000 devices only support a single queue.