Mellanox ethernet card failed when probing

Mellanox ethernet card failed when probing

[Jean-Michel Hautbois]

Hi all,

I am using a HP server with Mellanox ethernet cards in it.
I have compiled the latest linux 3.2 version, and I am encountering
the following error :

[ 1094.644193] mlx4_core: Mellanox ConnectX core driver v1.1 (Dec, 2011)
[ 1094.644196] mlx4_core: Initializing 0000:06:00.0
[ 1097.183671] mlx4_core 0000:06:00.0: irq 84 for MSI/MSI-X
[ 1097.183681] mlx4_core 0000:06:00.0: irq 85 for MSI/MSI-X
[ 1097.183690] mlx4_core 0000:06:00.0: irq 86 for MSI/MSI-X
[ 1097.183697] mlx4_core 0000:06:00.0: irq 87 for MSI/MSI-X
[ 1097.201267] mlx4_core: Initializing 0000:06:00.1
[ 1097.201509] mlx4_core 0000:06:00.1: Multiple PFs not yet supported.
Skipping PF.
[ 1097.201844] mlx4_core: probe of 0000:06:00.1 failed with error -22
[ 1097.201853] mlx4_core: Initializing 0000:06:00.2
[ 1097.202078] mlx4_core 0000:06:00.2: Multiple PFs not yet supported.
Skipping PF.
[ 1097.202237] mlx4_core: probe of 0000:06:00.2 failed with error -22
[ 1097.202246] mlx4_core: Initializing 0000:06:00.3
[ 1097.202460] mlx4_core 0000:06:00.3: Multiple PFs not yet supported.
Skipping PF.
[ 1097.202911] mlx4_core: probe of 0000:06:00.3 failed with error -22
[ 1097.202919] mlx4_core: Initializing 0000:06:00.4
[ 1097.203143] mlx4_core 0000:06:00.4: Multiple PFs not yet supported.
Skipping PF.
[ 1097.203301] mlx4_core: probe of 0000:06:00.4 failed with error -22
[ 1097.203308] mlx4_core: Initializing 0000:06:00.5
[ 1097.203521] mlx4_core 0000:06:00.5: Multiple PFs not yet supported.
Skipping PF.
[ 1097.203937] mlx4_core: probe of 0000:06:00.5 failed with error -22
[ 1097.203945] mlx4_core: Initializing 0000:06:00.6
[ 1097.204167] mlx4_core 0000:06:00.6: Multiple PFs not yet supported.
Skipping PF.
[ 1097.204322] mlx4_core: probe of 0000:06:00.6 failed with error -22
[ 1097.204328] mlx4_core: Initializing 0000:06:00.7
[ 1097.204541] mlx4_core 0000:06:00.7: Multiple PFs not yet supported.
Skipping PF.
[ 1097.209445] mlx4_core: probe of 0000:06:00.7 failed with error -22
[ 1097.239493] mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.0 (Dec 2011)
[ 1097.239682] mlx4_en 0000:06:00.0: Activating port:1
[ 1097.241144] mlx4_en: eth8: Using 8 TX rings
[ 1097.241147] mlx4_en: eth8: Using 4 RX rings
[ 1097.241285] mlx4_en: eth8: Initializing port
[ 1097.241355] mlx4_en 0000:06:00.0: Activating port:2
[ 1097.244236] mlx4_en: eth9: Using 8 TX rings
[ 1097.244239] mlx4_en: eth9: Using 4 RX rings
[ 1097.244316] mlx4_en: eth9: Initializing port
[ 1097.544700] mlx4_en: eth8: Link Up
[ 1097.798226] mlx4_en: eth9_rename: Link Up

I think this is firmware related, but I don't know for sure.
I can't find any firmware generated in the /lib/firmware directory...

Thanks in advance for your help.
Regards,
JM

Re: Mellanox ethernet card failed when probing

[Eric Dumazet]

Le vendredi 03 février 2012 à 16:13 +0100, Jean-Michel Hautbois a
écrit :

> I think this is firmware related, but I don't know for sure.
> I can't find any firmware generated in the /lib/firmware directory...

Take a look here :

http://git.kernel.org/?p=linux/kernel/git/firmware/linux-firmware.git;a=summary

Re: Mellanox ethernet card failed when probing

[Jean-Michel Hautbois]

2012/2/3 Eric Dumazet <eric.dumazet@gmail.com>:
> Le vendredi 03 février 2012 à 16:13 +0100, Jean-Michel Hautbois a
> écrit :
>
>> I think this is firmware related, but I don't know for sure.
>> I can't find any firmware generated in the /lib/firmware directory...
>
> Take a look here :
>
> http://git.kernel.org/?p=linux/kernel/git/firmware/linux-firmware.git;a=summary
>

I thought it could be a firmware problem, but when I activate
debug_level it tells me "FW version 2.8.000" so, it loads it...
I have a probe issue, -22 means invalid value, but I don't know why.
Maybe will I need to debug it, in order to find the line which fails
and why, but I wondered if some people in here could have answers to
it :).
My device is a 15b3:6764 (Mellanox MT2648).

JM

Re: Mellanox ethernet card failed when probing

[Rick Jones]

On 02/03/2012 07:13 AM, Jean-Michel Hautbois wrote:
> Hi all,
>
> I am using a HP server with Mellanox ethernet cards in it.
> I have compiled the latest linux 3.2 version, and I am encountering
> the following error :
>
> [ 1094.644193] mlx4_core: Mellanox ConnectX core driver v1.1 (Dec, 2011)
> [ 1094.644196] mlx4_core: Initializing 0000:06:00.0
> [ 1097.183671] mlx4_core 0000:06:00.0: irq 84 for MSI/MSI-X
> [ 1097.183681] mlx4_core 0000:06:00.0: irq 85 for MSI/MSI-X
> [ 1097.183690] mlx4_core 0000:06:00.0: irq 86 for MSI/MSI-X
> [ 1097.183697] mlx4_core 0000:06:00.0: irq 87 for MSI/MSI-X
> [ 1097.201267] mlx4_core: Initializing 0000:06:00.1
> [ 1097.201509] mlx4_core 0000:06:00.1: Multiple PFs not yet supported.
> Skipping PF.
> [ 1097.201844] mlx4_core: probe of 0000:06:00.1 failed with error -22
 > ...
> I think this is firmware related, but I don't know for sure.
> I can't find any firmware generated in the /lib/firmware directory...

That error message seems to relate to SRIOV support:

                 /* We reset the device and enable SRIOV only for physical
                  * devices.  Try to claim ownership on the device;
                  * if already taken, skip -- do not allow multiple PFs */
                 err = mlx4_get_ownership(dev);
                 if (err) {
                         if (err < 0)
                                 goto err_free_dev;
                         else {
                                 mlx4_warn(dev, "Multiple PFs not yet 
supported."
                                           " Skipping PF.\n");
                                 err = -EINVAL;
                                 goto err_free_dev;
                         }
                 }


Isn't SRIOV support a "work in progress" for the Mellanox devices?  The 
rest of the logging (snipped) seems to suggest that the ".0" PFs did 
initialize correctly, so are you simply worried about the messages in 
the log?

rick jones

Fwd: Mellanox ethernet card failed when probing

[Jean-Michel Hautbois]

2012/2/3 Rick Jones <rick.jones2@hp.com>
>
> On 02/03/2012 07:13 AM, Jean-Michel Hautbois wrote:
>>
>> Hi all,
>>
>> I am using a HP server with Mellanox ethernet cards in it.
>> I have compiled the latest linux 3.2 version, and I am encountering
>> the following error :
>>
>> [ 1094.644193] mlx4_core: Mellanox ConnectX core driver v1.1 (Dec, 2011)
>> [ 1094.644196] mlx4_core: Initializing 0000:06:00.0
>> [ 1097.183671] mlx4_core 0000:06:00.0: irq 84 for MSI/MSI-X
>> [ 1097.183681] mlx4_core 0000:06:00.0: irq 85 for MSI/MSI-X
>> [ 1097.183690] mlx4_core 0000:06:00.0: irq 86 for MSI/MSI-X
>> [ 1097.183697] mlx4_core 0000:06:00.0: irq 87 for MSI/MSI-X
>> [ 1097.201267] mlx4_core: Initializing 0000:06:00.1
>> [ 1097.201509] mlx4_core 0000:06:00.1: Multiple PFs not yet supported.
>> Skipping PF.
>> [ 1097.201844] mlx4_core: probe of 0000:06:00.1 failed with error -22
>
> > ...
>
>> I think this is firmware related, but I don't know for sure.
>> I can't find any firmware generated in the /lib/firmware directory...
>
>
> That error message seems to relate to SRIOV support:
>
>                /* We reset the device and enable SRIOV only for physical
>                 * devices.  Try to claim ownership on the device;
>                 * if already taken, skip -- do not allow multiple PFs */
>                err = mlx4_get_ownership(dev);
>                if (err) {
>                        if (err < 0)
>                                goto err_free_dev;
>                        else {
>                                mlx4_warn(dev, "Multiple PFs not yet supported."
>                                          " Skipping PF.\n");
>                                err = -EINVAL;
>                                goto err_free_dev;
>                        }
>                }
>
>
> Isn't SRIOV support a "work in progress" for the Mellanox devices?  The rest of the logging (snipped) seems to suggest that the ".0" PFs did initialize correctly, so are you simply worried about the messages in the log?
>
> rick jones


I am not worried only with this message, but when I am looking at my
devices, even if the mlx4_core module is loaded, I don't have my eth
device.
Initializing is ok, but probe fails, so I can't use the device, this
is what I am worried about :).
I have two types of devices : be2net and mlx4. I want to have bonding
between them, and using an old kernel (debian lenny, 2.6.26), and the
drivers directly recompiled from mellanox website I can get it work.
Using the latest kernel (3.4.0-rc2) and using "modprobe mlx4_en" I get
the same messages related to mlx4_core, but it loads some ethernet
ports.

When booting, it detects the mlx and udev loads mlx4_core, not mlx4_en...
I will try to force it (but I am sure there is a clean way to do that ;-)).

JM