Re: [Intel-wired-lan] Bonding problem on Intel X710 hardware

[Sven Anders <sven.anders@anduras.de>]

Hallo Dave,

I will setup a new test environment these days to go deeper in the
analysis. Then I can compare different kernel versions and I will
try to reproduce this issue with an small script. I will check, if
I can reproduce this even with a major linux distri (Ubuntu?).

If I have any result, I will get back to you...

Regards
  Sven

Am 21.06.22 um 23:18 schrieb Switzer, David:
> 
> 
>> -----Original Message-----
>> From: Switzer, David
>> Sent: Thursday, June 16, 2022 1:31 PM
>> To: Sven Anders <sven.anders@anduras.de>; intel-wired-lan@osuosl.org
>> Subject: RE: [Intel-wired-lan] Bonding problem on Intel X710 hardware
>>
>>> -----Original Message-----
>>> From: Sven Anders <sven.anders@anduras.de>
>>> Sent: Monday, June 13, 2022 12:44 AM
>>> To: Switzer, David <david.switzer@intel.com>;
>>> intel-wired-lan@osuosl.org
>>> Subject: Re: [Intel-wired-lan] Bonding problem on Intel X710 hardware
>>>
>>> Hallo Dave,
>> Hi Sven,
> 
> Hello Sven!
>>
>>>
>>> thanks for your answer.
>> You're very welcome!
>>
>>>
>>> I tried to disable the LLDP engine, as suggested, but it did not change
>>> anything.
>>>
>>> To remark: I do not use LACP, I'm trying to use "Adaptive Load Balancing"
>>> which works with ARP negotiation.
>>> If I'm using the "active-backup" bonding mode, it all works fine. But
>>> this is nearly the same as using only one interface, which works too.
>>>
>>> In my test setup, I'm using two "HPE 2530-48G-PoE+" switches
>>> (YA.16.10.0002), each is connected with one cable to the network card.
>>> My test is very simple:
>>> After configuring the bond, I used a continous ping to the Linux server
>>> and I see many drops. If I toggle the "ntuples support", the drops disappear.
>>>
>>> Can we compare the settings of your setup with mine? Maybe you missed
>>> something...
>> Just realized we missed the 2nd switch(we're working on setting that up now)
> I got my 2nd switch configured with 36 vlans on the bond, I still can't reproduce the issue.
> 
> I'm in a test environment so I don't have much of a "network" between the two switches.  I was wondering how your switches were configured and if there was anything else going on with  the network(people streaming, backups occurring, etc).
> 
>>
>>> How can I assist further?
>> Can you enable source pruning on the ports that are part of your bond?  There
>> might be some confusion in the fw/sw with the two interfaces and packets
>> are getting ignored:
>> ethtool --set-priv-flags <interface name> disable-source-pruning on
>>
> I was reading over this and realized I contradicted myself above.  I would like to see if disabling source pruning would help.
> 
> I hope you're having a good week!
> Dave
> 
>> Let me know if that helps.  It sounds like we'll be getting our two switches
>> later today so we can make sure we have our environment matching yours.
>>
>> Have a great day!
>> Dave
>>
>>
>>>
>>> Regards
>>>   Sven
>>>
>>> Am 11.06.22 um 03:27 schrieb Switzer, David:
>>>>> -----Original Message-----
>>>>> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf
>>>>> Of Sven Anders
>>>>> Sent: Monday, June 6, 2022 11:05 PM
>>>>> To: intel-wired-lan@osuosl.org
>>>>> Subject: Re: [Intel-wired-lan] Bonding problem on Intel X710
>>>>> hardware
>>>>>
>>>>> Hello!
>>>> Hi Sven!  My apologies on taking so long to reply.
>>>>>
>>>>> Was anybody able to reproduce the problem?
>>>> I haven't been able to reproduce your issue yet, but with your
>>>> description it sounds that the issue could be related to the X710
>>>> hardware LLDP engine consuming the LLDP packets.  To disable the
>>> hardware engine, user the following command on each of the ports that
>>> you're using in the bond:
>>>>
>>>> ethtool --set-priv-flags <interface name> disable-fw-lldp on
>>>>
>>>> Then the LLDP packets that the bonding modules relies on will be
>>>> forwarded
>>> by the hardware to the OS network stack.
>>>>
>>>> I hope you have a great weekend!
>>>> Dave
>>>>
>>>>> Do you need assistance or further information?
>>>>>
>>>>> Regards
>>>>>    Sven
>>>>>
>>>>> Am 30.05.22 um 17:53 schrieb Sven Anders:
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> This is a follow up to my question. I did not hear anything so
>>>>>>>>> far, but I tried to find some some information meanwhile.
>>>>>>>>>
>>>>>>>>> I've got a guess from somebody, that the error message "Error
>>>>>>>>> I40E_AQ_RC_EINVAL adding RX filters on PF, promiscuous mode
>>>>>>>>> forced on" maybe triggered, because I'm hitting a limit here.
>>>>>>>
>>>>>>> Yes, typically this is a response from our firmware that a table
>>>>>>> is full in hardware, and I'm guessing that you might be running
>>>>>>> into a filter
>>>>> limit due to using vlans?
>>>>>>
>>>>>> That's what I suspect, but I did know the hardware and firmware
>>>>>> insufficiently to be sure.
>>>>>> What I'm wondering is: Why this is only triggered if I toggle the
>>>>>> "ntuples
>>>>> support"
>>>>>> and not earlier when setting up the interface?
>>>>>>
>>>>>>>>> Data packets are only received sporadically. If I run the same
>>>>>>>>> test with only one slave port, it works without problems.
>>>>>>>
>>>>>>> And there are no counters going up in ethtool -S when you
>>>>>>> receive/drop
>>>>> packets?
>>>>>>
>>>>>> I'm not sure here. I tried to inspect the counters and it seems
>>>>>> that the counters are going up slower in this case, but it's
>>>>>> difficult to say, because there is some "noise" on the line and I
>>>>>> do not have direct access to the hardware at the moment.
>>>>>>
>>>>>> If you need any further information or tests, just contact me.
>>>>>
>>>>> Regards
>>>>>      Sven Anders
>>>>> --
>>>>>    Sven Anders                  () UTF-8 Ribbon Campaign
>>>>>                                                    /\ Support plain text e-mail
>>>>>    ANDURAS intranet security AG
>>>>>    Messestrasse 3 - 94036 Passau - Germany
>>>>>    Web: www.anduras.de - Tel: +49 (0)851-4 90 50-0 - Fax: +49
>>>>> (0)851-4
>>>>> 90 50-55
>>>>>
>>>>> Those who would give up essential Liberty, to purchase a little
>>>>> temporary Safety, deserve neither Liberty nor Safety.
>>>>>     - Benjamin Franklin
>>>>>
>>>>> _______________________________________________
>>>>> Intel-wired-lan mailing list
>>>>> Intel-wired-lan@osuosl.org
>>>>> https://eu-central-
>>> 1.protection.sophos.com?d=osuosl.org&u=aHR0cHM6Ly9
>>>>>
>>> saXN0cy5vc3Vvc2wub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwtd2lyZWQt
>> b
>>> GFu&i=N
>>>>>
>>> WRkZTc5MjQzMjM2YzExNjg0ZTI4MjM4&t=WHZUTUdjNVkySTVVcWVjUEpV
>> T
>>> GlkaXV3Nkx
>>>>>
>>> VQm1LSGdUOXMxeVVEQ1pyOD0=&h=4ed5ca43b14147eb8ae4891cf2a54f70
>>>
>>>
>>> Mit freundlichen Grüßen
>>>   Sven Anders
>>>
>>> --
>>>   Sven Anders <anders@anduras.de>                 () UTF-8 Ribbon Campaign
>>>                                                   /\ Support plain text
>>> e-mail
>>>   ANDURAS intranet security AG
>>>   Messestraße 3 - 94036 Passau - Germany
>>>   Web: www.anduras.de - Tel: +49 (0)851-4 90 50-0
>>>
>>> Rechtsform: Aktiengesellschaft - Sitz: Passau - Amtsgericht: Passau HRB
>>> 6032
>>> Vorstand: Marcus Junker
>>> Vorsitzender des Aufsichtsrats: Andreas Wagner


Mit freundlichen Grüßen
  Sven Anders

-- 
  Sven Anders <anders@anduras.de>                 () UTF-8 Ribbon Campaign
                                                  /\ Support plain text e-mail
  ANDURAS intranet security AG
  Messestraße 3 - 94036 Passau - Germany
  Web: www.anduras.de - Tel: +49 (0)851-4 90 50-0

Rechtsform: Aktiengesellschaft - Sitz: Passau - Amtsgericht: Passau HRB 6032
Vorstand: Marcus Junker
Vorsitzender des Aufsichtsrats: Andreas Wagner
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan