Re: mlx4: dropping multicast packets at promisc leave

[Marcelo Ricardo Leitner <mleitner@redhat.com>]

On 09/20/2012 10:21 AM, Or Gerlitz wrote:
> On 20/09/2012 03:43, Marcelo Ricardo Leitner wrote:
>> I have a report that our mlx4 driver (RHEL 6.3) is dropping multicast
>> packets when NIC leaves promisc mode. It seems this is being cause due
>> to the new steering mode that took place near by commit
>> 1679200f91da6a054b06954c9bd3eeed29b6731f. As it seems, the new
>> steering mode needs more commands/time to leave the promisc mode,
>> which may be leading to packet drops.
>
> Marcelo,
>
> The commit you point on below 6d19993 "net/mlx4_en: Re-design multicast
> attachments flow" makes sure to avoid
> doing extra firmware comments and not leave a window in time where
> "correct" addresses are not attached. Its hard to say what's the case on
> that RHEL 6.3 system, it would be very helpful through if you manage to
> reproduce the problem on an upstream kernel -- BTW you didn't say on

Okay, I understand that the commit prevents a window. I may be missing 
something, but isn't there another one in there? Between:
mlx4_SET_MCAST_FLTR MLX4_MCAST_DISABLE and
mlx4_SET_MCAST_FLTR MLX4_MCAST_ENABLE
because mlx4_multicast_promisc_remove() was called just before those.
Otherwise I don't how is the NIC would be receiving multicast packets in 
there.

I understand the difficulty about the kernel version, I am sorry for 
that. As I'm unable to reproduce the issue by myself, I couldn't run a 
test in a plain upstream kernel so far or experiment much.

I was holding this email: I just access to a server that seems to 
reproduce the issue. It has a MT27500 ConnectX-3 NIC. Only tried our 
RHEL 6.3 stock so far. Keep you posted on further tests!

This was the result of ifconfig mlx4_2 -promisc:
[  3] 34.0-35.0 sec  61.7 MBytes   517 Mbits/sec   0.024 ms  274/43502 
(0.63%)
[  3] 34.0-35.0 sec  756 datagrams received out-of-order

> which kernel you are trying to reproduce this, note that upstream has
> also commit 60d31c1475f2 "net/mlx4_core: Looking for promiscuous entries
> on the correct port" (reported by you...) , so if somehow this commit
> makes the diff you could use it also on their system.

Sorry, that would be 2.6.32-279.el6. It has additional commits up to 
somewhere near commit
58a3de0 - mlx4_core: fix race on comm channel
but maybe not all before that one. Can't tell you for sure.

And then I tried 3 additional patches applied at once:
- 60d31c1475f2 "net/mlx4_core: Looking for promiscuous entries on the 
correct port"
- f1f75f0 - mlx4: attach multicast with correct flag
   - Yes, this one wasn't in 2.6.32-279.el6.
- 6d19993 - net/mlx4_en: Re-design multicast attachments flow

And they still reported drops.

>> It takes 300ms to perform the change there against my 600us. Hitting
>> something like tcpdump -c 10 in a loop helps triggering it.
>
> Do you have any insight for this huge difference?

No idea. Couldn't track it yet.

Thanks,
Marcelo.