From mboxrd@z Thu Jan  1 00:00:00 1970
From: David Ahern <dsahern@gmail.com>
Subject: Re: Kernel 4.19 network performance - forwarding/routing normal users
 traffic
Date: Thu, 8 Nov 2018 17:42:23 -0700
Message-ID: <754d9d5d-efd2-52e0-cb2b-13caf15f0737@gmail.com>
References: <61697e49-e839-befc-8330-fc00187c48ee@itcare.pl>
 <3a88bb53-9d17-3e85-638e-a605f5bfe0fb@gmail.com>
 <20181101115522.10b0dd0a@redhat.com>
 <63198d68-6752-3695-f406-d86fb395c12b@itcare.pl>
 <7141e1e0-93e4-ab20-bce6-17f1e14682f1@gmail.com>
 <a63df254-3791-cd5c-ddfc-6c9b8d38c559@itcare.pl>
 <f6188534-38d7-fc3e-189d-443a76bcb47d@gmail.com>
 <394a0bf2-fa97-1085-2eda-98ddf476895c@itcare.pl>
 <6ed1666d-47bc-24e7-d432-a0c0027452ed@gmail.com>
 <8dde3b32-59ce-38f3-5913-2ce08264e9dc@itcare.pl>
 <6165513d-1e27-31dc-8f94-9de029a73f93@gmail.com>
 <11199f9f-da21-527b-f5db-0bbf1e448a8b@itcare.pl>
 <87a2a15c-f9bf-743b-b4c5-7d37da0bd887@itcare.pl>
 <68cc8279-5e3f-85c2-673c-aa3d4a47b353@gmail.com>
 <8cb2630e-e7fe-cd44-7798-070f2e6d348a@itcare.pl>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Cc: netdev <netdev@vger.kernel.org>, Yoel Caspersen <yoel@kviknet.dk>
To: =?UTF-8?Q?Pawe=c5=82_Staszewski?= <pstaszewski@itcare.pl>,
        Jesper Dangaard Brouer <brouer@redhat.com>
Return-path: <netdev-owner@vger.kernel.org>
Received: from mail-pg1-f195.google.com ([209.85.215.195]:34075 "EHLO
        mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1726599AbeKIKUd (ORCPT
        <rfc822;netdev@vger.kernel.org>); Fri, 9 Nov 2018 05:20:33 -0500
Received: by mail-pg1-f195.google.com with SMTP id k1-v6so90747pgq.1
        for <netdev@vger.kernel.org>; Thu, 08 Nov 2018 16:42:25 -0800 (PST)
In-Reply-To: <8cb2630e-e7fe-cd44-7798-070f2e6d348a@itcare.pl>
Content-Language: en-US
Sender: netdev-owner@vger.kernel.org
List-ID: <netdev.vger.kernel.org>

On 11/8/18 5:40 PM, Paweł Staszewski wrote:
> 
> 
> W dniu 08.11.2018 o 17:32, David Ahern pisze:
>> On 11/8/18 9:27 AM, Paweł Staszewski wrote:
>>>>> What hardware is this?
>>>>>
>>> mellanox connectx 4
>>> ethtool -i enp175s0f0
>>> driver: mlx5_core
>>> version: 5.0-0
>>> firmware-version: 12.21.1000 (SM_2001000001033)
>>> expansion-rom-version:
>>> bus-info: 0000:af:00.0
>>> supports-statistics: yes
>>> supports-test: yes
>>> supports-eeprom-access: no
>>> supports-register-dump: no
>>> supports-priv-flags: yes
>>>
>>> ethtool -i enp175s0f1
>>> driver: mlx5_core
>>> version: 5.0-0
>>> firmware-version: 12.21.1000 (SM_2001000001033)
>>> expansion-rom-version:
>>> bus-info: 0000:af:00.1
>>> supports-statistics: yes
>>> supports-test: yes
>>> supports-eeprom-access: no
>>> supports-register-dump: no
>>> supports-priv-flags: yes
>>>
>>>>> Start with:
>>>>>
>>>>> echo 1 > /sys/kernel/debug/tracing/events/xdp/enable
>>>>> cat /sys/kernel/debug/tracing/trace_pipe
>>>>   cat /sys/kernel/debug/tracing/trace_pipe
>>>>           <idle>-0     [045] ..s. 68469.467752: xdp_devmap_xmit:
>>>> ndo_xdp_xmit map_id=32 map_index=5 action=REDIRECT sent=0 drops=1
>>>> from_ifindex=4 to_ifindex=5 err=-6
>> FIB lookup is good, the redirect is happening, but the mlx5 driver does
>> not like it.
>>
>> I think the -6 is coming from the mlx5 driver and the packet is getting
>> dropped. Perhaps this check in mlx5e_xdp_xmit:
>>
>>         if (unlikely(sq_num >= priv->channels.num))
>>                  return -ENXIO;
> I removed that part and recompiled - but after running now xdp_fwd i
> have kernel pamic :)

Jesper or one of the Mellanox folks needs to respond about the config
needed to run XDP with this NIC. I don't have a 40G or 100G card to play
with.