* Re: [PATCH 11/11] xfs: rename the ondisk dquot d_flags to d_type
From: Dave Chinner @ 2020-07-17 0:15 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs
In-Reply-To: <159488199070.3813063.17484927860165624202.stgit@magnolia>
On Wed, Jul 15, 2020 at 11:46:30PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> The ondisk dquot stores the quota record type in the flags field.
> Rename this field to d_type to make the _type relationship between the
> ondisk and incore dquot more obvious.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> fs/xfs/libxfs/xfs_dquot_buf.c | 6 +++---
> fs/xfs/libxfs/xfs_format.h | 2 +-
> fs/xfs/xfs_dquot.c | 8 ++++----
> fs/xfs/xfs_dquot_item_recover.c | 4 ++--
> fs/xfs/xfs_qm.c | 4 ++--
> 5 files changed, 12 insertions(+), 12 deletions(-)
Looks good. nice and clear.
Reviewed-by: Dave Chinner <dchinner@redhat.com>
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply
* Dubios pointer casting with put_user()
From: Richard Sailer @ 2020-07-17 0:13 UTC (permalink / raw)
To: kernelnewbies
[-- Attachment #1.1.1: Type: text/plain, Size: 803 bytes --]
Hi,
In the following example from net/dccp/proto.c the pointer given
put_user() is casted to (int __user *) although the value to copy is a
unsigned long. Is this (correctness and security wise) sane? Because as
I understand it put_user() determines the amount it copies from the
pointer type.
```
unsigned long amount = 0;
skb = skb_peek(&sk->sk_receive_queue);
if (skb != NULL) {
amount = skb->len;
}
rc = put_user(amount, (int __user *)arg);
```
Also skb->len is an unsigned int realisticly in most cases < 9000 (and
in all cases I can imagine < int_max (with 16 bit)).
I would like to declare amount as int outside of the switch case
statement (because I need it in another case statement as signed int)
would it be safe to do so?
Thanks,
-- Richard
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
[-- Attachment #2: Type: text/plain, Size: 170 bytes --]
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
^ permalink raw reply
* Re: [PATCH 10/11] xfs: improve ondisk dquot flags checking
From: Dave Chinner @ 2020-07-17 0:13 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs
In-Reply-To: <159488198306.3813063.16348101518917273554.stgit@magnolia>
On Wed, Jul 15, 2020 at 11:46:23PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> Create an XFS_DQTYPE_ANY mask for ondisk dquots flags, and use that to
> ensure that we never accept any garbage flags when we're loading dquots.
> While we're at it, restructure the quota type flag checking to use the
> proper masking.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> fs/xfs/libxfs/xfs_dquot_buf.c | 11 ++++++++---
> fs/xfs/libxfs/xfs_format.h | 2 ++
> 2 files changed, 10 insertions(+), 3 deletions(-)
Ok, I looked at this and questioned why it existed and why the
code didn't just use XFS_DQTYPE_REC_MASK directly. I think this
change exists because you plan on adding a new on-disk flag for
bigtime support and hence XFS_DQTYPE_ANY will grow to include the
new flag, right?
If so, can you add that to the commit message?
Code looks fine assuming I've understood this correctly...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply
* Re: [RFC PATCH v3 16/18] gpu: host1x: mipi: Split tegra_mipi_calibrate and tegra_mipi_wait
From: Sowjanya Komatineni @ 2020-07-17 0:16 UTC (permalink / raw)
To: Dmitry Osipenko, thierry.reding, jonathanh, frankc, hverkuil,
sakari.ailus, robh+dt, helen.koike
Cc: sboyd, gregkh, linux-media, devicetree, linux-tegra, linux-kernel,
linux-i2c
In-Reply-To: <9b4fbf9d-d651-aa35-c0a6-b8f16aeb0900@gmail.com>
On 7/16/20 4:47 PM, Dmitry Osipenko wrote:
> 17.07.2020 02:09, Sowjanya Komatineni пишет:
>> On 7/16/20 4:06 PM, Sowjanya Komatineni wrote:
>>> On 7/16/20 4:01 PM, Dmitry Osipenko wrote:
>>>> 17.07.2020 01:49, Sowjanya Komatineni пишет:
>>>>>> What keeps MIPI clock enabled after completion of the
>>>>>> tegra_mipi_calibrate() invocation?
>>>>> MIPI clock is disabled at end of tegra_mipi_calibrate and is re-enabled
>>>>> during tegra_mipi_wait.
>>>>>
>>>>> I think I should fix this to keep the clock enabled till calibration
>>>>> results are latched.
>>>>>
>>>>> All consumers of tegra_mipi_calibrate() will call tegra_mipi_wait().
>>>>>
>>>>> So will remove clk_disable mipi clk at end of tegra_mipi_calibrate()
>>>>> and
>>>>> clk_enable mipi_clk at beginning of tegra_mipi_wait()
>>>> Isn't it possible to perform the calibration after enabling CSI and
>>>> before of starting the sensor streaming?
>>> Currently this is what I am doing. Triggering calibration start during
>>> CSI receiver being ready and then sensor streaming will happen where
>>> internal MIPI CAL detects for LP -> HS transition and applies results
>>> to pads. So checking for calibration results after sensor stream is
>>> enabled
>> 1. Calling tegra_mipi_calibrate() during CSI streaming where CSI pads
>> are enabled and receiver is kept ready
>>
>> 2. Start Sensor stream
>>
>> 3. Calling tegra_mipi_wait() to check for MIPI Cal status.
>>
>> So as mipi cal clk need to be kept enabled till 3rd step, we can enable
>> clock during tegra_mipi_calibrate() and leave it enabled and disable it
>> in tegra_mipi_wait after status check.
> From TRM:
>
> The following sequence is recommended for capturing a single frame:
>
> 1. Set up CSI registers for use case such as number of lanes, virtual
> channel, etc.
> 2. Initialize and power up CSI interface
> 3. Wait for initialization time or done signal from calibration logic
> 4. Power up camera through the I2C interface
> 5. All CSI data and clock lanes are in stop state, LP11
> 6. Initiate frame capture through the I2C
> 7. Frame done, CSI goes back to stop state, LP11
>
> Hence, is it really necessary to perform the manual calibration?
done signal from calibration logic will happen only when it sees LP to
HS transition as thats when calibration results are applied to pads and
then done signal is set.
Also MIPI Pads calibration need to be done on every power off/on. So
need to do calibration and trigger it along with CSI receiver
programming to keep it ready and then need to check/wait for status only
after sensor stream happens as thats where LP->HS transition happen.
^ permalink raw reply
* Re: [PATCH v4 net-next 3/3] octeontx2-pf: Add support for PTP clock
From: Jakub Kicinski @ 2020-07-17 0:11 UTC (permalink / raw)
To: Subbaraya Sundeep
Cc: davem, richardcochran, netdev, sgoutham, Aleksey Makarov
In-Reply-To: <1594816689-5935-4-git-send-email-sbhatta@marvell.com>
On Wed, 15 Jul 2020 18:08:09 +0530 Subbaraya Sundeep wrote:
> @@ -1730,10 +1745,149 @@ static void otx2_reset_task(struct work_struct *work)
> if (!netif_running(pf->netdev))
> return;
>
> + rtnl_lock();
> otx2_stop(pf->netdev);
> pf->reset_count++;
> otx2_open(pf->netdev);
> netif_trans_update(pf->netdev);
> + rtnl_unlock();
> +}
> +
This looks unrelated, otherwise for the patches:
Acked-by: Jakub Kicinski <kuba@kernel.org>
^ permalink raw reply
* Re: [PATCH] spapr_pci: Robustify support of PCI bridges
From: David Gibson @ 2020-07-16 23:57 UTC (permalink / raw)
To: Greg Kurz
Cc: Thomas Huth, Michael S. Tsirkin, Markus Armbruster, qemu-devel,
qemu-ppc
In-Reply-To: <20200716165754.50af735a@bahia.lan>
[-- Attachment #1: Type: text/plain, Size: 2850 bytes --]
On Thu, Jul 16, 2020 at 04:57:54PM +0200, Greg Kurz wrote:
> On Thu, 16 Jul 2020 16:23:52 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
> > David Gibson <david@gibson.dropbear.id.au> writes:
> >
> > > On Thu, Jul 16, 2020 at 12:32:44PM +0200, Greg Kurz wrote:
> > >> On Thu, 16 Jul 2020 14:45:40 +1000
> > >> David Gibson <david@gibson.dropbear.id.au> wrote:
> > >>
> > >> > On Thu, Jul 09, 2020 at 07:12:47PM +0200, Greg Kurz wrote:
> > >> > > Some recent error handling cleanups unveiled issues with our support of
> > >> > > PCI bridges:
> > >> > >
> > >> > > 1) QEMU aborts when using non-standard PCI bridge types,
> > >> > > unveiled by commit 7ef1553dac "spapr_pci: Drop some dead error handling"
> > >> > >
> > >> > > $ qemu-system-ppc64 -M pseries -device pcie-pci-bridge
> > >> > > Unexpected error in object_property_find() at qom/object.c:1240:
> > >> > > qemu-system-ppc64: -device pcie-pci-bridge: Property '.chassis_nr' not found
> > >> > > Aborted (core dumped)
> > >> >
> > >> > Oops, I thought we had a check that we actually had a "pci-bridge"
> > >> > device before continuing with the hotplug, but I guess not.
> > >>
> > >> Ah... are you suggesting we should explicitly check the actual type
> > >> of the bridge rather than looking for the "chassis_nr" property ?
> > >
> > > Uh.. I thought about it, but I don't think it matters much which way
> > > we do it.
> >
> > Would it make sense to add the "chassis_nr" property to *all* PCI
> > bridge devices?
> >
>
> I see that the "PCI Express to PCI/PCI-X Bridge Specification" mentions
> a "Chassis Number Register" which looks very similar to the what exists
> in standard PCI-to-PCI brdiges. This doesn't seem to be implemented in
> our "pcie-pci-bridge" device model though, but of course I have no idea
> why :)
We could consider it, but I don't think there's a lot to be gained by
it at this stage. I don't think there's really any reason to want to
use bridges other than plain "pci-bridge" on the pseries machine.
PCI is a bit weird on pseries, since it's explicitly paravirt.
Although you can use extended config space, and thereby PCI-E devices
on it, the topology really looks pretty much identical to vanilla
PCI. So, I don't think there's any reason to use PCI-E bridges on
pseries.
Other than PCI-E bridges of various sorts, a quick scan suggests all
the other bridge types in qemu are weird variants that are mostly
specific to some particular platform. I don't see any reason we'd
want those on pseries either.
> Maybe Michael or Marcel (cc'd) can share some thoughts about that ?
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [Intel-wired-lan] NAT performance issue 944mbit -> ~40mbit
From: Alexander Duyck @ 2020-07-17 0:09 UTC (permalink / raw)
To: Ian Kumlien
Cc: Jakub Kicinski, Linux Kernel Network Developers, intel-wired-lan
In-Reply-To: <CAA85sZt6B+rG8pUfRoNVOH=VqHn=rT-+2kHpFDzW+eBwvODxJA@mail.gmail.com>
On Thu, Jul 16, 2020 at 12:47 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
>
> Sorry, tried to respond via the phone, used the webbrowser version but
> still html mails... :/
>
> On Thu, Jul 16, 2020 at 5:18 PM Alexander Duyck
> <alexander.duyck@gmail.com> wrote:
> > On Wed, Jul 15, 2020 at 5:00 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > On Thu, Jul 16, 2020 at 1:42 AM Alexander Duyck
> > > <alexander.duyck@gmail.com> wrote:
> > > > On Wed, Jul 15, 2020 at 3:51 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > > > On Thu, Jul 16, 2020 at 12:32 AM Alexander Duyck
> > > > > <alexander.duyck@gmail.com> wrote:
> > > > > > On Wed, Jul 15, 2020 at 3:00 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > > > > > On Wed, Jul 15, 2020 at 11:40 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > > > > > > > On Wed, 15 Jul 2020 23:12:23 +0200 Ian Kumlien wrote:
> > > > > > > > > On Wed, Jul 15, 2020 at 11:02 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > > > > > > > > On Wed, Jul 15, 2020 at 10:31 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > > > > > > > > > > On Wed, 15 Jul 2020 22:05:58 +0200 Ian Kumlien wrote:
> > > > > > > > > > > > After a lot of debugging it turns out that the bug is in igb...
> > > > > > > > > > > >
> > > > > > > > > > > > driver: igb
> > > > > > > > > > > > version: 5.6.0-k
> > > > > > > > > > > > firmware-version: 0. 6-1
> > > > > > > > > > > >
> > > > > > > > > > > > 03:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network
> > > > > > > > > > > > Connection (rev 03)
> > > > > > > > > > >
> > > > > > > > > > > Unclear to me what you're actually reporting. Is this a regression
> > > > > > > > > > > after a kernel upgrade? Compared to no NAT?
> > > > > > > > > >
> > > > > > > > > > It only happens on "internet links"
> > > > > > > > > >
> > > > > > > > > > Lets say that A is client with ibg driver, B is a firewall running NAT
> > > > > > > > > > with ixgbe drivers, C is another local node with igb and
> > > > > > > > > > D is a remote node with a bridge backed by a bnx2 interface.
> > > > > > > > > >
> > > > > > > > > > A -> B -> C is ok (B and C is on the same switch)
> > > > > > > > > >
> > > > > > > > > > A -> B -> D -- 32-40mbit
> > > > > > > > > >
> > > > > > > > > > B -> D 944 mbit
> > > > > > > > > > C -> D 944 mbit
> > > > > > > > > >
> > > > > > > > > > A' -> D ~933 mbit (A with realtek nic -- also link is not idle atm)
> > > > > > > > >
> > > > > > > > > This should of course be A' -> B -> D
> > > > > > > > >
> > > > > > > > > Sorry, I've been scratching my head for about a week...
> > > > > > > >
> > > > > > > > Hm, only thing that comes to mind if A' works reliably and A doesn't is
> > > > > > > > that A has somehow broken TCP offloads. Could you try disabling things
> > > > > > > > via ethtool -K and see if those settings make a difference?
> > > > > > >
> > > > > > > It's a bit hard since it works like this, turned tso off:
> > > > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > > > [ 5] 0.00-1.00 sec 108 MBytes 902 Mbits/sec 0 783 KBytes
> > > > > > > [ 5] 1.00-2.00 sec 110 MBytes 923 Mbits/sec 31 812 KBytes
> > > > > > > [ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 92 772 KBytes
> > > > > > > [ 5] 3.00-4.00 sec 110 MBytes 923 Mbits/sec 0 834 KBytes
> > > > > > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 60 823 KBytes
> > > > > > > [ 5] 5.00-6.00 sec 110 MBytes 923 Mbits/sec 31 789 KBytes
> > > > > > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 0 786 KBytes
> > > > > > > [ 5] 7.00-8.00 sec 110 MBytes 923 Mbits/sec 0 761 KBytes
> > > > > > > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 0 772 KBytes
> > > > > > > [ 5] 9.00-10.00 sec 109 MBytes 912 Mbits/sec 0 868 KBytes
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > > > [ 5] 0.00-10.00 sec 1.07 GBytes 923 Mbits/sec 214 sender
> > > > > > > [ 5] 0.00-10.00 sec 1.07 GBytes 920 Mbits/sec receiver
> > > > > > >
> > > > > > > Continued running tests:
> > > > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > > > [ 5] 0.00-1.00 sec 5.82 MBytes 48.8 Mbits/sec 0 82.0 KBytes
> > > > > > > [ 5] 1.00-2.00 sec 4.97 MBytes 41.7 Mbits/sec 0 130 KBytes
> > > > > > > [ 5] 2.00-3.00 sec 5.28 MBytes 44.3 Mbits/sec 0 99.0 KBytes
> > > > > > > [ 5] 3.00-4.00 sec 5.28 MBytes 44.3 Mbits/sec 0 105 KBytes
> > > > > > > [ 5] 4.00-5.00 sec 5.28 MBytes 44.3 Mbits/sec 0 122 KBytes
> > > > > > > [ 5] 5.00-6.00 sec 5.28 MBytes 44.3 Mbits/sec 0 82.0 KBytes
> > > > > > > [ 5] 6.00-7.00 sec 5.28 MBytes 44.3 Mbits/sec 0 79.2 KBytes
> > > > > > > [ 5] 7.00-8.00 sec 5.28 MBytes 44.3 Mbits/sec 0 110 KBytes
> > > > > > > [ 5] 8.00-9.00 sec 5.28 MBytes 44.3 Mbits/sec 0 156 KBytes
> > > > > > > [ 5] 9.00-10.00 sec 5.28 MBytes 44.3 Mbits/sec 0 87.7 KBytes
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > > > [ 5] 0.00-10.00 sec 53.0 MBytes 44.5 Mbits/sec 0 sender
> > > > > > > [ 5] 0.00-10.00 sec 52.5 MBytes 44.1 Mbits/sec receiver
> > > > > > >
> > > > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > > > [ 5] 0.00-1.00 sec 7.08 MBytes 59.4 Mbits/sec 0 156 KBytes
> > > > > > > [ 5] 1.00-2.00 sec 5.97 MBytes 50.0 Mbits/sec 0 110 KBytes
> > > > > > > [ 5] 2.00-3.00 sec 4.97 MBytes 41.7 Mbits/sec 0 124 KBytes
> > > > > > > [ 5] 3.00-4.00 sec 5.47 MBytes 45.9 Mbits/sec 0 96.2 KBytes
> > > > > > > [ 5] 4.00-5.00 sec 5.47 MBytes 45.9 Mbits/sec 0 158 KBytes
> > > > > > > [ 5] 5.00-6.00 sec 4.97 MBytes 41.7 Mbits/sec 0 70.7 KBytes
> > > > > > > [ 5] 6.00-7.00 sec 5.47 MBytes 45.9 Mbits/sec 0 113 KBytes
> > > > > > > [ 5] 7.00-8.00 sec 5.47 MBytes 45.9 Mbits/sec 0 96.2 KBytes
> > > > > > > [ 5] 8.00-9.00 sec 4.97 MBytes 41.7 Mbits/sec 0 84.8 KBytes
> > > > > > > [ 5] 9.00-10.00 sec 5.47 MBytes 45.9 Mbits/sec 0 116 KBytes
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > > > [ 5] 0.00-10.00 sec 55.3 MBytes 46.4 Mbits/sec 0 sender
> > > > > > > [ 5] 0.00-10.00 sec 53.9 MBytes 45.2 Mbits/sec receiver
> > > > > > >
> > > > > > > And the low bandwidth continues with:
> > > > > > > ethtool -k enp3s0 |grep ": on"
> > > > > > > rx-vlan-offload: on
> > > > > > > tx-vlan-offload: on [requested off]
> > > > > > > highdma: on [fixed]
> > > > > > > rx-vlan-filter: on [fixed]
> > > > > > > tx-gre-segmentation: on
> > > > > > > tx-gre-csum-segmentation: on
> > > > > > > tx-ipxip4-segmentation: on
> > > > > > > tx-ipxip6-segmentation: on
> > > > > > > tx-udp_tnl-segmentation: on
> > > > > > > tx-udp_tnl-csum-segmentation: on
> > > > > > > tx-gso-partial: on
> > > > > > > tx-udp-segmentation: on
> > > > > > > hw-tc-offload: on
> > > > > > >
> > > > > > > Can't quite find how to turn those off since they aren't listed in
> > > > > > > ethtool (since the text is not what you use to enable/disable)
> > > > > >
> > > > > > To disable them you would just repeat the same string in the display
> > > > > > string. So it should just be "ethtool -K enp3s0 tx-gso-partial off"
> > > > > > and that would turn off a large chunk of them as all the encapsulated
> > > > > > support requires gso partial support.
> > > > >
> > > > > ethtool -k enp3s0 |grep ": on"
> > > > > highdma: on [fixed]
> > > > > rx-vlan-filter: on [fixed]
> > > > > ---
> > > > > And then back to back:
> > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > [ 5] 0.00-1.00 sec 4.91 MBytes 41.2 Mbits/sec 0 45.2 KBytes
> > > > > [ 5] 1.00-2.00 sec 4.47 MBytes 37.5 Mbits/sec 0 52.3 KBytes
> > > > > [ 5] 2.00-3.00 sec 4.47 MBytes 37.5 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 3.00-4.00 sec 4.47 MBytes 37.5 Mbits/sec 0 141 KBytes
> > > > > [ 5] 4.00-5.00 sec 111 MBytes 928 Mbits/sec 63 764 KBytes
> > > > > [ 5] 5.00-6.00 sec 86.2 MBytes 724 Mbits/sec 0 744 KBytes
> > > > > [ 5] 6.00-7.00 sec 98.8 MBytes 828 Mbits/sec 61 769 KBytes
> > > > > [ 5] 7.00-8.00 sec 110 MBytes 923 Mbits/sec 0 749 KBytes
> > > > > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 0 741 KBytes
> > > > > [ 5] 9.00-10.00 sec 110 MBytes 923 Mbits/sec 31 761 KBytes
> > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > [ 5] 0.00-10.00 sec 644 MBytes 540 Mbits/sec 155 sender
> > > > > [ 5] 0.00-10.01 sec 641 MBytes 537 Mbits/sec receiver
> > > > >
> > > > > and we're back at the not working bit:
> > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > [ 5] 0.00-1.00 sec 4.84 MBytes 40.6 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 1.00-2.00 sec 4.60 MBytes 38.6 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 2.00-3.00 sec 4.23 MBytes 35.4 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 3.00-4.00 sec 4.47 MBytes 37.5 Mbits/sec 0 67.9 KBytes
> > > > > [ 5] 4.00-5.00 sec 4.47 MBytes 37.5 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 5.00-6.00 sec 4.23 MBytes 35.4 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 6.00-7.00 sec 4.23 MBytes 35.4 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 7.00-8.00 sec 4.47 MBytes 37.5 Mbits/sec 0 67.9 KBytes
> > > > > [ 5] 8.00-9.00 sec 4.47 MBytes 37.5 Mbits/sec 0 53.7 KBytes
> > > > > [ 5] 9.00-10.00 sec 4.47 MBytes 37.5 Mbits/sec 0 79.2 KBytes
> > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > [ 5] 0.00-10.00 sec 44.5 MBytes 37.3 Mbits/sec 0 sender
> > > > > [ 5] 0.00-10.00 sec 43.9 MBytes 36.8 Mbits/sec receiver
> > > > >
> > > > > > > I was hoping that you'd have a clue of something that might introduce
> > > > > > > a regression - ie specific patches to try to revert
> > > > > > >
> > > > > > > Btw, the same issue applies to udp as werll
> > > > > > >
> > > > > > > [ ID] Interval Transfer Bitrate Total Datagrams
> > > > > > > [ 5] 0.00-1.00 sec 6.77 MBytes 56.8 Mbits/sec 4900
> > > > > > > [ 5] 1.00-2.00 sec 4.27 MBytes 35.8 Mbits/sec 3089
> > > > > > > [ 5] 2.00-3.00 sec 4.20 MBytes 35.2 Mbits/sec 3041
> > > > > > > [ 5] 3.00-4.00 sec 4.30 MBytes 36.1 Mbits/sec 3116
> > > > > > > [ 5] 4.00-5.00 sec 4.24 MBytes 35.6 Mbits/sec 3070
> > > > > > > [ 5] 5.00-6.00 sec 4.21 MBytes 35.3 Mbits/sec 3047
> > > > > > > [ 5] 6.00-7.00 sec 4.29 MBytes 36.0 Mbits/sec 3110
> > > > > > > [ 5] 7.00-8.00 sec 4.28 MBytes 35.9 Mbits/sec 3097
> > > > > > > [ 5] 8.00-9.00 sec 4.25 MBytes 35.6 Mbits/sec 3075
> > > > > > > [ 5] 9.00-10.00 sec 4.20 MBytes 35.2 Mbits/sec 3039
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Jitter
> > > > > > > Lost/Total Datagrams
> > > > > > > [ 5] 0.00-10.00 sec 45.0 MBytes 37.7 Mbits/sec 0.000 ms
> > > > > > > 0/32584 (0%) sender
> > > > > > > [ 5] 0.00-10.00 sec 45.0 MBytes 37.7 Mbits/sec 0.037 ms
> > > > > > > 0/32573 (0%) receiver
> > > > > > >
> > > > > > > vs:
> > > > > > >
> > > > > > > [ ID] Interval Transfer Bitrate Total Datagrams
> > > > > > > [ 5] 0.00-1.00 sec 114 MBytes 954 Mbits/sec 82342
> > > > > > > [ 5] 1.00-2.00 sec 114 MBytes 955 Mbits/sec 82439
> > > > > > > [ 5] 2.00-3.00 sec 114 MBytes 956 Mbits/sec 82507
> > > > > > > [ 5] 3.00-4.00 sec 114 MBytes 955 Mbits/sec 82432
> > > > > > > [ 5] 4.00-5.00 sec 114 MBytes 956 Mbits/sec 82535
> > > > > > > [ 5] 5.00-6.00 sec 114 MBytes 953 Mbits/sec 82240
> > > > > > > [ 5] 6.00-7.00 sec 114 MBytes 956 Mbits/sec 82512
> > > > > > > [ 5] 7.00-8.00 sec 114 MBytes 956 Mbits/sec 82503
> > > > > > > [ 5] 8.00-9.00 sec 114 MBytes 956 Mbits/sec 82532
> > > > > > > [ 5] 9.00-10.00 sec 114 MBytes 956 Mbits/sec 82488
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Jitter
> > > > > > > Lost/Total Datagrams
> > > > > > > [ 5] 0.00-10.00 sec 1.11 GBytes 955 Mbits/sec 0.000 ms
> > > > > > > 0/824530 (0%) sender
> > > > > > > [ 5] 0.00-10.01 sec 1.11 GBytes 949 Mbits/sec 0.014 ms
> > > > > > > 4756/824530 (0.58%) receiver
> > > > > >
> > > > > > The fact that it is impacting UDP seems odd. I wonder if we don't have
> > > > > > a qdisc somewhere that is misbehaving and throttling the Tx. Either
> > > > > > that or I wonder if we are getting spammed with flow control frames.
> > > > >
> > > > > it sometimes works, it looks like the cwindow just isn't increased -
> > > > > that's where i started...
> > > > >
> > > > > Example:
> > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > [ 5] 0.00-1.00 sec 4.86 MBytes 40.8 Mbits/sec 0 50.9 KBytes
> > > > > [ 5] 1.00-2.00 sec 4.66 MBytes 39.1 Mbits/sec 0 65.0 KBytes
> > > > > [ 5] 2.00-3.00 sec 4.29 MBytes 36.0 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 3.00-4.00 sec 4.66 MBytes 39.1 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 4.00-5.00 sec 23.1 MBytes 194 Mbits/sec 0 1.07 MBytes
> > > > > [ 5] 5.00-6.00 sec 110 MBytes 923 Mbits/sec 0 761 KBytes
> > > > > [ 5] 6.00-7.00 sec 98.8 MBytes 828 Mbits/sec 60 806 KBytes
> > > > > [ 5] 7.00-8.00 sec 82.5 MBytes 692 Mbits/sec 0 812 KBytes
> > > > > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 92 761 KBytes
> > > > > [ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec 0 755 KBytes
> > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > [ 5] 0.00-10.00 sec 554 MBytes 465 Mbits/sec 152 sender
> > > > > [ 5] 0.00-10.00 sec 550 MBytes 461 Mbits/sec receiver
> > > > >
> > > > > > It would be useful to include the output of just calling "ethtool
> > > > > > enp3s0" on the interface to verify the speed, "ethtool -a enp3s0" to
> > > > > > verify flow control settings, and "ethtool -S enp3s0 | grep -v :\ 0"
> > > > > > to output the statistics and dump anything that isn't zero.
> > > > >
> > > > > ethtool enp3s0
> > > > > Settings for enp3s0:
> > > > > Supported ports: [ TP ]
> > > > > Supported link modes: 10baseT/Half 10baseT/Full
> > > > > 100baseT/Half 100baseT/Full
> > > > > 1000baseT/Full
> > > > > Supported pause frame use: Symmetric
> > > > > Supports auto-negotiation: Yes
> > > > > Supported FEC modes: Not reported
> > > > > Advertised link modes: 10baseT/Half 10baseT/Full
> > > > > 100baseT/Half 100baseT/Full
> > > > > 1000baseT/Full
> > > > > Advertised pause frame use: Symmetric
> > > > > Advertised auto-negotiation: Yes
> > > > > Advertised FEC modes: Not reported
> > > > > Speed: 1000Mb/s
> > > > > Duplex: Full
> > > > > Auto-negotiation: on
> > > > > Port: Twisted Pair
> > > > > PHYAD: 1
> > > > > Transceiver: internal
> > > > > MDI-X: off (auto)
> > > > > Supports Wake-on: pumbg
> > > > > Wake-on: g
> > > > > Current message level: 0x00000007 (7)
> > > > > drv probe link
> > > > > Link detected: yes
> > > > > ---
> > > > > ethtool -a enp3s0
> > > > > Pause parameters for enp3s0:
> > > > > Autonegotiate: on
> > > > > RX: on
> > > > > TX: off
> > > > > ---
> > > > > ethtool -S enp3s0 |grep -v :\ 0
> > > > > NIC statistics:
> > > > > rx_packets: 15920618
> > > > > tx_packets: 17846725
> > > > > rx_bytes: 15676264423
> > > > > tx_bytes: 19925010639
> > > > > rx_broadcast: 119553
> > > > > tx_broadcast: 497
> > > > > rx_multicast: 330193
> > > > > tx_multicast: 18190
> > > > > multicast: 330193
> > > > > rx_missed_errors: 270102
> > > > > rx_long_length_errors: 6
> > > > > tx_tcp_seg_good: 1342561
> > > > > rx_long_byte_count: 15676264423
> > > > > rx_errors: 6
> > > > > rx_length_errors: 6
> > > > > rx_fifo_errors: 270102
> > > > > tx_queue_0_packets: 7651168
> > > > > tx_queue_0_bytes: 7823281566
> > > > > tx_queue_0_restart: 4920
> > > > > tx_queue_1_packets: 10195557
> > > > > tx_queue_1_bytes: 12027522118
> > > > > tx_queue_1_restart: 12718
> > > > > rx_queue_0_packets: 15920618
> > > > > rx_queue_0_bytes: 15612581951
> > > > > rx_queue_0_csum_err: 76
> > > > > (I've only run two runs since i reenabled the interface)
> > > >
> > > > So I am seeing three things here.
> > > >
> > > > The rx_long_length_errors are usually due to an MTU mismatch. Do you
> > > > have something on the network that is using jumbo frames, or is the
> > > > MTU on the NIC set to something smaller than what is supported on the
> > > > network?
> > >
> > > I'm using jumbo frames on the local network, internet side is the
> > > normal 1500 bytes mtu though
> > >
> > > > You are getting rx_missed_errors, that would seem to imply that the
> > > > DMA is not able to keep up. We may want to try disabling the L1 to see
> > > > if we get any boost from doing that.
> > >
> > > It used to work, I don't do benchmarks all the time and sometimes the first
> > > benchmarks turn out fine... so it's hard to say when this started happening...
> > >
> > > It could also be related to a bios upgrade, but I'm pretty sure I did
> > > successful benchmarks after that...
> > >
> > > How do I disable the l1? just echo 0 >
> > > /sys/bus/pci/drivers/igb/0000\:03\:00.0/link/l1_aspm ?
> > >
> > > > The last bit is that queue 0 is seeing packets with bad checksums. You
> > > > might want to run some tests and see where the bad checksums are
> > > > coming from. If they are being detected from a specific NIC such as
> > > > the ixgbe in your example it might point to some sort of checksum
> > > > error being created as a result of the NAT translation.
> > >
> > > But that should also affect A' and the A -> B -> C case, which it doesn't...
> > >
> > > It only seems to happen with higher rtt (6 hops, sub 3 ms in this case
> > > but still high enough somehow)
> > >
> > > > > ---
> > > > >
> > > > > > > lspci -s 03:00.0 -vvv
> > > > > > > 03:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network
> > > > > > > Connection (rev 03)
> > > > > > > Subsystem: ASUSTeK Computer Inc. I211 Gigabit Network Connection
> > > > > > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> > > > > > > Stepping- SERR- FastB2B- DisINTx+
> > > > > > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> > > > > > > <TAbort- <MAbort- >SERR- <PERR- INTx-
> > > > > > > Latency: 0
> > > > > > > Interrupt: pin A routed to IRQ 57
> > > > > > > IOMMU group: 20
> > > > > > > Region 0: Memory at fc900000 (32-bit, non-prefetchable) [size=128K]
> > > > > > > Region 2: I/O ports at e000 [size=32]
> > > > > > > Region 3: Memory at fc920000 (32-bit, non-prefetchable) [size=16K]
> > > > > > > Capabilities: [40] Power Management version 3
> > > > > > > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> > > > > > > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
> > > > > > > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> > > > > > > Address: 0000000000000000 Data: 0000
> > > > > > > Masking: 00000000 Pending: 00000000
> > > > > > > Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
> > > > > > > Vector table: BAR=3 offset=00000000
> > > > > > > PBA: BAR=3 offset=00002000
> > > > > > > Capabilities: [a0] Express (v2) Endpoint, MSI 00
> > > > > > > DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
> > > > > > > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
> > > > > > > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
> > > > > > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
> > > > > > > MaxPayload 128 bytes, MaxReadReq 512 bytes
> > > > > > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
> > > > > > > LnkCap: Port #3, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency
> > > > > > > L0s <2us, L1 <16us
> > > > > > > ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
> > > > > > > LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
> > > > > > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > > > > > > LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
> > > > > > > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> > > > > >
> > > > > > PCIe wise the connection is going to be pretty tight in terms of
> > > > > > bandwidth. It looks like we have 2.5GT/s with only a single lane of
> > > > > > PCIe. In addition we are running with ASPM enabled so that means that
> > > > > > if we don't have enough traffic we are shutting off the one PCIe lane
> > > > > > we have so if we are getting bursty traffic that can get ugly.
> > > > >
> > > > > Humm... is there a way to force disable ASPM in sysfs?
> > > >
> > > > Actually the easiest way to do this is to just use setpci.
> > > >
> > > > You should be able to dump the word containing the setting via:
> > > > # setpci -s 3:00.0 0xB0.w
> > > > 0042
> > > > # setpci -s 3:00.0 0xB0.w=0040
> > > >
> > > > Basically what you do is clear the lower 3 bits of the value so in
> > > > this case that means replacing the 2 with a 0 based on the output of
> > > > the first command.
> > >
> > > Well... I'll be damned... I used to force enable ASPM... this must be
> > > related to the change in PCIe bus ASPM
> > > Perhaps disable ASPM if there is only one link?
> >
> > Is there any specific reason why you are enabling ASPM? Is this system
> > a laptop where you are trying to conserve power when on battery? If
> > not disabling it probably won't hurt things too much since the power
> > consumption for a 2.5GT/s link operating in a width of one shouldn't
> > bee too high. Otherwise you are likely going to end up paying the
> > price for getting the interface out of L1 when the traffic goes idle
> > so you are going to see flows that get bursty paying a heavy penalty
> > when they start dropping packets.
>
> Ah, you misunderstand, I used to do this and everything worked - now
> Linux enables ASPM by default on all pcie controllers,
> so imho this should be a quirk, if there is only one lane, don't do
> ASPM due to latency and timing issues...
>
> > It is also possible this could be something that changed with the
> > physical PCIe link. Basically L1 works by powering down the link when
> > idle, and then powering it back up when there is activity. The problem
> > is bringing it back up can sometimes be a challenge when the physical
> > link starts to go faulty. I know I have seen that in some cases it can
> > even result in the device falling off of the PCIe bus if the link
> > training fails.
>
> It works fine without ASPM (and the machine is pretty new)
>
> I suspect we hit some timing race with aggressive ASPM (assumed as
> such since it works on local links but doesn't on ~3 ms Links)
Agreed. What is probably happening if you are using a NAT is that it
may be seeing some burstiness being introduced and as a result the
part is going to sleep and then being overrun when the traffic does
arrive.
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 113 MBytes 950 Mbits/sec 31 710 KBytes
> > > [ 5] 1.00-2.00 sec 110 MBytes 923 Mbits/sec 135 626 KBytes
> > > [ 5] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 18 713 KBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 0 798 KBytes
> > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 0 721 KBytes
> > > [ 5] 5.00-6.00 sec 112 MBytes 944 Mbits/sec 31 800 KBytes
> > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 0 730 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 19 730 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 0 701 KBytes
> > > [ 5] 9.00-10.00 sec 112 MBytes 944 Mbits/sec 12 701 KBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 937 Mbits/sec 246 sender
> > > [ 5] 0.00-10.01 sec 1.09 GBytes 933 Mbits/sec receiver
> > >
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 114 MBytes 956 Mbits/sec 0 749 KBytes
> > > [ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec 30 766 KBytes
> > > [ 5] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 7 749 KBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 11 707 KBytes
> > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 2 699 KBytes
> > > [ 5] 5.00-6.00 sec 111 MBytes 933 Mbits/sec 8 699 KBytes
> > > [ 5] 6.00-7.00 sec 112 MBytes 944 Mbits/sec 1 953 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 0 701 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 26 707 KBytes
> > > [ 5] 9.00-10.00 sec 112 MBytes 944 Mbits/sec 2 1.07 MBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec 87 sender
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec receiver
> > >
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 114 MBytes 953 Mbits/sec 16 908 KBytes
> > > [ 5] 1.00-2.00 sec 112 MBytes 944 Mbits/sec 0 693 KBytes
> > > [ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 0 713 KBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 0 687 KBytes
> > > [ 5] 4.00-5.00 sec 112 MBytes 944 Mbits/sec 15 687 KBytes
> > > [ 5] 5.00-6.00 sec 111 MBytes 933 Mbits/sec 2 888 KBytes
> > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 17 696 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 0 758 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 31 749 KBytes
> > > [ 5] 9.00-10.00 sec 112 MBytes 944 Mbits/sec 0 792 KBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 938 Mbits/sec 81 sender
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec receiver
> > >
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 114 MBytes 956 Mbits/sec 0 747 KBytes
> > > [ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec 0 744 KBytes
> > > [ 5] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 12 1.18 MBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 2 699 KBytes
> > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 28 699 KBytes
> > > [ 5] 5.00-6.00 sec 112 MBytes 944 Mbits/sec 0 684 KBytes
> > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 0 741 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 3 687 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 22 699 KBytes
> > > [ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec 11 707 KBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 938 Mbits/sec 78 sender
> > > [ 5] 0.00-10.01 sec 1.09 GBytes 934 Mbits/sec receiver
> > > ---
> > >
> > > ethtool -S enp3s0 |grep -v ": 0"
> > > NIC statistics:
> > > rx_packets: 16303520
> > > tx_packets: 21602840
> > > rx_bytes: 15711958157
> > > tx_bytes: 25599009212
> > > rx_broadcast: 122212
> > > tx_broadcast: 530
> > > rx_multicast: 333489
> > > tx_multicast: 18446
> > > multicast: 333489
> > > rx_missed_errors: 270143
> > > rx_long_length_errors: 6
> > > tx_tcp_seg_good: 1342561
> > > rx_long_byte_count: 15711958157
> > > rx_errors: 6
> > > rx_length_errors: 6
> > > rx_fifo_errors: 270143
> > > tx_queue_0_packets: 8963830
> > > tx_queue_0_bytes: 9803196683
> > > tx_queue_0_restart: 4920
> > > tx_queue_1_packets: 12639010
> > > tx_queue_1_bytes: 15706576814
> > > tx_queue_1_restart: 12718
> > > rx_queue_0_packets: 16303520
> > > rx_queue_0_bytes: 15646744077
> > > rx_queue_0_csum_err: 76
> >
> > Okay, so this result still has the same length and checksum errors,
> > were you resetting the system/statistics between runs?
>
> Ah, no.... Will reset and do more tests when I'm back home
>
> Am I blind or is this part missing from ethtools man page?
There isn't a reset that will reset the stats via ethtool. The device
stats will be persistent until the driver is unloaded and reloaded or
the system is reset. You can reset the queue stats by changing the
number of queues. So for example using "ethtool -L enp3s0 1; ethtool
-L enp3s0 2".
^ permalink raw reply
* Re: [PATCH] spapr_pci: Robustify support of PCI bridges
From: David Gibson @ 2020-07-16 23:50 UTC (permalink / raw)
To: Greg Kurz
Cc: Thomas Huth, Michael S. Tsirkin, qemu-ppc, Markus Armbruster,
qemu-devel
In-Reply-To: <20200716164200.2bea2977@bahia.lan>
[-- Attachment #1: Type: text/plain, Size: 5516 bytes --]
On Thu, Jul 16, 2020 at 04:42:00PM +0200, Greg Kurz wrote:
> On Thu, 16 Jul 2020 16:01:18 +0200
> Markus Armbruster <armbru@redhat.com> wrote:
>
> > David Gibson <david@gibson.dropbear.id.au> writes:
> >
> > > On Thu, Jul 09, 2020 at 07:12:47PM +0200, Greg Kurz wrote:
> > >> Some recent error handling cleanups unveiled issues with our support of
> > >> PCI bridges:
> > >>
> > >> 1) QEMU aborts when using non-standard PCI bridge types,
> > >> unveiled by commit 7ef1553dac "spapr_pci: Drop some dead error handling"
> > >>
> > >> $ qemu-system-ppc64 -M pseries -device pcie-pci-bridge
> > >> Unexpected error in object_property_find() at qom/object.c:1240:
> > >> qemu-system-ppc64: -device pcie-pci-bridge: Property '.chassis_nr' not found
> > >> Aborted (core dumped)
> > >
> > > Oops, I thought we had a check that we actually had a "pci-bridge"
> > > device before continuing with the hotplug, but I guess not.
> > >
> > >> This happens because we assume all PCI bridge types to have a "chassis_nr"
> > >> property. This property only exists with the standard PCI bridge type
> > >> "pci-bridge" actually. We could possibly revert 7ef1553dac but it seems
> > >> much simpler to check the presence of "chassis_nr" earlier.
> > >
> > > Hrm, right, 7ef1553dac was not really correct since add_drcs() really
> > > can fail.
> >
> > Right. I failed to see that we can run into a bridge without a
> > "chassis_nr" here.
And I missed it on review, as well.
> > >> 2) QEMU abort if same "chassis_nr" value is used several times,
> > >> unveiled by commit d2623129a7de "qom: Drop parameter @errp of
> > >> object_property_add() & friends"
> > >>
> > >> $ qemu-system-ppc64 -M pseries -device pci-bridge,chassis_nr=1 \
> > >> -device pci-bridge,chassis_nr=1
> > >> Unexpected error in object_property_try_add() at qom/object.c:1167:
> > >> qemu-system-ppc64: -device pci-bridge,chassis_nr=1: attempt to add duplicate property '40000100' to object (type 'container')
> > >> Aborted (core dumped)
> >
> > Before d2623129a7de, the error got *ignored* in
> > spapr_dr_connector_new():
> >
> > SpaprDrc *spapr_dr_connector_new(Object *owner, const char *type,
> > uint32_t id)
> > {
> > SpaprDrc *drc = SPAPR_DR_CONNECTOR(object_new(type));
> > char *prop_name;
> >
> > drc->id = id;
> > drc->owner = owner;
> > prop_name = g_strdup_printf("dr-connector[%"PRIu32"]",
> > spapr_drc_index(drc));
> > object_property_add_child(owner, prop_name, OBJECT(drc), &error_abort);
> > object_unref(OBJECT(drc));
> > ---> object_property_set_bool(OBJECT(drc), true, "realized", NULL);
> > g_free(prop_name);
> >
> > return drc;
> > }
> >
> > I doubt that's healthy.
Indeed.
> This isn't. The object_property_set_bool() was later converted to
> qdev_realize() (thanks again for the cleanups!) but the problem
> remains. Realize can fail and I see now reason we don't do proper
> error handling when it comes to the DRCs.
>
> I'll look into fixing that.
>
> > >> This happens because we assume that "chassis_nr" values are unique, but
> > >> nobody enforces that and we end up generating duplicate DRC ids. The PCI
> > >> code doesn't really care for duplicate "chassis_nr" properties since it
> > >> is only used to initialize the "Chassis Number Register" of the bridge,
> > >> with no functional impact on QEMU. So, even if passing the same value
> > >> several times might look weird, it never broke anything before, so
> > >> I guess we don't necessarily want to enforce strict checking in the PCI
> > >> code now.
> > >
> > > Yeah, I guess. I'm pretty sure that the chassis number of bridges is
> > > supposed to be system-unique (well, unique within the PCI domain at
> > > least, I guess) as part of the hardware spec. So specifying multiple
> > > chassis ids the same is a user error, but we need a better failure
> > > mode.
> > >
> > >> Workaround both issues in the PAPR code: check that the bridge has a
> > >> unique and non null "chassis_nr" when plugging it into its parent bus.
> > >>
> > >> Fixes: 05929a6c5dfe ("spapr: Don't use bus number for building DRC ids")
> > >
> > > Arguably, it's really fixing 7ef1553dac.
> >
> > I agree 7ef1553dac broke the "use a bridge that doesn't have property
> > 'chassis_nr' case.
> >
> > I suspect the "duplicate chassis_nr" case has always been broken, and
> > d2623129a7de merely uncovered it.
>
> Yes.
I agree.
> > If we can trigger the abort with hot-plug, then d2623129a7de made things
> > materially worse (new way to accidentally kill your guest and maybe lose
> > data), and I'd add a Fixes: blaming it.
> >
>
> Yes it does.
>
> David,
>
> Maybe consider folding a third Fixes: tag into this patch ?
Done.
> > >> Reported-by: Thomas Huth <thuth@redhat.com>
> > >> Signed-off-by: Greg Kurz <groug@kaod.org>
> > >
> > > I had a few misgivings about the details of this, but I think I've
> > > convinced myself they're fine. There's a couple of things I'd like to
> > > polish, but I'll do that as a follow up.
> >
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH v4] spapr: Add a new level of NUMA for GPUs
From: David Gibson @ 2020-07-17 0:07 UTC (permalink / raw)
To: Reza Arbab
Cc: Leonardo Augusto Guimaraes Garcia, Alexey Kardashevskiy,
Daniel Henrique Barboza, Daniel Henrique Barboza, qemu-devel,
Greg Kurz, qemu-ppc
In-Reply-To: <20200716225655.24289-1-arbab@linux.ibm.com>
[-- Attachment #1: Type: text/plain, Size: 7282 bytes --]
On Thu, Jul 16, 2020 at 05:56:55PM -0500, Reza Arbab wrote:
> NUMA nodes corresponding to GPU memory currently have the same
> affinity/distance as normal memory nodes. Add a third NUMA associativity
> reference point enabling us to give GPU nodes more distance.
>
> This is guest visible information, which shouldn't change under a
> running guest across migration between different qemu versions, so make
> the change effective only in new (pseries > 5.0) machine types.
>
> Before, `numactl -H` output in a guest with 4 GPUs (nodes 2-5):
>
> node distances:
> node 0 1 2 3 4 5
> 0: 10 40 40 40 40 40
> 1: 40 10 40 40 40 40
> 2: 40 40 10 40 40 40
> 3: 40 40 40 10 40 40
> 4: 40 40 40 40 10 40
> 5: 40 40 40 40 40 10
>
> After:
>
> node distances:
> node 0 1 2 3 4 5
> 0: 10 40 80 80 80 80
> 1: 40 10 80 80 80 80
> 2: 80 80 10 80 80 80
> 3: 80 80 80 10 80 80
> 4: 80 80 80 80 10 80
> 5: 80 80 80 80 80 10
>
> These are the same distances as on the host, mirroring the change made
> to host firmware in skiboot commit f845a648b8cb ("numa/associativity:
> Add a new level of NUMA for GPU's").
Applied to ppc-for-5.1.
>
> Signed-off-by: Reza Arbab <arbab@linux.ibm.com>
> ---
> v4:
> * Use nvslot->numa_id for distinction at all levels of ibm,associativity
> * Use ARRAY_SIZE(refpoints)
> * Rebase
>
> v3:
> * Squash into one patch
> * Add PHB compat property
> ---
> hw/ppc/spapr.c | 21 +++++++++++++++++++--
> hw/ppc/spapr_pci.c | 2 ++
> hw/ppc/spapr_pci_nvlink2.c | 13 ++++++++++---
> include/hw/pci-host/spapr.h | 1 +
> include/hw/ppc/spapr.h | 1 +
> 5 files changed, 33 insertions(+), 5 deletions(-)
>
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 299908cc7396..0ae293ec9431 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -890,10 +890,16 @@ static int spapr_dt_rng(void *fdt)
> static void spapr_dt_rtas(SpaprMachineState *spapr, void *fdt)
> {
> MachineState *ms = MACHINE(spapr);
> + SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(ms);
> int rtas;
> GString *hypertas = g_string_sized_new(256);
> GString *qemu_hypertas = g_string_sized_new(256);
> - uint32_t refpoints[] = { cpu_to_be32(0x4), cpu_to_be32(0x4) };
> + uint32_t refpoints[] = {
> + cpu_to_be32(0x4),
> + cpu_to_be32(0x4),
> + cpu_to_be32(0x2),
> + };
> + uint32_t nr_refpoints = ARRAY_SIZE(refpoints);
> uint64_t max_device_addr = MACHINE(spapr)->device_memory->base +
> memory_region_size(&MACHINE(spapr)->device_memory->mr);
> uint32_t lrdr_capacity[] = {
> @@ -945,8 +951,12 @@ static void spapr_dt_rtas(SpaprMachineState *spapr, void *fdt)
> qemu_hypertas->str, qemu_hypertas->len));
> g_string_free(qemu_hypertas, TRUE);
>
> + if (smc->pre_5_1_assoc_refpoints) {
> + nr_refpoints = 2;
> + }
> +
> _FDT(fdt_setprop(fdt, rtas, "ibm,associativity-reference-points",
> - refpoints, sizeof(refpoints)));
> + refpoints, nr_refpoints * sizeof(refpoints[0])));
>
> _FDT(fdt_setprop(fdt, rtas, "ibm,max-associativity-domains",
> maxdomains, sizeof(maxdomains)));
> @@ -4584,9 +4594,16 @@ DEFINE_SPAPR_MACHINE(5_1, "5.1", true);
> */
> static void spapr_machine_5_0_class_options(MachineClass *mc)
> {
> + SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc);
> + static GlobalProperty compat[] = {
> + { TYPE_SPAPR_PCI_HOST_BRIDGE, "pre-5.1-associativity", "on" },
> + };
> +
> spapr_machine_5_1_class_options(mc);
> compat_props_add(mc->compat_props, hw_compat_5_0, hw_compat_5_0_len);
> + compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat));
> mc->numa_mem_supported = true;
> + smc->pre_5_1_assoc_refpoints = true;
> }
>
> DEFINE_SPAPR_MACHINE(5_0, "5.0", false);
> diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
> index 2a6a48744aaa..16739334e35f 100644
> --- a/hw/ppc/spapr_pci.c
> +++ b/hw/ppc/spapr_pci.c
> @@ -2035,6 +2035,8 @@ static Property spapr_phb_properties[] = {
> pcie_ecs, true),
> DEFINE_PROP_UINT64("gpa", SpaprPhbState, nv2_gpa_win_addr, 0),
> DEFINE_PROP_UINT64("atsd", SpaprPhbState, nv2_atsd_win_addr, 0),
> + DEFINE_PROP_BOOL("pre-5.1-associativity", SpaprPhbState,
> + pre_5_1_assoc, false),
> DEFINE_PROP_END_OF_LIST(),
> };
>
> diff --git a/hw/ppc/spapr_pci_nvlink2.c b/hw/ppc/spapr_pci_nvlink2.c
> index dd8cd6db9654..76ae77ebc851 100644
> --- a/hw/ppc/spapr_pci_nvlink2.c
> +++ b/hw/ppc/spapr_pci_nvlink2.c
> @@ -362,9 +362,9 @@ void spapr_phb_nvgpu_ram_populate_dt(SpaprPhbState *sphb, void *fdt)
> &error_abort);
> uint32_t associativity[] = {
> cpu_to_be32(0x4),
> - SPAPR_GPU_NUMA_ID,
> - SPAPR_GPU_NUMA_ID,
> - SPAPR_GPU_NUMA_ID,
> + cpu_to_be32(nvslot->numa_id),
> + cpu_to_be32(nvslot->numa_id),
> + cpu_to_be32(nvslot->numa_id),
> cpu_to_be32(nvslot->numa_id)
> };
> uint64_t size = object_property_get_uint(nv_mrobj, "size", NULL);
> @@ -375,6 +375,13 @@ void spapr_phb_nvgpu_ram_populate_dt(SpaprPhbState *sphb, void *fdt)
> _FDT(off);
> _FDT((fdt_setprop_string(fdt, off, "device_type", "memory")));
> _FDT((fdt_setprop(fdt, off, "reg", mem_reg, sizeof(mem_reg))));
> +
> + if (sphb->pre_5_1_assoc) {
> + associativity[1] = SPAPR_GPU_NUMA_ID;
> + associativity[2] = SPAPR_GPU_NUMA_ID;
> + associativity[3] = SPAPR_GPU_NUMA_ID;
> + }
> +
> _FDT((fdt_setprop(fdt, off, "ibm,associativity", associativity,
> sizeof(associativity))));
>
> diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
> index 8877ff51fbf7..600eb55c3488 100644
> --- a/include/hw/pci-host/spapr.h
> +++ b/include/hw/pci-host/spapr.h
> @@ -94,6 +94,7 @@ struct SpaprPhbState {
> hwaddr nv2_gpa_win_addr;
> hwaddr nv2_atsd_win_addr;
> SpaprPhbPciNvGpuConfig *nvgpus;
> + bool pre_5_1_assoc;
> };
>
> #define SPAPR_PCI_MEM_WIN_BUS_OFFSET 0x80000000ULL
> diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
> index c421410e3fb8..3134d339e8fe 100644
> --- a/include/hw/ppc/spapr.h
> +++ b/include/hw/ppc/spapr.h
> @@ -129,6 +129,7 @@ struct SpaprMachineClass {
> bool linux_pci_probe;
> bool smp_threads_vsmt; /* set VSMT to smp_threads by default */
> hwaddr rma_limit; /* clamp the RMA to this size */
> + bool pre_5_1_assoc_refpoints;
>
> void (*phb_placement)(SpaprMachineState *spapr, uint32_t index,
> uint64_t *buid, hwaddr *pio,
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* [Intel-wired-lan] NAT performance issue 944mbit -> ~40mbit
From: Alexander Duyck @ 2020-07-17 0:09 UTC (permalink / raw)
To: intel-wired-lan
In-Reply-To: <CAA85sZt6B+rG8pUfRoNVOH=VqHn=rT-+2kHpFDzW+eBwvODxJA@mail.gmail.com>
On Thu, Jul 16, 2020 at 12:47 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
>
> Sorry, tried to respond via the phone, used the webbrowser version but
> still html mails... :/
>
> On Thu, Jul 16, 2020 at 5:18 PM Alexander Duyck
> <alexander.duyck@gmail.com> wrote:
> > On Wed, Jul 15, 2020 at 5:00 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > On Thu, Jul 16, 2020 at 1:42 AM Alexander Duyck
> > > <alexander.duyck@gmail.com> wrote:
> > > > On Wed, Jul 15, 2020 at 3:51 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > > > On Thu, Jul 16, 2020 at 12:32 AM Alexander Duyck
> > > > > <alexander.duyck@gmail.com> wrote:
> > > > > > On Wed, Jul 15, 2020 at 3:00 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > > > > > On Wed, Jul 15, 2020 at 11:40 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > > > > > > > On Wed, 15 Jul 2020 23:12:23 +0200 Ian Kumlien wrote:
> > > > > > > > > On Wed, Jul 15, 2020 at 11:02 PM Ian Kumlien <ian.kumlien@gmail.com> wrote:
> > > > > > > > > > On Wed, Jul 15, 2020 at 10:31 PM Jakub Kicinski <kuba@kernel.org> wrote:
> > > > > > > > > > > On Wed, 15 Jul 2020 22:05:58 +0200 Ian Kumlien wrote:
> > > > > > > > > > > > After a lot of debugging it turns out that the bug is in igb...
> > > > > > > > > > > >
> > > > > > > > > > > > driver: igb
> > > > > > > > > > > > version: 5.6.0-k
> > > > > > > > > > > > firmware-version: 0. 6-1
> > > > > > > > > > > >
> > > > > > > > > > > > 03:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network
> > > > > > > > > > > > Connection (rev 03)
> > > > > > > > > > >
> > > > > > > > > > > Unclear to me what you're actually reporting. Is this a regression
> > > > > > > > > > > after a kernel upgrade? Compared to no NAT?
> > > > > > > > > >
> > > > > > > > > > It only happens on "internet links"
> > > > > > > > > >
> > > > > > > > > > Lets say that A is client with ibg driver, B is a firewall running NAT
> > > > > > > > > > with ixgbe drivers, C is another local node with igb and
> > > > > > > > > > D is a remote node with a bridge backed by a bnx2 interface.
> > > > > > > > > >
> > > > > > > > > > A -> B -> C is ok (B and C is on the same switch)
> > > > > > > > > >
> > > > > > > > > > A -> B -> D -- 32-40mbit
> > > > > > > > > >
> > > > > > > > > > B -> D 944 mbit
> > > > > > > > > > C -> D 944 mbit
> > > > > > > > > >
> > > > > > > > > > A' -> D ~933 mbit (A with realtek nic -- also link is not idle atm)
> > > > > > > > >
> > > > > > > > > This should of course be A' -> B -> D
> > > > > > > > >
> > > > > > > > > Sorry, I've been scratching my head for about a week...
> > > > > > > >
> > > > > > > > Hm, only thing that comes to mind if A' works reliably and A doesn't is
> > > > > > > > that A has somehow broken TCP offloads. Could you try disabling things
> > > > > > > > via ethtool -K and see if those settings make a difference?
> > > > > > >
> > > > > > > It's a bit hard since it works like this, turned tso off:
> > > > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > > > [ 5] 0.00-1.00 sec 108 MBytes 902 Mbits/sec 0 783 KBytes
> > > > > > > [ 5] 1.00-2.00 sec 110 MBytes 923 Mbits/sec 31 812 KBytes
> > > > > > > [ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 92 772 KBytes
> > > > > > > [ 5] 3.00-4.00 sec 110 MBytes 923 Mbits/sec 0 834 KBytes
> > > > > > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 60 823 KBytes
> > > > > > > [ 5] 5.00-6.00 sec 110 MBytes 923 Mbits/sec 31 789 KBytes
> > > > > > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 0 786 KBytes
> > > > > > > [ 5] 7.00-8.00 sec 110 MBytes 923 Mbits/sec 0 761 KBytes
> > > > > > > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 0 772 KBytes
> > > > > > > [ 5] 9.00-10.00 sec 109 MBytes 912 Mbits/sec 0 868 KBytes
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > > > [ 5] 0.00-10.00 sec 1.07 GBytes 923 Mbits/sec 214 sender
> > > > > > > [ 5] 0.00-10.00 sec 1.07 GBytes 920 Mbits/sec receiver
> > > > > > >
> > > > > > > Continued running tests:
> > > > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > > > [ 5] 0.00-1.00 sec 5.82 MBytes 48.8 Mbits/sec 0 82.0 KBytes
> > > > > > > [ 5] 1.00-2.00 sec 4.97 MBytes 41.7 Mbits/sec 0 130 KBytes
> > > > > > > [ 5] 2.00-3.00 sec 5.28 MBytes 44.3 Mbits/sec 0 99.0 KBytes
> > > > > > > [ 5] 3.00-4.00 sec 5.28 MBytes 44.3 Mbits/sec 0 105 KBytes
> > > > > > > [ 5] 4.00-5.00 sec 5.28 MBytes 44.3 Mbits/sec 0 122 KBytes
> > > > > > > [ 5] 5.00-6.00 sec 5.28 MBytes 44.3 Mbits/sec 0 82.0 KBytes
> > > > > > > [ 5] 6.00-7.00 sec 5.28 MBytes 44.3 Mbits/sec 0 79.2 KBytes
> > > > > > > [ 5] 7.00-8.00 sec 5.28 MBytes 44.3 Mbits/sec 0 110 KBytes
> > > > > > > [ 5] 8.00-9.00 sec 5.28 MBytes 44.3 Mbits/sec 0 156 KBytes
> > > > > > > [ 5] 9.00-10.00 sec 5.28 MBytes 44.3 Mbits/sec 0 87.7 KBytes
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > > > [ 5] 0.00-10.00 sec 53.0 MBytes 44.5 Mbits/sec 0 sender
> > > > > > > [ 5] 0.00-10.00 sec 52.5 MBytes 44.1 Mbits/sec receiver
> > > > > > >
> > > > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > > > [ 5] 0.00-1.00 sec 7.08 MBytes 59.4 Mbits/sec 0 156 KBytes
> > > > > > > [ 5] 1.00-2.00 sec 5.97 MBytes 50.0 Mbits/sec 0 110 KBytes
> > > > > > > [ 5] 2.00-3.00 sec 4.97 MBytes 41.7 Mbits/sec 0 124 KBytes
> > > > > > > [ 5] 3.00-4.00 sec 5.47 MBytes 45.9 Mbits/sec 0 96.2 KBytes
> > > > > > > [ 5] 4.00-5.00 sec 5.47 MBytes 45.9 Mbits/sec 0 158 KBytes
> > > > > > > [ 5] 5.00-6.00 sec 4.97 MBytes 41.7 Mbits/sec 0 70.7 KBytes
> > > > > > > [ 5] 6.00-7.00 sec 5.47 MBytes 45.9 Mbits/sec 0 113 KBytes
> > > > > > > [ 5] 7.00-8.00 sec 5.47 MBytes 45.9 Mbits/sec 0 96.2 KBytes
> > > > > > > [ 5] 8.00-9.00 sec 4.97 MBytes 41.7 Mbits/sec 0 84.8 KBytes
> > > > > > > [ 5] 9.00-10.00 sec 5.47 MBytes 45.9 Mbits/sec 0 116 KBytes
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > > > [ 5] 0.00-10.00 sec 55.3 MBytes 46.4 Mbits/sec 0 sender
> > > > > > > [ 5] 0.00-10.00 sec 53.9 MBytes 45.2 Mbits/sec receiver
> > > > > > >
> > > > > > > And the low bandwidth continues with:
> > > > > > > ethtool -k enp3s0 |grep ": on"
> > > > > > > rx-vlan-offload: on
> > > > > > > tx-vlan-offload: on [requested off]
> > > > > > > highdma: on [fixed]
> > > > > > > rx-vlan-filter: on [fixed]
> > > > > > > tx-gre-segmentation: on
> > > > > > > tx-gre-csum-segmentation: on
> > > > > > > tx-ipxip4-segmentation: on
> > > > > > > tx-ipxip6-segmentation: on
> > > > > > > tx-udp_tnl-segmentation: on
> > > > > > > tx-udp_tnl-csum-segmentation: on
> > > > > > > tx-gso-partial: on
> > > > > > > tx-udp-segmentation: on
> > > > > > > hw-tc-offload: on
> > > > > > >
> > > > > > > Can't quite find how to turn those off since they aren't listed in
> > > > > > > ethtool (since the text is not what you use to enable/disable)
> > > > > >
> > > > > > To disable them you would just repeat the same string in the display
> > > > > > string. So it should just be "ethtool -K enp3s0 tx-gso-partial off"
> > > > > > and that would turn off a large chunk of them as all the encapsulated
> > > > > > support requires gso partial support.
> > > > >
> > > > > ethtool -k enp3s0 |grep ": on"
> > > > > highdma: on [fixed]
> > > > > rx-vlan-filter: on [fixed]
> > > > > ---
> > > > > And then back to back:
> > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > [ 5] 0.00-1.00 sec 4.91 MBytes 41.2 Mbits/sec 0 45.2 KBytes
> > > > > [ 5] 1.00-2.00 sec 4.47 MBytes 37.5 Mbits/sec 0 52.3 KBytes
> > > > > [ 5] 2.00-3.00 sec 4.47 MBytes 37.5 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 3.00-4.00 sec 4.47 MBytes 37.5 Mbits/sec 0 141 KBytes
> > > > > [ 5] 4.00-5.00 sec 111 MBytes 928 Mbits/sec 63 764 KBytes
> > > > > [ 5] 5.00-6.00 sec 86.2 MBytes 724 Mbits/sec 0 744 KBytes
> > > > > [ 5] 6.00-7.00 sec 98.8 MBytes 828 Mbits/sec 61 769 KBytes
> > > > > [ 5] 7.00-8.00 sec 110 MBytes 923 Mbits/sec 0 749 KBytes
> > > > > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 0 741 KBytes
> > > > > [ 5] 9.00-10.00 sec 110 MBytes 923 Mbits/sec 31 761 KBytes
> > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > [ 5] 0.00-10.00 sec 644 MBytes 540 Mbits/sec 155 sender
> > > > > [ 5] 0.00-10.01 sec 641 MBytes 537 Mbits/sec receiver
> > > > >
> > > > > and we're back at the not working bit:
> > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > [ 5] 0.00-1.00 sec 4.84 MBytes 40.6 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 1.00-2.00 sec 4.60 MBytes 38.6 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 2.00-3.00 sec 4.23 MBytes 35.4 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 3.00-4.00 sec 4.47 MBytes 37.5 Mbits/sec 0 67.9 KBytes
> > > > > [ 5] 4.00-5.00 sec 4.47 MBytes 37.5 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 5.00-6.00 sec 4.23 MBytes 35.4 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 6.00-7.00 sec 4.23 MBytes 35.4 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 7.00-8.00 sec 4.47 MBytes 37.5 Mbits/sec 0 67.9 KBytes
> > > > > [ 5] 8.00-9.00 sec 4.47 MBytes 37.5 Mbits/sec 0 53.7 KBytes
> > > > > [ 5] 9.00-10.00 sec 4.47 MBytes 37.5 Mbits/sec 0 79.2 KBytes
> > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > [ 5] 0.00-10.00 sec 44.5 MBytes 37.3 Mbits/sec 0 sender
> > > > > [ 5] 0.00-10.00 sec 43.9 MBytes 36.8 Mbits/sec receiver
> > > > >
> > > > > > > I was hoping that you'd have a clue of something that might introduce
> > > > > > > a regression - ie specific patches to try to revert
> > > > > > >
> > > > > > > Btw, the same issue applies to udp as werll
> > > > > > >
> > > > > > > [ ID] Interval Transfer Bitrate Total Datagrams
> > > > > > > [ 5] 0.00-1.00 sec 6.77 MBytes 56.8 Mbits/sec 4900
> > > > > > > [ 5] 1.00-2.00 sec 4.27 MBytes 35.8 Mbits/sec 3089
> > > > > > > [ 5] 2.00-3.00 sec 4.20 MBytes 35.2 Mbits/sec 3041
> > > > > > > [ 5] 3.00-4.00 sec 4.30 MBytes 36.1 Mbits/sec 3116
> > > > > > > [ 5] 4.00-5.00 sec 4.24 MBytes 35.6 Mbits/sec 3070
> > > > > > > [ 5] 5.00-6.00 sec 4.21 MBytes 35.3 Mbits/sec 3047
> > > > > > > [ 5] 6.00-7.00 sec 4.29 MBytes 36.0 Mbits/sec 3110
> > > > > > > [ 5] 7.00-8.00 sec 4.28 MBytes 35.9 Mbits/sec 3097
> > > > > > > [ 5] 8.00-9.00 sec 4.25 MBytes 35.6 Mbits/sec 3075
> > > > > > > [ 5] 9.00-10.00 sec 4.20 MBytes 35.2 Mbits/sec 3039
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Jitter
> > > > > > > Lost/Total Datagrams
> > > > > > > [ 5] 0.00-10.00 sec 45.0 MBytes 37.7 Mbits/sec 0.000 ms
> > > > > > > 0/32584 (0%) sender
> > > > > > > [ 5] 0.00-10.00 sec 45.0 MBytes 37.7 Mbits/sec 0.037 ms
> > > > > > > 0/32573 (0%) receiver
> > > > > > >
> > > > > > > vs:
> > > > > > >
> > > > > > > [ ID] Interval Transfer Bitrate Total Datagrams
> > > > > > > [ 5] 0.00-1.00 sec 114 MBytes 954 Mbits/sec 82342
> > > > > > > [ 5] 1.00-2.00 sec 114 MBytes 955 Mbits/sec 82439
> > > > > > > [ 5] 2.00-3.00 sec 114 MBytes 956 Mbits/sec 82507
> > > > > > > [ 5] 3.00-4.00 sec 114 MBytes 955 Mbits/sec 82432
> > > > > > > [ 5] 4.00-5.00 sec 114 MBytes 956 Mbits/sec 82535
> > > > > > > [ 5] 5.00-6.00 sec 114 MBytes 953 Mbits/sec 82240
> > > > > > > [ 5] 6.00-7.00 sec 114 MBytes 956 Mbits/sec 82512
> > > > > > > [ 5] 7.00-8.00 sec 114 MBytes 956 Mbits/sec 82503
> > > > > > > [ 5] 8.00-9.00 sec 114 MBytes 956 Mbits/sec 82532
> > > > > > > [ 5] 9.00-10.00 sec 114 MBytes 956 Mbits/sec 82488
> > > > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > > > [ ID] Interval Transfer Bitrate Jitter
> > > > > > > Lost/Total Datagrams
> > > > > > > [ 5] 0.00-10.00 sec 1.11 GBytes 955 Mbits/sec 0.000 ms
> > > > > > > 0/824530 (0%) sender
> > > > > > > [ 5] 0.00-10.01 sec 1.11 GBytes 949 Mbits/sec 0.014 ms
> > > > > > > 4756/824530 (0.58%) receiver
> > > > > >
> > > > > > The fact that it is impacting UDP seems odd. I wonder if we don't have
> > > > > > a qdisc somewhere that is misbehaving and throttling the Tx. Either
> > > > > > that or I wonder if we are getting spammed with flow control frames.
> > > > >
> > > > > it sometimes works, it looks like the cwindow just isn't increased -
> > > > > that's where i started...
> > > > >
> > > > > Example:
> > > > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > > > [ 5] 0.00-1.00 sec 4.86 MBytes 40.8 Mbits/sec 0 50.9 KBytes
> > > > > [ 5] 1.00-2.00 sec 4.66 MBytes 39.1 Mbits/sec 0 65.0 KBytes
> > > > > [ 5] 2.00-3.00 sec 4.29 MBytes 36.0 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 3.00-4.00 sec 4.66 MBytes 39.1 Mbits/sec 0 42.4 KBytes
> > > > > [ 5] 4.00-5.00 sec 23.1 MBytes 194 Mbits/sec 0 1.07 MBytes
> > > > > [ 5] 5.00-6.00 sec 110 MBytes 923 Mbits/sec 0 761 KBytes
> > > > > [ 5] 6.00-7.00 sec 98.8 MBytes 828 Mbits/sec 60 806 KBytes
> > > > > [ 5] 7.00-8.00 sec 82.5 MBytes 692 Mbits/sec 0 812 KBytes
> > > > > [ 5] 8.00-9.00 sec 110 MBytes 923 Mbits/sec 92 761 KBytes
> > > > > [ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec 0 755 KBytes
> > > > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > > > [ ID] Interval Transfer Bitrate Retr
> > > > > [ 5] 0.00-10.00 sec 554 MBytes 465 Mbits/sec 152 sender
> > > > > [ 5] 0.00-10.00 sec 550 MBytes 461 Mbits/sec receiver
> > > > >
> > > > > > It would be useful to include the output of just calling "ethtool
> > > > > > enp3s0" on the interface to verify the speed, "ethtool -a enp3s0" to
> > > > > > verify flow control settings, and "ethtool -S enp3s0 | grep -v :\ 0"
> > > > > > to output the statistics and dump anything that isn't zero.
> > > > >
> > > > > ethtool enp3s0
> > > > > Settings for enp3s0:
> > > > > Supported ports: [ TP ]
> > > > > Supported link modes: 10baseT/Half 10baseT/Full
> > > > > 100baseT/Half 100baseT/Full
> > > > > 1000baseT/Full
> > > > > Supported pause frame use: Symmetric
> > > > > Supports auto-negotiation: Yes
> > > > > Supported FEC modes: Not reported
> > > > > Advertised link modes: 10baseT/Half 10baseT/Full
> > > > > 100baseT/Half 100baseT/Full
> > > > > 1000baseT/Full
> > > > > Advertised pause frame use: Symmetric
> > > > > Advertised auto-negotiation: Yes
> > > > > Advertised FEC modes: Not reported
> > > > > Speed: 1000Mb/s
> > > > > Duplex: Full
> > > > > Auto-negotiation: on
> > > > > Port: Twisted Pair
> > > > > PHYAD: 1
> > > > > Transceiver: internal
> > > > > MDI-X: off (auto)
> > > > > Supports Wake-on: pumbg
> > > > > Wake-on: g
> > > > > Current message level: 0x00000007 (7)
> > > > > drv probe link
> > > > > Link detected: yes
> > > > > ---
> > > > > ethtool -a enp3s0
> > > > > Pause parameters for enp3s0:
> > > > > Autonegotiate: on
> > > > > RX: on
> > > > > TX: off
> > > > > ---
> > > > > ethtool -S enp3s0 |grep -v :\ 0
> > > > > NIC statistics:
> > > > > rx_packets: 15920618
> > > > > tx_packets: 17846725
> > > > > rx_bytes: 15676264423
> > > > > tx_bytes: 19925010639
> > > > > rx_broadcast: 119553
> > > > > tx_broadcast: 497
> > > > > rx_multicast: 330193
> > > > > tx_multicast: 18190
> > > > > multicast: 330193
> > > > > rx_missed_errors: 270102
> > > > > rx_long_length_errors: 6
> > > > > tx_tcp_seg_good: 1342561
> > > > > rx_long_byte_count: 15676264423
> > > > > rx_errors: 6
> > > > > rx_length_errors: 6
> > > > > rx_fifo_errors: 270102
> > > > > tx_queue_0_packets: 7651168
> > > > > tx_queue_0_bytes: 7823281566
> > > > > tx_queue_0_restart: 4920
> > > > > tx_queue_1_packets: 10195557
> > > > > tx_queue_1_bytes: 12027522118
> > > > > tx_queue_1_restart: 12718
> > > > > rx_queue_0_packets: 15920618
> > > > > rx_queue_0_bytes: 15612581951
> > > > > rx_queue_0_csum_err: 76
> > > > > (I've only run two runs since i reenabled the interface)
> > > >
> > > > So I am seeing three things here.
> > > >
> > > > The rx_long_length_errors are usually due to an MTU mismatch. Do you
> > > > have something on the network that is using jumbo frames, or is the
> > > > MTU on the NIC set to something smaller than what is supported on the
> > > > network?
> > >
> > > I'm using jumbo frames on the local network, internet side is the
> > > normal 1500 bytes mtu though
> > >
> > > > You are getting rx_missed_errors, that would seem to imply that the
> > > > DMA is not able to keep up. We may want to try disabling the L1 to see
> > > > if we get any boost from doing that.
> > >
> > > It used to work, I don't do benchmarks all the time and sometimes the first
> > > benchmarks turn out fine... so it's hard to say when this started happening...
> > >
> > > It could also be related to a bios upgrade, but I'm pretty sure I did
> > > successful benchmarks after that...
> > >
> > > How do I disable the l1? just echo 0 >
> > > /sys/bus/pci/drivers/igb/0000\:03\:00.0/link/l1_aspm ?
> > >
> > > > The last bit is that queue 0 is seeing packets with bad checksums. You
> > > > might want to run some tests and see where the bad checksums are
> > > > coming from. If they are being detected from a specific NIC such as
> > > > the ixgbe in your example it might point to some sort of checksum
> > > > error being created as a result of the NAT translation.
> > >
> > > But that should also affect A' and the A -> B -> C case, which it doesn't...
> > >
> > > It only seems to happen with higher rtt (6 hops, sub 3 ms in this case
> > > but still high enough somehow)
> > >
> > > > > ---
> > > > >
> > > > > > > lspci -s 03:00.0 -vvv
> > > > > > > 03:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network
> > > > > > > Connection (rev 03)
> > > > > > > Subsystem: ASUSTeK Computer Inc. I211 Gigabit Network Connection
> > > > > > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> > > > > > > Stepping- SERR- FastB2B- DisINTx+
> > > > > > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> > > > > > > <TAbort- <MAbort- >SERR- <PERR- INTx-
> > > > > > > Latency: 0
> > > > > > > Interrupt: pin A routed to IRQ 57
> > > > > > > IOMMU group: 20
> > > > > > > Region 0: Memory at fc900000 (32-bit, non-prefetchable) [size=128K]
> > > > > > > Region 2: I/O ports at e000 [size=32]
> > > > > > > Region 3: Memory at fc920000 (32-bit, non-prefetchable) [size=16K]
> > > > > > > Capabilities: [40] Power Management version 3
> > > > > > > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
> > > > > > > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
> > > > > > > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> > > > > > > Address: 0000000000000000 Data: 0000
> > > > > > > Masking: 00000000 Pending: 00000000
> > > > > > > Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
> > > > > > > Vector table: BAR=3 offset=00000000
> > > > > > > PBA: BAR=3 offset=00002000
> > > > > > > Capabilities: [a0] Express (v2) Endpoint, MSI 00
> > > > > > > DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <512ns, L1 <64us
> > > > > > > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
> > > > > > > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
> > > > > > > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ FLReset-
> > > > > > > MaxPayload 128 bytes, MaxReadReq 512 bytes
> > > > > > > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
> > > > > > > LnkCap: Port #3, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency
> > > > > > > L0s <2us, L1 <16us
> > > > > > > ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
> > > > > > > LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
> > > > > > > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> > > > > > > LnkSta: Speed 2.5GT/s (ok), Width x1 (ok)
> > > > > > > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
> > > > > >
> > > > > > PCIe wise the connection is going to be pretty tight in terms of
> > > > > > bandwidth. It looks like we have 2.5GT/s with only a single lane of
> > > > > > PCIe. In addition we are running with ASPM enabled so that means that
> > > > > > if we don't have enough traffic we are shutting off the one PCIe lane
> > > > > > we have so if we are getting bursty traffic that can get ugly.
> > > > >
> > > > > Humm... is there a way to force disable ASPM in sysfs?
> > > >
> > > > Actually the easiest way to do this is to just use setpci.
> > > >
> > > > You should be able to dump the word containing the setting via:
> > > > # setpci -s 3:00.0 0xB0.w
> > > > 0042
> > > > # setpci -s 3:00.0 0xB0.w=0040
> > > >
> > > > Basically what you do is clear the lower 3 bits of the value so in
> > > > this case that means replacing the 2 with a 0 based on the output of
> > > > the first command.
> > >
> > > Well... I'll be damned... I used to force enable ASPM... this must be
> > > related to the change in PCIe bus ASPM
> > > Perhaps disable ASPM if there is only one link?
> >
> > Is there any specific reason why you are enabling ASPM? Is this system
> > a laptop where you are trying to conserve power when on battery? If
> > not disabling it probably won't hurt things too much since the power
> > consumption for a 2.5GT/s link operating in a width of one shouldn't
> > bee too high. Otherwise you are likely going to end up paying the
> > price for getting the interface out of L1 when the traffic goes idle
> > so you are going to see flows that get bursty paying a heavy penalty
> > when they start dropping packets.
>
> Ah, you misunderstand, I used to do this and everything worked - now
> Linux enables ASPM by default on all pcie controllers,
> so imho this should be a quirk, if there is only one lane, don't do
> ASPM due to latency and timing issues...
>
> > It is also possible this could be something that changed with the
> > physical PCIe link. Basically L1 works by powering down the link when
> > idle, and then powering it back up when there is activity. The problem
> > is bringing it back up can sometimes be a challenge when the physical
> > link starts to go faulty. I know I have seen that in some cases it can
> > even result in the device falling off of the PCIe bus if the link
> > training fails.
>
> It works fine without ASPM (and the machine is pretty new)
>
> I suspect we hit some timing race with aggressive ASPM (assumed as
> such since it works on local links but doesn't on ~3 ms Links)
Agreed. What is probably happening if you are using a NAT is that it
may be seeing some burstiness being introduced and as a result the
part is going to sleep and then being overrun when the traffic does
arrive.
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 113 MBytes 950 Mbits/sec 31 710 KBytes
> > > [ 5] 1.00-2.00 sec 110 MBytes 923 Mbits/sec 135 626 KBytes
> > > [ 5] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 18 713 KBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 0 798 KBytes
> > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 0 721 KBytes
> > > [ 5] 5.00-6.00 sec 112 MBytes 944 Mbits/sec 31 800 KBytes
> > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 0 730 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 19 730 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 0 701 KBytes
> > > [ 5] 9.00-10.00 sec 112 MBytes 944 Mbits/sec 12 701 KBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 937 Mbits/sec 246 sender
> > > [ 5] 0.00-10.01 sec 1.09 GBytes 933 Mbits/sec receiver
> > >
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 114 MBytes 956 Mbits/sec 0 749 KBytes
> > > [ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec 30 766 KBytes
> > > [ 5] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 7 749 KBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 11 707 KBytes
> > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 2 699 KBytes
> > > [ 5] 5.00-6.00 sec 111 MBytes 933 Mbits/sec 8 699 KBytes
> > > [ 5] 6.00-7.00 sec 112 MBytes 944 Mbits/sec 1 953 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 0 701 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 26 707 KBytes
> > > [ 5] 9.00-10.00 sec 112 MBytes 944 Mbits/sec 2 1.07 MBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 939 Mbits/sec 87 sender
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec receiver
> > >
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 114 MBytes 953 Mbits/sec 16 908 KBytes
> > > [ 5] 1.00-2.00 sec 112 MBytes 944 Mbits/sec 0 693 KBytes
> > > [ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 0 713 KBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 0 687 KBytes
> > > [ 5] 4.00-5.00 sec 112 MBytes 944 Mbits/sec 15 687 KBytes
> > > [ 5] 5.00-6.00 sec 111 MBytes 933 Mbits/sec 2 888 KBytes
> > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 17 696 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 0 758 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 31 749 KBytes
> > > [ 5] 9.00-10.00 sec 112 MBytes 944 Mbits/sec 0 792 KBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 938 Mbits/sec 81 sender
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 934 Mbits/sec receiver
> > >
> > > [ ID] Interval Transfer Bitrate Retr Cwnd
> > > [ 5] 0.00-1.00 sec 114 MBytes 956 Mbits/sec 0 747 KBytes
> > > [ 5] 1.00-2.00 sec 111 MBytes 933 Mbits/sec 0 744 KBytes
> > > [ 5] 2.00-3.00 sec 112 MBytes 944 Mbits/sec 12 1.18 MBytes
> > > [ 5] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 2 699 KBytes
> > > [ 5] 4.00-5.00 sec 111 MBytes 933 Mbits/sec 28 699 KBytes
> > > [ 5] 5.00-6.00 sec 112 MBytes 944 Mbits/sec 0 684 KBytes
> > > [ 5] 6.00-7.00 sec 111 MBytes 933 Mbits/sec 0 741 KBytes
> > > [ 5] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 3 687 KBytes
> > > [ 5] 8.00-9.00 sec 111 MBytes 933 Mbits/sec 22 699 KBytes
> > > [ 5] 9.00-10.00 sec 111 MBytes 933 Mbits/sec 11 707 KBytes
> > > - - - - - - - - - - - - - - - - - - - - - - - - -
> > > [ ID] Interval Transfer Bitrate Retr
> > > [ 5] 0.00-10.00 sec 1.09 GBytes 938 Mbits/sec 78 sender
> > > [ 5] 0.00-10.01 sec 1.09 GBytes 934 Mbits/sec receiver
> > > ---
> > >
> > > ethtool -S enp3s0 |grep -v ": 0"
> > > NIC statistics:
> > > rx_packets: 16303520
> > > tx_packets: 21602840
> > > rx_bytes: 15711958157
> > > tx_bytes: 25599009212
> > > rx_broadcast: 122212
> > > tx_broadcast: 530
> > > rx_multicast: 333489
> > > tx_multicast: 18446
> > > multicast: 333489
> > > rx_missed_errors: 270143
> > > rx_long_length_errors: 6
> > > tx_tcp_seg_good: 1342561
> > > rx_long_byte_count: 15711958157
> > > rx_errors: 6
> > > rx_length_errors: 6
> > > rx_fifo_errors: 270143
> > > tx_queue_0_packets: 8963830
> > > tx_queue_0_bytes: 9803196683
> > > tx_queue_0_restart: 4920
> > > tx_queue_1_packets: 12639010
> > > tx_queue_1_bytes: 15706576814
> > > tx_queue_1_restart: 12718
> > > rx_queue_0_packets: 16303520
> > > rx_queue_0_bytes: 15646744077
> > > rx_queue_0_csum_err: 76
> >
> > Okay, so this result still has the same length and checksum errors,
> > were you resetting the system/statistics between runs?
>
> Ah, no.... Will reset and do more tests when I'm back home
>
> Am I blind or is this part missing from ethtools man page?
There isn't a reset that will reset the stats via ethtool. The device
stats will be persistent until the driver is unloaded and reloaded or
the system is reset. You can reset the queue stats by changing the
number of queues. So for example using "ethtool -L enp3s0 1; ethtool
-L enp3s0 2".
^ permalink raw reply
* Re: [PATCH 09/11] xfs: create xfs_dqtype_t to represent quota types
From: Dave Chinner @ 2020-07-17 0:08 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: linux-xfs
In-Reply-To: <159488197642.3813063.4673664984532713595.stgit@magnolia>
On Wed, Jul 15, 2020 at 11:46:16PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
>
> Create a new type (xfs_dqtype_t) to represent the type of an incore
> dquot (user, group, project, or none). Rename the incore dquot's
> dq_flags field to q_type.
>
> This allows us to replace all the "uint type" arguments to the quota
> functions with "xfs_dqtype_t type", to make it obvious when we're
> passing a quota type argument into a function.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> fs/xfs/libxfs/xfs_dquot_buf.c | 2 +-
> fs/xfs/libxfs/xfs_format.h | 9 +++++++++
> fs/xfs/libxfs/xfs_quota_defs.h | 27 ++++++++++++---------------
> fs/xfs/scrub/quota.c | 8 ++++----
> fs/xfs/scrub/repair.c | 4 ++--
> fs/xfs/scrub/repair.h | 4 +++-
> fs/xfs/xfs_dquot.c | 37 +++++++++++++++++++------------------
> fs/xfs/xfs_dquot.h | 33 +++++++++++++++++----------------
> fs/xfs/xfs_iomap.c | 24 ++++++++++++------------
> fs/xfs/xfs_qm.c | 22 +++++++++++-----------
> fs/xfs/xfs_qm.h | 26 ++++++++++++++++----------
> fs/xfs/xfs_qm_syscalls.c | 8 ++++----
> fs/xfs/xfs_quota.h | 4 ++--
> fs/xfs/xfs_quotaops.c | 2 +-
> fs/xfs/xfs_trace.h | 21 +++++++++++++++------
> 15 files changed, 128 insertions(+), 103 deletions(-)
Looks fine.
Reviewed-by: Dave Chinner <dchinner@redhat.com>
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply
* Re: [RFC PATCH 4/7] x86: use exit_lazy_tlb rather than membarrier_mm_sync_core_before_usermode
From: Nicholas Piggin @ 2020-07-17 0:00 UTC (permalink / raw)
To: Mathieu Desnoyers, paulmck, Alan Stern
Cc: linux-arch, Arnd Bergmann, Peter Zijlstra, x86, linux-kernel,
linux-mm, Andy Lutomirski, linuxppc-dev
In-Reply-To: <595582123.17106.1594925921537.JavaMail.zimbra@efficios.com>
Excerpts from Mathieu Desnoyers's message of July 17, 2020 4:58 am:
> ----- On Jul 16, 2020, at 12:03 PM, Mathieu Desnoyers mathieu.desnoyers@efficios.com wrote:
>
>> ----- On Jul 16, 2020, at 11:46 AM, Mathieu Desnoyers
>> mathieu.desnoyers@efficios.com wrote:
>>
>>> ----- On Jul 16, 2020, at 12:42 AM, Nicholas Piggin npiggin@gmail.com wrote:
>>>> I should be more complete here, especially since I was complaining
>>>> about unclear barrier comment :)
>>>>
>>>>
>>>> CPU0 CPU1
>>>> a. user stuff 1. user stuff
>>>> b. membarrier() 2. enter kernel
>>>> c. smp_mb() 3. smp_mb__after_spinlock(); // in __schedule
>>>> d. read rq->curr 4. rq->curr switched to kthread
>>>> e. is kthread, skip IPI 5. switch_to kthread
>>>> f. return to user 6. rq->curr switched to user thread
>>>> g. user stuff 7. switch_to user thread
>>>> 8. exit kernel
>>>> 9. more user stuff
>>>>
>>>> What you're really ordering is a, g vs 1, 9 right?
>>>>
>>>> In other words, 9 must see a if it sees g, g must see 1 if it saw 9,
>>>> etc.
>>>>
>>>> Userspace does not care where the barriers are exactly or what kernel
>>>> memory accesses might be being ordered by them, so long as there is a
>>>> mb somewhere between a and g, and 1 and 9. Right?
>>>
>>> This is correct.
>>
>> Actually, sorry, the above is not quite right. It's been a while
>> since I looked into the details of membarrier.
>>
>> The smp_mb() at the beginning of membarrier() needs to be paired with a
>> smp_mb() _after_ rq->curr is switched back to the user thread, so the
>> memory barrier is between store to rq->curr and following user-space
>> accesses.
>>
>> The smp_mb() at the end of membarrier() needs to be paired with the
>> smp_mb__after_spinlock() at the beginning of schedule, which is
>> between accesses to userspace memory and switching rq->curr to kthread.
>>
>> As to *why* this ordering is needed, I'd have to dig through additional
>> scenarios from https://lwn.net/Articles/573436/. Or maybe Paul remembers ?
>
> Thinking further about this, I'm beginning to consider that maybe we have been
> overly cautious by requiring memory barriers before and after store to rq->curr.
>
> If CPU0 observes a CPU1's rq->curr->mm which differs from its own process (current)
> while running the membarrier system call, it necessarily means that CPU1 had
> to issue smp_mb__after_spinlock when entering the scheduler, between any user-space
> loads/stores and update of rq->curr.
>
> Requiring a memory barrier between update of rq->curr (back to current process's
> thread) and following user-space memory accesses does not seem to guarantee
> anything more than what the initial barrier at the beginning of __schedule already
> provides, because the guarantees are only about accesses to user-space memory.
>
> Therefore, with the memory barrier at the beginning of __schedule, just observing that
> CPU1's rq->curr differs from current should guarantee that a memory barrier was issued
> between any sequentially consistent instructions belonging to the current process on
> CPU1.
>
> Or am I missing/misremembering an important point here ?
I might have mislead you.
CPU0 CPU1
r1=y x=1
membarrier() y=1
r2=x
membarrier provides if r1==1 then r2==1 (right?)
CPU0
r1=y
membarrier()
smp_mb();
t = cpu_rq(1)->curr;
if (t->mm == mm)
IPI(CPU1);
smp_mb()
r2=x
vs
CPU1
...
__schedule()
smp_mb__after_spinlock()
rq->curr = kthread
...
__schedule()
smp_mb__after_spinlock()
rq->curr = user thread
exit kernel
x=1
y=1
Now these last 3 stores are not ordered, so CPU0 might see y==1 but
rq->curr == kthread, right? Then it will skip the IPI and stores to x
and y will not be ordered.
So we do need a mb after rq->curr store when mm is switching.
I believe for the global membarrier PF_KTHREAD optimisation, we also
need a barrier when switching from a kernel thread to user, for the
same reason.
So I think I was wrong to say the barrier is not necessary.
I haven't quite worked out why two mb()s are required in membarrier(),
but at least that's less of a performance concern.
Thanks,
Nick
^ permalink raw reply
* Re: [PATCH v8 5/8] powerpc/vdso: Prepare for switching VDSO to generic C implementation.
From: Tulio Magno Quites Machado Filho @ 2020-07-16 23:18 UTC (permalink / raw)
To: Christophe Leroy, Michael Ellerman
Cc: linux-arch, nathanl, arnd, linux-kernel, Paul Mackerras,
Christophe Leroy, luto, tglx, vincenzo.frascino, linuxppc-dev
In-Reply-To: <20200715204725.Horde.5GZvsEv4ZkdzFHL76HZiFg8@messagerie.si.c-s.fr>
Christophe Leroy <christophe.leroy@csgroup.eu> writes:
> Michael Ellerman <mpe@ellerman.id.au> a écrit :
>
>> Christophe Leroy <christophe.leroy@c-s.fr> writes:
>>> Prepare for switching VDSO to generic C implementation in following
>>> patch. Here, we:
>>> - Modify __get_datapage() to take an offset
>>> - Prepare the helpers to call the C VDSO functions
>>> - Prepare the required callbacks for the C VDSO functions
>>> - Prepare the clocksource.h files to define VDSO_ARCH_CLOCKMODES
>>> - Add the C trampolines to the generic C VDSO functions
>>>
>>> powerpc is a bit special for VDSO as well as system calls in the
>>> way that it requires setting CR SO bit which cannot be done in C.
>>> Therefore, entry/exit needs to be performed in ASM.
>>>
>>> Implementing __arch_get_vdso_data() would clobber the link register,
>>> requiring the caller to save it. As the ASM calling function already
>>> has to set a stack frame and saves the link register before calling
>>> the C vdso function, retriving the vdso data pointer there is lighter.
>> ...
>>
>>> diff --git a/arch/powerpc/include/asm/vdso/gettimeofday.h
>>> b/arch/powerpc/include/asm/vdso/gettimeofday.h
>>> new file mode 100644
>>> index 000000000000..4452897f9bd8
>>> --- /dev/null
>>> +++ b/arch/powerpc/include/asm/vdso/gettimeofday.h
>>> @@ -0,0 +1,175 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +#ifndef __ASM_VDSO_GETTIMEOFDAY_H
>>> +#define __ASM_VDSO_GETTIMEOFDAY_H
>>> +
>>> +#include <asm/ptrace.h>
>>> +
>>> +#ifdef __ASSEMBLY__
>>> +
>>> +.macro cvdso_call funct
>>> + .cfi_startproc
>>> + PPC_STLU r1, -STACK_FRAME_OVERHEAD(r1)
>>> + mflr r0
>>> + .cfi_register lr, r0
>>> + PPC_STL r0, STACK_FRAME_OVERHEAD + PPC_LR_STKOFF(r1)
>>
>> This doesn't work for me on ppc64(le) with glibc.
>>
>> glibc doesn't create a stack frame before making the VDSO call, so the
>> store of r0 (LR) goes into the caller's frame, corrupting the saved LR,
>> leading to an infinite loop.
>
> Where should it be saved if it can't be saved in the standard location ?
As Michael pointed out, userspace doesn't treat the VDSO as a normal function
call. In order to keep compatibility with existent software, LR would need to
be saved on another stack frame.
--
Tulio Magno
^ permalink raw reply
* Re: [PATCH v3 07/21] ASoC: atmel: merge .digital_mute() into .mute_stream()
From: Mark Brown @ 2020-07-16 23:59 UTC (permalink / raw)
To: Kuninori Morimoto; +Cc: Linux-ALSA
In-Reply-To: <87mu3z2g97.wl-kuninori.morimoto.gx@renesas.com>
[-- Attachment #1: Type: text/plain, Size: 583 bytes --]
On Fri, Jul 17, 2020 at 08:27:46AM +0900, Kuninori Morimoto wrote:
> > > snd_soc_dai_digital_mute() is internally using both
> > > mute_stream() (1) or digital_mute() (2), but the difference between
> > > these 2 are only handling direction.
> > At least this patch needs a rebase against current code, possibly others.
> I noticed it yesterday.
> OK, I will rebase and re-post v4
Turned out it was just that one - everything else (except the final
removal patches) was fine, git fixed up the remaining issues itself.
> Thank you for your help !!
Thank you for your hard work!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* [net-next V2 15/15] net/mlx5e: CT: Map 128 bits labels to 32 bit map ID
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Eli Britstein, Oz Shlomo, Roi Dayan, Maor Dickman,
Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Eli Britstein <elibr@mellanox.com>
The 128 bits ct_label field is matched using a 32 bit hardware register.
As such, only the lower 32 bits of ct_label field are offloaded. Change
this logic to support setting and matching higher bits too.
Map the 128 bits data to a unique 32 bits ID. Matching is done as exact
match of the mapping ID of key & mask.
Signed-off-by: Eli Britstein <elibr@mellanox.com>
Reviewed-by: Oz Shlomo <ozsh@mellanox.com>
Reviewed-by: Roi Dayan <roid@mellanox.com>
Reviewed-by: Maor Dickman <maord@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../ethernet/mellanox/mlx5/core/en/tc_ct.c | 59 ++++++++++++-------
.../ethernet/mellanox/mlx5/core/en/tc_ct.h | 3 +
.../net/ethernet/mellanox/mlx5/core/en_tc.c | 3 +-
3 files changed, 42 insertions(+), 23 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
index 4c65677feaabf..c6bc9224c3b18 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c
@@ -32,6 +32,9 @@
#define MLX5_FTE_ID_MAX GENMASK(MLX5_FTE_ID_BITS - 1, 0)
#define MLX5_FTE_ID_MASK MLX5_FTE_ID_MAX
+#define MLX5_CT_LABELS_BITS (mlx5e_tc_attr_to_reg_mappings[LABELS_TO_REG].mlen * 8)
+#define MLX5_CT_LABELS_MASK GENMASK(MLX5_CT_LABELS_BITS - 1, 0)
+
#define ct_dbg(fmt, args...)\
netdev_dbg(ct_priv->netdev, "ct_debug: " fmt "\n", ##args)
@@ -48,6 +51,7 @@ struct mlx5_tc_ct_priv {
struct mlx5_flow_table *post_ct;
struct mutex control_lock; /* guards parallel adds/dels */
struct mapping_ctx *zone_mapping;
+ struct mapping_ctx *labels_mapping;
};
struct mlx5_ct_flow {
@@ -404,6 +408,7 @@ mlx5_tc_ct_entry_del_rule(struct mlx5_tc_ct_priv *ct_priv,
mlx5_eswitch_del_offloaded_rule(esw, zone_rule->rule, attr);
mlx5e_mod_hdr_detach(ct_priv->esw->dev,
&esw->offloads.mod_hdr, zone_rule->mh);
+ mapping_remove(ct_priv->labels_mapping, attr->ct_attr.ct_labels_id);
}
static void
@@ -436,7 +441,7 @@ mlx5_tc_ct_entry_set_registers(struct mlx5_tc_ct_priv *ct_priv,
struct mlx5e_tc_mod_hdr_acts *mod_acts,
u8 ct_state,
u32 mark,
- u32 label,
+ u32 labels_id,
u8 zone_restore_id)
{
struct mlx5_eswitch *esw = ct_priv->esw;
@@ -453,7 +458,7 @@ mlx5_tc_ct_entry_set_registers(struct mlx5_tc_ct_priv *ct_priv,
return err;
err = mlx5e_tc_match_to_reg_set(esw->dev, mod_acts,
- LABELS_TO_REG, label);
+ LABELS_TO_REG, labels_id);
if (err)
return err;
@@ -597,13 +602,10 @@ mlx5_tc_ct_entry_create_mod_hdr(struct mlx5_tc_ct_priv *ct_priv,
if (!meta)
return -EOPNOTSUPP;
- if (meta->ct_metadata.labels[1] ||
- meta->ct_metadata.labels[2] ||
- meta->ct_metadata.labels[3]) {
- ct_dbg("Failed to offload ct entry due to unsupported label");
+ err = mapping_add(ct_priv->labels_mapping, meta->ct_metadata.labels,
+ &attr->ct_attr.ct_labels_id);
+ if (err)
return -EOPNOTSUPP;
- }
-
if (nat) {
err = mlx5_tc_ct_entry_create_nat(ct_priv, flow_rule,
&mod_acts);
@@ -617,7 +619,7 @@ mlx5_tc_ct_entry_create_mod_hdr(struct mlx5_tc_ct_priv *ct_priv,
err = mlx5_tc_ct_entry_set_registers(ct_priv, &mod_acts,
ct_state,
meta->ct_metadata.mark,
- meta->ct_metadata.labels[0],
+ attr->ct_attr.ct_labels_id,
zone_restore_id);
if (err)
goto err_mapping;
@@ -637,6 +639,7 @@ mlx5_tc_ct_entry_create_mod_hdr(struct mlx5_tc_ct_priv *ct_priv,
err_mapping:
dealloc_mod_hdr_actions(&mod_acts);
+ mapping_remove(ct_priv->labels_mapping, attr->ct_attr.ct_labels_id);
return err;
}
@@ -959,6 +962,7 @@ int
mlx5_tc_ct_parse_match(struct mlx5e_priv *priv,
struct mlx5_flow_spec *spec,
struct flow_cls_offload *f,
+ struct mlx5_ct_attr *ct_attr,
struct netlink_ext_ack *extack)
{
struct mlx5_tc_ct_priv *ct_priv = mlx5_tc_ct_get_ct_priv(priv);
@@ -969,6 +973,7 @@ mlx5_tc_ct_parse_match(struct mlx5e_priv *priv,
u16 ct_state_on, ct_state_off;
u16 ct_state, ct_state_mask;
struct flow_match_ct match;
+ u32 ct_labels[4];
if (!flow_rule_match_key(rule, FLOW_DISSECTOR_KEY_CT))
return 0;
@@ -995,12 +1000,6 @@ mlx5_tc_ct_parse_match(struct mlx5e_priv *priv,
return -EOPNOTSUPP;
}
- if (mask->ct_labels[1] || mask->ct_labels[2] || mask->ct_labels[3]) {
- NL_SET_ERR_MSG_MOD(extack,
- "only lower 32bits of ct_labels are supported for offload");
- return -EOPNOTSUPP;
- }
-
ct_state_on = ct_state & ct_state_mask;
ct_state_off = (ct_state & ct_state_mask) ^ ct_state_mask;
trk = ct_state_on & TCA_FLOWER_KEY_CT_FLAGS_TRACKED;
@@ -1029,10 +1028,17 @@ mlx5_tc_ct_parse_match(struct mlx5e_priv *priv,
if (mask->ct_mark)
mlx5e_tc_match_to_reg_match(spec, MARK_TO_REG,
key->ct_mark, mask->ct_mark);
- if (mask->ct_labels[0])
- mlx5e_tc_match_to_reg_match(spec, LABELS_TO_REG,
- key->ct_labels[0],
- mask->ct_labels[0]);
+ if (mask->ct_labels[0] || mask->ct_labels[1] || mask->ct_labels[2] ||
+ mask->ct_labels[3]) {
+ ct_labels[0] = key->ct_labels[0] & mask->ct_labels[0];
+ ct_labels[1] = key->ct_labels[1] & mask->ct_labels[1];
+ ct_labels[2] = key->ct_labels[2] & mask->ct_labels[2];
+ ct_labels[3] = key->ct_labels[3] & mask->ct_labels[3];
+ if (mapping_add(ct_priv->labels_mapping, ct_labels, &ct_attr->ct_labels_id))
+ return -EOPNOTSUPP;
+ mlx5e_tc_match_to_reg_match(spec, LABELS_TO_REG, ct_attr->ct_labels_id,
+ MLX5_CT_LABELS_MASK);
+ }
return 0;
}
@@ -1398,7 +1404,7 @@ mlx5_tc_ct_del_ft_cb(struct mlx5_tc_ct_priv *ct_priv, struct mlx5_ct_ft *ft)
* + tuple + zone match +
* +--------------------+
* | set mark
- * | set label
+ * | set labels_id
* | set established
* | set zone_restore
* | do nat (if needed)
@@ -1789,7 +1795,13 @@ mlx5_tc_ct_init(struct mlx5_rep_uplink_priv *uplink_priv)
ct_priv->zone_mapping = mapping_create(sizeof(u16), 0, true);
if (IS_ERR(ct_priv->zone_mapping)) {
err = PTR_ERR(ct_priv->zone_mapping);
- goto err_mapping;
+ goto err_mapping_zone;
+ }
+
+ ct_priv->labels_mapping = mapping_create(sizeof(u32) * 4, 0, true);
+ if (IS_ERR(ct_priv->labels_mapping)) {
+ err = PTR_ERR(ct_priv->labels_mapping);
+ goto err_mapping_labels;
}
ct_priv->esw = esw;
@@ -1833,8 +1845,10 @@ mlx5_tc_ct_init(struct mlx5_rep_uplink_priv *uplink_priv)
err_ct_nat_tbl:
mlx5_esw_chains_destroy_global_table(esw, ct_priv->ct);
err_ct_tbl:
+ mapping_destroy(ct_priv->labels_mapping);
+err_mapping_labels:
mapping_destroy(ct_priv->zone_mapping);
-err_mapping:
+err_mapping_zone:
kfree(ct_priv);
err_alloc:
err_support:
@@ -1854,6 +1868,7 @@ mlx5_tc_ct_clean(struct mlx5_rep_uplink_priv *uplink_priv)
mlx5_esw_chains_destroy_global_table(ct_priv->esw, ct_priv->ct_nat);
mlx5_esw_chains_destroy_global_table(ct_priv->esw, ct_priv->ct);
mapping_destroy(ct_priv->zone_mapping);
+ mapping_destroy(ct_priv->labels_mapping);
rhashtable_destroy(&ct_priv->ct_tuples_ht);
rhashtable_destroy(&ct_priv->ct_tuples_nat_ht);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.h
index 5e10a72f5f240..3baef917a677a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.h
@@ -25,6 +25,7 @@ struct mlx5_ct_attr {
u16 ct_action;
struct mlx5_ct_flow *ct_flow;
struct nf_flowtable *nf_ft;
+ u32 ct_labels_id;
};
#define zone_to_reg_ct {\
@@ -90,6 +91,7 @@ int
mlx5_tc_ct_parse_match(struct mlx5e_priv *priv,
struct mlx5_flow_spec *spec,
struct flow_cls_offload *f,
+ struct mlx5_ct_attr *ct_attr,
struct netlink_ext_ack *extack);
int
mlx5_tc_ct_add_no_trk_match(struct mlx5e_priv *priv,
@@ -132,6 +134,7 @@ static inline int
mlx5_tc_ct_parse_match(struct mlx5e_priv *priv,
struct mlx5_flow_spec *spec,
struct flow_cls_offload *f,
+ struct mlx5_ct_attr *ct_attr,
struct netlink_ext_ack *extack)
{
struct flow_rule *rule = flow_cls_offload_flow_rule(f);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
index b366c46a56041..7a0c22d05575a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c
@@ -4401,7 +4401,8 @@ __mlx5e_add_fdb_flow(struct mlx5e_priv *priv,
goto err_free;
/* actions validation depends on parsing the ct matches first */
- err = mlx5_tc_ct_parse_match(priv, &parse_attr->spec, f, extack);
+ err = mlx5_tc_ct_parse_match(priv, &parse_attr->spec, f,
+ &flow->esw_attr->ct_attr, extack);
if (err)
goto err_free;
--
2.26.2
^ permalink raw reply related
* [net-next V2 09/15] net/mlx5e: IPsec: Add IPsec steering in local NIC RX
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Huy Nguyen, Boris Pismenny, Tariq Toukan, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Huy Nguyen <huyn@mellanox.com>
Introduce decrypt FT, the RX error FT and the default rules.
The IPsec RX decrypt flow table is pointed by the TTC
(Traffic Type Classifier) ESP steering rules.
The decrypt flow table has two flow groups. The first flow group
keeps the decrypt steering rule programmed via the "ip xfrm s" interface.
The second flow group has a default rule to forward all non-offloaded
ESP packet to the TTC ESP default RSS TIR.
The RX error flow table is the destination of the decrypt steering rules
in the IPsec RX decrypt flow table. It has a fixed rule with single
copy action that copies ipsec_syndrome to metadata_regB[0:6]. The IPsec
syndrome is used to filter out non-ipsec packet and to return the IPsec
crypto offload status in Rx flow. The destination of RX error flow table
is the TTC ESP default RSS TIR.
All the FTs (decrypt FT and error FT) are created only when IPsec SAs
are added. If there is no IPsec SAs, the FTs are removed.
Signed-off-by: Huy Nguyen <huyn@mellanox.com>
Reviewed-by: Boris Pismenny <borisp@mellanox.com>
Reviewed-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/Makefile | 2 +-
.../mellanox/mlx5/core/accel/ipsec_offload.c | 6 +
.../net/ethernet/mellanox/mlx5/core/en/fs.h | 6 +-
.../mellanox/mlx5/core/en_accel/ipsec.c | 37 +-
.../mellanox/mlx5/core/en_accel/ipsec.h | 10 +
.../mellanox/mlx5/core/en_accel/ipsec_fs.c | 544 ++++++++++++++++++
.../mellanox/mlx5/core/en_accel/ipsec_fs.h | 26 +
.../net/ethernet/mellanox/mlx5/core/fs_cmd.c | 2 +
.../net/ethernet/mellanox/mlx5/core/fs_core.c | 2 +-
9 files changed, 630 insertions(+), 5 deletions(-)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.h
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index 375c8a2d42fa8..10e6886c96ba8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -72,7 +72,7 @@ mlx5_core-$(CONFIG_MLX5_ACCEL) += lib/crypto.o accel/tls.o accel/ipsec.o
mlx5_core-$(CONFIG_MLX5_FPGA) += fpga/cmd.o fpga/core.o fpga/conn.o fpga/sdk.o
mlx5_core-$(CONFIG_MLX5_EN_IPSEC) += en_accel/ipsec.o en_accel/ipsec_rxtx.o \
- en_accel/ipsec_stats.o
+ en_accel/ipsec_stats.o en_accel/ipsec_fs.o
mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/tls.o en_accel/tls_rxtx.o en_accel/tls_stats.o \
en_accel/fs_tcp.o en_accel/ktls.o en_accel/ktls_txrx.o \
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
index 1c8923f42b093..c49699d580fff 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
@@ -4,9 +4,11 @@
#include "mlx5_core.h"
#include "ipsec_offload.h"
#include "lib/mlx5.h"
+#include "en_accel/ipsec_fs.h"
#define MLX5_IPSEC_DEV_BASIC_CAPS (MLX5_ACCEL_IPSEC_CAP_DEVICE | MLX5_ACCEL_IPSEC_CAP_IPV6 | \
MLX5_ACCEL_IPSEC_CAP_LSO)
+
struct mlx5_ipsec_sa_ctx {
struct rhash_head hash;
u32 enc_key_id;
@@ -30,6 +32,10 @@ static u32 mlx5_ipsec_offload_device_caps(struct mlx5_core_dev *mdev)
if (!mlx5_is_ipsec_device(mdev))
return 0;
+ if (!MLX5_CAP_FLOWTABLE_NIC_TX(mdev, ipsec_encrypt) ||
+ !MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ipsec_decrypt))
+ return 0;
+
if (MLX5_CAP_IPSEC(mdev, ipsec_crypto_esp_aes_gcm_128_encrypt) &&
MLX5_CAP_IPSEC(mdev, ipsec_crypto_esp_aes_gcm_128_decrypt))
caps |= MLX5_ACCEL_IPSEC_CAP_ESP;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
index 6f4767324044e..6fdcd5e694764 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h
@@ -129,7 +129,11 @@ enum {
MLX5E_ACCEL_FS_TCP_FT_LEVEL,
#endif
#ifdef CONFIG_MLX5_EN_ARFS
- MLX5E_ARFS_FT_LEVEL
+ MLX5E_ARFS_FT_LEVEL,
+#endif
+#ifdef CONFIG_MLX5_EN_IPSEC
+ MLX5E_ACCEL_FS_ESP_FT_LEVEL = MLX5E_INNER_TTC_FT_LEVEL + 1,
+ MLX5E_ACCEL_FS_ESP_FT_ERR_LEVEL,
#endif
};
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
index 147191b7a98a6..d39989cddd905 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
@@ -40,7 +40,7 @@
#include "en.h"
#include "en_accel/ipsec.h"
#include "en_accel/ipsec_rxtx.h"
-
+#include "en_accel/ipsec_fs.h"
static struct mlx5e_ipsec_sa_entry *to_ipsec_sa_entry(struct xfrm_state *x)
{
@@ -284,6 +284,27 @@ static inline int mlx5e_xfrm_validate_state(struct xfrm_state *x)
return 0;
}
+static int mlx5e_xfrm_fs_add_rule(struct mlx5e_priv *priv,
+ struct mlx5e_ipsec_sa_entry *sa_entry)
+{
+ if (!mlx5_is_ipsec_device(priv->mdev))
+ return 0;
+
+ return mlx5e_accel_ipsec_fs_add_rule(priv, &sa_entry->xfrm->attrs,
+ sa_entry->ipsec_obj_id,
+ &sa_entry->ipsec_rule);
+}
+
+static void mlx5e_xfrm_fs_del_rule(struct mlx5e_priv *priv,
+ struct mlx5e_ipsec_sa_entry *sa_entry)
+{
+ if (!mlx5_is_ipsec_device(priv->mdev))
+ return;
+
+ mlx5e_accel_ipsec_fs_del_rule(priv, &sa_entry->xfrm->attrs,
+ &sa_entry->ipsec_rule);
+}
+
static int mlx5e_xfrm_add_state(struct xfrm_state *x)
{
struct mlx5e_ipsec_sa_entry *sa_entry = NULL;
@@ -331,10 +352,15 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x)
goto err_xfrm;
}
+ sa_entry->ipsec_obj_id = sa_handle;
+ err = mlx5e_xfrm_fs_add_rule(priv, sa_entry);
+ if (err)
+ goto err_hw_ctx;
+
if (x->xso.flags & XFRM_OFFLOAD_INBOUND) {
err = mlx5e_ipsec_sadb_rx_add(sa_entry, sa_handle);
if (err)
- goto err_hw_ctx;
+ goto err_add_rule;
} else {
sa_entry->set_iv_op = (x->props.flags & XFRM_STATE_ESN) ?
mlx5e_ipsec_set_iv_esn : mlx5e_ipsec_set_iv;
@@ -343,6 +369,8 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x)
x->xso.offload_handle = (unsigned long)sa_entry;
goto out;
+err_add_rule:
+ mlx5e_xfrm_fs_del_rule(priv, sa_entry);
err_hw_ctx:
mlx5_accel_esp_free_hw_context(priv->mdev, sa_entry->hw_context);
err_xfrm:
@@ -368,12 +396,14 @@ static void mlx5e_xfrm_del_state(struct xfrm_state *x)
static void mlx5e_xfrm_free_state(struct xfrm_state *x)
{
struct mlx5e_ipsec_sa_entry *sa_entry = to_ipsec_sa_entry(x);
+ struct mlx5e_priv *priv = netdev_priv(x->xso.dev);
if (!sa_entry)
return;
if (sa_entry->hw_context) {
flush_workqueue(sa_entry->ipsec->wq);
+ mlx5e_xfrm_fs_del_rule(priv, sa_entry);
mlx5_accel_esp_free_hw_context(sa_entry->xfrm->mdev, sa_entry->hw_context);
mlx5_accel_esp_destroy_xfrm(sa_entry->xfrm);
}
@@ -407,6 +437,8 @@ int mlx5e_ipsec_init(struct mlx5e_priv *priv)
kfree(ipsec);
return -ENOMEM;
}
+
+ mlx5e_accel_ipsec_fs_init(priv);
netdev_dbg(priv->netdev, "IPSec attached to netdevice\n");
return 0;
}
@@ -418,6 +450,7 @@ void mlx5e_ipsec_cleanup(struct mlx5e_priv *priv)
if (!ipsec)
return;
+ mlx5e_accel_ipsec_fs_cleanup(priv);
destroy_workqueue(ipsec->wq);
ida_destroy(&ipsec->halloc);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h
index c85151a1e008b..0fc8b4d4f4a34 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h
@@ -75,6 +75,8 @@ struct mlx5e_ipsec_stats {
u64 ipsec_cmd_drop;
};
+struct mlx5e_accel_fs_esp;
+
struct mlx5e_ipsec {
struct mlx5e_priv *en_priv;
DECLARE_HASHTABLE(sadb_rx, MLX5E_IPSEC_SADB_RX_BITS);
@@ -84,6 +86,7 @@ struct mlx5e_ipsec {
struct mlx5e_ipsec_sw_stats sw_stats;
struct mlx5e_ipsec_stats stats;
struct workqueue_struct *wq;
+ struct mlx5e_accel_fs_esp *rx_fs;
};
struct mlx5e_ipsec_esn_state {
@@ -92,6 +95,11 @@ struct mlx5e_ipsec_esn_state {
u8 overlap: 1;
};
+struct mlx5e_ipsec_rule {
+ struct mlx5_flow_handle *rule;
+ struct mlx5_modify_hdr *set_modify_hdr;
+};
+
struct mlx5e_ipsec_sa_entry {
struct hlist_node hlist; /* Item in SADB_RX hashtable */
struct mlx5e_ipsec_esn_state esn_state;
@@ -102,6 +110,8 @@ struct mlx5e_ipsec_sa_entry {
void *hw_context;
void (*set_iv_op)(struct sk_buff *skb, struct xfrm_state *x,
struct xfrm_offload *xo);
+ u32 ipsec_obj_id;
+ struct mlx5e_ipsec_rule ipsec_rule;
};
void mlx5e_ipsec_build_inverse_table(void);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
new file mode 100644
index 0000000000000..429428bbc903c
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c
@@ -0,0 +1,544 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */
+
+#include <linux/netdevice.h>
+#include "accel/ipsec_offload.h"
+#include "ipsec_fs.h"
+#include "fs_core.h"
+
+#define NUM_IPSEC_FTE BIT(15)
+
+enum accel_fs_esp_type {
+ ACCEL_FS_ESP4,
+ ACCEL_FS_ESP6,
+ ACCEL_FS_ESP_NUM_TYPES,
+};
+
+struct mlx5e_ipsec_rx_err {
+ struct mlx5_flow_table *ft;
+ struct mlx5_flow_handle *rule;
+ struct mlx5_modify_hdr *copy_modify_hdr;
+};
+
+struct mlx5e_accel_fs_esp_prot {
+ struct mlx5_flow_table *ft;
+ struct mlx5_flow_group *miss_group;
+ struct mlx5_flow_handle *miss_rule;
+ struct mlx5_flow_destination default_dest;
+ struct mlx5e_ipsec_rx_err rx_err;
+ u32 refcnt;
+ struct mutex prot_mutex; /* protect ESP4/ESP6 protocol */
+};
+
+struct mlx5e_accel_fs_esp {
+ struct mlx5e_accel_fs_esp_prot fs_prot[ACCEL_FS_ESP_NUM_TYPES];
+};
+
+/* IPsec RX flow steering */
+static enum mlx5e_traffic_types fs_esp2tt(enum accel_fs_esp_type i)
+{
+ if (i == ACCEL_FS_ESP4)
+ return MLX5E_TT_IPV4_IPSEC_ESP;
+ return MLX5E_TT_IPV6_IPSEC_ESP;
+}
+
+static int rx_err_add_rule(struct mlx5e_priv *priv,
+ struct mlx5e_accel_fs_esp_prot *fs_prot,
+ struct mlx5e_ipsec_rx_err *rx_err)
+{
+ u8 action[MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)] = {};
+ struct mlx5_core_dev *mdev = priv->mdev;
+ struct mlx5_flow_act flow_act = {};
+ struct mlx5_modify_hdr *modify_hdr;
+ struct mlx5_flow_handle *fte;
+ struct mlx5_flow_spec *spec;
+ int err = 0;
+
+ spec = kzalloc(sizeof(*spec), GFP_KERNEL);
+ if (!spec)
+ return -ENOMEM;
+
+ /* Action to copy 7 bit ipsec_syndrome to regB[0:6] */
+ MLX5_SET(copy_action_in, action, action_type, MLX5_ACTION_TYPE_COPY);
+ MLX5_SET(copy_action_in, action, src_field, MLX5_ACTION_IN_FIELD_IPSEC_SYNDROME);
+ MLX5_SET(copy_action_in, action, src_offset, 0);
+ MLX5_SET(copy_action_in, action, length, 7);
+ MLX5_SET(copy_action_in, action, dst_field, MLX5_ACTION_IN_FIELD_METADATA_REG_B);
+ MLX5_SET(copy_action_in, action, dst_offset, 0);
+
+ modify_hdr = mlx5_modify_header_alloc(mdev, MLX5_FLOW_NAMESPACE_KERNEL,
+ 1, action);
+
+ if (IS_ERR(modify_hdr)) {
+ err = PTR_ERR(modify_hdr);
+ netdev_err(priv->netdev,
+ "fail to alloc ipsec copy modify_header_id err=%d\n", err);
+ goto out_spec;
+ }
+
+ /* create fte */
+ flow_act.action = MLX5_FLOW_CONTEXT_ACTION_MOD_HDR |
+ MLX5_FLOW_CONTEXT_ACTION_FWD_DEST;
+ flow_act.modify_hdr = modify_hdr;
+ fte = mlx5_add_flow_rules(rx_err->ft, spec, &flow_act,
+ &fs_prot->default_dest, 1);
+ if (IS_ERR(fte)) {
+ err = PTR_ERR(fte);
+ netdev_err(priv->netdev, "fail to add ipsec rx err copy rule err=%d\n", err);
+ goto out;
+ }
+
+ rx_err->rule = fte;
+ rx_err->copy_modify_hdr = modify_hdr;
+
+out:
+ if (err)
+ mlx5_modify_header_dealloc(mdev, modify_hdr);
+out_spec:
+ kfree(spec);
+ return err;
+}
+
+static void rx_err_del_rule(struct mlx5e_priv *priv,
+ struct mlx5e_ipsec_rx_err *rx_err)
+{
+ if (rx_err->rule) {
+ mlx5_del_flow_rules(rx_err->rule);
+ rx_err->rule = NULL;
+ }
+
+ if (rx_err->copy_modify_hdr) {
+ mlx5_modify_header_dealloc(priv->mdev, rx_err->copy_modify_hdr);
+ rx_err->copy_modify_hdr = NULL;
+ }
+}
+
+static void rx_err_destroy_ft(struct mlx5e_priv *priv, struct mlx5e_ipsec_rx_err *rx_err)
+{
+ rx_err_del_rule(priv, rx_err);
+
+ if (rx_err->ft) {
+ mlx5_destroy_flow_table(rx_err->ft);
+ rx_err->ft = NULL;
+ }
+}
+
+static int rx_err_create_ft(struct mlx5e_priv *priv,
+ struct mlx5e_accel_fs_esp_prot *fs_prot,
+ struct mlx5e_ipsec_rx_err *rx_err)
+{
+ struct mlx5_flow_table_attr ft_attr = {};
+ struct mlx5_flow_table *ft;
+ int err;
+
+ ft_attr.max_fte = 1;
+ ft_attr.autogroup.max_num_groups = 1;
+ ft_attr.level = MLX5E_ACCEL_FS_ESP_FT_ERR_LEVEL;
+ ft_attr.prio = MLX5E_NIC_PRIO;
+ ft = mlx5_create_auto_grouped_flow_table(priv->fs.ns, &ft_attr);
+ if (IS_ERR(ft)) {
+ err = PTR_ERR(ft);
+ netdev_err(priv->netdev, "fail to create ipsec rx inline ft err=%d\n", err);
+ return err;
+ }
+
+ rx_err->ft = ft;
+ err = rx_err_add_rule(priv, fs_prot, rx_err);
+ if (err)
+ goto out_err;
+
+ return 0;
+
+out_err:
+ mlx5_destroy_flow_table(ft);
+ rx_err->ft = NULL;
+ return err;
+}
+
+static void rx_fs_destroy(struct mlx5e_accel_fs_esp_prot *fs_prot)
+{
+ if (fs_prot->miss_rule) {
+ mlx5_del_flow_rules(fs_prot->miss_rule);
+ fs_prot->miss_rule = NULL;
+ }
+
+ if (fs_prot->miss_group) {
+ mlx5_destroy_flow_group(fs_prot->miss_group);
+ fs_prot->miss_group = NULL;
+ }
+
+ if (fs_prot->ft) {
+ mlx5_destroy_flow_table(fs_prot->ft);
+ fs_prot->ft = NULL;
+ }
+}
+
+static int rx_fs_create(struct mlx5e_priv *priv,
+ struct mlx5e_accel_fs_esp_prot *fs_prot)
+{
+ int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in);
+ struct mlx5_flow_table_attr ft_attr = {};
+ struct mlx5_flow_group *miss_group;
+ struct mlx5_flow_handle *miss_rule;
+ MLX5_DECLARE_FLOW_ACT(flow_act);
+ struct mlx5_flow_spec *spec;
+ struct mlx5_flow_table *ft;
+ u32 *flow_group_in;
+ int err = 0;
+
+ flow_group_in = kvzalloc(inlen, GFP_KERNEL);
+ spec = kvzalloc(sizeof(*spec), GFP_KERNEL);
+ if (!flow_group_in || !spec) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ /* Create FT */
+ ft_attr.max_fte = NUM_IPSEC_FTE;
+ ft_attr.level = MLX5E_ACCEL_FS_ESP_FT_LEVEL;
+ ft_attr.prio = MLX5E_NIC_PRIO;
+ ft_attr.autogroup.num_reserved_entries = 1;
+ ft_attr.autogroup.max_num_groups = 1;
+ ft = mlx5_create_auto_grouped_flow_table(priv->fs.ns, &ft_attr);
+ if (IS_ERR(ft)) {
+ err = PTR_ERR(ft);
+ netdev_err(priv->netdev, "fail to create ipsec rx ft err=%d\n", err);
+ goto out;
+ }
+ fs_prot->ft = ft;
+
+ /* Create miss_group */
+ MLX5_SET(create_flow_group_in, flow_group_in, start_flow_index, ft->max_fte - 1);
+ MLX5_SET(create_flow_group_in, flow_group_in, end_flow_index, ft->max_fte - 1);
+ miss_group = mlx5_create_flow_group(ft, flow_group_in);
+ if (IS_ERR(miss_group)) {
+ err = PTR_ERR(miss_group);
+ netdev_err(priv->netdev, "fail to create ipsec rx miss_group err=%d\n", err);
+ goto out;
+ }
+ fs_prot->miss_group = miss_group;
+
+ /* Create miss rule */
+ miss_rule = mlx5_add_flow_rules(ft, spec, &flow_act, &fs_prot->default_dest, 1);
+ if (IS_ERR(miss_rule)) {
+ err = PTR_ERR(miss_rule);
+ netdev_err(priv->netdev, "fail to create ipsec rx miss_rule err=%d\n", err);
+ goto out;
+ }
+ fs_prot->miss_rule = miss_rule;
+
+out:
+ kfree(flow_group_in);
+ kfree(spec);
+ return err;
+}
+
+static int rx_destroy(struct mlx5e_priv *priv, enum accel_fs_esp_type type)
+{
+ struct mlx5e_accel_fs_esp_prot *fs_prot;
+ struct mlx5e_accel_fs_esp *accel_esp;
+
+ accel_esp = priv->ipsec->rx_fs;
+
+ /* The netdev unreg already happened, so all offloaded rule are already removed */
+ fs_prot = &accel_esp->fs_prot[type];
+
+ rx_fs_destroy(fs_prot);
+
+ rx_err_destroy_ft(priv, &fs_prot->rx_err);
+
+ return 0;
+}
+
+static int rx_create(struct mlx5e_priv *priv, enum accel_fs_esp_type type)
+{
+ struct mlx5e_accel_fs_esp_prot *fs_prot;
+ struct mlx5e_accel_fs_esp *accel_esp;
+ int err;
+
+ accel_esp = priv->ipsec->rx_fs;
+ fs_prot = &accel_esp->fs_prot[type];
+
+ fs_prot->default_dest = mlx5e_ttc_get_default_dest(priv, fs_esp2tt(type));
+
+ err = rx_err_create_ft(priv, fs_prot, &fs_prot->rx_err);
+ if (err)
+ return err;
+
+ err = rx_fs_create(priv, fs_prot);
+ if (err)
+ rx_destroy(priv, type);
+
+ return err;
+}
+
+static int rx_ft_get(struct mlx5e_priv *priv, enum accel_fs_esp_type type)
+{
+ struct mlx5e_accel_fs_esp_prot *fs_prot;
+ struct mlx5_flow_destination dest = {};
+ struct mlx5e_accel_fs_esp *accel_esp;
+ int err = 0;
+
+ accel_esp = priv->ipsec->rx_fs;
+ fs_prot = &accel_esp->fs_prot[type];
+ mutex_lock(&fs_prot->prot_mutex);
+ if (fs_prot->refcnt++)
+ goto out;
+
+ /* create FT */
+ err = rx_create(priv, type);
+ if (err) {
+ fs_prot->refcnt--;
+ goto out;
+ }
+
+ /* connect */
+ dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE;
+ dest.ft = fs_prot->ft;
+ mlx5e_ttc_fwd_dest(priv, fs_esp2tt(type), &dest);
+
+out:
+ mutex_unlock(&fs_prot->prot_mutex);
+ return err;
+}
+
+static void rx_ft_put(struct mlx5e_priv *priv, enum accel_fs_esp_type type)
+{
+ struct mlx5e_accel_fs_esp_prot *fs_prot;
+ struct mlx5e_accel_fs_esp *accel_esp;
+
+ accel_esp = priv->ipsec->rx_fs;
+ fs_prot = &accel_esp->fs_prot[type];
+ mutex_lock(&fs_prot->prot_mutex);
+ if (--fs_prot->refcnt)
+ goto out;
+
+ /* disconnect */
+ mlx5e_ttc_fwd_default_dest(priv, fs_esp2tt(type));
+
+ /* remove FT */
+ rx_destroy(priv, type);
+
+out:
+ mutex_unlock(&fs_prot->prot_mutex);
+}
+
+static void setup_fte_common(struct mlx5_accel_esp_xfrm_attrs *attrs,
+ u32 ipsec_obj_id,
+ struct mlx5_flow_spec *spec,
+ struct mlx5_flow_act *flow_act)
+{
+ u8 ip_version = attrs->is_ipv6 ? 6 : 4;
+
+ spec->match_criteria_enable = MLX5_MATCH_OUTER_HEADERS | MLX5_MATCH_MISC_PARAMETERS;
+
+ /* ip_version */
+ MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.ip_version);
+ MLX5_SET(fte_match_param, spec->match_value, outer_headers.ip_version, ip_version);
+
+ /* Non fragmented */
+ MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.frag);
+ MLX5_SET(fte_match_param, spec->match_value, outer_headers.frag, 0);
+
+ /* ESP header */
+ MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, outer_headers.ip_protocol);
+ MLX5_SET(fte_match_param, spec->match_value, outer_headers.ip_protocol, IPPROTO_ESP);
+
+ /* SPI number */
+ MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria, misc_parameters.outer_esp_spi);
+ MLX5_SET(fte_match_param, spec->match_value, misc_parameters.outer_esp_spi,
+ be32_to_cpu(attrs->spi));
+
+ if (ip_version == 4) {
+ memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_value,
+ outer_headers.src_ipv4_src_ipv6.ipv4_layout.ipv4),
+ &attrs->saddr.a4, 4);
+ memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_value,
+ outer_headers.dst_ipv4_dst_ipv6.ipv4_layout.ipv4),
+ &attrs->daddr.a4, 4);
+ MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria,
+ outer_headers.src_ipv4_src_ipv6.ipv4_layout.ipv4);
+ MLX5_SET_TO_ONES(fte_match_param, spec->match_criteria,
+ outer_headers.dst_ipv4_dst_ipv6.ipv4_layout.ipv4);
+ } else {
+ memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_value,
+ outer_headers.src_ipv4_src_ipv6.ipv6_layout.ipv6),
+ &attrs->saddr.a6, 16);
+ memcpy(MLX5_ADDR_OF(fte_match_param, spec->match_value,
+ outer_headers.dst_ipv4_dst_ipv6.ipv6_layout.ipv6),
+ &attrs->daddr.a6, 16);
+ memset(MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
+ outer_headers.src_ipv4_src_ipv6.ipv6_layout.ipv6),
+ 0xff, 16);
+ memset(MLX5_ADDR_OF(fte_match_param, spec->match_criteria,
+ outer_headers.dst_ipv4_dst_ipv6.ipv6_layout.ipv6),
+ 0xff, 16);
+ }
+
+ flow_act->ipsec_obj_id = ipsec_obj_id;
+ flow_act->flags |= FLOW_ACT_NO_APPEND;
+}
+
+static int rx_add_rule(struct mlx5e_priv *priv,
+ struct mlx5_accel_esp_xfrm_attrs *attrs,
+ u32 ipsec_obj_id,
+ struct mlx5e_ipsec_rule *ipsec_rule)
+{
+ u8 action[MLX5_UN_SZ_BYTES(set_add_copy_action_in_auto)] = {};
+ struct mlx5_modify_hdr *modify_hdr = NULL;
+ struct mlx5e_accel_fs_esp_prot *fs_prot;
+ struct mlx5_flow_destination dest = {};
+ struct mlx5e_accel_fs_esp *accel_esp;
+ struct mlx5_flow_act flow_act = {};
+ struct mlx5_flow_handle *rule;
+ enum accel_fs_esp_type type;
+ struct mlx5_flow_spec *spec;
+ int err = 0;
+
+ accel_esp = priv->ipsec->rx_fs;
+ type = attrs->is_ipv6 ? ACCEL_FS_ESP6 : ACCEL_FS_ESP4;
+ fs_prot = &accel_esp->fs_prot[type];
+
+ err = rx_ft_get(priv, type);
+ if (err)
+ return err;
+
+ spec = kvzalloc(sizeof(*spec), GFP_KERNEL);
+ if (!spec) {
+ err = -ENOMEM;
+ goto out_err;
+ }
+
+ setup_fte_common(attrs, ipsec_obj_id, spec, &flow_act);
+
+ /* Set 1 bit ipsec marker */
+ /* Set 24 bit ipsec_obj_id */
+ MLX5_SET(set_action_in, action, action_type, MLX5_ACTION_TYPE_SET);
+ MLX5_SET(set_action_in, action, field, MLX5_ACTION_IN_FIELD_METADATA_REG_B);
+ MLX5_SET(set_action_in, action, data, (ipsec_obj_id << 1) | 0x1);
+ MLX5_SET(set_action_in, action, offset, 7);
+ MLX5_SET(set_action_in, action, length, 25);
+
+ modify_hdr = mlx5_modify_header_alloc(priv->mdev, MLX5_FLOW_NAMESPACE_KERNEL,
+ 1, action);
+ if (IS_ERR(modify_hdr)) {
+ err = PTR_ERR(modify_hdr);
+ netdev_err(priv->netdev,
+ "fail to alloc ipsec set modify_header_id err=%d\n", err);
+ modify_hdr = NULL;
+ goto out_err;
+ }
+
+ flow_act.action = MLX5_FLOW_CONTEXT_ACTION_FWD_DEST |
+ MLX5_FLOW_CONTEXT_ACTION_IPSEC_DECRYPT |
+ MLX5_FLOW_CONTEXT_ACTION_MOD_HDR;
+ dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE;
+ flow_act.modify_hdr = modify_hdr;
+ dest.ft = fs_prot->rx_err.ft;
+ rule = mlx5_add_flow_rules(fs_prot->ft, spec, &flow_act, &dest, 1);
+ if (IS_ERR(rule)) {
+ err = PTR_ERR(rule);
+ netdev_err(priv->netdev, "fail to add ipsec rule attrs->action=0x%x, err=%d\n",
+ attrs->action, err);
+ goto out_err;
+ }
+
+ ipsec_rule->rule = rule;
+ ipsec_rule->set_modify_hdr = modify_hdr;
+ goto out;
+
+out_err:
+ if (modify_hdr)
+ mlx5_modify_header_dealloc(priv->mdev, modify_hdr);
+ rx_ft_put(priv, type);
+
+out:
+ kvfree(spec);
+ return err;
+}
+
+static void rx_del_rule(struct mlx5e_priv *priv,
+ struct mlx5_accel_esp_xfrm_attrs *attrs,
+ struct mlx5e_ipsec_rule *ipsec_rule)
+{
+ mlx5_del_flow_rules(ipsec_rule->rule);
+ ipsec_rule->rule = NULL;
+
+ mlx5_modify_header_dealloc(priv->mdev, ipsec_rule->set_modify_hdr);
+ ipsec_rule->set_modify_hdr = NULL;
+
+ rx_ft_put(priv, attrs->is_ipv6 ? ACCEL_FS_ESP6 : ACCEL_FS_ESP4);
+}
+
+int mlx5e_accel_ipsec_fs_add_rule(struct mlx5e_priv *priv,
+ struct mlx5_accel_esp_xfrm_attrs *attrs,
+ u32 ipsec_obj_id,
+ struct mlx5e_ipsec_rule *ipsec_rule)
+{
+ if (!priv->ipsec->rx_fs || attrs->action != MLX5_ACCEL_ESP_ACTION_DECRYPT)
+ return -EOPNOTSUPP;
+
+ return rx_add_rule(priv, attrs, ipsec_obj_id, ipsec_rule);
+}
+
+void mlx5e_accel_ipsec_fs_del_rule(struct mlx5e_priv *priv,
+ struct mlx5_accel_esp_xfrm_attrs *attrs,
+ struct mlx5e_ipsec_rule *ipsec_rule)
+{
+ if (!priv->ipsec->rx_fs)
+ return;
+
+ rx_del_rule(priv, attrs, ipsec_rule);
+}
+
+static void fs_cleanup_rx(struct mlx5e_priv *priv)
+{
+ struct mlx5e_accel_fs_esp_prot *fs_prot;
+ struct mlx5e_accel_fs_esp *accel_esp;
+ enum accel_fs_esp_type i;
+
+ accel_esp = priv->ipsec->rx_fs;
+ for (i = 0; i < ACCEL_FS_ESP_NUM_TYPES; i++) {
+ fs_prot = &accel_esp->fs_prot[i];
+ mutex_destroy(&fs_prot->prot_mutex);
+ WARN_ON(fs_prot->refcnt);
+ }
+ kfree(priv->ipsec->rx_fs);
+ priv->ipsec->rx_fs = NULL;
+}
+
+static int fs_init_rx(struct mlx5e_priv *priv)
+{
+ struct mlx5e_accel_fs_esp_prot *fs_prot;
+ struct mlx5e_accel_fs_esp *accel_esp;
+ enum accel_fs_esp_type i;
+
+ priv->ipsec->rx_fs =
+ kzalloc(sizeof(struct mlx5e_accel_fs_esp), GFP_KERNEL);
+ if (!priv->ipsec->rx_fs)
+ return -ENOMEM;
+
+ accel_esp = priv->ipsec->rx_fs;
+ for (i = 0; i < ACCEL_FS_ESP_NUM_TYPES; i++) {
+ fs_prot = &accel_esp->fs_prot[i];
+ mutex_init(&fs_prot->prot_mutex);
+ }
+
+ return 0;
+}
+
+void mlx5e_accel_ipsec_fs_cleanup(struct mlx5e_priv *priv)
+{
+ if (!priv->ipsec->rx_fs)
+ return;
+
+ fs_cleanup_rx(priv);
+}
+
+int mlx5e_accel_ipsec_fs_init(struct mlx5e_priv *priv)
+{
+ if (!mlx5_is_ipsec_device(priv->mdev) || !priv->ipsec)
+ return -EOPNOTSUPP;
+
+ return fs_init_rx(priv);
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.h
new file mode 100644
index 0000000000000..3389b3bb3ef80
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */
+
+#ifndef __MLX5_IPSEC_STEERING_H__
+#define __MLX5_IPSEC_STEERING_H__
+
+#include "en.h"
+#include "ipsec.h"
+#include "accel/ipsec_offload.h"
+#include "en/fs.h"
+
+#ifdef CONFIG_MLX5_EN_IPSEC
+void mlx5e_accel_ipsec_fs_cleanup(struct mlx5e_priv *priv);
+int mlx5e_accel_ipsec_fs_init(struct mlx5e_priv *priv);
+int mlx5e_accel_ipsec_fs_add_rule(struct mlx5e_priv *priv,
+ struct mlx5_accel_esp_xfrm_attrs *attrs,
+ u32 ipsec_obj_id,
+ struct mlx5e_ipsec_rule *ipsec_rule);
+void mlx5e_accel_ipsec_fs_del_rule(struct mlx5e_priv *priv,
+ struct mlx5_accel_esp_xfrm_attrs *attrs,
+ struct mlx5e_ipsec_rule *ipsec_rule);
+#else
+static inline void mlx5e_accel_ipsec_fs_cleanup(struct mlx5e_priv *priv) {}
+static inline int mlx5e_accel_ipsec_fs_init(struct mlx5e_priv *priv) { return 0; }
+#endif
+#endif /* __MLX5_IPSEC_STEERING_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
index 465a1076a4777..fee169732de74 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_cmd.c
@@ -459,6 +459,8 @@ static int mlx5_cmd_set_fte(struct mlx5_core_dev *dev,
MLX5_SET(flow_context, in_flow_context, modify_header_id,
fte->action.modify_hdr->id);
+ MLX5_SET(flow_context, in_flow_context, ipsec_obj_id, fte->action.ipsec_obj_id);
+
vlan = MLX5_ADDR_OF(flow_context, in_flow_context, push_vlan);
MLX5_SET(vlan, vlan, ethtype, fte->action.vlan[0].ethtype);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
index 785b2960d6b59..6904ad96af48b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c
@@ -105,7 +105,7 @@
#define ETHTOOL_PRIO_NUM_LEVELS 1
#define ETHTOOL_NUM_PRIOS 11
#define ETHTOOL_MIN_LEVEL (KERNEL_MIN_LEVEL + ETHTOOL_NUM_PRIOS)
-/* Vlan, mac, ttc, inner ttc, {aRFS/accel} */
+/* Vlan, mac, ttc, inner ttc, {aRFS/accel and esp/esp_err} */
#define KERNEL_NIC_PRIO_NUM_LEVELS 6
#define KERNEL_NIC_NUM_PRIOS 1
/* One more level for tc */
--
2.26.2
^ permalink raw reply related
* [net-next V2 12/15] net/mlx5e: XDP, Avoid indirect call in TX flow
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Tariq Toukan, Maxim Mikityanskiy, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Tariq Toukan <tariqt@mellanox.com>
Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call
when/if CONFIG_RETPOLINE=y.
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Reviewed-by: Maxim Mikityanskiy <maximmi@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/en/xdp.c | 27 ++++++++++---------
.../net/ethernet/mellanox/mlx5/core/en/xdp.h | 13 +++++++++
.../ethernet/mellanox/mlx5/core/en/xsk/tx.c | 11 ++++++--
3 files changed, 37 insertions(+), 14 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
index c9d308e919655..e0c1b010d41ab 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c
@@ -34,6 +34,7 @@
#include <net/xdp_sock_drv.h>
#include "en/xdp.h"
#include "en/params.h"
+#include <linux/indirect_call_wrapper.h>
int mlx5e_xdp_max_mtu(struct mlx5e_params *params, struct mlx5e_xsk_param *xsk)
{
@@ -114,7 +115,8 @@ mlx5e_xmit_xdp_buff(struct mlx5e_xdpsq *sq, struct mlx5e_rq *rq,
xdpi.page.di = *di;
}
- return sq->xmit_xdp_frame(sq, &xdptxd, &xdpi, 0);
+ return INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe,
+ mlx5e_xmit_xdp_frame, sq, &xdptxd, &xdpi, 0);
}
/* returns true if packet was consumed by xdp */
@@ -237,7 +239,7 @@ enum {
MLX5E_XDP_CHECK_START_MPWQE = 2,
};
-static int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq *sq)
+INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq *sq)
{
if (unlikely(!sq->mpwqe.wqe)) {
const u16 stop_room = mlx5e_stop_room_for_wqe(MLX5_SEND_WQE_MAX_WQEBBS);
@@ -256,10 +258,9 @@ static int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq *sq)
return MLX5E_XDP_CHECK_OK;
}
-static bool mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq,
- struct mlx5e_xdp_xmit_data *xdptxd,
- struct mlx5e_xdp_info *xdpi,
- int check_result)
+INDIRECT_CALLABLE_SCOPE bool
+mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xdp_xmit_data *xdptxd,
+ struct mlx5e_xdp_info *xdpi, int check_result)
{
struct mlx5e_xdp_mpwqe *session = &sq->mpwqe;
struct mlx5e_xdpsq_stats *stats = sq->stats;
@@ -293,7 +294,7 @@ static bool mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq,
return true;
}
-static int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq)
+INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq)
{
if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, 1))) {
/* SQ is full, ring doorbell */
@@ -305,10 +306,9 @@ static int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq)
return MLX5E_XDP_CHECK_OK;
}
-static bool mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq,
- struct mlx5e_xdp_xmit_data *xdptxd,
- struct mlx5e_xdp_info *xdpi,
- int check_result)
+INDIRECT_CALLABLE_SCOPE bool
+mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xdp_xmit_data *xdptxd,
+ struct mlx5e_xdp_info *xdpi, int check_result)
{
struct mlx5_wq_cyc *wq = &sq->wq;
u16 pi = mlx5_wq_cyc_ctr2ix(wq, sq->pc);
@@ -506,6 +506,7 @@ int mlx5e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames,
struct xdp_frame *xdpf = frames[i];
struct mlx5e_xdp_xmit_data xdptxd;
struct mlx5e_xdp_info xdpi;
+ bool ret;
xdptxd.data = xdpf->data;
xdptxd.len = xdpf->len;
@@ -522,7 +523,9 @@ int mlx5e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames,
xdpi.frame.xdpf = xdpf;
xdpi.frame.dma_addr = xdptxd.dma_addr;
- if (unlikely(!sq->xmit_xdp_frame(sq, &xdptxd, &xdpi, 0))) {
+ ret = INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe,
+ mlx5e_xmit_xdp_frame, sq, &xdptxd, &xdpi, 0);
+ if (unlikely(!ret)) {
dma_unmap_single(sq->pdev, xdptxd.dma_addr,
xdptxd.len, DMA_TO_DEVICE);
xdp_return_frame_rx_napi(xdpf);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h
index ca48c293151be..e806c13d491f0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h
@@ -32,6 +32,8 @@
#ifndef __MLX5_EN_XDP_H__
#define __MLX5_EN_XDP_H__
+#include <linux/indirect_call_wrapper.h>
+
#include "en.h"
#include "en/txrx.h"
@@ -70,6 +72,17 @@ void mlx5e_xdp_rx_poll_complete(struct mlx5e_rq *rq);
int mlx5e_xdp_xmit(struct net_device *dev, int n, struct xdp_frame **frames,
u32 flags);
+INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq,
+ struct mlx5e_xdp_xmit_data *xdptxd,
+ struct mlx5e_xdp_info *xdpi,
+ int check_result));
+INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq,
+ struct mlx5e_xdp_xmit_data *xdptxd,
+ struct mlx5e_xdp_info *xdpi,
+ int check_result));
+INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq *sq));
+INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq));
+
static inline void mlx5e_xdp_tx_enable(struct mlx5e_priv *priv)
{
set_bit(MLX5E_STATE_XDP_TX_ENABLED, &priv->state);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
index e0b3c61af93ea..0dfbc96e952ab 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c
@@ -6,6 +6,7 @@
#include "en/xdp.h"
#include "en/params.h"
#include <net/xdp_sock_drv.h>
+#include <linux/indirect_call_wrapper.h>
int mlx5e_xsk_wakeup(struct net_device *dev, u32 qid, u32 flags)
{
@@ -75,8 +76,12 @@ bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget)
xdpi.mode = MLX5E_XDP_XMIT_MODE_XSK;
for (; budget; budget--) {
- int check_result = sq->xmit_xdp_frame_check(sq);
+ int check_result = INDIRECT_CALL_2(sq->xmit_xdp_frame_check,
+ mlx5e_xmit_xdp_frame_check_mpwqe,
+ mlx5e_xmit_xdp_frame_check,
+ sq);
struct xdp_desc desc;
+ bool ret;
if (unlikely(check_result < 0)) {
work_done = false;
@@ -98,7 +103,9 @@ bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget)
xsk_buff_raw_dma_sync_for_device(umem, xdptxd.dma_addr, xdptxd.len);
- if (unlikely(!sq->xmit_xdp_frame(sq, &xdptxd, &xdpi, check_result))) {
+ ret = INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe,
+ mlx5e_xmit_xdp_frame, sq, &xdptxd, &xdpi, check_result);
+ if (unlikely(!ret)) {
if (sq->mpwqe.wqe)
mlx5e_xdp_mpwqe_complete(sq);
--
2.26.2
^ permalink raw reply related
* [net-next V2 14/15] net/mlx5e: Do not request completion on every single UMR WQE
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Tariq Toukan, Maxim Mikityanskiy, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Tariq Toukan <tariqt@mellanox.com>
UMR WQEs are posted in bulks, and HW is notified once per a bulk.
Reduce the number of completions by requesting such only for
the last WQE of the bulk.
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Reviewed-by: Maxim Mikityanskiy <maximmi@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 88ea1908cb14a..9d5d8b28bcd81 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -232,7 +232,6 @@ static inline void mlx5e_build_umr_wqe(struct mlx5e_rq *rq,
cseg->qpn_ds = cpu_to_be32((sq->sqn << MLX5_WQE_CTRL_QPN_SHIFT) |
ds_cnt);
- cseg->fm_ce_se = MLX5_WQE_CTRL_CQ_UPDATE;
cseg->umr_mkey = rq->mkey_be;
ucseg->flags = MLX5_UMR_TRANSLATION_OFFSET_EN | MLX5_UMR_INLINE;
--
2.26.2
^ permalink raw reply related
* Re: [PATCH] ASoC: Intel: bytcht_es8316: Add missed put_device()
From: Mark Brown @ 2020-07-16 23:57 UTC (permalink / raw)
To: pierre-louis.bossart, kuninori.morimoto.gx, hdegoede, paul,
yang.jie, liam.r.girdwood, cezary.rojewski, Jing Xiangfeng, tiwai,
perex
Cc: alsa-devel, linux-kernel
In-Reply-To: <20200714080918.148196-1-jingxiangfeng@huawei.com>
On Tue, 14 Jul 2020 16:09:18 +0800, Jing Xiangfeng wrote:
> snd_byt_cht_es8316_mc_probe() misses to call put_device() in an error
> path. Add the missed function call to fix it.
Applied to
https://git.kernel.org/pub/scm/linux/kernel/git/broonie/sound.git for-next
Thanks!
[1/1] ASoC: Intel: bytcht_es8316: Add missed put_device()
commit: b3df80ab6d147d4738be242e1c91e5fdbb6b03ef
All being well this means that it will be integrated into the linux-next
tree (usually sometime in the next 24 hours) and sent to Linus during
the next merge window (or sooner if it is a bug fix), however if
problems are discovered then the patch may be dropped or reverted.
You may get further e-mails resulting from automated or manual testing
and review of the tree, please engage with people reporting problems and
send followup patches addressing any issues that are reported if needed.
If any updates are required or you are submitting further changes they
should be sent as incremental updates against current git, existing
patches will not be replaced.
Please add any relevant lists and maintainers to the CCs when replying
to this mail.
Thanks,
Mark
^ permalink raw reply
* [net-next V2 13/15] net/mlx5e: RX, Avoid indirect call in representor CQE handling
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Tariq Toukan, Maxim Mikityanskiy, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Tariq Toukan <tariqt@mellanox.com>
Use INDIRECT_CALL_2() helper to avoid the cost of the indirect call
when/if CONFIG_RETPOLINE=y.
Signed-off-by: Tariq Toukan <tariqt@mellanox.com>
Reviewed-by: Maxim Mikityanskiy <maximmi@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 8b24e44f860a8..74860f3827b1a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -1266,7 +1266,10 @@ void mlx5e_handle_rx_cqe_rep(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe)
goto free_wqe;
}
- skb = rq->wqe.skb_from_cqe(rq, cqe, wi, cqe_bcnt);
+ skb = INDIRECT_CALL_2(rq->wqe.skb_from_cqe,
+ mlx5e_skb_from_cqe_linear,
+ mlx5e_skb_from_cqe_nonlinear,
+ rq, cqe, wi, cqe_bcnt);
if (!skb) {
/* probably for XDP */
if (__test_and_clear_bit(MLX5E_RQ_FLAG_XDP_XMIT, rq->flags)) {
--
2.26.2
^ permalink raw reply related
* [net-next V2 10/15] net/mlx5e: IPsec: Add Connect-X IPsec Rx data path offload
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Raed Salem, Boris Pismenny, Tariq Toukan, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Raed Salem <raeds@mellanox.com>
On receive flow inspect received packets for IPsec offload indication
using the cqe, for IPsec offloaded packets propagate offload status
and stack handle to stack for further processing.
Supported statuses:
- Offload ok.
- Authentication failure.
- Bad trailer indication.
Connect-X IPsec does not use mlx5e_ipsec_handle_rx_cqe.
For RX only offload, we see the BW gain. Below is the iperf3
performance report on two server of 24 cores Intel(R) Xeon(R)
CPU E5-2620 v3 @ 2.40GHz with ConnectX6-DX.
We use one thread per IPsec tunnel.
---------------------------------------------------------------------
Mode | Num tunnel | BW | Send CPU util | Recv CPU util
| | (Gbps) | (Average %) | (Average %)
---------------------------------------------------------------------
Cryto offload | 1 | 4.6 | 4.2 | 14.5
---------------------------------------------------------------------
Cryto offload | 24 | 38 | 73 | 63
---------------------------------------------------------------------
Non-offload | 1 | 4 | 4 | 13
---------------------------------------------------------------------
Non-offload | 24 | 23 | 52 | 67
Signed-off-by: Raed Salem <raeds@mellanox.com>
Reviewed-by: Boris Pismenny <borisp@mellanox.com>
Reviewed-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../mellanox/mlx5/core/en_accel/ipsec_rxtx.c | 56 +++++++++++++++++++
.../mellanox/mlx5/core/en_accel/ipsec_rxtx.h | 22 +++++++-
.../net/ethernet/mellanox/mlx5/core/en_main.c | 4 +-
.../net/ethernet/mellanox/mlx5/core/en_rx.c | 10 +++-
4 files changed, 88 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.c
index 824b87ac8f9ee..93a8d68815ade 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.c
@@ -360,6 +360,62 @@ struct sk_buff *mlx5e_ipsec_handle_rx_skb(struct net_device *netdev,
return skb;
}
+enum {
+ MLX5E_IPSEC_OFFLOAD_RX_SYNDROME_DECRYPTED,
+ MLX5E_IPSEC_OFFLOAD_RX_SYNDROME_AUTH_FAILED,
+ MLX5E_IPSEC_OFFLOAD_RX_SYNDROME_BAD_TRAILER,
+};
+
+void mlx5e_ipsec_offload_handle_rx_skb(struct net_device *netdev,
+ struct sk_buff *skb,
+ struct mlx5_cqe64 *cqe)
+{
+ u32 ipsec_meta_data = be32_to_cpu(cqe->ft_metadata);
+ u8 ipsec_syndrome = ipsec_meta_data & 0xFF;
+ struct mlx5e_priv *priv;
+ struct xfrm_offload *xo;
+ struct xfrm_state *xs;
+ struct sec_path *sp;
+ u32 sa_handle;
+
+ sa_handle = MLX5_IPSEC_METADATA_HANDLE(ipsec_meta_data);
+ priv = netdev_priv(netdev);
+ sp = secpath_set(skb);
+ if (unlikely(!sp)) {
+ atomic64_inc(&priv->ipsec->sw_stats.ipsec_rx_drop_sp_alloc);
+ return;
+ }
+
+ xs = mlx5e_ipsec_sadb_rx_lookup(priv->ipsec, sa_handle);
+ if (unlikely(!xs)) {
+ atomic64_inc(&priv->ipsec->sw_stats.ipsec_rx_drop_sadb_miss);
+ return;
+ }
+
+ sp = skb_sec_path(skb);
+ sp->xvec[sp->len++] = xs;
+ sp->olen++;
+
+ xo = xfrm_offload(skb);
+ xo->flags = CRYPTO_DONE;
+
+ switch (ipsec_syndrome & MLX5_IPSEC_METADATA_SYNDROM_MASK) {
+ case MLX5E_IPSEC_OFFLOAD_RX_SYNDROME_DECRYPTED:
+ xo->status = CRYPTO_SUCCESS;
+ if (WARN_ON_ONCE(priv->ipsec->no_trailer))
+ xo->flags |= XFRM_ESP_NO_TRAILER;
+ break;
+ case MLX5E_IPSEC_OFFLOAD_RX_SYNDROME_AUTH_FAILED:
+ xo->status = CRYPTO_TUNNEL_ESP_AUTH_FAILED;
+ break;
+ case MLX5E_IPSEC_OFFLOAD_RX_SYNDROME_BAD_TRAILER:
+ xo->status = CRYPTO_INVALID_PACKET_SYNTAX;
+ break;
+ default:
+ atomic64_inc(&priv->ipsec->sw_stats.ipsec_rx_drop_syndrome);
+ }
+}
+
bool mlx5e_ipsec_feature_check(struct sk_buff *skb, struct net_device *netdev,
netdev_features_t features)
{
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h
index ba02643586a54..2a47673da5a4e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_rxtx.h
@@ -34,13 +34,17 @@
#ifndef __MLX5E_IPSEC_RXTX_H__
#define __MLX5E_IPSEC_RXTX_H__
-#ifdef CONFIG_MLX5_EN_IPSEC
-
#include <linux/skbuff.h>
#include <net/xfrm.h>
#include "en.h"
#include "en/txrx.h"
+#define MLX5_IPSEC_METADATA_MARKER_MASK (0x80)
+#define MLX5_IPSEC_METADATA_SYNDROM_MASK (0x7F)
+#define MLX5_IPSEC_METADATA_HANDLE(metadata) (((metadata) >> 8) & 0xFF)
+
+#ifdef CONFIG_MLX5_EN_IPSEC
+
struct sk_buff *mlx5e_ipsec_handle_rx_skb(struct net_device *netdev,
struct sk_buff *skb, u32 *cqe_bcnt);
void mlx5e_ipsec_handle_rx_cqe(struct mlx5e_rq *rq, struct mlx5_cqe64 *cqe);
@@ -55,7 +59,21 @@ void mlx5e_ipsec_set_iv(struct sk_buff *skb, struct xfrm_state *x,
bool mlx5e_ipsec_handle_tx_skb(struct mlx5e_priv *priv,
struct mlx5_wqe_eth_seg *eseg,
struct sk_buff *skb);
+void mlx5e_ipsec_offload_handle_rx_skb(struct net_device *netdev,
+ struct sk_buff *skb,
+ struct mlx5_cqe64 *cqe);
+static inline bool mlx5_ipsec_is_rx_flow(struct mlx5_cqe64 *cqe)
+{
+ return !!(MLX5_IPSEC_METADATA_MARKER_MASK & be32_to_cpu(cqe->ft_metadata));
+}
+#else
+static inline
+void mlx5e_ipsec_offload_handle_rx_skb(struct net_device *netdev,
+ struct sk_buff *skb,
+ struct mlx5_cqe64 *cqe)
+{}
+static inline bool mlx5_ipsec_is_rx_flow(struct mlx5_cqe64 *cqe) { return false; }
#endif /* CONFIG_MLX5_EN_IPSEC */
#endif /* __MLX5E_IPSEC_RXTX_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 4e5d83f6334a4..88ea1908cb14a 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -65,6 +65,7 @@
#include "en/hv_vhca_stats.h"
#include "en/devlink.h"
#include "lib/mlx5.h"
+#include "fpga/ipsec.h"
bool mlx5e_check_fragmented_striding_rq_cap(struct mlx5_core_dev *mdev)
{
@@ -496,7 +497,8 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,
rq->dealloc_wqe = mlx5e_dealloc_rx_wqe;
#ifdef CONFIG_MLX5_EN_IPSEC
- if (c->priv->ipsec)
+ if ((mlx5_fpga_ipsec_device_caps(mdev) & MLX5_ACCEL_IPSEC_CAP_DEVICE) &&
+ c->priv->ipsec)
rq->handle_rx_cqe = mlx5e_ipsec_handle_rx_cqe;
else
#endif
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 350f9c54e508f..8b24e44f860a8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -973,9 +973,14 @@ static inline void mlx5e_handle_csum(struct net_device *netdev,
goto csum_unnecessary;
if (likely(is_last_ethertype_ip(skb, &network_depth, &proto))) {
- if (unlikely(get_ip_proto(skb, network_depth, proto) == IPPROTO_SCTP))
+ u8 ipproto = get_ip_proto(skb, network_depth, proto);
+
+ if (unlikely(ipproto == IPPROTO_SCTP))
goto csum_unnecessary;
+ if (unlikely(mlx5_ipsec_is_rx_flow(cqe)))
+ goto csum_none;
+
stats->csum_complete++;
skb->ip_summed = CHECKSUM_COMPLETE;
skb->csum = csum_unfold((__force __sum16)cqe->check_sum);
@@ -1021,6 +1026,9 @@ static inline void mlx5e_build_rx_skb(struct mlx5_cqe64 *cqe,
mlx5e_tls_handle_rx_skb(rq, skb, cqe, &cqe_bcnt);
+ if (unlikely(mlx5_ipsec_is_rx_flow(cqe)))
+ mlx5e_ipsec_offload_handle_rx_skb(netdev, skb, cqe);
+
if (lro_num_seg > 1) {
mlx5e_lro_update_hdr(skb, cqe, cqe_bcnt);
skb_shinfo(skb)->gso_size = DIV_ROUND_UP(cqe_bcnt, lro_num_seg);
--
2.26.2
^ permalink raw reply related
* [net-next V2 11/15] net/mlx5e: IPsec: Add Connect-X IPsec ESN update offload support
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Raed Salem, Boris Pismenny, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Raed Salem <raeds@mellanox.com>
Synchronize offloading device ESN with xfrm received SN
by updating an existing IPsec HW context with the new SN.
Signed-off-by: Raed Salem <raeds@mellanox.com>
Reviewed-by: Boris Pismenny <borisp@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../mellanox/mlx5/core/accel/ipsec_offload.c | 88 +++++++++++++++++++
1 file changed, 88 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
index c49699d580fff..2f13a250aab3e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
@@ -279,6 +279,93 @@ static int mlx5_ipsec_offload_init(struct mlx5_core_dev *mdev)
return 0;
}
+static int mlx5_modify_ipsec_obj(struct mlx5_core_dev *mdev,
+ struct mlx5_ipsec_obj_attrs *attrs,
+ u32 ipsec_id)
+{
+ u32 in[MLX5_ST_SZ_DW(modify_ipsec_obj_in)] = {};
+ u32 out[MLX5_ST_SZ_DW(query_ipsec_obj_out)];
+ u64 modify_field_select = 0;
+ u64 general_obj_types;
+ void *obj;
+ int err;
+
+ if (!(attrs->accel_flags & MLX5_ACCEL_ESP_FLAGS_ESN_TRIGGERED))
+ return 0;
+
+ general_obj_types = MLX5_CAP_GEN_64(mdev, general_obj_types);
+ if (!(general_obj_types & MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_IPSEC))
+ return -EINVAL;
+
+ /* general object fields set */
+ MLX5_SET(general_obj_in_cmd_hdr, in, opcode, MLX5_CMD_OP_QUERY_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_type, MLX5_GENERAL_OBJECT_TYPES_IPSEC);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, ipsec_id);
+ err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+ if (err) {
+ mlx5_core_err(mdev, "Query IPsec object failed (Object id %d), err = %d\n",
+ ipsec_id, err);
+ return err;
+ }
+
+ obj = MLX5_ADDR_OF(query_ipsec_obj_out, out, ipsec_object);
+ modify_field_select = MLX5_GET64(ipsec_obj, obj, modify_field_select);
+
+ /* esn */
+ if (!(modify_field_select & MLX5_MODIFY_IPSEC_BITMASK_ESN_OVERLAP) ||
+ !(modify_field_select & MLX5_MODIFY_IPSEC_BITMASK_ESN_MSB))
+ return -EOPNOTSUPP;
+
+ obj = MLX5_ADDR_OF(modify_ipsec_obj_in, in, ipsec_object);
+ MLX5_SET(ipsec_obj, obj, esn_msb, attrs->esn_msb);
+ if (attrs->accel_flags & MLX5_ACCEL_ESP_FLAGS_ESN_STATE_OVERLAP)
+ MLX5_SET(ipsec_obj, obj, esn_overlap, 1);
+
+ /* general object fields set */
+ MLX5_SET(general_obj_in_cmd_hdr, in, opcode, MLX5_CMD_OP_MODIFY_GENERAL_OBJECT);
+
+ return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+static int mlx5_ipsec_offload_esp_modify_xfrm(struct mlx5_accel_esp_xfrm *xfrm,
+ const struct mlx5_accel_esp_xfrm_attrs *attrs)
+{
+ struct mlx5_ipsec_obj_attrs ipsec_attrs = {};
+ struct mlx5_core_dev *mdev = xfrm->mdev;
+ struct mlx5_ipsec_esp_xfrm *mxfrm;
+
+ int err = 0;
+
+ if (!memcmp(&xfrm->attrs, attrs, sizeof(xfrm->attrs)))
+ return 0;
+
+ if (mlx5_ipsec_offload_esp_validate_xfrm_attrs(mdev, attrs))
+ return -EOPNOTSUPP;
+
+ mxfrm = container_of(xfrm, struct mlx5_ipsec_esp_xfrm, accel_xfrm);
+
+ mutex_lock(&mxfrm->lock);
+
+ if (!mxfrm->sa_ctx)
+ /* Not bound xfrm, change only sw attrs */
+ goto change_sw_xfrm_attrs;
+
+ /* need to add find and replace in ipsec_rhash_sa the sa_ctx */
+ /* modify device with new hw_sa */
+ ipsec_attrs.accel_flags = attrs->flags;
+ ipsec_attrs.esn_msb = attrs->esn;
+ err = mlx5_modify_ipsec_obj(mdev,
+ &ipsec_attrs,
+ mxfrm->sa_ctx->ipsec_obj_id);
+
+change_sw_xfrm_attrs:
+ if (!err)
+ memcpy(&xfrm->attrs, attrs, sizeof(xfrm->attrs));
+
+ mutex_unlock(&mxfrm->lock);
+ return err;
+}
+
static const struct mlx5_accel_ipsec_ops ipsec_offload_ops = {
.device_caps = mlx5_ipsec_offload_device_caps,
.create_hw_context = mlx5_ipsec_offload_create_sa_ctx,
@@ -286,6 +373,7 @@ static const struct mlx5_accel_ipsec_ops ipsec_offload_ops = {
.init = mlx5_ipsec_offload_init,
.esp_create_xfrm = mlx5_ipsec_offload_esp_create_xfrm,
.esp_destroy_xfrm = mlx5_ipsec_offload_esp_destroy_xfrm,
+ .esp_modify_xfrm = mlx5_ipsec_offload_esp_modify_xfrm,
};
const struct mlx5_accel_ipsec_ops *mlx5_ipsec_offload_ops(struct mlx5_core_dev *mdev)
--
2.26.2
^ permalink raw reply related
* [net-next V2 07/15] net/mlx5: IPsec: Add HW crypto offload support
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Raed Salem, Huy Nguyen, Tariq Toukan, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Raed Salem <raeds@mellanox.com>
This patch adds support for Connect-X IPsec crypto offload
by implementing the IPsec acceleration layer needed routines,
which delegates IPsec offloads to Connect-X routines.
In Connect-X IPsec, a Security Association (SA) is added or deleted
via allocating a HW context of an encryption/decryption key and
a HW context of a matching SA (IPsec object).
The Security Policy (SP) is added or deleted by creating matching Tx/Rx
steering rules whith an action of encryption/decryption respectively,
executed using the previously allocated SA HW context.
When new xfrm state (SA) is added:
- Use a separate crypto key HW context.
- Create a separate IPsec context in HW to inlcude the SA properties:
- aes-gcm salt.
- ICV properties (ICV length, implicit IV).
- on supported devices also update ESN.
- associate the allocated crypto key with this IPsec context.
Introduce a new compilation flag MLX5_IPSEC for it.
Downstream patches will implement the Rx,Tx steering
and will add the update esn.
Signed-off-by: Raed Salem <raeds@mellanox.com>
Signed-off-by: Huy Nguyen <huyn@mellanox.com>
Reviewed-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../net/ethernet/mellanox/mlx5/core/Kconfig | 15 +-
.../net/ethernet/mellanox/mlx5/core/Makefile | 1 +
.../ethernet/mellanox/mlx5/core/accel/ipsec.c | 7 +-
.../mellanox/mlx5/core/accel/ipsec_offload.c | 291 ++++++++++++++++++
.../mellanox/mlx5/core/accel/ipsec_offload.h | 38 +++
.../ethernet/mellanox/mlx5/core/accel/tls.c | 4 +-
.../mellanox/mlx5/core/en_accel/ipsec.c | 6 +-
drivers/net/ethernet/mellanox/mlx5/core/fw.c | 6 +
.../ethernet/mellanox/mlx5/core/lib/crypto.c | 5 +-
.../ethernet/mellanox/mlx5/core/lib/mlx5.h | 8 +-
10 files changed, 372 insertions(+), 9 deletions(-)
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.h
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
index 7d7148c9b7440..99f1ec3b2575b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig
@@ -134,12 +134,25 @@ config MLX5_FPGA_IPSEC
mlx5_core driver will include the Innova FPGA core and allow building
sandbox-specific client drivers.
+config MLX5_IPSEC
+ bool "Mellanox Technologies IPsec Connect-X support"
+ depends on MLX5_CORE_EN
+ depends on XFRM_OFFLOAD
+ depends on INET_ESP_OFFLOAD || INET6_ESP_OFFLOAD
+ select MLX5_ACCEL
+ default n
+ help
+ Build IPsec support for the Connect-X family of network cards by Mellanox
+ Technologies.
+ Note: If you select this option, the mlx5_core driver will include
+ IPsec support for the Connect-X family.
+
config MLX5_EN_IPSEC
bool "IPSec XFRM cryptography-offload accelaration"
depends on MLX5_CORE_EN
depends on XFRM_OFFLOAD
depends on INET_ESP_OFFLOAD || INET6_ESP_OFFLOAD
- depends on MLX5_FPGA_IPSEC
+ depends on MLX5_FPGA_IPSEC || MLX5_IPSEC
default n
help
Build support for IPsec cryptography-offload accelaration in the NIC.
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
index 124caec65a347..375c8a2d42fa8 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile
@@ -64,6 +64,7 @@ mlx5_core-$(CONFIG_MLX5_CORE_IPOIB) += ipoib/ipoib.o ipoib/ethtool.o ipoib/ipoib
#
# Accelerations & FPGA
#
+mlx5_core-$(CONFIG_MLX5_IPSEC) += accel/ipsec_offload.o
mlx5_core-$(CONFIG_MLX5_FPGA_IPSEC) += fpga/ipsec.o
mlx5_core-$(CONFIG_MLX5_FPGA_TLS) += fpga/tls.o
mlx5_core-$(CONFIG_MLX5_ACCEL) += lib/crypto.o accel/tls.o accel/ipsec.o
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c
index 628c8887f0869..09f5ce97af46b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c
@@ -36,12 +36,17 @@
#include "accel/ipsec.h"
#include "mlx5_core.h"
#include "fpga/ipsec.h"
+#include "accel/ipsec_offload.h"
void mlx5_accel_ipsec_init(struct mlx5_core_dev *mdev)
{
- const struct mlx5_accel_ipsec_ops *ipsec_ops = mlx5_fpga_ipsec_ops(mdev);
+ const struct mlx5_accel_ipsec_ops *ipsec_ops;
int err = 0;
+ ipsec_ops = (mlx5_ipsec_offload_ops(mdev)) ?
+ mlx5_ipsec_offload_ops(mdev) :
+ mlx5_fpga_ipsec_ops(mdev);
+
if (!ipsec_ops || !ipsec_ops->init) {
mlx5_core_dbg(mdev, "IPsec ops is not supported\n");
return;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
new file mode 100644
index 0000000000000..1c8923f42b093
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.c
@@ -0,0 +1,291 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIBt
+/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */
+
+#include "mlx5_core.h"
+#include "ipsec_offload.h"
+#include "lib/mlx5.h"
+
+#define MLX5_IPSEC_DEV_BASIC_CAPS (MLX5_ACCEL_IPSEC_CAP_DEVICE | MLX5_ACCEL_IPSEC_CAP_IPV6 | \
+ MLX5_ACCEL_IPSEC_CAP_LSO)
+struct mlx5_ipsec_sa_ctx {
+ struct rhash_head hash;
+ u32 enc_key_id;
+ u32 ipsec_obj_id;
+ /* hw ctx */
+ struct mlx5_core_dev *dev;
+ struct mlx5_ipsec_esp_xfrm *mxfrm;
+};
+
+struct mlx5_ipsec_esp_xfrm {
+ /* reference counter of SA ctx */
+ struct mlx5_ipsec_sa_ctx *sa_ctx;
+ struct mutex lock; /* protects mlx5_ipsec_esp_xfrm */
+ struct mlx5_accel_esp_xfrm accel_xfrm;
+};
+
+static u32 mlx5_ipsec_offload_device_caps(struct mlx5_core_dev *mdev)
+{
+ u32 caps = MLX5_IPSEC_DEV_BASIC_CAPS;
+
+ if (!mlx5_is_ipsec_device(mdev))
+ return 0;
+
+ if (MLX5_CAP_IPSEC(mdev, ipsec_crypto_esp_aes_gcm_128_encrypt) &&
+ MLX5_CAP_IPSEC(mdev, ipsec_crypto_esp_aes_gcm_128_decrypt))
+ caps |= MLX5_ACCEL_IPSEC_CAP_ESP;
+
+ if (MLX5_CAP_IPSEC(mdev, ipsec_esn)) {
+ caps |= MLX5_ACCEL_IPSEC_CAP_ESN;
+ caps |= MLX5_ACCEL_IPSEC_CAP_TX_IV_IS_ESN;
+ }
+
+ /* We can accommodate up to 2^24 different IPsec objects
+ * because we use up to 24 bit in flow table metadata
+ * to hold the IPsec Object unique handle.
+ */
+ WARN_ON_ONCE(MLX5_CAP_IPSEC(mdev, log_max_ipsec_offload) > 24);
+ return caps;
+}
+
+static int
+mlx5_ipsec_offload_esp_validate_xfrm_attrs(struct mlx5_core_dev *mdev,
+ const struct mlx5_accel_esp_xfrm_attrs *attrs)
+{
+ if (attrs->replay_type != MLX5_ACCEL_ESP_REPLAY_NONE) {
+ mlx5_core_err(mdev, "Cannot offload xfrm states with anti replay (replay_type = %d)\n",
+ attrs->replay_type);
+ return -EOPNOTSUPP;
+ }
+
+ if (attrs->keymat_type != MLX5_ACCEL_ESP_KEYMAT_AES_GCM) {
+ mlx5_core_err(mdev, "Only aes gcm keymat is supported (keymat_type = %d)\n",
+ attrs->keymat_type);
+ return -EOPNOTSUPP;
+ }
+
+ if (attrs->keymat.aes_gcm.iv_algo !=
+ MLX5_ACCEL_ESP_AES_GCM_IV_ALGO_SEQ) {
+ mlx5_core_err(mdev, "Only iv sequence algo is supported (iv_algo = %d)\n",
+ attrs->keymat.aes_gcm.iv_algo);
+ return -EOPNOTSUPP;
+ }
+
+ if (attrs->keymat.aes_gcm.key_len != 128 &&
+ attrs->keymat.aes_gcm.key_len != 256) {
+ mlx5_core_err(mdev, "Cannot offload xfrm states with key length other than 128/256 bit (key length = %d)\n",
+ attrs->keymat.aes_gcm.key_len);
+ return -EOPNOTSUPP;
+ }
+
+ if ((attrs->flags & MLX5_ACCEL_ESP_FLAGS_ESN_TRIGGERED) &&
+ !MLX5_CAP_IPSEC(mdev, ipsec_esn)) {
+ mlx5_core_err(mdev, "Cannot offload xfrm states with ESN triggered\n");
+ return -EOPNOTSUPP;
+ }
+
+ return 0;
+}
+
+static struct mlx5_accel_esp_xfrm *
+mlx5_ipsec_offload_esp_create_xfrm(struct mlx5_core_dev *mdev,
+ const struct mlx5_accel_esp_xfrm_attrs *attrs,
+ u32 flags)
+{
+ struct mlx5_ipsec_esp_xfrm *mxfrm;
+ int err = 0;
+
+ err = mlx5_ipsec_offload_esp_validate_xfrm_attrs(mdev, attrs);
+ if (err)
+ return ERR_PTR(err);
+
+ mxfrm = kzalloc(sizeof(*mxfrm), GFP_KERNEL);
+ if (!mxfrm)
+ return ERR_PTR(-ENOMEM);
+
+ mutex_init(&mxfrm->lock);
+ memcpy(&mxfrm->accel_xfrm.attrs, attrs,
+ sizeof(mxfrm->accel_xfrm.attrs));
+
+ return &mxfrm->accel_xfrm;
+}
+
+static void mlx5_ipsec_offload_esp_destroy_xfrm(struct mlx5_accel_esp_xfrm *xfrm)
+{
+ struct mlx5_ipsec_esp_xfrm *mxfrm = container_of(xfrm, struct mlx5_ipsec_esp_xfrm,
+ accel_xfrm);
+
+ /* assuming no sa_ctx are connected to this xfrm_ctx */
+ WARN_ON(mxfrm->sa_ctx);
+ kfree(mxfrm);
+}
+
+struct mlx5_ipsec_obj_attrs {
+ const struct aes_gcm_keymat *aes_gcm;
+ u32 accel_flags;
+ u32 esn_msb;
+ u32 enc_key_id;
+};
+
+static int mlx5_create_ipsec_obj(struct mlx5_core_dev *mdev,
+ struct mlx5_ipsec_obj_attrs *attrs,
+ u32 *ipsec_id)
+{
+ const struct aes_gcm_keymat *aes_gcm = attrs->aes_gcm;
+ u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)];
+ u32 in[MLX5_ST_SZ_DW(create_ipsec_obj_in)] = {};
+ void *obj, *salt_p, *salt_iv_p;
+ int err;
+
+ obj = MLX5_ADDR_OF(create_ipsec_obj_in, in, ipsec_object);
+
+ /* salt and seq_iv */
+ salt_p = MLX5_ADDR_OF(ipsec_obj, obj, salt);
+ memcpy(salt_p, &aes_gcm->salt, sizeof(aes_gcm->salt));
+
+ switch (aes_gcm->icv_len) {
+ case 64:
+ MLX5_SET(ipsec_obj, obj, icv_length,
+ MLX5_IPSEC_OBJECT_ICV_LEN_8B);
+ break;
+ case 96:
+ MLX5_SET(ipsec_obj, obj, icv_length,
+ MLX5_IPSEC_OBJECT_ICV_LEN_12B);
+ break;
+ case 128:
+ MLX5_SET(ipsec_obj, obj, icv_length,
+ MLX5_IPSEC_OBJECT_ICV_LEN_16B);
+ break;
+ default:
+ return -EINVAL;
+ }
+ salt_iv_p = MLX5_ADDR_OF(ipsec_obj, obj, implicit_iv);
+ memcpy(salt_iv_p, &aes_gcm->seq_iv, sizeof(aes_gcm->seq_iv));
+ /* esn */
+ if (attrs->accel_flags & MLX5_ACCEL_ESP_FLAGS_ESN_TRIGGERED) {
+ MLX5_SET(ipsec_obj, obj, esn_en, 1);
+ MLX5_SET(ipsec_obj, obj, esn_msb, attrs->esn_msb);
+ if (attrs->accel_flags & MLX5_ACCEL_ESP_FLAGS_ESN_STATE_OVERLAP)
+ MLX5_SET(ipsec_obj, obj, esn_overlap, 1);
+ }
+
+ MLX5_SET(ipsec_obj, obj, dekn, attrs->enc_key_id);
+
+ /* general object fields set */
+ MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+ MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
+ MLX5_GENERAL_OBJECT_TYPES_IPSEC);
+
+ err = mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+ if (!err)
+ *ipsec_id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+
+ return err;
+}
+
+static void mlx5_destroy_ipsec_obj(struct mlx5_core_dev *mdev, u32 ipsec_id)
+{
+ u32 in[MLX5_ST_SZ_DW(general_obj_in_cmd_hdr)] = {};
+ u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)];
+
+ MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
+ MLX5_CMD_OP_DESTROY_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
+ MLX5_GENERAL_OBJECT_TYPES_IPSEC);
+ MLX5_SET(general_obj_in_cmd_hdr, in, obj_id, ipsec_id);
+
+ mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out));
+}
+
+static void *mlx5_ipsec_offload_create_sa_ctx(struct mlx5_core_dev *mdev,
+ struct mlx5_accel_esp_xfrm *accel_xfrm,
+ const __be32 saddr[4], const __be32 daddr[4],
+ const __be32 spi, bool is_ipv6, u32 *hw_handle)
+{
+ struct mlx5_accel_esp_xfrm_attrs *xfrm_attrs = &accel_xfrm->attrs;
+ struct aes_gcm_keymat *aes_gcm = &xfrm_attrs->keymat.aes_gcm;
+ struct mlx5_ipsec_obj_attrs ipsec_attrs = {};
+ struct mlx5_ipsec_esp_xfrm *mxfrm;
+ struct mlx5_ipsec_sa_ctx *sa_ctx;
+ int err;
+
+ /* alloc SA context */
+ sa_ctx = kzalloc(sizeof(*sa_ctx), GFP_KERNEL);
+ if (!sa_ctx)
+ return ERR_PTR(-ENOMEM);
+
+ sa_ctx->dev = mdev;
+
+ mxfrm = container_of(accel_xfrm, struct mlx5_ipsec_esp_xfrm, accel_xfrm);
+ mutex_lock(&mxfrm->lock);
+ sa_ctx->mxfrm = mxfrm;
+
+ /* key */
+ err = mlx5_create_encryption_key(mdev, aes_gcm->aes_key,
+ aes_gcm->key_len / BITS_PER_BYTE,
+ MLX5_ACCEL_OBJ_IPSEC_KEY,
+ &sa_ctx->enc_key_id);
+ if (err) {
+ mlx5_core_dbg(mdev, "Failed to create encryption key (err = %d)\n", err);
+ goto err_sa_ctx;
+ }
+
+ ipsec_attrs.aes_gcm = aes_gcm;
+ ipsec_attrs.accel_flags = accel_xfrm->attrs.flags;
+ ipsec_attrs.esn_msb = accel_xfrm->attrs.esn;
+ ipsec_attrs.enc_key_id = sa_ctx->enc_key_id;
+ err = mlx5_create_ipsec_obj(mdev, &ipsec_attrs,
+ &sa_ctx->ipsec_obj_id);
+ if (err) {
+ mlx5_core_dbg(mdev, "Failed to create IPsec object (err = %d)\n", err);
+ goto err_enc_key;
+ }
+
+ *hw_handle = sa_ctx->ipsec_obj_id;
+ mxfrm->sa_ctx = sa_ctx;
+ mutex_unlock(&mxfrm->lock);
+
+ return sa_ctx;
+
+err_enc_key:
+ mlx5_destroy_encryption_key(mdev, sa_ctx->enc_key_id);
+err_sa_ctx:
+ mutex_unlock(&mxfrm->lock);
+ kfree(sa_ctx);
+ return ERR_PTR(err);
+}
+
+static void mlx5_ipsec_offload_delete_sa_ctx(void *context)
+{
+ struct mlx5_ipsec_sa_ctx *sa_ctx = (struct mlx5_ipsec_sa_ctx *)context;
+ struct mlx5_ipsec_esp_xfrm *mxfrm = sa_ctx->mxfrm;
+
+ mutex_lock(&mxfrm->lock);
+ mlx5_destroy_ipsec_obj(sa_ctx->dev, sa_ctx->ipsec_obj_id);
+ mlx5_destroy_encryption_key(sa_ctx->dev, sa_ctx->enc_key_id);
+ kfree(sa_ctx);
+ mxfrm->sa_ctx = NULL;
+ mutex_unlock(&mxfrm->lock);
+}
+
+static int mlx5_ipsec_offload_init(struct mlx5_core_dev *mdev)
+{
+ return 0;
+}
+
+static const struct mlx5_accel_ipsec_ops ipsec_offload_ops = {
+ .device_caps = mlx5_ipsec_offload_device_caps,
+ .create_hw_context = mlx5_ipsec_offload_create_sa_ctx,
+ .free_hw_context = mlx5_ipsec_offload_delete_sa_ctx,
+ .init = mlx5_ipsec_offload_init,
+ .esp_create_xfrm = mlx5_ipsec_offload_esp_create_xfrm,
+ .esp_destroy_xfrm = mlx5_ipsec_offload_esp_destroy_xfrm,
+};
+
+const struct mlx5_accel_ipsec_ops *mlx5_ipsec_offload_ops(struct mlx5_core_dev *mdev)
+{
+ if (!mlx5_ipsec_offload_device_caps(mdev))
+ return NULL;
+
+ return &ipsec_offload_ops;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.h b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.h
new file mode 100644
index 0000000000000..970c66d19c1dc
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec_offload.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */
+
+#ifndef __MLX5_IPSEC_OFFLOAD_H__
+#define __MLX5_IPSEC_OFFLOAD_H__
+
+#include <linux/mlx5/driver.h>
+#include "accel/ipsec.h"
+
+#ifdef CONFIG_MLX5_IPSEC
+
+const struct mlx5_accel_ipsec_ops *mlx5_ipsec_offload_ops(struct mlx5_core_dev *mdev);
+static inline bool mlx5_is_ipsec_device(struct mlx5_core_dev *mdev)
+{
+ if (!MLX5_CAP_GEN(mdev, ipsec_offload))
+ return false;
+
+ if (!MLX5_CAP_GEN(mdev, log_max_dek))
+ return false;
+
+ if (!(MLX5_CAP_GEN_64(mdev, general_obj_types) &
+ MLX5_HCA_CAP_GENERAL_OBJECT_TYPES_IPSEC))
+ return false;
+
+ return MLX5_CAP_IPSEC(mdev, ipsec_crypto_offload) &&
+ MLX5_CAP_ETH(mdev, insert_trailer);
+}
+
+#else
+static inline const struct mlx5_accel_ipsec_ops *
+mlx5_ipsec_offload_ops(struct mlx5_core_dev *mdev) { return NULL; }
+static inline bool mlx5_is_ipsec_device(struct mlx5_core_dev *mdev)
+{
+ return false;
+}
+
+#endif /* CONFIG_MLX5_IPSEC */
+#endif /* __MLX5_IPSEC_OFFLOAD_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.c b/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.c
index cbf3d76c05a88..6c2b86a26863b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.c
@@ -113,7 +113,9 @@ int mlx5_ktls_create_key(struct mlx5_core_dev *mdev,
return -EINVAL;
}
- return mlx5_create_encryption_key(mdev, key, sz_bytes, p_key_id);
+ return mlx5_create_encryption_key(mdev, key, sz_bytes,
+ MLX5_ACCEL_OBJ_TLS_KEY,
+ p_key_id);
}
void mlx5_ktls_destroy_key(struct mlx5_core_dev *mdev, u32 key_id)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
index 8d797cd56e264..147191b7a98a6 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
@@ -111,7 +111,7 @@ static void mlx5e_ipsec_sadb_rx_del(struct mlx5e_ipsec_sa_entry *sa_entry)
static bool mlx5e_ipsec_update_esn_state(struct mlx5e_ipsec_sa_entry *sa_entry)
{
struct xfrm_replay_state_esn *replay_esn;
- u32 seq_bottom;
+ u32 seq_bottom = 0;
u8 overlap;
u32 *esn;
@@ -121,7 +121,9 @@ static bool mlx5e_ipsec_update_esn_state(struct mlx5e_ipsec_sa_entry *sa_entry)
}
replay_esn = sa_entry->x->replay_esn;
- seq_bottom = replay_esn->seq - replay_esn->replay_window + 1;
+ if (replay_esn->seq >= replay_esn->replay_window)
+ seq_bottom = replay_esn->seq - replay_esn->replay_window + 1;
+
overlap = sa_entry->esn_state.overlap;
sa_entry->esn_state.esn = xfrm_replay_seqhi(sa_entry->x,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c
index c3095863372cd..02558ac2ace69 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c
@@ -250,6 +250,12 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev)
return err;
}
+ if (MLX5_CAP_GEN(dev, ipsec_offload)) {
+ err = mlx5_core_get_caps(dev, MLX5_CAP_IPSEC);
+ if (err)
+ return err;
+ }
+
return 0;
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c b/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c
index dcea87ec59770..57eb91bcbca79 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c
@@ -6,7 +6,7 @@
int mlx5_create_encryption_key(struct mlx5_core_dev *mdev,
void *key, u32 sz_bytes,
- u32 *p_key_id)
+ u32 key_type, u32 *p_key_id)
{
u32 in[MLX5_ST_SZ_DW(create_encryption_key_in)] = {};
u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)];
@@ -41,8 +41,7 @@ int mlx5_create_encryption_key(struct mlx5_core_dev *mdev,
memcpy(key_p, key, sz_bytes);
MLX5_SET(encryption_key_obj, obj, key_size, general_obj_key_size);
- MLX5_SET(encryption_key_obj, obj, key_type,
- MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_TLS);
+ MLX5_SET(encryption_key_obj, obj, key_type, key_type);
MLX5_SET(general_obj_in_cmd_hdr, in, opcode,
MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
MLX5_SET(general_obj_in_cmd_hdr, in, obj_type,
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h b/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h
index 249539247e2e7..d046db7bb047d 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h
@@ -80,8 +80,14 @@ void mlx5_get_pme_stats(struct mlx5_core_dev *dev, struct mlx5_pme_stats *stats)
int mlx5_notifier_call_chain(struct mlx5_events *events, unsigned int event, void *data);
/* Crypto */
+enum {
+ MLX5_ACCEL_OBJ_TLS_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_TLS,
+ MLX5_ACCEL_OBJ_IPSEC_KEY = MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_TYPE_IPSEC,
+};
+
int mlx5_create_encryption_key(struct mlx5_core_dev *mdev,
- void *key, u32 sz_bytes, u32 *p_key_id);
+ void *key, u32 sz_bytes,
+ u32 key_type, u32 *p_key_id);
void mlx5_destroy_encryption_key(struct mlx5_core_dev *mdev, u32 key_id);
static inline struct net *mlx5_core_net(struct mlx5_core_dev *dev)
--
2.26.2
^ permalink raw reply related
* [net-next V2 08/15] net/mlx5: Add IPsec related Flow steering entry's fields
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Huy Nguyen, Raed Salem, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Huy Nguyen <huyn@mellanox.com>
Add FTE actions IPsec ENCRYPT/DECRYPT
Add ipsec_obj_id field in FTE
Add new action field MLX5_ACTION_IN_FIELD_IPSEC_SYNDROME
Signed-off-by: Huy Nguyen <huyn@mellanox.com>
Reviewed-by: Raed Salem <raeds@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
include/linux/mlx5/fs.h | 5 ++++-
include/linux/mlx5/mlx5_ifc.h | 12 ++++++++++--
2 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/include/linux/mlx5/fs.h b/include/linux/mlx5/fs.h
index 6c5aa0a214251..92d991d93757b 100644
--- a/include/linux/mlx5/fs.h
+++ b/include/linux/mlx5/fs.h
@@ -207,7 +207,10 @@ struct mlx5_flow_act {
u32 action;
struct mlx5_modify_hdr *modify_hdr;
struct mlx5_pkt_reformat *pkt_reformat;
- uintptr_t esp_id;
+ union {
+ u32 ipsec_obj_id;
+ uintptr_t esp_id;
+ };
u32 flags;
struct mlx5_fs_vlan vlan[MLX5_FS_VLAN_DEPTH];
struct ib_counters *counters;
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index 791766e15d5cf..9e64710bc54f4 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -416,7 +416,11 @@ struct mlx5_ifc_flow_table_prop_layout_bits {
u8 table_miss_action_domain[0x1];
u8 termination_table[0x1];
u8 reformat_and_fwd_to_table[0x1];
- u8 reserved_at_1a[0x6];
+ u8 reserved_at_1a[0x2];
+ u8 ipsec_encrypt[0x1];
+ u8 ipsec_decrypt[0x1];
+ u8 reserved_at_1e[0x2];
+
u8 termination_table_raw_traffic[0x1];
u8 reserved_at_21[0x1];
u8 log_max_ft_size[0x6];
@@ -2965,6 +2969,8 @@ enum {
MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH = 0x100,
MLX5_FLOW_CONTEXT_ACTION_VLAN_POP_2 = 0x400,
MLX5_FLOW_CONTEXT_ACTION_VLAN_PUSH_2 = 0x800,
+ MLX5_FLOW_CONTEXT_ACTION_IPSEC_DECRYPT = 0x1000,
+ MLX5_FLOW_CONTEXT_ACTION_IPSEC_ENCRYPT = 0x2000,
};
enum {
@@ -3006,7 +3012,8 @@ struct mlx5_ifc_flow_context_bits {
struct mlx5_ifc_vlan_bits push_vlan_2;
- u8 reserved_at_120[0xe0];
+ u8 ipsec_obj_id[0x20];
+ u8 reserved_at_140[0xc0];
struct mlx5_ifc_fte_match_param_bits match_value;
@@ -5752,6 +5759,7 @@ enum {
MLX5_ACTION_IN_FIELD_METADATA_REG_C_7 = 0x58,
MLX5_ACTION_IN_FIELD_OUT_TCP_SEQ_NUM = 0x59,
MLX5_ACTION_IN_FIELD_OUT_TCP_ACK_NUM = 0x5B,
+ MLX5_ACTION_IN_FIELD_IPSEC_SYNDROME = 0x5D,
};
struct mlx5_ifc_alloc_modify_header_context_out_bits {
--
2.26.2
^ permalink raw reply related
* [net-next V2 06/15] net/mlx5: Accel, Add core IPsec support for the Connect-X family
From: Saeed Mahameed @ 2020-07-17 0:04 UTC (permalink / raw)
To: David S. Miller, Jakub Kicinski
Cc: netdev, Raed Salem, Tariq Toukan, Saeed Mahameed
In-Reply-To: <20200717000410.55600-1-saeedm@mellanox.com>
From: Raed Salem <raeds@mellanox.com>
This to set the base for downstream patches to support
the new IPsec implementation of the Connect-X family.
Following modifications made:
- Remove accel layer dependency from MLX5_FPGA_IPSEC.
- Introduce accel_ipsec_ops, each IPsec device will
have to support these ops.
Signed-off-by: Raed Salem <raeds@mellanox.com>
Reviewed-by: Tariq Toukan <tariqt@mellanox.com>
Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
---
.../ethernet/mellanox/mlx5/core/accel/ipsec.c | 103 +++++++++++++-----
.../ethernet/mellanox/mlx5/core/accel/ipsec.h | 45 ++++----
.../mellanox/mlx5/core/en_accel/ipsec.c | 4 +-
.../ethernet/mellanox/mlx5/core/fpga/ipsec.c | 51 ++++++---
.../ethernet/mellanox/mlx5/core/fpga/ipsec.h | 37 ++-----
.../net/ethernet/mellanox/mlx5/core/main.c | 9 +-
include/linux/mlx5/accel.h | 6 +-
include/linux/mlx5/driver.h | 3 +
8 files changed, 154 insertions(+), 104 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c
index 8a4985d8cbfe5..628c8887f0869 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.c
@@ -31,37 +31,83 @@
*
*/
-#ifdef CONFIG_MLX5_FPGA_IPSEC
-
#include <linux/mlx5/device.h>
#include "accel/ipsec.h"
#include "mlx5_core.h"
#include "fpga/ipsec.h"
+void mlx5_accel_ipsec_init(struct mlx5_core_dev *mdev)
+{
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mlx5_fpga_ipsec_ops(mdev);
+ int err = 0;
+
+ if (!ipsec_ops || !ipsec_ops->init) {
+ mlx5_core_dbg(mdev, "IPsec ops is not supported\n");
+ return;
+ }
+
+ err = ipsec_ops->init(mdev);
+ if (err) {
+ mlx5_core_warn_once(mdev, "Failed to start IPsec device, err = %d\n", err);
+ return;
+ }
+
+ mdev->ipsec_ops = ipsec_ops;
+}
+
+void mlx5_accel_ipsec_cleanup(struct mlx5_core_dev *mdev)
+{
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mdev->ipsec_ops;
+
+ if (!ipsec_ops || !ipsec_ops->cleanup)
+ return;
+
+ ipsec_ops->cleanup(mdev);
+}
+
u32 mlx5_accel_ipsec_device_caps(struct mlx5_core_dev *mdev)
{
- return mlx5_fpga_ipsec_device_caps(mdev);
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mdev->ipsec_ops;
+
+ if (!ipsec_ops || !ipsec_ops->device_caps)
+ return 0;
+
+ return ipsec_ops->device_caps(mdev);
}
EXPORT_SYMBOL_GPL(mlx5_accel_ipsec_device_caps);
unsigned int mlx5_accel_ipsec_counters_count(struct mlx5_core_dev *mdev)
{
- return mlx5_fpga_ipsec_counters_count(mdev);
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mdev->ipsec_ops;
+
+ if (!ipsec_ops || !ipsec_ops->counters_count)
+ return -EOPNOTSUPP;
+
+ return ipsec_ops->counters_count(mdev);
}
int mlx5_accel_ipsec_counters_read(struct mlx5_core_dev *mdev, u64 *counters,
unsigned int count)
{
- return mlx5_fpga_ipsec_counters_read(mdev, counters, count);
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mdev->ipsec_ops;
+
+ if (!ipsec_ops || !ipsec_ops->counters_read)
+ return -EOPNOTSUPP;
+
+ return ipsec_ops->counters_read(mdev, counters, count);
}
void *mlx5_accel_esp_create_hw_context(struct mlx5_core_dev *mdev,
struct mlx5_accel_esp_xfrm *xfrm,
u32 *sa_handle)
{
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mdev->ipsec_ops;
__be32 saddr[4] = {}, daddr[4] = {};
+ if (!ipsec_ops || !ipsec_ops->create_hw_context)
+ return ERR_PTR(-EOPNOTSUPP);
+
if (!xfrm->attrs.is_ipv6) {
saddr[3] = xfrm->attrs.saddr.a4;
daddr[3] = xfrm->attrs.daddr.a4;
@@ -70,29 +116,18 @@ void *mlx5_accel_esp_create_hw_context(struct mlx5_core_dev *mdev,
memcpy(daddr, xfrm->attrs.daddr.a6, sizeof(daddr));
}
- return mlx5_fpga_ipsec_create_sa_ctx(mdev, xfrm, saddr,
- daddr, xfrm->attrs.spi,
- xfrm->attrs.is_ipv6, sa_handle);
+ return ipsec_ops->create_hw_context(mdev, xfrm, saddr, daddr, xfrm->attrs.spi,
+ xfrm->attrs.is_ipv6, sa_handle);
}
-void mlx5_accel_esp_free_hw_context(void *context)
+void mlx5_accel_esp_free_hw_context(struct mlx5_core_dev *mdev, void *context)
{
- mlx5_fpga_ipsec_delete_sa_ctx(context);
-}
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mdev->ipsec_ops;
-int mlx5_accel_ipsec_init(struct mlx5_core_dev *mdev)
-{
- return mlx5_fpga_ipsec_init(mdev);
-}
-
-void mlx5_accel_ipsec_build_fs_cmds(void)
-{
- mlx5_fpga_ipsec_build_fs_cmds();
-}
+ if (!ipsec_ops || !ipsec_ops->free_hw_context)
+ return;
-void mlx5_accel_ipsec_cleanup(struct mlx5_core_dev *mdev)
-{
- mlx5_fpga_ipsec_cleanup(mdev);
+ ipsec_ops->free_hw_context(context);
}
struct mlx5_accel_esp_xfrm *
@@ -100,9 +135,13 @@ mlx5_accel_esp_create_xfrm(struct mlx5_core_dev *mdev,
const struct mlx5_accel_esp_xfrm_attrs *attrs,
u32 flags)
{
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = mdev->ipsec_ops;
struct mlx5_accel_esp_xfrm *xfrm;
- xfrm = mlx5_fpga_esp_create_xfrm(mdev, attrs, flags);
+ if (!ipsec_ops || !ipsec_ops->esp_create_xfrm)
+ return ERR_PTR(-EOPNOTSUPP);
+
+ xfrm = ipsec_ops->esp_create_xfrm(mdev, attrs, flags);
if (IS_ERR(xfrm))
return xfrm;
@@ -113,15 +152,23 @@ EXPORT_SYMBOL_GPL(mlx5_accel_esp_create_xfrm);
void mlx5_accel_esp_destroy_xfrm(struct mlx5_accel_esp_xfrm *xfrm)
{
- mlx5_fpga_esp_destroy_xfrm(xfrm);
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = xfrm->mdev->ipsec_ops;
+
+ if (!ipsec_ops || !ipsec_ops->esp_destroy_xfrm)
+ return;
+
+ ipsec_ops->esp_destroy_xfrm(xfrm);
}
EXPORT_SYMBOL_GPL(mlx5_accel_esp_destroy_xfrm);
int mlx5_accel_esp_modify_xfrm(struct mlx5_accel_esp_xfrm *xfrm,
const struct mlx5_accel_esp_xfrm_attrs *attrs)
{
- return mlx5_fpga_esp_modify_xfrm(xfrm, attrs);
+ const struct mlx5_accel_ipsec_ops *ipsec_ops = xfrm->mdev->ipsec_ops;
+
+ if (!ipsec_ops || !ipsec_ops->esp_modify_xfrm)
+ return -EOPNOTSUPP;
+
+ return ipsec_ops->esp_modify_xfrm(xfrm, attrs);
}
EXPORT_SYMBOL_GPL(mlx5_accel_esp_modify_xfrm);
-
-#endif
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.h b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.h
index e89747674712c..fbb9c5415d539 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/ipsec.h
@@ -37,7 +37,7 @@
#include <linux/mlx5/driver.h>
#include <linux/mlx5/accel.h>
-#ifdef CONFIG_MLX5_FPGA_IPSEC
+#ifdef CONFIG_MLX5_ACCEL
#define MLX5_IPSEC_DEV(mdev) (mlx5_accel_ipsec_device_caps(mdev) & \
MLX5_ACCEL_IPSEC_CAP_DEVICE)
@@ -49,12 +49,30 @@ int mlx5_accel_ipsec_counters_read(struct mlx5_core_dev *mdev, u64 *counters,
void *mlx5_accel_esp_create_hw_context(struct mlx5_core_dev *mdev,
struct mlx5_accel_esp_xfrm *xfrm,
u32 *sa_handle);
-void mlx5_accel_esp_free_hw_context(void *context);
+void mlx5_accel_esp_free_hw_context(struct mlx5_core_dev *mdev, void *context);
-int mlx5_accel_ipsec_init(struct mlx5_core_dev *mdev);
-void mlx5_accel_ipsec_build_fs_cmds(void);
+void mlx5_accel_ipsec_init(struct mlx5_core_dev *mdev);
void mlx5_accel_ipsec_cleanup(struct mlx5_core_dev *mdev);
+struct mlx5_accel_ipsec_ops {
+ u32 (*device_caps)(struct mlx5_core_dev *mdev);
+ unsigned int (*counters_count)(struct mlx5_core_dev *mdev);
+ int (*counters_read)(struct mlx5_core_dev *mdev, u64 *counters, unsigned int count);
+ void* (*create_hw_context)(struct mlx5_core_dev *mdev,
+ struct mlx5_accel_esp_xfrm *xfrm,
+ const __be32 saddr[4], const __be32 daddr[4],
+ const __be32 spi, bool is_ipv6, u32 *sa_handle);
+ void (*free_hw_context)(void *context);
+ int (*init)(struct mlx5_core_dev *mdev);
+ void (*cleanup)(struct mlx5_core_dev *mdev);
+ struct mlx5_accel_esp_xfrm* (*esp_create_xfrm)(struct mlx5_core_dev *mdev,
+ const struct mlx5_accel_esp_xfrm_attrs *attrs,
+ u32 flags);
+ int (*esp_modify_xfrm)(struct mlx5_accel_esp_xfrm *xfrm,
+ const struct mlx5_accel_esp_xfrm_attrs *attrs);
+ void (*esp_destroy_xfrm)(struct mlx5_accel_esp_xfrm *xfrm);
+};
+
#else
#define MLX5_IPSEC_DEV(mdev) false
@@ -67,23 +85,12 @@ mlx5_accel_esp_create_hw_context(struct mlx5_core_dev *mdev,
return NULL;
}
-static inline void mlx5_accel_esp_free_hw_context(void *context)
-{
-}
-
-static inline int mlx5_accel_ipsec_init(struct mlx5_core_dev *mdev)
-{
- return 0;
-}
+static inline void mlx5_accel_esp_free_hw_context(struct mlx5_core_dev *mdev, void *context) {}
-static inline void mlx5_accel_ipsec_build_fs_cmds(void)
-{
-}
+static inline void mlx5_accel_ipsec_init(struct mlx5_core_dev *mdev) {}
-static inline void mlx5_accel_ipsec_cleanup(struct mlx5_core_dev *mdev)
-{
-}
+static inline void mlx5_accel_ipsec_cleanup(struct mlx5_core_dev *mdev) {}
-#endif
+#endif /* CONFIG_MLX5_ACCEL */
#endif /* __MLX5_ACCEL_IPSEC_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
index bc55c82b55ba8..8d797cd56e264 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
@@ -342,7 +342,7 @@ static int mlx5e_xfrm_add_state(struct xfrm_state *x)
goto out;
err_hw_ctx:
- mlx5_accel_esp_free_hw_context(sa_entry->hw_context);
+ mlx5_accel_esp_free_hw_context(priv->mdev, sa_entry->hw_context);
err_xfrm:
mlx5_accel_esp_destroy_xfrm(sa_entry->xfrm);
err_sa_entry:
@@ -372,7 +372,7 @@ static void mlx5e_xfrm_free_state(struct xfrm_state *x)
if (sa_entry->hw_context) {
flush_workqueue(sa_entry->ipsec->wq);
- mlx5_accel_esp_free_hw_context(sa_entry->hw_context);
+ mlx5_accel_esp_free_hw_context(sa_entry->xfrm->mdev, sa_entry->hw_context);
mlx5_accel_esp_destroy_xfrm(sa_entry->xfrm);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.c
index b463787d6ca16..cc67366495b09 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.c
@@ -359,7 +359,7 @@ u32 mlx5_fpga_ipsec_device_caps(struct mlx5_core_dev *mdev)
return ret;
}
-unsigned int mlx5_fpga_ipsec_counters_count(struct mlx5_core_dev *mdev)
+static unsigned int mlx5_fpga_ipsec_counters_count(struct mlx5_core_dev *mdev)
{
struct mlx5_fpga_device *fdev = mdev->fpga;
@@ -370,8 +370,8 @@ unsigned int mlx5_fpga_ipsec_counters_count(struct mlx5_core_dev *mdev)
number_of_ipsec_counters);
}
-int mlx5_fpga_ipsec_counters_read(struct mlx5_core_dev *mdev, u64 *counters,
- unsigned int counters_count)
+static int mlx5_fpga_ipsec_counters_read(struct mlx5_core_dev *mdev, u64 *counters,
+ unsigned int counters_count)
{
struct mlx5_fpga_device *fdev = mdev->fpga;
unsigned int i;
@@ -665,12 +665,10 @@ static bool mlx5_is_fpga_egress_ipsec_rule(struct mlx5_core_dev *dev,
return true;
}
-void *mlx5_fpga_ipsec_create_sa_ctx(struct mlx5_core_dev *mdev,
- struct mlx5_accel_esp_xfrm *accel_xfrm,
- const __be32 saddr[4],
- const __be32 daddr[4],
- const __be32 spi, bool is_ipv6,
- u32 *sa_handle)
+static void *mlx5_fpga_ipsec_create_sa_ctx(struct mlx5_core_dev *mdev,
+ struct mlx5_accel_esp_xfrm *accel_xfrm,
+ const __be32 saddr[4], const __be32 daddr[4],
+ const __be32 spi, bool is_ipv6, u32 *sa_handle)
{
struct mlx5_fpga_ipsec_sa_ctx *sa_ctx;
struct mlx5_fpga_esp_xfrm *fpga_xfrm =
@@ -862,7 +860,7 @@ mlx5_fpga_ipsec_release_sa_ctx(struct mlx5_fpga_ipsec_sa_ctx *sa_ctx)
mutex_unlock(&fipsec->sa_hash_lock);
}
-void mlx5_fpga_ipsec_delete_sa_ctx(void *context)
+static void mlx5_fpga_ipsec_delete_sa_ctx(void *context)
{
struct mlx5_fpga_esp_xfrm *fpga_xfrm =
((struct mlx5_fpga_ipsec_sa_ctx *)context)->fpga_xfrm;
@@ -1264,7 +1262,7 @@ const struct mlx5_flow_cmds *mlx5_fs_cmd_get_default_ipsec_fpga_cmds(enum fs_flo
}
}
-int mlx5_fpga_ipsec_init(struct mlx5_core_dev *mdev)
+static int mlx5_fpga_ipsec_init(struct mlx5_core_dev *mdev)
{
struct mlx5_fpga_conn_attr init_attr = {0};
struct mlx5_fpga_device *fdev = mdev->fpga;
@@ -1346,7 +1344,7 @@ static void destroy_rules_rb(struct rb_root *root)
}
}
-void mlx5_fpga_ipsec_cleanup(struct mlx5_core_dev *mdev)
+static void mlx5_fpga_ipsec_cleanup(struct mlx5_core_dev *mdev)
{
struct mlx5_fpga_device *fdev = mdev->fpga;
@@ -1451,7 +1449,7 @@ mlx5_fpga_esp_validate_xfrm_attrs(struct mlx5_core_dev *mdev,
return 0;
}
-struct mlx5_accel_esp_xfrm *
+static struct mlx5_accel_esp_xfrm *
mlx5_fpga_esp_create_xfrm(struct mlx5_core_dev *mdev,
const struct mlx5_accel_esp_xfrm_attrs *attrs,
u32 flags)
@@ -1479,7 +1477,7 @@ mlx5_fpga_esp_create_xfrm(struct mlx5_core_dev *mdev,
return &fpga_xfrm->accel_xfrm;
}
-void mlx5_fpga_esp_destroy_xfrm(struct mlx5_accel_esp_xfrm *xfrm)
+static void mlx5_fpga_esp_destroy_xfrm(struct mlx5_accel_esp_xfrm *xfrm)
{
struct mlx5_fpga_esp_xfrm *fpga_xfrm =
container_of(xfrm, struct mlx5_fpga_esp_xfrm,
@@ -1488,8 +1486,8 @@ void mlx5_fpga_esp_destroy_xfrm(struct mlx5_accel_esp_xfrm *xfrm)
kfree(fpga_xfrm);
}
-int mlx5_fpga_esp_modify_xfrm(struct mlx5_accel_esp_xfrm *xfrm,
- const struct mlx5_accel_esp_xfrm_attrs *attrs)
+static int mlx5_fpga_esp_modify_xfrm(struct mlx5_accel_esp_xfrm *xfrm,
+ const struct mlx5_accel_esp_xfrm_attrs *attrs)
{
struct mlx5_core_dev *mdev = xfrm->mdev;
struct mlx5_fpga_device *fdev = mdev->fpga;
@@ -1560,3 +1558,24 @@ int mlx5_fpga_esp_modify_xfrm(struct mlx5_accel_esp_xfrm *xfrm,
mutex_unlock(&fpga_xfrm->lock);
return err;
}
+
+static const struct mlx5_accel_ipsec_ops fpga_ipsec_ops = {
+ .device_caps = mlx5_fpga_ipsec_device_caps,
+ .counters_count = mlx5_fpga_ipsec_counters_count,
+ .counters_read = mlx5_fpga_ipsec_counters_read,
+ .create_hw_context = mlx5_fpga_ipsec_create_sa_ctx,
+ .free_hw_context = mlx5_fpga_ipsec_delete_sa_ctx,
+ .init = mlx5_fpga_ipsec_init,
+ .cleanup = mlx5_fpga_ipsec_cleanup,
+ .esp_create_xfrm = mlx5_fpga_esp_create_xfrm,
+ .esp_modify_xfrm = mlx5_fpga_esp_modify_xfrm,
+ .esp_destroy_xfrm = mlx5_fpga_esp_destroy_xfrm,
+};
+
+const struct mlx5_accel_ipsec_ops *mlx5_fpga_ipsec_ops(struct mlx5_core_dev *mdev)
+{
+ if (!mlx5_fpga_is_ipsec_device(mdev))
+ return NULL;
+
+ return &fpga_ipsec_ops;
+}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.h b/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.h
index 9ba637f0f0f27..db88eb4c49e34 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.h
@@ -38,44 +38,23 @@
#include "fs_cmd.h"
#ifdef CONFIG_MLX5_FPGA_IPSEC
+const struct mlx5_accel_ipsec_ops *mlx5_fpga_ipsec_ops(struct mlx5_core_dev *mdev);
u32 mlx5_fpga_ipsec_device_caps(struct mlx5_core_dev *mdev);
-unsigned int mlx5_fpga_ipsec_counters_count(struct mlx5_core_dev *mdev);
-int mlx5_fpga_ipsec_counters_read(struct mlx5_core_dev *mdev, u64 *counters,
- unsigned int counters_count);
-
-void *mlx5_fpga_ipsec_create_sa_ctx(struct mlx5_core_dev *mdev,
- struct mlx5_accel_esp_xfrm *accel_xfrm,
- const __be32 saddr[4],
- const __be32 daddr[4],
- const __be32 spi, bool is_ipv6,
- u32 *sa_handle);
-void mlx5_fpga_ipsec_delete_sa_ctx(void *context);
-
-int mlx5_fpga_ipsec_init(struct mlx5_core_dev *mdev);
-void mlx5_fpga_ipsec_cleanup(struct mlx5_core_dev *mdev);
-void mlx5_fpga_ipsec_build_fs_cmds(void);
-
-struct mlx5_accel_esp_xfrm *
-mlx5_fpga_esp_create_xfrm(struct mlx5_core_dev *mdev,
- const struct mlx5_accel_esp_xfrm_attrs *attrs,
- u32 flags);
-void mlx5_fpga_esp_destroy_xfrm(struct mlx5_accel_esp_xfrm *xfrm);
-int mlx5_fpga_esp_modify_xfrm(struct mlx5_accel_esp_xfrm *xfrm,
- const struct mlx5_accel_esp_xfrm_attrs *attrs);
-
const struct mlx5_flow_cmds *
mlx5_fs_cmd_get_default_ipsec_fpga_cmds(enum fs_flow_table_type type);
+void mlx5_fpga_ipsec_build_fs_cmds(void);
#else
-static inline u32 mlx5_fpga_ipsec_device_caps(struct mlx5_core_dev *mdev)
-{
- return 0;
-}
-
+static inline
+const struct mlx5_accel_ipsec_ops *mlx5_fpga_ipsec_ops(struct mlx5_core_dev *mdev)
+{ return NULL; }
+static inline u32 mlx5_fpga_ipsec_device_caps(struct mlx5_core_dev *mdev) { return 0; }
static inline const struct mlx5_flow_cmds *
mlx5_fs_cmd_get_default_ipsec_fpga_cmds(enum fs_flow_table_type type)
{
return mlx5_fs_cmd_get_default(type);
}
+static inline void mlx5_fpga_ipsec_build_fs_cmds(void) {};
+
#endif /* CONFIG_MLX5_FPGA_IPSEC */
#endif /* __MLX5_FPGA_IPSEC_H__ */
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 8b658908f0442..e32d46c337011 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -1089,11 +1089,7 @@ static int mlx5_load(struct mlx5_core_dev *dev)
goto err_fpga_start;
}
- err = mlx5_accel_ipsec_init(dev);
- if (err) {
- mlx5_core_err(dev, "IPSec device start failed %d\n", err);
- goto err_ipsec_start;
- }
+ mlx5_accel_ipsec_init(dev);
err = mlx5_accel_tls_init(dev);
if (err) {
@@ -1135,7 +1131,6 @@ static int mlx5_load(struct mlx5_core_dev *dev)
mlx5_accel_tls_cleanup(dev);
err_tls_start:
mlx5_accel_ipsec_cleanup(dev);
-err_ipsec_start:
mlx5_fpga_device_stop(dev);
err_fpga_start:
mlx5_rsc_dump_cleanup(dev);
@@ -1628,7 +1623,7 @@ static int __init init(void)
get_random_bytes(&sw_owner_id, sizeof(sw_owner_id));
mlx5_core_verify_params();
- mlx5_accel_ipsec_build_fs_cmds();
+ mlx5_fpga_ipsec_build_fs_cmds();
mlx5_register_debugfs();
err = pci_register_driver(&mlx5_core_driver);
diff --git a/include/linux/mlx5/accel.h b/include/linux/mlx5/accel.h
index 96ebaa94a92e5..dacf69516002e 100644
--- a/include/linux/mlx5/accel.h
+++ b/include/linux/mlx5/accel.h
@@ -126,7 +126,7 @@ enum mlx5_accel_ipsec_cap {
MLX5_ACCEL_IPSEC_CAP_TX_IV_IS_ESN = 1 << 7,
};
-#ifdef CONFIG_MLX5_FPGA_IPSEC
+#ifdef CONFIG_MLX5_ACCEL
u32 mlx5_accel_ipsec_device_caps(struct mlx5_core_dev *mdev);
@@ -152,5 +152,5 @@ static inline int
mlx5_accel_esp_modify_xfrm(struct mlx5_accel_esp_xfrm *xfrm,
const struct mlx5_accel_esp_xfrm_attrs *attrs) { return -EOPNOTSUPP; }
-#endif
-#endif
+#endif /* CONFIG_MLX5_ACCEL */
+#endif /* __MLX5_ACCEL_H__ */
diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h
index 1e6ca716635a9..6a97ad601991e 100644
--- a/include/linux/mlx5/driver.h
+++ b/include/linux/mlx5/driver.h
@@ -707,6 +707,9 @@ struct mlx5_core_dev {
} roce;
#ifdef CONFIG_MLX5_FPGA
struct mlx5_fpga_device *fpga;
+#endif
+#ifdef CONFIG_MLX5_ACCEL
+ const struct mlx5_accel_ipsec_ops *ipsec_ops;
#endif
struct mlx5_clock clock;
struct mlx5_ib_clock_info *clock_info;
--
2.26.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.