From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yb1-f171.google.com (mail-yb1-f171.google.com [209.85.219.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D914307AED for ; Wed, 2 Jul 2025 17:13:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.171 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751476425; cv=none; b=sPkMCZlI0Js7ZWav5Pgbhom0YQO75jtIFjUuYjTv0AoPbfjHHkeLq5JNs0iaI+9wagrzB0KCI1v3FUc1RReJ4tapIiM2vtzKTnE1CIW+jmwIUasdGJTgqilKjWfjqP0wXH6vjN9ZfR+sFIzlKd7JLRRdhh24mqOF2PR6X/0BNvg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751476425; c=relaxed/simple; bh=lR8sZYn1piOI9rb/eMaBw5zn0vMldVTl/zyYQeigi7s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FKbC4Nf4UzMsD3Ndg5/xvrAVA4nwFA5gVHkwEcMc1BOZ9p2bfr6E7JgOUbJYsMETHcOIbBXVjqv2/DihuYSCgzs+nulmzKa4LuUBQtY8MOKNuxe/OTUJbulYDQjaEpUz2KEQpok9NrUWp5CPysTI6S0Z5irp3EU+8RuQMBYpfzs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eRY7QDoD; arc=none smtp.client-ip=209.85.219.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eRY7QDoD" Received: by mail-yb1-f171.google.com with SMTP id 3f1490d57ef6-e812c817de0so4023282276.0 for ; Wed, 02 Jul 2025 10:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1751476422; x=1752081222; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PNKvn9ika6fvQ1y4/w4LelnjLc9DZeAjrdRp4Yq0EKk=; b=eRY7QDoDFR4k/dSx7pQ0omJwub5JOM4Vsue+vBLM4RwJhuEEXskvoEUrMF5CJOC/SZ Mh0ZO52wgoAa5fXh9ed1GEl+UwL7iWMgRaPdCNfbEnHpKrKMmOOcHDdMCEojv7efsf3G QG72S0orL8fehg++HgUEtUZ9dDlWJ24lqRIVVhMcH0fmdNqBhazZDJJX4S5piJNEyEJB FbZ4x70EDq8SweJtqhdaHhSt4Bysc5qR3NB20Q0okS+yZHD0BrWTgjsSga9cPXodaf+n 8jRiKAaTq7HkyoCRgdxKn2sOa8wsD9+I5yDDiTlDnHm0kxN/D/4fRj3b0bWMB6dVhYLe F0iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751476422; x=1752081222; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PNKvn9ika6fvQ1y4/w4LelnjLc9DZeAjrdRp4Yq0EKk=; b=Ncv3mbih+tdWrVNSTe8up2XIr9oDSpi+ogouol1GVXe9r0XLLgeJmkZvdOX1hxINAU 6/zzeamwRLw7l1GDBlUL4kWnBv4uh5XKW0CKpFIkipUHolMNu1X7zPlEYKaPD0QtaPit f4MMhl/WLFnLHzRxXOpe6Z9z2ChrnoE25abX9DW07Q3cAIiFbxSievIKJ4Xmk2/JnvCM UhaUgOaL3SED6sxTqeUNUiIYXtX4LnR69FgyWC05U2EdJTzZ2P/vi7jjWWEEKzqEmvM1 xt3/nGMpCFOU4VRX7SdgMeyKMmBHGS0hyt+MTw+joNyNi/CUjXbwwoKtkp9AavQlzgvG Vsdw== X-Forwarded-Encrypted: i=1; AJvYcCVk8Di9TJw6xpGeNmM97+hSD4fz4Vc/SwKl+J3/rVV6Dem6EPp8+Wk7r2iJkPqxyCa7w2wp6+c=@vger.kernel.org X-Gm-Message-State: AOJu0Yw4uNrP2/JgsattTibhGdt5bnJ08TQ6Wvtc6e9A25IPA3sp7jkq VjOiZp0j/Unvr9sOo73KW211yiyg7xLQi2L5k59epRvLN3FWiKKyOiOJ X-Gm-Gg: ASbGncue23kAAkjhHhPILMRvgXXJCKW/hr4JACrQpeTYLB5qtipaNyH0UlKzs6GeXOp fwRYPxQljLEoESgT05+J4fVYd+77y5irC39l2Uf3z/yRv9VWtSjM9lRmpRXQ1p4GLCD0Jj1Y+YZ feJLXXdNVXK5vZvjUDvYzaZP9628we7lnji13M5LcC08/l0EFvphXrNGPQD/URYytjI9VoTS+bh 1rDNVFXXjBxrKbWgyAuHWCYqLqm0WIRwAwgGLVvT3dCclz42/LtbjXq5ZDSbHlWu53Dx5XswaB+ Tg4uXtBgMzJ86QDWJJEzBSMAx1E8sobHg4oMSwUTy1ISFdNTHjJeK7bzSaWf X-Google-Smtp-Source: AGHT+IFLmpiiHnYy0kJY+ax11Yf2HnjqOs7QBdxDYigyd3lJdgQkIoFZqc2b9R5zVgoDGcKFEU4lXg== X-Received: by 2002:a25:bacf:0:b0:e7d:a7c7:3f34 with SMTP id 3f1490d57ef6-e897e213aeemr4087680276.32.1751476420186; Wed, 02 Jul 2025 10:13:40 -0700 (PDT) Received: from localhost ([2a03:2880:25ff:43::]) by smtp.gmail.com with ESMTPSA id 3f1490d57ef6-e87a6bf02bcsm3811740276.50.2025.07.02.10.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Jul 2025 10:13:39 -0700 (PDT) From: Daniel Zahka To: Donald Hunter , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Simon Horman , Jonathan Corbet , Andrew Lunn Cc: Saeed Mahameed , Leon Romanovsky , Tariq Toukan , Boris Pismenny , Kuniyuki Iwashima , Willem de Bruijn , David Ahern , Neal Cardwell , Patrisious Haddad , Raed Salem , Jianbo Liu , Dragos Tatulea , Rahul Rameshbabu , Stanislav Fomichev , =?UTF-8?q?Toke=20H=C3=B8iland-J=C3=B8rgensen?= , Alexander Lobakin , Jacob Keller , netdev@vger.kernel.org Subject: [PATCH v3 11/19] net/mlx5e: Support PSP offload functionality Date: Wed, 2 Jul 2025 10:13:16 -0700 Message-ID: <20250702171326.3265825-12-daniel.zahka@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250702171326.3265825-1-daniel.zahka@gmail.com> References: <20250702171326.3265825-1-daniel.zahka@gmail.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Raed Salem Add PSP offload related IFC structs, layouts, and enumerations. Implement .set_config and .rx_spi_alloc PSP device operations. Driver does not need to make use of the .set_config operation. Stub .assoc_add and .assoc_del PSP operations. Introduce the MLX5_EN_PSP configuration option for enabling PSP offload support on mlx5 devices. Signed-off-by: Raed Salem Signed-off-by: Rahul Rameshbabu Signed-off-by: Daniel Zahka --- Notes: v1: - https://lore.kernel.org/netdev/20240510030435.120935-10-kuba@kernel.org/ .../net/ethernet/mellanox/mlx5/core/Kconfig | 11 ++ .../net/ethernet/mellanox/mlx5/core/Makefile | 4 +- drivers/net/ethernet/mellanox/mlx5/core/en.h | 3 + .../ethernet/mellanox/mlx5/core/en/params.c | 4 +- .../mellanox/mlx5/core/en_accel/psp.c | 149 ++++++++++++++++++ .../mellanox/mlx5/core/en_accel/psp.h | 53 +++++++ .../mellanox/mlx5/core/en_accel/psp_offload.c | 52 ++++++ .../net/ethernet/mellanox/mlx5/core/en_main.c | 9 ++ drivers/net/ethernet/mellanox/mlx5/core/fw.c | 6 + .../net/ethernet/mellanox/mlx5/core/main.c | 5 + drivers/net/ethernet/mellanox/mlx5/core/psp.c | 24 +++ drivers/net/ethernet/mellanox/mlx5/core/psp.h | 15 ++ include/linux/mlx5/device.h | 4 + include/linux/mlx5/driver.h | 2 + include/linux/mlx5/mlx5_ifc.h | 94 ++++++++++- 15 files changed, 428 insertions(+), 7 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.h create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_offload.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/psp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/psp.h diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig index 6ec7d6e0181d..79a64489da89 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig +++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig @@ -208,3 +208,14 @@ config MLX5_DPLL help DPLL support in Mellanox Technologies ConnectX NICs. +config MLX5_EN_PSP + bool "Mellanox Technologies support for PSP cryptography-offload acceleration" + depends on INET_PSP + depends on MLX5_CORE_EN + default y + help + mlx5 device offload support for Google PSP Security Protocol offload. + Adds support for PSP encryption offload and for SPI and key generation + interfaces to PSP Stack which supports PSP crypto offload. + + If unsure, say Y. diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index d292e6a9e22c..e27de74ef028 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -17,7 +17,7 @@ mlx5_core-y := main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \ fs_counters.o fs_ft_pool.o rl.o lag/debugfs.o lag/lag.o dev.o events.o wq.o lib/gid.o \ lib/devcom.o lib/pci_vsc.o lib/dm.o lib/fs_ttc.o diag/fs_tracepoint.o \ diag/fw_tracer.o diag/crdump.o devlink.o diag/rsc_dump.o diag/reporter_vnic.o \ - fw_reset.o qos.o lib/tout.o lib/aso.o wc.o fs_pool.o + fw_reset.o qos.o lib/tout.o lib/aso.o wc.o fs_pool.o psp.o # # Netdev basic @@ -110,6 +110,8 @@ mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/ktls_stats.o \ en_accel/fs_tcp.o en_accel/ktls.o en_accel/ktls_txrx.o \ en_accel/ktls_tx.o en_accel/ktls_rx.o +mlx5_core-$(CONFIG_MLX5_EN_PSP) += en_accel/psp.o en_accel/psp_offload.o + # # SW Steering # diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 65a73913b9a2..9078becfd710 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -934,6 +934,9 @@ struct mlx5e_priv { #ifdef CONFIG_MLX5_EN_IPSEC struct mlx5e_ipsec *ipsec; #endif +#ifdef CONFIG_MLX5_EN_PSP + struct mlx5e_psp *psp; +#endif #ifdef CONFIG_MLX5_EN_TLS struct mlx5e_tls *tls; #endif diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c index fc945bce933a..68970f96fc7c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.c @@ -6,6 +6,7 @@ #include "en/port.h" #include "en_accel/en_accel.h" #include "en_accel/ipsec.h" +#include "en_accel/psp.h" #include #include #include @@ -1017,7 +1018,8 @@ void mlx5e_build_sq_param(struct mlx5_core_dev *mdev, bool allow_swp; allow_swp = mlx5_geneve_tx_allowed(mdev) || - (mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_CRYPTO); + (mlx5_ipsec_device_caps(mdev) & MLX5_IPSEC_CAP_CRYPTO) || + mlx5_is_psp_device(mdev); mlx5e_build_sq_param_common(mdev, param); MLX5_SET(wq, wq, log_wq_sz, params->log_sq_size); MLX5_SET(sqc, sqc, allow_swp, allow_swp); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c new file mode 100644 index 000000000000..482e2cdabdae --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.c @@ -0,0 +1,149 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ +#include +#include +#include +#include "mlx5_core.h" +#include "psp.h" +#include "lib/crypto.h" +#include "en_accel/psp.h" + +static int +mlx5e_psp_set_config(struct psp_dev *psd, struct psp_dev_config *conf, + struct netlink_ext_ack *extack) +{ + return 0; /* TODO: this should actually do things to the device */ +} + +static int +mlx5e_psp_rx_spi_alloc(struct psp_dev *psd, u32 version, + struct psp_key_parsed *assoc, + struct netlink_ext_ack *extack) +{ + struct mlx5e_priv *priv = netdev_priv(psd->main_netdev); + enum mlx5_psp_gen_spi_in_key_size keysz; + struct psp_key_spi key_spi = {}; + u8 keysz_bytes; + int err; + + switch (version) { + case PSP_VERSION_HDR0_AES_GCM_128: + keysz = MLX5_PSP_GEN_SPI_IN_KEY_SIZE_128; + keysz_bytes = 16; + break; + case PSP_VERSION_HDR0_AES_GCM_256: + keysz = MLX5_PSP_GEN_SPI_IN_KEY_SIZE_256; + keysz_bytes = 32; + break; + default: + return -EINVAL; + } + + err = mlx5e_psp_generate_key_spi(priv->mdev, keysz, keysz_bytes, + &key_spi); + if (err) + return err; + + assoc->spi = cpu_to_be32(key_spi.spi); + memcpy(assoc->key, key_spi.key, keysz_bytes); + return 0; +} + +static int mlx5e_psp_assoc_add(struct psp_dev *psd, struct psp_assoc *pas, + struct netlink_ext_ack *extack) +{ + struct mlx5e_priv *priv = netdev_priv(psd->main_netdev); + + mlx5_core_dbg(priv->mdev, "PSP assoc add: rx: %u, tx: %u\n", + be32_to_cpu(pas->rx.spi), be32_to_cpu(pas->tx.spi)); + + return -EINVAL; +} + +static void mlx5e_psp_assoc_del(struct psp_dev *psd, struct psp_assoc *pas) +{ +} + +static struct psp_dev_ops mlx5_psp_ops = { + .set_config = mlx5e_psp_set_config, + .rx_spi_alloc = mlx5e_psp_rx_spi_alloc, + .tx_key_add = mlx5e_psp_assoc_add, + .tx_key_del = mlx5e_psp_assoc_del, +}; + +void mlx5e_psp_unregister(struct mlx5e_priv *priv) +{ + if (!priv->psp || !priv->psp->psp) + return; + + psp_dev_unregister(priv->psp->psp); +} + +void mlx5e_psp_register(struct mlx5e_priv *priv) +{ + /* FW Caps missing */ + if (!priv->psp) + return; + + priv->psp->caps.assoc_drv_spc = sizeof(u32); + priv->psp->caps.versions = 1 << PSP_VERSION_HDR0_AES_GCM_128; + if (MLX5_CAP_PSP(priv->mdev, psp_crypto_esp_aes_gcm_256_encrypt) && + MLX5_CAP_PSP(priv->mdev, psp_crypto_esp_aes_gcm_256_decrypt)) + priv->psp->caps.versions |= 1 << PSP_VERSION_HDR0_AES_GCM_256; + + priv->psp->psp = psp_dev_create(priv->netdev, &mlx5_psp_ops, + &priv->psp->caps, NULL); + if (IS_ERR(priv->psp->psp)) + mlx5_core_err(priv->mdev, "PSP failed to register due to %pe\n", + priv->psp->psp); +} + +int mlx5e_psp_init(struct mlx5e_priv *priv) +{ + struct mlx5_core_dev *mdev = priv->mdev; + struct mlx5e_psp *psp; + + if (!mlx5_is_psp_device(mdev)) { + mlx5_core_dbg(mdev, "PSP offload not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp)) { + mlx5_core_dbg(mdev, "SWP not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp_csum)) { + mlx5_core_dbg(mdev, "SWP checksum not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp_csum_l4_partial)) { + mlx5_core_dbg(mdev, "SWP L4 partial checksum not supported\n"); + return -EOPNOTSUPP; + } + + if (!MLX5_CAP_ETH(mdev, swp_lso)) { + mlx5_core_dbg(mdev, "PSP LSO not supported\n"); + return -EOPNOTSUPP; + } + + psp = kzalloc(sizeof(*psp), GFP_KERNEL); + if (!psp) + return -ENOMEM; + + priv->psp = psp; + mlx5_core_dbg(priv->mdev, "PSP attached to netdevice\n"); + return 0; +} + +void mlx5e_psp_cleanup(struct mlx5e_priv *priv) +{ + struct mlx5e_psp *psp = priv->psp; + + if (!psp) + return; + + priv->psp = NULL; + kfree(psp); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.h new file mode 100644 index 000000000000..9707f50029ed --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5E_ACCEL_PSP_H__ +#define __MLX5E_ACCEL_PSP_H__ +#if IS_ENABLED(CONFIG_MLX5_EN_PSP) +#include +#include "en.h" + +struct mlx5e_psp { + struct psp_dev *psp; + struct psp_dev_caps caps; +}; + +struct psp_key_spi { + u32 spi; + __be32 key[PSP_MAX_KEY / sizeof(u32)]; + u16 keysz; +}; + +static inline bool mlx5_is_psp_device(struct mlx5_core_dev *mdev) +{ + if (!MLX5_CAP_GEN(mdev, psp)) + return false; + + if (!MLX5_CAP_PSP(mdev, psp_crypto_esp_aes_gcm_128_encrypt) || + !MLX5_CAP_PSP(mdev, psp_crypto_esp_aes_gcm_128_decrypt)) + return false; + + return true; +} + +void mlx5e_psp_register(struct mlx5e_priv *priv); +void mlx5e_psp_unregister(struct mlx5e_priv *priv); +int mlx5e_psp_init(struct mlx5e_priv *priv); +void mlx5e_psp_cleanup(struct mlx5e_priv *priv); +int mlx5e_psp_rotate_key(struct mlx5_core_dev *mdev); +int mlx5e_psp_generate_key_spi(struct mlx5_core_dev *mdev, + enum mlx5_psp_gen_spi_in_key_size keysz, + unsigned int keysz_bytes, + struct psp_key_spi *keys); +#else +static inline bool mlx5_is_psp_device(struct mlx5_core_dev *mdev) +{ + return false; +} + +static inline void mlx5e_psp_register(struct mlx5e_priv *priv) { } +static inline void mlx5e_psp_unregister(struct mlx5e_priv *priv) { } +static inline int mlx5e_psp_init(struct mlx5e_priv *priv) { return 0; } +static inline void mlx5e_psp_cleanup(struct mlx5e_priv *priv) { } +#endif /* CONFIG_MLX5_EN_PSP */ +#endif /* __MLX5E_ACCEL_PSP_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_offload.c new file mode 100644 index 000000000000..c3c21a99a92b --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/psp_offload.c @@ -0,0 +1,52 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ +#include +#include +#include "mlx5_core.h" +#include "en_accel/psp.h" + +int mlx5e_psp_rotate_key(struct mlx5_core_dev *mdev) +{ + u32 in[MLX5_ST_SZ_DW(psp_rotate_key_in)] = {}; + u32 out[MLX5_ST_SZ_DW(psp_rotate_key_out)]; + + MLX5_SET(psp_rotate_key_in, in, opcode, + MLX5_CMD_OP_PSP_ROTATE_KEY); + + return mlx5_cmd_exec(mdev, in, sizeof(in), out, sizeof(out)); +} + +int mlx5e_psp_generate_key_spi(struct mlx5_core_dev *mdev, + enum mlx5_psp_gen_spi_in_key_size keysz, + unsigned int keysz_bytes, + struct psp_key_spi *keys) +{ + u32 in[MLX5_ST_SZ_DW(psp_gen_spi_in)] = {}; + int err, outlen, i; + void *out, *outkey; + + WARN_ON_ONCE(keysz_bytes > PSP_MAX_KEY); + + outlen = MLX5_ST_SZ_BYTES(psp_gen_spi_out) + MLX5_ST_SZ_BYTES(key_spi); + out = kzalloc(outlen, GFP_KERNEL); + if (!out) + return -ENOMEM; + + MLX5_SET(psp_gen_spi_in, in, opcode, MLX5_CMD_OP_PSP_GEN_SPI); + MLX5_SET(psp_gen_spi_in, in, key_size, keysz); + MLX5_SET(psp_gen_spi_in, in, num_of_spi, 1); + err = mlx5_cmd_exec(mdev, in, sizeof(in), out, outlen); + if (err) + goto out; + + outkey = MLX5_ADDR_OF(psp_gen_spi_out, out, key_spi); + keys->keysz = keysz_bytes * BITS_PER_BYTE; + keys->spi = MLX5_GET(key_spi, outkey, spi); + for (i = 0; i < keysz_bytes / sizeof(*keys->key); ++i) + keys->key[i] = cpu_to_be32(MLX5_GET(key_spi, + outkey + (32 - keysz_bytes), key[i])); + +out: + kfree(out); + return err; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index dca5ca51a470..b651827e5321 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -52,6 +52,7 @@ #include "en_tc.h" #include "en_rep.h" #include "en_accel/ipsec.h" +#include "en_accel/psp.h" #include "en_accel/macsec.h" #include "en_accel/en_accel.h" #include "en_accel/ktls.h" @@ -5835,6 +5836,7 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, if (take_rtnl) rtnl_lock(); + mlx5e_psp_register(priv); /* update XDP supported features */ mlx5e_set_xdp_feature(netdev); @@ -5847,6 +5849,7 @@ static int mlx5e_nic_init(struct mlx5_core_dev *mdev, static void mlx5e_nic_cleanup(struct mlx5e_priv *priv) { mlx5e_health_destroy_reporters(priv); + mlx5e_psp_unregister(priv); mlx5e_ktls_cleanup(priv); mlx5e_fs_cleanup(priv->fs); debugfs_remove_recursive(priv->dfs_root); @@ -5974,6 +5977,10 @@ static void mlx5e_nic_enable(struct mlx5e_priv *priv) if (err) mlx5_core_err(mdev, "MACsec initialization failed, %d\n", err); + err = mlx5e_psp_init(priv); + if (err) + mlx5_core_err(mdev, "PSP initialization failed, %d\n", err); + /* Marking the link as currently not needed by the Driver */ if (!netif_running(netdev)) mlx5e_modify_admin_state(mdev, MLX5_PORT_DOWN); @@ -6035,6 +6042,7 @@ static void mlx5e_nic_disable(struct mlx5e_priv *priv) mlx5e_disable_async_events(priv); mlx5_lag_remove_netdev(mdev, priv->netdev); mlx5_vxlan_reset_to_default(mdev->vxlan); + mlx5e_psp_cleanup(priv); mlx5e_macsec_cleanup(priv); mlx5e_ipsec_cleanup(priv); } @@ -6693,6 +6701,7 @@ static void _mlx5e_remove(struct auxiliary_device *adev) * is already unregistered before changing to NIC profile. */ if (priv->netdev->reg_state == NETREG_REGISTERED) { + mlx5e_psp_unregister(priv); unregister_netdev(priv->netdev); _mlx5e_suspend(adev, false); } else { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c index 57476487e31f..eeb4437975f2 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c @@ -294,6 +294,12 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev) return err; } + if (MLX5_CAP_GEN(dev, psp)) { + err = mlx5_core_get_caps(dev, MLX5_CAP_PSP); + if (err) + return err; + } + return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 41e8660c819c..1890e4b09ffe 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -74,6 +74,7 @@ #include "mlx5_irq.h" #include "hwmon.h" #include "lag/lag.h" +#include "psp.h" MODULE_AUTHOR("Eli Cohen "); MODULE_DESCRIPTION("Mellanox 5th generation network adapters (ConnectX series) core driver"); @@ -1046,6 +1047,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev) dev->vxlan = mlx5_vxlan_create(dev); dev->geneve = mlx5_geneve_create(dev); + dev->psp = mlx5_psp_create(dev); err = mlx5_init_rl_table(dev); if (err) { @@ -1128,6 +1130,7 @@ static int mlx5_init_once(struct mlx5_core_dev *dev) err_rl_cleanup: mlx5_cleanup_rl_table(dev); err_clock_cleanup: + mlx5_psp_destroy(dev->psp); mlx5_geneve_destroy(dev->geneve); mlx5_vxlan_destroy(dev->vxlan); mlx5_cleanup_clock(dev); @@ -1163,6 +1166,7 @@ static void mlx5_cleanup_once(struct mlx5_core_dev *dev) mlx5_sriov_cleanup(dev); mlx5_mpfs_cleanup(dev); mlx5_cleanup_rl_table(dev); + mlx5_psp_destroy(dev->psp); mlx5_geneve_destroy(dev->geneve); mlx5_vxlan_destroy(dev->vxlan); mlx5_cleanup_clock(dev); @@ -1799,6 +1803,7 @@ static const int types[] = { MLX5_CAP_VDPA_EMULATION, MLX5_CAP_IPSEC, MLX5_CAP_PORT_SELECTION, + MLX5_CAP_PSP, MLX5_CAP_MACSEC, MLX5_CAP_ADV_VIRTUALIZATION, MLX5_CAP_CRYPTO, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/psp.c b/drivers/net/ethernet/mellanox/mlx5/core/psp.c new file mode 100644 index 000000000000..15df8bde3632 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/psp.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#include "psp.h" + +struct mlx5_psp *mlx5_psp_create(struct mlx5_core_dev *mdev) +{ + struct mlx5_psp *psp = kzalloc(sizeof(*psp), GFP_KERNEL); + + if (!psp) + return ERR_PTR(-ENOMEM); + + psp->mdev = mdev; + + return psp; +} + +void mlx5_psp_destroy(struct mlx5_psp *psp) +{ + if (IS_ERR_OR_NULL(psp)) + return; + + kfree(psp); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/psp.h b/drivers/net/ethernet/mellanox/mlx5/core/psp.h new file mode 100644 index 000000000000..1d7927c4ea72 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/psp.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. */ + +#ifndef __MLX5_PSP_H__ +#define __MLX5_PSP_H__ +#include + +struct mlx5_psp { + struct mlx5_core_dev *mdev; +}; + +struct mlx5_psp *mlx5_psp_create(struct mlx5_core_dev *mdev); +void mlx5_psp_destroy(struct mlx5_psp *psp); + +#endif /* __MLX5_PSP_H__ */ diff --git a/include/linux/mlx5/device.h b/include/linux/mlx5/device.h index 6822cfa5f4ad..c2809d3b094c 100644 --- a/include/linux/mlx5/device.h +++ b/include/linux/mlx5/device.h @@ -1247,6 +1247,7 @@ enum mlx5_cap_type { MLX5_CAP_IPSEC, MLX5_CAP_CRYPTO = 0x1a, MLX5_CAP_SHAMPO = 0x1d, + MLX5_CAP_PSP = 0x1e, MLX5_CAP_MACSEC = 0x1f, MLX5_CAP_GENERAL_2 = 0x20, MLX5_CAP_PORT_SELECTION = 0x25, @@ -1486,6 +1487,9 @@ enum mlx5_qcam_feature_groups { #define MLX5_CAP_SHAMPO(mdev, cap) \ MLX5_GET(shampo_cap, mdev->caps.hca[MLX5_CAP_SHAMPO]->cur, cap) +#define MLX5_CAP_PSP(mdev, cap)\ + MLX5_GET(psp_cap, (mdev)->caps.hca[MLX5_CAP_PSP]->cur, cap) + enum { MLX5_CMD_STAT_OK = 0x0, MLX5_CMD_STAT_INT_ERR = 0x1, diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index e6ba8f4f4bd1..671512699a92 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -491,6 +491,7 @@ struct mlx5_sf_dev_table; struct mlx5_sf_hw_table; struct mlx5_sf_table; struct mlx5_crypto_dek_priv; +struct mlx5_psp; struct mlx5_rate_limit { u32 rate; @@ -786,6 +787,7 @@ struct mlx5_core_dev { enum mlx5_wc_state wc_state; /* sync write combining state */ struct mutex wc_state_lock; + struct mlx5_psp *psp; }; struct mlx5_db { diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h index 2c09df4ee574..5478435506c3 100644 --- a/include/linux/mlx5/mlx5_ifc.h +++ b/include/linux/mlx5/mlx5_ifc.h @@ -311,6 +311,8 @@ enum { MLX5_CMD_OP_CREATE_UMEM = 0xa08, MLX5_CMD_OP_DESTROY_UMEM = 0xa0a, MLX5_CMD_OP_SYNC_STEERING = 0xb00, + MLX5_CMD_OP_PSP_GEN_SPI = 0xb10, + MLX5_CMD_OP_PSP_ROTATE_KEY = 0xb11, MLX5_CMD_OP_QUERY_VHCA_STATE = 0xb0d, MLX5_CMD_OP_MODIFY_VHCA_STATE = 0xb0e, MLX5_CMD_OP_SYNC_CRYPTO = 0xb12, @@ -481,12 +483,14 @@ struct mlx5_ifc_flow_table_prop_layout_bits { u8 execute_aso[0x1]; u8 reserved_at_47[0x19]; - u8 reserved_at_60[0x2]; + u8 reformat_l2_to_l3_psp_tunnel[0x1]; + u8 reformat_l3_psp_tunnel_to_l2[0x1]; u8 reformat_insert[0x1]; u8 reformat_remove[0x1]; u8 macsec_encrypt[0x1]; u8 macsec_decrypt[0x1]; - u8 reserved_at_66[0x2]; + u8 psp_encrypt[0x1]; + u8 psp_decrypt[0x1]; u8 reformat_add_macsec[0x1]; u8 reformat_remove_macsec[0x1]; u8 reparse[0x1]; @@ -688,7 +692,7 @@ struct mlx5_ifc_fte_match_set_misc2_bits { u8 metadata_reg_a[0x20]; - u8 reserved_at_1a0[0x8]; + u8 psp_syndrome[0x8]; u8 macsec_syndrome[0x8]; u8 ipsec_syndrome[0x8]; @@ -1497,6 +1501,19 @@ struct mlx5_ifc_macsec_cap_bits { u8 reserved_at_40[0x7c0]; }; +struct mlx5_ifc_psp_cap_bits { + u8 reserved_at_0[0x1]; + u8 psp_crypto_offload[0x1]; /* Set by the driver */ + u8 reserved_at_2[0x1]; + u8 psp_crypto_esp_aes_gcm_256_encrypt[0x1]; + u8 psp_crypto_esp_aes_gcm_128_encrypt[0x1]; + u8 psp_crypto_esp_aes_gcm_256_decrypt[0x1]; + u8 psp_crypto_esp_aes_gcm_128_decrypt[0x1]; + u8 reserved_at_7[0x4]; + u8 log_max_num_of_psp_spi[0x5]; + u8 reserved_at_10[0x7f0]; +}; + enum { MLX5_WQ_TYPE_LINKED_LIST = 0x0, MLX5_WQ_TYPE_CYCLIC = 0x1, @@ -1636,7 +1653,7 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 reg_c_preserve[0x1]; u8 reserved_at_aa[0x1]; u8 log_max_srq[0x5]; - u8 reserved_at_b0[0x1]; + u8 reserved_at_b0[0x1]; u8 uplink_follow[0x1]; u8 ts_cqe_to_dest_cqn[0x1]; u8 reserved_at_b3[0x6]; @@ -1859,7 +1876,8 @@ struct mlx5_ifc_cmd_hca_cap_bits { u8 reserved_at_280[0x10]; u8 max_wqe_sz_sq[0x10]; - u8 reserved_at_2a0[0xb]; + u8 reserved_at_2a0[0xa]; + u8 psp[0x1]; u8 shampo[0x1]; u8 reserved_at_2ac[0x4]; u8 max_wqe_sz_rq[0x10]; @@ -3772,6 +3790,7 @@ union mlx5_ifc_hca_cap_union_bits { struct mlx5_ifc_macsec_cap_bits macsec_cap; struct mlx5_ifc_crypto_cap_bits crypto_cap; struct mlx5_ifc_ipsec_cap_bits ipsec_cap; + struct mlx5_ifc_psp_cap_bits psp_cap; u8 reserved_at_0[0x8000]; }; @@ -3801,6 +3820,7 @@ enum { enum { MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_IPSEC = 0x0, MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_MACSEC = 0x1, + MLX5_FLOW_CONTEXT_ENCRYPT_DECRYPT_TYPE_PSP = 0x2, }; struct mlx5_ifc_vlan_bits { @@ -7094,6 +7114,8 @@ enum mlx5_reformat_ctx_type { MLX5_REFORMAT_TYPE_DEL_ESP_TRANSPORT_OVER_UDP = 0xa, MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_IPV6 = 0xb, MLX5_REFORMAT_TYPE_ADD_ESP_TRANSPORT_OVER_UDPV6 = 0xc, + MLX5_REFORMAT_TYPE_ADD_PSP_TUNNEL = 0xd, + MLX5_REFORMAT_TYPE_DEL_PSP_TUNNEL = 0xe, MLX5_REFORMAT_TYPE_INSERT_HDR = 0xf, MLX5_REFORMAT_TYPE_REMOVE_HDR = 0x10, MLX5_REFORMAT_TYPE_ADD_MACSEC = 0x11, @@ -7220,6 +7242,7 @@ enum { MLX5_ACTION_IN_FIELD_IPSEC_SYNDROME = 0x5D, MLX5_ACTION_IN_FIELD_OUT_EMD_47_32 = 0x6F, MLX5_ACTION_IN_FIELD_OUT_EMD_31_0 = 0x70, + MLX5_ACTION_IN_FIELD_PSP_SYNDROME = 0x71, }; struct mlx5_ifc_alloc_modify_header_context_out_bits { @@ -12899,6 +12922,7 @@ enum { MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_TLS = 0x1, MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_IPSEC = 0x2, MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_MACSEC = 0x4, + MLX5_GENERAL_OBJECT_TYPE_ENCRYPTION_KEY_PURPOSE_PSP = 0x6, }; struct mlx5_ifc_tls_static_params_bits { @@ -13279,4 +13303,64 @@ struct mlx5_ifc_mrtcq_reg_bits { u8 reserved_at_80[0x180]; }; +struct mlx5_ifc_psp_rotate_key_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + + u8 reserved_at_40[0x40]; +}; + +struct mlx5_ifc_psp_rotate_key_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + + u8 syndrome[0x20]; + + u8 reserved_at_40[0x40]; +}; + +enum mlx5_psp_gen_spi_in_key_size { + MLX5_PSP_GEN_SPI_IN_KEY_SIZE_128 = 0x0, + MLX5_PSP_GEN_SPI_IN_KEY_SIZE_256 = 0x1, +}; + +struct mlx5_ifc_key_spi_bits { + u8 spi[0x20]; + + u8 reserved_at_20[0x60]; + + u8 key[8][0x20]; +}; + +struct mlx5_ifc_psp_gen_spi_in_bits { + u8 opcode[0x10]; + u8 uid[0x10]; + + u8 reserved_at_20[0x10]; + u8 op_mod[0x10]; + + u8 reserved_at_40[0x20]; + + u8 key_size[0x2]; + u8 reserved_at_62[0xe]; + u8 num_of_spi[0x10]; +}; + +struct mlx5_ifc_psp_gen_spi_out_bits { + u8 status[0x8]; + u8 reserved_at_8[0x18]; + + u8 syndrome[0x20]; + + u8 reserved_at_40[0x10]; + u8 num_of_spi[0x10]; + + u8 reserved_at_60[0x20]; + + struct mlx5_ifc_key_spi_bits key_spi[0]; +}; + #endif /* MLX5_IFC_H */ -- 2.47.1