From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pdx-out-008.esa.us-west-2.outbound.mail-perimeter.amazon.com (pdx-out-008.esa.us-west-2.outbound.mail-perimeter.amazon.com [52.42.203.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D6CE37104C for ; Thu, 9 Apr 2026 07:49:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.42.203.116 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775720969; cv=none; b=L/YHHCaaljJWSfpfkwBVKK7mpW2OJCWdp2Z558snUv6LaA+C5TjUdHhEEqUnReko19rgkEHDHdgoakDLLvRguynaVTI60V1iSfl7J3rUBp5KqyNDDmMxWURM5ZGTcfgqku5o1ofHiIArge/ufF4l000jNEuaKTSnQUvFqwLnKcw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775720969; c=relaxed/simple; bh=nh0c8nnTcc31TM7DBAptWGaP/TkBB5P9ZLKwKVX9YbE=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=Ab9E8hgxyi3MYaFCneHDpIJ1z8az1IKIQbouvrRdadJSi1MFJXgaqiqW43Sjhc8+lm12xiDSKwOxQ9pFEvxr1l4XsxSoJ4xmBOsmacelVMu7AT81+eiuxXh/sozvrET0RsIRKODWPT9Wb6ixaGUs2QnzJvAx+iaVn6VCQwxddU4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.com; dkim=pass (2048-bit key) header.d=amazon.com header.i=@amazon.com header.b=KRFxFHjX; arc=none smtp.client-ip=52.42.203.116 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=amazon.com header.i=@amazon.com header.b="KRFxFHjX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazoncorp2; t=1775720968; x=1807256968; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=/BqNTm6lfwztRwTvMUVyQiZ3Y+aB3PrSsmeBTv64UeI=; b=KRFxFHjXjANIlYhm4Mqu5RoYBHnFETSbOU1DetxnnVUy4dCi8LcfIwaN FyaQB71+ED+ox9GGUpm2VQ9A9ayuQACqiPWQ1gHsC7p8WGYN5oNR9mM5W CbDIczcVD2dCuWyuqBSy7VEZILPyrLjKFYnKPYsKWMbo7XKF2c8IOzntV 8Xm79J3N2upJIOprZ8EtbR4eJuP9eDrVI7jIv2UgeWCdrcB/SGNFzmO1K ZIFPrfnzXVf1mzzLSgTf2sQL9IsnUgvO5uJIkDPthWFGE1derJhHtMonP ROzW7ohEpQwwbzPp6ercx1w6mn9abHy58W3SQS5HMEvXSt+nG0Tq9UNST A==; X-CSE-ConnectionGUID: kbYWEwjzSvatYhMlJh6DfA== X-CSE-MsgGUID: KhWeNnAuR7+XB1+wdewn6Q== X-IronPort-AV: E=Sophos;i="6.23,169,1770595200"; d="scan'208";a="16912032" Received: from ip-10-5-6-203.us-west-2.compute.internal (HELO smtpout.naws.us-west-2.prod.farcaster.email.amazon.dev) ([10.5.6.203]) by internal-pdx-out-008.esa.us-west-2.outbound.mail-perimeter.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Apr 2026 07:49:23 +0000 Received: from EX19MTAUWA002.ant.amazon.com [205.251.233.178:30570] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.60.232:2525] with esmtp (Farcaster) id fdc10c4b-b1c9-4de8-88c8-13ae3571319f; Thu, 9 Apr 2026 07:49:22 +0000 (UTC) X-Farcaster-Flow-ID: fdc10c4b-b1c9-4de8-88c8-13ae3571319f Received: from EX19D001UWA001.ant.amazon.com (10.13.138.214) by EX19MTAUWA002.ant.amazon.com (10.250.64.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.37; Thu, 9 Apr 2026 07:49:21 +0000 Received: from dev-dsk-ynachum-1b-0ecf7b87.eu-west-1.amazon.com (10.13.226.176) by EX19D001UWA001.ant.amazon.com (10.13.138.214) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.37; Thu, 9 Apr 2026 07:49:19 +0000 From: Yonatan Nachum To: , , CC: , , , , Yonatan Nachum , Firas Jahjah Subject: [PATCH for-next] RDMA/efa: Add checksum support for admin responses Date: Thu, 9 Apr 2026 07:49:05 +0000 Message-ID: <20260409074905.3126023-1-ynachum@amazon.com> X-Mailer: git-send-email 2.50.1 Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: EX19D033UWC004.ant.amazon.com (10.13.139.225) To EX19D001UWA001.ant.amazon.com (10.13.138.214) EFA devices added support for CRC16 checksum on admin responses and to expose it to the driver the API version increased to 0.2. Add a check for support on device init and if supported validate the checksum on each admin response the driver receives. If the checksum validation failed, drop the CQE. Add the CRC16 module to Kconfig to have the in-tree dependency. Reviewed-by: Firas Jahjah Reviewed-by: Michael Margolin Signed-off-by: Yonatan Nachum --- drivers/infiniband/hw/efa/Kconfig | 3 +- .../infiniband/hw/efa/efa_admin_cmds_defs.h | 3 -- drivers/infiniband/hw/efa/efa_admin_defs.h | 15 +++--- drivers/infiniband/hw/efa/efa_com.c | 50 ++++++++++++++++--- drivers/infiniband/hw/efa/efa_com.h | 4 +- 5 files changed, 55 insertions(+), 20 deletions(-) diff --git a/drivers/infiniband/hw/efa/Kconfig b/drivers/infiniband/hw/efa/Kconfig index 457e18ba1d57..ff7f7c0870b3 100644 --- a/drivers/infiniband/hw/efa/Kconfig +++ b/drivers/infiniband/hw/efa/Kconfig @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause -# Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All rights reserved. +# Copyright 2018-2026 Amazon.com, Inc. or its affiliates. All rights reserved. # # Amazon fabric device configuration # @@ -8,6 +8,7 @@ config INFINIBAND_EFA tristate "Amazon Elastic Fabric Adapter (EFA) support" depends on PCI_MSI && 64BIT && !CPU_BIG_ENDIAN depends on INFINIBAND_USER_ACCESS + select CRC16 help This driver supports Amazon Elastic Fabric Adapter (EFA). diff --git a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h index 57178dad5eb7..dd9bfcabe8c4 100644 --- a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h +++ b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h @@ -6,9 +6,6 @@ #ifndef _EFA_ADMIN_CMDS_H_ #define _EFA_ADMIN_CMDS_H_ -#define EFA_ADMIN_API_VERSION_MAJOR 0 -#define EFA_ADMIN_API_VERSION_MINOR 1 - /* EFA admin queue opcodes */ enum efa_admin_aq_opcode { EFA_ADMIN_CREATE_QP = 1, diff --git a/drivers/infiniband/hw/efa/efa_admin_defs.h b/drivers/infiniband/hw/efa/efa_admin_defs.h index 35700c93e639..02f86edabed8 100644 --- a/drivers/infiniband/hw/efa/efa_admin_defs.h +++ b/drivers/infiniband/hw/efa/efa_admin_defs.h @@ -1,18 +1,20 @@ /* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ /* - * Copyright 2018-2024 Amazon.com, Inc. or its affiliates. All rights reserved. + * Copyright 2018-2026 Amazon.com, Inc. or its affiliates. All rights reserved. */ #ifndef _EFA_ADMIN_H_ #define _EFA_ADMIN_H_ +#define EFA_ADMIN_API_VERSION_MAJOR 0 +#define EFA_ADMIN_API_VERSION_MINOR 2 + enum efa_admin_aq_completion_status { EFA_ADMIN_SUCCESS = 0, EFA_ADMIN_RESOURCE_ALLOCATION_FAILURE = 1, EFA_ADMIN_BAD_OPCODE = 2, EFA_ADMIN_UNSUPPORTED_OPCODE = 3, EFA_ADMIN_MALFORMED_REQUEST = 4, - /* Additional status is provided in ACQ entry extended_status */ EFA_ADMIN_ILLEGAL_PARAMETER = 5, EFA_ADMIN_UNKNOWN_ERROR = 6, EFA_ADMIN_RESOURCE_BUSY = 7, @@ -78,13 +80,10 @@ struct efa_admin_acq_common_desc { */ u8 flags; - u16 extended_status; + /* Poly 0x8005 CRC16 with initial value 0xFFFF and final XOR of 0xFFFF */ + u16 checksum; - /* - * indicates to the driver which AQ entry has been consumed by the - * device and could be reused - */ - u16 sq_head_indx; + u16 reserved; }; struct efa_admin_acq_entry { diff --git a/drivers/infiniband/hw/efa/efa_com.c b/drivers/infiniband/hw/efa/efa_com.c index e97b5f0d7003..7cc3f4af0bb9 100644 --- a/drivers/infiniband/hw/efa/efa_com.c +++ b/drivers/infiniband/hw/efa/efa_com.c @@ -3,6 +3,7 @@ * Copyright 2018-2026 Amazon.com, Inc. or its affiliates. All rights reserved. */ +#include #include #include "efa_com.h" @@ -22,6 +23,14 @@ #define EFA_CTRL_MINOR 0 #define EFA_CTRL_SUB_MINOR 1 +#define EFA_CRC16_INIT_VAL 0xffff + +#define EFA_CRC_MIN_ADMIN_API_VERSION_MAJOR 0 +#define EFA_CRC_MIN_ADMIN_API_VERSION_MINOR 2 + +#define EFA_MIN_ADMIN_API_VERSION_MAJOR 0 +#define EFA_MIN_ADMIN_API_VERSION_MINOR 1 + enum efa_cmd_status { EFA_CMD_UNUSED, EFA_CMD_ALLOCATED, @@ -167,9 +176,8 @@ static int efa_com_admin_init_cq(struct efa_com_dev *edev) struct efa_com_admin_queue *aq = &edev->aq; struct efa_com_admin_cq *cq = &aq->cq; u16 size = aq->depth * sizeof(*cq->entries); - u32 acq_caps = 0; - u32 addr_high; - u32 addr_low; + u32 acq_caps = 0, crc_min_ver = 0; + u32 addr_high, addr_low; cq->entries = dma_alloc_coherent(aq->dmadev, size, &cq->dma_addr, GFP_KERNEL); @@ -178,6 +186,11 @@ static int efa_com_admin_init_cq(struct efa_com_dev *edev) spin_lock_init(&cq->lock); + EFA_SET(&crc_min_ver, EFA_REGS_VERSION_MAJOR_VERSION, EFA_CRC_MIN_ADMIN_API_VERSION_MAJOR); + EFA_SET(&crc_min_ver, EFA_REGS_VERSION_MINOR_VERSION, EFA_CRC_MIN_ADMIN_API_VERSION_MINOR); + if (edev->dev_api_ver >= crc_min_ver) + cq->validate_checksum = true; + cq->cc = 0; cq->phase = 1; @@ -409,12 +422,35 @@ static int efa_com_submit_admin_cmd(struct efa_com_admin_queue *aq, return 0; } +static bool efa_com_cqe_checksum_valid(struct efa_com_admin_queue *aq, + struct efa_admin_acq_entry *cqe) +{ + u16 cqe_checksum = cqe->acq_common_descriptor.checksum; + u16 calc_checksum; + + cqe->acq_common_descriptor.checksum = 0; + + calc_checksum = crc16(EFA_CRC16_INIT_VAL, (u8 *)cqe, sizeof(*cqe)) ^ EFA_CRC16_INIT_VAL; + if (calc_checksum != cqe_checksum) { + ibdev_err(aq->efa_dev, + "Received completion with invalid checksum, cqe[%u], calc[%u], sq producer[%d], sq consumer[%d], cq consumer[%d]\n", + cqe_checksum, calc_checksum, aq->sq.pc, aq->sq.cc, + aq->cq.cc); + return false; + } + + return true; +} + static int efa_com_handle_single_admin_completion(struct efa_com_admin_queue *aq, struct efa_admin_acq_entry *cqe) { struct efa_comp_ctx *comp_ctx; u16 cmd_id; + if (aq->cq.validate_checksum && !efa_com_cqe_checksum_valid(aq, cqe)) + return -EINVAL; + cmd_id = EFA_GET(&cqe->acq_common_descriptor.command, EFA_ADMIN_ACQ_COMMON_DESC_COMMAND_ID); @@ -954,16 +990,16 @@ int efa_com_validate_version(struct efa_com_dev *edev) EFA_GET(&ver, EFA_REGS_VERSION_MAJOR_VERSION), EFA_GET(&ver, EFA_REGS_VERSION_MINOR_VERSION)); - EFA_SET(&min_ver, EFA_REGS_VERSION_MAJOR_VERSION, - EFA_ADMIN_API_VERSION_MAJOR); - EFA_SET(&min_ver, EFA_REGS_VERSION_MINOR_VERSION, - EFA_ADMIN_API_VERSION_MINOR); + EFA_SET(&min_ver, EFA_REGS_VERSION_MAJOR_VERSION, EFA_MIN_ADMIN_API_VERSION_MAJOR); + EFA_SET(&min_ver, EFA_REGS_VERSION_MINOR_VERSION, EFA_MIN_ADMIN_API_VERSION_MINOR); if (ver < min_ver) { ibdev_err(edev->efa_dev, "EFA version is lower than the minimal version the driver supports\n"); return -EOPNOTSUPP; } + edev->dev_api_ver = ver; + ibdev_dbg( edev->efa_dev, "efa controller version: %d.%d.%d implementation version %d\n", diff --git a/drivers/infiniband/hw/efa/efa_com.h b/drivers/infiniband/hw/efa/efa_com.h index 4d9ca97e4296..f8c692b0e092 100644 --- a/drivers/infiniband/hw/efa/efa_com.h +++ b/drivers/infiniband/hw/efa/efa_com.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */ /* - * Copyright 2018-2025 Amazon.com, Inc. or its affiliates. All rights reserved. + * Copyright 2018-2026 Amazon.com, Inc. or its affiliates. All rights reserved. */ #ifndef _EFA_COM_H_ @@ -25,6 +25,7 @@ struct efa_com_admin_cq { struct efa_admin_acq_entry *entries; dma_addr_t dma_addr; spinlock_t lock; /* Protects ACQ */ + bool validate_checksum; u16 cc; /* consumer counter */ u8 phase; @@ -112,6 +113,7 @@ struct efa_com_dev { u32 supported_features; u32 dma_addr_bits; + u32 dev_api_ver; struct efa_com_mmio_read mmio_read; }; -- 2.50.1