* Re: [PATCH RFC 01/18] accel/qda: Add Qualcomm QDA DSP accelerator driver docs
From: Ekansh Gupta @ 2026-04-02 8:41 UTC (permalink / raw)
To: Trilok Soni, Dmitry Baryshkov
Cc: Bjorn Andersson, Oded Gabbay, Jonathan Corbet, Shuah Khan,
Joerg Roedel, Will Deacon, Robin Murphy, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Sumit Semwal, Christian König, dri-devel, linux-doc,
linux-kernel, linux-arm-msm, iommu, linux-media, linaro-mm-sig,
Srinivas Kandagatla, Bharath Kumar, Chenna Kesava Raju
In-Reply-To: <3f06453a-ac7e-46e0-8d37-e0f9980b438d@oss.qualcomm.com>
On 2/26/2026 4:48 AM, Trilok Soni wrote:
> On 2/25/2026 11:40 AM, Dmitry Baryshkov wrote:
>> On Wed, Feb 25, 2026 at 11:16:26AM -0800, Trilok Soni wrote:
>>> On 2/25/2026 7:12 AM, Bjorn Andersson wrote:
>>>> On Wed, Feb 25, 2026 at 07:47:08PM +0530, Ekansh Gupta wrote:
>>>>>
>>>>> On 2/24/2026 9:03 AM, Trilok Soni wrote:
>>>>>> On 2/23/2026 11:08 AM, Ekansh Gupta wrote:
>>>>>>> Add initial documentation for the Qualcomm DSP Accelerator (QDA) driver
>>>>>>> integrated in the DRM accel subsystem.
>>>>>>>
>>>>>>> The new docs introduce QDA as a DRM/accel-based implementation of
>>>>>>> Hexagon DSP offload that is intended as a modern alternative to the
>>>>>>> legacy FastRPC driver in drivers/misc. The text describes the driver
>>>>>>> motivation, high-level architecture and interaction with IOMMU context
>>>>>>> banks, GEM-based buffer management and the RPMsg transport.
>>>>>>>
>>>>>>> The user-space facing section documents the main QDA IOCTLs used to
>>>>>>> establish DSP sessions, manage GEM buffer objects and invoke remote
>>>>>>> procedures using the FastRPC protocol, along with a typical lifecycle
>>>>>>> example for applications.
>>>>>>>
>>>>>>> Finally, the driver is wired into the Compute Accelerators
>>>>>>> documentation index under Documentation/accel, and a brief debugging
>>>>>>> section shows how to enable dynamic debug for the QDA implementation.
>>>>>> So existing applications written over character device UAPI needs to be
>>>>>> rewritten over new UAPI and it will be broken once this driver gets
>>>>>> merged? Are we going to keep both the drivers in the Linux kernel
>>>>>> and not deprecate the /char device one?
>>>>>>
>>>>>> Is Qualcomm going to provide the wrapper library in the userspace
>>>>>> so that existing applications by our customers and developers
>>>>>> keep working w/ the newer kernel if the char interface based
>>>>>> driver gets deprecated? It is not clear from your text above.
>>>>> Thanks for raising this, Trilok.
>>>>>
>>>>> This is one of the open items that I have. I'm not exactly sure what would be the
>>>>> acceptable way for this.
>>>>>
>>>>> As you mentioned, applications that rely on /dev/fastrpc* might not work on QDA
>>>>> without modification.
>>>>>
>>>>> I was thinking in the same lines as you have mentioned and having some shim/compat
>>>>> driver to translate FastRPC UAPI to QDA. The compat driver would expose the existing
>>>>> character devices and route the calls to QDA. The compat driver could be built via Kconfig.
>>>>>
>>>> This is a fundamental requirement, you need to address this in order for
>>>> this to move forward.
>>>>
>>>> Which makes me wonder if it would be possible to reach an accel driver
>>>> through incremental transition of the current driver, instead of just
>>>> dropping in a few thousand lines of new code/design.
>>>>
>>>>> However, I haven’t encountered an example of such a UAPI‑translation driver in the kernel
>>>>> before, so I would want guidance from maintainers on whether this is an acceptable
>>>>> model or not.
>>>>>
>>>>> Regarding your question about library, all the APIs exposed by github/fastrpc library are kept
>>>>> unchanged in terms of definitions and expectation. The same project can be build for both
>>>>> FastRPC and QDA based on configure options. So, the applications using github/fastrpc should
>>>>> not face any problem if the libs is built with proper configure options.
>>>>>
>>>> You're assuming that the kernel and userspace are a unified piece of
>>>> software, they are not. It must be possible for me to install a new
>>>> kernel package without having to replace the userspace libraries.
>>> Thank you Bjorn for providing the inputs.
>>>
>>> I also foresee that we will be stop adding (or already happened) new features
>>> into the existing fastrpc driver, so calling the new driver as an alternative
>>> is in oversold category.
>>>
>>> You are pretty much began the deprecating the existing fastrpc driver, so let's
>>> just mention it if that is the case and provide migration/shim path so that
>>> existing binaries doesn't break.
>> I agree that we need a migration path, but I'd really focus on it after
>> getting at least basic parts of the QDA reviewed and agreed upon.
>> Otherwise the shim layer will be reworked again and again with no
>> immediate added benefit.
>>
> I am fine with the review to be continued, this is RFC series anyway. We should also decide
> the design of the shim layer here as well. I prefer to not have multiple
> RFC revisions here if we don't agree on the basic requirements which
> leads to acceptance of this new driver.
Just wanted to provide an update here, I'm currently working on a new
version of this driver with majority of comments addressed.
I'm thinking of including limited functionalities (say, init, gem_alloc, invoke)
as of now along with a minimal compat/shim driver.
The compat driver is currently planned in the same drivers/accel/qda path which
will be exposing same interfaces(device nodes and ioctls) as fastrpc and simply
route the calls to QDA. Please let me know if you see any concerns with this. I
can rework my approach before sending the v1 of actual patch series.
//Ekansh
>
> ---Trilok Soni
^ permalink raw reply
* Re: [PATCH RFC 12/18] accel/qda: Add PRIME dma-buf import support
From: Christian König @ 2026-04-02 8:51 UTC (permalink / raw)
To: Ekansh Gupta, Oded Gabbay, Jonathan Corbet, Shuah Khan,
Joerg Roedel, Will Deacon, Robin Murphy, Maarten Lankhorst,
Maxime Ripard, Thomas Zimmermann, David Airlie, Simona Vetter,
Sumit Semwal
Cc: dri-devel, linux-doc, linux-kernel, linux-arm-msm, iommu,
linux-media, linaro-mm-sig, Srinivas Kandagatla, Dmitry Baryshkov,
Bharath Kumar, Chenna Kesava Raju
In-Reply-To: <998ce121-e027-441d-a3f4-2f3e41e10830@oss.qualcomm.com>
On 4/2/26 10:36, Ekansh Gupta wrote:
> On 3/9/2026 12:29 PM, Ekansh Gupta wrote:
>>
>> On 2/24/2026 2:42 PM, Christian König wrote:
>>> On 2/23/26 20:09, Ekansh Gupta wrote:
>>>> [Sie erhalten nicht häufig E-Mails von ekansh.gupta@oss.qualcomm.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]
>>>>
>>>> Add PRIME dma-buf import support for QDA GEM buffer objects and integrate
>>>> it with the existing per-process memory manager and IOMMU device model.
>>>>
>>>> The implementation extends qda_gem_obj to represent imported dma-bufs,
>>>> including dma_buf references, attachment state, scatter-gather tables
>>>> and an imported DMA address used for DSP-facing book-keeping. The
>>>> qda_gem_prime_import() path handles reimports of buffers originally
>>>> exported by QDA as well as imports of external dma-bufs, attaching them
>>>> to the assigned IOMMU device
>>> That is usually an absolutely clear NO-GO for DMA-bufs. Where exactly in the code is that?
>> dma_buf_attach* to comute-cb iommu devices are critical for DSPs to access the buffer.
>> This is needed if the buffer is exported by anyone other than QDA(say system heap). If this is not
>> the correct way, what should be the right way here? On the current fastrpc driver also,
>> the DMABUF is getting attached with iommu device[1] due to the same requirement.
>>
>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/misc/fastrpc.c#n779
>
> Hi Christian,
>
> Do you have any suggestions for the shared requirements?
Well I don't fully understand what you are trying to do with the iommu. Usually it is the job of the exporter to provide the importer with DMA addresses which are valid for its device structure, that includes IOMMU mapping.
Can you iterate what exactly this iommu group is and why you have to attach the imported buffers to it, how that attachment works and how lifetime is managed?
Regards,
Christian.
>
> I'm reworking on the next version and currently I don't see any other way
> to handle dma_buf_attach* cases.
>
> //Ekansh
>
>>>> and mapping them through the memory manager
>>>> for DSP access. The GEM free path is updated to unmap and detach
>>>> imported buffers while preserving the existing behaviour for locally
>>>> allocated memory.
>>>>
>>>> The PRIME fd-to-handle path is implemented in qda_prime_fd_to_handle(),
>>>> which records the calling drm_file in a driver-private import context
>>>> before invoking the core DRM helpers. The GEM import callback retrieves
>>>> this context to ensure that an IOMMU device is assigned to the process
>>>> and that imported buffers follow the same per-process IOMMU selection
>>>> rules as natively allocated GEM objects.
>>>>
>>>> This patch prepares the driver for interoperable buffer sharing between
>>>> QDA and other dma-buf capable subsystems while keeping IOMMU mapping and
>>>> lifetime handling consistent with the existing GEM allocation flow.
>>>>
>>>> Signed-off-by: Ekansh Gupta <ekansh.gupta@oss.qualcomm.com>
>>> ...
>>>
>>>> @@ -15,23 +16,29 @@ static int validate_gem_obj_for_mmap(struct qda_gem_obj *qda_gem_obj)
>>>> qda_err(NULL, "Invalid GEM object size\n");
>>>> return -EINVAL;
>>>> }
>>>> - if (!qda_gem_obj->iommu_dev || !qda_gem_obj->iommu_dev->dev) {
>>>> - qda_err(NULL, "Allocated buffer missing IOMMU device\n");
>>>> - return -EINVAL;
>>>> - }
>>>> - if (!qda_gem_obj->iommu_dev->dev) {
>>>> - qda_err(NULL, "Allocated buffer missing IOMMU device\n");
>>>> - return -EINVAL;
>>>> - }
>>>> - if (!qda_gem_obj->virt) {
>>>> - qda_err(NULL, "Allocated buffer missing virtual address\n");
>>>> - return -EINVAL;
>>>> - }
>>>> - if (qda_gem_obj->dma_addr == 0) {
>>>> - qda_err(NULL, "Allocated buffer missing DMA address\n");
>>>> - return -EINVAL;
>>>> + if (qda_gem_obj->is_imported) {
>>> Absolutely clear NAK to that. Imported buffers *can't* be mmaped through the importer!
>>>
>>> Userspace needs to mmap() them through the exporter.
>>>
>>> If you absolutely have to map them through the importer for uAPI backward compatibility then there is dma_buf_mmap() for that, but this is clearly not the case here.
>>>
>>> ...
>> Okay, the requirement is slightly different here. Any buffer which is not allocated using the
>> QDA GEM interface needs to be attached to the iommu device for that particular process to
>> enable DSP for the access. I should not call it `mmap` instead it should be called importing the
>> buffer to a particular iommu context bank. With this definition, is it fine to keep it this way? Or
>> should the dma_buf_attach* calls be moved to some other place?
>>>> +static int qda_memory_manager_map_imported(struct qda_memory_manager *mem_mgr,
>>>> + struct qda_gem_obj *gem_obj,
>>>> + struct qda_iommu_device *iommu_dev)
>>>> +{
>>>> + struct scatterlist *sg;
>>>> + dma_addr_t dma_addr;
>>>> + int ret = 0;
>>>> +
>>>> + if (!gem_obj->is_imported || !gem_obj->sgt || !iommu_dev) {
>>>> + qda_err(NULL, "Invalid parameters for imported buffer mapping\n");
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + gem_obj->iommu_dev = iommu_dev;
>>>> +
>>>> + sg = gem_obj->sgt->sgl;
>>>> + if (sg) {
>>>> + dma_addr = sg_dma_address(sg);
>>>> + dma_addr += ((u64)iommu_dev->sid << 32);
>>>> +
>>>> + gem_obj->imported_dma_addr = dma_addr;
>>> Well that looks like you are only using the first DMA address from the imported sgt. What about the others?
>> I might have a proper appach for this now, will update in the next spin.
>>> Regards,
>>> Christian.
>
^ permalink raw reply
* Re: [PATCH v10 net-next 02/11] net/nebula-matrix: add our driver architecture
From: Mohsin Bashir @ 2026-04-02 9:02 UTC (permalink / raw)
To: illusion.wang, dimon.zhao, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, linux-doc, lorenzo, pabeni, horms,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, jani.nikula, open list
In-Reply-To: <20260401022318.28550-3-illusion.wang@nebula-matrix.com>
On 3/31/26 7:23 PM, illusion.wang wrote:
> our driver architecture:
> Hardware (HW), Channel, Resource, Dispatch, and Device Layer
> Struct Initialization/Deinitialization, and Operation Set Registration/
> Unregistration
>
> Our driver architecture is relatively complex because the code is highly
> reusable and designed to support multiple features. Additionally, the
> codebase supports multiple chip variants, each with distinct
> hardware-software interactions.
> To ensure compatibility, our architecture is divided into the following
> layers:
>
> 1. Dev Layer (Device Layer)
> The top-level business logic layer where all operations are
> device-centric. Every operation is performed relative to the device
> context. The intergration of base functions encompasses:
> management(ctrl only for leonis pf0), network(net_dev,this time not
> contained),common.
>
> 2. Dispatch Layer
> The distribution from services to specific data operations is mainly
> divided into two types: direct pass-through and handling by the
> management PF. It shields the upper layer from the differences in
> specific underlying locations.
> It describes the processing locations and paths of the services.
>
> 3. Resource Layer
> Handles tasks dispatched from Dispatch Layer. These tasks fall into two
> categories:
> 3.1 Hardware control
> The Resource Layer further invokes the HW Layer when hardware access is
> needed, as only the HW Layer has OS-level privileges.
> 3.2 Software resource management
> Operations like packet statistics collection that don't require hardware
> access.
>
> 4. HW Layer (Hardware Layer)
> Serves the Resource Layer by interacting with different hardware
> chipsets.Writes to hardware registers to drive the hardware based on
> Resource Layer directives.
>
> 5. Channel Layer
>
> Handle communication between PF0(has ctrl func) and other PF,and provide
> basic interaction channels.
>
> 6. Common Layer
> Provides fundamental services
>
> Signed-off-by: illusion.wang <illusion.wang@nebula-matrix.com>
> ---
> .../net/ethernet/nebula-matrix/nbl/Makefile | 7 +-
> .../nbl/nbl_channel/nbl_channel.c | 82 ++++++++
> .../nbl/nbl_channel/nbl_channel.h | 29 +++
> .../net/ethernet/nebula-matrix/nbl/nbl_core.h | 43 +++++
> .../nebula-matrix/nbl/nbl_core/nbl_dev.c | 56 ++++++
> .../nebula-matrix/nbl/nbl_core/nbl_dev.h | 27 +++
> .../nebula-matrix/nbl/nbl_core/nbl_dispatch.c | 75 ++++++++
> .../nebula-matrix/nbl/nbl_core/nbl_dispatch.h | 25 +++
> .../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 140 ++++++++++++++
> .../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 14 ++
> .../nbl_hw_leonis/nbl_resource_leonis.c | 83 ++++++++
> .../nbl_hw_leonis/nbl_resource_leonis.h | 10 +
> .../nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h | 68 +++++++
> .../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 30 +++
> .../nbl/nbl_include/nbl_def_channel.h | 26 +++
> .../nbl/nbl_include/nbl_def_common.h | 35 ++++
> .../nbl/nbl_include/nbl_def_dev.h | 16 ++
> .../nbl/nbl_include/nbl_def_dispatch.h | 29 +++
> .../nbl/nbl_include/nbl_def_hw.h | 22 +++
> .../nbl/nbl_include/nbl_def_resource.h | 20 ++
> .../nbl/nbl_include/nbl_include.h | 11 ++
> .../nbl/nbl_include/nbl_product_base.h | 18 ++
> .../net/ethernet/nebula-matrix/nbl/nbl_main.c | 179 ++++++++++++++++++
> 23 files changed, 1044 insertions(+), 1 deletion(-)
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_product_base.h
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> index b90fba239401..271605920396 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> @@ -3,4 +3,9 @@
>
> obj-$(CONFIG_NBL) := nbl.o
>
> -nbl-objs += nbl_main.o
> +nbl-objs += nbl_channel/nbl_channel.o \
> + nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
> + nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
> + nbl_core/nbl_dispatch.o \
> + nbl_core/nbl_dev.o \
> + nbl_main.o
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> new file mode 100644
> index 000000000000..9f4603870cb4
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> @@ -0,0 +1,82 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#include <linux/device.h>
> +#include <linux/pci.h>
> +#include "nbl_channel.h"
> +
> +static struct nbl_channel_ops chan_ops = {
> +};
> +
> +static struct nbl_channel_mgt *
> +nbl_chan_setup_chan_mgt(struct nbl_adapter *adapter)
> +{
> + struct nbl_hw_ops_tbl *hw_ops_tbl = adapter->intf.hw_ops_tbl;
> + struct nbl_common_info *common = &adapter->common;
> + struct device *dev = &adapter->pdev->dev;
> + struct nbl_chan_info *mailbox;
> + struct nbl_channel_mgt *chan_mgt;
Reverse xmas violation here
> +
> + chan_mgt = devm_kzalloc(dev, sizeof(*chan_mgt), GFP_KERNEL);
> + if (!chan_mgt)
> + return ERR_PTR(-ENOMEM);
> +
> + chan_mgt->common = common;
> + chan_mgt->hw_ops_tbl = hw_ops_tbl;
> +
> + mailbox = devm_kzalloc(dev, sizeof(*mailbox), GFP_KERNEL);
> + if (!mailbox)
> + return ERR_PTR(-ENOMEM);
Here, if mailbox allocation fails, we return without freeing chan_mgt
resulting in a leak.
> + mailbox->chan_type = NBL_CHAN_TYPE_MAILBOX;
> + chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX] = mailbox;
> +
> + return chan_mgt;
> +}
> +
> +static struct nbl_channel_ops_tbl *
> +nbl_chan_setup_ops(struct device *dev, struct nbl_channel_mgt *chan_mgt)
> +{
> + struct nbl_channel_ops_tbl *chan_ops_tbl;
> +
> + chan_ops_tbl = devm_kzalloc(dev, sizeof(*chan_ops_tbl), GFP_KERNEL);
> + if (!chan_ops_tbl)
> + return ERR_PTR(-ENOMEM);
> +
> + chan_ops_tbl->ops = &chan_ops;
> + chan_ops_tbl->priv = chan_mgt;
> +
> + return chan_ops_tbl;
> +}
> +
> +int nbl_chan_init_common(struct nbl_adapter *adap)
> +{
> + struct nbl_channel_ops_tbl *chan_ops_tbl;
> + struct device *dev = &adap->pdev->dev;
> + struct nbl_channel_mgt *chan_mgt;
> + int ret;
> +
> + chan_mgt = nbl_chan_setup_chan_mgt(adap);
> + if (IS_ERR(chan_mgt)) {
> + ret = PTR_ERR(chan_mgt);
> + goto setup_mgt_fail;
> + }
> + adap->core.chan_mgt = chan_mgt;
> +
> + chan_ops_tbl = nbl_chan_setup_ops(dev, chan_mgt);
> + if (IS_ERR(chan_ops_tbl)) {
> + ret = PTR_ERR(chan_ops_tbl);
> + goto setup_ops_fail;
> + }
> + adap->intf.channel_ops_tbl = chan_ops_tbl;
> + return 0;
> +
> +setup_ops_fail:
> +setup_mgt_fail:
> + return ret;
> +}
Why do we need two labels here? Looks like some piece of code will use
two different clean-up paths in the later patches. Better to add related
label then?
> +
> +void nbl_chan_remove_common(struct nbl_adapter *adap)
> +{
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> new file mode 100644
> index 000000000000..637912d1e806
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_CHANNEL_H_
> +#define _NBL_CHANNEL_H_
> +
> +#include <linux/types.h>
> +
> +#include "../nbl_include/nbl_include.h"
> +#include "../nbl_include/nbl_product_base.h"
> +#include "../nbl_include/nbl_def_channel.h"
> +#include "../nbl_include/nbl_def_hw.h"
> +#include "../nbl_include/nbl_def_common.h"
> +#include "../nbl_core.h"
> +
> +struct nbl_chan_info {
> + u8 chan_type;
> +};
> +
> +struct nbl_channel_mgt {
> + struct nbl_common_info *common;
> + struct nbl_hw_ops_tbl *hw_ops_tbl;
> + struct nbl_chan_info *chan_info[NBL_CHAN_TYPE_MAX];
> + struct nbl_hash_tbl_mgt *handle_hash_tbl;
> +};
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
> index c525114297b4..aa4d24d04e2c 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core.h
> @@ -6,6 +6,20 @@
> #ifndef _NBL_CORE_H_
> #define _NBL_CORE_H_
>
> +#include <linux/pci.h>
> +#include "nbl_include/nbl_def_common.h"
> +#include "nbl_include/nbl_include.h"
> +
> +struct nbl_hw_mgt;
> +struct nbl_hw_ops_tbl;
> +struct nbl_resource_mgt;
> +struct nbl_resource_ops_tbl;
> +struct nbl_dispatch_mgt;
> +struct nbl_dispatch_ops_tbl;
> +struct nbl_channel_ops_tbl;
> +struct nbl_channel_mgt;
> +struct nbl_dev_mgt;
> +
> enum {
> NBL_CAP_HAS_CTRL_BIT,
> NBL_CAP_HAS_NET_BIT,
> @@ -13,4 +27,33 @@ enum {
> NBL_CAP_IS_LEONIS_BIT,
> };
>
> +struct nbl_interface {
> + struct nbl_hw_ops_tbl *hw_ops_tbl;
> + struct nbl_resource_ops_tbl *resource_ops_tbl;
> + struct nbl_dispatch_ops_tbl *dispatch_ops_tbl;
> + struct nbl_channel_ops_tbl *channel_ops_tbl;
> +};
> +
> +struct nbl_core {
> + struct nbl_hw_mgt *hw_mgt;
> + struct nbl_resource_mgt *res_mgt;
> + struct nbl_dispatch_mgt *disp_mgt;
> + struct nbl_dev_mgt *dev_mgt;
> + struct nbl_channel_mgt *chan_mgt;
> +};
> +
> +struct nbl_adapter {
> + struct pci_dev *pdev;
> + struct nbl_core core;
> + struct nbl_interface intf;
> + struct nbl_common_info common;
> + struct nbl_product_base_ops *product_base_ops;
> +};
> +
> +struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
> + struct nbl_init_param *param);
> +void nbl_core_remove(struct nbl_adapter *adapter);
> +int nbl_core_start(struct nbl_adapter *adapter);
> +void nbl_core_stop(struct nbl_adapter *adapter);
> +
> #endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> new file mode 100644
> index 000000000000..5deb21e35f8e
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.c
> @@ -0,0 +1,56 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include <linux/pci.h>
> +#include "nbl_dev.h"
> +
> +static struct nbl_dev_mgt *nbl_dev_setup_dev_mgt(struct nbl_common_info *common)
> +{
> + struct nbl_dev_mgt *dev_mgt;
> +
> + dev_mgt = devm_kzalloc(common->dev, sizeof(*dev_mgt), GFP_KERNEL);
> + if (!dev_mgt)
> + return ERR_PTR(-ENOMEM);
> +
> + dev_mgt->common = common;
> + return dev_mgt;
> +}
> +
> +int nbl_dev_init(struct nbl_adapter *adapter)
> +{
> + struct nbl_common_info *common = &adapter->common;
> + struct nbl_dispatch_ops_tbl *disp_ops_tbl =
> + adapter->intf.dispatch_ops_tbl;
> + struct nbl_channel_ops_tbl *chan_ops_tbl =
> + adapter->intf.channel_ops_tbl;
> + struct nbl_dev_mgt *dev_mgt;
> + int ret;
> +
> + dev_mgt = nbl_dev_setup_dev_mgt(common);
> + if (IS_ERR(dev_mgt)) {
> + ret = PTR_ERR(dev_mgt);
> + return ret;
> + }
> +
> + dev_mgt->disp_ops_tbl = disp_ops_tbl;
> + dev_mgt->chan_ops_tbl = chan_ops_tbl;
> + adapter->core.dev_mgt = dev_mgt;
> +
> + return 0;
> +}
> +
> +void nbl_dev_remove(struct nbl_adapter *adapter)
> +{
> +}
> +
> +/* ---------- Dev start process ---------- */
> +int nbl_dev_start(struct nbl_adapter *adapter)
> +{
> + return 0;
> +}
> +
> +void nbl_dev_stop(struct nbl_adapter *adapter)
> +{
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
> new file mode 100644
> index 000000000000..9b71092b99a0
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dev.h
> @@ -0,0 +1,27 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEV_H_
> +#define _NBL_DEV_H_
> +
> +#include <linux/types.h>
> +
> +#include "../nbl_include/nbl_include.h"
> +#include "../nbl_include/nbl_product_base.h"
> +#include "../nbl_include/nbl_def_channel.h"
> +#include "../nbl_include/nbl_def_hw.h"
> +#include "../nbl_include/nbl_def_resource.h"
> +#include "../nbl_include/nbl_def_dispatch.h"
> +#include "../nbl_include/nbl_def_dev.h"
> +#include "../nbl_include/nbl_def_common.h"
> +#include "../nbl_core.h"
> +
> +struct nbl_dev_mgt {
> + struct nbl_common_info *common;
> + struct nbl_dispatch_ops_tbl *disp_ops_tbl;
> + struct nbl_channel_ops_tbl *chan_ops_tbl;
> +};
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> new file mode 100644
> index 000000000000..8b031f6e963f
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.c
> @@ -0,0 +1,75 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include <linux/pci.h>
> +#include "nbl_dispatch.h"
> +
> +static struct nbl_dispatch_mgt *
> +nbl_disp_setup_disp_mgt(struct nbl_common_info *common)
> +{
> + struct nbl_dispatch_mgt *disp_mgt;
> + struct device *dev = common->dev;
> +
> + disp_mgt = devm_kzalloc(dev, sizeof(*disp_mgt), GFP_KERNEL);
> + if (!disp_mgt)
> + return ERR_PTR(-ENOMEM);
> +
> + disp_mgt->common = common;
> + return disp_mgt;
> +}
> +
> +static struct nbl_dispatch_ops_tbl *
> +nbl_disp_setup_ops(struct device *dev, struct nbl_dispatch_mgt *disp_mgt)
> +{
> + struct nbl_dispatch_ops_tbl *disp_ops_tbl;
> + struct nbl_dispatch_ops *disp_ops;
> +
> + disp_ops_tbl = devm_kzalloc(dev, sizeof(*disp_ops_tbl), GFP_KERNEL);
> + if (!disp_ops_tbl)
> + return ERR_PTR(-ENOMEM);
> +
> + disp_ops =
> + devm_kzalloc(dev, sizeof(*disp_ops), GFP_KERNEL);
> + if (!disp_ops)
> + return ERR_PTR(-ENOMEM);
> +
> + disp_ops_tbl->ops = disp_ops;
> + disp_ops_tbl->priv = disp_mgt;
> +
> + return disp_ops_tbl;
> +}
> +
> +int nbl_disp_init(struct nbl_adapter *adapter)
> +{
> + struct nbl_common_info *common = &adapter->common;
> + struct nbl_dispatch_ops_tbl *disp_ops_tbl;
> + struct nbl_resource_ops_tbl *res_ops_tbl =
> + adapter->intf.resource_ops_tbl;
> + struct nbl_channel_ops_tbl *chan_ops_tbl =
> + adapter->intf.channel_ops_tbl;
> + struct device *dev = &adapter->pdev->dev;
> + struct nbl_dispatch_mgt *disp_mgt;
> + int ret;
> +
> + disp_mgt = nbl_disp_setup_disp_mgt(common);
> + if (IS_ERR(disp_mgt)) {
> + ret = PTR_ERR(disp_mgt);
> + return ret;
> + }
> +
> + disp_ops_tbl = nbl_disp_setup_ops(dev, disp_mgt);
> + if (IS_ERR(disp_ops_tbl)) {
> + ret = PTR_ERR(disp_ops_tbl);
> + return ret;
> + }
> +
> + disp_mgt->res_ops_tbl = res_ops_tbl;
> + disp_mgt->chan_ops_tbl = chan_ops_tbl;
> + disp_mgt->disp_ops_tbl = disp_ops_tbl;
> + adapter->core.disp_mgt = disp_mgt;
> + adapter->intf.dispatch_ops_tbl = disp_ops_tbl;
> +
> + return 0;
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
> new file mode 100644
> index 000000000000..fa7f4597febe
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_core/nbl_dispatch.h
> @@ -0,0 +1,25 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DISPATCH_H_
> +#define _NBL_DISPATCH_H_
> +#include "../nbl_include/nbl_include.h"
> +#include "../nbl_include/nbl_product_base.h"
> +#include "../nbl_include/nbl_def_channel.h"
> +#include "../nbl_include/nbl_def_hw.h"
> +#include "../nbl_include/nbl_def_resource.h"
> +#include "../nbl_include/nbl_def_dispatch.h"
> +#include "../nbl_include/nbl_def_common.h"
> +#include "../nbl_core.h"
> +
> +struct nbl_dispatch_mgt {
> + struct nbl_common_info *common;
> + struct nbl_resource_ops_tbl *res_ops_tbl;
> + struct nbl_channel_ops_tbl *chan_ops_tbl;
> + struct nbl_dispatch_ops_tbl *disp_ops_tbl;
> + DECLARE_BITMAP(ctrl_lvl, NBL_DISP_CTRL_LVL_MAX);
> +};
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> new file mode 100644
> index 000000000000..6dd8d3a087dd
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -0,0 +1,140 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include <linux/pci.h>
> +#include <linux/bits.h>
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +#include "nbl_hw_leonis.h"
> +
> +static struct nbl_hw_ops hw_ops = {
> +};
> +
> +/* Structure starts here, adding an op should not modify anything below */
> +static struct nbl_hw_mgt *nbl_hw_setup_hw_mgt(struct nbl_common_info *common)
> +{
> + struct device *dev = common->dev;
> + struct nbl_hw_mgt *hw_mgt;
> +
> + hw_mgt = devm_kzalloc(dev, sizeof(*hw_mgt), GFP_KERNEL);
> + if (!hw_mgt)
> + return ERR_PTR(-ENOMEM);
> +
> + hw_mgt->common = common;
> +
> + return hw_mgt;
> +}
> +
> +static struct nbl_hw_ops_tbl *nbl_hw_setup_ops(struct nbl_common_info *common,
> + struct nbl_hw_mgt *hw_mgt)
> +{
> + struct nbl_hw_ops_tbl *hw_ops_tbl;
> + struct device *dev;
> +
> + dev = common->dev;
> + hw_ops_tbl =
> + devm_kzalloc(dev, sizeof(*hw_ops_tbl), GFP_KERNEL);
> + if (!hw_ops_tbl)
> + return ERR_PTR(-ENOMEM);
> +
> + hw_ops_tbl->ops = &hw_ops;
> + hw_ops_tbl->priv = hw_mgt;
> +
> + return hw_ops_tbl;
> +}
> +
> +int nbl_hw_init_leonis(struct nbl_adapter *adapter)
> +{
> + struct nbl_common_info *common = &adapter->common;
> + struct pci_dev *pdev = common->pdev;
> + struct nbl_hw_ops_tbl *hw_ops_tbl;
> + struct nbl_hw_mgt *hw_mgt;
> + int bar_mask;
> + int ret;
> +
> + hw_mgt = nbl_hw_setup_hw_mgt(common);
> + if (IS_ERR(hw_mgt)) {
> + ret = PTR_ERR(hw_mgt);
> + goto setup_mgt_fail;
> + }
> + bar_mask = BIT(NBL_MEMORY_BAR) | BIT(NBL_MAILBOX_BAR);
> + ret = pci_request_selected_regions(pdev, bar_mask, NBL_DRIVER_NAME);
> + if (ret) {
> + dev_err(&pdev->dev,
> + "Request memory bar and mailbox bar failed, err = %d\n",
> + ret);
> + goto request_bar_region_fail;
> + }
> +
> + if (common->has_ctrl) {
> + hw_mgt->hw_addr =
> + ioremap(pci_resource_start(pdev, NBL_MEMORY_BAR),
> + pci_resource_len(pdev, NBL_MEMORY_BAR) -
> + NBL_RDMA_NOTIFY_OFF);
> + if (!hw_mgt->hw_addr) {
> + dev_err(&pdev->dev, "Memory bar ioremap failed\n");
> + ret = -EIO;
> + goto ioremap_err;
> + }
> + hw_mgt->hw_size = pci_resource_len(pdev, NBL_MEMORY_BAR) -
> + NBL_RDMA_NOTIFY_OFF;
> + } else {
> + hw_mgt->hw_addr =
> + ioremap(pci_resource_start(pdev, NBL_MEMORY_BAR),
> + NBL_RDMA_NOTIFY_OFF);
> + if (!hw_mgt->hw_addr) {
> + dev_err(&pdev->dev, "Memory bar ioremap failed\n");
> + ret = -EIO;
> + goto ioremap_err;
> + }
> + hw_mgt->hw_size = NBL_RDMA_NOTIFY_OFF;
> + }
> +
> + hw_mgt->notify_offset = 0;
> + hw_mgt->mailbox_bar_hw_addr = pci_ioremap_bar(pdev, NBL_MAILBOX_BAR);
> + if (!hw_mgt->mailbox_bar_hw_addr) {
> + dev_err(&pdev->dev, "Mailbox bar ioremap failed\n");
> + ret = -EIO;
> + goto mailbox_ioremap_err;
> + }
> +
> + spin_lock_init(&hw_mgt->reg_lock);
> + adapter->core.hw_mgt = hw_mgt;
> +
> + hw_ops_tbl = nbl_hw_setup_ops(common, hw_mgt);
> + if (IS_ERR(hw_ops_tbl)) {
> + ret = PTR_ERR(hw_ops_tbl);
> + goto setup_ops_fail;
> + }
> + adapter->intf.hw_ops_tbl = hw_ops_tbl;
> +
> + return 0;
> +
> +setup_ops_fail:
> + iounmap(hw_mgt->mailbox_bar_hw_addr);
> +mailbox_ioremap_err:
> + iounmap(hw_mgt->hw_addr);
> +ioremap_err:
> + pci_release_selected_regions(pdev, bar_mask);
> +request_bar_region_fail:
> +setup_mgt_fail:
> + return ret;
> +}
> +
> +void nbl_hw_remove_leonis(struct nbl_adapter *adapter)
> +{
> + int bar_mask = BIT(NBL_MEMORY_BAR) | BIT(NBL_MAILBOX_BAR);
> + struct nbl_common_info *common = &adapter->common;
> + struct nbl_hw_mgt *hw_mgt = adapter->core.hw_mgt;
> + u8 __iomem *hw_addr = hw_mgt->hw_addr;
> + struct pci_dev *pdev = common->pdev;
> + u8 __iomem *mailbox_bar_hw_addr;
> +
> + mailbox_bar_hw_addr = hw_mgt->mailbox_bar_hw_addr;
> +
> + iounmap(mailbox_bar_hw_addr);
> + iounmap(hw_addr);
> + pci_release_selected_regions(pdev, bar_mask);
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> new file mode 100644
> index 000000000000..77c67b67ba31
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> @@ -0,0 +1,14 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_HW_LEONIS_H_
> +#define _NBL_HW_LEONIS_H_
> +
> +#include <linux/types.h>
> +
> +#include "../../nbl_include/nbl_include.h"
> +#include "../nbl_hw_reg.h"
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> new file mode 100644
> index 000000000000..355f7ea66b88
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> @@ -0,0 +1,83 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include <linux/pci.h>
> +#include "nbl_resource_leonis.h"
> +
> +static struct nbl_resource_ops res_ops = {
> +};
> +
> +static struct nbl_resource_mgt *
> +nbl_res_setup_res_mgt(struct nbl_common_info *common)
> +{
> + struct nbl_resource_info *resource_info;
> + struct nbl_resource_mgt *res_mgt;
> + struct device *dev = common->dev;
> +
> + res_mgt = devm_kzalloc(dev, sizeof(*res_mgt), GFP_KERNEL);
> + if (!res_mgt)
> + return ERR_PTR(-ENOMEM);
> + res_mgt->common = common;
> +
> + resource_info =
> + devm_kzalloc(dev, sizeof(*resource_info), GFP_KERNEL);
> + if (!resource_info)
> + return ERR_PTR(-ENOMEM);
> + res_mgt->resource_info = resource_info;
> +
> + return res_mgt;
> +}
> +
> +static struct nbl_resource_ops_tbl *
> +nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_resource_ops_tbl *res_ops_tbl;
> +
> + res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
> + if (!res_ops_tbl)
> + return ERR_PTR(-ENOMEM);
> +
> + res_ops_tbl->ops = &res_ops;
> + res_ops_tbl->priv = res_mgt;
> +
> + return res_ops_tbl;
> +}
> +
> +static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
> +{
> + return 0;
> +}
> +
> +int nbl_res_init_leonis(struct nbl_adapter *adap)
> +{
> + struct nbl_channel_ops_tbl *chan_ops_tbl = adap->intf.channel_ops_tbl;
> + struct nbl_hw_ops_tbl *hw_ops_tbl = adap->intf.hw_ops_tbl;
> + struct nbl_common_info *common = &adap->common;
> + struct nbl_resource_ops_tbl *res_ops_tbl;
> + struct device *dev = &adap->pdev->dev;
> + struct nbl_resource_mgt *res_mgt;
> + int ret;
> +
> + res_mgt = nbl_res_setup_res_mgt(common);
> + if (IS_ERR(res_mgt)) {
> + ret = PTR_ERR(res_mgt);
> + return ret;
> + }
> + res_mgt->chan_ops_tbl = chan_ops_tbl;
> + res_mgt->hw_ops_tbl = hw_ops_tbl;
> +
> + ret = nbl_res_start(res_mgt);
> + if (ret)
> + return ret;
> + adap->core.res_mgt = res_mgt;
> +
> + res_ops_tbl = nbl_res_setup_ops(dev, res_mgt);
> + if (IS_ERR(res_ops_tbl)) {
> + ret = PTR_ERR(res_ops_tbl);
> + return ret;
> + }
> + adap->intf.resource_ops_tbl = res_ops_tbl;
> + return 0;
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
> new file mode 100644
> index 000000000000..4e61a5c141e5
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.h
> @@ -0,0 +1,10 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_RESOURCE_LEONIS_H_
> +#define _NBL_RESOURCE_LEONIS_H_
> +
> +#include "../nbl_resource.h"
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
> new file mode 100644
> index 000000000000..46e58b4e73dc
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_reg.h
> @@ -0,0 +1,68 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_HW_REG_H_
> +#define _NBL_HW_REG_H_
> +
> +#include <linux/types.h>
> +
> +#include "../nbl_include/nbl_product_base.h"
> +#include "../nbl_include/nbl_def_channel.h"
> +#include "../nbl_include/nbl_def_hw.h"
> +#include "../nbl_include/nbl_def_common.h"
> +#include "../nbl_core.h"
> +
> +#define NBL_MEMORY_BAR 0
> +#define NBL_MAILBOX_BAR 2
> +#define NBL_RDMA_NOTIFY_OFF 8192
> +#define NBL_HW_DUMMY_REG 0x1300904
> +
> +struct nbl_hw_mgt {
> + struct nbl_common_info *common;
> + u8 __iomem *hw_addr;
> + u8 __iomem *mailbox_bar_hw_addr;
> + u64 notify_offset;
> + u32 version;
> + u32 hw_size;
> + spinlock_t reg_lock; /* Protect reg access */
> +};
> +
> +static inline u32 rd32(u8 __iomem *addr, u64 reg)
> +{
> + return readl(addr + reg);
> +}
> +
> +static inline void wr32(u8 __iomem *addr, u64 reg, u32 value)
> +{
> + writel(value, addr + reg);
> +}
> +
> +static inline void nbl_hw_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
> +{
> + /* Used for emu, make sure that we won't write too frequently */
> + wr32(hw_mgt->hw_addr, reg, value);
> +}
> +
> +static inline u32 nbl_hw_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
> +{
> + return rd32(hw_mgt->hw_addr, reg);
> +}
> +
> +static inline void nbl_mbx_wr32(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 value)
> +{
> + writel(value, hw_mgt->mailbox_bar_hw_addr + reg);
> +}
> +
> +static inline void nbl_flush_writes(struct nbl_hw_mgt *hw_mgt)
> +{
> + nbl_hw_rd32(hw_mgt, NBL_HW_DUMMY_REG);
> +}
> +
> +static inline u32 nbl_mbx_rd32(struct nbl_hw_mgt *hw_mgt, u64 reg)
> +{
> + return readl(hw_mgt->mailbox_bar_hw_addr + reg);
> +}
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> new file mode 100644
> index 000000000000..e08b6237da32
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> @@ -0,0 +1,30 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_RESOURCE_H_
> +#define _NBL_RESOURCE_H_
> +
> +#include <linux/types.h>
> +
> +#include "../nbl_include/nbl_include.h"
> +#include "../nbl_include/nbl_product_base.h"
> +#include "../nbl_include/nbl_def_channel.h"
> +#include "../nbl_include/nbl_def_hw.h"
> +#include "../nbl_include/nbl_def_resource.h"
> +#include "../nbl_include/nbl_def_common.h"
> +#include "../nbl_core.h"
> +
> +struct nbl_resource_info {
> +};
> +
> +struct nbl_resource_mgt {
> + struct nbl_common_info *common;
> + struct nbl_resource_info *resource_info;
> + struct nbl_channel_ops_tbl *chan_ops_tbl;
> + struct nbl_hw_ops_tbl *hw_ops_tbl;
> + struct nbl_interrupt_mgt *intr_mgt;
> +};
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> new file mode 100644
> index 000000000000..ff03a53b9f5d
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> @@ -0,0 +1,26 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEF_CHANNEL_H_
> +#define _NBL_DEF_CHANNEL_H_
> +
> +struct nbl_channel_mgt;
> +struct nbl_adapter;
> +enum nbl_channel_type {
> + NBL_CHAN_TYPE_MAILBOX,
> + NBL_CHAN_TYPE_MAX
> +};
> +
> +struct nbl_channel_ops {
> +};
> +
> +struct nbl_channel_ops_tbl {
> + struct nbl_channel_ops *ops;
> + struct nbl_channel_mgt *priv;
> +};
> +
> +int nbl_chan_init_common(struct nbl_adapter *adapter);
> +void nbl_chan_remove_common(struct nbl_adapter *adapter);
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> new file mode 100644
> index 000000000000..cc73d6a91f3c
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> @@ -0,0 +1,35 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEF_COMMON_H_
> +#define _NBL_DEF_COMMON_H_
> +
> +#include <linux/types.h>
> +#include <linux/pci.h>
> +#include <linux/device.h>
> +#include "nbl_include.h"
> +
> +struct nbl_common_info {
> + struct pci_dev *pdev;
> + struct device *dev;
> + u32 msg_enable;
> + u16 vsi_id;
> + u8 eth_id;
> + u8 logic_eth_id;
> + u8 eth_mode;
> +
> + u8 function;
> + u8 devid;
> + u8 bus;
> + u8 hw_bus;
> + u16 mgt_pf;
> +
> + bool pci_using_dac;
> + enum nbl_product_type product_type;
> + u8 has_ctrl;
> + u8 has_net;
> +};
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
> new file mode 100644
> index 000000000000..32e6cce38d39
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dev.h
> @@ -0,0 +1,16 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEF_DEV_H_
> +#define _NBL_DEF_DEV_H_
> +
> +struct nbl_adapter;
> +
> +int nbl_dev_init(struct nbl_adapter *adapter);
> +void nbl_dev_remove(struct nbl_adapter *adapter);
> +int nbl_dev_start(struct nbl_adapter *adapter);
> +void nbl_dev_stop(struct nbl_adapter *adapter);
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> new file mode 100644
> index 000000000000..2230c728079e
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_dispatch.h
> @@ -0,0 +1,29 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEF_DISPATCH_H_
> +#define _NBL_DEF_DISPATCH_H_
> +
> +struct nbl_dispatch_mgt;
> +struct nbl_adapter;
> +enum {
> + NBL_DISP_CTRL_LVL_NEVER = 0,
> + NBL_DISP_CTRL_LVL_MGT,
> + NBL_DISP_CTRL_LVL_NET,
> + NBL_DISP_CTRL_LVL_ALWAYS,
> + NBL_DISP_CTRL_LVL_MAX,
> +};
> +
> +struct nbl_dispatch_ops {
> +};
> +
> +struct nbl_dispatch_ops_tbl {
> + struct nbl_dispatch_ops *ops;
> + struct nbl_dispatch_mgt *priv;
> +};
> +
> +int nbl_disp_init(struct nbl_adapter *adapter);
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> new file mode 100644
> index 000000000000..168504b30973
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEF_HW_H_
> +#define _NBL_DEF_HW_H_
> +
> +struct nbl_hw_mgt;
> +struct nbl_adapter;
> +struct nbl_hw_ops {
> +};
> +
> +struct nbl_hw_ops_tbl {
> + struct nbl_hw_ops *ops;
> + struct nbl_hw_mgt *priv;
> +};
> +
> +int nbl_hw_init_leonis(struct nbl_adapter *adapter);
> +void nbl_hw_remove_leonis(struct nbl_adapter *adapter);
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
> new file mode 100644
> index 000000000000..16add7e25eae
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEF_RESOURCE_H_
> +#define _NBL_DEF_RESOURCE_H_
> +
> +struct nbl_resource_mgt;
> +struct nbl_adapter;
> +struct nbl_resource_ops {
> +};
> +
> +struct nbl_resource_ops_tbl {
> + struct nbl_resource_ops *ops;
> + struct nbl_resource_mgt *priv;
> +};
> +
> +int nbl_res_init_leonis(struct nbl_adapter *adapter);
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> index 1046e6517b15..50f30f756bf3 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> @@ -11,6 +11,11 @@
> /* ------ Basic definitions ------- */
> #define NBL_DRIVER_NAME "nbl"
>
> +enum nbl_product_type {
> + NBL_LEONIS_TYPE,
> + NBL_PRODUCT_MAX,
> +};
> +
> struct nbl_func_caps {
> u32 has_ctrl:1;
> u32 has_net:1;
> @@ -18,4 +23,10 @@ struct nbl_func_caps {
> u32 rsv:29;
> };
>
> +struct nbl_init_param {
> + struct nbl_func_caps caps;
> + enum nbl_product_type product_type;
> + bool pci_using_dac;
> +};
> +
> #endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_product_base.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_product_base.h
> new file mode 100644
> index 000000000000..4368d4f57a03
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_product_base.h
> @@ -0,0 +1,18 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_DEF_PRODUCT_BASE_H_
> +#define _NBL_DEF_PRODUCT_BASE_H_
> +
> +struct nbl_adapter;
> +struct nbl_product_base_ops {
> + int (*hw_init)(struct nbl_adapter *p);
> + void (*hw_remove)(struct nbl_adapter *p);
> + int (*res_init)(struct nbl_adapter *p);
> + int (*chan_init)(struct nbl_adapter *p);
> + void (*chan_remove)(struct nbl_adapter *p);
> +};
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> index 10c3536b327b..db035136e09c 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_main.c
> @@ -6,17 +6,196 @@
> #include <linux/device.h>
> #include <linux/pci.h>
> #include <linux/module.h>
> +#include <linux/bits.h>
> #include "nbl_include/nbl_include.h"
> +#include "nbl_include/nbl_product_base.h"
> +#include "nbl_include/nbl_def_channel.h"
> +#include "nbl_include/nbl_def_hw.h"
> +#include "nbl_include/nbl_def_resource.h"
> +#include "nbl_include/nbl_def_dispatch.h"
> +#include "nbl_include/nbl_def_dev.h"
> +#include "nbl_include/nbl_def_common.h"
> #include "nbl_core.h"
>
> +static struct nbl_product_base_ops nbl_product_base_ops[NBL_PRODUCT_MAX] = {
> + {
> + .hw_init = nbl_hw_init_leonis,
> + .hw_remove = nbl_hw_remove_leonis,
> + .res_init = nbl_res_init_leonis,
> + .chan_init = nbl_chan_init_common,
> + .chan_remove = nbl_chan_remove_common,
> + },
> +};
> +
> +int nbl_core_start(struct nbl_adapter *adapter)
> +{
> + return nbl_dev_start(adapter);
> +}
> +
> +void nbl_core_stop(struct nbl_adapter *adapter)
> +{
> + nbl_dev_stop(adapter);
> +}
> +
> +static struct nbl_product_base_ops *
> +nbl_core_setup_product_ops(struct nbl_adapter *adapter,
> + struct nbl_init_param *param)
> +{
> + adapter->product_base_ops = &nbl_product_base_ops[param->product_type];
> + return adapter->product_base_ops;
> +}
> +
> +struct nbl_adapter *nbl_core_init(struct pci_dev *pdev,
> + struct nbl_init_param *param)
> +{
> + struct nbl_product_base_ops *product_base_ops;
> + struct nbl_common_info *common;
> + struct nbl_adapter *adapter;
> + int ret;
> +
> + adapter = devm_kzalloc(&pdev->dev, sizeof(*adapter), GFP_KERNEL);
> + if (!adapter)
> + return NULL;
> +
> + adapter->pdev = pdev;
> + common = &adapter->common;
> +
> + common->pdev = pdev;
> + common->dev = &pdev->dev;
> + common->has_ctrl = param->caps.has_ctrl;
> + common->has_net = param->caps.has_net;
> + common->pci_using_dac = param->pci_using_dac;
> + common->function = PCI_FUNC(pdev->devfn);
> + common->devid = PCI_SLOT(pdev->devfn);
> + common->bus = pdev->bus->number;
> + common->product_type = param->product_type;
> +
> + product_base_ops = nbl_core_setup_product_ops(adapter, param);
> +
> + /*
> + *every product's hw/chan/res layer has a great difference,
> + *so call their own init ops
> + */
> + ret = product_base_ops->hw_init(adapter);
> + if (ret)
> + goto hw_init_fail;
> +
> + ret = product_base_ops->chan_init(adapter);
> + if (ret)
> + goto chan_init_fail;
> +
> + ret = product_base_ops->res_init(adapter);
> + if (ret)
> + goto res_init_fail;
> +
> + ret = nbl_disp_init(adapter);
> + if (ret)
> + goto res_init_fail;
> +
> + ret = nbl_dev_init(adapter);
> + if (ret)
> + goto res_init_fail;
> + return adapter;
> +
All the potential failures after res_init() do not need any other cleanup?
> +res_init_fail:
> + product_base_ops->chan_remove(adapter);
> +chan_init_fail:
> + product_base_ops->hw_remove(adapter);
> +hw_init_fail:
> + return NULL;
> +}
> +
> +void nbl_core_remove(struct nbl_adapter *adapter)
> +{
> + struct nbl_product_base_ops *product_base_ops;
> +
> + product_base_ops = adapter->product_base_ops;
> + nbl_dev_remove(adapter);
> + product_base_ops->chan_remove(adapter);
> + product_base_ops->hw_remove(adapter);
> +}
> +
> +static void nbl_get_func_param(struct pci_dev *pdev, kernel_ulong_t driver_data,
> + struct nbl_init_param *param)
> +{
> + param->caps.has_ctrl = !!(driver_data & BIT(NBL_CAP_HAS_CTRL_BIT));
> + param->caps.has_net = !!(driver_data & BIT(NBL_CAP_HAS_NET_BIT));
> + param->caps.is_nic = !!(driver_data & BIT(NBL_CAP_IS_NIC_BIT));
> +
> + if (!!(driver_data & BIT(NBL_CAP_IS_LEONIS_BIT)))
> + param->product_type = NBL_LEONIS_TYPE;
> +
> + /*
> + * Leonis only PF0 has ctrl capability, but PF0's pcie device_id
> + * is same with other PF.So handle it special.
> + */
> + if (param->product_type == NBL_LEONIS_TYPE &&
> + (PCI_FUNC(pdev->devfn) == 0))
> + param->caps.has_ctrl = 1;
> +}
> +
> static int nbl_probe(struct pci_dev *pdev,
> const struct pci_device_id *id)
> {
> + struct nbl_init_param param = { { 0 } };
> + struct device *dev = &pdev->dev;
> + struct nbl_adapter *adapter;
> + int err;
> +
> + if (pci_enable_device(pdev)) {
> + dev_err(&pdev->dev, "Failed to enable PCI device\n");
> + return -ENODEV;
> + }
> +
> + param.pci_using_dac = true;
> + nbl_get_func_param(pdev, id->driver_data, ¶m);
> +
> + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
> + if (err) {
> + dev_dbg(dev, "Configure DMA 64 bit mask failed, err = %d\n",
> + err);
> + param.pci_using_dac = false;
> + err = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(32));
> + if (err) {
> + dev_err(dev,
> + "Configure DMA 32 bit mask failed, err = %d\n",
> + err);
> + goto configure_dma_err;
> + }
> + }
> + pci_set_master(pdev);
> + pci_save_state(pdev);
> + adapter = nbl_core_init(pdev, ¶m);
> + if (!adapter) {
> + dev_err(dev, "Nbl adapter init fail\n");
> + err = -ENOMEM;
> + goto adapter_init_err;
> + }
> + pci_set_drvdata(pdev, adapter);
> + err = nbl_core_start(adapter);
> + if (err)
> + goto core_start_err;
> return 0;
> +core_start_err:
> + nbl_core_remove(adapter);
> +adapter_init_err:
> + pci_clear_master(pdev);
> +configure_dma_err:
> + pci_disable_device(pdev);
> + return err;
> }
>
> static void nbl_remove(struct pci_dev *pdev)
> {
> + struct nbl_adapter *adapter = pci_get_drvdata(pdev);
> +
> + pci_disable_sriov(pdev);
> +
> + nbl_core_stop(adapter);
> + nbl_core_remove(adapter);
> +
> + pci_clear_master(pdev);
> + pci_disable_device(pdev);
> }
>
> /*
^ permalink raw reply
* Re: [PATCH v10 net-next 04/11] net/nebula-matrix: channel msg value and msg struct
From: Mohsin Bashir @ 2026-04-02 9:03 UTC (permalink / raw)
To: illusion.wang, dimon.zhao, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, linux-doc, lorenzo, pabeni, horms,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, jani.nikula, open list
In-Reply-To: <20260401022318.28550-5-illusion.wang@nebula-matrix.com>
On 3/31/26 7:23 PM, illusion.wang wrote:
> For compatibility, the msg id value is fixed, and each msg struct can
> only have fields added (not removed), new fields must be appended
>
> Signed-off-by: illusion.wang <illusion.wang@nebula-matrix.com>
> ---
> .../nbl/nbl_include/nbl_def_channel.h | 249 ++++++++++++++++++
> 1 file changed, 249 insertions(+)
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> index ff03a53b9f5d..99e1d8455e0e 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> @@ -8,6 +8,255 @@
>
> struct nbl_channel_mgt;
> struct nbl_adapter;
> +enum nbl_chan_msg_type {
> + NBL_CHAN_MSG_ACK,
> + NBL_CHAN_MSG_ADD_MACVLAN,
> + NBL_CHAN_MSG_DEL_MACVLAN,
> + NBL_CHAN_MSG_ADD_MULTI_RULE,
> + NBL_CHAN_MSG_DEL_MULTI_RULE,
> + NBL_CHAN_MSG_SETUP_MULTI_GROUP,
> + NBL_CHAN_MSG_REMOVE_MULTI_GROUP,
> + NBL_CHAN_MSG_REGISTER_NET,
> + NBL_CHAN_MSG_UNREGISTER_NET,
> + NBL_CHAN_MSG_ALLOC_TXRX_QUEUES,
> + NBL_CHAN_MSG_FREE_TXRX_QUEUES,
> + NBL_CHAN_MSG_SETUP_QUEUE,
> + NBL_CHAN_MSG_REMOVE_ALL_QUEUES,
> + NBL_CHAN_MSG_CFG_DSCH,
> + NBL_CHAN_MSG_SETUP_CQS,
> + NBL_CHAN_MSG_REMOVE_CQS,
> + NBL_CHAN_MSG_CFG_QDISC_MQPRIO,
> + NBL_CHAN_MSG_CONFIGURE_MSIX_MAP,
> + NBL_CHAN_MSG_DESTROY_MSIX_MAP,
> + NBL_CHAN_MSG_MAILBOX_ENABLE_IRQ,
> + NBL_CHAN_MSG_GET_GLOBAL_VECTOR,
> + NBL_CHAN_MSG_GET_VSI_ID,
> + NBL_CHAN_MSG_SET_PROSISC_MODE,
> + NBL_CHAN_MSG_GET_FIRMWARE_VERSION,
> + NBL_CHAN_MSG_GET_QUEUE_ERR_STATS,
> + NBL_CHAN_MSG_GET_COALESCE,
> + NBL_CHAN_MSG_SET_COALESCE,
> + NBL_CHAN_MSG_SET_SPOOF_CHECK_ADDR,
> + NBL_CHAN_MSG_SET_VF_SPOOF_CHECK,
> + NBL_CHAN_MSG_GET_RXFH_INDIR_SIZE,
> + NBL_CHAN_MSG_GET_RXFH_INDIR,
> + NBL_CHAN_MSG_GET_RXFH_RSS_KEY,
> + NBL_CHAN_MSG_GET_RXFH_RSS_ALG_SEL,
> + NBL_CHAN_MSG_GET_HW_CAPS,
> + NBL_CHAN_MSG_GET_HW_STATE,
> + NBL_CHAN_MSG_REGISTER_RDMA,
> + NBL_CHAN_MSG_UNREGISTER_RDMA,
> + NBL_CHAN_MSG_GET_REAL_HW_ADDR,
> + NBL_CHAN_MSG_GET_REAL_BDF,
> + NBL_CHAN_MSG_GRC_PROCESS,
> + NBL_CHAN_MSG_SET_SFP_STATE,
> + NBL_CHAN_MSG_SET_ETH_LOOPBACK,
> + NBL_CHAN_MSG_CHECK_ACTIVE_VF,
> + NBL_CHAN_MSG_GET_PRODUCT_FLEX_CAP,
> + NBL_CHAN_MSG_ALLOC_KTLS_TX_INDEX,
> + NBL_CHAN_MSG_FREE_KTLS_TX_INDEX,
> + NBL_CHAN_MSG_CFG_KTLS_TX_KEYMAT,
> + NBL_CHAN_MSG_ALLOC_KTLS_RX_INDEX,
> + NBL_CHAN_MSG_FREE_KTLS_RX_INDEX,
> + NBL_CHAN_MSG_CFG_KTLS_RX_KEYMAT,
> + NBL_CHAN_MSG_CFG_KTLS_RX_RECORD,
> + NBL_CHAN_MSG_ADD_KTLS_RX_FLOW,
> + NBL_CHAN_MSG_DEL_KTLS_RX_FLOW,
> + NBL_CHAN_MSG_ALLOC_IPSEC_TX_INDEX,
> + NBL_CHAN_MSG_FREE_IPSEC_TX_INDEX,
> + NBL_CHAN_MSG_ALLOC_IPSEC_RX_INDEX,
> + NBL_CHAN_MSG_FREE_IPSEC_RX_INDEX,
> + NBL_CHAN_MSG_CFG_IPSEC_TX_SAD,
> + NBL_CHAN_MSG_CFG_IPSEC_RX_SAD,
> + NBL_CHAN_MSG_ADD_IPSEC_TX_FLOW,
> + NBL_CHAN_MSG_DEL_IPSEC_TX_FLOW,
> + NBL_CHAN_MSG_ADD_IPSEC_RX_FLOW,
> + NBL_CHAN_MSG_DEL_IPSEC_RX_FLOW,
> + NBL_CHAN_MSG_NOTIFY_IPSEC_HARD_EXPIRE,
> + NBL_CHAN_MSG_GET_MBX_IRQ_NUM,
> + NBL_CHAN_MSG_CLEAR_FLOW,
> + NBL_CHAN_MSG_CLEAR_QUEUE,
> + NBL_CHAN_MSG_GET_ETH_ID,
> + NBL_CHAN_MSG_SET_OFFLOAD_STATUS,
> + NBL_CHAN_MSG_INIT_OFLD,
> + NBL_CHAN_MSG_INIT_CMDQ,
> + NBL_CHAN_MSG_DESTROY_CMDQ,
> + NBL_CHAN_MSG_RESET_CMDQ,
> + NBL_CHAN_MSG_INIT_FLOW,
> + NBL_CHAN_MSG_DEINIT_FLOW,
> + NBL_CHAN_MSG_OFFLOAD_FLOW_RULE,
> + NBL_CHAN_MSG_GET_ACL_SWITCH,
> + NBL_CHAN_MSG_GET_VSI_GLOBAL_QUEUE_ID,
> + NBL_CHAN_MSG_INIT_REP,
> + NBL_CHAN_MSG_GET_LINE_RATE_INFO,
> + NBL_CHAN_MSG_REGISTER_NET_REP,
> + NBL_CHAN_MSG_UNREGISTER_NET_REP,
> + NBL_CHAN_MSG_REGISTER_ETH_REP,
> + NBL_CHAN_MSG_UNREGISTER_ETH_REP,
> + NBL_CHAN_MSG_REGISTER_UPCALL_PORT,
> + NBL_CHAN_MSG_UNREGISTER_UPCALL_PORT,
> + NBL_CHAN_MSG_GET_PORT_STATE,
> + NBL_CHAN_MSG_SET_PORT_ADVERTISING,
> + NBL_CHAN_MSG_GET_MODULE_INFO,
> + NBL_CHAN_MSG_GET_MODULE_EEPROM,
> + NBL_CHAN_MSG_GET_LINK_STATE,
> + NBL_CHAN_MSG_NOTIFY_LINK_STATE,
> + NBL_CHAN_MSG_GET_QUEUE_CXT,
> + NBL_CHAN_MSG_CFG_LOG,
> + NBL_CHAN_MSG_INIT_VDPAQ,
> + NBL_CHAN_MSG_DESTROY_VDPAQ,
> + NBL_CHAN_MSG_GET_UPCALL_PORT,
> + NBL_CHAN_MSG_NOTIFY_ETH_REP_LINK_STATE,
> + NBL_CHAN_MSG_SET_ETH_MAC_ADDR,
> + NBL_CHAN_MSG_GET_FUNCTION_ID,
> + NBL_CHAN_MSG_GET_CHIP_TEMPERATURE,
> + NBL_CHAN_MSG_DISABLE_HW_FLOW,
> + NBL_CHAN_MSG_ENABLE_HW_FLOW,
> + NBL_CHAN_MSG_SET_UPCALL_RULE,
> + NBL_CHAN_MSG_UNSET_UPCALL_RULE,
> + NBL_CHAN_MSG_GET_REG_DUMP,
> + NBL_CHAN_MSG_GET_REG_DUMP_LEN,
> + NBL_CHAN_MSG_CFG_LAG_HASH_ALGORITHM,
> + NBL_CHAN_MSG_CFG_LAG_MEMBER_FWD,
> + NBL_CHAN_MSG_CFG_LAG_MEMBER_LIST,
> + NBL_CHAN_MSG_CFG_LAG_MEMBER_UP_ATTR,
> + NBL_CHAN_MSG_ADD_LAG_FLOW,
> + NBL_CHAN_MSG_DEL_LAG_FLOW,
> + NBL_CHAN_MSG_SWITCHDEV_INIT_CMDQ,
> + NBL_CHAN_MSG_SWITCHDEV_DEINIT_CMDQ,
> + NBL_CHAN_MSG_SET_TC_FLOW_INFO,
> + NBL_CHAN_MSG_UNSET_TC_FLOW_INFO,
> + NBL_CHAN_MSG_INIT_ACL,
> + NBL_CHAN_MSG_UNINIT_ACL,
> + NBL_CHAN_MSG_CFG_LAG_MCC,
> + NBL_CHAN_MSG_REGISTER_VSI2Q,
> + NBL_CHAN_MSG_SETUP_Q2VSI,
> + NBL_CHAN_MSG_REMOVE_Q2VSI,
> + NBL_CHAN_MSG_SETUP_RSS,
> + NBL_CHAN_MSG_REMOVE_RSS,
> + NBL_CHAN_MSG_GET_REP_QUEUE_INFO,
> + NBL_CHAN_MSG_CTRL_PORT_LED,
> + NBL_CHAN_MSG_NWAY_RESET,
> + NBL_CHAN_MSG_SET_INTL_SUPPRESS_LEVEL,
> + NBL_CHAN_MSG_GET_ETH_STATS,
> + NBL_CHAN_MSG_GET_MODULE_TEMPERATURE,
> + NBL_CHAN_MSG_GET_BOARD_INFO,
> + NBL_CHAN_MSG_GET_P4_USED,
> + NBL_CHAN_MSG_GET_VF_BASE_VSI_ID,
> + NBL_CHAN_MSG_ADD_LLDP_FLOW,
> + NBL_CHAN_MSG_DEL_LLDP_FLOW,
> + NBL_CHAN_MSG_CFG_ETH_BOND_INFO,
> + NBL_CHAN_MSG_CFG_DUPPKT_MCC,
> + NBL_CHAN_MSG_ADD_ND_UPCALL_FLOW,
> + NBL_CHAN_MSG_DEL_ND_UPCALL_FLOW,
> + NBL_CHAN_MSG_GET_BOARD_ID,
> + NBL_CHAN_MSG_SET_SHAPING_DPORT_VLD,
> + NBL_CHAN_MSG_SET_DPORT_FC_TH_VLD,
> + NBL_CHAN_MSG_REGISTER_RDMA_BOND,
> + NBL_CHAN_MSG_UNREGISTER_RDMA_BOND,
> + NBL_CHAN_MSG_RESTORE_NETDEV_QUEUE,
> + NBL_CHAN_MSG_RESTART_NETDEV_QUEUE,
> + NBL_CHAN_MSG_RESTORE_HW_QUEUE,
> + NBL_CHAN_MSG_KEEP_ALIVE,
> + NBL_CHAN_MSG_GET_BASE_MAC_ADDR,
> + NBL_CHAN_MSG_CFG_BOND_SHAPING,
> + NBL_CHAN_MSG_CFG_BGID_BACK_PRESSURE,
> + NBL_CHAN_MSG_ALLOC_KT_BLOCK,
> + NBL_CHAN_MSG_FREE_KT_BLOCK,
> + NBL_CHAN_MSG_GET_USER_QUEUE_INFO,
> + NBL_CHAN_MSG_GET_ETH_BOND_INFO,
> + NBL_CHAN_MSG_CLEAR_ACCEL_FLOW,
> + NBL_CHAN_MSG_SET_BRIDGE_MODE,
> + NBL_CHAN_MSG_GET_VF_FUNCTION_ID,
> + NBL_CHAN_MSG_NOTIFY_LINK_FORCED,
> + NBL_CHAN_MSG_SET_PMD_DEBUG,
> + NBL_CHAN_MSG_REGISTER_FUNC_MAC,
> + NBL_CHAN_MSG_SET_TX_RATE,
> + NBL_CHAN_MSG_REGISTER_FUNC_LINK_FORCED,
> + NBL_CHAN_MSG_GET_LINK_FORCED,
> + NBL_CHAN_MSG_REGISTER_FUNC_VLAN,
> + NBL_CHAN_MSG_GET_FD_FLOW,
> + NBL_CHAN_MSG_GET_FD_FLOW_CNT,
> + NBL_CHAN_MSG_GET_FD_FLOW_ALL,
> + NBL_CHAN_MSG_GET_FD_FLOW_MAX,
> + NBL_CHAN_MSG_REPLACE_FD_FLOW,
> + NBL_CHAN_MSG_REMOVE_FD_FLOW,
> + NBL_CHAN_MSG_CFG_FD_FLOW_STATE,
> + NBL_CHAN_MSG_REGISTER_FUNC_RATE,
> + NBL_CHAN_MSG_NOTIFY_VLAN,
> + NBL_CHAN_MSG_GET_XDP_QUEUE_INFO,
> + NBL_CHAN_MSG_STOP_ABNORMAL_SW_QUEUE,
> + NBL_CHAN_MSG_STOP_ABNORMAL_HW_QUEUE,
> + NBL_CHAN_MSG_NOTIFY_RESET_EVENT,
> + NBL_CHAN_MSG_ACK_RESET_EVENT,
> + NBL_CHAN_MSG_GET_VF_VSI_ID,
> + NBL_CHAN_MSG_CONFIGURE_QOS,
> + NBL_CHAN_MSG_GET_PFC_BUFFER_SIZE,
> + NBL_CHAN_MSG_SET_PFC_BUFFER_SIZE,
> + NBL_CHAN_MSG_GET_VF_STATS,
> + NBL_CHAN_MSG_REGISTER_FUNC_TRUST,
> + NBL_CHAN_MSG_NOTIFY_TRUST,
> + NBL_CHAN_MSG_CHECK_VF_IS_ACTIVE,
> + NBL_CHAN_MSG_GET_ETH_ABNORMAL_STATS,
> + NBL_CHAN_MSG_GET_ETH_CTRL_STATS,
> + NBL_CHAN_MSG_GET_PAUSE_STATS,
> + NBL_CHAN_MSG_GET_ETH_MAC_STATS,
> + NBL_CHAN_MSG_GET_FEC_STATS,
> + NBL_CHAN_MSG_CFG_MULTI_MCAST_RULE,
> + NBL_CHAN_MSG_GET_LINK_DOWN_COUNT,
> + NBL_CHAN_MSG_GET_LINK_STATUS_OPCODE,
> + NBL_CHAN_MSG_GET_RMON_STATS,
> + NBL_CHAN_MSG_REGISTER_PF_NAME,
> + NBL_CHAN_MSG_GET_PF_NAME,
> + NBL_CHAN_MSG_CONFIGURE_RDMA_BW,
> + NBL_CHAN_MSG_SET_RATE_LIMIT,
> + NBL_CHAN_MSG_SET_TC_WGT,
> + NBL_CHAN_MSG_REMOVE_QUEUE,
> + NBL_CHAN_MSG_GET_MIRROR_TABLE_ID,
> + NBL_CHAN_MSG_CONFIGURE_MIRROR,
> + NBL_CHAN_MSG_CONFIGURE_MIRROR_TABLE,
> + NBL_CHAN_MSG_CLEAR_MIRROR_CFG,
> + NBL_CHAN_MSG_MIRROR_OUTPUTPORT_NOTIFY,
> + NBL_CHAN_MSG_CHECK_FLOWTABLE_SPEC,
> + NBL_CHAN_MSG_CHECK_VF_IS_VDPA,
> + NBL_CHAN_MSG_GET_VDPA_VF_STATS,
> + NBL_CHAN_MSG_SET_RX_RATE,
> + NBL_CHAN_MSG_GET_UVN_PKT_DROP_STATS,
> + NBL_CHAN_MSG_GET_USTORE_PKT_DROP_STATS,
> + NBL_CHAN_MSG_GET_USTORE_TOTAL_PKT_DROP_STATS,
> + NBL_CHAN_MSG_SET_WOL,
> + NBL_CHAN_MSG_INIT_VF_MSIX_MAP,
> + NBL_CHAN_MSG_GET_ST_NAME,
> + NBL_CHAN_MSG_MTU_SET = 501,
> + NBL_CHAN_MSG_SET_RXFH_INDIR = 506,
> + NBL_CHAN_MSG_SET_RXFH_RSS_ALG_SEL = 508,
> + /* mailbox msg end */
> + NBL_CHAN_MSG_MAILBOX_MAX,
> +};
> +
This is a big enum. Is there a possibility to group them by functionality?
> +struct nbl_chan_param_cfg_msix_map {
> + u16 num_net_msix;
> + u16 num_others_msix;
> + u16 msix_mask_en;
> +};
> +
> +struct nbl_chan_param_enable_mailbox_irq {
> + u16 vector_id;
> + bool enable_msix;
> +};
> +
> +struct nbl_chan_param_get_vsi_id {
> + u16 vsi_id;
> + u16 type;
> +};
> +
> +struct nbl_chan_param_get_eth_id {
> + u16 vsi_id;
> + u8 eth_mode;
> + u8 eth_id;
> + u8 logic_eth_id;
> +};
> +
> enum nbl_channel_type {
> NBL_CHAN_TYPE_MAILBOX,
> NBL_CHAN_TYPE_MAX
^ permalink raw reply
* Re: [PATCH v10 net-next 05/11] net/nebula-matrix: add channel layer
From: Mohsin Bashir @ 2026-04-02 9:04 UTC (permalink / raw)
To: illusion.wang, dimon.zhao, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, linux-doc, lorenzo, pabeni, horms,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, jani.nikula, open list
In-Reply-To: <20260401022318.28550-6-illusion.wang@nebula-matrix.com>
On 3/31/26 7:23 PM, illusion.wang wrote:
> a channel management layer provides structured approach to handle
> communication between different components and drivers. Here's a summary
> of its key functionalities:
>
> 1. Message Handling Framework
> Message Registration/Unregistration: Functions (nbl_chan_register_msg,
> nbl_chan_unregister_msg) allow dynamic registration of message handlers
> for specific message types, enabling extensible communication protocols.
> Message Sending/Acknowledgment: Core functions (nbl_chan_send_msg,
> nbl_chan_send_ack) handle message transmission, including asynchronous
> operations with acknowledgment (ACK) support.
> Received ACKs are processed via nbl_chan_recv_ack_msg.
> Hash-Based Handler Lookup: A hash table (handle_hash_tbl) stores message
> handlers for efficient O(1) lookup by message type.
>
> 2. Channel Types and Queue Management
> Mailbox Channel: For direct communication between PF0 and Other PF.
> Queue Initialization/Teardown: Functions (nbl_chan_init_queue,
> nbl_chan_teardown_queue) manage transmit (TX) and receive (RX) queues.
>
> Queue Configuration: Hardware-specific queue parameters (e.g., buffer
> sizes, entry counts) are set via nbl_chan_config_queue, with hardware
> interactions delegated to hw_ops.
>
> 3. Hardware Abstraction Layer (HW Ops)
> Hardware-Specific Operations: The nbl_hw_ops structure abstracts
> hardware interactions: queue configuration (config_mailbox_txq/rxq),
> tail pointer updates(update_mailbox_queue_tail_ptr).
>
> Signed-off-by: illusion.wang <illusion.wang@nebula-matrix.com>
> ---
> .../net/ethernet/nebula-matrix/nbl/Makefile | 3 +-
> .../nbl/nbl_channel/nbl_channel.c | 768 +++++++++++++++++-
> .../nbl/nbl_channel/nbl_channel.h | 129 +++
> .../nebula-matrix/nbl/nbl_common/nbl_common.c | 207 +++++
> .../nebula-matrix/nbl/nbl_common/nbl_common.h | 34 +
> .../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 143 ++++
> .../nbl/nbl_include/nbl_def_channel.h | 87 ++
> .../nbl/nbl_include/nbl_def_common.h | 30 +
> .../nbl/nbl_include/nbl_def_hw.h | 28 +
> .../nbl/nbl_include/nbl_include.h | 6 +
> 10 files changed, 1433 insertions(+), 2 deletions(-)
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> index 63116d1d7043..c9bc060732e7 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> @@ -3,7 +3,8 @@
>
> obj-$(CONFIG_NBL) := nbl.o
>
> -nbl-objs += nbl_channel/nbl_channel.o \
> +nbl-objs += nbl_common/nbl_common.o \
> + nbl_channel/nbl_channel.o \
> nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
> nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
> nbl_hw/nbl_hw_leonis/nbl_hw_leonis_regs.o \
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> index 9f4603870cb4..0da7968e93e8 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.c
> @@ -2,12 +2,763 @@
> /*
> * Copyright (c) 2025 Nebula Matrix Limited.
> */
> -
> +#include <linux/delay.h>
> #include <linux/device.h>
> #include <linux/pci.h>
> +#include <linux/bits.h>
> +#include <linux/dma-mapping.h>
> #include "nbl_channel.h"
>
> +static int nbl_chan_add_msg_handler(struct nbl_channel_mgt *chan_mgt,
> + u16 msg_type, nbl_chan_resp func,
> + void *priv)
> +{
> + struct nbl_chan_msg_node_data handler = { 0 };
> + int ret;
> +
> + handler.func = func;
> + handler.priv = priv;
> + ret = nbl_common_alloc_hash_node(chan_mgt->handle_hash_tbl, &msg_type,
> + &handler, NULL);
> +
> + return ret;
> +}
> +
> +static int nbl_chan_init_msg_handler(struct nbl_channel_mgt *chan_mgt)
> +{
> + struct nbl_common_info *common = chan_mgt->common;
> + struct nbl_hash_tbl_key tbl_key;
> +
> + tbl_key.dev = common->dev;
> + tbl_key.key_size = sizeof(u16);
> + tbl_key.data_size = sizeof(struct nbl_chan_msg_node_data);
> + tbl_key.bucket_size = NBL_CHAN_HANDLER_TBL_BUCKET_SIZE;
> +
> + chan_mgt->handle_hash_tbl = nbl_common_init_hash_table(&tbl_key);
> + if (!chan_mgt->handle_hash_tbl)
> + return -ENOMEM;
> +
> + return 0;
> +}
> +
> +static void nbl_chan_remove_msg_handler(struct nbl_channel_mgt *chan_mgt)
> +{
> + nbl_common_remove_hash_table(chan_mgt->handle_hash_tbl, NULL);
> +
> + chan_mgt->handle_hash_tbl = NULL;
> +}
> +
> +static void nbl_chan_init_queue_param(struct nbl_chan_info *chan_info,
> + u16 num_txq_entries, u16 num_rxq_entries,
> + u16 txq_buf_size, u16 rxq_buf_size)
> +{
> + mutex_init(&chan_info->txq_lock);
> + chan_info->num_txq_entries = num_txq_entries;
> + chan_info->num_rxq_entries = num_rxq_entries;
> + chan_info->txq_buf_size = txq_buf_size;
> + chan_info->rxq_buf_size = rxq_buf_size;
> +}
> +
> +static int nbl_chan_init_tx_queue(struct nbl_common_info *common,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_chan_ring *txq = &chan_info->txq;
> + struct device *dev = common->dev;
> + size_t size =
> + chan_info->num_txq_entries * sizeof(struct nbl_chan_tx_desc);
> +
> + txq->desc.tx_desc = dmam_alloc_coherent(dev, size, &txq->dma,
> + GFP_KERNEL);
> + if (!txq->desc.tx_desc)
> + return -ENOMEM;
> +
> + chan_info->wait = devm_kcalloc(dev, chan_info->num_txq_entries,
> + sizeof(*chan_info->wait),
> + GFP_KERNEL);
> + if (!chan_info->wait)
> + return -ENOMEM;
> +
> + txq->buf = devm_kcalloc(dev, chan_info->num_txq_entries,
> + sizeof(*txq->buf), GFP_KERNEL);
> + if (!txq->buf)
> + return -ENOMEM;
> +
> + return 0;
> +}
> +
> +static int nbl_chan_init_rx_queue(struct nbl_common_info *common,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_chan_ring *rxq = &chan_info->rxq;
> + struct device *dev = common->dev;
> + size_t size =
> + chan_info->num_rxq_entries * sizeof(struct nbl_chan_rx_desc);
> +
> + rxq->desc.rx_desc = dmam_alloc_coherent(dev, size, &rxq->dma,
> + GFP_KERNEL);
> + if (!rxq->desc.rx_desc) {
> + dev_err(dev,
> + "Allocate DMA for chan rx descriptor ring failed\n");
> + return -ENOMEM;
> + }
> +
> + rxq->buf = devm_kcalloc(dev, chan_info->num_rxq_entries,
> + sizeof(*rxq->buf), GFP_KERNEL);
> + if (!rxq->buf)
> + return -ENOMEM;
> +
> + return 0;
> +}
> +
> +static int nbl_chan_init_queue(struct nbl_common_info *common,
> + struct nbl_chan_info *chan_info)
> +{
> + int err;
> +
> + err = nbl_chan_init_tx_queue(common, chan_info);
> + if (err)
> + return err;
> +
> + err = nbl_chan_init_rx_queue(common, chan_info);
> + return err;
> +}
> +
> +static void nbl_chan_config_queue(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info, bool tx)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
> + int size_bwid = ilog2(chan_info->num_rxq_entries);
Looks like this assignment is dead code as we are overwriting it later
in both the branches.
> + struct nbl_chan_ring *ring;
> + dma_addr_t dma_addr;
> +
> + if (tx)
> + ring = &chan_info->txq;
> + else
> + ring = &chan_info->rxq;
> + dma_addr = ring->dma;
> + if (tx) {
> + size_bwid = ilog2(chan_info->num_txq_entries);
> + hw_ops->config_mailbox_txq(p, dma_addr, size_bwid);
> + } else {
> + size_bwid = ilog2(chan_info->num_rxq_entries);
> + hw_ops->config_mailbox_rxq(p, dma_addr, size_bwid);
> + }
> +}
> +
> +static int nbl_chan_alloc_all_tx_bufs(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_chan_ring *txq = &chan_info->txq;
> + struct device *dev = chan_mgt->common->dev;
> + struct nbl_chan_buf *buf;
> + u16 i;
> +
> + for (i = 0; i < chan_info->num_txq_entries; i++) {
> + buf = &txq->buf[i];
> + buf->va = dmam_alloc_coherent(dev, chan_info->txq_buf_size,
> + &buf->pa,
> + GFP_KERNEL);
> + if (!buf->va) {
> + dev_err(dev,
> + "Allocate buffer for chan tx queue failed\n");
> + return -ENOMEM;
> + }
> + }
> +
> + txq->next_to_clean = 0;
> + txq->next_to_use = 0;
> + txq->tail_ptr = 0;
> +
> + return 0;
> +}
> +
> +static int
> +nbl_chan_cfg_mailbox_qinfo_map_table(struct nbl_channel_mgt *chan_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_common_info *common = chan_mgt->common;
> + struct nbl_hw_mgt *p = chan_mgt->hw_ops_tbl->priv;
> + u16 func_id;
> + u32 pf_mask;
> +
> + pf_mask = hw_ops->get_host_pf_mask(p);
> + for (func_id = 0; func_id < NBL_MAX_PF; func_id++) {
> + if (!(pf_mask & (1 << func_id)))
> + hw_ops->cfg_mailbox_qinfo(p, func_id, common->hw_bus,
> + common->devid,
> + common->function + func_id);
> + }
> +
> + return 0;
> +}
> +
> +static int nbl_chan_cfg_qinfo_map_table(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type)
> +{
> + return nbl_chan_cfg_mailbox_qinfo_map_table(chan_mgt);
> +}
> +
> +#define NBL_UPDATE_QUEUE_TAIL_PTR(chan_info, hw_ops, chan_mgt, tail_ptr, qid)\
> +do { \
> + (void)(chan_info); \
> + typeof(hw_ops) _hw_ops = (hw_ops); \
> + typeof(chan_mgt) _chan_mgt = (chan_mgt); \
> + typeof(tail_ptr) _tail_ptr = (tail_ptr); \
> + typeof(qid) _qid = (qid); \
> + (_hw_ops)->update_mailbox_queue_tail_ptr( \
> + _chan_mgt->hw_ops_tbl->priv, _tail_ptr, _qid); \
> +} while (0)
> +
> +static int nbl_chan_alloc_all_rx_bufs(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_chan_ring *rxq = &chan_info->rxq;
> + struct device *dev = chan_mgt->common->dev;
> + struct nbl_chan_rx_desc *desc;
> + struct nbl_chan_buf *buf;
> + u32 retry_times;
> + u16 i;
> +
> + for (i = 0; i < chan_info->num_rxq_entries; i++) {
> + buf = &rxq->buf[i];
> + buf->va = dmam_alloc_coherent(dev, chan_info->rxq_buf_size,
> + &buf->pa,
> + GFP_KERNEL);
> + if (!buf->va) {
> + dev_err(dev,
> + "Allocate buffer for chan rx queue failed\n");
> + goto err;
> + }
> + }
> +
> + desc = rxq->desc.rx_desc;
> + for (i = 0; i < chan_info->num_rxq_entries - 1; i++) {
> + buf = &rxq->buf[i];
> + desc[i].flags = BIT(NBL_CHAN_RX_DESC_AVAIL);
> + desc[i].buf_addr = buf->pa;
> + desc[i].buf_len = chan_info->rxq_buf_size;
> + }
> +
> + rxq->next_to_clean = 0;
> + rxq->next_to_use = chan_info->num_rxq_entries - 1;
> + rxq->tail_ptr = chan_info->num_rxq_entries - 1;
> +
> + NBL_UPDATE_QUEUE_TAIL_PTR(chan_info, hw_ops, chan_mgt, rxq->tail_ptr,
> + NBL_MB_RX_QID);
> +
> + for (retry_times = 0; retry_times < 3; retry_times++) {
> + NBL_UPDATE_QUEUE_TAIL_PTR(chan_info, hw_ops, chan_mgt,
> + rxq->tail_ptr, NBL_MB_RX_QID);
> + usleep_range(NBL_CHAN_TX_WAIT_US * 50,
> + NBL_CHAN_TX_WAIT_US * 60);
> + }
> +
Admittedly I have limited understanding here, but this loop looks
suspicious. We are updating the tail ptr 3 times without any
check/condition. Is this some sort of workaround to handle some HW
behavior? if so, a comment here would really help.
> + return 0;
> +err:
> + return -ENOMEM;
> +}
> +
> +static int nbl_chan_alloc_all_bufs(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + int err;
> +
> + err = nbl_chan_alloc_all_tx_bufs(chan_mgt, chan_info);
> + if (err)
> + return err;
> + err = nbl_chan_alloc_all_rx_bufs(chan_mgt, chan_info);
> + if (err)
> + goto alloc_rx_bufs_err;
Why do we need a label here if no cleanup is needed? How about simply
return nbl_chan_alloc_all_rx_bufs(chan_mgt, chan_info);
> +
> + return 0;
> +
> +alloc_rx_bufs_err:
> + return err;
> +}
> +
> +static void nbl_chan_stop_queue(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> +
> + hw_ops->stop_mailbox_rxq(chan_mgt->hw_ops_tbl->priv);
> + hw_ops->stop_mailbox_txq(chan_mgt->hw_ops_tbl->priv);
> +}
> +
> +static int nbl_chan_teardown_queue(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> +
> + nbl_chan_stop_queue(chan_mgt, chan_info);
Minor nit: A space here perhaps?
> + return 0;
> +}
> +
> +static int nbl_chan_setup_queue(struct nbl_channel_mgt *chan_mgt, u8 chan_type)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> + struct nbl_common_info *common = chan_mgt->common;
> + int err;
> +
> + nbl_chan_init_queue_param(chan_info, NBL_CHAN_QUEUE_LEN,
> + NBL_CHAN_QUEUE_LEN, NBL_CHAN_BUF_LEN,
> + NBL_CHAN_BUF_LEN);
> + err = nbl_chan_init_queue(common, chan_info);
> + if (err)
> + return err;
> +
> + nbl_chan_config_queue(chan_mgt, chan_info, true); /* tx */
> + nbl_chan_config_queue(chan_mgt, chan_info, false); /* rx */
> +
> + err = nbl_chan_alloc_all_bufs(chan_mgt, chan_info);
> + if (err)
> + goto chan_q_setup_fail;
> +
> + return 0;
> +
> +chan_q_setup_fail:
> + nbl_chan_teardown_queue(chan_mgt, chan_type);
> + return err;
> +}
> +
> +static int nbl_chan_update_txqueue(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info,
> + struct nbl_chan_tx_param *param)
> +{
> + struct nbl_chan_ring *txq = &chan_info->txq;
> + struct nbl_chan_tx_desc *tx_desc =
> + NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_use);
> + struct nbl_chan_buf *tx_buf =
> + NBL_CHAN_TX_RING_TO_BUF(txq, txq->next_to_use);
> +
> + if (param->arg_len > NBL_CHAN_BUF_LEN - sizeof(*tx_desc))
> + return -EINVAL;
> +
> + tx_desc->dstid = param->dstid;
> + tx_desc->msg_type = param->msg_type;
> + tx_desc->msgid = param->msgid;
> +
> + if (param->arg_len > NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN) {
> + memcpy(tx_buf->va, param->arg, param->arg_len);
> + tx_desc->buf_addr = tx_buf->pa;
> + tx_desc->buf_len = param->arg_len;
> + tx_desc->data_len = 0;
> + } else {
> + memcpy(tx_desc->data, param->arg, param->arg_len);
> + tx_desc->buf_len = 0;
> + tx_desc->data_len = param->arg_len;
> + }
> + tx_desc->flags = BIT(NBL_CHAN_TX_DESC_AVAIL);
> +
> + txq->next_to_use =
> + NBL_NEXT_ID(txq->next_to_use, chan_info->num_txq_entries - 1);
> + txq->tail_ptr++;
> +
> + return 0;
> +}
> +
> +static int nbl_chan_kick_tx_ring(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_chan_ring *txq = &chan_info->txq;
> + struct device *dev = chan_mgt->common->dev;
> + struct nbl_chan_tx_desc *tx_desc;
> + int i = 0;
> +
> + NBL_UPDATE_QUEUE_TAIL_PTR(chan_info, hw_ops, chan_mgt, txq->tail_ptr,
> + NBL_MB_TX_QID);
> +
> + tx_desc = NBL_CHAN_TX_RING_TO_DESC(txq, txq->next_to_clean);
> +
> + while (!(tx_desc->flags & BIT(NBL_CHAN_TX_DESC_USED))) {
> + udelay(NBL_CHAN_TX_WAIT_US);
> + i++;
> +
> + if (!(i % NBL_CHAN_TX_REKICK_WAIT_TIMES))
> + NBL_UPDATE_QUEUE_TAIL_PTR(chan_info, hw_ops, chan_mgt,
> + txq->tail_ptr, NBL_MB_TX_QID);
> +
> + if (i == NBL_CHAN_TX_WAIT_TIMES) {
> + dev_err(dev, "chan send message type: %d timeout\n",
> + tx_desc->msg_type);
> + return -ETIMEDOUT;
> + }
> + }
> +
> + txq->next_to_clean = txq->next_to_use;
nit here as well. This looks like a common issue with this patch.
> + return 0;
> +}
> +
> +static void nbl_chan_recv_ack_msg(void *priv, u16 srcid, u16 msgid, void *data,
> + u32 data_len)
> +{
> + struct nbl_channel_mgt *chan_mgt = (struct nbl_channel_mgt *)priv;
> + struct nbl_chan_waitqueue_head *wait_head = NULL;
> + union nbl_chan_msg_id ack_msgid = { { 0 } };
> + struct device *dev = chan_mgt->common->dev;
> + struct nbl_chan_info *chan_info =
> + chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
> + u32 *payload = (u32 *)data;
> + u32 ack_datalen;
> + u32 copy_len;
> +
> + if (data_len < 3 * sizeof(u32)) {
> + dev_err(dev, "Invalid ACK data_len: %u\n", data_len);
> + return;
> + }
> + ack_datalen = data_len - 3 * sizeof(u32);
> + ack_msgid.id = *(u16 *)(payload + 1);
> + if (ack_msgid.info.loc >= NBL_CHAN_QUEUE_LEN) {
> + dev_err(dev, "chan recv msg loc: %d err\n", ack_msgid.info.loc);
> + return;
> + }
> + wait_head = &chan_info->wait[ack_msgid.info.loc];
> + wait_head->ack_err = *(payload + 2);
> +
> + copy_len = min_t(u32, wait_head->ack_data_len, ack_datalen);
> + if (wait_head->ack_err >= 0 && copy_len > 0)
> + memcpy((char *)wait_head->ack_data, payload + 3, copy_len);
> + wait_head->ack_data_len = (u16)copy_len;
> +
> + /*
> + * Ensure all writes to ack_data and ack_data_len are completed
> + * before setting the 'acked' flag. This prevents other threads
> + * from observing stale or partially updated data.
> + */
> + wmb();
> + wait_head->acked = 1;
> + if (wait_head->need_waked)
> + wake_up(&wait_head->wait_queue);
> +}
> +
> +static void nbl_chan_recv_msg(struct nbl_channel_mgt *chan_mgt, void *data)
> +{
> + struct device *dev = chan_mgt->common->dev;
> + struct nbl_chan_msg_node_data *msg_handler;
> + u16 msg_type, payload_len, srcid, msgid;
> + struct nbl_chan_tx_desc *tx_desc;
> + void *payload;
> +
> + tx_desc = data;
> + msg_type = tx_desc->msg_type;
> + dev_dbg(dev, "recv msg_type: %d\n", tx_desc->msg_type);
> +
> + srcid = tx_desc->srcid;
> + msgid = tx_desc->msgid;
> + /* Only check if the value exceeds the maximum, relying on the hash
> + * table to filter invalid message IDs.
> + * The gap values are reserved for future protocol extensions.
> + */
> + if (msg_type >= NBL_CHAN_MSG_MAILBOX_MAX)
> + return;
> +
> + if (tx_desc->data_len) {
> + payload = (void *)tx_desc->data;
> + payload_len = tx_desc->data_len;
> + } else {
> + payload = (void *)(tx_desc + 1);
> + payload_len = tx_desc->buf_len;
> + }
> +
> + msg_handler =
> + nbl_common_get_hash_node(chan_mgt->handle_hash_tbl, &msg_type);
> + if (!msg_handler) {
> + pr_err("Invalid msg_type: %d\n", msg_type);
> + return;
> + }
> + msg_handler->func(msg_handler->priv, srcid, msgid, payload,
> + payload_len);
> +}
> +
> +static void nbl_chan_advance_rx_ring(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info,
> + struct nbl_chan_ring *rxq)
> +{
> + struct nbl_hw_ops *hw_ops = chan_mgt->hw_ops_tbl->ops;
> + struct nbl_chan_rx_desc *rx_desc;
> + struct nbl_chan_buf *rx_buf;
> + u16 next_to_use;
> +
> + next_to_use = rxq->next_to_use;
> + rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_use);
> + rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_use);
> +
> + rx_desc->flags = BIT(NBL_CHAN_RX_DESC_AVAIL);
> + rx_desc->buf_addr = rx_buf->pa;
> + rx_desc->buf_len = chan_info->rxq_buf_size;
> +
> + rxq->next_to_use++;
> + if (rxq->next_to_use == chan_info->num_rxq_entries)
> + rxq->next_to_use = 0;
> + rxq->tail_ptr++;
> +
> + NBL_UPDATE_QUEUE_TAIL_PTR(chan_info, hw_ops, chan_mgt, rxq->tail_ptr,
> + NBL_MB_RX_QID);
> +}
> +
> +static void nbl_chan_clean_queue(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_info *chan_info)
> +{
> + struct nbl_chan_ring *rxq = &chan_info->rxq;
> + struct device *dev = chan_mgt->common->dev;
> + struct nbl_chan_rx_desc *rx_desc;
> + struct nbl_chan_buf *rx_buf;
> + u16 next_to_clean;
> +
> + next_to_clean = rxq->next_to_clean;
> + rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
> + rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
> + while (rx_desc->flags & BIT(NBL_CHAN_RX_DESC_USED)) {
> + if (!(rx_desc->flags & BIT(NBL_CHAN_RX_DESC_WRITE)))
> + dev_dbg(dev,
> + "mailbox rx flag 0x%x has no NBL_CHAN_RX_DESC_WRITE\n",
> + rx_desc->flags);
> +
> + dma_rmb();
> + nbl_chan_recv_msg(chan_mgt, rx_buf->va);
> + nbl_chan_advance_rx_ring(chan_mgt, chan_info, rxq);
> + next_to_clean++;
> + if (next_to_clean == chan_info->num_rxq_entries)
> + next_to_clean = 0;
> + rx_desc = NBL_CHAN_RX_RING_TO_DESC(rxq, next_to_clean);
> + rx_buf = NBL_CHAN_RX_RING_TO_BUF(rxq, next_to_clean);
> + }
> + rxq->next_to_clean = next_to_clean;
> +}
> +
> +static void nbl_chan_clean_queue_subtask(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> +
> + if (!test_bit(NBL_CHAN_INTERRUPT_READY, chan_info->state) ||
> + test_bit(NBL_CHAN_RESETTING, chan_info->state))
> + return;
> +
> + nbl_chan_clean_queue(chan_mgt, chan_info);
> +}
> +
> +static int nbl_chan_get_msg_id(struct nbl_chan_info *chan_info,
> + union nbl_chan_msg_id *msgid)
> +{
> + int valid_loc = chan_info->wait_head_index, i;
> + struct nbl_chan_waitqueue_head *wait = NULL;
> +
> + for (i = 0; i < NBL_CHAN_QUEUE_LEN; i++) {
> + wait = &chan_info->wait[valid_loc];
> +
> + if (wait->status != NBL_MBX_STATUS_WAITING) {
> + wait->msg_index = NBL_NEXT_ID(wait->msg_index,
> + NBL_CHAN_MSG_INDEX_MAX);
> + msgid->info.index = wait->msg_index;
> + msgid->info.loc = valid_loc;
> +
> + valid_loc = NBL_NEXT_ID(valid_loc,
> + chan_info->num_txq_entries - 1);
> + chan_info->wait_head_index = valid_loc;
> + return 0;
> + }
> +
> + valid_loc =
> + NBL_NEXT_ID(valid_loc, chan_info->num_txq_entries - 1);
> + }
> + /*
> + * the current NBL_CHAN_QUEUE_LEN configuration meets the design
> + * requirements and theoretically should not return errors, the
> + * following scenarios may still cause the waiting queue to
> + * become full:
> + * High-concurrency scenarios:
> + * If the sender (calling nbl_chan_send_msg()) generates messages
> + * at a rate far exceeding the receiver's ability to process
> + * acknowledgments (ACKs),the waiting queue may become fully occupied.
> + * Delayed or failed ACK handling by the receiver:
> + * The receiver may fail to send ACKs in a timely manner due to
> + * processing delays, blocking, or faults, causing the sender's
> + * waiting queue slots to remain occupied for an extended period.
> + */
> + return -EAGAIN;
> +}
> +
> +static int nbl_chan_send_msg(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_send_info *chan_send)
> +{
> + struct nbl_common_info *common = chan_mgt->common;
> + struct nbl_chan_waitqueue_head *wait_head;
> + struct nbl_chan_tx_param tx_param = { 0 };
> + union nbl_chan_msg_id msgid = { { 0 } };
> + int i = NBL_CHAN_TX_WAIT_ACK_TIMES, ret;
> + struct nbl_chan_info *chan_info =
> + chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX];
> + struct device *dev = common->dev;
> +
> + if (test_bit(NBL_CHAN_ABNORMAL, chan_info->state))
> + return -EIO;
> +
> + mutex_lock(&chan_info->txq_lock);
> +
> + ret = nbl_chan_get_msg_id(chan_info, &msgid);
> + if (ret) {
> + mutex_unlock(&chan_info->txq_lock);
> + dev_err(dev,
> + "Channel tx wait head full, send msgtype:%u to dstid:%u failed\n",
> + chan_send->msg_type, chan_send->dstid);
> + return ret;
> + }
> +
> + tx_param.msg_type = chan_send->msg_type;
> + tx_param.arg = chan_send->arg;
> + tx_param.arg_len = chan_send->arg_len;
> + tx_param.dstid = chan_send->dstid;
> + tx_param.msgid = msgid.id;
> +
> + ret = nbl_chan_update_txqueue(chan_mgt, chan_info, &tx_param);
> + if (ret) {
> + mutex_unlock(&chan_info->txq_lock);
> + dev_err(dev,
> + "Channel tx queue full, send msgtype:%u to dstid:%u failed\n",
> + chan_send->msg_type, chan_send->dstid);
> + return ret;
> + }
> +
> + wait_head = &chan_info->wait[msgid.info.loc];
> + init_waitqueue_head(&wait_head->wait_queue);
> + wait_head->acked = 0;
> + wait_head->ack_data = chan_send->resp;
> + wait_head->ack_data_len = chan_send->resp_len;
> + wait_head->msg_type = chan_send->msg_type;
> + wait_head->need_waked = chan_send->ack;
> + wait_head->msg_index = msgid.info.index;
> + wait_head->status = chan_send->ack ? NBL_MBX_STATUS_WAITING :
> + NBL_MBX_STATUS_IDLE;
> +
> + ret = nbl_chan_kick_tx_ring(chan_mgt, chan_info);
> +
> + mutex_unlock(&chan_info->txq_lock);
> + if (ret) {
> + wait_head->status = NBL_MBX_STATUS_TIMEOUT;
> + return ret;
> + }
> + if (!chan_send->ack)
> + return 0;
> +
> + if (test_bit(NBL_CHAN_INTERRUPT_READY, chan_info->state)) {
> + ret = wait_event_timeout(wait_head->wait_queue,
> + wait_head->acked,
> + NBL_CHAN_ACK_WAIT_TIME);
> + if (!ret) {
> + wait_head->status = NBL_MBX_STATUS_TIMEOUT;
> + dev_err(dev,
> + "Channel waiting ack failed, message type: %d, msg id: %u\n",
> + chan_send->msg_type, msgid.id);
> + return -ETIMEDOUT;
> + }
> +
> + /*
> + * ensure that after observing 'acked == 1', all subsequent
> + * reads (ack_data_len, ack_err) observe the latest values
> + * written by the sender (nbl_chan_recv_ack_msg()). This
> + * prevents stale reads of ACK data or status.
> + */
> + rmb();
> + chan_send->ack_len = wait_head->ack_data_len;
> + wait_head->status = NBL_MBX_STATUS_IDLE;
> + return wait_head->ack_err;
> + }
> +
> + /*polling wait mailbox ack*/
> + while (i--) {
> + nbl_chan_clean_queue(chan_mgt, chan_info);
> +
> + if (wait_head->acked) {
> + chan_send->ack_len = wait_head->ack_data_len;
> + wait_head->status = NBL_MBX_STATUS_IDLE;
> + return wait_head->ack_err;
> + }
> + usleep_range(NBL_CHAN_TX_WAIT_ACK_US_MIN,
> + NBL_CHAN_TX_WAIT_ACK_US_MAX);
> + }
> +
> + wait_head->status = NBL_MBX_STATUS_TIMEOUT;
> + dev_err(dev,
> + "Channel polling ack failed, message type: %d msg id: %u\n",
> + chan_send->msg_type, msgid.id);
> + return -EFAULT;
> +}
> +
> +static int nbl_chan_send_ack(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_ack_info *chan_ack)
> +{
> + u32 len = 3 * sizeof(u32) + chan_ack->data_len;
> + struct nbl_chan_send_info chan_send;
> + u32 *tmp;
> + int ret;
> +
> + tmp = kzalloc(len, GFP_KERNEL);
> + if (!tmp)
> + return -ENOMEM;
> +
> + tmp[0] = chan_ack->msg_type;
> + tmp[1] = chan_ack->msgid;
> + tmp[2] = (u32)chan_ack->err;
> + if (chan_ack->data && chan_ack->data_len)
> + memcpy(&tmp[3], chan_ack->data, chan_ack->data_len);
> +
> + NBL_CHAN_SEND(chan_send, chan_ack->dstid, NBL_CHAN_MSG_ACK, tmp, len,
> + NULL, 0, 0);
> + ret = nbl_chan_send_msg(chan_mgt, &chan_send);
> + kfree(tmp);
> +
> + return ret;
> +}
> +
> +static int nbl_chan_register_msg(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
> + nbl_chan_resp func, void *callback)
> +{
> + return nbl_chan_add_msg_handler(chan_mgt, msg_type, func, callback);
> +}
> +
> +static bool nbl_chan_check_queue_exist(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> +
> + return chan_info ? true : false;
Perhaps we can simplify this as return !!chan_info;
> +}
> +
> +static void nbl_chan_register_chan_task(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type, struct work_struct *task)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> +
> + chan_info->clean_task = task;
> +}
> +
> +static void nbl_chan_set_queue_state(struct nbl_channel_mgt *chan_mgt,
> + enum nbl_chan_state state, u8 chan_type,
> + u8 set)
> +{
> + struct nbl_chan_info *chan_info = chan_mgt->chan_info[chan_type];
> +
> + if (set)
> + set_bit(state, chan_info->state);
> + else
> + clear_bit(state, chan_info->state);
> +}
> +
> static struct nbl_channel_ops chan_ops = {
> + .send_msg = nbl_chan_send_msg,
> + .send_ack = nbl_chan_send_ack,
> + .register_msg = nbl_chan_register_msg,
> + .cfg_chan_qinfo_map_table = nbl_chan_cfg_qinfo_map_table,
> + .check_queue_exist = nbl_chan_check_queue_exist,
> + .setup_queue = nbl_chan_setup_queue,
> + .teardown_queue = nbl_chan_teardown_queue,
> + .clean_queue_subtask = nbl_chan_clean_queue_subtask,
> + .register_chan_task = nbl_chan_register_chan_task,
> + .set_queue_state = nbl_chan_set_queue_state,
> };
>
> static struct nbl_channel_mgt *
> @@ -18,6 +769,7 @@ nbl_chan_setup_chan_mgt(struct nbl_adapter *adapter)
> struct device *dev = &adapter->pdev->dev;
> struct nbl_chan_info *mailbox;
> struct nbl_channel_mgt *chan_mgt;
> + int ret;
>
> chan_mgt = devm_kzalloc(dev, sizeof(*chan_mgt), GFP_KERNEL);
> if (!chan_mgt)
> @@ -32,6 +784,10 @@ nbl_chan_setup_chan_mgt(struct nbl_adapter *adapter)
> mailbox->chan_type = NBL_CHAN_TYPE_MAILBOX;
> chan_mgt->chan_info[NBL_CHAN_TYPE_MAILBOX] = mailbox;
>
> + ret = nbl_chan_init_msg_handler(chan_mgt);
> + if (ret)
> + return ERR_PTR(-ENOMEM);
> +
> return chan_mgt;
> }
>
> @@ -39,6 +795,7 @@ static struct nbl_channel_ops_tbl *
> nbl_chan_setup_ops(struct device *dev, struct nbl_channel_mgt *chan_mgt)
> {
> struct nbl_channel_ops_tbl *chan_ops_tbl;
> + int ret;
>
> chan_ops_tbl = devm_kzalloc(dev, sizeof(*chan_ops_tbl), GFP_KERNEL);
> if (!chan_ops_tbl)
> @@ -47,6 +804,11 @@ nbl_chan_setup_ops(struct device *dev, struct nbl_channel_mgt *chan_mgt)
> chan_ops_tbl->ops = &chan_ops;
> chan_ops_tbl->priv = chan_mgt;
>
> + ret = nbl_chan_register_msg(chan_mgt, NBL_CHAN_MSG_ACK,
> + nbl_chan_recv_ack_msg, chan_mgt);
> + if (ret)
> + return ERR_PTR(-ENOMEM);
> +
> return chan_ops_tbl;
> }
>
> @@ -73,10 +835,14 @@ int nbl_chan_init_common(struct nbl_adapter *adap)
> return 0;
>
> setup_ops_fail:
> + nbl_chan_remove_msg_handler(chan_mgt);
> setup_mgt_fail:
> return ret;
> }
>
> void nbl_chan_remove_common(struct nbl_adapter *adap)
> {
> + struct nbl_channel_mgt *chan_mgt = adap->core.chan_mgt;
> +
> + nbl_chan_remove_msg_handler(chan_mgt);
> }
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> index 637912d1e806..901b1d7166fe 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_channel/nbl_channel.h
> @@ -15,10 +15,139 @@
> #include "../nbl_include/nbl_def_common.h"
> #include "../nbl_core.h"
>
> +#define NBL_CHAN_TX_RING_TO_DESC(tx_ring, i) \
> + (&((((tx_ring)->desc.tx_desc))[i]))
> +#define NBL_CHAN_RX_RING_TO_DESC(rx_ring, i) \
> + (&((((rx_ring)->desc.rx_desc))[i]))
> +#define NBL_CHAN_TX_RING_TO_BUF(tx_ring, i) (&(((tx_ring)->buf)[i]))
> +#define NBL_CHAN_RX_RING_TO_BUF(rx_ring, i) (&(((rx_ring)->buf)[i]))
> +
> +#define NBL_CHAN_TX_WAIT_US 100
> +#define NBL_CHAN_TX_REKICK_WAIT_TIMES 2000
> +#define NBL_CHAN_TX_WAIT_TIMES 30000
> +#define NBL_CHAN_TX_WAIT_ACK_US_MIN 100
> +#define NBL_CHAN_TX_WAIT_ACK_US_MAX 120
> +#define NBL_CHAN_TX_WAIT_ACK_TIMES 50000
> +#define NBL_CHAN_QUEUE_LEN 256
> +#define NBL_CHAN_BUF_LEN 4096
> +#define NBL_CHAN_TX_DESC_EMBEDDED_DATA_LEN 16
> +
> +#define NBL_CHAN_TX_DESC_AVAIL 0
> +#define NBL_CHAN_TX_DESC_USED 1
> +#define NBL_CHAN_RX_DESC_WRITE 1
> +#define NBL_CHAN_RX_DESC_AVAIL 3
> +#define NBL_CHAN_RX_DESC_USED 4
> +
> +#define NBL_CHAN_ACK_WAIT_TIME (3 * HZ)
> +
> +#define NBL_CHAN_HANDLER_TBL_BUCKET_SIZE 512
> +
> +enum {
> + NBL_MB_RX_QID = 0,
> + NBL_MB_TX_QID = 1,
> +};
> +
> +enum {
> + NBL_MBX_STATUS_IDLE = 0,
> + NBL_MBX_STATUS_WAITING,
> + NBL_MBX_STATUS_TIMEOUT = -1,
> +};
> +
> +struct nbl_chan_tx_param {
> + enum nbl_chan_msg_type msg_type;
> + void *arg;
> + size_t arg_len;
> + u16 dstid;
> + u16 msgid;
> +};
> +
> +struct nbl_chan_buf {
> + void *va;
> + dma_addr_t pa;
> + size_t size;
> +};
> +
> +struct nbl_chan_tx_desc {
> + u16 flags;
> + u16 srcid;
> + u16 dstid;
> + u16 data_len;
> + u16 buf_len;
> + u64 buf_addr;
> + u16 msg_type;
> + u8 data[16];
> + u16 msgid;
> + u8 rsv[26];
> +} __packed;
> +
> +struct nbl_chan_rx_desc {
> + u16 flags;
> + u32 buf_len;
> + u16 buf_id;
> + u64 buf_addr;
> +} __packed;
> +
> +union nbl_chan_desc_ptr {
> + struct nbl_chan_tx_desc *tx_desc;
> + struct nbl_chan_rx_desc *rx_desc;
> +};
> +
> +struct nbl_chan_ring {
> + union nbl_chan_desc_ptr desc;
> + struct nbl_chan_buf *buf;
> + u16 next_to_use;
> + u16 tail_ptr;
> + u16 next_to_clean;
> + dma_addr_t dma;
> +};
> +
> +#define NBL_CHAN_MSG_INDEX_MAX 63
> +
> +union nbl_chan_msg_id {
> + struct nbl_chan_msg_id_info {
> + u16 index : 6;
> + u16 loc : 10;
> + } info;
> + u16 id;
> +};
> +
> +struct nbl_chan_waitqueue_head {
> + struct wait_queue_head wait_queue;
> + char *ack_data;
> + int acked;
> + int ack_err;
> + u16 ack_data_len;
> + u16 need_waked;
> + u16 msg_type;
> + u8 status;
> + u8 msg_index;
> +};
> +
> struct nbl_chan_info {
> + struct nbl_chan_ring txq;
> + struct nbl_chan_ring rxq;
> + struct nbl_chan_waitqueue_head *wait;
> + /*
> + *Protects access to the TX queue (txq) and related metadata.
> + *This mutex ensures exclusive access when updating the TX queue
> + *or waiting for ACKs to prevent race conditions.
> + */
> + struct mutex txq_lock;
> + struct work_struct *clean_task;
> + u16 wait_head_index;
> + u16 num_txq_entries;
> + u16 num_rxq_entries;
> + u16 txq_buf_size;
> + u16 rxq_buf_size;
> + DECLARE_BITMAP(state, NBL_CHAN_STATE_NBITS);
> u8 chan_type;
> };
>
> +struct nbl_chan_msg_node_data {
> + nbl_chan_resp func;
> + void *priv;
> +};
> +
> struct nbl_channel_mgt {
> struct nbl_common_info *common;
> struct nbl_hw_ops_tbl *hw_ops_tbl;
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> new file mode 100644
> index 000000000000..5f75902bbdc0
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.c
> @@ -0,0 +1,207 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#include <linux/device.h>
> +#include "nbl_common.h"
> +
> +static struct nbl_common_wq_mgt *wq_mgt;
> +
> +void nbl_common_queue_work(struct work_struct *task)
> +{
> + queue_work(wq_mgt->ctrl_dev_wq, task);
> +}
> +
> +void nbl_common_destroy_wq(void)
> +{
> + destroy_workqueue(wq_mgt->ctrl_dev_wq);
> + kfree(wq_mgt);
> +}
> +
> +int nbl_common_create_wq(void)
> +{
> + wq_mgt = kzalloc_obj(*wq_mgt);
> + if (!wq_mgt)
> + return -ENOMEM;
> +
> + wq_mgt->ctrl_dev_wq = alloc_workqueue("%s", WQ_MEM_RECLAIM | WQ_UNBOUND,
> + 0, "nbl_ctrldev_wq");
> + if (!wq_mgt->ctrl_dev_wq) {
> + pr_err("Failed to create workqueue nbl_ctrldev_wq\n");
> + goto alloc_ctrl_dev_wq_failed;
> + }
> +
> + return 0;
> +alloc_ctrl_dev_wq_failed:
> + kfree(wq_mgt);
> + return -ENOMEM;
> +}
> +
> +u32 nbl_common_pf_id_subtraction_mgtpf_id(struct nbl_common_info *common,
> + u32 pf_id)
> +{
> + u32 diff = U32_MAX;
> +
> + if (pf_id >= common->mgt_pf)
> + diff = pf_id - common->mgt_pf;
> +
> + return diff;
> +}
> +
> +static u32 nbl_common_calc_hash_key(void *key, u32 key_size, u32 bucket_size)
> +{
> + u32 value = 0;
> + u32 hash_val;
> + u32 i;
> +
> + if (bucket_size == NBL_HASH_TBL_LIST_BUCKET_SIZE)
> + return 0;
> +
> + for (i = 0; i < key_size; i++)
> + value += *((u8 *)key + i);
> +
> + hash_val = __hash_32(value);
> +
> + return hash_val % bucket_size;
> +}
The hash looks pretty primitive. Looks like keys with same bytes but in
different order would collide. Mathmatically speaking, we are destroying
entropy before the actual hash even runs.
> +
> +/*
> + * alloc a hash table
> + * the table support multi thread
> + */
> +struct nbl_hash_tbl_mgt *
> +nbl_common_init_hash_table(struct nbl_hash_tbl_key *key)
> +{
> + struct nbl_hash_tbl_mgt *tbl_mgt;
> + int bucket_size;
> + int i;
> +
> + tbl_mgt = devm_kzalloc(key->dev, sizeof(*tbl_mgt), GFP_KERNEL);
> + if (!tbl_mgt)
> + return NULL;
> +
> + bucket_size = key->bucket_size;
> + tbl_mgt->hash = devm_kcalloc(key->dev, bucket_size,
> + sizeof(struct hlist_head), GFP_KERNEL);
> + if (!tbl_mgt->hash)
> + goto alloc_hash_failed;
> +
> + for (i = 0; i < bucket_size; i++)
> + INIT_HLIST_HEAD(tbl_mgt->hash + i);
> +
> + memcpy(&tbl_mgt->tbl_key, key, sizeof(struct nbl_hash_tbl_key));
> +
> + return tbl_mgt;
> +
> +alloc_hash_failed:
> + return NULL;
> +}
> +
> +/*
> + * The number of nodes in the hash table is guaranteed to be bounded
> + * (as defined in nbl_disp_setup_msg).
> + * So all hash nodes (struct nbl_hash_entry_node) and their associated keys/data
> + * are allocated via devm_kzalloc() and will be automatically freed when the
> + * device is removed. Functions like nbl_common_detach_hash_node() only remove
> + * nodes from the list but do not trigger immediate memory deallocation.
> + */
> +int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
> + void *data, void **out_data)
> +{
> + struct nbl_hash_entry_node *hash_node;
> + u16 data_size;
> + u32 hash_val;
> + u16 key_size;
> +
> + hash_node = devm_kzalloc(tbl_mgt->tbl_key.dev, sizeof(*hash_node),
> + GFP_KERNEL);
> + if (!hash_node)
> + return -ENOMEM;
> +
> + key_size = tbl_mgt->tbl_key.key_size;
> + hash_node->key =
> + devm_kzalloc(tbl_mgt->tbl_key.dev, key_size, GFP_KERNEL);
> + if (!hash_node->key)
> + return -ENOMEM;
> +
> + data_size = tbl_mgt->tbl_key.data_size;
> + hash_node->data =
> + devm_kzalloc(tbl_mgt->tbl_key.dev, data_size, GFP_KERNEL);
> + if (!hash_node->data)
> + return -ENOMEM;
> +
> + memcpy(hash_node->key, key, key_size);
> + memcpy(hash_node->data, data, data_size);
> +
> + hash_val = nbl_common_calc_hash_key(key, key_size,
> + tbl_mgt->tbl_key.bucket_size);
> +
> + hlist_add_head(&hash_node->node, tbl_mgt->hash + hash_val);
> + tbl_mgt->node_num++;
> + if (out_data)
> + *out_data = hash_node->data;
> +
> + return 0;
> +}
> +
> +/*
> + * get a hash node, return the data if node exist
> + */
> +void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key)
> +{
> + struct nbl_hash_entry_node *hash_node;
> + struct hlist_head *head;
> + void *data = NULL;
> + u32 hash_val;
> + u16 key_size;
> +
> + key_size = tbl_mgt->tbl_key.key_size;
> + hash_val = nbl_common_calc_hash_key(key, key_size,
> + tbl_mgt->tbl_key.bucket_size);
> + head = tbl_mgt->hash + hash_val;
> +
> + hlist_for_each_entry(hash_node, head, node)
> + if (!memcmp(hash_node->key, key, key_size)) {
> + data = hash_node->data;
> + break;
> + }
> +
> + return data;
> +}
> +
> +/*
> + * Detaches the node from the hash list but does NOT free the memory.
> + * Memory is managed by devm and will be released automatically
> + * when the device is removed.
> + */
> +static void nbl_common_detach_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt,
> + struct nbl_hash_entry_node *hash_node)
> +{
> + hlist_del(&hash_node->node);
> + tbl_mgt->node_num--;
> +}
> +
> +/*
> + * Detaches all nodes from the hash table but does NOT free their memory.
> + * Memory will be released automatically by devm when the device is removed.
> + */
> +void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt,
> + struct nbl_hash_tbl_del_key *key)
> +{
> + struct nbl_hash_entry_node *hash_node;
> + struct hlist_node *safe_node;
> + struct hlist_head *head;
> + u32 i;
> +
> + for (i = 0; i < tbl_mgt->tbl_key.bucket_size; i++) {
> + head = tbl_mgt->hash + i;
> + hlist_for_each_entry_safe(hash_node, safe_node, head, node) {
> + if (key && key->action_func)
> + key->action_func(key->action_priv,
> + hash_node->key,
> + hash_node->data);
> + nbl_common_detach_hash_node(tbl_mgt, hash_node);
> + }
> + }
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
> new file mode 100644
> index 000000000000..7d628d3556ee
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_common/nbl_common.h
> @@ -0,0 +1,34 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_COMMON_H_
> +#define _NBL_COMMON_H_
> +
> +#include <linux/types.h>
> +
> +#include "../nbl_include/nbl_include.h"
> +#include "../nbl_include/nbl_def_common.h"
> +
> +/* list only need one bucket size */
> +#define NBL_HASH_TBL_LIST_BUCKET_SIZE 1
> +
> +struct nbl_common_wq_mgt {
> + struct workqueue_struct *ctrl_dev_wq;
> +};
> +
> +struct nbl_hash_tbl_mgt {
> + struct nbl_hash_tbl_key tbl_key;
> + struct hlist_head *hash;
> + u16 node_num;
> +};
> +
> +/* it used for y_axis no necessay */
> +struct nbl_hash_entry_node {
> + struct hlist_node node;
> + void *key;
> + void *data;
> +};
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index 6dd8d3a087dd..83a4dc584f48 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -9,7 +9,150 @@
> #include <linux/spinlock.h>
> #include "nbl_hw_leonis.h"
>
> +static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
> + const u32 *data, u32 len)
> +{
> + u32 i;
> +
> + if (len % 4)
> + return;
> +
> + for (i = 0; i < len / 4; i++)
> + nbl_mbx_wr32(hw_mgt, reg + i * sizeof(u32), data[i]);
> +}
> +
> +static void nbl_hw_rd_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, u32 *data,
> + u32 len)
> +{
> + u32 size = len / 4;
> + u32 i;
> +
> + if (len % 4)
> + return;
> +
> + spin_lock(&hw_mgt->reg_lock);
> +
> + for (i = 0; i < size; i++)
> + data[i] = rd32(hw_mgt->hw_addr, reg + i * sizeof(u32));
> + spin_unlock(&hw_mgt->reg_lock);
> +}
> +
> +static void nbl_hw_wr_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, const u32 *data,
> + u32 len)
> +{
> + u32 size = len / 4;
> + u32 i;
> +
> + if (len % 4)
> + return;
> + spin_lock(&hw_mgt->reg_lock);
> + for (i = 0; i < size; i++)
> + wr32(hw_mgt->hw_addr, reg + i * sizeof(u32), data[i]);
> + spin_unlock(&hw_mgt->reg_lock);
> +}
> +
> +static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
> + u16 tail_ptr, u8 txrx)
> +{
> + /* local_qid 0 and 1 denote rx and tx queue respectively */
> + u32 local_qid = txrx;
> + u32 value = ((u32)tail_ptr << 16) | local_qid;
> +
> + /* wmb for doorbell */
> + wmb();
> + nbl_mbx_wr32(hw_mgt, NBL_MAILBOX_NOTIFY_ADDR, value);
> +}
> +
> +static void nbl_hw_config_mailbox_rxq(struct nbl_hw_mgt *hw_mgt,
> + dma_addr_t dma_addr, int size_bwid)
> +{
> + struct nbl_mailbox_qinfo_cfg_table qinfo_cfg_rx_table = { 0 };
> +
> + qinfo_cfg_rx_table.queue_rst = 1;
> + nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
> + (u32 *)&qinfo_cfg_rx_table,
> + sizeof(qinfo_cfg_rx_table));
> +
> + qinfo_cfg_rx_table.queue_base_addr_l = (u32)(dma_addr & 0xFFFFFFFF);
> + qinfo_cfg_rx_table.queue_base_addr_h = (u32)(dma_addr >> 32);
> + qinfo_cfg_rx_table.queue_size_bwind = (u32)size_bwid;
> + qinfo_cfg_rx_table.queue_rst = 0;
> + qinfo_cfg_rx_table.queue_en = 1;
> + nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
> + (u32 *)&qinfo_cfg_rx_table,
> + sizeof(qinfo_cfg_rx_table));
> +}
> +
> +static void nbl_hw_config_mailbox_txq(struct nbl_hw_mgt *hw_mgt,
> + dma_addr_t dma_addr, int size_bwid)
> +{
> + struct nbl_mailbox_qinfo_cfg_table qinfo_cfg_tx_table = { 0 };
> +
> + qinfo_cfg_tx_table.queue_rst = 1;
> + nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
> + (u32 *)&qinfo_cfg_tx_table,
> + sizeof(qinfo_cfg_tx_table));
> +
> + qinfo_cfg_tx_table.queue_base_addr_l = (u32)(dma_addr & 0xFFFFFFFF);
> + qinfo_cfg_tx_table.queue_base_addr_h = (u32)(dma_addr >> 32);
> + qinfo_cfg_tx_table.queue_size_bwind = (u32)size_bwid;
> + qinfo_cfg_tx_table.queue_rst = 0;
> + qinfo_cfg_tx_table.queue_en = 1;
> + nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
> + (u32 *)&qinfo_cfg_tx_table,
> + sizeof(qinfo_cfg_tx_table));
> +}
> +
> +static void nbl_hw_stop_mailbox_rxq(struct nbl_hw_mgt *hw_mgt)
> +{
> + struct nbl_mailbox_qinfo_cfg_table qinfo_cfg_rx_table = { 0 };
> +
> + nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_RX_TABLE_ADDR,
> + (u32 *)&qinfo_cfg_rx_table,
> + sizeof(qinfo_cfg_rx_table));
> +}
> +
> +static void nbl_hw_stop_mailbox_txq(struct nbl_hw_mgt *hw_mgt)
> +{
> + struct nbl_mailbox_qinfo_cfg_table qinfo_cfg_tx_table = { 0 };
> +
> + nbl_hw_write_mbx_regs(hw_mgt, NBL_MAILBOX_QINFO_CFG_TX_TABLE_ADDR,
> + (u32 *)&qinfo_cfg_tx_table,
> + sizeof(qinfo_cfg_tx_table));
> +}
> +
> +static u32 nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt)
> +{
> + u32 data;
> +
> + nbl_hw_rd_regs(hw_mgt, NBL_PCIE_HOST_K_PF_MASK_REG, &data,
> + sizeof(data));
> + return data;
> +}
> +
> +static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + u16 bus, u16 devid, u16 function)
> +{
> + struct nbl_mailbox_qinfo_map_table mb_qinfo_map;
> +
> + memset(&mb_qinfo_map, 0, sizeof(mb_qinfo_map));
> + mb_qinfo_map.function = function;
> + mb_qinfo_map.devid = devid;
> + mb_qinfo_map.bus = bus;
> + mb_qinfo_map.msix_idx_valid = 0;
> + nbl_hw_wr_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
> + (u32 *)&mb_qinfo_map, sizeof(mb_qinfo_map));
> +}
> +
> static struct nbl_hw_ops hw_ops = {
> + .update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
> + .config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
> + .config_mailbox_txq = nbl_hw_config_mailbox_txq,
> + .stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
> + .stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
> + .get_host_pf_mask = nbl_hw_get_host_pf_mask,
> + .cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
> +
> };
>
> /* Structure starts here, adding an op should not modify anything below */
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> index 99e1d8455e0e..39d23160a7a4 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_channel.h
> @@ -6,8 +6,42 @@
> #ifndef _NBL_DEF_CHANNEL_H_
> #define _NBL_DEF_CHANNEL_H_
>
> +#include <linux/types.h>
> +
> struct nbl_channel_mgt;
> struct nbl_adapter;
> +#define NBL_CHAN_SEND(chan_send, dst_id, mesg_type, argument, arg_length,\
> + response, resp_length, need_ack) \
> +do { \
> + typeof(chan_send) *__chan_send = &(chan_send); \
> + __chan_send->dstid = (dst_id); \
> + __chan_send->msg_type = (mesg_type); \
> + __chan_send->arg = (argument); \
> + __chan_send->arg_len = (arg_length); \
> + __chan_send->resp = (response); \
> + __chan_send->resp_len = (resp_length); \
> + __chan_send->ack = (need_ack); \
> +} while (0)
> +
> +#define NBL_CHAN_ACK(chan_ack, dst_id, mesg_type, msg_id, err_code, ack_data, \
> + data_length) \
> +do { \
> + typeof(chan_ack) *__chan_ack = &(chan_ack); \
> + __chan_ack->dstid = (dst_id); \
> + __chan_ack->msg_type = (mesg_type); \
> + __chan_ack->msgid = (msg_id); \
> + __chan_ack->err = (err_code); \
> + __chan_ack->data = (ack_data); \
> + __chan_ack->data_len = (data_length); \
> +} while (0)
> +
> +typedef void (*nbl_chan_resp)(void *, u16, u16, void *, u32);
> +
> +enum {
> + NBL_CHAN_RESP_OK,
> + NBL_CHAN_RESP_ERR,
> +};
> +
> enum nbl_chan_msg_type {
> NBL_CHAN_MSG_ACK,
> NBL_CHAN_MSG_ADD_MACVLAN,
> @@ -234,6 +268,13 @@ enum nbl_chan_msg_type {
> NBL_CHAN_MSG_MAILBOX_MAX,
> };
>
> +enum nbl_chan_state {
> + NBL_CHAN_INTERRUPT_READY,
> + NBL_CHAN_RESETTING,
> + NBL_CHAN_ABNORMAL,
> + NBL_CHAN_STATE_NBITS
> +};
> +
> struct nbl_chan_param_cfg_msix_map {
> u16 num_net_msix;
> u16 num_others_msix;
> @@ -257,12 +298,58 @@ struct nbl_chan_param_get_eth_id {
> u8 logic_eth_id;
> };
>
> +struct nbl_board_port_info {
> + u8 eth_num;
> + u8 eth_speed;
> + u8 p4_version;
> + u8 rsv[5];
> +};
> +
> +struct nbl_chan_send_info {
> + void *arg;
> + size_t arg_len;
> + void *resp;
> + size_t resp_len;
> + u16 dstid;
> + u16 msg_type;
> + u16 ack;
> + u16 ack_len;
> +};
> +
> +struct nbl_chan_ack_info {
> + void *data;
> + int err;
> + u32 data_len;
> + u16 dstid;
> + u16 msg_type;
> + u16 msgid;
> +};
> +
> enum nbl_channel_type {
> NBL_CHAN_TYPE_MAILBOX,
> NBL_CHAN_TYPE_MAX
> };
>
> struct nbl_channel_ops {
> + int (*send_msg)(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_send_info *chan_send);
> + int (*send_ack)(struct nbl_channel_mgt *chan_mgt,
> + struct nbl_chan_ack_info *chan_ack);
> + int (*register_msg)(struct nbl_channel_mgt *chan_mgt, u16 msg_type,
> + nbl_chan_resp func, void *callback_priv);
> + int (*cfg_chan_qinfo_map_table)(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type);
> + bool (*check_queue_exist)(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type);
> + int (*setup_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
> + int (*teardown_queue)(struct nbl_channel_mgt *chan_mgt, u8 chan_type);
> + void (*clean_queue_subtask)(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type);
> + void (*register_chan_task)(struct nbl_channel_mgt *chan_mgt,
> + u8 chan_type, struct work_struct *task);
> + void (*set_queue_state)(struct nbl_channel_mgt *chan_mgt,
> + enum nbl_chan_state state, u8 chan_type,
> + u8 set);
> };
>
> struct nbl_channel_ops_tbl {
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> index cc73d6a91f3c..5c532247c852 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> @@ -11,6 +11,8 @@
> #include <linux/device.h>
> #include "nbl_include.h"
>
> +struct nbl_hash_tbl_mgt;
> +
> struct nbl_common_info {
> struct pci_dev *pdev;
> struct device *dev;
> @@ -32,4 +34,32 @@ struct nbl_common_info {
> u8 has_net;
> };
>
> +struct nbl_hash_tbl_key {
> + struct device *dev;
> + u16 key_size;
> + u16 data_size; /* no include key or node member */
> + u16 bucket_size;
> + u16 resv;
> +};
> +
> +struct nbl_hash_tbl_del_key {
> + void *action_priv;
> + void (*action_func)(void *priv, void *key, void *data);
> +};
> +
> +void nbl_common_queue_work(struct work_struct *task);
> +
> +void nbl_common_destroy_wq(void);
> +int nbl_common_create_wq(void);
> +u32 nbl_common_pf_id_subtraction_mgtpf_id(struct nbl_common_info *common,
> + u32 pf_id);
> +
> +struct nbl_hash_tbl_mgt *
> +nbl_common_init_hash_table(struct nbl_hash_tbl_key *key);
> +void nbl_common_remove_hash_table(struct nbl_hash_tbl_mgt *tbl_mgt,
> + struct nbl_hash_tbl_del_key *key);
> +int nbl_common_alloc_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key,
> + void *data, void **out_data);
> +void *nbl_common_get_hash_node(struct nbl_hash_tbl_mgt *tbl_mgt, void *key);
> +
> #endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> index 168504b30973..f97f7a810ad0 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> @@ -6,9 +6,37 @@
> #ifndef _NBL_DEF_HW_H_
> #define _NBL_DEF_HW_H_
>
> +#include <linux/types.h>
> +
> struct nbl_hw_mgt;
> struct nbl_adapter;
> struct nbl_hw_ops {
> + void (*configure_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool valid, dma_addr_t dma_addr, u8 bus,
> + u8 devid, u8 function);
> + void (*configure_msix_info)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool valid, u16 interrupt_id, u8 bus,
> + u8 devid, u8 function,
> + bool net_msix_mask_en);
> + void (*update_mailbox_queue_tail_ptr)(struct nbl_hw_mgt *hw_mgt,
> + u16 tail_ptr, u8 txrx);
> + void (*config_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt,
> + dma_addr_t dma_addr, int size_bwid);
> + void (*config_mailbox_txq)(struct nbl_hw_mgt *hw_mgt,
> + dma_addr_t dma_addr, int size_bwid);
> + void (*stop_mailbox_rxq)(struct nbl_hw_mgt *hw_mgt);
> + void (*stop_mailbox_txq)(struct nbl_hw_mgt *hw_mgt);
> + u32 (*get_host_pf_mask)(struct nbl_hw_mgt *hw_mgt);
> + u32 (*get_real_bus)(struct nbl_hw_mgt *hw_mgt);
> +
> + void (*cfg_mailbox_qinfo)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + u16 bus, u16 devid, u16 function);
> + void (*enable_mailbox_irq)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool enable_msix, u16 global_vec_id);
> + u32 (*get_fw_eth_num)(struct nbl_hw_mgt *hw_mgt);
> + u32 (*get_fw_eth_map)(struct nbl_hw_mgt *hw_mgt);
> + void (*get_board_info)(struct nbl_hw_mgt *hw_mgt,
> + struct nbl_board_port_info *board);
> };
>
> struct nbl_hw_ops_tbl {
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> index 50f30f756bf3..a01c32f57d84 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> @@ -10,6 +10,12 @@
>
> /* ------ Basic definitions ------- */
> #define NBL_DRIVER_NAME "nbl"
> +#define NBL_MAX_PF 8
> +#define NBL_NEXT_ID(id, max) \
> + ({ \
> + typeof(id) _id = (id); \
> + ((_id) == (max) ? 0 : (_id) + 1); \
> + })
>
> enum nbl_product_type {
> NBL_LEONIS_TYPE,
^ permalink raw reply
* Re: [PATCH v10 net-next 06/11] net/nebula-matrix: add common resource implementation
From: Mohsin Bashir @ 2026-04-02 9:05 UTC (permalink / raw)
To: illusion.wang, dimon.zhao, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, linux-doc, lorenzo, pabeni, horms,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, jani.nikula, open list
In-Reply-To: <20260401022318.28550-7-illusion.wang@nebula-matrix.com>
On 3/31/26 7:23 PM, illusion.wang wrote:
> The Resource layer processes the entries/data of various modules within
> the processing chip to accomplish specific entry management operations,
> this describes the module business capabilities of the chip and the data
> it manages.
> The resource layer comprises the following sub-modules: common,
> interrupt, and vsi(txrx,queue not contained this time)
>
> This patch provides the common part, including the conversion
> relationships among vsi_id, func_id, eth_id, and pf_id. These
> relationships may be utilized in the upper layer or the resource layer.
>
> Signed-off-by: illusion.wang <illusion.wang@nebula-matrix.com>
> ---
> .../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
> .../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 55 ++++++
> .../nbl_hw_leonis/nbl_resource_leonis.c | 157 ++++++++++++++++++
> .../nebula-matrix/nbl/nbl_hw/nbl_resource.c | 110 ++++++++++++
> .../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 52 ++++++
> .../nbl/nbl_include/nbl_def_common.h | 15 ++
> .../nbl/nbl_include/nbl_def_resource.h | 16 ++
> .../nbl/nbl_include/nbl_include.h | 8 +
> 8 files changed, 414 insertions(+)
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> index c9bc060732e7..b03c20f9988e 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> @@ -8,6 +8,7 @@ nbl-objs += nbl_common/nbl_common.o \
> nbl_hw/nbl_hw_leonis/nbl_hw_leonis.o \
> nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
> nbl_hw/nbl_hw_leonis/nbl_hw_leonis_regs.o \
> + nbl_hw/nbl_resource.o \
> nbl_core/nbl_dispatch.o \
> nbl_core/nbl_dev.o \
> nbl_main.o
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index 83a4dc584f48..4ef0d5989a76 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -9,6 +9,18 @@
> #include <linux/spinlock.h>
> #include "nbl_hw_leonis.h"
>
> +static void nbl_hw_read_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
> + u32 *data, u32 len)
> +{
> + u32 i;
> +
> + if (len % 4)
> + return;
> +
> + for (i = 0; i < len / 4; i++)
> + data[i] = nbl_mbx_rd32(hw_mgt, reg + i * sizeof(u32));
> +}
> +
> static void nbl_hw_write_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
> const u32 *data, u32 len)
> {
> @@ -130,6 +142,14 @@ static u32 nbl_hw_get_host_pf_mask(struct nbl_hw_mgt *hw_mgt)
> return data;
> }
>
> +static u32 nbl_hw_get_real_bus(struct nbl_hw_mgt *hw_mgt)
> +{
> + u32 data;
> +
> + data = nbl_hw_rd32(hw_mgt, NBL_PCIE_HOST_TL_CFG_BUSDEV);
> + return data >> 5;
> +}
> +
> static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> u16 bus, u16 devid, u16 function)
> {
> @@ -144,6 +164,36 @@ static void nbl_hw_cfg_mailbox_qinfo(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> (u32 *)&mb_qinfo_map, sizeof(mb_qinfo_map));
> }
>
> +static void nbl_hw_get_board_info(struct nbl_hw_mgt *hw_mgt,
> + struct nbl_board_port_info *board_info)
> +{
> + union nbl_fw_board_cfg_dw3 dw3 = { .info = { 0 } };
> +
> + nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW3_OFFSET, (u32 *)&dw3,
> + sizeof(dw3));
> + board_info->eth_num = dw3.info.port_num;
> + board_info->eth_speed = dw3.info.port_speed;
> + board_info->p4_version = dw3.info.p4_version;
> +}
> +
> +static u32 nbl_hw_get_fw_eth_num(struct nbl_hw_mgt *hw_mgt)
> +{
> + union nbl_fw_board_cfg_dw3 dw3 = { .info = { 0 } };
> +
> + nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW3_OFFSET, (u32 *)&dw3,
> + sizeof(dw3));
> + return dw3.info.port_num;
> +}
> +
> +static u32 nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt)
> +{
> + union nbl_fw_board_cfg_dw6 dw6 = { .info = { 0 } };
> +
> + nbl_hw_read_mbx_regs(hw_mgt, NBL_FW_BOARD_DW6_OFFSET, (u32 *)&dw6,
> + sizeof(dw6));
> + return dw6.info.eth_bitmap;
> +}
> +
> static struct nbl_hw_ops hw_ops = {
> .update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
> .config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
> @@ -151,8 +201,13 @@ static struct nbl_hw_ops hw_ops = {
> .stop_mailbox_rxq = nbl_hw_stop_mailbox_rxq,
> .stop_mailbox_txq = nbl_hw_stop_mailbox_txq,
> .get_host_pf_mask = nbl_hw_get_host_pf_mask,
> + .get_real_bus = nbl_hw_get_real_bus,
> +
> .cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
>
> + .get_fw_eth_num = nbl_hw_get_fw_eth_num,
> + .get_fw_eth_map = nbl_hw_get_fw_eth_map,
> + .get_board_info = nbl_hw_get_board_info,
> };
>
> /* Structure starts here, adding an op should not modify anything below */
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> index 355f7ea66b88..00569aa3459a 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> @@ -4,9 +4,12 @@
> */
> #include <linux/device.h>
> #include <linux/pci.h>
> +#include <linux/bits.h>
> #include "nbl_resource_leonis.h"
>
> static struct nbl_resource_ops res_ops = {
> + .get_vsi_id = nbl_res_func_id_to_vsi_id,
> + .get_eth_id = nbl_res_get_eth_id,
> };
>
> static struct nbl_resource_mgt *
> @@ -45,8 +48,162 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
> return res_ops_tbl;
> }
>
> +static int nbl_res_ctrl_dev_setup_eth_info(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + struct device *dev = res_mgt->common->dev;
> + struct nbl_eth_info *eth_info;
> + u32 eth_bitmap, eth_id;
> + u32 eth_num = 0;
> + int i;
> +
> + eth_info = devm_kzalloc(dev, sizeof(*eth_info), GFP_KERNEL);
> + if (!eth_info)
> + return -ENOMEM;
> +
> + res_mgt->resource_info->eth_info = eth_info;
> +
> + eth_info->eth_num =
> + (u8)hw_ops->get_fw_eth_num(res_mgt->hw_ops_tbl->priv);
> + eth_bitmap = hw_ops->get_fw_eth_map(res_mgt->hw_ops_tbl->priv);
> + /* for 2 eth port board, the eth_id is 0, 2 */
> + for (i = 0; i < NBL_MAX_ETHERNET; i++) {
> + if ((1 << i) & eth_bitmap) {
> + set_bit(i, eth_info->eth_bitmap);
> + eth_info->eth_id[eth_num] = i;
> + eth_info->logic_eth_id[i] = eth_num;
> + eth_num++;
> + }
> + }
> +
> + for (i = 0; i < res_mgt->resource_info->max_pf; i++) {
> + /* if pf_id < eth_num, the pf relate corresponding eth_id*/
nit: /* if pf_id < eth_num, the pf relate corresponding eth_id */
> + if (i < eth_num) {
> + eth_id = eth_info->eth_id[i];
> + eth_info->pf_bitmap[eth_id] |= BIT(i);
> + }
> + /* if pf_id >= eth_num, the pf relate eth 0*/
> + else
> + eth_info->pf_bitmap[0] |= BIT(i);
> + }
> +
> + return 0;
> +}
> +
> +static int nbl_res_ctrl_dev_sriov_info_init(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
> + struct nbl_common_info *common = res_mgt->common;
> + struct nbl_sriov_info *sriov_info;
> + struct device *dev = common->dev;
> + u16 func_id, function;
> +
> + sriov_info = devm_kcalloc(dev, res_mgt->resource_info->max_pf,
> + sizeof(*sriov_info), GFP_KERNEL);
> + if (!sriov_info)
> + return -ENOMEM;
> +
> + res_mgt->resource_info->sriov_info = sriov_info;
> +
> + for (func_id = 0; func_id < res_mgt->resource_info->max_pf; func_id++) {
> + sriov_info = res_mgt->resource_info->sriov_info + func_id;
> + function = common->function + func_id;
> +
> + common->hw_bus = (u8)hw_ops->get_real_bus(p);
Why do we have to write to common->hw_bus in a loop? Can we do it once
outside the loop?
> + sriov_info->bdf = PCI_DEVID(common->hw_bus,
> + PCI_DEVFN(common->devid, function));
> + }
nit: space here
> + return 0;
> +}
> +
> +static int nbl_res_ctrl_dev_vsi_info_init(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
> + struct nbl_common_info *common = res_mgt->common;
> + struct device *dev = common->dev;
> + struct nbl_vsi_info *vsi_info;
> + int i;
> +
> + vsi_info = devm_kcalloc(dev, res_mgt->resource_info->max_pf,
> + sizeof(*vsi_info), GFP_KERNEL);
> + if (!vsi_info)
> + return -ENOMEM;
> +
> + res_mgt->resource_info->vsi_info = vsi_info;
> + /*
> + * case 1 two port(2pf)
> + * pf0,pf1(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,512
> +
> + * case 2 four port(4pf)
> + * pf0,pf1,pf2,pf3(NBL_VSI_SERV_PF_DATA_TYPE) vsi is 0,256,512,768
> +
> + */
> +
> + vsi_info->num = eth_info->eth_num;
> + for (i = 0; i < vsi_info->num; i++) {
> + vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].base_id =
> + i * NBL_VSI_ID_GAP(vsi_info->num);
> + vsi_info->serv_info[i][NBL_VSI_SERV_PF_DATA_TYPE].num = 1;
> + }
nit: space here
> + return 0;
> +}
> +
> +static int nbl_res_init_pf_num(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + u32 pf_num = 0;
> + u32 pf_mask;
> + int i;
> +
> + pf_mask = hw_ops->get_host_pf_mask(res_mgt->hw_ops_tbl->priv);
> + for (i = 0; i < NBL_MAX_PF; i++) {
> + if (!(pf_mask & (1 << i)))
> + pf_num++;
> + else
> + break;
> + }
> +
> + res_mgt->resource_info->max_pf = pf_num;
> +
> + if (!pf_num)
> + return -1;
Can we have an appropriate error code here instead of raw number? How
about -EINVAL?
> +
> + return 0;
> +}
> +
> +static void nbl_res_init_board_info(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> +
> + hw_ops->get_board_info(res_mgt->hw_ops_tbl->priv,
> + &res_mgt->resource_info->board_info);
> +}
> +
> static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
> {
> + struct nbl_common_info *common = res_mgt->common;
> + int ret = 0;
> +
> + if (common->has_ctrl) {
> + nbl_res_init_board_info(res_mgt);
> +
> + ret = nbl_res_init_pf_num(res_mgt);
> + if (ret)
> + return ret;
> +
> + ret = nbl_res_ctrl_dev_sriov_info_init(res_mgt);
> + if (ret)
> + return ret;
> +
> + ret = nbl_res_ctrl_dev_setup_eth_info(res_mgt);
> + if (ret)
> + return ret;
> +
> + ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
> + if (ret)
> + return ret;
> + }
> return 0;
> }
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
> new file mode 100644
> index 000000000000..2afec802b0e3
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.c
> @@ -0,0 +1,110 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#include <linux/pci.h>
> +#include "nbl_resource.h"
> +
> +u16 nbl_res_pfid_to_vsi_id(struct nbl_resource_mgt *res_mgt, int pfid, u16 type)
> +{
> + struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
> + enum nbl_vsi_serv_type dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
> + struct nbl_common_info *common = res_mgt->common;
> + u16 vsi_id = U16_MAX;
> + u32 diff;
> +
> + diff = nbl_common_pf_id_subtraction_mgtpf_id(common, pfid);
> + if (diff == U32_MAX)
> + return vsi_id;
> +
> + if (diff < vsi_info->num) {
> + nbl_res_pf_dev_vsi_type_to_hw_vsi_type(type, &dst_type);
> + vsi_id = vsi_info->serv_info[diff][dst_type].base_id;
> + }
> + if (vsi_id == U16_MAX)
> + pr_err("convert pfid %d to vsi_id(%d) failed!\n", pfid, type);
> + return vsi_id;
> +}
> +
> +u16 nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
> + u16 type)
> +{
> + int pfid = func_id;
> +
> + return nbl_res_pfid_to_vsi_id(res_mgt, pfid, type);
> +}
> +
> +int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id)
> +{
> + struct nbl_vsi_info *vsi_info = res_mgt->resource_info->vsi_info;
> + struct nbl_common_info *common = res_mgt->common;
> + bool vsi_find = false;
> + u32 pf_id = U32_MAX;
> + int i, j;
> +
> + for (i = 0; i < vsi_info->num; i++) {
> + for (j = 0; j < NBL_VSI_SERV_MAX_TYPE; j++)
> + if (vsi_id >= vsi_info->serv_info[i][j].base_id &&
> + (vsi_id < vsi_info->serv_info[i][j].base_id +
> + vsi_info->serv_info[i][j].num)) {
> + vsi_find = true;
> + break;
> + }
> +
> + if (vsi_find)
> + break;
> + }
> +
> + if (vsi_find) {
> + if (j == NBL_VSI_SERV_PF_DATA_TYPE)
> + pf_id = i + common->mgt_pf;
> + }
> +
> + return pf_id;
So we have a possible case of returning U32_MAX when the return type is int.
> +}
> +
> +int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
> + u8 *bus, u8 *dev, u8 *function)
> +{
> + struct nbl_common_info *common = res_mgt->common;
> + struct nbl_sriov_info *sriov_info;
> + int pfid = func_id;
> + u8 pf_bus, devfn;
> + u32 diff;
> +
> + diff = nbl_common_pf_id_subtraction_mgtpf_id(common, pfid);
> + if (diff == U32_MAX)
> + return U32_MAX;
> + sriov_info = res_mgt->resource_info->sriov_info + diff;
> + pf_bus = PCI_BUS_NUM(sriov_info->bdf);
> + devfn = sriov_info->bdf & 0xff;
> + *bus = pf_bus;
> + *dev = PCI_SLOT(devfn);
> + *function = PCI_FUNC(devfn);
nit: space here
> + return 0;
> +}
> +
> +void nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id,
> + u8 *eth_mode, u8 *eth_id, u8 *logic_eth_id)
> +{
> + struct nbl_eth_info *eth_info = res_mgt->resource_info->eth_info;
> + u16 pf_id = nbl_res_vsi_id_to_pf_id(res_mgt, vsi_id);
> +
> + *eth_mode = eth_info->eth_num;
> + if (pf_id < eth_info->eth_num) {
> + *eth_id = eth_info->eth_id[pf_id];
> + *logic_eth_id = pf_id;
> + } else {
> + /* if pf_id >= eth_num, use eth_id 0 */
Perhaps we can elaborate this a little more
> + *eth_id = eth_info->eth_id[0];
> + *logic_eth_id = 0;
> + }
> +}
> +
> +void nbl_res_pf_dev_vsi_type_to_hw_vsi_type(u16 src_type,
> + enum nbl_vsi_serv_type *dst_type)
> +{
> + if (src_type == NBL_VSI_DATA)
> + *dst_type = NBL_VSI_SERV_PF_DATA_TYPE;
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> index e08b6237da32..51b5b958cde8 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> @@ -16,7 +16,48 @@
> #include "../nbl_include/nbl_def_common.h"
> #include "../nbl_core.h"
>
> +struct nbl_resource_mgt;
> +
> +/* --------- INFO ---------- */
> +struct nbl_sriov_info {
> + unsigned int bdf;
> +};
> +
> +struct nbl_eth_info {
> + DECLARE_BITMAP(eth_bitmap, NBL_MAX_ETHERNET);
> + u8 pf_bitmap[NBL_MAX_ETHERNET];
> + u8 eth_num;
> + u8 resv[3];
> + u8 eth_id[NBL_MAX_PF];
> + u8 logic_eth_id[NBL_MAX_PF];
> +};
> +
> +enum nbl_vsi_serv_type {
> + NBL_VSI_SERV_PF_DATA_TYPE,
> + NBL_VSI_SERV_MAX_TYPE,
> +};
> +
> +struct nbl_vsi_serv_info {
> + u16 base_id;
> + u16 num;
> +};
> +
> +struct nbl_vsi_info {
> + u16 num;
> + struct nbl_vsi_serv_info serv_info[NBL_MAX_ETHERNET]
> + [NBL_VSI_SERV_MAX_TYPE];
> +};
> +
> struct nbl_resource_info {
> + /* ctrl-dev owned pfs */
> + DECLARE_BITMAP(func_bitmap, NBL_MAX_FUNC);
> + struct nbl_sriov_info *sriov_info;
> + struct nbl_eth_info *eth_info;
> + struct nbl_vsi_info *vsi_info;
> + u32 base_qid;
> + u32 max_vf_num;
> + u8 max_pf;
> + struct nbl_board_port_info board_info;
> };
>
> struct nbl_resource_mgt {
> @@ -27,4 +68,15 @@ struct nbl_resource_mgt {
> struct nbl_interrupt_mgt *intr_mgt;
> };
>
> +int nbl_res_vsi_id_to_pf_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id);
> +u16 nbl_res_pfid_to_vsi_id(struct nbl_resource_mgt *res_mgt, int pfid,
> + u16 type);
> +u16 nbl_res_func_id_to_vsi_id(struct nbl_resource_mgt *res_mgt, u16 func_id,
> + u16 type);
> +int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
> + u8 *bus, u8 *dev, u8 *function);
> +void nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id,
> + u8 *eth_mode, u8 *eth_id, u8 *logic_eth_id);
> +void nbl_res_pf_dev_vsi_type_to_hw_vsi_type(u16 src_type,
> + enum nbl_vsi_serv_type *dst_type);
> #endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> index 5c532247c852..04ffc1918a46 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_common.h
> @@ -12,6 +12,21 @@
> #include "nbl_include.h"
>
> struct nbl_hash_tbl_mgt;
> +#define NBL_TWO_ETHERNET_PORT 2
> +#define NBL_FOUR_ETHERNET_PORT 4
> +#define NBL_DEFAULT_VSI_ID_GAP 1024
> +#define NBL_TWO_ETHERNET_VSI_ID_GAP 512
> +#define NBL_FOUR_ETHERNET_VSI_ID_GAP 256
> +
> +#define NBL_VSI_ID_GAP(m) \
> + ({ \
> + typeof(m) _m = (m); \
> + _m == NBL_FOUR_ETHERNET_PORT ? \
> + NBL_FOUR_ETHERNET_VSI_ID_GAP : \
> + (_m == NBL_TWO_ETHERNET_PORT ? \
> + NBL_TWO_ETHERNET_VSI_ID_GAP : \
> + NBL_DEFAULT_VSI_ID_GAP); \
> + })
>
> struct nbl_common_info {
> struct pci_dev *pdev;
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
> index 16add7e25eae..249b8f633cee 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_resource.h
> @@ -6,9 +6,25 @@
> #ifndef _NBL_DEF_RESOURCE_H_
> #define _NBL_DEF_RESOURCE_H_
>
> +#include <linux/types.h>
> +
> struct nbl_resource_mgt;
> struct nbl_adapter;
> +
> struct nbl_resource_ops {
> + int (*init_chip_module)(struct nbl_resource_mgt *res_mgt);
> + void (*deinit_chip_module)(struct nbl_resource_mgt *res_mgt);
> +
> + int (*configure_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id,
> + u16 num_net_msix, u16 num_others_msix,
> + bool net_msix_mask_en);
> + int (*destroy_msix_map)(struct nbl_resource_mgt *res_mgt, u16 func_id);
> + int (*enable_mailbox_irq)(struct nbl_resource_mgt *res_mgt, u16 func_id,
> + u16 vector_id, bool enable_msix);
> + u16 (*get_vsi_id)(struct nbl_resource_mgt *res_mgt, u16 func_id,
> + u16 type);
> + void (*get_eth_id)(struct nbl_resource_mgt *res_mgt, u16 vsi_id,
> + u8 *eth_mode, u8 *eth_id, u8 *logic_eth_id);
> };
>
> struct nbl_resource_ops_tbl {
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> index a01c32f57d84..6a0bf5e8ca32 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> @@ -17,11 +17,19 @@
> ((_id) == (max) ? 0 : (_id) + 1); \
> })
>
> +#define NBL_MAX_FUNC 520
> +#define NBL_MAX_ETHERNET 4
> +
> enum nbl_product_type {
> NBL_LEONIS_TYPE,
> NBL_PRODUCT_MAX,
> };
>
> +enum {
> + NBL_VSI_DATA = 0,
> + NBL_VSI_MAX,
> +};
> +
> struct nbl_func_caps {
> u32 has_ctrl:1;
> u32 has_net:1;
^ permalink raw reply
* Re: [PATCH v10 net-next 07/11] net/nebula-matrix: add intr resource implementation
From: Mohsin Bashir @ 2026-04-02 9:05 UTC (permalink / raw)
To: illusion.wang, dimon.zhao, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, linux-doc, lorenzo, pabeni, horms,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, jani.nikula, open list
In-Reply-To: <20260401022318.28550-8-illusion.wang@nebula-matrix.com>
On 3/31/26 7:23 PM, illusion.wang wrote:
> MSI-X Interrupt Configuration:
> Dynamically allocate and manage MSI-X interrupt vectors, including
> network interrupts and other types of interrupts.
> Interrupt Mapping Table Management: Maintain the MSI-X mapping table
> (msix_map_table) to establish interrupt associations between hardware
> and software.
> Interrupt Enabling/Disabling:
> Support enabling or disabling specific interrupts through hardware
> operations.
> Interrupt Information Query: Provide interfaces to obtain the
> hardware register addresses and data of interrupts.
>
> Signed-off-by: illusion.wang <illusion.wang@nebula-matrix.com>
> ---
> .../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
> .../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 71 +++++
> .../nbl_hw_leonis/nbl_resource_leonis.c | 13 +
> .../nebula-matrix/nbl/nbl_hw/nbl_interrupt.c | 243 ++++++++++++++++++
> .../nebula-matrix/nbl/nbl_hw/nbl_interrupt.h | 12 +
> .../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 33 +++
> .../nbl/nbl_include/nbl_include.h | 2 +
> 7 files changed, 375 insertions(+)
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> index b03c20f9988e..a56e722a5ac7 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> @@ -9,6 +9,7 @@ nbl-objs += nbl_common/nbl_common.o \
> nbl_hw/nbl_hw_leonis/nbl_resource_leonis.o \
> nbl_hw/nbl_hw_leonis/nbl_hw_leonis_regs.o \
> nbl_hw/nbl_resource.o \
> + nbl_hw/nbl_interrupt.o \
> nbl_core/nbl_dispatch.o \
> nbl_core/nbl_dev.o \
> nbl_main.o
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index 4ef0d5989a76..aa5e91c2b278 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -63,6 +63,73 @@ static void nbl_hw_wr_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, const u32 *data,
> spin_unlock(&hw_mgt->reg_lock);
> }
>
> +static void nbl_hw_enable_mailbox_irq(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool enable_msix, u16 global_vec_id)
> +{
> + struct nbl_mailbox_qinfo_map_table mb_qinfo_map = { 0 };
> +
> + nbl_hw_rd_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
> + (u32 *)&mb_qinfo_map, sizeof(mb_qinfo_map));
> +
> + if (enable_msix) {
> + mb_qinfo_map.msix_idx = global_vec_id;
> + mb_qinfo_map.msix_idx_valid = 1;
> + } else {
> + mb_qinfo_map.msix_idx = 0;
> + mb_qinfo_map.msix_idx_valid = 0;
> + }
> +
> + nbl_hw_wr_regs(hw_mgt, NBL_MAILBOX_QINFO_MAP_REG_ARR(func_id),
> + (u32 *)&mb_qinfo_map, sizeof(mb_qinfo_map));
> +}
> +
> +static void nbl_hw_configure_msix_map(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool valid, dma_addr_t dma_addr, u8 bus,
> + u8 devid, u8 function)
> +{
> + struct nbl_function_msix_map function_msix_map = { 0 };
> +
> + if (valid) {
> + function_msix_map.msix_map_base_addr = dma_addr;
> + /* use af's bdf, because dma memmory is alloc by af */
> + function_msix_map.function = function;
> + function_msix_map.devid = devid;
> + function_msix_map.bus = bus;
> + function_msix_map.valid = 1;
> + }
> +
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_PCOMPLETER_FUNCTION_MSIX_MAP_REG_ARR(func_id),
> + (u32 *)&function_msix_map, sizeof(function_msix_map));
> +}
> +
> +static void nbl_hw_configure_msix_info(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> + bool valid, u16 interrupt_id, u8 bus,
> + u8 devid, u8 function, bool msix_mask_en)
> +{
> + struct nbl_pcompleter_host_msix_fid_table host_msix_fid_tbl = { 0 };
> + struct nbl_host_msix_info msix_info = { 0 };
> +
> + if (valid) {
> + host_msix_fid_tbl.vld = 1;
> + host_msix_fid_tbl.fid = func_id;
> +
> + msix_info.intrl_pnum = 0;
> + msix_info.intrl_rate = 0;
> + msix_info.function = function;
> + msix_info.devid = devid;
> + msix_info.bus = bus;
> + msix_info.valid = 1;
> + if (msix_mask_en)
> + msix_info.msix_mask_en = 1;
> + }
> +
> + nbl_hw_wr_regs(hw_mgt, NBL_PADPT_HOST_MSIX_INFO_REG_ARR(interrupt_id),
> + (u32 *)&msix_info, sizeof(msix_info));
> + nbl_hw_wr_regs(hw_mgt, NBL_PCOMPLETER_HOST_MSIX_FID_TABLE(interrupt_id),
> + (u32 *)&host_msix_fid_tbl, sizeof(host_msix_fid_tbl));
> +}
> +
> static void nbl_hw_update_mailbox_queue_tail_ptr(struct nbl_hw_mgt *hw_mgt,
> u16 tail_ptr, u8 txrx)
> {
> @@ -195,6 +262,9 @@ static u32 nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt)
> }
>
> static struct nbl_hw_ops hw_ops = {
> + .configure_msix_map = nbl_hw_configure_msix_map,
> + .configure_msix_info = nbl_hw_configure_msix_info,
> +
> .update_mailbox_queue_tail_ptr = nbl_hw_update_mailbox_queue_tail_ptr,
> .config_mailbox_rxq = nbl_hw_config_mailbox_rxq,
> .config_mailbox_txq = nbl_hw_config_mailbox_txq,
> @@ -204,6 +274,7 @@ static struct nbl_hw_ops hw_ops = {
> .get_real_bus = nbl_hw_get_real_bus,
>
> .cfg_mailbox_qinfo = nbl_hw_cfg_mailbox_qinfo,
> + .enable_mailbox_irq = nbl_hw_enable_mailbox_irq,
>
> .get_fw_eth_num = nbl_hw_get_fw_eth_num,
> .get_fw_eth_map = nbl_hw_get_fw_eth_map,
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> index 00569aa3459a..770a255ab06f 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> @@ -12,6 +12,7 @@ static struct nbl_resource_ops res_ops = {
> .get_eth_id = nbl_res_get_eth_id,
> };
>
> +static bool is_ops_inited;
> static struct nbl_resource_mgt *
> nbl_res_setup_res_mgt(struct nbl_common_info *common)
> {
> @@ -37,11 +38,19 @@ static struct nbl_resource_ops_tbl *
> nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
> {
> struct nbl_resource_ops_tbl *res_ops_tbl;
> + int ret;
>
> res_ops_tbl = devm_kzalloc(dev, sizeof(*res_ops_tbl), GFP_KERNEL);
> if (!res_ops_tbl)
> return ERR_PTR(-ENOMEM);
>
> + if (!is_ops_inited) {
> + ret = nbl_intr_setup_ops(&res_ops);
> + if (ret)
> + return ERR_PTR(-ENOMEM);
> + is_ops_inited = true;
> + }
> +
> res_ops_tbl->ops = &res_ops;
> res_ops_tbl->priv = res_mgt;
>
> @@ -203,6 +212,10 @@ static int nbl_res_start(struct nbl_resource_mgt *res_mgt)
> ret = nbl_res_ctrl_dev_vsi_info_init(res_mgt);
> if (ret)
> return ret;
> +
> + ret = nbl_intr_mgt_start(res_mgt);
> + if (ret)
> + return ret;
> }
> return 0;
> }
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
> new file mode 100644
> index 000000000000..aeea866be9c3
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.c
> @@ -0,0 +1,243 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include "nbl_interrupt.h"
> +
> +static int nbl_res_intr_destroy_msix_map(struct nbl_resource_mgt *res_mgt,
> + u16 func_id)
> +{
> + struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + struct device *dev = res_mgt->common->dev;
> + struct nbl_msix_map_table *msix_map_table;
> + u16 *interrupts;
> + u16 intr_num, i;
> +
> + /* use ctrl dev bdf */
> + hw_ops->configure_msix_map(res_mgt->hw_ops_tbl->priv, func_id, false, 0,
> + 0, 0, 0);
> +
> + intr_num = intr_mgt->func_intr_res[func_id].num_interrupts;
> + interrupts = intr_mgt->func_intr_res[func_id].interrupts;
> +
> + WARN_ON(!interrupts);
So we WARN_ON no interrupts and then proceed to iterate over
interrupts[]? This will surely crash. WARN_ON is not providing any
safety here.
> + for (i = 0; i < intr_num; i++) {
> + if (interrupts[i] >= NBL_MAX_OTHER_INTERRUPT)
> + clear_bit(interrupts[i] - NBL_MAX_OTHER_INTERRUPT,
> + intr_mgt->interrupt_net_bitmap);
> + else
> + clear_bit(interrupts[i],
> + intr_mgt->interrupt_others_bitmap);
> +
> + hw_ops->configure_msix_info(res_mgt->hw_ops_tbl->priv, func_id,
> + false, interrupts[i], 0, 0, 0,
> + false);
> + }
> +
> + kfree(interrupts);
> + intr_mgt->func_intr_res[func_id].interrupts = NULL;
> + intr_mgt->func_intr_res[func_id].num_interrupts = 0;
> +
> + msix_map_table = &intr_mgt->func_intr_res[func_id].msix_map_table;
> + dma_free_coherent(dev, msix_map_table->size, msix_map_table->base_addr,
> + msix_map_table->dma);
> + msix_map_table->size = 0;
> + msix_map_table->base_addr = NULL;
> + msix_map_table->dma = 0;
> +
> + return 0;
> +}
> +
> +static int nbl_res_intr_configure_msix_map(struct nbl_resource_mgt *res_mgt,
> + u16 func_id, u16 num_net_msix,
> + u16 num_others_msix,
> + bool net_msix_mask_en)
> +{
> + struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + struct nbl_common_info *common = res_mgt->common;
> + struct nbl_msix_map_table *msix_map_table;
> + struct nbl_msix_map *msix_map_entries;
> + struct device *dev = common->dev;
> + u16 requested, intr_index;
> + u8 bus, devid, function;
> + bool msix_mask_en;
> + u16 *interrupts;
> + int ret = 0;
> + u16 i;
> +
> + requested = num_net_msix + num_others_msix;
> + if (requested > NBL_MSIX_MAP_TABLE_MAX_ENTRIES)
> + return -EINVAL;
> + if (intr_mgt->func_intr_res[func_id].interrupts)
> + nbl_res_intr_destroy_msix_map(res_mgt, func_id);
> +
> + ret = nbl_res_func_id_to_bdf(res_mgt, func_id, &bus, &devid, &function);
> + if (ret == U32_MAX)
> + return -EINVAL;
> +
> + msix_map_table = &intr_mgt->func_intr_res[func_id].msix_map_table;
> + WARN_ON(msix_map_table->base_addr);
> + msix_map_table->size =
> + sizeof(struct nbl_msix_map) * NBL_MSIX_MAP_TABLE_MAX_ENTRIES;
> + msix_map_table->base_addr = dma_alloc_coherent(dev,
> + msix_map_table->size,
> + &msix_map_table->dma,
> + GFP_KERNEL);
> + if (!msix_map_table->base_addr) {
> + pr_err("Allocate DMA memory for function msix map table failed\n");
> + msix_map_table->size = 0;
> + return -ENOMEM;
> + }
> +
> + interrupts = kcalloc(requested, sizeof(interrupts[0]), GFP_KERNEL);
> + if (!interrupts) {
> + ret = -ENOMEM;
> + goto alloc_interrupts_err;
> + }
> +
> + intr_mgt->func_intr_res[func_id].interrupts = interrupts;
> + intr_mgt->func_intr_res[func_id].num_interrupts = requested;
> + intr_mgt->func_intr_res[func_id].num_net_interrupts = num_net_msix;
> +
> + for (i = 0; i < num_net_msix; i++) {
> + intr_index = find_first_zero_bit(intr_mgt->interrupt_net_bitmap,
> + NBL_MAX_NET_INTERRUPT);
> + if (intr_index == NBL_MAX_NET_INTERRUPT) {
> + pr_err("There is no available interrupt left\n");
Can we use dev_err instead of pr_err? Here and at other places as well?
> + ret = -EAGAIN;
> + goto get_interrupt_err;
> + }
> + interrupts[i] = intr_index + NBL_MAX_OTHER_INTERRUPT;
> + set_bit(intr_index, intr_mgt->interrupt_net_bitmap);
> + }
> +
> + for (i = num_net_msix; i < requested; i++) {
> + intr_index =
> + find_first_zero_bit(intr_mgt->interrupt_others_bitmap,
> + NBL_MAX_OTHER_INTERRUPT);
> + if (intr_index == NBL_MAX_OTHER_INTERRUPT) {
> + pr_err("There is no available interrupt left\n");
> + ret = -EAGAIN;
> + goto get_interrupt_err;
> + }
> + interrupts[i] = intr_index;
> + set_bit(intr_index, intr_mgt->interrupt_others_bitmap);
> + }
> +
> + msix_map_entries = msix_map_table->base_addr;
> + for (i = 0; i < requested; i++) {
> + msix_map_entries[i].global_msix_index = interrupts[i];
> + msix_map_entries[i].valid = 1;
> +
> + if (i < num_net_msix && net_msix_mask_en)
> + msix_mask_en = 1;
> + else
> + msix_mask_en = 0;
> + hw_ops->configure_msix_info(res_mgt->hw_ops_tbl->priv, func_id,
> + true, interrupts[i], bus, devid,
> + function, msix_mask_en);
> + }
> +
> + /* use ctrl dev bdf */
> + hw_ops->configure_msix_map(res_mgt->hw_ops_tbl->priv, func_id, true,
> + msix_map_table->dma, common->hw_bus,
> + common->devid, common->function);
> +
> + return 0;
> +
> +get_interrupt_err:
> + while (i--) {
> + intr_index = interrupts[i];
> + if (intr_index >= NBL_MAX_OTHER_INTERRUPT)
> + clear_bit(intr_index - NBL_MAX_OTHER_INTERRUPT,
> + intr_mgt->interrupt_net_bitmap);
> + else
> + clear_bit(intr_index,
> + intr_mgt->interrupt_others_bitmap);
> + }
> + kfree(interrupts);
> + intr_mgt->func_intr_res[func_id].num_interrupts = 0;
> + intr_mgt->func_intr_res[func_id].interrupts = NULL;
> +
> +alloc_interrupts_err:
> + dma_free_coherent(dev, msix_map_table->size, msix_map_table->base_addr,
> + msix_map_table->dma);
> + msix_map_table->size = 0;
> + msix_map_table->base_addr = NULL;
> + msix_map_table->dma = 0;
> +
> + return ret;
> +}
> +
> +static int nbl_res_intr_enable_mailbox_irq(struct nbl_resource_mgt *res_mgt,
> + u16 func_id, u16 vector_id,
> + bool enable_msix)
> +{
> + struct nbl_interrupt_mgt *intr_mgt = res_mgt->intr_mgt;
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + u16 global_vec_id;
> +
> + global_vec_id = intr_mgt->func_intr_res[func_id].interrupts[vector_id];
> + hw_ops->enable_mailbox_irq(res_mgt->hw_ops_tbl->priv, func_id,
> + enable_msix, global_vec_id);
> +
> + return 0;
> +}
> +
> +/* NBL_INTR_SET_OPS(ops_name, func)
> + *
> + * Use X Macros to reduce setup and remove codes.
> + */
> +#define NBL_INTR_OPS_TBL \
> +do { \
> + NBL_INTR_SET_OPS(configure_msix_map, \
> + nbl_res_intr_configure_msix_map); \
> + NBL_INTR_SET_OPS(destroy_msix_map, \
> + nbl_res_intr_destroy_msix_map); \
> + NBL_INTR_SET_OPS(enable_mailbox_irq, \
> + nbl_res_intr_enable_mailbox_irq); \
> +} while (0)
> +
> +/* Structure starts here, adding an op should not modify anything below */
> +static struct nbl_interrupt_mgt *nbl_intr_setup_mgt(struct device *dev)
> +{
> + struct nbl_interrupt_mgt *intr_mgt;
> +
> + intr_mgt = devm_kzalloc(dev, sizeof(*intr_mgt), GFP_KERNEL);
> + if (!intr_mgt)
> + return ERR_PTR(-ENOMEM);
> +
> + return intr_mgt;
> +}
> +
> +int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt)
> +{
> + struct device *dev = res_mgt->common->dev;
> + struct nbl_interrupt_mgt *intr_mgt;
> + int ret;
> +
> + intr_mgt = nbl_intr_setup_mgt(dev);
> + if (IS_ERR(intr_mgt)) {
> + ret = PTR_ERR(intr_mgt);
> + return ret;
> + }
> + res_mgt->intr_mgt = intr_mgt;
> + return 0;
> +}
> +
> +int nbl_intr_setup_ops(struct nbl_resource_ops *res_ops)
> +{
> +#define NBL_INTR_SET_OPS(name, func) \
> + do { \
> + res_ops->NBL_NAME(name) = func; \
> + ; \
> + } while (0)
> + NBL_INTR_OPS_TBL;
> +#undef NBL_INTR_SET_OPS
> +
> + return 0;
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
> new file mode 100644
> index 000000000000..b876bf30084b
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_interrupt.h
> @@ -0,0 +1,12 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_INTERRUPT_H_
> +#define _NBL_INTERRUPT_H_
> +
> +#include "nbl_resource.h"
> +
> +#define NBL_MSIX_MAP_TABLE_MAX_ENTRIES 1024
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> index 51b5b958cde8..1b80676cf19a 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> @@ -18,6 +18,37 @@
>
> struct nbl_resource_mgt;
>
> +/* --------- INTERRUPT ---------- */
> +#define NBL_MAX_OTHER_INTERRUPT 1024
> +#define NBL_MAX_NET_INTERRUPT 4096
> +
> +struct nbl_msix_map {
> + u16 valid:1;
> + u16 global_msix_index:13;
> + u16 rsv:2;
> +};
> +
> +struct nbl_msix_map_table {
> + struct nbl_msix_map *base_addr;
> + dma_addr_t dma;
> + size_t size;
> +};
> +
> +struct nbl_func_interrupt_resource_mng {
> + u16 num_interrupts;
> + u16 num_net_interrupts;
> + u16 msix_base;
> + u16 msix_max;
> + u16 *interrupts;
> + struct nbl_msix_map_table msix_map_table;
> +};
> +
> +struct nbl_interrupt_mgt {
> + DECLARE_BITMAP(interrupt_net_bitmap, NBL_MAX_NET_INTERRUPT);
> + DECLARE_BITMAP(interrupt_others_bitmap, NBL_MAX_OTHER_INTERRUPT);
> + struct nbl_func_interrupt_resource_mng func_intr_res[NBL_MAX_FUNC];
> +};
> +
> /* --------- INFO ---------- */
> struct nbl_sriov_info {
> unsigned int bdf;
> @@ -77,6 +108,8 @@ int nbl_res_func_id_to_bdf(struct nbl_resource_mgt *res_mgt, u16 func_id,
> u8 *bus, u8 *dev, u8 *function);
> void nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id,
> u8 *eth_mode, u8 *eth_id, u8 *logic_eth_id);
> +int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt);
> +int nbl_intr_setup_ops(struct nbl_resource_ops *resource_ops);
> void nbl_res_pf_dev_vsi_type_to_hw_vsi_type(u16 src_type,
> enum nbl_vsi_serv_type *dst_type);
> #endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> index 6a0bf5e8ca32..e4f11e6ded94 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> @@ -19,6 +19,8 @@
>
> #define NBL_MAX_FUNC 520
> #define NBL_MAX_ETHERNET 4
> +/* Used for macros to pass checkpatch */
> +#define NBL_NAME(x) x
>
> enum nbl_product_type {
> NBL_LEONIS_TYPE,
^ permalink raw reply
* Re: [PATCH v10 net-next 08/11] net/nebula-matrix: add vsi resource implementation
From: Mohsin Bashir @ 2026-04-02 9:06 UTC (permalink / raw)
To: illusion.wang, dimon.zhao, alvin.wang, sam.chen, netdev
Cc: andrew+netdev, corbet, kuba, linux-doc, lorenzo, pabeni, horms,
vadim.fedorenko, lukas.bulwahn, edumazet, enelsonmoore, skhan,
hkallweit1, jani.nikula, open list
In-Reply-To: <20260401022318.28550-9-illusion.wang@nebula-matrix.com>
On 3/31/26 7:23 PM, illusion.wang wrote:
> The HW (Hardware) layer code can have a quick review since it is
> highly chip-specific.
> Chip initialization includes the initialization of the DP module, the
> intf module, and the P4 registers.
> The initialization of the DP module encompasses the initialization of
> the dped(downstream pkt edit), uped(upstream pkt edit), dsch(downstream
> schedule), ustore, dstore, dvn, uvn, and uqm modules.
>
> Signed-off-by: illusion.wang <illusion.wang@nebula-matrix.com>
> ---
> .../net/ethernet/nebula-matrix/nbl/Makefile | 1 +
> .../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c | 406 ++++++++++++++++++
> .../nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h | 12 +-
> .../nbl_hw_leonis/nbl_resource_leonis.c | 4 +
> .../nebula-matrix/nbl/nbl_hw/nbl_resource.h | 1 +
> .../nebula-matrix/nbl/nbl_hw/nbl_vsi.c | 51 +++
> .../nebula-matrix/nbl/nbl_hw/nbl_vsi.h | 11 +
> .../nbl/nbl_include/nbl_def_hw.h | 4 +
> .../nbl/nbl_include/nbl_include.h | 31 ++
> 9 files changed, 520 insertions(+), 1 deletion(-)
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.c
> create mode 100644 drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.h
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/Makefile b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> index a56e722a5ac7..241bbb572b5e 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/Makefile
> @@ -10,6 +10,7 @@ nbl-objs += nbl_common/nbl_common.o \
> nbl_hw/nbl_hw_leonis/nbl_hw_leonis_regs.o \
> nbl_hw/nbl_resource.o \
> nbl_hw/nbl_interrupt.o \
> + nbl_hw/nbl_vsi.o \
> nbl_core/nbl_dispatch.o \
> nbl_core/nbl_dev.o \
> nbl_main.o
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> index aa5e91c2b278..37cfd322d422 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.c
> @@ -8,6 +8,7 @@
> #include <linux/io.h>
> #include <linux/spinlock.h>
> #include "nbl_hw_leonis.h"
> +#include "nbl_hw_leonis_regs.h"
>
> static void nbl_hw_read_mbx_regs(struct nbl_hw_mgt *hw_mgt, u64 reg,
> u32 *data, u32 len)
> @@ -63,6 +64,408 @@ static void nbl_hw_wr_regs(struct nbl_hw_mgt *hw_mgt, u64 reg, const u32 *data,
> spin_unlock(&hw_mgt->reg_lock);
> }
>
> +static u32 nbl_hw_get_quirks(struct nbl_hw_mgt *hw_mgt)
> +{
> + u32 quirks;
> +
> + nbl_hw_read_mbx_regs(hw_mgt, NBL_LEONIS_QUIRKS_OFFSET, &quirks,
> + sizeof(u32));
> +
> + if (quirks == NBL_LEONIS_ILLEGAL_REG_VALUE)
> + return 0;
> +
> + return quirks;
> +}
> +
> +static void nbl_configure_dped_checksum(struct nbl_hw_mgt *hw_mgt)
> +{
> + union dped_l4_ck_cmd_40_u l4_ck_cmd_40;
> +
> + /* DPED dped_l4_ck_cmd_40 for sctp */
> + nbl_hw_rd_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, (u32 *)&l4_ck_cmd_40,
> + sizeof(l4_ck_cmd_40));
> + l4_ck_cmd_40.info.en = 1;
> + nbl_hw_wr_regs(hw_mgt, NBL_DPED_L4_CK_CMD_40_ADDR, (u32 *)&l4_ck_cmd_40,
> + sizeof(l4_ck_cmd_40));
> +}
> +
> +static int nbl_dped_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + nbl_hw_wr32(hw_mgt, NBL_DPED_VLAN_OFFSET, 0xC);
> + nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_0, 0x8);
> + nbl_hw_wr32(hw_mgt, NBL_DPED_DSCP_OFFSET_1, 0x4);
> +
> + /* dped checksum offload */
> + nbl_configure_dped_checksum(hw_mgt);
> +
> + return 0;
> +}
> +
> +static int nbl_uped_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + struct ped_hw_edit_profile hw_edit;
> +
> + nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_DPED_V4_TCP_IDX),
> + (u32 *)&hw_edit, sizeof(hw_edit));
> + hw_edit.l3_len = 0;
> + nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_DPED_V4_TCP_IDX),
> + (u32 *)&hw_edit, sizeof(hw_edit));
> +
> + nbl_hw_rd_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_DPED_V6_TCP_IDX),
> + (u32 *)&hw_edit, sizeof(hw_edit));
> + hw_edit.l3_len = 1;
> + nbl_hw_wr_regs(hw_mgt, NBL_UPED_HW_EDT_PROF_TABLE(NBL_DPED_V6_TCP_IDX),
> + (u32 *)&hw_edit, sizeof(hw_edit));
> +
> + return 0;
> +}
> +
> +static void nbl_shaping_eth_init(struct nbl_hw_mgt *hw_mgt, u8 eth_id, u8 speed)
> +{
> + struct nbl_shaping_dvn_dport dvn_dport = { 0 };
> + struct nbl_shaping_dport dport = { 0 };
> + u32 rate, half_rate;
> +
> + if (speed == NBL_FW_PORT_SPEED_100G) {
> + rate = NBL_SHAPING_DPORT_100G_RATE;
> + half_rate = NBL_SHAPING_DPORT_HALF_100G_RATE;
> + } else {
> + rate = NBL_SHAPING_DPORT_25G_RATE;
> + half_rate = NBL_SHAPING_DPORT_HALF_25G_RATE;
> + }
> +
> + dport.cir = rate;
> + dport.pir = rate;
> + dport.depth = max(dport.cir * 2, NBL_LR_LEONIS_NET_BUCKET_DEPTH);
> + dport.cbs = dport.depth;
> + dport.pbs = dport.depth;
> + dport.valid = 1;
> +
> + dvn_dport.cir = half_rate;
> + dvn_dport.pir = rate;
> + dvn_dport.depth = dport.depth;
> + dvn_dport.cbs = dvn_dport.depth;
> + dvn_dport.pbs = dvn_dport.depth;
> + dvn_dport.valid = 1;
> +
> + nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DPORT_REG(eth_id), (u32 *)&dport,
> + sizeof(dport));
> + nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_DVN_DPORT_REG(eth_id),
> + (u32 *)&dvn_dport, sizeof(dvn_dport));
> +}
> +
> +static int nbl_shaping_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
> +{
> +#define NBL_SHAPING_FLUSH_INTERVAL 128
> + struct nbl_shaping_net net_shaping = { 0 };
> + struct dsch_psha_en psha_en = { 0 };
> + int i;
> +
> + for (i = 0; i < NBL_MAX_ETHERNET; i++)
> + nbl_shaping_eth_init(hw_mgt, i, speed);
> +
> + psha_en.en = 0xF;
> + nbl_hw_wr_regs(hw_mgt, NBL_DSCH_PSHA_EN_ADDR, (u32 *)&psha_en,
> + sizeof(psha_en));
> +
> + for (i = 0; i < NBL_MAX_FUNC; i++) {
> + nbl_hw_wr_regs(hw_mgt, NBL_SHAPING_NET_REG(i),
> + (u32 *)&net_shaping, sizeof(net_shaping));
> + if ((i % NBL_SHAPING_FLUSH_INTERVAL) == 0)
> + nbl_flush_writes(hw_mgt);
> + }
> + nbl_flush_writes(hw_mgt);
> + return 0;
> +}
> +
> +static int nbl_dsch_qid_max_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + struct dsch_vn_quanta quanta = { 0 };
> +
> + quanta.h_qua = NBL_HOST_QUANTA;
> + quanta.e_qua = NBL_ECPU_QUANTA;
> + nbl_hw_wr_regs(hw_mgt, NBL_DSCH_VN_QUANTA_ADDR, (u32 *)&quanta,
> + sizeof(quanta));
> + nbl_hw_wr32(hw_mgt, NBL_DSCH_HOST_QID_MAX, NBL_MAX_QUEUE_ID);
> +
> + nbl_hw_wr32(hw_mgt, NBL_DVN_ECPU_QUEUE_NUM, 0);
> + nbl_hw_wr32(hw_mgt, NBL_UVN_ECPU_QUEUE_NUM, 0);
> +
> + return 0;
> +}
> +
> +static int nbl_ustore_init(struct nbl_hw_mgt *hw_mgt, u8 eth_num)
> +{
> + struct nbl_ustore_port_drop_th drop_th = { 0 };
> + struct ustore_pkt_len pkt_len;
> + int i;
> +
> + nbl_hw_rd_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, (u32 *)&pkt_len,
> + sizeof(pkt_len));
Why are we issuing read while ignoring return values? Issuing read to
clear registers? A comment here would help.
> + /* min arp packet length 42 (14 + 28) */
> + pkt_len.min = 42;
> + nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PKT_LEN_ADDR, (u32 *)&pkt_len,
> + sizeof(pkt_len));
> +
> + drop_th.en = 1;
> + if (eth_num == 1)
> + drop_th.disc_th = NBL_USTORE_SINGLE_ETH_DROP_TH;
> + else if (eth_num == 2)
> + drop_th.disc_th = NBL_USTORE_DUAL_ETH_DROP_TH;
> + else
> + drop_th.disc_th = NBL_USTORE_QUAD_ETH_DROP_TH;
> +
> + for (i = 0; i < NBL_MAX_ETHERNET; i++)
> + nbl_hw_wr_regs(hw_mgt, NBL_USTORE_PORT_DROP_TH_REG_ARR(i),
> + (u32 *)&drop_th, sizeof(drop_th));
> +
> + for (i = 0; i < NBL_MAX_ETHERNET; i++) {
> + nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_DROP_PKT(i));
> + nbl_hw_rd32(hw_mgt, NBL_USTORE_BUF_PORT_TRUN_PKT(i));
> + }
> +
> + return 0;
> +}
> +
> +static int nbl_dstore_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
> +{
> + struct dstore_port_drop_th drop_th;
> + struct dstore_d_dport_fc_th fc_th;
> + struct dstore_disc_bp_th bp_th;
> + int i;
> +
> + for (i = 0; i < NBL_DSTORE_PORT_DROP_TH_DEPTH; i++) {
> + nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i),
> + (u32 *)&drop_th, sizeof(drop_th));
> + drop_th.en = 0;
> + nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_PORT_DROP_TH_REG(i),
> + (u32 *)&drop_th, sizeof(drop_th));
> + }
> +
> + nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, (u32 *)&bp_th,
> + sizeof(bp_th));
> + bp_th.en = 1;
> + nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_DISC_BP_TH, (u32 *)&bp_th,
> + sizeof(bp_th));
> +
> + for (i = 0; i < NBL_MAX_ETHERNET; i++) {
> + nbl_hw_rd_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i),
> + (u32 *)&fc_th, sizeof(fc_th));
> + if (speed == NBL_FW_PORT_SPEED_100G) {
> + fc_th.xoff_th = NBL_DSTORE_DROP_XOFF_TH_100G;
> + fc_th.xon_th = NBL_DSTORE_DROP_XON_TH_100G;
> + } else {
> + fc_th.xoff_th = NBL_DSTORE_DROP_XOFF_TH;
> + fc_th.xon_th = NBL_DSTORE_DROP_XON_TH;
> + }
> +
> + fc_th.fc_en = 1;
> + nbl_hw_wr_regs(hw_mgt, NBL_DSTORE_D_DPORT_FC_TH_REG(i),
> + (u32 *)&fc_th, sizeof(fc_th));
> + }
> +
> + return 0;
> +}
> +
> +static void nbl_dvn_descreq_num_cfg(struct nbl_hw_mgt *hw_mgt, u32 descreq_num)
> +{
> + u32 split_ring_num = (descreq_num >> 16) & 0xffff;
> + struct nbl_dvn_descreq_num_cfg num_cfg = { 0 };
> + u32 packet_ring_num = descreq_num & 0xffff;
> +
> + packet_ring_num =
> + clamp(packet_ring_num, PACKET_RING_MIN, PACKET_RING_MAX);
> + num_cfg.packed_l1_num =
> + (packet_ring_num - PACKET_RING_BASE) / PACKET_RING_DIV;
> +
> + split_ring_num = clamp(split_ring_num, SPLIT_RING_MIN,
> + SPLIT_RING_MAX);
> + num_cfg.avring_cfg_num = split_ring_num > SPLIT_RING_MIN ?
> + SPLIT_RING_CFG_16 :
> + SPLIT_RING_CFG_8;
> +
> + nbl_hw_wr_regs(hw_mgt, NBL_DVN_DESCREQ_NUM_CFG, (u32 *)&num_cfg,
> + sizeof(num_cfg));
> +}
> +
> +static int nbl_dvn_init(struct nbl_hw_mgt *hw_mgt, u8 speed)
> +{
> + struct nbl_dvn_desc_wr_merge_timeout timeout = { 0 };
> + struct nbl_dvn_dif_req_rd_ro_flag ro_flag = { 0 };
> +
> + timeout.cfg_cycle = DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX;
> + nbl_hw_wr_regs(hw_mgt, NBL_DVN_DESC_WR_MERGE_TIMEOUT, (u32 *)&timeout,
> + sizeof(timeout));
> +
> + ro_flag.rd_desc_ro_en = 1;
> + ro_flag.rd_data_ro_en = 1;
> + ro_flag.rd_avring_ro_en = 1;
> + nbl_hw_wr_regs(hw_mgt, NBL_DVN_DIF_REQ_RD_RO_FLAG, (u32 *)&ro_flag,
> + sizeof(ro_flag));
> +
> + if (speed == NBL_FW_PORT_SPEED_100G)
> + nbl_dvn_descreq_num_cfg(hw_mgt,
> + DEFAULT_DVN_100G_DESCREQ_NUMCFG);
> + else
> + nbl_dvn_descreq_num_cfg(hw_mgt, DEFAULT_DVN_DESCREQ_NUMCFG);
> +
> + return 0;
> +}
> +
> +static int nbl_uvn_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + struct uvn_desc_prefetch_init prefetch_init = { 0 };
> + struct uvn_desc_wr_timeout desc_wr_timeout = { 0 };
> + struct uvn_dif_req_ro_flag flag = { 0 };
> + struct uvn_queue_err_mask mask = { 0 };
> + u16 wr_timeout = 0x12c;
> + u32 timeout = 119760; /* 200us 200000/1.67 */
> + u32 quirks;
> +
> + nbl_hw_wr32(hw_mgt, NBL_UVN_DESC_RD_WAIT, timeout);
> +
> + desc_wr_timeout.num = wr_timeout;
> + nbl_hw_wr_regs(hw_mgt, NBL_UVN_DESC_WR_TIMEOUT, (u32 *)&desc_wr_timeout,
> + sizeof(desc_wr_timeout));
> +
> + flag.avail_rd = 1;
> + flag.desc_rd = 1;
> + flag.pkt_wr = 1;
> + flag.desc_wr = 0;
> + nbl_hw_wr_regs(hw_mgt, NBL_UVN_DIF_REQ_RO_FLAG, (u32 *)&flag,
> + sizeof(flag));
> +
> + nbl_hw_rd_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, (u32 *)&mask,
> + sizeof(mask));
> + mask.dif_err = 1;
> + nbl_hw_wr_regs(hw_mgt, NBL_UVN_QUEUE_ERR_MASK, (u32 *)&mask,
> + sizeof(mask));
> +
> + prefetch_init.num = NBL_UVN_DESC_PREFETCH_NUM;
> + prefetch_init.sel = 0;
> + quirks = nbl_hw_get_quirks(hw_mgt);
> + if (!(quirks & BIT(NBL_QUIRKS_UVN_PREFETCH_ALIGN)))
> + prefetch_init.sel = 1;
> + nbl_hw_wr_regs(hw_mgt, NBL_UVN_DESC_PREFETCH_INIT,
> + (u32 *)&prefetch_init, sizeof(prefetch_init));
> +
> + return 0;
> +}
> +
> +static int nbl_uqm_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + struct nbl_uqm_que_type que_type = { 0 };
> + u32 cnt = 0;
> + int i;
> +
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_FWD_DROP_CNT, &cnt, sizeof(cnt));
> +
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_CNT, &cnt, sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_SLICE_CNT, &cnt,
> + sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_PKT_LEN_ADD_CNT, &cnt,
> + sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_HEAD_PNTR_ADD_CNT, &cnt,
> + sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_DROP_WEIGHT_ADD_CNT, &cnt,
> + sizeof(cnt));
> +
> + for (i = 0; i < NBL_UQM_PORT_DROP_DEPTH; i++) {
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_UQM_PORT_DROP_PKT_CNT + (sizeof(cnt) * i),
> + &cnt, sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_UQM_PORT_DROP_PKT_SLICE_CNT +
> + (sizeof(cnt) * i),
> + &cnt, sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_UQM_PORT_DROP_PKT_LEN_ADD_CNT +
> + (sizeof(cnt) * i),
> + &cnt, sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_UQM_PORT_DROP_HEAD_PNTR_ADD_CNT +
> + (sizeof(cnt) * i),
> + &cnt, sizeof(cnt));
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_UQM_PORT_DROP_WEIGHT_ADD_CNT +
> + (sizeof(cnt) * i),
> + &cnt, sizeof(cnt));
> + }
> +
> + for (i = 0; i < NBL_UQM_DPORT_DROP_DEPTH; i++)
> + nbl_hw_wr_regs(hw_mgt,
> + NBL_UQM_DPORT_DROP_CNT + (sizeof(cnt) * i),
> + &cnt, sizeof(cnt));
> +
> + que_type.bp_drop = 0;
> + nbl_hw_wr_regs(hw_mgt, NBL_UQM_QUE_TYPE, (u32 *)&que_type,
> + sizeof(que_type));
> +
> + return 0;
> +}
> +
> +static int nbl_dp_init(struct nbl_hw_mgt *hw_mgt, u8 speed, u8 eth_num)
> +{
> + nbl_dped_init(hw_mgt);
> + nbl_uped_init(hw_mgt);
> + nbl_shaping_init(hw_mgt, speed);
> + nbl_dsch_qid_max_init(hw_mgt);
> + nbl_ustore_init(hw_mgt, eth_num);
> + nbl_dstore_init(hw_mgt, speed);
> + nbl_dvn_init(hw_mgt, speed);
> + nbl_uvn_init(hw_mgt);
> + nbl_uqm_init(hw_mgt);
> +
> + return 0;
> +}
> +
> +static int nbl_host_padpt_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + /* padpt flow control register */
> + nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_CPLH_UP, 0x10400);
> + nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PD_DN, 0x10080);
> + nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_PH_DN, 0x10010);
> + nbl_hw_wr32(hw_mgt, NBL_HOST_PADPT_HOST_CFG_FC_NPH_DN, 0x10010);
> +
> + return 0;
> +}
> +
> +static int nbl_intf_init(struct nbl_hw_mgt *hw_mgt)
> +{
> + nbl_host_padpt_init(hw_mgt);
> + return 0;
> +}
> +
> +static void nbl_hw_set_driver_status(struct nbl_hw_mgt *hw_mgt, bool active)
> +{
> + u32 status;
> +
> + status = nbl_hw_rd32(hw_mgt, NBL_DRIVER_STATUS_REG);
> +
> + status = (status & ~(1 << NBL_DRIVER_STATUS_BIT)) |
> + (active << NBL_DRIVER_STATUS_BIT);
> +
> + nbl_hw_wr32(hw_mgt, NBL_DRIVER_STATUS_REG, status);
> +}
> +
> +static void nbl_hw_deinit_chip_module(struct nbl_hw_mgt *hw_mgt)
> +{
> + nbl_hw_set_driver_status(hw_mgt, false);
> +}
> +
> +static int nbl_hw_init_chip_module(struct nbl_hw_mgt *hw_mgt, u8 eth_speed,
> + u8 eth_num)
> +{
> + nbl_dp_init(hw_mgt, eth_speed, eth_num);
> + nbl_intf_init(hw_mgt);
> +
> + nbl_write_all_regs(hw_mgt);
> + nbl_hw_set_driver_status(hw_mgt, true);
> + hw_mgt->version = nbl_hw_rd32(hw_mgt, NBL_HW_DUMMY_REG);
> +
> + return 0;
> +}
> +
> static void nbl_hw_enable_mailbox_irq(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> bool enable_msix, u16 global_vec_id)
> {
> @@ -262,6 +665,9 @@ static u32 nbl_hw_get_fw_eth_map(struct nbl_hw_mgt *hw_mgt)
> }
>
> static struct nbl_hw_ops hw_ops = {
> + .init_chip_module = nbl_hw_init_chip_module,
> + .deinit_chip_module = nbl_hw_deinit_chip_module,
> +
> .configure_msix_map = nbl_hw_configure_msix_map,
> .configure_msix_info = nbl_hw_configure_msix_info,
>
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> index 8831394ed11b..7487d0e757e3 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_hw_leonis.h
> @@ -128,7 +128,8 @@ struct nbl_function_msix_map {
> #define NBL_DPED_VLAN_OFFSET (NBL_DP_DPED_BASE + 0x000003F4)
> #define NBL_DPED_DSCP_OFFSET_0 (NBL_DP_DPED_BASE + 0x000003F8)
> #define NBL_DPED_DSCP_OFFSET_1 (NBL_DP_DPED_BASE + 0x000003FC)
> -
> +#define NBL_DPED_V4_TCP_IDX 5
> +#define NBL_DPED_V6_TCP_IDX 6
> /* DPED hw_edt_prof/ UPED hw_edt_prof */
> struct ped_hw_edit_profile {
> u32 l4_len:2;
> @@ -268,6 +269,15 @@ struct dsch_vn_quanta {
>
> #define DEFAULT_DVN_DESC_WR_MERGE_TIMEOUT_MAX 0x3FF
>
> +#define PACKET_RING_MIN 8U
> +#define PACKET_RING_MAX 32U
> +#define SPLIT_RING_MIN 8U
> +#define SPLIT_RING_MAX 16U
> +#define PACKET_RING_BASE 8U
> +#define PACKET_RING_DIV 4U
> +#define SPLIT_RING_CFG_8 0U
> +#define SPLIT_RING_CFG_16 1U
> +
> struct nbl_dvn_descreq_num_cfg {
> u32 avring_cfg_num:1; /* spilit ring descreq_num 0:8,1:16 */
> u32 rsv0:3;
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> index 770a255ab06f..403e74b9b1d0 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_hw_leonis/nbl_resource_leonis.c
> @@ -46,6 +46,10 @@ nbl_res_setup_ops(struct device *dev, struct nbl_resource_mgt *res_mgt)
>
> if (!is_ops_inited) {
> ret = nbl_intr_setup_ops(&res_ops);
> + if (ret)
> + return ERR_PTR(-ENOMEM);
> +
> + ret = nbl_vsi_setup_ops(&res_ops);
> if (ret)
> return ERR_PTR(-ENOMEM);
> is_ops_inited = true;
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> index 1b80676cf19a..675649ffb271 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_resource.h
> @@ -110,6 +110,7 @@ void nbl_res_get_eth_id(struct nbl_resource_mgt *res_mgt, u16 vsi_id,
> u8 *eth_mode, u8 *eth_id, u8 *logic_eth_id);
> int nbl_intr_mgt_start(struct nbl_resource_mgt *res_mgt);
> int nbl_intr_setup_ops(struct nbl_resource_ops *resource_ops);
> +int nbl_vsi_setup_ops(struct nbl_resource_ops *resource_ops);
> void nbl_res_pf_dev_vsi_type_to_hw_vsi_type(u16 src_type,
> enum nbl_vsi_serv_type *dst_type);
> #endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.c b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.c
> new file mode 100644
> index 000000000000..67b9b23ad012
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.c
> @@ -0,0 +1,51 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +#include <linux/device.h>
> +#include "nbl_vsi.h"
> +
> +static void nbl_res_vsi_deinit_chip_module(struct nbl_resource_mgt *res_mgt)
> +{
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> +
> + hw_ops->deinit_chip_module(res_mgt->hw_ops_tbl->priv);
> +}
> +
> +static int nbl_res_vsi_init_chip_module(struct nbl_resource_mgt *res_mgt)
> +{
> + u8 eth_speed = res_mgt->resource_info->board_info.eth_speed;
> + u8 eth_num = res_mgt->resource_info->board_info.eth_num;
> + struct nbl_hw_ops *hw_ops = res_mgt->hw_ops_tbl->ops;
> + struct nbl_hw_mgt *p = res_mgt->hw_ops_tbl->priv;
> + int ret;
> +
> + ret = hw_ops->init_chip_module(p, eth_speed, eth_num);
> +
> + return ret;
> +}
> +
> +/* NBL_VSI_SET_OPS(ops_name, func)
> + *
> + * Use X Macros to reduce setup and remove codes.
> + */
> +#define NBL_VSI_OPS_TBL \
> +do { \
> + NBL_VSI_SET_OPS(init_chip_module, \
> + nbl_res_vsi_init_chip_module); \
> + NBL_VSI_SET_OPS(deinit_chip_module, \
> + nbl_res_vsi_deinit_chip_module); \
> +} while (0)
> +
> +int nbl_vsi_setup_ops(struct nbl_resource_ops *res_ops)
> +{
> +#define NBL_VSI_SET_OPS(name, func) \
> + do { \
> + res_ops->NBL_NAME(name) = func; \
> + ; \
> + } while (0)
> + NBL_VSI_OPS_TBL;
> +#undef NBL_VSI_SET_OPS
> +
> + return 0;
> +}
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.h
> new file mode 100644
> index 000000000000..6fd79add57d6
> --- /dev/null
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_hw/nbl_vsi.h
> @@ -0,0 +1,11 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2025 Nebula Matrix Limited.
> + */
> +
> +#ifndef _NBL_VSI_H_
> +#define _NBL_VSI_H_
> +
> +#include "nbl_resource.h"
> +
> +#endif
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> index f97f7a810ad0..93dfe74d3643 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_def_hw.h
> @@ -11,6 +11,10 @@
> struct nbl_hw_mgt;
> struct nbl_adapter;
> struct nbl_hw_ops {
> + int (*init_chip_module)(struct nbl_hw_mgt *hw_mgt, u8 eth_speed,
> + u8 eth_num);
> + void (*deinit_chip_module)(struct nbl_hw_mgt *hw_mgt);
> +
> void (*configure_msix_map)(struct nbl_hw_mgt *hw_mgt, u16 func_id,
> bool valid, dma_addr_t dma_addr, u8 bus,
> u8 devid, u8 function);
> diff --git a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> index e4f11e6ded94..5203bb2a9a5f 100644
> --- a/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> +++ b/drivers/net/ethernet/nebula-matrix/nbl/nbl_include/nbl_include.h
> @@ -45,4 +45,35 @@ struct nbl_init_param {
> bool pci_using_dac;
> };
>
> +enum nbl_fw_port_speed {
> + NBL_FW_PORT_SPEED_10G,
> + NBL_FW_PORT_SPEED_25G,
> + NBL_FW_PORT_SPEED_50G,
> + NBL_FW_PORT_SPEED_100G,
> +};
> +
> +#define NBL_OPS_CALL(func, para) \
> +do { \
> + typeof(func) _func = (func); \
> + if (_func) \
> + _func para; \
> +} while (0)
> +
> +#define NBL_OPS_CALL_RET(func, para) \
> +({ \
> + typeof(func) _func = (func); \
> + _func ? _func para : 0; \
> +})
> +
> +#define NBL_OPS_CALL_RET_PTR(func, para) \
> +({ \
> + typeof(func) _func = (func); \
> + _func ? _func para : NULL; \
> +})
> +
> +enum nbl_performance_mode {
> + NBL_QUIRKS_NO_TOE,
> + NBL_QUIRKS_UVN_PREFETCH_ALIGN,
> +};
> +
> #endif
^ permalink raw reply
* Re: [PATCH 3/5] compiler_attributes: Add overflow_behavior macros __ob_trap and __ob_wrap
From: David Laight @ 2026-04-02 9:13 UTC (permalink / raw)
To: Kees Cook
Cc: Peter Zijlstra, Justin Stitt, Miguel Ojeda, Marco Elver,
Andrey Konovalov, Andrey Ryabinin, Jonathan Corbet, Shuah Khan,
Miguel Ojeda, Nathan Chancellor, kasan-dev, linux-doc, llvm,
Linus Torvalds, Nicolas Schier, Arnd Bergmann, Greg Kroah-Hartman,
Andrew Morton, linux-kernel, linux-hardening, linux-kbuild
In-Reply-To: <202604011313.AD471BC8@keescook>
On Wed, 1 Apr 2026 13:21:17 -0700
Kees Cook <kees@kernel.org> wrote:
> On Wed, Apr 01, 2026 at 11:08:15AM +0200, Peter Zijlstra wrote:
> > On Tue, Mar 31, 2026 at 12:52:10PM -0700, Kees Cook wrote:
> >
> > > I think for this series, __ob_trap/__ob_wrap is what should be used.
> > >
> > > And for other folks, the background here is that we originally wanted
> > > to use macros for "__trap" and "__wrap", but the powerpc C compiler
> > > (both Clang and GCC) have a builtin macro named "__trap" already. So
> > > I switched to just using the Clang-native type qualifier. We can use
> > > the attribute style too, but there was a lot of confusion during the
> > > Clang development phases where people kept forgetting this was a type
> > > qualifier, not an attribute (i.e. the attribute is an internal alias
> > > for the qualifier, and the qualifier is a new type).
> >
> > Since you mention qualifiers...
> >
> > What is the result of __typeof_unqual__(int __ob_trap) ?
>
> Hmm, it seems like "const" doesn't get peeled off. That can be fixed, if
> that's needed?
>
> 'typeof_unqual(int)' (aka 'int')
> 'typeof_unqual(__ob_trap int)' (aka '__ob_trap int')
> 'typeof_unqual(const int)' (aka 'int')
> 'typeof_unqual(__ob_trap const int)' (aka '__ob_trap const int')
>
> -Kees
>
Adding all the required cases to the _Generic() doesn't scale.
typeof_unqual() needs to die.
Just using 'auto a = b;' should remove const and volatile - but gcc is buggy.
There are some alternatives that work in many cases.
(It has all been discussed before.)
In most cases you can use 'auto a = (b) + 0'.
That does do integer promotions - but they happen as soon as 'a' is
used; so it pretty much doesn't change the type of value, just the
type of the variable.
David
^ permalink raw reply
* Re: [PATCH v2] doc: Add CPU Isolation documentation
From: Frederic Weisbecker @ 2026-04-02 9:15 UTC (permalink / raw)
To: Steven Rostedt
Cc: Randy Dunlap, LKML, Anna-Maria Behnsen, Gabriele Monaco,
Ingo Molnar, Jonathan Corbet, Marcelo Tosatti, Marco Crivellari,
Michal Hocko, Paul E . McKenney, Peter Zijlstra, Phil Auld,
Thomas Gleixner, Valentin Schneider, Vlastimil Babka, Waiman Long,
linux-doc, Sebastian Andrzej Siewior, Bagas Sanjaya
In-Reply-To: <20260401130855.02c161d8@gandalf.local.home>
Le Wed, Apr 01, 2026 at 01:08:55PM -0400, Steven Rostedt a écrit :
> On Wed, 1 Apr 2026 18:27:03 +0200
> Frederic Weisbecker <frederic@kernel.org> wrote:
>
> > > > +"CPU Isolation" means leaving a CPU exclusive to a given workload
> > > > +without any undesired code interference from the kernel.
> > > > +
> > > > +Those interferences, commonly pointed out as "noise", can be triggered
> > >
> > > nit: "noise,"
> >
> > Thanks! I have applied all your suggestions, except this one for now because I don't
> > really understand the typo rule behind. Any hint?
>
> So this looks to be an American English thing (placing commas within the
> quote), but from what I read, British English places the comma outside the
> quote.
So does french english :-)
>
> Here's one case I much rather go the British English way. This also means
> it's only incorrect to Americans ;-)
Ok thank you both!
--
Frederic Weisbecker
SUSE Labs
^ permalink raw reply
* [PATCH v3] doc: Add CPU Isolation documentation
From: Frederic Weisbecker @ 2026-04-02 9:47 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Anna-Maria Behnsen, Gabriele Monaco,
Ingo Molnar, Jonathan Corbet, Marcelo Tosatti, Marco Crivellari,
Michal Hocko, Paul E . McKenney, Peter Zijlstra, Phil Auld,
Steven Rostedt, Thomas Gleixner, Valentin Schneider,
Vlastimil Babka, Waiman Long, linux-doc,
Sebastian Andrzej Siewior, Bagas Sanjaya
nohz_full was introduced in v3.10 in 2013, which means this
documentation is overdue for 13 years.
Fortunately Paul wrote a part of the needed documentation a while ago,
especially concerning nohz_full in Documentation/timers/no_hz.rst and
also about per-CPU kthreads in
Documentation/admin-guide/kernel-per-CPU-kthreads.rst
Introduce a new page that gives an overview of CPU isolation in general.
Acked-by: Waiman Long <longman@redhat.com>
Reviewed-by: Valentin Schneider <vschneid@redhat.com>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
---
v3: Apply suggestions from Randy, Steven, Valentin, Waiman and also Sashiko!
Documentation/admin-guide/cpu-isolation.rst | 357 ++++++++++++++++++++
Documentation/admin-guide/index.rst | 1 +
2 files changed, 358 insertions(+)
create mode 100644 Documentation/admin-guide/cpu-isolation.rst
diff --git a/Documentation/admin-guide/cpu-isolation.rst b/Documentation/admin-guide/cpu-isolation.rst
new file mode 100644
index 000000000000..8c65d03fd28c
--- /dev/null
+++ b/Documentation/admin-guide/cpu-isolation.rst
@@ -0,0 +1,357 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=============
+CPU Isolation
+=============
+
+Introduction
+============
+
+"CPU Isolation" means leaving a CPU exclusive to a given workload
+without any undesired code interference from the kernel.
+
+Those interferences, commonly pointed out as "noise", can be triggered
+by asynchronous events (interrupts, timers, scheduler preemption by
+workqueues and kthreads, ...) or synchronous events (syscalls and page
+faults).
+
+Such noise usually goes unnoticed. After all, synchronous events are a
+component of the requested kernel service. And asynchronous events are
+either sufficiently well-distributed by the scheduler when executed
+as tasks or reasonably fast when executed as interrupt. The timer
+interrupt can even execute 1024 times per seconds without a significant
+and measurable impact most of the time.
+
+However some rare and extreme workloads can be quite sensitive to
+those kinds of noise. This is the case, for example, with high
+bandwidth network processing that can't afford losing a single packet
+or very low latency network processing. Typically those use cases
+involve DPDK, bypassing the kernel networking stack and performing
+direct access to the networking device from userspace.
+
+In order to run a CPU without or with limited kernel noise, the
+related housekeeping work needs to be either shut down, migrated or
+offloaded.
+
+Housekeeping
+============
+
+In the CPU isolation terminology, housekeeping is the work, often
+asynchronous, that the kernel needs to process in order to maintain
+all its services. It matches the noises and disturbances enumerated
+above except when at least one CPU is isolated. Then housekeeping may
+make use of further coping mechanisms if CPU-tied work must be
+offloaded.
+
+Housekeeping CPUs are the non-isolated CPUs where the kernel noise
+is moved away from isolated CPUs.
+
+The isolation can be implemented in several ways depending on the
+nature of the noise:
+
+- Unbound work, where "unbound" means not tied to any CPU, can be
+ simply migrated away from isolated CPUs to housekeeping CPUs.
+ This is the case of unbound workqueues, kthreads and timers.
+
+- Bound work, where "bound" means tied to a specific CPU, usually
+ can't be moved away as-is by nature. Either:
+
+ - The work must switch to a locked implementation. E.g.:
+ This is the case of RCU with CONFIG_RCU_NOCB_CPU.
+
+ - The related feature must be shut down and considered
+ incompatible with isolated CPUs. E.g.: Lockup watchdog,
+ unreliable clocksources, etc...
+
+ - An elaborate and heavyweight coping mechanism stands as a
+ replacement. E.g.: the timer tick is shut down on nohz_full
+ CPUs but with the constraint of running a single task on
+ them. A significant cost penalty is added on kernel entry/exit
+ and a residual 1Hz scheduler tick is offloaded to housekeeping
+ CPUs.
+
+In any case, housekeeping work has to be handled, which is why there
+must be at least one housekeeping CPU in the system, preferably more
+if the machine runs a lot of CPUs. For example one per node on NUMA
+systems.
+
+Also CPU isolation often means a tradeoff between noise-free isolated
+CPUs and added overhead on housekeeping CPUs, sometimes even on
+isolated CPUs entering the kernel.
+
+Isolation features
+==================
+
+Different levels of isolation can be configured in the kernel, each of
+which has its own drawbacks and tradeoffs.
+
+Scheduler domain isolation
+--------------------------
+
+This feature isolates a CPU from the scheduler topology. As a result,
+the target isn't part of the load balancing. Tasks won't migrate
+either from or to it unless affined explicitly.
+
+As a side effect the CPU is also isolated from unbound workqueues and
+unbound kthreads.
+
+Requirements
+~~~~~~~~~~~~
+
+- CONFIG_CPUSETS=y for the cpusets-based interface
+
+Tradeoffs
+~~~~~~~~~
+
+By nature, the system load is overall less distributed since some CPUs
+are extracted from the global load balancing.
+
+Interfaces
+~~~~~~~~~~
+
+- Documentation/admin-guide/cgroup-v2.rst cpuset isolated partitions are recommended
+ because they are tunable at runtime.
+
+- The 'isolcpus=' kernel boot parameter with the 'domain' flag is a
+ less flexible alternative that doesn't allow for runtime
+ reconfiguration.
+
+IRQs isolation
+--------------
+
+Isolate the IRQs whenever possible, so that they don't fire on the
+target CPUs.
+
+Interfaces
+~~~~~~~~~~
+
+- The file /proc/irq/\*/smp_affinity as explained in detail in
+ Documentation/core-api/irq/irq-affinity.rst page.
+
+- The "irqaffinity=" kernel boot parameter for a default setting.
+
+- The "managed_irq" flag in the "isolcpus=" kernel boot parameter
+ tries a best effort affinity override for managed IRQs.
+
+Full Dynticks (aka nohz_full)
+-----------------------------
+
+Full dynticks extends the dynticks idle mode, which stops the tick when
+the CPU is idle, to CPUs running a single task in userspace. That is,
+the timer tick is stopped if the environment allows it.
+
+Global timer callbacks are also isolated from the nohz_full CPUs.
+
+Requirements
+~~~~~~~~~~~~
+
+- CONFIG_NO_HZ_FULL=y
+
+Constraints
+~~~~~~~~~~~
+
+- The isolated CPUs must run a single task only. Multitask requires
+ the tick to maintain preemption. This is usually fine since the
+ workload usually can't stand the latency of random context switches.
+
+- No call to the kernel from isolated CPUs, at the risk of triggering
+ random noise.
+
+- No use of POSIX CPU timers on isolated CPUs.
+
+- Architecture must have a stable and reliable clocksource (no
+ unreliable TSC that requires the watchdog).
+
+
+Tradeoffs
+~~~~~~~~~
+
+In terms of cost, this is the most invasive isolation feature. It is
+assumed to be used when the workload spends most of its time in
+userspace and doesn't rely on the kernel except for preparatory
+work because:
+
+- RCU adds more overhead due to the locked, offloaded and threaded
+ callbacks processing (the same that would be obtained with "rcu_nocbs"
+ boot parameter).
+
+- Kernel entry/exit through syscalls, exceptions and IRQs are more
+ costly due to fully ordered RmW operations that maintain userspace
+ as RCU extended quiescent state. Also the CPU time is accounted on
+ kernel boundaries instead of periodically from the tick.
+
+- Housekeeping CPUs must run a 1Hz residual remote scheduler tick
+ on behalf of the isolated CPUs.
+
+Checklist
+=========
+
+You have set up each of the above isolation features but you still
+observe jitters that trash your workload? Make sure to check a few
+elements before proceeding.
+
+Some of these checklist items are similar to those of real-time
+workloads:
+
+- Use mlock() to prevent your pages from being swapped away. Page
+ faults are usually not compatible with jitter sensitive workloads.
+
+- Avoid SMT to prevent your hardware thread from being "preempted"
+ by another one.
+
+- CPU frequency changes may induce subtle sorts of jitter in a
+ workload. Cpufreq should be used and tuned with caution.
+
+- Deep C-states may result in latency issues upon wake-up. If this
+ happens to be a problem, C-states can be limited via kernel boot
+ parameters such as processor.max_cstate or intel_idle.max_cstate.
+ More finegrained tunings are described in
+ Documentation/admin-guide/pm/cpuidle.rst page
+
+- Your system may be subject to firmware-originating interrupts - x86 has
+ System Management Interrupts (SMIs) for example. Check your system BIOS
+ to disable such interference, and with some luck your vendor will have
+ a BIOS tuning guidance for low-latency operations.
+
+
+Full isolation example
+======================
+
+In this example, the system has 8 CPUs and the 8th is to be fully
+isolated. Since CPUs start from 0, the 8th CPU is CPU 7.
+
+Kernel parameters
+-----------------
+
+Set the following kernel boot parameters to disable SMT and setup tick
+and IRQ isolation:
+
+- Full dynticks: nohz_full=7
+
+- IRQs isolation: irqaffinity=0-6
+
+- Managed IRQs isolation: isolcpus=managed_irq,7
+
+- Prevent SMT: nosmt
+
+The full command line is then:
+
+ nohz_full=7 irqaffinity=0-6 isolcpus=managed_irq,7 nosmt
+
+CPUSET configuration (cgroup v2)
+--------------------------------
+
+Assuming cgroup v2 is mounted to /sys/fs/cgroup, the following script
+isolates CPU 7 from scheduler domains.
+
+::
+
+ cd /sys/fs/cgroup
+ # Activate the cpuset subsystem
+ echo +cpuset > cgroup.subtree_control
+ # Create partition to be isolated
+ mkdir test
+ cd test
+ echo +cpuset > cgroup.subtree_control
+ # Isolate CPU 7
+ echo 7 > cpuset.cpus
+ echo "isolated" > cpuset.cpus.partition
+
+The userspace workload
+----------------------
+
+Fake a pure userspace workload, the program below runs a dummy
+userspace loop on the isolated CPU 7.
+
+::
+
+ #include <stdio.h>
+ #include <fcntl.h>
+ #include <unistd.h>
+ #include <errno.h>
+ int main(void)
+ {
+ // Move the current task to the isolated cpuset (bind to CPU 7)
+ int fd = open("/sys/fs/cgroup/test/cgroup.procs", O_WRONLY);
+ if (fd < 0) {
+ perror("Can't open cpuset file...\n");
+ return 0;
+ }
+
+ write(fd, "0\n", 2);
+ close(fd);
+
+ // Run an endless dummy loop until the launcher kills us
+ while (1)
+ ;
+
+ return 0;
+ }
+
+Build it and save for later step:
+
+::
+
+ # gcc user_loop.c -o user_loop
+
+The launcher
+------------
+
+The below launcher runs the above program for 10 seconds and traces
+the noise resulting from preempting tasks and IRQs.
+
+::
+
+ TRACING=/sys/kernel/tracing/
+ # Make sure tracing is off for now
+ echo 0 > $TRACING/tracing_on
+ # Flush previous traces
+ echo > $TRACING/trace
+ # Record disturbance from other tasks
+ echo 1 > $TRACING/events/sched/sched_switch/enable
+ # Record disturbance from interrupts
+ echo 1 > $TRACING/events/irq_vectors/enable
+ # Now we can start tracing
+ echo 1 > $TRACING/tracing_on
+ # Run the dummy user_loop for 10 seconds on CPU 7
+ ./user_loop &
+ USER_LOOP_PID=$!
+ sleep 10
+ kill $USER_LOOP_PID
+ # Disable tracing and save traces from CPU 7 in a file
+ echo 0 > $TRACING/tracing_on
+ cat $TRACING/per_cpu/cpu7/trace > trace.7
+
+If no specific problem arose, the output of trace.7 should look like
+the following:
+
+::
+
+ <idle>-0 [007] d..2. 1980.976624: sched_switch: prev_comm=swapper/7 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=user_loop next_pid=1553 next_prio=120
+ user_loop-1553 [007] d.h.. 1990.946593: reschedule_entry: vector=253
+ user_loop-1553 [007] d.h.. 1990.946593: reschedule_exit: vector=253
+
+That is, no specific noise triggered between the first trace and the
+second during 10 seconds when user_loop was running.
+
+Debugging
+=========
+
+Of course things are never so easy, especially on this matter.
+Chances are that actual noise will be observed in the aforementioned
+trace.7 file.
+
+The best way to investigate further is to enable finer grained
+tracepoints such as those of subsystems producing asynchronous
+events: workqueue, timer, irq_vector, etc... It also can be
+interesting to enable the tick_stop event to diagnose why the tick is
+retained when that happens.
+
+Some tools may also be useful for higher level analysis:
+
+- Documentation/tools/rtla/rtla.rst provides a suite of tools to analyze
+ latency and noise in the system. For example Documentation/tools/rtla/rtla-osnoise.rst
+ runs a kernel tracer that analyzes and output a summary of the noises.
+
+- dynticks-testing does something similar to rtla-osnoise but in userspace. It is available
+ at git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
index b734f8a2a2c4..cd28dfe91b06 100644
--- a/Documentation/admin-guide/index.rst
+++ b/Documentation/admin-guide/index.rst
@@ -94,6 +94,7 @@ likely to be of interest on almost any system.
cgroup-v2
cgroup-v1/index
+ cpu-isolation
cpu-load
mm/index
module-signing
--
2.53.0
^ permalink raw reply related
* Re: [net-next,13/15] ice: add support for unmanaged DPLL on E830 NIC
From: Paolo Abeni @ 2026-04-02 10:26 UTC (permalink / raw)
To: anthony.l.nguyen
Cc: andriy.shevchenko, pmenzel, aleksandr.loktionov, mika.westerberg,
pabeni, mbloch, saeedm, piotr.kwapulinski, sx.rinitha, kuba,
grzegorz.nitka, netdev, przemyslaw.kitszel, arkadiusz.kubalewski,
tariqt, davem, dima.ruinskiy, bhelgaas, andrew+netdev, lukas,
jacob.e.keller, richardcochran, takkozu, enjuk,
maximilianpezzullo, joe, przemyslaw.korba, leon, vgrinber,
linux-pci, sunithax.d.mekala, corbet, vinicius.gomes,
ilpo.jarvinen, edumazet, linux-doc, kohei.enju, kohei, horms,
avigailx.dahan
In-Reply-To: <20260330230248.646900-14-anthony.l.nguyen@intel.com>
This is an AI-generated review of your patch. The human sending this
email has considered the AI review valid, or at least plausible.
For local reproduction steps see https://netdev-ai.bots.linux.dev/ai-local.html
---
ice: add support for unmanaged DPLL on E830 NIC
Hardware variants of E830 may support an unmanaged DPLL where the
configuration is hardcoded within the hardware and firmware, meaning
users cannot modify settings. However, users are able to check the DPLL
lock status and obtain configuration information through the Linux DPLL
and devlink health subsystem.
> diff --git a/drivers/net/ethernet/intel/ice/ice_dpll.c b/drivers/net/ethernet/intel/ice/ice_dpll.c
> index 62f75701d652..2a9eb233dbf4 100644
> --- a/drivers/net/ethernet/intel/ice/ice_dpll.c
> +++ b/drivers/net/ethernet/intel/ice/ice_dpll.c
[ ... ]
> @@ -3148,12 +3176,15 @@ ice_dpll_init_direct_pins(struct ice_pf *pf, bool cgu,
> int ret;
>
> ret = ice_dpll_get_pins(pf, pins, start_idx, count, pf->dplls.clock_id);
> - if (ret)
> + if (!cgu || ret)
> return ret;
> - if (cgu) {
> +
> + if (first) {
> ret = ice_dpll_register_pins(first, pins, ops, count);
> if (ret)
> goto release_pins;
> + }
> + if (second) {
> ret = ice_dpll_register_pins(second, pins, ops, count);
> if (ret)
> goto unregister_first;
> @@ -3162,7 +3193,8 @@ ice_dpll_init_direct_pins(struct ice_pf *pf, bool cgu,
> return 0;
>
> unregister_first:
> - ice_dpll_unregister_pins(first, pins, ops, count);
> + if (first)
> + ice_dpll_unregister_pins(first, pins, ops, count);
> release_pins:
> ice_dpll_release_pins(pins, count);
> return ret;
NULL checks were added to ice_dpll_init_direct_pins for handling unmanaged
DPLL initialization where only one DPLL device is active. However, is the
corresponding cleanup function ice_dpll_deinit_direct_pins missing the same
NULL checks?
In unmanaged mode, only pf->dplls.pps.dpll is initialized while
pf->dplls.eec.dpll remains NULL. When ice_dpll_init_pins calls
ice_dpll_init_direct_pins(pf, true, outputs, count, num_outputs,
output_ops, eec.dpll, pps.dpll), if this second call (for output pins)
fails, the error path calls ice_dpll_deinit_direct_pins(pf, true, ...,
pps.dpll, eec.dpll).
Looking at ice_dpll_deinit_direct_pins:
static void ice_dpll_deinit_direct_pins(struct ice_pf *pf, bool cgu,
struct ice_dpll_pin *pins, int count,
const struct dpll_pin_ops *ops,
struct dpll_device *first,
struct dpll_device *second)
{
if (cgu) {
ice_dpll_unregister_pins(first, pins, ops, count);
ice_dpll_unregister_pins(second, pins, ops, count);
}
ice_dpll_release_pins(pins, count);
}
When cgu=true and second=NULL, ice_dpll_unregister_pins is called with
dpll=NULL. Then dpll_pin_unregister(dpll, pin) dereferences dpll with
xa_empty(&dpll->pin_refs), causing a NULL pointer dereference.
Shouldn't ice_dpll_deinit_direct_pins add NULL checks for first and second,
matching the guards you added in ice_dpll_init_direct_pins?
> diff --git a/drivers/net/ethernet/intel/ice/ice_dpll.h b/drivers/net/ethernet/intel/ice/ice_dpll.h
> index ae42cdea0ee1..2c98b6c6deb0 100644
> --- a/drivers/net/ethernet/intel/ice/ice_dpll.h
> +++ b/drivers/net/ethernet/intel/ice/ice_dpll.h
[ ... ]
> @@ -173,3 +181,5 @@ static inline void ice_dpll_deinit(struct ice_pf *pf) { }
> #define ICE_CGU_R11_SYNCE_S_BYP_CLK GENMASK(6, 1)
>
> #define ICE_CGU_BYPASS_MUX_OFFSET_E825C 3
> +#define ICE_DPLL_UNMANAGED_PIN_NUM 4
> +#define ICE_DPLL_IN_ESYNC_ENABLED ICE_AQC_GET_CGU_IN_CFG_FLG2_ESYNC_EN
This isn't a bug, but ICE_DPLL_UNMANAGED_PIN_NUM is defined but never
used. Looking at the unmanaged pin descriptors, ice_e830_unmanaged_inputs
has 1 entry and ice_e830_unmanaged_outputs has 2 entries (totaling 3 pins,
not 4).
^ permalink raw reply
* Re: [PATCH v12 02/15] powerpc/crash: Fix possible memory leak in update_crash_elfcorehdr()
From: Sourabh Jain @ 2026-04-02 10:57 UTC (permalink / raw)
To: Jinjie Ruan, corbet, skhan, catalin.marinas, will, chenhuacai,
kernel, maddy, mpe, npiggin, chleroy, pjw, palmer, aou, alex,
tglx, mingo, bp, dave.hansen, hpa, robh, saravanak, akpm, bhe,
vgoyal, dyoung, rdunlap, peterz, pawan.kumar.gupta, feng.tang,
dapeng1.mi, kees, elver, paulmck, lirongqing, rppt, leitao, ardb,
jbohac, cfsworks, tangyouling, ritesh.list, hbathini, eajames,
guoren, songshuaishuai, kevin.brodsky, vishal.moola, junhui.liu,
coxu, fuqiang.wang, liaoyuanhong, takahiro.akashi, james.morse,
lizhengyu3, x86, linux-doc, linux-kernel, linux-arm-kernel,
loongarch, linuxppc-dev, linux-riscv, devicetree, kexec
In-Reply-To: <20260402072701.628293-3-ruanjinjie@huawei.com>
On 02/04/26 12:56, Jinjie Ruan wrote:
> In get_crash_memory_ranges(), if crash_exclude_mem_range() failed
> after realloc_mem_ranges() has successfully allocated the cmem
> memory, it just returns an error but leaves cmem pointing to
> the allocated memory, nor is it freed in the caller
> update_crash_elfcorehdr(), which cause a memory leak, goto out
> to free the cmem.
>
> Cc: Sourabh Jain <sourabhjain@linux.ibm.com>
> Cc: Hari Bathini <hbathini@linux.ibm.com>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Fixes: 849599b702ef ("powerpc/crash: add crash memory hotplug support")
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> ---
> arch/powerpc/kexec/crash.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/kexec/crash.c b/arch/powerpc/kexec/crash.c
> index a325c1c02f96..1d12cef8e1e0 100644
> --- a/arch/powerpc/kexec/crash.c
> +++ b/arch/powerpc/kexec/crash.c
> @@ -440,7 +440,7 @@ static void update_crash_elfcorehdr(struct kimage *image, struct memory_notify *
> ret = get_crash_memory_ranges(&cmem);
> if (ret) {
> pr_err("Failed to get crash mem range\n");
> - return;
> + goto out;
> }
>
> /*
Thanks for fixing this Jinjie.
Feel free to add:
Reviewed-by: Sourabh Jain <sourabhjain@linux.ibm.com>
- Sourabh Jain
^ permalink raw reply
* Re: [PATCH v3] doc: Add CPU Isolation documentation
From: Sebastian Andrzej Siewior @ 2026-04-02 11:01 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: LKML, Anna-Maria Behnsen, Gabriele Monaco, Ingo Molnar,
Jonathan Corbet, Marcelo Tosatti, Marco Crivellari, Michal Hocko,
Paul E . McKenney, Peter Zijlstra, Phil Auld, Steven Rostedt,
Thomas Gleixner, Valentin Schneider, Vlastimil Babka, Waiman Long,
linux-doc, Bagas Sanjaya
In-Reply-To: <20260402094749.18879-1-frederic@kernel.org>
On 2026-04-02 11:47:49 [+0200], Frederic Weisbecker wrote:
> nohz_full was introduced in v3.10 in 2013, which means this
> documentation is overdue for 13 years.
>
> Fortunately Paul wrote a part of the needed documentation a while ago,
> especially concerning nohz_full in Documentation/timers/no_hz.rst and
> also about per-CPU kthreads in
> Documentation/admin-guide/kernel-per-CPU-kthreads.rst
>
> Introduce a new page that gives an overview of CPU isolation in general.
>
> Acked-by: Waiman Long <longman@redhat.com>
> Reviewed-by: Valentin Schneider <vschneid@redhat.com>
> Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
This documents also isolcpus= boot argument. The only thing that this
argument does and runtime can not do is the managed_irq sub argument.
This sub argument is a story of its own and it is of quite limited for
me taste.
However, isolcpus= is marked as deprecated. I suggest to remove the
"Deprecated - use cpusets instead" note as the static configuration is
fine if the system is partitioned once never changed within its
lifetime.
Are there any objections and if so why needs this boot argument be
removed (assuming we have a runtime equivalent knob for managed_irq)?
Sebastian
^ permalink raw reply
* Re: [PATCH v12 00/15] arm64/riscv: Add support for crashkernel CMA reservation
From: Borislav Petkov @ 2026-04-02 11:31 UTC (permalink / raw)
To: Jinjie Ruan
Cc: corbet, skhan, catalin.marinas, will, chenhuacai, kernel, maddy,
mpe, npiggin, chleroy, pjw, palmer, aou, alex, tglx, mingo,
dave.hansen, hpa, robh, saravanak, akpm, bhe, vgoyal, dyoung,
rdunlap, peterz, pawan.kumar.gupta, feng.tang, dapeng1.mi, kees,
elver, paulmck, lirongqing, rppt, leitao, ardb, jbohac, cfsworks,
tangyouling, sourabhjain, ritesh.list, hbathini, eajames, guoren,
songshuaishuai, kevin.brodsky, vishal.moola, junhui.liu, coxu,
fuqiang.wang, liaoyuanhong, takahiro.akashi, james.morse,
lizhengyu3, x86, linux-doc, linux-kernel, linux-arm-kernel,
loongarch, linuxppc-dev, linux-riscv, devicetree, kexec
In-Reply-To: <20260402072701.628293-1-ruanjinjie@huawei.com>
On Thu, Apr 02, 2026 at 03:26:46PM +0800, Jinjie Ruan wrote:
> The crash memory allocation, and the exclude of crashk_res, crashk_low_res
> and crashk_cma memory are almost identical across different architectures,
> This patch set handle them in crash core in a general way, which eliminate
> a lot of duplication code.
From: Documentation/process/submitting-patches.rst
"Don't get discouraged - or impatient
------------------------------------
After you have submitted your change, be patient and wait. Reviewers are
busy people and may not get to your patch right away.
Once upon a time, patches used to disappear into the void without comment,
but the development process works more smoothly than that now. You should
receive comments within a week or so; if that does not happen, make sure
that you have sent your patches to the right place. Wait for a minimum of
^^^^^^^^^^^^^^^^^^^^^
one week before resubmitting or pinging reviewers - possibly longer during
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
busy times like merge windows."
You need to be patient and send once a week and not spam people:
Feb 04 Jinjie Ruan ( :1.0K|) [PATCH v3 0/3] arm64/riscv: Add support for crashkernel CMA reservation
Feb 09 Jinjie Ruan ( :1.5K|) [PATCH v4 0/3] arm64/riscv: Add support for crashkernel CMA reservation
Feb 12 Jinjie Ruan ( :2.1K|) [PATCH v5 0/4] arm64/riscv: Add support for crashkernel CMA reservation
Feb 24 Jinjie Ruan ( :2.3K|) [PATCH v6 0/5] arm64/riscv: Add support for crashkernel CMA reservation
Feb 26 Jinjie Ruan ( :2.5K|) [PATCH v7 0/5] arm64/riscv: Add support for crashkernel CMA reservation
Mar 02 Jinjie Ruan ( :2.8K|) [PATCH v8 0/5] arm64/riscv: Add support for crashkernel CMA reservation
Mar 23 Jinjie Ruan ( :2.9K|) [PATCH v9 0/5] arm64/riscv: Add support for crashkernel CMA reservation
Mar 25 Jinjie Ruan ( :3.7K|) [PATCH v10 0/8] arm64/riscv: Add support for crashkernel CMA reservation
Mar 28 Jinjie Ruan ( :4.3K|) [PATCH v11 00/11] arm64/riscv: Add support for crashkernel CMA reservation
Apr 02 Jinjie Ruan ( :4.5K|) [PATCH v12 00/15] arm64/riscv: Add support for crashkernel CMA reservation
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply
* Re: [PATCH v12 00/15] arm64/riscv: Add support for crashkernel CMA reservation
From: Jinjie Ruan @ 2026-04-02 11:47 UTC (permalink / raw)
To: Borislav Petkov
Cc: corbet, skhan, catalin.marinas, will, chenhuacai, kernel, maddy,
mpe, npiggin, chleroy, pjw, palmer, aou, alex, tglx, mingo,
dave.hansen, hpa, robh, saravanak, akpm, bhe, vgoyal, dyoung,
rdunlap, peterz, pawan.kumar.gupta, feng.tang, dapeng1.mi, kees,
elver, paulmck, lirongqing, rppt, leitao, ardb, jbohac, cfsworks,
tangyouling, sourabhjain, ritesh.list, hbathini, eajames, guoren,
songshuaishuai, kevin.brodsky, vishal.moola, junhui.liu, coxu,
fuqiang.wang, liaoyuanhong, takahiro.akashi, james.morse,
lizhengyu3, x86, linux-doc, linux-kernel, linux-arm-kernel,
loongarch, linuxppc-dev, linux-riscv, devicetree, kexec
In-Reply-To: <20260402113137.GAac5Tmc5b7SL98KdY@fat_crate.local>
On 2026/4/2 19:31, Borislav Petkov wrote:
> On Thu, Apr 02, 2026 at 03:26:46PM +0800, Jinjie Ruan wrote:
>> The crash memory allocation, and the exclude of crashk_res, crashk_low_res
>> and crashk_cma memory are almost identical across different architectures,
>> This patch set handle them in crash core in a general way, which eliminate
>> a lot of duplication code.
>
> From: Documentation/process/submitting-patches.rst
>
> "Don't get discouraged - or impatient
> ------------------------------------
>
> After you have submitted your change, be patient and wait. Reviewers are
> busy people and may not get to your patch right away.
>
> Once upon a time, patches used to disappear into the void without comment,
> but the development process works more smoothly than that now. You should
> receive comments within a week or so; if that does not happen, make sure
> that you have sent your patches to the right place. Wait for a minimum of
> ^^^^^^^^^^^^^^^^^^^^^
>
> one week before resubmitting or pinging reviewers - possibly longer during
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> busy times like merge windows."
>
> You need to be patient and send once a week and not spam people:
Thank you for the reminder and for your patience. I apologize for the
frequent updates; I am becoming more familiar with the community's
workflow. I will follow your advice, limit my follow-ups to once a week,
and avoid resending during busy periods like the merge window. I'll wait
for further feedback before taking the next step.
>
> Feb 04 Jinjie Ruan ( :1.0K|) [PATCH v3 0/3] arm64/riscv: Add support for crashkernel CMA reservation
> Feb 09 Jinjie Ruan ( :1.5K|) [PATCH v4 0/3] arm64/riscv: Add support for crashkernel CMA reservation
> Feb 12 Jinjie Ruan ( :2.1K|) [PATCH v5 0/4] arm64/riscv: Add support for crashkernel CMA reservation
> Feb 24 Jinjie Ruan ( :2.3K|) [PATCH v6 0/5] arm64/riscv: Add support for crashkernel CMA reservation
> Feb 26 Jinjie Ruan ( :2.5K|) [PATCH v7 0/5] arm64/riscv: Add support for crashkernel CMA reservation
> Mar 02 Jinjie Ruan ( :2.8K|) [PATCH v8 0/5] arm64/riscv: Add support for crashkernel CMA reservation
> Mar 23 Jinjie Ruan ( :2.9K|) [PATCH v9 0/5] arm64/riscv: Add support for crashkernel CMA reservation
> Mar 25 Jinjie Ruan ( :3.7K|) [PATCH v10 0/8] arm64/riscv: Add support for crashkernel CMA reservation
> Mar 28 Jinjie Ruan ( :4.3K|) [PATCH v11 00/11] arm64/riscv: Add support for crashkernel CMA reservation
> Apr 02 Jinjie Ruan ( :4.5K|) [PATCH v12 00/15] arm64/riscv: Add support for crashkernel CMA reservation
>
^ permalink raw reply
* Re: [PATCH v3 0/5] Support the FEAT_HDBSS introduced in Armv9.5
From: Leonardo Bras @ 2026-04-02 12:42 UTC (permalink / raw)
To: Tian Zheng
Cc: Leonardo Bras, maz, oupton, catalin.marinas, corbet, pbonzini,
will, yuzenghui, wangzhou1, liuyonglong, Jonathan.Cameron,
yezhenyu2, linuxarm, joey.gouly, kvmarm, kvm, linux-arm-kernel,
linux-doc, linux-kernel, skhan, suzuki.poulose
In-Reply-To: <730aaffa-9dfd-40c3-a372-c774d203b6e1@huawei.com>
On Thu, Apr 02, 2026 at 10:40:37AM +0800, Tian Zheng wrote:
>
> On 3/31/2026 10:13 PM, Leonardo Bras wrote:
> > On Wed, Feb 25, 2026 at 12:04:16PM +0800, Tian Zheng wrote:
> > > This series of patches add support to the Hardware Dirty state tracking
> > > Structure(HDBSS) feature, which is introduced by the ARM architecture
> > > in the DDI0601(ID121123) version.
> > >
> > > The HDBSS feature is an extension to the architecture that enhances
> > > tracking translation table descriptors' dirty state, identified as
> > > FEAT_HDBSS. This feature utilizes hardware assistance to achieve dirty
> > > page tracking, aiming to significantly reduce the overhead of scanning
> > > for dirty pages.
> > >
> > > The purpose of this feature is to make the execution overhead of live
> > > migration lower to both the guest and the host, compared to existing
> > > approaches (write-protect or search stage 2 tables).
> > >
> > > After these patches, users(such as qemu) can use the
> > > KVM_CAP_ARM_HW_DIRTY_STATE_TRACK ioctl to enable or disable the HDBSS
> > > feature before and after the live migration.
> > >
> > > v2:
> > > https://lore.kernel.org/linux-arm-kernel/20251121092342.3393318-1-zhengtian10@huawei.com/
> > >
> > > v2->v3 changes:
> > > - Remove the ARM64_HDBSS configuration option and ensure this feature
> > > is only enabled in VHE mode.
> > > - Move HDBSS-related variables to the arch-independent portion of the
> > > kvm structure.
> > > - Remove error messages during HDBSS enable/disable operations
> > > - Change HDBSS buffer flushing from handle_exit to vcpu_put,
> > > check_vcpu_requests, and kvm_handle_guest_abort.
> > > - Add fault handling for HDBSS including buffer full, external abort,
> > > and general protection fault (GPF).
> > > - Add support for a 4KB HDBSS buffer size, mapped to the value 0b0000.
> > > - Add a second argument to the ioctl to turn HDBSS on or off.
> > >
> > > Tian Zheng (1):
> > > KVM: arm64: Document HDBSS ioctl
> > >
> > > eillon (4):
> > > arm64/sysreg: Add HDBSS related register information
> > > KVM: arm64: Add support to set the DBM attr during memory abort
> > > KVM: arm64: Add support for FEAT_HDBSS
> > > KVM: arm64: Enable HDBSS support and handle HDBSSF events
> > >
> > > Documentation/virt/kvm/api.rst | 16 +++++
> > > arch/arm64/include/asm/cpufeature.h | 5 ++
> > > arch/arm64/include/asm/esr.h | 7 ++
> > > arch/arm64/include/asm/kvm_host.h | 17 +++++
> > > arch/arm64/include/asm/kvm_mmu.h | 1 +
> > > arch/arm64/include/asm/kvm_pgtable.h | 4 ++
> > > arch/arm64/include/asm/sysreg.h | 11 +++
> > > arch/arm64/kernel/cpufeature.c | 12 ++++
> > > arch/arm64/kvm/arm.c | 102 +++++++++++++++++++++++++++
> > > arch/arm64/kvm/hyp/pgtable.c | 6 ++
> > > arch/arm64/kvm/hyp/vhe/switch.c | 19 +++++
> > > arch/arm64/kvm/mmu.c | 70 ++++++++++++++++++
> > > arch/arm64/kvm/reset.c | 3 +
> > > arch/arm64/tools/cpucaps | 1 +
> > > arch/arm64/tools/sysreg | 29 ++++++++
> > > include/uapi/linux/kvm.h | 1 +
> > > tools/include/uapi/linux/kvm.h | 1 +
> > > 17 files changed, 305 insertions(+)
> > >
> > > --
> > > 2.33.0
> > >
> >
> > Hi Tian,
> >
> > I was thinking: maybe instead of putting the HDBSS (and HACDBS) stuff
> > across a bunch of KVM files, we should try to focus them all on a single
> > arch/arm64/kvm/dirty_bit.c file (plus a header such as
> > arch/arm64/include/asm/kvm_dirty_bit.h).
> >
> > What is your opinion on that?
> >
> > Thanks!
> > Leo
>
>
> Sorry for the late reply. Yes, I had the same thought before. In the next
> version, I will
>
> move all the HDBSS-related content into the same file, such as
> arch/arm64/kvm/dirty_bit.c
>
> and arch/arm64/include/asm/kvm_dirty_bit.h.
>
>
> Tian
>
>
> >
> >
Awesome! Then I will work my HACDBS enablement that way as well.
Thanks!
Leo
^ permalink raw reply
* [PATCH v7 0/4] Support runtime configuration for per-VM's HGATP mode
From: fangyu.yu @ 2026-04-02 13:22 UTC (permalink / raw)
To: pbonzini, corbet, anup, atish.patra, pjw, palmer, aou, alex,
skhan
Cc: guoren, radim.krcmar, andrew.jones, linux-doc, kvm, kvm-riscv,
linux-riscv, linux-kernel, Fangyu Yu
From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Currently, RISC-V KVM hardcodes the G-stage page table format (HGATP mode)
to the maximum mode detected at boot time (e.g., SV57x4 if supported). but
often such a wide GPA is unnecessary, just as a host sometimes doesn't need
sv57.
This patch introduces per-VM configurability of the G-stage mode via a new
KVM capability: KVM_CAP_RISCV_SET_HGATP_MODE. User-space can now explicitly
request a specific HGATP mode (SV39x4, SV48x4, SV57x4 or SV32x4) during
VM creation.
---
Changes in v7 (Anup's suggestions):
- Keep the original HGATP mode probing logic.
- Link to v6:
https://lore.kernel.org/linux-riscv/20260330122601.22140-1-fangyu.yu@linux.alibaba.com/
---
Changes in v6 (Anup's suggestions):
- Reworked kvm_riscv_gstage_gpa_bits() and kvm_riscv_gstage_gpa_size() to
take "unsigned long pgd_levels" instead of "struct kvm_arch *".
- Moved kvm_riscv_gstage_mode() helper from kvm_host.h to kvm_gstage.h.
- Renamed kvm->arch.kvm_riscv_gstage_pgd_levels to kvm->arch.pgd_levels.
- Added pgd_levels to struct kvm_gstage to avoid repeated
gstage->kvm->arch pointer chasing.
- Link to v5:
https://lore.kernel.org/linux-riscv/20260204134507.33912-1-fangyu.yu@linux.alibaba.com/
---
Changes in v5:
- Use architectural HGATP.MODE encodings as the bit index for the supported-mode
bitmap and for the VM-mode selection UAPI; no new UAPI mode/bit defines are
introduced(per Radim).
- Allow KVM_CAP_RISCV_SET_HGATP_MODE on RV32 as well(per Drew).
- Link to v4:
https://lore.kernel.org/linux-riscv/20260202140716.34323-1-fangyu.yu@linux.alibaba.com/
---
Changes in v4:
- Extend kvm_riscv_gstage_mode_detect() to probe all HGATP.MODE values
supported by the host and record them in a bitmask.
- Treat unexpected pgd_levels in kvm_riscv_gstage_mode() as an internal error
(e.g. WARN_ON_ONCE())(per Radim).
- Move kvm_riscv_gstage_gpa_bits() and kvm_riscv_gstage_gpa_size() to header
as static inline helpers(per Radim).
- Drop gstage_mode_user_initialized and Remove the kvm_debug() message from
KVM_CAP_RISCV_SET_HGATP_MODE(per Radim).
- Link to v3:
https://lore.kernel.org/linux-riscv/20260125150450.27068-1-fangyu.yu@linux.alibaba.com/
---
Changes in v3:
- Reworked the patch formatting (per Drew).
- Dropped kvm->arch.kvm_riscv_gstage_mode and derive HGATP.MODE from
kvm_riscv_gstage_pgd_levels via a helper, avoiding redundant per-VM state(per Drew).
- Removed kvm_riscv_gstage_max_mode and keep only kvm_riscv_gstage_max_pgd_levels
for host capability detection(per Drew).
- Other initialization and return value issues(per Drew).
- Enforce that KVM_CAP_RISCV_SET_HGATP_MODE can only be enabled before any vCPUs
are created by rejecting the ioctl once kvm->created_vcpus is non-zero(per Radim).
- Add a memslot safety check and reject the capability unless
kvm_are_all_memslots_empty(kvm) is true, ensuring the G-stage format is not
changed after any memslots have been installed(per Radim).
- Link to v2:
https://lore.kernel.org/linux-riscv/20260105143232.76715-1-fangyu.yu@linux.alibaba.com/
Fangyu Yu (4):
RISC-V: KVM: Support runtime configuration for per-VM's HGATP mode
RISC-V: KVM: Cache gstage pgd_levels in struct kvm_gstage
RISC-V: KVM: Detect and expose supported HGATP G-stage modes
RISC-V: KVM: add KVM_CAP_RISCV_SET_HGATP_MODE
Documentation/virt/kvm/api.rst | 27 ++++++++++
arch/riscv/include/asm/kvm_gstage.h | 58 ++++++++++++++++++---
arch/riscv/include/asm/kvm_host.h | 1 +
arch/riscv/kvm/gstage.c | 78 ++++++++++++++++-------------
arch/riscv/kvm/main.c | 12 ++---
arch/riscv/kvm/mmu.c | 70 ++++++++------------------
arch/riscv/kvm/vm.c | 20 ++++++--
arch/riscv/kvm/vmid.c | 3 +-
include/uapi/linux/kvm.h | 1 +
9 files changed, 169 insertions(+), 101 deletions(-)
--
2.50.1
^ permalink raw reply
* [PATCH v7 2/4] RISC-V: KVM: Cache gstage pgd_levels in struct kvm_gstage
From: fangyu.yu @ 2026-04-02 13:23 UTC (permalink / raw)
To: pbonzini, corbet, anup, atish.patra, pjw, palmer, aou, alex,
skhan
Cc: guoren, radim.krcmar, andrew.jones, linux-doc, kvm, kvm-riscv,
linux-riscv, linux-kernel, Fangyu Yu
In-Reply-To: <20260402132303.6252-1-fangyu.yu@linux.alibaba.com>
From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Gstage page-table helpers frequently chase gstage->kvm->arch to
fetch pgd_levels. This adds noise and repeats the same dereference
chain in hot paths.
Add pgd_levels to struct kvm_gstage and initialize it from kvm->arch
when setting up a gstage instance. Introduce kvm_riscv_gstage_init()
to centralize initialization and switch gstage code to use
gstage->pgd_levels.
Suggested-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
---
arch/riscv/include/asm/kvm_gstage.h | 10 ++++++
arch/riscv/kvm/gstage.c | 10 +++---
arch/riscv/kvm/mmu.c | 50 ++++++-----------------------
3 files changed, 25 insertions(+), 45 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index 5aa58d1f692a..70d9d483365e 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -15,6 +15,7 @@ struct kvm_gstage {
#define KVM_GSTAGE_FLAGS_LOCAL BIT(0)
unsigned long vmid;
pgd_t *pgd;
+ unsigned long pgd_levels;
};
struct kvm_gstage_mapping {
@@ -92,4 +93,13 @@ static inline unsigned long kvm_riscv_gstage_mode(unsigned long pgd_levels)
}
}
+static inline void kvm_riscv_gstage_init(struct kvm_gstage *gstage, struct kvm *kvm)
+{
+ gstage->kvm = kvm;
+ gstage->flags = 0;
+ gstage->vmid = READ_ONCE(kvm->arch.vmid.vmid);
+ gstage->pgd = kvm->arch.pgd;
+ gstage->pgd_levels = kvm->arch.pgd_levels;
+}
+
#endif
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index 4beb9322fe76..7c4c34bc191b 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -26,7 +26,7 @@ static inline unsigned long gstage_pte_index(struct kvm_gstage *gstage,
unsigned long mask;
unsigned long shift = HGATP_PAGE_SHIFT + (kvm_riscv_gstage_index_bits * level);
- if (level == gstage->kvm->arch.pgd_levels - 1)
+ if (level == gstage->pgd_levels - 1)
mask = (PTRS_PER_PTE * (1UL << kvm_riscv_gstage_pgd_xbits)) - 1;
else
mask = PTRS_PER_PTE - 1;
@@ -45,7 +45,7 @@ static int gstage_page_size_to_level(struct kvm_gstage *gstage, unsigned long pa
u32 i;
unsigned long psz = 1UL << 12;
- for (i = 0; i < gstage->kvm->arch.pgd_levels; i++) {
+ for (i = 0; i < gstage->pgd_levels; i++) {
if (page_size == (psz << (i * kvm_riscv_gstage_index_bits))) {
*out_level = i;
return 0;
@@ -58,7 +58,7 @@ static int gstage_page_size_to_level(struct kvm_gstage *gstage, unsigned long pa
static int gstage_level_to_page_order(struct kvm_gstage *gstage, u32 level,
unsigned long *out_pgorder)
{
- if (gstage->kvm->arch.pgd_levels < level)
+ if (gstage->pgd_levels < level)
return -EINVAL;
*out_pgorder = 12 + (level * kvm_riscv_gstage_index_bits);
@@ -83,7 +83,7 @@ bool kvm_riscv_gstage_get_leaf(struct kvm_gstage *gstage, gpa_t addr,
pte_t **ptepp, u32 *ptep_level)
{
pte_t *ptep;
- u32 current_level = gstage->kvm->arch.pgd_levels - 1;
+ u32 current_level = gstage->pgd_levels - 1;
*ptep_level = current_level;
ptep = (pte_t *)gstage->pgd;
@@ -127,7 +127,7 @@ int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
struct kvm_mmu_memory_cache *pcache,
const struct kvm_gstage_mapping *map)
{
- u32 current_level = gstage->kvm->arch.pgd_levels - 1;
+ u32 current_level = gstage->pgd_levels - 1;
pte_t *next_ptep = (pte_t *)gstage->pgd;
pte_t *ptep = &next_ptep[gstage_pte_index(gstage, map->addr, current_level)];
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index fbcdd75cb9af..2d3def024270 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -24,10 +24,7 @@ static void mmu_wp_memory_region(struct kvm *kvm, int slot)
phys_addr_t end = (memslot->base_gfn + memslot->npages) << PAGE_SHIFT;
struct kvm_gstage gstage;
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
spin_lock(&kvm->mmu_lock);
kvm_riscv_gstage_wp_range(&gstage, start, end);
@@ -49,10 +46,7 @@ int kvm_riscv_mmu_ioremap(struct kvm *kvm, gpa_t gpa, phys_addr_t hpa,
struct kvm_gstage_mapping map;
struct kvm_gstage gstage;
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
end = (gpa + size + PAGE_SIZE - 1) & PAGE_MASK;
pfn = __phys_to_pfn(hpa);
@@ -89,10 +83,7 @@ void kvm_riscv_mmu_iounmap(struct kvm *kvm, gpa_t gpa, unsigned long size)
{
struct kvm_gstage gstage;
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
spin_lock(&kvm->mmu_lock);
kvm_riscv_gstage_unmap_range(&gstage, gpa, size, false);
@@ -109,10 +100,7 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm,
phys_addr_t end = (base_gfn + __fls(mask) + 1) << PAGE_SHIFT;
struct kvm_gstage gstage;
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
kvm_riscv_gstage_wp_range(&gstage, start, end);
}
@@ -141,10 +129,7 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm,
phys_addr_t size = slot->npages << PAGE_SHIFT;
struct kvm_gstage gstage;
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
spin_lock(&kvm->mmu_lock);
kvm_riscv_gstage_unmap_range(&gstage, gpa, size, false);
@@ -250,10 +235,7 @@ bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range)
if (!kvm->arch.pgd)
return false;
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
mmu_locked = spin_trylock(&kvm->mmu_lock);
kvm_riscv_gstage_unmap_range(&gstage, range->start << PAGE_SHIFT,
(range->end - range->start) << PAGE_SHIFT,
@@ -275,10 +257,7 @@ bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
WARN_ON(size != PAGE_SIZE && size != PMD_SIZE && size != PUD_SIZE);
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
if (!kvm_riscv_gstage_get_leaf(&gstage, range->start << PAGE_SHIFT,
&ptep, &ptep_level))
return false;
@@ -298,10 +277,7 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range)
WARN_ON(size != PAGE_SIZE && size != PMD_SIZE && size != PUD_SIZE);
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
if (!kvm_riscv_gstage_get_leaf(&gstage, range->start << PAGE_SHIFT,
&ptep, &ptep_level))
return false;
@@ -463,10 +439,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
struct kvm_gstage gstage;
struct page *page;
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
/* Setup initial state of output mapping */
memset(out_map, 0, sizeof(*out_map));
@@ -587,10 +560,7 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
spin_lock(&kvm->mmu_lock);
if (kvm->arch.pgd) {
- gstage.kvm = kvm;
- gstage.flags = 0;
- gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
- gstage.pgd = kvm->arch.pgd;
+ kvm_riscv_gstage_init(&gstage, kvm);
kvm_riscv_gstage_unmap_range(&gstage, 0UL,
kvm_riscv_gstage_gpa_size(kvm->arch.pgd_levels), false);
pgd = READ_ONCE(kvm->arch.pgd);
--
2.50.1
^ permalink raw reply related
* [PATCH v7 1/4] RISC-V: KVM: Support runtime configuration for per-VM's HGATP mode
From: fangyu.yu @ 2026-04-02 13:23 UTC (permalink / raw)
To: pbonzini, corbet, anup, atish.patra, pjw, palmer, aou, alex,
skhan
Cc: guoren, radim.krcmar, andrew.jones, linux-doc, kvm, kvm-riscv,
linux-riscv, linux-kernel, Fangyu Yu
In-Reply-To: <20260402132303.6252-1-fangyu.yu@linux.alibaba.com>
From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Introduces one per-VM architecture-specific fields to support runtime
configuration of the G-stage page table format:
- kvm->arch.pgd_levels: the corresponding number of page table levels
for the selected mode.
These fields replace the previous global variables
kvm_riscv_gstage_mode and kvm_riscv_gstage_pgd_levels, enabling different
virtual machines to independently select their G-stage page table format
instead of being forced to share the maximum mode detected by the kernel
at boot time.
Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Reviewed-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/kvm_gstage.h | 37 ++++++++++++----
arch/riscv/include/asm/kvm_host.h | 1 +
arch/riscv/kvm/gstage.c | 65 ++++++++++++++---------------
arch/riscv/kvm/main.c | 12 +++---
arch/riscv/kvm/mmu.c | 20 +++++----
arch/riscv/kvm/vm.c | 2 +-
arch/riscv/kvm/vmid.c | 3 +-
7 files changed, 83 insertions(+), 57 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index 595e2183173e..5aa58d1f692a 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -29,16 +29,22 @@ struct kvm_gstage_mapping {
#define kvm_riscv_gstage_index_bits 10
#endif
-extern unsigned long kvm_riscv_gstage_mode;
-extern unsigned long kvm_riscv_gstage_pgd_levels;
+extern unsigned long kvm_riscv_gstage_max_pgd_levels;
#define kvm_riscv_gstage_pgd_xbits 2
#define kvm_riscv_gstage_pgd_size (1UL << (HGATP_PAGE_SHIFT + kvm_riscv_gstage_pgd_xbits))
-#define kvm_riscv_gstage_gpa_bits (HGATP_PAGE_SHIFT + \
- (kvm_riscv_gstage_pgd_levels * \
- kvm_riscv_gstage_index_bits) + \
- kvm_riscv_gstage_pgd_xbits)
-#define kvm_riscv_gstage_gpa_size ((gpa_t)(1ULL << kvm_riscv_gstage_gpa_bits))
+
+static inline unsigned long kvm_riscv_gstage_gpa_bits(unsigned long pgd_levels)
+{
+ return (HGATP_PAGE_SHIFT +
+ pgd_levels * kvm_riscv_gstage_index_bits +
+ kvm_riscv_gstage_pgd_xbits);
+}
+
+static inline gpa_t kvm_riscv_gstage_gpa_size(unsigned long pgd_levels)
+{
+ return BIT_ULL(kvm_riscv_gstage_gpa_bits(pgd_levels));
+}
bool kvm_riscv_gstage_get_leaf(struct kvm_gstage *gstage, gpa_t addr,
pte_t **ptepp, u32 *ptep_level);
@@ -69,4 +75,21 @@ void kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end
void kvm_riscv_gstage_mode_detect(void);
+static inline unsigned long kvm_riscv_gstage_mode(unsigned long pgd_levels)
+{
+ switch (pgd_levels) {
+ case 2:
+ return HGATP_MODE_SV32X4;
+ case 3:
+ return HGATP_MODE_SV39X4;
+ case 4:
+ return HGATP_MODE_SV48X4;
+ case 5:
+ return HGATP_MODE_SV57X4;
+ default:
+ WARN_ON_ONCE(1);
+ return HGATP_MODE_OFF;
+ }
+}
+
#endif
diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h
index 24585304c02b..478f699e9dec 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -94,6 +94,7 @@ struct kvm_arch {
/* G-stage page table */
pgd_t *pgd;
phys_addr_t pgd_phys;
+ unsigned long pgd_levels;
/* Guest Timer */
struct kvm_guest_timer timer;
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index b67d60d722c2..4beb9322fe76 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -12,22 +12,21 @@
#include <asm/kvm_gstage.h>
#ifdef CONFIG_64BIT
-unsigned long kvm_riscv_gstage_mode __ro_after_init = HGATP_MODE_SV39X4;
-unsigned long kvm_riscv_gstage_pgd_levels __ro_after_init = 3;
+unsigned long kvm_riscv_gstage_max_pgd_levels __ro_after_init = 3;
#else
-unsigned long kvm_riscv_gstage_mode __ro_after_init = HGATP_MODE_SV32X4;
-unsigned long kvm_riscv_gstage_pgd_levels __ro_after_init = 2;
+unsigned long kvm_riscv_gstage_max_pgd_levels __ro_after_init = 2;
#endif
#define gstage_pte_leaf(__ptep) \
(pte_val(*(__ptep)) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC))
-static inline unsigned long gstage_pte_index(gpa_t addr, u32 level)
+static inline unsigned long gstage_pte_index(struct kvm_gstage *gstage,
+ gpa_t addr, u32 level)
{
unsigned long mask;
unsigned long shift = HGATP_PAGE_SHIFT + (kvm_riscv_gstage_index_bits * level);
- if (level == (kvm_riscv_gstage_pgd_levels - 1))
+ if (level == gstage->kvm->arch.pgd_levels - 1)
mask = (PTRS_PER_PTE * (1UL << kvm_riscv_gstage_pgd_xbits)) - 1;
else
mask = PTRS_PER_PTE - 1;
@@ -40,12 +39,13 @@ static inline unsigned long gstage_pte_page_vaddr(pte_t pte)
return (unsigned long)pfn_to_virt(__page_val_to_pfn(pte_val(pte)));
}
-static int gstage_page_size_to_level(unsigned long page_size, u32 *out_level)
+static int gstage_page_size_to_level(struct kvm_gstage *gstage, unsigned long page_size,
+ u32 *out_level)
{
u32 i;
unsigned long psz = 1UL << 12;
- for (i = 0; i < kvm_riscv_gstage_pgd_levels; i++) {
+ for (i = 0; i < gstage->kvm->arch.pgd_levels; i++) {
if (page_size == (psz << (i * kvm_riscv_gstage_index_bits))) {
*out_level = i;
return 0;
@@ -55,21 +55,23 @@ static int gstage_page_size_to_level(unsigned long page_size, u32 *out_level)
return -EINVAL;
}
-static int gstage_level_to_page_order(u32 level, unsigned long *out_pgorder)
+static int gstage_level_to_page_order(struct kvm_gstage *gstage, u32 level,
+ unsigned long *out_pgorder)
{
- if (kvm_riscv_gstage_pgd_levels < level)
+ if (gstage->kvm->arch.pgd_levels < level)
return -EINVAL;
*out_pgorder = 12 + (level * kvm_riscv_gstage_index_bits);
return 0;
}
-static int gstage_level_to_page_size(u32 level, unsigned long *out_pgsize)
+static int gstage_level_to_page_size(struct kvm_gstage *gstage, u32 level,
+ unsigned long *out_pgsize)
{
int rc;
unsigned long page_order = PAGE_SHIFT;
- rc = gstage_level_to_page_order(level, &page_order);
+ rc = gstage_level_to_page_order(gstage, level, &page_order);
if (rc)
return rc;
@@ -81,11 +83,11 @@ bool kvm_riscv_gstage_get_leaf(struct kvm_gstage *gstage, gpa_t addr,
pte_t **ptepp, u32 *ptep_level)
{
pte_t *ptep;
- u32 current_level = kvm_riscv_gstage_pgd_levels - 1;
+ u32 current_level = gstage->kvm->arch.pgd_levels - 1;
*ptep_level = current_level;
ptep = (pte_t *)gstage->pgd;
- ptep = &ptep[gstage_pte_index(addr, current_level)];
+ ptep = &ptep[gstage_pte_index(gstage, addr, current_level)];
while (ptep && pte_val(ptep_get(ptep))) {
if (gstage_pte_leaf(ptep)) {
*ptep_level = current_level;
@@ -97,7 +99,7 @@ bool kvm_riscv_gstage_get_leaf(struct kvm_gstage *gstage, gpa_t addr,
current_level--;
*ptep_level = current_level;
ptep = (pte_t *)gstage_pte_page_vaddr(ptep_get(ptep));
- ptep = &ptep[gstage_pte_index(addr, current_level)];
+ ptep = &ptep[gstage_pte_index(gstage, addr, current_level)];
} else {
ptep = NULL;
}
@@ -110,7 +112,7 @@ static void gstage_tlb_flush(struct kvm_gstage *gstage, u32 level, gpa_t addr)
{
unsigned long order = PAGE_SHIFT;
- if (gstage_level_to_page_order(level, &order))
+ if (gstage_level_to_page_order(gstage, level, &order))
return;
addr &= ~(BIT(order) - 1);
@@ -125,9 +127,9 @@ int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
struct kvm_mmu_memory_cache *pcache,
const struct kvm_gstage_mapping *map)
{
- u32 current_level = kvm_riscv_gstage_pgd_levels - 1;
+ u32 current_level = gstage->kvm->arch.pgd_levels - 1;
pte_t *next_ptep = (pte_t *)gstage->pgd;
- pte_t *ptep = &next_ptep[gstage_pte_index(map->addr, current_level)];
+ pte_t *ptep = &next_ptep[gstage_pte_index(gstage, map->addr, current_level)];
if (current_level < map->level)
return -EINVAL;
@@ -151,7 +153,7 @@ int kvm_riscv_gstage_set_pte(struct kvm_gstage *gstage,
}
current_level--;
- ptep = &next_ptep[gstage_pte_index(map->addr, current_level)];
+ ptep = &next_ptep[gstage_pte_index(gstage, map->addr, current_level)];
}
if (pte_val(*ptep) != pte_val(map->pte)) {
@@ -175,7 +177,7 @@ int kvm_riscv_gstage_map_page(struct kvm_gstage *gstage,
out_map->addr = gpa;
out_map->level = 0;
- ret = gstage_page_size_to_level(page_size, &out_map->level);
+ ret = gstage_page_size_to_level(gstage, page_size, &out_map->level);
if (ret)
return ret;
@@ -217,7 +219,7 @@ void kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
u32 next_ptep_level;
unsigned long next_page_size, page_size;
- ret = gstage_level_to_page_size(ptep_level, &page_size);
+ ret = gstage_level_to_page_size(gstage, ptep_level, &page_size);
if (ret)
return;
@@ -229,7 +231,7 @@ void kvm_riscv_gstage_op_pte(struct kvm_gstage *gstage, gpa_t addr,
if (ptep_level && !gstage_pte_leaf(ptep)) {
next_ptep = (pte_t *)gstage_pte_page_vaddr(ptep_get(ptep));
next_ptep_level = ptep_level - 1;
- ret = gstage_level_to_page_size(next_ptep_level, &next_page_size);
+ ret = gstage_level_to_page_size(gstage, next_ptep_level, &next_page_size);
if (ret)
return;
@@ -263,7 +265,7 @@ void kvm_riscv_gstage_unmap_range(struct kvm_gstage *gstage,
while (addr < end) {
found_leaf = kvm_riscv_gstage_get_leaf(gstage, addr, &ptep, &ptep_level);
- ret = gstage_level_to_page_size(ptep_level, &page_size);
+ ret = gstage_level_to_page_size(gstage, ptep_level, &page_size);
if (ret)
break;
@@ -297,7 +299,7 @@ void kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end
while (addr < end) {
found_leaf = kvm_riscv_gstage_get_leaf(gstage, addr, &ptep, &ptep_level);
- ret = gstage_level_to_page_size(ptep_level, &page_size);
+ ret = gstage_level_to_page_size(gstage, ptep_level, &page_size);
if (ret)
break;
@@ -319,39 +321,34 @@ void __init kvm_riscv_gstage_mode_detect(void)
/* Try Sv57x4 G-stage mode */
csr_write(CSR_HGATP, HGATP_MODE_SV57X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV57X4) {
- kvm_riscv_gstage_mode = HGATP_MODE_SV57X4;
- kvm_riscv_gstage_pgd_levels = 5;
+ kvm_riscv_gstage_max_pgd_levels = 5;
goto done;
}
/* Try Sv48x4 G-stage mode */
csr_write(CSR_HGATP, HGATP_MODE_SV48X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV48X4) {
- kvm_riscv_gstage_mode = HGATP_MODE_SV48X4;
- kvm_riscv_gstage_pgd_levels = 4;
+ kvm_riscv_gstage_max_pgd_levels = 4;
goto done;
}
/* Try Sv39x4 G-stage mode */
csr_write(CSR_HGATP, HGATP_MODE_SV39X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV39X4) {
- kvm_riscv_gstage_mode = HGATP_MODE_SV39X4;
- kvm_riscv_gstage_pgd_levels = 3;
+ kvm_riscv_gstage_max_pgd_levels = 3;
goto done;
}
#else /* CONFIG_32BIT */
/* Try Sv32x4 G-stage mode */
csr_write(CSR_HGATP, HGATP_MODE_SV32X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV32X4) {
- kvm_riscv_gstage_mode = HGATP_MODE_SV32X4;
- kvm_riscv_gstage_pgd_levels = 2;
+ kvm_riscv_gstage_max_pgd_levels = 2;
goto done;
}
#endif
/* KVM depends on !HGATP_MODE_OFF */
- kvm_riscv_gstage_mode = HGATP_MODE_OFF;
- kvm_riscv_gstage_pgd_levels = 0;
+ kvm_riscv_gstage_max_pgd_levels = 0;
done:
csr_write(CSR_HGATP, 0);
diff --git a/arch/riscv/kvm/main.c b/arch/riscv/kvm/main.c
index 0f3fe3986fc0..90ee0a032b9a 100644
--- a/arch/riscv/kvm/main.c
+++ b/arch/riscv/kvm/main.c
@@ -105,17 +105,17 @@ static int __init riscv_kvm_init(void)
return rc;
kvm_riscv_gstage_mode_detect();
- switch (kvm_riscv_gstage_mode) {
- case HGATP_MODE_SV32X4:
+ switch (kvm_riscv_gstage_max_pgd_levels) {
+ case 2:
str = "Sv32x4";
break;
- case HGATP_MODE_SV39X4:
+ case 3:
str = "Sv39x4";
break;
- case HGATP_MODE_SV48X4:
+ case 4:
str = "Sv48x4";
break;
- case HGATP_MODE_SV57X4:
+ case 5:
str = "Sv57x4";
break;
default:
@@ -164,7 +164,7 @@ static int __init riscv_kvm_init(void)
(rc) ? slist : "no features");
}
- kvm_info("using %s G-stage page table format\n", str);
+ kvm_info("highest G-stage page table mode is %s\n", str);
kvm_info("VMID %ld bits available\n", kvm_riscv_gstage_vmid_bits());
diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c
index 088d33ba90ed..fbcdd75cb9af 100644
--- a/arch/riscv/kvm/mmu.c
+++ b/arch/riscv/kvm/mmu.c
@@ -67,7 +67,7 @@ int kvm_riscv_mmu_ioremap(struct kvm *kvm, gpa_t gpa, phys_addr_t hpa,
if (!writable)
map.pte = pte_wrprotect(map.pte);
- ret = kvm_mmu_topup_memory_cache(&pcache, kvm_riscv_gstage_pgd_levels);
+ ret = kvm_mmu_topup_memory_cache(&pcache, kvm->arch.pgd_levels);
if (ret)
goto out;
@@ -186,7 +186,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
* space addressable by the KVM guest GPA space.
*/
if ((new->base_gfn + new->npages) >=
- (kvm_riscv_gstage_gpa_size >> PAGE_SHIFT))
+ kvm_riscv_gstage_gpa_size(kvm->arch.pgd_levels) >> PAGE_SHIFT)
return -EFAULT;
hva = new->userspace_addr;
@@ -472,7 +472,7 @@ int kvm_riscv_mmu_map(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot,
memset(out_map, 0, sizeof(*out_map));
/* We need minimum second+third level pages */
- ret = kvm_mmu_topup_memory_cache(pcache, kvm_riscv_gstage_pgd_levels);
+ ret = kvm_mmu_topup_memory_cache(pcache, kvm->arch.pgd_levels);
if (ret) {
kvm_err("Failed to topup G-stage cache\n");
return ret;
@@ -575,6 +575,7 @@ int kvm_riscv_mmu_alloc_pgd(struct kvm *kvm)
return -ENOMEM;
kvm->arch.pgd = page_to_virt(pgd_page);
kvm->arch.pgd_phys = page_to_phys(pgd_page);
+ kvm->arch.pgd_levels = kvm_riscv_gstage_max_pgd_levels;
return 0;
}
@@ -590,10 +591,12 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
gstage.flags = 0;
gstage.vmid = READ_ONCE(kvm->arch.vmid.vmid);
gstage.pgd = kvm->arch.pgd;
- kvm_riscv_gstage_unmap_range(&gstage, 0UL, kvm_riscv_gstage_gpa_size, false);
+ kvm_riscv_gstage_unmap_range(&gstage, 0UL,
+ kvm_riscv_gstage_gpa_size(kvm->arch.pgd_levels), false);
pgd = READ_ONCE(kvm->arch.pgd);
kvm->arch.pgd = NULL;
kvm->arch.pgd_phys = 0;
+ kvm->arch.pgd_levels = 0;
}
spin_unlock(&kvm->mmu_lock);
@@ -603,11 +606,12 @@ void kvm_riscv_mmu_free_pgd(struct kvm *kvm)
void kvm_riscv_mmu_update_hgatp(struct kvm_vcpu *vcpu)
{
- unsigned long hgatp = kvm_riscv_gstage_mode << HGATP_MODE_SHIFT;
- struct kvm_arch *k = &vcpu->kvm->arch;
+ struct kvm_arch *ka = &vcpu->kvm->arch;
+ unsigned long hgatp = kvm_riscv_gstage_mode(ka->pgd_levels)
+ << HGATP_MODE_SHIFT;
- hgatp |= (READ_ONCE(k->vmid.vmid) << HGATP_VMID_SHIFT) & HGATP_VMID;
- hgatp |= (k->pgd_phys >> PAGE_SHIFT) & HGATP_PPN;
+ hgatp |= (READ_ONCE(ka->vmid.vmid) << HGATP_VMID_SHIFT) & HGATP_VMID;
+ hgatp |= (ka->pgd_phys >> PAGE_SHIFT) & HGATP_PPN;
ncsr_write(CSR_HGATP, hgatp);
diff --git a/arch/riscv/kvm/vm.c b/arch/riscv/kvm/vm.c
index 13c63ae1a78b..4d82a886102c 100644
--- a/arch/riscv/kvm/vm.c
+++ b/arch/riscv/kvm/vm.c
@@ -199,7 +199,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
r = KVM_USER_MEM_SLOTS;
break;
case KVM_CAP_VM_GPA_BITS:
- r = kvm_riscv_gstage_gpa_bits;
+ r = kvm_riscv_gstage_gpa_bits(kvm->arch.pgd_levels);
break;
default:
r = 0;
diff --git a/arch/riscv/kvm/vmid.c b/arch/riscv/kvm/vmid.c
index cf34d448289d..c15bdb1dd8be 100644
--- a/arch/riscv/kvm/vmid.c
+++ b/arch/riscv/kvm/vmid.c
@@ -26,7 +26,8 @@ static DEFINE_SPINLOCK(vmid_lock);
void __init kvm_riscv_gstage_vmid_detect(void)
{
/* Figure-out number of VMID bits in HW */
- csr_write(CSR_HGATP, (kvm_riscv_gstage_mode << HGATP_MODE_SHIFT) | HGATP_VMID);
+ csr_write(CSR_HGATP, (kvm_riscv_gstage_mode(kvm_riscv_gstage_max_pgd_levels) <<
+ HGATP_MODE_SHIFT) | HGATP_VMID);
vmid_bits = csr_read(CSR_HGATP);
vmid_bits = (vmid_bits & HGATP_VMID) >> HGATP_VMID_SHIFT;
vmid_bits = fls_long(vmid_bits);
--
2.50.1
^ permalink raw reply related
* [PATCH v7 3/4] RISC-V: KVM: Detect and expose supported HGATP G-stage modes
From: fangyu.yu @ 2026-04-02 13:23 UTC (permalink / raw)
To: pbonzini, corbet, anup, atish.patra, pjw, palmer, aou, alex,
skhan
Cc: guoren, radim.krcmar, andrew.jones, linux-doc, kvm, kvm-riscv,
linux-riscv, linux-kernel, Fangyu Yu
In-Reply-To: <20260402132303.6252-1-fangyu.yu@linux.alibaba.com>
From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Extend kvm_riscv_gstage_mode_detect() to record HGATP.MODE values in a
bitmask. Keep tracking the maximum supported G-stage page table level
for existing internal users.
Also provide lightweight helpers to retrieve the supported-mode bitmask
and validate a requested HGATP.MODE against it.
Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Guo Ren <guoren@kernel.org>
---
arch/riscv/include/asm/kvm_gstage.h | 11 +++++++++++
arch/riscv/kvm/gstage.c | 15 ++++++++++++---
2 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
index 70d9d483365e..bbf8f45c6563 100644
--- a/arch/riscv/include/asm/kvm_gstage.h
+++ b/arch/riscv/include/asm/kvm_gstage.h
@@ -31,6 +31,7 @@ struct kvm_gstage_mapping {
#endif
extern unsigned long kvm_riscv_gstage_max_pgd_levels;
+extern u32 kvm_riscv_gstage_supported_mode_mask;
#define kvm_riscv_gstage_pgd_xbits 2
#define kvm_riscv_gstage_pgd_size (1UL << (HGATP_PAGE_SHIFT + kvm_riscv_gstage_pgd_xbits))
@@ -102,4 +103,14 @@ static inline void kvm_riscv_gstage_init(struct kvm_gstage *gstage, struct kvm *
gstage->pgd_levels = kvm->arch.pgd_levels;
}
+static inline u32 kvm_riscv_get_hgatp_mode_mask(void)
+{
+ return kvm_riscv_gstage_supported_mode_mask;
+}
+
+static inline bool kvm_riscv_hgatp_mode_is_valid(unsigned long mode)
+{
+ return kvm_riscv_gstage_supported_mode_mask & BIT(mode);
+}
+
#endif
diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
index 7c4c34bc191b..9204e6427d2d 100644
--- a/arch/riscv/kvm/gstage.c
+++ b/arch/riscv/kvm/gstage.c
@@ -16,6 +16,8 @@ unsigned long kvm_riscv_gstage_max_pgd_levels __ro_after_init = 3;
#else
unsigned long kvm_riscv_gstage_max_pgd_levels __ro_after_init = 2;
#endif
+/* Bitmask of supported HGATP.MODE encodings (BIT(HGATP_MODE_*)). */
+u32 kvm_riscv_gstage_supported_mode_mask __ro_after_init;
#define gstage_pte_leaf(__ptep) \
(pte_val(*(__ptep)) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC))
@@ -317,11 +319,17 @@ void kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end
void __init kvm_riscv_gstage_mode_detect(void)
{
+ kvm_riscv_gstage_supported_mode_mask = 0;
+ kvm_riscv_gstage_max_pgd_levels = 0;
+
#ifdef CONFIG_64BIT
/* Try Sv57x4 G-stage mode */
csr_write(CSR_HGATP, HGATP_MODE_SV57X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV57X4) {
kvm_riscv_gstage_max_pgd_levels = 5;
+ kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV57X4) |
+ BIT(HGATP_MODE_SV48X4) |
+ BIT(HGATP_MODE_SV39X4);
goto done;
}
@@ -329,6 +337,8 @@ void __init kvm_riscv_gstage_mode_detect(void)
csr_write(CSR_HGATP, HGATP_MODE_SV48X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV48X4) {
kvm_riscv_gstage_max_pgd_levels = 4;
+ kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV48X4) |
+ BIT(HGATP_MODE_SV39X4);
goto done;
}
@@ -336,6 +346,7 @@ void __init kvm_riscv_gstage_mode_detect(void)
csr_write(CSR_HGATP, HGATP_MODE_SV39X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV39X4) {
kvm_riscv_gstage_max_pgd_levels = 3;
+ kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV39X4);
goto done;
}
#else /* CONFIG_32BIT */
@@ -343,13 +354,11 @@ void __init kvm_riscv_gstage_mode_detect(void)
csr_write(CSR_HGATP, HGATP_MODE_SV32X4 << HGATP_MODE_SHIFT);
if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV32X4) {
kvm_riscv_gstage_max_pgd_levels = 2;
+ kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV32X4);
goto done;
}
#endif
- /* KVM depends on !HGATP_MODE_OFF */
- kvm_riscv_gstage_max_pgd_levels = 0;
-
done:
csr_write(CSR_HGATP, 0);
kvm_riscv_local_hfence_gvma_all();
--
2.50.1
^ permalink raw reply related
* [PATCH v7 4/4] RISC-V: KVM: add KVM_CAP_RISCV_SET_HGATP_MODE
From: fangyu.yu @ 2026-04-02 13:23 UTC (permalink / raw)
To: pbonzini, corbet, anup, atish.patra, pjw, palmer, aou, alex,
skhan
Cc: guoren, radim.krcmar, andrew.jones, linux-doc, kvm, kvm-riscv,
linux-riscv, linux-kernel, Fangyu Yu
In-Reply-To: <20260402132303.6252-1-fangyu.yu@linux.alibaba.com>
From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Add a VM capability that allows userspace to select the G-stage page table
format by setting HGATP.MODE on a per-VM basis.
Userspace enables the capability via KVM_ENABLE_CAP, passing the requested
HGATP.MODE in args[0]. The request is rejected with -EINVAL if the mode is
not supported by the host, and with -EBUSY if the VM has already been
committed (e.g. vCPUs have been created or any memslot is populated).
KVM_CHECK_EXTENSION(KVM_CAP_RISCV_SET_HGATP_MODE) returns a bitmask of the
HGATP.MODE formats supported by the host.
Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Guo Ren <guoren@kernel.org>
---
Documentation/virt/kvm/api.rst | 27 +++++++++++++++++++++++++++
arch/riscv/kvm/vm.c | 18 ++++++++++++++++--
include/uapi/linux/kvm.h | 1 +
3 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 032516783e96..9d7f6958fa81 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -8902,6 +8902,33 @@ helpful if user space wants to emulate instructions which are not
This capability can be enabled dynamically even if VCPUs were already
created and are running.
+7.47 KVM_CAP_RISCV_SET_HGATP_MODE
+---------------------------------
+
+:Architectures: riscv
+:Type: VM
+:Parameters: args[0] contains the requested HGATP mode
+:Returns:
+ - 0 on success.
+ - -EINVAL if args[0] is outside the range of HGATP modes supported by the
+ hardware.
+ - -EBUSY if vCPUs have already been created for the VM, if the VM has any
+ non-empty memslots.
+
+This capability allows userspace to explicitly select the HGATP mode for
+the VM. The selected mode must be supported by both KVM and hardware. This
+capability must be enabled before creating any vCPUs or memslots.
+
+If this capability is not enabled, KVM will select the default HGATP mode
+automatically. The default is the highest HGATP.MODE value supported by
+hardware.
+
+``KVM_CHECK_EXTENSION(KVM_CAP_RISCV_SET_HGATP_MODE)`` returns a bitmask of
+HGATP.MODE values supported by the host. A return value of 0 indicates that
+the capability is not supported. Supported-mode bitmask use HGATP.MODE
+encodings as defined by the RISC-V privileged specification, such as Sv39x4
+corresponds to HGATP.MODE=8, so userspace should test bitmask & BIT(8).
+
8. Other capabilities.
======================
diff --git a/arch/riscv/kvm/vm.c b/arch/riscv/kvm/vm.c
index 4d82a886102c..5e82a3ad3ad0 100644
--- a/arch/riscv/kvm/vm.c
+++ b/arch/riscv/kvm/vm.c
@@ -201,6 +201,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_VM_GPA_BITS:
r = kvm_riscv_gstage_gpa_bits(kvm->arch.pgd_levels);
break;
+ case KVM_CAP_RISCV_SET_HGATP_MODE:
+ r = kvm_riscv_get_hgatp_mode_mask();
+ break;
default:
r = 0;
break;
@@ -211,12 +214,23 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
{
+ if (cap->flags)
+ return -EINVAL;
+
switch (cap->cap) {
case KVM_CAP_RISCV_MP_STATE_RESET:
- if (cap->flags)
- return -EINVAL;
kvm->arch.mp_state_reset = true;
return 0;
+ case KVM_CAP_RISCV_SET_HGATP_MODE:
+ if (!kvm_riscv_hgatp_mode_is_valid(cap->args[0]))
+ return -EINVAL;
+
+ if (kvm->created_vcpus || !kvm_are_all_memslots_empty(kvm))
+ return -EBUSY;
+#ifdef CONFIG_64BIT
+ kvm->arch.pgd_levels = 3 + cap->args[0] - HGATP_MODE_SV39X4;
+#endif
+ return 0;
default:
return -EINVAL;
}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 80364d4dbebb..a74a80fd4046 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -989,6 +989,7 @@ struct kvm_enable_cap {
#define KVM_CAP_ARM_SEA_TO_USER 245
#define KVM_CAP_S390_USER_OPEREXEC 246
#define KVM_CAP_S390_KEYOP 247
+#define KVM_CAP_RISCV_SET_HGATP_MODE 248
struct kvm_irq_routing_irqchip {
__u32 irqchip;
--
2.50.1
^ permalink raw reply related
* Re: [PATCH v12 00/15] arm64/riscv: Add support for crashkernel CMA reservation
From: Borislav Petkov @ 2026-04-02 13:36 UTC (permalink / raw)
To: Jinjie Ruan
Cc: corbet, skhan, catalin.marinas, will, chenhuacai, kernel, maddy,
mpe, npiggin, chleroy, pjw, palmer, aou, alex, tglx, mingo,
dave.hansen, hpa, robh, saravanak, akpm, bhe, vgoyal, dyoung,
rdunlap, peterz, pawan.kumar.gupta, feng.tang, dapeng1.mi, kees,
elver, paulmck, lirongqing, rppt, leitao, ardb, jbohac, cfsworks,
tangyouling, sourabhjain, ritesh.list, hbathini, eajames, guoren,
songshuaishuai, kevin.brodsky, vishal.moola, junhui.liu, coxu,
fuqiang.wang, liaoyuanhong, takahiro.akashi, james.morse,
lizhengyu3, x86, linux-doc, linux-kernel, linux-arm-kernel,
loongarch, linuxppc-dev, linux-riscv, devicetree, kexec
In-Reply-To: <c6eb858c-5275-6fdc-b6e9-bd6d4b169e05@huawei.com>
On Thu, Apr 02, 2026 at 07:47:53PM +0800, Jinjie Ruan wrote:
> Thank you for the reminder and for your patience. I apologize for the
> frequent updates; I am becoming more familiar with the community's
> workflow.
Yap, and you can use that time while waiting to learn about it:
Documentation/process/
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply
* Re: [PATCH v7 3/4] RISC-V: KVM: Detect and expose supported HGATP G-stage modes
From: Anup Patel @ 2026-04-02 14:40 UTC (permalink / raw)
To: fangyu.yu
Cc: pbonzini, corbet, atish.patra, pjw, palmer, aou, alex, skhan,
guoren, radim.krcmar, andrew.jones, linux-doc, kvm, kvm-riscv,
linux-riscv, linux-kernel
In-Reply-To: <20260402132303.6252-4-fangyu.yu@linux.alibaba.com>
On Thu, Apr 2, 2026 at 6:53 PM <fangyu.yu@linux.alibaba.com> wrote:
>
> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>
> Extend kvm_riscv_gstage_mode_detect() to record HGATP.MODE values in a
> bitmask. Keep tracking the maximum supported G-stage page table level
> for existing internal users.
>
> Also provide lightweight helpers to retrieve the supported-mode bitmask
> and validate a requested HGATP.MODE against it.
>
> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
> Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
> Reviewed-by: Guo Ren <guoren@kernel.org>
LGTM.
Reviewed-by: Anup Patel <anup@brainfault.org>
Thanks,
Anup
> ---
> arch/riscv/include/asm/kvm_gstage.h | 11 +++++++++++
> arch/riscv/kvm/gstage.c | 15 ++++++++++++---
> 2 files changed, 23 insertions(+), 3 deletions(-)
>
> diff --git a/arch/riscv/include/asm/kvm_gstage.h b/arch/riscv/include/asm/kvm_gstage.h
> index 70d9d483365e..bbf8f45c6563 100644
> --- a/arch/riscv/include/asm/kvm_gstage.h
> +++ b/arch/riscv/include/asm/kvm_gstage.h
> @@ -31,6 +31,7 @@ struct kvm_gstage_mapping {
> #endif
>
> extern unsigned long kvm_riscv_gstage_max_pgd_levels;
> +extern u32 kvm_riscv_gstage_supported_mode_mask;
>
> #define kvm_riscv_gstage_pgd_xbits 2
> #define kvm_riscv_gstage_pgd_size (1UL << (HGATP_PAGE_SHIFT + kvm_riscv_gstage_pgd_xbits))
> @@ -102,4 +103,14 @@ static inline void kvm_riscv_gstage_init(struct kvm_gstage *gstage, struct kvm *
> gstage->pgd_levels = kvm->arch.pgd_levels;
> }
>
> +static inline u32 kvm_riscv_get_hgatp_mode_mask(void)
> +{
> + return kvm_riscv_gstage_supported_mode_mask;
> +}
> +
> +static inline bool kvm_riscv_hgatp_mode_is_valid(unsigned long mode)
> +{
> + return kvm_riscv_gstage_supported_mode_mask & BIT(mode);
> +}
> +
> #endif
> diff --git a/arch/riscv/kvm/gstage.c b/arch/riscv/kvm/gstage.c
> index 7c4c34bc191b..9204e6427d2d 100644
> --- a/arch/riscv/kvm/gstage.c
> +++ b/arch/riscv/kvm/gstage.c
> @@ -16,6 +16,8 @@ unsigned long kvm_riscv_gstage_max_pgd_levels __ro_after_init = 3;
> #else
> unsigned long kvm_riscv_gstage_max_pgd_levels __ro_after_init = 2;
> #endif
> +/* Bitmask of supported HGATP.MODE encodings (BIT(HGATP_MODE_*)). */
> +u32 kvm_riscv_gstage_supported_mode_mask __ro_after_init;
>
> #define gstage_pte_leaf(__ptep) \
> (pte_val(*(__ptep)) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC))
> @@ -317,11 +319,17 @@ void kvm_riscv_gstage_wp_range(struct kvm_gstage *gstage, gpa_t start, gpa_t end
>
> void __init kvm_riscv_gstage_mode_detect(void)
> {
> + kvm_riscv_gstage_supported_mode_mask = 0;
> + kvm_riscv_gstage_max_pgd_levels = 0;
> +
> #ifdef CONFIG_64BIT
> /* Try Sv57x4 G-stage mode */
> csr_write(CSR_HGATP, HGATP_MODE_SV57X4 << HGATP_MODE_SHIFT);
> if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV57X4) {
> kvm_riscv_gstage_max_pgd_levels = 5;
> + kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV57X4) |
> + BIT(HGATP_MODE_SV48X4) |
> + BIT(HGATP_MODE_SV39X4);
> goto done;
> }
>
> @@ -329,6 +337,8 @@ void __init kvm_riscv_gstage_mode_detect(void)
> csr_write(CSR_HGATP, HGATP_MODE_SV48X4 << HGATP_MODE_SHIFT);
> if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV48X4) {
> kvm_riscv_gstage_max_pgd_levels = 4;
> + kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV48X4) |
> + BIT(HGATP_MODE_SV39X4);
> goto done;
> }
>
> @@ -336,6 +346,7 @@ void __init kvm_riscv_gstage_mode_detect(void)
> csr_write(CSR_HGATP, HGATP_MODE_SV39X4 << HGATP_MODE_SHIFT);
> if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV39X4) {
> kvm_riscv_gstage_max_pgd_levels = 3;
> + kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV39X4);
> goto done;
> }
> #else /* CONFIG_32BIT */
> @@ -343,13 +354,11 @@ void __init kvm_riscv_gstage_mode_detect(void)
> csr_write(CSR_HGATP, HGATP_MODE_SV32X4 << HGATP_MODE_SHIFT);
> if ((csr_read(CSR_HGATP) >> HGATP_MODE_SHIFT) == HGATP_MODE_SV32X4) {
> kvm_riscv_gstage_max_pgd_levels = 2;
> + kvm_riscv_gstage_supported_mode_mask |= BIT(HGATP_MODE_SV32X4);
> goto done;
> }
> #endif
>
> - /* KVM depends on !HGATP_MODE_OFF */
> - kvm_riscv_gstage_max_pgd_levels = 0;
> -
> done:
> csr_write(CSR_HGATP, 0);
> kvm_riscv_local_hfence_gvma_all();
> --
> 2.50.1
>
^ permalink raw reply
* Re: [PATCH v7 4/4] RISC-V: KVM: add KVM_CAP_RISCV_SET_HGATP_MODE
From: Anup Patel @ 2026-04-02 14:50 UTC (permalink / raw)
To: fangyu.yu
Cc: pbonzini, corbet, atish.patra, pjw, palmer, aou, alex, skhan,
guoren, radim.krcmar, andrew.jones, linux-doc, kvm, kvm-riscv,
linux-riscv, linux-kernel
In-Reply-To: <20260402132303.6252-5-fangyu.yu@linux.alibaba.com>
On Thu, Apr 2, 2026 at 6:53 PM <fangyu.yu@linux.alibaba.com> wrote:
>
> From: Fangyu Yu <fangyu.yu@linux.alibaba.com>
>
> Add a VM capability that allows userspace to select the G-stage page table
> format by setting HGATP.MODE on a per-VM basis.
>
> Userspace enables the capability via KVM_ENABLE_CAP, passing the requested
> HGATP.MODE in args[0]. The request is rejected with -EINVAL if the mode is
> not supported by the host, and with -EBUSY if the VM has already been
> committed (e.g. vCPUs have been created or any memslot is populated).
>
> KVM_CHECK_EXTENSION(KVM_CAP_RISCV_SET_HGATP_MODE) returns a bitmask of the
> HGATP.MODE formats supported by the host.
>
> Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
> Reviewed-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
> Reviewed-by: Guo Ren <guoren@kernel.org>
> ---
> Documentation/virt/kvm/api.rst | 27 +++++++++++++++++++++++++++
> arch/riscv/kvm/vm.c | 18 ++++++++++++++++--
> include/uapi/linux/kvm.h | 1 +
> 3 files changed, 44 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index 032516783e96..9d7f6958fa81 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -8902,6 +8902,33 @@ helpful if user space wants to emulate instructions which are not
> This capability can be enabled dynamically even if VCPUs were already
> created and are running.
>
> +7.47 KVM_CAP_RISCV_SET_HGATP_MODE
> +---------------------------------
> +
> +:Architectures: riscv
> +:Type: VM
> +:Parameters: args[0] contains the requested HGATP mode
> +:Returns:
> + - 0 on success.
> + - -EINVAL if args[0] is outside the range of HGATP modes supported by the
> + hardware.
> + - -EBUSY if vCPUs have already been created for the VM, if the VM has any
> + non-empty memslots.
> +
> +This capability allows userspace to explicitly select the HGATP mode for
> +the VM. The selected mode must be supported by both KVM and hardware. This
> +capability must be enabled before creating any vCPUs or memslots.
> +
> +If this capability is not enabled, KVM will select the default HGATP mode
> +automatically. The default is the highest HGATP.MODE value supported by
> +hardware.
> +
> +``KVM_CHECK_EXTENSION(KVM_CAP_RISCV_SET_HGATP_MODE)`` returns a bitmask of
> +HGATP.MODE values supported by the host. A return value of 0 indicates that
> +the capability is not supported. Supported-mode bitmask use HGATP.MODE
> +encodings as defined by the RISC-V privileged specification, such as Sv39x4
> +corresponds to HGATP.MODE=8, so userspace should test bitmask & BIT(8).
> +
> 8. Other capabilities.
> ======================
>
> diff --git a/arch/riscv/kvm/vm.c b/arch/riscv/kvm/vm.c
> index 4d82a886102c..5e82a3ad3ad0 100644
> --- a/arch/riscv/kvm/vm.c
> +++ b/arch/riscv/kvm/vm.c
> @@ -201,6 +201,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
> case KVM_CAP_VM_GPA_BITS:
> r = kvm_riscv_gstage_gpa_bits(kvm->arch.pgd_levels);
> break;
> + case KVM_CAP_RISCV_SET_HGATP_MODE:
> + r = kvm_riscv_get_hgatp_mode_mask();
> + break;
Introducing a new RISC-V capability looks a bit complex.
Instead of KVM_CAP_RISCV_SET_HGATP_MODE, we can
simply re-use KVM_CAP_VM_GPA_BITS.
The kvm_vm_ioctl_check_extension() for KVM_CAP_VM_GPA_BITS
return number of GPA bits which in-directly implies the underlying
hgatp.MODE. As we know, if it return 59 bits GPA then it means
Sv57x4 is the selected hgatp.MODE and Sv48x4 and Sv39x4 modes
are also supported as-per RISC-V privileged specification.
The kvm_vm_ioctl_enable_cap() for KVM_CAP_VM_GPA_BITS
will take the desired number of GPA bits and downsize the selected
hgatp.MODE. For example, if user-space ask GPA bits <= 50 and
GPA bits > 41 then we select Sv48x4. If user-space ask GPA
bits <= 41 then we select Sv39x4. If user-space ask GPA bits <= 59
and GPA bits > 50 then we select Sv57x4.
> default:
> r = 0;
> break;
> @@ -211,12 +214,23 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
>
> int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
> {
> + if (cap->flags)
> + return -EINVAL;
> +
> switch (cap->cap) {
> case KVM_CAP_RISCV_MP_STATE_RESET:
> - if (cap->flags)
> - return -EINVAL;
> kvm->arch.mp_state_reset = true;
> return 0;
> + case KVM_CAP_RISCV_SET_HGATP_MODE:
> + if (!kvm_riscv_hgatp_mode_is_valid(cap->args[0]))
> + return -EINVAL;
> +
> + if (kvm->created_vcpus || !kvm_are_all_memslots_empty(kvm))
> + return -EBUSY;
> +#ifdef CONFIG_64BIT
> + kvm->arch.pgd_levels = 3 + cap->args[0] - HGATP_MODE_SV39X4;
> +#endif
> + return 0;
> default:
> return -EINVAL;
> }
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 80364d4dbebb..a74a80fd4046 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -989,6 +989,7 @@ struct kvm_enable_cap {
> #define KVM_CAP_ARM_SEA_TO_USER 245
> #define KVM_CAP_S390_USER_OPEREXEC 246
> #define KVM_CAP_S390_KEYOP 247
> +#define KVM_CAP_RISCV_SET_HGATP_MODE 248
>
> struct kvm_irq_routing_irqchip {
> __u32 irqchip;
> --
> 2.50.1
>
Regards,
Anup
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox