* Re: [PATCH v3] killswitch: add per-function short-circuit mitigation primitive
From: Daniel Borkmann @ 2026-05-21 9:11 UTC (permalink / raw)
To: Sasha Levin
Cc: Song Liu, linux-kernel, linux-doc, linux-kselftest, bpf,
live-patching, Greg Kroah-Hartman, Andrew Morton, Jonathan Corbet,
Mathieu Desnoyers, Joshua Peisach, Florian Weimer, Breno Leitao,
Anthony Iliopoulos, Michal Hocko, Jiri Olsa, John Fastabend,
Christian Brauner, KP Singh
In-Reply-To: <agzAwjKhOhuANz_P@laps>
On 5/19/26 9:57 PM, Sasha Levin wrote:
> On Tue, May 19, 2026 at 02:13:26PM +0200, Daniel Borkmann wrote:
>> On 5/19/26 1:59 AM, Song Liu wrote:
>>> On Mon, May 18, 2026 at 6:33 AM Sasha Levin <sashal@kernel.org> wrote:
>>>> On Sun, May 17, 2026 at 11:37:36PM -0700, Song Liu wrote:
>>>>> On Sun, May 17, 2026 at 6:49 AM Sasha Levin <sashal@kernel.org> wrote:
>>>>>> * fail_function (CONFIG_FUNCTION_ERROR_INJECTION) is disabled in
>>>>>> most production kernels. Even where enabled, it only works on
>>>>>> functions pre-annotated with ALLOW_ERROR_INJECTION() in source -
>>>>>> no help for a freshly-disclosed CVE. The debugfs UI is blocked by
>>>>>> lockdown=integrity and the override is probabilistic.
>>>>>>
>>>>>> * BPF override (bpf_override_return) honors the same
>>>>>> ALLOW_ERROR_INJECTION() whitelist, and BPF itself is off in many
>>>>>> production kernels. Even where on, the operator interface is
>>>>>> "load a verified BPF program," not a one-line write.
>>>>>
>>>>> If it is OK for killswitch to attach to any kernel functions, do we still
>>>>> need ALLOW_ERROR_INJECTION() for fail_function and BPF
>>>>> override? Shall we instead also allow fail_function and BPF override
>>>>> to attach to any kernel functions?
>>>>
>>>> I don't think so. ALLOW_ERROR_INJECTION is not a security mechanism, it's an
>>>> integrity/safety mechanism for both bpf and fault injection.
>>>>
>>>> It protects against a "developer or CI script doing legitimate fault injection
>>>> accidentally panics the box" scenario, not an "attacker gets in" one.
>>>
>>> There really isn't a clear boundary between "security mechanism" and
>>> "non-security mechanism". As we are making killswitch available
>>> everywhere under root, users will soon learn to use it to do fault injection,
>>> and potentially much more scary things. (Think about agents with sudo
>>> access).
>>
>> Fully agree with Song here that there is no clear boundary, and that the
>> killswitch could lead to arbitrary, hard to debug breakage if applied to
>> the wrong function.. introducing worse bugs than the one being mitigated
>> or even /short-circuit LSM enforcement/ (engage security_file_open 0,
>> engage cap_capable 0, engage apparmor_* etc).
>
> This is similar to livepatch, right? Do we need guardrails there too?
>
> Or do we just trust root to do the right thing for it's systems without needing
> to be it's babysitter?
[See Song's reply.]
>> The ALLOW_ERROR_INJECTION() provides a curated white-list where you may
>> return with an error without causing more severe damage (assuming the
>> error handling code is right). The right thing would be to more widely
>> apply ALLOW_ERROR_INJECTION() or to figure out a better way to safely
>> enable the latter without explicit function annotation.
>
> Sure, this would also work. How do you see this happening? Can we let a certain
> user/pid/etc disable the allowlist if they choose to?
I don't think we should, given then we're back to square one where root
or some other user would be able to just override/bypass an LSM.
[...]
> How do you see this working with the allowlist?
We should look at the underlying areas where most of the CVE-like fixes
took place (these days should be more easily doable given Claude and friends)
and based on that either extend ALLOW_ERROR_INJECTION() or (better) create
new hooks which BPF LSM can consume where you can then have a policy to reject
requests and tighten the attack surface. For example, the AF_ALG stuff you
can already easily cover today ...
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#define AF_ALG 38
#define EPERM 1
char _license[] SEC("license") = "Dual BSD/GPL";
SEC("lsm/socket_create")
int BPF_PROG(block_af_alg, int family, int type, int protocol, int kern)
{
if (family == AF_ALG)
return -EPERM;
return 0;
}
... the problem is that distros enable and pull in all sort of crap which
then non-root could pull in via request_module() as an example; similarly
for netlink we want to have a BPF LSM policy to parse into netlink requests
and then reject based on certain attribute matching (both on our todo list)
which would have helped in case of exotic tc cls/act/qdisc modules to prevent
them to be pulled from userns. I bet there are a ton more examples once we
look further into the data.
Thanks,
Daniel
^ permalink raw reply
* Re: [PATCH v6 20/43] KVM: guest_memfd: Enable INIT_SHARED on guest_memfd for x86 Coco VMs
From: Fuad Tabba @ 2026-05-21 8:54 UTC (permalink / raw)
To: ackerleytng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <20260507-gmem-inplace-conversion-v6-20-91ab5a8b19a4@google.com>
On Thu, 7 May 2026 at 21:22, Ackerley Tng via B4 Relay
<devnull+ackerleytng.google.com@kernel.org> wrote:
>
> From: Sean Christopherson <seanjc@google.com>
>
> Now that guest_memfd supports tracking private vs. shared within gmem
> itself, allow userspace to specify INIT_SHARED on a guest_memfd instance
> for x86 Confidential Computing (CoCo) VMs, so long as per-VM attributes
> are disabled, i.e. when it's actually possible for a guest_memfd instance
> to contain shared memory.
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Cheers,
/fuad
> ---
> arch/x86/kvm/x86.c | 11 +++++------
> 1 file changed, 5 insertions(+), 6 deletions(-)
>
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 1560de1e95be0..6609957ecfea3 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -14172,14 +14172,13 @@ bool kvm_arch_no_poll(struct kvm_vcpu *vcpu)
> }
>
> #ifdef CONFIG_KVM_GUEST_MEMFD
> -/*
> - * KVM doesn't yet support initializing guest_memfd memory as shared for VMs
> - * with private memory (the private vs. shared tracking needs to be moved into
> - * guest_memfd).
> - */
> bool kvm_arch_supports_gmem_init_shared(struct kvm *kvm)
> {
> - return !kvm_arch_has_private_mem(kvm);
> + /*
> + * INIT_SHARED isn't supported if the memory attributes are per-VM,
> + * in which case guest_memfd can _only_ be used for private memory.
> + */
> + return !vm_memory_attributes || !kvm_arch_has_private_mem(kvm);
> }
>
> #ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE
>
> --
> 2.54.0.563.g4f69b47b94-goog
>
>
^ permalink raw reply
* Re: [PATCH v5 06/13] ima: Mediate open/release method of the measurements list
From: Roberto Sassu @ 2026-05-21 8:30 UTC (permalink / raw)
To: Mimi Zohar, corbet, skhan, dmitry.kasatkin, eric.snowberg, paul,
jmorris, serge
Cc: linux-doc, linux-kernel, linux-integrity, linux-security-module,
gregorylumen, chenste, nramas, Roberto Sassu
In-Reply-To: <db872f810f22bf25ff0ae7fe15b44f316b078079.camel@linux.ibm.com>
On Wed, 2026-05-20 at 22:07 -0400, Mimi Zohar wrote:
> On Wed, 2026-04-29 at 18:03 +0200, Roberto Sassu wrote:
> > From: Roberto Sassu <roberto.sassu@huawei.com>
> >
> > Introduce the ima_measure_users counter, to implement a semaphore-like
> > locking scheme where the binary and ASCII measurements list interfaces can
> > be concurrently open by multiple readers, or alternatively by a single
> > writer.
> >
> > A semaphore cannot be used because the kernel cannot return to user space
> > with a lock held.
> >
> > Introduce the ima_measure_lock() and ima_measure_unlock() primitives, to
> > respectively lock/unlock the interfaces (safely with the ima_measure_users
> > counter, without holding a lock).
> >
> > Finally, introduce _ima_measurements_open() to lock the interface before
> > seq_open(), and call it from ima_measurements_open() and
> > ima_ascii_measurements_open(). And, introduce ima_measurements_release(),
> > to unlock the interface.
> >
> > Require CAP_SYS_ADMIN if the interface is opened for write (not possible
> > for the current measurements interfaces, since they only have read
> > permission).
> >
> > No functional changes: multiple readers are allowed as before.
> >
> > Link: https://github.com/linux-integrity/linux/issues/1
> > Signed-off-by: Roberto Sassu <roberto.sassu@huawei.com>
> > ---
> > security/integrity/ima/ima_fs.c | 71 +++++++++++++++++++++++++++++++--
> > 1 file changed, 67 insertions(+), 4 deletions(-)
> >
> > diff --git a/security/integrity/ima/ima_fs.c b/security/integrity/ima/ima_fs.c
> > index 9a8dba14d82a..68edea7139d5 100644
> > --- a/security/integrity/ima/ima_fs.c
> > +++ b/security/integrity/ima/ima_fs.c
> > @@ -25,6 +25,8 @@
> > #include "ima.h"
> >
> > static DEFINE_MUTEX(ima_write_mutex);
> > +static DEFINE_MUTEX(ima_measure_mutex);
> > +static long ima_measure_users;
>
> long?
The limit pre process can be up to INT_MAX. Two processes could
overflow the counter if it was int.
Since privileged users can bypass the system wide max-file check (see
alloc_empty_file()), I will add an overflow check to be sure.
> >
> > bool ima_canonical_fmt;
> > static int __init default_canonical_fmt_setup(char *str)
> > @@ -209,16 +211,76 @@ static const struct seq_operations ima_measurments_seqops = {
> > .show = ima_measurements_show
> > };
> >
> > +static int ima_measure_lock(bool write)
> > +{
> > + mutex_lock(&ima_measure_mutex);
> > + if ((write && ima_measure_users != 0) ||
> > + (!write && ima_measure_users < 0)) {
> > + mutex_unlock(&ima_measure_mutex);
> > + return -EBUSY;
> > + }
>
> Thanks, Roberto. The code is really clear and well written. However, it could
> use a comment indicating the different ima_measure_users values as a reminder.
>
> ima_measure_users: > 0 open readers
> ima_meaasure_users: == -1 open writer
Ok.
> > +
> > + if (write)
> > + ima_measure_users--;
> > + else
> > + ima_measure_users++;
> > + mutex_unlock(&ima_measure_mutex);
> > + return 0;
> > +}
> > +
> > +static void ima_measure_unlock(bool write)
> > +{
> > + mutex_lock(&ima_measure_mutex);
> > + if (write)
> > + ima_measure_users++;
>
> There should only be one writer at a time. ima_measure_users could be set to
> zero.
Sure, but I find the code more clear this way.
Roberto
> > + else
> > + ima_measure_users--;
> > + mutex_unlock(&ima_measure_mutex);
> > +}
> > +
>
> thanks,
>
> Mimi
^ permalink raw reply
* Re: [PATCH v6 19/43] KVM: Let userspace disable per-VM mem attributes, enable per-gmem attributes
From: Fuad Tabba @ 2026-05-21 8:44 UTC (permalink / raw)
To: ackerleytng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <20260507-gmem-inplace-conversion-v6-19-91ab5a8b19a4@google.com>
Hi Ackerley,
On Thu, 7 May 2026 at 21:22, Ackerley Tng via B4 Relay
<devnull+ackerleytng.google.com@kernel.org> wrote:
>
> From: Sean Christopherson <seanjc@google.com>
>
> Make vm_memory_attributes a module parameter so that userspace can disable
> the use of memory attributes on the VM level.
>
> To avoid inconsistencies in the way memory attributes are tracked in KVM
> and guest_memfd, the vm_memory_attributes module_param is made
> read-only (0444).
>
> Make CONFIG_KVM_VM_MEMORY_ATTRIBUTES selectable, only for (CoCo) VM types
> that might use vm_memory_attributes.
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Config files always confuse me, but Sashiko might be onto something:
https://sashiko.dev/#/patchset/20260507-gmem-inplace-conversion-v6-0-91ab5a8b19a4%40google.com?part=19
I think this partially goes back to commit 6, the one I flagged
yesterday. But also adding "default y" to KVM_VM_MEMORY_ATTRIBUTES?
The default value should at least fix this issue, but I'm not sure if
it would cause other problems...
Cheers,
/fuad
> ---
> arch/x86/kvm/Kconfig | 13 +++++++++----
> virt/kvm/kvm_main.c | 1 +
> 2 files changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
> index b6d65ee664d0f..8b97d341bd33f 100644
> --- a/arch/x86/kvm/Kconfig
> +++ b/arch/x86/kvm/Kconfig
> @@ -82,13 +82,20 @@ config KVM_WERROR
>
> config KVM_VM_MEMORY_ATTRIBUTES
> select KVM_MEMORY_ATTRIBUTES
> - bool
> + depends on KVM_SW_PROTECTED_VM || KVM_INTEL_TDX || KVM_AMD_SEV
> + bool "Enable per-VM memory attributes (for CoCo VMs)"
> + help
> + Enable support for per-VM memory attributes, which are deprecated in
> + favor of tracking memory attributes in guest_memfd. Select this if
> + you need to run CoCo VMs using a VMM that doesn't support guest_memfd
> + memory attributes.
> +
> + If unsure, say N.
>
> config KVM_SW_PROTECTED_VM
> bool "Enable support for KVM software-protected VMs"
> depends on EXPERT
> depends on KVM_X86 && X86_64
> - select KVM_VM_MEMORY_ATTRIBUTES
> help
> Enable support for KVM software-protected VMs. Currently, software-
> protected VMs are purely a development and testing vehicle for
> @@ -139,7 +146,6 @@ config KVM_INTEL_TDX
> bool "Intel Trust Domain Extensions (TDX) support"
> default y
> depends on INTEL_TDX_HOST
> - select KVM_VM_MEMORY_ATTRIBUTES
> select HAVE_KVM_ARCH_GMEM_POPULATE
> help
> Provides support for launching Intel Trust Domain Extensions (TDX)
> @@ -163,7 +169,6 @@ config KVM_AMD_SEV
> depends on KVM_AMD && X86_64
> depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m)
> select ARCH_HAS_CC_PLATFORM
> - select KVM_VM_MEMORY_ATTRIBUTES
> select HAVE_KVM_ARCH_GMEM_PREPARE
> select HAVE_KVM_ARCH_GMEM_INVALIDATE
> select HAVE_KVM_ARCH_GMEM_POPULATE
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index cec02d68d7039..ba195bb239aaa 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -104,6 +104,7 @@ module_param(allow_unsafe_mappings, bool, 0444);
> #ifdef CONFIG_KVM_MEMORY_ATTRIBUTES
> #ifdef CONFIG_KVM_VM_MEMORY_ATTRIBUTES
> bool vm_memory_attributes = true;
> +module_param(vm_memory_attributes, bool, 0444);
> #endif
> DEFINE_STATIC_CALL_RET0(__kvm_get_memory_attributes, kvm_get_memory_attributes_t);
> EXPORT_SYMBOL_FOR_KVM_INTERNAL(STATIC_CALL_KEY(__kvm_get_memory_attributes));
>
> --
> 2.54.0.563.g4f69b47b94-goog
>
>
^ permalink raw reply
* [PATCH v2] Fail the build on RUST=y and RUST_IS_AVAILABLE=n
From: Sasha Finkelstein @ 2026-05-21 8:30 UTC (permalink / raw)
To: Alice Ryhl, Andreas Hindborg, Benno Lossin, Björn Roy Baron,
Boqun Feng, Danilo Krummrich, Gary Guo, Jonathan Corbet,
Miguel Ojeda, Shuah Khan, Trevor Gross
Cc: Neal Gompa, linux-doc, linux-kernel, rust-for-linux,
Sasha Finkelstein
The current approach of silently disabling all rust drivers if the
toolchain is missing results in users that try to compile their own
kernels getting a "successful" build and then being confused about where
did their drivers go. In comparison, missing openssl results in a build
failure, not a disappearance of everything that depends on it.
This also means that allyesconfig will depend on rust, but since the
rust experiment concluded with "rust is here to stay", i believe that
allyesconfig should be building rust drivers too.
Signed-off-by: Sasha Finkelstein <k@chaosmail.tech>
---
Changes in v2:
- No longer a RFC, let's make it happen.
- Update the docs.
- Link to v1: https://patch.msgid.link/20260510-evolve-to-crab-v1-1-208df84e67be@chaosmail.tech
---
Documentation/rust/quick-start.rst | 6 +++---
init/Kconfig | 1 -
2 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/Documentation/rust/quick-start.rst b/Documentation/rust/quick-start.rst
index a6ec3fa94d33..764c81d0dd59 100644
--- a/Documentation/rust/quick-start.rst
+++ b/Documentation/rust/quick-start.rst
@@ -321,9 +321,9 @@ Configuration
-------------
``Rust support`` (``CONFIG_RUST``) needs to be enabled in the ``General setup``
-menu. The option is only shown if a suitable Rust toolchain is found (see
-above), as long as the other requirements are met. In turn, this will make
-visible the rest of options that depend on Rust.
+menu. In turn, this will make visible the rest of options that depend on Rust.
+You can check the value of ``RUST_IS_AVAILABLE`` to determine if your toolchain
+is configured correctly.
Afterwards, go to::
diff --git a/init/Kconfig b/init/Kconfig
index 2937c4d308ae..f7d4c7ea764f 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2190,7 +2190,6 @@ config PROFILING
config RUST
bool "Rust support"
depends on HAVE_RUST
- depends on RUST_IS_AVAILABLE
select EXTENDED_MODVERSIONS if MODVERSIONS
depends on !MODVERSIONS || GENDWARFKSYMS
depends on !GCC_PLUGIN_RANDSTRUCT
---
base-commit: 8bc67e4db64aa72732c474b44ea8622062c903f0
change-id: 20260510-evolve-to-crab-8cba1768dcd5
Best regards,
--
Sasha Finkelstein <k@chaosmail.tech>
^ permalink raw reply related
* Re: [PATCH net-next v3 05/14] libie: add bookkeeping support for control queue messages
From: Larysa Zaremba @ 2026-05-21 8:25 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Tony Nguyen, davem, pabeni, edumazet, andrew+netdev, netdev,
Phani R Burra, przemyslaw.kitszel, aleksander.lobakin,
sridhar.samudrala, anjali.singhai, michal.swiatkowski,
maciej.fijalkowski, emil.s.tantilov, madhu.chittim, joshua.a.hay,
jacob.e.keller, jayaprakash.shanmugam, jiri, horms, corbet,
richardcochran, linux-doc, Bharath R, Samuel Salin,
Aleksandr Loktionov
In-Reply-To: <20260520185121.6f380ad0@kernel.org>
On Wed, May 20, 2026 at 06:51:21PM -0700, Jakub Kicinski wrote:
> On Fri, 15 May 2026 15:44:29 -0700 Tony Nguyen wrote:
> > + guard(spinlock)(&xnm->free_xns_bm_lock);
>
> Quoting documentation:
>
> Using device-managed and cleanup.h constructs
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Netdev remains skeptical about promises of all "auto-cleanup" APIs,
> including even ``devm_`` helpers, historically. They are not the preferred
> style of implementation, merely an acceptable one.
>
> Use of ``guard()`` is discouraged within any function longer than 20 lines,
> ``scoped_guard()`` is considered more readable. Using normal lock/unlock is
> still (weakly) preferred.
>
I agree that using guard in long functions is confusing, but the longest
function in this patchset that uses guard() is libie_ctlq_xn_pop_free(), which
has 18 lines between curly braces and represents a concrete atomic operation.
There is also scoped_guard() in libie_ctlq_xn_process_send(), which protects a
block of 22 lines, but I would consider it acceptable under the guidelines you
shared too.
> Low level cleanup constructs (such as ``__free()``) can be used when building
> APIs and helpers, especially scoped iterators. However, direct use of
> ``__free()`` within networking core and drivers is discouraged.
> Similar guidance applies to declaring variables mid-function.
>
> See: https://www.kernel.org/doc/html/next/process/maintainer-netdev.html#using-device-managed-and-cleanup-h-constructs
^ permalink raw reply
* Re: [PATCH 1/6] alloc_tag: add ioctl to /proc/allocinfo
From: Hao Ge @ 2026-05-21 8:19 UTC (permalink / raw)
To: Suren Baghdasaryan, Abhishek Bapat
Cc: Shuah Khan, Jonathan Corbet, linux-doc, linux-kernel, linux-mm,
Sourav Panda, Kent Overstreet, Andrew Morton
In-Reply-To: <CAJuCfpFn0Oefewvjp1jBhCOgxxwhHFy_RK08DwQywOjYcfr2pw@mail.gmail.com>
On 2026/5/20 01:42, Suren Baghdasaryan wrote:
> On Mon, May 18, 2026 at 7:53 PM Hao Ge <hao.ge@linux.dev> wrote:
>> Hi Abhishek
>>
>>
>> Thanks for the follow-up.
>>
>>
>> On 2026/5/19 07:41, Abhishek Bapat wrote:
>>> On Wed, May 13, 2026 at 9:38 PM Hao Ge<hao.ge@linux.dev> wrote:
>>>> Hi Suren and Abhishek
>>>>
>>>>
>>>> Thanks for the patch! A couple of minor comments below.
>>>>
>>>>
>>>> On 2026/5/5 07:36, Abhishek Bapat wrote:
>>>>> From: Suren Baghdasaryan<surenb@google.com>
>>>>>
>>>>> Add the following ioctl commands for /proc/allocinfo file:
>>>>>
>>>>> ALLOCINFO_IOC_CONTENT_ID - gets content identifier which can be used
>>>>> to check whether the file content has changed specifically due to module
>>>>> load/unload. Every time a module is loaded / unloaded, the returned
>>>>> value will be different. By comparing the identifier value at the
>>>>> beginning and at the end of the content retrieval operation, users can
>>>>> validate retrieved information for consistency.
>>>>>
>>>>> ALLOCINFO_IOC_GET_AT - gets the record at the specified position. This
>>>>> is the position of a record in /proc/allocinfo.
>>>>>
>>>>> ALLOCINFO_IOC_GET_NEXT - gets the record next to the last retrieved
>>>>> one. If no records were previously retrieved, returns the first
>>>>> record.
>>>>>
>>>>> Signed-off-by: Suren Baghdasaryan<surenb@google.com>
>>>>> Signed-off-by: Abhishek Bapat<abhishekbapat@google.com>
>>>>> ---
>>>>> .../userspace-api/ioctl/ioctl-number.rst | 2 +
>>>>> include/linux/codetag.h | 1 +
>>>>> include/uapi/linux/alloc_tag.h | 54 ++++++
>>>>> lib/alloc_tag.c | 178 +++++++++++++++++-
>>>>> lib/codetag.c | 11 ++
>>>>> 5 files changed, 244 insertions(+), 2 deletions(-)
>>>>> create mode 100644 include/uapi/linux/alloc_tag.h
>>>>>
>>>>> diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
>>>>> index 331223761fff..84f6808a8578 100644
>>>>> --- a/Documentation/userspace-api/ioctl/ioctl-number.rst
>>>>> +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
>>>>> @@ -349,6 +349,8 @@ Code Seq# Include File Comments
>>>>> <mailto:luzmaximilian@gmail.com>
>>>>> 0xA5 20-2F linux/surface_aggregator/dtx.h Microsoft Surface DTX driver
>>>>> <mailto:luzmaximilian@gmail.com>
>>>>> +0xA6 00-0F uapi/linux/alloc_tag.h Memory allocation profiling
>>>>> +<mailto:surenb@google.com>
>>>>> 0xAA 00-3F linux/uapi/linux/userfaultfd.h
>>>>> 0xAB 00-1F linux/nbd.h
>>>>> 0xAC 00-1F linux/raw.h
>>>>> diff --git a/include/linux/codetag.h b/include/linux/codetag.h
>>>>> index 8ea2a5f7c98a..2bcd4e7c809e 100644
>>>>> --- a/include/linux/codetag.h
>>>>> +++ b/include/linux/codetag.h
>>>>> @@ -76,6 +76,7 @@ struct codetag_iterator {
>>>>>
>>>>> void codetag_lock_module_list(struct codetag_type *cttype, bool lock);
>>>>> bool codetag_trylock_module_list(struct codetag_type *cttype);
>>>>> +unsigned long codetag_get_content_id(struct codetag_type *cttype);
>>>>> struct codetag_iterator codetag_get_ct_iter(struct codetag_type *cttype);
>>>>> struct codetag *codetag_next_ct(struct codetag_iterator *iter);
>>>>>
>>>>> diff --git a/include/uapi/linux/alloc_tag.h b/include/uapi/linux/alloc_tag.h
>>>>> new file mode 100644
>>>>> index 000000000000..e9a5b55fcc7a
>>>>> --- /dev/null
>>>>> +++ b/include/uapi/linux/alloc_tag.h
>>>>> @@ -0,0 +1,54 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
>>>>> +/*
>>>>> + * include/linux/alloc_tag.h
>>>>> + */
>>>>> +
>>>>> +#ifndef _UAPI_ALLOC_TAG_H
>>>>> +#define _UAPI_ALLOC_TAG_H
>>>>> +
>>>>> +#include <linux/types.h>
>>>>> +
>>>>> +#define ALLOCINFO_STR_SIZE 64
>>>>> +
>>>>> +struct allocinfo_content_id {
>>>>> + __u64 id;
>>>>> +};
>>>>> +
>>>>> +struct allocinfo_tag {
>>>>> + /* Longer names are trimmed */
>>>>> + char modname[ALLOCINFO_STR_SIZE];
>>>>> + char function[ALLOCINFO_STR_SIZE];
>>>>> + char filename[ALLOCINFO_STR_SIZE];
>>>>> + __u64 lineno;
>>>>> +};
>>>>> +
>>>>> +struct allocinfo_counter {
>>>>> + __u64 bytes;
>>>>> + __u64 calls;
>>>>> + __u8 accurate;
>>>>> + __u8 pad[7]; /* Add alignment to not break the 32-bit compatible interface */
>>>>> +};
>>>>> +
>>>>> +struct allocinfo_tag_data {
>>>>> + struct allocinfo_tag tag;
>>>>> + struct allocinfo_counter counter;
>>>>> +};
>>>>> +
>>>>> +struct allocinfo_get_at {
>>>>> + __u64 pos; /* input */
>>>>> + struct allocinfo_tag_data data;
>>>>> +};
>>>>> +
>>>>> +#define _ALLOCINFO_IOC_CONTENT_ID 0
>>>>> +#define _ALLOCINFO_IOC_GET_AT 1
>>>>> +#define _ALLOCINFO_IOC_GET_NEXT 2
>>>>> +
>>>>> +#define ALLOCINFO_IOC_BASE 0xA6
>>>>> +#define ALLOCINFO_IOC_CONTENT_ID _IOR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_CONTENT_ID, \
>>>>> + struct allocinfo_content_id)
>>>>> +#define ALLOCINFO_IOC_GET_AT _IOWR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_GET_AT, \
>>>>> + struct allocinfo_get_at)
>>>>> +#define ALLOCINFO_IOC_GET_NEXT _IOR(ALLOCINFO_IOC_BASE, _ALLOCINFO_IOC_GET_NEXT, \
>>>>> + struct allocinfo_tag_data)
>>>>> +
>>>>> +#endif /* _UAPI_ALLOC_TAG_H */
>>>>> diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
>>>>> index ed1bdcf1f8ab..5c24d2f954d4 100644
>>>>> --- a/lib/alloc_tag.c
>>>>> +++ b/lib/alloc_tag.c
>>>>> @@ -14,6 +14,7 @@
>>>>> #include <linux/string_choices.h>
>>>>> #include <linux/vmalloc.h>
>>>>> #include <linux/kmemleak.h>
>>>>> +#include <uapi/linux/alloc_tag.h>
>>>>>
>>>>> #define ALLOCINFO_FILE_NAME "allocinfo"
>>>>> #define MODULE_ALLOC_TAG_VMAP_SIZE (100000UL * sizeof(struct alloc_tag))
>>>>> @@ -46,6 +47,9 @@ int alloc_tag_ref_offs;
>>>>> struct allocinfo_private {
>>>>> struct codetag_iterator iter;
>>>>> bool print_header;
>>>>> + /* ioctl uses a separate iterator not to interfere with reads */
>>>>> + struct codetag_iterator ioctl_iter;
>>>>> + bool positioned; /* seq_open_private() sets to 0 */
>>>>> };
>>>>>
>>>>> static void *allocinfo_start(struct seq_file *m, loff_t *pos)
>>>>> @@ -125,6 +129,177 @@ static const struct seq_operations allocinfo_seq_op = {
>>>>> .show = allocinfo_show,
>>>>> };
>>>>>
>>>>> +static int allocinfo_open(struct inode *inode, struct file *file)
>>>>> +{
>>>>> + return seq_open_private(file, &allocinfo_seq_op,
>>>>> + sizeof(struct allocinfo_private));
>>>>> +}
>>>>> +
>>>>> +static int allocinfo_release(struct inode *inode, struct file *file)
>>>>> +{
>>>>> + return seq_release_private(inode, file);
>>>>> +}
>>>>> +
>>>>> +static const char *allocinfo_str(const char *str)
>>>>> +{
>>>>> + size_t len = strlen(str);
>>>>> +
>>>>> + /* Keep an extra space for the trailing NULL. */
>>>>> + if (len >= ALLOCINFO_STR_SIZE)
>>>>> + str += (len - ALLOCINFO_STR_SIZE) + 1;
>>>>> + return str;
>>>>> +}
>>>>> +
>>>>> +/* Copy a string and trim from the beginning if it's too long */
>>>>> +static void allocinfo_copy_str(char *dest, const char *src)
>>>>> +{
>>>>> + strscpy(dest, allocinfo_str(src), ALLOCINFO_STR_SIZE);
>>>>> +}
>>>>> +
>>>>> +static void allocinfo_to_params(struct codetag *ct,
>>>>> + struct allocinfo_tag_data *data)
>>>>> +{
>>>>> + struct alloc_tag *tag = ct_to_alloc_tag(ct);
>>>>> + struct alloc_tag_counters counter = alloc_tag_read(tag);
>>>>> +
>>>>> + if (ct->modname)
>>>>> + allocinfo_copy_str(data->tag.modname, ct->modname);
>>>>> + else
>>>>> + data->tag.modname[0] = '\0';
>>>> Minor nit about allocinfo_to_params():
>>>>
>>>> When modname is NULL (built-in kernel code), the current code sets it
>>>>
>>>> to an empty string:
>>>>
>>>> if (ct->modname)
>>>>
>>>> allocinfo_copy_str(data->tag.modname, ct->modname);
>>>>
>>>> else
>>>>
>>>> data->tag.modname[0] = '\0';
>>>>
>>>> This is of course workable in userspace by checking for an empty
>>>>
>>>> string, but I was wondering if it would be cleaner to use "vmlinux"
>>>>
>>>> as a default:
>>>>
>>>> else
>>>>
>>>> allocinfo_copy_str(data->tag.modname, "vmlinux");
>>>>
>>>>
>>>> For some context, in our memory analysis workflow we often group
>>>>
>>>> allocations by module to get a quick overview of where memory goes,
>>>>
>>>> for example:
>>>>
>>>> vmlinux: 2.1 GB (kernel core)
>>>>
>>>> nvidia: 1.2 GB (GPU driver)
>>>>
>>>> iwlwifi: 800 MB (WiFi driver)
>>>>
>>>> ext4: 500 MB (filesystem)
>>>>
>>>> Having a consistent identifier for kernel built-in allocations would
>>>>
>>>> avoid each userspace tool needing to handle the empty string as a
>>>>
>>>> special case. Totally fine if this is intentional though.
>>>>
>>> Thanks for bringing this up, I can certainly make this change.
>>> However, the information is not currently exposed this way through
>>> /proc/allocinfo. /proc/allocinfo does not categorize kernel non-module
>>> allocations as vmlinux, so there will a delta between how IOCTL and
>>> /proc/allocinfo behave. Suren, could you comment on whether this
>>> recommendation is fine by you?
>>>
>> Right, /proc/allocinfo indeed doesn't categorize them as vmlinux currently.
>>
>> It's just that in practice we often group allocations by module, so
>> having "vmlinux" as a default
>>
>> would be convenient. Let's wait for Suren's input.
> Hi Folks,
> I would prefer to keep it empty because vmlinux is not really a module
> and hardcoding this name also seems suboptimal (in case it ever
> changes). Empty string also aligns with how we output /proc/allocinfo
> data. If the symbol is in the kernel itself, we do not display the
> module name at all. So, all in all, unless there is a strong reason
> against it, I think we should keep it empty.
Hi Suren
Thanks for the clarification, that makes sense.
For userspace tools that want to group by module, we can always map an
empty modname to "vmlinux" at the
presentation layer — no need to hardcode that in the kernel.
Hi Abhishek
I noticed the new files (like include/uapi/linux/alloc_tag.h) were added
in this patchset.
Should they be reflected in the MAINTAINERS file for easier future
maintenance?
Thanks
Best Regards
Hao
>>>>> + allocinfo_copy_str(data->tag.function, ct->function);
>>>>> + allocinfo_copy_str(data->tag.filename, ct->filename);
>>>>> + data->tag.lineno = ct->lineno;
>>>>> + data->counter.bytes = counter.bytes;
>>>>> + data->counter.calls = counter.calls;
>>>>> + data->counter.accurate = !alloc_tag_is_inaccurate(tag);
>>>>> +}
>>>>> +
>>>>> +static int allocinfo_ioctl_get_content_id(struct seq_file *m, void __user *arg)
>>>>> +{
>>>>> + struct allocinfo_content_id params;
>>>>> +
>>>>> + codetag_lock_module_list(alloc_tag_cttype, true);
>>>>> + params.id = codetag_get_content_id(alloc_tag_cttype);
>>>>> + codetag_lock_module_list(alloc_tag_cttype, false);
>>>>> + if (copy_to_user(arg, ¶ms, sizeof(params)))
>>>>> + return -EFAULT;
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +static int allocinfo_ioctl_get_at(struct seq_file *m, void __user *arg)
>>>>> +{
>>>>> + struct allocinfo_private *priv;
>>>>> + struct codetag *ct;
>>>>> + __u64 pos;
>>>>> + struct allocinfo_get_at params = {0};
>>>>> +
>>>>> + if (copy_from_user(¶ms, arg, sizeof(params)))
>>>>> + return -EFAULT;
>>>>> +
>>>>> + priv = (struct allocinfo_private *)m->private;
>>>>> + pos = params.pos;
>>>>> +
>>>>> + codetag_lock_module_list(alloc_tag_cttype, true);
>>>>> +
>>>>> + /* Find the codetag */
>>>>> + priv->ioctl_iter = codetag_get_ct_iter(alloc_tag_cttype);
>>>>> + ct = codetag_next_ct(&priv->ioctl_iter);
>>>>> + while (ct && pos--)
>>>>> + ct = codetag_next_ct(&priv->ioctl_iter);
>>>> I noticed that codetag_next_ct(&priv->ioctl_iter) and
>>>>
>>>> priv->positioned are accessed without serialization in the ioctl
>>>>
>>>> path. Concurrent ioctl calls on the same fd could race on these
>>>>
>>>> fields. Just something I spotted while reading the code.
>>>>
>>>>
>>>> Thanks
>>>>
>>>> Best Regards
>>>>
>>>> Hao
>>>>
>>> I believe this should be prevented by `codetag_lock_module_list`; am I
>>> wrong in my understanding?
>> Thanks for the explanation! codetag_lock_module_list is designed to
>> protect the module list from concurrent load/unload, which it does
>>
>> correctly. However, it doesn't cover the race between concurrent ioctl
>> calls on the same fd, since it acquires cttype->mod_lock via
>>
>> down_read() and rwsem read locks allow multiple readers to proceed
>> concurrently:
>>
>> Thread A: ALLOCINFO_IOC_GET_AT
>>
>> down_read(&cttype->mod_lock) // read lock acquired
>>
>> priv->ioctl_iter = codetag_get_ct_iter(...)
>>
>> ct = codetag_next_ct(&priv->ioctl_iter)
>>
>> priv->positioned = true;
>>
>> Thread B: ALLOCINFO_IOC_GET_NEXT // concurrent ioctl on same fd
>>
>> down_read(&cttype->mod_lock) // read locks don't exclude
>> each other
>>
>> if (!priv->positioned) { // sees partial state from
>> Thread A
>>
>> priv->ioctl_iter = ... // overwrites Thread A's iterator
>>
>> }
>>
>> ct = codetag_next_ct(&priv->ioctl_iter) // corrupted iterator
>>
>> priv->ioctl_iter and priv->positioned are per-fd state with no
>> serialization in the ioctl path.
> Yep, you are right. codetag_lock_module_list() is not enough here to
> protect from such races. I guess allocinfo_private would need another
> lock.
> Thanks,
> Suren.
>
>
>> Just something I spotted.
>>
>> Thanks
>>
>> Best Regards
>>
>> Hao
>>
>>>>> + if (ct) {
>>>>> + allocinfo_to_params(ct, ¶ms.data);
>>>>> + priv->positioned = true;
>>>>> + }
>>>>> +
>>>>> + codetag_lock_module_list(alloc_tag_cttype, false);
>>>>> +
>>>>> + if (!ct)
>>>>> + return -ENOENT;
>>>>> +
>>>>> + if (copy_to_user(arg, ¶ms, sizeof(params)))
>>>>> + return -EFAULT;
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> +static int allocinfo_ioctl_get_next(struct seq_file *m, void __user *arg)
>>>>> +{
>>>>> + struct allocinfo_private *priv;
>>>>> + struct codetag *ct;
>>>>> + struct allocinfo_tag_data params = {0};
>>>>> + int ret = 0;
>>>>> +
>>>>> + priv = (struct allocinfo_private *)m->private;
>>>>> +
>>>>> + codetag_lock_module_list(alloc_tag_cttype, true);
>>>>> +
>>>>> + if (!priv->positioned) {
>>>>> + priv->ioctl_iter = codetag_get_ct_iter(alloc_tag_cttype);
>>>>> + priv->positioned = true;
>>>>> + }
>>>>> +
>>>>> + ct = codetag_next_ct(&priv->ioctl_iter);
>>>>> + if (ct)
>>>>> + allocinfo_to_params(ct, ¶ms);
>>>>> +
>>>>> + if (!ct) {
>>>>> + priv->positioned = false;
>>>>> + ret = -ENOENT;
>>>>> + }
>>>>> + codetag_lock_module_list(alloc_tag_cttype, false);
>>>>> +
>>>>> + if (ret == 0) {
>>>>> + if (copy_to_user(arg, ¶ms, sizeof(params)))
>>>>> + return -EFAULT;
>>>>> + }
>>>>> + return ret;
>>>>> +}
>>>>> +
>>>>> +static long allocinfo_ioctl(struct file *file, unsigned int cmd,
>>>>> + unsigned long __arg)
>>>>> +{
>>>>> + void __user *arg = (void __user *)__arg;
>>>>> + int ret;
>>>>> +
>>>>> + switch (cmd) {
>>>>> + case ALLOCINFO_IOC_CONTENT_ID:
>>>>> + ret = allocinfo_ioctl_get_content_id(file->private_data, arg);
>>>>> + break;
>>>>> + case ALLOCINFO_IOC_GET_AT:
>>>>> + ret = allocinfo_ioctl_get_at(file->private_data, arg);
>>>>> + break;
>>>>> + case ALLOCINFO_IOC_GET_NEXT:
>>>>> + ret = allocinfo_ioctl_get_next(file->private_data, arg);
>>>>> + break;
>>>>> + default:
>>>>> + ret = -ENOIOCTLCMD;
>>>>> + break;
>>>>> + }
>>>>> +
>>>>> + return ret;
>>>>> +}
>>>>> +
>>>>> +#ifdef CONFIG_COMPAT
>>>>> +static long allocinfo_compat_ioctl(struct file *file, unsigned int cmd,
>>>>> + unsigned long arg)
>>>>> +{
>>>>> + return allocinfo_ioctl(file, cmd, (unsigned long)compat_ptr(arg));
>>>>> +}
>>>>> +#endif
>>>>> +
>>>>> +static const struct proc_ops allocinfo_proc_ops = {
>>>>> + .proc_open = allocinfo_open,
>>>>> + .proc_read_iter = seq_read_iter,
>>>>> + .proc_lseek = seq_lseek,
>>>>> + .proc_release = allocinfo_release,
>>>>> + .proc_ioctl = allocinfo_ioctl,
>>>>> +#ifdef CONFIG_COMPAT
>>>>> + .proc_compat_ioctl = allocinfo_compat_ioctl,
>>>>> +#endif
>>>>> +
>>>>> +};
>>>>> +
>>>>> size_t alloc_tag_top_users(struct codetag_bytes *tags, size_t count, bool can_sleep)
>>>>> {
>>>>> struct codetag_iterator iter;
>>>>> @@ -946,8 +1121,7 @@ static int __init alloc_tag_init(void)
>>>>> return 0;
>>>>> }
>>>>>
>>>>> - if (!proc_create_seq_private(ALLOCINFO_FILE_NAME, 0400, NULL, &allocinfo_seq_op,
>>>>> - sizeof(struct allocinfo_private), NULL)) {
>>>>> + if (!proc_create(ALLOCINFO_FILE_NAME, 0400, NULL, &allocinfo_proc_ops)) {
>>>>> pr_err("Failed to create %s file\n", ALLOCINFO_FILE_NAME);
>>>>> shutdown_mem_profiling(false);
>>>>> return -ENOMEM;
>>>>> diff --git a/lib/codetag.c b/lib/codetag.c
>>>>> index 304667897ad4..93aa30991563 100644
>>>>> --- a/lib/codetag.c
>>>>> +++ b/lib/codetag.c
>>>>> @@ -48,6 +48,17 @@ bool codetag_trylock_module_list(struct codetag_type *cttype)
>>>>> return down_read_trylock(&cttype->mod_lock) != 0;
>>>>> }
>>>>>
>>>>> +unsigned long codetag_get_content_id(struct codetag_type *cttype)
>>>>> +{
>>>>> + lockdep_assert_held(&cttype->mod_lock);
>>>>> +
>>>>> + /*
>>>>> + * next_mod_seq is updated on every load, so can be used to identify
>>>>> + * content changes.
>>>>> + */
>>>>> + return cttype->next_mod_seq;
>>>>> +}
>>>>> +
>>>>> struct codetag_iterator codetag_get_ct_iter(struct codetag_type *cttype)
>>>>> {
>>>>> struct codetag_iterator iter = {
>>> Note, I will be following up with a v2 patchset with your feedback
>>> included. Please bring up any other points you'd want to clarify so
>>> that I can include all the changes in the v2 patchset. Thanks for
>>> reviewing!
^ permalink raw reply
* Re: [PATCH v5 04/13] ima: Introduce per binary measurements list type binary_runtime_size value
From: Roberto Sassu @ 2026-05-21 7:58 UTC (permalink / raw)
To: Mimi Zohar, corbet, skhan, dmitry.kasatkin, eric.snowberg, paul,
jmorris, serge
Cc: linux-doc, linux-kernel, linux-integrity, linux-security-module,
gregorylumen, chenste, nramas, Roberto Sassu
In-Reply-To: <b7f97a0a3b79b72a014d12514febc338d1ecd038.camel@linux.ibm.com>
On Wed, 2026-05-20 at 22:06 -0400, Mimi Zohar wrote:
> On Wed, 2026-04-29 at 18:03 +0200, Roberto Sassu wrote:
> > From: Roberto Sassu <roberto.sassu@huawei.com>
> >
> > Make binary_runtime_size as an array, to have separate counters per binary
> > measurements list type. Currently, define the BINARY type for the existing
> > binary measurements list.
> >
> > Introduce ima_update_binary_runtime_size() to facilitate updating a
> > binary_runtime_size value with a given binary measurement list type.
> >
> > Also add the binary measurements list type parameter to
> > ima_get_binary_runtime_size(), to retrieve the desired value. Retrieving
> > the value is now done under the ima_extend_list_mutex, since there can be
> > concurrent updates.
> >
> > No functional change (except for the mutex usage, that fixes the
> > concurrency issue): the BINARY array element is equivalent to the old
> > binary_runtime_size.
>
> The patch is really clear and well written, but I don't see a concurrency issue
> requiring taking the ima_extend_list_mutex at least in this patch.
binary_runtime_size is not an atomic variable. It is updated under the
ima_extend_list_mutex lock in ima_add_digest_entry(). The same lock
must be taken on the reader side, ima_get_binary_runtime_size().
Roberto
^ permalink raw reply
* Re: [PATCH v6 18/43] KVM: Move KVM_VM_MEMORY_ATTRIBUTES config definition to x86
From: Fuad Tabba @ 2026-05-21 8:07 UTC (permalink / raw)
To: ackerleytng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <20260507-gmem-inplace-conversion-v6-18-91ab5a8b19a4@google.com>
On Thu, 7 May 2026 at 21:22, Ackerley Tng via B4 Relay
<devnull+ackerleytng.google.com@kernel.org> wrote:
>
> From: Sean Christopherson <seanjc@google.com>
>
> Bury KVM_VM_MEMORY_ATTRIBUTES in x86 to discourage other architectures
> from adding support for per-VM memory attributes, because tracking private
> vs. shared memory on a per-VM basis is now deprecated in favor of tracking
> on a per-guest_memfd basis, and no other memory attributes are on the
> horizon.
>
> This will also allow modifying KVM_VM_MEMORY_ATTRIBUTES to be
> user-selectable (in x86) without creating weirdness in KVM's Kconfigs.
> Now that guest_memfd support memory attributes, it's entirely possible to
> run x86 CoCo VMs without support for KVM_VM_MEMORY_ATTRIBUTES.
>
> Leave the code itself in common KVM so that it's trivial to undo this
> change if new per-VM attributes do come along.
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Cheers,
/fuad
> ---
> arch/x86/kvm/Kconfig | 4 ++++
> virt/kvm/Kconfig | 4 ----
> 2 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
> index 26f6afd51bbdc..b6d65ee664d0f 100644
> --- a/arch/x86/kvm/Kconfig
> +++ b/arch/x86/kvm/Kconfig
> @@ -80,6 +80,10 @@ config KVM_WERROR
>
> If in doubt, say "N".
>
> +config KVM_VM_MEMORY_ATTRIBUTES
> + select KVM_MEMORY_ATTRIBUTES
> + bool
> +
> config KVM_SW_PROTECTED_VM
> bool "Enable support for KVM software-protected VMs"
> depends on EXPERT
> diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
> index e371e079e2c50..663de6421eda2 100644
> --- a/virt/kvm/Kconfig
> +++ b/virt/kvm/Kconfig
> @@ -103,10 +103,6 @@ config KVM_MMU_LOCKLESS_AGING
> config KVM_MEMORY_ATTRIBUTES
> bool
>
> -config KVM_VM_MEMORY_ATTRIBUTES
> - select KVM_MEMORY_ATTRIBUTES
> - bool
> -
> config KVM_GUEST_MEMFD
> select XARRAY_MULTI
> select KVM_MEMORY_ATTRIBUTES
>
> --
> 2.54.0.563.g4f69b47b94-goog
>
>
^ permalink raw reply
* Re: [PATCH v5 8/8] ARM: defconfig: Add a zx29 defconfig file
From: Stefan Dösinger @ 2026-05-21 8:00 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Linus Walleij, Jonathan Corbet, Shuah Khan, Russell King,
Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Krzysztof Kozlowski, Alexandre Belloni, Drew Fustini,
Greg Kroah-Hartman, Jiri Slaby, linux-doc, linux-kernel,
linux-arm-kernel, devicetree, soc, linux-serial
In-Reply-To: <30b96e0d-f296-4c31-8701-a15c568ebffc@app.fastmail.com>
[-- Attachment #1: Type: text/plain, Size: 3910 bytes --]
Hi Arnd,
I saw your reply to my defconfig pull request, but apparently never received your original reply. I only found this mail here. It looks like I have to look for a better E-Mail provider as gmail is choking on the volume of the linux-arm-kernel mailing list.
To answer your questions I found at https://lore.kernel.org/all/61452117-0cdc-4ec2-83eb-dc03ccbd410b@app.fastmail.com/ :
> Either way, the patch description above should at least explain
> why you think you need your own defconfig, as we don't normally
> take those.
It was more cluelessness / being new to kernel development that gave me the impression that boards should have defconfigs. Since then I ran across scripts/dt_to_config. I haven't tested it yet on my DT, but if it does the right thing I don't think this board needs a defconfig.
>> +CONFIG_CMDLINE="console=ttyAMA0 earlyprintk root=/dev/ram rw"
> A definconfig should normall not rely on earlyprintk, just add
> that when you actually need to debug the super-early boot
> stages. With "earlycon" it should pick up the right console
> from the stdout path and work almost as early.
>> +CONFIG_BINFMT_FLAT=y
> Are you actually using flat binaries? I wasn't aware that this
> is still possible on MMU-enabled kernels.
>> +CONFIG_BLK_DEV_RAM=y
>> +CONFIG_BLK_DEV_RAM_COUNT=4
> The old ramdisk boot is going away in the future, please use
> initramfs instead. This should also save a good amount of RAM.
I'll fix those in my tree and keep the defconfig around just in case, but otherwise drop it from the submission. We can revisit it later when the board is more complete.
>> +CONFIG_DEVTMPFS=y # FIXME: This is specific to my initrd. Remove
>> before upstream
>stale comment?
I believe I removed this in later versions though :-)
Cheers,
Stefan
> Am 24.04.2026 um 11:54 schrieb Arnd Bergmann <arnd@arndb.de>:
>
> On Fri, Apr 24, 2026, at 09:13, Linus Walleij wrote:
>> On Tue, Apr 21, 2026 at 10:24 PM Stefan Dösinger
>> <stefandoesinger@gmail.com> wrote:
>>
>>> This enables existing drivers that already are (UART) or will be (USB,
>>> GPIO) necessary to operate this board even if they aren't declared in
>>> the DTS yet.
>>>
>>> Signed-off-by: Stefan Dösinger <stefandoesinger@gmail.com>
>>
>> *I* personally (as SoC maintainer) think that having a few more defconfigs
>> is fine, even helpful.
>>
>> But I would defer this to the more senior SoC maintainers because I think
>> their stance is something like:
>>
>> - We have multi_v7_defconfig for compile testing
>>
>> - We know that binary gets way to big for your system: it's for build
>> testing and perhaps booting in QEMU or systems with many MB of
>> RAM, not for actually running it on products.
>>
>> - You are encouraged to keep your own defconfig out-of-tree.
>
> Right, we clearly need to do something better than what we are with
> the general defconfigs, as I'm sure many of the existing ones are
> never actually used for booting a machine, and are horribly out of
> date with the Kconfig options.
>
> I wouldn't object to adding another defconfig for a new (or revived)
> soc family, but I don't want to have more per-board ones.
> Overall, we have about 70 defconfigs and 55 soc families that have their
> own mach-* directory (plus a few without code), and the number of
> defconfigs alone makes it hard to keep them up to date.
>
>> However I even challenged this myself by adding a defconfig for memory
>> constrained Broadcoms a while back (NACKed/ignored ;) so if it was all
>> up to me I would merge this.
>
> I don't even remember that discussion ;-)
>
> One idea might be to have a tiny base defconfig, plus platform
> specific fragments that add drivers. The problem is agreeing
> what bits are essential enough to still get enabled in the
> tiny config.
>
> Arnd
[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH v6 17/43] KVM: guest_memfd: Determine invalidation filter from memory attributes
From: Fuad Tabba @ 2026-05-21 7:56 UTC (permalink / raw)
To: ackerleytng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <20260507-gmem-inplace-conversion-v6-17-91ab5a8b19a4@google.com>
On Thu, 7 May 2026 at 21:22, Ackerley Tng via B4 Relay
<devnull+ackerleytng.google.com@kernel.org> wrote:
>
> From: Ackerley Tng <ackerleytng@google.com>
>
> Before conversion, the range filter doesn't really matter:
>
> + For non-CoCo VMs that use guest_memfd, they have no mirrored tdp, so
> KVM_DIRECT_ROOTS would have been invalidated anyway.
> + CoCo VMs could not use INIT_SHARED, and there's no conversion support, so
> always using KVM_FILTER_PRIVATE would have worked.
>
> Now with conversion support, update kvm_gmem_get_invalidate_filter to
> inspect the memory attributes maple tree for a given range.
>
> Instead of determining the invalidation filter based on static inode
> flags, iterate through the attributes maple tree for the specific range
> being invalidated. This allows KVM to identify if the range contains
> private pages, shared pages, or both, and set the filter bits
> accordingly.
>
> Update kvm_gmem_invalidate_begin and kvm_gmem_release to pass the range
> parameters to the filter helper to ensure invalidation accurately
> targets the memory types present in the affected range.
>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Cheers,
/fuad
> ---
> virt/kvm/guest_memfd.c | 27 ++++++++++++++++++++-------
> 1 file changed, 20 insertions(+), 7 deletions(-)
>
> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 9f6eebfb68f6b..c9f155c2dc5c5 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
> @@ -193,12 +193,24 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
> return folio;
> }
>
> -static enum kvm_gfn_range_filter kvm_gmem_get_invalidate_filter(struct inode *inode)
> +static enum kvm_gfn_range_filter kvm_gmem_get_invalidate_filter(
> + struct inode *inode, pgoff_t start, pgoff_t end)
> {
> - if (GMEM_I(inode)->flags & GUEST_MEMFD_FLAG_INIT_SHARED)
> - return KVM_FILTER_SHARED;
> + struct gmem_inode *gi = GMEM_I(inode);
> + enum kvm_gfn_range_filter filter = 0;
> + void *entry;
> +
> + lockdep_assert(mt_lock_is_held(&gi->attributes));
> +
> + mt_for_each(&gi->attributes, entry, start, end - 1) {
> + filter |= (xa_to_value(entry) & KVM_MEMORY_ATTRIBUTE_PRIVATE) ?
> + KVM_FILTER_PRIVATE : KVM_FILTER_SHARED;
> +
> + if (filter == (KVM_FILTER_PRIVATE | KVM_FILTER_SHARED))
> + break;
> + }
>
> - return KVM_FILTER_PRIVATE;
> + return filter;
> }
>
> static void __kvm_gmem_invalidate_begin(struct gmem_file *f, pgoff_t start,
> @@ -244,7 +256,7 @@ static void kvm_gmem_invalidate_begin(struct inode *inode, pgoff_t start,
> enum kvm_gfn_range_filter attr_filter;
> struct gmem_file *f;
>
> - attr_filter = kvm_gmem_get_invalidate_filter(inode);
> + attr_filter = kvm_gmem_get_invalidate_filter(inode, start, end);
>
> kvm_gmem_for_each_file(f, inode)
> __kvm_gmem_invalidate_begin(f, start, end, attr_filter);
> @@ -367,6 +379,7 @@ static long kvm_gmem_fallocate(struct file *file, int mode, loff_t offset,
> static int kvm_gmem_release(struct inode *inode, struct file *file)
> {
> struct gmem_file *f = file->private_data;
> + enum kvm_gfn_range_filter filter;
> struct kvm_memory_slot *slot;
> struct kvm *kvm = f->kvm;
> unsigned long index;
> @@ -398,8 +411,8 @@ static int kvm_gmem_release(struct inode *inode, struct file *file)
> * memory, as its lifetime is associated with the inode, not the file.
> */
> end = i_size_read(inode) >> PAGE_SHIFT;
> - __kvm_gmem_invalidate_begin(f, 0, end,
> - kvm_gmem_get_invalidate_filter(inode));
> + filter = kvm_gmem_get_invalidate_filter(inode, 0, end);
> + __kvm_gmem_invalidate_begin(f, 0, end, filter);
> __kvm_gmem_invalidate_end(f, 0, end);
>
> list_del(&f->entry);
>
> --
> 2.54.0.563.g4f69b47b94-goog
>
>
^ permalink raw reply
* Re: [PATCH bpf-next v11 6/8] bpf: Add bpf_list_add to insert node after a given list node
From: Kaitao Cheng @ 2026-05-21 7:35 UTC (permalink / raw)
To: bot+bpf-ci
Cc: bpf, ast, corbet, martin.lau, daniel, andrii, eddyz87, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa, shuah,
chengkaitao, skhan, memxor, linux-kernel, linux-doc, vmalik,
linux-kselftest, martin.lau, clm, ihor.solodrai
In-Reply-To: <61871735d9151729b3092d42209dad08b1e38fa930bc1c9e452645240efbc40e@mail.kernel.org>
在 2026/5/21 12:08, bot+bpf-ci@kernel.org 写道:
>> diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
>> index 1c69476c8a09..89579165ef4d 100644
>> --- a/kernel/bpf/helpers.c
>> +++ b/kernel/bpf/helpers.c
>> @@ -2577,6 +2577,16 @@ __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
>> return bpf_list_push_back(head, node, meta__ign, off);
>> }
>>
>> +__bpf_kfunc int bpf_list_add(struct bpf_list_head *head, struct bpf_list_node *new,
>> + struct bpf_list_node *prev__nonown_allowed,
>> + struct btf_struct_meta *meta, u64 off)
>> +{
>> + struct bpf_list_node_kern *n = (void *)new, *p = (void *)prev__nonown_allowed;
>> + struct list_head *prev_ptr = &p->list_head;
>> +
>> + return __bpf_list_add(n, head, &prev_ptr, meta ? meta->record : NULL, off);
>> +}
>
> If __bpf_list_add() fails at runtime (e.g., because prev is not in the
> list indicated by head), it drops the 'new' node via __bpf_obj_drop_impl
> in the fail path.
>
> However, the verifier unconditionally converts 'new' into a non-owning
> reference during check_kfunc_call(). If __bpf_list_add() fails, the BPF
> program retains this non-owning reference to a freed node.
>
> Could this allow the program to trigger a use-after-free within the lock
> critical section?
>
> (sashiko-bot@kernel.org, v10 RESEND)
> https://lore.kernel.org/bpf/20260513204420.2BB22C19425@smtp.kernel.org/
This should be an acceptable case within the current design framework.
>> static struct bpf_list_node *__bpf_list_del(struct bpf_list_head *head,
>> struct list_head *n)
>
> [ ... ]
>
>> @@ -4756,6 +4766,7 @@ BTF_ID_FLAGS(func, bpf_list_push_front, KF_IMPLICIT_ARGS)
>> BTF_ID_FLAGS(func, bpf_list_push_front_impl)
>> BTF_ID_FLAGS(func, bpf_list_push_back, KF_IMPLICIT_ARGS)
>> BTF_ID_FLAGS(func, bpf_list_push_back_impl)
>> +BTF_ID_FLAGS(func, bpf_list_add, KF_IMPLICIT_ARGS)
>
> Since bpf_list_add is marked with KF_IMPLICIT_ARGS, won't the verifier
> look for a bpf_list_add_impl counterpart during program load?
>
> Without defining and registering a bpf_list_add_impl function,
> find_kfunc_impl_proto() will fail to find the required _impl suffix in
> the vmlinux BTF.
>
> Will this cause the verifier to unconditionally reject the kfunc?
>
> (sashiko-bot@kernel.org, v10 RESEND)
> https://lore.kernel.org/bpf/20260513204420.2BB22C19425@smtp.kernel.org/
The target prototype with the _impl suffix has already been deprecated.
>> BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL)
>> BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)
>> BTF_ID_FLAGS(func, bpf_list_del, KF_ACQUIRE | KF_RET_NULL)
>
> [ ... ]
>
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index 35eebb5e7769..662ad7312697 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
>
> [ ... ]
>
>> @@ -19524,8 +19527,11 @@ int bpf_fixup_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
>> int struct_meta_reg = BPF_REG_3;
>> int node_offset_reg = BPF_REG_4;
>>
>> - /* rbtree_add has extra 'less' arg, so args-to-fixup are in diff regs */
>> - if (is_bpf_rbtree_add_kfunc(desc->func_id)) {
>> + /* list_add/rbtree_add have an extra arg (prev/less),
>> + * so args-to-fixup are in diff regs.
>> + */
>
> This isn't a bug, but the multi-line comment format doesn't follow the
> BPF subsystem style. The preferred form in kernel/bpf/ has the opening
> /* on its own line.
>
> Additionally, consider adding an is_bpf_list_add_kfunc() helper instead
> of direct comparison with special_kfunc_list[KF_bpf_list_add].
>
> (bot+bpf-ci@kernel.org and Eduard Zingerman, v10 RESEND)
> https://lore.kernel.org/bpf/f573f7e0a5140fe6254d2ddfd3f69add455fa25c67c020c0312e2c2ceeca7c69@mail.kernel.org/
I provided a more detailed explanation in the link below.
https://lore.kernel.org/all/9e300669-8793-41a2-84a1-98fb8195e263@linux.dev/
>> + if (desc->func_id == special_kfunc_list[KF_bpf_list_add] ||
>> + is_bpf_rbtree_add_kfunc(desc->func_id)) {
>> struct_meta_reg = BPF_REG_4;
>> node_offset_reg = BPF_REG_5;
>> }
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/26204125015
--
Thanks
Kaitao Cheng
^ permalink raw reply
* Re: [PATCH v6 16/43] KVM: guest_memfd: Use actual size for invalidation in kvm_gmem_release()
From: Fuad Tabba @ 2026-05-21 7:30 UTC (permalink / raw)
To: ackerleytng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <20260507-gmem-inplace-conversion-v6-16-91ab5a8b19a4@google.com>
Hi Ackerley,
On Thu, 7 May 2026 at 21:22, Ackerley Tng via B4 Relay
<devnull+ackerleytng.google.com@kernel.org> wrote:
>
> From: Ackerley Tng <ackerleytng@google.com>
>
> __kvm_gmem_invalidate_begin() and __kvm_gmem_invalidate_end() actually do
> not specially handle -1ul. -1ul is used as a huge number, which legal
> indices do not exceed, and hence the invalidation works as expected.
>
> Since a later patch is going to make use of the exact range, calculate the
> size of the guest_memfd inode and use it as the end range for invalidating
> SPTEs.
>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Want to look at what Sashiko has to say? Seems to be a real issue:
https://sashiko.dev/#/patchset/20260507-gmem-inplace-conversion-v6-0-91ab5a8b19a4%40google.com?part=16
If I understand correctly, the fix should simple: use
check_add_overflow() to validate the offset and size parameters in
kvm_gmem_bind()
int kvm_gmem_bind(struct kvm *kvm, struct kvm_memory_slot *slot,
unsigned int fd, loff_t offset)
{
loff_t size = slot->npages << PAGE_SHIFT;
+ loff_t end;
unsigned long start, end_index;
struct gmem_file *f;
...
- if (offset < 0 || !PAGE_ALIGNED(offset) ||
- offset + size > i_size_read(inode))
+ if (offset < 0 || !PAGE_ALIGNED(offset) ||
+ check_add_overflow(offset, size, &end) ||
+ end > i_size_read(inode))
goto err;
What do you think?
/fuad
> ---
> virt/kvm/guest_memfd.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 050a8c092b1a3..9f6eebfb68f6b 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
> @@ -370,6 +370,7 @@ static int kvm_gmem_release(struct inode *inode, struct file *file)
> struct kvm_memory_slot *slot;
> struct kvm *kvm = f->kvm;
> unsigned long index;
> + pgoff_t end;
>
> /*
> * Prevent concurrent attempts to *unbind* a memslot. This is the last
> @@ -396,9 +397,10 @@ static int kvm_gmem_release(struct inode *inode, struct file *file)
> * Zap all SPTEs pointed at by this file. Do not free the backing
> * memory, as its lifetime is associated with the inode, not the file.
> */
> - __kvm_gmem_invalidate_begin(f, 0, -1ul,
> + end = i_size_read(inode) >> PAGE_SHIFT;
> + __kvm_gmem_invalidate_begin(f, 0, end,
> kvm_gmem_get_invalidate_filter(inode));
> - __kvm_gmem_invalidate_end(f, 0, -1ul);
> + __kvm_gmem_invalidate_end(f, 0, end);
>
> list_del(&f->entry);
>
>
> --
> 2.54.0.563.g4f69b47b94-goog
>
>
^ permalink raw reply
* [PATCH net-next 2/3] devlink: Add eswitch mode boot defaults
From: Tariq Toukan @ 2026-05-21 7:24 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Jonathan Corbet, Shuah Khan, Jiri Pirko, Simon Horman,
Saeed Mahameed, Leon Romanovsky, Tariq Toukan, Mark Bloch,
Borislav Petkov (AMD), Andrew Morton, Randy Dunlap,
Thomas Gleixner, Petr Mladek, Peter Zijlstra (Intel), Tejun Heo,
Vlastimil Babka, Feng Tang, Christian Brauner, Dave Hansen,
Dapeng Mi, Kees Cook, Marco Elver, Li RongQing, Eric Biggers,
Paul E. McKenney, linux-doc, linux-kernel, netdev, linux-rdma,
Gal Pressman, Dragos Tatulea, Jiri Pirko
In-Reply-To: <20260521072434.362624-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
Add devlink_eswitch_mode= command line support for setting an eswitch
mode during device initialization.
The supported syntax selects either all devlink handles or one explicit
comma-separated handle list:
devlink_eswitch_mode=[*]:<mode>
devlink_eswitch_mode=[<handle>[,<handle>...]]:<mode>
where <mode> is one of legacy, switchdev or switchdev_inactive. All
selected handles receive the same mode. Assigning different modes to
different handle lists in the same parameter value is not supported.
The default is applied through the existing eswitch_mode_set() devlink
operation, matching the userspace devlink eswitch set command.
Expose devl_apply_default_esw_mode() so drivers can apply the default at
the point where their devlink instance and eswitch operations are ready.
Document the devlink_eswitch_mode= syntax and duplicate handle handling.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../admin-guide/kernel-parameters.txt | 25 ++
.../networking/devlink/devlink-defaults.rst | 80 ++++++
Documentation/networking/devlink/index.rst | 1 +
include/net/devlink.h | 1 +
net/devlink/core.c | 255 ++++++++++++++++++
5 files changed, 362 insertions(+)
create mode 100644 Documentation/networking/devlink/devlink-defaults.rst
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 7834ee927310..f87ae561c0dc 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1278,6 +1278,31 @@ Kernel parameters
dell_smm_hwmon.fan_max=
[HW] Maximum configurable fan speed.
+ devlink_eswitch_mode=
+ [NET]
+ Format:
+ [<selector>]:<mode>
+
+ <selector>:
+ * | <handle>[,<handle>...]
+
+ <handle>:
+ <bus-name>/<dev-name>
+
+ Configure default devlink eswitch mode for matching
+ devlink instances during device initialization.
+
+ <mode>:
+ legacy | switchdev | switchdev_inactive
+
+ Examples:
+ devlink_eswitch_mode=[*]:switchdev
+ devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+ devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+ See Documentation/networking/devlink/devlink-defaults.rst
+ for the full syntax.
+
dfltcc= [HW,S390]
Format: { on | off | def_only | inf_only | always }
on: s390 zlib hardware support for compression on
diff --git a/Documentation/networking/devlink/devlink-defaults.rst b/Documentation/networking/devlink/devlink-defaults.rst
new file mode 100644
index 000000000000..b554e75eeeea
--- /dev/null
+++ b/Documentation/networking/devlink/devlink-defaults.rst
@@ -0,0 +1,80 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==============================
+Devlink Eswitch Mode Defaults
+==============================
+
+Devlink eswitch mode defaults allow the eswitch mode to be provided on the
+kernel command line and applied to matching devlink instances during device
+initialization.
+
+The devlink device is selected by its devlink handle. For PCI devices this is
+the same handle shown by ``devlink dev show``, for example
+``pci/0000:08:00.0``.
+
+Kernel command line syntax
+==========================
+
+Defaults are specified with the ``devlink_eswitch_mode=`` kernel command line
+parameter.
+
+The general syntax is::
+
+ devlink_eswitch_mode=[<selector>]:<mode>
+
+``<selector>`` is either ``*`` or one or more devlink handles::
+
+ * | <bus-name>/<dev-name>[,<bus-name>/<dev-name>...]
+
+``*`` applies the mode to every devlink instance. All handles in the same
+``[]`` list receive the same eswitch mode.
+
+``<mode>`` is one of ``legacy``, ``switchdev`` or ``switchdev_inactive``.
+
+Syntax rules
+------------
+
+The following syntax rules apply:
+
+* Specify the default in one ``devlink_eswitch_mode=`` parameter. Repeated
+ ``devlink_eswitch_mode=`` parameters are not accumulated.
+* The ``devlink_eswitch_mode=`` value is limited by the kernel command line
+ size.
+* Whitespace is not allowed within the parameter value.
+* ``<selector>`` must be either ``*`` or a handle list. ``*`` cannot be
+ combined with explicit handles.
+* ``<bus-name>`` and ``<dev-name>`` must not be empty.
+* ``<bus-name>`` must not contain ``:``.
+* ``<dev-name>`` may contain ``:``. This allows PCI names such as
+ ``0000:08:00.0``.
+* Handles must not contain whitespace, ``[``, ``]``, ``*`` or more than one
+ ``/``.
+* A comma inside ``[]`` separates handles.
+* Comma-separated default groups are not supported.
+* Duplicate handles are rejected and the devlink eswitch mode default is
+ ignored.
+
+The eswitch mode default corresponds to the userspace command::
+
+ devlink dev eswitch set <handle> mode <value>
+
+
+Examples
+========
+
+Set all devlink instances to switchdev mode::
+
+ devlink_eswitch_mode=[*]:switchdev
+
+Set one PCI devlink instance to switchdev mode::
+
+ devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev
+
+Set two PCI devlink instances to legacy mode::
+
+ devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:legacy
+
+The following is invalid because comma-separated default groups are not
+supported::
+
+ devlink_eswitch_mode=[pci/0000:08:00.0]:switchdev,[pci/0000:09:00.0]:switchdev_inactive
diff --git a/Documentation/networking/devlink/index.rst b/Documentation/networking/devlink/index.rst
index f7ba7dcf477d..0d27a7008b14 100644
--- a/Documentation/networking/devlink/index.rst
+++ b/Documentation/networking/devlink/index.rst
@@ -56,6 +56,7 @@ general.
:maxdepth: 1
devlink-dpipe
+ devlink-defaults
devlink-eswitch-attr
devlink-flash
devlink-health
diff --git a/include/net/devlink.h b/include/net/devlink.h
index bcd31de1f890..98885f7c6c10 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -1622,6 +1622,7 @@ int devl_trylock(struct devlink *devlink);
void devl_unlock(struct devlink *devlink);
void devl_assert_locked(struct devlink *devlink);
bool devl_lock_is_held(struct devlink *devlink);
+int devl_apply_default_esw_mode(struct devlink *devlink);
DEFINE_GUARD(devl, struct devlink *, devl_lock(_T), devl_unlock(_T));
struct ib_device;
diff --git a/net/devlink/core.c b/net/devlink/core.c
index eeb6a71f5f56..4bc1734878d1 100644
--- a/net/devlink/core.c
+++ b/net/devlink/core.c
@@ -4,6 +4,10 @@
* Copyright (c) 2016 Jiri Pirko <jiri@mellanox.com>
*/
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/slab.h>
+#include <linux/string.h>
#include <net/genetlink.h>
#define CREATE_TRACE_POINTS
#include <trace/events/devlink.h>
@@ -16,6 +20,233 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(devlink_trap_report);
DEFINE_XARRAY_FLAGS(devlinks, XA_FLAGS_ALLOC);
+static char *devlink_default_esw_mode_param;
+static bool devlink_default_esw_mode_match_all;
+static enum devlink_eswitch_mode devlink_default_esw_mode;
+static LIST_HEAD(devlink_default_esw_mode_nodes);
+
+struct devlink_default_esw_mode_node {
+ struct list_head list;
+ char *bus_name;
+ char *dev_name;
+};
+
+static int __init
+devlink_default_esw_mode_to_value(const char *str,
+ enum devlink_eswitch_mode *mode)
+{
+ if (!strcmp(str, "legacy")) {
+ *mode = DEVLINK_ESWITCH_MODE_LEGACY;
+ return 0;
+ }
+ if (!strcmp(str, "switchdev")) {
+ *mode = DEVLINK_ESWITCH_MODE_SWITCHDEV;
+ return 0;
+ }
+ if (!strcmp(str, "switchdev_inactive")) {
+ *mode = DEVLINK_ESWITCH_MODE_SWITCHDEV_INACTIVE;
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static int devlink_default_esw_mode_apply(struct devlink *devlink)
+{
+ const struct devlink_ops *ops = devlink->ops;
+
+ if (!ops->eswitch_mode_set)
+ return -EOPNOTSUPP;
+
+ return ops->eswitch_mode_set(devlink, devlink_default_esw_mode,
+ NULL);
+}
+
+static int __init
+devlink_default_esw_mode_handle_parse(char *handle, char **bus_name,
+ char **dev_name)
+{
+ char *slash;
+ char *p;
+
+ if (!handle || !*handle)
+ return -EINVAL;
+
+ for (p = handle; *p; p++) {
+ if (*p == '[' || *p == ']' || *p == '*')
+ return -EINVAL;
+ }
+
+ slash = strchr(handle, '/');
+ if (!slash || slash == handle || !slash[1])
+ return -EINVAL;
+ if (strchr(slash + 1, '/'))
+ return -EINVAL;
+
+ *slash = '\0';
+ if (strchr(handle, ':'))
+ return -EINVAL;
+
+ *bus_name = handle;
+ *dev_name = slash + 1;
+ return 0;
+}
+
+static struct devlink_default_esw_mode_node *
+devlink_default_esw_mode_node_find(const char *bus_name, const char *dev_name)
+{
+ struct devlink_default_esw_mode_node *node;
+
+ list_for_each_entry(node, &devlink_default_esw_mode_nodes, list) {
+ if (!strcmp(node->bus_name, bus_name) &&
+ !strcmp(node->dev_name, dev_name))
+ return node;
+ }
+
+ return NULL;
+}
+
+static int __init
+devlink_default_esw_mode_node_add(const char *bus_name, const char *dev_name)
+{
+ struct devlink_default_esw_mode_node *node;
+
+ if (devlink_default_esw_mode_node_find(bus_name, dev_name))
+ return -EEXIST;
+
+ node = kzalloc_obj(*node);
+ if (!node)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&node->list);
+ node->bus_name = kstrdup(bus_name, GFP_KERNEL);
+ node->dev_name = kstrdup(dev_name, GFP_KERNEL);
+ if (!node->bus_name || !node->dev_name) {
+ kfree(node->bus_name);
+ kfree(node->dev_name);
+ kfree(node);
+ return -ENOMEM;
+ }
+
+ list_add_tail(&node->list, &devlink_default_esw_mode_nodes);
+ return 0;
+}
+
+static int __init devlink_default_esw_mode_handles_parse(char *handles)
+{
+ char *handle;
+ int err;
+
+ if (!strcmp(handles, "*")) {
+ devlink_default_esw_mode_match_all = true;
+ return 0;
+ }
+
+ while ((handle = strsep(&handles, ",")) != NULL) {
+ char *bus_name;
+ char *dev_name;
+
+ err = devlink_default_esw_mode_handle_parse(handle, &bus_name,
+ &dev_name);
+ if (err)
+ return err;
+
+ err = devlink_default_esw_mode_node_add(bus_name, dev_name);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static void __init
+devlink_default_esw_mode_node_free(struct devlink_default_esw_mode_node *node)
+{
+ kfree(node->bus_name);
+ kfree(node->dev_name);
+ kfree(node);
+}
+
+static void __init devlink_default_esw_mode_nodes_clear(void)
+{
+ struct devlink_default_esw_mode_node *node;
+ struct devlink_default_esw_mode_node *node_tmp;
+
+ list_for_each_entry_safe(node, node_tmp,
+ &devlink_default_esw_mode_nodes, list) {
+ list_del(&node->list);
+ devlink_default_esw_mode_node_free(node);
+ }
+
+ devlink_default_esw_mode_match_all = false;
+}
+
+static int __init devlink_default_esw_mode_parse(char *str)
+{
+ char *handles_end;
+ char *handles;
+ char *mode;
+ int err;
+
+ if (!str || *str != '[')
+ return -EINVAL;
+
+ handles = str + 1;
+ handles_end = strchr(handles, ']');
+ if (!handles_end || handles_end[1] != ':' || !handles_end[2])
+ return -EINVAL;
+
+ *handles_end = '\0';
+ mode = handles_end + 2;
+ if (!*handles)
+ return -EINVAL;
+
+ err = devlink_default_esw_mode_to_value(mode,
+ &devlink_default_esw_mode);
+ if (err)
+ return err;
+
+ err = devlink_default_esw_mode_handles_parse(handles);
+ if (err)
+ devlink_default_esw_mode_nodes_clear();
+
+ return err;
+}
+
+/**
+ * devl_apply_default_esw_mode - Apply default eswitch mode to devlink instance
+ * @devlink: devlink
+ *
+ * The caller must hold the devlink instance lock.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int devl_apply_default_esw_mode(struct devlink *devlink)
+{
+ const char *bus_name = devlink_bus_name(devlink);
+ const char *dev_name = devlink_dev_name(devlink);
+ struct devlink_default_esw_mode_node *node;
+
+ devl_assert_locked(devlink);
+
+ if (devlink_default_esw_mode_match_all)
+ return devlink_default_esw_mode_apply(devlink);
+
+ node = devlink_default_esw_mode_node_find(bus_name, dev_name);
+ if (node)
+ return devlink_default_esw_mode_apply(devlink);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(devl_apply_default_esw_mode);
+
+static int __init devlink_default_esw_mode_setup(char *str)
+{
+ devlink_default_esw_mode_param = str;
+ return 1;
+}
+__setup("devlink_eswitch_mode=", devlink_default_esw_mode_setup);
+
static struct devlink *devlinks_xa_get(unsigned long index)
{
struct devlink *devlink;
@@ -578,6 +809,27 @@ static int __init devlink_init(void)
{
int err;
+ if (devlink_default_esw_mode_param) {
+ char *def;
+
+ def = kstrdup(devlink_default_esw_mode_param, GFP_KERNEL);
+ if (!def) {
+ err = -ENOMEM;
+ goto out;
+ }
+ err = devlink_default_esw_mode_parse(def);
+ kfree(def);
+ if (err == -EEXIST) {
+ devlink_default_esw_mode_param = NULL;
+ pr_warn("devlink: duplicate eswitch mode handles ignored\n");
+ } else if (err == -EINVAL) {
+ devlink_default_esw_mode_param = NULL;
+ pr_warn("devlink: invalid devlink_eswitch_mode parameter ignored\n");
+ } else if (err) {
+ goto out;
+ }
+ }
+
err = register_pernet_subsys(&devlink_pernet_ops);
if (err)
goto out;
@@ -593,7 +845,10 @@ static int __init devlink_init(void)
out_unreg_pernet_subsys:
unregister_pernet_subsys(&devlink_pernet_ops);
out:
+ if (err)
+ devlink_default_esw_mode_nodes_clear();
WARN_ON(err);
+
return err;
}
--
2.44.0
^ permalink raw reply related
* [PATCH net-next 3/3] net/mlx5: Apply devlink default eswitch mode during init
From: Tariq Toukan @ 2026-05-21 7:24 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Jonathan Corbet, Shuah Khan, Jiri Pirko, Simon Horman,
Saeed Mahameed, Leon Romanovsky, Tariq Toukan, Mark Bloch,
Borislav Petkov (AMD), Andrew Morton, Randy Dunlap,
Thomas Gleixner, Petr Mladek, Peter Zijlstra (Intel), Tejun Heo,
Vlastimil Babka, Feng Tang, Christian Brauner, Dave Hansen,
Dapeng Mi, Kees Cook, Marco Elver, Li RongQing, Eric Biggers,
Paul E. McKenney, linux-doc, linux-kernel, netdev, linux-rdma,
Gal Pressman, Dragos Tatulea, Jiri Pirko, Shay Drori,
Moshe Shemesh
In-Reply-To: <20260521072434.362624-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
Apply devlink default eswitch mode for mlx5 devices after successful
device initialization while holding the devlink instance lock.
At this point the devlink instance is registered and the mlx5 devlink
operations are available, so the default eswitch mode can be applied to
the matching PCI devlink handle.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Reviewed-by: Shay Drori <shayd@nvidia.com>
Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/main.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 0c6e4efe38c8..4528097f3d84 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -1391,6 +1391,21 @@ static void mlx5_unload(struct mlx5_core_dev *dev)
mlx5_free_bfreg(dev, &dev->priv.bfreg);
}
+static void mlx5_devl_apply_default_esw_mode(struct mlx5_core_dev *dev)
+{
+ struct devlink *devlink = priv_to_devlink(dev);
+ int err;
+
+ if (!MLX5_ESWITCH_MANAGER(dev))
+ return;
+
+ devl_assert_locked(devlink);
+ err = devl_apply_default_esw_mode(devlink);
+ if (err)
+ mlx5_core_warn(dev, "Couldn't apply default eswitch mode, err %d\n",
+ err);
+}
+
int mlx5_init_one_devl_locked(struct mlx5_core_dev *dev)
{
bool light_probe = mlx5_dev_is_lightweight(dev);
@@ -1437,6 +1452,7 @@ int mlx5_init_one_devl_locked(struct mlx5_core_dev *dev)
mlx5_core_err(dev, "mlx5_hwmon_dev_register failed with error code %d\n", err);
mutex_unlock(&dev->intf_state_mutex);
+ mlx5_devl_apply_default_esw_mode(dev);
return 0;
err_register:
@@ -1538,6 +1554,7 @@ int mlx5_load_one_devl_locked(struct mlx5_core_dev *dev, bool recovery)
goto err_attach;
mutex_unlock(&dev->intf_state_mutex);
+ mlx5_devl_apply_default_esw_mode(dev);
return 0;
err_attach:
--
2.44.0
^ permalink raw reply related
* [PATCH net-next 1/3] net/mlx5: Clear FW reset-in-progress bit before reload
From: Tariq Toukan @ 2026-05-21 7:24 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Jonathan Corbet, Shuah Khan, Jiri Pirko, Simon Horman,
Saeed Mahameed, Leon Romanovsky, Tariq Toukan, Mark Bloch,
Borislav Petkov (AMD), Andrew Morton, Randy Dunlap,
Thomas Gleixner, Petr Mladek, Peter Zijlstra (Intel), Tejun Heo,
Vlastimil Babka, Feng Tang, Christian Brauner, Dave Hansen,
Dapeng Mi, Kees Cook, Marco Elver, Li RongQing, Eric Biggers,
Paul E. McKenney, linux-doc, linux-kernel, netdev, linux-rdma,
Gal Pressman, Dragos Tatulea, Jiri Pirko, Shay Drori,
Moshe Shemesh
In-Reply-To: <20260521072434.362624-1-tariqt@nvidia.com>
From: Mark Bloch <mbloch@nvidia.com>
mlx5 sets MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS when acknowledging a
sync reset request. This bit blocks devlink reload and other devlink
operations while the firmware reset is running, but it was kept set
until after the driver reload finished.
Clear the reset-in-progress bit once the reset unload flow is done and
PCI access is back, before reloading the device. For a reset initiated
through devlink, clear it before completing the reload waiter. For a
reset reported through an asynchronous firmware event, keep the unload
flow outside devl_lock, then take devl_lock before clearing the bit and
reloading through the devl-locked load helper.
Signed-off-by: Mark Bloch <mbloch@nvidia.com>
Reviewed-by: Shay Drori <shayd@nvidia.com>
Reviewed-by: Moshe Shemesh <moshe@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
.../ethernet/mellanox/mlx5/core/fw_reset.c | 28 +++++++++++--------
1 file changed, 17 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c
index 07440c58713a..7283e5b49eed 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c
@@ -238,24 +238,30 @@ static void mlx5_fw_reset_complete_reload(struct mlx5_core_dev *dev)
{
struct mlx5_fw_reset *fw_reset = dev->priv.fw_reset;
struct devlink *devlink = priv_to_devlink(dev);
+ int err;
/* if this is the driver that initiated the fw reset, devlink completed the reload */
if (test_bit(MLX5_FW_RESET_FLAGS_PENDING_COMP, &fw_reset->reset_flags)) {
+ clear_bit(MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS,
+ &fw_reset->reset_flags);
complete(&fw_reset->done);
- } else {
- mlx5_sync_reset_unload_flow(dev, false);
- if (mlx5_health_wait_pci_up(dev))
- mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n");
- else
- mlx5_load_one(dev, true);
- devl_lock(devlink);
- devlink_remote_reload_actions_performed(devlink, 0,
- BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) |
- BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE));
- devl_unlock(devlink);
+ return;
}
+ mlx5_sync_reset_unload_flow(dev, false);
+ err = mlx5_health_wait_pci_up(dev);
+
+ devl_lock(devlink);
clear_bit(MLX5_FW_RESET_FLAGS_RESET_IN_PROGRESS, &fw_reset->reset_flags);
+ if (err)
+ mlx5_core_err(dev, "reset reload flow aborted, PCI reads still not working\n");
+ else
+ mlx5_load_one_devl_locked(dev, true);
+
+ devlink_remote_reload_actions_performed(devlink, 0,
+ BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT) |
+ BIT(DEVLINK_RELOAD_ACTION_FW_ACTIVATE));
+ devl_unlock(devlink);
}
static void mlx5_stop_sync_reset_poll(struct mlx5_core_dev *dev)
--
2.44.0
^ permalink raw reply related
* [PATCH net-next 0/3] devlink: Add boot-time eswitch mode defaults
From: Tariq Toukan @ 2026-05-21 7:24 UTC (permalink / raw)
To: Eric Dumazet, Jakub Kicinski, Paolo Abeni, Andrew Lunn,
David S. Miller
Cc: Jonathan Corbet, Shuah Khan, Jiri Pirko, Simon Horman,
Saeed Mahameed, Leon Romanovsky, Tariq Toukan, Mark Bloch,
Borislav Petkov (AMD), Andrew Morton, Randy Dunlap,
Thomas Gleixner, Petr Mladek, Peter Zijlstra (Intel), Tejun Heo,
Vlastimil Babka, Feng Tang, Christian Brauner, Dave Hansen,
Dapeng Mi, Kees Cook, Marco Elver, Li RongQing, Eric Biggers,
Paul E. McKenney, linux-doc, linux-kernel, netdev, linux-rdma,
Gal Pressman, Dragos Tatulea, Jiri Pirko
Hi,
See detailed feature description by Mark below.
Regards,
Tariq
This series adds a devlink_eswitch_mode= kernel command line parameter
for applying a default devlink eswitch mode during device
initialization.
Following the discussion with Jakub[1] and the feedback on the RFC
postings, this version implements the direction that was agreed on: keep
the scope limited to a boot-time devlink eswitch mode default only.
The implementation is intended to support the following properties:
- A system may have multiple devlink devices that usually need the same
configuration. For a configuration such as eswitch mode switchdev, a
user can specify either all devlink devices or an explicit list of
devices to which that mode applies.
- Deployments can set the devlink eswitch mode before normal userspace
orchestration runs, while still using devlink concepts and driver
callbacks rather than adding driver-specific module parameters.
A default is scoped to either all devlink handles or to a
comma-separated list of devlink handles, for example:
devlink_eswitch_mode=[*]:switchdev
devlink_eswitch_mode=[pci/0000:08:00.0,pci/0000:09:00.1]:switchdev_inactive
The supported modes are legacy, switchdev and switchdev_inactive.
mlx5 wires this into device initialization after the devlink instance is
registered and after mlx5 devlink operations are available, so eswitch
mode defaults can be applied to matching PCI devlink devices.
Patch 1 clears the mlx5 FW reset-in-progress bit before reloading after
a firmware reset.
Patch 2 adds the devlink eswitch mode boot-default parser, storage,
devl_apply_default_esw_mode() API and documentation for the
devlink_eswitch_mode= syntax.
Patch 3 calls devl_apply_default_esw_mode() from mlx5 device
initialization.
Changelog:
Since RFC v2:
- Replaced the generic devlink=[...]:esw:mode:<mode> command line API
with devlink_eswitch_mode=[...]:<mode>.
- Simplified the parser to handle one eswitch mode parameter instead of
a generic command/attribute grammar.
- Renamed devl_apply_defaults() to devl_apply_default_esw_mode().
- Added the mlx5 firmware reset cleanup as the first patch after the
cover letter.
[1] https://lore.kernel.org/all/20260502184153.4fd8d06f@kernel.org/
RFC V1:
https://lore.kernel.org/all/20260506123739.1959770-1-mbloch@nvidia.com/
RFC V2:
https://lore.kernel.org/all/20260510185424.2041415-1-mbloch@nvidia.com/
Mark Bloch (3):
net/mlx5: Clear FW reset-in-progress bit before reload
devlink: Add eswitch mode boot defaults
net/mlx5: Apply devlink default eswitch mode during init
.../admin-guide/kernel-parameters.txt | 25 ++
.../networking/devlink/devlink-defaults.rst | 80 ++++++
Documentation/networking/devlink/index.rst | 1 +
.../ethernet/mellanox/mlx5/core/fw_reset.c | 28 +-
.../net/ethernet/mellanox/mlx5/core/main.c | 17 ++
include/net/devlink.h | 1 +
net/devlink/core.c | 255 ++++++++++++++++++
7 files changed, 396 insertions(+), 11 deletions(-)
create mode 100644 Documentation/networking/devlink/devlink-defaults.rst
base-commit: 9bf93cb2e180a58d5984ba13daee95903ff4fc14
--
2.44.0
^ permalink raw reply
* Re: [PATCH v6 05/43] KVM: guest_memfd: Wire up kvm_get_memory_attributes() to per-gmem attributes
From: Fuad Tabba @ 2026-05-21 7:19 UTC (permalink / raw)
To: Ackerley Tng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <CAEvNRgGQvMdDmVfbk42EY_PGN0ybTp-x21Zj+pg_X1mk9iCRtA@mail.gmail.com>
On Wed, 20 May 2026 at 22:44, Ackerley Tng <ackerleytng@google.com> wrote:
>
> Fuad Tabba <tabba@google.com> writes:
>
> >
> > [...snip...]
> >
> >> +unsigned long kvm_gmem_get_memory_attributes(struct kvm *kvm, gfn_t gfn)
> >> +{
> >> + struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn);
> >> + struct inode *inode;
> >> +
> >> + /*
> >> + * If this gfn has no associated memslot, there's no chance of the gfn
> >> + * being backed by private memory, since guest_memfd must be used for
> >> + * private memory, and guest_memfd must be associated with some memslot.
> >> + */
> >> + if (!slot)
> >> + return 0;
> >> +
> >> + CLASS(gmem_get_file, file)(slot);
> >> + if (!file)
> >> + return 0;
> >> +
> >> + inode = file_inode(file);
> >> +
> >> + /*
> >> + * Rely on the maple tree's internal RCU lock to ensure a
> >> + * stable result. This result can become stale as soon as the
> >> + * lock is dropped, so the caller _must_ still protect
> >> + * consumption of private vs. shared by checking
> >> + * mmu_invalidate_retry_gfn() under mmu_lock to serialize
> >> + * against ongoing attribute updates.
> >> + */
> >> + return kvm_gmem_get_attributes(inode, kvm_gmem_get_index(slot, gfn));
> >> +}
> >
> > Doesn't this imply that all consumers of kvm_mem_is_private() should
> > validate the result using mmu_lock and the invalidation sequence?
>
> Let me know how I can improve the comment.
Given Sean's context, the comment is good I think. I would quibble
with the the "_must_ still protect" phrasing being a bit too strict.
Maybe just soften it slightly to acknowledge the exception? Something like:
* lock is dropped, so callers that require a strict result _must_ protect
* consumption of private vs. shared by checking mmu_invalidate_retry_gfn()
* under mmu_lock to serialize against ongoing attribute updates. Callers
* doing lockless reads must be able to tolerate a stale result.
That aligns the comment with how KVM is actually using it today. That
said, this is nitpicking. Feel free to use or ignore.
>
> I think the "consumption" of private vs shared here actually means
> something like "don't commit a page being faulted into page tables based
> on the result of kvm_gmem_get_memory_attributes() without checking
> kvm->mmu_invalidate_in_progress.", since a racing conversion may
> complete before you commit.
>
> kvm_mem_is_private() is used from these places:
>
> 1. Fault handling in KVM, like page_fault_can_be_fast(),
> kvm_mmu_faultin_pfn(), kvm_mmu_page_fault(): this already handles the
> entire mmu_lock and invalidation dance. No fault will be committed if
> a racing conversion happened after kvm_mem_is_private() but before
> the commit.
>
> 2. kvm_mmu_max_mapping_level() from recovering huge pages after
> disabling dirty logging: Other than that it can't be used with
> guest_memfd now since dirty logging can't be used with guest_memfd
> and guest_memfd memslots are not updatable, this holds mmu_lock
> throughout until the huge page recovery is done. invalidate_begin
> also involves zapping the pages in the range, so if the order of
> events is
>
> | Thread A | Thread B |
> |------------------------------|-------------------|
> | invalidate_begin + zap | |
> | update attributes maple_tree | recover huge page |
> | invalidate_end | |
>
> Then recovering will never see the zapped pages, nothing to
> recover, no kvm_mem_is_private() lookup.
>
> 3. kvm_arch_vcpu_pre_fault_memory()
>
> This eventually calls kvm_tdp_mmu_page_fault(), which checks
> is_page_fault_stale(), so it does check before committing.
>
> Were there any other calls I missed?
The one I was looking at was `sev_handle_rmp_fault()`, which does a lockless
read without the retry loop. But as Sean just pointed out, that path can
tolerate false positives/negatives and relies on the guest faulting again,
so the lack of synchronization there is existing behavior and considered "fine".
>
> > sev_handle_rmp_fault() calls kvm_mem_is_private() without holding
> > mmu_lock and without any retry mechanism. Is that a problem?
> >
>
> Sean already replied on your actual question separately :)
>
> > Cheers,
> > /fuad
> >
> >
> >>
> >> [...snip...]
> >>
^ permalink raw reply
* Re: [PATCH v6 15/43] KVM: guest_memfd: Handle lru_add fbatch refcounts during conversion safety check
From: Fuad Tabba @ 2026-05-21 7:13 UTC (permalink / raw)
To: ackerleytng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <20260507-gmem-inplace-conversion-v6-15-91ab5a8b19a4@google.com>
On Thu, 7 May 2026 at 21:22, Ackerley Tng via B4 Relay
<devnull+ackerleytng.google.com@kernel.org> wrote:
>
> From: Ackerley Tng <ackerleytng@google.com>
>
> When checking if a guest_memfd folio is safe for conversion, its refcount
> is examined. A folio may be present in a per-CPU lru_add fbatch, which
> temporarily increases its refcount. This can lead to a false positive,
> incorrectly indicating that the folio is in use and preventing the
> conversion, even if it is otherwise safe. The conversion process might not
> be on the same CPU that holds the folio in its fbatch, making a simple
> per-CPU check insufficient.
>
> To address this, drain all CPUs' lru_add fbatches if an unexpectedly high
> refcount is encountered during the safety check. This is performed at most
> once per conversion request. Draining only if the folio in question may be
> lru cached.
>
> guest_memfd folios are unevictable, so they can only reside in the lru_add
> fbatch. If the folio's refcount is still unsafe after draining, then the
> conversion is truly deemed unsafe.
>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>
Not an area I've worked with that much, but it seems right to me:
Reviewed-by: Fuad Tabba <tabba@google.com>
Cheers,
/fuad
> ---
> mm/swap.c | 2 ++
> virt/kvm/guest_memfd.c | 18 ++++++++++++++----
> 2 files changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/mm/swap.c b/mm/swap.c
> index 5cc44f0de9877..3134d9d3d7c30 100644
> --- a/mm/swap.c
> +++ b/mm/swap.c
> @@ -37,6 +37,7 @@
> #include <linux/page_idle.h>
> #include <linux/local_lock.h>
> #include <linux/buffer_head.h>
> +#include <linux/kvm_types.h>
>
> #include "internal.h"
>
> @@ -904,6 +905,7 @@ void lru_add_drain_all(void)
> lru_add_drain();
> }
> #endif /* CONFIG_SMP */
> +EXPORT_SYMBOL_FOR_KVM(lru_add_drain_all);
>
> atomic_t lru_disable_count = ATOMIC_INIT(0);
>
> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 034b72b4947fb..050a8c092b1a3 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
> @@ -8,6 +8,7 @@
> #include <linux/mempolicy.h>
> #include <linux/pseudo_fs.h>
> #include <linux/pagemap.h>
> +#include <linux/swap.h>
>
> #include "kvm_mm.h"
>
> @@ -596,18 +597,27 @@ static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start,
> const int filemap_get_folios_refcount = 1;
> pgoff_t last = start + nr_pages - 1;
> struct folio_batch fbatch;
> + bool lru_drained = false;
> bool safe = true;
> int i;
>
> folio_batch_init(&fbatch);
> while (safe && filemap_get_folios(mapping, &start, last, &fbatch)) {
>
> - for (i = 0; i < folio_batch_count(&fbatch); ++i) {
> + for (i = 0; i < folio_batch_count(&fbatch);) {
> struct folio *folio = fbatch.folios[i];
>
> - if (folio_ref_count(folio) !=
> - folio_nr_pages(folio) + filemap_get_folios_refcount) {
> - safe = false;
> + safe = (folio_ref_count(folio) ==
> + folio_nr_pages(folio) +
> + filemap_get_folios_refcount);
> +
> + if (safe) {
> + ++i;
> + } else if (folio_may_be_lru_cached(folio) &&
> + !lru_drained) {
> + lru_add_drain_all();
> + lru_drained = true;
> + } else {
> *err_index = folio->index;
> break;
> }
>
> --
> 2.54.0.563.g4f69b47b94-goog
>
>
^ permalink raw reply
* Re: [PATCH v6 11/43] KVM: guest_memfd: Ensure pages are not in use before conversion
From: Fuad Tabba @ 2026-05-21 7:09 UTC (permalink / raw)
To: ackerleytng
Cc: aik, andrew.jones, binbin.wu, brauner, chao.p.peng, david,
ira.weiny, jmattson, jthoughton, michael.roth, oupton,
pankaj.gupta, qperret, rick.p.edgecombe, rientjes, shivankg,
steven.price, willy, wyihan, yan.y.zhao, forkloop, pratyush,
suzuki.poulose, aneesh.kumar, liam, Paolo Bonzini,
Sean Christopherson, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, x86, H. Peter Anvin, Steven Rostedt,
Masami Hiramatsu, Mathieu Desnoyers, Jonathan Corbet, Shuah Khan,
Shuah Khan, Vishal Annapurve, Andrew Morton, Chris Li,
Kairui Song, Kemeng Shi, Nhat Pham, Baoquan He, Barry Song,
Axel Rasmussen, Yuanchu Xie, Wei Xu, Youngjun Park, Qi Zheng,
Shakeel Butt, Kiryl Shutsemau, Jason Gunthorpe, Vlastimil Babka,
kvm, linux-kernel, linux-trace-kernel, linux-doc, linux-kselftest,
linux-mm, linux-coco
In-Reply-To: <20260507-gmem-inplace-conversion-v6-11-91ab5a8b19a4@google.com>
Hi Ackerley,
On Thu, 7 May 2026 at 21:22, Ackerley Tng via B4 Relay
<devnull+ackerleytng.google.com@kernel.org> wrote:
>
> From: Ackerley Tng <ackerleytng@google.com>
>
> When converting memory to private in guest_memfd, it is necessary to ensure
> that the pages are not currently being accessed by any other part of the
> kernel or userspace to avoid any current user writing to guest private
> memory.
>
> guest_memfd checks for unexpected refcounts to determine whether a page is
> still in use. The only expected refcounts after unmapping the range
> requested for conversion are those that are held by guest_memfd itself.
>
> Update the kvm_memory_attributes2 structure to include an error_offset
> field. This allows KVM to report the exact offset where a conversion
> failed to userspace. If the safety check fails, return -EAGAIN and copy
> the error_offset back to userspace so that it can potentially retry the
> operation or handle the failure gracefully.
>
> Suggested-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: Ackerley Tng <ackerleytng@google.com>
> Co-developed-by: Vishal Annapurve <vannapurve@google.com>
> Signed-off-by: Vishal Annapurve <vannapurve@google.com>
> ---
> include/uapi/linux/kvm.h | 3 ++-
> virt/kvm/guest_memfd.c | 65 ++++++++++++++++++++++++++++++++++++++++++++----
> 2 files changed, 62 insertions(+), 6 deletions(-)
>
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index e6bbf68a83813..0b55258573d3d 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -1658,7 +1658,8 @@ struct kvm_memory_attributes2 {
> __u64 size;
> __u64 attributes;
> __u64 flags;
> - __u64 reserved[12];
> + __u64 error_offset;
> + __u64 reserved[11];
> };
>
> #define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)
> diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
> index 91e89b188f583..9d82642a025e9 100644
> --- a/virt/kvm/guest_memfd.c
> +++ b/virt/kvm/guest_memfd.c
> @@ -572,9 +572,42 @@ static int kvm_gmem_mas_preallocate(struct ma_state *mas, u64 attributes,
> return mas_preallocate(mas, xa_mk_value(attributes), GFP_KERNEL);
> }
>
> +static bool kvm_gmem_is_safe_for_conversion(struct inode *inode, pgoff_t start,
> + size_t nr_pages, pgoff_t *err_index)
> +{
> + struct address_space *mapping = inode->i_mapping;
> + const int filemap_get_folios_refcount = 1;
> + pgoff_t last = start + nr_pages - 1;
> + struct folio_batch fbatch;
> + bool safe = true;
> + int i;
> +
> + folio_batch_init(&fbatch);
> + while (safe && filemap_get_folios(mapping, &start, last, &fbatch)) {
> +
> + for (i = 0; i < folio_batch_count(&fbatch); ++i) {
> + struct folio *folio = fbatch.folios[i];
> +
> + if (folio_ref_count(folio) !=
> + folio_nr_pages(folio) + filemap_get_folios_refcount) {
> + safe = false;
> + *err_index = folio->index;
> + break;
https://sashiko.dev/#/patchset/20260507-gmem-inplace-conversion-v6-0-91ab5a8b19a4%40google.com?part=11
Sashiko raised a few issues here, but I think this one might be
genuine. Can you look into it please?
If that's right, when huge page support lands, if start falls in the
middle of a large folio, returning folio->index as the err_index will
return an offset strictly less than the requested start. A naive
userspace retry loop resuming from error_offset would step backwards
and corrupt attributes on memory it didn't intend to convert.
err_index should be clamped to max(start, folio->index).
Cheers,
/fuad
> + }
> + }
> +
> + folio_batch_release(&fbatch);
> + cond_resched();
> + }
> +
> + return safe;
> +}
> +
> static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
> - size_t nr_pages, uint64_t attrs)
> + size_t nr_pages, uint64_t attrs,
> + pgoff_t *err_index)
> {
> + bool to_private = attrs & KVM_MEMORY_ATTRIBUTE_PRIVATE;
> struct address_space *mapping = inode->i_mapping;
> struct gmem_inode *gi = GMEM_I(inode);
> pgoff_t end = start + nr_pages;
> @@ -588,8 +621,21 @@ static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start,
>
> mas_init(&mas, mt, start);
> r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages);
> - if (r)
> + if (r) {
> + *err_index = start;
> goto out;
> + }
> +
> + if (to_private) {
> + unmap_mapping_pages(mapping, start, nr_pages, false);
> +
> + if (!kvm_gmem_is_safe_for_conversion(inode, start, nr_pages,
> + err_index)) {
> + mas_destroy(&mas);
> + r = -EAGAIN;
> + goto out;
> + }
> + }
>
> /*
> * From this point on guest_memfd has performed necessary
> @@ -609,9 +655,10 @@ static long kvm_gmem_set_attributes(struct file *file, void __user *argp)
> struct gmem_file *f = file->private_data;
> struct inode *inode = file_inode(file);
> struct kvm_memory_attributes2 attrs;
> + pgoff_t err_index;
> size_t nr_pages;
> pgoff_t index;
> - int i;
> + int i, r;
>
> if (copy_from_user(&attrs, argp, sizeof(attrs)))
> return -EFAULT;
> @@ -635,8 +682,16 @@ static long kvm_gmem_set_attributes(struct file *file, void __user *argp)
>
> nr_pages = attrs.size >> PAGE_SHIFT;
> index = attrs.offset >> PAGE_SHIFT;
> - return __kvm_gmem_set_attributes(inode, index, nr_pages,
> - attrs.attributes);
> + r = __kvm_gmem_set_attributes(inode, index, nr_pages, attrs.attributes,
> + &err_index);
> + if (r) {
> + attrs.error_offset = ((uint64_t)err_index) << PAGE_SHIFT;
> +
> + if (copy_to_user(argp, &attrs, sizeof(attrs)))
> + return -EFAULT;
> + }
> +
> + return r;
> }
>
> static long kvm_gmem_ioctl(struct file *file, unsigned int ioctl,
>
> --
> 2.54.0.563.g4f69b47b94-goog
>
>
^ permalink raw reply
* Re: [PATCH bpf-next v11 4/8] bpf: Introduce the bpf_list_del kfunc.
From: Kaitao Cheng @ 2026-05-21 6:59 UTC (permalink / raw)
To: bot+bpf-ci
Cc: bpf, ast, corbet, martin.lau, daniel, andrii, eddyz87, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa, shuah,
chengkaitao, skhan, memxor, linux-kernel, linux-doc, vmalik,
linux-kselftest, martin.lau, clm, ihor.solodrai
In-Reply-To: <d1fb9e986434b92188b3b37514336794c49a807c0378badceda45c3c14bb9739@mail.kernel.org>
在 2026/5/21 12:08, bot+bpf-ci@kernel.org 写道:
>> diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
>> index 59855b434f0b..804c201c28f3 100644
>> --- a/kernel/bpf/helpers.c
>> +++ b/kernel/bpf/helpers.c
>
> [ ... ]
>
>> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
>> index f3cf8d85bea0..35eebb5e7769 100644
>> --- a/kernel/bpf/verifier.c
>> +++ b/kernel/bpf/verifier.c
>
> [ ... ]
>
>> @@ -11671,7 +11674,8 @@ static bool check_kfunc_is_graph_node_api(struct bpf_verifier_env *env,
>>
>> switch (node_field_type) {
>> case BPF_LIST_NODE:
>> - ret = is_bpf_list_push_kfunc(kfunc_btf_id);
>> + ret = is_bpf_list_push_kfunc(kfunc_btf_id) ||
>> + kfunc_btf_id == special_kfunc_list[KF_bpf_list_del];
>
> This isn't a bug, but Eduard Zingerman suggested in v10 adding a
> is_bpf_list_del_kfunc() helper for consistency with the existing code style
> (similar to is_bpf_list_push_kfunc()). This would maintain the pattern used
> throughout the codebase for checking kfunc types.
>
> Reference: https://lore.kernel.org/bpf/ca14a6fd8a1f3ce1e64c70b6d9ea2baf8df913ae.camel@gmail.com/
The reason for using is_bpf_list_push_kfunc() is that it groups multiple parallel
"kfunc_btf_id == special_kfunc_list[*]" checks into one helper, making the code
easier to reuse. If we added is_bpf_list_del_kfunc() here, it would only wrap a
single KF_bpf_list_del check, which is unnecessary. Also, more helpers such as
is_bpf_list_first/last may be added later, and overusing is_bpf_list_* helpers
would make the code more redundant.
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/26204125015
--
Thanks
Kaitao Cheng
^ permalink raw reply
* [PATCH 3/3] Documentation: PCI: Add PCI DMA endpoint function documentation
From: Koichiro Den @ 2026-05-21 6:36 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
In-Reply-To: <20260521063638.2843021-1-den@valinux.co.jp>
Add a function description and a user guide for pci-epf-dma. Describe
the BAR-resident metadata consumed by dw-edma-pcie, the configfs
attributes, endpoint controller requirements and the host-side DMAengine
usage model.
Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
Documentation/PCI/endpoint/index.rst | 2 +
.../PCI/endpoint/pci-dma-function.rst | 182 ++++++++++++++++
Documentation/PCI/endpoint/pci-dma-howto.rst | 200 ++++++++++++++++++
3 files changed, 384 insertions(+)
create mode 100644 Documentation/PCI/endpoint/pci-dma-function.rst
create mode 100644 Documentation/PCI/endpoint/pci-dma-howto.rst
diff --git a/Documentation/PCI/endpoint/index.rst b/Documentation/PCI/endpoint/index.rst
index dd1f62e731c9..cd4107e02ec2 100644
--- a/Documentation/PCI/endpoint/index.rst
+++ b/Documentation/PCI/endpoint/index.rst
@@ -15,6 +15,8 @@ PCI Endpoint Framework
pci-ntb-howto
pci-vntb-function
pci-vntb-howto
+ pci-dma-function
+ pci-dma-howto
pci-nvme-function
function/binding/pci-test
diff --git a/Documentation/PCI/endpoint/pci-dma-function.rst b/Documentation/PCI/endpoint/pci-dma-function.rst
new file mode 100644
index 000000000000..54caf4fafe00
--- /dev/null
+++ b/Documentation/PCI/endpoint/pci-dma-function.rst
@@ -0,0 +1,182 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+================
+PCI DMA Function
+================
+
+:Author: Koichiro Den <den@valinux.co.jp>
+
+The PCI DMA endpoint function exposes an endpoint-integrated DMA controller
+to the PCI host as a PCI DMA controller. A matching host-side driver
+discovers the endpoint DMA metadata and registers the delegated channels with
+the Linux DMAengine framework, so host DMAengine clients can submit
+transfers.
+
+An endpoint Linux system can already use an endpoint-integrated DMA
+controller locally through the normal DMAengine API, for example to transfer
+data between endpoint memory and host addresses reachable over PCI. The PCI
+DMA function provides a different ownership model: it delegates selected
+local DMA channels to the host, so a host DMAengine client can request and
+program those endpoint-side channels through the host's DMAengine API.
+
+To make that possible, the endpoint function publishes the DMA controller
+register window and descriptor memory layout to the host, reserves the
+selected local DMA channels on the endpoint side, and lets the host program
+those channels directly.
+
+Constructs Used for Implementing DMA
+====================================
+
+The PCI DMA function uses the following endpoint-side resources and
+configuration:
+
+ 1) DMA controller register window
+ 2) DMA descriptor memory for endpoint-to-RC channels
+ 3) DMA descriptor memory for RC-to-endpoint channels
+ 4) MSI or MSI-X interrupt vectors selected through configfs
+ 5) One endpoint BAR used to publish metadata
+ 6) If needed, one endpoint BAR used for dynamically mapped DMA windows
+
+The endpoint controller reports the DMA controller register and descriptor
+resources through the endpoint auxiliary resource interface. The PCI DMA
+function uses those descriptions to build the host-visible metadata and to map
+resources that are not already visible to the host.
+
+DMA Controller Register Window:
+-------------------------------
+
+It contains the DMA controller registers programmed by the host-side driver
+to submit transfers, control channels and handle DMA interrupts.
+
+DMA Descriptor Memory:
+----------------------
+
+It contains the descriptor memory used by the DMA controller. The PCI DMA
+function exposes descriptor memory for the delegated endpoint-to-RC and
+RC-to-endpoint channels.
+
+MSI/MSI-X Interrupt Vectors:
+----------------------------
+
+They are used by the delegated DMA channels to signal completion and error
+conditions to the host-side driver.
+
+Metadata BAR:
+-------------
+
+It is the endpoint BAR used to publish the endpoint DMA metadata and handshake
+bits. The BAR remains stable while the endpoint function programs the DMA
+windows.
+
+DMA Window BAR:
+---------------
+
+It is the endpoint BAR used for DMA resources that are not already visible
+through a fixed BAR. The endpoint function may switch this BAR to subrange
+mapping after the host-side driver has found the metadata BAR.
+
+BAR Metadata
+============
+
+The endpoint function places a small metadata block at the beginning of the
+selected metadata BAR. The format is defined in
+``include/linux/pci-ep-dma.h``.
+
+The host-side driver scans the function's assigned memory BARs, looks for the
+endpoint DMA metadata magic, requests DMA window programming, waits for the
+READY bit, and then parses the metadata to find the DMA register window and
+descriptor windows.
+
+::
+
+ +----------------------+ metadata BAR offset 0
+ | endpoint DMA metadata|
+ +----------------------+
+ | optional padding |
+ +----------------------+
+
+ +----------------------+ DMA window BAR offset 0
+ | mapped DMA resources |
+ +----------------------+
+ | optional padding |
+ +----------------------+
+
+The metadata can also reference resources that are already host-visible
+through fixed BARs. For example, an endpoint controller may expose the DMA
+controller register window at a fixed BAR offset while descriptor memories
+are mapped into the DMA window BAR by the endpoint function.
+
+The metadata is BAR-resident instead of a self-contained PCI Vendor-Specific
+Extended Capability (VSEC). Some endpoint controllers do not provide writable
+configuration-space backing storage large enough for a new VSEC payload, while
+they can map endpoint memory and controller resources into a BAR.
+
+Channel Ownership
+=================
+
+The ``wr_chans`` attribute exposes endpoint-to-RC DMA write channels. The
+``rd_chans`` attribute exposes RC-to-endpoint DMA read channels. The function
+reserves the selected endpoint-side DMAengine channels so that endpoint-side
+DMAengine clients cannot allocate and use the same hardware channels while
+they are delegated to the host.
+
+The current metadata revision describes channels in dense, zero-based order.
+For example, ``wr_chans = 2`` exposes write channels 0 and 1. Skipping a
+hardware channel in the middle of the exposed range is not supported.
+
+The current DesignWare eDMA unroll and HDMA compatible support also requires
+each exposed direction to be delegated as a whole. For example, on a controller
+with two write channels, ``wr_chans`` must be either 0 or 2.
+
+Interrupts
+==========
+
+The PCI DMA function exposes DMA interrupts through MSI or MSI-X. The common
+endpoint function ``msi_interrupts`` and ``msix_interrupts`` configfs attributes
+select the interrupt vector counts programmed into endpoint config space. At
+least one MSI or MSI-X vector must be configured before the function is bound
+to an endpoint controller.
+
+Transfer Addressing
+===================
+
+The host-side DMAengine client supplies the endpoint memory address as the
+DMA slave address. For example, the ``dw-edma-pcie`` endpoint DMA metadata
+parser passes that slave address to the DMA controller as a raw endpoint-side
+address instead of translating it through a host PCI BAR resource.
+
+The host memory buffer used as the other side of the transfer is still mapped
+using the normal DMA mapping API on the host.
+
+Endpoint Controller Requirements
+================================
+
+The endpoint controller driver must expose the DMA controller register
+window and per-channel descriptor memories through the endpoint auxiliary
+resource API. Endpoint controllers with other DMA register layouts also need
+matching metadata and host-side DMAengine driver support.
+
+If any DMA resource is not already host-visible through a fixed BAR, the
+endpoint controller must also support BAR subrange mapping and dynamic inbound
+mapping, because the DMA window BAR is assembled from those resources.
+
+Current Support
+===============
+
+The current host-side support is implemented in ``dw-edma-pcie`` for
+DesignWare eDMA unroll and HDMA compatible layouts. Other PCIe controller
+DMA implementations need corresponding host-side DMAengine driver support.
+
+The ``dw-edma-pcie`` PCI ID table does not contain a generic endpoint DMA PCI
+ID entry. Users need to bind the host-side driver explicitly using
+``driver_override``.
+
+The current metadata revision requires the exposed channels to be a dense
+prefix of the hardware channel numbers.
+
+Security Model
+==============
+
+The interface is intended for trusted endpoint/host deployments. A delegated
+DMA channel can access endpoint memory addresses supplied by a host DMAengine
+client.
diff --git a/Documentation/PCI/endpoint/pci-dma-howto.rst b/Documentation/PCI/endpoint/pci-dma-howto.rst
new file mode 100644
index 000000000000..84f322881aa7
--- /dev/null
+++ b/Documentation/PCI/endpoint/pci-dma-howto.rst
@@ -0,0 +1,200 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========================================
+PCI DMA Endpoint Function (EPF) User Guide
+==========================================
+
+:Author: Koichiro Den <den@valinux.co.jp>
+
+This guide shows how to configure the ``pci-epf-dma`` endpoint function driver.
+It uses ``dw-edma-pcie`` as the currently available host-side driver. For the
+hardware model and layout see Documentation/PCI/endpoint/pci-dma-function.rst.
+
+Endpoint Device
+===============
+
+Endpoint Controller Devices
+---------------------------
+
+To find the list of endpoint controller devices in the system::
+
+ # ls /sys/class/pci_epc/
+ e65d0000.pcie-ep
+
+If ``PCI_ENDPOINT_CONFIGFS`` is enabled::
+
+ # ls /sys/kernel/config/pci_ep/controllers
+ e65d0000.pcie-ep
+
+Endpoint Function Drivers
+-------------------------
+
+To find the list of endpoint function drivers in the system::
+
+ # ls /sys/bus/pci-epf/drivers
+ pci_epf_dma pci_epf_test
+
+If ``PCI_ENDPOINT_CONFIGFS`` is enabled::
+
+ # ls /sys/kernel/config/pci_ep/functions
+ pci_epf_dma pci_epf_test
+
+Creating pci-epf-dma Device
+---------------------------
+
+Create a ``pci-epf-dma`` device with configfs::
+
+ # mount -t configfs none /sys/kernel/config
+ # cd /sys/kernel/config/pci_ep/
+ # mkdir functions/pci_epf_dma/dma0
+
+The "mkdir dma0" above creates the ``pci-epf-dma`` function device that will
+be probed by the ``pci_epf_dma`` driver.
+
+The PCI endpoint framework populates the directory with the common
+configurable fields::
+
+ # ls functions/pci_epf_dma/dma0
+ baseclass_code msi_interrupts progif_code subsys_id
+ cache_line_size msix_interrupts revid subsys_vendor_id
+ deviceid pci_epf_dma.0 secondary vendorid
+ interrupt_pin primary subclass_code
+
+The PCI DMA function driver also creates a function-specific sub-directory.
+The numeric suffix depends on the endpoint function instance number::
+
+ # ls functions/pci_epf_dma/dma0/pci_epf_dma.0/
+ dma_window_bar metadata_bar rd_chans wr_chans
+
+Configuring pci-epf-dma Device
+------------------------------
+
+The host-side ``dw-edma-pcie`` PCI ID table does not contain a generic
+endpoint DMA PCI ID entry. Choose a PCI vendor/device ID for the endpoint
+device::
+
+ # echo <vendor-id> > functions/pci_epf_dma/dma0/vendorid
+ # echo <device-id> > functions/pci_epf_dma/dma0/deviceid
+ # echo 1 > functions/pci_epf_dma/dma0/msi_interrupts
+
+The PCI class defaults to ``PCI_BASE_CLASS_SYSTEM`` and
+``PCI_CLASS_SYSTEM_DMA``.
+
+The function-specific attributes are:
+
+============== ============================================================
+Attribute Description
+============== ============================================================
+metadata_bar BAR used to publish the endpoint DMA metadata and handshake
+ bits. It is kept as a stable BAR while the DMA windows are
+ programmed. If this is left unset, the first usable BAR that
+ does not already contain a fixed DMA resource is used.
+dma_window_bar BAR used for DMA resources that are not already host-visible,
+ such as the DMA register window or descriptor windows. This
+ BAR may be switched to subrange mapping after the host driver
+ has found the metadata. If this is left unset and a DMA
+ window is needed, the first usable BAR different from
+ ``metadata_bar`` and not already occupied by a fixed DMA
+ resource is used.
+wr_chans Number of endpoint-to-RC DMA write channels to expose.
+rd_chans Number of RC-to-endpoint DMA read channels to expose.
+============== ============================================================
+
+A sample configuration for a DesignWare eDMA/HDMA compatible controller with
+two write channels and two read channels is given below::
+
+ # echo 0 > functions/pci_epf_dma/dma0/pci_epf_dma.0/metadata_bar
+ # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/dma_window_bar
+ # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/wr_chans
+ # echo 2 > functions/pci_epf_dma/dma0/pci_epf_dma.0/rd_chans
+
+``wr_chans`` and ``rd_chans`` default to 0. At least one channel direction
+must be configured. The selected channels are exposed in dense, zero-based
+order; for example, ``wr_chans = 2`` exposes write channels 0 and 1.
+Current DesignWare eDMA unroll and HDMA compatible support requires each
+exposed direction to be delegated as a whole, so set a direction to either 0 or
+the number of hardware channels in that direction. If ``dma_window_bar`` is
+configured, it must be different from ``metadata_bar``.
+
+The common ``msi_interrupts`` and ``msix_interrupts`` attributes select the
+number of MSI and MSI-X vectors exposed to the host. At least one MSI or
+MSI-X vector must be configured.
+
+The function-specific attributes can only be changed before the endpoint
+function is bound to an endpoint controller.
+
+Binding pci-epf-dma Device to EP Controller
+-------------------------------------------
+
+The DMA function device should be attached to a PCI endpoint controller
+connected to the host::
+
+ # ln -s controllers/e65d0000.pcie-ep \
+ functions/pci_epf_dma/dma0/primary/
+
+Once the above step is completed, the PCI endpoint controller is ready to
+establish a link with the host.
+
+Start the Link
+--------------
+
+Start the endpoint controller by writing 1 to ``start``::
+
+ # echo 1 > controllers/e65d0000.pcie-ep/start
+
+Root Complex Device
+===================
+
+lspci Output
+------------
+
+Note that the device listed here corresponds to the values populated in the
+endpoint configuration above::
+
+ # lspci -nk
+ 01:00.1 0801: <vendor-id>:<device-id>
+
+If the host was already running while the endpoint function was configured,
+rescan the PCI bus after the endpoint side has completed the configfs setup
+and started the endpoint controller, if the platform supports it.
+
+Bind the endpoint DMA function to ``dw-edma-pcie`` explicitly with
+``driver_override``::
+
+ # modprobe dw_edma_pcie
+ # echo dw-edma-pcie > /sys/bus/pci/devices/0000:01:00.1/driver_override
+ # echo 0000:01:00.1 > /sys/bus/pci/drivers_probe
+
+The device should then be bound to ``dw-edma-pcie``::
+
+ # lspci -nk -s 01:00.1
+ 01:00.1 0801: <vendor-id>:<device-id>
+ Kernel driver in use: dw-edma-pcie
+
+Using pci-epf-dma Device
+------------------------
+
+The host side software uses the standard Linux DMAengine API. A DMAengine
+client driver running on the host must request one of the channels provided by
+``dw-edma-pcie`` and submit a transfer.
+
+For an endpoint-to-RC write transfer, the DMAengine client uses a host DMA
+buffer as the destination and an endpoint-side address as the slave source
+address. For an RC-to-endpoint read transfer, the DMAengine client uses a
+host DMA buffer as the source and an endpoint-side address as the slave
+destination address.
+
+Troubleshooting
+===============
+
+``pci-epf-dma`` requires endpoint controller support for DMA auxiliary
+resources and MSI or MSI-X. If any DMA resource must be mapped dynamically,
+the endpoint controller must also support BAR subrange mapping and dynamic
+inbound mapping. Binding the function to an endpoint controller fails if the
+required capabilities are not available, or if both ``msi_interrupts`` and
+``msix_interrupts`` are zero.
+
+If ``dw-edma-pcie`` fails to probe on the host, check that the endpoint was
+bound to the host driver, that the endpoint BARs were assigned by PCI
+enumeration, and that the endpoint DMA metadata READY bit was set after any
+DMA window BAR submaps were programmed.
--
2.51.0
^ permalink raw reply related
* [PATCH 2/3] PCI: endpoint: Add DMA endpoint function
From: Koichiro Den @ 2026-05-21 6:36 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
In-Reply-To: <20260521063638.2843021-1-den@valinux.co.jp>
Add pci-epf-dma, an endpoint function that exposes selected
endpoint-integrated DMA channels as a separate PCI DMA controller
function.
The function consumes EPC auxiliary DMA resources, publishes a stable
metadata BAR for host discovery, and uses a DMA window BAR for DMA
resources that are not already host-visible. After the host-side driver
finds the metadata and requests the final layout, the endpoint function
programs DMA window BAR submaps and marks the metadata ready.
The endpoint function does not bake in a vendor/device ID. As with other
generic endpoint functions, users provide the PCI IDs through the common
EPF configfs header attributes.
Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
drivers/pci/endpoint/functions/Kconfig | 14 +
drivers/pci/endpoint/functions/Makefile | 1 +
drivers/pci/endpoint/functions/pci-epf-dma.c | 1361 ++++++++++++++++++
3 files changed, 1376 insertions(+)
create mode 100644 drivers/pci/endpoint/functions/pci-epf-dma.c
diff --git a/drivers/pci/endpoint/functions/Kconfig b/drivers/pci/endpoint/functions/Kconfig
index bb5a23994288..078ac19dc772 100644
--- a/drivers/pci/endpoint/functions/Kconfig
+++ b/drivers/pci/endpoint/functions/Kconfig
@@ -39,6 +39,20 @@ config PCI_EPF_VNTB
If in doubt, say "N" to disable Endpoint NTB driver.
+config PCI_EPF_DMA
+ tristate "PCI Endpoint DMA driver"
+ depends on PCI_ENDPOINT
+ select CONFIGFS_FS
+ select DMA_ENGINE
+ help
+ Select this configuration option to expose an endpoint-integrated
+ DMA controller as a PCI endpoint function. The function advertises
+ the DMA controller layout to the host using BAR-resident metadata
+ and maps resources that are not already host-visible into the
+ DMA window BAR.
+
+ If in doubt, say "N" to disable Endpoint DMA driver.
+
config PCI_EPF_MHI
tristate "PCI Endpoint driver for MHI bus"
depends on PCI_ENDPOINT && MHI_BUS_EP
diff --git a/drivers/pci/endpoint/functions/Makefile b/drivers/pci/endpoint/functions/Makefile
index 696473fce50e..de92f6897b8f 100644
--- a/drivers/pci/endpoint/functions/Makefile
+++ b/drivers/pci/endpoint/functions/Makefile
@@ -6,4 +6,5 @@
obj-$(CONFIG_PCI_EPF_TEST) += pci-epf-test.o
obj-$(CONFIG_PCI_EPF_NTB) += pci-epf-ntb.o
obj-$(CONFIG_PCI_EPF_VNTB) += pci-epf-vntb.o
+obj-$(CONFIG_PCI_EPF_DMA) += pci-epf-dma.o
obj-$(CONFIG_PCI_EPF_MHI) += pci-epf-mhi.o
diff --git a/drivers/pci/endpoint/functions/pci-epf-dma.c b/drivers/pci/endpoint/functions/pci-epf-dma.c
new file mode 100644
index 000000000000..d7761966eca2
--- /dev/null
+++ b/drivers/pci/endpoint/functions/pci-epf-dma.c
@@ -0,0 +1,1361 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * PCI endpoint function that exposes an endpoint-integrated DMA controller
+ * to the PCI host.
+ *
+ * The host-side dw-edma-pcie driver consumes the BAR metadata published
+ * by this function.
+ */
+
+#include <linux/bitfield.h>
+#include <linux/configfs.h>
+#include <linux/dma/edma.h>
+#include <linux/dma-mapping.h>
+#include <linux/dmaengine.h>
+#include <linux/module.h>
+#include <linux/overflow.h>
+#include <linux/pci-ep-dma.h>
+#include <linux/pci-epc.h>
+#include <linux/pci-epf.h>
+#include <linux/pci_regs.h>
+#include <linux/slab.h>
+#include <linux/workqueue.h>
+
+/* HOST_REQ is set by the host driver, so poll it at a low idle rate. */
+#define PCI_EPF_DMA_HOST_REQ_POLL_MS 500
+
+struct pci_epf_dma_bar_map {
+ const struct pci_epc_aux_resource *res;
+ enum pci_barno bar;
+ u64 res_offset_in_bar;
+ u64 submap_offset_in_bar;
+ dma_addr_t phys_addr;
+ size_t map_size;
+ bool needs_submap;
+};
+
+struct pci_epf_dma {
+ struct pci_epf *epf;
+ struct config_group group;
+ struct delayed_work map_work;
+
+ enum pci_barno metadata_bar;
+ enum pci_barno dma_window_bar;
+ u16 wr_chans;
+ u16 rd_chans;
+ u8 reg_layout;
+ u8 reg_layout_data;
+
+ /* Backing storage for ctrl and descriptor resource pointers. */
+ struct pci_epc_aux_resource *resources;
+ unsigned int num_resources;
+ const struct pci_epc_aux_resource *ctrl;
+ const struct pci_epc_aux_resource *ep_to_rc_desc[EDMA_MAX_WR_CH];
+ const struct pci_epc_aux_resource *rc_to_ep_desc[EDMA_MAX_RD_CH];
+
+ /* Local DMAengine reservations for channels delegated to the host. */
+ struct dma_chan *ep_to_rc_chan[EDMA_MAX_WR_CH];
+ struct dma_chan *rc_to_ep_chan[EDMA_MAX_RD_CH];
+
+ void *metadata_addr;
+ void *dma_window_addr;
+ size_t msix_table_offset;
+ struct pci_epf_dma_bar_map *bar_maps;
+ unsigned int num_bar_maps;
+ struct pci_epf_bar_submap *submaps;
+ unsigned int num_submaps;
+
+ /* Cleared when a later event should retry programming the submaps. */
+ bool submaps_programmed;
+};
+
+#define to_epf_dma(epf_group) container_of((epf_group), struct pci_epf_dma, group)
+
+static struct pci_epf_header pci_epf_dma_header = {
+ .vendorid = PCI_ANY_ID,
+ .deviceid = PCI_ANY_ID,
+ .baseclass_code = PCI_BASE_CLASS_SYSTEM,
+ .subclass_code = PCI_CLASS_SYSTEM_DMA & 0xff,
+ .interrupt_pin = PCI_INTERRUPT_INTA,
+};
+
+static void pci_epf_dma_release_channels(struct pci_epf_dma *epf_dma)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(epf_dma->ep_to_rc_chan); i++) {
+ if (!epf_dma->ep_to_rc_chan[i])
+ continue;
+
+ dma_release_channel(epf_dma->ep_to_rc_chan[i]);
+ epf_dma->ep_to_rc_chan[i] = NULL;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(epf_dma->rc_to_ep_chan); i++) {
+ if (!epf_dma->rc_to_ep_chan[i])
+ continue;
+
+ dma_release_channel(epf_dma->rc_to_ep_chan[i]);
+ epf_dma->rc_to_ep_chan[i] = NULL;
+ }
+}
+
+static int pci_epf_dma_claim_channel(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *res,
+ struct dma_chan **chan)
+{
+ struct device *dev = &epf_dma->epf->dev;
+ struct dma_chan *dma_chan;
+
+ if (!res->u.dma_desc.dma_chan) {
+ dev_err(dev, "DMA channel %u cannot be reserved\n",
+ res->u.dma_desc.hw_ch);
+ return -EOPNOTSUPP;
+ }
+
+ dma_chan = dma_get_slave_channel(res->u.dma_desc.dma_chan);
+ if (!dma_chan) {
+ dev_err(dev, "DMA channel %u is already in use\n",
+ res->u.dma_desc.hw_ch);
+ return -EBUSY;
+ }
+
+ *chan = dma_chan;
+
+ return 0;
+}
+
+static int
+pci_epf_dma_validate_dw_edma_ctrl(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *ctrl)
+{
+ struct device *dev = &epf_dma->epf->dev;
+ enum dw_edma_map_format map = ctrl->u.dma_ctrl.reg_layout_data;
+ u16 total_wr_chans = ctrl->u.dma_ctrl.ep_to_rc_ch_cnt;
+ u16 total_rd_chans = ctrl->u.dma_ctrl.rc_to_ep_ch_cnt;
+
+ switch (map) {
+ case EDMA_MF_EDMA_LEGACY:
+ dev_err(dev, "legacy DesignWare eDMA layout cannot be delegated\n");
+ return -EOPNOTSUPP;
+ case EDMA_MF_EDMA_UNROLL:
+ case EDMA_MF_HDMA_COMPAT:
+ if ((epf_dma->wr_chans && epf_dma->wr_chans != total_wr_chans) ||
+ (epf_dma->rd_chans && epf_dma->rd_chans != total_rd_chans)) {
+ dev_err(dev, "DesignWare eDMA v0 delegation must cover the whole direction\n");
+ return -EOPNOTSUPP;
+ }
+ return 0;
+ case EDMA_MF_HDMA_NATIVE:
+ dev_err(dev, "DesignWare HDMA native layout cannot be delegated\n");
+ return -EOPNOTSUPP;
+ default:
+ return -EINVAL;
+ }
+}
+
+static bool pci_epf_dma_bar_usable(const struct pci_epc_features *epc_features,
+ enum pci_barno bar)
+{
+ if (bar < BAR_0 || bar >= PCI_STD_NUM_BARS)
+ return false;
+
+ return epc_features->bar[bar].type != BAR_RESERVED &&
+ epc_features->bar[bar].type != BAR_DISABLED;
+}
+
+static bool pci_epf_dma_bar_has_fixed_resource(struct pci_epf_dma *epf_dma,
+ enum pci_barno bar)
+{
+ unsigned int i;
+
+ for (i = 0; i < epf_dma->num_resources; i++) {
+ if (epf_dma->resources[i].bar == bar)
+ return true;
+ }
+
+ return false;
+}
+
+static enum pci_barno
+pci_epf_dma_first_usable_bar(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_features *epc_features,
+ enum pci_barno exclude)
+{
+ enum pci_barno bar;
+
+ for (bar = BAR_0; bar < PCI_STD_NUM_BARS; bar++) {
+ bar = pci_epc_get_next_free_bar(epc_features, bar);
+ if (bar == NO_BAR)
+ return NO_BAR;
+ if (bar != exclude &&
+ !pci_epf_dma_bar_has_fixed_resource(epf_dma, bar))
+ return bar;
+ }
+
+ return NO_BAR;
+}
+
+static size_t pci_epf_dma_align_size(size_t size, size_t align)
+{
+ if (!align)
+ return size;
+
+ return ALIGN(size, align);
+}
+
+static int pci_epf_dma_reuse_submap(struct pci_epf_dma *epf_dma,
+ unsigned int map_count,
+ dma_addr_t phys_addr, size_t map_size,
+ size_t offset, size_t *next_offset_in_bar,
+ u64 *res_offset_in_bar)
+{
+ struct pci_epf_dma_bar_map *map;
+ u64 delta;
+ size_t merged_size, next;
+ u64 res_map_end, submap_bar_end, submap_phys_end;
+ unsigned int i;
+
+ if (check_add_overflow(phys_addr, map_size, &res_map_end))
+ return -EOVERFLOW;
+
+ for (i = 0; i < map_count; i++) {
+ map = &epf_dma->bar_maps[i];
+ if (!map->needs_submap || map->bar != epf_dma->dma_window_bar)
+ continue;
+
+ if (check_add_overflow(map->phys_addr, map->map_size,
+ &submap_phys_end) ||
+ check_add_overflow(map->submap_offset_in_bar,
+ map->map_size, &submap_bar_end))
+ return -EOVERFLOW;
+
+ /*
+ * Reuse a submap that already covers this aligned resource
+ * window.
+ */
+ if (phys_addr >= map->phys_addr &&
+ res_map_end <= submap_phys_end) {
+ if (check_add_overflow(phys_addr - map->phys_addr,
+ offset, &delta) ||
+ check_add_overflow(map->submap_offset_in_bar,
+ delta, res_offset_in_bar))
+ return -EOVERFLOW;
+ return 1;
+ }
+
+ /*
+ * Extend only the BAR-tail submap when the physical ranges are
+ * contiguous.
+ */
+ if (submap_phys_end == phys_addr &&
+ submap_bar_end == *next_offset_in_bar) {
+ if (check_add_overflow(map->map_size, map_size,
+ &merged_size) ||
+ check_add_overflow(*next_offset_in_bar, map_size,
+ &next) ||
+ check_add_overflow(*next_offset_in_bar, offset,
+ res_offset_in_bar))
+ return -EOVERFLOW;
+
+ map->map_size = merged_size;
+ *next_offset_in_bar = next;
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+static int pci_epf_dma_add_map(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *res,
+ size_t align, size_t *next_offset_in_bar,
+ unsigned int *map_idx)
+{
+ dma_addr_t phys_addr;
+ size_t map_size, offset = 0, next;
+ u64 res_offset_in_bar;
+ int ret;
+
+ if (!res || !res->size)
+ return -EINVAL;
+
+ if (res->bar != NO_BAR) {
+ if (res->bar < BAR_0 || res->bar >= PCI_STD_NUM_BARS)
+ return -EINVAL;
+ if (res->bar == epf_dma->metadata_bar ||
+ res->bar == epf_dma->dma_window_bar)
+ return -EINVAL;
+
+ epf_dma->bar_maps[*map_idx] = (struct pci_epf_dma_bar_map) {
+ .res = res,
+ .bar = res->bar,
+ .res_offset_in_bar = res->bar_offset,
+ .map_size = res->size,
+ };
+ (*map_idx)++;
+
+ return 0;
+ }
+
+ if (epf_dma->dma_window_bar == NO_BAR)
+ return -EOPNOTSUPP;
+
+ phys_addr = res->phys_addr;
+ /* Map the aligned window that contains this resource. */
+ if (align) {
+ phys_addr = ALIGN_DOWN(res->phys_addr, align);
+ offset = res->phys_addr - phys_addr;
+ }
+
+ if (check_add_overflow(res->size, offset, &map_size))
+ return -EOVERFLOW;
+ map_size = pci_epf_dma_align_size(map_size, align);
+
+ ret = pci_epf_dma_reuse_submap(epf_dma, *map_idx, phys_addr, map_size,
+ offset, next_offset_in_bar,
+ &res_offset_in_bar);
+ if (ret < 0)
+ return ret;
+ if (ret) {
+ epf_dma->bar_maps[*map_idx] = (struct pci_epf_dma_bar_map) {
+ .res = res,
+ .bar = epf_dma->dma_window_bar,
+ .res_offset_in_bar = res_offset_in_bar,
+ .phys_addr = res->phys_addr,
+ .map_size = res->size,
+ };
+
+ (*map_idx)++;
+
+ return 0;
+ }
+
+ if (check_add_overflow(*next_offset_in_bar, map_size, &next))
+ return -EOVERFLOW;
+ if (check_add_overflow(*next_offset_in_bar, offset, &res_offset_in_bar))
+ return -EOVERFLOW;
+
+ epf_dma->bar_maps[*map_idx] = (struct pci_epf_dma_bar_map) {
+ .res = res,
+ .bar = epf_dma->dma_window_bar,
+ .res_offset_in_bar = res_offset_in_bar,
+ .submap_offset_in_bar = *next_offset_in_bar,
+ .phys_addr = phys_addr,
+ .map_size = map_size,
+ .needs_submap = true,
+ };
+
+ *next_offset_in_bar = next;
+ (*map_idx)++;
+
+ return 0;
+}
+
+static const struct pci_epf_dma_bar_map *
+pci_epf_dma_find_map(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_aux_resource *res)
+{
+ unsigned int i;
+
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ if (epf_dma->bar_maps[i].res == res)
+ return &epf_dma->bar_maps[i];
+ }
+
+ return NULL;
+}
+
+static bool pci_epf_dma_needs_dma_window(struct pci_epf_dma *epf_dma)
+{
+ unsigned int i;
+
+ if (epf_dma->ctrl && epf_dma->ctrl->bar == NO_BAR)
+ return true;
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ if (epf_dma->ep_to_rc_desc[i] &&
+ epf_dma->ep_to_rc_desc[i]->bar == NO_BAR)
+ return true;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ if (epf_dma->rc_to_ep_desc[i] &&
+ epf_dma->rc_to_ep_desc[i]->bar == NO_BAR)
+ return true;
+ }
+
+ return false;
+}
+
+static int pci_epf_dma_collect_resources(struct pci_epf_dma *epf_dma)
+{
+ const struct pci_epc_aux_resource *ep_to_rc_desc[EDMA_MAX_WR_CH] = {};
+ const struct pci_epc_aux_resource *rc_to_ep_desc[EDMA_MAX_RD_CH] = {};
+ const struct pci_epc_aux_resource *ctrl = NULL;
+ struct pci_epf *epf = epf_dma->epf;
+ struct pci_epc *epc = epf->epc;
+ struct device *dev = &epf->dev;
+ int count, i, ret;
+
+ count = pci_epc_get_aux_resources_count(epc, epf->func_no,
+ epf->vfunc_no);
+ if (count <= 0)
+ return count ?: -ENODEV;
+
+ struct pci_epc_aux_resource *res __free(kfree) =
+ kzalloc_objs(*res, count);
+ if (!res)
+ return -ENOMEM;
+
+ ret = pci_epc_get_aux_resources(epc, epf->func_no, epf->vfunc_no,
+ res, count);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < count; i++) {
+ switch (res[i].type) {
+ case PCI_EPC_AUX_DMA_CTRL_MMIO:
+ if (ctrl)
+ return -EINVAL;
+ ctrl = &res[i];
+ break;
+ case PCI_EPC_AUX_DMA_DESC_MEM: {
+ u16 hw_ch = res[i].u.dma_desc.hw_ch;
+
+ switch (res[i].u.dma_desc.dir) {
+ case PCI_EPC_AUX_DMA_EP_TO_RC:
+ if (hw_ch >= EDMA_MAX_WR_CH ||
+ ep_to_rc_desc[hw_ch])
+ return -EINVAL;
+ ep_to_rc_desc[hw_ch] = &res[i];
+ break;
+ case PCI_EPC_AUX_DMA_RC_TO_EP:
+ if (hw_ch >= EDMA_MAX_RD_CH ||
+ rc_to_ep_desc[hw_ch])
+ return -EINVAL;
+ rc_to_ep_desc[hw_ch] = &res[i];
+ break;
+ default:
+ return -EINVAL;
+ }
+ break;
+ }
+ default:
+ continue;
+ }
+ }
+
+ if (!ctrl)
+ return -ENODEV;
+
+ if (!epf_dma->wr_chans && !epf_dma->rd_chans)
+ return -EINVAL;
+
+ if (epf_dma->wr_chans > ctrl->u.dma_ctrl.ep_to_rc_ch_cnt ||
+ epf_dma->rd_chans > ctrl->u.dma_ctrl.rc_to_ep_ch_cnt)
+ return -EINVAL;
+
+ switch (ctrl->u.dma_ctrl.reg_layout) {
+ case PCI_EPC_AUX_DMA_REG_LAYOUT_DW_EDMA:
+ ret = pci_epf_dma_validate_dw_edma_ctrl(epf_dma, ctrl);
+ if (ret)
+ return ret;
+ epf_dma->reg_layout = PCI_EP_DMA_METADATA_REG_LAYOUT_DW_EDMA;
+ epf_dma->reg_layout_data = ctrl->u.dma_ctrl.reg_layout_data;
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ if (!ep_to_rc_desc[i]) {
+ dev_err(dev, "missing dense write DMA channel %d\n", i);
+ return -EINVAL;
+ }
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ if (!rc_to_ep_desc[i]) {
+ dev_err(dev, "missing dense read DMA channel %d\n", i);
+ return -EINVAL;
+ }
+ }
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ ret = pci_epf_dma_claim_channel(epf_dma, ep_to_rc_desc[i],
+ &epf_dma->ep_to_rc_chan[i]);
+ if (ret)
+ goto err_release_channels;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ ret = pci_epf_dma_claim_channel(epf_dma, rc_to_ep_desc[i],
+ &epf_dma->rc_to_ep_chan[i]);
+ if (ret)
+ goto err_release_channels;
+ }
+
+ epf_dma->resources = no_free_ptr(res);
+ epf_dma->num_resources = count;
+ epf_dma->ctrl = ctrl;
+ memcpy(epf_dma->ep_to_rc_desc, ep_to_rc_desc, sizeof(ep_to_rc_desc));
+ memcpy(epf_dma->rc_to_ep_desc, rc_to_ep_desc, sizeof(rc_to_ep_desc));
+
+ return 0;
+
+err_release_channels:
+ pci_epf_dma_release_channels(epf_dma);
+
+ return ret;
+}
+
+static void pci_epf_dma_metadata_write(__le32 *metadata, u16 metadata_off,
+ u32 val)
+{
+ metadata[metadata_off / sizeof(*metadata)] = cpu_to_le32(val);
+}
+
+static void pci_epf_dma_metadata_write64(__le32 *metadata, u16 metadata_off,
+ u64 val)
+{
+ pci_epf_dma_metadata_write(metadata, metadata_off, lower_32_bits(val));
+ pci_epf_dma_metadata_write(metadata, metadata_off + sizeof(u32),
+ upper_32_bits(val));
+}
+
+static int pci_epf_dma_build_ch_entry(const struct pci_epf_dma_bar_map *map,
+ __le32 *metadata, u16 entry)
+{
+ const struct pci_epc_aux_resource *res = map->res;
+ u32 ctrl;
+
+ if (res->size > U32_MAX)
+ return -EOVERFLOW;
+
+ ctrl = FIELD_PREP(PCI_EP_DMA_METADATA_CH_CTRL_HW_CH,
+ res->u.dma_desc.hw_ch) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CH_CTRL_DESC_BAR, map->bar);
+
+ pci_epf_dma_metadata_write(metadata, entry + PCI_EP_DMA_METADATA_CH_CTRL,
+ ctrl);
+ pci_epf_dma_metadata_write64(metadata,
+ entry + PCI_EP_DMA_METADATA_CH_DESC_OFF_LO,
+ map->res_offset_in_bar);
+ pci_epf_dma_metadata_write(metadata,
+ entry + PCI_EP_DMA_METADATA_CH_DESC_SIZE,
+ (u32)res->size);
+ pci_epf_dma_metadata_write64(metadata,
+ entry + PCI_EP_DMA_METADATA_CH_DESC_ADDR_LO,
+ res->phys_addr);
+
+ return 0;
+}
+
+static void pci_epf_dma_set_metadata_ready(struct pci_epf_dma *epf_dma,
+ bool ready)
+{
+ __le32 *metadata = epf_dma->metadata_addr;
+ __le32 *ctrl_ptr;
+ u32 ctrl;
+
+ if (!metadata)
+ return;
+
+ ctrl_ptr = &metadata[PCI_EP_DMA_METADATA_CTRL / sizeof(*metadata)];
+ ctrl = le32_to_cpu(READ_ONCE(*ctrl_ptr));
+ if (ready) {
+ dma_wmb();
+ ctrl |= PCI_EP_DMA_METADATA_CTRL_READY;
+ } else {
+ ctrl &= ~PCI_EP_DMA_METADATA_CTRL_READY;
+ }
+ WRITE_ONCE(*ctrl_ptr, cpu_to_le32(ctrl));
+}
+
+static bool pci_epf_dma_metadata_host_requested(struct pci_epf_dma *epf_dma)
+{
+ __le32 *metadata = epf_dma->metadata_addr;
+ u32 ctrl;
+
+ if (!metadata)
+ return false;
+
+ ctrl = le32_to_cpu(READ_ONCE(metadata[PCI_EP_DMA_METADATA_CTRL /
+ sizeof(*metadata)]));
+
+ return ctrl & PCI_EP_DMA_METADATA_CTRL_HOST_REQ;
+}
+
+static void pci_epf_dma_clear_metadata_status(struct pci_epf_dma *epf_dma)
+{
+ __le32 *metadata = epf_dma->metadata_addr;
+ __le32 *ctrl_ptr;
+ u32 ctrl;
+
+ if (!metadata)
+ return;
+
+ ctrl_ptr = &metadata[PCI_EP_DMA_METADATA_CTRL / sizeof(*metadata)];
+ ctrl = le32_to_cpu(READ_ONCE(*ctrl_ptr));
+ ctrl &= ~(PCI_EP_DMA_METADATA_CTRL_HOST_REQ |
+ PCI_EP_DMA_METADATA_CTRL_READY);
+ WRITE_ONCE(*ctrl_ptr, cpu_to_le32(ctrl));
+}
+
+static int pci_epf_dma_build_metadata(struct pci_epf_dma *epf_dma)
+{
+ const struct pci_epf_dma_bar_map *ctrl_map;
+ u16 entry_size = PCI_EP_DMA_METADATA_CH_ENTRY_SIZE;
+ u16 wr_table, rd_table, total_len;
+ __le32 *metadata = epf_dma->metadata_addr;
+ unsigned int i;
+ int ret;
+
+ if (!metadata)
+ return -EINVAL;
+
+ ctrl_map = pci_epf_dma_find_map(epf_dma, epf_dma->ctrl);
+ if (!ctrl_map)
+ return -EINVAL;
+ if (epf_dma->wr_chans > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT) ||
+ epf_dma->rd_chans > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT) ||
+ entry_size > FIELD_MAX(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE) ||
+ ctrl_map->res->size > U32_MAX)
+ return -EOVERFLOW;
+
+ wr_table = epf_dma->wr_chans ? PCI_EP_DMA_METADATA_HDR_LEN : 0;
+ rd_table = epf_dma->rd_chans ?
+ PCI_EP_DMA_METADATA_HDR_LEN + epf_dma->wr_chans * entry_size : 0;
+ total_len = PCI_EP_DMA_METADATA_HDR_LEN +
+ (epf_dma->wr_chans + epf_dma->rd_chans) * entry_size;
+
+ memset(metadata, 0, total_len);
+
+ pci_epf_dma_metadata_write(metadata, 0, PCI_EP_DMA_METADATA_MAGIC);
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_HDR,
+ FIELD_PREP(PCI_EP_DMA_METADATA_HDR_REV,
+ PCI_EP_DMA_METADATA_REV) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_HDR_LEN_FIELD,
+ total_len));
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_CTRL,
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_REG_BAR,
+ ctrl_map->bar) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT,
+ epf_dma->wr_chans) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT,
+ epf_dma->rd_chans) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE,
+ entry_size));
+ pci_epf_dma_metadata_write64(metadata,
+ PCI_EP_DMA_METADATA_REG_OFF_LO,
+ ctrl_map->res_offset_in_bar);
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_REG_LAYOUT,
+ FIELD_PREP(PCI_EP_DMA_METADATA_REG_LAYOUT_ID,
+ epf_dma->reg_layout) |
+ FIELD_PREP(PCI_EP_DMA_METADATA_REG_LAYOUT_DATA,
+ epf_dma->reg_layout_data));
+ pci_epf_dma_metadata_write(metadata, PCI_EP_DMA_METADATA_REG_SIZE,
+ (u32)ctrl_map->res->size);
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ const struct pci_epf_dma_bar_map *map;
+
+ map = pci_epf_dma_find_map(epf_dma,
+ epf_dma->ep_to_rc_desc[i]);
+ if (!map)
+ return -EINVAL;
+ ret = pci_epf_dma_build_ch_entry(map, metadata,
+ wr_table + i * entry_size);
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ const struct pci_epf_dma_bar_map *map;
+
+ map = pci_epf_dma_find_map(epf_dma,
+ epf_dma->rc_to_ep_desc[i]);
+ if (!map)
+ return -EINVAL;
+ ret = pci_epf_dma_build_ch_entry(map, metadata,
+ rd_table + i * entry_size);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int pci_epf_dma_reserve_msix(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_features *epc_features,
+ size_t *backing_size)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ size_t msix_table_size, pba_size, next;
+ unsigned int nvec = epf->msix_interrupts;
+
+ epf_dma->msix_table_offset = 0;
+
+ if (!epc_features->msix_capable || !nvec)
+ return 0;
+
+ next = ALIGN(*backing_size, 8);
+ if (next > U32_MAX)
+ return -EOVERFLOW;
+ epf_dma->msix_table_offset = next;
+
+ if (check_mul_overflow(PCI_MSIX_ENTRY_SIZE, nvec, &msix_table_size))
+ return -EOVERFLOW;
+
+ pba_size = ALIGN(DIV_ROUND_UP(nvec, 8), 8);
+ if (check_add_overflow(next, msix_table_size, &next) ||
+ next > U32_MAX ||
+ check_add_overflow(next, pba_size, &next))
+ return -EOVERFLOW;
+
+ *backing_size = next;
+
+ return 0;
+}
+
+static int pci_epf_dma_build_layout(struct pci_epf_dma *epf_dma,
+ const struct pci_epc_features *epc_features)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ struct device *dev = &epf->dev;
+ struct pci_epf_bar *bar;
+ unsigned int max_maps, map_idx = 0, sub_idx = 0;
+ size_t align = epc_features->align;
+ size_t metadata_size, metadata_backing_size, metadata_bar_size;
+ size_t mapped_size = 0, dma_window_bar_size;
+ int i, ret;
+
+ metadata_size = PCI_EP_DMA_METADATA_HDR_LEN;
+ metadata_size += (epf_dma->wr_chans + epf_dma->rd_chans) *
+ PCI_EP_DMA_METADATA_CH_ENTRY_SIZE;
+ metadata_backing_size = metadata_size;
+ ret = pci_epf_dma_reserve_msix(epf_dma, epc_features,
+ &metadata_backing_size);
+ if (ret)
+ return ret;
+ metadata_bar_size = pci_epf_dma_align_size(metadata_backing_size,
+ align);
+
+ epf_dma->metadata_addr = pci_epf_alloc_space(epf, metadata_bar_size,
+ epf_dma->metadata_bar,
+ epc_features,
+ PRIMARY_INTERFACE);
+ if (!epf_dma->metadata_addr) {
+ dev_err(dev, "failed to allocate BAR%d metadata space\n",
+ epf_dma->metadata_bar);
+ return -ENOMEM;
+ }
+ memset(epf_dma->metadata_addr, 0, epf->bar[epf_dma->metadata_bar].size);
+
+ /* One map for DMA controller registers, plus one per channel. */
+ max_maps = 1 + epf_dma->wr_chans + epf_dma->rd_chans;
+ epf_dma->bar_maps = kzalloc_objs(*epf_dma->bar_maps, max_maps);
+ if (!epf_dma->bar_maps)
+ return -ENOMEM;
+
+ ret = pci_epf_dma_add_map(epf_dma, epf_dma->ctrl, align,
+ &mapped_size, &map_idx);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < epf_dma->wr_chans; i++) {
+ ret = pci_epf_dma_add_map(epf_dma,
+ epf_dma->ep_to_rc_desc[i], align,
+ &mapped_size, &map_idx);
+ if (ret)
+ return ret;
+ }
+
+ for (i = 0; i < epf_dma->rd_chans; i++) {
+ ret = pci_epf_dma_add_map(epf_dma,
+ epf_dma->rc_to_ep_desc[i], align,
+ &mapped_size, &map_idx);
+ if (ret)
+ return ret;
+ }
+
+ epf_dma->num_bar_maps = map_idx;
+
+ ret = pci_epf_dma_build_metadata(epf_dma);
+ if (ret)
+ return ret;
+
+ /* Some DMA resources may already be visible through another map. */
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ if (epf_dma->bar_maps[i].needs_submap)
+ epf_dma->num_submaps++;
+ }
+ if (!epf_dma->num_submaps)
+ return 0;
+
+ dma_window_bar_size = mapped_size;
+ epf_dma->dma_window_addr =
+ pci_epf_alloc_space(epf, dma_window_bar_size,
+ epf_dma->dma_window_bar, epc_features,
+ PRIMARY_INTERFACE);
+ if (!epf_dma->dma_window_addr) {
+ dev_err(dev, "failed to allocate BAR%d DMA window space\n",
+ epf_dma->dma_window_bar);
+ return -ENOMEM;
+ }
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ memset(epf_dma->dma_window_addr, 0, bar->size);
+
+ if (bar->size > mapped_size)
+ epf_dma->num_submaps++;
+
+ epf_dma->submaps = kzalloc_objs(*epf_dma->submaps, epf_dma->num_submaps);
+ if (!epf_dma->submaps)
+ return -ENOMEM;
+
+ for (i = 0; i < epf_dma->num_bar_maps; i++) {
+ if (!epf_dma->bar_maps[i].needs_submap)
+ continue;
+
+ epf_dma->submaps[sub_idx++] = (struct pci_epf_bar_submap) {
+ .phys_addr = epf_dma->bar_maps[i].phys_addr,
+ .size = epf_dma->bar_maps[i].map_size,
+ };
+ }
+
+ /* Cover any BAR tail padding with the allocated scratch space. */
+ if (bar->size > mapped_size) {
+ epf_dma->submaps[sub_idx++] = (struct pci_epf_bar_submap) {
+ .phys_addr = bar->phys_addr + mapped_size,
+ .size = bar->size - mapped_size,
+ };
+ }
+
+ return 0;
+}
+
+static void pci_epf_dma_free_layout(struct pci_epf_dma *epf_dma)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ struct pci_epf_bar *bar;
+
+ if (epf_dma->dma_window_addr) {
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ bar->submap = NULL;
+ bar->num_submap = 0;
+ }
+ epf_dma->submaps_programmed = false;
+
+ kfree(epf_dma->submaps);
+ epf_dma->submaps = NULL;
+ epf_dma->num_submaps = 0;
+
+ kfree(epf_dma->bar_maps);
+ epf_dma->bar_maps = NULL;
+ epf_dma->num_bar_maps = 0;
+
+ pci_epf_dma_release_channels(epf_dma);
+
+ kfree(epf_dma->resources);
+ epf_dma->resources = NULL;
+ epf_dma->num_resources = 0;
+ epf_dma->ctrl = NULL;
+ memset(epf_dma->ep_to_rc_desc, 0, sizeof(epf_dma->ep_to_rc_desc));
+ memset(epf_dma->rc_to_ep_desc, 0, sizeof(epf_dma->rc_to_ep_desc));
+
+ if (epf_dma->dma_window_addr) {
+ pci_epf_free_space(epf, epf_dma->dma_window_addr,
+ epf_dma->dma_window_bar,
+ PRIMARY_INTERFACE);
+ epf_dma->dma_window_addr = NULL;
+ }
+
+ if (epf_dma->metadata_addr) {
+ pci_epf_free_space(epf, epf_dma->metadata_addr,
+ epf_dma->metadata_bar,
+ PRIMARY_INTERFACE);
+ epf_dma->metadata_addr = NULL;
+ }
+ epf_dma->msix_table_offset = 0;
+}
+
+static int pci_epf_dma_program_submaps(struct pci_epf_dma *epf_dma)
+{
+ struct pci_epf *epf = epf_dma->epf;
+ struct pci_epf_bar *bar;
+ int ret;
+
+ if (!epf_dma->dma_window_addr) {
+ pci_epf_dma_set_metadata_ready(epf_dma, true);
+ return 0;
+ }
+
+ if (epf_dma->submaps_programmed)
+ return 0;
+
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ bar->submap = epf_dma->submaps;
+ bar->num_submap = epf_dma->num_submaps;
+
+ ret = pci_epc_set_bar(epf->epc, epf->func_no, epf->vfunc_no, bar);
+ if (ret) {
+ bar->submap = NULL;
+ bar->num_submap = 0;
+ return ret;
+ }
+
+ epf_dma->submaps_programmed = true;
+ pci_epf_dma_set_metadata_ready(epf_dma, true);
+
+ return 0;
+}
+
+static void pci_epf_dma_map_work(struct work_struct *work)
+{
+ struct pci_epf_dma *epf_dma =
+ container_of(to_delayed_work(work), struct pci_epf_dma,
+ map_work);
+ struct pci_epf *epf = epf_dma->epf;
+ int ret;
+
+ if (!epf->epc)
+ return;
+
+ if (!epf->epc->init_complete) {
+ schedule_delayed_work(&epf_dma->map_work,
+ msecs_to_jiffies(PCI_EPF_DMA_HOST_REQ_POLL_MS));
+ return;
+ }
+
+ if (!pci_epf_dma_metadata_host_requested(epf_dma)) {
+ schedule_delayed_work(&epf_dma->map_work,
+ msecs_to_jiffies(PCI_EPF_DMA_HOST_REQ_POLL_MS));
+ return;
+ }
+
+ ret = pci_epf_dma_program_submaps(epf_dma);
+ if (ret)
+ dev_err(&epf->dev, "failed to program DMA window BAR submaps: %d\n",
+ ret);
+}
+
+static int pci_epf_dma_epc_init(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ const struct pci_epc_features *epc_features;
+ struct pci_epc *epc = epf->epc;
+ struct device *dev = &epf->dev;
+ int ret;
+
+ epc_features = pci_epc_get_features(epc, epf->func_no, epf->vfunc_no);
+ if (!epc_features)
+ return -EOPNOTSUPP;
+
+ pci_epf_dma_clear_metadata_status(epf_dma);
+
+ ret = pci_epc_write_header(epc, epf->func_no, epf->vfunc_no,
+ epf->header);
+ if (ret) {
+ dev_err(dev, "configuration header write failed\n");
+ return ret;
+ }
+
+ ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->metadata_bar]);
+ if (ret) {
+ dev_err(dev, "BAR%d setup failed: %d\n",
+ epf_dma->metadata_bar, ret);
+ return ret;
+ }
+
+ if (epf_dma->dma_window_addr) {
+ ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->dma_window_bar]);
+ if (ret) {
+ dev_err(dev, "BAR%d setup failed: %d\n",
+ epf_dma->dma_window_bar, ret);
+ goto err_clear_metadata_bar;
+ }
+ }
+
+ if (epc_features->msi_capable && epf->msi_interrupts) {
+ ret = pci_epc_set_msi(epc, epf->func_no, epf->vfunc_no,
+ epf->msi_interrupts);
+ if (ret) {
+ dev_err(dev, "MSI setup failed: %d\n", ret);
+ goto err_clear_dma_window_bar;
+ }
+ }
+
+ if (epc_features->msix_capable && epf->msix_interrupts) {
+ ret = pci_epc_set_msix(epc, epf->func_no, epf->vfunc_no,
+ epf->msix_interrupts,
+ epf_dma->metadata_bar,
+ epf_dma->msix_table_offset);
+ if (ret) {
+ dev_err(dev, "MSI-X setup failed: %d\n", ret);
+ goto err_clear_dma_window_bar;
+ }
+ }
+
+ schedule_delayed_work(&epf_dma->map_work, 0);
+
+ return 0;
+
+err_clear_dma_window_bar:
+ if (epf_dma->dma_window_addr)
+ pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->dma_window_bar]);
+err_clear_metadata_bar:
+ pci_epc_clear_bar(epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->metadata_bar]);
+ pci_epf_dma_clear_metadata_status(epf_dma);
+
+ return ret;
+}
+
+static void pci_epf_dma_epc_deinit(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ struct pci_epf_bar *bar;
+
+ cancel_delayed_work_sync(&epf_dma->map_work);
+
+ if (!epf_dma->metadata_addr)
+ return;
+
+ pci_epf_dma_clear_metadata_status(epf_dma);
+ if (epf_dma->dma_window_addr) {
+ bar = &epf->bar[epf_dma->dma_window_bar];
+ pci_epc_clear_bar(epf->epc, epf->func_no, epf->vfunc_no, bar);
+ bar->submap = NULL;
+ bar->num_submap = 0;
+ }
+ pci_epc_clear_bar(epf->epc, epf->func_no, epf->vfunc_no,
+ &epf->bar[epf_dma->metadata_bar]);
+ epf_dma->submaps_programmed = false;
+}
+
+static int pci_epf_dma_link_up(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+
+ schedule_delayed_work(&epf_dma->map_work, 0);
+
+ return 0;
+}
+
+static int pci_epf_dma_link_down(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+
+ cancel_delayed_work_sync(&epf_dma->map_work);
+ pci_epf_dma_clear_metadata_status(epf_dma);
+ /*
+ * Link down can invalidate non-sticky inbound ATU state without going
+ * through pci_epc_clear_bar(). Keep the BAR/submap description intact,
+ * but force the next link-up path to reprogram the subrange mappings.
+ */
+ epf_dma->submaps_programmed = false;
+
+ return 0;
+}
+
+static const struct pci_epc_event_ops pci_epf_dma_event_ops = {
+ .epc_init = pci_epf_dma_epc_init,
+ .epc_deinit = pci_epf_dma_epc_deinit,
+ .link_up = pci_epf_dma_link_up,
+ .link_down = pci_epf_dma_link_down,
+};
+
+static int pci_epf_dma_bind(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ const struct pci_epc_features *epc_features;
+ struct pci_epc *epc = epf->epc;
+ bool needs_dma_window;
+ int ret;
+
+ if (WARN_ON_ONCE(!epc))
+ return -EINVAL;
+
+ epc_features = pci_epc_get_features(epc, epf->func_no, epf->vfunc_no);
+ if (!epc_features)
+ return -EOPNOTSUPP;
+
+ if (!epc_features->msi_capable && !epc_features->msix_capable)
+ return -EOPNOTSUPP;
+
+ if ((!epc_features->msi_capable || !epf->msi_interrupts) &&
+ (!epc_features->msix_capable || !epf->msix_interrupts))
+ return -EINVAL;
+
+ ret = pci_epf_dma_collect_resources(epf_dma);
+ if (ret)
+ return ret;
+
+ if (epf_dma->metadata_bar == NO_BAR)
+ epf_dma->metadata_bar =
+ pci_epf_dma_first_usable_bar(epf_dma, epc_features,
+ NO_BAR);
+
+ if (epf_dma->metadata_bar == NO_BAR ||
+ !pci_epf_dma_bar_usable(epc_features, epf_dma->metadata_bar) ||
+ pci_epf_dma_bar_has_fixed_resource(epf_dma, epf_dma->metadata_bar)) {
+ ret = -EINVAL;
+ goto err_free;
+ }
+
+ needs_dma_window = pci_epf_dma_needs_dma_window(epf_dma);
+ if (needs_dma_window) {
+ if (!epc_features->subrange_mapping ||
+ !epc_features->dynamic_inbound_mapping) {
+ ret = -EOPNOTSUPP;
+ goto err_free;
+ }
+
+ if (epf_dma->dma_window_bar == NO_BAR)
+ epf_dma->dma_window_bar =
+ pci_epf_dma_first_usable_bar(epf_dma, epc_features,
+ epf_dma->metadata_bar);
+ if (epf_dma->dma_window_bar == NO_BAR) {
+ ret = -EOPNOTSUPP;
+ goto err_free;
+ }
+ }
+
+ if (epf_dma->dma_window_bar != NO_BAR) {
+ if (!pci_epf_dma_bar_usable(epc_features,
+ epf_dma->dma_window_bar)) {
+ ret = -EINVAL;
+ goto err_free;
+ }
+ if (epf_dma->metadata_bar == epf_dma->dma_window_bar ||
+ pci_epf_dma_bar_has_fixed_resource(epf_dma,
+ epf_dma->dma_window_bar)) {
+ ret = -EINVAL;
+ goto err_free;
+ }
+ }
+
+ ret = pci_epf_dma_build_layout(epf_dma, epc_features);
+ if (ret)
+ goto err_free;
+
+ return 0;
+
+err_free:
+ pci_epf_dma_free_layout(epf_dma);
+
+ return ret;
+}
+
+static void pci_epf_dma_unbind(struct pci_epf *epf)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+
+ cancel_delayed_work_sync(&epf_dma->map_work);
+ if (epf->epc && epf->epc->init_complete)
+ pci_epf_dma_epc_deinit(epf);
+ pci_epf_dma_free_layout(epf_dma);
+}
+
+#define PCI_EPF_DMA_SHOW(_name, _fmt, _val) \
+static ssize_t pci_epf_dma_##_name##_show(struct config_item *item, \
+ char *page) \
+{ \
+ struct config_group *group = to_config_group(item); \
+ struct pci_epf_dma *epf_dma = to_epf_dma(group); \
+ \
+ return sysfs_emit(page, _fmt "\n", (_val)); \
+}
+
+PCI_EPF_DMA_SHOW(metadata_bar, "%d", (int)epf_dma->metadata_bar)
+PCI_EPF_DMA_SHOW(dma_window_bar, "%d", (int)epf_dma->dma_window_bar)
+
+static ssize_t pci_epf_dma_metadata_bar_store(struct config_item *item, const char *page,
+ size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ int bar, ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtoint(page, 0, &bar);
+ if (ret)
+ return ret;
+
+ if (bar != NO_BAR && (bar < BAR_0 || bar >= PCI_STD_NUM_BARS))
+ return -EINVAL;
+ if (bar != NO_BAR && bar == epf_dma->dma_window_bar)
+ return -EINVAL;
+
+ epf_dma->metadata_bar = bar;
+
+ return len;
+}
+
+static ssize_t pci_epf_dma_dma_window_bar_store(struct config_item *item,
+ const char *page, size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ int bar, ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtoint(page, 0, &bar);
+ if (ret)
+ return ret;
+
+ if (bar != NO_BAR && (bar < BAR_0 || bar >= PCI_STD_NUM_BARS))
+ return -EINVAL;
+ if (bar != NO_BAR && bar == epf_dma->metadata_bar)
+ return -EINVAL;
+
+ epf_dma->dma_window_bar = bar;
+
+ return len;
+}
+
+PCI_EPF_DMA_SHOW(wr_chans, "%u", (unsigned int)epf_dma->wr_chans)
+
+static ssize_t pci_epf_dma_wr_chans_store(struct config_item *item,
+ const char *page, size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ u16 val;
+ int ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtou16(page, 0, &val);
+ if (ret)
+ return ret;
+ if (val > EDMA_MAX_WR_CH)
+ return -EINVAL;
+
+ epf_dma->wr_chans = val;
+
+ return len;
+}
+
+PCI_EPF_DMA_SHOW(rd_chans, "%u", (unsigned int)epf_dma->rd_chans)
+
+static ssize_t pci_epf_dma_rd_chans_store(struct config_item *item,
+ const char *page, size_t len)
+{
+ struct config_group *group = to_config_group(item);
+ struct pci_epf_dma *epf_dma = to_epf_dma(group);
+ u16 val;
+ int ret;
+
+ if (epf_dma->epf->epc)
+ return -EOPNOTSUPP;
+
+ ret = kstrtou16(page, 0, &val);
+ if (ret)
+ return ret;
+ if (val > EDMA_MAX_RD_CH)
+ return -EINVAL;
+
+ epf_dma->rd_chans = val;
+
+ return len;
+}
+
+CONFIGFS_ATTR(pci_epf_dma_, metadata_bar);
+CONFIGFS_ATTR(pci_epf_dma_, dma_window_bar);
+CONFIGFS_ATTR(pci_epf_dma_, wr_chans);
+CONFIGFS_ATTR(pci_epf_dma_, rd_chans);
+
+static struct configfs_attribute *pci_epf_dma_attrs[] = {
+ &pci_epf_dma_attr_metadata_bar,
+ &pci_epf_dma_attr_dma_window_bar,
+ &pci_epf_dma_attr_wr_chans,
+ &pci_epf_dma_attr_rd_chans,
+ NULL,
+};
+
+static const struct config_item_type pci_epf_dma_group_type = {
+ .ct_attrs = pci_epf_dma_attrs,
+ .ct_owner = THIS_MODULE,
+};
+
+static struct config_group *pci_epf_dma_add_cfs(struct pci_epf *epf,
+ struct config_group *group)
+{
+ struct pci_epf_dma *epf_dma = epf_get_drvdata(epf);
+ struct config_group *epf_group = &epf_dma->group;
+ struct device *dev = &epf->dev;
+
+ config_group_init_type_name(epf_group, dev_name(dev),
+ &pci_epf_dma_group_type);
+
+ return epf_group;
+}
+
+static const struct pci_epf_device_id pci_epf_dma_ids[] = {
+ {
+ .name = "pci_epf_dma",
+ },
+ {},
+};
+
+static int pci_epf_dma_probe(struct pci_epf *epf,
+ const struct pci_epf_device_id *id)
+{
+ struct pci_epf_dma *epf_dma;
+
+ epf_dma = devm_kzalloc(&epf->dev, sizeof(*epf_dma), GFP_KERNEL);
+ if (!epf_dma)
+ return -ENOMEM;
+
+ epf->header = &pci_epf_dma_header;
+ epf->event_ops = &pci_epf_dma_event_ops;
+
+ epf_dma->epf = epf;
+ epf_dma->metadata_bar = NO_BAR;
+ epf_dma->dma_window_bar = NO_BAR;
+ INIT_DELAYED_WORK(&epf_dma->map_work, pci_epf_dma_map_work);
+
+ epf_set_drvdata(epf, epf_dma);
+
+ return 0;
+}
+
+static const struct pci_epf_ops pci_epf_dma_ops = {
+ .unbind = pci_epf_dma_unbind,
+ .bind = pci_epf_dma_bind,
+ .add_cfs = pci_epf_dma_add_cfs,
+};
+
+static struct pci_epf_driver pci_epf_dma_driver = {
+ .driver.name = "pci_epf_dma",
+ .probe = pci_epf_dma_probe,
+ .id_table = pci_epf_dma_ids,
+ .ops = &pci_epf_dma_ops,
+ .owner = THIS_MODULE,
+};
+
+static int __init pci_epf_dma_init(void)
+{
+ return pci_epf_register_driver(&pci_epf_dma_driver);
+}
+module_init(pci_epf_dma_init);
+
+static void __exit pci_epf_dma_exit(void)
+{
+ pci_epf_unregister_driver(&pci_epf_dma_driver);
+}
+module_exit(pci_epf_dma_exit);
+
+MODULE_DESCRIPTION("PCI EPF DMA DRIVER");
+MODULE_AUTHOR("Koichiro Den <den@valinux.co.jp>");
+MODULE_LICENSE("GPL");
--
2.51.0
^ permalink raw reply related
* [PATCH 1/3] dmaengine: dw-edma-pcie: Discover endpoint DMA metadata
From: Koichiro Den @ 2026-05-21 6:36 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
In-Reply-To: <20260521063638.2843021-1-den@valinux.co.jp>
Teach dw-edma-pcie to discover a PCI endpoint DMA function from
BAR-resident metadata. The metadata supplies the DMA register window,
channel counts, descriptor windows, optional auxiliary windows, and
endpoint-local descriptor and auxiliary addresses.
Endpoint-provided DMA channels use raw slave addresses because the host
programs transfers against endpoint physical addresses, not PCI BAR
addresses. Scope the default remote interrupt mode to the endpoint DMA
metadata match entry so EDDA and MDB keep their existing local interrupt
behavior.
Endpoint DMA metadata can be discovered after an explicit bind through
driver_override or a dynamic ID. For such binds, there is no static
match data, so the driver falls back to the generic endpoint DMA
metadata parser.
The endpoint polls HOST_REQ at a low idle rate before programming DMA
window submaps and setting READY. Let the host wait for several endpoint
poll periods before treating the READY handshake as timed out.
Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
drivers/dma/dw-edma/dw-edma-pcie.c | 369 ++++++++++++++++++++++++++++-
1 file changed, 368 insertions(+), 1 deletion(-)
diff --git a/drivers/dma/dw-edma/dw-edma-pcie.c b/drivers/dma/dw-edma/dw-edma-pcie.c
index 2f752e8fb999..d4ae6df36858 100644
--- a/drivers/dma/dw-edma/dw-edma-pcie.c
+++ b/drivers/dma/dw-edma/dw-edma-pcie.c
@@ -11,9 +11,13 @@
#include <linux/pci.h>
#include <linux/device.h>
#include <linux/dma/edma.h>
+#include <linux/iopoll.h>
#include <linux/pci-epf.h>
#include <linux/msi.h>
#include <linux/bitfield.h>
+#include <linux/io.h>
+#include <linux/overflow.h>
+#include <linux/pci-ep-dma.h>
#include <linux/sizes.h>
#include "dw-edma-core.h"
@@ -44,6 +48,9 @@
#define DW_PCIE_XILINX_MDB_DT_OFF_GAP 0x100000
#define DW_PCIE_XILINX_MDB_DT_SIZE 0x800
+#define DW_PCIE_EP_DMA_READY_POLL_US 1000
+#define DW_PCIE_EP_DMA_READY_TIMEOUT_US 2000000
+
#define DW_BLOCK(a, b, c) \
{ \
.bar = a, \
@@ -93,6 +100,12 @@ struct dw_edma_pcie_match_data {
#define DW_EDMA_PCIE_F_RAW_SLAVE_ADDR BIT(1)
#define DW_EDMA_PCIE_F_REG_OFFSET BIT(2)
+struct dw_edma_pcie_ep_dma_view {
+ struct pci_dev *pdev;
+ void __iomem *base;
+ resource_size_t limit;
+};
+
static const struct dw_edma_pcie_data snps_edda_data = {
/* eDMA registers location */
.rg.bar = BAR_0,
@@ -144,6 +157,13 @@ static const struct dw_edma_pcie_data xilinx_mdb_data = {
.rd_ch_cnt = 8,
};
+static const struct dw_edma_pcie_data ep_dma_data = {
+ .mf = EDMA_MF_EDMA_UNROLL,
+ .irqs = EDMA_MAX_WR_CH + EDMA_MAX_RD_CH,
+ .wr_ch_cnt = EDMA_MAX_WR_CH,
+ .rd_ch_cnt = EDMA_MAX_RD_CH,
+};
+
static void dw_edma_set_chan_region_offset(struct dw_edma_pcie_data *pdata,
enum pci_barno bar, off_t start_off,
off_t ll_off_gap, size_t ll_size,
@@ -217,6 +237,82 @@ static const struct dw_edma_plat_ops dw_edma_pcie_raw_addr_plat_ops = {
.irq_vector = dw_edma_pcie_irq_vector,
};
+static bool dw_edma_pcie_valid_bar(enum pci_barno bar)
+{
+ return bar >= BAR_0 && bar <= BAR_5;
+}
+
+static bool dw_edma_pcie_valid_bar_range(struct pci_dev *pdev,
+ enum pci_barno bar, u64 off,
+ size_t sz)
+{
+ resource_size_t bar_len;
+
+ if (!dw_edma_pcie_valid_bar(bar) || !sz)
+ return false;
+
+ bar_len = pci_resource_len(pdev, bar);
+
+ return off <= bar_len && sz <= bar_len - off;
+}
+
+static bool dw_edma_pcie_valid_block(struct pci_dev *pdev,
+ const struct dw_edma_block *block)
+{
+ return dw_edma_pcie_valid_bar_range(pdev, block->bar, block->off,
+ block->sz);
+}
+
+static bool dw_edma_pcie_ep_dma_bar_scannable(struct pci_dev *pdev,
+ enum pci_barno bar)
+{
+ unsigned long flags = pci_resource_flags(pdev, bar);
+
+ if (!(flags & IORESOURCE_MEM))
+ return false;
+
+ if (flags & (IORESOURCE_UNSET | IORESOURCE_DISABLED))
+ return false;
+
+ return pci_resource_len(pdev, bar) >= PCI_EP_DMA_METADATA_HDR_LEN;
+}
+
+static u32 dw_edma_pcie_ep_dma_readl(struct dw_edma_pcie_ep_dma_view *view,
+ u16 off)
+{
+ return readl(view->base + off);
+}
+
+static void dw_edma_pcie_ep_dma_writel(struct dw_edma_pcie_ep_dma_view *view,
+ u16 off, u32 val)
+{
+ writel(val, view->base + off);
+}
+
+static u64 dw_edma_pcie_ep_dma_read64(struct dw_edma_pcie_ep_dma_view *view,
+ u16 lo, u16 hi)
+{
+ u64 val;
+
+ val = dw_edma_pcie_ep_dma_readl(view, hi);
+
+ return (val << 32) | dw_edma_pcie_ep_dma_readl(view, lo);
+}
+
+static int dw_edma_pcie_ep_dma_read_off(struct dw_edma_pcie_ep_dma_view *view,
+ u16 lo, u16 hi, off_t *off)
+{
+ u64 val;
+
+ val = dw_edma_pcie_ep_dma_read64(view, lo, hi);
+ if (val > type_max(*off))
+ return -EINVAL;
+
+ *off = val;
+
+ return 0;
+}
+
static void dw_edma_pcie_get_synopsys_dma_data(struct pci_dev *pdev,
struct dw_edma_pcie_data *pdata)
{
@@ -318,6 +414,265 @@ static void dw_edma_pcie_get_xilinx_dma_data(struct pci_dev *pdev,
pdata->devmem_phys_off = off;
}
+static int
+dw_edma_pcie_parse_ep_dma_ch_table(struct dw_edma_pcie_ep_dma_view *view,
+ struct dw_edma_pcie_data *pdata,
+ u16 table_off, u16 entry_size, u16 ch_cnt,
+ bool write)
+{
+ struct dw_edma_block *desc_blocks = write ? pdata->ll_wr : pdata->ll_rd;
+ struct dw_edma_block *data_blocks = write ? pdata->dt_wr : pdata->dt_rd;
+ u32 ctrl;
+ u16 i;
+ int ret;
+
+ for (i = 0; i < ch_cnt; i++) {
+ struct dw_edma_block *desc_block = &desc_blocks[i];
+ struct dw_edma_block *data_block = &data_blocks[i];
+ u16 off = table_off + i * entry_size;
+ u16 field, lo, hi;
+
+ field = off + PCI_EP_DMA_METADATA_CH_CTRL;
+ ctrl = dw_edma_pcie_ep_dma_readl(view, field);
+ if (FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_HW_CH, ctrl) != i)
+ return -EOPNOTSUPP;
+
+ desc_block->bar =
+ FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_DESC_BAR, ctrl);
+ lo = off + PCI_EP_DMA_METADATA_CH_DESC_OFF_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_DESC_OFF_HI;
+ ret = dw_edma_pcie_ep_dma_read_off(view, lo, hi,
+ &desc_block->off);
+ if (ret)
+ return ret;
+ field = off + PCI_EP_DMA_METADATA_CH_DESC_SIZE;
+ desc_block->sz = dw_edma_pcie_ep_dma_readl(view, field);
+ lo = off + PCI_EP_DMA_METADATA_CH_DESC_ADDR_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_DESC_ADDR_HI;
+ desc_block->paddr =
+ dw_edma_pcie_ep_dma_read64(view, lo, hi);
+ desc_block->paddr_valid = true;
+ if (!dw_edma_pcie_valid_block(view->pdev, desc_block))
+ return -EINVAL;
+
+ *data_block = (struct dw_edma_block) { .bar = NO_BAR };
+ if (!(ctrl & PCI_EP_DMA_METADATA_CH_CTRL_AUX_VALID))
+ continue;
+
+ data_block->bar =
+ FIELD_GET(PCI_EP_DMA_METADATA_CH_CTRL_AUX_BAR, ctrl);
+ lo = off + PCI_EP_DMA_METADATA_CH_AUX_OFF_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_AUX_OFF_HI;
+ ret = dw_edma_pcie_ep_dma_read_off(view, lo, hi,
+ &data_block->off);
+ if (ret)
+ return ret;
+ field = off + PCI_EP_DMA_METADATA_CH_AUX_SIZE;
+ data_block->sz = dw_edma_pcie_ep_dma_readl(view, field);
+ lo = off + PCI_EP_DMA_METADATA_CH_AUX_ADDR_LO;
+ hi = off + PCI_EP_DMA_METADATA_CH_AUX_ADDR_HI;
+ data_block->paddr =
+ dw_edma_pcie_ep_dma_read64(view, lo, hi);
+ data_block->paddr_valid = true;
+ if (!dw_edma_pcie_valid_block(view->pdev, data_block))
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int
+dw_edma_pcie_ep_dma_wait_ready(struct dw_edma_pcie_ep_dma_view *view)
+{
+ u32 val;
+
+ return read_poll_timeout(dw_edma_pcie_ep_dma_readl, val,
+ val & PCI_EP_DMA_METADATA_CTRL_READY,
+ DW_PCIE_EP_DMA_READY_POLL_US,
+ DW_PCIE_EP_DMA_READY_TIMEOUT_US, false,
+ view, PCI_EP_DMA_METADATA_CTRL);
+}
+
+static int
+dw_edma_pcie_validate_ep_dma_metadata(struct dw_edma_pcie_ep_dma_view *view,
+ u32 *metadata_ctrl, u8 *reg_layout_data)
+{
+ size_t table_size, table_end;
+ enum pci_barno reg_bar;
+ u16 len, entry_size;
+ u16 wr_ch_cnt, rd_ch_cnt;
+ u8 layout, layout_data;
+ u32 val;
+
+ val = dw_edma_pcie_ep_dma_readl(view, 0);
+ if (val != PCI_EP_DMA_METADATA_MAGIC)
+ return -ENODEV;
+
+ val = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_HDR);
+ if (FIELD_GET(PCI_EP_DMA_METADATA_HDR_REV, val) !=
+ PCI_EP_DMA_METADATA_REV)
+ return -EINVAL;
+
+ len = FIELD_GET(PCI_EP_DMA_METADATA_HDR_LEN_FIELD, val);
+ if (len < PCI_EP_DMA_METADATA_HDR_LEN)
+ return -EINVAL;
+ if (len > view->limit)
+ return -EINVAL;
+
+ val = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_REG_LAYOUT);
+ layout = FIELD_GET(PCI_EP_DMA_METADATA_REG_LAYOUT_ID, val);
+ if (layout != PCI_EP_DMA_METADATA_REG_LAYOUT_DW_EDMA)
+ return -EOPNOTSUPP;
+
+ layout_data = FIELD_GET(PCI_EP_DMA_METADATA_REG_LAYOUT_DATA, val);
+ if (layout_data == EDMA_MF_EDMA_LEGACY ||
+ layout_data == EDMA_MF_HDMA_NATIVE)
+ return -EOPNOTSUPP;
+ if (layout_data != EDMA_MF_EDMA_UNROLL &&
+ layout_data != EDMA_MF_HDMA_COMPAT)
+ return -EINVAL;
+
+ val = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_CTRL);
+ reg_bar = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_REG_BAR, val);
+ if (!dw_edma_pcie_valid_bar(reg_bar))
+ return -EINVAL;
+
+ wr_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT, val);
+ rd_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT, val);
+ if (!wr_ch_cnt && !rd_ch_cnt)
+ return -EINVAL;
+ if (wr_ch_cnt > EDMA_MAX_WR_CH || rd_ch_cnt > EDMA_MAX_RD_CH)
+ return -EINVAL;
+
+ entry_size = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE, val);
+ if (entry_size < PCI_EP_DMA_METADATA_CH_ENTRY_SIZE ||
+ entry_size % sizeof(u32))
+ return -EINVAL;
+
+ if (check_mul_overflow((size_t)(wr_ch_cnt + rd_ch_cnt),
+ (size_t)entry_size, &table_size) ||
+ check_add_overflow((size_t)PCI_EP_DMA_METADATA_HDR_LEN,
+ table_size, &table_end) ||
+ table_end > len)
+ return -EINVAL;
+
+ if (metadata_ctrl)
+ *metadata_ctrl = val;
+ if (reg_layout_data)
+ *reg_layout_data = layout_data;
+
+ return 0;
+}
+
+static int
+dw_edma_pcie_parse_ep_dma_data(struct dw_edma_pcie_ep_dma_view *view,
+ struct dw_edma_pcie_data *pdata)
+{
+ u32 ctrl, reg_sz;
+ u8 reg_layout_data;
+ u64 reg_off;
+ u16 wr_table, rd_table, entry_size;
+ u16 wr_ch_cnt, rd_ch_cnt;
+ int ret;
+
+ ret = dw_edma_pcie_validate_ep_dma_metadata(view, &ctrl,
+ ®_layout_data);
+ if (ret)
+ return ret;
+
+ pci_dbg(view->pdev, "Detected PCI endpoint DMA BAR metadata\n");
+
+ pdata->mf = reg_layout_data;
+ pdata->rg.bar = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_REG_BAR, ctrl);
+
+ wr_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_WR_CH_COUNT, ctrl);
+ rd_ch_cnt = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_RD_CH_COUNT, ctrl);
+ pdata->wr_ch_cnt = min_t(u16, pdata->wr_ch_cnt, wr_ch_cnt);
+ pdata->rd_ch_cnt = min_t(u16, pdata->rd_ch_cnt, rd_ch_cnt);
+ pdata->irqs = pdata->wr_ch_cnt + pdata->rd_ch_cnt;
+ reg_off = dw_edma_pcie_ep_dma_read64(view,
+ PCI_EP_DMA_METADATA_REG_OFF_LO,
+ PCI_EP_DMA_METADATA_REG_OFF_HI);
+ reg_sz = dw_edma_pcie_ep_dma_readl(view, PCI_EP_DMA_METADATA_REG_SIZE);
+ if (reg_off > type_max(pdata->rg.off) ||
+ !dw_edma_pcie_valid_bar_range(view->pdev, pdata->rg.bar,
+ reg_off, reg_sz))
+ return -EINVAL;
+ pdata->rg.off = reg_off;
+ pdata->rg.sz = reg_sz;
+
+ entry_size = FIELD_GET(PCI_EP_DMA_METADATA_CTRL_CH_ENTRY_SIZE, ctrl);
+ wr_table = PCI_EP_DMA_METADATA_HDR_LEN;
+ rd_table = PCI_EP_DMA_METADATA_HDR_LEN + wr_ch_cnt * entry_size;
+
+ ret = dw_edma_pcie_parse_ep_dma_ch_table(view, pdata, wr_table,
+ entry_size, pdata->wr_ch_cnt,
+ true);
+ if (ret)
+ return ret;
+
+ return dw_edma_pcie_parse_ep_dma_ch_table(view, pdata, rd_table,
+ entry_size,
+ pdata->rd_ch_cnt, false);
+}
+
+static int
+dw_edma_pcie_parse_ep_dma_caps(struct pci_dev *pdev,
+ struct dw_edma_pcie_data *pdata, bool *non_ll)
+{
+ struct dw_edma_pcie_ep_dma_view metadata_view;
+ void __iomem *base;
+ resource_size_t bar_len;
+ enum pci_barno bar;
+ u32 ctrl;
+ int ret;
+
+ for (bar = BAR_0; bar < PCI_STD_NUM_BARS; bar++) {
+ if (!dw_edma_pcie_ep_dma_bar_scannable(pdev, bar))
+ continue;
+
+ bar_len = pci_resource_len(pdev, bar);
+ base = pci_iomap_range(pdev, bar, 0, 0);
+ if (!base)
+ continue;
+
+ metadata_view = (struct dw_edma_pcie_ep_dma_view) {
+ .pdev = pdev,
+ .base = base,
+ .limit = bar_len,
+ };
+ ret = dw_edma_pcie_validate_ep_dma_metadata(&metadata_view,
+ NULL, NULL);
+ if (ret == -ENODEV) {
+ pci_iounmap(metadata_view.pdev, base);
+ continue;
+ }
+ if (ret) {
+ pci_iounmap(metadata_view.pdev, base);
+ return ret;
+ }
+
+ ctrl = dw_edma_pcie_ep_dma_readl(&metadata_view,
+ PCI_EP_DMA_METADATA_CTRL);
+ ctrl |= PCI_EP_DMA_METADATA_CTRL_HOST_REQ;
+ dw_edma_pcie_ep_dma_writel(&metadata_view,
+ PCI_EP_DMA_METADATA_CTRL, ctrl);
+
+ ret = dw_edma_pcie_ep_dma_wait_ready(&metadata_view);
+ if (ret) {
+ pci_iounmap(metadata_view.pdev, base);
+ return ret;
+ }
+
+ ret = dw_edma_pcie_parse_ep_dma_data(&metadata_view, pdata);
+ pci_iounmap(metadata_view.pdev, base);
+
+ return ret;
+ }
+
+ return -ENODEV;
+}
+
static int
dw_edma_pcie_parse_synopsys_caps(struct pci_dev *pdev,
struct dw_edma_pcie_data *pdata, bool *non_ll)
@@ -357,6 +712,14 @@ dw_edma_pcie_parse_xilinx_caps(struct pci_dev *pdev,
return 0;
}
+static const struct dw_edma_pcie_match_data ep_dma_match_data = {
+ .data = &ep_dma_data,
+ .parse_caps = dw_edma_pcie_parse_ep_dma_caps,
+ .flags = DW_EDMA_PCIE_F_REG_OFFSET | DW_EDMA_PCIE_F_RAW_SLAVE_ADDR,
+ .chip_flags = DW_EDMA_CHIP_PARTIAL,
+ .default_irq_mode = DW_EDMA_CH_IRQ_REMOTE,
+};
+
static u64 dw_edma_get_phys_addr(struct pci_dev *pdev,
const struct dw_edma_pcie_match_data *match,
struct dw_edma_pcie_data *pdata,
@@ -384,7 +747,7 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev,
const struct pci_device_id *pid)
{
const struct dw_edma_pcie_match_data *match = (void *)pid->driver_data;
- const struct dw_edma_pcie_data *pdata = match->data;
+ const struct dw_edma_pcie_data *pdata;
struct device *dev = &pdev->dev;
struct dw_edma_chip *chip;
int err, nr_irqs;
@@ -398,6 +761,10 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev,
return err;
}
+ if (!match)
+ match = &ep_dma_match_data;
+ pdata = match->data;
+
struct dw_edma_pcie_data *dma_data __free(kfree) =
kmemdup(pdata, sizeof(*dma_data), GFP_KERNEL);
if (!dma_data)
--
2.51.0
^ permalink raw reply related
* [PATCH 0/3] PCI: endpoint: Add PCI DMA endpoint function (part 3/3)
From: Koichiro Den @ 2026-05-21 6:36 UTC (permalink / raw)
To: Manivannan Sadhasivam, Krzysztof Wilczyński,
Kishon Vijay Abraham I, Bjorn Helgaas, Jonathan Corbet,
Shuah Khan, Vinod Koul, Frank Li, Arnd Bergmann, Damien Le Moal,
Niklas Cassel
Cc: Marek Vasut, Yoshihiro Shimoda, linux-pci, linux-doc,
linux-kernel, dmaengine
Hi,
This is part 3 of three series for PCI endpoint DMA.
The three series are:
* part 1: dmaengine: dw-edma: Prepare for PCI EP DMA
* part 2: PCI: endpoint: Expose endpoint DMA resources
* part 3: PCI: endpoint: Add PCI DMA endpoint function
This series adds the host-side metadata parser, the pci-epf-dma endpoint
function driver, and documentation.
The endpoint function exposes selected endpoint-integrated DMA channels as
a separate PCI DMA controller function. The host-side dw-edma-pcie driver
discovers the BAR metadata, requests the final layout, and registers the
exposed channels with DMAengine. Host clients then submit transfers through
the regular DMAengine API. The endpoint function keeps the metadata BAR
stable and uses a separate DMA window BAR for resources that need dynamic
subrange mappings.
No fixed PCI ID is assigned by this series. Users provide the PCI
vendor/device ID through configfs and bind dw-edma-pcie explicitly, for
example with driver_override.
Dependencies
============
This series depends on parts 1 and 2, applied on top of pci/endpoint:
[PATCH 00/12] dmaengine: dw-edma: Prepare for PCI EP DMA (part 1/3)
https://lore.kernel.org/all/20260521063115.2842238-1-den@valinux.co.jp/
[PATCH 0/3] PCI: endpoint: Expose endpoint DMA resources (part 2/3)
https://lore.kernel.org/all/20260521063405.2842644-1-den@valinux.co.jp/
Note
====
This series touches both dmaengine and PCI endpoint code. I kept the
dw-edma-pcie metadata parser together with the endpoint function so the
metadata producer and consumer can be reviewed in one place.
If the general direction looks acceptable, the dw-edma-pcie patch may need
a dmaengine Ack if this series is routed through the PCI endpoint tree.
Tested on
=========
The RC-to-EP data path was tested with a small out-of-tree DMAengine
client. The host submits a DMA_MEM_TO_DEV transfer through dw-edma-pcie,
which uses a DesignWare eDMA read channel to copy host memory into
endpoint memory.
Tested with:
* R-Car S4 as endpoint and R-Car S4 as root complex
* RK3588 as endpoint and CD8180 as root complex
Best regards,
Koichiro
Koichiro Den (3):
dmaengine: dw-edma-pcie: Discover endpoint DMA metadata
PCI: endpoint: Add DMA endpoint function
Documentation: PCI: Add PCI DMA endpoint function documentation
Documentation/PCI/endpoint/index.rst | 2 +
.../PCI/endpoint/pci-dma-function.rst | 182 +++
Documentation/PCI/endpoint/pci-dma-howto.rst | 200 +++
drivers/dma/dw-edma/dw-edma-pcie.c | 369 ++++-
drivers/pci/endpoint/functions/Kconfig | 14 +
drivers/pci/endpoint/functions/Makefile | 1 +
drivers/pci/endpoint/functions/pci-epf-dma.c | 1361 +++++++++++++++++
7 files changed, 2128 insertions(+), 1 deletion(-)
create mode 100644 Documentation/PCI/endpoint/pci-dma-function.rst
create mode 100644 Documentation/PCI/endpoint/pci-dma-howto.rst
create mode 100644 drivers/pci/endpoint/functions/pci-epf-dma.c
--
2.51.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox