[PATCH bpf-next v3 0/5] bpf, x86: enable EXECMEM_ROX_CACHE for BPF allocations

[PATCH bpf-next v3 0/5] bpf, x86: enable EXECMEM_ROX_CACHE for BPF allocations

[Mike Rapoport (Microsoft)]

Hi,

BPF allocations of executable memory on x86 are essentially read-only. Most
paths that call bpf_jit_alloc_exec() immediately make it ROX with
set_memory_rox().

The code generation, at least on x86, uses separately allocated writable
buffers and then updates the actual text memory with text_poke().

These patches do several small adjustments to how BPF allocates executable
memory and enable EXECMEM_ROX_CACHE for BPF allocations on x86.

---
v3 changes:
* replace vmalloc() with vzalloc() in bpf_dispatcher_change_prog()
* rebase on the current bpf-next

v2: https://patch.msgid.link/20260711-execmem-x86-rox-bpf-v0-v2-0-bfd956d35119@kernel.org
* rebase on the current bpf-next

v1: https://patch.msgid.link/20260626-execmem-x86-rox-bpf-v0-v1-0-45a0b0ed4fe9@kernel.org

---

---
Mike Rapoport (Microsoft) (5):
      bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()
      bpf: drop __weak from bpf_jit_alloc_exec() and bpf_jit_free_exec()
      bpf: alloc_prog_pack(): skip ROX management for already ROX memory
      bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable
      x86/bpf: enable EXECMEM_ROX_CACHE for BPF allocations

 arch/x86/mm/init.c          |  4 ++--
 arch/x86/net/bpf_jit_comp.c |  5 ++---
 include/linux/filter.h      |  1 +
 kernel/bpf/core.c           | 30 +++++++++++++++++++++---------
 kernel/bpf/dispatcher.c     |  5 ++++-
 5 files changed, 30 insertions(+), 15 deletions(-)
---
base-commit: d1f4b56417a3dc1a0600f960b14f46bd25eda89d
change-id: 20260626-execmem-x86-rox-bpf-v0-b4241ade80df

--
Sincerely yours,
Mike.

[PATCH bpf-next v3 1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

[Mike Rapoport (Microsoft)]

bpf_dispatcher->rw_image is a temporary writable buffer that
arch_prepare_bpf_dispatcher() fills and then copies into
bpf_dispatcher->image using bpf_arch_text_copy().

The rel32 offsets emitted by emit_bpf_dispatcher() are calculated against
->image, so ->rw_image does not need to live in the module address range.

Allocate ->rw_image with vzalloc() to avoid permissions dance when
EXECMEM_BPF will be backed by ROX caches.

Using vzalloc() rather than vmalloc() ensures that the memory that
bpf_dispatcher_update() unconditionally copies into the executable buffer
is zeroed, which is not ideal but still better than random memory returned
by the existing bpf_jit_alloc_exec() or plain vmalloc().

Switching from bpf_jit_alloc_exec() to vzalloc() also saves a bit of
space in the more scarce module address space.

Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 kernel/bpf/dispatcher.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/dispatcher.c b/kernel/bpf/dispatcher.c
index ea2d60dc1fee..79f0c222c583 100644
--- a/kernel/bpf/dispatcher.c
+++ b/kernel/bpf/dispatcher.c
@@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
 		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
 		if (!d->image)
 			goto out;
-		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
+		/* d->rw_image doesn't need to be in module memory range, so we
+		 * can use vzalloc.
+		 */
+		d->rw_image = vzalloc(PAGE_SIZE);
 		if (!d->rw_image) {
 			bpf_prog_pack_free(d->image, PAGE_SIZE);
 			d->image = NULL;

-- 
2.53.0

[PATCH bpf-next v3 2/5] bpf: drop __weak from bpf_jit_alloc_exec() and bpf_jit_free_exec()

[Mike Rapoport (Microsoft)]

bpf_jit_alloc_exec() and bpf_jit_free_exec() are wrappers for the
corresponding execmem APIs.

Architectures define the properties of the memory range needed by BPF in
their initialization of execmem and don't need to override neither of
them.

Drop the __weak qualifier from bpf_jit_alloc_exec() and
bpf_jit_free_exec().

Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 kernel/bpf/core.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 47fe047ad30b..fc75625dc951 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1116,12 +1116,12 @@ void bpf_jit_uncharge_modmem(u32 size)
 	atomic_long_sub(size, &bpf_jit_current);
 }
 
-void *__weak bpf_jit_alloc_exec(unsigned long size)
+void *bpf_jit_alloc_exec(unsigned long size)
 {
 	return execmem_alloc(EXECMEM_BPF, size);
 }
 
-void __weak bpf_jit_free_exec(void *addr)
+void bpf_jit_free_exec(void *addr)
 {
 	execmem_free(addr);
 }

-- 
2.53.0

[PATCH bpf-next v3 3/5] bpf: alloc_prog_pack(): skip ROX management for already ROX memory

[Mike Rapoport (Microsoft)]

execmem_alloc() can return ROX memory that is already filled with
architecture defined trapping instructions.

In preparation for enabling this mode for BPF on x86, make sure that there
is no redundant management of the ROX memory.

There is no need to fill allocated memory with trapping instructions, to
request permissions reset on free and to set ROX permissions as this all
is handled by execmem_alloc().

Add bpf_jit_mem_is_rox() wrapper for execmem_is_rox(), use it to check if
execmem_alloc() returns ROX memory and skip the redundant steps in that
case.

Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 kernel/bpf/core.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index fc75625dc951..1b89c18cf246 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -916,6 +916,11 @@ static LIST_HEAD(pack_list);
 
 #define BPF_PROG_CHUNK_COUNT (BPF_PROG_PACK_SIZE / BPF_PROG_CHUNK_SIZE)
 
+static bool bpf_jit_mem_is_rox(void)
+{
+	return execmem_is_rox(EXECMEM_BPF);
+}
+
 static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_insns)
 {
 	struct bpf_prog_pack *pack;
@@ -927,16 +932,18 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins
 	pack->ptr = bpf_jit_alloc_exec(BPF_PROG_PACK_SIZE);
 	if (!pack->ptr)
 		goto out;
-	bpf_fill_ill_insns(pack->ptr, BPF_PROG_PACK_SIZE);
 	bitmap_zero(pack->bitmap, BPF_PROG_PACK_SIZE / BPF_PROG_CHUNK_SIZE);
 
 	if (static_branch_unlikely(&bpf_pred_flush_enabled))
 		pack->arch_flush_needed = true;
-	set_vm_flush_reset_perms(pack->ptr);
-	err = set_memory_rox((unsigned long)pack->ptr,
-			     BPF_PROG_PACK_SIZE / PAGE_SIZE);
-	if (err)
-		goto out;
+	if (!bpf_jit_mem_is_rox()) {
+		bpf_fill_ill_insns(pack->ptr, BPF_PROG_PACK_SIZE);
+		set_vm_flush_reset_perms(pack->ptr);
+		err = set_memory_rox((unsigned long)pack->ptr,
+				     BPF_PROG_PACK_SIZE / PAGE_SIZE);
+		if (err)
+			goto out;
+	}
 	list_add_tail(&pack->list, &pack_list);
 	return pack;
 
@@ -965,7 +972,7 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool
 			pr_warn_once("BPF: Predictors not flushed for allocations greater than BPF_PROG_PACK_SIZE\n");
 		size = round_up(size, PAGE_SIZE);
 		ptr = bpf_jit_alloc_exec(size);
-		if (ptr) {
+		if (ptr && !bpf_jit_mem_is_rox()) {
 			int err;
 
 			bpf_fill_ill_insns(ptr, size);

-- 
2.53.0

[PATCH bpf-next v3 4/5] bpf, x86: make sure allocation in arch_bpf_trampoline_size() is writable

[Mike Rapoport (Microsoft)]

arch_bpf_trampoline_size() allocates a buffer to get actual size required
for a trampoline.

This buffer must be in the module address space because
__arch_prepare_bpf_trampoline() calculates  rel32 offsets relatively to
that buffer.

In preparation for enabling ROX mode for EXECMEM_BPF make sure that the
allocated memory is writable.

Add bpf_jit_alloc_exec_rw() wrapper for execmem_alloc_rw() and use it for
buffer allocation in arch_bpf_trampoline_size().

Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 arch/x86/net/bpf_jit_comp.c | 5 ++---
 include/linux/filter.h      | 1 +
 kernel/bpf/core.c           | 5 +++++
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index de7515ea1bea..b2feec81e231 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3703,13 +3703,12 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
 	int ret;
 
 	/* Allocate a temporary buffer for __arch_prepare_bpf_trampoline().
-	 * This will NOT cause fragmentation in direct map, as we do not
-	 * call set_memory_*() on this buffer.
 	 *
 	 * We cannot use kvmalloc here, because we need image to be in
 	 * module memory range.
+	 * Since it must be writable use bpf_jit_alloc_exec_rw().
 	 */
-	image = bpf_jit_alloc_exec(PAGE_SIZE);
+	image = bpf_jit_alloc_exec_rw(PAGE_SIZE);
 	if (!image)
 		return -ENOMEM;
 
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 14acb2455746..32d5297c557e 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1333,6 +1333,7 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr,
 void bpf_jit_binary_free(struct bpf_binary_header *hdr);
 u64 bpf_jit_alloc_exec_limit(void);
 void *bpf_jit_alloc_exec(unsigned long size);
+void *bpf_jit_alloc_exec_rw(unsigned long size);
 void bpf_jit_free_exec(void *addr);
 void bpf_jit_free(struct bpf_prog *fp);
 struct bpf_binary_header *
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 1b89c18cf246..e2076667b245 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -1128,6 +1128,11 @@ void *bpf_jit_alloc_exec(unsigned long size)
 	return execmem_alloc(EXECMEM_BPF, size);
 }
 
+void *bpf_jit_alloc_exec_rw(unsigned long size)
+{
+	return execmem_alloc_rw(EXECMEM_BPF, size);
+}
+
 void bpf_jit_free_exec(void *addr)
 {
 	execmem_free(addr);

-- 
2.53.0

[PATCH bpf-next v3 5/5] x86/bpf: enable EXECMEM_ROX_CACHE for BPF allocations

[Mike Rapoport (Microsoft)]

BPF core and x86 JIT use text poking and temporary writable buffers and
thus can handle ROX memory.

Enable ROX cache for EXECMEM_BPF when configuration and CPU features allow
that.

Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
---
 arch/x86/mm/init.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index fb67217fddcd..079f8c7e9e3c 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -1107,10 +1107,10 @@ struct execmem_info __init *execmem_arch_setup(void)
 				.alignment = MODULE_ALIGN,
 			},
 			[EXECMEM_BPF] = {
-				.flags	= EXECMEM_KASAN_SHADOW,
+				.flags	= flags,
 				.start	= start,
 				.end	= MODULES_END,
-				.pgprot	= PAGE_KERNEL,
+				.pgprot	= pgprot,
 				.alignment = MODULE_ALIGN,
 			},
 			[EXECMEM_MODULE_DATA] = {

-- 
2.53.0

Re: [PATCH bpf-next v3 1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1166 bytes --]

> diff --git a/kernel/bpf/dispatcher.c b/kernel/bpf/dispatcher.c
> index ea2d60dc1fee..79f0c222c583 100644
> --- a/kernel/bpf/dispatcher.c
> +++ b/kernel/bpf/dispatcher.c
> @@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
>  		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
>  		if (!d->image)
>  			goto out;
> -		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
> +		/* d->rw_image doesn't need to be in module memory range, so we
> +		 * can use vzalloc.
> +		 */
> +		d->rw_image = vzalloc(PAGE_SIZE);

This isn't a bug, but for multi-line comments under kernel/bpf/ the opening
/* is expected to sit on its own line, so would this read better as:

	/*
	 * d->rw_image doesn't need to be in module memory range, so we
	 * can use vzalloc.
	 */

>  		if (!d->rw_image) {
>  			bpf_prog_pack_free(d->image, PAGE_SIZE);
>  			d->image = NULL;


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29482648692

Re: [PATCH bpf-next v3 1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

[Mike Rapoport]

On Thu, Jul 16, 2026 at 08:52:25AM +0000, bot+bpf-ci@kernel.org wrote:
> > diff --git a/kernel/bpf/dispatcher.c b/kernel/bpf/dispatcher.c
> > index ea2d60dc1fee..79f0c222c583 100644
> > --- a/kernel/bpf/dispatcher.c
> > +++ b/kernel/bpf/dispatcher.c
> > @@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
> >  		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
> >  		if (!d->image)
> >  			goto out;
> > -		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
> > +		/* d->rw_image doesn't need to be in module memory range, so we
> > +		 * can use vzalloc.
> > +		 */
> > +		d->rw_image = vzalloc(PAGE_SIZE);
> 
> This isn't a bug, but for multi-line comments under kernel/bpf/ the opening
> /* is expected to sit on its own line, so would this read better as:
> 
> 	/*
> 	 * d->rw_image doesn't need to be in module memory range, so we
> 	 * can use vzalloc.
> 	 */

This really contradicts what I see in the code :/
 
> >  		if (!d->rw_image) {
> >  			bpf_prog_pack_free(d->image, PAGE_SIZE);
> >  			d->image = NULL;
> 
> 
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
> 
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29482648692


-- 
Sincerely yours,
Mike.

Re: [PATCH bpf-next v3 1/5] bpf: dispatcher: allocate bpf_dispatcher->rw_image with vzalloc()

[Kumar Kartikeya Dwivedi]

On Thu Jul 16, 2026 at 11:27 AM CEST, Mike Rapoport wrote:
> On Thu, Jul 16, 2026 at 08:52:25AM +0000, bot+bpf-ci@kernel.org wrote:
>> > diff --git a/kernel/bpf/dispatcher.c b/kernel/bpf/dispatcher.c
>> > index ea2d60dc1fee..79f0c222c583 100644
>> > --- a/kernel/bpf/dispatcher.c
>> > +++ b/kernel/bpf/dispatcher.c
>> > @@ -148,7 +148,10 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
>> >  		d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
>> >  		if (!d->image)
>> >  			goto out;
>> > -		d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
>> > +		/* d->rw_image doesn't need to be in module memory range, so we
>> > +		 * can use vzalloc.
>> > +		 */
>> > +		d->rw_image = vzalloc(PAGE_SIZE);
>>
>> This isn't a bug, but for multi-line comments under kernel/bpf/ the opening
>> /* is expected to sit on its own line, so would this read better as:
>>
>> 	/*
>> 	 * d->rw_image doesn't need to be in module memory range, so we
>> 	 * can use vzalloc.
>> 	 */
>
> This really contradicts what I see in the code :/
>

We ended up changing to the multi-line convention, but you can let it be. People
are out on vacation, hence things are moving slow, but I plan to get to this
series soon. Sorry about the delay.

>> >  		if (!d->rw_image) {
>> >  			bpf_prog_pack_free(d->image, PAGE_SIZE);
>> >  			d->image = NULL;
>>
>>
>> ---
>> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
>> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>>
>> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/29482648692