Re: [PATCH v3 5/5] LoongArch: BPF: Add bpf trampoline support for Loongarch

[Chenghao Duan <duanchenghao@kylinos.cn>]

On Wed, Jul 16, 2025 at 03:15:31PM -0700, Vincent Li wrote:
> On Wed, Jul 16, 2025 at 8:35 AM Vincent Li <vincent.mc.li@gmail.com> wrote:
> >
> > On Wed, Jul 16, 2025 at 7:51 AM Vincent Li <vincent.mc.li@gmail.com> wrote:
> > >
> > > On Tue, Jul 15, 2025 at 8:11 PM Vincent Li <vincent.mc.li@gmail.com> wrote:
> > > >
> > > > On Tue, Jul 15, 2025 at 2:54 PM Vincent Li <vincent.mc.li@gmail.com> wrote:
> > > > >
> > > > > On Tue, Jul 15, 2025 at 2:19 PM Vincent Li <vincent.mc.li@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Jul 10, 2025 at 8:03 AM Vincent Li <vincent.mc.li@gmail.com> wrote:
> > > > > > >
> > > > > > > On Thu, Jul 10, 2025 at 1:05 AM Chenghao Duan <duanchenghao@kylinos.cn> wrote:
> > > > > > > >
> > > > > > > > On Wed, Jul 09, 2025 at 12:08:27PM -0700, Vincent Li wrote:
> > > > > > > > > Hi Chenghao,
> > > > > > > > >
> > > > > > > > > I removed many email recipients for my question because I don't want
> > > > > > > > > to bother others, so you could  reply to this and not CC others who
> > > > > > > > > may not want to be bothered.  Thanks for working on this BPF
> > > > > > > > > trampoline feature for loongarch, I have been waiting for the bpf
> > > > > > > > > trampoline feature for loongarch for a long time.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Jul 8, 2025 at 10:52 PM Chenghao Duan <duanchenghao@kylinos.cn> wrote:
> > > > > > > > > >
> > > > > > > > > > BPF trampoline is the critical infrastructure of the BPF subsystem, acting
> > > > > > > > > > as a mediator between kernel functions and BPF programs. Numerous important
> > > > > > > > > > features, such as using BPF program for zero overhead kernel introspection,
> > > > > > > > > > rely on this key component.
> > > > > > > > > >
> > > > > > > > > > The related tests have passed, Including the following technical points:
> > > > > > > > > > 1. fentry
> > > > > > > > > > 2. fmod_ret
> > > > > > > > > > 3. fexit
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > My firewall project https://github.com/vincentmli/loongfire relies on
> > > > > > > > > https://github.com/xdp-project/xdp-tools/tree/main/lib/libxdp which
> > > > > > > > > relies on "BPF program extension" to attach multiple XDP program to
> > > > > > > > > one single network interface. Now I can't attach multiple XDP programs
> > > > > > > > > to one network interface through libxdp so I use bpf tail call as a
> > > > > > > > > workaround.  I am not a kernel developer and not  familiar with the
> > > > > > > > > kernel code so I am not sure if the "BPF program extension" is
> > > > > > > > > automatically supported after bpf trampoline is supported for
> > > > > > > > > loongarch, could you shed some light on this? thanks!
> > > > > > > > >
> > > > > > > >
> > > > > > > > Hi Vincent,
> > > > > > > >
> > > > > > > > I sincerely apologize, but I'm not quite sure about the specific
> > > > > > > > scenarios where the "BPF Program Extension" you mentioned is applied.
> > > > > > > > The trampoline feature allows multiple BPF programs to be attached to
> > > > > > > > the fentry/fexit points of kernel functions at runtime. I'm not sure if
> > > > > > > > this aligns with your technical approach.
> > > > > > > >
> > > > > > > > Chenghao
> > > > > > > >
> > > > > > >
> > > > > > > Thanks for the feedback, the
> > > > > > > https://github.com/xdp-project/xdp-tools/tree/main/lib/libxdp has
> > > > > > > section
> > > > > > > Kernel and BPF program feature compatibility.
> > > > > > >
> > > > > > > "The ability to attach multiple BPF programs to a single interface
> > > > > > > relies on the kernel “BPF program extension” feature which was
> > > > > > > introduced by commit be8704ff07d2 (“bpf: Introduce dynamic program
> > > > > > > extensions”) in the upstream kernel and first appeared in kernel
> > > > > > > release 5.6. To incrementally attach multiple programs, a further
> > > > > > > refinement added by commit 4a1e7c0c63e0 (“bpf: Support attaching
> > > > > > > freplace programs to multiple attach points”) is needed; this first
> > > > > > > appeared in the upstream kernel version 5.10. The functionality relies
> > > > > > > on the “BPF trampolines” feature which is unfortunately only available
> > > > > > > on the x86_64 architecture. In other words, kernels before 5.6 can
> > > > > > > only attach a single XDP program to each interface, kernels 5.6+ can
> > > > > > > attach multiple programs if they are all attached at the same time,
> > > > > > > and kernels 5.10 have full support for XDP multiprog on x86_64. On
> > > > > > > other architectures, only a single program can be attached to each
> > > > > > > interface."
> > > > > > >
> > > > > > > According to the above mentioned commit and  paragraph, I think “BPF
> > > > > > > program extension” should be  supported if the architecture supports
> > > > > > > trampoline, when libxdp author wrote the above paragraph, probably
> > > > > > > only X86_64 architecture supported BPF trampolines at that time. I
> > > > > > > will be glad to help testing that when loongarch BPF trampoline
> > > > > > > feature is supported :)
> > > > > > >
> > > > > >
> > > > > > I applied the patches on top of 6.16.0-rc6 on my local loongarch
> > > > > > machine, then tried to run xdp-filter program from xdp-tools project,
> > > > > > it looks not working with xdp-tools libxdp, got "Bad address" error:
> > > > > >
> > > > > > ./xdp-filter/xdp-filter load -m skb -vvv lo
> > > > > >
> > > > > >  libxdp: Loaded XDP program xdp_pass, got fd 22
> > > > > >  libxdp: Duplicated fd 22 to 23 for prog xdp_pass
> > > > > >  libxdp: Failed to attach test program to dispatcher: Bad address <====
> > > > > >  libxdp: Compatibility check for dispatcher program failed: Bad address
> > > > > >  libxdp: Falling back to loading single prog without dispatcher
> > > > > >  libxdp: Checking for kernel frags support
> > > > > >  libxdp: Loading XDP program 'xdp-dispatcher.o' from embedded object file
> > > > > >
> > > > > > the error is from xdp-tools lib/libxdp/libxdp.c
> > > > > >
> > > > > >         bpf_program__set_type(test_prog->bpf_prog, BPF_PROG_TYPE_EXT);
> > > > > >         bpf_program__set_expected_attach_type(test_prog->bpf_prog, 0);
> > > > > >         err = xdp_program__load(test_prog);
> > > > > >         if (err) {
> > > > > >                 char buf[100] = {};
> > > > > >                 libxdp_strerror(err, buf, sizeof(buf));
> > > > > >                 pr_debug("Failed to load program %s: %s\n",
> > > > > >                         xdp_program__name(test_prog), buf);
> > > > > >                 goto out;
> > > > > >         }
> > > > > >
> > > > > >         test_prog->link_fd = bpf_raw_tracepoint_open(NULL, test_prog->prog_fd);
> > > > > >         if (test_prog->link_fd < 0) {
> > > > > >                 err = -errno;
> > > > > >                 pr_debug("Failed to attach test program to dispatcher:
> > > > > > %s\n",  <======error Bad address
> > > > > >                          strerror(-err));
> > > > > >                 goto out;
> > > > > >         }
> > > > > >
> > > > >
> > > > > here is strace:
> > > > >
> > > > > 2754  write(2, "libxdp: Duplicated fd 22 to 23 for prog xdp_pass\n", 49) = 49
> > > > > 2754  bpf(BPF_RAW_TRACEPOINT_OPEN, {raw_tracepoint={name=NULL,
> > > > > prog_fd=23}}, 24) = -1 EFAULT (Bad address)
> > > > > 2754  write(2, " ", 1)                  = 1
> > > > > 2754  write(2, "libxdp: Failed to attach test program to dispatcher:
> > > > > Bad address\n", 65) = 65
> > > > >
> > > >
> > > > Sorry to make more noise, if I understand the code right, bpf syscall
> > > > of BPF_RAW_TRACEPOINT_OPEN code path looks like below, and it
> > > > eventually may call bpf_arch_text_poke() which would invoke the
> > > > loongarch bpf_arch_text_poke().  raw tracepoint could rely on the bpf
> > > > trampoline. I wonder if there is a way to print out the kernel stack
> > > > trace when executing xdp-filter command.
> > > >
> > > >
> > > > static int __sys_bpf(enum bpf_cmd cmd, bpfptr_t uattr, unsigned int size)
> > > > {
> > > >         union bpf_attr attr;
> > > >         int err;
> > > > ...
> > > >         case BPF_RAW_TRACEPOINT_OPEN:
> > > >                 err = bpf_raw_tracepoint_open(&attr); <==============
> > > >                 break;
> > > > ...
> > > > }
> > > >
> > > > static int bpf_raw_tracepoint_open(const union bpf_attr *attr)
> > > > {
> > > >         struct bpf_prog *prog;
> > > >         void __user *tp_name;
> > > >         __u64 cookie;
> > > > ...
> > > >         fd = bpf_raw_tp_link_attach(prog, tp_name, cookie); <===========
> > > >                 bpf_prog_put(prog);
> > > > ...
> > > > }
> > > >
> > > > static int bpf_raw_tp_link_attach(struct bpf_prog *prog,
> > > >                                   const char __user *user_tp_name, u64 cookie)
> > > > {
> > > >         struct bpf_link_primer link_primer;
> > > >         struct bpf_raw_tp_link *link;
> > > >         struct bpf_raw_event_map *btp;
> > > >         const char *tp_name;
> > > >         char buf[128];
> > > >         int err;
> > > >
> > > >         switch (prog->type) {
> > > >         case BPF_PROG_TYPE_TRACING:
> > > >         case BPF_PROG_TYPE_EXT: <==========
> > > >         case BPF_PROG_TYPE_LSM:
> > > >                 if (user_tp_name)
> > > >                         /* The attach point for this category of programs
> > > >                          * should be specified via btf_id during program load.
> > > >                          */
> > > >                         return -EINVAL;
> > > >                 if (prog->type == BPF_PROG_TYPE_TRACING &&
> > > >                     prog->expected_attach_type == BPF_TRACE_RAW_TP) {
> > > >                         tp_name = prog->aux->attach_func_name;
> > > >                         break;
> > > >                 }
> > > >                 return bpf_tracing_prog_attach(prog, 0, 0, 0);
> > > > <=====================
> > > > ...
> > > > }
> > > >
> > > >
> > > > static int bpf_tracing_prog_attach(struct bpf_prog *prog,
> > > >                                    int tgt_prog_fd,
> > > >                                    u32 btf_id,
> > > >                                    u64 bpf_cookie)
> > > > {
> > > >
> > > >
> > > >        err = bpf_trampoline_link_prog(&link->link, tr, tgt_prog);
> > > >
> > > > }
> > > >
> > > > int bpf_trampoline_link_prog(struct bpf_tramp_link *link,
> > > >                              struct bpf_trampoline *tr,
> > > >                              struct bpf_prog *tgt_prog)
> > > > {
> > > >         int err;
> > > >
> > > >         mutex_lock(&tr->mutex);
> > > >         err = __bpf_trampoline_link_prog(link, tr, tgt_prog);
> > > >         mutex_unlock(&tr->mutex);
> > > >         return err;
> > > > }
> > > >
> > > > static int __bpf_trampoline_link_prog(struct bpf_tramp_link *link,
> > > >                                       struct bpf_trampoline *tr,
> > > >                                       struct bpf_prog *tgt_prog)
> > > > {
> > > >         enum bpf_tramp_prog_type kind;
> > > >         struct bpf_tramp_link *link_exiting;
> > > >         int err = 0;
> > > >         int cnt = 0, i;
> > > >
> > > >         kind = bpf_attach_type_to_tramp(link->link.prog);
> > > >         if (tr->extension_prog)
> > > >                 /* cannot attach fentry/fexit if extension prog is attached.
> > > >                  * cannot overwrite extension prog either.
> > > >                  */
> > > >                 return -EBUSY;
> > > >
> > > >         for (i = 0; i < BPF_TRAMP_MAX; i++)
> > > >                 cnt += tr->progs_cnt[i];
> > > >
> > > >         if (kind == BPF_TRAMP_REPLACE) { <==========
> > > >                 /* Cannot attach extension if fentry/fexit are in use. */
> > > >                 if (cnt)
> > > >                         return -EBUSY;
> > > >                 err = bpf_freplace_check_tgt_prog(tgt_prog);
> > > >                 if (err)
> > > >                         return err;
> > > >                 tr->extension_prog = link->link.prog;
> > > >                 return bpf_arch_text_poke(tr->func.addr, BPF_MOD_JUMP, NULL,
> > > >                                           link->link.prog->bpf_func);
> > > > <===============
> > > >         }
> > > >
> > > > ...
> > > > }
> > > >
> > >
> > > To continue the fun of hacking :), to find a good use of this patch
> > > series to test out fentry, DeepSeek helped me  come up with the
> > > following method to fentry the bpf_trampoline_link_prog().
> > >
> > > SEC("fentry/bpf_trampoline_link_prog")
> > > int BPF_PROG(trace_trampoline_link,
> > >             struct bpf_tramp_link *link,
> > >             struct bpf_trampoline *tr,
> > >             struct bpf_prog *tgt_prog)
> > > {
> > >     enum bpf_tramp_prog_type kind = BPF_TRAMP_MAX;
> > >     const char *kind_str = "UNKNOWN";
> > >     char prog_name[16] = {0};
> > >     bool has_prog_name = false;
> > >
> > >     if (link && link->link.prog) {
> > >         // Get program name if available
> > >         if (link->link.prog->aux) {
> > >             bpf_probe_read_kernel_str(prog_name, sizeof(prog_name),
> > >                                     link->link.prog->aux->name);
> > >             has_prog_name = true;
> > >         }
> > >
> > >         // Determine kind
> > >         switch (link->link.prog->expected_attach_type) {
> > >             case BPF_TRACE_FENTRY:
> > >                 kind = BPF_TRAMP_FENTRY;
> > >                 kind_str = "FENTRY";
> > >                 break;
> > >             case BPF_TRACE_FEXIT:
> > >                 kind = BPF_TRAMP_FEXIT;
> > >                 kind_str = "FEXIT";
> > >                 break;
> > >             case BPF_MODIFY_RETURN:
> > >                 kind = BPF_TRAMP_MODIFY_RETURN;
> > >                 kind_str = "MODIFY_RETURN";
> > >                 break;
> > >             default:
> > >                 if (link->link.prog->type == BPF_PROG_TYPE_EXT) {
> > >                     kind = BPF_TRAMP_REPLACE;
> > >                     kind_str = "REPLACE";
> > >                 }
> > >                 break;
> > >         }
> > >     }
> > >
> > >     bpf_printk("__bpf_trampoline_link_prog: kind=%d (%s) prog=%s",
> > >                kind,
> > >                kind_str,
> > >                has_prog_name ? prog_name : "N/A");
> > >
> > >     return 0;
> > > }
> > >
> > > It confirms the xdp_pass program is set to  BPF_PROG_TYPE_EXT as
> > > expected and kind is BPF_TRAMP_REPLACE and bpf_arch_text_poke() would
> > > be invoked when I run 'xdp-filter load -m skb lo'
> > >
> > > [root@fedora c]# cat /sys/kernel/debug/tracing/trace_pipe
> > >            <...>-10081   [003] ...11 64926.801233: bpf_trace_printk:
> > > __bpf_trampoline_link_prog: kind=4 (REPLACE) prog=xdp_pass
> > >
> > >
> >
> > I think I found where the error coming from, now I used fentry on the
> > bpf_arch_text_poke()
> >
> > SEC("fentry/bpf_arch_text_poke")
> > int BPF_PROG(trace_text_poke_entry, void *ip, enum bpf_text_poke_type t,
> >              void *old_addr, void *new_addr)
> > {
> >     bpf_printk("bpf_arch_text_poke INVOKED: ip=%pS, type=%d, old=%pS, new=%pS",
> >                ip, t, old_addr, new_addr);
> >     return 0;
> > }
> >
> > SEC("fexit/bpf_arch_text_poke")
> > int BPF_PROG(trace_text_poke_exit,
> >              void *ip, enum bpf_text_poke_type t,
> >              void *old_addr, void *new_addr,
> >              int retval)  // <-- Return value is automatically passed
> > as last arg
> > {
> >     bpf_printk("bpf_arch_text_poke RETURNED: %d (ip=%pS)", retval, ip);
> >     return 0;
> > }
> >
> > it prints:
> > [root@fedora c]# cat /sys/kernel/debug/tracing/trace_pipe
> >            <...>-10426   [001] ...11 67878.110342: bpf_trace_printk:
> > bpf_arch_text_poke INVOKED:
> > ip=bpf_prog_3b185187f1855c4c_xdp_pass+0x0/0x60, type=1, old=0x0,
> > new=bpf_prog_3b185187f1855c4c_xdp_pass+0x0/0x60
> >            <...>-10426   [001] ...11 67878.110350: bpf_trace_printk:
> > bpf_arch_text_poke RETURNED: -14
> > (ip=bpf_prog_3b185187f1855c4c_xdp_pass+0x0/0x60)
> >
> > the error code -14 (-EFAULT) is  likely from line 1434 from this patch
> > series I applied
> >
> > 1417 int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type,
> > 1418                        void *old_addr, void *new_addr)
> > 1419 {
> > 1420         u32 old_insns[5] = {[0 ... 4] = INSN_NOP};
> > 1421         u32 new_insns[5] = {[0 ... 4] = INSN_NOP};
> > 1422         bool is_call = poke_type == BPF_MOD_CALL;
> > 1423         int ret;
> > 1424
> > 1425         if (!is_kernel_text((unsigned long)ip) &&
> > 1426                 !is_bpf_text_address((unsigned long)ip))
> > 1427                 return -ENOTSUPP;
> > 1428
> > 1429         ret = gen_jump_or_nops(old_addr, ip, old_insns, is_call);
> > 1430         if (ret)
> > 1431                 return ret;
> > 1432
> > 1433         if (memcmp(ip, old_insns, 5 * 4))
> > 1434                 return -EFAULT;
> > 1435
> > 1436         ret = gen_jump_or_nops(new_addr, ip, new_insns, is_call);
> > 1437         if (ret)
> > 1438                 return ret;
> > 1439
> > 1440         mutex_lock(&text_mutex);
> > 1441         if (memcmp(ip, new_insns, 5 * 4))
> > 1442                 ret = larch_insn_text_copy(ip, new_insns, 5 * 4);
> > 1443         mutex_unlock(&text_mutex);
> > 1444         return ret;
> > 1445 }
> >
> 
> in case it helps, I added following diff to print out the ip,
> old_insns before and after ret = gen_jump_or_nops(old_addr, ip,
> old_insns, is_call);
> ip and old_insns do not match before and after
> 
>  [   48.611736] INITIAL: ip [0x02c0840a 0x02ff0063 0x29c0e061
> 0x29c0c076 0x29c0a077]
> [   48.611743] INITIAL: old_insns [0x03400000 0x03400000 0x03400000
> 0x03400000 0x03400000]
> [   48.611745] POST_GEN: old_insns [0x03400000 0x03400000 0x03400000
> 0x03400000 0x03400000]
> [   48.611747] POST_GEN: ip [0x02c0840a 0x02ff0063 0x29c0e061
> 0x29c0c076 0x29c0a077]
> 
> diff below:
> 
> @@ -1426,10 +1432,26 @@ int bpf_arch_text_poke(void *ip, enum
> bpf_text_poke_type poke_type,
>                 !is_bpf_text_address((unsigned long)ip))
>                 return -ENOTSUPP;
> 
> +       /* DEBUG: Print initial state */
> +        pr_info("INITIAL: ip [0x%08x 0x%08x 0x%08x 0x%08x 0x%08x]\n",
> +               ((u32 *)ip)[0], ((u32 *)ip)[1], ((u32 *)ip)[2],
> +               ((u32 *)ip)[3], ((u32 *)ip)[4]);
> +        pr_info("INITIAL: old_insns [0x%08x 0x%08x 0x%08x 0x%08x 0x%08x]\n",
> +               old_insns[0], old_insns[1], old_insns[2],
> +               old_insns[3], old_insns[4]);
> +
>         ret = gen_jump_or_nops(old_addr, ip, old_insns, is_call);
>         if (ret)
>                 return ret;
> 
> +        /* DEBUG: Print post-gen_jump_or_nops */
> +        pr_info("POST_GEN: old_insns [0x%08x 0x%08x 0x%08x 0x%08x 0x%08x]\n",
> +               old_insns[0], old_insns[1], old_insns[2],
> +               old_insns[3], old_insns[4]);
> +        pr_info("POST_GEN: ip [0x%08x 0x%08x 0x%08x 0x%08x 0x%08x]\n",
> +               ((u32 *)ip)[0], ((u32 *)ip)[1], ((u32 *)ip)[2],
> +               ((u32 *)ip)[3], ((u32 *)ip)[4]);
> +
> 

Hi Vincent,

Thank you very much for your testing. We encountered the same situation
when testing fexit_bpf2bpf. After analysis, I think this is not an
issue with the trampoline. The problem you encountered is caused by
the old_addr=NULL passed to the bpf_arch_text_poke function, which
means the old address was not found. We are still analyzing this issue.

Chenghao
> > > >
> > > >
> > > > > deepseek says the  bpf syscall raw tracepoint name should not be NULL,
> > > > > something related to raw tracepoint?
> > > > >
> > > > >
> > > > > > xdp-tools/libxdp is a popular project to attach multiple XDP programs
> > > > > > to a single network interface, packaged in mainstream Linux
> > > > > > distributions, so it would be nice to get the loongarch kernel bpf
> > > > > > trampoline working with xdp-tools/libxdp :).
> > > > > >
> > > > > > > > > > Co-developed-by: George Guo <guodongtai@kylinos.cn>
> > > > > > > > > > Signed-off-by: George Guo <guodongtai@kylinos.cn>
> > > > > > > > > > Signed-off-by: Chenghao Duan <duanchenghao@kylinos.cn>
> > > > > > > > > > ---
> > > > > > > > > >  arch/loongarch/net/bpf_jit.c | 391 +++++++++++++++++++++++++++++++++++
> > > > > > > > > >  arch/loongarch/net/bpf_jit.h |   6 +
> > > > > > > > > >  2 files changed, 397 insertions(+)
> > > > > > > > > >
> > > > > > > > > > diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
> > > > > > > > > > index 9cb01f0b0..6820558af 100644
> > > > > > > > > > --- a/arch/loongarch/net/bpf_jit.c
> > > > > > > > > > +++ b/arch/loongarch/net/bpf_jit.c
> > > > > > > > > > @@ -7,6 +7,10 @@
> > > > > > > > > >  #include <linux/memory.h>
> > > > > > > > > >  #include "bpf_jit.h"
> > > > > > > > > >
> > > > > > > > > > +#define LOONGARCH_MAX_REG_ARGS 8
> > > > > > > > > > +#define LOONGARCH_FENTRY_NINSNS 2
> > > > > > > > > > +#define LOONGARCH_FENTRY_NBYTES (LOONGARCH_FENTRY_NINSNS * 4)
> > > > > > > > > > +
> > > > > > > > > >  #define REG_TCC                LOONGARCH_GPR_A6
> > > > > > > > > >  #define TCC_SAVED      LOONGARCH_GPR_S5
> > > > > > > > > >
> > > > > > > > > > @@ -1400,6 +1404,16 @@ static int gen_jump_or_nops(void *target, void *ip, u32 *insns, bool is_call)
> > > > > > > > > >                                   (unsigned long)ip, (unsigned long)target);
> > > > > > > > > >  }
> > > > > > > > > >
> > > > > > > > > > +static int emit_call(struct jit_ctx *ctx, u64 addr)
> > > > > > > > > > +{
> > > > > > > > > > +       u64 ip;
> > > > > > > > > > +
> > > > > > > > > > +       if (addr && ctx->image && ctx->ro_image)
> > > > > > > > > > +               ip = (u64)(ctx->image + ctx->idx);
> > > > > > > > > > +
> > > > > > > > > > +       return emit_jump_and_link(ctx, LOONGARCH_GPR_RA, ip, addr);
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > >  int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type,
> > > > > > > > > >                        void *old_addr, void *new_addr)
> > > > > > > > > >  {
> > > > > > > > > > @@ -1457,3 +1471,380 @@ void *bpf_arch_text_copy(void *dst, void *src, size_t len)
> > > > > > > > > >
> > > > > > > > > >         return dst;
> > > > > > > > > >  }
> > > > > > > > > > +
> > > > > > > > > > +static void store_args(struct jit_ctx *ctx, int nargs, int args_off)
> > > > > > > > > > +{
> > > > > > > > > > +       int i;
> > > > > > > > > > +
> > > > > > > > > > +       for (i = 0; i < nargs; i++) {
> > > > > > > > > > +               emit_insn(ctx, std, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off);
> > > > > > > > > > +               args_off -= 8;
> > > > > > > > > > +       }
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +static void restore_args(struct jit_ctx *ctx, int nargs, int args_off)
> > > > > > > > > > +{
> > > > > > > > > > +       int i;
> > > > > > > > > > +
> > > > > > > > > > +       for (i = 0; i < nargs; i++) {
> > > > > > > > > > +               emit_insn(ctx, ldd, LOONGARCH_GPR_A0 + i, LOONGARCH_GPR_FP, -args_off);
> > > > > > > > > > +               args_off -= 8;
> > > > > > > > > > +       }
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_link *l,
> > > > > > > > > > +                          int args_off, int retval_off,
> > > > > > > > > > +                          int run_ctx_off, bool save_ret)
> > > > > > > > > > +{
> > > > > > > > > > +       int ret;
> > > > > > > > > > +       u32 *branch;
> > > > > > > > > > +       struct bpf_prog *p = l->link.prog;
> > > > > > > > > > +       int cookie_off = offsetof(struct bpf_tramp_run_ctx, bpf_cookie);
> > > > > > > > > > +
> > > > > > > > > > +       if (l->cookie) {
> > > > > > > > > > +               move_imm(ctx, LOONGARCH_GPR_T1, l->cookie, false);
> > > > > > > > > > +               emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -run_ctx_off + cookie_off);
> > > > > > > > > > +       } else {
> > > > > > > > > > +               emit_insn(ctx, std, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_FP,
> > > > > > > > > > +                         -run_ctx_off + cookie_off);
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       /* arg1: prog */
> > > > > > > > > > +       move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false);
> > > > > > > > > > +       /* arg2: &run_ctx */
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_A1, LOONGARCH_GPR_FP, -run_ctx_off);
> > > > > > > > > > +       ret = emit_call(ctx, (const u64)bpf_trampoline_enter(p));
> > > > > > > > > > +       if (ret)
> > > > > > > > > > +               return ret;
> > > > > > > > > > +
> > > > > > > > > > +       /* store prog start time */
> > > > > > > > > > +       move_reg(ctx, LOONGARCH_GPR_S1, LOONGARCH_GPR_A0);
> > > > > > > > > > +
> > > > > > > > > > +       /* if (__bpf_prog_enter(prog) == 0)
> > > > > > > > > > +        *      goto skip_exec_of_prog;
> > > > > > > > > > +        *
> > > > > > > > > > +        */
> > > > > > > > > > +       branch = (u32 *)ctx->image + ctx->idx;
> > > > > > > > > > +       /* nop reserved for conditional jump */
> > > > > > > > > > +       emit_insn(ctx, nop);
> > > > > > > > > > +
> > > > > > > > > > +       /* arg1: &args_off */
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -args_off);
> > > > > > > > > > +       if (!p->jited)
> > > > > > > > > > +               move_imm(ctx, LOONGARCH_GPR_A1, (const s64)p->insnsi, false);
> > > > > > > > > > +       ret = emit_call(ctx, (const u64)p->bpf_func);
> > > > > > > > > > +       if (ret)
> > > > > > > > > > +               return ret;
> > > > > > > > > > +
> > > > > > > > > > +       if (save_ret) {
> > > > > > > > > > +               emit_insn(ctx, std, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -retval_off);
> > > > > > > > > > +               emit_insn(ctx, std, regmap[BPF_REG_0], LOONGARCH_GPR_FP, -(retval_off - 8));
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       /* update branch with beqz */
> > > > > > > > > > +       if (ctx->image) {
> > > > > > > > > > +               int offset = (void *)(&ctx->image[ctx->idx]) - (void *)branch;
> > > > > > > > > > +               *branch = larch_insn_gen_beq(LOONGARCH_GPR_A0, LOONGARCH_GPR_ZERO, offset);
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       /* arg1: prog */
> > > > > > > > > > +       move_imm(ctx, LOONGARCH_GPR_A0, (const s64)p, false);
> > > > > > > > > > +       /* arg2: prog start time */
> > > > > > > > > > +       move_reg(ctx, LOONGARCH_GPR_A1, LOONGARCH_GPR_S1);
> > > > > > > > > > +       /* arg3: &run_ctx */
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_A2, LOONGARCH_GPR_FP, -run_ctx_off);
> > > > > > > > > > +       ret = emit_call(ctx, (const u64)bpf_trampoline_exit(p));
> > > > > > > > > > +
> > > > > > > > > > +       return ret;
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_links *tl,
> > > > > > > > > > +                              int args_off, int retval_off, int run_ctx_off, u32 **branches)
> > > > > > > > > > +{
> > > > > > > > > > +       int i;
> > > > > > > > > > +
> > > > > > > > > > +       emit_insn(ctx, std, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_FP, -retval_off);
> > > > > > > > > > +       for (i = 0; i < tl->nr_links; i++) {
> > > > > > > > > > +               invoke_bpf_prog(ctx, tl->links[i], args_off, retval_off,
> > > > > > > > > > +                               run_ctx_off, true);
> > > > > > > > > > +               emit_insn(ctx, ldd, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -retval_off);
> > > > > > > > > > +               branches[i] = (u32 *)ctx->image + ctx->idx;
> > > > > > > > > > +               emit_insn(ctx, nop);
> > > > > > > > > > +       }
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +u64 bpf_jit_alloc_exec_limit(void)
> > > > > > > > > > +{
> > > > > > > > > > +       return VMALLOC_END - VMALLOC_START;
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +void *arch_alloc_bpf_trampoline(unsigned int size)
> > > > > > > > > > +{
> > > > > > > > > > +       return bpf_prog_pack_alloc(size, jit_fill_hole);
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +void arch_free_bpf_trampoline(void *image, unsigned int size)
> > > > > > > > > > +{
> > > > > > > > > > +       bpf_prog_pack_free(image, size);
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
> > > > > > > > > > +                                        const struct btf_func_model *m,
> > > > > > > > > > +                                        struct bpf_tramp_links *tlinks,
> > > > > > > > > > +                                        void *func_addr, u32 flags)
> > > > > > > > > > +{
> > > > > > > > > > +       int i;
> > > > > > > > > > +       int stack_size = 0, nargs = 0;
> > > > > > > > > > +       int retval_off, args_off, nargs_off, ip_off, run_ctx_off, sreg_off;
> > > > > > > > > > +       struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY];
> > > > > > > > > > +       struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT];
> > > > > > > > > > +       struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN];
> > > > > > > > > > +       int ret, save_ret;
> > > > > > > > > > +       void *orig_call = func_addr;
> > > > > > > > > > +       u32 **branches = NULL;
> > > > > > > > > > +
> > > > > > > > > > +       if (flags & (BPF_TRAMP_F_ORIG_STACK | BPF_TRAMP_F_SHARE_IPMODIFY))
> > > > > > > > > > +               return -ENOTSUPP;
> > > > > > > > > > +
> > > > > > > > > > +       /*
> > > > > > > > > > +        * FP + 8       [ RA to parent func ] return address to parent
> > > > > > > > > > +        *                    function
> > > > > > > > > > +        * FP + 0       [ FP of parent func ] frame pointer of parent
> > > > > > > > > > +        *                    function
> > > > > > > > > > +        * FP - 8       [ T0 to traced func ] return address of traced
> > > > > > > > > > +        *                    function
> > > > > > > > > > +        * FP - 16      [ FP of traced func ] frame pointer of traced
> > > > > > > > > > +        *                    function
> > > > > > > > > > +        *
> > > > > > > > > > +        * FP - retval_off  [ return value      ] BPF_TRAMP_F_CALL_ORIG or
> > > > > > > > > > +        *                    BPF_TRAMP_F_RET_FENTRY_RET
> > > > > > > > > > +        *                  [ argN              ]
> > > > > > > > > > +        *                  [ ...               ]
> > > > > > > > > > +        * FP - args_off    [ arg1              ]
> > > > > > > > > > +        *
> > > > > > > > > > +        * FP - nargs_off   [ regs count        ]
> > > > > > > > > > +        *
> > > > > > > > > > +        * FP - ip_off      [ traced func   ] BPF_TRAMP_F_IP_ARG
> > > > > > > > > > +        *
> > > > > > > > > > +        * FP - run_ctx_off [ bpf_tramp_run_ctx ]
> > > > > > > > > > +        *
> > > > > > > > > > +        * FP - sreg_off    [ callee saved reg  ]
> > > > > > > > > > +        *
> > > > > > > > > > +        */
> > > > > > > > > > +
> > > > > > > > > > +       if (m->nr_args > LOONGARCH_MAX_REG_ARGS)
> > > > > > > > > > +               return -ENOTSUPP;
> > > > > > > > > > +
> > > > > > > > > > +       if (flags & (BPF_TRAMP_F_ORIG_STACK | BPF_TRAMP_F_SHARE_IPMODIFY))
> > > > > > > > > > +               return -ENOTSUPP;
> > > > > > > > > > +
> > > > > > > > > > +       stack_size = 0;
> > > > > > > > > > +
> > > > > > > > > > +       /* room of trampoline frame to store return address and frame pointer */
> > > > > > > > > > +       stack_size += 16;
> > > > > > > > > > +
> > > > > > > > > > +       save_ret = flags & (BPF_TRAMP_F_CALL_ORIG | BPF_TRAMP_F_RET_FENTRY_RET);
> > > > > > > > > > +       if (save_ret) {
> > > > > > > > > > +               /* Save BPF R0 and A0 */
> > > > > > > > > > +               stack_size += 16;
> > > > > > > > > > +               retval_off = stack_size;
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       /* room of trampoline frame to store args */
> > > > > > > > > > +       nargs = m->nr_args;
> > > > > > > > > > +       stack_size += nargs * 8;
> > > > > > > > > > +       args_off = stack_size;
> > > > > > > > > > +
> > > > > > > > > > +       /* room of trampoline frame to store args number */
> > > > > > > > > > +       stack_size += 8;
> > > > > > > > > > +       nargs_off = stack_size;
> > > > > > > > > > +
> > > > > > > > > > +       /* room of trampoline frame to store ip address */
> > > > > > > > > > +       if (flags & BPF_TRAMP_F_IP_ARG) {
> > > > > > > > > > +               stack_size += 8;
> > > > > > > > > > +               ip_off = stack_size;
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       /* room of trampoline frame to store struct bpf_tramp_run_ctx */
> > > > > > > > > > +       stack_size += round_up(sizeof(struct bpf_tramp_run_ctx), 8);
> > > > > > > > > > +       run_ctx_off = stack_size;
> > > > > > > > > > +
> > > > > > > > > > +       stack_size += 8;
> > > > > > > > > > +       sreg_off = stack_size;
> > > > > > > > > > +
> > > > > > > > > > +       stack_size = round_up(stack_size, 16);
> > > > > > > > > > +
> > > > > > > > > > +       /* For the trampoline called from function entry */
> > > > > > > > > > +       /* RA and FP for parent function*/
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, -16);
> > > > > > > > > > +       emit_insn(ctx, std, LOONGARCH_GPR_RA, LOONGARCH_GPR_SP, 8);
> > > > > > > > > > +       emit_insn(ctx, std, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, 0);
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, 16);
> > > > > > > > > > +
> > > > > > > > > > +       /* RA and FP for traced function*/
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, -stack_size);
> > > > > > > > > > +       emit_insn(ctx, std, LOONGARCH_GPR_T0, LOONGARCH_GPR_SP, stack_size - 8);
> > > > > > > > > > +       emit_insn(ctx, std, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, stack_size - 16);
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, stack_size);
> > > > > > > > > > +
> > > > > > > > > > +       /* callee saved register S1 to pass start time */
> > > > > > > > > > +       emit_insn(ctx, std, LOONGARCH_GPR_S1, LOONGARCH_GPR_FP, -sreg_off);
> > > > > > > > > > +
> > > > > > > > > > +       /* store ip address of the traced function */
> > > > > > > > > > +       if (flags & BPF_TRAMP_F_IP_ARG) {
> > > > > > > > > > +               move_imm(ctx, LOONGARCH_GPR_T1, (const s64)func_addr, false);
> > > > > > > > > > +               emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -ip_off);
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       /* store nargs number*/
> > > > > > > > > > +       move_imm(ctx, LOONGARCH_GPR_T1, nargs, false);
> > > > > > > > > > +       emit_insn(ctx, std, LOONGARCH_GPR_T1, LOONGARCH_GPR_FP, -nargs_off);
> > > > > > > > > > +
> > > > > > > > > > +       store_args(ctx, nargs, args_off);
> > > > > > > > > > +
> > > > > > > > > > +       /* To traced function */
> > > > > > > > > > +       orig_call += LOONGARCH_FENTRY_NBYTES;
> > > > > > > > > > +       if (flags & BPF_TRAMP_F_CALL_ORIG) {
> > > > > > > > > > +               move_imm(ctx, LOONGARCH_GPR_A0, (const s64)im, false);
> > > > > > > > > > +               ret = emit_call(ctx, (const u64)__bpf_tramp_enter);
> > > > > > > > > > +               if (ret)
> > > > > > > > > > +                       return ret;
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       for (i = 0; i < fentry->nr_links; i++) {
> > > > > > > > > > +               ret = invoke_bpf_prog(ctx, fentry->links[i], args_off, retval_off,
> > > > > > > > > > +                                     run_ctx_off, flags & BPF_TRAMP_F_RET_FENTRY_RET);
> > > > > > > > > > +               if (ret)
> > > > > > > > > > +                       return ret;
> > > > > > > > > > +       }
> > > > > > > > > > +       if (fmod_ret->nr_links) {
> > > > > > > > > > +               branches  = kcalloc(fmod_ret->nr_links, sizeof(u32 *), GFP_KERNEL);
> > > > > > > > > > +               if (!branches)
> > > > > > > > > > +                       return -ENOMEM;
> > > > > > > > > > +
> > > > > > > > > > +               invoke_bpf_mod_ret(ctx, fmod_ret, args_off, retval_off,
> > > > > > > > > > +                                  run_ctx_off, branches);
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       if (flags & BPF_TRAMP_F_CALL_ORIG) {
> > > > > > > > > > +               restore_args(ctx, m->nr_args, args_off);
> > > > > > > > > > +               ret = emit_call(ctx, (const u64)orig_call);
> > > > > > > > > > +               if (ret)
> > > > > > > > > > +                       goto out;
> > > > > > > > > > +               emit_insn(ctx, std, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -retval_off);
> > > > > > > > > > +               emit_insn(ctx, std, regmap[BPF_REG_0], LOONGARCH_GPR_FP, -(retval_off - 8));
> > > > > > > > > > +               im->ip_after_call = ctx->ro_image + ctx->idx;
> > > > > > > > > > +               /* Reserve space for the move_imm + jirl instruction */
> > > > > > > > > > +               emit_insn(ctx, nop);
> > > > > > > > > > +               emit_insn(ctx, nop);
> > > > > > > > > > +               emit_insn(ctx, nop);
> > > > > > > > > > +               emit_insn(ctx, nop);
> > > > > > > > > > +               emit_insn(ctx, nop);
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       for (i = 0; ctx->image && i < fmod_ret->nr_links; i++) {
> > > > > > > > > > +               int offset = (void *)(&ctx->image[ctx->idx]) - (void *)branches[i];
> > > > > > > > > > +               *branches[i] = larch_insn_gen_bne(LOONGARCH_GPR_T1, LOONGARCH_GPR_ZERO, offset);
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       for (i = 0; i < fexit->nr_links; i++) {
> > > > > > > > > > +               ret = invoke_bpf_prog(ctx, fexit->links[i], args_off, retval_off,
> > > > > > > > > > +                                     run_ctx_off, false);
> > > > > > > > > > +               if (ret)
> > > > > > > > > > +                       goto out;
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       if (flags & BPF_TRAMP_F_CALL_ORIG) {
> > > > > > > > > > +               im->ip_epilogue = ctx->ro_image + ctx->idx;
> > > > > > > > > > +               move_imm(ctx, LOONGARCH_GPR_A0, (const s64)im, false);
> > > > > > > > > > +               ret = emit_call(ctx, (const u64)__bpf_tramp_exit);
> > > > > > > > > > +               if (ret)
> > > > > > > > > > +                       goto out;
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       if (flags & BPF_TRAMP_F_RESTORE_REGS)
> > > > > > > > > > +               restore_args(ctx, m->nr_args, args_off);
> > > > > > > > > > +
> > > > > > > > > > +       if (save_ret) {
> > > > > > > > > > +               emit_insn(ctx, ldd, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -retval_off);
> > > > > > > > > > +               emit_insn(ctx, ldd, regmap[BPF_REG_0], LOONGARCH_GPR_FP, -(retval_off - 8));
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       emit_insn(ctx, ldd, LOONGARCH_GPR_S1, LOONGARCH_GPR_FP, -sreg_off);
> > > > > > > > > > +
> > > > > > > > > > +       /* trampoline called from function entry */
> > > > > > > > > > +       emit_insn(ctx, ldd, LOONGARCH_GPR_T0, LOONGARCH_GPR_SP, stack_size - 8);
> > > > > > > > > > +       emit_insn(ctx, ldd, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, stack_size - 16);
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, stack_size);
> > > > > > > > > > +
> > > > > > > > > > +       emit_insn(ctx, ldd, LOONGARCH_GPR_RA, LOONGARCH_GPR_SP, 8);
> > > > > > > > > > +       emit_insn(ctx, ldd, LOONGARCH_GPR_FP, LOONGARCH_GPR_SP, 0);
> > > > > > > > > > +       emit_insn(ctx, addid, LOONGARCH_GPR_SP, LOONGARCH_GPR_SP, 16);
> > > > > > > > > > +
> > > > > > > > > > +       if (flags & BPF_TRAMP_F_SKIP_FRAME)
> > > > > > > > > > +               /* return to parent function */
> > > > > > > > > > +               emit_insn(ctx, jirl, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_RA, 0);
> > > > > > > > > > +       else
> > > > > > > > > > +               /* return to traced function */
> > > > > > > > > > +               emit_insn(ctx, jirl, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_T0, 0);
> > > > > > > > > > +
> > > > > > > > > > +       ret = ctx->idx;
> > > > > > > > > > +out:
> > > > > > > > > > +       kfree(branches);
> > > > > > > > > > +
> > > > > > > > > > +       return ret;
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *ro_image,
> > > > > > > > > > +                               void *ro_image_end, const struct btf_func_model *m,
> > > > > > > > > > +                               u32 flags, struct bpf_tramp_links *tlinks,
> > > > > > > > > > +                               void *func_addr)
> > > > > > > > > > +{
> > > > > > > > > > +       int ret;
> > > > > > > > > > +       void *image, *tmp;
> > > > > > > > > > +       u32 size = ro_image_end - ro_image;
> > > > > > > > > > +
> > > > > > > > > > +       image = kvmalloc(size, GFP_KERNEL);
> > > > > > > > > > +       if (!image)
> > > > > > > > > > +               return -ENOMEM;
> > > > > > > > > > +
> > > > > > > > > > +       struct jit_ctx ctx = {
> > > > > > > > > > +               .image = (union loongarch_instruction *)image,
> > > > > > > > > > +               .ro_image = (union loongarch_instruction *)ro_image,
> > > > > > > > > > +               .idx = 0,
> > > > > > > > > > +       };
> > > > > > > > > > +
> > > > > > > > > > +       jit_fill_hole(image, (unsigned int)(ro_image_end - ro_image));
> > > > > > > > > > +       ret = __arch_prepare_bpf_trampoline(&ctx, im, m, tlinks, func_addr, flags);
> > > > > > > > > > +       if (ret > 0 && validate_code(&ctx) < 0) {
> > > > > > > > > > +               ret = -EINVAL;
> > > > > > > > > > +               goto out;
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       tmp = bpf_arch_text_copy(ro_image, image, size);
> > > > > > > > > > +       if (IS_ERR(tmp)) {
> > > > > > > > > > +               ret = PTR_ERR(tmp);
> > > > > > > > > > +               goto out;
> > > > > > > > > > +       }
> > > > > > > > > > +
> > > > > > > > > > +       bpf_flush_icache(ro_image, ro_image_end);
> > > > > > > > > > +out:
> > > > > > > > > > +       kvfree(image);
> > > > > > > > > > +       return ret < 0 ? ret : size;
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
> > > > > > > > > > +                            struct bpf_tramp_links *tlinks, void *func_addr)
> > > > > > > > > > +{
> > > > > > > > > > +       struct bpf_tramp_image im;
> > > > > > > > > > +       struct jit_ctx ctx;
> > > > > > > > > > +       int ret;
> > > > > > > > > > +
> > > > > > > > > > +       ctx.image = NULL;
> > > > > > > > > > +       ctx.idx = 0;
> > > > > > > > > > +
> > > > > > > > > > +       ret = __arch_prepare_bpf_trampoline(&ctx, &im, m, tlinks, func_addr, flags);
> > > > > > > > > > +
> > > > > > > > > > +       /* Page align */
> > > > > > > > > > +       return ret < 0 ? ret : round_up(ret * LOONGARCH_INSN_SIZE, PAGE_SIZE);
> > > > > > > > > > +}
> > > > > > > > > > diff --git a/arch/loongarch/net/bpf_jit.h b/arch/loongarch/net/bpf_jit.h
> > > > > > > > > > index f9c569f53..5697158fd 100644
> > > > > > > > > > --- a/arch/loongarch/net/bpf_jit.h
> > > > > > > > > > +++ b/arch/loongarch/net/bpf_jit.h
> > > > > > > > > > @@ -18,6 +18,7 @@ struct jit_ctx {
> > > > > > > > > >         u32 *offset;
> > > > > > > > > >         int num_exentries;
> > > > > > > > > >         union loongarch_instruction *image;
> > > > > > > > > > +       union loongarch_instruction *ro_image;
> > > > > > > > > >         u32 stack_size;
> > > > > > > > > >  };
> > > > > > > > > >
> > > > > > > > > > @@ -308,3 +309,8 @@ static inline int emit_tailcall_jmp(struct jit_ctx *ctx, u8 cond, enum loongarch
> > > > > > > > > >
> > > > > > > > > >         return -EINVAL;
> > > > > > > > > >  }
> > > > > > > > > > +
> > > > > > > > > > +static inline void bpf_flush_icache(void *start, void *end)
> > > > > > > > > > +{
> > > > > > > > > > +       flush_icache_range((unsigned long)start, (unsigned long)end);
> > > > > > > > > > +}
> > > > > > > > > > --
> > > > > > > > > > 2.43.0
> > > > > > > > > >
> > > > > > > > > >