[PATCH bpf] libbpf: fix CO-RE relocs against .text section

[PATCH bpf] libbpf: fix CO-RE relocs against .text section

[Andrii Nakryiko]

bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't
return .text "BPF program", if it is a function storage for sub-programs.
Because of that, any CO-RE relocation in helper non-inlined functions will
fail. Fix this by searching for .text-corresponding BPF program manually.

Adjust one of bpf_iter selftest to exhibit this pattern.

Reported-by: Yonghong Song <yhs@fb.com>
Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
 tools/lib/bpf/libbpf.c                               | 8 +++++++-
 tools/testing/selftests/bpf/progs/bpf_iter_netlink.c | 2 +-
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 477c679ed945..f17151d866e6 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -4818,7 +4818,13 @@ bpf_core_reloc_fields(struct bpf_object *obj, const char *targ_btf_path)
 			err = -EINVAL;
 			goto out;
 		}
-		prog = bpf_object__find_program_by_title(obj, sec_name);
+		prog = NULL;
+		for (i = 0; i < obj->nr_programs; i++) {
+			if (!strcmp(obj->programs[i].section_name, sec_name)) {
+				prog = &obj->programs[i];
+				break;
+			}
+		}
 		if (!prog) {
 			pr_warn("failed to find program '%s' for CO-RE offset relocation\n",
 				sec_name);
diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c b/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c
index e7b8753eac0b..75ecf956a2df 100644
--- a/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c
+++ b/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c
@@ -25,7 +25,7 @@ struct bpf_iter__netlink {
 	struct netlink_sock *sk;
 } __attribute__((preserve_access_index));
 
-static inline struct inode *SOCK_INODE(struct socket *socket)
+static __attribute__((noinline)) struct inode *SOCK_INODE(struct socket *socket)
 {
 	return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
 }
-- 
2.24.1

Re: [PATCH bpf] libbpf: fix CO-RE relocs against .text section

[Yonghong Song]

On 6/19/20 4:04 PM, Andrii Nakryiko wrote:
> bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't
> return .text "BPF program", if it is a function storage for sub-programs.
> Because of that, any CO-RE relocation in helper non-inlined functions will
> fail. Fix this by searching for .text-corresponding BPF program manually.
> 
> Adjust one of bpf_iter selftest to exhibit this pattern.
> 
> Reported-by: Yonghong Song <yhs@fb.com>
> Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")
> Signed-off-by: Andrii Nakryiko <andriin@fb.com>

Acked-by: Yonghong Song <yhs@fb.com>

But the fix here only fixed the issue for interpreter mode.
For jit only mode, we still have issues. The following patch can fix
the jit mode issue,

=============

 From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001
From: Yonghong Song <yhs@fb.com>
Date: Fri, 19 Jun 2020 23:26:13 -0700
Subject: [PATCH bpf] bpf: set the number of exception entries properly for
  subprograms

Currently, if a bpf program has more than one subprograms, each
program will be jitted separately. For tracing problem, the
prog->aux->num_exentries is not setup properly. For example,
with bpf_iter_netlink.c modified to force one function not inlined,
and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON,
we will have error like below:
   $ ./test_progs -n 3/3
   ...
   libbpf: failed to load program 'iter/netlink'
   libbpf: failed to load object 'bpf_iter_netlink'
   libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007
   test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton 
open_and_load failed
   #3/3 netlink:FAIL
The dmesg shows the following errors:
   ex gen bug
which is triggered by the following code in arch/x86/net/bpf_jit_comp.c:
   if (excnt >= bpf_prog->aux->num_exentries) {
     pr_err("ex gen bug\n");
     return -EFAULT;
   }

If the program has more than one subprograms, num_exentries is actually
0 since it is not setup.

This patch fixed the issue by setuping proper num_exentries for
each subprogram before calling jit function.

Signed-off-by: Yonghong Song <yhs@fb.com>
---
  kernel/bpf/verifier.c | 12 +++++++++++-
  1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 34cde841ab68..7d8b23ba825c 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -9801,7 +9801,7 @@ static int jit_subprogs(struct bpf_verifier_env *env)
  	int i, j, subprog_start, subprog_end = 0, len, subprog;
  	struct bpf_insn *insn;
  	void *old_bpf_func;
-	int err;
+	int err, num_exentries;

  	if (env->subprog_cnt <= 1)
  		return 0;
@@ -9876,6 +9876,16 @@ static int jit_subprogs(struct bpf_verifier_env *env)
  		func[i]->aux->nr_linfo = prog->aux->nr_linfo;
  		func[i]->aux->jited_linfo = prog->aux->jited_linfo;
  		func[i]->aux->linfo_idx = env->subprog_info[i].linfo_idx;
+
+		num_exentries = 0;
+		insn = func[i]->insnsi;
+		for (j = 0; j < func[i]->len; j++, insn++) {
+			if (BPF_CLASS(insn->code) == BPF_LDX &&
+			    BPF_MODE(insn->code) == BPF_PROBE_MEM)
+				num_exentries++;
+		}
+		func[i]->aux->num_exentries = num_exentries;
+
  		func[i] = bpf_int_jit_compile(func[i]);
  		if (!func[i]->jited) {
  			err = -ENOTSUPP;
-- 
2.24.1

================

We need this (or similar fixes) go in together with libbpf fix
to avoid bpf_iter_netlink.c test failure at jit only mode.

Do we need a separate patch for the above fix? Or Andrii can
fold this into his patch and resubmit? Maybe the latter is better.

> ---
>   tools/lib/bpf/libbpf.c                               | 8 +++++++-
>   tools/testing/selftests/bpf/progs/bpf_iter_netlink.c | 2 +-
>   2 files changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 477c679ed945..f17151d866e6 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -4818,7 +4818,13 @@ bpf_core_reloc_fields(struct bpf_object *obj, const char *targ_btf_path)
>   			err = -EINVAL;
>   			goto out;
>   		}
> -		prog = bpf_object__find_program_by_title(obj, sec_name);
> +		prog = NULL;
> +		for (i = 0; i < obj->nr_programs; i++) {
> +			if (!strcmp(obj->programs[i].section_name, sec_name)) {
> +				prog = &obj->programs[i];
> +				break;
> +			}
> +		}
>   		if (!prog) {
>   			pr_warn("failed to find program '%s' for CO-RE offset relocation\n",
>   				sec_name);
> diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c b/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c
> index e7b8753eac0b..75ecf956a2df 100644
> --- a/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c
> +++ b/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c
> @@ -25,7 +25,7 @@ struct bpf_iter__netlink {
>   	struct netlink_sock *sk;
>   } __attribute__((preserve_access_index));
>   
> -static inline struct inode *SOCK_INODE(struct socket *socket)
> +static __attribute__((noinline)) struct inode *SOCK_INODE(struct socket *socket)
>   {
>   	return &container_of(socket, struct socket_alloc, socket)->vfs_inode;
>   }
> 

Re: [PATCH bpf] libbpf: fix CO-RE relocs against .text section

[Alexei Starovoitov]

On Sat, Jun 20, 2020 at 12:06 AM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 6/19/20 4:04 PM, Andrii Nakryiko wrote:
> > bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't
> > return .text "BPF program", if it is a function storage for sub-programs.
> > Because of that, any CO-RE relocation in helper non-inlined functions will
> > fail. Fix this by searching for .text-corresponding BPF program manually.
> >
> > Adjust one of bpf_iter selftest to exhibit this pattern.
> >
> > Reported-by: Yonghong Song <yhs@fb.com>
> > Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")
> > Signed-off-by: Andrii Nakryiko <andriin@fb.com>
>
> Acked-by: Yonghong Song <yhs@fb.com>
>
> But the fix here only fixed the issue for interpreter mode.
> For jit only mode, we still have issues. The following patch can fix
> the jit mode issue,
>
> =============
>
>  From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001
> From: Yonghong Song <yhs@fb.com>
> Date: Fri, 19 Jun 2020 23:26:13 -0700
> Subject: [PATCH bpf] bpf: set the number of exception entries properly for
>   subprograms
>
> Currently, if a bpf program has more than one subprograms, each
> program will be jitted separately. For tracing problem, the
> prog->aux->num_exentries is not setup properly. For example,
> with bpf_iter_netlink.c modified to force one function not inlined,
> and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON,
> we will have error like below:
>    $ ./test_progs -n 3/3
>    ...
>    libbpf: failed to load program 'iter/netlink'
>    libbpf: failed to load object 'bpf_iter_netlink'
>    libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007
>    test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton
> open_and_load failed
>    #3/3 netlink:FAIL
> The dmesg shows the following errors:
>    ex gen bug
> which is triggered by the following code in arch/x86/net/bpf_jit_comp.c:
>    if (excnt >= bpf_prog->aux->num_exentries) {
>      pr_err("ex gen bug\n");
>      return -EFAULT;
>    }
>
> If the program has more than one subprograms, num_exentries is actually
> 0 since it is not setup.
>
> This patch fixed the issue by setuping proper num_exentries for
> each subprogram before calling jit function.
>
> Signed-off-by: Yonghong Song <yhs@fb.com>

Thanks for fixing. Applied both to bpf tree.
Yonghong, next time please submit the patch properly.
It was very awkward to copy-paste it manually from the thread.
I've edited the commit log a bit.

Re: [PATCH bpf] libbpf: fix CO-RE relocs against .text section

[Yonghong Song]

On 6/23/20 5:40 PM, Alexei Starovoitov wrote:
> On Sat, Jun 20, 2020 at 12:06 AM Yonghong Song <yhs@fb.com> wrote:
>>
>>
>>
>> On 6/19/20 4:04 PM, Andrii Nakryiko wrote:
>>> bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't
>>> return .text "BPF program", if it is a function storage for sub-programs.
>>> Because of that, any CO-RE relocation in helper non-inlined functions will
>>> fail. Fix this by searching for .text-corresponding BPF program manually.
>>>
>>> Adjust one of bpf_iter selftest to exhibit this pattern.
>>>
>>> Reported-by: Yonghong Song <yhs@fb.com>
>>> Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm")
>>> Signed-off-by: Andrii Nakryiko <andriin@fb.com>
>>
>> Acked-by: Yonghong Song <yhs@fb.com>
>>
>> But the fix here only fixed the issue for interpreter mode.
>> For jit only mode, we still have issues. The following patch can fix
>> the jit mode issue,
>>
>> =============
>>
>>   From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001
>> From: Yonghong Song <yhs@fb.com>
>> Date: Fri, 19 Jun 2020 23:26:13 -0700
>> Subject: [PATCH bpf] bpf: set the number of exception entries properly for
>>    subprograms
>>
>> Currently, if a bpf program has more than one subprograms, each
>> program will be jitted separately. For tracing problem, the
>> prog->aux->num_exentries is not setup properly. For example,
>> with bpf_iter_netlink.c modified to force one function not inlined,
>> and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON,
>> we will have error like below:
>>     $ ./test_progs -n 3/3
>>     ...
>>     libbpf: failed to load program 'iter/netlink'
>>     libbpf: failed to load object 'bpf_iter_netlink'
>>     libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007
>>     test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton
>> open_and_load failed
>>     #3/3 netlink:FAIL
>> The dmesg shows the following errors:
>>     ex gen bug
>> which is triggered by the following code in arch/x86/net/bpf_jit_comp.c:
>>     if (excnt >= bpf_prog->aux->num_exentries) {
>>       pr_err("ex gen bug\n");
>>       return -EFAULT;
>>     }
>>
>> If the program has more than one subprograms, num_exentries is actually
>> 0 since it is not setup.
>>
>> This patch fixed the issue by setuping proper num_exentries for
>> each subprogram before calling jit function.
>>
>> Signed-off-by: Yonghong Song <yhs@fb.com>
> 
> Thanks for fixing. Applied both to bpf tree.
> Yonghong, next time please submit the patch properly.
> It was very awkward to copy-paste it manually from the thread.
> I've edited the commit log a bit.

Thanks. I posted original commit as I am not sure how to proceed as
this and Andrii's patch belongs to the same patch set to fix 
bpf_iter_netlink problem. I guess next time I will go ahead with
patch submit with proper description in the patch, which
sounds better for review and to get notice from other people.