[PATCH 6.6.y] selftests/bpf: Remove flaky test_btf_id test

[PATCH 6.6.y] selftests/bpf: Remove flaky test_btf_id test

[Samasth Norway Ananda]

From: Yonghong Song <yonghong.song@linux.dev>

[ Upstream commit 56925f389e152dcb8d093435d43b78a310539c23 ]

With previous patch, one of subtests in test_btf_id becomes
flaky and may fail. The following is a failing example:

  Error: #26 btf
  Error: #26/174 btf/BTF ID
    Error: #26/174 btf/BTF ID
    btf_raw_create:PASS:check 0 nsec
    btf_raw_create:PASS:check 0 nsec
    test_btf_id:PASS:check 0 nsec
    ...
    test_btf_id:PASS:check 0 nsec
    test_btf_id:FAIL:check BTF lingersdo_test_get_info:FAIL:check failed: -1

The test tries to prove a btf_id not available after the map is closed.
But btf_id is freed only after workqueue and a rcu grace period, compared
to previous case just after a rcu grade period.
Depending on system workload, workqueue could take quite some time
to execute function bpf_map_free_deferred() which may cause the test failure.
Instead of adding arbitrary delays, let us remove the logic to
check btf_id availability after map is closed.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20231214203820.1469402-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
[Samasth: backport for 6.6.y]
Signed-off-by: Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
---
Above patch is a fix for 59e5791f59dd ("bpf: Fix a race condition between 
btf_put() and map_free()"). While the commit causing the error is
present in 6.6.y the fix is not present. 
---
 tools/testing/selftests/bpf/prog_tests/btf.c | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/btf.c b/tools/testing/selftests/bpf/prog_tests/btf.c
index 92d51f377fe5..0c0881c7eb1a 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf.c
@@ -4630,11 +4630,6 @@ static int test_btf_id(unsigned int test_num)
 	/* The map holds the last ref to BTF and its btf_id */
 	close(map_fd);
 	map_fd = -1;
-	btf_fd[0] = bpf_btf_get_fd_by_id(map_info.btf_id);
-	if (CHECK(btf_fd[0] >= 0, "BTF lingers")) {
-		err = -1;
-		goto done;
-	}
 
 	fprintf(stderr, "OK");
 
-- 
2.42.0

Re: [PATCH 6.6.y] selftests/bpf: Remove flaky test_btf_id test

[Greg KH]

On Thu, Feb 01, 2024 at 07:45:45PM -0800, Samasth Norway Ananda wrote:
> From: Yonghong Song <yonghong.song@linux.dev>
> 
> [ Upstream commit 56925f389e152dcb8d093435d43b78a310539c23 ]
> 
> With previous patch, one of subtests in test_btf_id becomes
> flaky and may fail. The following is a failing example:
> 
>   Error: #26 btf
>   Error: #26/174 btf/BTF ID
>     Error: #26/174 btf/BTF ID
>     btf_raw_create:PASS:check 0 nsec
>     btf_raw_create:PASS:check 0 nsec
>     test_btf_id:PASS:check 0 nsec
>     ...
>     test_btf_id:PASS:check 0 nsec
>     test_btf_id:FAIL:check BTF lingersdo_test_get_info:FAIL:check failed: -1
> 
> The test tries to prove a btf_id not available after the map is closed.
> But btf_id is freed only after workqueue and a rcu grace period, compared
> to previous case just after a rcu grade period.
> Depending on system workload, workqueue could take quite some time
> to execute function bpf_map_free_deferred() which may cause the test failure.
> Instead of adding arbitrary delays, let us remove the logic to
> check btf_id availability after map is closed.
> 
> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
> Link: https://lore.kernel.org/r/20231214203820.1469402-1-yonghong.song@linux.dev
> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
> [Samasth: backport for 6.6.y]
> Signed-off-by: Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
> ---
> Above patch is a fix for 59e5791f59dd ("bpf: Fix a race condition between 
> btf_put() and map_free()"). While the commit causing the error is
> present in 6.6.y the fix is not present. 
> ---
>  tools/testing/selftests/bpf/prog_tests/btf.c | 5 -----
>  1 file changed, 5 deletions(-)

What about 6.7 as well?  Shouldn't this change be there too?

thanks,

greg k-h

Re: [External] : Re: [PATCH 6.6.y] selftests/bpf: Remove flaky test_btf_id test

[samasth.norway.ananda]

On 2/2/24 5:31 PM, Greg KH wrote:
> On Thu, Feb 01, 2024 at 07:45:45PM -0800, Samasth Norway Ananda wrote:
>> From: Yonghong Song <yonghong.song@linux.dev>
>>
>> [ Upstream commit 56925f389e152dcb8d093435d43b78a310539c23 ]
>>
>> With previous patch, one of subtests in test_btf_id becomes
>> flaky and may fail. The following is a failing example:
>>
>>    Error: #26 btf
>>    Error: #26/174 btf/BTF ID
>>      Error: #26/174 btf/BTF ID
>>      btf_raw_create:PASS:check 0 nsec
>>      btf_raw_create:PASS:check 0 nsec
>>      test_btf_id:PASS:check 0 nsec
>>      ...
>>      test_btf_id:PASS:check 0 nsec
>>      test_btf_id:FAIL:check BTF lingersdo_test_get_info:FAIL:check failed: -1
>>
>> The test tries to prove a btf_id not available after the map is closed.
>> But btf_id is freed only after workqueue and a rcu grace period, compared
>> to previous case just after a rcu grade period.
>> Depending on system workload, workqueue could take quite some time
>> to execute function bpf_map_free_deferred() which may cause the test failure.
>> Instead of adding arbitrary delays, let us remove the logic to
>> check btf_id availability after map is closed.
>>
>> Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
>> Link: https://urldefense.com/v3/__https://lore.kernel.org/r/20231214203820.1469402-1-yonghong.song@linux.dev__;!!ACWV5N9M2RV99hQ!I-3G5NyOo-Xom0b0NmUHYWm_hs6Ai1qD4A9smNew_5_8jyyEXbISpxoAPa4wSD_eXQr-IOAd4_TM2NBscejS$
>> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
>> [Samasth: backport for 6.6.y]
>> Signed-off-by: Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
>> ---
>> Above patch is a fix for 59e5791f59dd ("bpf: Fix a race condition between
>> btf_put() and map_free()"). While the commit causing the error is
>> present in 6.6.y the fix is not present.
>> ---
>>   tools/testing/selftests/bpf/prog_tests/btf.c | 5 -----
>>   1 file changed, 5 deletions(-)
> 
> What about 6.7 as well?  Shouldn't this change be there too?

Yes, it should be there on 6.7 as well. Sorry, I should have mentioned that.

Thanks,
Samasth.

> 
> thanks,
> 
> greg k-h

Re: [External] : Re: [PATCH 6.6.y] selftests/bpf: Remove flaky test_btf_id test

[Greg KH]

On Fri, Feb 02, 2024 at 05:33:36PM -0800, samasth.norway.ananda@oracle.com wrote:
> 
> 
> On 2/2/24 5:31 PM, Greg KH wrote:
> > On Thu, Feb 01, 2024 at 07:45:45PM -0800, Samasth Norway Ananda wrote:
> > > From: Yonghong Song <yonghong.song@linux.dev>
> > > 
> > > [ Upstream commit 56925f389e152dcb8d093435d43b78a310539c23 ]
> > > 
> > > With previous patch, one of subtests in test_btf_id becomes
> > > flaky and may fail. The following is a failing example:
> > > 
> > >    Error: #26 btf
> > >    Error: #26/174 btf/BTF ID
> > >      Error: #26/174 btf/BTF ID
> > >      btf_raw_create:PASS:check 0 nsec
> > >      btf_raw_create:PASS:check 0 nsec
> > >      test_btf_id:PASS:check 0 nsec
> > >      ...
> > >      test_btf_id:PASS:check 0 nsec
> > >      test_btf_id:FAIL:check BTF lingersdo_test_get_info:FAIL:check failed: -1
> > > 
> > > The test tries to prove a btf_id not available after the map is closed.
> > > But btf_id is freed only after workqueue and a rcu grace period, compared
> > > to previous case just after a rcu grade period.
> > > Depending on system workload, workqueue could take quite some time
> > > to execute function bpf_map_free_deferred() which may cause the test failure.
> > > Instead of adding arbitrary delays, let us remove the logic to
> > > check btf_id availability after map is closed.
> > > 
> > > Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
> > > Link: https://urldefense.com/v3/__https://lore.kernel.org/r/20231214203820.1469402-1-yonghong.song@linux.dev__;!!ACWV5N9M2RV99hQ!I-3G5NyOo-Xom0b0NmUHYWm_hs6Ai1qD4A9smNew_5_8jyyEXbISpxoAPa4wSD_eXQr-IOAd4_TM2NBscejS$
> > > Signed-off-by: Alexei Starovoitov <ast@kernel.org>
> > > [Samasth: backport for 6.6.y]
> > > Signed-off-by: Samasth Norway Ananda <samasth.norway.ananda@oracle.com>
> > > ---
> > > Above patch is a fix for 59e5791f59dd ("bpf: Fix a race condition between
> > > btf_put() and map_free()"). While the commit causing the error is
> > > present in 6.6.y the fix is not present.
> > > ---
> > >   tools/testing/selftests/bpf/prog_tests/btf.c | 5 -----
> > >   1 file changed, 5 deletions(-)
> > 
> > What about 6.7 as well?  Shouldn't this change be there too?
> 
> Yes, it should be there on 6.7 as well. Sorry, I should have mentioned that.

thanks, now queued up.

greg k-h