Re: [PATCH v2] cgroup/bpf: fast path for not loaded skb BPF filtering

[Pavel Begunkov <asml.silence@gmail.com>]

On 12/14/21 07:27, Martin KaFai Lau wrote:
> On Sat, Dec 11, 2021 at 07:17:49PM +0000, Pavel Begunkov wrote:
>> cgroup_bpf_enabled_key static key guards from overhead in cases where
>> no cgroup bpf program of a specific type is loaded in any cgroup. Turn
>> out that's not always good enough, e.g. when there are many cgroups but
>> ones that we're interesting in are without bpf. It's seen in server
>> environments, but the problem seems to be even wider as apparently
>> systemd loads some BPF affecting my laptop.
>>
>> Profiles for small packet or zerocopy transmissions over fast network
>> show __cgroup_bpf_run_filter_skb() taking 2-3%, 1% of which is from
>> migrate_disable/enable(), and similarly on the receiving side. Also
>> got +4-5% of t-put for local testing.
> What is t-put?  throughput?

yes

> Local testing means sending to lo/dummy?

yes, it was dummy specifically

> 
> [ ... ]
> 
>> diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
>> index 11820a430d6c..793e4f65ccb5 100644
>> --- a/include/linux/bpf-cgroup.h
>> +++ b/include/linux/bpf-cgroup.h
>> @@ -219,11 +219,28 @@ int bpf_percpu_cgroup_storage_copy(struct bpf_map *map, void *key, void *value);
>>   int bpf_percpu_cgroup_storage_update(struct bpf_map *map, void *key,
>>   				     void *value, u64 flags);
>>   
>> +static inline bool
>> +__cgroup_bpf_prog_array_is_empty(struct cgroup_bpf *cgrp_bpf,
>> +				 enum cgroup_bpf_attach_type type)
> Lets remove this.
> 
>> +{
>> +	struct bpf_prog_array *array = rcu_access_pointer(cgrp_bpf->effective[type]);
>> +
>> +	return array == &empty_prog_array.hdr;
>> +}
>> +
>> +#define CGROUP_BPF_TYPE_ENABLED(sk, atype)				       \
> and change cgroup.c to directly use this instead, so
> everywhere holding a fullsock sk will use this instead
> of having two helpers for empty check.

Why? CGROUP_BPF_TYPE_ENABLED can't be a function atm because of header
dependency hell, and so it'd kill some of typization, which doesn't add
clarity. And also it imposes some extra overhead to *sockopt using
the first helper directly.

I think it's better with two of them. I could inline the second
one, but it wouldn't have been pretty.

> 
> [ ... ]
> 
>> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
>> index 2405e39d800f..fedc7b44a1a9 100644
>> --- a/kernel/bpf/core.c
>> +++ b/kernel/bpf/core.c
>> @@ -1967,18 +1967,10 @@ static struct bpf_prog_dummy {
>>   	},
>>   };
>>   
>> -/* to avoid allocating empty bpf_prog_array for cgroups that
>> - * don't have bpf program attached use one global 'empty_prog_array'
>> - * It will not be modified the caller of bpf_prog_array_alloc()
>> - * (since caller requested prog_cnt == 0)
>> - * that pointer should be 'freed' by bpf_prog_array_free()
>> - */
>> -static struct {
>> -	struct bpf_prog_array hdr;
>> -	struct bpf_prog *null_prog;
>> -} empty_prog_array = {
>> +struct bpf_empty_prog_array empty_prog_array = {
>>   	.null_prog = NULL,
>>   };
>> +EXPORT_SYMBOL(empty_prog_array);
> nit. Since it is exported, may be prefix it with 'bpf_'.

yeah, sure


-- 
Pavel Begunkov