* [PATCH bpf-next v5 1/3] add includes masked by cgroup -> bpf dependency
2021-12-16 2:55 [PATCH bpf-next v5 0/3] bpf: remove the cgroup -> bpf header dependecy Jakub Kicinski
@ 2021-12-16 2:55 ` Jakub Kicinski
2021-12-16 2:55 ` [PATCH bpf-next v5 2/3] add missing bpf-cgroup.h includes Jakub Kicinski
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Jakub Kicinski @ 2021-12-16 2:55 UTC (permalink / raw)
To: daniel, ast, andrii
Cc: bpf, Jakub Kicinski, hca, gor, borntraeger, agordeev, peterz,
mingo, acme, mark.rutland, alexander.shishkin, jolsa, namhyung,
kafai, songliubraving, yhs, john.fastabend, kpsingh, peterx, akpm,
linux-s390, linux-perf-users
cgroup pulls in BPF which pulls in a lot of includes.
We're about to break that chain so fix those who were
depending on it.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: hca@linux.ibm.com
CC: gor@linux.ibm.com
CC: borntraeger@linux.ibm.com
CC: agordeev@linux.ibm.com
CC: peterz@infradead.org
CC: mingo@redhat.com
CC: acme@kernel.org
CC: mark.rutland@arm.com
CC: alexander.shishkin@linux.intel.com
CC: jolsa@redhat.com
CC: namhyung@kernel.org
CC: ast@kernel.org
CC: daniel@iogearbox.net
CC: andrii@kernel.org
CC: kafai@fb.com
CC: songliubraving@fb.com
CC: yhs@fb.com
CC: john.fastabend@gmail.com
CC: kpsingh@kernel.org
CC: peterx@redhat.com
CC: akpm@linux-foundation.org
CC: linux-s390@vger.kernel.org
CC: linux-perf-users@vger.kernel.org
CC: bpf@vger.kernel.org
---
arch/s390/mm/hugetlbpage.c | 1 +
include/linux/perf_event.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/s390/mm/hugetlbpage.c b/arch/s390/mm/hugetlbpage.c
index da36d13ffc16..082793d497ec 100644
--- a/arch/s390/mm/hugetlbpage.c
+++ b/arch/s390/mm/hugetlbpage.c
@@ -9,6 +9,7 @@
#define KMSG_COMPONENT "hugetlb"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+#include <asm/pgalloc.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
#include <linux/mman.h>
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 0dcfd265beed..4a021149eaf0 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -611,6 +611,7 @@ struct swevent_hlist {
#define PERF_ATTACH_SCHED_CB 0x20
#define PERF_ATTACH_CHILD 0x40
+struct bpf_prog;
struct perf_cgroup;
struct perf_buffer;
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH bpf-next v5 2/3] add missing bpf-cgroup.h includes
2021-12-16 2:55 [PATCH bpf-next v5 0/3] bpf: remove the cgroup -> bpf header dependecy Jakub Kicinski
2021-12-16 2:55 ` [PATCH bpf-next v5 1/3] add includes masked by cgroup -> bpf dependency Jakub Kicinski
@ 2021-12-16 2:55 ` Jakub Kicinski
2021-12-16 2:55 ` [PATCH bpf-next v5 3/3] bpf: remove the cgroup -> bpf header dependecy Jakub Kicinski
2021-12-16 23:05 ` [PATCH bpf-next v5 0/3] " Alexei Starovoitov
3 siblings, 0 replies; 6+ messages in thread
From: Jakub Kicinski @ 2021-12-16 2:55 UTC (permalink / raw)
To: daniel, ast, andrii
Cc: bpf, Jakub Kicinski, Tejun Heo, kafai, songliubraving, yhs,
john.fastabend, kpsingh, lizefan.x, hannes, rostedt, mingo,
yoshfuji, dsahern, jmorris, serge, cgroups, linux-security-module
We're about to break the cgroup-defs.h -> bpf-cgroup.h dependency,
make sure those who actually need more than the definition of
struct cgroup_bpf include bpf-cgroup.h explicitly.
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: ast@kernel.org
CC: daniel@iogearbox.net
CC: andrii@kernel.org
CC: kafai@fb.com
CC: songliubraving@fb.com
CC: yhs@fb.com
CC: john.fastabend@gmail.com
CC: kpsingh@kernel.org
CC: lizefan.x@bytedance.com
CC: hannes@cmpxchg.org
CC: rostedt@goodmis.org
CC: mingo@redhat.com
CC: yoshfuji@linux-ipv6.org
CC: dsahern@kernel.org
CC: jmorris@namei.org
CC: serge@hallyn.com
CC: bpf@vger.kernel.org
CC: cgroups@vger.kernel.org
CC: linux-security-module@vger.kernel.org
---
kernel/bpf/helpers.c | 1 +
kernel/bpf/syscall.c | 1 +
kernel/bpf/verifier.c | 1 +
kernel/cgroup/cgroup.c | 1 +
kernel/trace/trace_kprobe.c | 1 +
kernel/trace/trace_uprobe.c | 1 +
net/ipv4/udp.c | 1 +
net/ipv6/udp.c | 1 +
net/socket.c | 1 +
security/device_cgroup.c | 1 +
10 files changed, 10 insertions(+)
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 8babae03d30a..34d6f91dec1c 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com
*/
#include <linux/bpf.h>
+#include <linux/bpf-cgroup.h>
#include <linux/rcupdate.h>
#include <linux/random.h>
#include <linux/smp.h>
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index ddd81d543203..da07bdf71697 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2011-2014 PLUMgrid, http://plumgrid.com
*/
#include <linux/bpf.h>
+#include <linux/bpf-cgroup.h>
#include <linux/bpf_trace.h>
#include <linux/bpf_lirc.h>
#include <linux/bpf_verifier.h>
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index d74e8a99412e..f0604796132f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -4,6 +4,7 @@
* Copyright (c) 2018 Covalent IO, Inc. http://covalent.io
*/
#include <uapi/linux/btf.h>
+#include <linux/bpf-cgroup.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/slab.h>
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 919194de39c8..cd4c23f7e3df 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -30,6 +30,7 @@
#include "cgroup-internal.h"
+#include <linux/bpf-cgroup.h>
#include <linux/cred.h>
#include <linux/errno.h>
#include <linux/init_task.h>
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index 33272a7b6912..4e1257f50aa3 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -7,6 +7,7 @@
*/
#define pr_fmt(fmt) "trace_kprobe: " fmt
+#include <linux/bpf-cgroup.h>
#include <linux/security.h>
#include <linux/module.h>
#include <linux/uaccess.h>
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index f5f0039d31e5..4f35514a48f3 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -7,6 +7,7 @@
*/
#define pr_fmt(fmt) "trace_uprobe: " fmt
+#include <linux/bpf-cgroup.h>
#include <linux/security.h>
#include <linux/ctype.h>
#include <linux/module.h>
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 69d30053fed9..99536127650b 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -74,6 +74,7 @@
#define pr_fmt(fmt) "UDP: " fmt
+#include <linux/bpf-cgroup.h>
#include <linux/uaccess.h>
#include <asm/ioctls.h>
#include <linux/memblock.h>
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 6a0e569f0bb8..ba8986d12413 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -17,6 +17,7 @@
* YOSHIFUJI Hideaki @USAGI: convert /proc/net/udp6 to seq_file.
*/
+#include <linux/bpf-cgroup.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/socket.h>
diff --git a/net/socket.c b/net/socket.c
index 7f64a6eccf63..721a5a1b1106 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -52,6 +52,7 @@
* Based upon Swansea University Computer Society NET3.039
*/
+#include <linux/bpf-cgroup.h>
#include <linux/ethtool.h>
#include <linux/mm.h>
#include <linux/socket.h>
diff --git a/security/device_cgroup.c b/security/device_cgroup.c
index 04375df52fc9..842889f3dcb7 100644
--- a/security/device_cgroup.c
+++ b/security/device_cgroup.c
@@ -5,6 +5,7 @@
* Copyright 2007 IBM Corp
*/
+#include <linux/bpf-cgroup.h>
#include <linux/device_cgroup.h>
#include <linux/cgroup.h>
#include <linux/ctype.h>
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH bpf-next v5 3/3] bpf: remove the cgroup -> bpf header dependecy
2021-12-16 2:55 [PATCH bpf-next v5 0/3] bpf: remove the cgroup -> bpf header dependecy Jakub Kicinski
2021-12-16 2:55 ` [PATCH bpf-next v5 1/3] add includes masked by cgroup -> bpf dependency Jakub Kicinski
2021-12-16 2:55 ` [PATCH bpf-next v5 2/3] add missing bpf-cgroup.h includes Jakub Kicinski
@ 2021-12-16 2:55 ` Jakub Kicinski
2021-12-16 23:05 ` [PATCH bpf-next v5 0/3] " Alexei Starovoitov
3 siblings, 0 replies; 6+ messages in thread
From: Jakub Kicinski @ 2021-12-16 2:55 UTC (permalink / raw)
To: daniel, ast, andrii
Cc: bpf, Jakub Kicinski, Tejun Heo, kafai, songliubraving, yhs,
john.fastabend, kpsingh, lizefan.x, hannes, cgroups
Remove the dependency from cgroup-defs.h to bpf-cgroup.h and bpf.h.
This reduces the incremental build size of x86 allmodconfig after
bpf.h was touched from ~17k objects rebuilt to ~5k objects.
bpf.h is 2.2kLoC and is modified relatively often.
We need a new header with just the definition of struct cgroup_bpf
and enum cgroup_bpf_attach_type, this is akin to cgroup-defs.h.
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
---
CC: ast@kernel.org
CC: daniel@iogearbox.net
CC: andrii@kernel.org
CC: kafai@fb.com
CC: songliubraving@fb.com
CC: yhs@fb.com
CC: john.fastabend@gmail.com
CC: kpsingh@kernel.org
CC: lizefan.x@bytedance.com
CC: hannes@cmpxchg.org
CC: bpf@vger.kernel.org
CC: cgroups@vger.kernel.org
---
include/linux/bpf-cgroup-defs.h | 70 +++++++++++++++++++++++++++++++++
include/linux/bpf-cgroup.h | 57 +--------------------------
include/linux/cgroup-defs.h | 2 +-
3 files changed, 72 insertions(+), 57 deletions(-)
create mode 100644 include/linux/bpf-cgroup-defs.h
diff --git a/include/linux/bpf-cgroup-defs.h b/include/linux/bpf-cgroup-defs.h
new file mode 100644
index 000000000000..695d1224a71b
--- /dev/null
+++ b/include/linux/bpf-cgroup-defs.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _BPF_CGROUP_DEFS_H
+#define _BPF_CGROUP_DEFS_H
+
+#ifdef CONFIG_CGROUP_BPF
+
+#include <linux/list.h>
+#include <linux/percpu-refcount.h>
+#include <linux/workqueue.h>
+
+struct bpf_prog_array;
+
+enum cgroup_bpf_attach_type {
+ CGROUP_BPF_ATTACH_TYPE_INVALID = -1,
+ CGROUP_INET_INGRESS = 0,
+ CGROUP_INET_EGRESS,
+ CGROUP_INET_SOCK_CREATE,
+ CGROUP_SOCK_OPS,
+ CGROUP_DEVICE,
+ CGROUP_INET4_BIND,
+ CGROUP_INET6_BIND,
+ CGROUP_INET4_CONNECT,
+ CGROUP_INET6_CONNECT,
+ CGROUP_INET4_POST_BIND,
+ CGROUP_INET6_POST_BIND,
+ CGROUP_UDP4_SENDMSG,
+ CGROUP_UDP6_SENDMSG,
+ CGROUP_SYSCTL,
+ CGROUP_UDP4_RECVMSG,
+ CGROUP_UDP6_RECVMSG,
+ CGROUP_GETSOCKOPT,
+ CGROUP_SETSOCKOPT,
+ CGROUP_INET4_GETPEERNAME,
+ CGROUP_INET6_GETPEERNAME,
+ CGROUP_INET4_GETSOCKNAME,
+ CGROUP_INET6_GETSOCKNAME,
+ CGROUP_INET_SOCK_RELEASE,
+ MAX_CGROUP_BPF_ATTACH_TYPE
+};
+
+struct cgroup_bpf {
+ /* array of effective progs in this cgroup */
+ struct bpf_prog_array __rcu *effective[MAX_CGROUP_BPF_ATTACH_TYPE];
+
+ /* attached progs to this cgroup and attach flags
+ * when flags == 0 or BPF_F_ALLOW_OVERRIDE the progs list will
+ * have either zero or one element
+ * when BPF_F_ALLOW_MULTI the list can have up to BPF_CGROUP_MAX_PROGS
+ */
+ struct list_head progs[MAX_CGROUP_BPF_ATTACH_TYPE];
+ u32 flags[MAX_CGROUP_BPF_ATTACH_TYPE];
+
+ /* list of cgroup shared storages */
+ struct list_head storages;
+
+ /* temp storage for effective prog array used by prog_attach/detach */
+ struct bpf_prog_array *inactive;
+
+ /* reference counter used to detach bpf programs after cgroup removal */
+ struct percpu_ref refcnt;
+
+ /* cgroup_bpf is released using a work queue */
+ struct work_struct release_work;
+};
+
+#else /* CONFIG_CGROUP_BPF */
+struct cgroup_bpf {};
+#endif /* CONFIG_CGROUP_BPF */
+
+#endif
diff --git a/include/linux/bpf-cgroup.h b/include/linux/bpf-cgroup.h
index 11820a430d6c..b525d8cdc25b 100644
--- a/include/linux/bpf-cgroup.h
+++ b/include/linux/bpf-cgroup.h
@@ -3,10 +3,10 @@
#define _BPF_CGROUP_H
#include <linux/bpf.h>
+#include <linux/bpf-cgroup-defs.h>
#include <linux/errno.h>
#include <linux/jump_label.h>
#include <linux/percpu.h>
-#include <linux/percpu-refcount.h>
#include <linux/rbtree.h>
#include <uapi/linux/bpf.h>
@@ -23,33 +23,6 @@ struct ctl_table_header;
struct task_struct;
#ifdef CONFIG_CGROUP_BPF
-enum cgroup_bpf_attach_type {
- CGROUP_BPF_ATTACH_TYPE_INVALID = -1,
- CGROUP_INET_INGRESS = 0,
- CGROUP_INET_EGRESS,
- CGROUP_INET_SOCK_CREATE,
- CGROUP_SOCK_OPS,
- CGROUP_DEVICE,
- CGROUP_INET4_BIND,
- CGROUP_INET6_BIND,
- CGROUP_INET4_CONNECT,
- CGROUP_INET6_CONNECT,
- CGROUP_INET4_POST_BIND,
- CGROUP_INET6_POST_BIND,
- CGROUP_UDP4_SENDMSG,
- CGROUP_UDP6_SENDMSG,
- CGROUP_SYSCTL,
- CGROUP_UDP4_RECVMSG,
- CGROUP_UDP6_RECVMSG,
- CGROUP_GETSOCKOPT,
- CGROUP_SETSOCKOPT,
- CGROUP_INET4_GETPEERNAME,
- CGROUP_INET6_GETPEERNAME,
- CGROUP_INET4_GETSOCKNAME,
- CGROUP_INET6_GETSOCKNAME,
- CGROUP_INET_SOCK_RELEASE,
- MAX_CGROUP_BPF_ATTACH_TYPE
-};
#define CGROUP_ATYPE(type) \
case BPF_##type: return type
@@ -127,33 +100,6 @@ struct bpf_prog_list {
struct bpf_cgroup_storage *storage[MAX_BPF_CGROUP_STORAGE_TYPE];
};
-struct bpf_prog_array;
-
-struct cgroup_bpf {
- /* array of effective progs in this cgroup */
- struct bpf_prog_array __rcu *effective[MAX_CGROUP_BPF_ATTACH_TYPE];
-
- /* attached progs to this cgroup and attach flags
- * when flags == 0 or BPF_F_ALLOW_OVERRIDE the progs list will
- * have either zero or one element
- * when BPF_F_ALLOW_MULTI the list can have up to BPF_CGROUP_MAX_PROGS
- */
- struct list_head progs[MAX_CGROUP_BPF_ATTACH_TYPE];
- u32 flags[MAX_CGROUP_BPF_ATTACH_TYPE];
-
- /* list of cgroup shared storages */
- struct list_head storages;
-
- /* temp storage for effective prog array used by prog_attach/detach */
- struct bpf_prog_array *inactive;
-
- /* reference counter used to detach bpf programs after cgroup removal */
- struct percpu_ref refcnt;
-
- /* cgroup_bpf is released using a work queue */
- struct work_struct release_work;
-};
-
int cgroup_bpf_inherit(struct cgroup *cgrp);
void cgroup_bpf_offline(struct cgroup *cgrp);
@@ -451,7 +397,6 @@ int cgroup_bpf_prog_query(const union bpf_attr *attr,
union bpf_attr __user *uattr);
#else
-struct cgroup_bpf {};
static inline int cgroup_bpf_inherit(struct cgroup *cgrp) { return 0; }
static inline void cgroup_bpf_offline(struct cgroup *cgrp) {}
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index db2e147e069f..411684c80cf3 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -19,7 +19,7 @@
#include <linux/percpu-rwsem.h>
#include <linux/u64_stats_sync.h>
#include <linux/workqueue.h>
-#include <linux/bpf-cgroup.h>
+#include <linux/bpf-cgroup-defs.h>
#include <linux/psi_types.h>
#ifdef CONFIG_CGROUPS
--
2.31.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH bpf-next v5 0/3] bpf: remove the cgroup -> bpf header dependecy
2021-12-16 2:55 [PATCH bpf-next v5 0/3] bpf: remove the cgroup -> bpf header dependecy Jakub Kicinski
` (2 preceding siblings ...)
2021-12-16 2:55 ` [PATCH bpf-next v5 3/3] bpf: remove the cgroup -> bpf header dependecy Jakub Kicinski
@ 2021-12-16 23:05 ` Alexei Starovoitov
2021-12-16 23:44 ` Jakub Kicinski
3 siblings, 1 reply; 6+ messages in thread
From: Alexei Starovoitov @ 2021-12-16 23:05 UTC (permalink / raw)
To: Jakub Kicinski; +Cc: Daniel Borkmann, Alexei Starovoitov, Andrii Nakryiko, bpf
On Wed, Dec 15, 2021 at 6:55 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> Changes to bpf.h tend to clog up our build systems. The netdev/bpf
> build bot does incremental builds to save time (reusing the build
> directory to only rebuild changed objects).
>
> This is the rough breakdown of how many objects needs to be rebuilt
> based on file touched:
>
> kernel.h 40633
> bpf.h 17881
> bpf-cgroup.h 17875
> skbuff.h 10696
> bpf-netns.h 7604
> netdevice.h 7452
> filter.h 5003
> sock.h 4959
> tcp.h 4048
>
> As the stats show touching bpf.h is _very_ expensive.
>
> Bulk of the objects get rebuilt because MM includes cgroup headers.
> Luckily bpf-cgroup.h does not fundamentally depend on bpf.h so we
> can break that dependency and reduce the number of objects.
>
> With the patches applied touching bpf.h causes 5019 objects to be rebuilt
> (17881 / 5019 = 3.56x). That's pretty much down to filter.h plus noise.
>
> v2:
> Try to make the new headers wider in scope. Collapse bpf-link and
> bpf-cgroup-types into one header, which may serve as "BPF kernel
> API" header in the future if needed. Rename bpf-cgroup-storage.h
> to bpf-inlines.h.
>
> Add a fix for the s390 build issue.
>
> v3: https://lore.kernel.org/all/20211215061916.715513-1-kuba@kernel.org/
> Merge bpf-includes.h into bpf.h.
>
> v4: https://lore.kernel.org/all/20211215181231.1053479-1-kuba@kernel.org/
> Change course - break off cgroup instead of breaking off bpf.
>
> v5:
> Add forward declaration of struct bpf_prog to perf_event.h
> when !CONFIG_BPF_SYSCALL (kbuild bot).
Applied. Thanks
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH bpf-next v5 0/3] bpf: remove the cgroup -> bpf header dependecy
2021-12-16 23:05 ` [PATCH bpf-next v5 0/3] " Alexei Starovoitov
@ 2021-12-16 23:44 ` Jakub Kicinski
0 siblings, 0 replies; 6+ messages in thread
From: Jakub Kicinski @ 2021-12-16 23:44 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Daniel Borkmann, Alexei Starovoitov, Andrii Nakryiko, bpf
On Thu, 16 Dec 2021 15:05:00 -0800 Alexei Starovoitov wrote:
> On Wed, Dec 15, 2021 at 6:55 PM Jakub Kicinski <kuba@kernel.org> wrote:
> >
> > Changes to bpf.h tend to clog up our build systems. The netdev/bpf
> > build bot does incremental builds to save time (reusing the build
> > directory to only rebuild changed objects).
> >
> > This is the rough breakdown of how many objects needs to be rebuilt
> > based on file touched:
> >
> > kernel.h 40633
> > bpf.h 17881
> > bpf-cgroup.h 17875
> > skbuff.h 10696
> > bpf-netns.h 7604
> > netdevice.h 7452
> > filter.h 5003
> > sock.h 4959
> > tcp.h 4048
>
> Applied. Thanks
Thanks! I got another patch to get to 1k objects, by taking filter.h
include out of sock.h but I'll give it a week for the dust to settle
here.
^ permalink raw reply [flat|nested] 6+ messages in thread