[PATCH bpf-next v3 2/4] bpf: Introduce __bpf_kfunc_mark_deprecated annotation

[Kumar Kartikeya Dwivedi <memxor@gmail.com>]

Documentation for KF_DEPRECATED documents the expectations and
recommendations for deprecating existing kfuncs. Until now, such a flag
never materialized. Unlike other kfunc flags, more context needs to be
carried with such a flag, thus we introduce __bpf_kfunc_mark_deprecated
as a replacement for the (non-existent) kfunc flag which takes a
replacement kfunc to provide recommendations to the user.

Apply it to all existing impl-suffixed kfuncs that are preserved for
backwards compatibility for a few kernel releases, after we introduced
versions with KF_IMPLICIT_ARGS.

Note that this is only supported by LLVM-compiled kernels as of writing,
since GCC does not support for BTF declaration tags.

Cc: Vineet Gupta <vineet.gupta@linux.dev>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 Documentation/bpf/kfuncs.rst                  | 42 ++++++++++---------
 include/linux/bpf_verifier.h                  |  1 +
 include/linux/btf.h                           |  5 +++
 include/linux/compiler_types.h                | 14 ++++++-
 kernel/bpf/helpers.c                          |  8 ++++
 kernel/bpf/verifier.c                         | 41 ++++++++++++++++++
 .../selftests/bpf/test_kmods/bpf_testmod.c    |  1 +
 7 files changed, 90 insertions(+), 22 deletions(-)

diff --git a/Documentation/bpf/kfuncs.rst b/Documentation/bpf/kfuncs.rst
index 47d60011d8eb..5beef4432a8d 100644
--- a/Documentation/bpf/kfuncs.rst
+++ b/Documentation/bpf/kfuncs.rst
@@ -344,22 +344,23 @@ RCU protection but do not take RCU protected arguments.
 
 .. _KF_deprecated_flag:
 
-2.5.8 KF_DEPRECATED flag
-------------------------
-
-The KF_DEPRECATED flag is used for kfuncs which are scheduled to be
-changed or removed in a subsequent kernel release. A kfunc that is
-marked with KF_DEPRECATED should also have any relevant information
-captured in its kernel doc. Such information typically includes the
-kfunc's expected remaining lifespan, a recommendation for new
-functionality that can replace it if any is available, and possibly a
-rationale for why it is being removed. When verifier warning logging is
-requested, the verifier will emit a warning when a BPF program uses a
-deprecated kfunc.
-
-Note that while on some occasions, a KF_DEPRECATED kfunc may continue to be
-supported and have its KF_DEPRECATED flag removed, it is likely to be far more
-difficult to remove a KF_DEPRECATED flag after it's been added than it is to
+2.5.8 Deprecated kfunc annotation
+---------------------------------
+
+The ``__bpf_kfunc_mark_deprecated(replacement)`` annotation is used for
+kfuncs which are scheduled to be changed or removed in a subsequent
+kernel release. A kfunc annotated with
+``__bpf_kfunc_mark_deprecated(replacement)`` should also have any
+relevant information captured in its kernel doc. Such information
+typically includes the kfunc's expected remaining lifespan, a
+recommendation for new functionality that can replace it if any is
+available, and possibly a rationale for why it is being removed. When
+verifier warning logging is requested, the verifier will emit a warning
+when a BPF program uses such a deprecated kfunc.
+
+Note that while on some occasions, a deprecated kfunc may continue to be
+supported and have its deprecation annotation removed, it is likely to be far
+more difficult to remove such an annotation after it's been added than it is to
 prevent it from being added in the first place. As described in
 :ref:`BPF_kfunc_lifecycle_expectations`, users that rely on specific kfuncs are
 encouraged to make their use-cases known as early as possible, and participate
@@ -534,10 +535,11 @@ maintainer if it turns out that the kfunc is actually being used.
 
 It's expected that the common case will be that kfuncs will go through a
 deprecation period rather than being changed or removed without warning. As
-described in :ref:`KF_deprecated_flag`, the kfunc framework provides the
-KF_DEPRECATED flag to kfunc developers to signal to users that a kfunc has been
-deprecated. Once a kfunc has been marked with KF_DEPRECATED, the following
-procedure is followed for removal:
+described in :ref:`KF_deprecated_flag`, the kfunc framework provides
+the ``__bpf_kfunc_mark_deprecated(replacement)`` annotation to signal to users
+that a kfunc has been deprecated and to identify the replacement kfunc. Once a
+kfunc has been marked deprecated, the following procedure is followed for
+removal:
 
 1. Any relevant information for deprecated kfuncs is documented in the kfunc's
    kernel docs. This documentation will typically include the kfunc's expected
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 515afcb83ec7..b70081fe24d8 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -1461,6 +1461,7 @@ struct bpf_kfunc_desc {
 	u32 func_id;
 	s32 imm;
 	u16 offset;
+	bool warned_deprecated;
 	unsigned long addr;
 };
 
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 48108471c5b1..c75ecef67096 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -88,6 +88,11 @@
  */
 #define __bpf_kfunc __used __retain __noclone noinline
 
+#define BPF_KFUNC_DECL_TAG_DEPRECATED "bpf:kfunc:deprecated:"
+
+#define __bpf_kfunc_mark_deprecated(replacement) \
+	BTF_DECL_TAG(BPF_KFUNC_DECL_TAG_DEPRECATED __stringify(replacement))
+
 #define __bpf_kfunc_start_defs()					       \
 	__diag_push();							       \
 	__diag_ignore_all("-Wmissing-declarations",			       \
diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h
index e8fd77593b68..ba5b20e30d4d 100644
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -35,10 +35,20 @@
  * see https://github.com/rust-lang/rust-bindgen/issues/2244.
  */
 #if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && \
-	__has_attribute(btf_type_tag) && !defined(__BINDGEN__)
-# define BTF_TYPE_TAG(value) __attribute__((btf_type_tag(#value)))
+	!defined(__BINDGEN__)
+# if __has_attribute(btf_type_tag)
+#  define BTF_TYPE_TAG(value) __attribute__((btf_type_tag(#value)))
+# else
+#  define BTF_TYPE_TAG(value) /* nothing */
+# endif
+# if __has_attribute(btf_decl_tag)
+#  define BTF_DECL_TAG(value) __attribute__((btf_decl_tag(value)))
+# else
+#  define BTF_DECL_TAG(value) /* nothing */
+# endif
 #else
 # define BTF_TYPE_TAG(value) /* nothing */
+# define BTF_DECL_TAG(value) /* nothing */
 #endif
 
 #include <linux/compiler-context-analysis.h>
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 2bb60200c266..69b4eae257a1 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -2343,6 +2343,7 @@ __bpf_kfunc void *bpf_obj_new(u64 local_type_id__k, struct btf_struct_meta *meta
 	return p;
 }
 
+__bpf_kfunc_mark_deprecated(bpf_obj_new)
 __bpf_kfunc void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign)
 {
 	return bpf_obj_new(local_type_id__k, meta__ign);
@@ -2367,6 +2368,7 @@ __bpf_kfunc void *bpf_percpu_obj_new(u64 local_type_id__k, struct btf_struct_met
 	return bpf_mem_alloc(&bpf_global_percpu_ma, size);
 }
 
+__bpf_kfunc_mark_deprecated(bpf_percpu_obj_new)
 __bpf_kfunc void *bpf_percpu_obj_new_impl(u64 local_type_id__k, void *meta__ign)
 {
 	return bpf_percpu_obj_new(local_type_id__k, meta__ign);
@@ -2410,6 +2412,7 @@ __bpf_kfunc void bpf_obj_drop(void *p__alloc, struct btf_struct_meta *meta)
 	__bpf_obj_drop_impl(p, meta ? meta->record : NULL, false);
 }
 
+__bpf_kfunc_mark_deprecated(bpf_obj_drop)
 __bpf_kfunc void bpf_obj_drop_impl(void *p__alloc, void *meta__ign)
 {
 	return bpf_obj_drop(p__alloc, meta__ign);
@@ -2428,6 +2431,7 @@ __bpf_kfunc void bpf_percpu_obj_drop(void *p__alloc, struct btf_struct_meta *met
 	bpf_mem_free_rcu(&bpf_global_percpu_ma, p__alloc);
 }
 
+__bpf_kfunc_mark_deprecated(bpf_percpu_obj_drop)
 __bpf_kfunc void bpf_percpu_obj_drop_impl(void *p__alloc, void *meta__ign)
 {
 	bpf_percpu_obj_drop(p__alloc, meta__ign);
@@ -2460,6 +2464,7 @@ __bpf_kfunc void *bpf_refcount_acquire(void *p__refcounted_kptr, struct btf_stru
 	return (void *)p__refcounted_kptr;
 }
 
+__bpf_kfunc_mark_deprecated(bpf_refcount_acquire)
 __bpf_kfunc void *bpf_refcount_acquire_impl(void *p__refcounted_kptr, void *meta__ign)
 {
 	return bpf_refcount_acquire(p__refcounted_kptr, meta__ign);
@@ -2514,6 +2519,7 @@ __bpf_kfunc int bpf_list_push_front(struct bpf_list_head *head,
 	return __bpf_list_add(n, head, false, meta ? meta->record : NULL, off);
 }
 
+__bpf_kfunc_mark_deprecated(bpf_list_push_front)
 __bpf_kfunc int bpf_list_push_front_impl(struct bpf_list_head *head,
 					 struct bpf_list_node *node,
 					 void *meta__ign, u64 off)
@@ -2543,6 +2549,7 @@ __bpf_kfunc int bpf_list_push_back(struct bpf_list_head *head,
 	return __bpf_list_add(n, head, true, meta ? meta->record : NULL, off);
 }
 
+__bpf_kfunc_mark_deprecated(bpf_list_push_back)
 __bpf_kfunc int bpf_list_push_back_impl(struct bpf_list_head *head,
 					struct bpf_list_node *node,
 					void *meta__ign, u64 off)
@@ -2683,6 +2690,7 @@ __bpf_kfunc int bpf_rbtree_add(struct bpf_rb_root *root,
 	return __bpf_rbtree_add(root, n, (void *)less, meta ? meta->record : NULL, off);
 }
 
+__bpf_kfunc_mark_deprecated(bpf_rbtree_add)
 __bpf_kfunc int bpf_rbtree_add_impl(struct bpf_rb_root *root, struct bpf_rb_node *node,
 				    bool (less)(struct bpf_rb_node *a, const struct bpf_rb_node *b),
 				    void *meta__ign, u64 off)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 8de2a4e5f5de..1f3b44f47949 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -12987,6 +12987,45 @@ static int check_return_code(struct bpf_verifier_env *env, int regno, const char
 static int process_bpf_exit_full(struct bpf_verifier_env *env,
 				 bool *do_print_state, bool exception_exit);
 
+static void warn_for_deprecated_kfuncs(struct bpf_verifier_env *env,
+				       struct bpf_kfunc_call_arg_meta *meta,
+				       int insn_idx, s16 offset)
+{
+	const struct bpf_line_info *linfo;
+	const char *replacement;
+	const struct btf_type *t;
+	struct bpf_kfunc_desc *desc;
+	const char *file;
+	int line_num;
+
+	if (!env->prog->aux->btf)
+		return;
+
+	t = btf_type_by_id(meta->btf, meta->func_id);
+	replacement = btf_find_decl_tag_value(meta->btf, t, -1, BPF_KFUNC_DECL_TAG_DEPRECATED);
+	if (IS_ERR(replacement) || str_is_empty(replacement))
+		return;
+
+	desc = find_kfunc_desc(env->prog, meta->func_id, offset);
+	if (!desc || desc->warned_deprecated)
+		return;
+
+	linfo = bpf_find_linfo(env->prog, insn_idx);
+	if (linfo) {
+		bpf_get_linfo_file_line(env->prog->aux->btf, linfo, &file, NULL, &line_num);
+		warn(env, "%s:%d (insn #%d) uses deprecated kfunc %s(), which will be removed.\n",
+		     file, line_num, insn_idx, meta->func_name);
+	} else {
+		warn(env, "(insn #%d) uses deprecated kfunc %s(), which will be removed.\n",
+		     insn_idx, meta->func_name);
+	}
+
+	warn(env, "Switch to kfunc %s() instead.\n", replacement);
+	warn(env, "For older kernels, choose the correct kfunc using bpf_ksym_exists().\n");
+
+	desc->warned_deprecated = true;
+}
+
 static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 			    int *insn_idx_p)
 {
@@ -13016,6 +13055,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 
 	insn_aux->is_iter_next = bpf_is_iter_next_kfunc(&meta);
 
+	warn_for_deprecated_kfuncs(env, &meta, insn_idx, insn->off);
+
 	if (!insn->off &&
 	    (insn->imm == special_kfunc_list[KF_bpf_res_spin_lock] ||
 	     insn->imm == special_kfunc_list[KF_bpf_res_spin_lock_irqsave])) {
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index d876314a4d67..5a57a92a8f8c 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -1252,6 +1252,7 @@ __bpf_kfunc int bpf_kfunc_multi_st_ops_test_1_assoc(struct st_ops_args *args, st
 
 __bpf_kfunc int bpf_kfunc_implicit_arg(int a, struct bpf_prog_aux *aux);
 __bpf_kfunc int bpf_kfunc_implicit_arg_legacy(int a, int b, struct bpf_prog_aux *aux);
+__bpf_kfunc_mark_deprecated(bpf_kfunc_implicit_arg_legacy)
 __bpf_kfunc int bpf_kfunc_implicit_arg_legacy_impl(int a, int b, struct bpf_prog_aux *aux);
 
 /* hook targets */
-- 
2.52.0