Re: [RFC PATCH bpf-next 02/12] libbpf: Introduce bpf SDT probe macros

Re: [RFC PATCH bpf-next 02/12] libbpf: Introduce bpf SDT probe macros

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] Hardcoded named labels in inline assembly break compilation during routine compiler optimizations.
- [High] Missing `static` keyword on dummy type-checking variables causes linker errors.
- [High] Missing `"memory"` clobber in inline assembly allows the compiler to reorder memory operations past the probe site.
- [Medium] Macro local variable names (`__a0`, `__a1`, etc.) are insufficiently obfuscated, leading to variable shadowing and trace data corruption.
- [Low] BPF subsystem multi-line comment style violation.
--

commit 225a1b2d44558cc189a9e559d3f30514eab101a7
Author: Xu Kuohai <xukuohai@huawei.com>

libbpf: Introduce bpf SDT probe macros

Introduce macros to declare and define bpf SDT probes. The declaration is
used to generate BTF ID of FUNC_PROTO for each probe site, so the probe
sites can be validated by the kernel, and observer programs can be attached
to the FUNC_PROTO like normal tracing programs.

> diff --git a/tools/lib/bpf/bpf_sdt.h b/tools/lib/bpf/bpf_sdt.h
> new file mode 100644
> index 0000000000000..23f84460aa368
> --- /dev/null
> +++ b/tools/lib/bpf/bpf_sdt.h

[ ... ]

> +#define BPF_SDT_DECLARE1(name, t0)					\
> +	___sdt_decl_attr(name, 1)					\
> +	static void (*const __sdt_type_##name)(t0);			\
> +	___sdt_chk_attr t0 __sdt_chk_##name##_0

[Severity: High]
Could this missing static keyword cause linker errors?

If a developer places BPF_SDT_DECLARE<N> in a shared header included in
multiple translation units, modern compilers defaulting to -fno-common might
throw multiple strong definition errors for these dummy variables since they
lack the static keyword.

[ ... ]

> +/* probe macros, 0-5 arguments
> + *
> + * Each probe emits a NOP (goto +0) in the program stream plus a
> + * .bpf_sdt_notes entry.  The entry begins with the ___sdt_jt_<name>

[Severity: Low]
This isn't a bug, but the BPF subsystem multi-line comment style requires
the opening /* to be on its own line.

> + * label so that libbpf can locate each entry boundary at load time.
> + * One BPF_SDT_PROBE<N> invocation per probe name is required; invoking
> + * the same name twice produces a duplicate symbol.
> + */
> +#define BPF_SDT_PROBE(name)						\
> +({									\
> +	asm volatile(							\
> +		"0:\n"							\
> +		"goto +0\n"						\
> +		".pushsection .bpf_sdt_notes,\"\",@progbits\n"		\
> +		"___sdt_jt_" #name ":\n"				\

[Severity: High]
Does this inline assembly block need a "memory" clobber?

Without it, could the compiler reorder memory operations past the probe
site, causing attached observer programs to read stale or uncommitted state
when triggered?

[Severity: High]
Can the fixed label ___sdt_jt_ #name cause a build failure?

If the compiler unrolls a loop containing this macro or inlines an
__always_inline caller multiple times, it seems the assembler would
encounter multiple identical labels and fail with a redefinition error.

[ ... ]

> +#define BPF_SDT_PROBE2(name, a, b)					\
> +({									\
> +	_Static_assert(__builtin_types_compatible_p(typeof(a),		\
> +			typeof(__sdt_chk_##name##_0)),			\
> +			"BPF_SDT_PROBE2: arg0 type mismatch");		\
> +	_Static_assert(sizeof(typeof(a)) <= 8,				\
> +			"BPF_SDT_PROBE2: arg0 too large, use pointer");	\
> +	_Static_assert(__builtin_types_compatible_p(typeof(b),		\
> +			typeof(__sdt_chk_##name##_1)),			\
> +			"BPF_SDT_PROBE2: arg1 type mismatch");		\
> +	_Static_assert(sizeof(typeof(b)) <= 8,				\
> +			"BPF_SDT_PROBE2: arg1 too large, use pointer");	\
> +	typeof(a) __a0 = (a);						\
> +	typeof(b) __a1 = (b);						\

[Severity: Medium]
Can these local variable names silently shadow caller variables?

If a caller passes a variable named __a0 as the second argument (b), does
the declaration of __a1 capture the macro's newly declared __a0 instead of
the caller's variable, resulting in the probe recording the first argument
twice?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=2

Re: [RFC PATCH bpf-next 03/12] libbpf: Add bpf_sdt_notes section parser

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] Integer overflow in `sdt_calc_nargs` bypasses bounds checks, leading to an out-of-bounds memory read when parsing SDT notes.
- [High] Missing bounds check on the ELF relocation symbol index leads to an out-of-bounds read in the symbol table.
- [Medium] Memory leak when parsing multiple `.bpf_sdt_notes` ELF sections.
--

commit f001c05c32f475aab765178535ef62a015937785
Author: Xu Kuohai <xukuohai@huawei.com>

libbpf: Add bpf_sdt_notes section parser

This commit parses the bpf_sdt_notes section to get the probe call site
index in the bpf prog, and argument registers for each probe.

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 7162146280a80..327c9d412fe89 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c

[ ... ]

> @@ -3892,6 +3909,254 @@ static int cmp_progs(const void *_a, const void *_b)
> +static int sdt_calc_nargs(struct sdt_sym *syms, int sym_idx, size_t sym_cnt, size_t last_off)
> +{
> +	size_t off = syms[sym_idx].off;
> +	size_t next_off = (sym_idx + 1 < sym_cnt) ? syms[sym_idx + 1].off : last_off;
> +
> +	if (next_off > last_off)
> +		return -EINVAL;
> +
> +	if (off + 8 > next_off)

[Severity: High]
Does this code allow an integer overflow if off is close to UINT64_MAX?

If off (which comes from the ELF symbol's st_value) is large enough
(e.g., UINT64_MAX - 7), off + 8 will wrap around to a small value,
bypassing this bounds check. This can later result in an out-of-bounds
memory read when memcpy() is called with data + off in
bpf_object__collect_sdt_notes().

> +		return -EINVAL;
> +
> +	if ((next_off - off - 8) % sizeof(struct bpf_insn))
> +		return -EINVAL;

[ ... ]

> +static int sdt_find_sec_idx(struct bpf_object *obj, Elf_Data *sdt_relo,
> +			    size_t sdt_entry_off)
> +{
> +	int i;
> +	Elf64_Sym *tgt;
> +	Elf64_Rel *rel = sdt_relo->d_buf;
> +	Elf64_Sym *elf_syms = obj->efile.symbols->d_buf;
> +
> +	for (i = 0; i < sdt_relo->d_size / sizeof(Elf64_Rel); i++, rel++) {
> +		if (rel->r_offset != sdt_entry_off)
> +			continue;
> +		if (ELF64_R_TYPE(rel->r_info) != R_BPF_64_ABS64)
> +			continue;
> +
> +		tgt = &elf_syms[ELF64_R_SYM(rel->r_info)];

[Severity: High]
Could this array access go out of bounds?

ELF64_R_SYM(rel->r_info) extracts the symbol index directly from the ELF
relocation entry, but there doesn't appear to be a check to ensure this
index is less than the total number of symbols in the table. A maliciously
crafted ELF could provide a large index and trigger an out-of-bounds memory
read.

> +		return tgt->st_shndx;
> +	}
> +
> +	return -1;
> +}

[ ... ]

> @@ -4034,6 +4299,13 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
>  				memcpy(obj->jumptables_data, data->d_buf, data->d_size);
>  				obj->jumptables_data_sz = data->d_size;
>  				obj->efile.jumptables_data_shndx = idx;
> +			} else if (strcmp(name, SDT_NOTES_SEC) == 0) {
> +				obj->sdt_notes_data = malloc(data->d_size);

[Severity: Medium]
Can this code leak memory if there are multiple ELF sections named
".bpf_sdt_notes"?

If a malformed ELF file contains multiple sections with the name
SDT_NOTES_SEC, malloc() will unconditionally allocate memory and overwrite
obj->sdt_notes_data, leaking the previously allocated buffer.

> +				if (!obj->sdt_notes_data)
> +					return -ENOMEM;
> +				memcpy(obj->sdt_notes_data, data->d_buf, data->d_size);
> +				obj->sdt_notes_data_sz = data->d_size;
> +				obj->efile.sdt_notes_shndx = idx;
>  			} else {
>  				pr_info("elf: skipping unrecognized data section(%d) %s\n",
>  					idx, name);

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=3

Re: [RFC PATCH bpf-next 06/12] bpf: Add type check for SDT probe site

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Out-of-bounds read of the verifier state `regs` array in `check_sdt_probe` due to unvalidated `val->arg_reg[i]`.
- [High] Missing validation of constant offset for `PTR_TO_BTF_ID` arguments in `check_sdt_probe` leading to type confusion.
- [Medium] Out-of-bounds read in `compute_insn_live_regs` due to unvalidated `sdt->nargs`.
- [Medium] If `bpf_insn_array_init` fails during SDT probe validation, it leaves the map permanently locked (`used = 1`).
--

commit 1e24bd6e76a99cfb094092a5071e910e30cca308
Author: Xu Kuohai <xukuohai@huawei.com>

bpf: Add type check for SDT probe site

The NOP instruction at SDT probe site will be patched to a call
instruction to observer programs. To ensure the arguments passed
to observers are as expected, add type check for bpf SDT probe site.

> diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
> index aca8676ab5633..fe30b08712fff 100644
> --- a/kernel/bpf/bpf_insn_array.c
> +++ b/kernel/bpf/bpf_insn_array.c

[ ... ]

> @@ -224,6 +227,15 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
>  	for (i = 0; i < map->max_entries; i++)
>  		values[i].xlated_off = values[i].orig_off;
>  
> +	if (insn_array->subtype == BPF_INSN_ARRAY_SUBTYPE_SDT) {
> +		for (i = 0; i < map->max_entries; i++) {
> +			insn = &prog->insnsi[values[i].orig_off];
> +			if (insn->code != (BPF_JMP | BPF_JA) || insn->off != 0)
> +				return -EINVAL;

[Severity: Medium]
If the instruction check fails and returns -EINVAL here, does it leave
the map permanently locked?

Earlier in bpf_insn_array_init() we do:

	if (atomic_xchg(&insn_array->used, 1))
		return -EBUSY;

If we return -EINVAL without clearing this flag or adding the map to
env->insn_array_maps for standard cleanup, it seems the map will remain
unusable. Should this error path clear the used flag?

> +			env->insn_aux_data[values[i].orig_off].sdt_entry = &values[i];
> +		}
> +	}
> +
>  	return 0;
>  }

[ ... ]

> diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
> index 0aadfbae0acc5..50ab8c29c0ebd 100644
> --- a/kernel/bpf/liveness.c
> +++ b/kernel/bpf/liveness.c

[ ... ]

> @@ -2163,10 +2164,27 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
>  		switch (code) {
>  		case BPF_JA:
>  			def = 0;
> -			if (BPF_SRC(insn->code) == BPF_X)
> +			if (BPF_SRC(insn->code) == BPF_X) {
>  				use = dst;
> -			else
> +			} else if (env->insn_aux_data[insn_idx].sdt_entry) {
> +				struct bpf_insn_array_value *sdt;
> +				int i;
> +
> +				/*
> +				 * Without marking the argument registers arg_reg[]
> +				 * as live, the liveness pass would clear them before
> +				 * the probe site, causing check_sdt_probe() to reject
> +				 * the prog with "arg is uninitialized".
> +				 */
>  				use = 0;
> +				sdt = env->insn_aux_data[insn_idx].sdt_entry;
> +				for (i = 0; i < sdt->nargs; i++) {
> +					if (sdt->arg_reg[i] < BPF_REG_FP)
> +						use |= BIT(sdt->arg_reg[i]);
> +				}

[Severity: Medium]
Can this loop read out of bounds of the arg_reg array?

The sdt->nargs value is provided by userspace and doesn't appear to be
validated prior to this liveness pass. Since arg_reg is defined as a
5-element array in struct bpf_insn_array_value, a malicious userspace
program could provide a large nargs value, causing this loop to read past
the end of arg_reg and corrupt the verifier's liveness state.

Should we validate sdt->nargs against the maximum allowed arguments
before iterating?

> +			} else {
> +				use = 0;
> +			}
>  			break;

[ ... ]

> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 05734163650a7..bc972beb80cf4 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c

[ ... ]

> +static int check_sdt_probe(struct bpf_verifier_env *env, int insn_idx)
> +{

[ ... ]

> +	frame = env->cur_state->frame[env->cur_state->curframe];
> +	regs = frame->regs;
> +
> +	for (i = 0; i < val->nargs; i++) {
> +		u8 reg = val->arg_reg[i];
> +		struct bpf_reg_state *rs = &regs[reg];

[Severity: High]
Could this lead to an out-of-bounds read on the verifier's regs array?

The val->arg_reg[i] values come directly from a BPF_MAP_TYPE_INSN_ARRAY
populated by userspace. If a user provides an index larger than or equal to
MAX_BPF_REG, regs[reg] will access memory past the end of the regs array in
the bpf_func_state structure, potentially causing a kernel panic.

Does val->arg_reg[i] need a bounds check against MAX_BPF_REG before
it is used as an index?

> +
> +		if (rs->type == NOT_INIT) {
> +			verbose(env, "SDT arg%d (r%d) is uninitialized\n", i, reg);
> +			return -EINVAL;
> +		}
> +	}
> +

[ ... ]

> +		if (base_type(rs->type) != PTR_TO_BTF_ID) {
> +			verbose(env, "SDT arg%d (r%d) type %s expected PTR_TO_BTF_ID\n",
> +				i, reg, reg_type_str(env, rs->type));
> +			return -EACCES;
> +		}
> +
> +		if (!btf_struct_ids_match(&env->log, rs->btf, rs->btf_id,
> +					  rs->var_off.value, btf, arg_btf_id, false)) {

[Severity: High]
Is it safe to pass rs->var_off.value directly without ensuring the offset
is constant?

If a BPF program adds a variable offset to a PTR_TO_BTF_ID register,
rs->var_off.value will just represent the lower bound or known bits of
that offset. Without enforcing tnum_is_const(rs->var_off), the verifier
might accept a variable-offset pointer as if it points to the base object.

Could this cause type confusion or out-of-bounds reads in the observer
program? Should we verify the offset is strictly constant first?

> +			verbose(env, "SDT arg%d (r%d) btf_id %u does not match expected %u\n",
> +				i, reg, rs->btf_id, arg_btf_id);
> +			return -EACCES;
> +		}
> +	}
> +	return 0;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=6

Re: [RFC PATCH bpf-next 07/12] bpf: Record probe name in SDT map

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] Strict `value_size` validation on the expanded `struct bpf_insn_array_value` breaks backwards and forwards compatibility.
- [Medium] Memory leak of `e->name` on the error path in `bpf_object__collect_sdt_notes` when a probe name exceeds `BPF_SDT_MAX_NAME_LEN`.
--

commit 33292e7627dcc830a8b1dc17788ae56ec52849c6
Author: Xu Kuohai <xukuohai@huawei.com>

bpf: Record probe name in SDT map

This commit adds a probe name field to the bpf_insn_array_value structure to
help the kernel locate target programs and probe sites. It enforces a maximum
name length of 64 characters and updates libbpf to pass the name during map
creation.

> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index 773a7def0fbd3..95ca41bf75018 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
[ ... ]
> @@ -7743,6 +7744,7 @@ struct bpf_insn_array_value {
>  	__u8  nargs;		/* argument count (0..5) */
>  	__u8  arg_reg[5];	/* BPF register for each argument */
>  	__u8  pad[2];
> +	char  name[BPF_SDT_MAX_NAME_LEN]; /* probe name */
>  };

[Severity: Critical]
Does changing the size of struct bpf_insn_array_value break backwards
compatibility with older userspace applications?

It appears the kernel enforces a strict value_size check during map creation
in insn_array_alloc_check():

	if (attr->max_entries == 0 || attr->key_size != 4 ||
	    attr->value_size != value_size ||
	    attr->map_flags & ~BPF_F_INSN_ARRAY_SDT)
		return -EINVAL;

Because value_size is derived from sizeof(struct bpf_insn_array_value),
older binaries passing a smaller value_size of 16 to
bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY) might now be rejected with -EINVAL.

[ ... ]
> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index badac128e8adc..dbc08a1931015 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
> @@ -4146,6 +4146,12 @@ static int bpf_object__collect_sdt_notes(struct bpf_object *obj)
>  			err = -ENOMEM;
>  			goto out;
>  		}
> +		if (strlen(e->name) >= BPF_SDT_MAX_NAME_LEN) {
> +			pr_warn("sdt: probe name '%s' too long (max %d chars)\n",
> +				e->name, BPF_SDT_MAX_NAME_LEN - 1);
> +			err = -EINVAL;
> +			goto out;
> +		}

[Severity: Medium]
Can this goto leak the newly allocated e->name string?

Just above this snippet, e->name is allocated:

		e->name = strdup(sdt_syms[i].name + 10);
		if (!e->name) {
			err = -ENOMEM;
			goto out;
		}

If the length exceeds BPF_SDT_MAX_NAME_LEN, we jump to the out label.
However, the cleanup loop in out:

out:
	if (err) {
		for (i = 0; i < obj->sdt_entry_cnt; i++)
			free(obj->sdt_entries[i].name);

only frees up to obj->sdt_entry_cnt, which hasn't been incremented yet for
this entry, potentially causing the string to be leaked.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=7

Re: [RFC PATCH bpf-next 08/12] libbpf: Add libbpf support to load SDT observer program

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The SEC_DEF for the newly added `bpf_sdt` section is missing its attach function (`attach_trace`), which completely breaks auto-attaching functionality for SDT programs via libbpf and BPF skeletons.
--

commit 507bbcea213558e6c107385fb1d4d27318f9df5b
Author: Xu Kuohai <xukuohai@huawei.com>

libbpf: Add libbpf support to load SDT observer program

This commit adds libbpf support for loading SDT observer programs. It allows
passing the target BPF program file descriptor and probe name to the kernel
so that the kernel can look up the probe in the target program's SDT map.

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index dbc08a1931015..273991b80f8f7 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c

[ ... ]

> @@ -10587,6 +10595,7 @@ static const struct bpf_sec_def section_defs[] = {
>  	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
>  	SEC_DEF("usdt+",		KPROBE,	0, SEC_USDT, attach_usdt),
>  	SEC_DEF("usdt.s+",		KPROBE,	0, SEC_USDT | SEC_SLEEPABLE, attach_usdt),
> +	SEC_DEF("bpf_sdt",		TRACING, BPF_TRACE_SDT, SEC_NONE),

[Severity: Medium]
Is the attach callback missing from this SEC_DEF? Without a fifth argument
like attach_trace, the prog->sec_def->prog_attach_fn pointer will be NULL,
which silently skips auto-attach functionality for SDT programs when users
call a BPF skeleton's __attach() function, forcing developers to manually
manage BPF link creation.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=8

Re: [RFC PATCH bpf-next 09/12] bpf: Add kernel support to load SDT observer program

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [High] Reference count leak of `dst_prog` due to pointer overwrite in `bpf_prog_load()`.
- [High] Error paths in `BPF_TRACE_SDT` handling bypass `put_sdt` cleanup, leaking `attach_btf` and `dst_prog`.
- [High] Memory leak of `prog->aux->sdt_probe` on early load failures in `bpf_prog_load()`.
- [Medium] bpf_insn_array_get_sdt_probe_by_name returns an incorrect semantic error code when a probe is not found.
--

commit 02476bf5b9d80f3ff0471539243d877d096c0806
Author: Xu Kuohai <xukuohai@huawei.com>

bpf: Add kernel support to load SDT observer program

Add kernel support to load SDT observer program. The program is
verified as a normal tracing type program while the target FUNC_PROTO
is resolved in the SDT map of the target program using the probe name
passed by the user.

> diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
> index a9564ef4b2efa..067cc1b817d5f 100644
> --- a/kernel/bpf/bpf_insn_array.c
> +++ b/kernel/bpf/bpf_insn_array.c
[ ... ]
> +int bpf_insn_array_get_sdt_probe_by_name(struct bpf_prog *prog, const char *name,
> +					 struct bpf_insn_array_value *val,
> +					 unsigned long *ip)
> +{
[ ... ]
> +		for (j = 0; j < map->max_entries; j++) {
> +			if (insn_array->values[j].xlated_off == INSN_DELETED)
> +				continue;
> +			if (!strcmp(insn_array->values[j].name, name)) {
> +				*val = insn_array->values[j];
> +				*ip = (unsigned long)insn_array->ips[j];
> +				return 0;
> +			}
> +		}
> +	}
> +
> +	return -EEXIST;

[Severity: Medium]
Is -EEXIST the correct error code when a probe is not found? Would -ENOENT be
more appropriate to indicate a missing entry?

> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index dc881e5ad4114..8b2c73bb6c2a8 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
[ ... ]
> @@ -3082,33 +3085,75 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_at
>  		btf_get(attach_btf);
>  	}
>  
> +	/*
> +	 * find the probe site in the target program at load time of the
> +	 * observer program, so the verifier can check the observer's context
> +	 * arguments from the probe's FUNC_PROTO.
> +	 */
> +	if (attr->expected_attach_type == BPF_TRACE_SDT) {
> +		struct bpf_insn_array_value sdt_val;
> +		struct bpf_sdt_probe_info *sdt_probe;
> +		unsigned long probe_ip;
> +
> +		if (!bpf_jit_supports_sdt_probe()) {
> +			err = -EOPNOTSUPP;
> +			goto put_token;
> +		}
> +		if (!attr->sdt.target_prog_fd) {
> +			err = -EINVAL;
> +			goto put_token;

[Severity: High]
Does jumping to put_token here bypass the cleanup of attach_btf and dst_prog?
If those resources were acquired earlier in bpf_prog_load(), it looks like
they might be leaked since the put_sdt label is skipped.

> +		}
> +		dst_prog = bpf_prog_get(attr->sdt.target_prog_fd);

[Severity: High]
Could overwriting dst_prog here leak the BPF program reference? If the caller
populated both attr->attach_prog_fd and attr->sdt.target_prog_fd, dst_prog
would already hold a reference that seems to get lost on this assignment.

> +		if (IS_ERR(dst_prog)) {
> +			err = PTR_ERR(dst_prog);
> +			dst_prog = NULL;
> +			goto put_token;
> +		}
[ ... ]
>  	prog->expected_attach_type = attr->expected_attach_type;
>  	prog->sleepable = !!(attr->prog_flags & BPF_F_SLEEPABLE);
>  	prog->aux->attach_btf = attach_btf;
> -	prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id;
> +	if (prog_sdt_probe)
> +		prog->aux->attach_btf_id = attach_btf_id;
> +	else
> +		prog->aux->attach_btf_id =
> +			multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id;
>  	prog->aux->dst_prog = dst_prog;
> +	prog->aux->sdt_probe = prog_sdt_probe;
> +	/* ownership of dst_prog/attach_btf/prog_sdt_probe moved to prog->aux */
> +	dst_prog = NULL;
> +	attach_btf = NULL;
> +	prog_sdt_probe = NULL;

[Severity: High]
Will early load failures after this point leak prog->aux->sdt_probe?

On subsequent errors, the code jumps to free_prog which calls bpf_prog_free().
Since bpf_prog_free() and its deferred cleanup routines don't seem to free
sdt_probe (unlike __bpf_prog_put_noref() which is bypassed on early failures),
does this leave the probe info allocated?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=9

Re: [RFC PATCH bpf-next 11/12] bpf, x86: Add JIT support SDT for probe

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [Critical] NULL pointer dereference in the x86 JIT compiler for classic BPF (cBPF) programs
- [High] Stack buffer overflow in `bpf_sdt_link_attach()` due to unvalidated `nargs` field
- [High] Out-of-bounds read in the Verifier and JIT compiler via unvalidated SDT argument registers
- [High] The SDT probe trampoline clobbers x86 caller-saved registers, corrupting the execution state of the target BPF program
- [High] Verifier bypass of SDT probe constraints via duplicate `orig_off` map entries
--

commit e3ca8ab99711f4294e8b0182251c50473a63e6d7
Author: Xu Kuohai <xukuohai@huawei.com>
Subject: bpf, x86: Add JIT support SDT for probe

This commit adds x86 JIT support for SDT probes. It jits SDT probe sites to
a 5-byte NOP and patches them to a CALL instruction when attached to an
observer program. It also modifies the BPF trampoline to save and restore
arguments from the func_model's arg_regs instead of fixed BPF registers.

> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> index 054e043ffcd22..262f1bd74d53f 100644
> --- a/arch/x86/net/bpf_jit_comp.c
> +++ b/arch/x86/net/bpf_jit_comp.c
> @@ -2797,6 +2797,16 @@ st:			insn_off = insn->off;
>  			}
>  
>  			if (!jmp_offset) {
> +				int index = bpf_prog->aux->subprog_start + i - 1;
> +
> +				/*
> +				 * Emit a 5-byte NOP for SDT probe site, so it can be patched to
> +				 * a CALL instruction when observer program is attached.
> +				 */
> +				if (env->insn_aux_data[index].sdt_entry) {

[Severity: Critical]
Does this code dereference a NULL pointer when JIT compiling classic BPF
programs?

When an unprivileged user attaches a classic BPF filter containing a jump to
the next instruction, the kernel translates it to eBPF and invokes
bpf_int_jit_compile() with a NULL verifier environment.

During compilation, do_jit() processes the jump with a zero offset and
evaluates env->insn_aux_data[index].sdt_entry. Since env is NULL, could this
lead to an immediate kernel panic?

> +					emit_nops(&prog, X86_PATCH_SIZE);
> +					break;
> +				}
>  				/*
>  				 * If jmp_padding is enabled, the extra nops will
>  				 * be inserted. Otherwise, optimize out nop jumps.

[ ... ]

> @@ -3087,9 +3101,12 @@ static void restore_regs(const struct btf_func_model *m, u8 **prog,
>  		arg_regs = (m->arg_size[i] + 7) / 8;
>  		if (nr_regs + arg_regs <= 6) {
>  			for (j = 0; j < arg_regs; j++) {
> -				emit_ldx(prog, BPF_DW,
> -					 nr_regs == 5 ? X86_REG_R9 : BPF_REG_1 + nr_regs,
> -					 BPF_REG_FP,
> +				u8 reg = (flags & BPF_TRAMP_F_SDT_PROBE)
> +					? m->arg_regs[nr_regs]
> +					: (nr_regs == 5 ? X86_REG_R9
> +					   : BPF_REG_1 + nr_regs);
> +
> +				emit_ldx(prog, BPF_DW, reg, BPF_REG_FP,

[Severity: High]
Can this lead to an out-of-bounds read if the register index is unvalidated?

If a user provides an out-of-bounds value for arg_reg in an insn_array map
entry, and the verifier is bypassed, could m->arg_regs[nr_regs] hold a value
exceeding the expected register range?

When this unvalidated index is passed to emit_ldx(), it appears to cause an
out-of-bounds read on the reg2hex array during x86 opcode generation,
potentially corrupting memory.

Is there a similar issue in check_sdt_probe() during verification?

kernel/bpf/verifier.c:check_sdt_probe() {
    ...
    u8 reg = val->arg_reg[i];
    struct bpf_reg_state *rs = &regs[reg];
    ...
}

If reg is unvalidated, could this also cause an out-of-bounds read on the
verifier's regs array?

>  					 -stack_size);
>  				stack_size -= 8;
>  				nr_regs++;

[ ... ]

> @@ -3603,7 +3620,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
>  	}
>  
>  	if (flags & BPF_TRAMP_F_RESTORE_REGS)
> -		restore_regs(m, &prog, regs_off);
> +		restore_regs(m, &prog, regs_off, flags);

[Severity: High]
Does the SDT probe trampoline clobber x86 caller-saved registers here?

The verifier treats the 5-byte NOP as a jump with a zero offset, assuming all
BPF caller-saved registers remain intact. However, when an observer attaches,
the NOP is patched to call the BPF trampoline, which executes the observer
program.

Could the observer program clobber the x86 caller-saved registers (RAX, RCX,
RDX, RDI, RSI, R8)? Since restore_regs() only restores the explicitly tracked
SDT argument registers, the remaining caller-saved registers might remain
corrupted when control returns to the target program, potentially corrupting
its execution state.

>  
>  	/* This needs to be done regardless. If there were fmod_ret programs,
>  	 * the return value is only updated on the stack and still needs to be

[ ... ]

[Severity: High]
Is it possible to bypass the verifier's SDT probe constraints using duplicate
offsets in the insn_array map?

In bpf_insn_array_init(), multiple map entries with the same offset appear to
overwrite each other sequentially:

kernel/bpf/bpf_insn_array.c:bpf_insn_array_init() {
    ...
    env->insn_aux_data[values[i].orig_off].sdt_entry = &values[i];
    ...
}

This means the verifier might only validate the last matching element.
However, bpf_insn_array_get_sdt_probe_by_name() seems to return the first
matching element by scanning sequentially:

kernel/bpf/bpf_insn_array.c:bpf_insn_array_get_sdt_probe_by_name() {
    ...
    if (!strcmp(insn_array->values[j].name, name)) {
        *val = insn_array->values[j];
    ...
}

Could this allow a malicious user to place a malformed, unverified probe
entry before a valid one with the same offset, completely subverting the
verifier?

[Severity: High]
If the verifier is bypassed, could bpf_sdt_link_attach() suffer from a stack
buffer overflow?

When looking up a probe by name, bpf_sdt_link_attach() might retrieve an
unverified entry with an arbitrary nargs value (up to 255):

kernel/bpf/syscall.c:bpf_sdt_link_attach() {
    ...
    for (i = 0; i < sdt->val.nargs; i++) {
        tgt_info.fmodel.arg_size[i] = 8;
        tgt_info.fmodel.arg_regs[i] = sdt->val.arg_reg[i];
    }
    ...
}

Because tgt_info.fmodel.arg_size and arg_regs are statically sized arrays
on the kernel stack, could this loop write out-of-bounds if nargs exceeds
the arrays' bounds?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=11

Re: [RFC PATCH bpf-next 04/12] bpf: Create insn_array map for bpf SDT probe

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] Size of struct bpf_insn_array_value changes, breaking UAPI backwards compatibility for BPF_MAP_TYPE_INSN_ARRAY map creation.
- [High] SDT probes in shared subprograms are dropped for all but the first main program that absorbs the subprogram.
- [High] Incorrect instruction offset calculation for SDT probes located in subprograms, leading to offset corruption and verifier rejection.
--

commit 4c79777e0549d710e07ecb99bf27d3c4bb18d256
Author: Xu Kuohai <xukuohai@huawei.com>

bpf: Create insn_array map for bpf SDT probe

Create one BPF_MAP_TYPE_INSN_ARRAY map per program from the parsed
SDT entries. Each map is tagged with BPF_F_INSN_ARRAY_SDT and
populated with the instruction offset, argument count, and per-argument
registers.

> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
[ ... ]
> @@ -7732,7 +7739,9 @@ struct bpf_insn_array_value {
>  	__u32 orig_off;
>  	__u32 xlated_off;
>  	__u32 jitted_off;
> -	__u32 :32;
> +	__u8  nargs;		/* argument count (0..5) */
> +	__u8  arg_reg[5];	/* BPF register for each argument */
> +	__u8  pad[2];
>  };

[Severity: High]
Does this struct size change break UAPI backwards compatibility?

Because insn_array_alloc_check() unconditionally enforces a strict size check:

kernel/bpf/bpf_insn_array.c:insn_array_alloc_check() {
    u32 value_size = sizeof(struct bpf_insn_array_value);
    
    if (attr->max_entries == 0 || attr->key_size != 4 ||
        attr->value_size != value_size ||
        attr->map_flags & ~BPF_F_INSN_ARRAY_SDT)
        return -EINVAL;
}

Older userspace binaries calling bpf_map_create() with the old value_size
will be rejected with -EINVAL.

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c
[ ... ]
> @@ -6686,6 +6692,131 @@ static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struc
>  	return err;
>  }
>  
> +static int bpf_object__resolve_sdt_progs(struct bpf_object *obj)
> +{
> +	int i, j;
> +	struct bpf_light_subprog *sp;
> +
> +	for (i = 0; i < obj->sdt_entry_cnt; i++) {
> +		struct sdt_entry *e = &obj->sdt_entries[i];
> +		struct bpf_program *prog, *subprog;
> +
> +		prog = find_prog_by_sec_insn(obj, e->sec_idx, e->insn_idx);
[ ... ]
> +		/* resolve subprog probe in the main prog that absorbed the subprog */
> +		if (prog_is_subprog(obj, prog) && prog->absorbed_by != -1) {
> +			subprog = prog;
> +			prog = &obj->programs[prog->absorbed_by];

[Severity: High]
Does this logic drop SDT probes in shared subprograms for all but the first
main program that absorbed it?

By resolving the shared SDT entry exclusively to the main program indicated
by prog->absorbed_by, any tracing tools relying on SDT probes in this
subprogram will silently miss executions when the subprogram is executed
through alternate main programs.

> +			for (j = 0; j < prog->subprog_cnt; j++) {
> +				sp = &prog->subprogs[j];
> +				if (sp->sec_insn_off == subprog->sec_insn_off) {
> +					e->insn_idx = sp->sub_insn_off +
> +						       (e->insn_idx - sp->sec_insn_off);
> +					break;
> +				}
> +			}
[ ... ]
> +static int bpf_object__create_sdt_maps(struct bpf_object *obj)
> +{
[ ... ]
> +	/* create insn_array maps per program and populate entries */
> +	for (i = 0; i < obj->sdt_entry_cnt; i++) {
> +		__u32 key;
> +		struct sdt_entry *e = &obj->sdt_entries[i];
> +		__u32 sdt_cnt = prog_sdt[e->prog_idx].sdt_cnt;
> +
> +		if (!sdt_cnt)
> +			continue;
> +
> +		prog = &obj->programs[e->prog_idx];
[ ... ]
> +		key = prog_sdt[e->prog_idx].next_key++;
> +
> +		memset(&val, 0, sizeof(val));
> +		val.nargs = e->nargs;
> +		val.orig_off = e->insn_idx - prog->sec_insn_off;

[Severity: High]
Can this offset calculation lead to verifier rejection later?

Earlier in bpf_object__resolve_sdt_progs(), e->insn_idx is updated to a 0-based
local offset relative to the main program.

Here in bpf_object__create_sdt_maps(), subtracting prog->sec_insn_off again
from an already localized offset will cause an integer underflow or offset
corruption if the main program's section offset is greater than 0. This will
cause the verifier to reject the BPF object.

> +		memcpy(val.arg_reg, e->arg_reg, sizeof(val.arg_reg));
> +
> +		err = bpf_map_update_elem(prog->sdt_map_fd, &key, &val, 0);
> +		if (err)
> +			goto out_free;
> +	}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=4

Re: [RFC PATCH bpf-next 05/12] bpf: Collect SDT probe BTF IDs from BTF decl tags

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [Critical] Missing bounds check for `val.nargs` during `BPF_MAP_TYPE_INSN_ARRAY` map update allows a massive out-of-bounds stack write (buffer overflow).
- [High] Modifying the size of `struct bpf_insn_array_value` breaks ABI compatibility.
- [Medium] `sdt_collect_btf_ids()` unconditionally dereferences `obj->btf`, causing a segmentation fault when an ELF file lacks BTF info.
- [Medium] The `<nargs>` parameter in the `bpf_sdt:<name>:<nargs>` BTF tag is ignored during parsing, potentially causing incorrect BTF ID assignments.
--

commit f74606f5fc3911fea2fa7d9d52875deb5fbf475d
Author: Xu Kuohai <xukuohai@huawei.com>

bpf: Collect SDT probe BTF IDs from BTF decl tags

Collect SDT probe BTF IDs from BTF decl tags matching name pattern
"bpf_sdt:<name>:<nargs>". The IDs are ultimately stored in each probe
entry of the SDT map.

> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -7739,6 +7739,7 @@ struct bpf_insn_array_value {
>  	__u32 orig_off;
>  	__u32 xlated_off;
>  	__u32 jitted_off;
> +	__u32 btf_id;		/* BTF FUNC_PROTO type ID (0 if none) */

[Severity: High]
Does changing the size of struct bpf_insn_array_value break ABI backward
compatibility?

When libbpf attempts to load a BPF program on an older kernel, create_jt_map()
will use the new sizeof(struct bpf_insn_array_value) for the .jumptables map
value size.

Will older kernels reject this map creation since they expect the value size
to be exactly the old size?

>  	__u8  nargs;		/* argument count (0..5) */
>  	__u8  arg_reg[5];	/* BPF register for each argument */
>  	__u8  pad[2];

[ ... ]

> diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
> --- a/kernel/bpf/bpf_insn_array.c
> +++ b/kernel/bpf/bpf_insn_array.c
> @@ -103,6 +103,7 @@ static long insn_array_update_elem(struct bpf_map *map, void *key, void *value,
>  	insn_array->values[index].orig_off = val.orig_off;
>  
>  	if (insn_array->subtype == BPF_INSN_ARRAY_SUBTYPE_SDT) {
> +		insn_array->values[index].btf_id = val.btf_id;
>  		insn_array->values[index].nargs = val.nargs;

[Severity: Critical]
Is there a bounds check missing for val.nargs before it is copied here?

If a malicious user provides a large value for val.nargs during map update,
could this cause an out-of-bounds write later in the attach path?

For example, in bpf_sdt_link_attach(), the loop iterates based on this value:

bpf_sdt_link_attach() {
    ...
    tgt_info.fmodel.nr_args = sdt->val.nargs;
    for (i = 0; i < sdt->val.nargs; i++) {
        tgt_info.fmodel.arg_size[i] = 8;
        tgt_info.fmodel.arg_regs[i] = sdt->val.arg_reg[i];
    }
    ...
}

Since the target arrays have a fixed capacity, would a large nargs value
overflow the kernel stack buffer?

>  		memcpy(insn_array->values[index].arg_reg, val.arg_reg, sizeof(val.arg_reg));
>  	}

[ ... ]

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c

[ ... ]

> @@ -6692,6 +6693,57 @@ static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struc
>  	return err;
>  }
>  
> +static void sdt_collect_btf_ids(struct bpf_object *obj)
> +{
> +	int n = btf__type_cnt(obj->btf);

[Severity: Medium]
Can obj->btf be NULL here if the ELF object was compiled without BTF
information?

If so, would calling btf__type_cnt() on a NULL pointer cause a segmentation
fault during bpf_object__open() when parsing SDT notes?

> +	const struct btf_type *t;
> +	int i;

[ ... ]

> +		/* match by name (tag format: "bpf_sdt:<name>:<nargs>") */
> +		for (j = 0; j < obj->sdt_entry_cnt; j++) {
> +			struct sdt_entry *e = &obj->sdt_entries[j];
> +
> +			if (!e->btf_id && strlen(e->name) == name_len &&
> +			    strncmp(tag + 8, e->name, name_len) == 0)
> +				e->btf_id = func_proto_id;

[Severity: Medium]
Does this string comparison logic correctly handle overloaded SDT probes?

The code parses the tag format "bpf_sdt:<name>:<nargs>" and compares the
probe name, but it appears to ignore the nargs substring when matching
the tag.

If multiple probes share the same name but have different argument counts,
could this assign the incorrect BTF ID to a probe, causing the kernel to fail
the program load when verifying the argument count?

> +		}
> +	}
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=5

Re: [RFC PATCH bpf-next 11/12] bpf, x86: Add JIT support SDT for probe

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2544 bytes --]

> diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
> index 054e043ffcd2..262f1bd74d53 100644
> --- a/arch/x86/net/bpf_jit_comp.c
> +++ b/arch/x86/net/bpf_jit_comp.c
> @@ -2797,6 +2797,16 @@ st:			insn_off = insn->off;
>  		}
>
>  		if (!jmp_offset) {
> +			int index = bpf_prog->aux->subprog_start + i - 1;
> +
> +			/*
> +			 * Emit a 5-byte NOP for SDT probe site, so it can be patched to
> +			 * a CALL instruction when observer program is attached.
> +			 */
> +			if (env->insn_aux_data[index].sdt_entry) {
> +				emit_nops(&prog, X86_PATCH_SIZE);
> +				break;
> +			}
>  			/*
>  			 * If jmp_padding is enabled, the extra nops will
>  			 * be inserted. Otherwise, optimize out nop jumps.

Can this dereference env when it's NULL?  The do_jit() function can be
invoked with env == NULL. The call path from bpf_prog_select_runtime() in
kernel/bpf/core.c goes through __bpf_prog_select_runtime(NULL, fp, err)
to bpf_prog_jit_compile(NULL, prog) and ultimately to
bpf_int_jit_compile(NULL, prog) -> do_jit(NULL, prog, ...) in
arch/x86/net/bpf_jit_comp.c.

This path is used by classic-BPF socket filters (net/core/filter.c in
bpf_prog_create), seccomp, and other internally-built programs that are
JITed without running the verifier. The code at line 2806 sits inside
the generic BPF_JMP|BPF_JA / BPF_JMP32|BPF_JA handling, reached whenever
a jump resolves to zero displacement (jmp_offset == 0):

net/core/filter.c:bpf_prog_create() {
    ...
    fp = bpf_prog_alloc(bpf_prog_size(new_len), 0);
    ...
    old_prog = bpf_migrate_filter(old_prog);
    ...
    *pfp = bpf_prog_select_runtime(fp, &err);
    ...
}

kernel/bpf/core.c:bpf_prog_select_runtime() {
    struct bpf_prog *err;
    err = __bpf_prog_select_runtime(NULL, fp, err);
    ...
}

cBPF->eBPF conversion emits BPF_JMP|BPF_JA instructions (net/core/filter.c),
so a program reaching do_jit with env==NULL can hit this branch and crash
the kernel.

The same function guards other env accesses against NULL: bpf_out_stack_arg_cnt(env, ...)
at line 1703 returns 0 if !env, and bpf_insn_is_indirect_target(env, ...) at line 1830
returns false if !env (kernel/bpf/core.c).

Also, when i==0 and subprog_start==0 for the main program, index becomes -1,
which is an out-of-bounds access into insn_aux_data[].

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28292842584

Re: [RFC PATCH bpf-next 06/12] bpf: Add type check for SDT probe site

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

>     bpf: Add type check for SDT probe site
>
>     The NOP instruction at SDT probe site will be patched to a call
>     instruction to observer programs. To ensure the arguments passed
>     to observers are as expected, add type check for bpf SDT probe site.
>
>     The check for probes with no arguments always succeeds.
>
>     For probes with arguments, argument registers are checked against
>     the types declared by FUNC_PROTO for the probe site.
>
>     Signed-off-by: Xu Kuohai <xukuohai@huawei.com>

This looks like a bug fix for SDT probe support. The commit adds type
checking and protection for SDT probe sites to prevent them from being
incorrectly optimized away by bpf_opt_remove_nops() and to ensure their
argument registers remain live during liveness analysis. These issues
were introduced when SDT probe support was added. Should this include:

  Fixes: b96ff657266c ("bpf: Create insn_array map for bpf SDT probe")


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28292842584

Re: [RFC PATCH bpf-next 09/12] bpf: Add kernel support to load SDT observer program

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4948 bytes --]

> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index dc881e5ad411..8b2c73bb6c2a 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -2454,6 +2454,7 @@ static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)
>  	kvfree(prog->aux->linfo);
>  	kfree(prog->aux->kfunc_tab);
>  	kfree(prog->aux->ctx_arg_info);
> +	kfree(prog->aux->sdt_probe);
>  	if (prog->aux->attach_btf)
>  		btf_put(prog->aux->attach_btf);

[ ... ]

> @@ -3082,33 +3085,75 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_at
>  		btf_get(attach_btf);
>  	}
>
> +	/*
> +	 * find the probe site in the target program at load time of the
> +	 * observer program, so the verifier can check the observer's context
> +	 * arguments from the probe's FUNC_PROTO.
> +	 */
> +	if (attr->expected_attach_type == BPF_TRACE_SDT) {
> +		struct bpf_insn_array_value sdt_val;
> +		struct bpf_sdt_probe_info *sdt_probe;
> +		unsigned long probe_ip;
> +
> +		if (!bpf_jit_supports_sdt_probe()) {
> +			err = -EOPNOTSUPP;
> +			goto put_token;
> +		}
> +		if (!attr->sdt.target_prog_fd) {
> +			err = -EINVAL;
> +			goto put_token;
> +		}
> +		dst_prog = bpf_prog_get(attr->sdt.target_prog_fd);
> +		if (IS_ERR(dst_prog)) {
> +			err = PTR_ERR(dst_prog);
> +			dst_prog = NULL;
> +			goto put_token;
> +		}
> +		err = bpf_insn_array_get_sdt_probe_by_name(dst_prog, attr->sdt.name,
> +							   &sdt_val, &probe_ip);
> +		if (err)
> +			goto put_sdt;
> +		sdt_probe = kzalloc_obj(struct bpf_sdt_probe_info, GFP_USER);
> +		if (!sdt_probe) {
> +			err = -ENOMEM;
> +			goto put_sdt;
> +		}
> +		sdt_probe->val = sdt_val;
> +		sdt_probe->probe_ip = probe_ip;
> +		prog_sdt_probe = sdt_probe;
> +		attach_btf_id = sdt_val.btf_id;
> +	}
> +
> +
>  	if (bpf_prog_load_check_attach(type, attr->expected_attach_type,
> -				       attach_btf, attr->attach_btf_id,
> +				       attach_btf,
> +				       prog_sdt_probe ? attach_btf_id : attr->attach_btf_id,
>  				       dst_prog, multi_func)) {
> -		if (dst_prog)
> -			bpf_prog_put(dst_prog);
> -		if (attach_btf)
> -			btf_put(attach_btf);
>  		err = -EINVAL;
> -		goto put_token;
> +		goto put_sdt;
>  	}
>
>  	/* plain bpf_prog allocation */
>  	prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER);
>  	if (!prog) {
> -		if (dst_prog)
> -			bpf_prog_put(dst_prog);
> -		if (attach_btf)
> -			btf_put(attach_btf);
>  		err = -EINVAL;
> -		goto put_token;
> +		goto put_sdt;
>  	}
>
>  	prog->expected_attach_type = attr->expected_attach_type;
>  	prog->sleepable = !!(attr->prog_flags & BPF_F_SLEEPABLE);
>  	prog->aux->attach_btf = attach_btf;
> -	prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id;
> +	if (prog_sdt_probe)
> +		prog->aux->attach_btf_id = attach_btf_id;
> +	else
> +		prog->aux->attach_btf_id =
> +			multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id;
>  	prog->aux->dst_prog = dst_prog;
> +	prog->aux->sdt_probe = prog_sdt_probe;
> +	/* ownership of dst_prog/attach_btf/prog_sdt_probe moved to prog->aux */
> +	dst_prog = NULL;
> +	attach_btf = NULL;
> +	prog_sdt_probe = NULL;

After this ownership transfer, prog_sdt_probe is NULL. Several error paths
below this point (copy_from_bpfptr failures, license checks, signature
verification, dev_bound initialization, find_prog_type, bpf_obj_name_cpy,
security_bpf_prog_load) all jump to the free_prog label.

Does the free_prog path correctly free prog->aux->sdt_probe?

Looking at the error paths:

kernel/bpf/syscall.c:bpf_prog_load() {
    err = copy_from_bpfptr(...);
    if (err)
        goto free_prog;
    ...
    err = bpf_prog_verify_signature(prog);
    if (err < 0)
        goto free_prog;
    ...
    err = bpf_prog_dev_bound_init(prog, attr);
    if (err)
        goto free_prog;
    ...
}

> @@ -3242,6 +3287,12 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_at
>  	if (prog->aux->attach_btf)
>  		btf_put(prog->aux->attach_btf);
>  	bpf_prog_free(prog);

The free_prog label calls bpf_prog_free(), which in turn calls
bpf_prog_free_deferred(). That function frees used_maps, used_btfs,
kfunc tables, and the JIT image, but not aux->sdt_probe. The only place
that frees aux->sdt_probe is __bpf_prog_put_noref(), which is reached
through the normal bpf_prog_put() teardown (the free_used_maps label),
not through this error path.

> +put_sdt:
> +	kfree(prog_sdt_probe);

This kfree handles the local prog_sdt_probe, which is NULL after the
ownership transfer above. It won't free the allocation now owned by
prog->aux->sdt_probe.

Can prog->aux->sdt_probe leak when a BPF_TRACE_SDT load succeeds past
the probe resolution but fails before bpf_check() completes?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28292842584

Re: [RFC PATCH bpf-next 12/12] selftests/bpf: Add tests for bpf SDT probe

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Incorrect relative include path for `bpf_sdt.h` causes compilation failure.
--

commit e66dde4c8db25d960e4cc2ec8a78a2fbb3113bee
Author: Xu Kuohai <xukuohai@huawei.com>

selftests/bpf: Add tests for bpf SDT probe

This commit adds selftests for BPF SDT probe sites and observers. It
includes tests for argument checking of the observer program, multiple
target programs in the same ELF, and programs with subprograms.

> diff --git a/tools/testing/selftests/bpf/progs/bpf_sdt_target.c b/tools/testing/selftests/bpf/progs/bpf_sdt_target.c
> new file mode 100644
> index 0000000000000..e1c14b9d7e9a2
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/bpf_sdt_target.c
> @@ -0,0 +1,34 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include "vmlinux.h"
> +#include <bpf/bpf_helpers.h>
> +#include <bpf/bpf_tracing.h>
> +#include "../../../../tools/lib/bpf/bpf_sdt.h"

[Severity: Low]
Will this relative include path cause a compilation failure?

Starting from tools/testing/selftests/bpf/progs/, going up four directories
with ../../../../ lands in the tools/ directory. Adding tools/lib/bpf/bpf_sdt.h
makes the resolved path tools/tools/lib/bpf/bpf_sdt.h, which doesn't exist.

Should this be ../../../../lib/bpf/bpf_sdt.h or perhaps just <bpf/bpf_sdt.h>?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=12

Re: [RFC PATCH bpf-next 04/12] bpf: Create insn_array map for bpf SDT probe

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2512 bytes --]

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 327c9d412fe8..3f8b12a1eb8b 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c

[ ... ]

> @@ -6686,6 +6692,131 @@ static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struc
>  	return err;
>  }
>
> +static int bpf_object__resolve_sdt_progs(struct bpf_object *obj)
> +{
> +	int i, j;
> +	struct bpf_light_subprog *sp;
> +
> +	for (i = 0; i < obj->sdt_entry_cnt; i++) {
> +		struct sdt_entry *e = &obj->sdt_entries[i];
> +		struct bpf_program *prog, *subprog;
> +
> +		prog = find_prog_by_sec_insn(obj, e->sec_idx, e->insn_idx);
> +		if (!prog) {
> +			pr_warn("sdt: probe '%s' at sec %u insn %llu not found in any program\n",
> +				e->name, (unsigned)e->sec_idx, (unsigned long long)e->insn_idx);
> +			return -EINVAL;
> +		}
> +
> +		/* resolve subprog probe in the main prog that absorbed the subprog */
> +		if (prog_is_subprog(obj, prog) && prog->absorbed_by != -1) {
> +			subprog = prog;
> +			prog = &obj->programs[prog->absorbed_by];

Can a subprog that holds an SDT probe be absorbed into more than one main
program?

bpf_object__relocate_calls() resets every subprog's sub_insn_off to 0
before relocating each main program, and bpf_object__reloc_code() then
appends a fresh copy of the subprog's code into every main program that
calls it.  In that case the same probe instruction ends up in several main
programs.

Here resolve only attributes the probe to a single program via
prog->absorbed_by, and create_sdt_maps below populates only that one
program's map.  Would the other main programs that also contain a copy of
the probe instruction end up with no entry in their SDT map, so the probe
is silently missing there?

[ ... ]

> @@ -7135,6 +7266,8 @@ bpf_object__reloc_code(struct bpf_object *obj, struct bpf_program *main_prog,
>  			err = bpf_object__append_subprog_code(obj, main_prog, subprog);
>  			if (err)
>  				return err;
> +			if (subprog->absorbed_by == -1)
> +				subprog->absorbed_by = main_prog - obj->programs;

Since absorbed_by is only set when it is still -1, it records the first
main program that absorbs the subprog.  Is that the reason later main
programs that also append a copy of the subprog do not get an SDT entry?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28292842584

Re: [RFC PATCH bpf-next 05/12] bpf: Collect SDT probe BTF IDs from BTF decl tags

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1665 bytes --]

> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
> index 3f8b12a1eb8b..badac128e8ad 100644
> --- a/tools/lib/bpf/libbpf.c
> +++ b/tools/lib/bpf/libbpf.c

[ ... ]

> @@ -6692,6 +6693,57 @@ static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struc
>  	return err;
>  }
>
> +static void sdt_collect_btf_ids(struct bpf_object *obj)
> +{
> +	int n = btf__type_cnt(obj->btf);
                                ^^^^^^^^

Can sdt_collect_btf_ids() handle an object with no BTF?

btf__type_cnt(obj->btf) dereferences obj->btf. obj->btf is NULL when
the object carries no .BTF section - stripped or built without BTF.

SDT entries come from the .bpf_sdt_notes section, which is independent of
BTF presence. An object with probe sites but stripped BTF would have
sdt_entry_cnt > 0 and obj->btf == NULL, triggering a NULL dereference
here.

Existing libbpf functions guard this explicitly:

tools/lib/bpf/libbpf.c:bpf_object_fixup_btf() {
    if (!obj->btf)
        return 0;
    n = btf__type_cnt(obj->btf);
    ...
}

Should sdt_collect_btf_ids() return early if obj->btf is NULL, leaving
btf_id as 0 (the intended value per the bpf_insn_array_value comment)?

> +	const struct btf_type *t;
> +	int i;
> +
> +	for (i = 1; i < n; i++) {

[ ... ]

> @@ -6749,6 +6801,8 @@ static int bpf_object__create_sdt_maps(struct bpf_object *obj)
>  	if (err)
>  		return err;
>
> +	sdt_collect_btf_ids(obj);
> +


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28292842584

Re: [RFC PATCH bpf-next 01/12] libbpf: Prepare bpf SDT probe section for the linker

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:

New issues:
- [High] Cross-endian linking corrupts target-endian addresses in `.bpf_sdt_notes`.
- [High] Unaligned memory access via direct pointer cast of `.bpf_sdt_notes` relocations causes undefined behavior.
- [Medium] Blind assumption of 64-bit relocation type for `.bpf_sdt_notes` can cause memory corruption.
- [Medium] Redundant symbol validation bypass allows malicious global symbols in `.bpf_sdt_notes`.

Pre-existing issues:
- [High] Missing bounds check for ELF relocation offsets allows out-of-bounds heap memory access.
--

commit c10bd6264b90c8115e40b5bcf69e608dd418c585
Author: Xu Kuohai <xukuohai@huawei.com>

libbpf: Prepare bpf SDT probe section for the linker

This patch prepares the BPF linker to handle the .bpf_sdt_notes section,
treating it similarly to jumptables. It bypasses section-type checks for
SDT symbols and relocates 64-bit absolute addresses inside each entry.

> diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> index 04cd303fb5a87..ed7a587a61ca4 100644
> --- a/tools/lib/bpf/libbpf_internal.h
> +++ b/tools/lib/bpf/libbpf_internal.h
[ ... ]
> diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c
> index 78f92c39290af..3a72276244c72 100644
> --- a/tools/lib/bpf/linker.c
> +++ b/tools/lib/bpf/linker.c
> @@ -2028,6 +2028,8 @@ static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
>  
>  		if (strcmp(src_sec->sec_name, JUMPTABLES_SEC) == 0)
>  			goto add_sym;
> +		if (strcmp(src_sec->sec_name, SDT_NOTES_SEC) == 0)
> +			goto add_sym;

[Severity: Medium]
Could this bypass of global symbol resolution cause issues? If a symbol in
the SDT notes section has STB_GLOBAL binding, it will hit this branch and
skip the deduplication and resolution logic entirely. 

For STB_LOCAL symbols, this check appears to be redundant because of the
subsequent check for STB_LOCAL binding.

>  	}
>  
>  	if (sym_bind == STB_LOCAL)
[ ... ]
> @@ -2274,6 +2276,13 @@ static int linker_append_elf_relos(struct bpf_linker *linker, struct src_obj *ob
>  						insn->imm += sec->dst_off / sizeof(struct bpf_insn);
>  					else
>  						insn->imm += sec->dst_off;
> +				} else if (strcmp(src_linked_sec->sec_name, SDT_NOTES_SEC) == 0) {
> +					/* .bpf_sdt_notes contains R_BPF_64_ABS64 relocations
> +					 * referencing code section symbols. Adjust the 64-bit
> +					 * address by dst_off.
> +					 */
> +					__u64 *addr = dst_linked_sec->raw_data + dst_rel->r_offset;

[Severity: High]
This is a pre-existing issue, but does this lack a bounds check for the
relocation offset? The code uses the r_offset from the relocation to index
into raw_data. If a malformed ELF provides an r_offset that exceeds the
section size, it could result in an out-of-bounds heap memory access.

[Severity: High]
Can this direct pointer cast result in unaligned memory access? The r_offset
value might not be naturally aligned, which could lead to undefined
behavior or crashes on strict-alignment architectures when dereferencing the
cast pointer.

> +					*addr += sec->dst_off;

[Severity: High]
Does this perform host-endian arithmetic on target-endian data without
conversion? The data section is not byte-swapped to native endianness, so
modifying the address using native host-endian addition could corrupt the
relocated address during cross-compiling.

[Severity: Medium]
Is it safe to blindly assume the relocation type is 64-bit? If an ELF object
contains a 32-bit relocation type targeting this section, unconditionally
reading and writing 64 bits without verifying the relocation type from
the relocation info could corrupt adjacent data.

>  				} else {
>  					pr_warn("relocation against STT_SECTION in non-exec section is not supported!\n");
>  					return -EINVAL;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=1

Re: [RFC PATCH bpf-next 10/12] bpf: Support attach and detach for SDT observer program

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Critical] Trampoline key space collision between SDT probe offsets and BTF Type IDs leading to verifier bypass and arbitrary memory access.
- [Low] Multi-line comment opening is not on its own line, violating BPF subsystem guidelines.
--

commit b1194a14a242f8613863b3524830be07aca1a7e7
Author: Xu Kuohai <xukuohai@huawei.com>

bpf: Support attach and detach for SDT observer program

This commit adds support to attach an SDT observer program by reusing the
bpf trampoline attach and detach mechanism. The trampoline is keyed by
the target program fd and the offset of the probe. The function prototype
is resolved from prog->aux->sdt_probe.

> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index 93c196f38b878..6548a2dd76da6 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -1270,6 +1270,12 @@ struct btf_func_model {
>   */
>  #define BPF_TRAMP_F_INDIRECT		BIT(8)
>  
> +/* Trampoline is for an SDT probe site: arguments arrive in
> + * BPF registers recorded by the SDT macro at build time,
> + * described by btf_func_model.arg_regs[] instead of ABI order.
> + */

[Severity: Low]
This isn't a bug, but does this multi-line comment follow the preferred
BPF subsystem style?

The opening block comment normally has the text starting on the next line:

/*
 * Trampoline is for an SDT probe site: arguments arrive in
 * ...

[ ... ]

> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index 8b2c73bb6c2a8..53897ff1100fa 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c

[ ... ]

> +static int bpf_sdt_link_attach(struct bpf_prog *prog)
> +{

[ ... ]

> +	err = bpf_link_prime(&sdt_link->link.link, &primer);
> +	if (err) {
> +		kfree(sdt_link);
> +		return err;
> +	}
> +
> +	/* use orig_off to compute the key as orig_off is unique for each probe */
> +	tr_key = bpf_trampoline_compute_key(tgt_prog, NULL, sdt->val.orig_off);

[Severity: Critical]
Does using sdt->val.orig_off here create a risk of trampoline key collisions
with BTF Type IDs?

Looking at bpf_trampoline_compute_key():

include/linux/bpf_verifier.h:bpf_trampoline_compute_key() {
    if (tgt_prog)
        return ((u64)tgt_prog->aux->id << 32) | btf_id;
    ...
}

Because the instruction offset is passed in place of the btf_id, an SDT probe
at an offset could generate the exact same 64-bit key as a subprogram with a
BTF Type ID matching that offset.

If both an SDT observer program and an fentry program attach using this same
key, they would share the same trampoline.

Could this allow the verifier to treat arbitrary scalars from the SDT probe
context as trusted pointers if they are passed into the fentry program?

> +	tgt_info.tgt_addr = sdt->probe_ip;
> +	tgt_info.tgt_name = sdt->val.name;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1782571533.git.xukuohai@huawei.com?part=10

[syzbot ci] Re: bpf: Introduce static-defined tracing probe for BPF

[syzbot ci]

syzbot ci has tested the following series

[v1] bpf: Introduce static-defined tracing probe for BPF
https://lore.kernel.org/all/cover.1782571533.git.xukuohai@huawei.com
* [RFC PATCH bpf-next 01/12] libbpf: Prepare bpf SDT probe section for the linker
* [RFC PATCH bpf-next 02/12] libbpf: Introduce bpf SDT probe macros
* [RFC PATCH bpf-next 03/12] libbpf: Add bpf_sdt_notes section parser
* [RFC PATCH bpf-next 04/12] bpf: Create insn_array map for bpf SDT probe
* [RFC PATCH bpf-next 05/12] bpf: Collect SDT probe BTF IDs from BTF decl tags
* [RFC PATCH bpf-next 06/12] bpf: Add type check for SDT probe site
* [RFC PATCH bpf-next 07/12] bpf: Record probe name in SDT map
* [RFC PATCH bpf-next 08/12] libbpf: Add libbpf support to load SDT observer program
* [RFC PATCH bpf-next 09/12] bpf: Add kernel support to load SDT observer program
* [RFC PATCH bpf-next 10/12] bpf: Support attach and detach for SDT observer program
* [RFC PATCH bpf-next 11/12] bpf, x86: Add JIT support SDT for probe
* [RFC PATCH bpf-next 12/12] selftests/bpf: Add tests for bpf SDT probe

and found the following issue:
general protection fault in do_jit

Full report is available here:
https://ci.syzbot.org/series/86d21ab6-d0e1-4dd3-b7e1-af4571d27460

***

general protection fault in do_jit

tree:      bpf-next
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/bpf/bpf-next.git
base:      53435562a725962e4de0c29653223129ba11643a
arch:      amd64
compiler:  Debian clang version 22.1.6 (++20260514074242+fc4aad7b5db3-1~exp1~20260514074407.73), Debian LLD 22.1.6
config:    https://ci.syzbot.org/builds/549153a4-d4b8-46a4-8266-df26ad835e2f/config
syz repro: https://ci.syzbot.org/findings/2efaa7e4-eae0-45d0-b336-f0e311f8356e/syz_repro

Oops: general protection fault, probably for non-canonical address 0xdffffc000000010e: 0000 [#1] SMP KASAN PTI
KASAN: null-ptr-deref in range [0x0000000000000870-0x0000000000000877]
CPU: 1 UID: 0 PID: 5818 Comm: syz.2.19 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
RIP: 0010:do_jit+0x7c8c/0x12a90 arch/x86/net/bpf_jit_comp.c:2806
Code: 49 83 c4 44 4c 89 e0 48 c1 e8 03 0f b6 04 10 84 c0 0f 85 0a 84 00 00 45 03 34 24 48 8b bc 24 a0 01 00 00 48 89 f8 48 c1 e8 03 <80> 3c 10 00 74 17 e8 49 30 b0 00 48 8b bc 24 a0 01 00 00 48 ba 00
RSP: 0018:ffffc900038cf640 EFLAGS: 00010202
RAX: 000000000000010e RBX: 000000000000001c RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000000 RDI: 0000000000000870
RBP: ffffc900038cfa00 R08: ffff88816bb0d940 R09: 0000000000000096
R10: 00000000000000fb R11: 0000000000000000 R12: ffff8881749b8044
R13: ffff88816c445450 R14: 0000000000000003 R15: 0000000000000000
FS:  00007f06814986c0(0000) GS:ffff8882a9224000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0680472780 CR3: 000000001cfc8000 CR4: 00000000000006f0
Call Trace:
 <TASK>
 bpf_int_jit_compile+0x8af/0x1620 arch/x86/net/bpf_jit_comp.c:3946
 bpf_prog_jit_compile kernel/bpf/core.c:2571 [inline]
 __bpf_prog_select_runtime+0x4e2/0xb20 kernel/bpf/core.c:2640
 bpf_migrate_filter net/core/filter.c:1318 [inline]
 bpf_prepare_filter+0x10ef/0x1280 net/core/filter.c:1366
 sk_attach_filter+0x24/0x140 net/core/filter.c:1550
 tun_attach_filter+0x176/0x280 drivers/net/tun.c:2992
 __tun_chr_ioctl+0x15f1/0x1e10 drivers/net/tun.c:3344
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:597 [inline]
 __se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f068059ce59
Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 e8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f0681498028 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f0680815fa0 RCX: 00007f068059ce59
RDX: 0000200000000300 RSI: 00000000401054d5 RDI: 0000000000000003
RBP: 00007f0680632e6f R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f0680816038 R14: 00007f0680815fa0 R15: 00007fff056e7c98
 </TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:do_jit+0x7c8c/0x12a90 arch/x86/net/bpf_jit_comp.c:2806
Code: 49 83 c4 44 4c 89 e0 48 c1 e8 03 0f b6 04 10 84 c0 0f 85 0a 84 00 00 45 03 34 24 48 8b bc 24 a0 01 00 00 48 89 f8 48 c1 e8 03 <80> 3c 10 00 74 17 e8 49 30 b0 00 48 8b bc 24 a0 01 00 00 48 ba 00
RSP: 0018:ffffc900038cf640 EFLAGS: 00010202
RAX: 000000000000010e RBX: 000000000000001c RCX: 0000000000000000
RDX: dffffc0000000000 RSI: 0000000000000000 RDI: 0000000000000870
RBP: ffffc900038cfa00 R08: ffff88816bb0d940 R09: 0000000000000096
R10: 00000000000000fb R11: 0000000000000000 R12: ffff8881749b8044
R13: ffff88816c445450 R14: 0000000000000003 R15: 0000000000000000
FS:  00007f06814986c0(0000) GS:ffff8882a9224000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f06805ea540 CR3: 000000001cfc8000 CR4: 00000000000006f0
----------------
Code disassembly (best guess):
   0:	49 83 c4 44          	add    $0x44,%r12
   4:	4c 89 e0             	mov    %r12,%rax
   7:	48 c1 e8 03          	shr    $0x3,%rax
   b:	0f b6 04 10          	movzbl (%rax,%rdx,1),%eax
   f:	84 c0                	test   %al,%al
  11:	0f 85 0a 84 00 00    	jne    0x8421
  17:	45 03 34 24          	add    (%r12),%r14d
  1b:	48 8b bc 24 a0 01 00 	mov    0x1a0(%rsp),%rdi
  22:	00
  23:	48 89 f8             	mov    %rdi,%rax
  26:	48 c1 e8 03          	shr    $0x3,%rax
* 2a:	80 3c 10 00          	cmpb   $0x0,(%rax,%rdx,1) <-- trapping instruction
  2e:	74 17                	je     0x47
  30:	e8 49 30 b0 00       	call   0xb0307e
  35:	48 8b bc 24 a0 01 00 	mov    0x1a0(%rsp),%rdi
  3c:	00
  3d:	48                   	rex.W
  3e:	ba                   	.byte 0xba


***

If these findings have caused you to resend the series or submit a
separate fix, please add the following tag to your commit message:
  Tested-by: syzbot@syzkaller.appspotmail.com

---
This report is generated by a bot. It may contain errors.
syzbot ci engineers can be reached at syzkaller@googlegroups.com.

To test a patch for this bug, please reply with `#syz test`
(should be on a separate line).

The patch should be attached to the email.
Note: arguments like custom git repos and branches are not supported.

[RFC PATCH bpf-next 00/12] bpf: Introduce static-defined tracing probe for BPF

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

This series introduces static-defined tracing probes for BPF programs.
BPF SDT (static-defined tracing) works similarly to USDT. User defines
probes in the BPF source code. The probes are built into NOP instructions
in the ELF. At runtime, when an observer is attached, the NOP instruction
is patched to a CALL instruction to the observer prog trampoline.

Unlike USDT, BPF SDT requires explicit macros to generate the function
prototype BTF for each probe. This allows the verifier to validate the
probe sites against the declared types, and observer programs can be
attached similarly to normal tracing programs using the function prototype
information.

A probe with two arguments in the target program can be declared and defined
like:

    BPF_SDT_DECLARE2(my_trace, int, int);

    SEC("xdp")
    int xdp_prog(struct xdp_md *ctx)
    {
        int len = ctx->data_end - ctx->data;
        int ret = XDP_DROP;
        ...
        BPF_SDT_PROBE2(my_trace, len, ret);
        ...
    }

An observer would be like:

    SEC("bpf_sdt")
    int BPF_PROG(observer_prog, int len, int ret)
    {
        bpf_printk("len=%d ret=%d\n", len, ret);
        return 0;
    }

The target program and probe site for the observer program should be set
at load time via bpf_program__set_attach_target() manually, since program
names are not unique and program IDs are allocated at runtime - there is
no static way to identify the target program.

For BPF_SDT_DECLARE2(my_trace, int, int) and BPF_SDT_PROBE2(my_trace, len, ret)
macros, the compiler produces:

  [code section, e.g. xdp]
      goto +0                  // NOP, patched to CALL at attach time

  [.bpf_sdt_notes section]
      ___sdt_jt_my_trace:      // symbol marking this entry's boundary
      .quad  0b                // 8 bytes: offset of the NOP in the code
                               // section (resolved by the linker via
                               // R_BPF_64_ABS64 relocation)
      r1 = %[arg1_reg]         // 8 bytes per argument: BPF move insn
                               // whose src_reg field encodes the BPF
                               // register holding each probe argument
      r2 = %[arg2_reg]

      ....

  [.BTF section]
      FUNC_PROTO (int, int) -> void   // from BPF_SDT_DECLARE2 stub
      DECL_TAG "bpf_sdt:my_trace:2"   // keyed by name + nargs

The .BTF and .bpf_sdt_notes sections are parsed by libbpf and ultimately
converted to an insn_array map per bpf program. Each map slot records the
instruction offset, jited address, FUNC_PROTO BTF ID, and argument register
for each probe site.

When an observer prog is loaded, the probe is resolved against the target
program's SDT map using the target fd and probe name passed by the user.

For attach, the existing bpf trampoline is reused with per-argument register
mappings declared by the SDT macro.

Xu Kuohai (12):
  libbpf: Prepare bpf SDT probe section for the linker
  libbpf: Introduce bpf SDT probe macros
  libbpf: Add bpf_sdt_notes section parser
  bpf: Create insn_array map for bpf SDT probe
  bpf: Collect SDT probe BTF IDs from BTF decl tags
  bpf: Add type check for SDT probe site
  bpf: Record probe name in SDT map
  libbpf: Add libbpf support to load SDT observer program
  bpf: Add kernel support to load SDT observer program
  bpf: Support attach and detach for SDT observer program
  bpf, x86: Add JIT support SDT for probe
  selftests/bpf: Add tests for bpf SDT probe

 arch/x86/net/bpf_jit_comp.c                   |  38 +-
 include/linux/bpf.h                           |  26 +-
 include/linux/bpf_types.h                     |   1 +
 include/linux/bpf_verifier.h                  |   3 +
 include/linux/filter.h                        |   1 +
 include/uapi/linux/bpf.h                      |  21 +-
 kernel/bpf/bpf_insn_array.c                   |  64 ++-
 kernel/bpf/cfg.c                              |   3 +
 kernel/bpf/core.c                             |   5 +
 kernel/bpf/fixups.c                           |   4 +
 kernel/bpf/liveness.c                         |  24 +-
 kernel/bpf/syscall.c                          | 163 +++++-
 kernel/bpf/trampoline.c                       |   4 +
 kernel/bpf/verifier.c                         | 174 +++++-
 tools/include/uapi/linux/bpf.h                |  21 +-
 tools/lib/bpf/Makefile                        |   2 +-
 tools/lib/bpf/bpf.c                           |  10 +-
 tools/lib/bpf/bpf.h                           |   9 +-
 tools/lib/bpf/bpf_sdt.h                       | 346 ++++++++++++
 tools/lib/bpf/libbpf.c                        | 513 +++++++++++++++++-
 tools/lib/bpf/libbpf_internal.h               |   1 +
 tools/lib/bpf/linker.c                        |   9 +
 .../selftests/bpf/prog_tests/test_bpf_sdt.c   | 151 ++++++
 .../selftests/bpf/progs/bpf_sdt_observer.c    |  30 +
 .../selftests/bpf/progs/bpf_sdt_target.c      |  34 ++
 25 files changed, 1622 insertions(+), 35 deletions(-)
 create mode 100644 tools/lib/bpf/bpf_sdt.h
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_sdt.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_observer.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_target.c

-- 
2.47.3

[RFC PATCH bpf-next 01/12] libbpf: Prepare bpf SDT probe section for the linker

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

The follow up patch introduces bpf SDT probe macros that emit per-probe
metadata into a new data section named .bpf_sdt_notes. Each entry begins
with a ___sdt_jt_* symbol and contains an R_BPF_64_ABS64 relocation
referencing the probe site address in the bpf prog.

Prepare the linker for this by treating .bpf_sdt_notes like .jumptables.
Bypass the section-type check when appending ___sdt_jt_* symbols so they
can survive from linking, and relocate to real address inside each entry.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 tools/lib/bpf/libbpf_internal.h | 1 +
 tools/lib/bpf/linker.c          | 9 +++++++++
 2 files changed, 10 insertions(+)

diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index 04cd303fb5a8..ed7a587a61ca 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -75,6 +75,7 @@
 #endif
 
 #define JUMPTABLES_SEC ".jumptables"
+#define SDT_NOTES_SEC ".bpf_sdt_notes"
 
 #define BTF_INFO_ENC(kind, kind_flag, vlen) \
 	((!!(kind_flag) << 31) | ((kind) << 24) | ((vlen) & BTF_MAX_VLEN))
diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c
index 78f92c39290a..3a72276244c7 100644
--- a/tools/lib/bpf/linker.c
+++ b/tools/lib/bpf/linker.c
@@ -2028,6 +2028,8 @@ static int linker_append_elf_sym(struct bpf_linker *linker, struct src_obj *obj,
 
 		if (strcmp(src_sec->sec_name, JUMPTABLES_SEC) == 0)
 			goto add_sym;
+		if (strcmp(src_sec->sec_name, SDT_NOTES_SEC) == 0)
+			goto add_sym;
 	}
 
 	if (sym_bind == STB_LOCAL)
@@ -2274,6 +2276,13 @@ static int linker_append_elf_relos(struct bpf_linker *linker, struct src_obj *ob
 						insn->imm += sec->dst_off / sizeof(struct bpf_insn);
 					else
 						insn->imm += sec->dst_off;
+				} else if (strcmp(src_linked_sec->sec_name, SDT_NOTES_SEC) == 0) {
+					/* .bpf_sdt_notes contains R_BPF_64_ABS64 relocations
+					 * referencing code section symbols. Adjust the 64-bit
+					 * address by dst_off.
+					 */
+					__u64 *addr = dst_linked_sec->raw_data + dst_rel->r_offset;
+					*addr += sec->dst_off;
 				} else {
 					pr_warn("relocation against STT_SECTION in non-exec section is not supported!\n");
 					return -EINVAL;
-- 
2.47.3

[RFC PATCH bpf-next 02/12] libbpf: Introduce bpf SDT probe macros

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Introduce macros to declare and define bpf SDT probes. The declaration is
used to generate BTF ID of FUNC_PROTO for each probe site, so the probe
sites can be validated by the kernel, and observer programs can be attached
to the FUNC_PROTO like normal tracing programs.

1. Example

A probe in the target program can be declared and defined like:

    BPF_SDT_DECLARE2(my_trace, int, int);

    SEC("xdp")
    int xdp_prog(struct xdp_md *ctx)
    {
        int len = ctx->data_end - ctx->data;
        int ret = XDP_DROP;
        ...
        BPF_SDT_PROBE2(my_trace, len, ret);
        ...
    }

An observer would be like:

    SEC("bpf_sdt")
    int BPF_PROG(observer_prog, int len, int ret)
    {
        bpf_printk("len=%d ret=%d\n", len, ret);
        return 0;
    }

The target program and probe site is set at attach time dynamically.

2. Design

2.1 declaration macro

BPF_SDT_DECLARE<N>(name, types...) generates:

  - A BTF function-pointer variable in a .debug_* section:

        static void (*const __sdt_type_##name)(types...);

    which produces a BTF VAR -> PTR -> FUNC_PROTO(types...) chain
    tagged with btf_decl_tag("bpf_sdt:<name>:<nargs>"). The BPF
    linker discards the .debug_* section but preserves the BTF
    entries, so the argument types survive into the final object.
    The type will be passed to the kernel for the verifier to check
    if it matches the real value type in the argument register.

  - One dummy static variable per argument:

        __attribute__((section(".debug_bpf_sdt_chk")))
        typeof(declared_type) __sdt_chk_##name##_N;

    The dummy variable carries the declared type. At the
    BPF_SDT_PROBE call site, _Static_assert with
    __builtin_types_compatible_p() compares the caller's actual
    argument type against the dummy variable's type:

        _Static_assert(__builtin_types_compatible_p(
                        typeof(caller_arg),
                        typeof(__sdt_chk_##name##_N)),
                        "type mismatch");

    For example, if the caller passes an int where the declaration
    said char *, the compiler fails with a message naming the
    mismatched argument.

2.2 probe macro

BPF_SDT_PROBE<N> is an inline assembly sequence that emits into two
ELF sections simultaneously. A goto+0 instruction is inserted at the
call site in the bpf prog code section. It works as a NOP, and will be
patched to a jump instruction when an observer is attached. The call
site address and argument information are recorded into a separate
section .bpf_sdt_notes. This section is not executed, it is used as
metadata. libbpf will convert the metadata to an insn_array map at
load time.

An interesting note is that the argument registers are recorded in
BPF move instructions, like r1 = %[arg_reg0]. The instruction is not
executed, it is only used by the linker to extract the argument
register from the src_reg.

For BPF_SDT_PROBE2(my_trace, len, ret) the compiler produces:

  [code section, e.g. xdp]
      goto +0                  // NOP, patched to CALL at attach time

  [.bpf_sdt_notes]
      ___sdt_jt_my_trace:      // symbol marking this entry's boundary
      .quad  0b                // 8 bytes: offset of the NOP in the code
                               // section (resolved by the linker via
                               // R_BPF_64_ABS64 relocation)
      r1 = %[arg1_reg]         // 8 bytes per argument: BPF move insn
                               // whose src_reg field encodes the BPF
                               // register holding each probe argument
      r2 = %[arg2_reg]

      ....

  [.BTF]
      FUNC_PROTO (int, int) -> void   // from BPF_SDT_DECLARE2 stub
      DECL_TAG "bpf_sdt:my_trace:2"   // keyed by name + nargs

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 tools/lib/bpf/Makefile  |   2 +-
 tools/lib/bpf/bpf_sdt.h | 346 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 347 insertions(+), 1 deletion(-)
 create mode 100644 tools/lib/bpf/bpf_sdt.h

diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index eca584fb061e..1f6f027a327b 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -235,7 +235,7 @@ install_lib: all_cmd
 
 SRC_HDRS := bpf.h libbpf.h btf.h libbpf_common.h libbpf_legacy.h	     \
 	    bpf_helpers.h bpf_tracing.h bpf_endian.h bpf_core_read.h	     \
-	    skel_internal.h libbpf_version.h usdt.bpf.h
+	    skel_internal.h libbpf_version.h usdt.bpf.h bpf_sdt.h
 GEN_HDRS := $(BPF_GENERATED)
 
 INSTALL_PFX := $(DESTDIR)$(prefix)/include/bpf
diff --git a/tools/lib/bpf/bpf_sdt.h b/tools/lib/bpf/bpf_sdt.h
new file mode 100644
index 000000000000..23f84460aa36
--- /dev/null
+++ b/tools/lib/bpf/bpf_sdt.h
@@ -0,0 +1,346 @@
+/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */
+
+/*
+ * USDT-like probe for BPF programs.
+ *
+ * 1. Example
+ *
+ * A probe in the target program can be declared and defined like:
+ *
+ *   BPF_SDT_DECLARE2(my_trace, int, int);
+ *
+ *   SEC("xdp")
+ *   int xdp_prog(struct xdp_md *ctx)
+ *   {
+ *       int len = ctx->data_end - ctx->data;
+ *       int ret = XDP_DROP;
+ *       ...
+ *       BPF_SDT_PROBE2(my_trace, len, ret);
+ *       ...
+ *   }
+ *
+ * An observer would be like:
+ *
+ *   SEC("bpf_sdt")
+ *   int BPF_PROG(observer_prog, int len, int ret)
+ *   {
+ *       bpf_printk("len=%d ret=%d\n", len, ret);
+ *       return 0;
+ *   }
+ *
+ * The target program and probe site is set at attach time dynamically.
+ *
+ * 2. Design
+ *
+ * 2.1 declaration macro
+ *
+ * BPF_SDT_DECLARE<N>(name, types...) generates:
+ *
+ *  - A BTF function-pointer variable in a .debug_* section:
+ *
+ *       static void (*const __sdt_type_##name)(types...);
+ *
+ *   which produces a BTF VAR -> PTR -> FUNC_PROTO(types...) chain
+ *   tagged with btf_decl_tag("bpf_sdt:<name>:<nargs>"). The BPF
+ *   linker discards the .debug_* section but preserves the BTF
+ *   entries, so the argument types survive into the final object.
+ *   The type will be passed to the kernel for the verifier to check
+ *   if it matches the real value type in the argument register.
+ *
+ *  - One dummy static variable per argument:
+ *
+ *       __attribute__((section(".debug_bpf_sdt_chk")))
+ *       typeof(declared_type) __sdt_chk_##name##_N;
+ *
+ *   The dummy variable carries the declared type. At the
+ *   BPF_SDT_PROBE call site, _Static_assert with
+ *   __builtin_types_compatible_p() compares the caller's actual
+ *   argument type against the dummy variable's type:
+ *
+ *       _Static_assert(__builtin_types_compatible_p(
+ *                       typeof(caller_arg),
+ *                       typeof(__sdt_chk_##name##_N)),
+ *                       "type mismatch");
+ *
+ *   For example, if the caller passes an int where the declaration
+ *   said char *, the compiler fails with a message naming the
+ *   mismatched argument.
+ *
+ * 2.2 probe macro
+ *
+ * BPF_SDT_PROBE<N> is an inline assembly sequence that emits into two
+ * ELF sections simultaneously. A goto+0 instruction is inserted at the
+ * call site in the bpf prog code section. It works as a NOP, and will be
+ * patched to a jump instruction when an observer is attached. The call
+ * site address and argument information are recorded into a separate
+ * section .bpf_sdt_notes. This section is not executed, it is used as
+ * metadata. libbpf will convert the metadata to an insn_array map at
+ * load time.
+ *
+ * An interesting note is that the argument registers are recorded in
+ * BPF move instructions, like r1 = %[arg_reg0]. The instruction is not
+ * executed, it is only used by the linker to extract the argument
+ * register from the src_reg.
+ *
+ * For BPF_SDT_PROBE2(my_trace, len, ret) the compiler produces:
+ *
+ * [code section, e.g. xdp]
+ *     goto +0                  // NOP, patched to CALL at attach time
+ *
+ * [.bpf_sdt_notes]
+ *     ___sdt_jt_my_trace:      // symbol marking this entry's boundary
+ *     .quad  0b                // 8 bytes: offset of the NOP in the code
+ *                              // section (resolved by the linker via
+ *                              // R_BPF_64_ABS64 relocation)
+ *     r1 = %[arg1_reg]         // 8 bytes per argument: BPF move instruction
+ *                              // whose src_reg field encodes the BPF
+ *                              // register holding each probe argument
+ *     r2 = %[arg2_reg]
+ *
+ *     ....
+ *
+ *  [.BTF]
+ *     FUNC_PROTO (int, int) -> void   // from BPF_SDT_DECLARE2 stub
+ *     DECL_TAG "bpf_sdt:my_trace:2"   // keyed by name + nargs
+ */
+
+#ifndef __BPF_SDT_PROBE_H
+#define __BPF_SDT_PROBE_H
+
+#define ___sdt_decl_attr(name, nargs)					\
+	__attribute__((used, section(".debug_bpf_sdt_types"),		\
+			btf_decl_tag("bpf_sdt:" #name ":" #nargs)))
+
+#define ___sdt_chk_attr							\
+	 __attribute__((unused, section(".debug_bpf_sdt_chk")))
+
+#define BPF_SDT_DECLARE(name)						\
+	___sdt_decl_attr(name, 0)					\
+	static void (*const __sdt_type_##name)(void)
+
+#define BPF_SDT_DECLARE1(name, t0)					\
+	___sdt_decl_attr(name, 1)					\
+	static void (*const __sdt_type_##name)(t0);			\
+	___sdt_chk_attr t0 __sdt_chk_##name##_0
+
+#define BPF_SDT_DECLARE2(name, t0, t1)					\
+	___sdt_decl_attr(name, 2)					\
+	static void (*const __sdt_type_##name)(t0, t1);			\
+	___sdt_chk_attr t0 __sdt_chk_##name##_0;			\
+	___sdt_chk_attr t1 __sdt_chk_##name##_1
+
+#define BPF_SDT_DECLARE3(name, t0, t1, t2)				\
+	___sdt_decl_attr(name, 3)					\
+	static void (*const __sdt_type_##name)(t0, t1, t2);		\
+	___sdt_chk_attr t0 __sdt_chk_##name##_0;			\
+	___sdt_chk_attr t1 __sdt_chk_##name##_1;			\
+	___sdt_chk_attr t2 __sdt_chk_##name##_2
+
+#define BPF_SDT_DECLARE4(name, t0, t1, t2, t3)				\
+	___sdt_decl_attr(name, 4)					\
+	static void (*const __sdt_type_##name)(t0, t1, t2, t3);		\
+	___sdt_chk_attr t0 __sdt_chk_##name##_0;			\
+	___sdt_chk_attr t1 __sdt_chk_##name##_1;			\
+	___sdt_chk_attr t2 __sdt_chk_##name##_2;			\
+	___sdt_chk_attr t3 __sdt_chk_##name##_3
+
+#define BPF_SDT_DECLARE5(name, t0, t1, t2, t3, t4)			\
+	___sdt_decl_attr(name, 5)					\
+	static void (*const __sdt_type_##name)(t0, t1, t2, t3, t4);	\
+	___sdt_chk_attr t0 __sdt_chk_##name##_0;			\
+	___sdt_chk_attr t1 __sdt_chk_##name##_1;			\
+	___sdt_chk_attr t2 __sdt_chk_##name##_2;			\
+	___sdt_chk_attr t3 __sdt_chk_##name##_3;			\
+	___sdt_chk_attr t4 __sdt_chk_##name##_4
+
+/* probe macros, 0-5 arguments
+ *
+ * Each probe emits a NOP (goto +0) in the program stream plus a
+ * .bpf_sdt_notes entry.  The entry begins with the ___sdt_jt_<name>
+ * label so that libbpf can locate each entry boundary at load time.
+ * One BPF_SDT_PROBE<N> invocation per probe name is required; invoking
+ * the same name twice produces a duplicate symbol.
+ */
+#define BPF_SDT_PROBE(name)						\
+({									\
+	asm volatile(							\
+		"0:\n"							\
+		"goto +0\n"						\
+		".pushsection .bpf_sdt_notes,\"\",@progbits\n"		\
+		"___sdt_jt_" #name ":\n"				\
+		".quad 0b\n"						\
+		".popsection\n"						\
+	);								\
+})
+
+#define BPF_SDT_PROBE1(name, a)						\
+({									\
+	_Static_assert(__builtin_types_compatible_p(typeof(a),		\
+			typeof(__sdt_chk_##name##_0)),			\
+			"BPF_SDT_PROBE1: arg0 type mismatch");		\
+	_Static_assert(sizeof(typeof(a)) <= 8,				\
+			"BPF_SDT_PROBE1: arg0 too large, use pointer");	\
+	typeof(a) __a0 = (a);						\
+	asm volatile(							\
+		"0:\n"							\
+		"goto +0\n"						\
+		".pushsection .bpf_sdt_notes,\"\",@progbits\n"		\
+		"___sdt_jt_" #name ":\n"				\
+		".quad 0b\n"						\
+		"r1 = %0\n"						\
+		".popsection\n"						\
+		:: "r"(__a0)						\
+	);								\
+})
+
+#define BPF_SDT_PROBE2(name, a, b)					\
+({									\
+	_Static_assert(__builtin_types_compatible_p(typeof(a),		\
+			typeof(__sdt_chk_##name##_0)),			\
+			"BPF_SDT_PROBE2: arg0 type mismatch");		\
+	_Static_assert(sizeof(typeof(a)) <= 8,				\
+			"BPF_SDT_PROBE2: arg0 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(b),		\
+			typeof(__sdt_chk_##name##_1)),			\
+			"BPF_SDT_PROBE2: arg1 type mismatch");		\
+	_Static_assert(sizeof(typeof(b)) <= 8,				\
+			"BPF_SDT_PROBE2: arg1 too large, use pointer");	\
+	typeof(a) __a0 = (a);						\
+	typeof(b) __a1 = (b);						\
+	asm volatile(							\
+		"0:\n"							\
+		"goto +0\n"						\
+		".pushsection .bpf_sdt_notes,\"\",@progbits\n"		\
+		"___sdt_jt_" #name ":\n"				\
+		".quad 0b\n"						\
+		"r1 = %0\n"						\
+		"r2 = %1\n"						\
+		".popsection\n"						\
+		:: "r"(__a0), "r"(__a1)					\
+	);								\
+})
+
+#define BPF_SDT_PROBE3(name, a, b, c)					\
+({									\
+	_Static_assert(__builtin_types_compatible_p(typeof(a),		\
+			typeof(__sdt_chk_##name##_0)),			\
+			"BPF_SDT_PROBE3: arg0 type mismatch");		\
+	_Static_assert(sizeof(typeof(a)) <= 8,				\
+			"BPF_SDT_PROBE3: arg0 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(b),		\
+			typeof(__sdt_chk_##name##_1)),			\
+			"BPF_SDT_PROBE3: arg1 type mismatch");		\
+	_Static_assert(sizeof(typeof(b)) <= 8,				\
+			"BPF_SDT_PROBE3: arg1 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(c),		\
+			typeof(__sdt_chk_##name##_2)),			\
+			"BPF_SDT_PROBE3: arg2 type mismatch");		\
+	_Static_assert(sizeof(typeof(c)) <= 8,				\
+			"BPF_SDT_PROBE3: arg2 too large, use pointer");	\
+	typeof(a) __a0 = (a);						\
+	typeof(b) __a1 = (b);						\
+	typeof(c) __a2 = (c);						\
+	asm volatile(							\
+		"0:\n"							\
+		"goto +0\n"						\
+		".pushsection .bpf_sdt_notes,\"\",@progbits\n"		\
+		"___sdt_jt_" #name ":\n"				\
+		".quad 0b\n"						\
+		"r1 = %0\n"						\
+		"r2 = %1\n"						\
+		"r3 = %2\n"						\
+		".popsection\n"						\
+		:: "r"(__a0), "r"(__a1), "r"(__a2)			\
+	);								\
+})
+
+#define BPF_SDT_PROBE4(name, a, b, c, d)				\
+({									\
+	_Static_assert(__builtin_types_compatible_p(typeof(a),		\
+			typeof(__sdt_chk_##name##_0)),			\
+			"BPF_SDT_PROBE4: arg0 type mismatch");		\
+	_Static_assert(sizeof(typeof(a)) <= 8,				\
+			"BPF_SDT_PROBE4: arg0 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(b),		\
+			typeof(__sdt_chk_##name##_1)),			\
+			"BPF_SDT_PROBE4: arg1 type mismatch");		\
+	_Static_assert(sizeof(typeof(b)) <= 8,				\
+			"BPF_SDT_PROBE4: arg1 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(c),		\
+			typeof(__sdt_chk_##name##_2)),			\
+			"BPF_SDT_PROBE4: arg2 type mismatch");		\
+	_Static_assert(sizeof(typeof(c)) <= 8,				\
+			"BPF_SDT_PROBE4: arg2 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(d),		\
+			typeof(__sdt_chk_##name##_3)),			\
+			"BPF_SDT_PROBE4: arg3 type mismatch");		\
+	_Static_assert(sizeof(typeof(d)) <= 8,				\
+			"BPF_SDT_PROBE4: arg3 too large, use pointer");	\
+	typeof(a) __a0 = (a);						\
+	typeof(b) __a1 = (b);						\
+	typeof(c) __a2 = (c);						\
+	typeof(d) __a3 = (d);						\
+	asm volatile(							\
+		"0:\n"							\
+		"goto +0\n"						\
+		".pushsection .bpf_sdt_notes,\"\",@progbits\n"		\
+		"___sdt_jt_" #name ":\n"				\
+		".quad 0b\n"						\
+		"r1 = %0\n"						\
+		"r2 = %1\n"						\
+		"r3 = %2\n"						\
+		"r4 = %3\n"						\
+		".popsection\n"						\
+		:: "r"(__a0), "r"(__a1), "r"(__a2), "r"(__a3)		\
+	);								\
+})
+
+#define BPF_SDT_PROBE5(name, a, b, c, d, e)				\
+({									\
+	_Static_assert(__builtin_types_compatible_p(typeof(a),		\
+			typeof(__sdt_chk_##name##_0)),			\
+			"BPF_SDT_PROBE5: arg0 type mismatch");		\
+	_Static_assert(sizeof(typeof(a)) <= 8,				\
+			"BPF_SDT_PROBE5: arg0 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(b),		\
+			typeof(__sdt_chk_##name##_1)),			\
+			"BPF_SDT_PROBE5: arg1 type mismatch");		\
+	_Static_assert(sizeof(typeof(b)) <= 8,				\
+			"BPF_SDT_PROBE5: arg1 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(c),		\
+			typeof(__sdt_chk_##name##_2)),			\
+			"BPF_SDT_PROBE5: arg2 type mismatch");		\
+	_Static_assert(sizeof(typeof(c)) <= 8,				\
+			"BPF_SDT_PROBE5: arg2 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(d),		\
+			typeof(__sdt_chk_##name##_3)),			\
+			"BPF_SDT_PROBE5: arg3 type mismatch");		\
+	_Static_assert(sizeof(typeof(d)) <= 8,				\
+			"BPF_SDT_PROBE5: arg3 too large, use pointer");	\
+	_Static_assert(__builtin_types_compatible_p(typeof(e),		\
+			typeof(__sdt_chk_##name##_4)),			\
+			"BPF_SDT_PROBE5: arg4 type mismatch");		\
+	_Static_assert(sizeof(typeof(e)) <= 8,				\
+			"BPF_SDT_PROBE5: arg4 too large, use pointer");	\
+	typeof(a) __a0 = (a);						\
+	typeof(b) __a1 = (b);						\
+	typeof(c) __a2 = (c);						\
+	typeof(d) __a3 = (d);						\
+	typeof(e) __a4 = (e);						\
+	asm volatile(							\
+		"0:\n"							\
+		"goto +0\n"						\
+		".pushsection .bpf_sdt_notes,\"\",@progbits\n"		\
+		"___sdt_jt_" #name ":\n"				\
+		".quad 0b\n"						\
+		"r1 = %0\n"						\
+		"r2 = %1\n"						\
+		"r3 = %2\n"						\
+		"r4 = %3\n"						\
+		"r5 = %4\n"						\
+		".popsection\n"						\
+		:: "r"(__a0), "r"(__a1), "r"(__a2), "r"(__a3), "r"(__a4)\
+	);								\
+})
+
+#endif /* __BPF_SDT_PROBE_H */
-- 
2.47.3

[RFC PATCH bpf-next 03/12] libbpf: Add bpf_sdt_notes section parser

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Parse bpf_sdt_notes section to get probe call site index in the bpf
prog, and argument registers for each probe.

Layout of SDT entry:
      0: insn_off         // 64 bits of the NOP instruction offset
      8: r1 = %[arg0_reg]  //  bpf move instruction for arg0
     16: r2 = %[arg1_reg]  //  bpf move instruction for arg1
    ...
  8 * N: rN = %[argN_reg]  //  bpf move instruction for argN

N + 1 is the argument number, up to 5.

The argument number is calculated by walking the ELF symbol table
for ___sdt_jt_* symbols that mark each probe entry boundary in the
bpf_sdt_notes section: (next_off - off - 8) / sizeof(struct bpf_insn),
where off is the current entry offset, next_off is the next entry
offset.

The owning program for each probe is identified by walking the relocation
entries for each probe. Each relocation entry records the section index
of the program that contains the probe.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 tools/lib/bpf/libbpf.c | 290 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 290 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 7162146280a8..327c9d412fe8 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -693,6 +693,7 @@ struct elf_state {
 	bool has_st_ops;
 	int arena_data_shndx;
 	int jumptables_data_shndx;
+	int sdt_notes_shndx;
 };
 
 struct usdt_manager;
@@ -703,6 +704,15 @@ enum bpf_object_state {
 	OBJ_LOADED,
 };
 
+struct sdt_entry {
+	char *name;      /* probe name */
+	__u16 prog_idx;  /* resolved in bpf_object__resolve_sdt_progs */
+	__u16 sec_idx;   /* ELF section index of the probe site (from reloc) */
+	__u64 insn_idx;  /* insn index of probe site within its ELF section */
+	__u8 nargs;      /* how many arguments */
+	__u8 arg_reg[5]; /* which register the argument locates in */
+};
+
 struct bpf_object {
 	char name[BPF_OBJ_NAME_LEN];
 	char license[64];
@@ -768,6 +778,13 @@ struct bpf_object {
 	void *jumptables_data;
 	size_t jumptables_data_sz;
 
+	void *sdt_notes_data;
+	size_t sdt_notes_data_sz;
+
+	struct sdt_entry *sdt_entries;
+	size_t sdt_entry_cnt;
+	size_t sdt_entry_cap;
+
 	struct {
 		struct bpf_program *prog;
 		unsigned int sym_off;
@@ -3892,6 +3909,254 @@ static int cmp_progs(const void *_a, const void *_b)
 	return a->sec_insn_off < b->sec_insn_off ? -1 : 1;
 }
 
+static struct bpf_program *find_prog_by_sec_insn(const struct bpf_object *obj,
+						 size_t sec_idx, size_t insn_idx);
+
+struct sdt_sym {
+	const char *name;
+	size_t off; /* offset in the sdt entry section */
+};
+
+static int sdt_sym_cmp(const void *a, const void *b)
+{
+	const struct sdt_sym *sym1 = a, *sym2 = b;
+
+	if (sym1->off == sym2->off)
+		return 0;
+
+	return sym1->off < sym2->off ? -1 : 1;
+}
+
+static Elf_Data *sdt_find_relo(struct bpf_object *obj)
+{
+	GElf_Shdr shdr;
+	Elf_Scn *scn = NULL;
+
+	while ((scn = elf_nextscn(obj->efile.elf, scn)) != NULL) {
+		if (!gelf_getshdr(scn, &shdr) || shdr.sh_type != SHT_REL)
+			continue;
+
+		if (shdr.sh_info == obj->efile.sdt_notes_shndx)
+			return elf_getdata(scn, NULL);
+	}
+
+	return NULL;
+}
+
+static int sdt_collect_syms(struct bpf_object *obj, struct sdt_sym **sdt_syms)
+{
+	int i, err, cnt = 0;
+	size_t nr_syms;
+	size_t sym_cap = 0;
+	const char *name;
+	Elf64_Sym *sym, *syms;
+	struct sdt_sym *ssyms = NULL;
+
+	syms = obj->efile.symbols->d_buf;
+	nr_syms = obj->efile.symbols->d_size / sizeof(Elf64_Sym);
+
+	for (i = 0; i < nr_syms; i++) {
+		sym = &syms[i];
+		if (sym->st_shndx != obj->efile.sdt_notes_shndx)
+			continue;
+
+		if (ELF64_ST_TYPE(sym->st_info) != STT_NOTYPE)
+			continue;
+
+		name = elf_sym_str(obj, sym->st_name);
+		if (!name || strncmp(name, "___sdt_jt_", 10))
+			continue;
+
+		err = libbpf_ensure_mem((void **)&ssyms, &sym_cap, sizeof(*ssyms), cnt + 1);
+		if (err) {
+			free(ssyms);
+			return -ENOMEM;
+		}
+
+		ssyms[cnt].name = name;
+		ssyms[cnt].off = sym->st_value;
+		cnt++;
+	}
+
+	if (cnt)
+		qsort(ssyms, cnt, sizeof(*ssyms), sdt_sym_cmp);
+
+	*sdt_syms = ssyms;
+
+	return cnt;
+}
+
+/*
+ * Layout of SDT entry:
+ *
+ *   off + 0:  64 bits of the NOP instruction offset
+ *   off + 8:  r1 = %[arg0_reg]  //  bpf move instruction for arg0
+ *   off + 16: r2 = %[arg1_reg]  //  bpf move instruction for arg1
+ *       ...
+ *   off + 8 * N: rN = %[argN_reg]  //  bpf move instruction for argN
+ *
+ *   next_off + 0: // start of the next entry
+ *
+ *   so argument number N = (next_entry_off - off - 8) / sizeof(bpf_insn).
+ */
+static int sdt_calc_nargs(struct sdt_sym *syms, int sym_idx, size_t sym_cnt, size_t last_off)
+{
+	size_t off = syms[sym_idx].off;
+	size_t next_off = (sym_idx + 1 < sym_cnt) ? syms[sym_idx + 1].off : last_off;
+
+	if (next_off > last_off)
+		return -EINVAL;
+
+	if (off + 8 > next_off)
+		return -EINVAL;
+
+	if ((next_off - off - 8) % sizeof(struct bpf_insn))
+		return -EINVAL;
+
+	if ((next_off - off - 8) / sizeof(struct bpf_insn) > 5)
+		return -EINVAL;
+
+	return (next_off - off - 8) / sizeof(struct bpf_insn);
+}
+
+/*
+ * Look up the ELF section index of the code section containing the probe
+ * site, via the R_BPF_64_ABS64 relocation on this entry's .quad.  The
+ * target symbol's st_shndx is the code section; this is stored in the
+ * sdt_entry and used later (after subprogram linking) to resolve prog_idx.
+ *
+ * Returns the section index on success, -1 on failure.
+ */
+static int sdt_find_sec_idx(struct bpf_object *obj, Elf_Data *sdt_relo,
+			    size_t sdt_entry_off)
+{
+	int i;
+	Elf64_Sym *tgt;
+	Elf64_Rel *rel = sdt_relo->d_buf;
+	Elf64_Sym *elf_syms = obj->efile.symbols->d_buf;
+
+	for (i = 0; i < sdt_relo->d_size / sizeof(Elf64_Rel); i++, rel++) {
+		if (rel->r_offset != sdt_entry_off)
+			continue;
+		if (ELF64_R_TYPE(rel->r_info) != R_BPF_64_ABS64)
+			continue;
+
+		tgt = &elf_syms[ELF64_R_SYM(rel->r_info)];
+		return tgt->st_shndx;
+	}
+
+	return -1;
+}
+
+static int sdt_parse_arg(const void *data, int nargs, struct sdt_entry *e)
+{
+	int i;
+	const struct bpf_insn *insn;
+
+	insn = (const struct bpf_insn *)data;
+	for (i = 0; i < nargs; i++, insn++) {
+		if (insn->code != (BPF_ALU64 | BPF_MOV | BPF_X))
+			return -EINVAL;
+		e->arg_reg[i] = insn->src_reg;
+	}
+
+	return 0;
+}
+
+
+/*
+ * Parse .bpf_sdt_notes into per-probe SDT entries.
+ *
+ * Each entry starts at a ___sdt_jt_<name> label and contains .quad nop_off
+ * (8 bytes, with R_BPF_64_ABS64 reloc) followed by one 8-byte BPF_MOV
+ * instruction per argument whose src_reg field encodes the register.
+ *
+ * nargs is derived from the gap between consecutive labels: the assembler
+ * lays entries out sequentially, so nargs = (next_off - off - 8) / 8.
+ */
+static int bpf_object__collect_sdt_notes(struct bpf_object *obj)
+{
+	int i, sym_cnt, err = 0;
+	struct sdt_sym *sdt_syms = NULL;
+	Elf_Data *sdt_relo = NULL;
+	const void *data = obj->sdt_notes_data;
+
+	if (!obj->sdt_notes_data || !obj->sdt_notes_data_sz)
+		return 0;
+
+	sdt_relo = sdt_find_relo(obj);
+	if (!sdt_relo)
+		return -EINVAL;
+
+	sym_cnt = sdt_collect_syms(obj, &sdt_syms);
+	if (sym_cnt < 0)
+		return sym_cnt;
+
+	for (i = 0; i < sym_cnt; i++) {
+		__u64 nop_idx;
+		int nargs, sec_idx;
+		struct sdt_entry *e;
+		size_t off = sdt_syms[i].off;
+
+		nargs = sdt_calc_nargs(sdt_syms, i, sym_cnt, obj->sdt_notes_data_sz);
+		if (nargs < 0) {
+			err = -EINVAL;
+			goto out;
+		}
+
+		memcpy(&nop_idx, data + off, sizeof(__u64));
+		nop_idx = nop_idx / sizeof(struct bpf_insn);
+
+		/*
+		 * Record the ELF section index of the code section containing
+		 * the probe site.  prog_idx is resolved later in
+		 * bpf_object__resolve_sdt_progs(), after static subprograms
+		 * in .text have been absorbed into their calling main program;
+		 * resolving here would return the subprogram's own entry, not
+		 * the main program that ends up owning the probe site.
+		 */
+		sec_idx = sdt_find_sec_idx(obj, sdt_relo, off);
+		if (sec_idx < 0) {
+			err = -EINVAL;
+			goto out;
+		}
+
+		/* Read arg_reg[] from cold instructions */
+		err = libbpf_ensure_mem((void **)&obj->sdt_entries, &obj->sdt_entry_cap,
+					sizeof(*obj->sdt_entries), i + 1);
+		if (err) {
+			err = -ENOMEM;
+			goto out;
+		}
+
+		e = &obj->sdt_entries[i];
+		err = sdt_parse_arg(data + off + 8, nargs, e);
+		if (err < 0)
+			goto out;
+
+		e->name = strdup(sdt_syms[i].name + 10); /* skip "___sdt_jt_" */
+		if (!e->name) {
+			err = -ENOMEM;
+			goto out;
+		}
+		e->sec_idx = sec_idx;
+		e->insn_idx = nop_idx;
+		e->nargs = nargs;
+		obj->sdt_entry_cnt++;
+	}
+
+out:
+	if (err) {
+		for (i = 0; i < obj->sdt_entry_cnt; i++)
+			free(obj->sdt_entries[i].name);
+		zfree(&obj->sdt_entries);
+		obj->sdt_entry_cnt = obj->sdt_entry_cap = 0;
+	}
+
+	free(sdt_syms);
+	return err;
+}
+
 static int bpf_object__elf_collect(struct bpf_object *obj)
 {
 	struct elf_sec_desc *sec_desc;
@@ -4034,6 +4299,13 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
 				memcpy(obj->jumptables_data, data->d_buf, data->d_size);
 				obj->jumptables_data_sz = data->d_size;
 				obj->efile.jumptables_data_shndx = idx;
+			} else if (strcmp(name, SDT_NOTES_SEC) == 0) {
+				obj->sdt_notes_data = malloc(data->d_size);
+				if (!obj->sdt_notes_data)
+					return -ENOMEM;
+				memcpy(obj->sdt_notes_data, data->d_buf, data->d_size);
+				obj->sdt_notes_data_sz = data->d_size;
+				obj->efile.sdt_notes_shndx = idx;
 			} else {
 				pr_info("elf: skipping unrecognized data section(%d) %s\n",
 					idx, name);
@@ -4087,6 +4359,13 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
 	if (obj->nr_programs)
 		qsort(obj->programs, obj->nr_programs, sizeof(*obj->programs), cmp_progs);
 
+	err = bpf_object__collect_sdt_notes(obj);
+	if (err) {
+		zfree(&obj->sdt_notes_data);
+		obj->sdt_notes_data_sz = 0;
+		return err;
+	}
+
 	return bpf_object__init_btf(obj, btf_data, btf_ext_data);
 }
 
@@ -9629,6 +9908,17 @@ void bpf_object__close(struct bpf_object *obj)
 	zfree(&obj->jumptables_data);
 	obj->jumptables_data_sz = 0;
 
+	zfree(&obj->sdt_notes_data);
+	obj->sdt_notes_data_sz = 0;
+
+	if (obj->sdt_entries) {
+		for (i = 0; i < obj->sdt_entry_cnt; i++)
+			free(obj->sdt_entries[i].name);
+		zfree(&obj->sdt_entries);
+		obj->sdt_entry_cnt = 0;
+		obj->sdt_entry_cap = 0;
+	}
+
 	for (i = 0; i < obj->jumptable_map_cnt; i++)
 		close(obj->jumptable_maps[i].fd);
 	zfree(&obj->jumptable_maps);
-- 
2.47.3

[RFC PATCH bpf-next 04/12] bpf: Create insn_array map for bpf SDT probe

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Create one BPF_MAP_TYPE_INSN_ARRAY map per program from the parsed
SDT entries. Each map is tagged with BPF_F_INSN_ARRAY_SDT and
populated with the instruction offset, argument count, and per-argument
registers.

Since the SDT map is not explicitly referenced by any instruction in the
bpf program, add a new sdt_map_fd attr field to pass the map fd to the
kernel, so the verifier can bind it to the program's used_maps.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 include/uapi/linux/bpf.h       |  11 ++-
 kernel/bpf/bpf_insn_array.c    |  19 ++++-
 kernel/bpf/cfg.c               |   3 +
 kernel/bpf/syscall.c           |   2 +-
 kernel/bpf/verifier.c          |  23 ++++++
 tools/include/uapi/linux/bpf.h |  11 ++-
 tools/lib/bpf/bpf.c            |   3 +-
 tools/lib/bpf/bpf.h            |   4 +-
 tools/lib/bpf/libbpf.c         | 141 ++++++++++++++++++++++++++++++++-
 9 files changed, 210 insertions(+), 7 deletions(-)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index c91b5a4bda03..6a03e3f0506e 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1466,6 +1466,9 @@ enum {
 
 /* Enable BPF ringbuf overwrite mode */
 	BPF_F_RB_OVERWRITE	= (1U << 19),
+
+/* insn_array map is used for bpf SDT probe */
+	BPF_F_INSN_ARRAY_SDT	= (1U << 20),
 };
 
 /* Flags for BPF_PROG_QUERY. */
@@ -1669,6 +1672,10 @@ union bpf_attr {
 		 * verification.
 		 */
 		__s32		keyring_id;
+		/* fd of the BPF_MAP_TYPE_INSN_ARRAY map created with
+		 * BPF_F_INSN_ARRAY_SDT, used for SDT probe
+		 */
+		__u32		sdt_map_fd;
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
@@ -7732,7 +7739,9 @@ struct bpf_insn_array_value {
 	__u32 orig_off;
 	__u32 xlated_off;
 	__u32 jitted_off;
-	__u32 :32;
+	__u8  nargs;		/* argument count (0..5) */
+	__u8  arg_reg[5];	/* BPF register for each argument */
+	__u8  pad[2];
 };
 
 #endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index a2f84afe6f7c..f43777c5cd07 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -7,6 +7,7 @@ struct bpf_insn_array {
 	struct bpf_map map;
 	atomic_t used;
 	long *ips;
+	int subtype;
 	DECLARE_FLEX_ARRAY(struct bpf_insn_array_value, values);
 };
 
@@ -15,6 +16,11 @@ struct bpf_insn_array {
 
 #define INSN_DELETED ((u32)-1)
 
+enum bpf_insn_array_subtype {
+	BPF_INSN_ARRAY_SUBTYPE_JUMP_TABLE = 0,
+	BPF_INSN_ARRAY_SUBTYPE_SDT = 1,
+};
+
 static inline u64 insn_array_alloc_size(u32 max_entries)
 {
 	const u64 base_size = sizeof(struct bpf_insn_array);
@@ -28,7 +34,8 @@ static int insn_array_alloc_check(union bpf_attr *attr)
 	u32 value_size = sizeof(struct bpf_insn_array_value);
 
 	if (attr->max_entries == 0 || attr->key_size != 4 ||
-	    attr->value_size != value_size || attr->map_flags != 0)
+	    attr->value_size != value_size ||
+	    attr->map_flags & ~BPF_F_INSN_ARRAY_SDT)
 		return -EINVAL;
 
 	return 0;
@@ -55,6 +62,11 @@ static struct bpf_map *insn_array_alloc(union bpf_attr *attr)
 
 	bpf_map_init_from_attr(&insn_array->map, attr);
 
+	if (attr->map_flags & BPF_F_INSN_ARRAY_SDT)
+		insn_array->subtype = BPF_INSN_ARRAY_SUBTYPE_SDT;
+	else
+		insn_array->subtype = BPF_INSN_ARRAY_SUBTYPE_JUMP_TABLE;
+
 	/* BPF programs aren't allowed to write to the map */
 	insn_array->map.map_flags |= BPF_F_RDONLY_PROG;
 
@@ -90,6 +102,11 @@ static long insn_array_update_elem(struct bpf_map *map, void *key, void *value,
 
 	insn_array->values[index].orig_off = val.orig_off;
 
+	if (insn_array->subtype == BPF_INSN_ARRAY_SUBTYPE_SDT) {
+		insn_array->values[index].nargs = val.nargs;
+		memcpy(insn_array->values[index].arg_reg, val.arg_reg, sizeof(val.arg_reg));
+	}
+
 	return 0;
 }
 
diff --git a/kernel/bpf/cfg.c b/kernel/bpf/cfg.c
index 26d37066465f..1b8734ee4bf3 100644
--- a/kernel/bpf/cfg.c
+++ b/kernel/bpf/cfg.c
@@ -251,6 +251,9 @@ static struct bpf_iarray *jt_from_map(struct bpf_map *map)
 	int err;
 	int n;
 
+	if (map->map_flags & BPF_F_INSN_ARRAY_SDT)
+		return ERR_PTR(-EINVAL);
+
 	jt = bpf_iarray_realloc(NULL, map->max_entries);
 	if (!jt)
 		return ERR_PTR(-ENOMEM);
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 6db306d23b47..dc881e5ad411 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2967,7 +2967,7 @@ int __init __used bpf_multi_func(void) { return 0; }
 BTF_ID_LIST_GLOBAL_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func)
 
 /* last field in 'union bpf_attr' used by this command */
-#define BPF_PROG_LOAD_LAST_FIELD keyring_id
+#define BPF_PROG_LOAD_LAST_FIELD sdt_map_fd
 
 static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_attr *attr_log)
 {
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 25aea4271cd0..05734163650a 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -19800,6 +19800,29 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,
 	if (ret)
 		goto skip_full_check;
 
+	if (attr->sdt_map_fd) {
+		CLASS(fd, f)(attr->sdt_map_fd);
+		struct bpf_map *sdt_map = __bpf_map_get(f);
+
+		if (IS_ERR(sdt_map)) {
+			verbose(env, "sdt_map_fd %d is not a valid bpf_map\n",
+				attr->sdt_map_fd);
+			ret = PTR_ERR(sdt_map);
+			goto skip_full_check;
+		}
+		if (sdt_map->map_type != BPF_MAP_TYPE_INSN_ARRAY) {
+			verbose(env, "sdt_map_fd %d is not an INSN_ARRAY map\n",
+				attr->sdt_map_fd);
+			ret = -EINVAL;
+			goto skip_full_check;
+		}
+		ret = __add_used_map(env, sdt_map);
+		if (ret < 0) {
+			verbose(env, "failed to bind SDT map to program: %d\n", ret);
+			goto skip_full_check;
+		}
+	}
+
 	mark_verifier_state_clean(env);
 
 	if (IS_ERR(btf_vmlinux)) {
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index c91b5a4bda03..6a03e3f0506e 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1466,6 +1466,9 @@ enum {
 
 /* Enable BPF ringbuf overwrite mode */
 	BPF_F_RB_OVERWRITE	= (1U << 19),
+
+/* insn_array map is used for bpf SDT probe */
+	BPF_F_INSN_ARRAY_SDT	= (1U << 20),
 };
 
 /* Flags for BPF_PROG_QUERY. */
@@ -1669,6 +1672,10 @@ union bpf_attr {
 		 * verification.
 		 */
 		__s32		keyring_id;
+		/* fd of the BPF_MAP_TYPE_INSN_ARRAY map created with
+		 * BPF_F_INSN_ARRAY_SDT, used for SDT probe
+		 */
+		__u32		sdt_map_fd;
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
@@ -7732,7 +7739,9 @@ struct bpf_insn_array_value {
 	__u32 orig_off;
 	__u32 xlated_off;
 	__u32 jitted_off;
-	__u32 :32;
+	__u8  nargs;		/* argument count (0..5) */
+	__u8  arg_reg[5];	/* BPF register for each argument */
+	__u8  pad[2];
 };
 
 #endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index 96819c082c77..cb006bca97c6 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -295,7 +295,7 @@ int bpf_prog_load(enum bpf_prog_type prog_type,
 		  const struct bpf_insn *insns, size_t insn_cnt,
 		  struct bpf_prog_load_opts *opts)
 {
-	const size_t attr_sz = offsetofend(union bpf_attr, keyring_id);
+	const size_t attr_sz = offsetofend(union bpf_attr, sdt_map_fd);
 	void *finfo = NULL, *linfo = NULL;
 	const char *func_info, *line_info;
 	__u32 log_size, log_level, attach_prog_fd, attach_btf_obj_fd;
@@ -369,6 +369,7 @@ int bpf_prog_load(enum bpf_prog_type prog_type,
 
 	attr.fd_array = ptr_to_u64(OPTS_GET(opts, fd_array, NULL));
 	attr.fd_array_cnt = OPTS_GET(opts, fd_array_cnt, 0);
+	attr.sdt_map_fd = OPTS_GET(opts, sdt_map_fd, 0);
 
 	if (log_level) {
 		attr.log_buf = ptr_to_u64(log_buf);
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 7534a593edae..88294bb6b120 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -128,9 +128,11 @@ struct bpf_prog_load_opts {
 
 	/* if set, provides the length of fd_array */
 	__u32 fd_array_cnt;
+	/* if set, FD of the program's BPF_MAP_TYPE_INSN_ARRAY SDT map */
+	__u32 sdt_map_fd;
 	size_t :0;
 };
-#define bpf_prog_load_opts__last_field fd_array_cnt
+#define bpf_prog_load_opts__last_field sdt_map_fd
 
 LIBBPF_API int bpf_prog_load(enum bpf_prog_type prog_type,
 			     const char *prog_name, const char *license,
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 327c9d412fe8..3f8b12a1eb8b 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -494,6 +494,7 @@ struct bpf_program {
 	struct bpf_object *obj;
 
 	int fd;
+	int sdt_map_fd;
 	bool autoload;
 	bool autoattach;
 	bool sym_global;
@@ -519,6 +520,9 @@ struct bpf_program {
 
 	struct bpf_light_subprog *subprogs;
 	__u32 subprog_cnt;
+
+	/* index of the main program that absorbed this subprog */
+	int absorbed_by;
 };
 
 struct bpf_struct_ops {
@@ -879,7 +883,9 @@ bpf_object__init_prog(struct bpf_object *obj, struct bpf_program *prog,
 
 	prog->type = BPF_PROG_TYPE_UNSPEC;
 	prog->fd = -1;
+	prog->sdt_map_fd = -1;
 	prog->exception_cb_idx = -1;
+	prog->absorbed_by = -1;
 
 	/* libbpf's convention for SEC("?abc...") is that it's just like
 	 * SEC("abc...") but the corresponding bpf_program starts out with
@@ -6686,6 +6692,131 @@ static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struc
 	return err;
 }
 
+static int bpf_object__resolve_sdt_progs(struct bpf_object *obj)
+{
+	int i, j;
+	struct bpf_light_subprog *sp;
+
+	for (i = 0; i < obj->sdt_entry_cnt; i++) {
+		struct sdt_entry *e = &obj->sdt_entries[i];
+		struct bpf_program *prog, *subprog;
+
+		prog = find_prog_by_sec_insn(obj, e->sec_idx, e->insn_idx);
+		if (!prog) {
+			pr_warn("sdt: probe '%s' at sec %u insn %llu not found in any program\n",
+				e->name, (unsigned)e->sec_idx, (unsigned long long)e->insn_idx);
+			return -EINVAL;
+		}
+
+		/* resolve subprog probe in the main prog that absorbed the subprog */
+		if (prog_is_subprog(obj, prog) && prog->absorbed_by != -1) {
+			subprog = prog;
+			prog = &obj->programs[prog->absorbed_by];
+			for (j = 0; j < prog->subprog_cnt; j++) {
+				sp = &prog->subprogs[j];
+				if (sp->sec_insn_off == subprog->sec_insn_off) {
+					e->insn_idx = sp->sub_insn_off +
+						       (e->insn_idx - sp->sec_insn_off);
+					break;
+				}
+			}
+			if (j >= prog->subprog_cnt) {
+				pr_warn("sdt: subprog probe '%s' not found\n", e->name);
+				return -EINVAL;
+			}
+		}
+
+		e->prog_idx = prog - obj->programs;
+	}
+	return 0;
+}
+
+static int bpf_object__create_sdt_maps(struct bpf_object *obj)
+{
+	const __u32 value_size = sizeof(struct bpf_insn_array_value);
+	struct bpf_insn_array_value val = {};
+	struct bpf_program *prog;
+	struct {
+		__u32 sdt_cnt;
+		__u32 next_key;
+	} *prog_sdt;
+	int i, err = 0;
+
+	if (!obj->sdt_entry_cnt)
+		return 0;
+
+	err = bpf_object__resolve_sdt_progs(obj);
+	if (err)
+		return err;
+
+	prog_sdt = calloc(obj->nr_programs, sizeof(*prog_sdt));
+	if (!prog_sdt)
+		return -ENOMEM;
+
+	/* count entries per program */
+	for (i = 0; i < obj->sdt_entry_cnt; i++)
+		prog_sdt[obj->sdt_entries[i].prog_idx].sdt_cnt++;
+
+	/* create insn_array maps per program and populate entries */
+	for (i = 0; i < obj->sdt_entry_cnt; i++) {
+		__u32 key;
+		struct sdt_entry *e = &obj->sdt_entries[i];
+		__u32 sdt_cnt = prog_sdt[e->prog_idx].sdt_cnt;
+
+		if (!sdt_cnt)
+			continue;
+
+		prog = &obj->programs[e->prog_idx];
+
+		if (prog->sdt_map_fd < 0) {
+			int map_fd;
+			LIBBPF_OPTS(bpf_map_create_opts, map_opts);
+
+			map_opts.map_flags = BPF_F_INSN_ARRAY_SDT;
+			map_fd = bpf_map_create(BPF_MAP_TYPE_INSN_ARRAY, ".bpf_sdt_notes",
+						sizeof(key), value_size, sdt_cnt, &map_opts);
+			if (map_fd < 0) {
+				err = map_fd;
+				goto out_free;
+			}
+			prog->sdt_map_fd = map_fd;
+		}
+
+		key = prog_sdt[e->prog_idx].next_key++;
+
+		memset(&val, 0, sizeof(val));
+		val.nargs = e->nargs;
+		val.orig_off = e->insn_idx - prog->sec_insn_off;
+		memcpy(val.arg_reg, e->arg_reg, sizeof(val.arg_reg));
+
+		err = bpf_map_update_elem(prog->sdt_map_fd, &key, &val, 0);
+		if (err)
+			goto out_free;
+	}
+
+	/* freeze maps */
+	for (i = 0; i < obj->nr_programs; i++) {
+		prog = &obj->programs[i];
+		if (prog->sdt_map_fd >= 0) {
+			err = bpf_map_freeze(prog->sdt_map_fd);
+			if (err)
+				goto out_free;
+		}
+	}
+
+out_free:
+	free(prog_sdt);
+	if (err) {
+		for (i = 0; i < obj->nr_programs; i++) {
+			if (obj->programs[i].sdt_map_fd >= 0) {
+				close(obj->programs[i].sdt_map_fd);
+				obj->programs[i].sdt_map_fd = -1;
+			}
+		}
+	}
+	return err;
+}
+
 /* Relocate data references within program code:
  *  - map references;
  *  - global variable references;
@@ -7135,6 +7266,8 @@ bpf_object__reloc_code(struct bpf_object *obj, struct bpf_program *main_prog,
 			err = bpf_object__append_subprog_code(obj, main_prog, subprog);
 			if (err)
 				return err;
+			if (subprog->absorbed_by == -1)
+				subprog->absorbed_by = main_prog - obj->programs;
 			err = bpf_object__reloc_code(obj, main_prog, subprog);
 			if (err)
 				return err;
@@ -8254,6 +8387,8 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
 	load_attr.log_level = log_level;
 	load_attr.prog_flags = prog->prog_flags;
 	load_attr.fd_array = obj->fd_array;
+	if (prog->sdt_map_fd >= 0)
+		load_attr.sdt_map_fd = prog->sdt_map_fd;
 
 	load_attr.token_fd = obj->token_fd;
 	if (obj->token_fd)
@@ -9343,6 +9478,7 @@ static int bpf_object_prepare(struct bpf_object *obj, const char *target_btf_pat
 	err = err ? : bpf_object__relocate(obj, obj->btf_custom_path ? : target_btf_path);
 	err = err ? : bpf_object__sanitize_and_load_btf(obj);
 	err = err ? : bpf_object__create_maps(obj);
+	err = err ? : bpf_object__create_sdt_maps(obj);
 	err = err ? : bpf_object_prepare_progs(obj);
 
 	if (err) {
@@ -9893,8 +10029,11 @@ void bpf_object__close(struct bpf_object *obj)
 	obj->nr_maps = 0;
 
 	if (obj->programs && obj->nr_programs) {
-		for (i = 0; i < obj->nr_programs; i++)
+		for (i = 0; i < obj->nr_programs; i++) {
+			if (obj->programs[i].sdt_map_fd >= 0)
+				close(obj->programs[i].sdt_map_fd);
 			bpf_program__exit(&obj->programs[i]);
+		}
 	}
 	zfree(&obj->programs);
 
-- 
2.47.3

[RFC PATCH bpf-next 05/12] bpf: Collect SDT probe BTF IDs from BTF decl tags

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Collect SDT probe BTF IDs from BTF decl tags matching name pattern
"bpf_sdt:<name>:<nargs>". The IDs are ultimately stored in each probe
entry of the SDT map.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 include/uapi/linux/bpf.h       |  1 +
 kernel/bpf/bpf_insn_array.c    |  1 +
 tools/include/uapi/linux/bpf.h |  1 +
 tools/lib/bpf/libbpf.c         | 55 ++++++++++++++++++++++++++++++++++
 4 files changed, 58 insertions(+)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 6a03e3f0506e..773a7def0fbd 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -7739,6 +7739,7 @@ struct bpf_insn_array_value {
 	__u32 orig_off;
 	__u32 xlated_off;
 	__u32 jitted_off;
+	__u32 btf_id;		/* BTF FUNC_PROTO type ID (0 if none) */
 	__u8  nargs;		/* argument count (0..5) */
 	__u8  arg_reg[5];	/* BPF register for each argument */
 	__u8  pad[2];
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index f43777c5cd07..aca8676ab563 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -103,6 +103,7 @@ static long insn_array_update_elem(struct bpf_map *map, void *key, void *value,
 	insn_array->values[index].orig_off = val.orig_off;
 
 	if (insn_array->subtype == BPF_INSN_ARRAY_SUBTYPE_SDT) {
+		insn_array->values[index].btf_id = val.btf_id;
 		insn_array->values[index].nargs = val.nargs;
 		memcpy(insn_array->values[index].arg_reg, val.arg_reg, sizeof(val.arg_reg));
 	}
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 6a03e3f0506e..773a7def0fbd 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -7739,6 +7739,7 @@ struct bpf_insn_array_value {
 	__u32 orig_off;
 	__u32 xlated_off;
 	__u32 jitted_off;
+	__u32 btf_id;		/* BTF FUNC_PROTO type ID (0 if none) */
 	__u8  nargs;		/* argument count (0..5) */
 	__u8  arg_reg[5];	/* BPF register for each argument */
 	__u8  pad[2];
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 3f8b12a1eb8b..badac128e8ad 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -715,6 +715,7 @@ struct sdt_entry {
 	__u64 insn_idx;  /* insn index of probe site within its ELF section */
 	__u8 nargs;      /* how many arguments */
 	__u8 arg_reg[5]; /* which register the argument locates in */
+	__u32 btf_id;    /* BTF FUNC_PROTO type ID from decl tag */
 };
 
 struct bpf_object {
@@ -6692,6 +6693,57 @@ static int create_jt_map(struct bpf_object *obj, struct bpf_program *prog, struc
 	return err;
 }
 
+static void sdt_collect_btf_ids(struct bpf_object *obj)
+{
+	int n = btf__type_cnt(obj->btf);
+	const struct btf_type *t;
+	int i;
+
+	for (i = 1; i < n; i++) {
+		const char *tag, *colon;
+		size_t name_len;
+		__u32 func_proto_id;
+		int j;
+
+		t = btf_type_by_id(obj->btf, i);
+		if (!btf_is_decl_tag(t) || btf_decl_tag(t)->component_idx != -1)
+			continue;
+
+		tag = btf__str_by_offset(obj->btf, t->name_off);
+		if (strncmp(tag, "bpf_sdt:", 8) != 0)
+			continue;
+
+		/* tag format: "bpf_sdt:<name>:<nargs>"; <name> has no colons */
+		colon = strchr(tag + 8, ':');
+		if (!colon)
+			continue;
+		name_len = colon - tag - 8;
+
+		/* type chain: DECL_TAG -> VAR -> (CONST) -> PTR -> FUNC_PROTO */
+		t = btf_type_by_id(obj->btf, t->type);
+		if (!btf_is_var(t))
+			continue;
+		t = btf_type_by_id(obj->btf, t->type);
+		if (btf_kind(t) == BTF_KIND_CONST)
+			t = btf_type_by_id(obj->btf, t->type);
+		if (!btf_is_ptr(t))
+			continue;
+		func_proto_id = t->type;
+		t = btf_type_by_id(obj->btf, func_proto_id);
+		if (!btf_is_func_proto(t))
+			continue;
+
+		/* match by name (tag format: "bpf_sdt:<name>:<nargs>") */
+		for (j = 0; j < obj->sdt_entry_cnt; j++) {
+			struct sdt_entry *e = &obj->sdt_entries[j];
+
+			if (!e->btf_id && strlen(e->name) == name_len &&
+			    strncmp(tag + 8, e->name, name_len) == 0)
+				e->btf_id = func_proto_id;
+		}
+	}
+}
+
 static int bpf_object__resolve_sdt_progs(struct bpf_object *obj)
 {
 	int i, j;
@@ -6749,6 +6801,8 @@ static int bpf_object__create_sdt_maps(struct bpf_object *obj)
 	if (err)
 		return err;
 
+	sdt_collect_btf_ids(obj);
+
 	prog_sdt = calloc(obj->nr_programs, sizeof(*prog_sdt));
 	if (!prog_sdt)
 		return -ENOMEM;
@@ -6785,6 +6839,7 @@ static int bpf_object__create_sdt_maps(struct bpf_object *obj)
 		key = prog_sdt[e->prog_idx].next_key++;
 
 		memset(&val, 0, sizeof(val));
+		val.btf_id = e->btf_id;
 		val.nargs = e->nargs;
 		val.orig_off = e->insn_idx - prog->sec_insn_off;
 		memcpy(val.arg_reg, e->arg_reg, sizeof(val.arg_reg));
-- 
2.47.3

[RFC PATCH bpf-next 06/12] bpf: Add type check for SDT probe site

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

The NOP instruction at SDT probe site will be patched to a call
instruction to observer programs. To ensure the arguments passed
to observers are as expected, add type check for bpf SDT probe site.

The check for probes with no arguments always succeeds.

For probes with arguments, argument registers are checked against
the types declared by FUNC_PROTO for the probe site.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 include/linux/bpf.h          |   2 +-
 include/linux/bpf_verifier.h |   3 +
 kernel/bpf/bpf_insn_array.c  |  14 +++-
 kernel/bpf/fixups.c          |   4 ++
 kernel/bpf/liveness.c        |  24 ++++++-
 kernel/bpf/verifier.c        | 132 ++++++++++++++++++++++++++++++++++-
 6 files changed, 173 insertions(+), 6 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 7719f6528445..cb43792afdee 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -4154,7 +4154,7 @@ int bpf_prog_get_file_line(struct bpf_prog *prog, unsigned long ip, const char *
 			   const char **linep, int *nump);
 struct bpf_prog *bpf_prog_find_from_stack(void);
 
-int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog);
+int bpf_insn_array_init(struct bpf_map *map, struct bpf_verifier_env *env);
 int bpf_insn_array_ready(struct bpf_map *map);
 void bpf_insn_array_release(struct bpf_map *map);
 void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h
index 76b8b7627a10..e824b7b64690 100644
--- a/include/linux/bpf_verifier.h
+++ b/include/linux/bpf_verifier.h
@@ -691,6 +691,9 @@ struct bpf_insn_aux_data {
 	u8 fastcall_spills_num:3;
 	u8 arg_prog:4;
 
+	/* set when the instruction is a SDT probe site */
+	struct bpf_insn_array_value *sdt_entry;
+
 	/* below fields are initialized once */
 	unsigned int orig_idx; /* original instruction index */
 	u32 jmp_point:1;
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index aca8676ab563..fe30b08712ff 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -2,6 +2,7 @@
 /* Copyright (c) 2025 Isovalent */
 
 #include <linux/bpf.h>
+#include <linux/bpf_verifier.h>
 
 struct bpf_insn_array {
 	struct bpf_map map;
@@ -198,10 +199,12 @@ static inline bool valid_offsets(const struct bpf_insn_array *insn_array,
 	return true;
 }
 
-int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
+int bpf_insn_array_init(struct bpf_map *map, struct bpf_verifier_env *env)
 {
 	struct bpf_insn_array *insn_array = cast_insn_array(map);
 	struct bpf_insn_array_value *values = insn_array->values;
+	const struct bpf_prog *prog = env->prog;
+	const struct bpf_insn *insn;
 	int i;
 
 	if (!is_frozen(map))
@@ -224,6 +227,15 @@ int bpf_insn_array_init(struct bpf_map *map, const struct bpf_prog *prog)
 	for (i = 0; i < map->max_entries; i++)
 		values[i].xlated_off = values[i].orig_off;
 
+	if (insn_array->subtype == BPF_INSN_ARRAY_SUBTYPE_SDT) {
+		for (i = 0; i < map->max_entries; i++) {
+			insn = &prog->insnsi[values[i].orig_off];
+			if (insn->code != (BPF_JMP | BPF_JA) || insn->off != 0)
+				return -EINVAL;
+			env->insn_aux_data[values[i].orig_off].sdt_entry = &values[i];
+		}
+	}
+
 	return 0;
 }
 
diff --git a/kernel/bpf/fixups.c b/kernel/bpf/fixups.c
index 3cf2cc6e3ab6..4281c71cde4c 100644
--- a/kernel/bpf/fixups.c
+++ b/kernel/bpf/fixups.c
@@ -568,6 +568,10 @@ int bpf_opt_remove_nops(struct bpf_verifier_env *env)
 		if (!is_may_goto_0 && !is_ja)
 			continue;
 
+		/* SDT probes are NOPs kept for text_poke at attach time. */
+		if (env->insn_aux_data[i].sdt_entry)
+			continue;
+
 		err = verifier_remove_insns(env, i, 1);
 		if (err)
 			return err;
diff --git a/kernel/bpf/liveness.c b/kernel/bpf/liveness.c
index 0aadfbae0acc..50ab8c29c0eb 100644
--- a/kernel/bpf/liveness.c
+++ b/kernel/bpf/liveness.c
@@ -2062,6 +2062,7 @@ struct insn_live_regs {
 /* Compute info->{use,def} fields for the instruction */
 static void compute_insn_live_regs(struct bpf_verifier_env *env,
 				   struct bpf_insn *insn,
+				   int insn_idx,
 				   struct insn_live_regs *info)
 {
 	struct bpf_call_summary cs;
@@ -2163,10 +2164,27 @@ static void compute_insn_live_regs(struct bpf_verifier_env *env,
 		switch (code) {
 		case BPF_JA:
 			def = 0;
-			if (BPF_SRC(insn->code) == BPF_X)
+			if (BPF_SRC(insn->code) == BPF_X) {
 				use = dst;
-			else
+			} else if (env->insn_aux_data[insn_idx].sdt_entry) {
+				struct bpf_insn_array_value *sdt;
+				int i;
+
+				/*
+				 * Without marking the argument registers arg_reg[]
+				 * as live, the liveness pass would clear them before
+				 * the probe site, causing check_sdt_probe() to reject
+				 * the prog with "arg is uninitialized".
+				 */
 				use = 0;
+				sdt = env->insn_aux_data[insn_idx].sdt_entry;
+				for (i = 0; i < sdt->nargs; i++) {
+					if (sdt->arg_reg[i] < BPF_REG_FP)
+						use |= BIT(sdt->arg_reg[i]);
+				}
+			} else {
+				use = 0;
+			}
 			break;
 		case BPF_JCOND:
 			def = 0;
@@ -2238,7 +2256,7 @@ int bpf_compute_live_registers(struct bpf_verifier_env *env)
 	}
 
 	for (i = 0; i < insn_cnt; ++i)
-		compute_insn_live_regs(env, &insns[i], &state[i]);
+		compute_insn_live_regs(env, &insns[i], i, &state[i]);
 
 	/* Forward pass: resolve stack access through FP-derived pointers */
 	err = bpf_compute_subprog_arg_access(env);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 05734163650a..bc972beb80cf 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -17242,6 +17242,133 @@ static int check_indirect_jump(struct bpf_verifier_env *env, struct bpf_insn *in
 	return INSN_IDX_UPDATED;
 }
 
+static int check_sdt_probe(struct bpf_verifier_env *env, int insn_idx)
+{
+	enum bpf_prog_type prog_type = resolve_prog_type(env->prog);
+	struct bpf_insn_array_value *val;
+	struct bpf_func_state *frame;
+	struct bpf_reg_state *regs;
+	const struct btf *btf;
+	const struct btf_type *proto;
+	const struct btf_param *args;
+	int i, nargs;
+
+	val = env->insn_aux_data[insn_idx].sdt_entry;
+	if (!val->nargs)
+		return 0;
+
+	if (val->nargs > MAX_BPF_FUNC_REG_ARGS) {
+		verbose(env, "SDT probe nargs %u > 5\n", val->nargs);
+		return -EINVAL;
+	}
+
+	frame = env->cur_state->frame[env->cur_state->curframe];
+	regs = frame->regs;
+
+	for (i = 0; i < val->nargs; i++) {
+		u8 reg = val->arg_reg[i];
+		struct bpf_reg_state *rs = &regs[reg];
+
+		if (rs->type == NOT_INIT) {
+			verbose(env, "SDT arg%d (r%d) is uninitialized\n", i, reg);
+			return -EINVAL;
+		}
+	}
+
+	btf = env->prog->aux->btf;
+	if (!btf || !val->btf_id) {
+		verbose(env, "BTF is required for SDT probe with %u arguments\n", val->nargs);
+		return -EINVAL;
+	}
+
+	proto = btf_type_by_id(btf, val->btf_id);
+	if (!proto || !btf_type_is_func_proto(proto)) {
+		verbose(env, "SDT btf_id %u is not a FUNC_PROTO\n", val->btf_id);
+		return -EINVAL;
+	}
+
+	nargs = btf_type_vlen(proto);
+	if (nargs != val->nargs) {
+		verbose(env, "SDT nargs %u != BTF FUNC_PROTO nargs %d\n",
+			val->nargs, nargs);
+		return -EINVAL;
+	}
+
+	args = (const struct btf_param *)(proto + 1);
+	for (i = 0; i < nargs; i++) {
+		u8 reg = val->arg_reg[i];
+		struct bpf_reg_state *rs = &regs[reg];
+		const struct btf_type *t;
+		u32 arg_btf_id;
+		u32 t_size;
+
+		t = btf_type_skip_modifiers(btf, args[i].type, NULL);
+		t_size = t ? t->size : 0;
+
+		if (btf_type_is_scalar(t)) {
+			if (base_type(rs->type) != SCALAR_VALUE) {
+				verbose(env, "SDT arg%d (r%d) type %s expected scalar\n",
+					i, reg, reg_type_str(env, rs->type));
+				return -EACCES;
+			}
+			continue;
+		}
+
+		/*
+		 * Small structs/unions (<= 8 bytes) are passed by value in
+		 * a register as a SCALAR_VALUE carrying the raw bytes.  The
+		 * BPF_SDT_PROBE<N> macro enforces sizeof(arg) <= 8 at build
+		 * time, so the FUNC_PROTO parameter type is the struct itself
+		 * (not a pointer) and the verifier sees SCALAR_VALUE.
+		 */
+		if (btf_type_is_struct(t) && t_size <= 8) {
+			if (base_type(rs->type) != SCALAR_VALUE) {
+				verbose(env,
+				"SDT arg%d (r%d) type %s expected scalar (small struct by value)\n",
+					i, reg, reg_type_str(env, rs->type));
+				return -EACCES;
+			}
+			continue;
+		}
+
+		if (!btf_type_is_ptr(t)) {
+			verbose(env, "SDT arg%d (r%d) unsupported BTF parameter kind\n",
+				i, reg);
+			return -EACCES;
+		}
+
+		/*
+		 * When the probe argument is the target program's context
+		 * type (e.g. struct xdp_md * for XDP), the register at the
+		 * probe site is PTR_TO_CTX, not PTR_TO_BTF_ID.
+		 */
+		if (base_type(rs->type) == PTR_TO_CTX &&
+		    btf_is_prog_ctx_type(&env->log, btf, t, prog_type, i))
+			continue;
+
+		t = btf_type_skip_modifiers(btf, t->type, &arg_btf_id);
+		if (!btf_type_is_struct(t)) {
+			verbose(env, "SDT arg%d (r%d) unsupported BTF pointer target\n",
+				i, reg);
+			return -EACCES;
+		}
+
+		if (base_type(rs->type) != PTR_TO_BTF_ID) {
+			verbose(env, "SDT arg%d (r%d) type %s expected PTR_TO_BTF_ID\n",
+				i, reg, reg_type_str(env, rs->type));
+			return -EACCES;
+		}
+
+		if (!btf_struct_ids_match(&env->log, rs->btf, rs->btf_id,
+					  rs->var_off.value, btf, arg_btf_id, false)) {
+			verbose(env, "SDT arg%d (r%d) btf_id %u does not match expected %u\n",
+				i, reg, rs->btf_id, arg_btf_id);
+			return -EACCES;
+		}
+	}
+	return 0;
+}
+
 static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
 {
 	int err;
@@ -17317,6 +17444,9 @@ static int do_check_insn(struct bpf_verifier_env *env, bool *do_print_state)
 			if (BPF_SRC(insn->code) == BPF_X)
 				return check_indirect_jump(env, insn);
 
+			if (env->insn_aux_data[env->insn_idx].sdt_entry)
+				return check_sdt_probe(env, env->insn_idx);
+
 			if (class == BPF_JMP)
 				env->insn_idx += insn->off + 1;
 			else
@@ -17882,7 +18012,7 @@ static int __add_used_map(struct bpf_verifier_env *env, struct bpf_map *map)
 	env->used_maps[env->used_map_cnt++] = map;
 
 	if (map->map_type == BPF_MAP_TYPE_INSN_ARRAY) {
-		err = bpf_insn_array_init(map, env->prog);
+		err = bpf_insn_array_init(map, env);
 		if (err) {
 			verbose(env, "Failed to properly initialize insn array\n");
 			return err;
-- 
2.47.3

[RFC PATCH bpf-next 07/12] bpf: Record probe name in SDT map

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

When an SDT observer program is loaded, kernel needs to know the target
program and the target probe site. As user specifies probe site by probe
name, record probe name in SDT map for kernel to find the probe.

The max probe name length is set to 64. Programs with probe name longer
than 64 will be denied by libbpf.

Assisted-by: OpenCode:GLM-5.2
Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 include/uapi/linux/bpf.h       | 2 ++
 kernel/bpf/bpf_insn_array.c    | 2 ++
 tools/include/uapi/linux/bpf.h | 2 ++
 tools/lib/bpf/libbpf.c         | 7 +++++++
 4 files changed, 13 insertions(+)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 773a7def0fbd..95ca41bf7501 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1521,6 +1521,7 @@ struct bpf_common_attr {
 };
 
 #define BPF_OBJ_NAME_LEN 16U
+#define BPF_SDT_MAX_NAME_LEN 64
 
 enum {
 	BPF_STREAM_STDOUT = 1,
@@ -7743,6 +7744,7 @@ struct bpf_insn_array_value {
 	__u8  nargs;		/* argument count (0..5) */
 	__u8  arg_reg[5];	/* BPF register for each argument */
 	__u8  pad[2];
+	char  name[BPF_SDT_MAX_NAME_LEN]; /* probe name */
 };
 
 #endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index fe30b08712ff..a9564ef4b2ef 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -3,6 +3,7 @@
 
 #include <linux/bpf.h>
 #include <linux/bpf_verifier.h>
+#include <linux/string.h>
 
 struct bpf_insn_array {
 	struct bpf_map map;
@@ -107,6 +108,7 @@ static long insn_array_update_elem(struct bpf_map *map, void *key, void *value,
 		insn_array->values[index].btf_id = val.btf_id;
 		insn_array->values[index].nargs = val.nargs;
 		memcpy(insn_array->values[index].arg_reg, val.arg_reg, sizeof(val.arg_reg));
+		strscpy(insn_array->values[index].name, val.name, sizeof(val.name));
 	}
 
 	return 0;
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 773a7def0fbd..95ca41bf7501 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1521,6 +1521,7 @@ struct bpf_common_attr {
 };
 
 #define BPF_OBJ_NAME_LEN 16U
+#define BPF_SDT_MAX_NAME_LEN 64
 
 enum {
 	BPF_STREAM_STDOUT = 1,
@@ -7743,6 +7744,7 @@ struct bpf_insn_array_value {
 	__u8  nargs;		/* argument count (0..5) */
 	__u8  arg_reg[5];	/* BPF register for each argument */
 	__u8  pad[2];
+	char  name[BPF_SDT_MAX_NAME_LEN]; /* probe name */
 };
 
 #endif /* _UAPI__LINUX_BPF_H__ */
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index badac128e8ad..dbc08a193101 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -4146,6 +4146,12 @@ static int bpf_object__collect_sdt_notes(struct bpf_object *obj)
 			err = -ENOMEM;
 			goto out;
 		}
+		if (strlen(e->name) >= BPF_SDT_MAX_NAME_LEN) {
+			pr_warn("sdt: probe name '%s' too long (max %d chars)\n",
+				e->name, BPF_SDT_MAX_NAME_LEN - 1);
+			err = -EINVAL;
+			goto out;
+		}
 		e->sec_idx = sec_idx;
 		e->insn_idx = nop_idx;
 		e->nargs = nargs;
@@ -6843,6 +6849,7 @@ static int bpf_object__create_sdt_maps(struct bpf_object *obj)
 		val.nargs = e->nargs;
 		val.orig_off = e->insn_idx - prog->sec_insn_off;
 		memcpy(val.arg_reg, e->arg_reg, sizeof(val.arg_reg));
+		strncpy(val.name, e->name, sizeof(val.name) - 1);
 
 		err = bpf_map_update_elem(prog->sdt_map_fd, &key, &val, 0);
 		if (err)
-- 
2.47.3

[RFC PATCH bpf-next 08/12] libbpf: Add libbpf support to load SDT observer program

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Add libbpf support to load SDT observer program. To ensure the
verifier can find which probe the observer program is attached,
the target bpf prog fd and probe name are passed to the kernel.
Kernel looks up probe against the target program's SDT map using
the probe name.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 include/uapi/linux/bpf.h       |  7 +++++++
 tools/include/uapi/linux/bpf.h |  7 +++++++
 tools/lib/bpf/bpf.c            |  9 ++++++++-
 tools/lib/bpf/bpf.h            |  7 ++++++-
 tools/lib/bpf/libbpf.c         | 20 ++++++++++++++++++++
 5 files changed, 48 insertions(+), 2 deletions(-)

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 95ca41bf7501..6490eb5dfb68 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1159,6 +1159,7 @@ enum bpf_attach_type {
 	BPF_TRACE_FENTRY_MULTI,
 	BPF_TRACE_FEXIT_MULTI,
 	BPF_TRACE_FSESSION_MULTI,
+	BPF_TRACE_SDT,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1184,6 +1185,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_NETKIT = 13,
 	BPF_LINK_TYPE_SOCKMAP = 14,
 	BPF_LINK_TYPE_TRACING_MULTI = 15,
+	BPF_LINK_TYPE_SDT,
 	__MAX_BPF_LINK_TYPE,
 };
 
@@ -1677,6 +1679,11 @@ union bpf_attr {
 		 * BPF_F_INSN_ARRAY_SDT, used for SDT probe
 		 */
 		__u32		sdt_map_fd;
+		/* Kernel looks up the probe using target prog and probe name */
+		struct {
+			__u32	target_prog_fd;
+			char	name[BPF_SDT_MAX_NAME_LEN]; /* probe name */
+		} sdt;
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 95ca41bf7501..6490eb5dfb68 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -1159,6 +1159,7 @@ enum bpf_attach_type {
 	BPF_TRACE_FENTRY_MULTI,
 	BPF_TRACE_FEXIT_MULTI,
 	BPF_TRACE_FSESSION_MULTI,
+	BPF_TRACE_SDT,
 	__MAX_BPF_ATTACH_TYPE
 };
 
@@ -1184,6 +1185,7 @@ enum bpf_link_type {
 	BPF_LINK_TYPE_NETKIT = 13,
 	BPF_LINK_TYPE_SOCKMAP = 14,
 	BPF_LINK_TYPE_TRACING_MULTI = 15,
+	BPF_LINK_TYPE_SDT,
 	__MAX_BPF_LINK_TYPE,
 };
 
@@ -1677,6 +1679,11 @@ union bpf_attr {
 		 * BPF_F_INSN_ARRAY_SDT, used for SDT probe
 		 */
 		__u32		sdt_map_fd;
+		/* Kernel looks up the probe using target prog and probe name */
+		struct {
+			__u32	target_prog_fd;
+			char	name[BPF_SDT_MAX_NAME_LEN]; /* probe name */
+		} sdt;
 	};
 
 	struct { /* anonymous struct used by BPF_OBJ_* commands */
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index cb006bca97c6..adf951df6a3e 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -295,7 +295,7 @@ int bpf_prog_load(enum bpf_prog_type prog_type,
 		  const struct bpf_insn *insns, size_t insn_cnt,
 		  struct bpf_prog_load_opts *opts)
 {
-	const size_t attr_sz = offsetofend(union bpf_attr, sdt_map_fd);
+	const size_t attr_sz = offsetofend(union bpf_attr, sdt.name);
 	void *finfo = NULL, *linfo = NULL;
 	const char *func_info, *line_info;
 	__u32 log_size, log_level, attach_prog_fd, attach_btf_obj_fd;
@@ -303,6 +303,7 @@ int bpf_prog_load(enum bpf_prog_type prog_type,
 	int fd, attempts;
 	union bpf_attr attr;
 	char *log_buf;
+	const char *sdt_name;
 
 	bump_rlimit_memlock();
 
@@ -370,6 +371,11 @@ int bpf_prog_load(enum bpf_prog_type prog_type,
 	attr.fd_array = ptr_to_u64(OPTS_GET(opts, fd_array, NULL));
 	attr.fd_array_cnt = OPTS_GET(opts, fd_array_cnt, 0);
 	attr.sdt_map_fd = OPTS_GET(opts, sdt_map_fd, 0);
+	attr.sdt.target_prog_fd = OPTS_GET(opts, sdt.target_prog_fd, 0);
+
+	sdt_name = OPTS_GET(opts, sdt.name, NULL);
+	if (sdt_name)
+		snprintf(attr.sdt.name, sizeof(attr.sdt.name), "%s", sdt_name);
 
 	if (log_level) {
 		attr.log_buf = ptr_to_u64(log_buf);
@@ -862,6 +868,7 @@ int bpf_link_create(int prog_fd, int target_fd,
 	case BPF_MODIFY_RETURN:
 	case BPF_TRACE_FSESSION:
 	case BPF_LSM_MAC:
+	case BPF_TRACE_SDT:
 		attr.link_create.tracing.cookie = OPTS_GET(opts, tracing.cookie, 0);
 		if (!OPTS_ZEROED(opts, tracing))
 			return libbpf_err(-EINVAL);
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 88294bb6b120..bf3665789420 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -130,9 +130,14 @@ struct bpf_prog_load_opts {
 	__u32 fd_array_cnt;
 	/* if set, FD of the program's BPF_MAP_TYPE_INSN_ARRAY SDT map */
 	__u32 sdt_map_fd;
+	/* target program fd and probe name for SDT observer program */
+	struct {
+		__u32 target_prog_fd;
+		const char *name;
+	} sdt;
 	size_t :0;
 };
-#define bpf_prog_load_opts__last_field sdt_map_fd
+#define bpf_prog_load_opts__last_field sdt.name
 
 LIBBPF_API int bpf_prog_load(enum bpf_prog_type prog_type,
 			     const char *prog_name, const char *license,
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index dbc08a193101..273991b80f8f 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -139,6 +139,7 @@ static const char * const attach_type_name[] = {
 	[BPF_TRACE_FENTRY_MULTI]	= "trace_fentry_multi",
 	[BPF_TRACE_FEXIT_MULTI]		= "trace_fexit_multi",
 	[BPF_TRACE_FSESSION_MULTI]	= "trace_fsession_multi",
+	[BPF_TRACE_SDT]			= "trace_sdt",
 };
 
 static const char * const link_type_name[] = {
@@ -158,6 +159,7 @@ static const char * const link_type_name[] = {
 	[BPF_LINK_TYPE_NETKIT]			= "netkit",
 	[BPF_LINK_TYPE_SOCKMAP]			= "sockmap",
 	[BPF_LINK_TYPE_TRACING_MULTI]		= "tracing_multi",
+	[BPF_LINK_TYPE_SDT]			= "sdt",
 };
 
 static const char * const map_type_name[] = {
@@ -507,6 +509,7 @@ struct bpf_program {
 	__u32 attach_btf_obj_fd;
 	__u32 attach_btf_id;
 	__u32 attach_prog_fd;
+	char *sdt_probe_name;
 
 	void *func_info;
 	__u32 func_info_rec_size;
@@ -836,6 +839,7 @@ static void bpf_program__exit(struct bpf_program *prog)
 	zfree(&prog->sec_name);
 	zfree(&prog->insns);
 	zfree(&prog->reloc_desc);
+	zfree(&prog->sdt_probe_name);
 
 	prog->nr_reloc = 0;
 	prog->insns_cnt = 0;
@@ -8451,6 +8455,10 @@ static int bpf_object_load_prog(struct bpf_object *obj, struct bpf_program *prog
 	load_attr.fd_array = obj->fd_array;
 	if (prog->sdt_map_fd >= 0)
 		load_attr.sdt_map_fd = prog->sdt_map_fd;
+	if (prog->expected_attach_type == BPF_TRACE_SDT && prog->sdt_probe_name) {
+		load_attr.sdt.target_prog_fd = prog->attach_prog_fd;
+		load_attr.sdt.name = prog->sdt_probe_name;
+	}
 
 	load_attr.token_fd = obj->token_fd;
 	if (obj->token_fd)
@@ -10587,6 +10595,7 @@ static const struct bpf_sec_def section_defs[] = {
 	SEC_DEF("kretsyscall+",		KPROBE, 0, SEC_NONE, attach_ksyscall),
 	SEC_DEF("usdt+",		KPROBE,	0, SEC_USDT, attach_usdt),
 	SEC_DEF("usdt.s+",		KPROBE,	0, SEC_USDT | SEC_SLEEPABLE, attach_usdt),
+	SEC_DEF("bpf_sdt",		TRACING, BPF_TRACE_SDT, SEC_NONE),
 	SEC_DEF("tc/ingress",		SCHED_CLS, BPF_TCX_INGRESS, SEC_NONE), /* alias for tcx */
 	SEC_DEF("tc/egress",		SCHED_CLS, BPF_TCX_EGRESS, SEC_NONE),  /* alias for tcx */
 	SEC_DEF("tcx/ingress",		SCHED_CLS, BPF_TCX_INGRESS, SEC_NONE),
@@ -15203,6 +15212,17 @@ int bpf_program__set_attach_target(struct bpf_program *prog,
 	if (prog->obj->state >= OBJ_LOADED)
 		return libbpf_err(-EINVAL);
 
+	if (prog->expected_attach_type == BPF_TRACE_SDT) {
+		if (!attach_func_name)
+			return libbpf_err(-EINVAL);
+		free(prog->sdt_probe_name);
+		prog->sdt_probe_name = strdup(attach_func_name);
+		if (!prog->sdt_probe_name)
+			return libbpf_err(-ENOMEM);
+		prog->attach_prog_fd = attach_prog_fd;
+		return 0;
+	}
+
 	if (attach_prog_fd && !attach_func_name) {
 		/* Store attach_prog_fd. The BTF ID will be resolved later during
 		 * the normal object/program load phase.
-- 
2.47.3

[RFC PATCH bpf-next 09/12] bpf: Add kernel support to load SDT observer program

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Add kernel support to load SDT observer program. The program is
verified as a normal tracing type program while the target FUNC_PROTO
is resolved in the SDT map of the target program using the probe name
passed by the user. The probe data is preserved in prog->aux->sdt_probe,
so that the later attach can find the target probe information easily.

Also introduce a returning false weak function bpf_jit_supports_sdt_probe()
to hint if SDT probe is supported by the JIT.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 include/linux/bpf.h         | 15 +++++++
 include/linux/bpf_types.h   |  1 +
 include/linux/filter.h      |  1 +
 kernel/bpf/bpf_insn_array.c | 28 +++++++++++++
 kernel/bpf/core.c           |  5 +++
 kernel/bpf/syscall.c        | 78 ++++++++++++++++++++++++++++++-------
 kernel/bpf/verifier.c       | 19 +++++++++
 7 files changed, 134 insertions(+), 13 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index cb43792afdee..93c196f38b87 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1220,6 +1220,8 @@ struct btf_func_model {
 	u8 nr_args;
 	u8 arg_size[MAX_BPF_FUNC_ARGS];
 	u8 arg_flags[MAX_BPF_FUNC_ARGS];
+	/* argument registers for bpf SDT probe site */
+	u8 arg_regs[MAX_BPF_FUNC_REG_ARGS];
 };
 
 /* Restore arguments before returning from trampoline to let original function
@@ -1419,6 +1421,15 @@ struct bpf_attach_target_info {
 	const struct btf_type *tgt_type;
 };
 
+/*
+ * cached in prog->aux, used by the verifier to type the observer's context
+ * arguments, and by link_create to build the trampoline without re-resolving
+ */
+struct bpf_sdt_probe_info {
+	struct bpf_insn_array_value val;
+	unsigned long probe_ip;
+};
+
 #define BPF_DISPATCHER_MAX 48 /* Fits in 2048B */
 
 struct bpf_dispatcher_prog {
@@ -1762,6 +1773,7 @@ struct bpf_prog_aux {
 	void __percpu *priv_stack_ptr;
 	struct mutex dst_mutex; /* protects dst_* pointers below, *after* prog becomes visible */
 	struct bpf_prog *dst_prog;
+	struct bpf_sdt_probe_info *sdt_probe;
 	struct bpf_trampoline *dst_trampoline;
 	enum bpf_prog_type saved_dst_prog_type;
 	enum bpf_attach_type saved_dst_attach_type;
@@ -4159,6 +4171,9 @@ int bpf_insn_array_ready(struct bpf_map *map);
 void bpf_insn_array_release(struct bpf_map *map);
 void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len);
 void bpf_insn_array_adjust_after_remove(struct bpf_map *map, u32 off, u32 len);
+int bpf_insn_array_get_sdt_probe_by_name(struct bpf_prog *prog, const char *name,
+					 struct bpf_insn_array_value *val,
+					 unsigned long *ip);
 
 #ifdef CONFIG_BPF_SYSCALL
 void bpf_prog_update_insn_ptrs(struct bpf_prog *prog, u32 *offsets, void *image);
diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
index e5906829aa6f..1aac6bc71c6f 100644
--- a/include/linux/bpf_types.h
+++ b/include/linux/bpf_types.h
@@ -157,3 +157,4 @@ BPF_LINK_TYPE(BPF_LINK_TYPE_KPROBE_MULTI, kprobe_multi)
 BPF_LINK_TYPE(BPF_LINK_TYPE_STRUCT_OPS, struct_ops)
 BPF_LINK_TYPE(BPF_LINK_TYPE_UPROBE_MULTI, uprobe_multi)
 BPF_LINK_TYPE(BPF_LINK_TYPE_TRACING_MULTI, tracing_multi)
+BPF_LINK_TYPE(BPF_LINK_TYPE_SDT, sdt)
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 67d337ede91b..ba4025160a6f 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1190,6 +1190,7 @@ bool bpf_jit_supports_insn(struct bpf_insn *insn, bool in_arena);
 bool bpf_jit_supports_private_stack(void);
 bool bpf_jit_supports_timed_may_goto(void);
 bool bpf_jit_supports_fsession(void);
+bool bpf_jit_supports_sdt_probe(void);
 u64 bpf_arch_uaddress_limit(void);
 void arch_bpf_stack_walk(bool (*consume_fn)(void *cookie, u64 ip, u64 sp, u64 bp), void *cookie);
 u64 arch_bpf_timed_may_goto(void);
diff --git a/kernel/bpf/bpf_insn_array.c b/kernel/bpf/bpf_insn_array.c
index a9564ef4b2ef..067cc1b817d5 100644
--- a/kernel/bpf/bpf_insn_array.c
+++ b/kernel/bpf/bpf_insn_array.c
@@ -263,6 +263,34 @@ void bpf_insn_array_release(struct bpf_map *map)
 	atomic_set(&insn_array->used, 0);
 }
 
+int bpf_insn_array_get_sdt_probe_by_name(struct bpf_prog *prog, const char *name,
+					 struct bpf_insn_array_value *val,
+					 unsigned long *ip)
+{
+	int i, j;
+	struct bpf_map *map;
+	struct bpf_insn_array *insn_array;
+
+	for (i = 0; i < prog->aux->used_map_cnt; i++) {
+		map = prog->aux->used_maps[i];
+		if (map->map_type != BPF_MAP_TYPE_INSN_ARRAY ||
+		    !(map->map_flags & BPF_F_INSN_ARRAY_SDT))
+			continue;
+		insn_array = cast_insn_array(map);
+		for (j = 0; j < map->max_entries; j++) {
+			if (insn_array->values[j].xlated_off == INSN_DELETED)
+				continue;
+			if (!strcmp(insn_array->values[j].name, name)) {
+				*val = insn_array->values[j];
+				*ip = (unsigned long)insn_array->ips[j];
+				return 0;
+			}
+		}
+	}
+
+	return -EEXIST;
+}
+
 void bpf_insn_array_adjust(struct bpf_map *map, u32 off, u32 len)
 {
 	struct bpf_insn_array *insn_array = cast_insn_array(map);
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 649cce41e13f..78c7ad3b4e2f 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -3263,6 +3263,11 @@ bool __weak bpf_jit_supports_fsession(void)
 	return false;
 }
 
+bool __weak bpf_jit_supports_sdt_probe(void)
+{
+	return false;
+}
+
 u64 __weak bpf_arch_uaddress_limit(void)
 {
 #if defined(CONFIG_64BIT) && defined(CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE)
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index dc881e5ad411..8b2c73bb6c2a 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2454,6 +2454,7 @@ static void __bpf_prog_put_noref(struct bpf_prog *prog, bool deferred)
 	kvfree(prog->aux->linfo);
 	kfree(prog->aux->kfunc_tab);
 	kfree(prog->aux->ctx_arg_info);
+	kfree(prog->aux->sdt_probe);
 	if (prog->aux->attach_btf)
 		btf_put(prog->aux->attach_btf);
 
@@ -2967,13 +2968,15 @@ int __init __used bpf_multi_func(void) { return 0; }
 BTF_ID_LIST_GLOBAL_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func)
 
 /* last field in 'union bpf_attr' used by this command */
-#define BPF_PROG_LOAD_LAST_FIELD sdt_map_fd
+#define BPF_PROG_LOAD_LAST_FIELD sdt.name
 
 static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_attr *attr_log)
 {
 	enum bpf_prog_type type = attr->prog_type;
+	struct bpf_sdt_probe_info *prog_sdt_probe = NULL;
 	struct bpf_prog *prog, *dst_prog = NULL;
 	struct btf *attach_btf = NULL;
+	u32 attach_btf_id = 0;
 	struct bpf_token *token = NULL;
 	bool bpf_cap;
 	int err;
@@ -3082,33 +3085,75 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_at
 		btf_get(attach_btf);
 	}
 
+	/*
+	 * find the probe site in the target program at load time of the
+	 * observer program, so the verifier can check the observer's context
+	 * arguments from the probe's FUNC_PROTO.
+	 */
+	if (attr->expected_attach_type == BPF_TRACE_SDT) {
+		struct bpf_insn_array_value sdt_val;
+		struct bpf_sdt_probe_info *sdt_probe;
+		unsigned long probe_ip;
+
+		if (!bpf_jit_supports_sdt_probe()) {
+			err = -EOPNOTSUPP;
+			goto put_token;
+		}
+		if (!attr->sdt.target_prog_fd) {
+			err = -EINVAL;
+			goto put_token;
+		}
+		dst_prog = bpf_prog_get(attr->sdt.target_prog_fd);
+		if (IS_ERR(dst_prog)) {
+			err = PTR_ERR(dst_prog);
+			dst_prog = NULL;
+			goto put_token;
+		}
+		err = bpf_insn_array_get_sdt_probe_by_name(dst_prog, attr->sdt.name,
+							   &sdt_val, &probe_ip);
+		if (err)
+			goto put_sdt;
+		sdt_probe = kzalloc_obj(struct bpf_sdt_probe_info, GFP_USER);
+		if (!sdt_probe) {
+			err = -ENOMEM;
+			goto put_sdt;
+		}
+		sdt_probe->val = sdt_val;
+		sdt_probe->probe_ip = probe_ip;
+		prog_sdt_probe = sdt_probe;
+		attach_btf_id = sdt_val.btf_id;
+	}
+
+
 	if (bpf_prog_load_check_attach(type, attr->expected_attach_type,
-				       attach_btf, attr->attach_btf_id,
+				       attach_btf,
+				       prog_sdt_probe ? attach_btf_id : attr->attach_btf_id,
 				       dst_prog, multi_func)) {
-		if (dst_prog)
-			bpf_prog_put(dst_prog);
-		if (attach_btf)
-			btf_put(attach_btf);
 		err = -EINVAL;
-		goto put_token;
+		goto put_sdt;
 	}
 
 	/* plain bpf_prog allocation */
 	prog = bpf_prog_alloc(bpf_prog_size(attr->insn_cnt), GFP_USER);
 	if (!prog) {
-		if (dst_prog)
-			bpf_prog_put(dst_prog);
-		if (attach_btf)
-			btf_put(attach_btf);
 		err = -EINVAL;
-		goto put_token;
+		goto put_sdt;
 	}
 
 	prog->expected_attach_type = attr->expected_attach_type;
 	prog->sleepable = !!(attr->prog_flags & BPF_F_SLEEPABLE);
 	prog->aux->attach_btf = attach_btf;
-	prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id;
+	if (prog_sdt_probe)
+		prog->aux->attach_btf_id = attach_btf_id;
+	else
+		prog->aux->attach_btf_id =
+			multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id;
 	prog->aux->dst_prog = dst_prog;
+	prog->aux->sdt_probe = prog_sdt_probe;
+	/* ownership of dst_prog/attach_btf/prog_sdt_probe moved to prog->aux */
+	dst_prog = NULL;
+	attach_btf = NULL;
+	prog_sdt_probe = NULL;
 	prog->aux->dev_bound = !!attr->prog_ifindex;
 	prog->aux->xdp_has_frags = attr->prog_flags & BPF_F_XDP_HAS_FRAGS;
 
@@ -3242,6 +3287,12 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, struct bpf_log_at
 	if (prog->aux->attach_btf)
 		btf_put(prog->aux->attach_btf);
 	bpf_prog_free(prog);
+put_sdt:
+	kfree(prog_sdt_probe);
+	if (attach_btf)
+		btf_put(attach_btf);
+	if (dst_prog)
+		bpf_prog_put(dst_prog);
 put_token:
 	bpf_token_put(token);
 	return err;
@@ -4507,6 +4558,7 @@ attach_type_to_prog_type(enum bpf_attach_type attach_type)
 	case BPF_TRACE_FENTRY_MULTI:
 	case BPF_TRACE_FEXIT_MULTI:
 	case BPF_MODIFY_RETURN:
+	case BPF_TRACE_SDT:
 		return BPF_PROG_TYPE_TRACING;
 	case BPF_LSM_MAC:
 		return BPF_PROG_TYPE_LSM;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index bc972beb80cf..a875d64a0950 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -16451,6 +16451,7 @@ static bool return_retval_range(struct bpf_verifier_env *env, struct bpf_retval_
 			break;
 		case BPF_TRACE_RAW_TP:
 		case BPF_MODIFY_RETURN:
+		case BPF_TRACE_SDT:
 			return false;
 		case BPF_TRACE_ITER:
 		default:
@@ -19458,6 +19459,24 @@ static int check_attach_btf_id(struct bpf_verifier_env *env)
 	    prog->type != BPF_PROG_TYPE_EXT)
 		return 0;
 
+	if (prog->expected_attach_type == BPF_TRACE_SDT) {
+		const struct btf_type *sdt_t;
+		struct btf *sdt_btf = tgt_prog ? tgt_prog->aux->btf : NULL;
+
+		if (!sdt_btf) {
+			verbose(env, "SDT observer requires a target program with BTF\n");
+			return -EINVAL;
+		}
+		sdt_t = btf_type_by_id(sdt_btf, btf_id);
+		if (!sdt_t || !btf_type_is_func_proto(sdt_t)) {
+			verbose(env, "SDT attach_btf_id %u is not a FUNC_PROTO\n", btf_id);
+			return -EINVAL;
+		}
+		prog->aux->attach_func_proto = sdt_t;
+		prog->aux->attach_func_name = "sdt_probe";
+		return 0;
+	}
+
 	ret = bpf_check_attach_target(&env->log, prog, tgt_prog, btf_id, &tgt_info);
 	if (ret)
 		return ret;
-- 
2.47.3

[RFC PATCH bpf-next 10/12] bpf: Support attach and detach for SDT observer program

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Add support to attach SDT observer program by reusing the bpf trampoline
attach and detach mechanism. The trampoline is keyed by target program
fd and offset of the probe in the target program. Unlike normal tracing
program, the target function address is the jited address of the probe site,
and function prototype is resolved from prog->aux->sdt_probe.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 include/linux/bpf.h     |  9 ++++-
 kernel/bpf/syscall.c    | 85 +++++++++++++++++++++++++++++++++++++++++
 kernel/bpf/trampoline.c |  4 ++
 3 files changed, 97 insertions(+), 1 deletion(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 93c196f38b87..6548a2dd76da 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1270,6 +1270,12 @@ struct btf_func_model {
  */
 #define BPF_TRAMP_F_INDIRECT		BIT(8)
 
+/* Trampoline is for an SDT probe site: arguments arrive in
+ * BPF registers recorded by the SDT macro at build time,
+ * described by btf_func_model.arg_regs[] instead of ABI order.
+ */
+#define BPF_TRAMP_F_SDT_PROBE		BIT(9)
+
 /* Each call __bpf_prog_enter + call bpf_func + call __bpf_prog_exit is ~50
  * bytes on x86.
  */
@@ -1393,7 +1399,8 @@ struct bpf_trampoline {
 	struct {
 		struct btf_func_model model;
 		void *addr;
-		bool ftrace_managed;
+		u8 ftrace_managed:1;
+		u8 sdt_probe_site:1;
 	} func;
 	/* if !NULL this is BPF_PROG_TYPE_EXT program that extends another BPF
 	 * program by replacing one of its functions. func.addr is the address
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 8b2c73bb6c2a..53897ff1100f 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -3737,6 +3737,89 @@ static const struct bpf_link_ops bpf_tracing_link_lops = {
 	.fill_link_info = bpf_tracing_link_fill_link_info,
 };
 
+struct bpf_sdt_link {
+	struct bpf_tramp_link link;
+	struct bpf_trampoline *trampoline;
+};
+
+static void bpf_sdt_link_release(struct bpf_link *link)
+{
+	struct bpf_sdt_link *sdt_link = container_of(link, struct bpf_sdt_link, link.link);
+	struct bpf_prog *prog = sdt_link->link.link.prog;
+	struct bpf_prog *tgt_prog = prog->aux->dst_prog;
+
+	WARN_ON_ONCE(bpf_trampoline_unlink_prog(&sdt_link->link.node, sdt_link->trampoline,
+						tgt_prog));
+	bpf_trampoline_put(sdt_link->trampoline);
+}
+
+static void bpf_sdt_link_dealloc(struct bpf_link *link)
+{
+	struct bpf_sdt_link *sdt_link = container_of(link, struct bpf_sdt_link, link.link);
+
+	kfree(sdt_link);
+}
+
+static const struct bpf_link_ops bpf_sdt_link_lops = {
+	.release = bpf_sdt_link_release,
+	.dealloc = bpf_sdt_link_dealloc,
+};
+
+static int bpf_sdt_link_attach(struct bpf_prog *prog)
+{
+	struct bpf_sdt_probe_info *sdt = prog->aux->sdt_probe;
+	struct bpf_attach_target_info tgt_info = {};
+	struct bpf_link_primer primer;
+	struct bpf_sdt_link *sdt_link;
+	struct bpf_prog *tgt_prog = prog->aux->dst_prog;
+	u64 tr_key;
+	int i, err;
+
+	if (!sdt || !tgt_prog)
+		return -EINVAL;
+
+	sdt_link = kzalloc_obj(struct bpf_sdt_link, GFP_USER);
+	if (!sdt_link)
+		return -ENOMEM;
+
+	bpf_tramp_link_init(&sdt_link->link, BPF_LINK_TYPE_SDT, &bpf_sdt_link_lops, prog,
+			    BPF_TRACE_SDT, 0);
+	err = bpf_link_prime(&sdt_link->link.link, &primer);
+	if (err) {
+		kfree(sdt_link);
+		return err;
+	}
+
+	/* use orig_off to compute the key as orig_off is unique for each probe */
+	tr_key = bpf_trampoline_compute_key(tgt_prog, NULL, sdt->val.orig_off);
+	tgt_info.tgt_addr = sdt->probe_ip;
+	tgt_info.tgt_name = sdt->val.name;
+	tgt_info.fmodel.nr_args = sdt->val.nargs;
+	for (i = 0; i < sdt->val.nargs; i++) {
+		tgt_info.fmodel.arg_size[i] = 8;
+		tgt_info.fmodel.arg_regs[i] = sdt->val.arg_reg[i];
+	}
+
+	sdt_link->trampoline = bpf_trampoline_get(tr_key, &tgt_info);
+	if (!sdt_link->trampoline) {
+		err = -ENOMEM;
+		goto err_cleanup;
+	}
+	sdt_link->trampoline->func.sdt_probe_site = 1;
+
+	err = bpf_trampoline_link_prog(&sdt_link->link.node, sdt_link->trampoline, tgt_prog);
+	if (err)
+		goto err_put_tramp;
+
+	return bpf_link_settle(&primer);
+
+err_put_tramp:
+	bpf_trampoline_put(sdt_link->trampoline);
+err_cleanup:
+	bpf_link_cleanup(&primer);
+	return err;
+}
+
 static int bpf_tracing_prog_attach(struct bpf_prog *prog,
 				   int tgt_prog_fd,
 				   u32 btf_id,
@@ -5945,6 +6028,8 @@ static int link_create(union bpf_attr *attr, bpfptr_t uattr)
 			ret = cgroup_bpf_link_attach(attr, prog);
 		else if (is_tracing_multi(prog->expected_attach_type))
 			ret = bpf_tracing_multi_attach(prog, attr);
+		else if (prog->expected_attach_type == BPF_TRACE_SDT)
+			ret = bpf_sdt_link_attach(prog);
 		else
 			ret = bpf_tracing_prog_attach(prog,
 						      attr->link_create.target_fd,
diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index 1a721fc4bef5..68f87e7b901f 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c
@@ -691,6 +691,9 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
 	/* clear all bits except SHARE_IPMODIFY and TAIL_CALL_CTX */
 	tr->flags &= (BPF_TRAMP_F_SHARE_IPMODIFY | BPF_TRAMP_F_TAIL_CALL_CTX);
 
+	if (tr->func.sdt_probe_site)
+		tr->flags |= BPF_TRAMP_F_SDT_PROBE;
+
 	if (tnodes[BPF_TRAMP_FEXIT].nr_nodes ||
 	    tnodes[BPF_TRAMP_MODIFY_RETURN].nr_nodes) {
 		/* NOTE: BPF_TRAMP_F_RESTORE_REGS and BPF_TRAMP_F_SKIP_FRAME
@@ -784,6 +787,7 @@ static enum bpf_tramp_prog_type bpf_attach_type_to_tramp(struct bpf_prog *prog)
 	switch (prog->expected_attach_type) {
 	case BPF_TRACE_FENTRY:
 	case BPF_TRACE_FENTRY_MULTI:
+	case BPF_TRACE_SDT:
 		return BPF_TRAMP_FENTRY;
 	case BPF_MODIFY_RETURN:
 		return BPF_TRAMP_MODIFY_RETURN;
-- 
2.47.3

[RFC PATCH bpf-next 11/12] bpf, x86: Add JIT support SDT for probe

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Add x86 JIT support for SDT probe. For the target side, each probe site
is jited to a 5-byte NOP so it can be patched to a CALL instruction when
an observer program is attached. For the observer side, the normal bpf
trampoline is reused, except that the argument registers come from the
arg_regs[] array in the func_model, rather than the fixed BPF R1~R5.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 arch/x86/net/bpf_jit_comp.c | 38 +++++++++++++++++++++++++++++--------
 1 file changed, 30 insertions(+), 8 deletions(-)

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 054e043ffcd2..262f1bd74d53 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -2797,6 +2797,16 @@ st:			insn_off = insn->off;
 			}
 
 			if (!jmp_offset) {
+				int index = bpf_prog->aux->subprog_start + i - 1;
+
+				/*
+				 * Emit a 5-byte NOP for SDT probe site, so it can be patched to
+				 * a CALL instruction when observer program is attached.
+				 */
+				if (env->insn_aux_data[index].sdt_entry) {
+					emit_nops(&prog, X86_PATCH_SIZE);
+					break;
+				}
 				/*
 				 * If jmp_padding is enabled, the extra nops will
 				 * be inserted. Otherwise, optimize out nop jumps.
@@ -3059,9 +3069,13 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
 
 			/* copy the arguments from regs into stack */
 			for (j = 0; j < arg_regs; j++) {
+				u8 reg = (flags & BPF_TRAMP_F_SDT_PROBE)
+					? m->arg_regs[nr_regs]
+					: (nr_regs == 5 ? X86_REG_R9
+					   : BPF_REG_1 + nr_regs);
+
 				emit_stx(prog, BPF_DW, BPF_REG_FP,
-					 nr_regs == 5 ? X86_REG_R9 : BPF_REG_1 + nr_regs,
-					 -stack_size);
+					 reg, -stack_size);
 				stack_size -= 8;
 				nr_regs++;
 			}
@@ -3072,7 +3086,7 @@ static void save_args(const struct btf_func_model *m, u8 **prog,
 }
 
 static void restore_regs(const struct btf_func_model *m, u8 **prog,
-			 int stack_size)
+			 int stack_size, u32 flags)
 {
 	int i, j, arg_regs, nr_regs = 0;
 
@@ -3087,9 +3101,12 @@ static void restore_regs(const struct btf_func_model *m, u8 **prog,
 		arg_regs = (m->arg_size[i] + 7) / 8;
 		if (nr_regs + arg_regs <= 6) {
 			for (j = 0; j < arg_regs; j++) {
-				emit_ldx(prog, BPF_DW,
-					 nr_regs == 5 ? X86_REG_R9 : BPF_REG_1 + nr_regs,
-					 BPF_REG_FP,
+				u8 reg = (flags & BPF_TRAMP_F_SDT_PROBE)
+					? m->arg_regs[nr_regs]
+					: (nr_regs == 5 ? X86_REG_R9
+					   : BPF_REG_1 + nr_regs);
+
+				emit_ldx(prog, BPF_DW, reg, BPF_REG_FP,
 					 -stack_size);
 				stack_size -= 8;
 				nr_regs++;
@@ -3547,7 +3564,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
 	}
 
 	if (flags & BPF_TRAMP_F_CALL_ORIG) {
-		restore_regs(m, &prog, regs_off);
+		restore_regs(m, &prog, regs_off, flags);
 		save_args(m, &prog, arg_stack_off, true, flags);
 
 		if (flags & BPF_TRAMP_F_TAIL_CALL_CTX) {
@@ -3603,7 +3620,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
 	}
 
 	if (flags & BPF_TRAMP_F_RESTORE_REGS)
-		restore_regs(m, &prog, regs_off);
+		restore_regs(m, &prog, regs_off, flags);
 
 	/* This needs to be done regardless. If there were fmod_ret programs,
 	 * the return value is only updated on the stack and still needs to be
@@ -4220,3 +4237,8 @@ bool bpf_jit_supports_fsession(void)
 {
 	return true;
 }
+
+bool bpf_jit_supports_sdt_probe(void)
+{
+	return true;
+}
-- 
2.47.3

[RFC PATCH bpf-next 12/12] selftests/bpf: Add tests for bpf SDT probe

[Xu Kuohai]

From: Xu Kuohai <xukuohai@huawei.com>

Add selftests for bpf SDT probe sites and observers.

The tests include argument check for observer program, multiple target
programs in the same ELF, and programs with subprog.

Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
---
 .../selftests/bpf/prog_tests/test_bpf_sdt.c   | 151 ++++++++++++++++++
 .../selftests/bpf/progs/bpf_sdt_observer.c    |  30 ++++
 .../selftests/bpf/progs/bpf_sdt_target.c      |  34 ++++
 3 files changed, 215 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/test_bpf_sdt.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_observer.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_sdt_target.c

diff --git a/tools/testing/selftests/bpf/prog_tests/test_bpf_sdt.c b/tools/testing/selftests/bpf/prog_tests/test_bpf_sdt.c
new file mode 100644
index 000000000000..b2752e7b2b09
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/test_bpf_sdt.c
@@ -0,0 +1,151 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include <bpf/bpf.h>
+#include "bpf_sdt_observer.skel.h"
+#include "bpf_sdt_target.skel.h"
+
+static int sdt_attach(struct bpf_program *obs)
+{
+	LIBBPF_OPTS(bpf_link_create_opts, opts);
+
+	return bpf_link_create(bpf_program__fd(obs), 0, BPF_TRACE_SDT, &opts);
+}
+
+static void read_args(void)
+{
+	LIBBPF_OPTS(bpf_test_run_opts, topts);
+	struct bpf_sdt_target *tgt_skel;
+	struct bpf_sdt_observer *skel;
+	char pkt[64] = {};
+	int link, err;
+
+	tgt_skel = bpf_sdt_target__open_and_load();
+	if (!ASSERT_OK_PTR(tgt_skel, "target open_and_load"))
+		return;
+
+	skel = bpf_sdt_observer__open();
+	if (!ASSERT_OK_PTR(skel, "observer open"))
+		goto out_tgt;
+
+	bpf_program__set_autoload(skel->progs.tc_trace_prog, false);
+	bpf_program__set_autoload(skel->progs.subprog_trace_prog, false);
+	bpf_program__set_attach_target(skel->progs.xdp_trace_prog,
+				       bpf_program__fd(tgt_skel->progs.xdp_prog),
+				       "xdp_probe_len_ret");
+	err = bpf_sdt_observer__load(skel);
+	if (!ASSERT_OK(err, "observer load"))
+		goto out_obs;
+
+	link = sdt_attach(skel->progs.xdp_trace_prog);
+	if (!ASSERT_GE(link, 0, "attach xdp_probe_len_ret"))
+		goto out_obs;
+
+	topts.data_in = pkt;
+	topts.data_size_in = sizeof(pkt);
+	topts.data_size_out = sizeof(pkt);
+	err = bpf_prog_test_run_opts(bpf_program__fd(tgt_skel->progs.xdp_prog), &topts);
+	if (!ASSERT_OK(err, "prog_test_run"))
+		goto out_link;
+
+	ASSERT_EQ(skel->bss->xdp_len, sizeof(pkt), "xdp_len");
+	ASSERT_EQ(skel->bss->xdp_ret, XDP_PASS, "xdp_ret");
+
+out_link:
+	close(link);
+out_obs:
+	bpf_sdt_observer__destroy(skel);
+out_tgt:
+	bpf_sdt_target__destroy(tgt_skel);
+}
+
+static void multi_prog(void)
+{
+	struct bpf_sdt_target *tgt_skel;
+	struct bpf_sdt_observer *skel;
+	int link1, link2, err;
+
+	tgt_skel = bpf_sdt_target__open_and_load();
+	if (!ASSERT_OK_PTR(tgt_skel, "target open_and_load"))
+		return;
+
+	skel = bpf_sdt_observer__open();
+	if (!ASSERT_OK_PTR(skel, "observer open"))
+		goto out_tgt;
+
+	bpf_program__set_autoload(skel->progs.subprog_trace_prog, false);
+	bpf_program__set_attach_target(skel->progs.xdp_trace_prog,
+				       bpf_program__fd(tgt_skel->progs.xdp_prog),
+				       "xdp_probe_len_ret");
+	bpf_program__set_attach_target(skel->progs.tc_trace_prog,
+				       bpf_program__fd(tgt_skel->progs.tc_prog),
+				       "tc_probe");
+	err = bpf_sdt_observer__load(skel);
+	if (!ASSERT_OK(err, "observer load"))
+		goto out_obs;
+
+	link1 = sdt_attach(skel->progs.xdp_trace_prog);
+	ASSERT_GE(link1, 0, "attach xdp xdp_probe_len_ret");
+
+	link2 = sdt_attach(skel->progs.tc_trace_prog);
+	ASSERT_GE(link2, 0, "attach tc_probe");
+
+	close(link1);
+	close(link2);
+out_obs:
+	bpf_sdt_observer__destroy(skel);
+out_tgt:
+	bpf_sdt_target__destroy(tgt_skel);
+}
+
+static void subprog_probe(void)
+{
+	LIBBPF_OPTS(bpf_test_run_opts, topts);
+	struct bpf_sdt_target *tgt_skel;
+	struct bpf_sdt_observer *skel;
+	char pkt[64] = {0, 1, 2, 3};
+	int link, err;
+
+	tgt_skel = bpf_sdt_target__open_and_load();
+	if (!ASSERT_OK_PTR(tgt_skel, "target open_and_load"))
+		return;
+
+	skel = bpf_sdt_observer__open();
+	if (!ASSERT_OK_PTR(skel, "observer open"))
+		goto out_tgt;
+
+	bpf_program__set_autoload(skel->progs.tc_trace_prog, false);
+	bpf_program__set_autoload(skel->progs.xdp_trace_prog, false);
+	bpf_program__set_attach_target(skel->progs.subprog_trace_prog,
+				       bpf_program__fd(tgt_skel->progs.xdp_prog),
+				       "xdp_probe_ctx");
+	err = bpf_sdt_observer__load(skel);
+	if (!ASSERT_OK(err, "observer load"))
+		goto out_obs;
+
+	link = sdt_attach(skel->progs.subprog_trace_prog);
+	if (!ASSERT_GE(link, 0, "attach xdp_probe_ctx"))
+		goto out_obs;
+
+	topts.data_in = pkt;
+	topts.data_size_in = sizeof(pkt);
+	topts.data_size_out = sizeof(pkt);
+	err = bpf_prog_test_run_opts(bpf_program__fd(tgt_skel->progs.xdp_prog), &topts);
+	ASSERT_OK(err, "prog_test_run");
+	ASSERT_EQ(skel->bss->xdp_len, sizeof(pkt), "xdp_len");
+
+	close(link);
+out_obs:
+	bpf_sdt_observer__destroy(skel);
+out_tgt:
+	bpf_sdt_target__destroy(tgt_skel);
+}
+
+void test_bpf_sdt(void)
+{
+	if (test__start_subtest("read_args"))
+		read_args();
+	if (test__start_subtest("multi_prog"))
+		multi_prog();
+	if (test__start_subtest("subprog_probe"))
+		subprog_probe();
+}
diff --git a/tools/testing/selftests/bpf/progs/bpf_sdt_observer.c b/tools/testing/selftests/bpf/progs/bpf_sdt_observer.c
new file mode 100644
index 000000000000..ec059e86b154
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_sdt_observer.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+volatile int xdp_len;
+volatile int xdp_ret;
+
+SEC("bpf_sdt")
+int BPF_PROG(tc_trace_prog)
+{
+	return 0;
+}
+
+SEC("bpf_sdt")
+int BPF_PROG(xdp_trace_prog, int len, int ret)
+{
+	xdp_len = len;
+	xdp_ret = ret;
+	return 0;
+}
+
+SEC("bpf_sdt")
+int BPF_PROG(subprog_trace_prog, struct xdp_md *xdp_ctx)
+{
+	xdp_len = bpf_xdp_get_buff_len((struct xdp_md *)xdp_ctx);
+	return 0;
+}
+
+char LICENSE[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/bpf_sdt_target.c b/tools/testing/selftests/bpf/progs/bpf_sdt_target.c
new file mode 100644
index 000000000000..e1c14b9d7e9a
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/bpf_sdt_target.c
@@ -0,0 +1,34 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "../../../../tools/lib/bpf/bpf_sdt.h"
+
+BPF_SDT_DECLARE(tc_probe);
+BPF_SDT_DECLARE1(xdp_probe_ctx, struct xdp_md *);
+BPF_SDT_DECLARE2(xdp_probe_len_ret, int, int);
+
+static __noinline int xdp_process(struct xdp_md *ctx)
+{
+	BPF_SDT_PROBE1(xdp_probe_ctx, ctx);
+	return XDP_PASS;
+}
+
+SEC("xdp")
+int xdp_prog(struct xdp_md *ctx)
+{
+	int len = (int)(ctx->data_end - ctx->data);
+
+	BPF_SDT_PROBE2(xdp_probe_len_ret, len, XDP_PASS);
+
+	return xdp_process(ctx);
+}
+
+SEC("tc")
+int tc_prog(struct __sk_buff *skb)
+{
+	BPF_SDT_PROBE(tc_probe);
+	return 0;
+}
+
+char LICENSE[] SEC("license") = "GPL";
-- 
2.47.3