* [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
@ 2023-10-23 9:57 Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 1/5] btf_encoder, pahole: move btf encoding options into conf_load Alan Maguire
` (5 more replies)
0 siblings, 6 replies; 16+ messages in thread
From: Alan Maguire @ 2023-10-23 9:57 UTC (permalink / raw)
To: acme, andrii.nakryiko
Cc: jolsa, ast, daniel, eddyz87, martin.lau, song, yhs,
john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf, Alan Maguire
Currently, the kernel uses pahole version checking as the way to
determine which BTF encoding features to request from pahole. This
means that such features have to be tied to a specific version and
as new features are added, additional clauses in scripts/pahole-flags.sh
have to be added; for example
if [ "${pahole_ver}" -ge "125" ]; then
extra_paholeopt="${extra_paholeopt} --skip_encoding_btf_inconsistent_proto --btf_gen_optimized"
fi
To better future-proof this process, this series introduces a
single "btf_features" parameter that uses a comma-separated list
of encoding options. This is helpful because
- the semantics are simpler for the user; the list comprises the set of
BTF features asked for, rather than having to specify a combination of
--skip_encoding_btf_feature and --btf_gen_feature options; and
- any version of pahole that supports --btf_features can accept the
option list; unknown options are silently ignored. As a result, there
would be no need to add additional version clauses beyond
if [ "${pahole_ver}" -ge "126" ]; then
extra_pahole_opt="-j --lang_exclude=rust
--btf_features=encode_force,var,float,decl_tag,type_tag,enum64,optimized_func,consistent_func"
fi
Newly-supported features would simply be appended to the btf_features
list, and these would have impact on BTF encoding only if the features
were supported by pahole. This means pahole will not require a version
bump when new BTF features are added, and should ease the burden of
coordinating such changes between bpf-next and dwarves.
Patches 1 and 2 are preparatory work, while patch 3 adds the
--btf_features support. Patch 4 provides a means of querying
the supported feature set since --btf_features will not error
out when it encounters unrecognized features (this ensures
an older pahole without a requested feature will not dump warnings
in the build log for kernel/module BTF generation). Patch 5
adds --btf_features_strict, which is identical to --btf_features
aside from the fact it will fail if an unrecognized feature is used.
See [1] for more background on this topic.
Changes since v3 [2]:
- Merged strtok_r() and feature-processing into a single loop to
avoid the need to store feature list in an array (Eduard, patch 3)
- Added ack from Eduard (patches 1-5)
Changes since v2 [3]:
- added acks from Andrii and Jiri (patches 1-5)
- merged suggestions from Eduard which simplify and clean up code
considerably; these changes fix issues with --btf_features_strict
while providing better diagnostic output when an unknown feature
is encountered (specifying the unknown feature if in verbose
or strict modes). Added Suggested-bys from Eduard for the
relevant patches (Eduard, patches 3,5)
Changes since RFC [4]:
- ensure features are disabled unless requested; use "default" field in
"struct btf_features" to specify the conf_load default value which
corresponds to the feature being disabled. For
conf_load->btf_gen_floats for example, the default value is false,
while for conf_load->skip_encoding_btf_type_tags the default is
true; in both cases the intent is to _not_ encode the associated
feature by default. However if the user specifies "float" or
"type_tag" in --btf_features, the default conf_load value is negated,
resulting in a BTF encoding that contains floats and type tags
(Eduard, patch 3)
- clarify feature default/setting behaviour and how it only applies
when --btf_features is used (Eduard, patch 3)
- ensure we do not run off the end of the feature_list[] array
(Eduard, patch 3)
- rather than having each struct btf_feature record the offset in the
conf_load structure of the boolean (requiring us to later do pointer
math to update it), record the pointers to the boolean conf_load
values associated with each feature (Jiri, patch 3)
- allow for multiple specifications of --btf_features, enabling the
union of all features specified (Andrii, patch 3)
- rename function-related optimized/consistent to optimized_func and
consistent_func in recognition of the fact they are function-specific
(Andrii, patch 3)
- add a strict version of --btf_features, --btf_features_strict that
will error out if an unrecognized feature is used (Andrii, patch 5)
[1] https://lore.kernel.org/bpf/CAEf4Bzaz1UqqxuZ7Q+KQee-HLyY1nwhAurBE2n9YTWchqoYLbg@mail.gmail.com/
[2] https://lore.kernel.org/bpf/20231018122926.735416-1-alan.maguire@oracle.com/
[3] https://lore.kernel.org/bpf/20231013153359.88274-1-alan.maguire@oracle.com/
[4] https://lore.kernel.org/bpf/20231011091732.93254-1-alan.maguire@oracle.com/
Alan Maguire (5):
btf_encoder, pahole: move btf encoding options into conf_load
dwarves: move ARRAY_SIZE() to dwarves.h
pahole: add --btf_features support
pahole: add --supported_btf_features
pahole: add --btf_features_strict to reject unknown BTF features
btf_encoder.c | 8 +-
btf_encoder.h | 2 +-
dwarves.c | 16 ----
dwarves.h | 19 +++++
man-pages/pahole.1 | 32 ++++++++
pahole.c | 186 +++++++++++++++++++++++++++++++++++++++++----
6 files changed, 229 insertions(+), 34 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v4 dwarves 1/5] btf_encoder, pahole: move btf encoding options into conf_load
2023-10-23 9:57 [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Alan Maguire
@ 2023-10-23 9:57 ` Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 2/5] dwarves: move ARRAY_SIZE() to dwarves.h Alan Maguire
` (4 subsequent siblings)
5 siblings, 0 replies; 16+ messages in thread
From: Alan Maguire @ 2023-10-23 9:57 UTC (permalink / raw)
To: acme, andrii.nakryiko
Cc: jolsa, ast, daniel, eddyz87, martin.lau, song, yhs,
john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf, Alan Maguire,
Andrii Nakryiko
...rather than passing them to btf_encoder__new(); this tidies
up the encoder API and also allows us to use generalized methods
to translate from a BTF feature (forthcoming) to a conf_load
value.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
btf_encoder.c | 8 ++++----
btf_encoder.h | 2 +-
dwarves.h | 3 +++
pahole.c | 21 ++++++++-------------
4 files changed, 16 insertions(+), 18 deletions(-)
diff --git a/btf_encoder.c b/btf_encoder.c
index 65f6e71..fd04008 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -1625,7 +1625,7 @@ out:
return err;
}
-struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filename, struct btf *base_btf, bool skip_encoding_vars, bool force, bool gen_floats, bool verbose)
+struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filename, struct btf *base_btf, bool verbose, struct conf_load *conf_load)
{
struct btf_encoder *encoder = zalloc(sizeof(*encoder));
@@ -1639,9 +1639,9 @@ struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filenam
if (encoder->btf == NULL)
goto out_delete;
- encoder->force = force;
- encoder->gen_floats = gen_floats;
- encoder->skip_encoding_vars = skip_encoding_vars;
+ encoder->force = conf_load->btf_encode_force;
+ encoder->gen_floats = conf_load->btf_gen_floats;
+ encoder->skip_encoding_vars = conf_load->skip_encoding_btf_vars;
encoder->verbose = verbose;
encoder->has_index_type = false;
encoder->need_index_type = false;
diff --git a/btf_encoder.h b/btf_encoder.h
index 34516bb..f54c95a 100644
--- a/btf_encoder.h
+++ b/btf_encoder.h
@@ -16,7 +16,7 @@ struct btf;
struct cu;
struct list_head;
-struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filename, struct btf *base_btf, bool skip_encoding_vars, bool force, bool gen_floats, bool verbose);
+struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filename, struct btf *base_btf, bool verbose, struct conf_load *conf_load);
void btf_encoder__delete(struct btf_encoder *encoder);
int btf_encoder__encode(struct btf_encoder *encoder);
diff --git a/dwarves.h b/dwarves.h
index eb1a6df..db68161 100644
--- a/dwarves.h
+++ b/dwarves.h
@@ -68,6 +68,9 @@ struct conf_load {
bool skip_encoding_btf_enum64;
bool btf_gen_optimized;
bool skip_encoding_btf_inconsistent_proto;
+ bool skip_encoding_btf_vars;
+ bool btf_gen_floats;
+ bool btf_encode_force;
uint8_t hashtable_bits;
uint8_t max_hashtable_bits;
uint16_t kabi_prefix_len;
diff --git a/pahole.c b/pahole.c
index e843999..7a41dc3 100644
--- a/pahole.c
+++ b/pahole.c
@@ -32,13 +32,10 @@
static struct btf_encoder *btf_encoder;
static char *detached_btf_filename;
static bool btf_encode;
-static bool btf_gen_floats;
static bool ctf_encode;
static bool sort_output;
static bool need_resort;
static bool first_obj_only;
-static bool skip_encoding_btf_vars;
-static bool btf_encode_force;
static const char *base_btf_file;
static const char *prettify_input_filename;
@@ -1786,9 +1783,9 @@ static error_t pahole__options_parser(int key, char *arg,
case ARGP_header_type:
conf.header_type = arg; break;
case ARGP_skip_encoding_btf_vars:
- skip_encoding_btf_vars = true; break;
+ conf_load.skip_encoding_btf_vars = true; break;
case ARGP_btf_encode_force:
- btf_encode_force = true; break;
+ conf_load.btf_encode_force = true; break;
case ARGP_btf_base:
base_btf_file = arg; break;
case ARGP_kabi_prefix:
@@ -1797,9 +1794,9 @@ static error_t pahole__options_parser(int key, char *arg,
case ARGP_numeric_version:
print_numeric_version = true; break;
case ARGP_btf_gen_floats:
- btf_gen_floats = true; break;
+ conf_load.btf_gen_floats = true; break;
case ARGP_btf_gen_all:
- btf_gen_floats = true; break;
+ conf_load.btf_gen_floats = true; break;
case ARGP_with_flexible_array:
show_with_flexible_array = true; break;
case ARGP_prettify_input_filename:
@@ -3063,8 +3060,8 @@ static enum load_steal_kind pahole_stealer(struct cu *cu,
* And, it is used by the thread
* create it.
*/
- btf_encoder = btf_encoder__new(cu, detached_btf_filename, conf_load->base_btf, skip_encoding_btf_vars,
- btf_encode_force, btf_gen_floats, global_verbose);
+ btf_encoder = btf_encoder__new(cu, detached_btf_filename, conf_load->base_btf,
+ global_verbose, conf_load);
if (btf_encoder && thr_data) {
struct thread_data *thread = thr_data;
@@ -3093,10 +3090,8 @@ static enum load_steal_kind pahole_stealer(struct cu *cu,
thread->encoder =
btf_encoder__new(cu, detached_btf_filename,
NULL,
- skip_encoding_btf_vars,
- btf_encode_force,
- btf_gen_floats,
- global_verbose);
+ global_verbose,
+ conf_load);
thread->btf = btf_encoder__btf(thread->encoder);
}
encoder = thread->encoder;
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v4 dwarves 2/5] dwarves: move ARRAY_SIZE() to dwarves.h
2023-10-23 9:57 [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 1/5] btf_encoder, pahole: move btf encoding options into conf_load Alan Maguire
@ 2023-10-23 9:57 ` Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 3/5] pahole: add --btf_features support Alan Maguire
` (3 subsequent siblings)
5 siblings, 0 replies; 16+ messages in thread
From: Alan Maguire @ 2023-10-23 9:57 UTC (permalink / raw)
To: acme, andrii.nakryiko
Cc: jolsa, ast, daniel, eddyz87, martin.lau, song, yhs,
john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf, Alan Maguire,
Andrii Nakryiko
...so it can be used by pahole.c too.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
dwarves.c | 16 ----------------
dwarves.h | 16 ++++++++++++++++
2 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/dwarves.c b/dwarves.c
index 218367b..9f97eda 100644
--- a/dwarves.c
+++ b/dwarves.c
@@ -2094,22 +2094,6 @@ int cus__load_file(struct cus *cus, struct conf_load *conf,
_min1 < _min2 ? _min1 : _min2; })
#endif
-/* Force a compilation error if condition is true, but also produce a
- result (of value 0 and type size_t), so the expression can be used
- e.g. in a structure initializer (or where-ever else comma expressions
- aren't permitted). */
-#define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); }))
-
-/* Are two types/vars the same type (ignoring qualifiers)? */
-#ifndef __same_type
-# define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
-#endif
-
-/* &a[0] degrades to a pointer: a different type from an array */
-#define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
-
-#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
-
#ifndef DW_LANG_C89
#define DW_LANG_C89 0x0001
#endif
diff --git a/dwarves.h b/dwarves.h
index db68161..857b37c 100644
--- a/dwarves.h
+++ b/dwarves.h
@@ -19,6 +19,22 @@
#include "list.h"
#include "rbtree.h"
+/* Force a compilation error if condition is true, but also produce a
+ result (of value 0 and type size_t), so the expression can be used
+ e.g. in a structure initializer (or where-ever else comma expressions
+ aren't permitted). */
+#define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); }))
+
+/* Are two types/vars the same type (ignoring qualifiers)? */
+#ifndef __same_type
+# define __same_type(a, b) __builtin_types_compatible_p(typeof(a), typeof(b))
+#endif
+
+/* &a[0] degrades to a pointer: a different type from an array */
+#define __must_be_array(a) BUILD_BUG_ON_ZERO(__same_type((a), &(a)[0]))
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]) + __must_be_array(arr))
+
struct cu;
enum load_steal_kind {
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v4 dwarves 3/5] pahole: add --btf_features support
2023-10-23 9:57 [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 1/5] btf_encoder, pahole: move btf encoding options into conf_load Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 2/5] dwarves: move ARRAY_SIZE() to dwarves.h Alan Maguire
@ 2023-10-23 9:57 ` Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 4/5] pahole: add --supported_btf_features Alan Maguire
` (2 subsequent siblings)
5 siblings, 0 replies; 16+ messages in thread
From: Alan Maguire @ 2023-10-23 9:57 UTC (permalink / raw)
To: acme, andrii.nakryiko
Cc: jolsa, ast, daniel, eddyz87, martin.lau, song, yhs,
john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf, Alan Maguire,
Andrii Nakryiko
This allows consumers to specify an opt-in set of features
they want to use in BTF encoding.
Supported features are a comma-separated combination of
encode_force Ignore invalid symbols when encoding BTF.
var Encode variables using BTF_KIND_VAR in BTF.
float Encode floating-point types in BTF.
decl_tag Encode declaration tags using BTF_KIND_DECL_TAG.
type_tag Encode type tags using BTF_KIND_TYPE_TAG.
enum64 Encode enum64 values with BTF_KIND_ENUM64.
optimized_func Encode representations of optimized functions
with suffixes like ".isra.0" etc
consistent_func Avoid encoding inconsistent static functions.
These occur when a parameter is optimized out
in some CUs and not others, or when the same
function name has inconsistent BTF descriptions
in different CUs.
Specifying "--btf_features=all" is the equivalent to setting
all of the above. If pahole does not know about a feature
specified in --btf_features it silently ignores it.
The --btf_features can either be specified via a single comma-separated
list
--btf_features=enum64,float
...or via multiple --btf_features values
--btf_features=enum64 --btf_features=float
These properties allow us to use the --btf_features option in
the kernel scripts/pahole_flags.sh script to specify the desired
set of BTF features.
If a feature named in --btf_features is not present in the version
of pahole used, BTF encoding will not complain. This is desired
because it means we no longer have to tie new features to a specific
pahole version.
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
man-pages/pahole.1 | 24 +++++++++
pahole.c | 132 ++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 155 insertions(+), 1 deletion(-)
diff --git a/man-pages/pahole.1 b/man-pages/pahole.1
index c1b48de..a09885f 100644
--- a/man-pages/pahole.1
+++ b/man-pages/pahole.1
@@ -273,6 +273,30 @@ Generate BTF for functions with optimization-related suffixes (.isra, .constprop
.B \-\-btf_gen_all
Allow using all the BTF features supported by pahole.
+.TP
+.B \-\-btf_features=FEATURE_LIST
+Encode BTF using the specified feature list, or specify 'all' for all features supported. This option can be used as an alternative to unsing multiple BTF-related options. Supported features are
+
+.nf
+ encode_force Ignore invalid symbols when encoding BTF; for example
+ if a symbol has an invalid name, it will be ignored
+ and BTF encoding will continue.
+ var Encode variables using BTF_KIND_VAR in BTF.
+ float Encode floating-point types in BTF.
+ decl_tag Encode declaration tags using BTF_KIND_DECL_TAG.
+ type_tag Encode type tags using BTF_KIND_TYPE_TAG.
+ enum64 Encode enum64 values with BTF_KIND_ENUM64.
+ optimized_func Encode representations of optimized functions
+ with suffixes like ".isra.0".
+ consistent_func Avoid encoding inconsistent static functions.
+ These occur when a parameter is optimized out
+ in some CUs and not others, or when the same
+ function name has inconsistent BTF descriptions
+ in different CUs.
+.fi
+
+So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values.
+
.TP
.B \-l, \-\-show_first_biggest_size_base_type_member
Show first biggest size base_type member.
diff --git a/pahole.c b/pahole.c
index 7a41dc3..fe1cc6a 100644
--- a/pahole.c
+++ b/pahole.c
@@ -1229,6 +1229,128 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;
#define ARGP_skip_emitting_atomic_typedefs 338
#define ARGP_btf_gen_optimized 339
#define ARGP_skip_encoding_btf_inconsistent_proto 340
+#define ARGP_btf_features 341
+
+/* --btf_features=feature1[,feature2,..] allows us to specify
+ * a list of requested BTF features or "all" to enable all features.
+ * These are translated into the appropriate conf_load values via a
+ * struct btf_feature which specifies the associated conf_load
+ * boolean field and whether its default (representing the feature being
+ * off) is false or true.
+ *
+ * btf_features is for opting _into_ features so for a case like
+ * conf_load->btf_gen_floats, the translation is simple; the presence
+ * of the "float" feature in --btf_features sets conf_load->btf_gen_floats
+ * to true.
+ *
+ * The more confusing case is for features that are enabled unless
+ * skipping them is specified; for example
+ * conf_load->skip_encoding_btf_type_tag. By default - to support
+ * the opt-in model of only enabling features the user asks for -
+ * conf_load->skip_encoding_btf_type_tag is set to true (meaning no
+ * type_tags) and it is only set to false if --btf_features contains
+ * the "type_tag" keyword.
+ *
+ * So from the user perspective, all features specified via
+ * --btf_features are enabled, and if a feature is not specified,
+ * it is disabled.
+ *
+ * If --btf_features is not used, the usual pahole defaults for
+ * BTF encoding apply; we encode type/decl tags, do not encode
+ * floats, etc. This ensures backwards compatibility.
+ */
+#define BTF_FEATURE(name, alias, default_value) \
+ { #name, #alias, &conf_load.alias, default_value }
+
+struct btf_feature {
+ const char *name;
+ const char *option_alias;
+ bool *conf_value;
+ bool default_value;
+} btf_features[] = {
+ BTF_FEATURE(encode_force, btf_encode_force, false),
+ BTF_FEATURE(var, skip_encoding_btf_vars, true),
+ BTF_FEATURE(float, btf_gen_floats, false),
+ BTF_FEATURE(decl_tag, skip_encoding_btf_decl_tag, true),
+ BTF_FEATURE(type_tag, skip_encoding_btf_type_tag, true),
+ BTF_FEATURE(enum64, skip_encoding_btf_enum64, true),
+ BTF_FEATURE(optimized_func, btf_gen_optimized, false),
+ BTF_FEATURE(consistent_func, skip_encoding_btf_inconsistent_proto, false),
+};
+
+#define BTF_MAX_FEATURE_STR 1024
+
+bool set_btf_features_defaults;
+
+static void init_btf_features(void)
+{
+ int i;
+
+ /* Only set default values once, as multiple --btf_features=
+ * may be specified on command-line, and setting defaults
+ * again could clobber values. The aim is to enable
+ * all features set across all --btf_features options.
+ */
+ if (set_btf_features_defaults)
+ return;
+ for (i = 0; i < ARRAY_SIZE(btf_features); i++)
+ *btf_features[i].conf_value = btf_features[i].default_value;
+ set_btf_features_defaults = true;
+}
+
+static struct btf_feature *find_btf_feature(char *name)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(btf_features); i++) {
+ if (strcmp(name, btf_features[i].name) == 0)
+ return &btf_features[i];
+ }
+ return NULL;
+}
+
+static void enable_btf_feature(struct btf_feature *feature)
+{
+ /* switch "default-off" features on, and "default-on" features
+ * off; i.e. negate the default value.
+ */
+ *feature->conf_value = !feature->default_value;
+}
+
+/* Translate --btf_features=feature1[,feature2] into conf_load values.
+ * Explicitly ignores unrecognized features to allow future specification
+ * of new opt-in features.
+ */
+static void parse_btf_features(const char *features)
+{
+ char *saveptr = NULL, *s, *feature_name;
+ char f[BTF_MAX_FEATURE_STR];
+
+ init_btf_features();
+
+ if (strcmp(features, "all") == 0) {
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(btf_features); i++)
+ enable_btf_feature(&btf_features[i]);
+ return;
+ }
+
+ strncpy(f, features, sizeof(f));
+ s = f;
+ while ((feature_name = strtok_r(s, ",", &saveptr)) != NULL) {
+ struct btf_feature *feature = find_btf_feature(feature_name);
+
+ if (!feature) {
+ if (global_verbose)
+ fprintf(stderr, "Ignoring unsupported feature '%s'\n",
+ feature_name);
+ } else {
+ enable_btf_feature(feature);
+ }
+ s = NULL;
+ }
+}
static const struct argp_option pahole__options[] = {
{
@@ -1651,6 +1773,12 @@ static const struct argp_option pahole__options[] = {
.key = ARGP_skip_encoding_btf_inconsistent_proto,
.doc = "Skip functions that have multiple inconsistent function prototypes sharing the same name, or that use unexpected registers for parameter values."
},
+ {
+ .name = "btf_features",
+ .key = ARGP_btf_features,
+ .arg = "FEATURE_LIST",
+ .doc = "Specify supported BTF features in FEATURE_LIST or 'all' for all supported features. See the pahole manual page for the list of supported features."
+ },
{
.name = NULL,
}
@@ -1796,7 +1924,7 @@ static error_t pahole__options_parser(int key, char *arg,
case ARGP_btf_gen_floats:
conf_load.btf_gen_floats = true; break;
case ARGP_btf_gen_all:
- conf_load.btf_gen_floats = true; break;
+ parse_btf_features("all"); break;
case ARGP_with_flexible_array:
show_with_flexible_array = true; break;
case ARGP_prettify_input_filename:
@@ -1826,6 +1954,8 @@ static error_t pahole__options_parser(int key, char *arg,
conf_load.btf_gen_optimized = true; break;
case ARGP_skip_encoding_btf_inconsistent_proto:
conf_load.skip_encoding_btf_inconsistent_proto = true; break;
+ case ARGP_btf_features:
+ parse_btf_features(arg); break;
default:
return ARGP_ERR_UNKNOWN;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v4 dwarves 4/5] pahole: add --supported_btf_features
2023-10-23 9:57 [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Alan Maguire
` (2 preceding siblings ...)
2023-10-23 9:57 ` [PATCH v4 dwarves 3/5] pahole: add --btf_features support Alan Maguire
@ 2023-10-23 9:57 ` Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 5/5] pahole: add --btf_features_strict to reject unknown BTF features Alan Maguire
2023-10-25 17:43 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
5 siblings, 0 replies; 16+ messages in thread
From: Alan Maguire @ 2023-10-23 9:57 UTC (permalink / raw)
To: acme, andrii.nakryiko
Cc: jolsa, ast, daniel, eddyz87, martin.lau, song, yhs,
john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf, Alan Maguire,
Andrii Nakryiko
By design --btf_features=FEATURE1[,FEATURE2,...] will not complain
if an unrecognized feature is specified. This allows the kernel
build process to specify new features regardless of whether they
are supported by the version of pahole used; in such cases we do
not wish for every invocation of pahole to complain. However it is
still valuable to have a way of knowing which BTF features pahole
supports; this could be logged as part of the build process for
example. By specifying --supported_btf_features a comma-separated
list is returned; for example:
$ pahole --supported_btf_features
encode_force,var,float,decl_tag,type_tag,enum64,optimized_func,consistent_func
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
man-pages/pahole.1 | 4 ++++
pahole.c | 20 ++++++++++++++++++++
2 files changed, 24 insertions(+)
diff --git a/man-pages/pahole.1 b/man-pages/pahole.1
index a09885f..6148915 100644
--- a/man-pages/pahole.1
+++ b/man-pages/pahole.1
@@ -297,6 +297,10 @@ Encode BTF using the specified feature list, or specify 'all' for all features s
So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values.
+.TP
+.B \-\-supported_btf_features
+Show set of BTF features supported by \-\-btf_features option and exit. Useful for checking which features are supported since \-\-btf_features will not emit an error if an unrecognized feature is specified.
+
.TP
.B \-l, \-\-show_first_biggest_size_base_type_member
Show first biggest size base_type member.
diff --git a/pahole.c b/pahole.c
index fe1cc6a..37fd2a4 100644
--- a/pahole.c
+++ b/pahole.c
@@ -1230,6 +1230,7 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;
#define ARGP_btf_gen_optimized 339
#define ARGP_skip_encoding_btf_inconsistent_proto 340
#define ARGP_btf_features 341
+#define ARGP_supported_btf_features 342
/* --btf_features=feature1[,feature2,..] allows us to specify
* a list of requested BTF features or "all" to enable all features.
@@ -1317,6 +1318,18 @@ static void enable_btf_feature(struct btf_feature *feature)
*feature->conf_value = !feature->default_value;
}
+static void show_supported_btf_features(FILE *output)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(btf_features); i++) {
+ if (i > 0)
+ fprintf(output, ",");
+ fprintf(output, "%s", btf_features[i].name);
+ }
+ fprintf(output, "\n");
+}
+
/* Translate --btf_features=feature1[,feature2] into conf_load values.
* Explicitly ignores unrecognized features to allow future specification
* of new opt-in features.
@@ -1779,6 +1792,11 @@ static const struct argp_option pahole__options[] = {
.arg = "FEATURE_LIST",
.doc = "Specify supported BTF features in FEATURE_LIST or 'all' for all supported features. See the pahole manual page for the list of supported features."
},
+ {
+ .name = "supported_btf_features",
+ .key = ARGP_supported_btf_features,
+ .doc = "Show list of btf_features supported by pahole and exit."
+ },
{
.name = NULL,
}
@@ -1956,6 +1974,8 @@ static error_t pahole__options_parser(int key, char *arg,
conf_load.skip_encoding_btf_inconsistent_proto = true; break;
case ARGP_btf_features:
parse_btf_features(arg); break;
+ case ARGP_supported_btf_features:
+ show_supported_btf_features(stdout); exit(0);
default:
return ARGP_ERR_UNKNOWN;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v4 dwarves 5/5] pahole: add --btf_features_strict to reject unknown BTF features
2023-10-23 9:57 [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Alan Maguire
` (3 preceding siblings ...)
2023-10-23 9:57 ` [PATCH v4 dwarves 4/5] pahole: add --supported_btf_features Alan Maguire
@ 2023-10-23 9:57 ` Alan Maguire
2023-10-25 17:43 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
5 siblings, 0 replies; 16+ messages in thread
From: Alan Maguire @ 2023-10-23 9:57 UTC (permalink / raw)
To: acme, andrii.nakryiko
Cc: jolsa, ast, daniel, eddyz87, martin.lau, song, yhs,
john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf, Alan Maguire,
Andrii Nakryiko
--btf_features is used to specify the list of requested features
for BTF encoding. However, it is not strict in rejecting requests
with unknown features; this allows us to use the same parameters
regardless of pahole version. --btf_features_strict carries out
the same encoding with the same feature set, but will fail if an
unrecognized feature is specified.
So
pahole -J --btf_features=enum64,foo
will succeed, while
pahole -J --btf_features_strict=enum64,foo
will not.
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
man-pages/pahole.1 | 4 ++++
pahole.c | 21 ++++++++++++++++++---
2 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/man-pages/pahole.1 b/man-pages/pahole.1
index 6148915..ea9045c 100644
--- a/man-pages/pahole.1
+++ b/man-pages/pahole.1
@@ -297,6 +297,10 @@ Encode BTF using the specified feature list, or specify 'all' for all features s
So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values.
+.TP
+.B \-\-btf_features_strict
+Identical to \-\-btf_features above, but pahole will exit if it encounters an unrecognized feature.
+
.TP
.B \-\-supported_btf_features
Show set of BTF features supported by \-\-btf_features option and exit. Useful for checking which features are supported since \-\-btf_features will not emit an error if an unrecognized feature is specified.
diff --git a/pahole.c b/pahole.c
index 37fd2a4..768a2fe 100644
--- a/pahole.c
+++ b/pahole.c
@@ -1231,6 +1231,7 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;
#define ARGP_skip_encoding_btf_inconsistent_proto 340
#define ARGP_btf_features 341
#define ARGP_supported_btf_features 342
+#define ARGP_btf_features_strict 343
/* --btf_features=feature1[,feature2,..] allows us to specify
* a list of requested BTF features or "all" to enable all features.
@@ -1334,7 +1335,7 @@ static void show_supported_btf_features(FILE *output)
* Explicitly ignores unrecognized features to allow future specification
* of new opt-in features.
*/
-static void parse_btf_features(const char *features)
+static void parse_btf_features(const char *features, bool strict)
{
char *saveptr = NULL, *s, *feature_name;
char f[BTF_MAX_FEATURE_STR];
@@ -1355,6 +1356,12 @@ static void parse_btf_features(const char *features)
struct btf_feature *feature = find_btf_feature(feature_name);
if (!feature) {
+ if (strict) {
+ fprintf(stderr, "Feature '%s' in '%s' is not supported. Supported BTF features are:\n",
+ feature_name, features);
+ show_supported_btf_features(stderr);
+ exit(EXIT_FAILURE);
+ }
if (global_verbose)
fprintf(stderr, "Ignoring unsupported feature '%s'\n",
feature_name);
@@ -1797,6 +1804,12 @@ static const struct argp_option pahole__options[] = {
.key = ARGP_supported_btf_features,
.doc = "Show list of btf_features supported by pahole and exit."
},
+ {
+ .name = "btf_features_strict",
+ .key = ARGP_btf_features_strict,
+ .arg = "FEATURE_LIST_STRICT",
+ .doc = "Specify supported BTF features in FEATURE_LIST_STRICT or 'all' for all supported features. Unlike --btf_features, unrecognized features will trigger an error."
+ },
{
.name = NULL,
}
@@ -1942,7 +1955,7 @@ static error_t pahole__options_parser(int key, char *arg,
case ARGP_btf_gen_floats:
conf_load.btf_gen_floats = true; break;
case ARGP_btf_gen_all:
- parse_btf_features("all"); break;
+ parse_btf_features("all", false); break;
case ARGP_with_flexible_array:
show_with_flexible_array = true; break;
case ARGP_prettify_input_filename:
@@ -1973,9 +1986,11 @@ static error_t pahole__options_parser(int key, char *arg,
case ARGP_skip_encoding_btf_inconsistent_proto:
conf_load.skip_encoding_btf_inconsistent_proto = true; break;
case ARGP_btf_features:
- parse_btf_features(arg); break;
+ parse_btf_features(arg, false); break;
case ARGP_supported_btf_features:
show_supported_btf_features(stdout); exit(0);
+ case ARGP_btf_features_strict:
+ parse_btf_features(arg, true); break;
default:
return ARGP_ERR_UNKNOWN;
}
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-23 9:57 [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Alan Maguire
` (4 preceding siblings ...)
2023-10-23 9:57 ` [PATCH v4 dwarves 5/5] pahole: add --btf_features_strict to reject unknown BTF features Alan Maguire
@ 2023-10-25 17:43 ` Arnaldo Carvalho de Melo
2023-10-25 17:48 ` Arnaldo Carvalho de Melo
2023-10-25 18:18 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
5 siblings, 2 replies; 16+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-10-25 17:43 UTC (permalink / raw)
To: Alan Maguire
Cc: andrii.nakryiko, jolsa, ast, daniel, eddyz87, martin.lau, song,
yhs, john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf
Em Mon, Oct 23, 2023 at 10:57:21AM +0100, Alan Maguire escreveu:
> Currently, the kernel uses pahole version checking as the way to
> determine which BTF encoding features to request from pahole. This
> means that such features have to be tied to a specific version and
> as new features are added, additional clauses in scripts/pahole-flags.sh
> have to be added; for example
Finally trying to test this:
I started with a random vmlinux file, already with BTF:
$ bpftool btf dump file vmlinux.v5.19.0-rc5+ format raw > vmlinux.v5.19.0-rc5+.dump.original
$ wc -l vmlinux.v5.19.0-rc5+.dump.original
291961 vmlinux.v5.19.0-rc5+.dump.original
$ grep -i enum64 vmlinux.v5.19.0-rc5+.dump.original | wc -l
0
$
$ grep -i enum vmlinux.v5.19.0-rc5+.dump.original | wc -l
2175
$
Ok, now I want to encode just with enum64, i.e. all the other features
will not be enabled via this new option and only the enum64 will, if
present in the DWARF info:
$ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.enum64 ; pahole --btf_encode --btf_features=enum64 vmlinux.v5.19.0-rc5+.enum64
$
I tried using --btf_encode_detached=file but then couldn't find a way to
make 'bpftool btf' to consume detached BTF, it seems that "file" means
"ELF file containing BTF" so I copied the original file to then reencode
BTF selecting just the enum64 feature, the resulting file continues to
have the original DWARF and the BTF using that --btf_features set:
[acme@quaco pahole]$ pahole -F btf vmlinux.v5.19.0-rc5+.enum64 | wc -l
143161
[acme@quaco pahole]$ pahole -F dwarf vmlinux.v5.19.0-rc5+.enum64 | wc -l
143589
[acme@quaco pahole]$
[acme@quaco pahole]$ pahole --expand_types -F btf vmlinux.v5.19.0-rc5+.enum64 -C spinlock
struct spinlock {
union {
struct raw_spinlock {
/* typedef arch_spinlock_t */ struct qspinlock {
union {
/* typedef atomic_t */ struct {
int counter; /* 0 4 */
} val; /* 0 4 */
struct {
/* typedef u8 -> __u8 */ unsigned char locked; /* 0 1 */
/* typedef u8 -> __u8 */ unsigned char pending; /* 1 1 */
}; /* 0 2 */
struct {
/* typedef u16 -> __u16 */ short unsigned int locked_pending; /* 0 2 */
/* typedef u16 -> __u16 */ short unsigned int tail; /* 2 2 */
}; /* 0 4 */
}; /* 0 4 */
} raw_lock; /* 0 4 */
}rlock; /* 0 4 */
}; /* 0 4 */
/* size: 4, cachelines: 1, members: 1 */
/* last cacheline: 4 bytes */
};
[acme@quaco pahole]$
But 'bpftool bpf' doesn't like it:
$ bpftool btf dump file vmlinux.v5.19.0-rc5+.enum64 raw
Error: failed to load BTF from vmlinux.v5.19.0-rc5+.enum64: Invalid argument
$
But it doesn't like it even when not using --btf_features :-\
$ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.default_btf_encode ; pahole --btf_encode vmlinux.v5.19.0-rc5+.default_btf_encode
$ bpftool btf dump file vmlinux.v5.19.0-rc5+.default_btf_encode raw | wc -l
Error: failed to load BTF from vmlinux.v5.19.0-rc5+.default_btf_encode: Invalid argument
0
$
I'll try to root cause this problem...
- Arnaldo
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-25 17:43 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
@ 2023-10-25 17:48 ` Arnaldo Carvalho de Melo
2023-10-25 18:12 ` Arnaldo Carvalho de Melo
2023-10-25 18:18 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
1 sibling, 1 reply; 16+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-10-25 17:48 UTC (permalink / raw)
To: Alan Maguire
Cc: andrii.nakryiko, jolsa, ast, daniel, eddyz87, martin.lau, song,
yhs, john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf
Em Wed, Oct 25, 2023 at 02:43:02PM -0300, Arnaldo Carvalho de Melo escreveu:
> But 'bpftool bpf' doesn't like it:
> $ bpftool btf dump file vmlinux.v5.19.0-rc5+.enum64 raw
> Error: failed to load BTF from vmlinux.v5.19.0-rc5+.enum64: Invalid argument
> $
> But it doesn't like it even when not using --btf_features :-\
>
> $ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.default_btf_encode ; pahole --btf_encode vmlinux.v5.19.0-rc5+.default_btf_encode
> $ bpftool btf dump file vmlinux.v5.19.0-rc5+.default_btf_encode raw | wc -l
> Error: failed to load BTF from vmlinux.v5.19.0-rc5+.default_btf_encode: Invalid argument
> 0
> $
> I'll try to root cause this problem...
Random old bpftool on this notebook was the cause, nevermind, I'm back
testing this, sorry for the noise :-)
- Arnaldo
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-25 17:48 ` Arnaldo Carvalho de Melo
@ 2023-10-25 18:12 ` Arnaldo Carvalho de Melo
2023-10-25 18:30 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 16+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-10-25 18:12 UTC (permalink / raw)
To: Alan Maguire
Cc: andrii.nakryiko, jolsa, ast, daniel, eddyz87, martin.lau, song,
yhs, john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf
Em Wed, Oct 25, 2023 at 02:48:50PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Oct 25, 2023 at 02:43:02PM -0300, Arnaldo Carvalho de Melo escreveu:
> > But 'bpftool bpf' doesn't like it:
>
> > $ bpftool btf dump file vmlinux.v5.19.0-rc5+.enum64 raw
> > Error: failed to load BTF from vmlinux.v5.19.0-rc5+.enum64: Invalid argument
> > $
>
> > But it doesn't like it even when not using --btf_features :-\
> >
> > $ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.default_btf_encode ; pahole --btf_encode vmlinux.v5.19.0-rc5+.default_btf_encode
> > $ bpftool btf dump file vmlinux.v5.19.0-rc5+.default_btf_encode raw | wc -l
> > Error: failed to load BTF from vmlinux.v5.19.0-rc5+.default_btf_encode: Invalid argument
> > 0
> > $
>
> > I'll try to root cause this problem...
>
> Random old bpftool on this notebook was the cause, nevermind, I'm back
> testing this, sorry for the noise :-)
Now things look better:
$ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.default_btf_encode ; pahole --btf_encode vmlinux.v5.19.0-rc5+.default_btf_encode
$ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.enum64 ; pahole --btf_encode --btf_features=enum64 vmlinux.v5.19.0-rc5+.enum64
$ bpftool btf dump file vmlinux.v5.19.0-rc5+.enum64 format raw > vmlinux.v5.19.0-rc5+.dump.enum64
$ bpftool btf dump file vmlinux.v5.19.0-rc5+.default_btf_encode format raw > vmlinux.v5.19.0-rc5+.dump.original.default_btf_encode
$ grep '^\[' vmlinux.v5.19.0-rc5+.dump.enum64 | cut -d ' ' -f 2 | sort | uniq -c | sort -k2 > enum64
$ grep '^\[' vmlinux.v5.19.0-rc5+.dump.original.default_btf_encode | cut -d ' ' -f 2 | sort | uniq -c | sort -k2 > original.default_btf_encode
$
$ diff -u original.default_btf_encode enum64
--- original.default_btf_encode 2023-10-25 14:56:45.027645981 -0300
+++ enum64 2023-10-25 14:54:15.024317995 -0300
@@ -1,6 +1,5 @@
3677 ARRAY
3362 CONST
- 1 DATASEC
2109 ENUM
11 ENUM64
54147 FUNC
@@ -12,5 +11,4 @@
10788 STRUCT
2394 TYPEDEF
1806 UNION
- 386 VAR
23 VOLATILE
$
Now with just "var"
$ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.var ; pahole --btf_encode --btf_features=var vmlinux.v5.19.0-rc5+.var
$ bpftool btf dump file vmlinux.v5.19.0-rc5+.var format raw > vmlinux.v5.19.0-rc5+.dump.var
$ grep '^\[' vmlinux.v5.19.0-rc5+.dump.var | cut -d ' ' -f 2 | sort | uniq -c | sort -k2 > var
$ diff -u original.default_btf_encode var
--- original.default_btf_encode 2023-10-25 14:56:45.027645981 -0300
+++ var 2023-10-25 15:04:24.231228667 -0300
@@ -1,8 +1,7 @@
3677 ARRAY
3362 CONST
1 DATASEC
- 2109 ENUM
- 11 ENUM64
+ 2120 ENUM
54147 FUNC
29055 FUNC_PROTO
111 FWD
$
vars/datasecs are not removed and enum64 is not encoded, remaining as
enum, so adding the 11 enum64 to the 2109 enums to get 2120 enums.
$ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.float ; pahole --btf_encode --btf_features=float vmlinux.v5.19.0-rc5+.float
$ bpftool btf dump file vmlinux.v5.19.0-rc5+.float format raw > vmlinux.v5.19.0-rc5+.dump.float
$ grep '^\[' vmlinux.v5.19.0-rc5+.dump.float | cut -d ' ' -f 2 | sort | uniq -c | sort -k2 > float
$ diff -u original.default_btf_encode float
--- original.default_btf_encode 2023-10-25 14:56:45.027645981 -0300
+++ float 2023-10-25 15:06:57.441315272 -0300
@@ -1,16 +1,14 @@
3677 ARRAY
3362 CONST
- 1 DATASEC
- 2109 ENUM
- 11 ENUM64
+ 2120 ENUM
+ 2 FLOAT
54147 FUNC
29055 FUNC_PROTO
111 FWD
- 17 INT
+ 15 INT
15345 PTR
4 RESTRICT
10788 STRUCT
2394 TYPEDEF
1806 UNION
- 386 VAR
23 VOLATILE
$
vars/datasecs are gone, enums combined, and floats are produced out of ints.
I'll try to script all this so that we we can have it in btfdiff or
another script to compare BTF from two files and then use in another
script to check that the differences are the ones expected for the
combinations of btf_features.
But I guess the acks/reviews + my tests are enough to merge this as-is,
thanks for your work on this!
- Arnaldo
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-25 17:43 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
2023-10-25 17:48 ` Arnaldo Carvalho de Melo
@ 2023-10-25 18:18 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 16+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-10-25 18:18 UTC (permalink / raw)
To: Alan Maguire
Cc: andrii.nakryiko, jolsa, ast, daniel, eddyz87, martin.lau, song,
yhs, john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf
Em Wed, Oct 25, 2023 at 02:43:02PM -0300, Arnaldo Carvalho de Melo escreveu:
> $ cp vmlinux.v5.19.0-rc5+ vmlinux.v5.19.0-rc5+.enum64 ; pahole --btf_encode --btf_features=enum64 vmlinux.v5.19.0-rc5+.enum64
> $
> I tried using --btf_encode_detached=file but then couldn't find a way to
> make 'bpftool btf' to consume detached BTF, it seems that "file" means
> "ELF file containing BTF" so I copied the original file to then reencode
> BTF selecting just the enum64 feature, the resulting file continues to
> have the original DWARF and the BTF using that --btf_features set:
This was another symptom of me using a random old bpftool, using
upstream I get what is expected:
$ pahole --btf_encode_detached=vmlinux.v5.19.0-rc5+.enum64 --btf_features=enum64 vmlinux.v5.19.0-rc5+
$ bpftool btf dump file vmlinux.v5.19.0-rc5+.enum64 format raw | wc -l
290975
$ file vmlinux.v5.19.0-rc5+.enum64
vmlinux.v5.19.0-rc5+.enum64: data
$
[acme@quaco pahole]$ bpftool btf dump file vmlinux.v5.19.0-rc5+.enum64 format raw | grep -w ENUM64
[4266] ENUM64 'perf_event_sample_format' encoding=UNSIGNED size=8 vlen=27
[5089] ENUM64 '(anon)' encoding=UNSIGNED size=8 vlen=11
[6727] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=28
[27943] ENUM64 '(anon)' encoding=UNSIGNED size=8 vlen=3
[31242] ENUM64 'netdev_priv_flags' encoding=UNSIGNED size=8 vlen=33
[31438] ENUM64 'perf_callchain_context' encoding=UNSIGNED size=8 vlen=7
[38853] ENUM64 'hmm_pfn_flags' encoding=UNSIGNED size=8 vlen=7
[56830] ENUM64 'ib_uverbs_device_cap_flags' encoding=UNSIGNED size=8 vlen=25
[60295] ENUM64 'blake2b_iv' encoding=UNSIGNED size=8 vlen=8
[63498] ENUM64 '(anon)' encoding=UNSIGNED size=8 vlen=31
[93914] ENUM64 '(anon)' encoding=SIGNED size=8 vlen=172
[acme@quaco pahole]$
So sorry for the noise about this.
- Arnaldo
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-25 18:12 ` Arnaldo Carvalho de Melo
@ 2023-10-25 18:30 ` Arnaldo Carvalho de Melo
2023-10-25 22:28 ` Eduard Zingerman
0 siblings, 1 reply; 16+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-10-25 18:30 UTC (permalink / raw)
To: Alan Maguire, Andrii Nakryiko
Cc: Jiri Olsa, ast, daniel, eddyz87, martin.lau, song, yhs,
john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf
Em Wed, Oct 25, 2023 at 03:12:49PM -0300, Arnaldo Carvalho de Melo escreveu:
> But I guess the acks/reviews + my tests are enough to merge this as-is,
> thanks for your work on this!
Ok, its in the 'next' branch so that it can go thru:
https://github.com/libbpf/libbpf/actions/workflows/pahole.yml
But the previous days are all failures, probably something else is
preventing this test from succeeding? Andrii?
- Arnaldo
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-25 18:30 ` Arnaldo Carvalho de Melo
@ 2023-10-25 22:28 ` Eduard Zingerman
2023-10-26 22:06 ` Andrii Nakryiko
0 siblings, 1 reply; 16+ messages in thread
From: Eduard Zingerman @ 2023-10-25 22:28 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Alan Maguire, Andrii Nakryiko
Cc: Jiri Olsa, ast, daniel, martin.lau, song, yhs, john.fastabend,
kpsingh, sdf, haoluo, mykolal, bpf
On Wed, 2023-10-25 at 15:30 -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Oct 25, 2023 at 03:12:49PM -0300, Arnaldo Carvalho de Melo escreveu:
> > But I guess the acks/reviews + my tests are enough to merge this as-is,
> > thanks for your work on this!
>
> Ok, its in the 'next' branch so that it can go thru:
>
> https://github.com/libbpf/libbpf/actions/workflows/pahole.yml
>
> But the previous days are all failures, probably something else is
> preventing this test from succeeding? Andrii?
It looks like the latest run succeeded, while a number of previous
runs got locked up for some reason. All using the same kernel
checkpoint commit. I know how to setup local github runner,
so I can try to replicate this by forking the repo,
redirecting CI to my machine and executing it several times.
Will do this over the weekend, need to work on some verifier
bugs first.
Thanks,
Eduard.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-25 22:28 ` Eduard Zingerman
@ 2023-10-26 22:06 ` Andrii Nakryiko
2023-10-27 13:25 ` Arnaldo Carvalho de Melo
2023-10-27 14:24 ` RCU stall issues in bpf-next (was: Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features) Alan Maguire
0 siblings, 2 replies; 16+ messages in thread
From: Andrii Nakryiko @ 2023-10-26 22:06 UTC (permalink / raw)
To: Eduard Zingerman
Cc: Arnaldo Carvalho de Melo, Alan Maguire, Jiri Olsa, ast, daniel,
martin.lau, song, yhs, john.fastabend, kpsingh, sdf, haoluo,
mykolal, bpf
On Wed, Oct 25, 2023 at 3:28 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Wed, 2023-10-25 at 15:30 -0300, Arnaldo Carvalho de Melo wrote:
> > Em Wed, Oct 25, 2023 at 03:12:49PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > But I guess the acks/reviews + my tests are enough to merge this as-is,
> > > thanks for your work on this!
> >
> > Ok, its in the 'next' branch so that it can go thru:
> >
> > https://github.com/libbpf/libbpf/actions/workflows/pahole.yml
> >
> > But the previous days are all failures, probably something else is
> > preventing this test from succeeding? Andrii?
>
> It looks like the latest run succeeded, while a number of previous
> runs got locked up for some reason. All using the same kernel
> checkpoint commit. I know how to setup local github runner,
> so I can try to replicate this by forking the repo,
> redirecting CI to my machine and executing it several times.
> Will do this over the weekend, need to work on some verifier
> bugs first.
>
BPF selftests are extremely unreliable under slow Github runners,
unfortunately. Kernel either crashes or locks up very frequently. It
has nothing to do with libbpf and we don't seem to see this in BPF CI
due to having much faster runners there.
I'm not sure what to do about this apart from trying to identify a
selftest that causes lock up (extremely time consuming endeavor) or
just wait till libbpf CI will be privileged enough to gain its own
fast AWS-based worker :)
But it seems like the last scheduled run succeeded, I think you are good.
> Thanks,
> Eduard.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features
2023-10-26 22:06 ` Andrii Nakryiko
@ 2023-10-27 13:25 ` Arnaldo Carvalho de Melo
2023-10-27 14:24 ` RCU stall issues in bpf-next (was: Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features) Alan Maguire
1 sibling, 0 replies; 16+ messages in thread
From: Arnaldo Carvalho de Melo @ 2023-10-27 13:25 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Eduard Zingerman, Arnaldo Carvalho de Melo, Alan Maguire,
Jiri Olsa, ast, daniel, martin.lau, song, yhs, john.fastabend,
kpsingh, sdf, haoluo, mykolal, bpf
Em Thu, Oct 26, 2023 at 03:06:15PM -0700, Andrii Nakryiko escreveu:
> On Wed, Oct 25, 2023 at 3:28 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> >
> > On Wed, 2023-10-25 at 15:30 -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Wed, Oct 25, 2023 at 03:12:49PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > But I guess the acks/reviews + my tests are enough to merge this as-is,
> > > > thanks for your work on this!
> > >
> > > Ok, its in the 'next' branch so that it can go thru:
> > >
> > > https://github.com/libbpf/libbpf/actions/workflows/pahole.yml
> > >
> > > But the previous days are all failures, probably something else is
> > > preventing this test from succeeding? Andrii?
> >
> > It looks like the latest run succeeded, while a number of previous
> > runs got locked up for some reason. All using the same kernel
> > checkpoint commit. I know how to setup local github runner,
> > so I can try to replicate this by forking the repo,
> > redirecting CI to my machine and executing it several times.
> > Will do this over the weekend, need to work on some verifier
> > bugs first.
> >
>
> BPF selftests are extremely unreliable under slow Github runners,
> unfortunately. Kernel either crashes or locks up very frequently. It
> has nothing to do with libbpf and we don't seem to see this in BPF CI
> due to having much faster runners there.
>
> I'm not sure what to do about this apart from trying to identify a
> selftest that causes lock up (extremely time consuming endeavor) or
> just wait till libbpf CI will be privileged enough to gain its own
> fast AWS-based worker :)
>
> But it seems like the last scheduled run succeeded, I think you are good.
I'm not sure it got the btf_features patch, I'll try to change the cmake
files to print the HEAD so that when looking at the output in the github
actions I can be sure that it is using what needs to be tested.
- Arnaldo
^ permalink raw reply [flat|nested] 16+ messages in thread
* RCU stall issues in bpf-next (was: Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features)
2023-10-26 22:06 ` Andrii Nakryiko
2023-10-27 13:25 ` Arnaldo Carvalho de Melo
@ 2023-10-27 14:24 ` Alan Maguire
2023-10-27 19:54 ` Jiri Olsa
1 sibling, 1 reply; 16+ messages in thread
From: Alan Maguire @ 2023-10-27 14:24 UTC (permalink / raw)
To: Andrii Nakryiko, Eduard Zingerman
Cc: Arnaldo Carvalho de Melo, Jiri Olsa, ast, daniel, martin.lau,
song, yhs, john.fastabend, kpsingh, sdf, haoluo, mykolal, bpf
On 26/10/2023 23:06, Andrii Nakryiko wrote:
> On Wed, Oct 25, 2023 at 3:28 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>>
>> On Wed, 2023-10-25 at 15:30 -0300, Arnaldo Carvalho de Melo wrote:
>>> Em Wed, Oct 25, 2023 at 03:12:49PM -0300, Arnaldo Carvalho de Melo escreveu:
>>>> But I guess the acks/reviews + my tests are enough to merge this as-is,
>>>> thanks for your work on this!
>>>
>>> Ok, its in the 'next' branch so that it can go thru:
>>>
>>> https://github.com/libbpf/libbpf/actions/workflows/pahole.yml
>>>
>>> But the previous days are all failures, probably something else is
>>> preventing this test from succeeding? Andrii?
>>
>> It looks like the latest run succeeded, while a number of previous
>> runs got locked up for some reason. All using the same kernel
>> checkpoint commit. I know how to setup local github runner,
>> so I can try to replicate this by forking the repo,
>> redirecting CI to my machine and executing it several times.
>> Will do this over the weekend, need to work on some verifier
>> bugs first.
>>
>
> BPF selftests are extremely unreliable under slow Github runners,
> unfortunately. Kernel either crashes or locks up very frequently. It
> has nothing to do with libbpf and we don't seem to see this in BPF CI
> due to having much faster runners there.
>
> I'm not sure what to do about this apart from trying to identify a
> selftest that causes lock up (extremely time consuming endeavor) or
> just wait till libbpf CI will be privileged enough to gain its own
> fast AWS-based worker :)
>
> But it seems like the last scheduled run succeeded, I think you are good.
>
Perhaps related, I've starting seeing regular RCU stalls with bpf-next
based kernels when running selftests that seem to have the
kprobe_multi_link_prog_run tests as a common factor. The system goes out
to lunch for a while and sometimes returns depending on which CPUs are hit.
In the same run I've also just run into a test hang after the
send_signal_sched_switch test completes; judging by the stack
traces and the alphabetical order it looks like setget_sockopt.c
is the culprit there. Seems to be a socket locking issue.
dmesg output follows for both issues..
[ 4151.256451] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 4151.273835] rcu: 2-....: (4 ticks this GP)
idle=2d34/1/0x4000000000000000 softirq=194674/194674 fqs=3185
[ 4151.293115] rcu: (detected by 0, t=60043 jiffies, g=201485, q=5607
ncpus=8)
[ 4151.309961] Sending NMI from CPU 0 to CPUs 2:
[ 4151.322250] NMI backtrace for cpu 2
[ 4151.322295] CPU: 2 PID: 1101 Comm: systemd-journal Kdump: loaded
Tainted: G O 6.6.0-rc7+ #159
[ 4151.322327] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.5.1 06/16/2021
[ 4151.322362] RIP: 0010:fprobe_handler+0x2d/0x2b0
[ 4151.322430] Code: fa 55 48 89 e5 41 57 49 89 cf 41 56 41 55 49 89 f5
41 54 49 89 fc 53 48 89 d3 48 83 ec 08 48 85 d2 74 0d f6 82 c8 00 00 00
01 <0f> 85 4f 01 00 00 65 4c 8b 34 25 80 27 03 00 49 8b b6 d0 13 00 00
[ 4151.322463] RSP: 0018:ffffc900001208a0 EFLAGS: 00000046
[ 4151.322525] RAX: ffff88811336b380 RBX: ffff88810614ca40 RCX:
ffffc900001208e0
[ 4151.322558] RDX: ffff88810614ca40 RSI: ffffffffb4666b5d RDI:
ffffffffb4584884
[ 4151.322585] RBP: ffffc900001208d0 R08: 0000000000000000 R09:
0000000000000000
[ 4151.322619] R10: 0000000000000000 R11: 0000000000000000 R12:
ffffffffb4584884
[ 4151.322658] R13: ffffffffb4666b5d R14: ffffffffffffffef R15:
ffffc900001208e0
[ 4151.322694] FS: 00007f7ec44b4a40(0000) GS:ffff888237c80000(0000)
knlGS:0000000000000000
[ 4151.322725] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4151.322757] CR2: 00007f7ebb9462d0 CR3: 0000000101b10003 CR4:
0000000000770ee0
[ 4151.322789] PKRU: 55555554
[ 4151.322818] Call Trace:
[ 4151.322856] <NMI>
[ 4151.322926] ? show_regs+0x69/0x80
[ 4151.323045] ? nmi_cpu_backtrace+0xa0/0x110
[ 4151.323271] ? nmi_cpu_backtrace_handler+0x15/0x20
[ 4151.323356] ? nmi_handle+0x64/0x160
[ 4151.323526] ? fprobe_handler+0x2d/0x2b0
[ 4151.323687] ? default_do_nmi+0x6e/0x180
[ 4151.323855] ? exc_nmi+0x151/0x1d0
[ 4151.324015] ? end_repeat_nmi+0x16/0x67
[ 4151.324114] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
[ 4151.324172] ? __rcu_read_unlock+0x4/0x40
[ 4151.324459] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
[ 4151.324509] ? __rcu_read_unlock+0x4/0x40
[ 4151.324585] ? fprobe_handler+0x2d/0x2b0
[ 4151.324755] ? fprobe_handler+0x2d/0x2b0
[ 4151.324958] ? fprobe_handler+0x2d/0x2b0
[ 4151.325102] </NMI>
[ 4151.325130] <IRQ>
[ 4151.325749] ? __rcu_read_unlock+0x9/0x40
[ 4151.325918] ? __rcu_read_unlock+0x9/0x40
[ 4151.325996] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
[ 4151.326097] ? __rcu_read_unlock+0x9/0x40
[ 4151.326146] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
[ 4151.326224] ? __pfx_copy_from_kernel_nofault_allowed+0x10/0x10
[ 4151.326360] ? copy_from_kernel_nofault+0x26/0xf0
[ 4151.326460] ? kprobe_multi_link_handler+0x28/0x40
[ 4151.326542] ? copy_from_kernel_nofault_allowed+0x4/0x50
[ 4151.326615] ? fprobe_handler+0x12f/0x2b0
[ 4151.326696] ? __pfx_irq_enter_rcu+0x10/0x10
[ 4151.327030] ? __pfx_irq_enter_rcu+0x10/0x10
[ 4151.327282] ? __pfx_irq_enter_rcu+0x10/0x10
[ 4151.327357] ? copy_from_kernel_nofault_allowed+0x9/0x50
[ 4151.327545] ? copy_from_kernel_nofault_allowed+0x9/0x50
[ 4151.327631] ? copy_from_kernel_nofault+0x26/0xf0
[ 4151.327737] ? copy_from_kernel_nofault_allowed+0x9/0x50
[ 4151.327786] ? copy_from_kernel_nofault+0x26/0xf0
[ 4151.327833] ? irq_enter_rcu+0x4/0x80
[ 4151.327884] ? __pfx_irq_enter_rcu+0x10/0x10
[ 4151.327934] ? sysvec_irq_work+0x4f/0xe0
[ 4151.328011] ? get_entry_ip+0x32/0x80
[ 4151.328191] ? kprobe_multi_link_handler+0x19/0x40
[ 4151.328271] ? irq_enter_rcu+0x4/0x80
[ 4151.328349] ? fprobe_handler+0x12f/0x2b0
[ 4151.328904] ? early_xen_iret_patch+0xc/0xc
[ 4151.329053] ? irq_enter_rcu+0x9/0x80
[ 4151.329230] ? irq_enter_rcu+0x9/0x80
[ 4151.329308] ? sysvec_irq_work+0x4f/0xe0
[ 4151.329415] ? irq_enter_rcu+0x9/0x80
[ 4151.329465] ? sysvec_irq_work+0x4f/0xe0
[ 4151.329599] ? asm_sysvec_irq_work+0x1f/0x30
[ 4151.330062] ? rcu_read_unlock_special+0xd5/0x190
[ 4151.330211] ? rcu_read_unlock_special+0xcf/0x190
[ 4151.330419] ? __rcu_read_unlock+0x33/0x40
[ 4151.330495] ? uncharge_batch+0xe8/0x140
[ 4151.330618] ? __mem_cgroup_uncharge+0x70/0x90
[ 4151.330860] ? __folio_put+0x36/0x80
[ 4151.330964] ? free_page_and_swap_cache+0x87/0x90
[ 4151.331064] ? tlb_remove_table_rcu+0x2c/0x50
[ 4151.331189] ? rcu_do_batch+0x1bf/0x550
[ 4151.331264] ? rcu_do_batch+0x159/0x550
[ 4151.331614] ? rcu_core+0x183/0x370
[ 4151.331794] ? rcu_core_si+0x12/0x20
[ 4151.331870] ? __do_softirq+0x10e/0x327
[ 4151.332187] ? __irq_exit_rcu+0xa8/0x110
[ 4151.332295] ? irq_exit_rcu+0x12/0x20
[ 4151.332374] ? sysvec_apic_timer_interrupt+0xb0/0xe0
[ 4151.332450] </IRQ>
[ 4151.332479] <TASK>
[ 4151.332595] ? asm_sysvec_apic_timer_interrupt+0x1f/0x30
[ 4151.333055] ? _raw_spin_unlock_irqrestore+0x21/0x50
[ 4151.333227] ? __wake_up_common_lock+0x8a/0xc0
[ 4151.333644] ? __wake_up+0x17/0x20
[ 4151.333717] ? fsnotify_insert_event+0x121/0x1a0
[ 4151.334004] ? inotify_handle_inode_event+0x158/0x240
[ 4151.334336] ? fsnotify_handle_inode_event.isra.0+0x6f/0xe0
[ 4151.334410] ? send_to_group+0x2ad/0x320
[ 4151.334771] ? fsnotify+0x25c/0x5a0
[ 4151.335385] ? __fsnotify_parent+0x21b/0x330
[ 4151.335698] ? __fsnotify_parent+0x9/0x330
[ 4151.335963] ? notify_change+0x42f/0x4c0
[ 4151.336018] ? __fsnotify_parent+0x9/0x330
[ 4151.336070] ? notify_change+0x42f/0x4c0
[ 4151.336173] ? notify_change+0x9/0x4c0
[ 4151.336474] ? do_truncate+0x87/0xe0
[ 4151.336528] ? do_truncate+0x87/0xe0
[ 4151.336960] ? do_sys_ftruncate+0x1af/0x210
[ 4151.337209] ? __x64_sys_ftruncate+0x1e/0x30
[ 4151.337285] ? do_syscall_64+0x3e/0x90
[ 4151.337382] ? entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[ 4151.337984] </TASK>
[ 4151.338018] INFO: NMI handler (nmi_cpu_backtrace_handler) took too
long to run: 15.770 msecs
[ 4154.063929] rcu: INFO: rcu_preempt detected expedited stalls on
CPUs/tasks: { } 72585 jiffies s: 2429 root: 0x0/.
[ 4200.980354] 8021q: 802.1Q VLAN Support v1.8
[ 4200.982157] 8021q: adding VLAN 0 to HW filter on device ens3
[ 4254.123516] test_progs[47486] is installing a program with
bpf_probe_write_user helper that may corrupt user memory!
[ 4254.123530] test_progs[47486] is installing a program with
bpf_probe_write_user helper that may corrupt user memory!
This second issue causes a hang in selftest execution as
described above after send_signal_sched_switch
[ 4263.776211] ------------[ cut here ]------------
[ 4263.781022] Voluntary context switch within RCU read-side critical
section!
[ 4263.781036] WARNING: CPU: 2 PID: 47486 at
kernel/rcu/tree_plugin.h:320 rcu_note_context_switch+0x2dd/0x320
[ 4263.783303] Modules linked in: sch_fq 8021q garp mrp dummy tun ipip
tunnel4 ip_tunnel bpf_preload bpf_testmod(O) veth cls_bpf sch_ingress
ipt_REJECT xt_comment xt_owner nft_compat nft_fib_inet nft_fib_ipv4
nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
nft_reject nft_ct nft_chain_nat nf_nat ip_set cuse vfat fat
intel_rapl_msr intel_rapl_common bochs kvm_amd drm_vram_helper ccp
drm_ttm_helper ttm kvm drm_kms_helper irqbypass drm joydev pcspkr
i2c_piix4 sch_fq_codel xfs iscsi_tcp libiscsi_tcp libiscsi nvme_tcp
nvme_fabrics nvme nvme_core sd_mod t10_pi crc64_rocksoft_generic
crc64_rocksoft sg virtio_net net_failover failover virtio_scsi
crct10dif_pclmul crc32_pclmul ghash_clmulni_intel sha512_ssse3
ata_generic pata_acpi ata_piix aesni_intel crypto_simd cryptd virtio_pci
virtio_pci_legacy_dev serio_raw virtio_pci_modern_dev libata floppy
qemu_fw_cfg dm_multipath sunrpc dm_mirror dm_region_hash dm_log dm_mod
scsi_transport_iscsi ipmi_devintf ipmi_msghandler fuse [last unloaded:
bpf_testmod(O)]
[ 4263.795254] CPU: 2 PID: 47486 Comm: new_name Kdump: loaded Tainted: G
O 6.6.0-rc7+ #159
[ 4263.796602] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.5.1 06/16/2021
[ 4263.797804] RIP: 0010:rcu_note_context_switch+0x2dd/0x320
[ 4263.798772] Code: 08 f0 83 44 24 fc 00 48 89 de 4c 89 f7 e8 5b 98 ff
ff e9 f0 fd ff ff 48 c7 c7 d0 00 d3 b5 c6 05 96 1b 3b 02 01 e8 c3 19 f4
ff <0f> 0b e9 83 fd ff ff a9 ff ff ff 7f 0f 84 3f fe ff ff 65 48 8b 3c
[ 4263.801495] RSP: 0018:ffffc9000f4f7b68 EFLAGS: 00010082
[ 4263.802493] RAX: 0000000000000000 RBX: ffff888237cb3e00 RCX:
0000000000000000
[ 4263.803676] RDX: 0000000000000003 RSI: ffffffffb5cdb4f1 RDI:
00000000ffffffff
[ 4263.804842] RBP: ffffc9000f4f7b88 R08: 0000000000000000 R09:
ffffc9000f4f79d0
[ 4263.805989] R10: 0000000000000003 R11: ffffffffb67f4908 R12:
0000000000000000
[ 4263.807158] R13: ffff8881003acd40 R14: 0000000000000001 R15:
00000000000000e1
[ 4263.808334] FS: 00007fd0577417c0(0000) GS:ffff888237c80000(0000)
knlGS:0000000000000000
[ 4263.809625] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4263.810690] CR2: 0000000006954568 CR3: 000000010ecb2004 CR4:
0000000000770ee0
[ 4263.811903] PKRU: 55555554
[ 4263.812656] Call Trace:
[ 4263.813401] <TASK>
[ 4263.814095] ? show_regs+0x69/0x80
[ 4263.814897] ? __warn+0x8d/0x150
[ 4263.815713] ? rcu_note_context_switch+0x2dd/0x320
[ 4263.816686] ? report_bug+0x196/0x1c0
[ 4263.817549] ? srso_alias_return_thunk+0x5/0x7f
[ 4263.818503] ? irq_work_queue+0x13/0x60
[ 4263.819387] ? handle_bug+0x45/0x80
[ 4263.820219] ? exc_invalid_op+0x1c/0x70
[ 4263.821089] ? asm_exc_invalid_op+0x1f/0x30
[ 4263.821994] ? rcu_note_context_switch+0x2dd/0x320
[ 4263.822961] ? rcu_note_context_switch+0x2dd/0x320
[ 4263.823916] __schedule+0x60/0x770
[ 4263.824765] ? __raw_spin_lock_irqsave+0x23/0x60
[ 4263.825720] ? srso_alias_return_thunk+0x5/0x7f
[ 4263.826662] ? _raw_spin_unlock_irqrestore+0x2b/0x50
[ 4263.827617] schedule+0x6a/0xf0
[ 4263.828383] __lock_sock+0x7d/0xc0
[ 4263.829178] ? __pfx_autoremove_wake_function+0x10/0x10
[ 4263.830145] lock_sock_nested+0x54/0x60
[ 4263.830982] do_ip_setsockopt+0x10b4/0x11a0
[ 4263.831859] ? srso_alias_return_thunk+0x5/0x7f
[ 4263.832766] ? srso_alias_return_thunk+0x5/0x7f
[ 4263.833655] sol_ip_sockopt+0x3c/0x80
[ 4263.834444] __bpf_setsockopt+0x5f/0xa0
[ 4263.835222] bpf_sock_ops_setsockopt+0x19/0x30
[ 4263.836064] bpf_prog_e2095e3611d57c9f_bpf_test_sockopt_int+0xbc/0x127
[ 4263.837110] bpf_prog_493685a3bae00bbd_bpf_test_ip_sockopt+0x49/0x4f
[ 4263.838124] bpf_prog_e80acd4ed535c5e2_skops_sockopt+0x433/0xe95
[ 4263.839102] ? migrate_disable+0x7b/0x90
[ 4263.839865] __cgroup_bpf_run_filter_sock_ops+0x91/0x140
[ 4263.840769] __inet_listen_sk+0x106/0x130
[ 4263.841529] ? lock_sock_nested+0x43/0x60
[ 4263.842259] inet_listen+0x4d/0x70
[ 4263.842935] __sys_listen+0x75/0xc0
[ 4263.843624] __x64_sys_listen+0x18/0x20
[ 4263.844309] do_syscall_64+0x3e/0x90
[ 4263.844969] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[ 4263.845788] RIP: 0033:0x7fd156f9c82b
[ 4263.846461] Code: f0 ff ff 73 01 c3 48 8b 0d 52 46 38 00 f7 d8 64 89
01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 32 00 00 00 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 25 46 38 00 f7 d8 64 89 01 48
[ 4263.848837] RSP: 002b:00007ffda4b93ed8 EFLAGS: 00000246 ORIG_RAX:
0000000000000032
[ 4263.849892] RAX: ffffffffffffffda RBX: 00000000068d85b0 RCX:
00007fd156f9c82b
[ 4263.850912] RDX: 0000000000000010 RSI: 0000000000000001 RDI:
000000000000002c
[ 4263.851932] RBP: 00007ffda4b93f20 R08: 0000000000000010 R09:
0000000000000000
[ 4263.852953] R10: 00007ffda4b93eb0 R11: 0000000000000246 R12:
0000000000411fa0
[ 4263.853975] R13: 00007ffda4b94370 R14: 0000000000000000 R15:
0000000000000000
[ 4263.855011] </TASK>
[ 4263.855564] ---[ end trace 0000000000000000 ]---
[ 4323.847899] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 4323.849648] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7):
P47486/1:b..l
[ 4323.850749] rcu: (detected by 4, t=60004 jiffies, g=212293, q=44343
ncpus=8)
[ 4323.851852] task:new_name state:D stack:0 pid:47486
ppid:47485 flags:0x00000000
[ 4323.853085] Call Trace:
[ 4323.853711] <TASK>
[ 4323.854296] __schedule+0x265/0x770
[ 4323.855023] ? __raw_spin_lock_irqsave+0x23/0x60
[ 4323.855861] schedule+0x6a/0xf0
[ 4323.856547] __lock_sock+0x7d/0xc0
[ 4323.857271] ? __pfx_autoremove_wake_function+0x10/0x10
[ 4323.858168] lock_sock_nested+0x54/0x60
[ 4323.858923] do_ip_setsockopt+0x10b4/0x11a0
[ 4323.859710] ? srso_alias_return_thunk+0x5/0x7f
[ 4323.860545] ? srso_alias_return_thunk+0x5/0x7f
[ 4323.861373] sol_ip_sockopt+0x3c/0x80
[ 4323.862093] __bpf_setsockopt+0x5f/0xa0
[ 4323.862837] bpf_sock_ops_setsockopt+0x19/0x30
[ 4323.863643] bpf_prog_e2095e3611d57c9f_bpf_test_sockopt_int+0xbc/0x127
[ 4323.864668] bpf_prog_493685a3bae00bbd_bpf_test_ip_sockopt+0x49/0x4f
[ 4323.865678] bpf_prog_e80acd4ed535c5e2_skops_sockopt+0x433/0xe95
[ 4323.866664] ? migrate_disable+0x7b/0x90
[ 4323.867441] __cgroup_bpf_run_filter_sock_ops+0x91/0x140
[ 4323.868356] __inet_listen_sk+0x106/0x130
[ 4323.869117] ? lock_sock_nested+0x43/0x60
[ 4323.869885] inet_listen+0x4d/0x70
[ 4323.870595] __sys_listen+0x75/0xc0
[ 4323.871300] __x64_sys_listen+0x18/0x20
[ 4323.872042] do_syscall_64+0x3e/0x90
[ 4323.872761] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[ 4323.873637] RIP: 0033:0x7fd156f9c82b
[ 4323.874387] RSP: 002b:00007ffda4b93ed8 EFLAGS: 00000246 ORIG_RAX:
0000000000000032
[ 4323.875496] RAX: ffffffffffffffda RBX: 00000000068d85b0 RCX:
00007fd156f9c82b
[ 4323.876583] RDX: 0000000000000010 RSI: 0000000000000001 RDI:
000000000000002c
[ 4323.877679] RBP: 00007ffda4b93f20 R08: 0000000000000010 R09:
0000000000000000
[ 4323.878774] R10: 00007ffda4b93eb0 R11: 0000000000000246 R12:
0000000000411fa0
[ 4323.879866] R13: 00007ffda4b94370 R14: 0000000000000000 R15:
0000000000000000
[ 4323.880952] </TASK>
[ 4334.246892] rcu: INFO: rcu_preempt detected expedited stalls on
CPUs/tasks: { P47486 } 61561 jiffies s: 3197 root: 0x0/T
[ 4334.248460] rcu: blocking rcu_node structures (internal RCU debug):
[ 4424.359789] INFO: task new_name:47486 blocked for more than 122 seconds.
[ 4424.361458] Tainted: G W O 6.6.0-rc7+ #159
[ 4424.362418] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 4424.363570] task:new_name state:D stack:0 pid:47486
ppid:47485 flags:0x00000000
[ 4424.364799] Call Trace:
[ 4424.365430] <TASK>
[ 4424.368934] __schedule+0x265/0x770
[ 4424.369663] ? __raw_spin_lock_irqsave+0x23/0x60
[ 4424.370499] schedule+0x6a/0xf0
[ 4424.371189] __lock_sock+0x7d/0xc0
[ 4424.371906] ? __pfx_autoremove_wake_function+0x10/0x10
[ 4424.372794] lock_sock_nested+0x54/0x60
[ 4424.373532] do_ip_setsockopt+0x10b4/0x11a0
[ 4424.374302] ? srso_alias_return_thunk+0x5/0x7f
[ 4424.375127] ? srso_alias_return_thunk+0x5/0x7f
[ 4424.375947] sol_ip_sockopt+0x3c/0x80
[ 4424.376653] __bpf_setsockopt+0x5f/0xa0
[ 4424.377381] bpf_sock_ops_setsockopt+0x19/0x30
[ 4424.378184] bpf_prog_e2095e3611d57c9f_bpf_test_sockopt_int+0xbc/0x127
[ 4424.379190] bpf_prog_493685a3bae00bbd_bpf_test_ip_sockopt+0x49/0x4f
[ 4424.380188] bpf_prog_e80acd4ed535c5e2_skops_sockopt+0x433/0xe95
[ 4424.381150] ? migrate_disable+0x7b/0x90
[ 4424.381885] __cgroup_bpf_run_filter_sock_ops+0x91/0x140
[ 4424.382762] __inet_listen_sk+0x106/0x130
[ 4424.383515] ? lock_sock_nested+0x43/0x60
[ 4424.384290] inet_listen+0x4d/0x70
[ 4424.384992] __sys_listen+0x75/0xc0
[ 4424.385690] __x64_sys_listen+0x18/0x20
[ 4424.386425] do_syscall_64+0x3e/0x90
[ 4424.387150] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[ 4424.387991] RIP: 0033:0x7fd156f9c82b
[ 4424.388702] RSP: 002b:00007ffda4b93ed8 EFLAGS: 00000246 ORIG_RAX:
0000000000000032
[ 4424.389804] RAX: ffffffffffffffda RBX: 00000000068d85b0 RCX:
00007fd156f9c82b
[ 4424.390862] RDX: 0000000000000010 RSI: 0000000000000001 RDI:
000000000000002c
[ 4424.391913] RBP: 00007ffda4b93f20 R08: 0000000000000010 R09:
0000000000000000
[ 4424.392987] R10: 00007ffda4b93eb0 R11: 0000000000000246 R12:
0000000000411fa0
[ 4424.394047] R13: 00007ffda4b94370 R14: 0000000000000000 R15:
0000000000000000
[ 4424.395117] </TASK>
[ 4424.395701] INFO: task dtprobed:49514 blocked for more than 122 seconds.
[ 4424.396751] Tainted: G W O 6.6.0-rc7+ #159
[ 4424.397696] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 4424.398858] task:dtprobed state:D stack:0 pid:49514 ppid:1
flags:0x00000002
[ 4424.400036] Call Trace:
[ 4424.400651] <TASK>
[ 4424.401258] __schedule+0x265/0x770
[ 4424.401975] ? srso_alias_return_thunk+0x5/0x7f
[ 4424.402795] schedule+0x6a/0xf0
[ 4424.403472] synchronize_rcu_expedited+0x185/0x200
[ 4424.404313] ? __pfx_wait_rcu_exp_gp+0x10/0x10
[ 4424.405101] ? __pfx_autoremove_wake_function+0x10/0x10
[ 4424.405966] ? srso_alias_return_thunk+0x5/0x7f
[ 4424.406758] ? preempt_count_add+0x52/0xc0
[ 4424.407499] namespace_unlock+0xd6/0x1b0
[ 4424.408234] put_mnt_ns+0x74/0xa0
[ 4424.408903] free_nsproxy+0x1f/0x1b0
[ 4424.409609] exit_task_namespaces+0x6f/0x90
[ 4424.410388] do_exit+0x28b/0x520
[ 4424.411041] ? srso_alias_return_thunk+0x5/0x7f
[ 4424.411837] ? srso_alias_return_thunk+0x5/0x7f
[ 4424.412635] do_group_exit+0x39/0x90
[ 4424.413351] __x64_sys_exit_group+0x1c/0x20
[ 4424.414106] do_syscall_64+0x3e/0x90
[ 4424.414810] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[ 4424.415650] RIP: 0033:0x7ff70a952cf6
[ 4424.416373] RSP: 002b:00007ffeecec40b8 EFLAGS: 00000246 ORIG_RAX:
00000000000000e7
[ 4424.417494] RAX: ffffffffffffffda RBX: 00007ff70ac14820 RCX:
00007ff70a952cf6
[ 4424.418560] RDX: 0000000000000002 RSI: 000000000000003c RDI:
0000000000000002
[ 4424.419636] RBP: 0000000000000002 R08: 00000000000000e7 R09:
ffffffffffffff68
[ 4424.420689] R10: 00007ffeecec3fbf R11: 0000000000000246 R12:
00007ff70ac14820
[ 4424.421768] R13: 0000000000000001 R14: 00007ff70ac1a310 R15:
0000000000000000
[ 4424.422837] </TASK>
>> Thanks,
>> Eduard.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: RCU stall issues in bpf-next (was: Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features)
2023-10-27 14:24 ` RCU stall issues in bpf-next (was: Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features) Alan Maguire
@ 2023-10-27 19:54 ` Jiri Olsa
0 siblings, 0 replies; 16+ messages in thread
From: Jiri Olsa @ 2023-10-27 19:54 UTC (permalink / raw)
To: Alan Maguire
Cc: Andrii Nakryiko, Eduard Zingerman, Arnaldo Carvalho de Melo, ast,
daniel, martin.lau, song, yhs, john.fastabend, kpsingh, sdf,
haoluo, mykolal, bpf
On Fri, Oct 27, 2023 at 03:24:44PM +0100, Alan Maguire wrote:
> On 26/10/2023 23:06, Andrii Nakryiko wrote:
> > On Wed, Oct 25, 2023 at 3:28 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> >>
> >> On Wed, 2023-10-25 at 15:30 -0300, Arnaldo Carvalho de Melo wrote:
> >>> Em Wed, Oct 25, 2023 at 03:12:49PM -0300, Arnaldo Carvalho de Melo escreveu:
> >>>> But I guess the acks/reviews + my tests are enough to merge this as-is,
> >>>> thanks for your work on this!
> >>>
> >>> Ok, its in the 'next' branch so that it can go thru:
> >>>
> >>> https://github.com/libbpf/libbpf/actions/workflows/pahole.yml
> >>>
> >>> But the previous days are all failures, probably something else is
> >>> preventing this test from succeeding? Andrii?
> >>
> >> It looks like the latest run succeeded, while a number of previous
> >> runs got locked up for some reason. All using the same kernel
> >> checkpoint commit. I know how to setup local github runner,
> >> so I can try to replicate this by forking the repo,
> >> redirecting CI to my machine and executing it several times.
> >> Will do this over the weekend, need to work on some verifier
> >> bugs first.
> >>
> >
> > BPF selftests are extremely unreliable under slow Github runners,
> > unfortunately. Kernel either crashes or locks up very frequently. It
> > has nothing to do with libbpf and we don't seem to see this in BPF CI
> > due to having much faster runners there.
> >
> > I'm not sure what to do about this apart from trying to identify a
> > selftest that causes lock up (extremely time consuming endeavor) or
> > just wait till libbpf CI will be privileged enough to gain its own
> > fast AWS-based worker :)
> >
> > But it seems like the last scheduled run succeeded, I think you are good.
> >
>
> Perhaps related, I've starting seeing regular RCU stalls with bpf-next
> based kernels when running selftests that seem to have the
> kprobe_multi_link_prog_run tests as a common factor. The system goes out
> to lunch for a while and sometimes returns depending on which CPUs are hit.
>
> In the same run I've also just run into a test hang after the
> send_signal_sched_switch test completes; judging by the stack
> traces and the alphabetical order it looks like setget_sockopt.c
> is the culprit there. Seems to be a socket locking issue.
hi,
looks like this might got fixed by this one?
https://lore.kernel.org/bpf/20231027182424.1444845-1-yonghong.song@linux.dev/
thanks,
jirka
>
> dmesg output follows for both issues..
>
> [ 4151.256451] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> [ 4151.273835] rcu: 2-....: (4 ticks this GP)
> idle=2d34/1/0x4000000000000000 softirq=194674/194674 fqs=3185
> [ 4151.293115] rcu: (detected by 0, t=60043 jiffies, g=201485, q=5607
> ncpus=8)
> [ 4151.309961] Sending NMI from CPU 0 to CPUs 2:
> [ 4151.322250] NMI backtrace for cpu 2
> [ 4151.322295] CPU: 2 PID: 1101 Comm: systemd-journal Kdump: loaded
> Tainted: G O 6.6.0-rc7+ #159
> [ 4151.322327] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS 1.5.1 06/16/2021
> [ 4151.322362] RIP: 0010:fprobe_handler+0x2d/0x2b0
> [ 4151.322430] Code: fa 55 48 89 e5 41 57 49 89 cf 41 56 41 55 49 89 f5
> 41 54 49 89 fc 53 48 89 d3 48 83 ec 08 48 85 d2 74 0d f6 82 c8 00 00 00
> 01 <0f> 85 4f 01 00 00 65 4c 8b 34 25 80 27 03 00 49 8b b6 d0 13 00 00
> [ 4151.322463] RSP: 0018:ffffc900001208a0 EFLAGS: 00000046
> [ 4151.322525] RAX: ffff88811336b380 RBX: ffff88810614ca40 RCX:
> ffffc900001208e0
> [ 4151.322558] RDX: ffff88810614ca40 RSI: ffffffffb4666b5d RDI:
> ffffffffb4584884
> [ 4151.322585] RBP: ffffc900001208d0 R08: 0000000000000000 R09:
> 0000000000000000
> [ 4151.322619] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffffffffb4584884
> [ 4151.322658] R13: ffffffffb4666b5d R14: ffffffffffffffef R15:
> ffffc900001208e0
> [ 4151.322694] FS: 00007f7ec44b4a40(0000) GS:ffff888237c80000(0000)
> knlGS:0000000000000000
> [ 4151.322725] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 4151.322757] CR2: 00007f7ebb9462d0 CR3: 0000000101b10003 CR4:
> 0000000000770ee0
> [ 4151.322789] PKRU: 55555554
> [ 4151.322818] Call Trace:
> [ 4151.322856] <NMI>
> [ 4151.322926] ? show_regs+0x69/0x80
> [ 4151.323045] ? nmi_cpu_backtrace+0xa0/0x110
> [ 4151.323271] ? nmi_cpu_backtrace_handler+0x15/0x20
> [ 4151.323356] ? nmi_handle+0x64/0x160
> [ 4151.323526] ? fprobe_handler+0x2d/0x2b0
> [ 4151.323687] ? default_do_nmi+0x6e/0x180
> [ 4151.323855] ? exc_nmi+0x151/0x1d0
> [ 4151.324015] ? end_repeat_nmi+0x16/0x67
> [ 4151.324114] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
> [ 4151.324172] ? __rcu_read_unlock+0x4/0x40
> [ 4151.324459] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
> [ 4151.324509] ? __rcu_read_unlock+0x4/0x40
> [ 4151.324585] ? fprobe_handler+0x2d/0x2b0
> [ 4151.324755] ? fprobe_handler+0x2d/0x2b0
> [ 4151.324958] ? fprobe_handler+0x2d/0x2b0
> [ 4151.325102] </NMI>
> [ 4151.325130] <IRQ>
> [ 4151.325749] ? __rcu_read_unlock+0x9/0x40
> [ 4151.325918] ? __rcu_read_unlock+0x9/0x40
> [ 4151.325996] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
> [ 4151.326097] ? __rcu_read_unlock+0x9/0x40
> [ 4151.326146] ? kprobe_multi_link_prog_run.isra.0+0x9d/0x130
> [ 4151.326224] ? __pfx_copy_from_kernel_nofault_allowed+0x10/0x10
> [ 4151.326360] ? copy_from_kernel_nofault+0x26/0xf0
> [ 4151.326460] ? kprobe_multi_link_handler+0x28/0x40
> [ 4151.326542] ? copy_from_kernel_nofault_allowed+0x4/0x50
> [ 4151.326615] ? fprobe_handler+0x12f/0x2b0
> [ 4151.326696] ? __pfx_irq_enter_rcu+0x10/0x10
> [ 4151.327030] ? __pfx_irq_enter_rcu+0x10/0x10
> [ 4151.327282] ? __pfx_irq_enter_rcu+0x10/0x10
> [ 4151.327357] ? copy_from_kernel_nofault_allowed+0x9/0x50
> [ 4151.327545] ? copy_from_kernel_nofault_allowed+0x9/0x50
> [ 4151.327631] ? copy_from_kernel_nofault+0x26/0xf0
> [ 4151.327737] ? copy_from_kernel_nofault_allowed+0x9/0x50
> [ 4151.327786] ? copy_from_kernel_nofault+0x26/0xf0
> [ 4151.327833] ? irq_enter_rcu+0x4/0x80
> [ 4151.327884] ? __pfx_irq_enter_rcu+0x10/0x10
> [ 4151.327934] ? sysvec_irq_work+0x4f/0xe0
> [ 4151.328011] ? get_entry_ip+0x32/0x80
> [ 4151.328191] ? kprobe_multi_link_handler+0x19/0x40
> [ 4151.328271] ? irq_enter_rcu+0x4/0x80
> [ 4151.328349] ? fprobe_handler+0x12f/0x2b0
> [ 4151.328904] ? early_xen_iret_patch+0xc/0xc
> [ 4151.329053] ? irq_enter_rcu+0x9/0x80
> [ 4151.329230] ? irq_enter_rcu+0x9/0x80
> [ 4151.329308] ? sysvec_irq_work+0x4f/0xe0
> [ 4151.329415] ? irq_enter_rcu+0x9/0x80
> [ 4151.329465] ? sysvec_irq_work+0x4f/0xe0
> [ 4151.329599] ? asm_sysvec_irq_work+0x1f/0x30
> [ 4151.330062] ? rcu_read_unlock_special+0xd5/0x190
> [ 4151.330211] ? rcu_read_unlock_special+0xcf/0x190
> [ 4151.330419] ? __rcu_read_unlock+0x33/0x40
> [ 4151.330495] ? uncharge_batch+0xe8/0x140
> [ 4151.330618] ? __mem_cgroup_uncharge+0x70/0x90
> [ 4151.330860] ? __folio_put+0x36/0x80
> [ 4151.330964] ? free_page_and_swap_cache+0x87/0x90
> [ 4151.331064] ? tlb_remove_table_rcu+0x2c/0x50
> [ 4151.331189] ? rcu_do_batch+0x1bf/0x550
> [ 4151.331264] ? rcu_do_batch+0x159/0x550
> [ 4151.331614] ? rcu_core+0x183/0x370
> [ 4151.331794] ? rcu_core_si+0x12/0x20
> [ 4151.331870] ? __do_softirq+0x10e/0x327
> [ 4151.332187] ? __irq_exit_rcu+0xa8/0x110
> [ 4151.332295] ? irq_exit_rcu+0x12/0x20
> [ 4151.332374] ? sysvec_apic_timer_interrupt+0xb0/0xe0
> [ 4151.332450] </IRQ>
> [ 4151.332479] <TASK>
> [ 4151.332595] ? asm_sysvec_apic_timer_interrupt+0x1f/0x30
> [ 4151.333055] ? _raw_spin_unlock_irqrestore+0x21/0x50
> [ 4151.333227] ? __wake_up_common_lock+0x8a/0xc0
> [ 4151.333644] ? __wake_up+0x17/0x20
> [ 4151.333717] ? fsnotify_insert_event+0x121/0x1a0
> [ 4151.334004] ? inotify_handle_inode_event+0x158/0x240
> [ 4151.334336] ? fsnotify_handle_inode_event.isra.0+0x6f/0xe0
> [ 4151.334410] ? send_to_group+0x2ad/0x320
> [ 4151.334771] ? fsnotify+0x25c/0x5a0
> [ 4151.335385] ? __fsnotify_parent+0x21b/0x330
> [ 4151.335698] ? __fsnotify_parent+0x9/0x330
> [ 4151.335963] ? notify_change+0x42f/0x4c0
> [ 4151.336018] ? __fsnotify_parent+0x9/0x330
> [ 4151.336070] ? notify_change+0x42f/0x4c0
> [ 4151.336173] ? notify_change+0x9/0x4c0
> [ 4151.336474] ? do_truncate+0x87/0xe0
> [ 4151.336528] ? do_truncate+0x87/0xe0
> [ 4151.336960] ? do_sys_ftruncate+0x1af/0x210
> [ 4151.337209] ? __x64_sys_ftruncate+0x1e/0x30
> [ 4151.337285] ? do_syscall_64+0x3e/0x90
> [ 4151.337382] ? entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> [ 4151.337984] </TASK>
> [ 4151.338018] INFO: NMI handler (nmi_cpu_backtrace_handler) took too
> long to run: 15.770 msecs
> [ 4154.063929] rcu: INFO: rcu_preempt detected expedited stalls on
> CPUs/tasks: { } 72585 jiffies s: 2429 root: 0x0/.
> [ 4200.980354] 8021q: 802.1Q VLAN Support v1.8
> [ 4200.982157] 8021q: adding VLAN 0 to HW filter on device ens3
> [ 4254.123516] test_progs[47486] is installing a program with
> bpf_probe_write_user helper that may corrupt user memory!
> [ 4254.123530] test_progs[47486] is installing a program with
> bpf_probe_write_user helper that may corrupt user memory!
>
>
> This second issue causes a hang in selftest execution as
> described above after send_signal_sched_switch
>
> [ 4263.776211] ------------[ cut here ]------------
> [ 4263.781022] Voluntary context switch within RCU read-side critical
> section!
> [ 4263.781036] WARNING: CPU: 2 PID: 47486 at
> kernel/rcu/tree_plugin.h:320 rcu_note_context_switch+0x2dd/0x320
> [ 4263.783303] Modules linked in: sch_fq 8021q garp mrp dummy tun ipip
> tunnel4 ip_tunnel bpf_preload bpf_testmod(O) veth cls_bpf sch_ingress
> ipt_REJECT xt_comment xt_owner nft_compat nft_fib_inet nft_fib_ipv4
> nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6
> nft_reject nft_ct nft_chain_nat nf_nat ip_set cuse vfat fat
> intel_rapl_msr intel_rapl_common bochs kvm_amd drm_vram_helper ccp
> drm_ttm_helper ttm kvm drm_kms_helper irqbypass drm joydev pcspkr
> i2c_piix4 sch_fq_codel xfs iscsi_tcp libiscsi_tcp libiscsi nvme_tcp
> nvme_fabrics nvme nvme_core sd_mod t10_pi crc64_rocksoft_generic
> crc64_rocksoft sg virtio_net net_failover failover virtio_scsi
> crct10dif_pclmul crc32_pclmul ghash_clmulni_intel sha512_ssse3
> ata_generic pata_acpi ata_piix aesni_intel crypto_simd cryptd virtio_pci
> virtio_pci_legacy_dev serio_raw virtio_pci_modern_dev libata floppy
> qemu_fw_cfg dm_multipath sunrpc dm_mirror dm_region_hash dm_log dm_mod
> scsi_transport_iscsi ipmi_devintf ipmi_msghandler fuse [last unloaded:
> bpf_testmod(O)]
> [ 4263.795254] CPU: 2 PID: 47486 Comm: new_name Kdump: loaded Tainted: G
> O 6.6.0-rc7+ #159
> [ 4263.796602] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS 1.5.1 06/16/2021
> [ 4263.797804] RIP: 0010:rcu_note_context_switch+0x2dd/0x320
> [ 4263.798772] Code: 08 f0 83 44 24 fc 00 48 89 de 4c 89 f7 e8 5b 98 ff
> ff e9 f0 fd ff ff 48 c7 c7 d0 00 d3 b5 c6 05 96 1b 3b 02 01 e8 c3 19 f4
> ff <0f> 0b e9 83 fd ff ff a9 ff ff ff 7f 0f 84 3f fe ff ff 65 48 8b 3c
> [ 4263.801495] RSP: 0018:ffffc9000f4f7b68 EFLAGS: 00010082
> [ 4263.802493] RAX: 0000000000000000 RBX: ffff888237cb3e00 RCX:
> 0000000000000000
> [ 4263.803676] RDX: 0000000000000003 RSI: ffffffffb5cdb4f1 RDI:
> 00000000ffffffff
> [ 4263.804842] RBP: ffffc9000f4f7b88 R08: 0000000000000000 R09:
> ffffc9000f4f79d0
> [ 4263.805989] R10: 0000000000000003 R11: ffffffffb67f4908 R12:
> 0000000000000000
> [ 4263.807158] R13: ffff8881003acd40 R14: 0000000000000001 R15:
> 00000000000000e1
> [ 4263.808334] FS: 00007fd0577417c0(0000) GS:ffff888237c80000(0000)
> knlGS:0000000000000000
> [ 4263.809625] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 4263.810690] CR2: 0000000006954568 CR3: 000000010ecb2004 CR4:
> 0000000000770ee0
> [ 4263.811903] PKRU: 55555554
> [ 4263.812656] Call Trace:
> [ 4263.813401] <TASK>
> [ 4263.814095] ? show_regs+0x69/0x80
> [ 4263.814897] ? __warn+0x8d/0x150
> [ 4263.815713] ? rcu_note_context_switch+0x2dd/0x320
> [ 4263.816686] ? report_bug+0x196/0x1c0
> [ 4263.817549] ? srso_alias_return_thunk+0x5/0x7f
> [ 4263.818503] ? irq_work_queue+0x13/0x60
> [ 4263.819387] ? handle_bug+0x45/0x80
> [ 4263.820219] ? exc_invalid_op+0x1c/0x70
> [ 4263.821089] ? asm_exc_invalid_op+0x1f/0x30
> [ 4263.821994] ? rcu_note_context_switch+0x2dd/0x320
> [ 4263.822961] ? rcu_note_context_switch+0x2dd/0x320
> [ 4263.823916] __schedule+0x60/0x770
> [ 4263.824765] ? __raw_spin_lock_irqsave+0x23/0x60
> [ 4263.825720] ? srso_alias_return_thunk+0x5/0x7f
> [ 4263.826662] ? _raw_spin_unlock_irqrestore+0x2b/0x50
> [ 4263.827617] schedule+0x6a/0xf0
> [ 4263.828383] __lock_sock+0x7d/0xc0
> [ 4263.829178] ? __pfx_autoremove_wake_function+0x10/0x10
> [ 4263.830145] lock_sock_nested+0x54/0x60
> [ 4263.830982] do_ip_setsockopt+0x10b4/0x11a0
> [ 4263.831859] ? srso_alias_return_thunk+0x5/0x7f
> [ 4263.832766] ? srso_alias_return_thunk+0x5/0x7f
> [ 4263.833655] sol_ip_sockopt+0x3c/0x80
> [ 4263.834444] __bpf_setsockopt+0x5f/0xa0
> [ 4263.835222] bpf_sock_ops_setsockopt+0x19/0x30
> [ 4263.836064] bpf_prog_e2095e3611d57c9f_bpf_test_sockopt_int+0xbc/0x127
> [ 4263.837110] bpf_prog_493685a3bae00bbd_bpf_test_ip_sockopt+0x49/0x4f
> [ 4263.838124] bpf_prog_e80acd4ed535c5e2_skops_sockopt+0x433/0xe95
> [ 4263.839102] ? migrate_disable+0x7b/0x90
> [ 4263.839865] __cgroup_bpf_run_filter_sock_ops+0x91/0x140
> [ 4263.840769] __inet_listen_sk+0x106/0x130
> [ 4263.841529] ? lock_sock_nested+0x43/0x60
> [ 4263.842259] inet_listen+0x4d/0x70
> [ 4263.842935] __sys_listen+0x75/0xc0
> [ 4263.843624] __x64_sys_listen+0x18/0x20
> [ 4263.844309] do_syscall_64+0x3e/0x90
> [ 4263.844969] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> [ 4263.845788] RIP: 0033:0x7fd156f9c82b
> [ 4263.846461] Code: f0 ff ff 73 01 c3 48 8b 0d 52 46 38 00 f7 d8 64 89
> 01 48 83 c8 ff c3 0f 1f 84 00 00 00 00 00 f3 0f 1e fa b8 32 00 00 00 0f
> 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 25 46 38 00 f7 d8 64 89 01 48
> [ 4263.848837] RSP: 002b:00007ffda4b93ed8 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000032
> [ 4263.849892] RAX: ffffffffffffffda RBX: 00000000068d85b0 RCX:
> 00007fd156f9c82b
> [ 4263.850912] RDX: 0000000000000010 RSI: 0000000000000001 RDI:
> 000000000000002c
> [ 4263.851932] RBP: 00007ffda4b93f20 R08: 0000000000000010 R09:
> 0000000000000000
> [ 4263.852953] R10: 00007ffda4b93eb0 R11: 0000000000000246 R12:
> 0000000000411fa0
> [ 4263.853975] R13: 00007ffda4b94370 R14: 0000000000000000 R15:
> 0000000000000000
> [ 4263.855011] </TASK>
> [ 4263.855564] ---[ end trace 0000000000000000 ]---
> [ 4323.847899] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> [ 4323.849648] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-7):
> P47486/1:b..l
> [ 4323.850749] rcu: (detected by 4, t=60004 jiffies, g=212293, q=44343
> ncpus=8)
> [ 4323.851852] task:new_name state:D stack:0 pid:47486
> ppid:47485 flags:0x00000000
> [ 4323.853085] Call Trace:
> [ 4323.853711] <TASK>
> [ 4323.854296] __schedule+0x265/0x770
> [ 4323.855023] ? __raw_spin_lock_irqsave+0x23/0x60
> [ 4323.855861] schedule+0x6a/0xf0
> [ 4323.856547] __lock_sock+0x7d/0xc0
> [ 4323.857271] ? __pfx_autoremove_wake_function+0x10/0x10
> [ 4323.858168] lock_sock_nested+0x54/0x60
> [ 4323.858923] do_ip_setsockopt+0x10b4/0x11a0
> [ 4323.859710] ? srso_alias_return_thunk+0x5/0x7f
> [ 4323.860545] ? srso_alias_return_thunk+0x5/0x7f
> [ 4323.861373] sol_ip_sockopt+0x3c/0x80
> [ 4323.862093] __bpf_setsockopt+0x5f/0xa0
> [ 4323.862837] bpf_sock_ops_setsockopt+0x19/0x30
> [ 4323.863643] bpf_prog_e2095e3611d57c9f_bpf_test_sockopt_int+0xbc/0x127
> [ 4323.864668] bpf_prog_493685a3bae00bbd_bpf_test_ip_sockopt+0x49/0x4f
> [ 4323.865678] bpf_prog_e80acd4ed535c5e2_skops_sockopt+0x433/0xe95
> [ 4323.866664] ? migrate_disable+0x7b/0x90
> [ 4323.867441] __cgroup_bpf_run_filter_sock_ops+0x91/0x140
> [ 4323.868356] __inet_listen_sk+0x106/0x130
> [ 4323.869117] ? lock_sock_nested+0x43/0x60
> [ 4323.869885] inet_listen+0x4d/0x70
> [ 4323.870595] __sys_listen+0x75/0xc0
> [ 4323.871300] __x64_sys_listen+0x18/0x20
> [ 4323.872042] do_syscall_64+0x3e/0x90
> [ 4323.872761] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> [ 4323.873637] RIP: 0033:0x7fd156f9c82b
> [ 4323.874387] RSP: 002b:00007ffda4b93ed8 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000032
> [ 4323.875496] RAX: ffffffffffffffda RBX: 00000000068d85b0 RCX:
> 00007fd156f9c82b
> [ 4323.876583] RDX: 0000000000000010 RSI: 0000000000000001 RDI:
> 000000000000002c
> [ 4323.877679] RBP: 00007ffda4b93f20 R08: 0000000000000010 R09:
> 0000000000000000
> [ 4323.878774] R10: 00007ffda4b93eb0 R11: 0000000000000246 R12:
> 0000000000411fa0
> [ 4323.879866] R13: 00007ffda4b94370 R14: 0000000000000000 R15:
> 0000000000000000
> [ 4323.880952] </TASK>
> [ 4334.246892] rcu: INFO: rcu_preempt detected expedited stalls on
> CPUs/tasks: { P47486 } 61561 jiffies s: 3197 root: 0x0/T
> [ 4334.248460] rcu: blocking rcu_node structures (internal RCU debug):
> [ 4424.359789] INFO: task new_name:47486 blocked for more than 122 seconds.
> [ 4424.361458] Tainted: G W O 6.6.0-rc7+ #159
> [ 4424.362418] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 4424.363570] task:new_name state:D stack:0 pid:47486
> ppid:47485 flags:0x00000000
> [ 4424.364799] Call Trace:
> [ 4424.365430] <TASK>
> [ 4424.368934] __schedule+0x265/0x770
> [ 4424.369663] ? __raw_spin_lock_irqsave+0x23/0x60
> [ 4424.370499] schedule+0x6a/0xf0
> [ 4424.371189] __lock_sock+0x7d/0xc0
> [ 4424.371906] ? __pfx_autoremove_wake_function+0x10/0x10
> [ 4424.372794] lock_sock_nested+0x54/0x60
> [ 4424.373532] do_ip_setsockopt+0x10b4/0x11a0
> [ 4424.374302] ? srso_alias_return_thunk+0x5/0x7f
> [ 4424.375127] ? srso_alias_return_thunk+0x5/0x7f
> [ 4424.375947] sol_ip_sockopt+0x3c/0x80
> [ 4424.376653] __bpf_setsockopt+0x5f/0xa0
> [ 4424.377381] bpf_sock_ops_setsockopt+0x19/0x30
> [ 4424.378184] bpf_prog_e2095e3611d57c9f_bpf_test_sockopt_int+0xbc/0x127
> [ 4424.379190] bpf_prog_493685a3bae00bbd_bpf_test_ip_sockopt+0x49/0x4f
> [ 4424.380188] bpf_prog_e80acd4ed535c5e2_skops_sockopt+0x433/0xe95
> [ 4424.381150] ? migrate_disable+0x7b/0x90
> [ 4424.381885] __cgroup_bpf_run_filter_sock_ops+0x91/0x140
> [ 4424.382762] __inet_listen_sk+0x106/0x130
> [ 4424.383515] ? lock_sock_nested+0x43/0x60
> [ 4424.384290] inet_listen+0x4d/0x70
> [ 4424.384992] __sys_listen+0x75/0xc0
> [ 4424.385690] __x64_sys_listen+0x18/0x20
> [ 4424.386425] do_syscall_64+0x3e/0x90
> [ 4424.387150] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> [ 4424.387991] RIP: 0033:0x7fd156f9c82b
> [ 4424.388702] RSP: 002b:00007ffda4b93ed8 EFLAGS: 00000246 ORIG_RAX:
> 0000000000000032
> [ 4424.389804] RAX: ffffffffffffffda RBX: 00000000068d85b0 RCX:
> 00007fd156f9c82b
> [ 4424.390862] RDX: 0000000000000010 RSI: 0000000000000001 RDI:
> 000000000000002c
> [ 4424.391913] RBP: 00007ffda4b93f20 R08: 0000000000000010 R09:
> 0000000000000000
> [ 4424.392987] R10: 00007ffda4b93eb0 R11: 0000000000000246 R12:
> 0000000000411fa0
> [ 4424.394047] R13: 00007ffda4b94370 R14: 0000000000000000 R15:
> 0000000000000000
> [ 4424.395117] </TASK>
> [ 4424.395701] INFO: task dtprobed:49514 blocked for more than 122 seconds.
> [ 4424.396751] Tainted: G W O 6.6.0-rc7+ #159
> [ 4424.397696] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 4424.398858] task:dtprobed state:D stack:0 pid:49514 ppid:1
> flags:0x00000002
> [ 4424.400036] Call Trace:
> [ 4424.400651] <TASK>
> [ 4424.401258] __schedule+0x265/0x770
> [ 4424.401975] ? srso_alias_return_thunk+0x5/0x7f
> [ 4424.402795] schedule+0x6a/0xf0
> [ 4424.403472] synchronize_rcu_expedited+0x185/0x200
> [ 4424.404313] ? __pfx_wait_rcu_exp_gp+0x10/0x10
> [ 4424.405101] ? __pfx_autoremove_wake_function+0x10/0x10
> [ 4424.405966] ? srso_alias_return_thunk+0x5/0x7f
> [ 4424.406758] ? preempt_count_add+0x52/0xc0
> [ 4424.407499] namespace_unlock+0xd6/0x1b0
> [ 4424.408234] put_mnt_ns+0x74/0xa0
> [ 4424.408903] free_nsproxy+0x1f/0x1b0
> [ 4424.409609] exit_task_namespaces+0x6f/0x90
> [ 4424.410388] do_exit+0x28b/0x520
> [ 4424.411041] ? srso_alias_return_thunk+0x5/0x7f
> [ 4424.411837] ? srso_alias_return_thunk+0x5/0x7f
> [ 4424.412635] do_group_exit+0x39/0x90
> [ 4424.413351] __x64_sys_exit_group+0x1c/0x20
> [ 4424.414106] do_syscall_64+0x3e/0x90
> [ 4424.414810] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
> [ 4424.415650] RIP: 0033:0x7ff70a952cf6
> [ 4424.416373] RSP: 002b:00007ffeecec40b8 EFLAGS: 00000246 ORIG_RAX:
> 00000000000000e7
> [ 4424.417494] RAX: ffffffffffffffda RBX: 00007ff70ac14820 RCX:
> 00007ff70a952cf6
> [ 4424.418560] RDX: 0000000000000002 RSI: 000000000000003c RDI:
> 0000000000000002
> [ 4424.419636] RBP: 0000000000000002 R08: 00000000000000e7 R09:
> ffffffffffffff68
> [ 4424.420689] R10: 00007ffeecec3fbf R11: 0000000000000246 R12:
> 00007ff70ac14820
> [ 4424.421768] R13: 0000000000000001 R14: 00007ff70ac1a310 R15:
> 0000000000000000
> [ 4424.422837] </TASK>
>
>
> >> Thanks,
> >> Eduard.
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2023-10-27 19:54 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-23 9:57 [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 1/5] btf_encoder, pahole: move btf encoding options into conf_load Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 2/5] dwarves: move ARRAY_SIZE() to dwarves.h Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 3/5] pahole: add --btf_features support Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 4/5] pahole: add --supported_btf_features Alan Maguire
2023-10-23 9:57 ` [PATCH v4 dwarves 5/5] pahole: add --btf_features_strict to reject unknown BTF features Alan Maguire
2023-10-25 17:43 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
2023-10-25 17:48 ` Arnaldo Carvalho de Melo
2023-10-25 18:12 ` Arnaldo Carvalho de Melo
2023-10-25 18:30 ` Arnaldo Carvalho de Melo
2023-10-25 22:28 ` Eduard Zingerman
2023-10-26 22:06 ` Andrii Nakryiko
2023-10-27 13:25 ` Arnaldo Carvalho de Melo
2023-10-27 14:24 ` RCU stall issues in bpf-next (was: Re: [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features) Alan Maguire
2023-10-27 19:54 ` Jiri Olsa
2023-10-25 18:18 ` [PATCH v4 dwarves 0/5] pahole, btf_encoder: support --btf_features Arnaldo Carvalho de Melo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox