From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-188.mta1.migadu.com (out-188.mta1.migadu.com [95.215.58.188]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 006FC38E8A2 for ; Mon, 3 Aug 2026 16:51:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.188 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785775904; cv=none; b=usArh93/J5NFHv2LS0J0vg8AtxB4MANQel+4OIFVgqs/crXaZ6DBJ6usDbMf99oIgJPVhExMl8XbTlCfSS9+fRUkqn99/BiayxC98xmISNUxtddkC4rVXB8gMS6HQThV+aWhL2Ew4rq0FlV968Nbi7KALEppswfbB/MeRnUKhzw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785775904; c=relaxed/simple; bh=YueUsyAPM50kr+tvjkqG3/IXAA21hvRsOkbvnXAcEk0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iLjlMG/yyAwiUPZuYRVE964eQUWCHSoJcPKbXQKqGn2kiBL2kapSIc8Y6jids7vEPLTDc3V45A2pJdJJKh1YzA88mGdHCXfKwHNWrIF51N3pj6mvtbmC4mW8AnPPUWZvOqGWrK9pFy/QbZatF4/HnBSjvPPHwKlPELrpYwYdQgc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=OZNVnlQz; arc=none smtp.client-ip=95.215.58.188 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="OZNVnlQz" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1785775900; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xJo3x4oawUqd8sXksvOFjNdRI29zIVI6gG2AIfHhMaU=; b=OZNVnlQzc3mcrIBH+cevf9uGFUEejFgP9wuhX02gpQTZ5gAnEUXTfeYb0d1EgtCB9FKhIk meKPaDsKJ9gAzJ7Yi0SQ8/dOWNvMy3zO0GGUZ1FERvhxPeDSnTuExa0m9pOT6tbhEO3lOu jO8uVjjIuPzL9d8Hx/KqvPHugtyFqBw= From: Vineet Gupta To: bpf@gcc.gnu.org, ast@kernel.org, Eduard Zingerman , Andrii Nakryiko , Ihor Solodrai Cc: linux-kernel@vger.kernel.org, Vineet Gupta Subject: [bpf-next 3/4] selftests/bpf: report failed subtest count in test_progs summary Date: Mon, 3 Aug 2026 09:51:21 -0700 Message-ID: <20260803165122.1884825-4-vineet.gupta@linux.dev> In-Reply-To: <20260803165122.1884825-1-vineet.gupta@linux.dev> References: <20260803165122.1884825-1-vineet.gupta@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT The final summary line is asymmetric: the PASSED field reports both the number of top-level tests and the number of subtests within them, while the FAILED field reports only top-level tests: Summary: 640/5750 PASSED, 7760 SKIPPED, 100 FAILED There is no way to tell whether those 100 failing tests amount to 100 broken subtests or 1000. So count subtests with a non-zero error_cnt into a new sub_fail_cnt and print it alongside fail_cnt: Summary: 640/5750 PASSED, 7760 SKIPPED, 100/342 FAILED ^^^^^ This is correct for -j runs, as subtest_states[] is populated both in sequential and parallel modes. A test that fails without declaring any subtests contributes 0 to sub_fail_cnt. That mirrors the existing behaviour of sub_succ_cnt for tests that pass without subtests, keeping the two numerators comparable. Also emit the new count as a "failed_subtest" field in the JSON output, for parity with the existing "success_subtest". Note that this changes the trailing field of the summary line from a bare integer to "A/B", so anything scraping "N FAILED" out of it needs updating. While here, fix the fail_cnt comment in struct test_env, which claims it counts "total failed tests + sub-tests". Signed-off-by: Vineet Gupta --- tools/testing/selftests/bpf/test_progs.c | 21 +++++++++++++-------- tools/testing/selftests/bpf/test_progs.h | 2 +- 2 files changed, 14 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/bpf/test_progs.c b/tools/testing/selftests/bpf/test_progs.c index 8ee46745e2e8..5e2f6ec2e212 100644 --- a/tools/testing/selftests/bpf/test_progs.c +++ b/tools/testing/selftests/bpf/test_progs.c @@ -1656,8 +1656,8 @@ static void *dispatch_thread(void *ctx) static void calculate_summary_and_print_errors(struct test_env *env) { - int i; - int succ_cnt = 0, fail_cnt = 0, sub_succ_cnt = 0, skip_cnt = 0; + int i, j; + int succ_cnt = 0, fail_cnt = 0, sub_succ_cnt = 0, sub_fail_cnt = 0, skip_cnt = 0; json_writer_t *w = NULL; for (i = 0; i < prog_test_cnt; i++) { @@ -1670,10 +1670,14 @@ static void calculate_summary_and_print_errors(struct test_env *env) sub_succ_cnt += state->sub_succ_cnt; skip_cnt += state->skip_cnt; - if (state->error_cnt) + if (state->error_cnt) { fail_cnt++; - else if (!test->not_built) + for (j = 0; j < state->subtest_num; j++) + if (state->subtest_states[j].error_cnt) + sub_fail_cnt++; + } else if (!test->not_built) { succ_cnt++; + } } if (env->json) { @@ -1688,6 +1692,7 @@ static void calculate_summary_and_print_errors(struct test_env *env) jsonw_uint_field(w, "success_subtest", sub_succ_cnt); jsonw_uint_field(w, "skipped", skip_cnt); jsonw_uint_field(w, "failed", fail_cnt); + jsonw_uint_field(w, "failed_subtest", sub_fail_cnt); jsonw_name(w, "results"); jsonw_start_array(w); } @@ -1726,12 +1731,12 @@ static void calculate_summary_and_print_errors(struct test_env *env) fclose(env->json); if (env->not_built_cnt) - printf("Summary: %d/%d PASSED, %d SKIPPED (%d not built), %d FAILED\n", + printf("Summary: %d/%d PASSED, %d SKIPPED (%d not built), %d/%d FAILED\n", succ_cnt, sub_succ_cnt, skip_cnt, env->not_built_cnt, - fail_cnt); + fail_cnt, sub_fail_cnt); else - printf("Summary: %d/%d PASSED, %d SKIPPED, %d FAILED\n", - succ_cnt, sub_succ_cnt, skip_cnt, fail_cnt); + printf("Summary: %d/%d PASSED, %d SKIPPED, %d/%d FAILED\n", + succ_cnt, sub_succ_cnt, skip_cnt, fail_cnt, sub_fail_cnt); env->succ_cnt = succ_cnt; env->sub_succ_cnt = sub_succ_cnt; diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h index e66d5c457901..ea493c477fbd 100644 --- a/tools/testing/selftests/bpf/test_progs.h +++ b/tools/testing/selftests/bpf/test_progs.h @@ -124,7 +124,7 @@ struct test_env { int succ_cnt; /* successful tests */ int sub_succ_cnt; /* successful sub-tests */ - int fail_cnt; /* total failed tests + sub-tests */ + int fail_cnt; /* failed tests */ int skip_cnt; /* skipped tests */ int not_built_cnt; /* tests not built */ -- 2.55.0