From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 690FCCA0EEB for ; Tue, 19 Aug 2025 16:11:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uoOvb-0003Sx-HT; Tue, 19 Aug 2025 12:11:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uoOvV-0003Q4-81 for qemu-devel@nongnu.org; Tue, 19 Aug 2025 12:11:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uoOvS-00080o-5g for qemu-devel@nongnu.org; Tue, 19 Aug 2025 12:11:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1755619863; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=f76GXYC3FIrjBGGqNOGDIgKcuddooQ3uT1XbwZDVf/g=; b=I9nzV7PIBrRnwS4s5Sxoc0ekEKHlkB4w+RJ/3Xqfx+90ZFDkfjOMGJvZ+GmYrJcR4EMzp7 d0KYchz1ulJjuEj/iPqtFMC3S9YkC+mTLWDjuG++avu7lvDDsDWWTUoWQn5dwJ9gUePRzD s7otCfOCmgN/ZBpjvk3PclxfCT8NvZQ= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-604-kj2cydF0PtO-gL0kt6Jq-Q-1; Tue, 19 Aug 2025 12:10:59 -0400 X-MC-Unique: kj2cydF0PtO-gL0kt6Jq-Q-1 X-Mimecast-MFC-AGG-ID: kj2cydF0PtO-gL0kt6Jq-Q_1755619858 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6D3FF1800299; Tue, 19 Aug 2025 16:10:58 +0000 (UTC) Received: from toolbx.redhat.com (unknown [10.42.28.237]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 6F316180028A; Tue, 19 Aug 2025 16:10:55 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: qemu-devel@nongnu.org Cc: Michael Roth , Paolo Bonzini , Thomas Huth , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Markus Armbruster , Mads Ynddal , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Stefan Hajnoczi , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= Subject: [PATCH v4 0/7] tracetool: add test suite to improve reviewability Date: Tue, 19 Aug 2025 17:10:46 +0100 Message-ID: <20250819161053.464641-1-berrange@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Received-SPF: pass client-ip=170.10.133.124; envelope-from=berrange@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org To repeat the start of the commit message in patch 5.... When reviewing tracetool patches it is often very unclear what the expected output will be for the generated backends. Compounding this is that a default build will only enable the 'log' trace backend, so developers won't see generated code for other backends without making a special effort. Some backends are also platform specific, so can't be enabled in QEMU builds, even though tracetool could generate the code. To address this, introduce a test suite for tracetool which is conceptually similar to the qapi-schema test. It is a simple python program that runs tracetool and compares the actual output to historical reference output kept in git. This was inspired by noticing the now pointless "__nocheck__" method wrapping in the inline probe functions. Changed in v4: - Drop patch adding '-' as an output filename for tracetool - Rewrite test to always use relative filenames for both inputs and outputs - Pass python code through 'black' - Minor docs punctuation tweak - Drop unused python method Changed in v3: - Also modify the QAPI schema test to use QEMU_TEST_REGENERATE=1 env & document / hint this - Make tracetool accept '-' as shorthand for stdout to avoid relative paths in the reference output Changed in v2: - Add tracetool info to docs/devel/testing.rst Daniel P. Berrangé (7): checkpatch: cull trailing '*/' in SPDX check tracetool: eliminate trailing whitespace in C format tracetool: avoid space after "*" in arg types tracetool: include SPDX-License-Identifier in generated files tracetool: add test suite for tracetool with reference output tracetool: drop the probe "__nocheck__" wrapping qapi: switch to use QEMU_TEST_REGENERATE env var MAINTAINERS | 1 + docs/devel/testing/main.rst | 40 +++++++ scripts/checkpatch.pl | 3 + scripts/tracetool/__init__.py | 9 +- scripts/tracetool/format/c.py | 3 +- scripts/tracetool/format/d.py | 3 +- scripts/tracetool/format/h.py | 17 +-- scripts/tracetool/format/log_stap.py | 1 + scripts/tracetool/format/simpletrace_stap.py | 1 + scripts/tracetool/format/stap.py | 1 + scripts/tracetool/format/ust_events_c.py | 1 + scripts/tracetool/format/ust_events_h.py | 1 + tests/Makefile.include | 1 + tests/meson.build | 1 + tests/qapi-schema/test-qapi.py | 7 +- tests/tracetool/dtrace.c | 32 ++++++ tests/tracetool/dtrace.d | 10 ++ tests/tracetool/dtrace.h | 45 ++++++++ tests/tracetool/dtrace.log-stap | 15 +++ tests/tracetool/dtrace.simpletrace-stap | 16 +++ tests/tracetool/dtrace.stap | 14 +++ tests/tracetool/ftrace.c | 32 ++++++ tests/tracetool/ftrace.h | 59 +++++++++++ tests/tracetool/log.c | 32 ++++++ tests/tracetool/log.h | 43 ++++++++ tests/tracetool/meson.build | 25 +++++ tests/tracetool/simple.c | 61 +++++++++++ tests/tracetool/simple.h | 40 +++++++ tests/tracetool/syslog.c | 32 ++++++ tests/tracetool/syslog.h | 43 ++++++++ tests/tracetool/trace-events | 5 + tests/tracetool/tracetool-test.py | 105 +++++++++++++++++++ tests/tracetool/ust.c | 32 ++++++ tests/tracetool/ust.h | 41 ++++++++ tests/tracetool/ust.ust-events-c | 14 +++ tests/tracetool/ust.ust-events-h | 56 ++++++++++ 36 files changed, 821 insertions(+), 21 deletions(-) create mode 100644 tests/tracetool/dtrace.c create mode 100644 tests/tracetool/dtrace.d create mode 100644 tests/tracetool/dtrace.h create mode 100644 tests/tracetool/dtrace.log-stap create mode 100644 tests/tracetool/dtrace.simpletrace-stap create mode 100644 tests/tracetool/dtrace.stap create mode 100644 tests/tracetool/ftrace.c create mode 100644 tests/tracetool/ftrace.h create mode 100644 tests/tracetool/log.c create mode 100644 tests/tracetool/log.h create mode 100644 tests/tracetool/meson.build create mode 100644 tests/tracetool/simple.c create mode 100644 tests/tracetool/simple.h create mode 100644 tests/tracetool/syslog.c create mode 100644 tests/tracetool/syslog.h create mode 100644 tests/tracetool/trace-events create mode 100755 tests/tracetool/tracetool-test.py create mode 100644 tests/tracetool/ust.c create mode 100644 tests/tracetool/ust.h create mode 100644 tests/tracetool/ust.ust-events-c create mode 100644 tests/tracetool/ust.ust-events-h -- 2.50.1