From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAEABC4332F for ; Sun, 15 May 2022 17:36:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237902AbiEORgU (ORCPT ); Sun, 15 May 2022 13:36:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237920AbiEORgS (ORCPT ); Sun, 15 May 2022 13:36:18 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 875B1EB0 for ; Sun, 15 May 2022 10:36:11 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 9-20020a250909000000b006484b89c979so11281685ybj.21 for ; Sun, 15 May 2022 10:36:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=8sHlDEs1q4JukqBHuWMP1ZMhpXOvq75/48oBKXyDEuU=; b=ez8Lk1k+iXKPT37+NcdQsVblaqAIufwRTtUy5BPvC35lWYU9gPN+ZQi8VjLVuN+a+y A0seUY8QY4ZWDV6IUcp+rSfHtK62dplL89ZTQ2ejb7YzFPGWzL9rDSKZKN2WKlkuBi/W gEP3qAMq3emNrET8UkeCKNknCCKjNTkZOV3a7OAa35D+ZWWxWs5o+ZobiU9eUrJ9ie6G gXLHY4VWX8CDExJM1t77gkUC9UeLmkb22WLVkrr8A3C/Cwrgt2FcoNA8XGfyAt8lz1R6 z55i71hw5TjflUKBfdDJobdkP1x4/sCdpUrAv9X7Xfqch1d5hjo+JJ+DxufxjtZq4kgw hAmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=8sHlDEs1q4JukqBHuWMP1ZMhpXOvq75/48oBKXyDEuU=; b=0BzJVqVsNTAZtj+4Yj/MjcUCWKJJ4uLRlb/eecBNqYYNLERpPEXweloEEPl8qNsgIk ycMrGDtIXBVnBlSDOBEFunxZeYg8tG7UJAxWC+yGQv/mgBeOuXpCDEYKHFUdItXHHQCC nTqvRlm9+/pM6aDbsf+Fy8SZh0YRRBrTe4EszkbWQYVpe24xdlIlOw6Okp3XZBAuf1ua k1n3TLV1zyopcTrO+XnHXBbjuTxMIivsen2VuKXXlQvgQbwKg+H6b4rmXJnJODV2Tj2N 6gcBxNnzj9D2/U4ORQEya+mPLPmp2hZ27qBTw3RfAXn8+AcypV8aOIyvCurx4coMky8P fumQ== X-Gm-Message-State: AOAM530sWBMXdqUL0r3o7FVO/B93cIPsfCvENg6SjXk5F+lbyb3Oa4uz +I86bkhwzAYLTaTg/CWsPHW1m0bdFlG0 X-Google-Smtp-Source: ABdhPJxqE0FK36N/zbPltFvdN3+0L1NaNwJqBtogZodPV1uz0vELjSDJB177+wPeghZR0WKsOq6O+qU4Wrfv X-Received: from irogers.svl.corp.google.com ([2620:15c:2cd:202:2a5e:db66:d206:cbaf]) (user=irogers job=sendgmr) by 2002:a25:3757:0:b0:64d:775d:7036 with SMTP id e84-20020a253757000000b0064d775d7036mr5091438yba.635.1652636170500; Sun, 15 May 2022 10:36:10 -0700 (PDT) Date: Sun, 15 May 2022 10:35:58 -0700 In-Reply-To: <20220515173558.2289702-1-irogers@google.com> Message-Id: <20220515173558.2289702-3-irogers@google.com> Mime-Version: 1.0 References: <20220515173558.2289702-1-irogers@google.com> X-Mailer: git-send-email 2.36.0.550.gb090851708-goog Subject: [PATCH 2/2] perf test: Json format checking From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Kan Liang , Zhengjun Xing , Sandipan Das , Claire Jensen , Alyssa Ross , Like Xu , James Clark , Florian Fischer , linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Claire Jensen Cc: Stephane Eranian , Ian Rogers Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-perf-users@vger.kernel.org From: Claire Jensen Add field checking tests for perf stat JSON output. Sanity checks the expected number of fields are present, that the expected keys are present and they have the correct values. Signed-off-by: Claire Jensen Signed-off-by: Ian Rogers --- .../tests/shell/lib/perf_json_output_lint.py | 90 +++++++++++ tools/perf/tests/shell/stat+json_output.sh | 141 ++++++++++++++++++ 2 files changed, 231 insertions(+) create mode 100644 tools/perf/tests/shell/lib/perf_json_output_lint.py create mode 100755 tools/perf/tests/shell/stat+json_output.sh diff --git a/tools/perf/tests/shell/lib/perf_json_output_lint.py b/tools/perf/tests/shell/lib/perf_json_output_lint.py new file mode 100644 index 000000000000..3d7404bf6e0b --- /dev/null +++ b/tools/perf/tests/shell/lib/perf_json_output_lint.py @@ -0,0 +1,90 @@ +#!/usr/bin/python +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +# Basic sanity check of perf JSON output as specified in the man page. + +from __future__ import print_function +import argparse +import sys +import json + +ap = argparse.ArgumentParser() +ap.add_argument('--no-args', action='store_true') +ap.add_argument('--interval', action='store_true') +ap.add_argument('--system-wide-no-aggr', action='store_true') +ap.add_argument('--system-wide', action='store_true') +ap.add_argument('--event', action='store_true') +ap.add_argument('--per-core', action='store_true') +ap.add_argument('--per-thread', action='store_true') +ap.add_argument('--per-die', action='store_true') +ap.add_argument('--per-node', action='store_true') +ap.add_argument('--per-socket', action='store_true') +args = ap.parse_args() + +Lines = sys.stdin.readlines() + +def isfloat(num): + try: + float(num) + return True + except ValueError: + return False + + +def isint(num): + try: + int(num) + return True + except ValueError: + return False + + +def check_json_output(expected_items): + if expected_items != -1: + for line in Lines: + if 'failed' not in line: + count = 0 + count = line.count(',') + if count != expected_items: + raise RuntimeError('wrong number of fields. counted {0}' + ' expected {1} in {2}\n'.format(count, exp, line)) + checks = { + 'aggregate-number': lambda x: isfloat(x), + 'core': lambda x: True, + 'counter-value': lambda x: isfloat(x), + 'cgroup': lambda x: True, + 'cpu': lambda x: isint(x), + 'die': lambda x: True, + 'event': lambda x: True, + 'event-runtime': lambda x: isfloat(x), + 'interval': lambda x: isfloat(x), + 'metric-unit': lambda x: True, + 'metric-value': lambda x: isfloat(x), + 'node': lambda x: True, + 'pcnt-running': lambda x: isfloat(x), + 'socket': lambda x: True, + 'thread': lambda x: True, + 'unit': lambda x: True, + } + input = '[\n' + ','.join(Lines) + '\n]' + for item in json.loads(input): + for key, value in item.items(): + if not key in checks: + raise RuntimeError(f'Unexpected key: key={key} value={value}') + if not checks[key](value): + raise RuntimeError(f'Check failed for: key={key} value={value}') + + +try: + if args.no_args or args.system_wide or args.event: + expected_items = 6 + elif args.interval or args.per_thread or args.system_wide_no_aggr: + expected_items = 7 + elif args.per_core or args.per_socket or args.per_node or args.per_die: + expected_items = 8 + else: + # If no option is specified, don't check the number of items. + expected_items = -1 + check_json_output(expected_items) +except: + print('Test failed for input:\n' + '\n'.join(Lines)) + raise diff --git a/tools/perf/tests/shell/stat+json_output.sh b/tools/perf/tests/shell/stat+json_output.sh new file mode 100755 index 000000000000..7b3745f3eeb8 --- /dev/null +++ b/tools/perf/tests/shell/stat+json_output.sh @@ -0,0 +1,141 @@ +#!/bin/bash +# perf stat JSON output linter +# SPDX-License-Identifier: GPL-2.0 +# Checks various perf stat JSON output commands for the +# correct number of fields. + +set -e + +pythonchecker=$(dirname $0)/lib/perf_json_output_lint.py +if [ "x$PYTHON" == "x" ] +then + if which python3 > /dev/null + then + PYTHON=python3 + elif which python > /dev/null + then + PYTHON=python + else + echo Skipping test, python not detected please set environment variable PYTHON. + exit 2 + fi +fi + +# Return true if perf_event_paranoid is > $1 and not running as root. +function ParanoidAndNotRoot() +{ + [ $(id -u) != 0 ] && [ $(cat /proc/sys/kernel/perf_event_paranoid) -gt $1 ] +} + +check_no_args() +{ + echo -n "Checking json output: no args " + perf stat -j true 2>&1 | $PYTHON $pythonchecker --no-args + echo "[Success]" +} + +check_system_wide() +{ + echo -n "Checking json output: system wide " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -j -a true 2>&1 | $PYTHON $pythonchecker --system-wide + echo "[Success]" +} + +check_system_wide_no_aggr() +{ + echo -n "Checking json output: system wide no aggregation " + perf stat -j -A -a --no-merge true 2>&1 | $PYTHON $pythonchecker --system-wide-no-aggr + echo "[Success]" +} + +check_interval() +{ + echo -n "Checking json output: interval " + perf stat -j -I 1000 true 2>&1 | $PYTHON $pythonchecker --interval + echo "[Success]" +} + + +check_event() +{ + echo -n "Checking json output: event " + perf stat -j -e cpu-clock true 2>&1 | $PYTHON $pythonchecker --event + echo "[Success]" +} + +check_per_core() +{ + echo -n "Checking json output: per core " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -j --per-core -a true 2>&1 | $PYTHON $pythonchecker --per-core + echo "[Success]" +} + +check_per_thread() +{ + echo -n "Checking json output: per thread " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -j --per-thread -a true 2>&1 | $PYTHON $pythonchecker --per-thread + echo "[Success]" +} + +check_per_die() +{ + echo -n "Checking json output: per die " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -j --per-die -a true 2>&1 | $PYTHON $pythonchecker --per-die + echo "[Success]" +} + +check_per_node() +{ + echo -n "Checking json output: per node " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -j --per-node -a true 2>&1 | $PYTHON $pythonchecker --per-node + echo "[Success]" +} + +check_per_socket() +{ + echo -n "Checking json output: per socket " + if ParanoidAndNotRoot 0 + then + echo "[Skip] parnoia and not root" + return + fi + perf stat -j --per-socket -a true 2>&1 | $PYTHON $pythonchecker --per-socket + echo "[Success]" +} + +check_no_args +check_system_wide +check_system_wide_no_aggr +check_interval +check_event +check_per_core +check_per_thread +check_per_die +check_per_node +check_per_socket +exit 0 -- 2.36.0.550.gb090851708-goog