* [RFC PATCH 0/7] Add script for real-time telemetry monitoring
@ 2025-12-10 16:55 Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
` (11 more replies)
0 siblings, 12 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
TL;DR
------
For a quick demo, apply patces, run e.g. testpmd and then in a separate
terminal run:
./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
Output, updated once per second, will be traffic rate per port e.g.:
Connected to application: "dpdk-testpmd"
Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
16:29:12 5,213,119 5,214,304 10,427,423
Fuller details
--------------
While we have the dpdk-telemetry.py CLI app for interactive querying of
telemetry on the commandline, and a telemetry exporter script for
sending telemetry to external tools for real-time monitoring, we don't
have an app that can print real-time stats for DPDK apps on the
terminal. This patchset adds such a script, developed with the help of
Github copilot to fill a need that I found in my testing. Submitting it
here in the hopes that others find it of use.
The script acts as a wrapper around the existing dpdk-telemetry.py
script, and pipes the commands to that script and reads the responses,
querying it once per second. It takes a number of flag parameters, such
as the ones above:
- "-d" for delta values, i.e. PPS rather than total packets
- "-1" for single-line output, i.e. no scrolling up the screen
- "-T" to display a total column
Other flag parameters can be seen by looking at the help output.
Beyond the flags, the script also takes a number of positional
parameters, which refer to specific stats to display. These stats must
be numeric values, and should take the form of the telemetry command to
send, followed by a "." and the stat within the result which is to be
tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
part of the result.
However, specifying individual stats can be awkward, so some shortcuts
are provided too for the common case of monitoring ethernet ports. Any
positional arg starting with "eth" will be replaced by the set of
equivalent values for each port, e.g. "eth.imissed" will track the
imissed value on all ports in use in the app. The ipackets and opackets
values, as common metrics, are also available as shortened values as
just "rx" and "tx", so in the example above, "eth.tx" means to track the
opackets stat for every ethdev port.
Finally, the script also has reconnection support so you can leave it
running while you start and stop your application in another terminal.
The watcher will try and reconnect to a running instance every second.
Bruce Richardson (7):
usertools: add new script to monitor telemetry on terminal
usertools/telemetry-watcher: add displaying stats
usertools/telemetry-watcher: add delta and timeout opts
usertools/telemetry-watcher: add total and one-line opts
usertools/telemetry-watcher: add thousands separator
usertools/telemetry-watcher: add eth name shortcuts
usertools/telemetry-watcher: support reconnection
usertools/dpdk-telemetry-watcher.py | 429 ++++++++++++++++++++++++++++
usertools/meson.build | 1 +
2 files changed, 430 insertions(+)
create mode 100755 usertools/dpdk-telemetry-watcher.py
--
2.51.0
^ permalink raw reply [flat|nested] 41+ messages in thread
* [RFC PATCH 1/7] usertools: add new script to monitor telemetry on terminal
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
@ 2025-12-10 16:55 ` Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
` (10 subsequent siblings)
11 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
The dpdk-telemetry.py script is useful for getting telemetry
interactively, but sometimes we want to monitor stats over time, so add
a telemetry-watcher script to do so.
Start off such a script with basic arg processing, and connecting to
dpdk-telemetry as a subprocess, so that we can send-receive commands
from it.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 200 ++++++++++++++++++++++++++++
usertools/meson.build | 1 +
2 files changed, 201 insertions(+)
create mode 100755 usertools/dpdk-telemetry-watcher.py
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
new file mode 100755
index 0000000000..01c8683d33
--- /dev/null
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation
+
+"""
+Script to monitor DPDK telemetry statistics on the command line.
+Wraps dpdk-telemetry.py to provide continuous monitoring capabilities.
+"""
+
+import argparse
+import subprocess
+import sys
+import os
+import shutil
+import errno
+import json
+
+
+def get_app_name(pid):
+ """return the app name for a given PID, for printing"""
+ proc_cmdline = os.path.join("/proc", str(pid), "cmdline")
+ try:
+ with open(proc_cmdline) as f:
+ argv0 = f.read(1024).split("\0")[0]
+ return os.path.basename(argv0)
+ except IOError as e:
+ # ignore file not found errors
+ if e.errno != errno.ENOENT:
+ raise
+ return None
+
+
+def find_telemetry_script():
+ """Find the dpdk-telemetry.py script in the script directory or PATH.
+
+ Returns:
+ str: Path to the dpdk-telemetry.py script
+
+ Exits:
+ If the script cannot be found
+ """
+ # First, try to find it in the same directory as this script
+ script_dir = os.path.dirname(os.path.abspath(__file__))
+ telemetry_script = os.path.join(script_dir, "dpdk-telemetry.py")
+
+ # If not found locally, check if it's in PATH
+ if not os.path.exists(telemetry_script):
+ telemetry_in_path = shutil.which("dpdk-telemetry.py")
+ if telemetry_in_path:
+ telemetry_script = telemetry_in_path
+ else:
+ print(
+ "Error: dpdk-telemetry.py not found in script directory or PATH",
+ file=sys.stderr,
+ )
+ sys.exit(1)
+
+ return telemetry_script
+
+
+def create_telemetry_process(telemetry_script, args_list):
+ """Create a subprocess for dpdk-telemetry.py with pipes.
+
+ Args:
+ telemetry_script: Path to the dpdk-telemetry.py script
+ args_list: List of arguments to pass to the script
+
+ Returns:
+ subprocess.Popen: Process handle with stdin/stdout/stderr pipes
+
+ Exits:
+ If the process cannot be created
+ """
+ # Build the command
+ cmd = [sys.executable, telemetry_script] + args_list
+
+ try:
+ process = subprocess.Popen(
+ cmd,
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ text=True,
+ bufsize=1, # Line buffered
+ )
+ return process
+ except FileNotFoundError:
+ print(f"Error: Python interpreter or script not found", file=sys.stderr)
+ sys.exit(1)
+ except Exception as e:
+ print(f"Error running dpdk-telemetry.py: {e}", file=sys.stderr)
+ sys.exit(1)
+
+
+def query_telemetry(process, command):
+ """Send a telemetry command and return the parsed JSON response.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
+
+ Returns:
+ dict: The parsed JSON response with the command wrapper stripped,
+ or None if there was an error
+ """
+ # Send the command
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+
+ # Read the JSON response
+ response = process.stdout.readline()
+ try:
+ data = json.loads(response)
+ # When run non-interactively, the response is wrapped with the command
+ # e.g., {"/info": {"version": ..., "pid": ...}}
+ # or {"/ethdev/stats,0": {...}}
+ # The response should have exactly one key which is the command
+ if len(data) == 1:
+ # Extract the value, ignoring the key
+ return next(iter(data.values()))
+ else:
+ return data
+ except (json.JSONDecodeError, KeyError):
+ return None
+
+
+def print_connected_app(process):
+ """Query and print the name of the connected DPDK application.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ """
+ info = query_telemetry(process, "/info")
+ if info and "pid" in info:
+ app_name = get_app_name(info["pid"])
+ if app_name:
+ print(f'Connected to application: "{app_name}"')
+
+
+def main():
+ """Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+
+ # Parse command line arguments - matching dpdk-telemetry.py parameters
+ parser = argparse.ArgumentParser(
+ description="Monitor DPDK telemetry statistics on the command line"
+ )
+ parser.add_argument(
+ "-f",
+ "--file-prefix",
+ default="rte",
+ help="Provide file-prefix for DPDK runtime directory",
+ )
+ parser.add_argument(
+ "-i",
+ "--instance",
+ default=0,
+ type=int,
+ help="Provide instance number for DPDK application",
+ )
+ parser.add_argument(
+ "-l",
+ "--list",
+ action="store_true",
+ default=False,
+ help="List all possible file-prefixes and exit",
+ )
+
+ args = parser.parse_args()
+
+ # Find the dpdk-telemetry.py script
+ telemetry_script = find_telemetry_script()
+
+ # Build arguments list
+ args_list = ["-f", args.file_prefix, "-i", str(args.instance)]
+
+ if args.list:
+ args_list.append("-l")
+ # For --list, just run the command directly without pipes
+ cmd = [sys.executable, telemetry_script] + args_list
+ return subprocess.run(cmd).returncode
+
+ # Run dpdk-telemetry.py with pipes for stdin and stdout
+ process = create_telemetry_process(telemetry_script, args_list)
+
+ # Get and display the connected application name
+ print_connected_app(process)
+
+ # TODO: Add monitoring logic here
+
+ # Clean up
+ process.stdin.close()
+ process.stdout.close()
+ process.stderr.close()
+ process.wait()
+
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/usertools/meson.build b/usertools/meson.build
index eb48e2f440..114d0a65b4 100644
--- a/usertools/meson.build
+++ b/usertools/meson.build
@@ -12,6 +12,7 @@ install_data([
'dpdk-hugepages.py',
'dpdk-rss-flows.py',
'dpdk-telemetry-exporter.py',
+ 'dpdk-telemetry-watcher.py',
],
install_dir: 'bin')
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [RFC PATCH 2/7] usertools/telemetry-watcher: add displaying stats
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
@ 2025-12-10 16:55 ` Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
` (9 subsequent siblings)
11 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Add support for printing out particular stats once every second. The
stats to be queried are given on the commandline, in the format of
<command>.<result-field>, for example /ethdev/stats,0.ipackets.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 101 +++++++++++++++++++++++++++-
1 file changed, 100 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 01c8683d33..824c9a37be 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -14,6 +14,7 @@
import shutil
import errno
import json
+import time
def get_app_name(pid):
@@ -137,6 +138,91 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def validate_stats(process, stat_specs):
+ """Validate stat specifications and check that fields are numeric.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+
+ Returns:
+ List of tuples (spec, command, field) for valid specs, or None on error
+ """
+ parsed_specs = []
+ for spec in stat_specs:
+ # Parse the stat specification
+ if "." not in spec:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ command, field = spec.rsplit(".", 1)
+ if not command or not field:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ # Query the stat once to validate it exists and is numeric
+ data = query_telemetry(process, command)
+ if not isinstance(data, dict):
+ print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
+ return None
+ if field not in data:
+ print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
+ return None
+ value = data[field]
+ if not isinstance(value, (int, float)):
+ print(
+ f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
+ file=sys.stderr,
+ )
+ return None
+
+ parsed_specs.append((spec, command, field))
+
+ return parsed_specs
+
+
+def monitor_stats(process, stat_specs):
+ """Monitor and display statistics in columns.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+ """
+ # Validate all stat specifications
+ parsed_specs = validate_stats(process, stat_specs)
+ if not parsed_specs:
+ return
+
+ # Print header
+ header = "Time".ljust(10)
+ for spec, _, _ in parsed_specs:
+ header += spec.rjust(25)
+ print(header)
+
+ # Monitor loop - once per second
+ try:
+ while True:
+ timestamp = time.strftime("%H:%M:%S")
+ row = timestamp.ljust(10)
+
+ for spec, command, field in parsed_specs:
+ data = query_telemetry(process, command)
+ row += str(data[field]).rjust(25)
+
+ print(row)
+ time.sleep(1)
+ except KeyboardInterrupt:
+ print("\nMonitoring stopped")
+
+
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
@@ -164,6 +250,11 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "stats",
+ nargs="*",
+ help="Statistics to monitor in format 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ )
args = parser.parse_args()
@@ -179,13 +270,21 @@ def main():
cmd = [sys.executable, telemetry_script] + args_list
return subprocess.run(cmd).returncode
+ # Check if stats were provided
+ if not args.stats:
+ print("Error: No statistics to monitor specified", file=sys.stderr)
+ print("Usage: dpdk-telemetry-watcher.py [options] stat1 stat2 ...", file=sys.stderr)
+ print("Example: dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets", file=sys.stderr)
+ return 1
+
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
# Get and display the connected application name
print_connected_app(process)
- # TODO: Add monitoring logic here
+ # Monitor the requested statistics
+ monitor_stats(process, args.stats)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [RFC PATCH 3/7] usertools/telemetry-watcher: add delta and timeout opts
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
@ 2025-12-10 16:55 ` Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
` (8 subsequent siblings)
11 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Add options to the script to time out and quit after a certain number of
seconds. Also, add an option to display delta instead of absolute
values, allowing easy display of e.g. PPS.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 64 +++++++++++++++++++++--------
1 file changed, 48 insertions(+), 16 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 824c9a37be..8af9af867c 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -146,9 +146,13 @@ def validate_stats(process, stat_specs):
stat_specs: List of stat specifications in format "command.field"
Returns:
- List of tuples (spec, command, field) for valid specs, or None on error
+ Tuple of (parsed_specs, initial_values) where:
+ parsed_specs: List of tuples (spec, command, field) for valid specs
+ initial_values: List of initial values for each stat
+ Returns (None, None) on error
"""
parsed_specs = []
+ initial_values = []
for spec in stat_specs:
# Parse the stat specification
if "." not in spec:
@@ -157,7 +161,7 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
command, field = spec.rsplit(".", 1)
if not command or not field:
@@ -166,38 +170,39 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
# Query the stat once to validate it exists and is numeric
data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
- return None
+ return None, None
if field not in data:
print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
- return None
+ return None, None
value = data[field]
if not isinstance(value, (int, float)):
print(
f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
file=sys.stderr,
)
- return None
+ return None, None
parsed_specs.append((spec, command, field))
+ initial_values.append(value)
- return parsed_specs
+ return parsed_specs, initial_values
-def monitor_stats(process, stat_specs):
+def monitor_stats(process, args):
"""Monitor and display statistics in columns.
Args:
process: The subprocess.Popen handle to the telemetry process
- stat_specs: List of stat specifications in format "command.field"
+ args: Parsed command line arguments
"""
- # Validate all stat specifications
- parsed_specs = validate_stats(process, stat_specs)
+ # Validate all stat specifications and get initial values
+ parsed_specs, prev_values = validate_stats(process, args.stats)
if not parsed_specs:
return
@@ -208,17 +213,30 @@ def monitor_stats(process, stat_specs):
print(header)
# Monitor loop - once per second
+ count = 0
try:
- while True:
+ while args.timeout is None or count < args.timeout:
+ time.sleep(1)
+ count += 1
+
timestamp = time.strftime("%H:%M:%S")
row = timestamp.ljust(10)
- for spec, command, field in parsed_specs:
+ current_values = []
+ for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
- row += str(data[field]).rjust(25)
+ current_value = data[field]
+ current_values.append(current_value)
+
+ if args.delta:
+ display_value = current_value - prev_values[i]
+ else:
+ display_value = current_value
+
+ row += str(display_value).rjust(25)
print(row)
- time.sleep(1)
+ prev_values = current_values
except KeyboardInterrupt:
print("\nMonitoring stopped")
@@ -250,6 +268,20 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "-t",
+ "--timeout",
+ type=int,
+ default=None,
+ help="Number of iterations to run before stopping (default: run indefinitely)",
+ )
+ parser.add_argument(
+ "-d",
+ "--delta",
+ action="store_true",
+ default=False,
+ help="Display delta values instead of absolute values",
+ )
parser.add_argument(
"stats",
nargs="*",
@@ -284,7 +316,7 @@ def main():
print_connected_app(process)
# Monitor the requested statistics
- monitor_stats(process, args.stats)
+ monitor_stats(process, args)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [RFC PATCH 4/7] usertools/telemetry-watcher: add total and one-line opts
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (2 preceding siblings ...)
2025-12-10 16:55 ` [RFC PATCH 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
@ 2025-12-10 16:55 ` Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
` (7 subsequent siblings)
11 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Add options to the script to print out totals at the end of each line,
and to print each line replacing the previous one, saving the output
scrolling up the screen constantly.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 27 ++++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 8af9af867c..bdf1423dd3 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -210,10 +210,13 @@ def monitor_stats(process, args):
header = "Time".ljust(10)
for spec, _, _ in parsed_specs:
header += spec.rjust(25)
+ if args.total:
+ header += "Total".rjust(25)
print(header)
# Monitor loop - once per second
count = 0
+ line_ending = "\r" if args.single_line else "\n"
try:
while args.timeout is None or count < args.timeout:
time.sleep(1)
@@ -223,6 +226,7 @@ def monitor_stats(process, args):
row = timestamp.ljust(10)
current_values = []
+ total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
current_value = data[field]
@@ -233,11 +237,17 @@ def monitor_stats(process, args):
else:
display_value = current_value
+ total += display_value
row += str(display_value).rjust(25)
- print(row)
+ if args.total:
+ row += str(total).rjust(25)
+
+ print(row, end=line_ending, flush=True)
prev_values = current_values
except KeyboardInterrupt:
+ if args.single_line:
+ print() # Add newline before exit message
print("\nMonitoring stopped")
@@ -282,6 +292,21 @@ def main():
default=False,
help="Display delta values instead of absolute values",
)
+ parser.add_argument(
+ "-T",
+ "--total",
+ action="store_true",
+ default=False,
+ help="Display a total column at the end of each row",
+ )
+ parser.add_argument(
+ "-1",
+ "--single-line",
+ action="store_true",
+ default=False,
+ dest="single_line",
+ help="Display output on a single line, replacing the previous output",
+ )
parser.add_argument(
"stats",
nargs="*",
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [RFC PATCH 5/7] usertools/telemetry-watcher: add thousands separator
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (3 preceding siblings ...)
2025-12-10 16:55 ` [RFC PATCH 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
@ 2025-12-10 16:55 ` Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
` (6 subsequent siblings)
11 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
When printing large values of packets, generally in the millions, things
are far more readable with thousands and millions separators.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index bdf1423dd3..59ec4c85b6 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -15,6 +15,7 @@
import errno
import json
import time
+import locale
def get_app_name(pid):
@@ -238,10 +239,10 @@ def monitor_stats(process, args):
display_value = current_value
total += display_value
- row += str(display_value).rjust(25)
+ row += f"{display_value:n}".rjust(25)
if args.total:
- row += str(total).rjust(25)
+ row += f"{total:n}".rjust(25)
print(row, end=line_ending, flush=True)
prev_values = current_values
@@ -254,6 +255,9 @@ def monitor_stats(process, args):
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+ # Set locale for number formatting
+ locale.setlocale(locale.LC_ALL, "")
+
# Parse command line arguments - matching dpdk-telemetry.py parameters
parser = argparse.ArgumentParser(
description="Monitor DPDK telemetry statistics on the command line"
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [RFC PATCH 6/7] usertools/telemetry-watcher: add eth name shortcuts
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (4 preceding siblings ...)
2025-12-10 16:55 ` [RFC PATCH 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
@ 2025-12-10 16:55 ` Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
` (5 subsequent siblings)
11 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Since an expected main use of the script is to monitor ethdev packet
stats, provide a shortened form of parameters to make it easier to
monitor all ports on the system. Any stat starting with "eth." is taken
not as a direct command, but instead as a shortcut for getting the stats
for all ports on the system. For example: eth.ibytes shows the byte
counts for all ports.
Beyond that, provide a shortcut for ipackets and opackets as just rx and
tx respectively. Therefore, to monitor the output rate of an app, one
can use "dpdk-telemetry-watcher -dT eth.tx"
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 49 ++++++++++++++++++++++++++++-
1 file changed, 48 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 59ec4c85b6..6beb67d29a 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -139,6 +139,48 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def expand_shortcuts(process, stat_specs):
+ """Expand special shortcuts like eth.rx and eth.tx into actual stat specifications.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications, possibly including shortcuts
+
+ Returns:
+ List of expanded stat specifications
+ """
+ expanded = []
+ for spec in stat_specs:
+ if not spec.startswith("eth."):
+ expanded.append(spec)
+ continue
+
+ # Extract the field name after "eth."
+ field = spec[4:] # Remove "eth." prefix
+ if not field:
+ print(f"Error: Invalid shortcut '{spec}' - missing field name", file=sys.stderr)
+ return None
+
+ # Map common shortcuts to actual field names
+ field_map = {
+ "rx": "ipackets",
+ "tx": "opackets",
+ }
+ field = field_map.get(field, field)
+
+ # Get list of ethernet devices
+ port_list = query_telemetry(process, "/ethdev/list")
+ if not isinstance(port_list, list):
+ print(f"Error: Failed to get ethernet device list", file=sys.stderr)
+ return None
+
+ # Create stat specs for each port
+ for port in port_list:
+ expanded.append(f"/ethdev/stats,{port}.{field}")
+
+ return expanded
+
+
def validate_stats(process, stat_specs):
"""Validate stat specifications and check that fields are numeric.
@@ -202,8 +244,13 @@ def monitor_stats(process, args):
process: The subprocess.Popen handle to the telemetry process
args: Parsed command line arguments
"""
+ # Expand any shortcuts like eth-rx, eth-tx
+ expanded_stats = expand_shortcuts(process, args.stats)
+ if not expanded_stats:
+ return
+
# Validate all stat specifications and get initial values
- parsed_specs, prev_values = validate_stats(process, args.stats)
+ parsed_specs, prev_values = validate_stats(process, expanded_stats)
if not parsed_specs:
return
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [RFC PATCH 7/7] usertools/telemetry-watcher: support reconnection
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (5 preceding siblings ...)
2025-12-10 16:55 ` [RFC PATCH 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
@ 2025-12-10 16:55 ` Bruce Richardson
2025-12-11 1:09 ` [RFC PATCH 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
` (4 subsequent siblings)
11 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-10 16:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Allow the watcher binary to run even when there is no DPDK process
running. In that case, wait for a suitable process to start and begin
monitoring then. In case of disconnection, keep trying to reconnect and
resume once reconnection succeeds.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 46 +++++++++++++++++++++--------
1 file changed, 34 insertions(+), 12 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 6beb67d29a..e5f470e08c 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -85,6 +85,12 @@ def create_telemetry_process(telemetry_script, args_list):
text=True,
bufsize=1, # Line buffered
)
+
+ # Get and display the connected application name
+ if not print_connected_app(process):
+ return None
+ process.script = telemetry_script # Store script path for reference
+ process.args = args_list # Store args for reference
return process
except FileNotFoundError:
print(f"Error: Python interpreter or script not found", file=sys.stderr)
@@ -102,15 +108,28 @@ def query_telemetry(process, command):
command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
Returns:
- dict: The parsed JSON response with the command wrapper stripped,
+ (process, dict): The process handle, in case of reconnection, and the
+ parsed JSON response with the command wrapper stripped,
or None if there was an error
"""
# Send the command
process.stdin.write(f"{command}\n")
process.stdin.flush()
- # Read the JSON response
+ # Read the JSON response, reconnecting if necessary
response = process.stdout.readline()
+ while not response:
+ script = process.script
+ args_list = process.args
+ process = None
+ print("Application disconnected, retrying...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(script, args_list)
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+ response = process.stdout.readline()
+
try:
data = json.loads(response)
# When run non-interactively, the response is wrapped with the command
@@ -119,11 +138,11 @@ def query_telemetry(process, command):
# The response should have exactly one key which is the command
if len(data) == 1:
# Extract the value, ignoring the key
- return next(iter(data.values()))
+ return (process, next(iter(data.values())))
else:
- return data
+ return (process, data)
except (json.JSONDecodeError, KeyError):
- return None
+ return (None, None)
def print_connected_app(process):
@@ -132,11 +151,12 @@ def print_connected_app(process):
Args:
process: The subprocess.Popen handle to the telemetry process
"""
- info = query_telemetry(process, "/info")
+ process, info = query_telemetry(process, "/info")
if info and "pid" in info:
app_name = get_app_name(info["pid"])
if app_name:
print(f'Connected to application: "{app_name}"')
+ return process
def expand_shortcuts(process, stat_specs):
@@ -169,7 +189,7 @@ def expand_shortcuts(process, stat_specs):
field = field_map.get(field, field)
# Get list of ethernet devices
- port_list = query_telemetry(process, "/ethdev/list")
+ process, port_list = query_telemetry(process, "/ethdev/list")
if not isinstance(port_list, list):
print(f"Error: Failed to get ethernet device list", file=sys.stderr)
return None
@@ -216,7 +236,7 @@ def validate_stats(process, stat_specs):
return None, None
# Query the stat once to validate it exists and is numeric
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
return None, None
@@ -276,7 +296,7 @@ def monitor_stats(process, args):
current_values = []
total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
current_value = data[field]
current_values.append(current_value)
@@ -387,9 +407,11 @@ def main():
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
-
- # Get and display the connected application name
- print_connected_app(process)
+ if not process:
+ print("Waiting for connection to DPDK application...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(telemetry_script, args_list)
# Monitor the requested statistics
monitor_stats(process, args)
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* Re: [RFC PATCH 0/7] Add script for real-time telemetry monitoring
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (6 preceding siblings ...)
2025-12-10 16:55 ` [RFC PATCH 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
@ 2025-12-11 1:09 ` Stephen Hemminger
2025-12-11 9:10 ` Bruce Richardson
2025-12-12 5:32 ` Stephen Hemminger
` (3 subsequent siblings)
11 siblings, 1 reply; 41+ messages in thread
From: Stephen Hemminger @ 2025-12-11 1:09 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On Wed, 10 Dec 2025 16:55:25 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> TL;DR
> ------
>
> For a quick demo, apply patces, run e.g. testpmd and then in a separate
> terminal run:
>
> ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
>
> Output, updated once per second, will be traffic rate per port e.g.:
>
> Connected to application: "dpdk-testpmd"
> Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> 16:29:12 5,213,119 5,214,304 10,427,423
>
>
> Fuller details
> --------------
>
> While we have the dpdk-telemetry.py CLI app for interactive querying of
> telemetry on the commandline, and a telemetry exporter script for
> sending telemetry to external tools for real-time monitoring, we don't
> have an app that can print real-time stats for DPDK apps on the
> terminal. This patchset adds such a script, developed with the help of
> Github copilot to fill a need that I found in my testing. Submitting it
> here in the hopes that others find it of use.
>
> The script acts as a wrapper around the existing dpdk-telemetry.py
> script, and pipes the commands to that script and reads the responses,
> querying it once per second. It takes a number of flag parameters, such
> as the ones above:
> - "-d" for delta values, i.e. PPS rather than total packets
> - "-1" for single-line output, i.e. no scrolling up the screen
> - "-T" to display a total column
>
> Other flag parameters can be seen by looking at the help output.
>
> Beyond the flags, the script also takes a number of positional
> parameters, which refer to specific stats to display. These stats must
> be numeric values, and should take the form of the telemetry command to
> send, followed by a "." and the stat within the result which is to be
> tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
> where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
> part of the result.
>
> However, specifying individual stats can be awkward, so some shortcuts
> are provided too for the common case of monitoring ethernet ports. Any
> positional arg starting with "eth" will be replaced by the set of
> equivalent values for each port, e.g. "eth.imissed" will track the
> imissed value on all ports in use in the app. The ipackets and opackets
> values, as common metrics, are also available as shortened values as
> just "rx" and "tx", so in the example above, "eth.tx" means to track the
> opackets stat for every ethdev port.
>
> Finally, the script also has reconnection support so you can leave it
> running while you start and stop your application in another terminal.
> The watcher will try and reconnect to a running instance every second.
>
There was a previous submission of a nice telemetry TUI which seems to have
been abandoned, is there overlap with this?
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [RFC PATCH 0/7] Add script for real-time telemetry monitoring
2025-12-11 1:09 ` [RFC PATCH 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
@ 2025-12-11 9:10 ` Bruce Richardson
0 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2025-12-11 9:10 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
On Thu, Dec 11, 2025 at 10:09:10AM +0900, Stephen Hemminger wrote:
> On Wed, 10 Dec 2025 16:55:25 +0000
> Bruce Richardson <bruce.richardson@intel.com> wrote:
>
> > TL;DR
> > ------
> >
> > For a quick demo, apply patces, run e.g. testpmd and then in a separate
> > terminal run:
> >
> > ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
> >
> > Output, updated once per second, will be traffic rate per port e.g.:
> >
> > Connected to application: "dpdk-testpmd"
> > Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> > 16:29:12 5,213,119 5,214,304 10,427,423
> >
> >
> > Fuller details
> > --------------
> >
> > While we have the dpdk-telemetry.py CLI app for interactive querying of
> > telemetry on the commandline, and a telemetry exporter script for
> > sending telemetry to external tools for real-time monitoring, we don't
> > have an app that can print real-time stats for DPDK apps on the
> > terminal. This patchset adds such a script, developed with the help of
> > Github copilot to fill a need that I found in my testing. Submitting it
> > here in the hopes that others find it of use.
> >
> > The script acts as a wrapper around the existing dpdk-telemetry.py
> > script, and pipes the commands to that script and reads the responses,
> > querying it once per second. It takes a number of flag parameters, such
> > as the ones above:
> > - "-d" for delta values, i.e. PPS rather than total packets
> > - "-1" for single-line output, i.e. no scrolling up the screen
> > - "-T" to display a total column
> >
> > Other flag parameters can be seen by looking at the help output.
> >
> > Beyond the flags, the script also takes a number of positional
> > parameters, which refer to specific stats to display. These stats must
> > be numeric values, and should take the form of the telemetry command to
> > send, followed by a "." and the stat within the result which is to be
> > tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
> > where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
> > part of the result.
> >
> > However, specifying individual stats can be awkward, so some shortcuts
> > are provided too for the common case of monitoring ethernet ports. Any
> > positional arg starting with "eth" will be replaced by the set of
> > equivalent values for each port, e.g. "eth.imissed" will track the
> > imissed value on all ports in use in the app. The ipackets and opackets
> > values, as common metrics, are also available as shortened values as
> > just "rx" and "tx", so in the example above, "eth.tx" means to track the
> > opackets stat for every ethdev port.
> >
> > Finally, the script also has reconnection support so you can leave it
> > running while you start and stop your application in another terminal.
> > The watcher will try and reconnect to a running instance every second.
> >
>
>
> There was a previous submission of a nice telemetry TUI which seems to have
> been abandoned, is there overlap with this?
>
Possibly. I didn't remember that being submitted in the past. This was just
something I came up with a while back for my own use, and find it really
handy for quickly getting some perf numbers. Therefore I thought others
might find it of use too.
/Bruce
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [RFC PATCH 0/7] Add script for real-time telemetry monitoring
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (7 preceding siblings ...)
2025-12-11 1:09 ` [RFC PATCH 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
@ 2025-12-12 5:32 ` Stephen Hemminger
2025-12-12 17:52 ` Bruce Richardson
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
` (2 subsequent siblings)
11 siblings, 1 reply; 41+ messages in thread
From: Stephen Hemminger @ 2025-12-12 5:32 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On Wed, 10 Dec 2025 16:55:25 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> TL;DR
> ------
>
> For a quick demo, apply patces, run e.g. testpmd and then in a separate
> terminal run:
>
> ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
>
> Output, updated once per second, will be traffic rate per port e.g.:
>
> Connected to application: "dpdk-testpmd"
> Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> 16:29:12 5,213,119 5,214,304 10,427,423
>
>
> Fuller details
> --------------
>
> While we have the dpdk-telemetry.py CLI app for interactive querying of
> telemetry on the commandline, and a telemetry exporter script for
> sending telemetry to external tools for real-time monitoring, we don't
> have an app that can print real-time stats for DPDK apps on the
> terminal. This patchset adds such a script, developed with the help of
> Github copilot to fill a need that I found in my testing. Submitting it
> here in the hopes that others find it of use.
>
> The script acts as a wrapper around the existing dpdk-telemetry.py
> script, and pipes the commands to that script and reads the responses,
> querying it once per second. It takes a number of flag parameters, such
> as the ones above:
> - "-d" for delta values, i.e. PPS rather than total packets
> - "-1" for single-line output, i.e. no scrolling up the screen
> - "-T" to display a total column
>
> Other flag parameters can be seen by looking at the help output.
>
> Beyond the flags, the script also takes a number of positional
> parameters, which refer to specific stats to display. These stats must
> be numeric values, and should take the form of the telemetry command to
> send, followed by a "." and the stat within the result which is to be
> tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
> where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
> part of the result.
>
> However, specifying individual stats can be awkward, so some shortcuts
> are provided too for the common case of monitoring ethernet ports. Any
> positional arg starting with "eth" will be replaced by the set of
> equivalent values for each port, e.g. "eth.imissed" will track the
> imissed value on all ports in use in the app. The ipackets and opackets
> values, as common metrics, are also available as shortened values as
> just "rx" and "tx", so in the example above, "eth.tx" means to track the
> opackets stat for every ethdev port.
>
> Finally, the script also has reconnection support so you can leave it
> running while you start and stop your application in another terminal.
> The watcher will try and reconnect to a running instance every second.
>
>
> Bruce Richardson (7):
> usertools: add new script to monitor telemetry on terminal
> usertools/telemetry-watcher: add displaying stats
> usertools/telemetry-watcher: add delta and timeout opts
> usertools/telemetry-watcher: add total and one-line opts
> usertools/telemetry-watcher: add thousands separator
> usertools/telemetry-watcher: add eth name shortcuts
> usertools/telemetry-watcher: support reconnection
>
> usertools/dpdk-telemetry-watcher.py | 429 ++++++++++++++++++++++++++++
> usertools/meson.build | 1 +
> 2 files changed, 430 insertions(+)
> create mode 100755 usertools/dpdk-telemetry-watcher.py
>
> --
> 2.51.0
The TUI patch was
https://patchwork.dpdk.org/project/dpdk/patch/20220831115250.362189-2-conor.walsh@intel.com/
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [RFC PATCH 0/7] Add script for real-time telemetry monitoring
2025-12-12 5:32 ` Stephen Hemminger
@ 2025-12-12 17:52 ` Bruce Richardson
2025-12-12 23:23 ` Stephen Hemminger
0 siblings, 1 reply; 41+ messages in thread
From: Bruce Richardson @ 2025-12-12 17:52 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
On Fri, Dec 12, 2025 at 02:32:49PM +0900, Stephen Hemminger wrote:
> On Wed, 10 Dec 2025 16:55:25 +0000
> Bruce Richardson <bruce.richardson@intel.com> wrote:
>
> > TL;DR
> > ------
> >
> > For a quick demo, apply patces, run e.g. testpmd and then in a separate
> > terminal run:
> >
> > ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
> >
> > Output, updated once per second, will be traffic rate per port e.g.:
> >
> > Connected to application: "dpdk-testpmd"
> > Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> > 16:29:12 5,213,119 5,214,304 10,427,423
> >
> >
<snip>
> >
> > usertools/dpdk-telemetry-watcher.py | 429 ++++++++++++++++++++++++++++
> > usertools/meson.build | 1 +
> > 2 files changed, 430 insertions(+)
> > create mode 100755 usertools/dpdk-telemetry-watcher.py
> >
> > --
> > 2.51.0
>
>
> The TUI patch was
> https://patchwork.dpdk.org/project/dpdk/patch/20220831115250.362189-2-conor.walsh@intel.com/
>
Took a look at that patch proposal, and here are some of the main
differences between the two scripts:
* the telemetry-tui is a full-screen app which is a lot prettier and
probably much more suitable for non-experts.
* the script I provided is FAR less user friendly and operates at a much
more "techie" level
* on the other hand, it's more flexible because of that:
- can be used to display any numeric stats - the tui is hard-coded to show
particular NIC stats - including e.g. cryptodev, or dmadev ones
- can show deltas and totals as desired
- can quit after a certain fixed amount of time
* my script is probably more limited in terms of what can be displayed,
since it assumes that you want one-line of output per update. TUI
displays lots of (fixed)stats simultaneously
* TUI as proposed has a few glitches - got a divide by zero error when run
against a testpmd instance that didn't have traffic running, so all stats
were zero. (That's an easy fix though)
* The new script I propose has one major feature that I would look to add
to the TUI if we merged that - the reconnection support. I find it very
handy to run the watcher script and leave it running while I start and
stop whatever app(s) I'm using.
* My script has no additional dependencies beyond stock python, while the
TUI requires one additional dependency "rich" to manage the TUI. It also
can use a second dependency "plotext" (not Ubuntu packaged) to display
charts if so desired.
Just my quick assessment of the two from a quick try of the TUI one.
/Bruce
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [RFC PATCH 0/7] Add script for real-time telemetry monitoring
2025-12-12 17:52 ` Bruce Richardson
@ 2025-12-12 23:23 ` Stephen Hemminger
0 siblings, 0 replies; 41+ messages in thread
From: Stephen Hemminger @ 2025-12-12 23:23 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On Fri, 12 Dec 2025 17:52:15 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> On Fri, Dec 12, 2025 at 02:32:49PM +0900, Stephen Hemminger wrote:
> > On Wed, 10 Dec 2025 16:55:25 +0000
> > Bruce Richardson <bruce.richardson@intel.com> wrote:
> >
> > > TL;DR
> > > ------
> > >
> > > For a quick demo, apply patces, run e.g. testpmd and then in a separate
> > > terminal run:
> > >
> > > ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
> > >
> > > Output, updated once per second, will be traffic rate per port e.g.:
> > >
> > > Connected to application: "dpdk-testpmd"
> > > Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> > > 16:29:12 5,213,119 5,214,304 10,427,423
> > >
> > >
> <snip>
> > >
> > > usertools/dpdk-telemetry-watcher.py | 429 ++++++++++++++++++++++++++++
> > > usertools/meson.build | 1 +
> > > 2 files changed, 430 insertions(+)
> > > create mode 100755 usertools/dpdk-telemetry-watcher.py
> > >
> > > --
> > > 2.51.0
> >
> >
> > The TUI patch was
> > https://patchwork.dpdk.org/project/dpdk/patch/20220831115250.362189-2-conor.walsh@intel.com/
> >
>
> Took a look at that patch proposal, and here are some of the main
> differences between the two scripts:
>
> * the telemetry-tui is a full-screen app which is a lot prettier and
> probably much more suitable for non-experts.
> * the script I provided is FAR less user friendly and operates at a much
> more "techie" level
> * on the other hand, it's more flexible because of that:
> - can be used to display any numeric stats - the tui is hard-coded to show
> particular NIC stats - including e.g. cryptodev, or dmadev ones
> - can show deltas and totals as desired
> - can quit after a certain fixed amount of time
> * my script is probably more limited in terms of what can be displayed,
> since it assumes that you want one-line of output per update. TUI
> displays lots of (fixed)stats simultaneously
> * TUI as proposed has a few glitches - got a divide by zero error when run
> against a testpmd instance that didn't have traffic running, so all stats
> were zero. (That's an easy fix though)
> * The new script I propose has one major feature that I would look to add
> to the TUI if we merged that - the reconnection support. I find it very
> handy to run the watcher script and leave it running while I start and
> stop whatever app(s) I'm using.
> * My script has no additional dependencies beyond stock python, while the
> TUI requires one additional dependency "rich" to manage the TUI. It also
> can use a second dependency "plotext" (not Ubuntu packaged) to display
> charts if so desired.
>
> Just my quick assessment of the two from a quick try of the TUI one.
>
> /Bruce
It is not a case of A or B. Both tools are useful.
I can see uses for both. But original submission of TUI seems to have
been a one off patch with no long term followup
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v2 0/7] Add script for real-time telemetry monitoring
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (8 preceding siblings ...)
2025-12-12 5:32 ` Stephen Hemminger
@ 2026-01-05 17:55 ` Bruce Richardson
2026-01-05 17:55 ` [PATCH v2 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
` (7 more replies)
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
11 siblings, 8 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
TL;DR
------
For a quick demo, apply patches, run e.g. testpmd and then in a separate
terminal run:
./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
Output, updated once per second, will be traffic rate per port e.g.:
Connected to application: "dpdk-testpmd"
Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
16:29:12 5,213,119 5,214,304 10,427,423
Fuller details
--------------
While we have the dpdk-telemetry.py CLI app for interactive querying of
telemetry on the commandline, and a telemetry exporter script for
sending telemetry to external tools for real-time monitoring, we don't
have an app that can print real-time stats for DPDK apps on the
terminal. This patchset adds such a script, developed with the help of
Github copilot to fill a need that I found in my testing. Submitting it
here in the hopes that others find it of use.
The script acts as a wrapper around the existing dpdk-telemetry.py
script, and pipes the commands to that script and reads the responses,
querying it once per second. It takes a number of flag parameters, such
as the ones above:
- "-d" for delta values, i.e. PPS rather than total packets
- "-1" for single-line output, i.e. no scrolling up the screen
- "-T" to display a total column
Other flag parameters can be seen by looking at the help output.
Beyond the flags, the script also takes a number of positional
parameters, which refer to specific stats to display. These stats must
be numeric values, and should take the form of the telemetry command to
send, followed by a "." and the stat within the result which is to be
tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
part of the result.
However, specifying individual stats can be awkward, so some shortcuts
are provided too for the common case of monitoring ethernet ports. Any
positional arg starting with "eth" will be replaced by the set of
equivalent values for each port, e.g. "eth.imissed" will track the
imissed value on all ports in use in the app. The ipackets and opackets
values, as common metrics, are also available as shortened values as
just "rx" and "tx", so in the example above, "eth.tx" means to track the
opackets stat for every ethdev port.
Finally, the script also has reconnection support so you can leave it
running while you start and stop your application in another terminal.
The watcher will try and reconnect to a running instance every second.
v2:
- improve reconnection handling, eliminating some crashes seen in testing.
Bruce Richardson (7):
usertools: add new script to monitor telemetry on terminal
usertools/telemetry-watcher: add displaying stats
usertools/telemetry-watcher: add delta and timeout opts
usertools/telemetry-watcher: add total and one-line opts
usertools/telemetry-watcher: add thousands separator
usertools/telemetry-watcher: add eth name shortcuts
usertools/telemetry-watcher: support reconnection
usertools/dpdk-telemetry-watcher.py | 435 ++++++++++++++++++++++++++++
usertools/meson.build | 1 +
2 files changed, 436 insertions(+)
create mode 100755 usertools/dpdk-telemetry-watcher.py
--
2.51.0
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v2 1/7] usertools: add new script to monitor telemetry on terminal
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
@ 2026-01-05 17:55 ` Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
` (6 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:55 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
The dpdk-telemetry.py script is useful for getting telemetry
interactively, but sometimes we want to monitor stats over time, so add
a telemetry-watcher script to do so.
Start off such a script with basic arg processing, and connecting to
dpdk-telemetry as a subprocess, so that we can send-receive commands
from it.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 200 ++++++++++++++++++++++++++++
usertools/meson.build | 1 +
2 files changed, 201 insertions(+)
create mode 100755 usertools/dpdk-telemetry-watcher.py
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
new file mode 100755
index 0000000000..01c8683d33
--- /dev/null
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation
+
+"""
+Script to monitor DPDK telemetry statistics on the command line.
+Wraps dpdk-telemetry.py to provide continuous monitoring capabilities.
+"""
+
+import argparse
+import subprocess
+import sys
+import os
+import shutil
+import errno
+import json
+
+
+def get_app_name(pid):
+ """return the app name for a given PID, for printing"""
+ proc_cmdline = os.path.join("/proc", str(pid), "cmdline")
+ try:
+ with open(proc_cmdline) as f:
+ argv0 = f.read(1024).split("\0")[0]
+ return os.path.basename(argv0)
+ except IOError as e:
+ # ignore file not found errors
+ if e.errno != errno.ENOENT:
+ raise
+ return None
+
+
+def find_telemetry_script():
+ """Find the dpdk-telemetry.py script in the script directory or PATH.
+
+ Returns:
+ str: Path to the dpdk-telemetry.py script
+
+ Exits:
+ If the script cannot be found
+ """
+ # First, try to find it in the same directory as this script
+ script_dir = os.path.dirname(os.path.abspath(__file__))
+ telemetry_script = os.path.join(script_dir, "dpdk-telemetry.py")
+
+ # If not found locally, check if it's in PATH
+ if not os.path.exists(telemetry_script):
+ telemetry_in_path = shutil.which("dpdk-telemetry.py")
+ if telemetry_in_path:
+ telemetry_script = telemetry_in_path
+ else:
+ print(
+ "Error: dpdk-telemetry.py not found in script directory or PATH",
+ file=sys.stderr,
+ )
+ sys.exit(1)
+
+ return telemetry_script
+
+
+def create_telemetry_process(telemetry_script, args_list):
+ """Create a subprocess for dpdk-telemetry.py with pipes.
+
+ Args:
+ telemetry_script: Path to the dpdk-telemetry.py script
+ args_list: List of arguments to pass to the script
+
+ Returns:
+ subprocess.Popen: Process handle with stdin/stdout/stderr pipes
+
+ Exits:
+ If the process cannot be created
+ """
+ # Build the command
+ cmd = [sys.executable, telemetry_script] + args_list
+
+ try:
+ process = subprocess.Popen(
+ cmd,
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ text=True,
+ bufsize=1, # Line buffered
+ )
+ return process
+ except FileNotFoundError:
+ print(f"Error: Python interpreter or script not found", file=sys.stderr)
+ sys.exit(1)
+ except Exception as e:
+ print(f"Error running dpdk-telemetry.py: {e}", file=sys.stderr)
+ sys.exit(1)
+
+
+def query_telemetry(process, command):
+ """Send a telemetry command and return the parsed JSON response.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
+
+ Returns:
+ dict: The parsed JSON response with the command wrapper stripped,
+ or None if there was an error
+ """
+ # Send the command
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+
+ # Read the JSON response
+ response = process.stdout.readline()
+ try:
+ data = json.loads(response)
+ # When run non-interactively, the response is wrapped with the command
+ # e.g., {"/info": {"version": ..., "pid": ...}}
+ # or {"/ethdev/stats,0": {...}}
+ # The response should have exactly one key which is the command
+ if len(data) == 1:
+ # Extract the value, ignoring the key
+ return next(iter(data.values()))
+ else:
+ return data
+ except (json.JSONDecodeError, KeyError):
+ return None
+
+
+def print_connected_app(process):
+ """Query and print the name of the connected DPDK application.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ """
+ info = query_telemetry(process, "/info")
+ if info and "pid" in info:
+ app_name = get_app_name(info["pid"])
+ if app_name:
+ print(f'Connected to application: "{app_name}"')
+
+
+def main():
+ """Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+
+ # Parse command line arguments - matching dpdk-telemetry.py parameters
+ parser = argparse.ArgumentParser(
+ description="Monitor DPDK telemetry statistics on the command line"
+ )
+ parser.add_argument(
+ "-f",
+ "--file-prefix",
+ default="rte",
+ help="Provide file-prefix for DPDK runtime directory",
+ )
+ parser.add_argument(
+ "-i",
+ "--instance",
+ default=0,
+ type=int,
+ help="Provide instance number for DPDK application",
+ )
+ parser.add_argument(
+ "-l",
+ "--list",
+ action="store_true",
+ default=False,
+ help="List all possible file-prefixes and exit",
+ )
+
+ args = parser.parse_args()
+
+ # Find the dpdk-telemetry.py script
+ telemetry_script = find_telemetry_script()
+
+ # Build arguments list
+ args_list = ["-f", args.file_prefix, "-i", str(args.instance)]
+
+ if args.list:
+ args_list.append("-l")
+ # For --list, just run the command directly without pipes
+ cmd = [sys.executable, telemetry_script] + args_list
+ return subprocess.run(cmd).returncode
+
+ # Run dpdk-telemetry.py with pipes for stdin and stdout
+ process = create_telemetry_process(telemetry_script, args_list)
+
+ # Get and display the connected application name
+ print_connected_app(process)
+
+ # TODO: Add monitoring logic here
+
+ # Clean up
+ process.stdin.close()
+ process.stdout.close()
+ process.stderr.close()
+ process.wait()
+
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/usertools/meson.build b/usertools/meson.build
index eb48e2f440..114d0a65b4 100644
--- a/usertools/meson.build
+++ b/usertools/meson.build
@@ -12,6 +12,7 @@ install_data([
'dpdk-hugepages.py',
'dpdk-rss-flows.py',
'dpdk-telemetry-exporter.py',
+ 'dpdk-telemetry-watcher.py',
],
install_dir: 'bin')
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v2 2/7] usertools/telemetry-watcher: add displaying stats
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
2026-01-05 17:55 ` [PATCH v2 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
@ 2026-01-05 17:56 ` Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
` (5 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:56 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Add support for printing out particular stats once every second. The
stats to be queried are given on the commandline, in the format of
<command>.<result-field>, for example /ethdev/stats,0.ipackets.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 101 +++++++++++++++++++++++++++-
1 file changed, 100 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 01c8683d33..824c9a37be 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -14,6 +14,7 @@
import shutil
import errno
import json
+import time
def get_app_name(pid):
@@ -137,6 +138,91 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def validate_stats(process, stat_specs):
+ """Validate stat specifications and check that fields are numeric.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+
+ Returns:
+ List of tuples (spec, command, field) for valid specs, or None on error
+ """
+ parsed_specs = []
+ for spec in stat_specs:
+ # Parse the stat specification
+ if "." not in spec:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ command, field = spec.rsplit(".", 1)
+ if not command or not field:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ # Query the stat once to validate it exists and is numeric
+ data = query_telemetry(process, command)
+ if not isinstance(data, dict):
+ print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
+ return None
+ if field not in data:
+ print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
+ return None
+ value = data[field]
+ if not isinstance(value, (int, float)):
+ print(
+ f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
+ file=sys.stderr,
+ )
+ return None
+
+ parsed_specs.append((spec, command, field))
+
+ return parsed_specs
+
+
+def monitor_stats(process, stat_specs):
+ """Monitor and display statistics in columns.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+ """
+ # Validate all stat specifications
+ parsed_specs = validate_stats(process, stat_specs)
+ if not parsed_specs:
+ return
+
+ # Print header
+ header = "Time".ljust(10)
+ for spec, _, _ in parsed_specs:
+ header += spec.rjust(25)
+ print(header)
+
+ # Monitor loop - once per second
+ try:
+ while True:
+ timestamp = time.strftime("%H:%M:%S")
+ row = timestamp.ljust(10)
+
+ for spec, command, field in parsed_specs:
+ data = query_telemetry(process, command)
+ row += str(data[field]).rjust(25)
+
+ print(row)
+ time.sleep(1)
+ except KeyboardInterrupt:
+ print("\nMonitoring stopped")
+
+
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
@@ -164,6 +250,11 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "stats",
+ nargs="*",
+ help="Statistics to monitor in format 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ )
args = parser.parse_args()
@@ -179,13 +270,21 @@ def main():
cmd = [sys.executable, telemetry_script] + args_list
return subprocess.run(cmd).returncode
+ # Check if stats were provided
+ if not args.stats:
+ print("Error: No statistics to monitor specified", file=sys.stderr)
+ print("Usage: dpdk-telemetry-watcher.py [options] stat1 stat2 ...", file=sys.stderr)
+ print("Example: dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets", file=sys.stderr)
+ return 1
+
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
# Get and display the connected application name
print_connected_app(process)
- # TODO: Add monitoring logic here
+ # Monitor the requested statistics
+ monitor_stats(process, args.stats)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v2 3/7] usertools/telemetry-watcher: add delta and timeout opts
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
2026-01-05 17:55 ` [PATCH v2 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
@ 2026-01-05 17:56 ` Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
` (4 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:56 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Add options to the script to time out and quit after a certain number of
seconds. Also, add an option to display delta instead of absolute
values, allowing easy display of e.g. PPS.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 64 +++++++++++++++++++++--------
1 file changed, 48 insertions(+), 16 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 824c9a37be..8af9af867c 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -146,9 +146,13 @@ def validate_stats(process, stat_specs):
stat_specs: List of stat specifications in format "command.field"
Returns:
- List of tuples (spec, command, field) for valid specs, or None on error
+ Tuple of (parsed_specs, initial_values) where:
+ parsed_specs: List of tuples (spec, command, field) for valid specs
+ initial_values: List of initial values for each stat
+ Returns (None, None) on error
"""
parsed_specs = []
+ initial_values = []
for spec in stat_specs:
# Parse the stat specification
if "." not in spec:
@@ -157,7 +161,7 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
command, field = spec.rsplit(".", 1)
if not command or not field:
@@ -166,38 +170,39 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
# Query the stat once to validate it exists and is numeric
data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
- return None
+ return None, None
if field not in data:
print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
- return None
+ return None, None
value = data[field]
if not isinstance(value, (int, float)):
print(
f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
file=sys.stderr,
)
- return None
+ return None, None
parsed_specs.append((spec, command, field))
+ initial_values.append(value)
- return parsed_specs
+ return parsed_specs, initial_values
-def monitor_stats(process, stat_specs):
+def monitor_stats(process, args):
"""Monitor and display statistics in columns.
Args:
process: The subprocess.Popen handle to the telemetry process
- stat_specs: List of stat specifications in format "command.field"
+ args: Parsed command line arguments
"""
- # Validate all stat specifications
- parsed_specs = validate_stats(process, stat_specs)
+ # Validate all stat specifications and get initial values
+ parsed_specs, prev_values = validate_stats(process, args.stats)
if not parsed_specs:
return
@@ -208,17 +213,30 @@ def monitor_stats(process, stat_specs):
print(header)
# Monitor loop - once per second
+ count = 0
try:
- while True:
+ while args.timeout is None or count < args.timeout:
+ time.sleep(1)
+ count += 1
+
timestamp = time.strftime("%H:%M:%S")
row = timestamp.ljust(10)
- for spec, command, field in parsed_specs:
+ current_values = []
+ for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
- row += str(data[field]).rjust(25)
+ current_value = data[field]
+ current_values.append(current_value)
+
+ if args.delta:
+ display_value = current_value - prev_values[i]
+ else:
+ display_value = current_value
+
+ row += str(display_value).rjust(25)
print(row)
- time.sleep(1)
+ prev_values = current_values
except KeyboardInterrupt:
print("\nMonitoring stopped")
@@ -250,6 +268,20 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "-t",
+ "--timeout",
+ type=int,
+ default=None,
+ help="Number of iterations to run before stopping (default: run indefinitely)",
+ )
+ parser.add_argument(
+ "-d",
+ "--delta",
+ action="store_true",
+ default=False,
+ help="Display delta values instead of absolute values",
+ )
parser.add_argument(
"stats",
nargs="*",
@@ -284,7 +316,7 @@ def main():
print_connected_app(process)
# Monitor the requested statistics
- monitor_stats(process, args.stats)
+ monitor_stats(process, args)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v2 4/7] usertools/telemetry-watcher: add total and one-line opts
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
` (2 preceding siblings ...)
2026-01-05 17:56 ` [PATCH v2 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
@ 2026-01-05 17:56 ` Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
` (3 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:56 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Add options to the script to print out totals at the end of each line,
and to print each line replacing the previous one, saving the output
scrolling up the screen constantly.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 27 ++++++++++++++++++++++++++-
1 file changed, 26 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 8af9af867c..bdf1423dd3 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -210,10 +210,13 @@ def monitor_stats(process, args):
header = "Time".ljust(10)
for spec, _, _ in parsed_specs:
header += spec.rjust(25)
+ if args.total:
+ header += "Total".rjust(25)
print(header)
# Monitor loop - once per second
count = 0
+ line_ending = "\r" if args.single_line else "\n"
try:
while args.timeout is None or count < args.timeout:
time.sleep(1)
@@ -223,6 +226,7 @@ def monitor_stats(process, args):
row = timestamp.ljust(10)
current_values = []
+ total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
current_value = data[field]
@@ -233,11 +237,17 @@ def monitor_stats(process, args):
else:
display_value = current_value
+ total += display_value
row += str(display_value).rjust(25)
- print(row)
+ if args.total:
+ row += str(total).rjust(25)
+
+ print(row, end=line_ending, flush=True)
prev_values = current_values
except KeyboardInterrupt:
+ if args.single_line:
+ print() # Add newline before exit message
print("\nMonitoring stopped")
@@ -282,6 +292,21 @@ def main():
default=False,
help="Display delta values instead of absolute values",
)
+ parser.add_argument(
+ "-T",
+ "--total",
+ action="store_true",
+ default=False,
+ help="Display a total column at the end of each row",
+ )
+ parser.add_argument(
+ "-1",
+ "--single-line",
+ action="store_true",
+ default=False,
+ dest="single_line",
+ help="Display output on a single line, replacing the previous output",
+ )
parser.add_argument(
"stats",
nargs="*",
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v2 5/7] usertools/telemetry-watcher: add thousands separator
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
` (3 preceding siblings ...)
2026-01-05 17:56 ` [PATCH v2 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
@ 2026-01-05 17:56 ` Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
` (2 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:56 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
When printing large values of packets, generally in the millions, things
are far more readable with thousands and millions separators.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index bdf1423dd3..59ec4c85b6 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -15,6 +15,7 @@
import errno
import json
import time
+import locale
def get_app_name(pid):
@@ -238,10 +239,10 @@ def monitor_stats(process, args):
display_value = current_value
total += display_value
- row += str(display_value).rjust(25)
+ row += f"{display_value:n}".rjust(25)
if args.total:
- row += str(total).rjust(25)
+ row += f"{total:n}".rjust(25)
print(row, end=line_ending, flush=True)
prev_values = current_values
@@ -254,6 +255,9 @@ def monitor_stats(process, args):
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+ # Set locale for number formatting
+ locale.setlocale(locale.LC_ALL, "")
+
# Parse command line arguments - matching dpdk-telemetry.py parameters
parser = argparse.ArgumentParser(
description="Monitor DPDK telemetry statistics on the command line"
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v2 6/7] usertools/telemetry-watcher: add eth name shortcuts
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
` (4 preceding siblings ...)
2026-01-05 17:56 ` [PATCH v2 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
@ 2026-01-05 17:56 ` Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
2026-01-14 2:00 ` [PATCH v2 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:56 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Since an expected main use of the script is to monitor ethdev packet
stats, provide a shortened form of parameters to make it easier to
monitor all ports on the system. Any stat starting with "eth." is taken
not as a direct command, but instead as a shortcut for getting the stats
for all ports on the system. For example: eth.ibytes shows the byte
counts for all ports.
Beyond that, provide a shortcut for ipackets and opackets as just rx and
tx respectively. Therefore, to monitor the output rate of an app, one
can use "dpdk-telemetry-watcher -dT eth.tx"
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 49 ++++++++++++++++++++++++++++-
1 file changed, 48 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 59ec4c85b6..6beb67d29a 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -139,6 +139,48 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def expand_shortcuts(process, stat_specs):
+ """Expand special shortcuts like eth.rx and eth.tx into actual stat specifications.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications, possibly including shortcuts
+
+ Returns:
+ List of expanded stat specifications
+ """
+ expanded = []
+ for spec in stat_specs:
+ if not spec.startswith("eth."):
+ expanded.append(spec)
+ continue
+
+ # Extract the field name after "eth."
+ field = spec[4:] # Remove "eth." prefix
+ if not field:
+ print(f"Error: Invalid shortcut '{spec}' - missing field name", file=sys.stderr)
+ return None
+
+ # Map common shortcuts to actual field names
+ field_map = {
+ "rx": "ipackets",
+ "tx": "opackets",
+ }
+ field = field_map.get(field, field)
+
+ # Get list of ethernet devices
+ port_list = query_telemetry(process, "/ethdev/list")
+ if not isinstance(port_list, list):
+ print(f"Error: Failed to get ethernet device list", file=sys.stderr)
+ return None
+
+ # Create stat specs for each port
+ for port in port_list:
+ expanded.append(f"/ethdev/stats,{port}.{field}")
+
+ return expanded
+
+
def validate_stats(process, stat_specs):
"""Validate stat specifications and check that fields are numeric.
@@ -202,8 +244,13 @@ def monitor_stats(process, args):
process: The subprocess.Popen handle to the telemetry process
args: Parsed command line arguments
"""
+ # Expand any shortcuts like eth-rx, eth-tx
+ expanded_stats = expand_shortcuts(process, args.stats)
+ if not expanded_stats:
+ return
+
# Validate all stat specifications and get initial values
- parsed_specs, prev_values = validate_stats(process, args.stats)
+ parsed_specs, prev_values = validate_stats(process, expanded_stats)
if not parsed_specs:
return
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v2 7/7] usertools/telemetry-watcher: support reconnection
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
` (5 preceding siblings ...)
2026-01-05 17:56 ` [PATCH v2 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
@ 2026-01-05 17:56 ` Bruce Richardson
2026-01-14 2:00 ` [PATCH v2 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-05 17:56 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
Allow the watcher binary to run even when there is no DPDK process
running. In that case, wait for a suitable process to start and begin
monitoring then. In case of disconnection, keep trying to reconnect and
resume once reconnection succeeds.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
usertools/dpdk-telemetry-watcher.py | 52 ++++++++++++++++++++++-------
1 file changed, 40 insertions(+), 12 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 6beb67d29a..5c568c1e88 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -85,6 +85,12 @@ def create_telemetry_process(telemetry_script, args_list):
text=True,
bufsize=1, # Line buffered
)
+
+ # Get and display the connected application name
+ if not print_connected_app(process):
+ return None
+ process.script = telemetry_script # Store script path for reference
+ process.args = args_list # Store args for reference
return process
except FileNotFoundError:
print(f"Error: Python interpreter or script not found", file=sys.stderr)
@@ -102,15 +108,32 @@ def query_telemetry(process, command):
command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
Returns:
- dict: The parsed JSON response with the command wrapper stripped,
+ (process, dict): The process handle, in case of reconnection, and the
+ parsed JSON response with the command wrapper stripped,
or None if there was an error
"""
+ # Handle case where process is None
+ if process is None:
+ return (None, None)
+
# Send the command
process.stdin.write(f"{command}\n")
process.stdin.flush()
- # Read the JSON response
+ # Read the JSON response, reconnecting if necessary
response = process.stdout.readline()
+ while not response:
+ script = process.script
+ args_list = process.args
+ process = None
+ print("Application disconnected, retrying...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(script, args_list)
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+ response = process.stdout.readline()
+
try:
data = json.loads(response)
# When run non-interactively, the response is wrapped with the command
@@ -119,11 +142,11 @@ def query_telemetry(process, command):
# The response should have exactly one key which is the command
if len(data) == 1:
# Extract the value, ignoring the key
- return next(iter(data.values()))
+ return (process, next(iter(data.values())))
else:
- return data
+ return (process, data)
except (json.JSONDecodeError, KeyError):
- return None
+ return (None, None)
def print_connected_app(process):
@@ -132,11 +155,12 @@ def print_connected_app(process):
Args:
process: The subprocess.Popen handle to the telemetry process
"""
- info = query_telemetry(process, "/info")
+ process, info = query_telemetry(process, "/info")
if info and "pid" in info:
app_name = get_app_name(info["pid"])
if app_name:
print(f'Connected to application: "{app_name}"')
+ return process
def expand_shortcuts(process, stat_specs):
@@ -169,7 +193,7 @@ def expand_shortcuts(process, stat_specs):
field = field_map.get(field, field)
# Get list of ethernet devices
- port_list = query_telemetry(process, "/ethdev/list")
+ process, port_list = query_telemetry(process, "/ethdev/list")
if not isinstance(port_list, list):
print(f"Error: Failed to get ethernet device list", file=sys.stderr)
return None
@@ -216,7 +240,7 @@ def validate_stats(process, stat_specs):
return None, None
# Query the stat once to validate it exists and is numeric
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
return None, None
@@ -276,7 +300,9 @@ def monitor_stats(process, args):
current_values = []
total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
+ if not data:
+ continue
current_value = data[field]
current_values.append(current_value)
@@ -387,9 +413,11 @@ def main():
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
-
- # Get and display the connected application name
- print_connected_app(process)
+ if not process:
+ print("Waiting for connection to DPDK application...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(telemetry_script, args_list)
# Monitor the requested statistics
monitor_stats(process, args)
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* Re: [PATCH v2 0/7] Add script for real-time telemetry monitoring
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
` (6 preceding siblings ...)
2026-01-05 17:56 ` [PATCH v2 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
@ 2026-01-14 2:00 ` Stephen Hemminger
7 siblings, 0 replies; 41+ messages in thread
From: Stephen Hemminger @ 2026-01-14 2:00 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On Mon, 5 Jan 2026 17:55:58 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> TL;DR
> ------
>
> For a quick demo, apply patches, run e.g. testpmd and then in a separate
> terminal run:
>
> ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
>
> Output, updated once per second, will be traffic rate per port e.g.:
>
> Connected to application: "dpdk-testpmd"
> Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> 16:29:12 5,213,119 5,214,304 10,427,423
>
>
> Fuller details
> --------------
>
> While we have the dpdk-telemetry.py CLI app for interactive querying of
> telemetry on the commandline, and a telemetry exporter script for
> sending telemetry to external tools for real-time monitoring, we don't
> have an app that can print real-time stats for DPDK apps on the
> terminal. This patchset adds such a script, developed with the help of
> Github copilot to fill a need that I found in my testing. Submitting it
> here in the hopes that others find it of use.
>
> The script acts as a wrapper around the existing dpdk-telemetry.py
> script, and pipes the commands to that script and reads the responses,
> querying it once per second. It takes a number of flag parameters, such
> as the ones above:
> - "-d" for delta values, i.e. PPS rather than total packets
> - "-1" for single-line output, i.e. no scrolling up the screen
> - "-T" to display a total column
>
> Other flag parameters can be seen by looking at the help output.
>
> Beyond the flags, the script also takes a number of positional
> parameters, which refer to specific stats to display. These stats must
> be numeric values, and should take the form of the telemetry command to
> send, followed by a "." and the stat within the result which is to be
> tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
> where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
> part of the result.
>
> However, specifying individual stats can be awkward, so some shortcuts
> are provided too for the common case of monitoring ethernet ports. Any
> positional arg starting with "eth" will be replaced by the set of
> equivalent values for each port, e.g. "eth.imissed" will track the
> imissed value on all ports in use in the app. The ipackets and opackets
> values, as common metrics, are also available as shortened values as
> just "rx" and "tx", so in the example above, "eth.tx" means to track the
> opackets stat for every ethdev port.
>
> Finally, the script also has reconnection support so you can leave it
> running while you start and stop your application in another terminal.
> The watcher will try and reconnect to a running instance every second.
>
> v2:
> - improve reconnection handling, eliminating some crashes seen in testing.
>
> Bruce Richardson (7):
> usertools: add new script to monitor telemetry on terminal
> usertools/telemetry-watcher: add displaying stats
> usertools/telemetry-watcher: add delta and timeout opts
> usertools/telemetry-watcher: add total and one-line opts
> usertools/telemetry-watcher: add thousands separator
> usertools/telemetry-watcher: add eth name shortcuts
> usertools/telemetry-watcher: support reconnection
>
> usertools/dpdk-telemetry-watcher.py | 435 ++++++++++++++++++++++++++++
> usertools/meson.build | 1 +
> 2 files changed, 436 insertions(+)
> create mode 100755 usertools/dpdk-telemetry-watcher.py
>
> --
> 2.51.0
Series-Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Automated patch review reports some things that could be added.
## DPDK Patch Review: Telemetry Watcher (v2, 7 patches)
**Submitter:** Bruce Richardson <bruce.richardson@intel.com>
**Series:** [PATCH v2 1-7/7] usertools: add telemetry watcher script
---
### Overall Summary
This is a well-structured patch series adding a new Python script (`dpdk-telemetry-watcher.py`) to monitor DPDK telemetry statistics on the command line. The series progressively builds up functionality across 7 commits, making the series bisectable.
---
### Patch-by-Patch Review
#### **Patch 1/7: usertools: add new script to monitor telemetry on terminal**
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 53 characters |
| Lowercase after colon | ✅ PASS | |
| Imperative mood | ✅ PASS | "add new script" |
| No trailing period | ✅ PASS | |
| Signed-off-by | ✅ PASS | |
| Body doesn't start with "It" | ✅ PASS | Starts with "The" |
| SPDX license | ✅ PASS | BSD-3-Clause on line 2 (correct for shebang script) |
| Copyright line | ✅ PASS | |
| Blank line after copyright | ✅ PASS | |
**Warning (should fix):**
- **Line 178**: Unnecessary f-string prefix:
```python
print(f"Error: Python interpreter or script not found", file=sys.stderr)
```
Should be:
```python
print("Error: Python interpreter or script not found", file=sys.stderr)
```
#### **Patch 2/7: usertools/telemetry-watcher: add displaying stats**
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 47 characters |
| Signed-off-by | ✅ PASS | |
| Body wrapping | ✅ PASS | Wrapped at 75 chars |
✅ No issues found.
#### **Patch 3/7: usertools/telemetry-watcher: add delta and timeout opts**
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 55 characters |
| Signed-off-by | ✅ PASS | |
✅ No issues found.
#### **Patch 4/7: usertools/telemetry-watcher: add total and one-line opts**
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 56 characters |
| Signed-off-by | ✅ PASS | |
✅ No issues found.
#### **Patch 5/7: usertools/telemetry-watcher: add thousands separator**
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 51 characters |
| Signed-off-by | ✅ PASS | |
✅ No issues found.
#### **Patch 6/7: usertools/telemetry-watcher: add eth name shortcuts**
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 50 characters |
| Signed-off-by | ✅ PASS | |
**Info (consider):**
- Line 1124: f-string without interpolation:
```python
print(f"Error: Failed to get ethernet device list", file=sys.stderr)
```
#### **Patch 7/7: usertools/telemetry-watcher: support reconnection**
| Check | Status | Notes |
|-------|--------|-------|
| Subject ≤60 chars | ✅ PASS | 48 characters |
| Signed-off-by | ✅ PASS | |
**Info (consider):**
- Lines 1243-1244: Dynamically adding attributes to `subprocess.Popen` object:
```python
process.script = telemetry_script
process.args = args_list
```
This works but could be cleaner using a wrapper class or dataclass to hold the process and its metadata together.
---
### Series-Level Issues
#### **Warning: Missing Documentation**
Per AGENTS.md: *"Code and documentation must be updated atomically in same patch"* and *"Documentation must match the code"*
The series adds a new usertool but does not include documentation updates. Consider adding:
- Usage documentation in `doc/guides/tools/` describing the new script
- Entry in the tools index
#### **Warning: Missing Release Notes**
Per AGENTS.md: *"New drivers or subsystems must have release notes"* and *"Current release notes updated for significant changes"*
A new usertools script should be mentioned in `doc/guides/rel_notes/release_XX_XX.rst` under a section like "New Features" or similar.
#### **Info: meson.build List Ordering**
In patch 1/7, the addition to `usertools/meson.build`:
```python
'dpdk-hugepages.py',
'dpdk-rss-flows.py',
'dpdk-telemetry-exporter.py',
+ 'dpdk-telemetry-watcher.py',
```
Per AGENTS.md: *"Lists alphabetically ordered"*. The insertion is correctly placed alphabetically between `dpdk-telemetry-exporter.py` and the end. ✅
---
### Code Quality Observations
**Positive:**
- Clean incremental commits that are individually bisectable
- Good docstrings on functions
- Proper error handling with messages to stderr
- Sensible defaults for optional arguments
- Good use of argparse for CLI handling
**Info (minor suggestions):**
1. The reconnection logic in patch 7/7 could benefit from exponential backoff instead of fixed 1-second retries
2. Consider adding `--help` examples in the epilog of argparse
---
### Summary Table
| Severity | Count | Items |
|----------|-------|-------|
| **Error** | 0 | None |
| **Warning** | 3 | Missing documentation, missing release notes, unnecessary f-string (1/7 line 178) |
| **Info** | 3 | Unnecessary f-string (6/7), dynamic attribute assignment style (7/7), reconnection backoff |
---
### Recommendation
**Acked-by candidate** with minor fixes needed:
1. Remove unnecessary f-string prefix in patch 1/7 line 178
2. Add documentation for the new tool
3. Add release notes entry
The code itself is well-written and the series structure is good. The issues are primarily around the documentation requirements rather than technical problems with the implementation.
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v3 0/7] Add script for real-time telemetry monitoring
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (9 preceding siblings ...)
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
` (8 more replies)
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
11 siblings, 9 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
TL;DR
------
For a quick demo, apply patches, run e.g. testpmd and then in a separate
terminal run:
./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
Output, updated once per second, will be traffic rate per port e.g.:
Connected to application: "dpdk-testpmd"
Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
16:29:12 5,213,119 5,214,304 10,427,423
Fuller details
--------------
While we have the dpdk-telemetry.py CLI app for interactive querying of
telemetry on the commandline, and a telemetry exporter script for
sending telemetry to external tools for real-time monitoring, we don't
have an app that can print real-time stats for DPDK apps on the
terminal. This patchset adds such a script, developed with the help of
Github copilot to fill a need that I found in my testing. Submitting it
here in the hopes that others find it of use.
The script acts as a wrapper around the existing dpdk-telemetry.py
script, and pipes the commands to that script and reads the responses,
querying it once per second. It takes a number of flag parameters, such
as the ones above:
- "-d" for delta values, i.e. PPS rather than total packets
- "-1" for single-line output, i.e. no scrolling up the screen
- "-T" to display a total column
Other flag parameters can be seen by looking at the help output.
Beyond the flags, the script also takes a number of positional
parameters, which refer to specific stats to display. These stats must
be numeric values, and should take the form of the telemetry command to
send, followed by a "." and the stat within the result which is to be
tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
part of the result.
However, specifying individual stats can be awkward, so some shortcuts
are provided too for the common case of monitoring ethernet ports. Any
positional arg starting with "eth" will be replaced by the set of
equivalent values for each port, e.g. "eth.imissed" will track the
imissed value on all ports in use in the app. The ipackets and opackets
values, as common metrics, are also available as shortened values as
just "rx" and "tx", so in the example above, "eth.tx" means to track the
opackets stat for every ethdev port.
Finally, the script also has reconnection support so you can leave it
running while you start and stop your application in another terminal.
The watcher will try and reconnect to a running instance every second.
v3:
Updated following AI review
- removed unnecessary f-string
- added documnentation in guides/tools
- added release note entry
v2:
- improve reconnection handling, eliminating some crashes seen in testing.
Bruce Richardson (7):
usertools: add new script to monitor telemetry on terminal
usertools/telemetry-watcher: add displaying stats
usertools/telemetry-watcher: add delta and timeout opts
usertools/telemetry-watcher: add total and one-line opts
usertools/telemetry-watcher: add thousands separator
usertools/telemetry-watcher: add eth name shortcuts
usertools/telemetry-watcher: support reconnection
doc/guides/rel_notes/release_26_03.rst | 7 +
doc/guides/tools/index.rst | 1 +
doc/guides/tools/telemetrywatcher.rst | 184 +++++++++++
usertools/dpdk-telemetry-watcher.py | 435 +++++++++++++++++++++++++
usertools/meson.build | 1 +
5 files changed, 628 insertions(+)
create mode 100644 doc/guides/tools/telemetrywatcher.rst
create mode 100755 usertools/dpdk-telemetry-watcher.py
--
2.51.0
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v3 1/7] usertools: add new script to monitor telemetry on terminal
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
` (7 subsequent siblings)
8 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
The dpdk-telemetry.py script is useful for getting telemetry
interactively, but sometimes we want to monitor stats over time, so add
a telemetry-watcher script to do so.
Start off such a script with basic arg processing, and connecting to
dpdk-telemetry as a subprocess, so that we can send-receive commands
from it.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/index.rst | 1 +
doc/guides/tools/telemetrywatcher.rst | 56 ++++++++
usertools/dpdk-telemetry-watcher.py | 200 ++++++++++++++++++++++++++
usertools/meson.build | 1 +
4 files changed, 258 insertions(+)
create mode 100644 doc/guides/tools/telemetrywatcher.rst
create mode 100755 usertools/dpdk-telemetry-watcher.py
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index 8ec429ec53..13f75a5bc6 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -14,6 +14,7 @@ DPDK Tools User Guides
pmdinfo
dumpcap
pdump
+ telemetrywatcher
dmaperf
flow-perf
securityperf
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
new file mode 100644
index 0000000000..e813bf0207
--- /dev/null
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -0,0 +1,56 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2026 Intel Corporation
+
+dpdk-telemetry-watcher Application
+===================================
+
+The ``dpdk-telemetry-watcher`` tool is a Data Plane Development Kit (DPDK) utility
+that provides continuous monitoring of DPDK telemetry statistics on the command line.
+It wraps the ``dpdk-telemetry.py`` script to provide real-time statistics display capabilities.
+
+
+Running the Application
+-----------------------
+
+The tool has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-telemetry-watcher.py [options] stat1 stat2 ...
+
+
+Options
+-------
+
+.. program:: dpdk-telemetry-watcher.py
+
+.. option:: -h, --help
+
+ Display usage information and quit
+
+.. option:: -f FILE_PREFIX, --file-prefix FILE_PREFIX
+
+ Provide file-prefix for DPDK runtime directory.
+ Passed to ``dpdk-telemetry.py`` to identify the target DPDK application.
+ Default is ``rte``.
+
+.. option:: -i INSTANCE, --instance INSTANCE
+
+ Provide instance number for DPDK application when multiple applications are running with the same file-prefix.
+ Passed to ``dpdk-telemetry.py`` to identify the target DPDK application instance.
+ Default is ``0``.
+
+.. option:: -l, --list
+
+ List all possible file-prefixes and exit.
+ This is useful to discover which DPDK applications are currently running.
+
+Dependencies
+------------
+
+The tool requires:
+
+* Python 3
+* The ``dpdk-telemetry.py`` script must be available in the same directory
+ or in the system PATH
+* A running DPDK application with telemetry enabled
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
new file mode 100755
index 0000000000..a5ac293d06
--- /dev/null
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation
+
+"""
+Script to monitor DPDK telemetry statistics on the command line.
+Wraps dpdk-telemetry.py to provide continuous monitoring capabilities.
+"""
+
+import argparse
+import subprocess
+import sys
+import os
+import shutil
+import errno
+import json
+
+
+def get_app_name(pid):
+ """return the app name for a given PID, for printing"""
+ proc_cmdline = os.path.join("/proc", str(pid), "cmdline")
+ try:
+ with open(proc_cmdline) as f:
+ argv0 = f.read(1024).split("\0")[0]
+ return os.path.basename(argv0)
+ except IOError as e:
+ # ignore file not found errors
+ if e.errno != errno.ENOENT:
+ raise
+ return None
+
+
+def find_telemetry_script():
+ """Find the dpdk-telemetry.py script in the script directory or PATH.
+
+ Returns:
+ str: Path to the dpdk-telemetry.py script
+
+ Exits:
+ If the script cannot be found
+ """
+ # First, try to find it in the same directory as this script
+ script_dir = os.path.dirname(os.path.abspath(__file__))
+ telemetry_script = os.path.join(script_dir, "dpdk-telemetry.py")
+
+ # If not found locally, check if it's in PATH
+ if not os.path.exists(telemetry_script):
+ telemetry_in_path = shutil.which("dpdk-telemetry.py")
+ if telemetry_in_path:
+ telemetry_script = telemetry_in_path
+ else:
+ print(
+ "Error: dpdk-telemetry.py not found in script directory or PATH",
+ file=sys.stderr,
+ )
+ sys.exit(1)
+
+ return telemetry_script
+
+
+def create_telemetry_process(telemetry_script, args_list):
+ """Create a subprocess for dpdk-telemetry.py with pipes.
+
+ Args:
+ telemetry_script: Path to the dpdk-telemetry.py script
+ args_list: List of arguments to pass to the script
+
+ Returns:
+ subprocess.Popen: Process handle with stdin/stdout/stderr pipes
+
+ Exits:
+ If the process cannot be created
+ """
+ # Build the command
+ cmd = [sys.executable, telemetry_script] + args_list
+
+ try:
+ process = subprocess.Popen(
+ cmd,
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ text=True,
+ bufsize=1, # Line buffered
+ )
+ return process
+ except FileNotFoundError:
+ print("Error: Python interpreter or script not found", file=sys.stderr)
+ sys.exit(1)
+ except Exception as e:
+ print(f"Error running dpdk-telemetry.py: {e}", file=sys.stderr)
+ sys.exit(1)
+
+
+def query_telemetry(process, command):
+ """Send a telemetry command and return the parsed JSON response.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
+
+ Returns:
+ dict: The parsed JSON response with the command wrapper stripped,
+ or None if there was an error
+ """
+ # Send the command
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+
+ # Read the JSON response
+ response = process.stdout.readline()
+ try:
+ data = json.loads(response)
+ # When run non-interactively, the response is wrapped with the command
+ # e.g., {"/info": {"version": ..., "pid": ...}}
+ # or {"/ethdev/stats,0": {...}}
+ # The response should have exactly one key which is the command
+ if len(data) == 1:
+ # Extract the value, ignoring the key
+ return next(iter(data.values()))
+ else:
+ return data
+ except (json.JSONDecodeError, KeyError):
+ return None
+
+
+def print_connected_app(process):
+ """Query and print the name of the connected DPDK application.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ """
+ info = query_telemetry(process, "/info")
+ if info and "pid" in info:
+ app_name = get_app_name(info["pid"])
+ if app_name:
+ print(f'Connected to application: "{app_name}"')
+
+
+def main():
+ """Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+
+ # Parse command line arguments - matching dpdk-telemetry.py parameters
+ parser = argparse.ArgumentParser(
+ description="Monitor DPDK telemetry statistics on the command line"
+ )
+ parser.add_argument(
+ "-f",
+ "--file-prefix",
+ default="rte",
+ help="Provide file-prefix for DPDK runtime directory",
+ )
+ parser.add_argument(
+ "-i",
+ "--instance",
+ default=0,
+ type=int,
+ help="Provide instance number for DPDK application",
+ )
+ parser.add_argument(
+ "-l",
+ "--list",
+ action="store_true",
+ default=False,
+ help="List all possible file-prefixes and exit",
+ )
+
+ args = parser.parse_args()
+
+ # Find the dpdk-telemetry.py script
+ telemetry_script = find_telemetry_script()
+
+ # Build arguments list
+ args_list = ["-f", args.file_prefix, "-i", str(args.instance)]
+
+ if args.list:
+ args_list.append("-l")
+ # For --list, just run the command directly without pipes
+ cmd = [sys.executable, telemetry_script] + args_list
+ return subprocess.run(cmd).returncode
+
+ # Run dpdk-telemetry.py with pipes for stdin and stdout
+ process = create_telemetry_process(telemetry_script, args_list)
+
+ # Get and display the connected application name
+ print_connected_app(process)
+
+ # TODO: Add monitoring logic here
+
+ # Clean up
+ process.stdin.close()
+ process.stdout.close()
+ process.stderr.close()
+ process.wait()
+
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/usertools/meson.build b/usertools/meson.build
index eb48e2f440..114d0a65b4 100644
--- a/usertools/meson.build
+++ b/usertools/meson.build
@@ -12,6 +12,7 @@ install_data([
'dpdk-hugepages.py',
'dpdk-rss-flows.py',
'dpdk-telemetry-exporter.py',
+ 'dpdk-telemetry-watcher.py',
],
install_dir: 'bin')
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v3 2/7] usertools/telemetry-watcher: add displaying stats
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
` (6 subsequent siblings)
8 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Add support for printing out particular stats once every second. The
stats to be queried are given on the commandline, in the format of
<command>.<result-field>, for example /ethdev/stats,0.ipackets.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 51 +++++++++++++
usertools/dpdk-telemetry-watcher.py | 101 +++++++++++++++++++++++++-
2 files changed, 151 insertions(+), 1 deletion(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index e813bf0207..94f9e31721 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -45,6 +45,57 @@ Options
List all possible file-prefixes and exit.
This is useful to discover which DPDK applications are currently running.
+.. option:: stat
+
+ Statistics to monitor in format ``command.field``.
+ Multiple statistics can be specified and will be displayed in columns.
+ See the `Statistics Format`_ section below for details on specifying statistics.
+
+
+Statistics Format
+-----------------
+
+Statistics are specified in the format ``command.field`` where:
+
+* ``command`` is a telemetry command (e.g., ``/ethdev/stats,0``)
+* ``field`` is a field name from the command's JSON response (e.g., ``ipackets``)
+
+To discover available commands and fields:
+
+1. Use ``dpdk-telemetry.py`` interactively to explore available commands
+2. Use the ``/`` command to list all available telemetry endpoints
+3. Query specific commands to see their response format
+
+Example telemetry commands:
+
+* ``/ethdev/list`` - List all ethernet devices
+* ``/ethdev/stats,N`` - Get statistics for ethernet device N
+* ``/ethdev/xstats,N`` - Get extended statistics for ethernet device N
+* ``/eal/mempool_list`` - List all mempools
+* ``/mempool/info,N`` - Get information about mempool N
+
+See `Examples`_ section for usage examples based on the results of these telemetry commands.
+
+Examples
+--------
+
+Monitor received packets on ethernet device 0::
+
+ dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets
+
+Monitor received and transmitted packets on device 0::
+
+ dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets /ethdev/stats,0.opackets
+
+Monitor a DPDK application with a custom file-prefix::
+
+ dpdk-telemetry-watcher.py -f myapp /ethdev/stats,0.ipackets
+
+List all running DPDK applications::
+
+ dpdk-telemetry-watcher.py -l
+
+
Dependencies
------------
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index a5ac293d06..dad4e60475 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -14,6 +14,7 @@
import shutil
import errno
import json
+import time
def get_app_name(pid):
@@ -137,6 +138,91 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def validate_stats(process, stat_specs):
+ """Validate stat specifications and check that fields are numeric.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+
+ Returns:
+ List of tuples (spec, command, field) for valid specs, or None on error
+ """
+ parsed_specs = []
+ for spec in stat_specs:
+ # Parse the stat specification
+ if "." not in spec:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ command, field = spec.rsplit(".", 1)
+ if not command or not field:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ # Query the stat once to validate it exists and is numeric
+ data = query_telemetry(process, command)
+ if not isinstance(data, dict):
+ print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
+ return None
+ if field not in data:
+ print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
+ return None
+ value = data[field]
+ if not isinstance(value, (int, float)):
+ print(
+ f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
+ file=sys.stderr,
+ )
+ return None
+
+ parsed_specs.append((spec, command, field))
+
+ return parsed_specs
+
+
+def monitor_stats(process, stat_specs):
+ """Monitor and display statistics in columns.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+ """
+ # Validate all stat specifications
+ parsed_specs = validate_stats(process, stat_specs)
+ if not parsed_specs:
+ return
+
+ # Print header
+ header = "Time".ljust(10)
+ for spec, _, _ in parsed_specs:
+ header += spec.rjust(25)
+ print(header)
+
+ # Monitor loop - once per second
+ try:
+ while True:
+ timestamp = time.strftime("%H:%M:%S")
+ row = timestamp.ljust(10)
+
+ for spec, command, field in parsed_specs:
+ data = query_telemetry(process, command)
+ row += str(data[field]).rjust(25)
+
+ print(row)
+ time.sleep(1)
+ except KeyboardInterrupt:
+ print("\nMonitoring stopped")
+
+
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
@@ -164,6 +250,11 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "stats",
+ nargs="*",
+ help="Statistics to monitor in format 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ )
args = parser.parse_args()
@@ -179,13 +270,21 @@ def main():
cmd = [sys.executable, telemetry_script] + args_list
return subprocess.run(cmd).returncode
+ # Check if stats were provided
+ if not args.stats:
+ print("Error: No statistics to monitor specified", file=sys.stderr)
+ print("Usage: dpdk-telemetry-watcher.py [options] stat1 stat2 ...", file=sys.stderr)
+ print("Example: dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets", file=sys.stderr)
+ return 1
+
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
# Get and display the connected application name
print_connected_app(process)
- # TODO: Add monitoring logic here
+ # Monitor the requested statistics
+ monitor_stats(process, args.stats)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v3 3/7] usertools/telemetry-watcher: add delta and timeout opts
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
` (5 subsequent siblings)
8 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Add options to the script to time out and quit after a certain number of
seconds. Also, add an option to display delta instead of absolute
values, allowing easy display of e.g. PPS.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 11 +++++
usertools/dpdk-telemetry-watcher.py | 64 ++++++++++++++++++++-------
2 files changed, 59 insertions(+), 16 deletions(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index 94f9e31721..137a4803b2 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -45,6 +45,17 @@ Options
List all possible file-prefixes and exit.
This is useful to discover which DPDK applications are currently running.
+.. option:: -t TIMEOUT, --timeout TIMEOUT
+
+ Number of iterations to run before stopping.
+ If not specified, the tool runs indefinitely until interrupted with Ctrl+C.
+
+.. option:: -d, --delta
+
+ Display delta (incremental) values instead of absolute values.
+ This shows the change in statistics since the last iteration,
+ which is useful for monitoring rates of change.
+
.. option:: stat
Statistics to monitor in format ``command.field``.
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index dad4e60475..5f4aa05431 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -146,9 +146,13 @@ def validate_stats(process, stat_specs):
stat_specs: List of stat specifications in format "command.field"
Returns:
- List of tuples (spec, command, field) for valid specs, or None on error
+ Tuple of (parsed_specs, initial_values) where:
+ parsed_specs: List of tuples (spec, command, field) for valid specs
+ initial_values: List of initial values for each stat
+ Returns (None, None) on error
"""
parsed_specs = []
+ initial_values = []
for spec in stat_specs:
# Parse the stat specification
if "." not in spec:
@@ -157,7 +161,7 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
command, field = spec.rsplit(".", 1)
if not command or not field:
@@ -166,38 +170,39 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
# Query the stat once to validate it exists and is numeric
data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
- return None
+ return None, None
if field not in data:
print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
- return None
+ return None, None
value = data[field]
if not isinstance(value, (int, float)):
print(
f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
file=sys.stderr,
)
- return None
+ return None, None
parsed_specs.append((spec, command, field))
+ initial_values.append(value)
- return parsed_specs
+ return parsed_specs, initial_values
-def monitor_stats(process, stat_specs):
+def monitor_stats(process, args):
"""Monitor and display statistics in columns.
Args:
process: The subprocess.Popen handle to the telemetry process
- stat_specs: List of stat specifications in format "command.field"
+ args: Parsed command line arguments
"""
- # Validate all stat specifications
- parsed_specs = validate_stats(process, stat_specs)
+ # Validate all stat specifications and get initial values
+ parsed_specs, prev_values = validate_stats(process, args.stats)
if not parsed_specs:
return
@@ -208,17 +213,30 @@ def monitor_stats(process, stat_specs):
print(header)
# Monitor loop - once per second
+ count = 0
try:
- while True:
+ while args.timeout is None or count < args.timeout:
+ time.sleep(1)
+ count += 1
+
timestamp = time.strftime("%H:%M:%S")
row = timestamp.ljust(10)
- for spec, command, field in parsed_specs:
+ current_values = []
+ for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
- row += str(data[field]).rjust(25)
+ current_value = data[field]
+ current_values.append(current_value)
+
+ if args.delta:
+ display_value = current_value - prev_values[i]
+ else:
+ display_value = current_value
+
+ row += str(display_value).rjust(25)
print(row)
- time.sleep(1)
+ prev_values = current_values
except KeyboardInterrupt:
print("\nMonitoring stopped")
@@ -250,6 +268,20 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "-t",
+ "--timeout",
+ type=int,
+ default=None,
+ help="Number of iterations to run before stopping (default: run indefinitely)",
+ )
+ parser.add_argument(
+ "-d",
+ "--delta",
+ action="store_true",
+ default=False,
+ help="Display delta values instead of absolute values",
+ )
parser.add_argument(
"stats",
nargs="*",
@@ -284,7 +316,7 @@ def main():
print_connected_app(process)
# Monitor the requested statistics
- monitor_stats(process, args.stats)
+ monitor_stats(process, args)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v3 4/7] usertools/telemetry-watcher: add total and one-line opts
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
` (2 preceding siblings ...)
2026-01-15 19:03 ` [PATCH v3 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
` (4 subsequent siblings)
8 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Add options to the script to print out totals at the end of each line,
and to print each line replacing the previous one, saving the output
scrolling up the screen constantly.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 27 +++++++++++++++++++++++++++
usertools/dpdk-telemetry-watcher.py | 27 ++++++++++++++++++++++++++-
2 files changed, 53 insertions(+), 1 deletion(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index 137a4803b2..3d22818260 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -56,6 +56,15 @@ Options
This shows the change in statistics since the last iteration,
which is useful for monitoring rates of change.
+.. option:: -T, --total
+
+ Display a total column at the end of each row that sums all monitored statistics.
+
+.. option:: -1, --single-line
+
+ Display output on a single line, replacing the previous output.
+ This is useful for reducing scrolling and keeping the display compact.
+
.. option:: stat
Statistics to monitor in format ``command.field``.
@@ -107,6 +116,24 @@ List all running DPDK applications::
dpdk-telemetry-watcher.py -l
+Output Format
+-------------
+
+The tool displays statistics in a tabular format with:
+
+* **Time column** - Current timestamp (HH:MM:SS)
+* **Statistics columns** - One column per specified statistic
+* **Total column** - Optional sum of all statistics (when ``-T`` is used)
+
+Values are formatted with locale-specific number formatting (e.g., comma separators).
+
+When ``--delta`` mode is enabled, the tool displays the change in each statistic
+since the last iteration, which typically represents the rate per second.
+
+When ``--single-line`` mode is enabled, each new output line replaces the previous one,
+similar to tools like ``top``.
+
+
Dependencies
------------
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 5f4aa05431..e4cd292515 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -210,10 +210,13 @@ def monitor_stats(process, args):
header = "Time".ljust(10)
for spec, _, _ in parsed_specs:
header += spec.rjust(25)
+ if args.total:
+ header += "Total".rjust(25)
print(header)
# Monitor loop - once per second
count = 0
+ line_ending = "\r" if args.single_line else "\n"
try:
while args.timeout is None or count < args.timeout:
time.sleep(1)
@@ -223,6 +226,7 @@ def monitor_stats(process, args):
row = timestamp.ljust(10)
current_values = []
+ total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
current_value = data[field]
@@ -233,11 +237,17 @@ def monitor_stats(process, args):
else:
display_value = current_value
+ total += display_value
row += str(display_value).rjust(25)
- print(row)
+ if args.total:
+ row += str(total).rjust(25)
+
+ print(row, end=line_ending, flush=True)
prev_values = current_values
except KeyboardInterrupt:
+ if args.single_line:
+ print() # Add newline before exit message
print("\nMonitoring stopped")
@@ -282,6 +292,21 @@ def main():
default=False,
help="Display delta values instead of absolute values",
)
+ parser.add_argument(
+ "-T",
+ "--total",
+ action="store_true",
+ default=False,
+ help="Display a total column at the end of each row",
+ )
+ parser.add_argument(
+ "-1",
+ "--single-line",
+ action="store_true",
+ default=False,
+ dest="single_line",
+ help="Display output on a single line, replacing the previous output",
+ )
parser.add_argument(
"stats",
nargs="*",
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v3 5/7] usertools/telemetry-watcher: add thousands separator
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
` (3 preceding siblings ...)
2026-01-15 19:03 ` [PATCH v3 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
` (3 subsequent siblings)
8 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
When printing large values of packets, generally in the millions, things
are far more readable with thousands and millions separators.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
usertools/dpdk-telemetry-watcher.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index e4cd292515..7ec7267a38 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -15,6 +15,7 @@
import errno
import json
import time
+import locale
def get_app_name(pid):
@@ -238,10 +239,10 @@ def monitor_stats(process, args):
display_value = current_value
total += display_value
- row += str(display_value).rjust(25)
+ row += f"{display_value:n}".rjust(25)
if args.total:
- row += str(total).rjust(25)
+ row += f"{total:n}".rjust(25)
print(row, end=line_ending, flush=True)
prev_values = current_values
@@ -254,6 +255,9 @@ def monitor_stats(process, args):
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+ # Set locale for number formatting
+ locale.setlocale(locale.LC_ALL, "")
+
# Parse command line arguments - matching dpdk-telemetry.py parameters
parser = argparse.ArgumentParser(
description="Monitor DPDK telemetry statistics on the command line"
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v3 6/7] usertools/telemetry-watcher: add eth name shortcuts
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
` (4 preceding siblings ...)
2026-01-15 19:03 ` [PATCH v3 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
` (2 subsequent siblings)
8 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Since an expected main use of the script is to monitor ethdev packet
stats, provide a shortened form of parameters to make it easier to
monitor all ports on the system. Any stat starting with "eth." is taken
not as a direct command, but instead as a shortcut for getting the stats
for all ports on the system. For example: eth.ibytes shows the byte
counts for all ports.
Beyond that, provide a shortcut for ipackets and opackets as just rx and
tx respectively. Therefore, to monitor the output rate of an app, one
can use "dpdk-telemetry-watcher -dT eth.tx"
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 33 ++++++++++++++++++
usertools/dpdk-telemetry-watcher.py | 49 ++++++++++++++++++++++++++-
2 files changed, 81 insertions(+), 1 deletion(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index 3d22818260..99e51881ff 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -96,6 +96,19 @@ Example telemetry commands:
See `Examples`_ section for usage examples based on the results of these telemetry commands.
+Shortcuts
+---------
+
+The tool provides convenient shortcuts for common statistics:
+
+* ``eth.rx`` - Expands to ``/ethdev/stats,N.ipackets`` for all ethernet devices
+* ``eth.tx`` - Expands to ``/ethdev/stats,N.opackets`` for all ethernet devices
+* ``eth.FIELD`` - Expands to ``/ethdev/stats,N.FIELD`` for all ethernet devices
+
+These shortcuts automatically detect all available ethernet devices
+and create a column for each one.
+
+
Examples
--------
@@ -107,10 +120,30 @@ Monitor received and transmitted packets on device 0::
dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets /ethdev/stats,0.opackets
+Monitor received packets on all ethernet devices using shortcut::
+
+ dpdk-telemetry-watcher.py eth.rx
+
+Monitor packet deltas (rates) for device 0::
+
+ dpdk-telemetry-watcher.py -d /ethdev/stats,0.ipackets /ethdev/stats,0.opackets
+
+Monitor with a total column showing aggregate traffic::
+
+ dpdk-telemetry-watcher.py -d -T eth.rx eth.tx
+
+Monitor for a specific duration (60 iterations = 60 seconds)::
+
+ dpdk-telemetry-watcher.py -t 60 /ethdev/stats,0.ipackets
+
Monitor a DPDK application with a custom file-prefix::
dpdk-telemetry-watcher.py -f myapp /ethdev/stats,0.ipackets
+Monitor in single-line mode (no scrolling)::
+
+ dpdk-telemetry-watcher.py -1 -d eth.rx eth.tx
+
List all running DPDK applications::
dpdk-telemetry-watcher.py -l
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 7ec7267a38..9685ffa01f 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -139,6 +139,48 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def expand_shortcuts(process, stat_specs):
+ """Expand special shortcuts like eth.rx and eth.tx into actual stat specifications.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications, possibly including shortcuts
+
+ Returns:
+ List of expanded stat specifications
+ """
+ expanded = []
+ for spec in stat_specs:
+ if not spec.startswith("eth."):
+ expanded.append(spec)
+ continue
+
+ # Extract the field name after "eth."
+ field = spec[4:] # Remove "eth." prefix
+ if not field:
+ print(f"Error: Invalid shortcut '{spec}' - missing field name", file=sys.stderr)
+ return None
+
+ # Map common shortcuts to actual field names
+ field_map = {
+ "rx": "ipackets",
+ "tx": "opackets",
+ }
+ field = field_map.get(field, field)
+
+ # Get list of ethernet devices
+ port_list = query_telemetry(process, "/ethdev/list")
+ if not isinstance(port_list, list):
+ print(f"Error: Failed to get ethernet device list", file=sys.stderr)
+ return None
+
+ # Create stat specs for each port
+ for port in port_list:
+ expanded.append(f"/ethdev/stats,{port}.{field}")
+
+ return expanded
+
+
def validate_stats(process, stat_specs):
"""Validate stat specifications and check that fields are numeric.
@@ -202,8 +244,13 @@ def monitor_stats(process, args):
process: The subprocess.Popen handle to the telemetry process
args: Parsed command line arguments
"""
+ # Expand any shortcuts like eth-rx, eth-tx
+ expanded_stats = expand_shortcuts(process, args.stats)
+ if not expanded_stats:
+ return
+
# Validate all stat specifications and get initial values
- parsed_specs, prev_values = validate_stats(process, args.stats)
+ parsed_specs, prev_values = validate_stats(process, expanded_stats)
if not parsed_specs:
return
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v3 7/7] usertools/telemetry-watcher: support reconnection
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
` (5 preceding siblings ...)
2026-01-15 19:03 ` [PATCH v3 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
@ 2026-01-15 19:03 ` Bruce Richardson
2026-01-16 6:53 ` [PATCH v3 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
2026-01-16 6:57 ` Stephen Hemminger
8 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-01-15 19:03 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Allow the watcher binary to run even when there is no DPDK process
running. In that case, wait for a suitable process to start and begin
monitoring then. In case of disconnection, keep trying to reconnect and
resume once reconnection succeeds.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/rel_notes/release_26_03.rst | 7 ++++
doc/guides/tools/telemetrywatcher.rst | 6 +++
usertools/dpdk-telemetry-watcher.py | 52 ++++++++++++++++++++------
3 files changed, 53 insertions(+), 12 deletions(-)
diff --git a/doc/guides/rel_notes/release_26_03.rst b/doc/guides/rel_notes/release_26_03.rst
index 15dabee7a1..a418491daf 100644
--- a/doc/guides/rel_notes/release_26_03.rst
+++ b/doc/guides/rel_notes/release_26_03.rst
@@ -55,6 +55,13 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added Script for Real-time Telemetry Monitoring.**
+
+ Introduced the `dpdk-telemetry-watcher.py` script, enabling users to monitor
+ real-time telemetry statistics from running DPDK applications.
+ The tool supports customizable display options, including delta values,
+ total statistics, and single-line output for compact monitoring.
+
Removed Items
-------------
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index 99e51881ff..070a93223f 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -12,6 +12,12 @@ It wraps the ``dpdk-telemetry.py`` script to provide real-time statistics displa
Running the Application
-----------------------
+The watcher tool can be run at any time, whether or not a DPDK application is currently running.
+When a DPDK application with telemetry enabled starts
+(assuming correct file-prefix and instance are specified),
+the watcher will automatically connect and begin displaying the requested statistics.
+If the DPDK application stops, the watcher will attempt to reconnect when the application restarts.
+
The tool has a number of command line options:
.. code-block:: console
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 9685ffa01f..bd5dc92fe6 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -85,6 +85,12 @@ def create_telemetry_process(telemetry_script, args_list):
text=True,
bufsize=1, # Line buffered
)
+
+ # Get and display the connected application name
+ if not print_connected_app(process):
+ return None
+ process.script = telemetry_script # Store script path for reference
+ process.args = args_list # Store args for reference
return process
except FileNotFoundError:
print("Error: Python interpreter or script not found", file=sys.stderr)
@@ -102,15 +108,32 @@ def query_telemetry(process, command):
command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
Returns:
- dict: The parsed JSON response with the command wrapper stripped,
+ (process, dict): The process handle, in case of reconnection, and the
+ parsed JSON response with the command wrapper stripped,
or None if there was an error
"""
+ # Handle case where process is None
+ if process is None:
+ return (None, None)
+
# Send the command
process.stdin.write(f"{command}\n")
process.stdin.flush()
- # Read the JSON response
+ # Read the JSON response, reconnecting if necessary
response = process.stdout.readline()
+ while not response:
+ script = process.script
+ args_list = process.args
+ process = None
+ print("Application disconnected, retrying...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(script, args_list)
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+ response = process.stdout.readline()
+
try:
data = json.loads(response)
# When run non-interactively, the response is wrapped with the command
@@ -119,11 +142,11 @@ def query_telemetry(process, command):
# The response should have exactly one key which is the command
if len(data) == 1:
# Extract the value, ignoring the key
- return next(iter(data.values()))
+ return (process, next(iter(data.values())))
else:
- return data
+ return (process, data)
except (json.JSONDecodeError, KeyError):
- return None
+ return (None, None)
def print_connected_app(process):
@@ -132,11 +155,12 @@ def print_connected_app(process):
Args:
process: The subprocess.Popen handle to the telemetry process
"""
- info = query_telemetry(process, "/info")
+ process, info = query_telemetry(process, "/info")
if info and "pid" in info:
app_name = get_app_name(info["pid"])
if app_name:
print(f'Connected to application: "{app_name}"')
+ return process
def expand_shortcuts(process, stat_specs):
@@ -169,7 +193,7 @@ def expand_shortcuts(process, stat_specs):
field = field_map.get(field, field)
# Get list of ethernet devices
- port_list = query_telemetry(process, "/ethdev/list")
+ process, port_list = query_telemetry(process, "/ethdev/list")
if not isinstance(port_list, list):
print(f"Error: Failed to get ethernet device list", file=sys.stderr)
return None
@@ -216,7 +240,7 @@ def validate_stats(process, stat_specs):
return None, None
# Query the stat once to validate it exists and is numeric
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
return None, None
@@ -276,7 +300,9 @@ def monitor_stats(process, args):
current_values = []
total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
+ if not data:
+ continue
current_value = data[field]
current_values.append(current_value)
@@ -387,9 +413,11 @@ def main():
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
-
- # Get and display the connected application name
- print_connected_app(process)
+ if not process:
+ print("Waiting for connection to DPDK application...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(telemetry_script, args_list)
# Monitor the requested statistics
monitor_stats(process, args)
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* Re: [PATCH v3 0/7] Add script for real-time telemetry monitoring
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
` (6 preceding siblings ...)
2026-01-15 19:03 ` [PATCH v3 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
@ 2026-01-16 6:53 ` Stephen Hemminger
2026-01-16 6:57 ` Stephen Hemminger
8 siblings, 0 replies; 41+ messages in thread
From: Stephen Hemminger @ 2026-01-16 6:53 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On Thu, 15 Jan 2026 19:03:24 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> TL;DR
> ------
>
> For a quick demo, apply patches, run e.g. testpmd and then in a separate
> terminal run:
>
> ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
>
> Output, updated once per second, will be traffic rate per port e.g.:
>
> Connected to application: "dpdk-testpmd"
> Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> 16:29:12 5,213,119 5,214,304 10,427,423
>
>
> Fuller details
> --------------
>
> While we have the dpdk-telemetry.py CLI app for interactive querying of
> telemetry on the commandline, and a telemetry exporter script for
> sending telemetry to external tools for real-time monitoring, we don't
> have an app that can print real-time stats for DPDK apps on the
> terminal. This patchset adds such a script, developed with the help of
> Github copilot to fill a need that I found in my testing. Submitting it
> here in the hopes that others find it of use.
>
> The script acts as a wrapper around the existing dpdk-telemetry.py
> script, and pipes the commands to that script and reads the responses,
> querying it once per second. It takes a number of flag parameters, such
> as the ones above:
> - "-d" for delta values, i.e. PPS rather than total packets
> - "-1" for single-line output, i.e. no scrolling up the screen
> - "-T" to display a total column
>
> Other flag parameters can be seen by looking at the help output.
>
> Beyond the flags, the script also takes a number of positional
> parameters, which refer to specific stats to display. These stats must
> be numeric values, and should take the form of the telemetry command to
> send, followed by a "." and the stat within the result which is to be
> tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
> where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
> part of the result.
>
> However, specifying individual stats can be awkward, so some shortcuts
> are provided too for the common case of monitoring ethernet ports. Any
> positional arg starting with "eth" will be replaced by the set of
> equivalent values for each port, e.g. "eth.imissed" will track the
> imissed value on all ports in use in the app. The ipackets and opackets
> values, as common metrics, are also available as shortened values as
> just "rx" and "tx", so in the example above, "eth.tx" means to track the
> opackets stat for every ethdev port.
>
> Finally, the script also has reconnection support so you can leave it
> running while you start and stop your application in another terminal.
> The watcher will try and reconnect to a running instance every second.
>
> v3:
> Updated following AI review
> - removed unnecessary f-string
> - added documnentation in guides/tools
> - added release note entry
>
> v2:
> - improve reconnection handling, eliminating some crashes seen in testing.
>
>
> Bruce Richardson (7):
> usertools: add new script to monitor telemetry on terminal
> usertools/telemetry-watcher: add displaying stats
> usertools/telemetry-watcher: add delta and timeout opts
> usertools/telemetry-watcher: add total and one-line opts
> usertools/telemetry-watcher: add thousands separator
> usertools/telemetry-watcher: add eth name shortcuts
> usertools/telemetry-watcher: support reconnection
>
> doc/guides/rel_notes/release_26_03.rst | 7 +
> doc/guides/tools/index.rst | 1 +
> doc/guides/tools/telemetrywatcher.rst | 184 +++++++++++
> usertools/dpdk-telemetry-watcher.py | 435 +++++++++++++++++++++++++
> usertools/meson.build | 1 +
> 5 files changed, 628 insertions(+)
> create mode 100644 doc/guides/tools/telemetrywatcher.rst
> create mode 100755 usertools/dpdk-telemetry-watcher.py
Automated AI review of this patch series.
Yes AI seems to be pickier about python code (wonder what it is trained on?)
---
## DPDK Patch Review: Telemetry Watcher (v3, 7 patches)
**Series**: `[PATCH v3 1/7]` through `[PATCH v3 7/7]`
**Author**: Bruce Richardson <bruce.richardson@intel.com>
**Reviewed against**: AGENTS.md review criteria
---
### Summary
| Category | Status |
|----------|--------|
| Commit Messages | ✓ Pass |
| SPDX/License | ⚠ Warning (minor) |
| Code Style | ⚠ Warning (minor) |
| Documentation | ✓ Pass |
| Release Notes | ✓ Pass |
| Build Integration | ✓ Pass |
---
### Commit Message Review
**All 7 patches pass commit message requirements:**
| Patch | Subject | Length | Format |
|-------|---------|--------|--------|
| 1/7 | `usertools: add new script to monitor telemetry on terminal` | 52 chars | ✓ |
| 2/7 | `usertools/telemetry-watcher: add displaying stats` | 47 chars | ✓ |
| 3/7 | `usertools/telemetry-watcher: add delta and timeout opts` | 53 chars | ✓ |
| 4/7 | `usertools/telemetry-watcher: add total and one-line opts` | 55 chars | ✓ |
| 5/7 | `usertools/telemetry-watcher: add thousands separator` | 51 chars | ✓ |
| 6/7 | `usertools/telemetry-watcher: add eth name shortcuts` | 51 chars | ✓ |
| 7/7 | `usertools/telemetry-watcher: support reconnection` | 49 chars | ✓ |
- ✓ All subjects ≤60 characters
- ✓ Lowercase after colon (except acronyms)
- ✓ Imperative mood
- ✓ No trailing periods
- ✓ `Signed-off-by:` present with valid name/email
- ✓ `Acked-by:` present (your ack is included)
- ✓ Body lines appear wrapped at 75 characters
- ✓ Body does not start with "It"
---
### License/SPDX Review
**Python script** (`dpdk-telemetry-watcher.py`):
```python
#!/usr/bin/env python3
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2025 Intel Corporation
```
✓ SPDX on line 2 (correct for shebang scripts)
✓ Copyright follows immediately
✓ Blank line before code
✓ BSD-3-Clause (appropriate for usertools)
**Documentation** (`telemetrywatcher.rst`):
```rst
.. SPDX-License-Identifier: BSD-3-Clause
Copyright(c) 2026 Intel Corporation
```
✓ SPDX on first line
✓ Copyright follows
⚠ **Warning**: Copyright year mismatch — Python script says **2025**, documentation says **2026**. Should be consistent (likely both should be 2025 or 2026 depending on when work started).
---
### Code Style Review
**Patch 5** (thousands separator):
⚠ **Warning**: The patch adds `import locale` and uses `f"{display_value:n}"` for locale-aware number formatting, but **does not call `locale.setlocale(locale.LC_ALL, '')`** to activate locale-aware formatting. Without this call, the `:n` format specifier may not produce thousands separators on all systems.
**Suggested fix** — add to `main()`:
```python
import locale
locale.setlocale(locale.LC_ALL, '')
```
**Patch 6** (eth shortcuts):
⚠ **Info** (minor): Unnecessary f-string at line 1401:
```python
print(f"Error: Failed to get ethernet device list", file=sys.stderr)
```
Should be:
```python
print("Error: Failed to get ethernet device list", file=sys.stderr)
```
**General Python style**:
- ✓ Good docstrings throughout
- ✓ Consistent formatting (appears black-compliant)
- ✓ Line lengths within 100 characters
- ✓ Good error handling with informative messages
---
### Documentation Review
✓ **Comprehensive documentation** in `telemetrywatcher.rst`:
- Application description
- Command-line options documented
- Statistics format explained
- Shortcuts documented
- Multiple usage examples
- Dependencies listed
- Output format described
✓ **Documentation matches code behavior**
✓ **Index updated** in `doc/guides/tools/index.rst`
---
### Release Notes Review
✓ **Release notes added** in patch 7/7 (`doc/guides/rel_notes/release_26_03.rst`):
```rst
* **Added Script for Real-time Telemetry Monitoring.**
Introduced the `dpdk-telemetry-watcher.py` script, enabling users to monitor
real-time telemetry statistics from running DPDK applications.
The tool supports customizable display options, including delta values,
total statistics, and single-line output for compact monitoring.
```
✓ Appropriate placement — release notes in final patch when feature is complete
---
### Build Integration Review
✓ **meson.build updated** (patch 1/7):
```python
install_data([
...
'dpdk-telemetry-watcher.py',
],
install_dir: 'bin')
```
---
### Structural Review
✓ Each patch appears independently functional (incremental development)
✓ Code and documentation updated together in relevant patches
✓ No forward dependencies within patchset
**Note**: No automated tests added for this usertools script. This is common practice for Python usertools in DPDK, though the `black` formatter should be run on the code.
---
### Issues Summary
| Severity | Issue | Location |
|----------|-------|----------|
| **Warning** | Copyright year mismatch (2025 vs 2026) | Python script vs RST doc |
| **Warning** | Missing `locale.setlocale()` call for `:n` format | Patch 5/7 |
| **Info** | Unnecessary f-string | Patch 6/7, line with "Failed to get ethernet device list" |
---
### Recommendation
**Acceptable with minor fixes**. The patch series is well-structured, properly documented, and follows DPDK conventions. The two warnings should be addressed:
1. Harmonize copyright years
2. Add `locale.setlocale(locale.LC_ALL, '')` to ensure thousands separators work correctly
^ permalink raw reply [flat|nested] 41+ messages in thread
* Re: [PATCH v3 0/7] Add script for real-time telemetry monitoring
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
` (7 preceding siblings ...)
2026-01-16 6:53 ` [PATCH v3 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
@ 2026-01-16 6:57 ` Stephen Hemminger
8 siblings, 0 replies; 41+ messages in thread
From: Stephen Hemminger @ 2026-01-16 6:57 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On Thu, 15 Jan 2026 19:03:24 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> TL;DR
> ------
>
> For a quick demo, apply patches, run e.g. testpmd and then in a separate
> terminal run:
>
> ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
>
> Output, updated once per second, will be traffic rate per port e.g.:
>
> Connected to application: "dpdk-testpmd"
> Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> 16:29:12 5,213,119 5,214,304 10,427,423
>
>
> Fuller details
> --------------
>
> While we have the dpdk-telemetry.py CLI app for interactive querying of
> telemetry on the commandline, and a telemetry exporter script for
> sending telemetry to external tools for real-time monitoring, we don't
> have an app that can print real-time stats for DPDK apps on the
> terminal. This patchset adds such a script, developed with the help of
> Github copilot to fill a need that I found in my testing. Submitting it
> here in the hopes that others find it of use.
>
> The script acts as a wrapper around the existing dpdk-telemetry.py
> script, and pipes the commands to that script and reads the responses,
> querying it once per second. It takes a number of flag parameters, such
> as the ones above:
> - "-d" for delta values, i.e. PPS rather than total packets
> - "-1" for single-line output, i.e. no scrolling up the screen
> - "-T" to display a total column
>
> Other flag parameters can be seen by looking at the help output.
>
> Beyond the flags, the script also takes a number of positional
> parameters, which refer to specific stats to display. These stats must
> be numeric values, and should take the form of the telemetry command to
> send, followed by a "." and the stat within the result which is to be
> tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
> where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
> part of the result.
>
> However, specifying individual stats can be awkward, so some shortcuts
> are provided too for the common case of monitoring ethernet ports. Any
> positional arg starting with "eth" will be replaced by the set of
> equivalent values for each port, e.g. "eth.imissed" will track the
> imissed value on all ports in use in the app. The ipackets and opackets
> values, as common metrics, are also available as shortened values as
> just "rx" and "tx", so in the example above, "eth.tx" means to track the
> opackets stat for every ethdev port.
>
> Finally, the script also has reconnection support so you can leave it
> running while you start and stop your application in another terminal.
> The watcher will try and reconnect to a running instance every second.
>
> v3:
> Updated following AI review
> - removed unnecessary f-string
> - added documnentation in guides/tools
> - added release note entry
>
> v2:
> - improve reconnection handling, eliminating some crashes seen in testing.
>
>
> Bruce Richardson (7):
> usertools: add new script to monitor telemetry on terminal
> usertools/telemetry-watcher: add displaying stats
> usertools/telemetry-watcher: add delta and timeout opts
> usertools/telemetry-watcher: add total and one-line opts
> usertools/telemetry-watcher: add thousands separator
> usertools/telemetry-watcher: add eth name shortcuts
> usertools/telemetry-watcher: support reconnection
>
> doc/guides/rel_notes/release_26_03.rst | 7 +
> doc/guides/tools/index.rst | 1 +
> doc/guides/tools/telemetrywatcher.rst | 184 +++++++++++
> usertools/dpdk-telemetry-watcher.py | 435 +++++++++++++++++++++++++
> usertools/meson.build | 1 +
> 5 files changed, 628 insertions(+)
> create mode 100644 doc/guides/tools/telemetrywatcher.rst
> create mode 100755 usertools/dpdk-telemetry-watcher.py
>
> --
> 2.51.0
AI also had some good feedback on the documentation here.
### Spelling Issues
✓ No spelling errors found.
---
### Grammar Issues
| Line/Section | Issue | Suggestion |
|--------------|-------|------------|
| Options: `-i` | "...when multiple applications are running with the same file-prefix." | Consider: "...when multiple applications **share** the same file-prefix." (more concise) |
| Options: `-t` | "Number of iterations to run before stopping." | Should be: "Number of iterations to run before **exiting**." (matches the behavior—tool exits, not just stops) |
| Statistics Format | "To discover available commands and fields:" followed by numbered list | The colon introduces an incomplete sentence. Better: "To discover available commands and fields, use the following methods:" or remove the colon and use "you can:" |
---
### Technical Writing Style Issues
| Location | Issue | Recommendation |
|----------|-------|----------------|
| Title | "dpdk-telemetry-watcher Application" | Consider: "dpdk-telemetry-watcher Tool" — the word "application" might confuse readers since DPDK applications are the *targets* being monitored |
| Opening paragraph | "Data Plane Development Kit (DPDK)" | Unnecessary expansion—DPDK docs assume readers know what DPDK is. Simplify to just "DPDK" |
| Options: `-d` | "Display delta (incremental) values" | Redundant parenthetical. Use either "delta values" or "incremental values", not both |
| Options: `-d` | "which is useful for monitoring rates of change" | Vague. Better: "which is useful for monitoring per-second rates" |
| Shortcuts section | "These shortcuts automatically detect all available ethernet devices" | Should be "Ethernet" (capital E) per DPDK style, or better: "ethdev ports" to match DPDK terminology |
| Output Format | "Values are formatted with locale-specific number formatting (e.g., comma separators)." | Awkward repetition of "formatting". Rewrite: "Values use locale-specific formatting (e.g., comma as thousands separator)." |
| Dependencies | "A running DPDK application with telemetry enabled" | Contradicts the "Running the Application" section which says the tool "can be run at any time, whether or not a DPDK application is currently running." Remove this item or clarify it's needed for actual monitoring. |
---
### Passive Voice (per DPDK documentation standards)
| Location | Passive Construction | Active Alternative |
|----------|---------------------|-------------------|
| Opening | "Statistics are specified in the format..." | "Specify statistics in the format..." |
| Output Format | "Values are formatted with..." | "The tool formats values with..." |
| Output Format | "the tool displays the change in each statistic" | ✓ Already active (good) |
---
### Consistency Issues
| Issue | Details |
|-------|---------|
| "ethernet" vs "Ethernet" | Line uses lowercase "ethernet devices" — DPDK typically uses "Ethernet" or avoids it entirely in favor of "ethdev" |
| "file-prefix" hyphenation | Used consistently (good), but DPDK code often uses "file_prefix" — verify against dpdk-telemetry.py docs |
| Option argument style | Mix of `FILE_PREFIX` (all caps) and `TIMEOUT` (all caps) — consistent (good) |
| Example commands | Some use full paths (`/ethdev/stats,0.ipackets`), shortcuts (`eth.rx`) — good variety showing both |
---
### RST Formatting Issues
| Location | Issue | Fix |
|----------|-------|-----|
| Line 1 | `.. SPDX` has two spaces after `..` | Should be single space: `.. SPDX` |
| Cross-references | `` `Statistics Format`_ `` and `` `Examples`_ `` | These RST references work but would be more robust as `:ref:` targets for cross-document linking |
---
### Missing Information
| Gap | Suggestion |
|-----|------------|
| No mention of Ctrl+C | Add note that Ctrl+C cleanly exits the monitoring loop (already in code) |
| No sample output | Consider adding a brief example of what the output looks like |
| No error handling description | What happens if a stat doesn't exist? Document the error messages |
---
### Suggested Rewrites
**Opening paragraph** (current):
> The ``dpdk-telemetry-watcher`` tool is a Data Plane Development Kit (DPDK) utility that provides continuous monitoring of DPDK telemetry statistics on the command line. It wraps the ``dpdk-telemetry.py`` script to provide real-time statistics display capabilities.
**Suggested**:
> The ``dpdk-telemetry-watcher`` tool monitors DPDK telemetry statistics continuously on the command line. It wraps ``dpdk-telemetry.py`` to provide real-time display of selected statistics.
---
**Dependencies section** (current):
> The tool requires:
> * Python 3
> * The ``dpdk-telemetry.py`` script must be available in the same directory or in the system PATH
> * A running DPDK application with telemetry enabled
**Suggested**:
> The tool requires:
>
> * Python 3
> * The ``dpdk-telemetry.py`` script (in the same directory or in PATH)
>
> For monitoring, a DPDK application with telemetry enabled must be running, though the watcher can start before the application and will connect automatically.
---
### Summary
| Category | Count |
|----------|-------|
| Grammar issues | 3 |
| Style issues | 7 |
| Passive voice | 2 |
| Consistency issues | 1 |
| RST formatting | 1 |
| Missing information | 3 |
**Overall**: The documentation is functional and reasonably clear. The issues are minor and mostly stylistic. The most important fix is resolving the contradiction in the Dependencies section about whether a running DPDK application is required.
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v4 0/7] Add script for real-time telemetry monitoring
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
` (10 preceding siblings ...)
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
` (7 more replies)
11 siblings, 8 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson
TL;DR
------
For a quick demo, apply patches, run e.g. testpmd and then in a separate
terminal run:
./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
Output, updated once per second, will be traffic rate per port e.g.:
Connected to application: "dpdk-testpmd"
Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
16:29:12 5,213,119 5,214,304 10,427,423
Fuller details
--------------
While we have the dpdk-telemetry.py CLI app for interactive querying of
telemetry on the commandline, and a telemetry exporter script for
sending telemetry to external tools for real-time monitoring, we don't
have an app that can print real-time stats for DPDK apps on the
terminal. This patchset adds such a script, developed with the help of
Github copilot to fill a need that I found in my testing. Submitting it
here in the hopes that others find it of use.
The script acts as a wrapper around the existing dpdk-telemetry.py
script, and pipes the commands to that script and reads the responses,
querying it once per second. It takes a number of flag parameters, such
as the ones above:
- "-d" for delta values, i.e. PPS rather than total packets
- "-1" for single-line output, i.e. no scrolling up the screen
- "-T" to display a total column
Other flag parameters can be seen by looking at the help output.
Beyond the flags, the script also takes a number of positional
parameters, which refer to specific stats to display. These stats must
be numeric values, and should take the form of the telemetry command to
send, followed by a "." and the stat within the result which is to be
tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
part of the result.
However, specifying individual stats can be awkward, so some shortcuts
are provided too for the common case of monitoring ethernet ports. Any
positional arg starting with "eth" will be replaced by the set of
equivalent values for each port, e.g. "eth.imissed" will track the
imissed value on all ports in use in the app. The ipackets and opackets
values, as common metrics, are also available as shortened values as
just "rx" and "tx", so in the example above, "eth.tx" means to track the
opackets stat for every ethdev port.
Finally, the script also has reconnection support so you can leave it
running while you start and stop your application in another terminal.
The watcher will try and reconnect to a running instance every second.
v4:
- Updated docs following AI review
- Converted one missed f-string to regular string
v3:
Updated following AI review
- removed unnecessary f-string
- added documnentation in guides/tools
- added release note entry
v2:
- improve reconnection handling, eliminating some crashes seen in testing.
Bruce Richardson (7):
usertools: add new script to monitor telemetry on terminal
usertools/telemetry-watcher: add displaying stats
usertools/telemetry-watcher: add delta and timeout opts
usertools/telemetry-watcher: add total and one-line opts
usertools/telemetry-watcher: add thousands separator
usertools/telemetry-watcher: add eth name shortcuts
usertools/telemetry-watcher: support reconnection
doc/guides/rel_notes/release_26_03.rst | 7 +
doc/guides/tools/index.rst | 1 +
doc/guides/tools/telemetrywatcher.rst | 184 +++++++++++
usertools/dpdk-telemetry-watcher.py | 435 +++++++++++++++++++++++++
usertools/meson.build | 1 +
5 files changed, 628 insertions(+)
create mode 100644 doc/guides/tools/telemetrywatcher.rst
create mode 100755 usertools/dpdk-telemetry-watcher.py
--
2.51.0
^ permalink raw reply [flat|nested] 41+ messages in thread
* [PATCH v4 1/7] usertools: add new script to monitor telemetry on terminal
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
` (6 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
The dpdk-telemetry.py script is useful for getting telemetry
interactively, but sometimes we want to monitor stats over time, so add
a telemetry-watcher script to do so.
Start off such a script with basic arg processing, and connecting to
dpdk-telemetry as a subprocess, so that we can send-receive commands
from it.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/index.rst | 1 +
doc/guides/tools/telemetrywatcher.rst | 56 ++++++++
usertools/dpdk-telemetry-watcher.py | 200 ++++++++++++++++++++++++++
usertools/meson.build | 1 +
4 files changed, 258 insertions(+)
create mode 100644 doc/guides/tools/telemetrywatcher.rst
create mode 100755 usertools/dpdk-telemetry-watcher.py
diff --git a/doc/guides/tools/index.rst b/doc/guides/tools/index.rst
index 8ec429ec53..13f75a5bc6 100644
--- a/doc/guides/tools/index.rst
+++ b/doc/guides/tools/index.rst
@@ -14,6 +14,7 @@ DPDK Tools User Guides
pmdinfo
dumpcap
pdump
+ telemetrywatcher
dmaperf
flow-perf
securityperf
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
new file mode 100644
index 0000000000..b337e4114f
--- /dev/null
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -0,0 +1,56 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2026 Intel Corporation
+
+dpdk-telemetry-watcher Tool
+===========================
+
+The ``dpdk-telemetry-watcher`` tool monitors DPDK telemetry statistics continuously on the command line.
+It wraps the ``dpdk-telemetry.py`` script to provide real-time statistics display capabilities.
+
+
+Running the Tool
+----------------
+
+The tool has a number of command line options:
+
+.. code-block:: console
+
+ dpdk-telemetry-watcher.py [options] stat1 stat2 ...
+
+
+Options
+-------
+
+.. program:: dpdk-telemetry-watcher.py
+
+.. option:: -h, --help
+
+ Display usage information and quit
+
+.. option:: -f FILE_PREFIX, --file-prefix FILE_PREFIX
+
+ Provide file-prefix for DPDK runtime directory.
+ Passed to ``dpdk-telemetry.py`` to identify the target DPDK application.
+ Default is ``rte``.
+
+.. option:: -i INSTANCE, --instance INSTANCE
+
+ Provide instance number for DPDK application when multiple applications share the same file-prefix.
+ Passed to ``dpdk-telemetry.py`` to identify the target DPDK application instance.
+ Default is ``0``.
+
+.. option:: -l, --list
+
+ List all possible file-prefixes and exit.
+ This is useful to discover which DPDK applications are currently running.
+
+Dependencies
+------------
+
+The tool requires:
+
+* Python 3
+* The ``dpdk-telemetry.py`` script (in the same directory or in PATH)
+
+For monitoring, a DPDK application with telemetry enabled must be running,
+though the watcher can start before the application and will connect automatically.
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
new file mode 100755
index 0000000000..a5ac293d06
--- /dev/null
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -0,0 +1,200 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2025 Intel Corporation
+
+"""
+Script to monitor DPDK telemetry statistics on the command line.
+Wraps dpdk-telemetry.py to provide continuous monitoring capabilities.
+"""
+
+import argparse
+import subprocess
+import sys
+import os
+import shutil
+import errno
+import json
+
+
+def get_app_name(pid):
+ """return the app name for a given PID, for printing"""
+ proc_cmdline = os.path.join("/proc", str(pid), "cmdline")
+ try:
+ with open(proc_cmdline) as f:
+ argv0 = f.read(1024).split("\0")[0]
+ return os.path.basename(argv0)
+ except IOError as e:
+ # ignore file not found errors
+ if e.errno != errno.ENOENT:
+ raise
+ return None
+
+
+def find_telemetry_script():
+ """Find the dpdk-telemetry.py script in the script directory or PATH.
+
+ Returns:
+ str: Path to the dpdk-telemetry.py script
+
+ Exits:
+ If the script cannot be found
+ """
+ # First, try to find it in the same directory as this script
+ script_dir = os.path.dirname(os.path.abspath(__file__))
+ telemetry_script = os.path.join(script_dir, "dpdk-telemetry.py")
+
+ # If not found locally, check if it's in PATH
+ if not os.path.exists(telemetry_script):
+ telemetry_in_path = shutil.which("dpdk-telemetry.py")
+ if telemetry_in_path:
+ telemetry_script = telemetry_in_path
+ else:
+ print(
+ "Error: dpdk-telemetry.py not found in script directory or PATH",
+ file=sys.stderr,
+ )
+ sys.exit(1)
+
+ return telemetry_script
+
+
+def create_telemetry_process(telemetry_script, args_list):
+ """Create a subprocess for dpdk-telemetry.py with pipes.
+
+ Args:
+ telemetry_script: Path to the dpdk-telemetry.py script
+ args_list: List of arguments to pass to the script
+
+ Returns:
+ subprocess.Popen: Process handle with stdin/stdout/stderr pipes
+
+ Exits:
+ If the process cannot be created
+ """
+ # Build the command
+ cmd = [sys.executable, telemetry_script] + args_list
+
+ try:
+ process = subprocess.Popen(
+ cmd,
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ text=True,
+ bufsize=1, # Line buffered
+ )
+ return process
+ except FileNotFoundError:
+ print("Error: Python interpreter or script not found", file=sys.stderr)
+ sys.exit(1)
+ except Exception as e:
+ print(f"Error running dpdk-telemetry.py: {e}", file=sys.stderr)
+ sys.exit(1)
+
+
+def query_telemetry(process, command):
+ """Send a telemetry command and return the parsed JSON response.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
+
+ Returns:
+ dict: The parsed JSON response with the command wrapper stripped,
+ or None if there was an error
+ """
+ # Send the command
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+
+ # Read the JSON response
+ response = process.stdout.readline()
+ try:
+ data = json.loads(response)
+ # When run non-interactively, the response is wrapped with the command
+ # e.g., {"/info": {"version": ..., "pid": ...}}
+ # or {"/ethdev/stats,0": {...}}
+ # The response should have exactly one key which is the command
+ if len(data) == 1:
+ # Extract the value, ignoring the key
+ return next(iter(data.values()))
+ else:
+ return data
+ except (json.JSONDecodeError, KeyError):
+ return None
+
+
+def print_connected_app(process):
+ """Query and print the name of the connected DPDK application.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ """
+ info = query_telemetry(process, "/info")
+ if info and "pid" in info:
+ app_name = get_app_name(info["pid"])
+ if app_name:
+ print(f'Connected to application: "{app_name}"')
+
+
+def main():
+ """Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+
+ # Parse command line arguments - matching dpdk-telemetry.py parameters
+ parser = argparse.ArgumentParser(
+ description="Monitor DPDK telemetry statistics on the command line"
+ )
+ parser.add_argument(
+ "-f",
+ "--file-prefix",
+ default="rte",
+ help="Provide file-prefix for DPDK runtime directory",
+ )
+ parser.add_argument(
+ "-i",
+ "--instance",
+ default=0,
+ type=int,
+ help="Provide instance number for DPDK application",
+ )
+ parser.add_argument(
+ "-l",
+ "--list",
+ action="store_true",
+ default=False,
+ help="List all possible file-prefixes and exit",
+ )
+
+ args = parser.parse_args()
+
+ # Find the dpdk-telemetry.py script
+ telemetry_script = find_telemetry_script()
+
+ # Build arguments list
+ args_list = ["-f", args.file_prefix, "-i", str(args.instance)]
+
+ if args.list:
+ args_list.append("-l")
+ # For --list, just run the command directly without pipes
+ cmd = [sys.executable, telemetry_script] + args_list
+ return subprocess.run(cmd).returncode
+
+ # Run dpdk-telemetry.py with pipes for stdin and stdout
+ process = create_telemetry_process(telemetry_script, args_list)
+
+ # Get and display the connected application name
+ print_connected_app(process)
+
+ # TODO: Add monitoring logic here
+
+ # Clean up
+ process.stdin.close()
+ process.stdout.close()
+ process.stderr.close()
+ process.wait()
+
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/usertools/meson.build b/usertools/meson.build
index eb48e2f440..114d0a65b4 100644
--- a/usertools/meson.build
+++ b/usertools/meson.build
@@ -12,6 +12,7 @@ install_data([
'dpdk-hugepages.py',
'dpdk-rss-flows.py',
'dpdk-telemetry-exporter.py',
+ 'dpdk-telemetry-watcher.py',
],
install_dir: 'bin')
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v4 2/7] usertools/telemetry-watcher: add displaying stats
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
` (5 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Add support for printing out particular stats once every second. The
stats to be queried are given on the commandline, in the format of
<command>.<result-field>, for example /ethdev/stats,0.ipackets.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 51 +++++++++++++
usertools/dpdk-telemetry-watcher.py | 101 +++++++++++++++++++++++++-
2 files changed, 151 insertions(+), 1 deletion(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index b337e4114f..44780e28c0 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -44,6 +44,57 @@ Options
List all possible file-prefixes and exit.
This is useful to discover which DPDK applications are currently running.
+.. option:: stat
+
+ Statistics to monitor in format ``command.field``.
+ Multiple statistics can be specified and will be displayed in columns.
+ See the `Statistics Format`_ section below for details on specifying statistics.
+
+
+Statistics Format
+-----------------
+
+Specify statistics in the format ``command.field`` where:
+
+* ``command`` is a telemetry command (e.g., ``/ethdev/stats,0``)
+* ``field`` is a field name from the command's JSON response (e.g., ``ipackets``)
+
+To discover available commands and fields, follow the steps below:
+
+1. Use ``dpdk-telemetry.py`` interactively to explore available commands
+2. Use the ``/`` command to list all available telemetry endpoints
+3. Query specific commands to see their response format
+
+Example telemetry commands:
+
+* ``/ethdev/list`` - List all Ethernet devices
+* ``/ethdev/stats,N`` - Get statistics for Ethernet device N
+* ``/ethdev/xstats,N`` - Get extended statistics for Ethernet device N
+* ``/eal/mempool_list`` - List all mempools
+* ``/mempool/info,N`` - Get information about mempool N
+
+See `Examples`_ section for usage examples based on the results of these telemetry commands.
+
+Examples
+--------
+
+Monitor received packets on Ethernet device 0::
+
+ dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets
+
+Monitor received and transmitted packets on device 0::
+
+ dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets /ethdev/stats,0.opackets
+
+Monitor a DPDK application with a custom file-prefix::
+
+ dpdk-telemetry-watcher.py -f myapp /ethdev/stats,0.ipackets
+
+List all running DPDK applications::
+
+ dpdk-telemetry-watcher.py -l
+
+
Dependencies
------------
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index a5ac293d06..dad4e60475 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -14,6 +14,7 @@
import shutil
import errno
import json
+import time
def get_app_name(pid):
@@ -137,6 +138,91 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def validate_stats(process, stat_specs):
+ """Validate stat specifications and check that fields are numeric.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+
+ Returns:
+ List of tuples (spec, command, field) for valid specs, or None on error
+ """
+ parsed_specs = []
+ for spec in stat_specs:
+ # Parse the stat specification
+ if "." not in spec:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ command, field = spec.rsplit(".", 1)
+ if not command or not field:
+ print(f"Error: Invalid stat specification '{spec}'", file=sys.stderr)
+ print(
+ "Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ file=sys.stderr,
+ )
+ return None
+
+ # Query the stat once to validate it exists and is numeric
+ data = query_telemetry(process, command)
+ if not isinstance(data, dict):
+ print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
+ return None
+ if field not in data:
+ print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
+ return None
+ value = data[field]
+ if not isinstance(value, (int, float)):
+ print(
+ f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
+ file=sys.stderr,
+ )
+ return None
+
+ parsed_specs.append((spec, command, field))
+
+ return parsed_specs
+
+
+def monitor_stats(process, stat_specs):
+ """Monitor and display statistics in columns.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications in format "command.field"
+ """
+ # Validate all stat specifications
+ parsed_specs = validate_stats(process, stat_specs)
+ if not parsed_specs:
+ return
+
+ # Print header
+ header = "Time".ljust(10)
+ for spec, _, _ in parsed_specs:
+ header += spec.rjust(25)
+ print(header)
+
+ # Monitor loop - once per second
+ try:
+ while True:
+ timestamp = time.strftime("%H:%M:%S")
+ row = timestamp.ljust(10)
+
+ for spec, command, field in parsed_specs:
+ data = query_telemetry(process, command)
+ row += str(data[field]).rjust(25)
+
+ print(row)
+ time.sleep(1)
+ except KeyboardInterrupt:
+ print("\nMonitoring stopped")
+
+
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
@@ -164,6 +250,11 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "stats",
+ nargs="*",
+ help="Statistics to monitor in format 'command.field' (e.g., /ethdev/stats,0.ipackets)",
+ )
args = parser.parse_args()
@@ -179,13 +270,21 @@ def main():
cmd = [sys.executable, telemetry_script] + args_list
return subprocess.run(cmd).returncode
+ # Check if stats were provided
+ if not args.stats:
+ print("Error: No statistics to monitor specified", file=sys.stderr)
+ print("Usage: dpdk-telemetry-watcher.py [options] stat1 stat2 ...", file=sys.stderr)
+ print("Example: dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets", file=sys.stderr)
+ return 1
+
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
# Get and display the connected application name
print_connected_app(process)
- # TODO: Add monitoring logic here
+ # Monitor the requested statistics
+ monitor_stats(process, args.stats)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v4 3/7] usertools/telemetry-watcher: add delta and timeout opts
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
` (4 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Add options to the script to time out and quit after a certain number of
seconds. Also, add an option to display delta instead of absolute
values, allowing easy display of e.g. PPS.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 11 +++++
usertools/dpdk-telemetry-watcher.py | 64 ++++++++++++++++++++-------
2 files changed, 59 insertions(+), 16 deletions(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index 44780e28c0..5a9c60946b 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -44,6 +44,17 @@ Options
List all possible file-prefixes and exit.
This is useful to discover which DPDK applications are currently running.
+.. option:: -t TIMEOUT, --timeout TIMEOUT
+
+ Number of iterations to run before exiting.
+ If not specified, the tool runs indefinitely until interrupted with Ctrl+C.
+
+.. option:: -d, --delta
+
+ Display delta values instead of absolute values.
+ This shows the change in statistics since the last iteration,
+ which is useful for monitoring per-second rates.
+
.. option:: stat
Statistics to monitor in format ``command.field``.
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index dad4e60475..5f4aa05431 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -146,9 +146,13 @@ def validate_stats(process, stat_specs):
stat_specs: List of stat specifications in format "command.field"
Returns:
- List of tuples (spec, command, field) for valid specs, or None on error
+ Tuple of (parsed_specs, initial_values) where:
+ parsed_specs: List of tuples (spec, command, field) for valid specs
+ initial_values: List of initial values for each stat
+ Returns (None, None) on error
"""
parsed_specs = []
+ initial_values = []
for spec in stat_specs:
# Parse the stat specification
if "." not in spec:
@@ -157,7 +161,7 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
command, field = spec.rsplit(".", 1)
if not command or not field:
@@ -166,38 +170,39 @@ def validate_stats(process, stat_specs):
"Expected format: 'command.field' (e.g., /ethdev/stats,0.ipackets)",
file=sys.stderr,
)
- return None
+ return None, None
# Query the stat once to validate it exists and is numeric
data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
- return None
+ return None, None
if field not in data:
print(f"Error: Field '{field}' not found in '{command}' response", file=sys.stderr)
- return None
+ return None, None
value = data[field]
if not isinstance(value, (int, float)):
print(
f"Error: Field '{field}' in '{command}' is not numeric (got {type(value).__name__})",
file=sys.stderr,
)
- return None
+ return None, None
parsed_specs.append((spec, command, field))
+ initial_values.append(value)
- return parsed_specs
+ return parsed_specs, initial_values
-def monitor_stats(process, stat_specs):
+def monitor_stats(process, args):
"""Monitor and display statistics in columns.
Args:
process: The subprocess.Popen handle to the telemetry process
- stat_specs: List of stat specifications in format "command.field"
+ args: Parsed command line arguments
"""
- # Validate all stat specifications
- parsed_specs = validate_stats(process, stat_specs)
+ # Validate all stat specifications and get initial values
+ parsed_specs, prev_values = validate_stats(process, args.stats)
if not parsed_specs:
return
@@ -208,17 +213,30 @@ def monitor_stats(process, stat_specs):
print(header)
# Monitor loop - once per second
+ count = 0
try:
- while True:
+ while args.timeout is None or count < args.timeout:
+ time.sleep(1)
+ count += 1
+
timestamp = time.strftime("%H:%M:%S")
row = timestamp.ljust(10)
- for spec, command, field in parsed_specs:
+ current_values = []
+ for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
- row += str(data[field]).rjust(25)
+ current_value = data[field]
+ current_values.append(current_value)
+
+ if args.delta:
+ display_value = current_value - prev_values[i]
+ else:
+ display_value = current_value
+
+ row += str(display_value).rjust(25)
print(row)
- time.sleep(1)
+ prev_values = current_values
except KeyboardInterrupt:
print("\nMonitoring stopped")
@@ -250,6 +268,20 @@ def main():
default=False,
help="List all possible file-prefixes and exit",
)
+ parser.add_argument(
+ "-t",
+ "--timeout",
+ type=int,
+ default=None,
+ help="Number of iterations to run before stopping (default: run indefinitely)",
+ )
+ parser.add_argument(
+ "-d",
+ "--delta",
+ action="store_true",
+ default=False,
+ help="Display delta values instead of absolute values",
+ )
parser.add_argument(
"stats",
nargs="*",
@@ -284,7 +316,7 @@ def main():
print_connected_app(process)
# Monitor the requested statistics
- monitor_stats(process, args.stats)
+ monitor_stats(process, args)
# Clean up
process.stdin.close()
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v4 4/7] usertools/telemetry-watcher: add total and one-line opts
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
` (2 preceding siblings ...)
2026-02-05 15:02 ` [PATCH v4 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
` (3 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Add options to the script to print out totals at the end of each line,
and to print each line replacing the previous one, saving the output
scrolling up the screen constantly.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 27 +++++++++++++++++++++++++++
usertools/dpdk-telemetry-watcher.py | 27 ++++++++++++++++++++++++++-
2 files changed, 53 insertions(+), 1 deletion(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index 5a9c60946b..b4ca1fcfc0 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -55,6 +55,15 @@ Options
This shows the change in statistics since the last iteration,
which is useful for monitoring per-second rates.
+.. option:: -T, --total
+
+ Display a total column at the end of each row that sums all monitored statistics.
+
+.. option:: -1, --single-line
+
+ Display output on a single line, replacing the previous output.
+ This is useful for reducing scrolling and keeping the display compact.
+
.. option:: stat
Statistics to monitor in format ``command.field``.
@@ -106,6 +115,24 @@ List all running DPDK applications::
dpdk-telemetry-watcher.py -l
+Output Format
+-------------
+
+The tool displays statistics in a tabular format with:
+
+* **Time column** - Current timestamp (HH:MM:SS)
+* **Statistics columns** - One column per specified statistic
+* **Total column** - Optional sum of all statistics (when ``-T`` is used)
+
+Displayed values use locale-specific number formatting (e.g. comma as thousands separator).
+
+When ``--delta`` mode is enabled, the tool displays the change in each statistic
+since the last iteration, which typically represents the rate per second.
+
+When ``--single-line`` mode is enabled, each new output line replaces the previous one,
+similar to tools like ``top``.
+
+
Dependencies
------------
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 5f4aa05431..e4cd292515 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -210,10 +210,13 @@ def monitor_stats(process, args):
header = "Time".ljust(10)
for spec, _, _ in parsed_specs:
header += spec.rjust(25)
+ if args.total:
+ header += "Total".rjust(25)
print(header)
# Monitor loop - once per second
count = 0
+ line_ending = "\r" if args.single_line else "\n"
try:
while args.timeout is None or count < args.timeout:
time.sleep(1)
@@ -223,6 +226,7 @@ def monitor_stats(process, args):
row = timestamp.ljust(10)
current_values = []
+ total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
data = query_telemetry(process, command)
current_value = data[field]
@@ -233,11 +237,17 @@ def monitor_stats(process, args):
else:
display_value = current_value
+ total += display_value
row += str(display_value).rjust(25)
- print(row)
+ if args.total:
+ row += str(total).rjust(25)
+
+ print(row, end=line_ending, flush=True)
prev_values = current_values
except KeyboardInterrupt:
+ if args.single_line:
+ print() # Add newline before exit message
print("\nMonitoring stopped")
@@ -282,6 +292,21 @@ def main():
default=False,
help="Display delta values instead of absolute values",
)
+ parser.add_argument(
+ "-T",
+ "--total",
+ action="store_true",
+ default=False,
+ help="Display a total column at the end of each row",
+ )
+ parser.add_argument(
+ "-1",
+ "--single-line",
+ action="store_true",
+ default=False,
+ dest="single_line",
+ help="Display output on a single line, replacing the previous output",
+ )
parser.add_argument(
"stats",
nargs="*",
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v4 5/7] usertools/telemetry-watcher: add thousands separator
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
` (3 preceding siblings ...)
2026-02-05 15:02 ` [PATCH v4 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
` (2 subsequent siblings)
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
When printing large values of packets, generally in the millions, things
are far more readable with thousands and millions separators.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
usertools/dpdk-telemetry-watcher.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index e4cd292515..7ec7267a38 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -15,6 +15,7 @@
import errno
import json
import time
+import locale
def get_app_name(pid):
@@ -238,10 +239,10 @@ def monitor_stats(process, args):
display_value = current_value
total += display_value
- row += str(display_value).rjust(25)
+ row += f"{display_value:n}".rjust(25)
if args.total:
- row += str(total).rjust(25)
+ row += f"{total:n}".rjust(25)
print(row, end=line_ending, flush=True)
prev_values = current_values
@@ -254,6 +255,9 @@ def monitor_stats(process, args):
def main():
"""Main function to parse arguments and run dpdk-telemetry.py with a pipe"""
+ # Set locale for number formatting
+ locale.setlocale(locale.LC_ALL, "")
+
# Parse command line arguments - matching dpdk-telemetry.py parameters
parser = argparse.ArgumentParser(
description="Monitor DPDK telemetry statistics on the command line"
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v4 6/7] usertools/telemetry-watcher: add eth name shortcuts
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
` (4 preceding siblings ...)
2026-02-05 15:02 ` [PATCH v4 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
2026-03-31 18:10 ` [PATCH v4 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Since an expected main use of the script is to monitor ethdev packet
stats, provide a shortened form of parameters to make it easier to
monitor all ports on the system. Any stat starting with "eth." is taken
not as a direct command, but instead as a shortcut for getting the stats
for all ports on the system. For example: eth.ibytes shows the byte
counts for all ports.
Beyond that, provide a shortcut for ipackets and opackets as just rx and
tx respectively. Therefore, to monitor the output rate of an app, one
can use "dpdk-telemetry-watcher -dT eth.tx"
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/tools/telemetrywatcher.rst | 33 ++++++++++++++++++
usertools/dpdk-telemetry-watcher.py | 49 ++++++++++++++++++++++++++-
2 files changed, 81 insertions(+), 1 deletion(-)
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index b4ca1fcfc0..251d99a085 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -95,6 +95,19 @@ Example telemetry commands:
See `Examples`_ section for usage examples based on the results of these telemetry commands.
+Shortcuts
+---------
+
+The tool provides convenient shortcuts for common statistics:
+
+* ``eth.rx`` - Expands to ``/ethdev/stats,N.ipackets`` for all Ethernet devices
+* ``eth.tx`` - Expands to ``/ethdev/stats,N.opackets`` for all Ethernet devices
+* ``eth.FIELD`` - Expands to ``/ethdev/stats,N.FIELD`` for all Ethernet devices
+
+These shortcuts automatically detect all available Ethernet devices
+and create a column for each one.
+
+
Examples
--------
@@ -106,10 +119,30 @@ Monitor received and transmitted packets on device 0::
dpdk-telemetry-watcher.py /ethdev/stats,0.ipackets /ethdev/stats,0.opackets
+Monitor received packets on all Ethernet devices using shortcut::
+
+ dpdk-telemetry-watcher.py eth.rx
+
+Monitor packet deltas (rates) for device 0::
+
+ dpdk-telemetry-watcher.py -d /ethdev/stats,0.ipackets /ethdev/stats,0.opackets
+
+Monitor with a total column showing aggregate traffic::
+
+ dpdk-telemetry-watcher.py -d -T eth.rx eth.tx
+
+Monitor for a specific duration (60 iterations = 60 seconds)::
+
+ dpdk-telemetry-watcher.py -t 60 /ethdev/stats,0.ipackets
+
Monitor a DPDK application with a custom file-prefix::
dpdk-telemetry-watcher.py -f myapp /ethdev/stats,0.ipackets
+Monitor in single-line mode (no scrolling)::
+
+ dpdk-telemetry-watcher.py -1 -d eth.rx eth.tx
+
List all running DPDK applications::
dpdk-telemetry-watcher.py -l
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index 7ec7267a38..eda57e5ba5 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -139,6 +139,48 @@ def print_connected_app(process):
print(f'Connected to application: "{app_name}"')
+def expand_shortcuts(process, stat_specs):
+ """Expand special shortcuts like eth.rx and eth.tx into actual stat specifications.
+
+ Args:
+ process: The subprocess.Popen handle to the telemetry process
+ stat_specs: List of stat specifications, possibly including shortcuts
+
+ Returns:
+ List of expanded stat specifications
+ """
+ expanded = []
+ for spec in stat_specs:
+ if not spec.startswith("eth."):
+ expanded.append(spec)
+ continue
+
+ # Extract the field name after "eth."
+ field = spec[4:] # Remove "eth." prefix
+ if not field:
+ print(f"Error: Invalid shortcut '{spec}' - missing field name", file=sys.stderr)
+ return None
+
+ # Map common shortcuts to actual field names
+ field_map = {
+ "rx": "ipackets",
+ "tx": "opackets",
+ }
+ field = field_map.get(field, field)
+
+ # Get list of ethernet devices
+ port_list = query_telemetry(process, "/ethdev/list")
+ if not isinstance(port_list, list):
+ print("Error: Failed to get ethernet device list", file=sys.stderr)
+ return None
+
+ # Create stat specs for each port
+ for port in port_list:
+ expanded.append(f"/ethdev/stats,{port}.{field}")
+
+ return expanded
+
+
def validate_stats(process, stat_specs):
"""Validate stat specifications and check that fields are numeric.
@@ -202,8 +244,13 @@ def monitor_stats(process, args):
process: The subprocess.Popen handle to the telemetry process
args: Parsed command line arguments
"""
+ # Expand any shortcuts like eth-rx, eth-tx
+ expanded_stats = expand_shortcuts(process, args.stats)
+ if not expanded_stats:
+ return
+
# Validate all stat specifications and get initial values
- parsed_specs, prev_values = validate_stats(process, args.stats)
+ parsed_specs, prev_values = validate_stats(process, expanded_stats)
if not parsed_specs:
return
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* [PATCH v4 7/7] usertools/telemetry-watcher: support reconnection
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
` (5 preceding siblings ...)
2026-02-05 15:02 ` [PATCH v4 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
@ 2026-02-05 15:02 ` Bruce Richardson
2026-03-31 18:10 ` [PATCH v4 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
7 siblings, 0 replies; 41+ messages in thread
From: Bruce Richardson @ 2026-02-05 15:02 UTC (permalink / raw)
To: dev; +Cc: Bruce Richardson, Stephen Hemminger
Allow the watcher binary to run even when there is no DPDK process
running. In that case, wait for a suitable process to start and begin
monitoring then. In case of disconnection, keep trying to reconnect and
resume once reconnection succeeds.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
---
doc/guides/rel_notes/release_26_03.rst | 7 ++++
doc/guides/tools/telemetrywatcher.rst | 6 +++
usertools/dpdk-telemetry-watcher.py | 52 ++++++++++++++++++++------
3 files changed, 53 insertions(+), 12 deletions(-)
diff --git a/doc/guides/rel_notes/release_26_03.rst b/doc/guides/rel_notes/release_26_03.rst
index 031eaa657e..cd284a3ba6 100644
--- a/doc/guides/rel_notes/release_26_03.rst
+++ b/doc/guides/rel_notes/release_26_03.rst
@@ -55,6 +55,13 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Added Script for Real-time Telemetry Monitoring.**
+
+ Introduced the `dpdk-telemetry-watcher.py` script, enabling users to monitor
+ real-time telemetry statistics from running DPDK applications.
+ The tool supports customizable display options, including delta values,
+ total statistics, and single-line output for compact monitoring.
+
* **Updated AMD axgbe ethernet driver.**
* Added support for V4000 Krackan2e.
diff --git a/doc/guides/tools/telemetrywatcher.rst b/doc/guides/tools/telemetrywatcher.rst
index 251d99a085..f3be49982f 100644
--- a/doc/guides/tools/telemetrywatcher.rst
+++ b/doc/guides/tools/telemetrywatcher.rst
@@ -11,6 +11,12 @@ It wraps the ``dpdk-telemetry.py`` script to provide real-time statistics displa
Running the Tool
----------------
+The watcher tool can be run at any time, whether or not a DPDK application is currently running.
+When a DPDK application with telemetry enabled starts
+(assuming correct file-prefix and instance are specified),
+the watcher will automatically connect and begin displaying the requested statistics.
+If the DPDK application stops, the watcher will attempt to reconnect when the application restarts.
+
The tool has a number of command line options:
.. code-block:: console
diff --git a/usertools/dpdk-telemetry-watcher.py b/usertools/dpdk-telemetry-watcher.py
index eda57e5ba5..2dac325e90 100755
--- a/usertools/dpdk-telemetry-watcher.py
+++ b/usertools/dpdk-telemetry-watcher.py
@@ -85,6 +85,12 @@ def create_telemetry_process(telemetry_script, args_list):
text=True,
bufsize=1, # Line buffered
)
+
+ # Get and display the connected application name
+ if not print_connected_app(process):
+ return None
+ process.script = telemetry_script # Store script path for reference
+ process.args = args_list # Store args for reference
return process
except FileNotFoundError:
print("Error: Python interpreter or script not found", file=sys.stderr)
@@ -102,15 +108,32 @@ def query_telemetry(process, command):
command: The telemetry command to send (e.g., "/info" or "/ethdev/stats,0")
Returns:
- dict: The parsed JSON response with the command wrapper stripped,
+ (process, dict): The process handle, in case of reconnection, and the
+ parsed JSON response with the command wrapper stripped,
or None if there was an error
"""
+ # Handle case where process is None
+ if process is None:
+ return (None, None)
+
# Send the command
process.stdin.write(f"{command}\n")
process.stdin.flush()
- # Read the JSON response
+ # Read the JSON response, reconnecting if necessary
response = process.stdout.readline()
+ while not response:
+ script = process.script
+ args_list = process.args
+ process = None
+ print("Application disconnected, retrying...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(script, args_list)
+ process.stdin.write(f"{command}\n")
+ process.stdin.flush()
+ response = process.stdout.readline()
+
try:
data = json.loads(response)
# When run non-interactively, the response is wrapped with the command
@@ -119,11 +142,11 @@ def query_telemetry(process, command):
# The response should have exactly one key which is the command
if len(data) == 1:
# Extract the value, ignoring the key
- return next(iter(data.values()))
+ return (process, next(iter(data.values())))
else:
- return data
+ return (process, data)
except (json.JSONDecodeError, KeyError):
- return None
+ return (None, None)
def print_connected_app(process):
@@ -132,11 +155,12 @@ def print_connected_app(process):
Args:
process: The subprocess.Popen handle to the telemetry process
"""
- info = query_telemetry(process, "/info")
+ process, info = query_telemetry(process, "/info")
if info and "pid" in info:
app_name = get_app_name(info["pid"])
if app_name:
print(f'Connected to application: "{app_name}"')
+ return process
def expand_shortcuts(process, stat_specs):
@@ -169,7 +193,7 @@ def expand_shortcuts(process, stat_specs):
field = field_map.get(field, field)
# Get list of ethernet devices
- port_list = query_telemetry(process, "/ethdev/list")
+ process, port_list = query_telemetry(process, "/ethdev/list")
if not isinstance(port_list, list):
print("Error: Failed to get ethernet device list", file=sys.stderr)
return None
@@ -216,7 +240,7 @@ def validate_stats(process, stat_specs):
return None, None
# Query the stat once to validate it exists and is numeric
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
if not isinstance(data, dict):
print(f"Error: Command '{command}' did not return a dictionary", file=sys.stderr)
return None, None
@@ -276,7 +300,9 @@ def monitor_stats(process, args):
current_values = []
total = 0
for i, (spec, command, field) in enumerate(parsed_specs):
- data = query_telemetry(process, command)
+ process, data = query_telemetry(process, command)
+ if not data:
+ continue
current_value = data[field]
current_values.append(current_value)
@@ -387,9 +413,11 @@ def main():
# Run dpdk-telemetry.py with pipes for stdin and stdout
process = create_telemetry_process(telemetry_script, args_list)
-
- # Get and display the connected application name
- print_connected_app(process)
+ if not process:
+ print("Waiting for connection to DPDK application...", file=sys.stderr)
+ while not process:
+ time.sleep(1)
+ process = create_telemetry_process(telemetry_script, args_list)
# Monitor the requested statistics
monitor_stats(process, args)
--
2.51.0
^ permalink raw reply related [flat|nested] 41+ messages in thread
* Re: [PATCH v4 0/7] Add script for real-time telemetry monitoring
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
` (6 preceding siblings ...)
2026-02-05 15:02 ` [PATCH v4 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
@ 2026-03-31 18:10 ` Stephen Hemminger
7 siblings, 0 replies; 41+ messages in thread
From: Stephen Hemminger @ 2026-03-31 18:10 UTC (permalink / raw)
To: Bruce Richardson; +Cc: dev
On Thu, 5 Feb 2026 15:02:23 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:
> TL;DR
> ------
>
> For a quick demo, apply patches, run e.g. testpmd and then in a separate
> terminal run:
>
> ./usertools/dpdk-telemetry-watcher.py -d1T eth.tx
>
> Output, updated once per second, will be traffic rate per port e.g.:
>
> Connected to application: "dpdk-testpmd"
> Time /ethdev/stats,0.opackets /ethdev/stats,1.opackets Total
> 16:29:12 5,213,119 5,214,304 10,427,423
>
>
> Fuller details
> --------------
>
> While we have the dpdk-telemetry.py CLI app for interactive querying of
> telemetry on the commandline, and a telemetry exporter script for
> sending telemetry to external tools for real-time monitoring, we don't
> have an app that can print real-time stats for DPDK apps on the
> terminal. This patchset adds such a script, developed with the help of
> Github copilot to fill a need that I found in my testing. Submitting it
> here in the hopes that others find it of use.
>
> The script acts as a wrapper around the existing dpdk-telemetry.py
> script, and pipes the commands to that script and reads the responses,
> querying it once per second. It takes a number of flag parameters, such
> as the ones above:
> - "-d" for delta values, i.e. PPS rather than total packets
> - "-1" for single-line output, i.e. no scrolling up the screen
> - "-T" to display a total column
>
> Other flag parameters can be seen by looking at the help output.
>
> Beyond the flags, the script also takes a number of positional
> parameters, which refer to specific stats to display. These stats must
> be numeric values, and should take the form of the telemetry command to
> send, followed by a "." and the stat within the result which is to be
> tracked. As above, a stat would be e.g. "/ethdev/stats,0.opackets",
> where we send "/ethdev/stats,0" to telemetry and extract the "opackets"
> part of the result.
>
> However, specifying individual stats can be awkward, so some shortcuts
> are provided too for the common case of monitoring ethernet ports. Any
> positional arg starting with "eth" will be replaced by the set of
> equivalent values for each port, e.g. "eth.imissed" will track the
> imissed value on all ports in use in the app. The ipackets and opackets
> values, as common metrics, are also available as shortened values as
> just "rx" and "tx", so in the example above, "eth.tx" means to track the
> opackets stat for every ethdev port.
>
> Finally, the script also has reconnection support so you can leave it
> running while you start and stop your application in another terminal.
> The watcher will try and reconnect to a running instance every second.
>
> v4:
> - Updated docs following AI review
> - Converted one missed f-string to regular string
>
> v3:
> Updated following AI review
> - removed unnecessary f-string
> - added documnentation in guides/tools
> - added release note entry
>
> v2:
> - improve reconnection handling, eliminating some crashes seen in testing.
>
> Bruce Richardson (7):
> usertools: add new script to monitor telemetry on terminal
> usertools/telemetry-watcher: add displaying stats
> usertools/telemetry-watcher: add delta and timeout opts
> usertools/telemetry-watcher: add total and one-line opts
> usertools/telemetry-watcher: add thousands separator
> usertools/telemetry-watcher: add eth name shortcuts
> usertools/telemetry-watcher: support reconnection
>
> doc/guides/rel_notes/release_26_03.rst | 7 +
> doc/guides/tools/index.rst | 1 +
> doc/guides/tools/telemetrywatcher.rst | 184 +++++++++++
> usertools/dpdk-telemetry-watcher.py | 435 +++++++++++++++++++++++++
> usertools/meson.build | 1 +
> 5 files changed, 628 insertions(+)
> create mode 100644 doc/guides/tools/telemetrywatcher.rst
> create mode 100755 usertools/dpdk-telemetry-watcher.py
>
> --
> 2.51.0
>
This didn't get merged so will need to be rebased.
You may want to address these AI review comments.
Review of [PATCH v4 1-7/7] usertools: dpdk-telemetry-watcher
============================================================
Nice tool — having a continuous monitoring wrapper around
dpdk-telemetry.py is a practical addition. Patches 1-6 are
clean and well-structured. Patch 7 (reconnection support)
has several correctness issues described below.
Patch 7/7: usertools/telemetry-watcher: support reconnection
------------------------------------------------------------
Error: monitor_stats `continue` on failed query causes
IndexError on next delta iteration.
When `query_telemetry` returns (process, None), the code
does `continue`, skipping `current_values.append(...)`.
At the end of the loop body, `prev_values = current_values`
stores a shorter list. On the next iteration,
`prev_values[i]` raises IndexError for the missing indices.
Suggested fix: when data is None, append prev_values[i]
(or 0) as the current_value so the list length is preserved:
process, data = query_telemetry(process, command)
if not data:
current_values.append(prev_values[i] if i < len(prev_values) else 0)
row += "N/A".rjust(25)
continue
Error: BrokenPipeError not handled in query_telemetry.
When the DPDK application dies, the subprocess's stdin pipe
breaks. The initial `process.stdin.write()` / `.flush()`
before the reconnection loop will raise BrokenPipeError
instead of returning an empty readline(). The reconnection
logic never triggers.
Suggested fix: wrap the write+flush+readline in a
try/except (BrokenPipeError, OSError) and treat it the
same as an empty response — fall into the reconnection
loop.
Warning: old subprocess not cleaned up on reconnection.
In query_telemetry, when readline() returns empty and
reconnection begins, the old process object is replaced
without calling process.terminate() or process.wait().
The dead subprocess accumulates as a zombie. Similarly,
create_telemetry_process now calls print_connected_app
which can fail and return None, leaking the just-created
Popen object.
Suggested fix: add a small helper to clean up a process
(terminate, close pipes, wait), and call it before setting
process = None in the reconnection path. In
create_telemetry_process, if print_connected_app fails,
terminate the process before returning None.
Warning: expand_shortcuts and validate_stats lose the
reconnected process handle.
Both functions update their local `process` variable via
query_telemetry's return value, but neither returns the
(possibly new) process to the caller. If a reconnection
happens during shortcut expansion or validation, the
caller in monitor_stats still holds the old dead process
object.
Suggested fix: have expand_shortcuts and validate_stats
return the process alongside their current return values,
or restructure so monitor_stats passes process by
reference (e.g., as a mutable container).
Info: recursive call between create_telemetry_process and
print_connected_app.
create_telemetry_process calls print_connected_app, which
calls query_telemetry, which on disconnect calls
create_telemetry_process again. This indirect recursion
works in practice (Python has a high default recursion
limit and the retry loop in query_telemetry breaks the
chain), but it is fragile and hard to follow. Consider
separating the "connect" step from the "verify connection"
step to avoid the recursive dependency.
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
^ permalink raw reply [flat|nested] 41+ messages in thread
end of thread, other threads:[~2026-03-31 18:10 UTC | newest]
Thread overview: 41+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-10 16:55 [RFC PATCH 0/7] Add script for real-time telemetry monitoring Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
2025-12-10 16:55 ` [RFC PATCH 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
2025-12-11 1:09 ` [RFC PATCH 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
2025-12-11 9:10 ` Bruce Richardson
2025-12-12 5:32 ` Stephen Hemminger
2025-12-12 17:52 ` Bruce Richardson
2025-12-12 23:23 ` Stephen Hemminger
2026-01-05 17:55 ` [PATCH v2 " Bruce Richardson
2026-01-05 17:55 ` [PATCH v2 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
2026-01-05 17:56 ` [PATCH v2 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
2026-01-14 2:00 ` [PATCH v2 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
2026-01-15 19:03 ` [PATCH v3 " Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
2026-01-15 19:03 ` [PATCH v3 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
2026-01-16 6:53 ` [PATCH v3 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
2026-01-16 6:57 ` Stephen Hemminger
2026-02-05 15:02 ` [PATCH v4 " Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 1/7] usertools: add new script to monitor telemetry on terminal Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 2/7] usertools/telemetry-watcher: add displaying stats Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 3/7] usertools/telemetry-watcher: add delta and timeout opts Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 4/7] usertools/telemetry-watcher: add total and one-line opts Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 5/7] usertools/telemetry-watcher: add thousands separator Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 6/7] usertools/telemetry-watcher: add eth name shortcuts Bruce Richardson
2026-02-05 15:02 ` [PATCH v4 7/7] usertools/telemetry-watcher: support reconnection Bruce Richardson
2026-03-31 18:10 ` [PATCH v4 0/7] Add script for real-time telemetry monitoring Stephen Hemminger
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox