* [PATCH] ansible: add lucid callback plugin for clean output
@ 2025-11-21 19:44 Daniel Gomez
2025-11-25 0:07 ` Luis Chamberlain
0 siblings, 1 reply; 2+ messages in thread
From: Daniel Gomez @ 2025-11-21 19:44 UTC (permalink / raw)
To: Luis Chamberlain, Chuck Lever; +Cc: kdevops, Daniel Gomez
From: Daniel Gomez <da.gomez@samsung.com>
Add the lucid callback plugin to provide clean, minimal Ansible output
for kdevops workflows. Current Ansible output is verbose by default
making it difficult to see what matters during long-running test
operations. Lucid solves this by showing only changes and errors at
default verbosity while providing progressive detail at higher levels.
The plugin includes dynamic mode with live updates for interactive
terminals showing running tasks with spinners and elapsed time similar
to build system output. For CI/CD and piped contexts it automatically
falls back to static output. All runs generate comprehensive logs with
playbook names and timestamps allowing multiple executions to coexist
without conflicts.
Task-level output control is available via the output_verbosity variable
allowing playbooks to specify per-task visibility. Execution time
tracking highlights slow tasks exceeding configurable thresholds. Logs
always capture full verbosity independent of display settings ensuring
complete audit trails.
Integrate lucid with kdevops Kconfig system providing options for
time threshold, timestamp format, output mode (auto/static/dynamic),
and log behavior. Add defconfig fragment for quick enablement. Update
ansible.cfg template to generate callback configuration when selected.
Make lucid the default callback for new configurations.
Add documentation following Ansible's standard format covering
requirements, parameters, usage examples, and behavior notes.
Generated-by: Claude AI
Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
---
The Ansible stdout callback ecosystem is quite wide, supporting multiple
output formats and options. However, after testing a variety of them
with kdevops workflows, none fit well for our use cases. Most existing
callbacks provide verbose output that makes it difficult for users to
understand what's actually happening during playbook executions.
kdevops relies on Ansible's idempotency principle [1], meaning playbooks
are designed to reach the desired state regardless of the current state.
Users shouldn't need to worry about which tasks are running or being
skipped, the output should simply guide them to the expected result.
With this principle in mind and inspired by other projects [2] like
bitbake (from OpenEmbedded/Yocto), we can focus the default output
on what actually matters: the currently running task, a short list of
recent tasks, and elapsed time for the current operation.
This is the core principle behind lucid, a new Ansible stdout callback
plugin tailored for kdevops where less is more. Errors are always
displayed in human-readable format on stdout, while comprehensive logs
remain available for detailed inspection of playbook execution.
Lucid is designed to be flexible and evolve with community feedback.
Users who prefer different output styles can easily configure the plugin
through kdevops' Kconfig system. For example, if the default dynamic
mode with live updates doesn't suit a particular workflow, users can
switch to static scrollable output. The goal is to provide sensible
defaults that work for most kernel and kdevops testing scenarios while
allowing individual customization when needed.
By making lucid the default callback, all kdevops users benefit from
cleaner, more actionable output during their kernel development and
testing workflows.
Here's a linux workflow output demo using lucid. The video is long, but
the first few seconds already give you a sense of how the plugin works.
Jump to 3:36 to see the output logs. The video will be deleted after
7 days:
https://asciinema.org/a/fCqZ8S8F0vNBfjkA8FLDGUue5
[1] https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_intro.html#desired-state-and-idempotency
[2] https://docs.yoctoproject.org/bitbake/
---
.gitignore | 4 +
CLAUDE.md | 16 +
callback_plugins/__init__.py | 0
callback_plugins/lucid.py | 734 +++++++++++++++++++++
defconfigs/configs/lucid.config | 6 +
docs/ansible-callbacks.md | 77 +++
kconfigs/Kconfig.ansible_cfg | 148 ++++-
.../roles/ansible_cfg/templates/ansible.cfg.j2 | 18 +-
8 files changed, 1001 insertions(+), 2 deletions(-)
diff --git a/.gitignore b/.gitignore
index 7472860e..ddc19b17 100644
--- a/.gitignore
+++ b/.gitignore
@@ -26,6 +26,7 @@ extra_vars.json
.config.old
ansible.cfg
+callback_plugins/__pycache__/
scripts/kconfig/.mconf-cfg
scripts/workflows/fstests/lib/__pycache__/
@@ -131,3 +132,6 @@ terraform/oci/scripts/__pycache__/
scripts/__pycache__/
docs/contrib/kdevops_contributions*
+__pycache__/
+
+.ansible
diff --git a/CLAUDE.md b/CLAUDE.md
index 28920130..01eb9af3 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -128,6 +128,22 @@ make fix-whitespace-last-commit # Fixes commit white space damage
make mrproper # Clean everything and restart from scratch
```
+### Ansible Callbacks
+
+kdevops uses the lucid callback plugin for clean Ansible output.
+
+Enable via:
+```bash
+./scripts/kconfig/merge_config.sh -n defconfigs/configs/lucid.config
+make
+```
+
+See [docs/ansible-callbacks.md](docs/ansible-callbacks.md) for:
+- Configuration options
+- Task-level output control
+- Dynamic mode behavior
+- Logging details
+
## Key Workflows
### fstests (Filesystem Testing)
diff --git a/callback_plugins/__init__.py b/callback_plugins/__init__.py
new file mode 100644
index 00000000..e69de29b
diff --git a/callback_plugins/lucid.py b/callback_plugins/lucid.py
new file mode 100644
index 00000000..c9c1f0ab
--- /dev/null
+++ b/callback_plugins/lucid.py
@@ -0,0 +1,734 @@
+#!/usr/bin/env python3
+"""
+Lucid Ansible Callback Plugin
+
+A modern stdout callback plugin providing:
+- Clean, minimal output with progressive verbosity levels
+- Task-level output control via output_verbosity variable
+- Comprehensive logging (always max verbosity)
+- Dynamic terminal display (Yocto/BitBake style) for interactive use
+- Static output for CI/CD and non-interactive terminals
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+import time
+import threading
+from datetime import datetime
+from typing import Dict, List, Tuple, Optional, Any
+from collections import deque
+
+from ansible.plugins.callback import CallbackBase
+from ansible import constants as C
+
+DOCUMENTATION = '''
+ name: lucid
+ type: stdout
+ short_description: Clean, minimal Ansible output with dynamic display
+ version_added: "2.10"
+ description:
+ - Provides clean, minimal output by default
+ - Progressive verbosity levels (-v, -vv, -vvv)
+ - Task-level output control via output_verbosity variable
+ - Comprehensive logging independent of display verbosity
+ - Dynamic live display for interactive terminals
+ - Static output for CI/CD environments
+ requirements:
+ - Ansible 2.10+
+ - Python 3.8+
+ options:
+ time_threshold:
+ description: Only show task times if duration exceeds this value (seconds)
+ default: 3.0
+ type: float
+ ini:
+ - section: callback_lucid
+ key: time_threshold
+ show_all_times:
+ description: Show all task times regardless of threshold
+ default: False
+ type: bool
+ ini:
+ - section: callback_lucid
+ key: show_all_times
+ show_timestamps:
+ description: Show timestamps in stdout
+ default: False
+ type: bool
+ ini:
+ - section: callback_lucid
+ key: show_timestamps
+ timestamp_format:
+ description: Timestamp format (time, datetime, iso8601)
+ default: time
+ type: str
+ ini:
+ - section: callback_lucid
+ key: timestamp_format
+ output_mode:
+ description: Output display mode (auto, static, dynamic)
+ default: auto
+ type: str
+ ini:
+ - section: callback_lucid
+ key: output_mode
+ choices: ['auto', 'static', 'dynamic']
+ log_file:
+ description: >
+ Path to log file (empty for auto-detect).
+ Auto-detect tries: .ansible/logs/, ~/.ansible/logs/, /var/log/ansible/
+ default: ''
+ type: str
+ ini:
+ - section: callback_lucid
+ key: log_file
+ log_append:
+ description: Append to existing log file instead of creating new
+ default: False
+ type: bool
+ ini:
+ - section: callback_lucid
+ key: log_append
+'''
+
+
+class CallbackModule(CallbackBase):
+ """
+ Lucid callback plugin for clean, minimal Ansible output
+ with dynamic terminal display and comprehensive logging.
+ """
+
+ CALLBACK_VERSION = 2.0
+ CALLBACK_TYPE = 'stdout'
+ CALLBACK_NAME = 'lucid'
+
+ # Spinner animation frames
+ SPINNER_FRAMES = ['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏']
+
+ def __init__(self):
+ super(CallbackModule, self).__init__()
+
+ # State tracking
+ self.running_tasks: Dict[Tuple[str, str], Dict[str, Any]] = {} # (host, task_uuid) -> task_info
+ self.completed_tasks = deque(maxlen=3) # Keep last 3 completed
+ self.failed_tasks: List[Tuple] = []
+ self.current_task_name: str = ''
+ self.current_task_hosts: List[str] = []
+
+ # Dynamic display state
+ self.display_lines = 0
+ self.last_update = 0.0
+ self.spinner_index = 0
+ self.is_interactive = False
+ self.dynamic_mode = False
+ self.update_thread: Optional[threading.Thread] = None
+ self.update_thread_stop: Optional[threading.Event] = None
+ self.task_lock = threading.Lock()
+
+ # Will be set in set_options()
+ self.time_threshold = 3.0
+ self.show_all_times = False
+ self.show_timestamps = False
+ self.timestamp_format = 'time'
+ self.output_mode = 'auto'
+ self.log_file_path: Optional[str] = None
+ self.log_append = False
+ self.log_timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
+ self.log_write_failed = False
+
+ def set_options(self, task_keys=None, var_options=None, direct=None):
+ """Set plugin options from ansible.cfg"""
+ super(CallbackModule, self).set_options(task_keys=task_keys, var_options=var_options, direct=direct)
+
+ # Load configuration
+ self.time_threshold = self.get_option('time_threshold')
+ self.show_all_times = self.get_option('show_all_times')
+ self.show_timestamps = self.get_option('show_timestamps')
+ self.timestamp_format = self.get_option('timestamp_format')
+ self.output_mode = self.get_option('output_mode')
+ self.log_file_path = self.get_option('log_file')
+ self.log_append = self.get_option('log_append')
+
+ # Determine display mode based on configuration
+ self.is_interactive = self._detect_interactive()
+ if self.output_mode == 'static':
+ self.dynamic_mode = False
+ elif self.output_mode == 'dynamic':
+ self.dynamic_mode = True
+ else: # 'auto' or any other value
+ self.dynamic_mode = self.is_interactive
+
+ # Initialize logging
+ self._init_log_file()
+
+ # Start update thread for dynamic mode
+ if self.dynamic_mode:
+ self._start_update_thread()
+
+ def _detect_interactive(self) -> bool:
+ """Detect if running in interactive terminal"""
+ return (
+ sys.stdout.isatty() and
+ sys.stderr.isatty() and
+ os.getenv('TERM') != 'dumb' and
+ os.getenv('CI') is None and
+ os.getenv('JENKINS_HOME') is None and
+ os.getenv('GITHUB_ACTIONS') is None and
+ os.getenv('GITLAB_CI') is None
+ )
+
+ def _init_log_file(self):
+ """Initialize log file if custom path provided, otherwise defer until playbook name known"""
+ if self.log_file_path:
+ # Custom path provided, initialize immediately
+ mode = 'a' if self.log_append else 'w'
+ try:
+ with open(self.log_file_path, mode) as f:
+ f.write(f"=== Ansible Playbook Log Started: {datetime.now().isoformat()} ===\n\n")
+ except (PermissionError, OSError) as e:
+ self._display.warning(f"Could not initialize log file {self.log_file_path}: {e}")
+ self.log_file_path = None
+ # If no custom path, defer log file creation until we have playbook name
+
+ def _create_log_file(self, playbook_name: str):
+ """Create log file with playbook name and timestamp"""
+ if self.log_file_path:
+ # Already initialized with custom path
+ return
+
+ # Remove extension from playbook name
+ playbook_base = os.path.splitext(playbook_name)[0]
+
+ # Try default locations in order (project dir first, then user home, then system)
+ attempts = [
+ f'.ansible/logs/{playbook_base}-{self.log_timestamp}.log',
+ os.path.expanduser(f'~/.ansible/logs/{playbook_base}-{self.log_timestamp}.log'),
+ f'/var/log/ansible/{playbook_base}-{self.log_timestamp}.log'
+ ]
+
+ for path in attempts:
+ try:
+ os.makedirs(os.path.dirname(path), exist_ok=True)
+ # Test write access
+ with open(path, 'a'):
+ pass
+ self.log_file_path = path
+ break
+ except (PermissionError, OSError):
+ continue
+
+ if self.log_file_path:
+ mode = 'a' if self.log_append else 'w'
+ try:
+ with open(self.log_file_path, mode) as f:
+ f.write(f"=== Ansible Playbook Log Started: {datetime.now().isoformat()} ===\n")
+ f.write(f"=== Playbook: {playbook_name} ===\n\n")
+ except (PermissionError, OSError) as e:
+ self._display.warning(f"Could not initialize log file {self.log_file_path}: {e}")
+ self.log_file_path = None
+
+ def _write_to_log(self, message: str):
+ """Write to log file with timestamp (always max verbosity)"""
+ if self.log_file_path:
+ timestamp = datetime.now().isoformat()
+ try:
+ with open(self.log_file_path, 'a') as f:
+ f.write(f"[{timestamp}] {message}\n")
+ except (PermissionError, OSError) as e:
+ # Warn once on first failure, then disable logging
+ if not self.log_write_failed:
+ self._display.warning(f"Log write failed, disabling logging: {e}")
+ self.log_write_failed = True
+ self.log_file_path = None
+
+ def _start_update_thread(self):
+ """Start background thread for live display updates"""
+ self.update_thread_stop = threading.Event()
+ self.update_thread = threading.Thread(
+ target=self._update_loop,
+ daemon=True
+ )
+ self.update_thread.start()
+
+ def _update_loop(self):
+ """Update display every 0.5 seconds in dynamic mode"""
+ while not self.update_thread_stop.is_set():
+ if self.running_tasks:
+ self._redraw_display()
+ time.sleep(0.5)
+
+ def _should_display_output(self, result, status: str) -> bool:
+ """
+ Determine if task output should be shown based on verbosity.
+
+ Rules:
+ - Changed tasks: ALWAYS show
+ - Failed tasks: ALWAYS show
+ - OK tasks: Show if current_verbosity >= task's output_verbosity
+ - Skipped tasks: Only at -v or higher
+ """
+ # Changed and failed always show
+ if status == 'changed' or status == 'failed' or status == 'unreachable':
+ return True
+
+ # Skipped only at -v
+ if status == 'skipped':
+ return self._display.verbosity >= 1
+
+ # Get task verbosity setting (default is 1)
+ task_verbosity = 1
+ if hasattr(result, '_task_fields'):
+ task_vars = result._task_fields.get('vars', {})
+ task_verbosity = task_vars.get('output_verbosity', 1)
+ elif hasattr(result, '_task'):
+ task_vars = getattr(result._task, 'vars', {})
+ task_verbosity = task_vars.get('output_verbosity', 1)
+
+ # Compare with current verbosity
+ current_verbosity = self._display.verbosity
+
+ return current_verbosity >= task_verbosity
+
+ def _format_timestamp(self) -> str:
+ """Format timestamp based on configuration"""
+ now = datetime.now()
+ if self.timestamp_format == 'time':
+ return now.strftime('%H:%M:%S')
+ elif self.timestamp_format == 'datetime':
+ return now.strftime('%Y-%m-%d %H:%M:%S')
+ elif self.timestamp_format == 'iso8601':
+ return now.isoformat()
+ else:
+ return now.strftime('%H:%M:%S')
+
+ def _format_duration(self, seconds: float) -> str:
+ """Format duration as human readable"""
+ if seconds < 60:
+ return f"{seconds:.1f}s"
+ elif seconds < 3600:
+ mins = int(seconds // 60)
+ secs = int(seconds % 60)
+ return f"{mins}m {secs}s"
+ else:
+ hours = int(seconds // 3600)
+ mins = int((seconds % 3600) // 60)
+ return f"{hours}h {mins}m"
+
+ def _display_message(self, message: str, color=None):
+ """Display message with optional timestamp and color"""
+ if self.show_timestamps:
+ timestamp = self._format_timestamp()
+ message = f"[{timestamp}] {message}"
+
+ if color:
+ self._display.display(message, color=color)
+ else:
+ self._display.display(message)
+
+ # ========================================================================
+ # Ansible v2 Callback Methods
+ # ========================================================================
+
+ def v2_playbook_on_start(self, playbook):
+ """Playbook started"""
+ playbook_name = os.path.basename(playbook._file_name)
+
+ # Create log file now that we have playbook name
+ self._create_log_file(playbook_name)
+
+ msg = f"PLAYBOOK: {playbook_name}"
+ self._display_message(msg, C.COLOR_HIGHLIGHT)
+ self._write_to_log(msg)
+
+ def v2_playbook_on_play_start(self, play):
+ """Play started"""
+ name = play.get_name().strip()
+ hosts = play.hosts
+ if isinstance(hosts, list):
+ hosts_str = ', '.join(hosts[:3])
+ if len(hosts) > 3:
+ hosts_str += f' (+{len(hosts) - 3} more)'
+ else:
+ hosts_str = str(hosts)
+
+ msg = f"\nPLAY: {name} [{hosts_str}]"
+ self._display_message(msg, C.COLOR_HIGHLIGHT)
+ self._write_to_log(msg)
+
+ def v2_playbook_on_task_start(self, task, is_conditional):
+ """Task started"""
+ self.current_task_name = task.get_name().strip()
+ self.current_task_hosts = []
+
+ # In static mode, print immediately (compact - no leading newline)
+ if not self.dynamic_mode:
+ msg = f"TASK: {self.current_task_name}"
+ self._display_message(msg, C.COLOR_HIGHLIGHT)
+
+ self._write_to_log(f"TASK: {self.current_task_name}")
+
+ def v2_runner_on_start(self, host, task):
+ """Task started on a host (for dynamic tracking)"""
+ key = (host.name, task._uuid)
+ with self.task_lock:
+ self.running_tasks[key] = {
+ 'start_time': time.time(),
+ 'host': host.name,
+ 'task_name': task.get_name().strip()
+ }
+
+ if host.name not in self.current_task_hosts:
+ self.current_task_hosts.append(host.name)
+
+ if self.dynamic_mode:
+ self._redraw_display()
+
+ def v2_runner_on_ok(self, result):
+ """Task succeeded"""
+ changed = result._result.get('changed', False)
+ status = 'changed' if changed else 'ok'
+ self._handle_result(result, status)
+
+ def v2_runner_on_failed(self, result, ignore_errors=False):
+ """Task failed"""
+ self._handle_result(result, 'failed', ignore_errors=ignore_errors)
+
+ def v2_runner_on_skipped(self, result):
+ """Task skipped"""
+ self._handle_result(result, 'skipped')
+
+ def v2_runner_on_unreachable(self, result):
+ """Host unreachable"""
+ self._handle_result(result, 'unreachable')
+
+ def v2_playbook_on_stats(self, stats):
+ """Final summary"""
+ # Stop update thread
+ if self.update_thread_stop:
+ self.update_thread_stop.set()
+ if self.update_thread:
+ self.update_thread.join(timeout=1.0)
+
+ # Clear dynamic display if active
+ if self.dynamic_mode and self.display_lines > 0:
+ self._clear_display()
+
+ # Print recap
+ self._display_recap(stats)
+
+ # Log file footer
+ self._write_to_log(f"\n=== Playbook Completed: {datetime.now().isoformat()} ===")
+
+ if self.log_file_path:
+ self._display.display(f"\nLog: {self.log_file_path}")
+
+ # ========================================================================
+ # Result Handling
+ # ========================================================================
+
+ def _handle_result(self, result, status: str, ignore_errors: bool = False):
+ """Unified result handler for all task outcomes"""
+ host = result._host.name
+ task_uuid = result._task._uuid
+ key = (host, task_uuid)
+
+ # Calculate duration
+ with self.task_lock:
+ task_info = self.running_tasks.pop(key, None)
+ start_time = task_info['start_time'] if task_info else time.time()
+ duration = time.time() - start_time
+
+ # Store result data
+ result_data = {
+ 'result': result,
+ 'status': status,
+ 'duration': duration,
+ 'host': host,
+ 'task_name': result._task.get_name().strip(),
+ 'ignore_errors': ignore_errors
+ }
+
+ # Track failures
+ if status == 'failed' and not ignore_errors:
+ self.failed_tasks.append(result_data)
+ else:
+ # Add to completed tasks (last 3 for dynamic mode)
+ self.completed_tasks.append(result_data)
+
+ # Log everything (max verbosity)
+ self._log_result(result, status, duration)
+
+ # Display based on mode
+ if self.dynamic_mode:
+ # Dynamic mode will update on next refresh
+ # But if failed and not ignoring, freeze display
+ if status == 'failed' and not ignore_errors:
+ self._freeze_and_show_failure(result_data)
+ else:
+ # Static mode - display immediately
+ self._display_result_static(result, status, duration)
+
+ def _display_result_static(self, result, status: str, duration: float):
+ """Display result in static mode"""
+ host = result._host.name
+
+ # Determine if we should show this result at all
+ # At verbosity 0: only show changed, failed, unreachable
+ # At verbosity 1+: show based on output_verbosity
+ if status == 'ok' and self._display.verbosity < 1:
+ # Hide OK tasks at verbosity 0
+ return
+
+ # Determine if we should show output
+ show_output = self._should_display_output(result, status)
+
+ # Skipped tasks only at verbosity 1+
+ if status == 'skipped' and self._display.verbosity < 1:
+ return
+
+ # Format status line
+ symbols = {
+ 'ok': '✓',
+ 'changed': '⚡',
+ 'failed': '✗',
+ 'skipped': '⊘',
+ 'unreachable': '⚠'
+ }
+
+ colors = {
+ 'ok': C.COLOR_OK,
+ 'changed': C.COLOR_CHANGED,
+ 'failed': C.COLOR_ERROR,
+ 'skipped': C.COLOR_SKIP,
+ 'unreachable': C.COLOR_UNREACHABLE
+ }
+
+ symbol = symbols.get(status, '?')
+ color = colors.get(status, C.COLOR_OK)
+
+ # Format time
+ time_str = ''
+ if (duration > self.time_threshold or self.show_all_times) and status not in ['skipped']:
+ time_str = f" ({self._format_duration(duration)})"
+
+ # Status line with unicode spacing (using regular spaces, not braille)
+ status_line = f" {symbol} [{host}]{time_str}"
+ self._display_message(status_line, color)
+
+ # Show output if conditions met
+ if show_output:
+ self._display_output(result)
+
+ def _display_output(self, result):
+ """Display stdout/stderr/msg from task result"""
+ output = []
+ res = result._result
+
+ # stdout
+ if 'stdout' in res and res['stdout']:
+ output.append(f"\nSTDOUT:\n{res['stdout']}")
+
+ # stderr
+ if 'stderr' in res and res['stderr']:
+ output.append(f"\nSTDERR:\n{res['stderr']}")
+
+ # msg (only if no stdout)
+ if 'msg' in res and res['msg'] and 'stdout' not in res:
+ msg_text = res['msg']
+ # Handle lists/dicts in msg
+ if isinstance(msg_text, (list, dict)):
+ import json
+ msg_text = json.dumps(msg_text, indent=2)
+ output.append(f"\nMSG:\n{msg_text}")
+
+ if output:
+ self._display.display(''.join(output))
+
+ def _log_result(self, result, status: str, duration: float):
+ """Write result to log file (always max verbosity)"""
+ host = result._host.name
+ res = result._result
+
+ # Status line
+ symbols = {
+ 'ok': '✓',
+ 'changed': '⚡',
+ 'failed': '✗',
+ 'skipped': '⊘',
+ 'unreachable': '⚠'
+ }
+
+ symbol = symbols.get(status, '?')
+ time_str = self._format_duration(duration)
+ log_line = f" {symbol} [{host}] ({time_str})"
+ self._write_to_log(log_line)
+
+ # Always log full output
+ if 'stdout' in res and res['stdout']:
+ self._write_to_log(f"\nSTDOUT:\n{res['stdout']}\n")
+
+ if 'stderr' in res and res['stderr']:
+ self._write_to_log(f"\nSTDERR:\n{res['stderr']}\n")
+
+ if 'msg' in res and res['msg']:
+ msg_text = res['msg']
+ if isinstance(msg_text, (list, dict)):
+ import json
+ msg_text = json.dumps(msg_text, indent=2)
+ self._write_to_log(f"\nMSG:\n{msg_text}\n")
+
+ if status == 'failed' and 'exception' in res:
+ self._write_to_log(f"\nEXCEPTION:\n{res['exception']}\n")
+
+ def _display_recap(self, stats):
+ """Display final statistics"""
+ self._display_message("\nPLAY RECAP", C.COLOR_HIGHLIGHT)
+
+ hosts = sorted(stats.processed.keys())
+ for host in hosts:
+ summary = stats.summarize(host)
+
+ # More compact format
+ msg = (
+ f"{host:25s} : "
+ f"ok={summary['ok']:<3d} "
+ f"changed={summary['changed']:<3d} "
+ f"unreachable={summary['unreachable']:<3d} "
+ f"failed={summary['failures']:<3d} "
+ f"skipped={summary['skipped']:<3d}"
+ )
+
+ # Color based on status
+ if summary['failures'] > 0 or summary['unreachable'] > 0:
+ color = C.COLOR_ERROR
+ elif summary['changed'] > 0:
+ color = C.COLOR_CHANGED
+ else:
+ color = C.COLOR_OK
+
+ self._display_message(msg, color)
+
+ # ========================================================================
+ # Dynamic Mode Display
+ # ========================================================================
+
+ def _redraw_display(self):
+ """Redraw entire display in dynamic mode"""
+ # Throttle updates (max once per 0.1s)
+ now = time.time()
+ if now - self.last_update < 0.1:
+ return
+ self.last_update = now
+
+ # Increment spinner
+ self.spinner_index = (self.spinner_index + 1) % len(self.SPINNER_FRAMES)
+
+ # Clear previous display
+ self._clear_display()
+
+ # Build new display
+ lines = []
+
+ # Task header
+ if self.current_task_name:
+ lines.append(f"TASK: {self.current_task_name}")
+ lines.append("")
+
+ # Running tasks (first)
+ # Create snapshot under lock to avoid holding lock during iteration
+ with self.task_lock:
+ tasks_snapshot = list(self.running_tasks.items())
+
+ if tasks_snapshot:
+ running_count = len(tasks_snapshot)
+ total_hosts = len(self.current_task_hosts)
+ lines.append(f"Running {running_count}/{total_hosts} host(s):")
+
+ spinner = self.SPINNER_FRAMES[self.spinner_index]
+
+ for (host, task_uuid), task_info in sorted(tasks_snapshot):
+ elapsed = time.time() - task_info['start_time']
+ duration_str = self._format_duration(elapsed)
+
+ # Show spinner for tasks > 1 second
+ if elapsed > 1.0:
+ lines.append(f" ├─ [{host}] [{spinner}] {duration_str}")
+ else:
+ lines.append(f" ├─ [{host}] {duration_str}")
+
+ lines.append("")
+
+ # Recently completed (after running tasks)
+ if self.completed_tasks:
+ lines.append("Recent:")
+ for task_data in self.completed_tasks:
+ status = task_data['status']
+ host = task_data['host']
+ duration = task_data['duration']
+ task_name = task_data.get('task_name', 'Unknown task')
+
+ symbols = {'ok': '✓', 'changed': '⚡', 'skipped': '⊘'}
+ symbol = symbols.get(status, '✓')
+
+ time_str = self._format_duration(duration)
+ # Truncate task name if too long
+ max_task_len = 50
+ if len(task_name) > max_task_len:
+ task_name = task_name[:max_task_len-3] + '...'
+ lines.append(f" {symbol} {task_name} [{host}] ({time_str})")
+ lines.append("")
+
+ # Display all lines
+ if lines:
+ output = '\n'.join(lines)
+ # Write directly to avoid extra newlines
+ sys.stdout.write(output)
+ sys.stdout.flush()
+ # Count actual lines printed (number of newlines + 1 for the last line)
+ self.display_lines = output.count('\n') + 1
+ else:
+ self.display_lines = 0
+
+ def _clear_display(self):
+ """Clear dynamic display using ANSI escape codes"""
+ if self.display_lines > 0:
+ # Clear current line first
+ sys.stdout.write('\r\033[2K')
+ # Move up and clear remaining lines
+ for _ in range(self.display_lines - 1):
+ sys.stdout.write('\033[1A') # Move cursor up one line
+ sys.stdout.write('\033[2K') # Clear entire line
+ sys.stdout.flush()
+ self.display_lines = 0
+
+ def _freeze_and_show_failure(self, result_data):
+ """Freeze display and show failure in dynamic mode"""
+ # Clear dynamic display
+ if self.display_lines > 0:
+ self._clear_display()
+
+ # Show failure in static format
+ result = result_data['result']
+ status = result_data['status']
+ duration = result_data['duration']
+
+ # Print task name if not already shown
+ msg = f"TASK: {self.current_task_name}"
+ self._display_message(msg, C.COLOR_HIGHLIGHT)
+
+ # Show failure
+ self._display_result_static(result, status, duration)
+
+ # Disable dynamic mode for rest of playbook
+ self.dynamic_mode = False
+ if self.update_thread_stop:
+ self.update_thread_stop.set()
+
+ def __del__(self):
+ """Cleanup when plugin is destroyed"""
+ if self.update_thread_stop:
+ self.update_thread_stop.set()
+ if self.update_thread:
+ self.update_thread.join(timeout=1.0)
diff --git a/defconfigs/configs/lucid.config b/defconfigs/configs/lucid.config
new file mode 100644
index 00000000..8dde6f3e
--- /dev/null
+++ b/defconfigs/configs/lucid.config
@@ -0,0 +1,6 @@
+CONFIG_ANSIBLE_CFG_CALLBACK_PLUGIN_LUCID=y
+CONFIG_ANSIBLE_CFG_LUCID_TIME_THRESHOLD="3.0"
+CONFIG_ANSIBLE_CFG_LUCID_SHOW_ALL_TIMES=n
+CONFIG_ANSIBLE_CFG_LUCID_SHOW_TIMESTAMPS=n
+CONFIG_ANSIBLE_CFG_LUCID_OUTPUT_MODE_AUTO=y
+CONFIG_ANSIBLE_CFG_LUCID_LOG_APPEND=n
diff --git a/docs/ansible-callbacks.md b/docs/ansible-callbacks.md
new file mode 100644
index 00000000..90a4ef0e
--- /dev/null
+++ b/docs/ansible-callbacks.md
@@ -0,0 +1,77 @@
+# lucid callback
+
+Clean, minimal Ansible output with progressive verbosity levels and optional dynamic terminal display.
+
+## Requirements
+
+- Ansible 2.10+
+- Python 3.8+
+- Set as stdout callback in ansible.cfg
+
+## Parameters
+
+| Parameter | Choices/Defaults | Configuration | Comments |
+|-----------|------------------|---------------|----------|
+| time_threshold | Default: 3.0 | ini: [callback_lucid] time_threshold | Only show task times if duration exceeds this value (seconds) |
+| show_all_times | Choices: true/false<br>Default: false | ini: [callback_lucid] show_all_times | Show all task times regardless of threshold |
+| show_timestamps | Choices: true/false<br>Default: false | ini: [callback_lucid] show_timestamps | Show timestamps in output |
+| timestamp_format | Choices: time/datetime/iso8601<br>Default: time | ini: [callback_lucid] timestamp_format | Timestamp format when show_timestamps is enabled |
+| output_mode | Choices: auto/static/dynamic<br>Default: auto | ini: [callback_lucid] output_mode | Display mode: auto detects terminal, static for CI/CD, dynamic for live updates |
+| log_file | Default: (empty) | ini: [callback_lucid] log_file | Path to log file. Empty uses auto-detect: .ansible/logs/, ~/.ansible/logs/, /var/log/ansible/ |
+| log_append | Choices: true/false<br>Default: false | ini: [callback_lucid] log_append | Append to existing log file instead of creating new timestamped file |
+
+## Notes
+
+- Only shows changed and failed tasks by default. Use `-v` to show all tasks.
+- Task-level control available via `output_verbosity` variable (0=always show, 1=show at -v, 2=show at -vv).
+- Dynamic mode provides live updates with spinner when running in interactive terminals.
+- Automatically detects CI/CD environments (GitHub Actions, Jenkins, GitLab CI) and uses static mode.
+- Logs are always full verbosity regardless of display verbosity.
+- Log files include playbook name and timestamp: `<playbook>-YYYY-MM-DD_HH-MM-SS.log`
+- Compatible with ansible.posix.profile_tasks callback.
+
+## Examples
+
+### Enable in kdevops
+
+```bash
+./scripts/kconfig/merge_config.sh -n defconfigs/configs/lucid.config
+make
+```
+
+### Task-level output control
+
+```yaml
+- name: Always visible task
+ debug:
+ msg: "Shown at all verbosity levels"
+ vars:
+ output_verbosity: 0
+
+- name: Verbose task
+ debug:
+ msg: "Shown with -v flag"
+ vars:
+ output_verbosity: 1
+
+- name: Debug task
+ debug:
+ msg: "Shown with -vv flag"
+ vars:
+ output_verbosity: 2
+```
+
+### Force static mode
+
+```bash
+# Via environment
+TERM=dumb ansible-playbook playbooks/test.yml
+```
+
+## See Also
+
+Other Ansible stdout callback plugins integrated in kdevops:
+
+- [community.general.diy callback](https://docs.ansible.com/projects/ansible/latest/collections/community/general/diy_callback.html)
+- [community.general.dense callback](https://docs.ansible.com/projects/ansible/latest/collections/community/general/dense_callback.html)
+- [ansible.posix.debug callback](https://docs.ansible.com/projects/ansible/latest/collections/ansible/posix/debug_callback.html)
diff --git a/kconfigs/Kconfig.ansible_cfg b/kconfigs/Kconfig.ansible_cfg
index e3fd02f1..388631e6 100644
--- a/kconfigs/Kconfig.ansible_cfg
+++ b/kconfigs/Kconfig.ansible_cfg
@@ -56,9 +56,26 @@ endif # ANSIBLE_CFG_FILE_CUSTOM
menu "Ansible Callback Plugin Configuration"
choice
prompt "Ansible Callback Plugin"
- default ANSIBLE_CFG_CALLBACK_PLUGIN_DENSE if !ANSIBLE_CFG_CALLBACK_PLUGIN_SET_BY_CLI
+ default ANSIBLE_CFG_CALLBACK_PLUGIN_LUCID if !ANSIBLE_CFG_CALLBACK_PLUGIN_SET_BY_CLI
default ANSIBLE_CFG_CALLBACK_PLUGIN_CUSTOM if ANSIBLE_CFG_CALLBACK_PLUGIN_SET_BY_CLI
+config ANSIBLE_CFG_CALLBACK_PLUGIN_LUCID
+ bool "Ansible Lucid Callback Plugin (recommended)"
+ help
+ Lucid: Modern, clean Ansible output with progressive verbosity levels.
+
+ Features:
+ - Clean, minimal output by default (shows only changes and errors)
+ - Progressive verbosity: -v shows all tasks, -vv shows debug tasks
+ - Dynamic live display for interactive terminals (BitBake/Yocto style)
+ - Static output for CI/CD and piped output
+ - Task-level output control via output_verbosity variable
+ - Comprehensive logging (always max verbosity, independent of display)
+ - Time tracking for tasks > 3 seconds
+
+ Lucid is designed by kernel developers for kernel testing workflows
+ where human-readable output and silent operation are essential.
+
config ANSIBLE_CFG_CALLBACK_PLUGIN_DEBUG
bool "Ansible Debug Callback Plugin"
help
@@ -93,6 +110,7 @@ endif # ANSIBLE_CFG_CALLBACK_PLUGIN_CUSTOM
config ANSIBLE_CFG_CALLBACK_PLUGIN_STRING
string
output yaml
+ default "lucid" if ANSIBLE_CFG_CALLBACK_PLUGIN_LUCID
default "debug" if ANSIBLE_CFG_CALLBACK_PLUGIN_DEBUG
default "dense" if ANSIBLE_CFG_CALLBACK_PLUGIN_DENSE
default "community.general.diy" if ANSIBLE_CFG_CALLBACK_PLUGIN_DIY
@@ -170,6 +188,134 @@ config ANSIBLE_CFG_CALLBACK_PLUGIN_PROFILE_TASKS
and generates a "TASKS RECAP" section showing execution times.
https://docs.ansible.com/ansible/latest/collections/ansible/posix/profile_tasks_callback.html
+
+if ANSIBLE_CFG_CALLBACK_PLUGIN_LUCID
+
+menu "Lucid Callback Configuration"
+
+config ANSIBLE_CFG_LUCID_TIME_THRESHOLD
+ string "Time threshold for displaying task durations (seconds)"
+ output yaml
+ default "3.0"
+ help
+ Only show task execution times if the duration exceeds this threshold.
+ This keeps output clean by hiding times for quick tasks.
+
+ Default: 3.0 seconds
+
+ Set to 0 to show all task times.
+
+config ANSIBLE_CFG_LUCID_SHOW_ALL_TIMES
+ bool "Show all task times regardless of threshold"
+ output yaml
+ default n
+ help
+ When enabled, show execution times for all tasks regardless of the
+ time_threshold setting. Useful for performance analysis.
+
+config ANSIBLE_CFG_LUCID_SHOW_TIMESTAMPS
+ bool "Show timestamps in output"
+ output yaml
+ default n
+ help
+ Add timestamps to console output. Timestamps are always included
+ in log files regardless of this setting.
+
+choice
+ prompt "Timestamp format"
+ default ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_TIME
+ depends on ANSIBLE_CFG_LUCID_SHOW_TIMESTAMPS
+
+config ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_TIME
+ bool "Time only (HH:MM:SS)"
+ help
+ Format: [14:30:45]
+
+config ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_DATETIME
+ bool "Date and time (YYYY-MM-DD HH:MM:SS)"
+ help
+ Format: [2025-11-13 14:30:45]
+
+config ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_ISO8601
+ bool "ISO 8601 (YYYY-MM-DDTHH:MM:SS.mmmmm)"
+ help
+ Format: [2025-11-13T14:30:45.123456]
+
+endchoice
+
+config ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_STRING
+ string
+ output yaml
+ default "time" if ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_TIME
+ default "datetime" if ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_DATETIME
+ default "iso8601" if ANSIBLE_CFG_LUCID_TIMESTAMP_FORMAT_ISO8601
+
+choice
+ prompt "Output display mode"
+ default ANSIBLE_CFG_LUCID_OUTPUT_MODE_AUTO
+
+config ANSIBLE_CFG_LUCID_OUTPUT_MODE_AUTO
+ bool "Auto (dynamic if interactive, static otherwise)"
+ help
+ Automatically select output mode based on terminal detection.
+
+ Uses dynamic live updating display (similar to BitBake/Yocto) when
+ running in an interactive terminal with spinner animations, live
+ task timers, and parallel task visualization.
+
+ Automatically falls back to traditional static output for CI/CD,
+ piped output, and non-TTY contexts.
+
+ This is the recommended default for most use cases.
+
+config ANSIBLE_CFG_LUCID_OUTPUT_MODE_STATIC
+ bool "Static (always use traditional output)"
+ help
+ Force static output mode regardless of terminal type.
+
+ Always use traditional static output with no live updates or
+ animations. Each task result is printed as it completes.
+
+ Useful for users who prefer traditional Ansible output or when
+ dynamic mode causes issues with specific terminal emulators.
+
+config ANSIBLE_CFG_LUCID_OUTPUT_MODE_DYNAMIC
+ bool "Dynamic (always attempt live display)"
+ help
+ Force dynamic output mode even in non-interactive contexts.
+
+ Always attempt to use live updating display with ANSI escape codes
+ even if terminal detection suggests static mode should be used.
+
+ Warning: May produce garbled output in CI/CD, piped contexts, or
+ terminals that don't support ANSI escape sequences. Only use this
+ if you know your environment supports dynamic output but the
+ automatic detection fails.
+
+endchoice
+
+config ANSIBLE_CFG_LUCID_OUTPUT_MODE_STRING
+ string
+ output yaml
+ default "auto" if ANSIBLE_CFG_LUCID_OUTPUT_MODE_AUTO
+ default "static" if ANSIBLE_CFG_LUCID_OUTPUT_MODE_STATIC
+ default "dynamic" if ANSIBLE_CFG_LUCID_OUTPUT_MODE_DYNAMIC
+
+config ANSIBLE_CFG_LUCID_LOG_APPEND
+ bool "Append to existing log file instead of creating new"
+ output yaml
+ default n
+ help
+ When enabled, append to existing log file. When disabled (default),
+ create a new timestamped log file for each playbook run.
+
+ Log file location is auto-detected (tries .ansible/logs/ in project
+ directory first, then ~/.ansible/logs/, then /var/log/ansible/).
+
+endmenu
+
+endif # ANSIBLE_CFG_CALLBACK_PLUGIN_LUCID
+
endmenu
config ANSIBLE_CFG_DEPRECATION_WARNINGS
diff --git a/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2 b/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
index deb1a559..b9ec8cea 100644
--- a/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
+++ b/playbooks/roles/ansible_cfg/templates/ansible.cfg.j2
@@ -1,6 +1,10 @@
[defaults]
+{% if ansible_cfg_callback_plugin_string == 'lucid' %}
+callback_plugins = ./callback_plugins
+{% endif %}
{% if ansible_cfg_callback_plugin_profile_tasks %}
-callbacks_enabled = ansible.posix.profile_tasks, {{ ansible_cfg_callback_plugin_string }}
+callbacks_enabled = ansible.posix.profile_tasks{% if ansible_cfg_callback_plugin_string == 'lucid' %}, {{ ansible_cfg_callback_plugin_string }}{% endif %}
+
{% endif %}
deprecation_warnings = {{ ansible_cfg_deprecation_warnings }}
stdout_callback = {{ ansible_cfg_callback_plugin_string }}
@@ -47,6 +51,18 @@ playbook_on_stats_msg_color = bright green
[callback_profile_tasks]
summary_only = true
{% endif %}
+{% if ansible_cfg_callback_plugin_string == 'lucid' %}
+
+[callback_lucid]
+time_threshold = {{ ansible_cfg_lucid_time_threshold | default('3.0') }}
+show_all_times = {{ ansible_cfg_lucid_show_all_times | default('False') }}
+show_timestamps = {{ ansible_cfg_lucid_show_timestamps | default('False') }}
+timestamp_format = {{ ansible_cfg_lucid_timestamp_format_string | default('time') }}
+output_mode = {{ ansible_cfg_lucid_output_mode_string | default('auto') }}
+log_file =
+log_append = {{ ansible_cfg_lucid_log_append | default('False') }}
+{% endif %}
+
[ssh_connection]
remote_port = {{ ansible_cfg_ssh_port }}
{% if ansible_facts['distribution'] == 'openSUSE' %}
---
base-commit: 2674eeee8a274f456d9d7c14c8967dc568d74b28
change-id: 20251121-lucid-adeb42a71d42
Best regards,
--
Daniel Gomez <da.gomez@samsung.com>
^ permalink raw reply related [flat|nested] 2+ messages in thread* Re: [PATCH] ansible: add lucid callback plugin for clean output
2025-11-21 19:44 [PATCH] ansible: add lucid callback plugin for clean output Daniel Gomez
@ 2025-11-25 0:07 ` Luis Chamberlain
0 siblings, 0 replies; 2+ messages in thread
From: Luis Chamberlain @ 2025-11-25 0:07 UTC (permalink / raw)
To: Daniel Gomez; +Cc: Chuck Lever, kdevops, Daniel Gomez
On Fri, Nov 21, 2025 at 08:44:59PM +0100, Daniel Gomez wrote:
> From: Daniel Gomez <da.gomez@samsung.com>
>
> Add the lucid callback plugin to provide clean, minimal Ansible output
> for kdevops workflows. Current Ansible output is verbose by default
> making it difficult to see what matters during long-running test
> operations. Lucid solves this by showing only changes and errors at
> default verbosity while providing progressive detail at higher levels.
>
> The plugin includes dynamic mode with live updates for interactive
> terminals showing running tasks with spinners and elapsed time similar
> to build system output. For CI/CD and piped contexts it automatically
> falls back to static output. All runs generate comprehensive logs with
> playbook names and timestamps allowing multiple executions to coexist
> without conflicts.
>
> Task-level output control is available via the output_verbosity variable
> allowing playbooks to specify per-task visibility. Execution time
> tracking highlights slow tasks exceeding configurable thresholds. Logs
> always capture full verbosity independent of display settings ensuring
> complete audit trails.
>
> Integrate lucid with kdevops Kconfig system providing options for
> time threshold, timestamp format, output mode (auto/static/dynamic),
> and log behavior. Add defconfig fragment for quick enablement. Update
> ansible.cfg template to generate callback configuration when selected.
> Make lucid the default callback for new configurations.
>
> Add documentation following Ansible's standard format covering
> requirements, parameters, usage examples, and behavior notes.
>
> Generated-by: Claude AI
> Signed-off-by: Daniel Gomez <da.gomez@samsung.com>
I saw the demo and am sold please yes!
Luis
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2025-11-25 0:07 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-11-21 19:44 [PATCH] ansible: add lucid callback plugin for clean output Daniel Gomez
2025-11-25 0:07 ` Luis Chamberlain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox