trace-cmd No such device Errors

[Vishal Chourasia <vishalc@linux.ibm.com>]

When running trace-cmd record, I encounter multiple "No such device"
errors due to the splice syscall failing with ENODEV for
/sys/kernel/tracing/per_cpu/cpuX/trace_pipe_raw files corresponding to
CPUs that are possible but not present.

Environment:
- Operating System: Red Hat Enterprise Linux 9.5 (Plow)
- Kernel: Linux 6.12.0-55.9.1.el10_0.ppc64le
- Architecture: ppc64le
- Command Executed: trace-cmd record -o /dev/null -v -e sched_switch -- sleep 1
- System CPU Configuration:
  - /sys/devices/system/cpu/possible: 0-223
  - /sys/devices/system/cpu/present: 0-63
  - /sys/kernel/tracing/per_cpu/: Contains directories cpu0 to cpu223

Steps to Reproduce: 
1. On a system with a mismatch between possible and present CPUs (e.g.,
   possible: 0-223, present: 0-63). 
2. Ensure /sys/kernel/tracing/per_cpu/ contains directories for all possible 
   CPUs (e.g., cpu0 to cpu223). 
3. Run the command:
strace --decode-fds -kk -o strace/trace-cmd -ff -- trace-cmd record -o /dev/null -v -e sched_switch -- sleep 1
4. Observe the output and strace logs.

Expected Behavior: trace-cmd should only attempt to access
trace_pipe_raw files for CPUs that are present (e.g., cpu0 to cpu63),
completing the recording without errors.

Actual Behavior:
trace-cmd fails with multiple errors:
trace-cmd: No such device
recorder error in splice input

The strace output shows splice syscalls failing with ENODEV for
trace_pipe_raw files of non-present CPUs 
(e.g., /sys/kernel/tracing/per_cpu/cpu223/trace_pipe_raw). 

Example:
trace-cmd.47140:splice(5</sys/kernel/tracing/per_cpu/cpu223/trace_pipe_raw>, NULL, 7<pipe:[96498]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)


Analysis:
- trace-cmd iterates over all directories in
  /sys/kernel/tracing/per_cpu/ (which includes cpu0 to cpu223 for all 
  possible CPUs).
- It attempts to invoke splice on trace_pipe_raw for each CPU, including
  those not present (e.g., cpu64 to cpu223).
- The splice syscall fails with ENODEV for non-present CPUs.
- Stack trace (from strace -kk):
splice(5</sys/kernel/tracing/per_cpu/cpu223/trace_pipe_raw>, ...)
  > /usr/lib64/glibc-hwcaps/power10/libc.so.6(splice+0x74)
  > /usr/bin/trace-cmd(splice_data+0x3b)
  > /usr/bin/trace-cmd(tracecmd_start_recording+0x93)
  > /usr/bin/trace-cmd(create_recorder+0x14f)
  > /usr/bin/trace-cmd(start_threads+0x81b)
  > /usr/bin/trace-cmd(record_trace.isra.0+0xf5f)
  > /usr/bin/trace-cmd(trace_record+0x47)
  > /usr/bin/trace-cmd(main+0xff)

# strace --decode-fds -kk -o strace/trace-cmd -ff -- trace-cmd record -o /dev/null -v -e sched_switch -- sleep 1
trace-cmd: No such device
    trace-cmd: No such device
recorder error in splice input  recorder error in splice input
recorder error in splice inputtrace-cmd: No such device
recorder error in splice input

trace-cmd: No such device
<snipped>
recorder error in splice input
trace-cmd: No such device
recorder error in splice input
recorder error in splice inputtrace-cmd: No such device

  recorder error in splice inputtrace-cmd: No such device

  recorder error in splice input
trace-cmd: No such device
  trace-cmd: No such device
recorder error in splice input
  recorder error in splice inputtrace-cmd: No such device

  recorder error in splice input
trace-cmd: No such device
  recorder error in splice inputtrace-cmd: No such device

recorder error in splice inputtrace-cmd: No such device
<snipped>
recorder error in splice input
recorder error in splice inputtrace-cmd: No such device

  recorder error in splice inputtrace-cmd: No such device

  recorder error in splice inputtrace-cmd: No such device

  recorder error in splice input
CPU0 data recorded at offset=0x10000
    0 bytes in size
CPU1 data recorded at offset=0x10000
    0 bytes in size
<snipped>
CPU10 data recorded at offset=0x10000
    0 bytes in size
CPU11 data recorded at offset=0x10000
<snipped>
    0 bytes in size
CPU63 data recorded at offset=0x10000
    0 bytes in size
CPU64 data recorded at offset=0x10000
    0 bytes in size
<snipped?
CPU213 data recorded at offset=0x10000
    0 bytes in size
<snipped>
CPU222 data recorded at offset=0x10000
    0 bytes in size
CPU223 data recorded at offset=0x10000
    0 bytes in size

# grep -i "such device" * | grep splice
trace-cmd.46981:splice(5</sys/kernel/tracing/per_cpu/cpu64/trace_pipe_raw>, NULL, 7<pipe:[86367]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.46982:splice(5</sys/kernel/tracing/per_cpu/cpu65/trace_pipe_raw>, NULL, 7<pipe:[82351]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.46983:splice(5</sys/kernel/tracing/per_cpu/cpu66/trace_pipe_raw>, NULL, 7<pipe:[81717]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.46984:splice(5</sys/kernel/tracing/per_cpu/cpu67/trace_pipe_raw>, NULL, 7<pipe:[1932]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.46985:splice(5</sys/kernel/tracing/per_cpu/cpu68/trace_pipe_raw>, NULL, 7<pipe:[92520]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.46986:splice(5</sys/kernel/tracing/per_cpu/cpu69/trace_pipe_raw>, NULL, 7<pipe:[93494]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
<snipped>
trace-cmd.47136:splice(5</sys/kernel/tracing/per_cpu/cpu219/trace_pipe_raw>, NULL, 7<pipe:[19742]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.47137:splice(5</sys/kernel/tracing/per_cpu/cpu220/trace_pipe_raw>, NULL, 7<pipe:[29868]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.47138:splice(5</sys/kernel/tracing/per_cpu/cpu221/trace_pipe_raw>, NULL, 7<pipe:[42069]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.47139:splice(5</sys/kernel/tracing/per_cpu/cpu222/trace_pipe_raw>, NULL, 7<pipe:[81737]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)
trace-cmd.47140:splice(5</sys/kernel/tracing/per_cpu/cpu223/trace_pipe_raw>, NULL, 7<pipe:[96498]>, NULL, 65536, SPLICE_F_MOVE) = -1 ENODEV (No such device)