public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Nam Cao <namcao@linutronix.de>
To: Gabriele Monaco <gmonaco@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>,
	linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org,
	john.ogness@linutronix.de
Subject: Re: [PATCH v4 20/22] rv: Add rtapp_sleep monitor
Date: Fri, 25 Apr 2025 08:34:56 +0200	[thread overview]
Message-ID: <20250425063456.NBE35YHR@linutronix.de> (raw)
In-Reply-To: <c321c7350ec10f9f358695acd765d2dbd067eeb2.camel@redhat.com>

On Thu, Apr 24, 2025 at 03:55:34PM +0200, Gabriele Monaco wrote:
> I've been playing with these monitors, code-wise they look good.
> I tested a bit and they seem to work without many surprises by doing
> something as simple as:
> 
> perf stat -e rv:error_sleep stress-ng --cpu-sched 1 -t 10s
>   -- shows several errors --

This one is a monitor's bug.

The monitor mistakenly sees the task getting woken up, *then* sees it going
to sleep.

This is due to trace_sched_switch() being called with a stale 'prev_state'.
'prev_state' is read at the beginning of __schedule(), but
trace_sched_switch() is invoked a bit later. Therefore if task->__state is
changed inbetween, 'prev_state' is not the value of task->__state.

The monitor checks (prev_state & TASK_INTERRUPTIBLE) to determine if the
task is going to sleep. This can be incorrect due to the race above. The
monitor sees the task going to sleep, but actually it is just preempted.

I think this also answers the race you observed with the srs monitor?

> perf stat -e rv:error_sleep stress-ng --prio-inv 1 --prio-inv-policy rr
>   -- shows only 1 error (normal while starting the program?) --
> 
> Not quite sound, but does it look a reasonable test to you?

The above command use mutexes with priority inheritance. That is good for
real-time. The errors are due to real-time tasks being delayed by
waitpid().

Priority inheritance can be disabled with "--prio-inv-type none". Then you
will see lots of errors with mutexes.

> I quickly tried the same with the other monitor comparing the number of
> errors with the page_faults generated by perf, but that didn't make too
> much sense. Perhaps I'm doing something wrong here though (the number
> reported by perf for page faults feels a bit too high).
> 
> perf stat -e page-faults -e rv:error_pagefault stress-ng --cyclic 1

This command run a non-real-time thread to do setup, and a cyclic real-time
thread. The number of pagefaults of each thread would be roughly
proportional to the code size executed by each thread. As the non-real-time
thread's code size is bigger, it sounds reasonable that the number of
pagefaults is greater than the number of monitor's warnings.
> 
> Anyway, the monitor looks good to me
> 
>   Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
> 
> but it'd be nice if you have tips to share how to quickly test it (e.g.
> without writing a custom workload).

I tested the monitor on a real system. My system has some real-time audio
processing processes (pipewire, firefox running youtube), yours also
should.

But thanks so much for testing with stress-ng. My testing didn't stress the
system enough for the above race to happen. I will give stress-ng a few
runs before the next version.

Best regards,
Nam

  reply	other threads:[~2025-04-25  6:35 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-04-23  6:49 [PATCH v4 00/22] RV: Linear temporal logic monitors for RT application Nam Cao
2025-04-23  6:49 ` [PATCH v4 01/22] rv: Add #undef TRACE_INCLUDE_FILE Nam Cao
2025-04-23  6:49 ` [PATCH v4 02/22] printk: Make vprintk_deferred() public Nam Cao
2025-04-23  6:49 ` [PATCH v4 03/22] panic: Add vpanic() Nam Cao
2025-04-23  6:49 ` [PATCH v4 04/22] rv: Let the reactors take care of buffers Nam Cao
2025-04-23  6:50 ` [PATCH v4 05/22] verification/dot2k: Make a separate dot2k_templates/Kconfig_container Nam Cao
2025-04-23  6:50 ` [PATCH v4 06/22] verification/dot2k: Remove __buff_to_string() Nam Cao
2025-04-23  6:50 ` [PATCH v4 07/22] verification/dot2k: Replace is_container() hack with subparsers Nam Cao
2025-04-23  6:50 ` [PATCH v4 08/22] rv: rename CONFIG_DA_MON_EVENTS to CONFIG_RV_MON_EVENTS Nam Cao
2025-04-23  6:50 ` [PATCH v4 09/22] verification/dot2k: Prepare the frontend for LTL inclusion Nam Cao
2025-04-23  6:50 ` [PATCH v4 10/22] Documentation/rv: Prepare monitor synthesis document " Nam Cao
2025-04-23  6:50 ` [PATCH v4 11/22] verification/rvgen: Restructure the templates files Nam Cao
2025-04-23  6:50 ` [PATCH v4 12/22] verification/rvgen: Restructure the classes to prepare for LTL inclusion Nam Cao
2025-04-23  6:50 ` [PATCH v4 13/22] rv: Add support for LTL monitors Nam Cao
2025-04-23  6:50 ` [PATCH v4 14/22] rv: Add rtapp container monitor Nam Cao
2025-04-23  6:50 ` [PATCH v4 15/22] x86/tracing: Remove redundant trace_pagefault_key Nam Cao
2025-04-23  6:50 ` [PATCH v4 16/22] x86/tracing: Move page fault trace points to generic Nam Cao
2025-04-23  6:50 ` [PATCH v4 17/22] arm64: mm: Add page fault trace points Nam Cao
2025-04-23  6:50 ` [PATCH v4 18/22] riscv: " Nam Cao
2025-04-23  6:50 ` [PATCH v4 19/22] rv: Add rtapp_pagefault monitor Nam Cao
2025-04-23 10:37   ` Gabriele Monaco
2025-04-24  3:40     ` Nam Cao
2025-04-23  6:50 ` [PATCH v4 20/22] rv: Add rtapp_sleep monitor Nam Cao
2025-04-24 13:55   ` Gabriele Monaco
2025-04-25  6:34     ` Nam Cao [this message]
2025-04-25  7:35       ` Gabriele Monaco
2025-04-25  9:33         ` Nam Cao
2025-04-25  7:45       ` John Ogness
2025-04-25  7:48         ` John Ogness
2025-04-25  9:23           ` Nam Cao
2025-04-23  6:50 ` [PATCH v4 21/22] rv: Add documentation for rtapp monitor Nam Cao
2025-04-23  6:50 ` [PATCH v4 22/22] rv: Allow to configure the number of per-task monitor Nam Cao
2025-08-10 21:12 ` [PATCH v4 00/22] RV: Linear temporal logic monitors for RT application patchwork-bot+linux-riscv

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250425063456.NBE35YHR@linutronix.de \
    --to=namcao@linutronix.de \
    --cc=gmonaco@redhat.com \
    --cc=john.ogness@linutronix.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox