[libevl][PATCH 1/2] tests: sched-quota-accuracy: Augment workload with busy-spinning

[libevl][PATCH 1/2] tests: sched-quota-accuracy: Augment workload with busy-spinning

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This increases calibration accuracy, specifically over virtual
environments.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 tests/sched-quota-accuracy.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/tests/sched-quota-accuracy.c b/tests/sched-quota-accuracy.c
index 9ca496d..278ff1b 100644
--- a/tests/sched-quota-accuracy.c
+++ b/tests/sched-quota-accuracy.c
@@ -94,6 +94,13 @@ static const struct option options[] = {
 static unsigned long __attribute__(( noinline ))
 __do_work(unsigned long count)
 {
+	struct timespec now, to;
+	int ret;
+
+	__Tcall_assert(ret, evl_read_clock(EVL_CLOCK_MONOTONIC, &now));
+	timespec_add_ns(&to, &now, 100);
+	while (timespec_sub_ns(&to, &now) > 0)
+		__Tcall_assert(ret, evl_read_clock(EVL_CLOCK_MONOTONIC, &now));
 	return count + 1;
 }
 
-- 
2.47.3

[libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

Test that higher-prio SCHED_FIFO threads do not run on the bill of
SCHED_QUOTA threads and that their accounting is not otherwise
disturbed.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 tests/sched-quota-accuracy.c | 55 +++++++++++++++++++++++++++++++-----
 1 file changed, 48 insertions(+), 7 deletions(-)

diff --git a/tests/sched-quota-accuracy.c b/tests/sched-quota-accuracy.c
index 278ff1b..f48f034 100644
--- a/tests/sched-quota-accuracy.c
+++ b/tests/sched-quota-accuracy.c
@@ -34,7 +34,7 @@ static struct quota_thread_desc {
 	pthread_t tid;
 	struct evl_sched_attrs attrs;
 	unsigned long count;
-} threads[MAX_THREADS];
+} threads[MAX_THREADS + 1];
 
 static int nrthreads = 3;
 
@@ -165,6 +165,41 @@ static void *quota_thread(void *arg)
 	return NULL;
 }
 
+static void *disruption_thread(void *arg)
+{
+	int serial = (int)(long)arg;
+	struct quota_thread_desc *t = threads + serial;
+	struct timespec now, to;
+	unsigned long loops;
+	int ret;
+
+	set_thread_affinity();
+
+	loops = crunch_per_sec / 1000; /* yield every 1 ms */
+
+	__Tcall_assert(t->efd, evl_attach_self("disrupt"));
+
+	evl_put_sem(&ready);
+
+	__Tcall_assert(ret, evl_lock_mutex(&lock));
+	for (;;) {
+		if (started)
+			break;
+		__Tcall_assert(ret, evl_wait_event(&barrier, &lock));
+	}
+	__Tcall_assert(ret, evl_unlock_mutex(&lock));
+
+	while (!done) {
+		do_work(loops, &t->count);
+
+		__Tcall_assert(ret, evl_read_clock(EVL_CLOCK_MONOTONIC, &now));
+		timespec_add_ns(&to, &now, 1000000);
+		__Tcall_assert(ret, evl_sleep_until(EVL_CLOCK_MONOTONIC, &to));
+	}
+
+	return NULL;
+}
+
 static void create_quota_thread(int tgid, int serial)
 {
 	struct quota_thread_desc *t = threads + serial;
@@ -175,13 +210,13 @@ static void create_quota_thread(int tgid, int serial)
 	new_thread(&t->tid, SCHED_FIFO, 1, quota_thread, (void *)(long)serial);
 }
 
-static void create_fifo_thread(int serial)
+static void create_fifo_thread(void *(*thread_func)(void *), int serial)
 {
 	struct quota_thread_desc *t = threads + serial;
 
 	t->attrs.sched_policy = SCHED_FIFO;
-	t->attrs.sched_priority = 1;
-	new_thread(&t->tid, SCHED_FIFO, 1, quota_thread, (void *)(long)serial);
+	t->attrs.sched_priority = 2;
+	new_thread(&t->tid, SCHED_FIFO, 2, thread_func, (void *)(long)serial);
 }
 
 static int cleanup_group(void)
@@ -235,6 +270,9 @@ static double run_quota(int quota)
 		__Tcall_assert(ret, evl_get_sem(&ready));
 	}
 
+	create_fifo_thread(disruption_thread, n);
+	__Tcall_assert(ret, evl_get_sem(&ready));
+
 	__Tcall_assert(ret, evl_lock_mutex(&lock));
 	started = true;
 	__Tcall_assert(ret, evl_broadcast_event(&barrier));
@@ -256,6 +294,8 @@ static double run_quota(int quota)
 	for (n = 0; n < nrthreads; n++)
 		pthread_join(threads[n].tid, NULL);
 
+	pthread_join(threads[nrthreads].tid, NULL);
+
 	/*
 	 * Percentage of completion of the SCHED_QUOTA run for the
 	 * given quota value, compared to the calibration run which
@@ -301,7 +341,7 @@ static unsigned long long calibrate(void)
 	crunch_per_sec = (((unsigned long long)count) * ONE_BILLION) / ns;
 
 	for (n = 0; n < nrthreads; n++) {
-		create_fifo_thread(n);
+		create_fifo_thread(quota_thread, n);
 		__Tcall_assert(ret, evl_get_sem(&ready));
 	}
 
@@ -400,12 +440,13 @@ int main(int argc, char *argv[])
 	effective = run_quota(quota);
 
 	if (verbose)	  /* Percentage of quota actually obtained. */
-		do_trace("CPU%d: %d thread%s: cap=%d%%, effective=%.1f%%",
+		do_trace("CPU%d: %d thread%s: cap=%d%%, effective=%.1f%%, disruption=%.1f%%",
 			test_cpu,
 			nrthreads,
 			nrthreads > 1 ?	"s": "",
 			quota,
-			effective);
+			effective,
+			threads[nrthreads].count * 100.0 / loops_per_sec);
 
 	/*
 	 * The accuracy value is the alignment of the observed runtime
-- 
2.47.3

Re: [libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Philippe Gerum]

Jan Kiszka <jan.kiszka@siemens.com> writes:

> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> Test that higher-prio SCHED_FIFO threads do not run on the bill of
> SCHED_QUOTA threads and that their accounting is not otherwise
> disturbed.
>
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  tests/sched-quota-accuracy.c | 55 +++++++++++++++++++++++++++++++-----
>  1 file changed, 48 insertions(+), 7 deletions(-)
>
> diff --git a/tests/sched-quota-accuracy.c b/tests/sched-quota-accuracy.c
> index 278ff1b..f48f034 100644
> --- a/tests/sched-quota-accuracy.c
> +++ b/tests/sched-quota-accuracy.c
> @@ -34,7 +34,7 @@ static struct quota_thread_desc {
>  	pthread_t tid;
>  	struct evl_sched_attrs attrs;
>  	unsigned long count;
> -} threads[MAX_THREADS];
> +} threads[MAX_THREADS + 1];
>  
>  static int nrthreads = 3;
>  
> @@ -165,6 +165,41 @@ static void *quota_thread(void *arg)
>  	return NULL;
>  }
>  
> +static void *disruption_thread(void *arg)
> +{
> +	int serial = (int)(long)arg;
> +	struct quota_thread_desc *t = threads + serial;
> +	struct timespec now, to;
> +	unsigned long loops;
> +	int ret;
> +
> +	set_thread_affinity();
> +
> +	loops = crunch_per_sec / 1000; /* yield every 1 ms */
> +
> +	__Tcall_assert(t->efd, evl_attach_self("disrupt"));
> +
> +	evl_put_sem(&ready);
> +
> +	__Tcall_assert(ret, evl_lock_mutex(&lock));
> +	for (;;) {
> +		if (started)
> +			break;
> +		__Tcall_assert(ret, evl_wait_event(&barrier, &lock));
> +	}
> +	__Tcall_assert(ret, evl_unlock_mutex(&lock));
> +
> +	while (!done) {
> +		do_work(loops, &t->count);
> +
> +		__Tcall_assert(ret, evl_read_clock(EVL_CLOCK_MONOTONIC, &now));
> +		timespec_add_ns(&to, &now, 1000000);
> +		__Tcall_assert(ret, evl_sleep_until(EVL_CLOCK_MONOTONIC, &to));
> +	}
> +
> +	return NULL;
> +}
> +
>  static void create_quota_thread(int tgid, int serial)
>  {
>  	struct quota_thread_desc *t = threads + serial;
> @@ -175,13 +210,13 @@ static void create_quota_thread(int tgid, int serial)
>  	new_thread(&t->tid, SCHED_FIFO, 1, quota_thread, (void *)(long)serial);
>  }
>  
> -static void create_fifo_thread(int serial)
> +static void create_fifo_thread(void *(*thread_func)(void *), int serial)
>  {
>  	struct quota_thread_desc *t = threads + serial;
>  
>  	t->attrs.sched_policy = SCHED_FIFO;
> -	t->attrs.sched_priority = 1;
> -	new_thread(&t->tid, SCHED_FIFO, 1, quota_thread, (void *)(long)serial);
> +	t->attrs.sched_priority = 2;
> +	new_thread(&t->tid, SCHED_FIFO, 2, thread_func, (void *)(long)serial);
>  }
>  
>  static int cleanup_group(void)
> @@ -235,6 +270,9 @@ static double run_quota(int quota)
>  		__Tcall_assert(ret, evl_get_sem(&ready));
>  	}
>  
> +	create_fifo_thread(disruption_thread, n);
> +	__Tcall_assert(ret, evl_get_sem(&ready));
> +
>  	__Tcall_assert(ret, evl_lock_mutex(&lock));
>  	started = true;
>  	__Tcall_assert(ret, evl_broadcast_event(&barrier));
> @@ -256,6 +294,8 @@ static double run_quota(int quota)
>  	for (n = 0; n < nrthreads; n++)
>  		pthread_join(threads[n].tid, NULL);
>  
> +	pthread_join(threads[nrthreads].tid, NULL);
> +
>  	/*
>  	 * Percentage of completion of the SCHED_QUOTA run for the
>  	 * given quota value, compared to the calibration run which
> @@ -301,7 +341,7 @@ static unsigned long long calibrate(void)
>  	crunch_per_sec = (((unsigned long long)count) * ONE_BILLION) / ns;
>  
>  	for (n = 0; n < nrthreads; n++) {
> -		create_fifo_thread(n);
> +		create_fifo_thread(quota_thread, n);
>  		__Tcall_assert(ret, evl_get_sem(&ready));
>  	}
>  
> @@ -400,12 +440,13 @@ int main(int argc, char *argv[])
>  	effective = run_quota(quota);
>  
>  	if (verbose)	  /* Percentage of quota actually obtained. */
> -		do_trace("CPU%d: %d thread%s: cap=%d%%, effective=%.1f%%",
> +		do_trace("CPU%d: %d thread%s: cap=%d%%, effective=%.1f%%, disruption=%.1f%%",
>  			test_cpu,
>  			nrthreads,
>  			nrthreads > 1 ?	"s": "",
>  			quota,
> -			effective);
> +			effective,
> +			threads[nrthreads].count * 100.0 / loops_per_sec);
>  
>  	/*
>  	 * The accuracy value is the alignment of the observed runtime

Merged, thanks.

-- 
Philippe.

Re: [libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Philippe Gerum]

Jan Kiszka <jan.kiszka@siemens.com> writes:

> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> Test that higher-prio SCHED_FIFO threads do not run on the bill of
> SCHED_QUOTA threads and that their accounting is not otherwise
> disturbed.
>

This patch introduced a regression when determining the accuracy of the
policy with respect to allotting threads the expected runtime budget:

root@phytec-mira-evl:~# evl test sched-quota-accuracy
sched-quota-accuracy: 494.0%

The proper result would rather be close to the following:

root@phytec-mira-evl:~# evl test sched-quota-accuracy
sched-quota-accuracy: 99.1%

-- 
Philippe.

Re: [libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Philippe Gerum]

Philippe Gerum <rpm@xenomai.org> writes:

> Jan Kiszka <jan.kiszka@siemens.com> writes:
>
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> Test that higher-prio SCHED_FIFO threads do not run on the bill of
>> SCHED_QUOTA threads and that their accounting is not otherwise
>> disturbed.
>>
>
> This patch introduced a regression when determining the accuracy of the
> policy with respect to allotting threads the expected runtime budget:
>
> root@phytec-mira-evl:~# evl test sched-quota-accuracy
> sched-quota-accuracy: 494.0%
>
> The proper result would rather be close to the following:
>
> root@phytec-mira-evl:~# evl test sched-quota-accuracy
> sched-quota-accuracy: 99.1%

Ok, I guess the fact that the calibration process does not factor in the
disruptor explains the full breakage we have with the accounting now:

root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
picked CPU1 for execution
CPU1: calibrating: 893547 loops/sec
CPU1: new thread group #0, quota sum is 10%
CPU1: done quota_thread[0], count=150399
CPU1: done quota_thread[1], count=145636
CPU1: done quota_thread[2], count=141552
CPU1: 3 threads: cap=10%, effective=49.0%, disruption=49.6%
sched-quota-accuracy: 489.7%

Which does not make any sense since the effective value should be capped
at 10% in the above case, compared to the nominal (full) report which
should be:

root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
picked CPU1 for execution
CPU1: calibrating: 899203 loops/sec
CPU1: new thread group #0, quota sum is 10%
CPU1: done quota_thread[0], count=26818
CPU1: done quota_thread[1], count=26709
CPU1: done quota_thread[2], count=35612
CPU1: 3 threads: cap=10%, effective=9.9%
sched-quota-accuracy: 99.1%

Dropping this patch from the -next branch for now.

-- 
Philippe.

Re: [libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Jan Kiszka]

On 20.06.26 13:17, Philippe Gerum wrote:
> Philippe Gerum <rpm@xenomai.org> writes:
> 
>> Jan Kiszka <jan.kiszka@siemens.com> writes:
>>
>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>
>>> Test that higher-prio SCHED_FIFO threads do not run on the bill of
>>> SCHED_QUOTA threads and that their accounting is not otherwise
>>> disturbed.
>>>
>>
>> This patch introduced a regression when determining the accuracy of the
>> policy with respect to allotting threads the expected runtime budget:
>>
>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>> sched-quota-accuracy: 494.0%
>>
>> The proper result would rather be close to the following:
>>
>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>> sched-quota-accuracy: 99.1%
> 
> Ok, I guess the fact that the calibration process does not factor in the
> disruptor explains the full breakage we have with the accounting now:
> 
> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
> picked CPU1 for execution
> CPU1: calibrating: 893547 loops/sec
> CPU1: new thread group #0, quota sum is 10%
> CPU1: done quota_thread[0], count=150399
> CPU1: done quota_thread[1], count=145636
> CPU1: done quota_thread[2], count=141552
> CPU1: 3 threads: cap=10%, effective=49.0%, disruption=49.6%
> sched-quota-accuracy: 489.7%
> 
> Which does not make any sense since the effective value should be capped
> at 10% in the above case, compared to the nominal (full) report which
> should be:
> 
> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
> picked CPU1 for execution
> CPU1: calibrating: 899203 loops/sec
> CPU1: new thread group #0, quota sum is 10%
> CPU1: done quota_thread[0], count=26818
> CPU1: done quota_thread[1], count=26709
> CPU1: done quota_thread[2], count=35612
> CPU1: 3 threads: cap=10%, effective=9.9%
> sched-quota-accuracy: 99.1%
> 
> Dropping this patch from the -next branch for now.
> 

I would rather recommend taking a trace and debugging the scheduler -
this could very likely be remaining issue in the quota fix.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

Re: [libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Jan Kiszka]

On 20.06.26 14:40, Jan Kiszka wrote:
> On 20.06.26 13:17, Philippe Gerum wrote:
>> Philippe Gerum <rpm@xenomai.org> writes:
>>
>>> Jan Kiszka <jan.kiszka@siemens.com> writes:
>>>
>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>
>>>> Test that higher-prio SCHED_FIFO threads do not run on the bill of
>>>> SCHED_QUOTA threads and that their accounting is not otherwise
>>>> disturbed.
>>>>
>>>
>>> This patch introduced a regression when determining the accuracy of the
>>> policy with respect to allotting threads the expected runtime budget:
>>>
>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>>> sched-quota-accuracy: 494.0%
>>>
>>> The proper result would rather be close to the following:
>>>
>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>>> sched-quota-accuracy: 99.1%
>>
>> Ok, I guess the fact that the calibration process does not factor in the
>> disruptor explains the full breakage we have with the accounting now:
>>
>> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
>> picked CPU1 for execution
>> CPU1: calibrating: 893547 loops/sec
>> CPU1: new thread group #0, quota sum is 10%
>> CPU1: done quota_thread[0], count=150399
>> CPU1: done quota_thread[1], count=145636
>> CPU1: done quota_thread[2], count=141552
>> CPU1: 3 threads: cap=10%, effective=49.0%, disruption=49.6%
>> sched-quota-accuracy: 489.7%
>>
>> Which does not make any sense since the effective value should be capped
>> at 10% in the above case, compared to the nominal (full) report which
>> should be:
>>
>> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
>> picked CPU1 for execution
>> CPU1: calibrating: 899203 loops/sec
>> CPU1: new thread group #0, quota sum is 10%
>> CPU1: done quota_thread[0], count=26818
>> CPU1: done quota_thread[1], count=26709
>> CPU1: done quota_thread[2], count=35612
>> CPU1: 3 threads: cap=10%, effective=9.9%
>> sched-quota-accuracy: 99.1%
>>
>> Dropping this patch from the -next branch for now.
>>
> 
> I would rather recommend taking a trace and debugging the scheduler -
> this could very likely be remaining issue in the quota fix.
> 

What is the state of this? How can I reproduce the issue you saw? What
is specific to the target, what could also be seen over qemu[/kvm]?

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

Re: [libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Philippe Gerum]

Jan Kiszka <jan.kiszka@siemens.com> writes:

> On 20.06.26 14:40, Jan Kiszka wrote:
>> On 20.06.26 13:17, Philippe Gerum wrote:
>>> Philippe Gerum <rpm@xenomai.org> writes:
>>>
>>>> Jan Kiszka <jan.kiszka@siemens.com> writes:
>>>>
>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>
>>>>> Test that higher-prio SCHED_FIFO threads do not run on the bill of
>>>>> SCHED_QUOTA threads and that their accounting is not otherwise
>>>>> disturbed.
>>>>>
>>>>
>>>> This patch introduced a regression when determining the accuracy of the
>>>> policy with respect to allotting threads the expected runtime budget:
>>>>
>>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>>>> sched-quota-accuracy: 494.0%
>>>>
>>>> The proper result would rather be close to the following:
>>>>
>>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>>>> sched-quota-accuracy: 99.1%
>>>
>>> Ok, I guess the fact that the calibration process does not factor in the
>>> disruptor explains the full breakage we have with the accounting now:
>>>
>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
>>> picked CPU1 for execution
>>> CPU1: calibrating: 893547 loops/sec
>>> CPU1: new thread group #0, quota sum is 10%
>>> CPU1: done quota_thread[0], count=150399
>>> CPU1: done quota_thread[1], count=145636
>>> CPU1: done quota_thread[2], count=141552
>>> CPU1: 3 threads: cap=10%, effective=49.0%, disruption=49.6%
>>> sched-quota-accuracy: 489.7%
>>>
>>> Which does not make any sense since the effective value should be capped
>>> at 10% in the above case, compared to the nominal (full) report which
>>> should be:
>>>
>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
>>> picked CPU1 for execution
>>> CPU1: calibrating: 899203 loops/sec
>>> CPU1: new thread group #0, quota sum is 10%
>>> CPU1: done quota_thread[0], count=26818
>>> CPU1: done quota_thread[1], count=26709
>>> CPU1: done quota_thread[2], count=35612
>>> CPU1: 3 threads: cap=10%, effective=9.9%
>>> sched-quota-accuracy: 99.1%
>>>
>>> Dropping this patch from the -next branch for now.
>>>
>> 
>> I would rather recommend taking a trace and debugging the scheduler -
>> this could very likely be remaining issue in the quota fix.
>> 
>
> What is the state of this? How can I reproduce the issue you saw? What
> is specific to the target, what could also be seen over qemu[/kvm]?
>

The task of writing a proper test for evl is ongoing, the overall sched
issue is very much on my radar. I'll follow up when this test code is
available.

-- 
Philippe.

Re: [libevl][PATCH 2/2] tests: sched-quota-accuracy: Add preempting FIFO thread

[Philippe Gerum]

Philippe Gerum <rpm@xenomai.org> writes:

> Jan Kiszka <jan.kiszka@siemens.com> writes:
>
>> On 20.06.26 14:40, Jan Kiszka wrote:
>>> On 20.06.26 13:17, Philippe Gerum wrote:
>>>> Philippe Gerum <rpm@xenomai.org> writes:
>>>>
>>>>> Jan Kiszka <jan.kiszka@siemens.com> writes:
>>>>>
>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>
>>>>>> Test that higher-prio SCHED_FIFO threads do not run on the bill of
>>>>>> SCHED_QUOTA threads and that their accounting is not otherwise
>>>>>> disturbed.
>>>>>>
>>>>>
>>>>> This patch introduced a regression when determining the accuracy of the
>>>>> policy with respect to allotting threads the expected runtime budget:
>>>>>
>>>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>>>>> sched-quota-accuracy: 494.0%
>>>>>
>>>>> The proper result would rather be close to the following:
>>>>>
>>>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy
>>>>> sched-quota-accuracy: 99.1%
>>>>
>>>> Ok, I guess the fact that the calibration process does not factor in the
>>>> disruptor explains the full breakage we have with the accounting now:
>>>>
>>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
>>>> picked CPU1 for execution
>>>> CPU1: calibrating: 893547 loops/sec
>>>> CPU1: new thread group #0, quota sum is 10%
>>>> CPU1: done quota_thread[0], count=150399
>>>> CPU1: done quota_thread[1], count=145636
>>>> CPU1: done quota_thread[2], count=141552
>>>> CPU1: 3 threads: cap=10%, effective=49.0%, disruption=49.6%
>>>> sched-quota-accuracy: 489.7%
>>>>
>>>> Which does not make any sense since the effective value should be capped
>>>> at 10% in the above case, compared to the nominal (full) report which
>>>> should be:
>>>>
>>>> root@phytec-mira-evl:~# evl test sched-quota-accuracy -- -v
>>>> picked CPU1 for execution
>>>> CPU1: calibrating: 899203 loops/sec
>>>> CPU1: new thread group #0, quota sum is 10%
>>>> CPU1: done quota_thread[0], count=26818
>>>> CPU1: done quota_thread[1], count=26709
>>>> CPU1: done quota_thread[2], count=35612
>>>> CPU1: 3 threads: cap=10%, effective=9.9%
>>>> sched-quota-accuracy: 99.1%
>>>>
>>>> Dropping this patch from the -next branch for now.
>>>>
>>> 
>>> I would rather recommend taking a trace and debugging the scheduler -
>>> this could very likely be remaining issue in the quota fix.
>>> 
>>
>> What is the state of this? How can I reproduce the issue you saw? What
>> is specific to the target, what could also be seen over qemu[/kvm]?
>>
>
> The task of writing a proper test for evl is ongoing, the overall sched
> issue is very much on my radar. I'll follow up when this test code is
> available.

You need to pick e5e876da..3e32fa6f from [1] for the kernel bits, and
198a21b7..HEAD from [2] for an improved sched-quota-accuracy test which
also measures the preemption noise, with a significantly simpler
implementation. The topmost commits in [2] also introduce an interface
to the ftrace-based 'evl_trace' event which I used for debugging.

Basically, I merged the gist of our past proposals, refining here and
there as issues were uncovered.

Heavily tested on real hardware, which uncovered a couple of other
serious issues. I plan to write a separate test program for testing the
runtime credit and peak management.

[1]
https://gitlab.com/Xenomai/xenomai4/linux-evl/-/commits/next/v6.12.y-cip-evl-rebase?ref_type=heads

[2] https://gitlab.com/Xenomai/xenomai4/libevl/-/commits/next?ref_type=heads

-- 
Philippe.