Re: [BUG?] Possible scheduler bug on Rock 5B RK3588 on 7.2.5

Re: [BUG?] Possible scheduler bug on Rock 5B RK3588 on 7.2.5

[Chaoyi Chen]

On 9/18/2026 3:15 AM, No SPAM here wrote:
> Hi
> 
> On Thursday, 17 September 2026 at 03:37, Chaoyi Chen <chaoyi.chen@rock-chips.com> wrote:
> 
>> Hello Trevor,
>>
>> On 9/17/2026 9:21 AM, No SPAM here wrote:
>>>> Would you mind trying to disable `CONFIG_SCHED_CLUSTER` and checking whether
>>>> the problem still reproduces?
> 
> trevor@rock-5b:~$ uname -a
> Linux rock-5b 6.18.52-current-rockchip64 #1 SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 aarch64 GNU/Linux
> trevor@rock-5b:~$ grep CONFIG_SCHED_CLU /boot/config-6.18.52-current-rockchip64 
> # CONFIG_SCHED_CLUSTER is not set
> trevor@rock-5b:~$ sar | tail -5
> 
> 18:50:15        CPU     %user     %nice   %system   %iowait    %steal     %idle
> 19:00:15        all     99.89      0.00      0.11      0.00      0.00      0.00
> 19:10:15        all     99.89      0.00      0.11      0.00      0.00      0.00
> Average:        all     99.89      0.00      0.11      0.00      0.00      0.00
> 
> Been running stress --cpu 8 since 18:47 and so far it is using all 8 cores at 100%. It would normally have shown the 7/8ths problem after 20s or so, maybe 2 or 3 minutes at max. So I'd say this setting is definitely involved.
> 

Thank you very much for testing. I'm not a scheduler expert, but I see 
that CONFIG_SCHED_CLUSTER is enabled by default on arm64 [0]. 

Peter is the maintainer and author of this series. 
@Peter, do you have any further insights? Thank you!

This thread background is here[1].

[0]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7bd291abe2da09f59dca81f35a4ec220e5e138a2
[1]: https://lore.kernel.org/all/f15f8166-c1e4-45dc-9ddd-4ec887e415eb@gdead.co.uk/

-- 
Best, 
Chaoyi

Re: [BUG?] Possible scheduler bug on Rock 5B RK3588 on 7.2.5

[No SPAM here]

[-- Attachment #1: Type: text/plain, Size: 2586 bytes --]

Thanks Chaoyi, Hello Peter.

I have run `openssl speed -multi 8 rsa md5 sha256 sha512 aes sha1  camellia  des rmd160` while booted to the same 6.18.52 kernel, one built with CONFIG_SCHED_CLUSTER=y and one with it =n. This produces a lot of output and runs for about 8.5 minutes and puts out a 21 line table of statistics. I've compared a fair few of the numbers from both runs and most of the numbers from the cluster=n run are between 8 and 11% higher which seems to back up the idea that it's just not using one whole core of the 8 available. I've attached the two 21 line summaries here, CLUSTER=n results are in openssl-speed-nocluster.txt and CLUSTER=y in openssl-speed-cluster.txt.

Trevor

On Friday, 18 September 2026 at 02:22, Chaoyi Chen <chaoyi.chen@rock-chips.com> wrote:

> On 9/18/2026 3:15 AM, No SPAM here wrote:
> > Hi
> >
> > On Thursday, 17 September 2026 at 03:37, Chaoyi Chen <chaoyi.chen@rock-chips.com> wrote:
> >
> >> Hello Trevor,
> >>
> >> On 9/17/2026 9:21 AM, No SPAM here wrote:
> >>>> Would you mind trying to disable `CONFIG_SCHED_CLUSTER` and checking whether
> >>>> the problem still reproduces?
> >
> > trevor@rock-5b:~$ uname -a
> > Linux rock-5b 6.18.52-current-rockchip64 #1 SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 aarch64 GNU/Linux
> > trevor@rock-5b:~$ grep CONFIG_SCHED_CLU /boot/config-6.18.52-current-rockchip64
> > # CONFIG_SCHED_CLUSTER is not set
> > trevor@rock-5b:~$ sar | tail -5
> >
> > 18:50:15        CPU     %user     %nice   %system   %iowait    %steal     %idle
> > 19:00:15        all     99.89      0.00      0.11      0.00      0.00      0.00
> > 19:10:15        all     99.89      0.00      0.11      0.00      0.00      0.00
> > Average:        all     99.89      0.00      0.11      0.00      0.00      0.00
> >
> > Been running stress --cpu 8 since 18:47 and so far it is using all 8 cores at 100%. It would normally have shown the 7/8ths problem after 20s or so, maybe 2 or 3 minutes at max. So I'd say this setting is definitely involved.
> >
> 
> Thank you very much for testing. I'm not a scheduler expert, but I see
> that CONFIG_SCHED_CLUSTER is enabled by default on arm64 [0].
> 
> Peter is the maintainer and author of this series.
> @Peter, do you have any further insights? Thank you!
> 
> This thread background is here[1].
> 
> [0]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7bd291abe2da09f59dca81f35a4ec220e5e138a2
> [1]: https://lore.kernel.org/all/f15f8166-c1e4-45dc-9ddd-4ec887e415eb@gdead.co.uk/
> 
> --
> Best,
> Chaoyi
> 

[-- Attachment #2: openssl-speed-nocluster.txt --]
[-- Type: text/plain, Size: 1933 bytes --]

md5             206646.70k   682082.69k  1627733.76k  2515694.25k  3003411.11k  3047533.23k
sha1            257265.18k   945564.91k  2826501.12k  5732223.32k  8417888.94k  8727172.44k
rmd160          168775.57k   482578.01k  1047489.96k  1490576.73k  1703146.84k  1721171.97k
sha256          256593.18k   949375.79k  2850635.69k  5781916.67k  8441995.26k  8740929.54k
sha512          139462.71k   558478.93k  1124027.14k  1811831.47k  2209688.23k  2246688.77k
des-cbc              0.00         0.00         0.00         0.00         0.00         0.00 
des-ede3        132712.38k   138286.40k   139840.77k   140261.44k   140372.65k   140245.64k
aes-128-cbc    2887098.42k  6649536.64k 10139807.06k 11986084.86k 12734630.57k 12803970.39k
aes-192-cbc    2751396.09k  5874383.27k  8473588.05k  9655262.55k 10161837.40k 10203824.13k
aes-256-cbc    2674905.48k  5340891.67k  7373585.83k  8255187.97k  8579080.19k  8609371.48k
camellia-128-cbc   582227.33k   679838.68k   710770.01k   721328.47k   723978.92k   724489.56k
camellia-192-cbc   468416.18k   530513.96k   550038.44k   556270.25k   557826.05k   558000.81k
camellia-256-cbc   467775.22k   530535.47k   550066.77k   556213.59k   556876.29k   557940.74k
                   sign    verify    encrypt   decrypt   sign/s verify/s  encr./s  decr./s
rsa   512 bits 0.000016s 0.000001s 0.000002s 0.000017s  64304.1 755378.7 645816.8  57525.3
rsa  1024 bits 0.000083s 0.000004s 0.000005s 0.000086s  11990.6 233562.6 218317.7  11679.9
rsa  2048 bits 0.000571s 0.000016s 0.000016s 0.000574s   1752.4  64301.8  62614.2   1743.0
rsa  3072 bits 0.001760s 0.000034s 0.000035s 0.001765s    568.1  29361.9  28935.6    566.5
rsa  4096 bits 0.003995s 0.000060s 0.000060s 0.004000s    250.3  16738.8  16563.8    250.0
rsa  7680 bits 0.031106s 0.000207s 0.000208s 0.031131s     32.1   4839.0   4814.8     32.1
rsa 15360 bits 0.189898s 0.000820s 0.000821s 0.184244s      5.3   1219.4   1217.3      5.4

[-- Attachment #3: openssl-speed-cluster.txt --]
[-- Type: text/plain, Size: 1933 bytes --]

md5             207893.48k   681828.80k  1624703.66k  2514480.47k  3003370.15k  3047145.47k
sha1            257001.97k   944134.21k  2818819.67k  5729986.56k  8258613.03k  7898929.54k
rmd160          157569.82k   446793.28k   963480.75k  1360659.80k  1549350.23k  1565089.82k
sha256          241875.40k   894078.82k  2667497.15k  5343714.37k  7705458.01k  7961350.38k
sha512          130139.93k   520255.22k  1040071.77k  1665936.38k  2023230.12k  2055602.18k
des-cbc              0.00         0.00         0.00         0.00         0.00         0.00 
des-ede3        122223.67k   126929.64k   128361.86k   128859.14k   128809.72k   128849.24k
aes-128-cbc    2737165.32k  6205807.32k  9220290.30k 10736674.05k 11360862.21k 11410365.80k
aes-192-cbc    2609197.06k  5462457.41k  7717889.27k  8693462.87k  9127941.46k  9163740.50k
aes-256-cbc    2531961.38k  4973017.60k  6722701.40k  7453166.55k  7728204.46k  7748107.85k
camellia-128-cbc   537508.58k   621512.36k   648128.60k   655762.20k   658434.73k   658625.88k
camellia-192-cbc   433255.91k   484146.24k   500183.47k   506022.57k   507199.49k   507164.29k
camellia-256-cbc   430679.27k   483797.80k   500581.66k   505632.77k   507202.22k   507150.34k
                   sign    verify    encrypt   decrypt   sign/s verify/s  encr./s  decr./s
rsa   512 bits 0.000017s 0.000001s 0.000002s 0.000019s  58896.7 690687.0 594987.3  52892.2
rsa  1024 bits 0.000092s 0.000005s 0.000005s 0.000094s  10854.4 211381.1 198357.9  10589.4
rsa  2048 bits 0.000633s 0.000017s 0.000018s 0.000636s   1580.0  57918.6  56515.5   1573.2
rsa  3072 bits 0.001957s 0.000038s 0.000038s 0.001959s    511.0  26431.1  26051.8    510.3
rsa  4096 bits 0.004442s 0.000066s 0.000067s 0.004449s    225.1  15041.4  14899.8    224.8
rsa  7680 bits 0.034616s 0.000230s 0.000231s 0.034496s     28.9   4351.1   4323.8     29.0
rsa 15360 bits 0.211541s 0.000911s 0.000917s 0.211495s      4.7   1097.8   1090.9      4.7

Re: [BUG?] Possible scheduler bug on Rock 5B RK3588 on 7.2.5

[No SPAM here]

I have now retested with kernel 7.3.0-rc3 and this bug appears to have been fixed. The same command now runs all 8 cores at 100% for the duration of the test and the resulting numbers differ (in favour of the 6.18.52 CLUSTER=n kernel) by a maximum of 0.25% which is measurement error.

Thanks for your attention. This can be marked as solved.

Trevor


Sent with Proton Mail secure email.

On Friday, 18 September 2026 at 02:22, Chaoyi Chen <chaoyi.chen@rock-chips.com> wrote:

> On 9/18/2026 3:15 AM, No SPAM here wrote:
> > Hi
> >
> > On Thursday, 17 September 2026 at 03:37, Chaoyi Chen <chaoyi.chen@rock-chips.com> wrote:
> >
> >> Hello Trevor,
> >>
> >> On 9/17/2026 9:21 AM, No SPAM here wrote:
> >>>> Would you mind trying to disable `CONFIG_SCHED_CLUSTER` and checking whether
> >>>> the problem still reproduces?
> >
> > trevor@rock-5b:~$ uname -a
> > Linux rock-5b 6.18.52-current-rockchip64 #1 SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 aarch64 GNU/Linux
> > trevor@rock-5b:~$ grep CONFIG_SCHED_CLU /boot/config-6.18.52-current-rockchip64
> > # CONFIG_SCHED_CLUSTER is not set
> > trevor@rock-5b:~$ sar | tail -5
> >
> > 18:50:15        CPU     %user     %nice   %system   %iowait    %steal     %idle
> > 19:00:15        all     99.89      0.00      0.11      0.00      0.00      0.00
> > 19:10:15        all     99.89      0.00      0.11      0.00      0.00      0.00
> > Average:        all     99.89      0.00      0.11      0.00      0.00      0.00
> >
> > Been running stress --cpu 8 since 18:47 and so far it is using all 8 cores at 100%. It would normally have shown the 7/8ths problem after 20s or so, maybe 2 or 3 minutes at max. So I'd say this setting is definitely involved.
> >
> 
> Thank you very much for testing. I'm not a scheduler expert, but I see
> that CONFIG_SCHED_CLUSTER is enabled by default on arm64 [0].
> 
> Peter is the maintainer and author of this series.
> @Peter, do you have any further insights? Thank you!
> 
> This thread background is here[1].
> 
> [0]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7bd291abe2da09f59dca81f35a4ec220e5e138a2
> [1]: https://lore.kernel.org/all/f15f8166-c1e4-45dc-9ddd-4ec887e415eb@gdead.co.uk/
> 
> --
> Best,
> Chaoyi
>

Re: [BUG?] Possible scheduler bug on Rock 5B RK3588 on 7.2.5

[Chaoyi Chen]

On 9/19/2026 12:11 AM, No SPAM here wrote:
> I have now retested with kernel 7.3.0-rc3 and this bug appears to have been fixed. The same command now runs all 8 cores at 100% for the duration of the test and the resulting numbers differ (in favour of the 6.18.52 CLUSTER=n kernel) by a maximum of 0.25% which is measurement error.
> 

Thanks! Looks like this series fixes the problem:

https://lore.kernel.org/all/20260720-rneri-fix-cas-clusters-v6-0-bb500bf4afd4@linux.intel.com/


> Thanks for your attention. This can be marked as solved.
> 
> Trevor
> 
> 
> Sent with Proton Mail secure email.
> 
> On Friday, 18 September 2026 at 02:22, Chaoyi Chen <chaoyi.chen@rock-chips.com> wrote:
> 
>> On 9/18/2026 3:15 AM, No SPAM here wrote:
>>> Hi
>>>
>>> On Thursday, 17 September 2026 at 03:37, Chaoyi Chen <chaoyi.chen@rock-chips.com> wrote:
>>>
>>>> Hello Trevor,
>>>>
>>>> On 9/17/2026 9:21 AM, No SPAM here wrote:
>>>>>> Would you mind trying to disable `CONFIG_SCHED_CLUSTER` and checking whether
>>>>>> the problem still reproduces?
>>>
>>> trevor@rock-5b:~$ uname -a
>>> Linux rock-5b 6.18.52-current-rockchip64 #1 SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 aarch64 GNU/Linux
>>> trevor@rock-5b:~$ grep CONFIG_SCHED_CLU /boot/config-6.18.52-current-rockchip64
>>> # CONFIG_SCHED_CLUSTER is not set
>>> trevor@rock-5b:~$ sar | tail -5
>>>
>>> 18:50:15        CPU     %user     %nice   %system   %iowait    %steal     %idle
>>> 19:00:15        all     99.89      0.00      0.11      0.00      0.00      0.00
>>> 19:10:15        all     99.89      0.00      0.11      0.00      0.00      0.00
>>> Average:        all     99.89      0.00      0.11      0.00      0.00      0.00
>>>
>>> Been running stress --cpu 8 since 18:47 and so far it is using all 8 cores at 100%. It would normally have shown the 7/8ths problem after 20s or so, maybe 2 or 3 minutes at max. So I'd say this setting is definitely involved.
>>>
>>
>> Thank you very much for testing. I'm not a scheduler expert, but I see
>> that CONFIG_SCHED_CLUSTER is enabled by default on arm64 [0].
>>
>> Peter is the maintainer and author of this series.
>> @Peter, do you have any further insights? Thank you!
>>
>> This thread background is here[1].
>>
>> [0]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7bd291abe2da09f59dca81f35a4ec220e5e138a2
>> [1]: https://lore.kernel.org/all/f15f8166-c1e4-45dc-9ddd-4ec887e415eb@gdead.co.uk/
>>
>> --
>> Best,
>> Chaoyi
>>
> 

-- 
Best, 
Chaoyi