Re: [PATCH] perf tests record: allow for some difference in cycle count in leader sampling test on aarch64

[James Clark <james.clark@linaro.org>]

On 13/10/2025 4:36 pm, Arnaldo Carvalho de Melo wrote:
> On Thu, Oct 09, 2025 at 03:59:54PM +0100, James Clark wrote:
>>
>>
>> On 09/10/2025 3:43 pm, Anubhav Shelat wrote:
>>> The first machine was running kernel 6.12.0-55.37.1.el10_0.aarch64 on a KVM
>>> virtual machine.
>>> The second machine was running kernel 6.12.0-119.el10.aarch64 also on a KVM.
>   
>> That's quite old. Make sure you test on the latest kernel before sending
> 
> While I agree with you I think that 6.12 is not really that 6.12ish, as
> its a disto kernel, an enterprise one at that, so tons of backports.
> 
> Having said that, yeah, the right thing is to build the latest upstream
> kernel and see if it works, if it works, try to identify backports, and
> only when its determined that it is something present on upstream,
> report it publicly.
> 
>> patches. The tests in mainline should be targeting the latest kernel,
>> especially in this case because the throttling fix didn't have a fixes tag
>> so won't be backported.
> 
> Right.
>   
>> That change to fix throttling and group sampling is only from v6.16.
>   
>> Also what hardware is the VM running on?
> 
> Anubhav, please provide this info,

On the adjacent thread:

 > Both were running on an HPE Apollo CN99XX server.

But now that I've seen the older kernel version, I think the issue is 
that the throttling fix isn't present and the specific hardware isn't 
important. If Anubhav confirms a newer kernel fixes it we should look 
into possibly removing the tolerance from the test and start looking for 
exact matches.

James

> 
> Thanks,
> 
> - Arnaldo
>   
>>> On Thu, Oct 9, 2025 at 3:17 PM James Clark <james.clark@linaro.org> wrote:
>>>> After reading that patch it seems like we should actually be removing
>>>> the 80% tolerance from the leader sampling test. Both instances of the
>>>> cycles counts should be the same now.
> 
>>> If there's no tolerance then the leader sampling test would fail much more
>>> often. In most of the runs there's at least one case where the leader event
>>> has much fewer cycles.
>   
>> That's assuming that we've agreed that any difference in cycle counts is
>> expected and valid. I don't agree that's the case yet and I think it's a
>> bug. I only see identical counts, and the commit message in Kan's fix
>> describes that the values should be the same for all architectures.
>   
>>>> (Excluding s390) I'm starting to think you were hitting this bug on an
>>>> older kernel? Or something else is going wrong that we should  get to
>>>> the bottom of. The test could have found something and we shouldn't
>>>> ignore it yet.
> 
>>> I agree that the first bug I mentioned might be from an older kernel, but
>>> there's still the case here where the cycle counts don't match. I'll keep
>>> looking into it.