Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

[Jon Hunter]

On Mon, 02 Jun 2025 15:47:17 +0200, Greg Kroah-Hartman wrote:
> This is the start of the stable review cycle for the 6.12.32 release.
> There are 55 patches in this series, all will be posted as a response
> to this one.  If anyone has any issues with these being applied, please
> let me know.
> 
> Responses should be made by Wed, 04 Jun 2025 13:42:20 +0000.
> Anything received after that time might be too late.
> 
> The whole patch series can be found in one patch at:
> 	https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.12.32-rc1.gz
> or in the git tree and branch at:
> 	git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.12.y
> and the diffstat can be found below.
> 
> thanks,
> 
> greg k-h

Failures detected for Tegra ...

Test results for stable-v6.12:
    10 builds:	10 pass, 0 fail
    28 boots:	28 pass, 0 fail
    116 tests:	115 pass, 1 fail

Linux version:	6.12.32-rc1-gce2ebbe0294c
Boards tested:	tegra124-jetson-tk1, tegra186-p2771-0000,
                tegra186-p3509-0000+p3636-0001, tegra194-p2972-0000,
                tegra194-p3509-0000+p3668-0000, tegra20-ventana,
                tegra210-p2371-2180, tegra210-p3450-0000,
                tegra30-cardhu-a04

Test failures:	tegra186-p2771-0000: pm-system-suspend.sh


Jon

Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

[Jon Hunter]

Hi Greg,

On 04/06/2025 10:41, Jon Hunter wrote:
> On Mon, 02 Jun 2025 15:47:17 +0200, Greg Kroah-Hartman wrote:
>> This is the start of the stable review cycle for the 6.12.32 release.
>> There are 55 patches in this series, all will be posted as a response
>> to this one.  If anyone has any issues with these being applied, please
>> let me know.
>>
>> Responses should be made by Wed, 04 Jun 2025 13:42:20 +0000.
>> Anything received after that time might be too late.
>>
>> The whole patch series can be found in one patch at:
>> 	https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.12.32-rc1.gz
>> or in the git tree and branch at:
>> 	git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.12.y
>> and the diffstat can be found below.
>>
>> thanks,
>>
>> greg k-h
> 
> Failures detected for Tegra ...
> 
> Test results for stable-v6.12:
>      10 builds:	10 pass, 0 fail
>      28 boots:	28 pass, 0 fail
>      116 tests:	115 pass, 1 fail
> 
> Linux version:	6.12.32-rc1-gce2ebbe0294c
> Boards tested:	tegra124-jetson-tk1, tegra186-p2771-0000,
>                  tegra186-p3509-0000+p3636-0001, tegra194-p2972-0000,
>                  tegra194-p3509-0000+p3668-0000, tegra20-ventana,
>                  tegra210-p2371-2180, tegra210-p3450-0000,
>                  tegra30-cardhu-a04
> 
> Test failures:	tegra186-p2771-0000: pm-system-suspend.sh


I have been looking at this and this appears to be an intermittent 
failure that has crept in. Bisect is point to the following change which 
landed in v6.12.31 and we did not catch it ...

# first bad commit: [d95fdee2253e612216e72f29c65b92ec42d254eb] cpufreq: 
tegra186: Share policy per cluster

I have tested v6.15 which has this change and I don't see the same issue 
there. I have also tested v6.6.y because this was backported to the 
various stable branches and I don't see any problems there. Only v6.12.y 
appears to be impacted which is odd (although this test only runs on 
v6.6+ kernels for this board). However, the testing is conclusive that 
this change is a problem for v6.12.y.

So I think we do need to revert the above change for v6.12.y but I am 
not sure if it makes sense to revert for earlier stable branches too?

Let me know your thoughts.

However, given that this is not a new failure for this stable update we 
can handle in subsequent updates. So for this update ...

Tested-by: Jon Hunter <jonathanh@nvidia.com>

Cheers
Jon

-- 
nvpublic

Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

[Greg Kroah-Hartman]

On Wed, Jun 04, 2025 at 10:57:29AM +0100, Jon Hunter wrote:
> Hi Greg,
> 
> On 04/06/2025 10:41, Jon Hunter wrote:
> > On Mon, 02 Jun 2025 15:47:17 +0200, Greg Kroah-Hartman wrote:
> > > This is the start of the stable review cycle for the 6.12.32 release.
> > > There are 55 patches in this series, all will be posted as a response
> > > to this one.  If anyone has any issues with these being applied, please
> > > let me know.
> > > 
> > > Responses should be made by Wed, 04 Jun 2025 13:42:20 +0000.
> > > Anything received after that time might be too late.
> > > 
> > > The whole patch series can be found in one patch at:
> > > 	https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.12.32-rc1.gz
> > > or in the git tree and branch at:
> > > 	git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.12.y
> > > and the diffstat can be found below.
> > > 
> > > thanks,
> > > 
> > > greg k-h
> > 
> > Failures detected for Tegra ...
> > 
> > Test results for stable-v6.12:
> >      10 builds:	10 pass, 0 fail
> >      28 boots:	28 pass, 0 fail
> >      116 tests:	115 pass, 1 fail
> > 
> > Linux version:	6.12.32-rc1-gce2ebbe0294c
> > Boards tested:	tegra124-jetson-tk1, tegra186-p2771-0000,
> >                  tegra186-p3509-0000+p3636-0001, tegra194-p2972-0000,
> >                  tegra194-p3509-0000+p3668-0000, tegra20-ventana,
> >                  tegra210-p2371-2180, tegra210-p3450-0000,
> >                  tegra30-cardhu-a04
> > 
> > Test failures:	tegra186-p2771-0000: pm-system-suspend.sh
> 
> 
> I have been looking at this and this appears to be an intermittent failure
> that has crept in. Bisect is point to the following change which landed in
> v6.12.31 and we did not catch it ...
> 
> # first bad commit: [d95fdee2253e612216e72f29c65b92ec42d254eb] cpufreq:
> tegra186: Share policy per cluster
> 
> I have tested v6.15 which has this change and I don't see the same issue
> there. I have also tested v6.6.y because this was backported to the various
> stable branches and I don't see any problems there. Only v6.12.y appears to
> be impacted which is odd (although this test only runs on v6.6+ kernels for
> this board). However, the testing is conclusive that this change is a
> problem for v6.12.y.
> 
> So I think we do need to revert the above change for v6.12.y but I am not
> sure if it makes sense to revert for earlier stable branches too?

Yes, let's revert it for the older ones as well as it would look odd,
and our tools might notice that we had "skipped" a stable release tree.

Can you send the revert or do you need us to?

> Let me know your thoughts.
> 
> However, given that this is not a new failure for this stable update we can
> handle in subsequent updates. So for this update ...
> 
> Tested-by: Jon Hunter <jonathanh@nvidia.com>

Wonderful, thanks for testing!

greg k-h

Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

[Jon Hunter]

On 04/06/2025 11:19, Greg Kroah-Hartman wrote:
> On Wed, Jun 04, 2025 at 10:57:29AM +0100, Jon Hunter wrote:
>> Hi Greg,
>>
>> On 04/06/2025 10:41, Jon Hunter wrote:
>>> On Mon, 02 Jun 2025 15:47:17 +0200, Greg Kroah-Hartman wrote:
>>>> This is the start of the stable review cycle for the 6.12.32 release.
>>>> There are 55 patches in this series, all will be posted as a response
>>>> to this one.  If anyone has any issues with these being applied, please
>>>> let me know.
>>>>
>>>> Responses should be made by Wed, 04 Jun 2025 13:42:20 +0000.
>>>> Anything received after that time might be too late.
>>>>
>>>> The whole patch series can be found in one patch at:
>>>> 	https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.12.32-rc1.gz
>>>> or in the git tree and branch at:
>>>> 	git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.12.y
>>>> and the diffstat can be found below.
>>>>
>>>> thanks,
>>>>
>>>> greg k-h
>>>
>>> Failures detected for Tegra ...
>>>
>>> Test results for stable-v6.12:
>>>       10 builds:	10 pass, 0 fail
>>>       28 boots:	28 pass, 0 fail
>>>       116 tests:	115 pass, 1 fail
>>>
>>> Linux version:	6.12.32-rc1-gce2ebbe0294c
>>> Boards tested:	tegra124-jetson-tk1, tegra186-p2771-0000,
>>>                   tegra186-p3509-0000+p3636-0001, tegra194-p2972-0000,
>>>                   tegra194-p3509-0000+p3668-0000, tegra20-ventana,
>>>                   tegra210-p2371-2180, tegra210-p3450-0000,
>>>                   tegra30-cardhu-a04
>>>
>>> Test failures:	tegra186-p2771-0000: pm-system-suspend.sh
>>
>>
>> I have been looking at this and this appears to be an intermittent failure
>> that has crept in. Bisect is point to the following change which landed in
>> v6.12.31 and we did not catch it ...
>>
>> # first bad commit: [d95fdee2253e612216e72f29c65b92ec42d254eb] cpufreq:
>> tegra186: Share policy per cluster
>>
>> I have tested v6.15 which has this change and I don't see the same issue
>> there. I have also tested v6.6.y because this was backported to the various
>> stable branches and I don't see any problems there. Only v6.12.y appears to
>> be impacted which is odd (although this test only runs on v6.6+ kernels for
>> this board). However, the testing is conclusive that this change is a
>> problem for v6.12.y.
>>
>> So I think we do need to revert the above change for v6.12.y but I am not
>> sure if it makes sense to revert for earlier stable branches too?
> 
> Yes, let's revert it for the older ones as well as it would look odd,
> and our tools might notice that we had "skipped" a stable release tree.
> 
> Can you send the revert or do you need us to?

I can no problem. Do you need a revert for each stable branch or just 
one email with the commit to revert for each stable branch?

Jon

-- 
nvpublic

Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

[Greg Kroah-Hartman]

On Wed, Jun 04, 2025 at 11:22:53AM +0100, Jon Hunter wrote:
> 
> On 04/06/2025 11:19, Greg Kroah-Hartman wrote:
> > On Wed, Jun 04, 2025 at 10:57:29AM +0100, Jon Hunter wrote:
> > > Hi Greg,
> > > 
> > > On 04/06/2025 10:41, Jon Hunter wrote:
> > > > On Mon, 02 Jun 2025 15:47:17 +0200, Greg Kroah-Hartman wrote:
> > > > > This is the start of the stable review cycle for the 6.12.32 release.
> > > > > There are 55 patches in this series, all will be posted as a response
> > > > > to this one.  If anyone has any issues with these being applied, please
> > > > > let me know.
> > > > > 
> > > > > Responses should be made by Wed, 04 Jun 2025 13:42:20 +0000.
> > > > > Anything received after that time might be too late.
> > > > > 
> > > > > The whole patch series can be found in one patch at:
> > > > > 	https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.12.32-rc1.gz
> > > > > or in the git tree and branch at:
> > > > > 	git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.12.y
> > > > > and the diffstat can be found below.
> > > > > 
> > > > > thanks,
> > > > > 
> > > > > greg k-h
> > > > 
> > > > Failures detected for Tegra ...
> > > > 
> > > > Test results for stable-v6.12:
> > > >       10 builds:	10 pass, 0 fail
> > > >       28 boots:	28 pass, 0 fail
> > > >       116 tests:	115 pass, 1 fail
> > > > 
> > > > Linux version:	6.12.32-rc1-gce2ebbe0294c
> > > > Boards tested:	tegra124-jetson-tk1, tegra186-p2771-0000,
> > > >                   tegra186-p3509-0000+p3636-0001, tegra194-p2972-0000,
> > > >                   tegra194-p3509-0000+p3668-0000, tegra20-ventana,
> > > >                   tegra210-p2371-2180, tegra210-p3450-0000,
> > > >                   tegra30-cardhu-a04
> > > > 
> > > > Test failures:	tegra186-p2771-0000: pm-system-suspend.sh
> > > 
> > > 
> > > I have been looking at this and this appears to be an intermittent failure
> > > that has crept in. Bisect is point to the following change which landed in
> > > v6.12.31 and we did not catch it ...
> > > 
> > > # first bad commit: [d95fdee2253e612216e72f29c65b92ec42d254eb] cpufreq:
> > > tegra186: Share policy per cluster
> > > 
> > > I have tested v6.15 which has this change and I don't see the same issue
> > > there. I have also tested v6.6.y because this was backported to the various
> > > stable branches and I don't see any problems there. Only v6.12.y appears to
> > > be impacted which is odd (although this test only runs on v6.6+ kernels for
> > > this board). However, the testing is conclusive that this change is a
> > > problem for v6.12.y.
> > > 
> > > So I think we do need to revert the above change for v6.12.y but I am not
> > > sure if it makes sense to revert for earlier stable branches too?
> > 
> > Yes, let's revert it for the older ones as well as it would look odd,
> > and our tools might notice that we had "skipped" a stable release tree.
> > 
> > Can you send the revert or do you need us to?
> 
> I can no problem. Do you need a revert for each stable branch or just one
> email with the commit to revert for each stable branch?

Which ever is easier for you, I can handle the git id "fixups" when
applying them to the different branches if you don't want to have to dig
for them.

thanks,

greg k-h

Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

[Greg Kroah-Hartman]

On Wed, Jun 04, 2025 at 02:41:11AM -0700, Jon Hunter wrote:
> On Mon, 02 Jun 2025 15:47:17 +0200, Greg Kroah-Hartman wrote:
> > This is the start of the stable review cycle for the 6.12.32 release.
> > There are 55 patches in this series, all will be posted as a response
> > to this one.  If anyone has any issues with these being applied, please
> > let me know.
> > 
> > Responses should be made by Wed, 04 Jun 2025 13:42:20 +0000.
> > Anything received after that time might be too late.
> > 
> > The whole patch series can be found in one patch at:
> > 	https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.12.32-rc1.gz
> > or in the git tree and branch at:
> > 	git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.12.y
> > and the diffstat can be found below.
> > 
> > thanks,
> > 
> > greg k-h
> 
> Failures detected for Tegra ...
> 
> Test results for stable-v6.12:
>     10 builds:	10 pass, 0 fail
>     28 boots:	28 pass, 0 fail
>     116 tests:	115 pass, 1 fail
> 
> Linux version:	6.12.32-rc1-gce2ebbe0294c
> Boards tested:	tegra124-jetson-tk1, tegra186-p2771-0000,
>                 tegra186-p3509-0000+p3636-0001, tegra194-p2972-0000,
>                 tegra194-p3509-0000+p3668-0000, tegra20-ventana,
>                 tegra210-p2371-2180, tegra210-p3450-0000,
>                 tegra30-cardhu-a04
> 
> Test failures:	tegra186-p2771-0000: pm-system-suspend.sh

Any hints as to what is causing the failure?

thanks,

greg k-h

Re: [PATCH 6.12 00/55] 6.12.32-rc1 review

[Jon Hunter]

On 04/06/2025 10:58, Greg Kroah-Hartman wrote:
> On Wed, Jun 04, 2025 at 02:41:11AM -0700, Jon Hunter wrote:
>> On Mon, 02 Jun 2025 15:47:17 +0200, Greg Kroah-Hartman wrote:
>>> This is the start of the stable review cycle for the 6.12.32 release.
>>> There are 55 patches in this series, all will be posted as a response
>>> to this one.  If anyone has any issues with these being applied, please
>>> let me know.
>>>
>>> Responses should be made by Wed, 04 Jun 2025 13:42:20 +0000.
>>> Anything received after that time might be too late.
>>>
>>> The whole patch series can be found in one patch at:
>>> 	https://www.kernel.org/pub/linux/kernel/v6.x/stable-review/patch-6.12.32-rc1.gz
>>> or in the git tree and branch at:
>>> 	git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-6.12.y
>>> and the diffstat can be found below.
>>>
>>> thanks,
>>>
>>> greg k-h
>>
>> Failures detected for Tegra ...
>>
>> Test results for stable-v6.12:
>>      10 builds:	10 pass, 0 fail
>>      28 boots:	28 pass, 0 fail
>>      116 tests:	115 pass, 1 fail
>>
>> Linux version:	6.12.32-rc1-gce2ebbe0294c
>> Boards tested:	tegra124-jetson-tk1, tegra186-p2771-0000,
>>                  tegra186-p3509-0000+p3636-0001, tegra194-p2972-0000,
>>                  tegra194-p3509-0000+p3668-0000, tegra20-ventana,
>>                  tegra210-p2371-2180, tegra210-p3450-0000,
>>                  tegra30-cardhu-a04
>>
>> Test failures:	tegra186-p2771-0000: pm-system-suspend.sh
> 
> Any hints as to what is causing the failure?


Yes, I had just responded to my initial email reporting the failure with 
the details when you sent this.

Cheers
Jon

-- 
nvpublic