* /sys/kernel/debug/kmemleak empty despite kmemleak reports
@ 2020-07-09 14:37 Paul Menzel
2020-07-09 17:57 ` Catalin Marinas
0 siblings, 1 reply; 5+ messages in thread
From: Paul Menzel @ 2020-07-09 14:37 UTC (permalink / raw)
To: Catalin Marinas, Michael Ellerman; +Cc: linuxppc-dev
Dear Linux folks,
Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC,
the file does not contain more information.
> $ dmesg
> […] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering
kernel.perf_event_max_sample_rate to 77750
> [48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
> [52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
> [52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
> [53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
> [53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
> [57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
> [57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
> [61608.697728] Can't find PMC that caused IRQ
> [71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> [92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
> [92849.380672] FS-Cache: Loaded
> [92849.417269] FS-Cache: Netfs 'nfs' registered for caching
> [92849.595974] NFS: Registering the id_resolver key type
> [92849.596000] Key type id_resolver registered
> [92849.596000] Key type id_legacy registered
> [101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> [106904.323471] Can't find PMC that caused IRQ
> [129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> [158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> $ sudo cat /sys/kernel/debug/kmemleak
> $
Kind regards,
Paul
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: /sys/kernel/debug/kmemleak empty despite kmemleak reports
2020-07-09 14:37 /sys/kernel/debug/kmemleak empty despite kmemleak reports Paul Menzel
@ 2020-07-09 17:57 ` Catalin Marinas
2020-07-09 21:08 ` Paul Menzel
0 siblings, 1 reply; 5+ messages in thread
From: Catalin Marinas @ 2020-07-09 17:57 UTC (permalink / raw)
To: Paul Menzel; +Cc: linuxppc-dev
On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
> Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
> file does not contain more information.
>
> > $ dmesg
> > […] > [48662.953323] perf: interrupt took too long (2570 > 2500),
> > lowering
> kernel.perf_event_max_sample_rate to 77750
> > [48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
> > [52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
> > [52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
> > [53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
> > [53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
> > [57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
> > [57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
> > [61608.697728] Can't find PMC that caused IRQ
> > [71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > [92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
> > [92849.380672] FS-Cache: Loaded
> > [92849.417269] FS-Cache: Netfs 'nfs' registered for caching
> > [92849.595974] NFS: Registering the id_resolver key type
> > [92849.596000] Key type id_resolver registered
> > [92849.596000] Key type id_legacy registered
> > [101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > [106904.323471] Can't find PMC that caused IRQ
> > [129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > [158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > $ sudo cat /sys/kernel/debug/kmemleak
When they are no longer present, they are most likely false positives.
Was this triggered during boot? Or under some workload?
--
Catalin
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: /sys/kernel/debug/kmemleak empty despite kmemleak reports
2020-07-09 17:57 ` Catalin Marinas
@ 2020-07-09 21:08 ` Paul Menzel
2020-07-13 18:27 ` Catalin Marinas
0 siblings, 1 reply; 5+ messages in thread
From: Paul Menzel @ 2020-07-09 21:08 UTC (permalink / raw)
To: Catalin Marinas; +Cc: linuxppc-dev
Dear Catalin,
Am 09.07.20 um 19:57 schrieb Catalin Marinas:
> On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
>> Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
>> file does not contain more information.
>>
>>> $ dmesg
>>> […] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering kernel.perf_event_max_sample_rate to 77750
>>> [48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
>>> [52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
>>> [52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
>>> [53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
>>> [53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
>>> [57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
>>> [57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
>>> [61608.697728] Can't find PMC that caused IRQ
>>> [71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>> [92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
>>> [92849.380672] FS-Cache: Loaded
>>> [92849.417269] FS-Cache: Netfs 'nfs' registered for caching
>>> [92849.595974] NFS: Registering the id_resolver key type
>>> [92849.596000] Key type id_resolver registered
>>> [92849.596000] Key type id_legacy registered
>>> [101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>> [106904.323471] Can't find PMC that caused IRQ
>>> [129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>> [158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>> $ sudo cat /sys/kernel/debug/kmemleak
>
> When they are no longer present, they are most likely false positives.
How can this be? Shouldn’t the false positive also be logged in
`/sys/kernel/debug/kmemleak`?
> Was this triggered during boot? Or under some workload?
From the timestamps it looks like under some load.
Kind regards,
Paul
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: /sys/kernel/debug/kmemleak empty despite kmemleak reports
2020-07-09 21:08 ` Paul Menzel
@ 2020-07-13 18:27 ` Catalin Marinas
2020-07-14 6:59 ` Paul Menzel
0 siblings, 1 reply; 5+ messages in thread
From: Catalin Marinas @ 2020-07-13 18:27 UTC (permalink / raw)
To: Paul Menzel; +Cc: linuxppc-dev
On Thu, Jul 09, 2020 at 11:08:52PM +0200, Paul Menzel wrote:
> Am 09.07.20 um 19:57 schrieb Catalin Marinas:
> > On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
> > > Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
> > > file does not contain more information.
> > >
> > > > $ dmesg
> > > > […] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering kernel.perf_event_max_sample_rate to 77750
> > > > [48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
> > > > [52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
> > > > [52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
> > > > [53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
> > > > [53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
> > > > [57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
> > > > [57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
> > > > [61608.697728] Can't find PMC that caused IRQ
> > > > [71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > > > [92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
> > > > [92849.380672] FS-Cache: Loaded
> > > > [92849.417269] FS-Cache: Netfs 'nfs' registered for caching
> > > > [92849.595974] NFS: Registering the id_resolver key type
> > > > [92849.596000] Key type id_resolver registered
> > > > [92849.596000] Key type id_legacy registered
> > > > [101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > > > [106904.323471] Can't find PMC that caused IRQ
> > > > [129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > > > [158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
> > > > $ sudo cat /sys/kernel/debug/kmemleak
> >
> > When they are no longer present, they are most likely false positives.
>
> How can this be? Shouldn’t the false positive also be logged in
> `/sys/kernel/debug/kmemleak`?
Sorry, I wasn't clear. It can be a transient false positive. At a
subsequent scan, kmemleak found pointer referring the previously
reported objects and no longer shows them.
> > Was this triggered during boot? Or under some workload?
>
> From the timestamps it looks like under some load.
Was it during boot? I put a delay of 60s to avoid this but, depending on
the platform, it can still trigger.
--
Catalin
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: /sys/kernel/debug/kmemleak empty despite kmemleak reports
2020-07-13 18:27 ` Catalin Marinas
@ 2020-07-14 6:59 ` Paul Menzel
0 siblings, 0 replies; 5+ messages in thread
From: Paul Menzel @ 2020-07-14 6:59 UTC (permalink / raw)
To: Catalin Marinas; +Cc: linuxppc-dev
Dear Catalin,
Am 13.07.20 um 20:27 schrieb Catalin Marinas:
> On Thu, Jul 09, 2020 at 11:08:52PM +0200, Paul Menzel wrote:
>> Am 09.07.20 um 19:57 schrieb Catalin Marinas:
>>> On Thu, Jul 09, 2020 at 04:37:10PM +0200, Paul Menzel wrote:
>>>> Despite Linux 5.8-rc4 reporting memory leaks on the IBM POWER 8 S822LC, the
>>>> file does not contain more information.
>>>>
>>>>> $ dmesg
>>>>> […] > [48662.953323] perf: interrupt took too long (2570 > 2500), lowering kernel.perf_event_max_sample_rate to 77750
>>>>> [48854.810636] perf: interrupt took too long (3216 > 3212), lowering kernel.perf_event_max_sample_rate to 62000
>>>>> [52300.044518] perf: interrupt took too long (4244 > 4020), lowering kernel.perf_event_max_sample_rate to 47000
>>>>> [52751.373083] perf: interrupt took too long (5373 > 5305), lowering kernel.perf_event_max_sample_rate to 37000
>>>>> [53354.000363] perf: interrupt took too long (6793 > 6716), lowering kernel.perf_event_max_sample_rate to 29250
>>>>> [53850.215606] perf: interrupt took too long (8672 > 8491), lowering kernel.perf_event_max_sample_rate to 23000
>>>>> [57542.266099] perf: interrupt took too long (10940 > 10840), lowering kernel.perf_event_max_sample_rate to 18250
>>>>> [57559.645404] perf: interrupt took too long (13714 > 13675), lowering kernel.perf_event_max_sample_rate to 14500
>>>>> [61608.697728] Can't find PMC that caused IRQ
>>>>> [71774.463111] kmemleak: 12 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>>>> [92372.044785] process '@/usr/bin/gnatmake-5' started with executable stack
>>>>> [92849.380672] FS-Cache: Loaded
>>>>> [92849.417269] FS-Cache: Netfs 'nfs' registered for caching
>>>>> [92849.595974] NFS: Registering the id_resolver key type
>>>>> [92849.596000] Key type id_resolver registered
>>>>> [92849.596000] Key type id_legacy registered
>>>>> [101808.079143] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>>>> [106904.323471] Can't find PMC that caused IRQ
>>>>> [129416.391456] kmemleak: 1 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>>>> [158171.604221] kmemleak: 34 new suspected memory leaks (see /sys/kernel/debug/kmemleak)
>>>>> $ sudo cat /sys/kernel/debug/kmemleak
>>>
>>> When they are no longer present, they are most likely false positives.
>>
>> How can this be? Shouldn’t the false positive also be logged in
>> `/sys/kernel/debug/kmemleak`?
>
> Sorry, I wasn't clear. It can be a transient false positive. At a
> subsequent scan, kmemleak found pointer referring the previously
> reported objects and no longer shows them.
Interesting. Is it possible to print a message in that case to avoid
confusion?
>>> Was this triggered during boot? Or under some workload?
>>
>> From the timestamps it looks like under some load.
>
> Was it during boot? I put a delay of 60s to avoid this but, depending on
> the platform, it can still trigger.
No, it happened after several hours of runtime.
Kind regards,
Paul
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-07-14 7:01 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-07-09 14:37 /sys/kernel/debug/kmemleak empty despite kmemleak reports Paul Menzel
2020-07-09 17:57 ` Catalin Marinas
2020-07-09 21:08 ` Paul Menzel
2020-07-13 18:27 ` Catalin Marinas
2020-07-14 6:59 ` Paul Menzel
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).