4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

[Jens Rottmann]

Hi,

4.1.y stable commit c5ad33184354260be6d05de57e46a5498692f6d6 (Upstream
commit 8f182270dfec432e93fae14f9208a6b9af01009f) "mm/swap.c: flush lru
pvecs on compound page arrival" in 4.1.28 introduces a memory leak.

Simply running

while sleep 0.1; do clear; free; done

shows mem continuously going down, eventually system panics with no
killable processes left. Using "unxz -t some.xz" instead of sleep brings
system down within minutes.

Kmemleak did not report anything. Bisect ended at named commit, and
reverting only this commit is indeed sufficient to fix the leak. Swap
partition on/off makes no difference.

My set-up:
i.MX6 (ARM Cortex-A9) dual-core, 2 GB RAM. Kernel sources are from
git.freescale.com i.e. heavily modified by Freescale for i.MX SoCs,
kernel.org stable patches up to 4.1.28 manually added.

I tried to reproduce with vanilla 4.1.28, but that wouldn't boot at all
on my hardware, hangs immediately after "Starting kernel", sorry.
However there is not a single difference between Freescale and vanilla
in the whole mm/ subdirectory, so I don't think it's i.MX-specific. I
didn't cross-check with an x86 system (yet).

Regards,
Jens

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

[Jens Rottmann]

[-- Attachment #1: Type: text/plain, Size: 54 bytes --]

Attached my .config, in case that helps.
Cheers, Jens

[-- Attachment #2: config.xz --]
[-- Type: application/x-xz, Size: 18448 bytes --]

Re: 4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

[Jens Rottmann]

Hi again,

took lack of response to express reluctance examining vendor kernels. Therefore reproduced and can confirm memory leak on 4.1.28 vanilla x86. Identical symptoms.

Regards,
Jens
________________________________________
From: Jens Rottmann <Jens.Rottmann@ADLINKtech.com>
Sent: Friday, July 15, 2016 21:27
To: Lukasz Odzioba; Sasha Levin
Cc: stable@vger.kernel.org; Michal Hocko; linux-mm@kvack.org; linux-kernel@vger.kernel.org
Subject: 4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

Hi,

4.1.y stable commit c5ad33184354260be6d05de57e46a5498692f6d6 (Upstream
commit 8f182270dfec432e93fae14f9208a6b9af01009f) "mm/swap.c: flush lru
pvecs on compound page arrival" in 4.1.28 introduces a memory leak.

Simply running

while sleep 0.1; do clear; free; done

shows mem continuously going down, eventually system panics with no
killable processes left. Replacing "sleep" with "unxz -t some.xz" brings
system down within minutes.

Kmemleak did not report anything. Bisect ended at named commit, and
reverting only this commit is indeed sufficient to fix the leak. Swap
partition on/off makes no difference.

My set-up:
i.MX6 (ARM Cortex-A9) dual-core, 2 GB RAM. Kernel sources are from
git.freescale.com i.e. heavily modified by Freescale for i.MX SoCs,
kernel.org stable patches up to 4.1.28 manually added.

I tried to reproduce with vanilla 4.1.28, but that wouldn't boot at all
on my i.MX hardware, hangs immediately after "Starting kernel", sorry.
However there is not a single difference between Freescale and vanilla
in the whole mm/ subdirectory, so I don't think it's i.MX-specific. I
didn't cross-check with an x86 system (yet).

Regards,
Jens

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

[Minchan Kim]

On Fri, Jul 15, 2016 at 09:27:55PM +0200, Jens Rottmann wrote:
> Hi,
> 
> 4.1.y stable commit c5ad33184354260be6d05de57e46a5498692f6d6 (Upstream
> commit 8f182270dfec432e93fae14f9208a6b9af01009f) "mm/swap.c: flush lru
> pvecs on compound page arrival" in 4.1.28 introduces a memory leak.
> 
> Simply running
> 
> while sleep 0.1; do clear; free; done
> 
> shows mem continuously going down, eventually system panics with no
> killable processes left. Using "unxz -t some.xz" instead of sleep brings
> system down within minutes.
> 
> Kmemleak did not report anything. Bisect ended at named commit, and
> reverting only this commit is indeed sufficient to fix the leak. Swap
> partition on/off makes no difference.
> 
> My set-up:
> i.MX6 (ARM Cortex-A9) dual-core, 2 GB RAM. Kernel sources are from
> git.freescale.com i.e. heavily modified by Freescale for i.MX SoCs,
> kernel.org stable patches up to 4.1.28 manually added.
> 
> I tried to reproduce with vanilla 4.1.28, but that wouldn't boot at all
> on my hardware, hangs immediately after "Starting kernel", sorry.
> However there is not a single difference between Freescale and vanilla
> in the whole mm/ subdirectory, so I don't think it's i.MX-specific. I
> didn't cross-check with an x86 system (yet).

I didn't have 4.1 stable tree in my local so just looked at git web
and found __lru_cache_add has a bug.

Please change

static void __lru_cache_add(struct page *page)
{
        struct pagevec *pvec = &get_cpu_var(lru_add_pvec);

        page_cache_get(page);
        if (!pagevec_space(pvec) || PageCompound(page)) <==
                __pagevec_lru_add(pvec);
        put_cpu_var(lru_add_pvec);
}

with

static void __lru_cache_add(struct page *page)
{
        struct pagevec *pvec = &get_cpu_var(lru_add_pvec);

        page_cache_get(page);
        if (!pagevec_add(pvec, page) || PageCompound(page)) <==
                __pagevec_lru_add(pvec);
        put_cpu_var(lru_add_pvec);
}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

[Jens Rottmann]

Hi Minchan (& all),

Minchan Kim wrote:
> [...] found __lru_cache_add has a bug. [...]
[-]     if (!pagevec_space(pvec) || PageCompound(page))
[+]     if (!pagevec_add(pvec, page) || PageCompound(page))

Confirm that did plug the leak, thanks!

Also I just saw this was known already:
https://marc.info/?l=linux-kernel&m=146858368215856
Sorry for not noticing earlier, I did search for "4.1.28 memory leak", but not for "memleak".

Many thanks,
Jens

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

[Mikulas Patocka]

On Sat, 16 Jul 2016, Jens Rottmann wrote:

> Hi Minchan (& all),
> 
> Minchan Kim wrote:
> > [...] found __lru_cache_add has a bug. [...]
> [-]     if (!pagevec_space(pvec) || PageCompound(page))
> [+]     if (!pagevec_add(pvec, page) || PageCompound(page))
> 
> Confirm that did plug the leak, thanks!
> 
> Also I just saw this was known already:
> https://marc.info/?l=linux-kernel&m=146858368215856
> Sorry for not noticing earlier, I did search for "4.1.28 memory leak", but not for "memleak".
> 
> Many thanks,
> Jens

For me it fixed the bug too.

Mikulas

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: 4.1.28: memory leak introduced by "mm/swap.c: flush lru pvecs on compound page arrival"

[Michal Hocko]

On Sat 16-07-16 23:47:40, Minchan Kim wrote:
> On Fri, Jul 15, 2016 at 09:27:55PM +0200, Jens Rottmann wrote:
> > Hi,
> > 
> > 4.1.y stable commit c5ad33184354260be6d05de57e46a5498692f6d6 (Upstream
> > commit 8f182270dfec432e93fae14f9208a6b9af01009f) "mm/swap.c: flush lru
> > pvecs on compound page arrival" in 4.1.28 introduces a memory leak.
> > 
> > Simply running
> > 
> > while sleep 0.1; do clear; free; done
> > 
> > shows mem continuously going down, eventually system panics with no
> > killable processes left. Using "unxz -t some.xz" instead of sleep brings
> > system down within minutes.
> > 
> > Kmemleak did not report anything. Bisect ended at named commit, and
> > reverting only this commit is indeed sufficient to fix the leak. Swap
> > partition on/off makes no difference.
> > 
> > My set-up:
> > i.MX6 (ARM Cortex-A9) dual-core, 2 GB RAM. Kernel sources are from
> > git.freescale.com i.e. heavily modified by Freescale for i.MX SoCs,
> > kernel.org stable patches up to 4.1.28 manually added.
> > 
> > I tried to reproduce with vanilla 4.1.28, but that wouldn't boot at all
> > on my hardware, hangs immediately after "Starting kernel", sorry.
> > However there is not a single difference between Freescale and vanilla
> > in the whole mm/ subdirectory, so I don't think it's i.MX-specific. I
> > didn't cross-check with an x86 system (yet).
> 
> I didn't have 4.1 stable tree in my local so just looked at git web
> and found __lru_cache_add has a bug.
> 
> Please change
> 
> static void __lru_cache_add(struct page *page)
> {
>         struct pagevec *pvec = &get_cpu_var(lru_add_pvec);
> 
>         page_cache_get(page);
>         if (!pagevec_space(pvec) || PageCompound(page)) <==
>                 __pagevec_lru_add(pvec);
>         put_cpu_var(lru_add_pvec);
> }
> 
> with
> 
> static void __lru_cache_add(struct page *page)
> {
>         struct pagevec *pvec = &get_cpu_var(lru_add_pvec);
> 
>         page_cache_get(page);
>         if (!pagevec_add(pvec, page) || PageCompound(page)) <==
>                 __pagevec_lru_add(pvec);
>         put_cpu_var(lru_add_pvec);
> }
> 

Yes this is it. Steven has reported that last week and Sasha should be
aware of that http://lkml.kernel.org/r/20160714175521.3675e3d6@gandalf.local.home

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>