From: kirill@shutemov.name (Kirill A. Shutemov)
To: linux-arm-kernel@lists.infradead.org
Subject: [BUG] Read-Only THP causes stalls (commit 10359213d)
Date: Mon, 25 May 2015 13:05:15 +0300 [thread overview]
Message-ID: <20150525100515.GA8275@node.dhcp.inet.fi> (raw)
In-Reply-To: <20150524193404.GD16910@cbox>
On Sun, May 24, 2015 at 09:34:04PM +0200, Christoffer Dall wrote:
> Hi all,
>
> I noticed a regression on my arm64 APM X-Gene system a couple
> of weeks back. I would occassionally see the system lock up and see RCU
> stalls during the caching phase of kernbench. I then wrote a small
> script that does nothing but cache the files
> (http://paste.ubuntu.com/11324767/) and ran that in a loop. On a known
> bad commit (v4.1-rc2), out of 25 boots, I never saw it get past 21
> iterations of the loop. I have since tried to run a bisect from v3.19 to
> v4.0 using 100 iterations as my criteria for a good commit.
>
> This resulted in the following first bad commit:
>
> 10359213d05acf804558bda7cc9b8422a828d1cd
> (mm: incorporate read-only pages into transparent huge pages, 2015-02-11)
>
> Indeed, running the workload on v4.1-rc4 still produced the behavior,
> but reverting the above commit gets me through 100 iterations of the
> loop.
>
> I have not tried to reproduce on an x86 system. Turning on a bunch
> of kernel debugging features *seems* to hide the problem. My config for
> the XGene system is defconfig + CONFIG_BRIDGE and
> CONFIG_POWER_RESET_XGENE.
>
> Please let me know if I can help test patches or other things I can
> do to help. I'm afraid that by simply reading the patch I didn't see
> anything obviously wrong with it which would cause this behavior.
I don't see the problem on x86.
Some backtraces could help to track it down.
--
Kirill A. Shutemov
WARNING: multiple messages have this Message-ID (diff)
From: "Kirill A. Shutemov" <kirill@shutemov.name>
To: Christoffer Dall <christoffer.dall@linaro.org>
Cc: linux-mm@kvack.org, ebru.akagunduz@gmail.com,
akpm@linux-foundation.org, linux-kernel@vger.kernel.org,
kirill.shutemov@linux.intel.com, riel@redhat.com, vbabka@suse.cz,
zhangyanfei@cn.fujitsu.com, aarcange@redhat.com,
Will Deacon <will.deacon@arm.com>,
Andre Przywara <andre.przywara@arm.com>,
Marc Zyngier <marc.zyngier@arm.com>,
linux-arm-kernel@lists.infradead.org
Subject: Re: [BUG] Read-Only THP causes stalls (commit 10359213d)
Date: Mon, 25 May 2015 13:05:15 +0300 [thread overview]
Message-ID: <20150525100515.GA8275@node.dhcp.inet.fi> (raw)
In-Reply-To: <20150524193404.GD16910@cbox>
On Sun, May 24, 2015 at 09:34:04PM +0200, Christoffer Dall wrote:
> Hi all,
>
> I noticed a regression on my arm64 APM X-Gene system a couple
> of weeks back. I would occassionally see the system lock up and see RCU
> stalls during the caching phase of kernbench. I then wrote a small
> script that does nothing but cache the files
> (http://paste.ubuntu.com/11324767/) and ran that in a loop. On a known
> bad commit (v4.1-rc2), out of 25 boots, I never saw it get past 21
> iterations of the loop. I have since tried to run a bisect from v3.19 to
> v4.0 using 100 iterations as my criteria for a good commit.
>
> This resulted in the following first bad commit:
>
> 10359213d05acf804558bda7cc9b8422a828d1cd
> (mm: incorporate read-only pages into transparent huge pages, 2015-02-11)
>
> Indeed, running the workload on v4.1-rc4 still produced the behavior,
> but reverting the above commit gets me through 100 iterations of the
> loop.
>
> I have not tried to reproduce on an x86 system. Turning on a bunch
> of kernel debugging features *seems* to hide the problem. My config for
> the XGene system is defconfig + CONFIG_BRIDGE and
> CONFIG_POWER_RESET_XGENE.
>
> Please let me know if I can help test patches or other things I can
> do to help. I'm afraid that by simply reading the patch I didn't see
> anything obviously wrong with it which would cause this behavior.
I don't see the problem on x86.
Some backtraces could help to track it down.
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: "Kirill A. Shutemov" <kirill@shutemov.name>
To: Christoffer Dall <christoffer.dall@linaro.org>
Cc: linux-mm@kvack.org, ebru.akagunduz@gmail.com,
akpm@linux-foundation.org, linux-kernel@vger.kernel.org,
kirill.shutemov@linux.intel.com, riel@redhat.com, vbabka@suse.cz,
zhangyanfei@cn.fujitsu.com, aarcange@redhat.com,
Will Deacon <will.deacon@arm.com>,
Andre Przywara <andre.przywara@arm.com>,
Marc Zyngier <marc.zyngier@arm.com>,
linux-arm-kernel@lists.infradead.org
Subject: Re: [BUG] Read-Only THP causes stalls (commit 10359213d)
Date: Mon, 25 May 2015 13:05:15 +0300 [thread overview]
Message-ID: <20150525100515.GA8275@node.dhcp.inet.fi> (raw)
In-Reply-To: <20150524193404.GD16910@cbox>
On Sun, May 24, 2015 at 09:34:04PM +0200, Christoffer Dall wrote:
> Hi all,
>
> I noticed a regression on my arm64 APM X-Gene system a couple
> of weeks back. I would occassionally see the system lock up and see RCU
> stalls during the caching phase of kernbench. I then wrote a small
> script that does nothing but cache the files
> (http://paste.ubuntu.com/11324767/) and ran that in a loop. On a known
> bad commit (v4.1-rc2), out of 25 boots, I never saw it get past 21
> iterations of the loop. I have since tried to run a bisect from v3.19 to
> v4.0 using 100 iterations as my criteria for a good commit.
>
> This resulted in the following first bad commit:
>
> 10359213d05acf804558bda7cc9b8422a828d1cd
> (mm: incorporate read-only pages into transparent huge pages, 2015-02-11)
>
> Indeed, running the workload on v4.1-rc4 still produced the behavior,
> but reverting the above commit gets me through 100 iterations of the
> loop.
>
> I have not tried to reproduce on an x86 system. Turning on a bunch
> of kernel debugging features *seems* to hide the problem. My config for
> the XGene system is defconfig + CONFIG_BRIDGE and
> CONFIG_POWER_RESET_XGENE.
>
> Please let me know if I can help test patches or other things I can
> do to help. I'm afraid that by simply reading the patch I didn't see
> anything obviously wrong with it which would cause this behavior.
I don't see the problem on x86.
Some backtraces could help to track it down.
--
Kirill A. Shutemov
next prev parent reply other threads:[~2015-05-25 10:05 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-05-24 19:34 [BUG] Read-Only THP causes stalls (commit 10359213d) Christoffer Dall
2015-05-24 19:34 ` Christoffer Dall
2015-05-24 19:34 ` Christoffer Dall
2015-05-25 10:05 ` Kirill A. Shutemov [this message]
2015-05-25 10:05 ` Kirill A. Shutemov
2015-05-25 10:05 ` Kirill A. Shutemov
2015-05-25 10:19 ` Christoffer Dall
2015-05-25 10:19 ` Christoffer Dall
2015-05-25 10:19 ` Christoffer Dall
2015-05-25 14:15 ` Andrea Arcangeli
2015-05-25 14:15 ` Andrea Arcangeli
2015-05-25 14:15 ` Andrea Arcangeli
2015-05-26 8:08 ` Christoffer Dall
2015-05-26 8:08 ` Christoffer Dall
2015-05-26 8:08 ` Christoffer Dall
2015-05-26 10:57 ` Marc Zyngier
2015-05-26 10:57 ` Marc Zyngier
2015-05-26 10:57 ` Marc Zyngier
2015-05-26 14:24 ` Steve Capper
2015-05-26 14:24 ` Steve Capper
2015-05-26 14:24 ` Steve Capper
2015-05-26 14:35 ` Christoffer Dall
2015-05-26 14:35 ` Christoffer Dall
2015-05-26 14:35 ` Christoffer Dall
2015-05-26 14:48 ` Steve Capper
2015-05-26 14:48 ` Steve Capper
2015-05-26 14:48 ` Steve Capper
2015-05-26 14:58 ` Andrea Arcangeli
2015-05-26 14:58 ` Andrea Arcangeli
2015-05-26 14:58 ` Andrea Arcangeli
2015-05-26 14:42 ` Andrea Arcangeli
2015-05-26 14:42 ` Andrea Arcangeli
2015-05-26 14:42 ` Andrea Arcangeli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150525100515.GA8275@node.dhcp.inet.fi \
--to=kirill@shutemov.name \
--cc=linux-arm-kernel@lists.infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.