Please backport shrink dentry list fixes to stable

Please backport shrink dentry list fixes to stable

[Cong Wang]

Hello,


Sorry to request for backporting another large fixes to stable.

We have seen a list corruption on 3.14 stable kernel (see the end of
this email), which I think is probably fixed by the following list of
patches from Al:

fe91522a7ba82ca1a51b07e19954b3825e4aaa22 don't remove from shrink list
in select_collect()
41edf278fc2f042f4e22a12ed87d19c5201210e1 dentry_kill(): don't try to
remove from shrink list
01b6035190b024240a43ac1d8e9c6f964f5f1c63 expand the call of
dentry_lru_del() in dentry_kill()
b4f0354e968f5fabd39bc85b99fedae4a97589fe new helper: dentry_free()
5c47e6d0ad608987b91affbcf7d1fc12dfbe8fb4 fold try_prune_one_dentry()
03b3b889e79cdb6b806fc0ba9be0d71c186bbfaa fold d_kill() and d_free()


And there are 7 patches to fix the above patches:


00fe425bc28f0ccba034a77783c09c15af4 dcache: add missing lockdep annotation
8cbf74da435d1bd13dbb790f94c7ff67b2fb6af4 dentry_kill() doesn't need
the second argument now
b2b80195d8829921506880f6dccd21cabd163d0d dealing with the rest of
shrink_dentry_list() livelock
046b961b45f93a92e4c70525a12f3d378bced130 shrink_dentry_list(): take
parent's ->d_lock earlier
ff2fde9929feb2aef45377ce56b8b12df85dda69 expand dentry_kill(dentry, 0)
in shrink_dentry_list()
e55fd011549eae01a230e3cace6f4d031b6a3453 split dentry_kill()
64fd72e0a44bdd62c5ca277cb24d0d02b2d8e9dc lift the "already marked
killed" case into shrink_dentry_list()

...and one more to fix up from Linus:

9f12600fe425bc28f0ccba034a77783c09c15af4    dcache: add missing
lockdep annotation

I don't follow vfs development so could easily miss something here, Al
should know much better than I do and may come up with a much easier
way to fix it. Please evaluate this.

Thanks!


------------->

[45204.957357] ------------[ cut here ]------------
[45204.957370] WARNING: CPU: 6 PID: 26375 at lib/list_debug.c:33
__list_add+0x63/0xa7()
[45204.957373] list_add corruption. prev->next should be next
(ffff880c0205d280), but was ffff880cfa3d9d00. (prev=ffff8807c5ad7c08).
[45204.957375] Modules linked in: cls_basic act_mirred cls_u32 veth
sch_ingress netconsole configfs ipv6 dm_multipath scsi_dh video sbs
sbshc acpi_p
ad acpi_ipmi parport_pc lp parport tcp_diag inet_diag ipmi_devintf sg
iTCO_wdt iTCO_vendor_support dcdbas igb i2c_algo_bit ptp pps_core wmi
ipmi_si sb_edac hed edac_core ipmi_msghandler i2c_i801
 ioatdma i2c_core microcode lpc_ich mfd_core dca shpchp ahci libahci
libata sd_mod scsi_mod
[45204.957421] CPU: 6 PID: 26375 Comm: python2.6 Not tainted 3.14.14 #1
[45204.957423] Hardware name: Dell Inc. PowerEdge C6220/03C9JJ, BIOS
1.2.1 05/27/2013
[45204.957426]  0000000000000000 ffff8807c5ad7ad0 ffffffff814a9b67
ffff8807c5ad7b18
[45204.957453]  ffff8807c5ad7b08 ffffffff8105b901 ffffffff812787ff
ffff8807c5ad7c08
[45204.957459]  ffff880c0205d280 ffff880be8bbc8c0 ffff880850fd3680
ffff8807c5ad7b68
[45204.957465] Call Trace:
[45204.957475]  [<ffffffff814a9b67>] dump_stack+0x45/0x56
[45204.957482]  [<ffffffff8105b901>] warn_slowpath_common+0x7f/0x98
[45204.957486]  [<ffffffff812787ff>] ? __list_add+0x63/0xa7
[45204.957490]  [<ffffffff8105b966>] warn_slowpath_fmt+0x4c/0x4e
[45204.957494]  [<ffffffff812787ff>] __list_add+0x63/0xa7
[45204.957500]  [<ffffffff811630fa>] list_add+0xc/0xe
[45204.957504]  [<ffffffff81163146>] d_shrink_add+0x4a/0x5e
[45204.957509]  [<ffffffff81163b0b>] shrink_dentry_list+0xaa/0xbb
[45204.957513]  [<ffffffff81164485>] shrink_dcache_parent+0x50/0x60
[45204.957520]  [<ffffffff811a5070>] proc_flush_task+0x14c/0x18b
[45204.957526]  [<ffffffff8105c49f>] release_task+0x30/0x36d
[45204.957530]  [<ffffffff8105dd43>] do_exit+0x809/0x8ee
[45204.957537]  [<ffffffff810afa48>] ? __unqueue_futex+0x53/0x56
[45204.957541]  [<ffffffff8105dea2>] do_group_exit+0x44/0x9a
[45204.957548]  [<ffffffff81069cc7>] get_signal_to_deliver+0x44b/0x4bc
[45204.957555]  [<ffffffff8100224f>] do_signal+0x49/0x5d5
[45204.957561]  [<ffffffff811442de>] ? kmem_cache_free+0x100/0x138
[45204.957565]  [<ffffffff8105915f>] ? __mmdrop+0x82/0x89
[45204.957570]  [<ffffffff81002804>] do_notify_resume+0x29/0x65
[45204.957576]  [<ffffffff814b5662>] int_signal+0x12/0x17
[45204.957579] ---[ end trace 0decfbe793859fa2 ]---
[45204.959359] ------------[ cut here ]------------

Re: Please backport shrink dentry list fixes to stable

[Holger Hoffstätte]

On Thu, 02 Oct 2014 13:50:24 -0700, Cong Wang wrote:

> Hello,
> 
> 
> Sorry to request for backporting another large fixes to stable.
> 
> We have seen a list corruption on 3.14 stable kernel (see the end of
> this email), which I think is probably fixed by the following list of
> patches from Al:
> 
> fe91522a7ba82ca1a51b07e19954b3825e4aaa22 don't remove from shrink list in select_collect()
> 41edf278fc2f042f4e22a12ed87d19c5201210e1 dentry_kill(): don't try to remove from shrink list
> 01b6035190b024240a43ac1d8e9c6f964f5f1c63 expand the call of dentry_lru_del() in dentry_kill()
> b4f0354e968f5fabd39bc85b99fedae4a97589fe new helper: dentry_free()
> 5c47e6d0ad608987b91affbcf7d1fc12dfbe8fb4 fold try_prune_one_dentry()
> 03b3b889e79cdb6b806fc0ba9be0d71c186bbfaa fold d_kill() and d_free()
> 
> And there are 7 patches to fix the above patches:
> 
> 00fe425bc28f0ccba034a77783c09c15af4 dcache: add missing lockdep annotation

This commit does not exist and seems to be the truncated hash of the
last one (9f12600f..). I guess copypasta salad.

> 8cbf74da435d1bd13dbb790f94c7ff67b2fb6af4 dentry_kill() doesn't need the second argument now
> b2b80195d8829921506880f6dccd21cabd163d0d dealing with the rest of shrink_dentry_list() livelock
> 046b961b45f93a92e4c70525a12f3d378bced130 shrink_dentry_list(): take parent's ->d_lock earlier
> ff2fde9929feb2aef45377ce56b8b12df85dda69 expand dentry_kill(dentry, 0) in shrink_dentry_list()
> e55fd011549eae01a230e3cace6f4d031b6a3453 split dentry_kill()
> 64fd72e0a44bdd62c5ca277cb24d0d02b2d8e9dc lift the "already marked killed" case into shrink_dentry_list()
> 
> ...and one more to fix up from Linus:
> 
> 9f12600fe425bc28f0ccba034a77783c09c15af4    dcache: add missing lockdep annotation
> 
> I don't follow vfs development so could easily miss something here, Al
> should know much better than I do and may come up with a much easier
> way to fix it. Please evaluate this.

I can confirm that these apply correctly (in reverse order per paragraph) over
3.14.19 and that it still compiles.

-h

Re: Please backport shrink dentry list fixes to stable

[Greg KH]

On Fri, Oct 03, 2014 at 08:24:34PM +0000, Holger Hoffstätte wrote:
> 
> On Thu, 02 Oct 2014 13:50:24 -0700, Cong Wang wrote:
> 
> > Hello,
> > 
> > 
> > Sorry to request for backporting another large fixes to stable.
> > 
> > We have seen a list corruption on 3.14 stable kernel (see the end of
> > this email), which I think is probably fixed by the following list of
> > patches from Al:
> > 
> > fe91522a7ba82ca1a51b07e19954b3825e4aaa22 don't remove from shrink list in select_collect()
> > 41edf278fc2f042f4e22a12ed87d19c5201210e1 dentry_kill(): don't try to remove from shrink list
> > 01b6035190b024240a43ac1d8e9c6f964f5f1c63 expand the call of dentry_lru_del() in dentry_kill()
> > b4f0354e968f5fabd39bc85b99fedae4a97589fe new helper: dentry_free()
> > 5c47e6d0ad608987b91affbcf7d1fc12dfbe8fb4 fold try_prune_one_dentry()
> > 03b3b889e79cdb6b806fc0ba9be0d71c186bbfaa fold d_kill() and d_free()
> > 
> > And there are 7 patches to fix the above patches:
> > 
> > 00fe425bc28f0ccba034a77783c09c15af4 dcache: add missing lockdep annotation
> 
> This commit does not exist and seems to be the truncated hash of the
> last one (9f12600f..). I guess copypasta salad.
> 
> > 8cbf74da435d1bd13dbb790f94c7ff67b2fb6af4 dentry_kill() doesn't need the second argument now
> > b2b80195d8829921506880f6dccd21cabd163d0d dealing with the rest of shrink_dentry_list() livelock
> > 046b961b45f93a92e4c70525a12f3d378bced130 shrink_dentry_list(): take parent's ->d_lock earlier
> > ff2fde9929feb2aef45377ce56b8b12df85dda69 expand dentry_kill(dentry, 0) in shrink_dentry_list()
> > e55fd011549eae01a230e3cace6f4d031b6a3453 split dentry_kill()
> > 64fd72e0a44bdd62c5ca277cb24d0d02b2d8e9dc lift the "already marked killed" case into shrink_dentry_list()
> > 
> > ...and one more to fix up from Linus:
> > 
> > 9f12600fe425bc28f0ccba034a77783c09c15af4    dcache: add missing lockdep annotation
> > 
> > I don't follow vfs development so could easily miss something here, Al
> > should know much better than I do and may come up with a much easier
> > way to fix it. Please evaluate this.
> 
> I can confirm that these apply correctly (in reverse order per paragraph) over
> 3.14.19 and that it still compiles.

Compiling is nice, what I would like to see is a way to reproduce the
original problem and proof that backporting all of these intrusive
patches is worth it.  Given that the git commit ids in the list aren't
even correct, I'm a bit loath to do so, sorry.

It's just too big of a chance for -stable.

thanks,

greg k-h

Re: Please backport shrink dentry list fixes to stable

[Holger Hoffstätte]

On 11/19/14 18:39, Greg KH wrote:
> On Fri, Oct 03, 2014 at 08:24:34PM +0000, Holger Hoffstätte wrote:
>>
>> On Thu, 02 Oct 2014 13:50:24 -0700, Cong Wang wrote:
>>
>>> Hello,
>>>
>>>
>>> Sorry to request for backporting another large fixes to stable.
>>>
>>> We have seen a list corruption on 3.14 stable kernel (see the end of
>>> this email), which I think is probably fixed by the following list of
>>> patches from Al:
>>>
>>> fe91522a7ba82ca1a51b07e19954b3825e4aaa22 don't remove from shrink list in select_collect()
>>> 41edf278fc2f042f4e22a12ed87d19c5201210e1 dentry_kill(): don't try to remove from shrink list
>>> 01b6035190b024240a43ac1d8e9c6f964f5f1c63 expand the call of dentry_lru_del() in dentry_kill()
>>> b4f0354e968f5fabd39bc85b99fedae4a97589fe new helper: dentry_free()
>>> 5c47e6d0ad608987b91affbcf7d1fc12dfbe8fb4 fold try_prune_one_dentry()
>>> 03b3b889e79cdb6b806fc0ba9be0d71c186bbfaa fold d_kill() and d_free()
>>>
>>> And there are 7 patches to fix the above patches:
>>>
>>> 00fe425bc28f0ccba034a77783c09c15af4 dcache: add missing lockdep annotation
>>
>> This commit does not exist and seems to be the truncated hash of the
>> last one (9f12600f..). I guess copypasta salad.
>>
>>> 8cbf74da435d1bd13dbb790f94c7ff67b2fb6af4 dentry_kill() doesn't need the second argument now
>>> b2b80195d8829921506880f6dccd21cabd163d0d dealing with the rest of shrink_dentry_list() livelock
>>> 046b961b45f93a92e4c70525a12f3d378bced130 shrink_dentry_list(): take parent's ->d_lock earlier
>>> ff2fde9929feb2aef45377ce56b8b12df85dda69 expand dentry_kill(dentry, 0) in shrink_dentry_list()
>>> e55fd011549eae01a230e3cace6f4d031b6a3453 split dentry_kill()
>>> 64fd72e0a44bdd62c5ca277cb24d0d02b2d8e9dc lift the "already marked killed" case into shrink_dentry_list()
>>>
>>> ...and one more to fix up from Linus:
>>>
>>> 9f12600fe425bc28f0ccba034a77783c09c15af4    dcache: add missing lockdep annotation
>>>
>>> I don't follow vfs development so could easily miss something here, Al
>>> should know much better than I do and may come up with a much easier
>>> way to fix it. Please evaluate this.
>>
>> I can confirm that these apply correctly (in reverse order per paragraph) over
>> 3.14.19 and that it still compiles.
> 
> Compiling is nice, what I would like to see is a way to reproduce the
> original problem and proof that backporting all of these intrusive
> patches is worth it.  Given that the git commit ids in the list aren't
> even correct, I'm a bit loath to do so, sorry.

I should have been more clear - by "still compiles" I meant that it not
only applies & compiles but also runs and doesn't result in any regressions,
at least I have not noticed any on my 3 machines.

As for the hashes: I picked all commits from Linus' main tree and just
now checked via cgit that e.g. the last commit 64fd72 is exactly where
it is supposed to be.

Not sure if that helps..still your call. :)

thanks
Holger

Re: Please backport shrink dentry list fixes to stable

[Greg KH]

On Wed, Nov 19, 2014 at 07:44:43PM +0100, Holger Hoffstätte wrote:
> On 11/19/14 18:39, Greg KH wrote:
> > On Fri, Oct 03, 2014 at 08:24:34PM +0000, Holger Hoffstätte wrote:
> >>
> >> On Thu, 02 Oct 2014 13:50:24 -0700, Cong Wang wrote:
> >>
> >>> Hello,
> >>>
> >>>
> >>> Sorry to request for backporting another large fixes to stable.
> >>>
> >>> We have seen a list corruption on 3.14 stable kernel (see the end of
> >>> this email), which I think is probably fixed by the following list of
> >>> patches from Al:
> >>>
> >>> fe91522a7ba82ca1a51b07e19954b3825e4aaa22 don't remove from shrink list in select_collect()
> >>> 41edf278fc2f042f4e22a12ed87d19c5201210e1 dentry_kill(): don't try to remove from shrink list
> >>> 01b6035190b024240a43ac1d8e9c6f964f5f1c63 expand the call of dentry_lru_del() in dentry_kill()
> >>> b4f0354e968f5fabd39bc85b99fedae4a97589fe new helper: dentry_free()
> >>> 5c47e6d0ad608987b91affbcf7d1fc12dfbe8fb4 fold try_prune_one_dentry()
> >>> 03b3b889e79cdb6b806fc0ba9be0d71c186bbfaa fold d_kill() and d_free()
> >>>
> >>> And there are 7 patches to fix the above patches:
> >>>
> >>> 00fe425bc28f0ccba034a77783c09c15af4 dcache: add missing lockdep annotation
> >>
> >> This commit does not exist and seems to be the truncated hash of the
> >> last one (9f12600f..). I guess copypasta salad.
> >>
> >>> 8cbf74da435d1bd13dbb790f94c7ff67b2fb6af4 dentry_kill() doesn't need the second argument now
> >>> b2b80195d8829921506880f6dccd21cabd163d0d dealing with the rest of shrink_dentry_list() livelock
> >>> 046b961b45f93a92e4c70525a12f3d378bced130 shrink_dentry_list(): take parent's ->d_lock earlier
> >>> ff2fde9929feb2aef45377ce56b8b12df85dda69 expand dentry_kill(dentry, 0) in shrink_dentry_list()
> >>> e55fd011549eae01a230e3cace6f4d031b6a3453 split dentry_kill()
> >>> 64fd72e0a44bdd62c5ca277cb24d0d02b2d8e9dc lift the "already marked killed" case into shrink_dentry_list()
> >>>
> >>> ...and one more to fix up from Linus:
> >>>
> >>> 9f12600fe425bc28f0ccba034a77783c09c15af4    dcache: add missing lockdep annotation
> >>>
> >>> I don't follow vfs development so could easily miss something here, Al
> >>> should know much better than I do and may come up with a much easier
> >>> way to fix it. Please evaluate this.
> >>
> >> I can confirm that these apply correctly (in reverse order per paragraph) over
> >> 3.14.19 and that it still compiles.
> > 
> > Compiling is nice, what I would like to see is a way to reproduce the
> > original problem and proof that backporting all of these intrusive
> > patches is worth it.  Given that the git commit ids in the list aren't
> > even correct, I'm a bit loath to do so, sorry.
> 
> I should have been more clear - by "still compiles" I meant that it not
> only applies & compiles but also runs and doesn't result in any regressions,
> at least I have not noticed any on my 3 machines.
> 
> As for the hashes: I picked all commits from Linus' main tree and just
> now checked via cgit that e.g. the last commit 64fd72 is exactly where
> it is supposed to be.
> 
> Not sure if that helps..still your call. :)

Ok, I'll consider it for a future 3.14 stable release when things slow
down a bit, right now I have plenty for this next release already.

thanks,

greg k-h

Re: Please backport shrink dentry list fixes to stable

[Greg KH]

On Wed, Nov 19, 2014 at 12:38:19PM -0800, Greg KH wrote:
> On Wed, Nov 19, 2014 at 07:44:43PM +0100, Holger Hoffstätte wrote:
> > On 11/19/14 18:39, Greg KH wrote:
> > > On Fri, Oct 03, 2014 at 08:24:34PM +0000, Holger Hoffstätte wrote:
> > >>
> > >> On Thu, 02 Oct 2014 13:50:24 -0700, Cong Wang wrote:
> > >>
> > >>> Hello,
> > >>>
> > >>>
> > >>> Sorry to request for backporting another large fixes to stable.
> > >>>
> > >>> We have seen a list corruption on 3.14 stable kernel (see the end of
> > >>> this email), which I think is probably fixed by the following list of
> > >>> patches from Al:
> > >>>
> > >>> fe91522a7ba82ca1a51b07e19954b3825e4aaa22 don't remove from shrink list in select_collect()
> > >>> 41edf278fc2f042f4e22a12ed87d19c5201210e1 dentry_kill(): don't try to remove from shrink list
> > >>> 01b6035190b024240a43ac1d8e9c6f964f5f1c63 expand the call of dentry_lru_del() in dentry_kill()
> > >>> b4f0354e968f5fabd39bc85b99fedae4a97589fe new helper: dentry_free()
> > >>> 5c47e6d0ad608987b91affbcf7d1fc12dfbe8fb4 fold try_prune_one_dentry()
> > >>> 03b3b889e79cdb6b806fc0ba9be0d71c186bbfaa fold d_kill() and d_free()
> > >>>
> > >>> And there are 7 patches to fix the above patches:
> > >>>
> > >>> 00fe425bc28f0ccba034a77783c09c15af4 dcache: add missing lockdep annotation
> > >>
> > >> This commit does not exist and seems to be the truncated hash of the
> > >> last one (9f12600f..). I guess copypasta salad.
> > >>
> > >>> 8cbf74da435d1bd13dbb790f94c7ff67b2fb6af4 dentry_kill() doesn't need the second argument now
> > >>> b2b80195d8829921506880f6dccd21cabd163d0d dealing with the rest of shrink_dentry_list() livelock
> > >>> 046b961b45f93a92e4c70525a12f3d378bced130 shrink_dentry_list(): take parent's ->d_lock earlier
> > >>> ff2fde9929feb2aef45377ce56b8b12df85dda69 expand dentry_kill(dentry, 0) in shrink_dentry_list()
> > >>> e55fd011549eae01a230e3cace6f4d031b6a3453 split dentry_kill()
> > >>> 64fd72e0a44bdd62c5ca277cb24d0d02b2d8e9dc lift the "already marked killed" case into shrink_dentry_list()
> > >>>
> > >>> ...and one more to fix up from Linus:
> > >>>
> > >>> 9f12600fe425bc28f0ccba034a77783c09c15af4    dcache: add missing lockdep annotation
> > >>>
> > >>> I don't follow vfs development so could easily miss something here, Al
> > >>> should know much better than I do and may come up with a much easier
> > >>> way to fix it. Please evaluate this.
> > >>
> > >> I can confirm that these apply correctly (in reverse order per paragraph) over
> > >> 3.14.19 and that it still compiles.
> > > 
> > > Compiling is nice, what I would like to see is a way to reproduce the
> > > original problem and proof that backporting all of these intrusive
> > > patches is worth it.  Given that the git commit ids in the list aren't
> > > even correct, I'm a bit loath to do so, sorry.
> > 
> > I should have been more clear - by "still compiles" I meant that it not
> > only applies & compiles but also runs and doesn't result in any regressions,
> > at least I have not noticed any on my 3 machines.
> > 
> > As for the hashes: I picked all commits from Linus' main tree and just
> > now checked via cgit that e.g. the last commit 64fd72 is exactly where
> > it is supposed to be.
> > 
> > Not sure if that helps..still your call. :)
> 
> Ok, I'll consider it for a future 3.14 stable release when things slow
> down a bit, right now I have plenty for this next release already.

It looks like Cong provided a better documented / tested series for this
as well, so that will make it easier to accept for a future kernel than
this assorted list of git commit ids...

thanks,

greg k-h

Re: Please backport shrink dentry list fixes to stable

[Jiri Slaby]

On 11/19/2014, 09:42 PM, Greg KH wrote:
> It looks like Cong provided a better documented / tested series for this
> as well, so that will make it easier to accept for a future kernel than
> this assorted list of git commit ids...

On the top of that I only wanted to add here, that it has some testing
coverage from the SLE12 tree too. We were dragging the series in there
for some time until I finally pushed them into 3.12.32.

thanks,
-- 
js
suse labs