From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:1170 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752060AbcGSCF6 (ORCPT ); Mon, 18 Jul 2016 22:05:58 -0400 From: Calvin Owens Subject: Re: [BUG] Slab corruption during XFS writeback under memory pressure To: Dave Chinner References: <28f77d74-5ab4-d913-2921-df90da53f393@fb.com> <20160717000003.GW1922@dastard> <20160718060215.GB16044@dastard> CC: , , , , Message-ID: <24d2f83f-5281-ab3c-9e91-985a4b8e2f8b@fb.com> Date: Mon, 18 Jul 2016 19:05:44 -0700 MIME-Version: 1.0 In-Reply-To: <20160718060215.GB16044@dastard> Content-Type: text/plain; charset="windows-1252"; format=flowed Sender: linux-block-owner@vger.kernel.org List-Id: linux-block@vger.kernel.org On 07/17/2016 11:02 PM, Dave Chinner wrote: > On Sun, Jul 17, 2016 at 10:00:03AM +1000, Dave Chinner wrote: >> On Fri, Jul 15, 2016 at 05:18:02PM -0700, Calvin Owens wrote: >>> Hello all, >>> >>> I've found a nasty source of slab corruption. Based on seeing similar symptoms >>> on boxes at Facebook, I suspect it's been around since at least 3.10. >>> >>> It only reproduces under memory pressure so far as I can tell: the issue seems >>> to be that XFS reclaims pages from buffers that are still in use by >>> scsi/block. I'm not sure which side the bug lies on, but I've only observed it >>> with XFS. > [....] >> But this indicates that the page is under writeback at this point, >> so that tends to indicate that the above freeing was incorrect. >> >> Hmmm - it's clear we've got direct reclaim involved here, and the >> suspicion of a dirty page that has had it's bufferheads cleared. >> Are there any other warnings in the log from XFS prior to kasan >> throwing the error? > > Can you try the patch below? Thanks for getting this out so quickly :) So far so good: I booted Linus' tree as of this morning and reproduced the ASAN splat. After applying your patch I haven't triggered it. I'm a bit wary since it was hard to trigger reliably in the first place... so I lined up a few dozen boxes to run the test case overnight. I'll confirm in the morning (-0700) they look good. Thanks, Calvin > -Dave.