From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2413CC5DF60 for ; Tue, 5 Nov 2019 17:05:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CFC7E2087E for ; Tue, 5 Nov 2019 17:05:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XUNB8vdx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CFC7E2087E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7DC576B0005; Tue, 5 Nov 2019 12:05:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 78D2C6B0006; Tue, 5 Nov 2019 12:05:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A2C06B0007; Tue, 5 Nov 2019 12:05:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0046.hostedemail.com [216.40.44.46]) by kanga.kvack.org (Postfix) with ESMTP id 55FED6B0005 for ; Tue, 5 Nov 2019 12:05:18 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 1121E181AEF07 for ; Tue, 5 Nov 2019 17:05:18 +0000 (UTC) X-FDA: 76122849516.03.wheel89_4b4ca1495aa26 X-HE-Tag: wheel89_4b4ca1495aa26 X-Filterd-Recvd-Size: 5625 Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Tue, 5 Nov 2019 17:05:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1572973517; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IsOu4q2X4kFfjJDkAgnsGgmj0ZxKuSVx1mHE6HO+CpA=; b=XUNB8vdxb1Po7L0C3e48ffAVPHkLP76nwyUBE35PotFoQSi4l5/ewocqk6XE3QKJEd3h9e PjB1IgyK53AAXciYlNKFvvqlQfYfifYpfgR0U66T4mFQ3iZIZr00l9JKccmqx8UcjzqfaH cwq23hkevQe/S6aF1ER9GR7mebAHxBE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-37-OAqz_rIDNxGv8KLlhjQsQQ-1; Tue, 05 Nov 2019 12:05:15 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6949E107ACC3; Tue, 5 Nov 2019 17:05:14 +0000 (UTC) Received: from bfoster (dhcp-41-2.bos.redhat.com [10.18.41.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CD02B608AC; Tue, 5 Nov 2019 17:05:13 +0000 (UTC) Date: Tue, 5 Nov 2019 12:05:12 -0500 From: Brian Foster To: Dave Chinner Cc: linux-xfs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 19/28] xfs: reduce kswapd blocking on inode locking. Message-ID: <20191105170512.GC28493@bfoster> References: <20191031234618.15403-1-david@fromorbit.com> <20191031234618.15403-20-david@fromorbit.com> MIME-Version: 1.0 In-Reply-To: <20191031234618.15403-20-david@fromorbit.com> User-Agent: Mutt/1.12.1 (2019-06-15) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: OAqz_rIDNxGv8KLlhjQsQQ-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Nov 01, 2019 at 10:46:09AM +1100, Dave Chinner wrote: > From: Dave Chinner >=20 > When doing async node reclaiming, we grab a batch of inodes that we > are likely able to reclaim and ignore those that are already > flushing. However, when we actually go to reclaim them, the first > thing we do is lock the inode. If we are racing with something > else reclaiming the inode or flushing it because it is dirty, > we block on the inode lock. Hence we can still block kswapd here. >=20 > Further, if we flush an inode, we also cluster all the other dirty > inodes in that cluster into the same IO, flush locking them all. > However, if the workload is operating on sequential inodes (e.g. > created by a tarball extraction) most of these inodes will be > sequntial in the cache and so in the same batch > we've already grabbed for reclaim scanning. >=20 > As a result, it is common for all the inodes in the batch to be > dirty and it is common for the first inode flushed to also flush all > the inodes in the reclaim batch. In which case, they are now all > going to be flush locked and we do not want to block on them. >=20 > Hence, for async reclaim (SYNC_TRYLOCK) make sure we always use > trylock semantics and abort reclaim of an inode as quickly as we can > without blocking kswapd. This will be necessary for the upcoming > conversion to LRU lists for inode reclaim tracking. >=20 > Found via tracing and finding big batches of repeated lock/unlock > runs on inodes that we just flushed by write clustering during > reclaim. >=20 > Signed-off-by: Dave Chinner > Reviewed-by: Christoph Hellwig > --- Reviewed-by: Brian Foster > fs/xfs/xfs_icache.c | 23 ++++++++++++++++++----- > 1 file changed, 18 insertions(+), 5 deletions(-) >=20 > diff --git a/fs/xfs/xfs_icache.c b/fs/xfs/xfs_icache.c > index edcc3f6bb3bf..189cf423fe8f 100644 > --- a/fs/xfs/xfs_icache.c > +++ b/fs/xfs/xfs_icache.c > @@ -1104,11 +1104,23 @@ xfs_reclaim_inode( > =20 > restart: > =09error =3D 0; > -=09xfs_ilock(ip, XFS_ILOCK_EXCL); > -=09if (!xfs_iflock_nowait(ip)) { > -=09=09if (!(sync_mode & SYNC_WAIT)) > +=09/* > +=09 * Don't try to flush the inode if another inode in this cluster has > +=09 * already flushed it after we did the initial checks in > +=09 * xfs_reclaim_inode_grab(). > +=09 */ > +=09if (sync_mode & SYNC_TRYLOCK) { > +=09=09if (!xfs_ilock_nowait(ip, XFS_ILOCK_EXCL)) > =09=09=09goto out; > -=09=09xfs_iflock(ip); > +=09=09if (!xfs_iflock_nowait(ip)) > +=09=09=09goto out_unlock; > +=09} else { > +=09=09xfs_ilock(ip, XFS_ILOCK_EXCL); > +=09=09if (!xfs_iflock_nowait(ip)) { > +=09=09=09if (!(sync_mode & SYNC_WAIT)) > +=09=09=09=09goto out_unlock; > +=09=09=09xfs_iflock(ip); > +=09=09} > =09} > =20 > =09if (XFS_FORCED_SHUTDOWN(ip->i_mount)) { > @@ -1215,9 +1227,10 @@ xfs_reclaim_inode( > =20 > out_ifunlock: > =09xfs_ifunlock(ip); > +out_unlock: > +=09xfs_iunlock(ip, XFS_ILOCK_EXCL); > out: > =09xfs_iflags_clear(ip, XFS_IRECLAIM); > -=09xfs_iunlock(ip, XFS_ILOCK_EXCL); > =09/* > =09 * We could return -EAGAIN here to make reclaim rescan the inode tree= in > =09 * a short while. However, this just burns CPU time scanning the tree > --=20 > 2.24.0.rc0 >=20