From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B648C35247 for ; Fri, 7 Feb 2020 00:09:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C141720838 for ; Fri, 7 Feb 2020 00:09:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="kd6ZoNv7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C141720838 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5A6D46B0003; Thu, 6 Feb 2020 19:09:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5583F6B0006; Thu, 6 Feb 2020 19:09:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46D926B0007; Thu, 6 Feb 2020 19:09:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id 2CAEF6B0003 for ; Thu, 6 Feb 2020 19:09:04 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id D9A56181AC1E9 for ; Fri, 7 Feb 2020 00:09:03 +0000 (UTC) X-FDA: 76461395766.11.baby14_14d7cdd0d1523 X-HE-Tag: baby14_14d7cdd0d1523 X-Filterd-Recvd-Size: 3584 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Fri, 7 Feb 2020 00:09:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=pN0iI2R6rQdmuuVfN/ywYGLta/+ZSKXcgSvDvEZwm1M=; b=kd6ZoNv7jTlQE1C/YZx3xGUfRc AWMtKofCtt9Emz1cr9AUHLgv9drPc/yXf/eSDD8FUNBCGsdEQkIYeAJPjCG98+i8xiJ6eF4ZPhtaX Pvbfx4gp2+lreUyGZhJIKP7uFfxaRZ1YfGSiYiWUWCUzMqD0+g9Zz32od65nlIio3Brnr3qjzSz1L y7pgjrZvl/SQUmcNM7UHq7zcTWsaruRoYht/2muWj6NVmTNOr98gL6PO+8CpA8XZdpehzpcALsW0u 0IKiSmVmhua2ewmV055wJiHFVESufRRbnfadamMvUqtcCjM+LcZxZQ2aF9Y7qgToAv+yx6KrDYtWy WvcVdf9A==; Received: from willy by bombadil.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1izrCX-0007WL-L8; Fri, 07 Feb 2020 00:08:53 +0000 Date: Thu, 6 Feb 2020 16:08:53 -0800 From: Matthew Wilcox To: Dave Chinner Cc: Mel Gorman , Jan Kara , Michal Hocko , lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Mel Gorman Subject: Re: [Lsf-pc] [LSF/MM TOPIC] Congestion Message-ID: <20200207000853.GD8731@bombadil.infradead.org> References: <20191231125908.GD6788@bombadil.infradead.org> <20200106115514.GG12699@dhcp22.suse.cz> <20200106232100.GL23195@dread.disaster.area> <20200109110751.GF27035@quack2.suse.cz> <20200109230043.GS23195@dread.disaster.area> <20200205160551.GI3466@techsingularity.net> <20200206231928.GA21953@dread.disaster.area> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200206231928.GA21953@dread.disaster.area> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Feb 07, 2020 at 10:19:28AM +1100, Dave Chinner wrote: > But detecting an abundance dirty pages/inodes on the LRU doesn't > really solve the problem of determining if and/or how long we should > wait for IO before we try to free more objects. There is no problem > with having lots of dirty pages/inodes on the LRU as long as the IO > subsystem keeps up with the rate at which reclaim is asking them to > be written back via async mechanisms (bdi writeback, metadata > writeback, etc). > > The problem comes when we cannot make efficient progress cleaning > pages/inodes on the LRU because the IO subsystem is overloaded and > cannot clean pages/inodes any faster. At this point, we have to wait > for the IO subsystem to make progress and without feedback from the > IO subsystem, we have no idea how fast that progress is made. Hence > we have no idea how long we need to wait before trying to reclaim > again. i.e. the answer can be different depending on hardware > behaviour, not just the current instantaneous reclaim and IO state. > > That's the fundamental problem we need to solve, and realistically > it can only be done with some level of feedback from the IO > subsystem. That triggered a memory for me. Jeremy Kerr presented a paper at LCA2006 on a different model where the device driver pulls dirty things from the VM rather than having the VM push dirty things to the device driver. It was prototyped in K42 rather than Linux, but the idea might be useful. http://jk.ozlabs.org/projects/k42/ http://jk.ozlabs.org/projects/k42/device-driven-IO-lca06.pdf