From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66270C43603 for ; Wed, 18 Dec 2019 08:33:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 44085218AC for ; Wed, 18 Dec 2019 08:33:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725955AbfLRIdQ (ORCPT ); Wed, 18 Dec 2019 03:33:16 -0500 Received: from mx2.suse.de ([195.135.220.15]:34558 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725535AbfLRIdP (ORCPT ); Wed, 18 Dec 2019 03:33:15 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id A4111B274; Wed, 18 Dec 2019 08:33:12 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id AC4EC1E0B2D; Wed, 18 Dec 2019 09:33:01 +0100 (CET) Date: Wed, 18 Dec 2019 09:33:01 +0100 From: Jan Kara To: Paul Richards Cc: Jan Kara , linux-ext4@vger.kernel.org Subject: Re: Query about ext4 commit interval vs dirty_expire_centisecs Message-ID: <20191218083301.GA4083@quack2.suse.cz> References: <20191213155912.GH15474@quack2.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org On Tue 17-12-19 14:42:48, Paul Richards wrote: > On Fri, 13 Dec 2019 at 15:59, Jan Kara wrote: > > > > Hello! > > > > On Tue 19-11-19 08:47:31, Paul Richards wrote: > > > I'm trying to understand the interaction between the ext4 `commit` > > > interval option, and the `vm.dirty_expire_centisecs` tuneable. > > > > > > The ext4 `commit` documentation says: > > > > > > > Ext4 can be told to sync all its data and metadata every 'nrsec' seconds. The default value is 5 seconds. This means that if you lose your power, you will lose as much as the latest 5 seconds of work (your filesystem will not be damaged though, thanks to the journaling). > > > > > > The `dirty_expire_centisecs` documentation says: > > > > > > > This tunable is used to define when dirty data is old enough to be eligible for writeout by the kernel flusher threads. It is expressed in 100'ths of a second. Data which has been dirty in-memory for longer than this interval will be written out next time a flusher thread wakes up. > > > > > > > > > Superficially these sound like they have a very similar effect. They > > > periodically flush out data that hasn't been explicitly fsync'd by the > > > application. I'd like to understand a bit more the interaction > > > between these. > > > > Yes, the effect is rather similar but not quite the same. The first thing > > to observe is kind of obvious fact that ext4 commit interval influences > > just the particular filesystem while dirty_expire_centisecs influences > > behavior of global writeback over all filesystems. > > > > Secondly, commit interval is really the maximum age of ext4 transation. So > > if there is metadata change pending in the journal, it will become > > persistent at latest after this time. So for say 'mkdir' that will be > > persistent at latest after this time. For data operations things are more > > complex. E.g. when delayed allocation is used (which is the default), the > > change gets logged in the journal only during writeback. So it can take up > > to dirty_expire_centisecs for data to be written back from page cache, that > > results in filesystem journalling block allocations etc. and then it can > > take upto commit interval for these changes to become persistent. So in > > this case the intervals add up. There are also other special cases > > somewhere in between but generally it is reasonable to assume that data gets > > automatically persistent in dirty_expire_centisecs + commit_interval time. > > Note both these times are actually times when writeback is triggered so > > if the disk gets too busy, the actual time when data is completely on disk > > may be much higher. > > > > Thanks for taking the time to reply! > > Since automatic persisting of data occurs only after > dirty_expire_centisecs + commit_interval, > should the ext4 docs be corrected? They currently state (for the > commit interval option): > > "The default value is 5 seconds. This means that if you lose > your power, you will lose as much as the latest 5 seconds of work" Yes, probably that should be clarified. Where did you find this wording? Because my ext4 manpage just states: commit=nrsec Start a journal commit every nrsec seconds. The default value is 5 seconds. Zero means default. Honza -- Jan Kara SUSE Labs, CR