From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAB79C4360F for ; Tue, 19 Mar 2019 15:17:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8D50C20850 for ; Tue, 19 Mar 2019 15:17:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727572AbfCSPRf (ORCPT ); Tue, 19 Mar 2019 11:17:35 -0400 Received: from outgoing-auth-1.mit.edu ([18.9.28.11]:40499 "EHLO outgoing.mit.edu" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726703AbfCSPRe (ORCPT ); Tue, 19 Mar 2019 11:17:34 -0400 Received: from callcc.thunk.org (guestnat-104-133-0-99.corp.google.com [104.133.0.99] (may be forged)) (authenticated bits=0) (User authenticated as tytso@ATHENA.MIT.EDU) by outgoing.mit.edu (8.14.7/8.12.4) with ESMTP id x2JFHAEM010711 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Mar 2019 11:17:11 -0400 Received: by callcc.thunk.org (Postfix, from userid 15806) id ED39E420AA8; Tue, 19 Mar 2019 11:17:09 -0400 (EDT) Date: Tue, 19 Mar 2019 11:17:09 -0400 From: "Theodore Ts'o" To: Vijay Chidambaram Cc: Amir Goldstein , Dave Chinner , Jayashree , fstests , linux-fsdevel , linux-doc@vger.kernel.org, chao@kernel.org, Filipe Manana , Jonathan Corbet , Josef Bacik , Anna Schumaker Subject: Re: [PATCH v2] Documenting the crash-recovery guarantees of Linux file systems Message-ID: <20190319151709.GB23187@mit.edu> References: <1552418820-18102-1-git-send-email-jaya@cs.utexas.edu> <20190314011925.GG23020@dastard> <20190315030313.GP26298@dastard> <20190317221652.GQ26298@dastard> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org On Mon, Mar 18, 2019 at 09:37:28PM -0500, Vijay Chidambaram wrote: > For new folks on the thread, I'm Vijay Chidambaram, prof at UT Austin > and Jayashree's advisor. We recently developed CrashMonkey, a tool for > finding crash-consistency bugs in file systems. As part of the > research effort, we had a lot of conversations with file-system > developers to understand the guarantees provided by different file > systems. This patch was inspired by the thought that we should quickly > document what we know about the data integrity guarantees of different > file systems. We did not expect to spur debate! > > Thanks Dave, Amir, and Ted for the discussion. We will incorporate > these comments into the next patch. If it is better to wait until a > consensus is reached after the LSF meeting, we'd be happy to do so. Something to consider is that certain side effects of what fsync(2) or fdatasync(2) might drag into the jbd2 transaction might change if we were to implement (for example) something like Daejun Park and Dongkun Shin's "iJournaling: Fine-grained journaling for improving the latency of fsync system call" published in Usenix, ATC 2017: https://www.usenix.org/system/files/conference/atc17/atc17-park.pdf That's an example of how if we document synchronization that goes beyond POSIX, it might change in the future. So if it gets documented, applications might start becoming unreliable on FreeBSD, MacOS, etc. And maybe as Linux developers we won't care about that; since it increases Linux lock-in. Win! (If you think like Steve Ballmer, anyway. :-) But then if we were to implement something like incremental journaling for fsync, and applications were to start assuming that it would also work, application authors might complain that we had broken their application So they might call the new feature a *BUG* which broke backwards compatibility, and then demand that we either withdraw the new feature, or complicate our testing matrix by adding Yet Another Mount Option. (That's especially true since iJournaling is a performance improvement that doesn't require an on-disk format change. So this is the sort of thing that we might want to enable by default eventually, even if initially it's only enabled via a mount option while we are stablizing the new feature.) So my concerns are not a theoretical, abstract concern, but something which is very real. Implementing something like what Park and Shin has proposed is something that is very much that we are thinking about. - Ted