From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BDDBC433F5 for ; Mon, 14 Mar 2022 22:53:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245671AbiCNWye (ORCPT ); Mon, 14 Mar 2022 18:54:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33238 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238823AbiCNWyc (ORCPT ); Mon, 14 Mar 2022 18:54:32 -0400 Received: from drax.kayaks.hungrycats.org (drax.kayaks.hungrycats.org [174.142.148.226]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 62BCC13F25 for ; Mon, 14 Mar 2022 15:53:21 -0700 (PDT) Received: by drax.kayaks.hungrycats.org (Postfix, from userid 1002) id 76AF125991B; Mon, 14 Mar 2022 18:52:10 -0400 (EDT) Date: Mon, 14 Mar 2022 18:51:36 -0400 From: Zygo Blaxell To: Remi Gauvin Cc: linux-btrfs Subject: Re: Btrfs autodefrag wrote 5TB in one day to a 0.5TB SSD without a measurable benefit Message-ID: References: <87tuc9q1fc.fsf@vps.thesusis.net> <87tuc7gdzp.fsf@vps.thesusis.net> <87ee34cnaq.fsf@vps.thesusis.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Mon, Mar 14, 2022 at 06:24:43PM -0400, Remi Gauvin wrote: > On 2022-03-14 5:53 p.m., Jan Ziak wrote: > > > > .... > > > > In this test case, "Disk Usage" is 60% higher than the file's size: > > > > $ compsize data > > Processed 1 file, 612 regular extents (1221 refs), 0 inline. > > Type Perc Disk Usage Uncompressed Referenced > > TOTAL 100% 16M 16M 10M > > > It would be nice if we could get a mount option to specify maximum > extent size, so this effect could be minimized on SSD without having to > use compress-force. (Or maybe this should be the default When ssd mode > is automaticallyd detected.) If you never use prealloc or defrag, it's usually not a problem. Files mostly fall into two categories: big sequential writes (where big extents are better) or small random writes (where big extents are bad, but you don't have any of those because you're doing small random writes all the time). Writeback gets this right most of the time, so the extents end up the right sizes on disk. If all your writes are random, short, and aligned to a multiple of 4K, then you'll end up in a steady state with a lot of short extents and little to no wasted space. If you run defrag on that, you end up with half the space wasted, and if the writes continue, lots of small extents either way. Prealloc's bad effects are similar to defrag, but with more reliable losses. A mount option to disable prealloc globally might be very useful--I run a number of apps that think prealloc doesn't waste huge amounts of CPU time and disk space on datacow files, and I grow weary of patching or LD_PRELOAD-hacking them all the time to not call fallocate().