From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 254F5C3A5A9 for ; Sat, 2 May 2020 07:52:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 007732173E for ; Sat, 2 May 2020 07:52:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726654AbgEBHwg convert rfc822-to-8bit (ORCPT ); Sat, 2 May 2020 03:52:36 -0400 Received: from james.kirk.hungrycats.org ([174.142.39.145]:39824 "EHLO james.kirk.hungrycats.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726520AbgEBHwf (ORCPT ); Sat, 2 May 2020 03:52:35 -0400 Received: by james.kirk.hungrycats.org (Postfix, from userid 1002) id D5EFC69FD92; Sat, 2 May 2020 03:52:33 -0400 (EDT) Date: Sat, 2 May 2020 03:52:33 -0400 From: Zygo Blaxell To: Phil Karn Cc: Paul Jones , Jean-Denis Girard , "linux-btrfs@vger.kernel.org" Subject: Re: Extremely slow device removals Message-ID: <20200502075233.GN10769@hungrycats.org> References: <8b647a7f-1223-fa9f-57c0-9a81a9bbeb27@ka9q.net> <14a8e382-0541-0f18-b969-ccf4b3254461@ka9q.net> <20200502033509.GG10769@hungrycats.org> <20200502072053.GL10769@hungrycats.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 8BIT In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Sat, May 02, 2020 at 12:27:27AM -0700, Phil Karn wrote: > > deleted the originals in logical extent order. Sometimes people call this > > "defrag free space" but the use of the word "defrag" can be confusing. > > > > balance is not btrfs defrag. defrag is concerned with making data extents > > contiguous, while balance is concerned with making free space contiguous. > > Got it. I actually would have understood "defrag free space" and that > it differed from file defragmentation (btrfs defrag, xfs_fsr, > e4defrag, etc). "Balance" confused me. > > How do you balance free space when you've got drives of unequal sizes, > like my (current) case of a 4-drive array consisting of two 16-TB > drives and two 6-TB drives? Depends on the RAID profile. For single, dup, raid1, raid1c3, and raid1c4, the drives with the most unallocated space are filled first, using devid to break ties. For raid0, raid5, and raid6, drives with free space are filled equally. raid10 fills disks in even-numbered groups of 4 or more drives at a time, filling disks with the most unallocated space first. There are some other rules (e.g. at most 10 disks are used in a single block group) but they're not relevant at this scale. raid1 and single profiles would fill the 16TB drives first, until there was only 6 TB remaining. At this point all the drives would have equal free space, then all drives fill equally until they are all full. raid0 and raid5 would fill all the disks at first, until the 6TB drives are full and the 16TB drives have 10TB of free space, then they'd fill the 16TB drives the rest of the way. raid1c3, raid1c4, raid6 and raid10 would fill all the drives at first, then stop with ENOSPC when the 6TB disks are full and the 16TB disks have 10TB of free space. These profiles have a minimum of 3 or more disks, and you don't have that number of the largest size disks. Once all the smaller disks are full no further allocation can be done. > Phil