Re: Btrfs balance

Re: Btrfs balance

[Tomasz Chmielewski]

> Here (Kernel 2.6.37, btrfs git Nov. 2010): balancing two disks/
> partitions with 2 and 1.5 TByte needs about 24 hours (CPU 1.5 GHz).
>
> What do you mean with "lots of IO"? the messages in "/var/log/messages"
> and "/var/log/warn"?

Simply disk activity.

See i.e. "iostat -k 1".


-- 
Tomasz Chmielewski
http://wpkg.org

Btrfs balance

[Andreas Philipp]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
 
Hi,

Maybe it is a very stupid question but I want to ask it anyway. In
general, 'btrfs filesystem balance' takes very long to finish and
produces lots of IO. So what are the classical usage scenarios, when
it is (really) worth doing a balance?

Thanks,
Andreas Philipp
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
 
iQIcBAEBAgAGBQJNODOKAAoJEJIcBJ3+XkgiUisP/A6PywwxIBFj9NuiQ9UAA5vY
QJRn/tXT+Ue2wqgZjhGaqD71Q36ALchSXonX9EXoHWRl0VtrX/2MKkPfEZvBXcDs
yVLOj647HgFu6e51gmdgGvx3qtLsQlFpXBtqYXU8+aeMUUicUZsY8ym+7w6L3jKl
0mKF7nRkdbF0JyBg4NqvbABtBrXEXIEz7hY0oQtFkjuu72iiAriLuKRRkiTnAck9
gkQ1aqcCLLUq7TPQsCdto6s7uSoQPQnZ1zXQ1G1Ij0jUItkU9h9bIjkAcbAjHV4n
cVVF5Zz2lk+z4eAHZwAKgyzzaYW5DT7+ZNrCDOls1CPQr0xka/giRa1dYjS2sas0
GZhPKtCljfROFsOGRs//DcFyu6P3X9WUHM7JRl3v5m0O1EyKlFKMP5O++BNEiB+I
9xUDnYqXzh64MiJBUk5WwGIJuHzZtlfq/PPznWoA2BpO/0/yODKyvd8MkXyEs3ht
3aHi9nbLBe7KqF7CrwH1epchoGKQw/iWopLJMFT6YVL+fHu3CC1aclzekdUh7ktX
ybdW+x575XixhCMrOw9xl0a2dRRIozBYWi5t56rpx8UMkwJwBmvwUOzS1ap9SZLx
c3jSiph8VR8cxtDu/NNFs46iXD/5I7EYlgin3FxD2qrYJOJAcR4d5V08CwGxM+/s
rnskJ6vWv4P71hvevI+R
=XwNW
-----END PGP SIGNATURE-----

Re: Btrfs balance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1087 bytes --]

On Thu, Jan 20, 2011 at 02:07:23PM +0100, Andreas Philipp wrote:
> Hi,
> 
> Maybe it is a very stupid question but I want to ask it anyway. In
> general, 'btrfs filesystem balance' takes very long to finish and
> produces lots of IO. So what are the classical usage scenarios, when
> it is (really) worth doing a balance?

   The primary use-cases for balancing are to even out the filesystem
after adding, removing or changing the size of one of the underlying
volumes.

   It will also be of use when we finally get around to allowing you
to change RAID settings on the whole volume, to implement the
requested changes to the RAID level.

   I'm in the process of implementing balance filters, so that some
other cases where balancing is useful (reclaiming unused block groups)
can be run more efficiently by only balancing the bits that need
doing.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
               --- Prisoner unknown:  Return to Zenda. ---               

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Btrfs balance

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2285 bytes --]

On Thu, Jan 20, 2011 at 03:53:41PM +0100, Andreas Philipp wrote:
> On 20.01.2011 14:39, Hugo Mills wrote:
> > On Thu, Jan 20, 2011 at 02:07:23PM +0100, Andreas Philipp wrote:
> >> Hi,
> >>
> >> Maybe it is a very stupid question but I want to ask it anyway. In
> >> general, 'btrfs filesystem balance' takes very long to finish and
> >> produces lots of IO. So what are the classical usage scenarios, when
> >> it is (really) worth doing a balance?
> >    The primary use-cases for balancing are to even out the filesystem
> > after adding, removing or changing the size of one of the underlying
> > volumes.
> Ok, so this is "a little bit" like for example resyncing a "classical"
> raid after it was in degraded mode etc.

   Pretty much exactly that.

> >    It will also be of use when we finally get around to allowing you
> > to change RAID settings on the whole volume, to implement the
> > requested changes to the RAID level.
> >
> Definitely, a nice feature.
> >    I'm in the process of implementing balance filters, so that some
> > other cases where balancing is useful (reclaiming unused block groups)
> > can be run more efficiently by only balancing the bits that need
> > doing.
> I have seen your post on balance filters. So then it will be (much)
> faster just because less is done? 

   Yes, that's the idea. If you've lost and replaced a drive from a
2-drive RAID-1 array, there's not much that filters can do for you:
all your data will have to be read and rebuilt. However, if you're
changing just your metadata from DUP to RAID-1, say, or recovering
from the loss of one drive in an 8-drive RAID-1 array, it should be an
awful lot faster with filters.

> When you have a version for trying it out and you need someone for
> testing I will give it a try.

   Thanks. I've got quite a bit reworked now to support multiple
filter types, but I need to do a full review of what I'm doing, and
test it myself first. I probably won't have much time to work on it
before Monday, now.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
                 --- emacs: Eats Memory and Crashes. ---                 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]

Re: Btrfs balance

[Helmut Hullen]

Hallo, Andreas,

Du meintest am 20.01.11:

> Maybe it is a very stupid question but I want to ask it anyway. In
> general, 'btrfs filesystem balance' takes very long to finish and
> produces lots of IO. So what are the classical usage scenarios, when
> it is (really) worth doing a balance?

Here (Kernel 2.6.37, btrfs git Nov. 2010): balancing two disks/ 
partitions with 2 and 1.5 TByte needs about 24 hours (CPU 1.5 GHz).

What do you mean with "lots of IO"? the messages in "/var/log/messages"  
and "/var/log/warn"?

Viele Gruesse!
Helmut

Re: Btrfs balance

[Hubert Kario]

On Thursday, January 20, 2011 14:40:00 Helmut Hullen wrote:
> Hallo, Andreas,
>=20
> Du meintest am 20.01.11:
> > Maybe it is a very stupid question but I want to ask it anyway. In
> > general, 'btrfs filesystem balance' takes very long to finish and
> > produces lots of IO. So what are the classical usage scenarios, whe=
n
> > it is (really) worth doing a balance?
>=20
> Here (Kernel 2.6.37, btrfs git Nov. 2010): balancing two disks/
> partitions with 2 and 1.5 TByte needs about 24 hours (CPU 1.5 GHz).
>=20
That's the effect of lots of IO. IMHO it shouldn't take more than 4-8h =
for 1TB=20
7200rpm disks in an otherwise idle system, as such it's rather ineffici=
ent=20
right now.

--=20
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawer=F3w 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Btrfs balance

[Chris Mason]

Excerpts from Andreas Philipp's message of 2011-01-20 08:07:23 -0500:
> 
> Hi,
> 
> Maybe it is a very stupid question but I want to ask it anyway. In
> general, 'btrfs filesystem balance' takes very long to finish and
> produces lots of IO. So what are the classical usage scenarios, when
> it is (really) worth doing a balance?

The idea behind the balance is to spread the used space across all your
drives evenly.  So you would usually run it after adding a new drive.

It's also useful if your disk has a lot of space allocated to either
data or metadata and you want to return to a more reasonable default.
Josef has a patch to do this more dynamically that we're fixing up
(hopefully for 2.6.38-rc2).

In general, it isn't something that you want to do very often.

-chris