Linux Btrfs filesystem development
 help / color / mirror / Atom feed
* btrfs raid1 balance slow
@ 2020-01-20 22:08 kjansen387
  2020-01-20 22:13 ` Hugo Mills
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: kjansen387 @ 2020-01-20 22:08 UTC (permalink / raw)
  To: linux-btrfs

Hello,

I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is 
about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and 
added one 2 TB disk and started the balance:

btrfs device add -f /dev/sdb /export
btrfs filesystem balance /export

It's now running for 24 hours, 70% remaining:

# btrfs balance status -v /export
Balance on '/export' is running
1057 out of about 3561 chunks balanced (1058 considered),  70% left
Dumping filters: flags 0x7, state 0x1, force is off
   DATA (flags 0x0): balancing
   METADATA (flags 0x0): balancing
   SYSTEM (flags 0x0): balancing

I have searched for similar cases, but, I do not have quotas enabled, I 
do not have compression enabled, and my CPU supports sse4_2 . CPU 
(i7-8700K) is doing fine, 80% idle (average over all threads).

Is this normal ? I have to repeat this process 2 times (adding more 2TB 
disks), any way I can make it faster ?

Thanks!
Klaas


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs raid1 balance slow
  2020-01-20 22:08 btrfs raid1 balance slow kjansen387
@ 2020-01-20 22:13 ` Hugo Mills
  2020-01-20 23:59 ` Qu Wenruo
  2020-01-21  9:51 ` kjansen387
  2 siblings, 0 replies; 5+ messages in thread
From: Hugo Mills @ 2020-01-20 22:13 UTC (permalink / raw)
  To: kjansen387; +Cc: linux-btrfs

On Mon, Jan 20, 2020 at 11:08:40PM +0100, kjansen387 wrote:
> I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is
> about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and added
> one 2 TB disk and started the balance:
> 
> btrfs device add -f /dev/sdb /export
> btrfs filesystem balance /export
> 
> It's now running for 24 hours, 70% remaining:
> 
> # btrfs balance status -v /export
> Balance on '/export' is running
> 1057 out of about 3561 chunks balanced (1058 considered),  70% left
> Dumping filters: flags 0x7, state 0x1, force is off
>   DATA (flags 0x0): balancing
>   METADATA (flags 0x0): balancing
>   SYSTEM (flags 0x0): balancing
> 
> I have searched for similar cases, but, I do not have quotas enabled, I do
> not have compression enabled, and my CPU supports sse4_2 . CPU (i7-8700K) is
> doing fine, 80% idle (average over all threads).

   Do you have lots of snapshots? It can take a lot of time on some of
the metadata chunks if there's lots of shared extents.

> Is this normal ? I have to repeat this process 2 times (adding more 2TB
> disks), any way I can make it faster ?

   Cancel the current balance, add the remaining disks, and then
balance only once you've added them all.

   Hugo.

-- 
Hugo Mills             | If it's December 1941 in Casablanca, what time is it
hugo@... carfax.org.uk | in New York?
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                               Rick Blaine, Casablanca

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs raid1 balance slow
  2020-01-20 22:08 btrfs raid1 balance slow kjansen387
  2020-01-20 22:13 ` Hugo Mills
@ 2020-01-20 23:59 ` Qu Wenruo
  2020-01-21  9:51 ` kjansen387
  2 siblings, 0 replies; 5+ messages in thread
From: Qu Wenruo @ 2020-01-20 23:59 UTC (permalink / raw)
  To: kjansen387, linux-btrfs


[-- Attachment #1.1: Type: text/plain, Size: 1293 bytes --]



On 2020/1/21 上午6:08, kjansen387 wrote:
> Hello,
> 
> I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is
> about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and
> added one 2 TB disk and started the balance:
> 
> btrfs device add -f /dev/sdb /export
> btrfs filesystem balance /export
> 
> It's now running for 24 hours, 70% remaining:
> 
> # btrfs balance status -v /export
> Balance on '/export' is running
> 1057 out of about 3561 chunks balanced (1058 considered),  70% left
> Dumping filters: flags 0x7, state 0x1, force is off
>   DATA (flags 0x0): balancing
>   METADATA (flags 0x0): balancing
>   SYSTEM (flags 0x0): balancing
> 
> I have searched for similar cases, but, I do not have quotas enabled, I
> do not have compression enabled, and my CPU supports sse4_2 . CPU
> (i7-8700K) is doing fine, 80% idle (average over all threads).

Are you using qgroup?

If so, that would be the cause.
Please use kernel newer than v5.2, which contains the optimization.

> 
> Is this normal ? I have to repeat this process 2 times (adding more 2TB
> disks), any way I can make it faster ?

Just as Hugo said, cancel the current one, add all device, then rebalance.

Thanks,
Qu

> 
> Thanks!
> Klaas
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs raid1 balance slow
  2020-01-20 22:08 btrfs raid1 balance slow kjansen387
  2020-01-20 22:13 ` Hugo Mills
  2020-01-20 23:59 ` Qu Wenruo
@ 2020-01-21  9:51 ` kjansen387
  2020-01-22  8:18   ` kjansen387
  2 siblings, 1 reply; 5+ messages in thread
From: kjansen387 @ 2020-01-21  9:51 UTC (permalink / raw)
  To: linux-btrfs

Forgot to mention; I'm running kernel 5.4.10-200.fc31.x86_64 .

Op 20-Jan-20 om 23:08 schreef kjansen387:
> Hello,
> 
> I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is 
> about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and 
> added one 2 TB disk and started the balance:
> 
> btrfs device add -f /dev/sdb /export
> btrfs filesystem balance /export
> 
> It's now running for 24 hours, 70% remaining:
> 
> # btrfs balance status -v /export
> Balance on '/export' is running
> 1057 out of about 3561 chunks balanced (1058 considered),  70% left
> Dumping filters: flags 0x7, state 0x1, force is off
>    DATA (flags 0x0): balancing
>    METADATA (flags 0x0): balancing
>    SYSTEM (flags 0x0): balancing
> 
> I have searched for similar cases, but, I do not have quotas enabled, I 
> do not have compression enabled, and my CPU supports sse4_2 . CPU 
> (i7-8700K) is doing fine, 80% idle (average over all threads).
> 
> Is this normal ? I have to repeat this process 2 times (adding more 2TB 
> disks), any way I can make it faster ?
> 
> Thanks!
> Klaas
> 

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs raid1 balance slow
  2020-01-21  9:51 ` kjansen387
@ 2020-01-22  8:18   ` kjansen387
  0 siblings, 0 replies; 5+ messages in thread
From: kjansen387 @ 2020-01-22  8:18 UTC (permalink / raw)
  To: linux-btrfs

Hmm, for some reason, I did not get the mail from Hugo and Qu (had too 
look it up in the archive). I think there is a delay when subscribing to 
the email list.

 > Do you have lots of snapshots? It can take a lot of time on some of
the metadata chunks if there's lots of shared extents.

No, I do not have any snapshots. Never made one - just got started with 
btrfs.
# btrfs subvolume show /export | grep Snap
         Snapshot(s):


 > Are you using qgroup?

No, quotas are disabled:
# btrfs qgroup show /export
ERROR: can't list qgroups: quotas not enabled


 >Please use kernel newer than v5.2, which contains the optimization.

I'm running 5.4.10-200.fc31.x86_64, that should be good


 > Just as Hugo said, cancel the current one, add all device, then 
rebalance.

Yes, that would be the best. I'm moving data however.. Expand btrfs > 
move the content of a 2TB disk > use that disk to expand btrfs > move 
the content of a 2TB disk etc. When this balance is done I'll move 2TB, 
find a temp drive to store the other 2TB and add the remaining 2 drives 
in 1 go.

I'm not in a hurry.. just wondering if this is normal. As I had 2x4TB 
with 150GB free and am adding 1x2TB (~1800GB, ~900 after raid1) I would 
expect that I will end up with 150+900=1050 GB free (usable), spread 
like (approx.):
4TB disk 1: 420GB (previously: 75GB)
4TB disk 2: 420GB (previously: 75GB)
2TB disk: 210GB

Thinking very simplisticly -and that's probably where I'm wrong, just 
wondering if it's a factor 8 wrong- this would mean 420-75 = 345GB read 
from each 4TB disks and written to the new 2TB disk. At a rate of 20MB/s 
(pretty conservative, but I do have a little bit of load) this should 
take like 10 hours. It will, however, at this rate, take approx. 82 
hours. Even if all data needs to be read (does it? ~3500GB net) it 
should only take ~50 hours..

Klaas

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-01-22  8:18 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-01-20 22:08 btrfs raid1 balance slow kjansen387
2020-01-20 22:13 ` Hugo Mills
2020-01-20 23:59 ` Qu Wenruo
2020-01-21  9:51 ` kjansen387
2020-01-22  8:18   ` kjansen387

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox