Status of rebalance and scrub support

Status of rebalance and scrub support

[Pavin Joseph]

Hello everyone,

Is there an ETA available for full rebalance and scrub support?

I recently migrated from btrfs to bcachefs and so far it has been great, 
mostly due to the performance improvements on pinning metadata to faster 
drives. But as I started off with 2+1+1 TB background target drives with 
replicas=2 that had around 70% usage and then added 4+1 TB drives 
recently, my disk usage are all messed up without a rebalance.

I also perform weekly scrubs on my btrfs and zfs pools, so feeling a bit 
naked here without it!

Thank you for the awesome work on bcachefs!

Kind regards,
Pavin Joseph.

Re: Status of rebalance and scrub support

[Pavin Joseph]

Gentle ping, anyone? 🥺

On 8/7/24 11:07 AM, Pavin Joseph wrote:
> Hello everyone,
> 
> Is there an ETA available for full rebalance and scrub support?
> 
> I recently migrated from btrfs to bcachefs and so far it has been great, 
> mostly due to the performance improvements on pinning metadata to faster 
> drives. But as I started off with 2+1+1 TB background target drives with 
> replicas=2 that had around 70% usage and then added 4+1 TB drives 
> recently, my disk usage are all messed up without a rebalance.
> 
> I also perform weekly scrubs on my btrfs and zfs pools, so feeling a bit 
> naked here without it!
> 
> Thank you for the awesome work on bcachefs!
> 
> Kind regards,
> Pavin Joseph.

Re: Status of rebalance and scrub support

[Martin Steigerwald]

Hi.

Pavin Joseph - 17.08.24, 19:17:18 CEST:
> Gentle ping, anyone? 🥺
> 
> On 8/7/24 11:07 AM, Pavin Joseph wrote:
> > Hello everyone,
> > 
> > Is there an ETA available for full rebalance and scrub support?

I take it that scrub support is not yet fully implemented?

I am not really asking for an ETA – it will be done when it is done – 
however I am not sure whether scrub is not yet fully implemented or 
whether I am facing a bug:

% bcachefs data scrub /mnt/tmp
Starting scrub on 1 devices:  dm-6
device   checked   corrected uncorrected  total
dm-6     0 B         0 B         0 B      17.2 GiB    0%  0 B/sec

(quick test filesystem formatted with bcachefs format without additional 
options on LUKS encrypted LVM)

Stays there until I Ctrl-C it. I waited for a bit more than 5 minutes. No 
disk I/O to the logical volume.

Self-compiled bcachefs-tools from 1.25.2 tag (displays version 1.25.1 
still) on self-compiled kernel 6.14.6 on Devuan Ceres. bcachefs-tools with 
tiny patch to Makefile to recognize eudev for determining udev directory.

Best,
-- 
Martin

Re: Status of rebalance and scrub support

[Malte Schröder]

On 17/05/2025 11:12, Martin Steigerwald wrote:
> Hi.
>
> Pavin Joseph - 17.08.24, 19:17:18 CEST:
>> Gentle ping, anyone? 🥺
>>
>> On 8/7/24 11:07 AM, Pavin Joseph wrote:
>>> Hello everyone,
>>>
>>> Is there an ETA available for full rebalance and scrub support?
> I take it that scrub support is not yet fully implemented?
>
> I am not really asking for an ETA – it will be done when it is done – 
> however I am not sure whether scrub is not yet fully implemented or 
> whether I am facing a bug:
>
> % bcachefs data scrub /mnt/tmp
> Starting scrub on 1 devices:  dm-6
> device   checked   corrected uncorrected  total
> dm-6     0 B         0 B         0 B      17.2 GiB    0%  0 B/sec
>
> (quick test filesystem formatted with bcachefs format without additional 
> options on LUKS encrypted LVM)
>
> Stays there until I Ctrl-C it. I waited for a bit more than 5 minutes. No 
> disk I/O to the logical volume.
>
> Self-compiled bcachefs-tools from 1.25.2 tag (displays version 1.25.1 
> still) on self-compiled kernel 6.14.6 on Devuan Ceres. bcachefs-tools with 
> tiny patch to Makefile to recognize eudev for determining udev directory.
>
> Best,

Can you retest on 6.15-rc? I didn't try that on 6.14, but on 6.15 it is
working for me (tm).

/Malte

Re: Status of rebalance and scrub support

[Martin Steigerwald]

Hi.

Malte Schröder - 17.05.25, 11:15:39 CEST:
> > % bcachefs data scrub /mnt/tmp
> > Starting scrub on 1 devices:  dm-6
> > device   checked   corrected uncorrected  total
> > dm-6     0 B         0 B         0 B      17.2 GiB    0%  0 B/sec
[…]
> Can you retest on 6.15-rc? I didn't try that on 6.14, but on 6.15 it is
> working for me (tm).

Thanks for letting me know!

Wonderful, seems to be working just fine:

% bcachefs data scrub /mnt/tmp 
Starting scrub on 1 devices:  dm-6
device    checked   corrected uncorrected     total
dm-6      17.2 GiB        0 B         0 B  17.2 GiB  100%  complete

So seems since 6.15 there is scrub support :)

Best,
-- 
Martin

Re: Status of rebalance and scrub support

[Pavin Joseph]

On 5/17/25 4:11 PM, Martin Steigerwald wrote:
> So seems since 6.15 there is scrub support :)

That's wonderful news!
Could you check if rebalance is also working in 6.15?

Thanks,
Pavin.

Re: Status of rebalance and scrub support

[Malte Schröder]

On 17/05/2025 14:18, Pavin Joseph wrote:
> On 5/17/25 4:11 PM, Martin Steigerwald wrote:
>> So seems since 6.15 there is scrub support :)
>
> That's wonderful news!
> Could you check if rebalance is also working in 6.15?
>
> Thanks,
> Pavin.

If by "rebalance" you mean equalize the amount of data between drives
(like btrfs): No, it can't do that at the moment. There is a thing
called rebalance on bcachefs, but it does something different (I think
it moves data between tiers).

/Malte

Re: Status of rebalance and scrub support

[Kent Overstreet]

On Sat, May 17, 2025 at 02:21:34PM +0200, Malte Schröder wrote:
> On 17/05/2025 14:18, Pavin Joseph wrote:
> > On 5/17/25 4:11 PM, Martin Steigerwald wrote:
> >> So seems since 6.15 there is scrub support :)
> >
> > That's wonderful news!
> > Could you check if rebalance is also working in 6.15?
> >
> > Thanks,
> > Pavin.
> 
> If by "rebalance" you mean equalize the amount of data between drives
> (like btrfs): No, it can't do that at the moment. There is a thing
> called rebalance on bcachefs, but it does something different (I think
> it moves data between tiers).

I was thinking about renaming "rebalance" to "background_data".

Re: Status of rebalance and scrub support

[John Stoffel]

>>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:

> On Sat, May 17, 2025 at 02:21:34PM +0200, Malte Schröder wrote:
>> On 17/05/2025 14:18, Pavin Joseph wrote:
>> > On 5/17/25 4:11 PM, Martin Steigerwald wrote:
>> >> So seems since 6.15 there is scrub support :)
>> >
>> > That's wonderful news!
>> > Could you check if rebalance is also working in 6.15?
>> >
>> > Thanks,
>> > Pavin.
>> 
>> If by "rebalance" you mean equalize the amount of data between drives
>> (like btrfs): No, it can't do that at the moment. There is a thing
>> called rebalance on bcachefs, but it does something different (I think
>> it moves data between tiers).

> I was thinking about renaming "rebalance" to "background_data".

tier_migrate?  migrate?  

Re: Status of rebalance and scrub support

[Kent Overstreet]

On Mon, May 19, 2025 at 10:43:17AM -0400, John Stoffel wrote:
> >>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:
> 
> > On Sat, May 17, 2025 at 02:21:34PM +0200, Malte Schröder wrote:
> >> On 17/05/2025 14:18, Pavin Joseph wrote:
> >> > On 5/17/25 4:11 PM, Martin Steigerwald wrote:
> >> >> So seems since 6.15 there is scrub support :)
> >> >
> >> > That's wonderful news!
> >> > Could you check if rebalance is also working in 6.15?
> >> >
> >> > Thanks,
> >> > Pavin.
> >> 
> >> If by "rebalance" you mean equalize the amount of data between drives
> >> (like btrfs): No, it can't do that at the moment. There is a thing
> >> called rebalance on bcachefs, but it does something different (I think
> >> it moves data between tiers).
> 
> > I was thinking about renaming "rebalance" to "background_data".
> 
> tier_migrate?  migrate?  

It's not just for migrating to different devices, background compression
is quite popular.

Re: Status of rebalance and scrub support

[John Stoffel]

>>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:

> On Mon, May 19, 2025 at 10:43:17AM -0400, John Stoffel wrote:
>> >>>>> "Kent" == Kent Overstreet <kent.overstreet@linux.dev> writes:
>> 
>> > On Sat, May 17, 2025 at 02:21:34PM +0200, Malte Schröder wrote:
>> >> On 17/05/2025 14:18, Pavin Joseph wrote:
>> >> > On 5/17/25 4:11 PM, Martin Steigerwald wrote:
>> >> >> So seems since 6.15 there is scrub support :)
>> >> >
>> >> > That's wonderful news!
>> >> > Could you check if rebalance is also working in 6.15?
>> >> >
>> >> > Thanks,
>> >> > Pavin.
>> >> 
>> >> If by "rebalance" you mean equalize the amount of data between drives
>> >> (like btrfs): No, it can't do that at the moment. There is a thing
>> >> called rebalance on bcachefs, but it does something different (I think
>> >> it moves data between tiers).
>> 
>> > I was thinking about renaming "rebalance" to "background_data".
>> 
>> tier_migrate?  migrate?  

> It's not just for migrating to different devices, background
> compression is quite popular.

Ah... then maybe 'reorganize' or something like that?  I still need to
try and build the bcachefs tools for myself that are upto date, but
your development is so bleeding edge that it's a real pain.  

Yes, I'm runnign Debian 13, which is certainly not bleeding edge.  

Re: Status of rebalance and scrub support

[Martin Steigerwald]

Kent Overstreet - 19.05.25, 16:54:17 CEST:
> > >> If by "rebalance" you mean equalize the amount of data between
> > >> drives
> > >> (like btrfs): No, it can't do that at the moment. There is a thing
> > >> called rebalance on bcachefs, but it does something different (I
> > >> think
> > >> it moves data between tiers).
> > > 
> > > I was thinking about renaming "rebalance" to "background_data".
> > 
> > tier_migrate?  migrate?
> 
> It's not just for migrating to different devices, background compression
> is quite popular.

How about "background_data" as new command group (maybe you meant it this 
way to begin with)? Like in:

- background_data migrate
- background_data compression

Only caveat is: It is a bit lengthy this way.

Maybe there is a good shortcut for "background_data" like "bg_data" or an 
alternative word for it that is short but also matches the meaning. But 
maybe these are more rare command calls and then length may not really 
matter.

Best,
-- 
Martin