balance stuck in loop on linux 6.1.7

balance stuck in loop on linux 6.1.7

[Ian Kelling]

Jan 19 11:48:50 frodo kernel: BTRFS info (device dm-13): balance: start -dconvert=raid1c3 -mconvert=raid1c3 -sconvert=raid1c3
...
Jan 22 05:03:23 frodo kernel: BTRFS info (device dm-13): relocating block group 80026774405120 flags data|raid1c3
Jan 22 05:03:48 frodo kernel: BTRFS info (device dm-13): found 41 extents, stage: move data extents
Jan 22 05:03:57 frodo kernel: BTRFS info (device dm-13): found 41 extents, stage: update data pointers
Jan 22 05:04:06 frodo kernel: BTRFS info (device dm-13): relocating block group 80025700663296 flags data|raid1c3
Jan 22 05:04:29 frodo kernel: BTRFS info (device dm-13): found 71 extents, stage: move data extents
Jan 22 05:04:48 frodo kernel: BTRFS info (device dm-13): found 71 extents, stage: update data pointers
Jan 22 05:05:10 frodo kernel: BTRFS info (device dm-13): relocating block group 42827839111168 flags data|raid10
Jan 22 05:05:24 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: move data extents
Jan 22 05:05:42 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: update data pointers
Jan 22 05:05:55 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: update data pointers
Jan 22 05:06:08 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: update data pointers
Jan 22 05:06:21 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: update data pointers
Jan 22 05:06:29 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: update data pointers
...
Jan 24 19:48:19 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: update data pointers
Jan 24 19:48:35 frodo kernel: BTRFS info (device dm-13): found 1 extents, stage: update data pointers

Since the ..., all the lines have been repeats.

# btrfs balance status /mnt/i
Balance on '/mnt/i' is running
6650 out of about 7480 chunks balanced (6651 considered),  11% left

It has been stuck at 11% for the last 2 days and I assume since the loop
started.

I first experienced this on an earlier kernel so I canceled it and
upgraded to the latest but it is still getting stuck.

# cat /proc/version
Linux version 6.1.7-gnu (rms@mit-oz) (x86_64-linux-gcc (GCC) 12.2.0, GNU ld (GNU Binutils) 2.40) #1.0 SMP PREEMPT_DYNAMIC Tue Sep 27 12:35:59 EST 1983

# btrfs --version
btrfs-progs v6.1.2

If this is a bug, I would like to help the btrfs developers solve it. I
can run whatever commands needed to get more information and try out
patches. But I cannot provide a copy of the filesystem because it has
personal data.

If I restart the balance, it will take about 2.8 days to get back to the
stuck 11% point. The balance is still running, but I'm going to cancel
it morning unless I hear from the list that there is some use to letting
it run since it is wearing out the mechanical drives.

-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org

Re: balance stuck in loop on linux 6.1.7

[Patrik Lundquist]

On Wed, 25 Jan 2023 at 02:48, Ian Kelling <iank@fsf.org> wrote:
>
> If I restart the balance, it will take about 2.8 days to get back to the
> stuck 11% point. The balance is still running, but I'm going to cancel
> it morning unless I hear from the list that there is some use to letting
> it run since it is wearing out the mechanical drives.

You can pause it instead of cancelling it.

Re: balance stuck in loop on linux 6.1.7

[Ian Kelling]

Patrik Lundquist <patrik.lundquist@gmail.com> writes:

> On Wed, 25 Jan 2023 at 02:48, Ian Kelling <iank@fsf.org> wrote:
>>
>> If I restart the balance, it will take about 2.8 days to get back to the
>> stuck 11% point. The balance is still running, but I'm going to cancel
>> it morning unless I hear from the list that there is some use to letting
>> it run since it is wearing out the mechanical drives.
>
> You can pause it instead of cancelling it.

I ran btrfs balance pause but the command is taking a very long time. My
ssh session died about half an hour after the command started for
unrelated reasons, but I see the command still running 5 hours after it
started. Is that normal?

Balance status says:

Balance on '/mnt/i' is running, pause requested

Re: balance stuck in loop on linux 6.1.7

[Patrik Lundquist]

On Wed, 25 Jan 2023 at 20:51, Ian Kelling <iank@fsf.org> wrote:
> Patrik Lundquist <patrik.lundquist@gmail.com> writes:
>
> > On Wed, 25 Jan 2023 at 02:48, Ian Kelling <iank@fsf.org> wrote:
> >>
> >> If I restart the balance, it will take about 2.8 days to get back to the
> >> stuck 11% point. The balance is still running, but I'm going to cancel
> >> it morning unless I hear from the list that there is some use to letting
> >> it run since it is wearing out the mechanical drives.
> >
> > You can pause it instead of cancelling it.
>
> I ran btrfs balance pause but the command is taking a very long time. My
> ssh session died about half an hour after the command started for
> unrelated reasons, but I see the command still running 5 hours after it
> started. Is that normal?

I don't think so. Now is probably a good time to add "skip_balance" to
your mount options.

>
> Balance status says:
>
> Balance on '/mnt/i' is running, pause requested