Re: With Linux 5.5: Filesystem full while still 90 GiB free

[Martin Steigerwald <martin@lichtvoll.de>]

Chris Murphy - 30.01.20, 21:18:41 CET:
> On Thu, Jan 30, 2020 at 1:02 PM Martin Steigerwald 
<martin@lichtvoll.de> wrote:
> > Chris Murphy - 30.01.20, 17:37:42 CET:
> > > On Thu, Jan 30, 2020 at 3:41 AM Martin Steigerwald
> > 
> > <martin@lichtvoll.de> wrote:
> > > > Chris Murphy - 29.01.20, 23:55:06 CET:
> > > > > On Wed, Jan 29, 2020 at 2:20 PM Martin Steigerwald
> > > > 
> > > > <martin@lichtvoll.de> wrote:
> > > > > > So if its just a cosmetic issue then I can wait for the
> > > > > > patch to
> > > > > > land in linux-stable. Or does it still need testing?
> > > > > 
> > > > > I'm not seeing it in linux-next. A reasonable short term work
> > > > > around
> > > > > is mount option 'metadata_ratio=1' and that's what needs more
> > > > > testing, because it seems decently likely mortal users will
> > > > > need
> > > > > an easy work around until a fix gets backported to stable. And
> > > > > that's gonna be a while, me thinks.
> > > > > 
> > > > > Is that mount option sufficient? Or does it take a filtered
> > > > > balance?
> > > > > What's the most minimal balance needed? I'm hoping -dlimit=1
> > > > 
> > > > Does not make a difference. I did:
> > > > 
> > > > - mount -o remount,metadata_ratio=1 /daten
> > > > - touch /daten/somefile
> > > > - dd if=/dev/zero of=/daten/someotherfile bs=1M count=500
> > > > - sync
> > > > - df still reporting zero space free
> > > > 
> > > > > I can't figure out a way to trigger this though, otherwise I'd
> > > > > be
> > > > > doing more testing.
> > > > 
> > > > Sure.
> > > > 
> > > > I am doing the balance -dlimit=1 thing next. With
> > > > metadata_ratio=0
> > > > again.
> > > > 
> > > > % btrfs balance start -dlimit=1 /daten
> > > > Done, had to relocate 1 out of 312 chunks
> > > > 
> > > > % LANG=en df -hT /daten
> > > > Filesystem             Type   Size  Used Avail Use% Mounted on
> > > > /dev/mapper/sata-daten btrfs  400G  311G     0 100% /daten
> > > > 
> > > > Okay, doing with metadata_ratio=1:
> > > > 
> > > > % mount -o remount,metadata_ratio=1 /daten
> > > > 
> > > > % btrfs balance start -dlimit=1 /daten
> > > > Done, had to relocate 1 out of 312 chunks
> > > > 
> > > > % LANG=en df -hT /daten
> > > > Filesystem             Type   Size  Used Avail Use% Mounted on
> > > > /dev/mapper/sata-daten btrfs  400G  311G     0 100% /daten
> > > > 
> > > > 
> > > > Okay, other suggestions? I'd like to avoid shuffling 311 GiB
> > > > data
> > > > around using a full balance.
> > > 
> > > There's earlier anecdotal evidence that -dlimit=10 will work. But
> > > you
> > > can just keep using -dlimit=1 and it'll balance a different block
> > > group each time (you can confirm/deny this with the block group
> > > address and extent count in dmesg for each balance). Count how
> > > many it takes to get df to stop misreporting. It may be a file
> > > system specific value.
> > 
> > Lost the patience after 25 attempts:
> > 
> > date; let I=I+1; echo "Balance $I"; btrfs balance start -dlimit=1
> > /daten ; LANG=en df -hT /daten
> > Do 30. Jan 20:59:17 CET 2020
> > Balance 25
> > Done, had to relocate 1 out of 312 chunks
> > Filesystem             Type   Size  Used Avail Use% Mounted on
> > /dev/mapper/sata-daten btrfs  400G  311G     0 100% /daten
> > 
> > 
> > Doing the -dlimit=10 balance now:
> > 
> > % btrfs balance start -dlimit=10 /daten ; LANG=en df -hT /daten
> > Done, had to relocate 10 out of 312 chunks
> > Filesystem             Type   Size  Used Avail Use% Mounted on
> > /dev/mapper/sata-daten btrfs  400G  311G     0 100% /daten
> > 
> > Okay, enough of balancing for today.
> > 
> > I bet I just wait for a proper fix, instead of needlessly shuffling
> > data around.
> 
> What about unmounting and remounting?

Does not help.
 
> There is a proposed patch that David referenced in this thread, but
> it's looking like it papers over the real problem. But even if so,
> that'd get your file system working sooner than a proper fix, which I
> think (?) needs to be demonstrated to at least cause no new
> regressions in 5.6, before it'll be backported to stable.

I am done with re-balancing experiments.

-- 
Martin