Re: Questions related to BCacheFS

[Martin Steigerwald <martin@lichtvoll.de>]

Take your time and have a great Sunday :)

Kent Overstreet - 19.11.23, 00:42:05 CET:
> On Sun, Nov 19, 2023 at 12:15:19AM +0100, Martin Steigerwald wrote:
> > Kent Overstreet - 18.11.23, 22:07:27 CET:
> > > > As far as I understand one specific performance related aspect of
> > > > BCacheFS would be low latencies due to the frontend / backend
> > > > architecture which in principle is based on what has been there in
> > > > BCache already. I am intending to explore a bit into that concept
> > > > in
> > > > my article.
> > > 
> > > The low latency stuff actually wasn't in bcache - that work came
> > > later.
> > 
> > So the frontend / backend architecture is not that much of what makes
> > BCacheFS unique? Important to know as it seems I may have
> > misunderstood
> > something here.
> 
> The "filesystem on top of a database" is the main thing that makes
> bcachefs unique - you have that right.

Phew! Seems I did not get it completely wrong then :)

> bcache had much of the core btree design - log structured btree nodes
> with eytzinger search trees; that's how we got a high enough performance
> btree to make the "filesystem on top of a database" thing practical.
> 
> But the btree in bcache was, from a performance POV, prototype quality -
> stable, but a lot of performance corner cases unfinished.
> 
> The latency work, real iterators, and the whole transaction layer came
> later :)

So it is fair to say that being based on BCache enabled that low latency 
work?

I am trying to find a balance here. The audience of the article are 
experienced sys admins working in small and large organizations, but not 
(primarily) kernel hackers. So I like to explain possible reasons to 
consider BCacheFS while also having BTRFS and ZFS and XFS, but without 
going into so much detail that one needs to be a kernel developer to 
understand it. For XFS it is easy as while there was a proof of concept 
with subvolumes and snapshots based on XFS in files on XFS, AFAIR from 
Dave Chinner, some years back, it does not have those advanced features 
(yet). For ZFS one can always argue it is not in the mainline kernel. But 
regarding BTRFS it becomes important to really explain something. I 
started to do some BCacheFS / BTRFS feature comparison chart, but I also 
like to explain on the benefits BCacheFS can have in the text and a bit 
about the background of those benefits. Of course also mentioning that 
BCacheFS is in development for more than 10 years, even without taking all 
the work for BCache itself into account.

So my idea currently is to explain that the BCache BTree and/or frontend/
backend architecture, not sure how to best word it, enabled a database 
approach to a filesystem to be feasible. And that in a sense it also 
enabled the latency work. And I can mention sixlocks and all the nice 
other stuff you mentioned. However… I may not explain exactly what those 
nice things are and how they work for example. For two reasons: 1. I need 
to understand those nice features myself, 2. limit of pages I may use and 
scope of the article. Currently I understood that sixlocks for certain use 
cases are the next best thing since the invention of a wheel or something 
like that. But not much more :)

Would something like that be accurate enough in your opinion?

I will review the user manual once again, I read about the database 
approach, and aim to find a good balance. Cause for certain I won't be 
getting 24 pages for the article :)


I got two setbacks regarding trying BCacheFS on my laptop yesterday and 
today:

1) Linux 6.7-rc1, as mentioned almost rc2, did not hibernate on ThinkPad 
T14 AMD Gen 1. It just blanked screen and nothing happened. So I went back 
to 6.6.1 temporarily. I do not really intend to do a git bisect between 
rc1 and 6.6 on a production laptop. It would be very time-consuming and 
possibly be dangerous. I may go back to 6.7 even without hibernation, just 
using standby over night for a while. I bet BCacheFS is compatible with 
hibernation (as in writing memory contents to encrypted swap and resuming 
from it)?

2) But after I removed 6.7-rc1 yesterday night after having booted into 
6.6.1 this morning I was greeted by GRUB command line. As I had a meeting 
scheduled my primary aim was to get things running again. I certainly did 
not really get the humorous aspect about it. I thought I'd had copied etc 
in the broken state to a another place, but do not find it anymore, maybe 
I copied it to /root or the GRML live distro accidently and so it is gone. 
Also I missed to copy the broken /boot to another place. So I am not 
really able to do any forensic analysis of what might have happened. I do 
not recall having seen any error messages either, but I may have missed 
something. I reviewed hook and script for initial RAM disk in bcachefs-
tools repo, but did not find anything in there that could have caused grub 
2 not finding its config file and modules anymore. Also it booted into 6.7 
and even 6.6.1 then after having executed "make install" from bcachefs-
tools directory. Also I see nothing being done with grub itself within 
bcachefs-tools. So this really is quite the mystery for me. I am on Devuan 
Ceres (based on Debian Sid) so maybe something else got messed up. Will 
review their bug trackers. I really have no idea what went wrong here, 
luckily I was able to recover with GRML. I use LVM on LUKS for BTRFS and 
test BCacheFS filesystem.

Maybe I need to continue testing BCacheFS on a virtual machine, but I'd 
really love to have a BCacheFS filesystem on my laptop and actually really 
using it for something. Well I am going to leave it at that and probably 
research on this after the weekend. Now it is time to have a Sunday off.

Best,
-- 
Martin