Re: Experiences: Why BTRFS had to yield for ZFS

[Chris Mason <chris.mason@fusionio.com>]

On Mon, Sep 17, 2012 at 02:45:08AM -0600, Casper Bang wrote:
> Abstract
> For database testing purposes, a COW filesystem was needed in order to
> facilitate snapshotting and rollback, such as to provide mirrors of
> our production database at fixed intervals (every night and by
> demand).

Thanks for taking the time to write this up follow through the thread.
It's always interesting to hear situations where btrfs doesn't work
well.

There are three basic problems with the database workloads on btrfs.
First is that we have higher latencies on writes because we are feeding
everything through helper threads for crcs.  Usually the extra latencies
don't show up because we have enough work in the pipeline to keep the
drive busy.

I don't believe the UEK kernels have the recent changes to do some of
the crc work inline (without handing off) for smaller synchronous IOs.

Second, on O_SYNC writes btrfs will write both the file metadata and
data into a special tree so we can be crash safe.  For big files this
tends to spend a lot of time looking for the extents in the file that
have changed.

Josef fixed that up and it is queued for the next merge window.

The third problem is that lots of random writes tend to make lots of
metadata.  If this doesn't fit in ram, we can end up doing many reads
that slow things down.  We're working on this now as well, but recent
kernels change how we cache things and should improve the results.

-chris