Re: new database files not compressed

[Nikolay Borisov <nborisov@suse.com>]

On 2.09.20 г. 3:32 ч., Hamish Moffatt wrote:
> On 1/9/20 6:55 pm, Hamish Moffatt wrote:
>> On 1/9/20 3:15 pm, Nikolay Borisov wrote:
>>>
>>> On 1.09.20 г. 2:50 ч., Hamish Moffatt wrote:
>>>> On 31/8/20 10:57 pm, Nikolay Borisov wrote:
>>>>> This means the data being passed to btrfs is not compressible. I.e
>>>>> after
>>>>> coompression the data is not smaller than the original, input data.
>>>> It is though - if I copy it, or run defrag, it compresses very well:
>>>>
>>>>
>>> As Zygo explained - with 16k writes you'd need at least 25% compression
>>>   in order for btrfs to deem it useful. If firebird's 16k writes are not
>>> 25% compressible then it won't compress. It also depends on whether it
>>> issues fsync after every write to ensure consistency meaning it won't
>>> allow more data to accumulate.
> 
> I've been able to reproduce this with a trivial test program which
> mimics the I/O behaviour of Firebird.
> 
> It is calling fallocate() to set up a bunch of blocks and then writing
> them with pwrite(). It seems to be the fallocate() step which is
> preventing compression.
> 
> Here is my trivial test program which just writes zeroes to a file. The
> output file does not get compressed by btrfs.

Ag yes, this makes sense, because fallocate creates PREALLOC extents
which are NOCOW (since they are essentially empty so it makes no sense
to CoW them) hence they go through a different path which doesn't
perform compression.

> 
> 
> #define _GNU_SOURCE
> 
> #include <stdio.h>
> #include <unistd.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <string.h>
> 
> #define BLOCK_SIZE 16384
> 
> int main()
> {
>     unlink("fill");
>     int fd = open("fill", O_RDWR | O_CREAT | O_EXCL, 0666);
> 
>     char buf[BLOCK_SIZE];
>     memset(buf, 0, BLOCK_SIZE);
> 
>     for (int count = 0; count < 256; ++count) {
>         if (count % 8 == 0)
>             fallocate(fd, 0, count * BLOCK_SIZE, 8 * BLOCK_SIZE);
>         pwrite(fd, buf, BLOCK_SIZE, count * BLOCK_SIZE);
>     }
> 
>     close(fd);
> 
>     return 0;
> }
> 
> 
> Hamish
>