On Sun, 2 Jan 2022 11:32:53 +0000 (UTC)
"Konstantin V. Gavrilenko" <k.gavrilenko@arhont.com> wrote:

> Hi list, 
> 
> I have noticed an interesting and surprising behaviour of my BTRFS with regards to compression of the files and NFS. 
> 
> I have BTRFS RAID10 with 8 disks , that is mounted with the " nofail,noatime,space_cache=v2,compress-force=zstd:9,subvol=@cloudstack-secondary" flags and is exported via NFS. 
> 
> When I create a snapshot of a disk in Cloudstack from CEPH and save it to a secondary storage to this BTRFS RAID10 over NFS, the file does not compress, despite the compress-force mount option being set on FS 
> So in the below example, the file eeceaf0e-9780-408b-a748-1495d517a9b6 was copied over NFS and is not compressed. When I copy the same file directly on a host, it does get compressed pretty well, as per example below. 
> 
> [...]
> 
> 
> So what I have checked so far what works 
> - after the original files is copied over NFS, the copy of the file using #cp gets compressed. 
> - after the original files is copied over NFS, the original file can be compressed using #btrfs defrag -czstd option 
> - If I copy the original file to some other host, and copy it back via NFS using cp, it does get compressed. 
> 
> So the problem seems to appear only when the file is exported from Ceph and copied to NFS. 
> 
> Any hints what could be causing such a behaviour? 
> 
> 
> 
> Yours sincerely, 
> Kos
> 

Btrfs doesn't compress direct-io writes. This suggests that whatever you use for "I create a snapshot of a disk in Cloudstack from CEPH and save it to a secondary storage to this BTRFS RAID10 over NFS", it writes with O_DIRECT.

Regards,
Lukas Straub

--