Re: [Lsf-pc] [LSF/MM ATTEND] Over-the-wire data compression

[David Disseldorp <ddiss@suse.de>]

Hi Enzo,

...
> On Thu 14-03-24 15:14:49, Enzo Matsumiya wrote:
> > Hello,
> > 
> > Having implemented data compression for SMB2 messages in cifs.ko, I'd
> > like to attend LSF/MM to discuss:
> > 
> > - implementation decisions, both in the protocol level and in the
> >   compression algorithms; e.g. performance improvements, what could,
> >   if possible/wanted, turn into a lib/ module, etc
> > 
> > - compression algorithms in general; talk about algorithms to determine
> >   if/how compressible a blob of data is
> >     * several such algorithms already exist and are used by on-disk
> >       compression tools, but for over-the-wire compression maybe the
> >       fastest one with good (not great nor best) predictability
> >       could work?

Ideally there could be some overlap between on-disk and over-the-wire
compression algorithm support. That could allow optimally aligned /
sized IOs to avoid unnecessary compression / decompression cycles on an
SMB server / client if the underlying filesystem supports encoded I/O
via e.g. BTRFS_IOC_ENCODED_READ/WRITE.

IIUC, we currently have:
SMB: LZ77, LZ77+Huffman (DEFLATE?), LZNT1, LZ4
Btrfs: zlib/DEFLATE, LZO, Zstd
Bcachefs: zlib/DEFLATE, LZ4, Zstd. Currently no encoded I/O support.

Cheers, David