[PATCH v14 0/7] btrfs: add send/receive support for reading/writing compressed data

Omar Sandoval
2022-03-17T17:26:10Z

From: Omar Sandoval <osandov@fb.com>

This series adds support for sending compressed data via Btrfs send and
btrfs-progs support for sending/receiving compressed data and writing it
with BTRFS_IOC_ENCODED_WRITE, which was previously merged into
misc-next. See the previous posting for more details and benchmarks [1].

Patches 1 and 2 are cleanups for Btrfs send. Patches 3-5 prepare some
protocol changes for send stream v2. Patch 6 implements compressed send.
Patch 7 enables send stream v2 and compressed send in the send ioctl
when requested.

Changes since v13 [2]:

- Rebased on latest misc-next branch.
- Dropped ioctl patches which are already in misc-next.

1: https://lore.kernel.org/linux-btrfs/cover.1615922753.git.osandov@fb.com/
2: https://lore.kernel.org/linux-btrfs/cover.1644519257.git.osandov@fb.com/

Omar Sandoval (7):
btrfs: send: remove unused send_ctx::{total,cmd}_send_size
btrfs: send: explicitly number commands and attributes
btrfs: add send stream v2 definitions
btrfs: send: write larger chunks when using stream v2
btrfs: send: allocate send buffer with alloc_page() and vmap() for v2
btrfs: send: send compressed extents with encoded writes
btrfs: send: enable support for stream v2 and compressed writes

fs/btrfs/ctree.h | 6 +
fs/btrfs/inode.c | 13 +-
fs/btrfs/send.c | 324 +++++++++++++++++++++++++++++++++----
fs/btrfs/send.h | 142 +++++++++-------
include/uapi/linux/btrfs.h | 10 +-
5 files changed, 395 insertions(+), 100 deletions(-)

The btrfs-progs patches were written by Boris Burkov with some updates
from me. Patches 1-4 are preparation. Patch 5 implements encoded writes.
Patch 6 implements the fallback to decompressing. Patches 7 and 8
implement the other commands. Patch 9 adds the new `btrfs send` options.
Patch 10 adds a test case.

Changes since v13:

- Rebased on latest devel branch.
- Updated the btrfs_ioctl_encoded_io_args definition to the version that
was merged into misc-next.

Boris Burkov (8):
btrfs-progs: receive: support v2 send stream larger tlv_len
btrfs-progs: receive: dynamically allocate sctx->read_buf
btrfs-progs: receive: support v2 send stream DATA tlv format
btrfs-progs: receive: process encoded_write commands
btrfs-progs: receive: encoded_write fallback to explicit decode and
write
btrfs-progs: receive: process fallocate commands
btrfs-progs: receive: process setflags ioctl commands
btrfs-progs: receive: add tests for basic encoded_write send/receive

Omar Sandoval (2):
btrfs-progs: receive: add send stream v2 cmds and attrs to send.h
btrfs-progs: send: stream v2 ioctl flags

Documentation/btrfs-receive.rst | 5 +
Documentation/btrfs-send.rst | 22 ++
cmds/receive-dump.c | 31 +-
cmds/receive.c | 347 +++++++++++++++++-
cmds/send.c | 100 ++++-
common/send-stream.c | 165 +++++++--
common/send-stream.h | 7 +
ioctl.h | 151 +++++++-
kernel-shared/send.h | 146 +++++---
libbtrfs/send-stream.c | 2 +-
.../052-receive-write-encoded/test.sh | 114 ++++++
11 files changed, 993 insertions(+), 97 deletions(-)
create mode 100755 tests/misc-tests/052-receive-write-encoded/test.sh

--
2.35.1