ANNOUNCE: SCSI XCOPY support for the kernel and device mapper

ANNOUNCE: SCSI XCOPY support for the kernel and device mapper

[Mikulas Patocka]

Hi

I annouce that I released the first version of a patch set that makes it 
possible to use SCSI XCOPY offload for the block layer and device mapper.

The patchset is at 
http://people.redhat.com/~mpatocka/patches/kernel/xcopy/series.html It 
requires kernel version at least 3.16-rc4. It was tested on target-core 
iSCSI implementation.

It is based on Martin Petersen's work 
https://git.kernel.org/cgit/linux/kernel/git/mkp/linux.git/commit/?h=xcopy&id=0bdeed274e16b3038a851552188512071974eea8, 
but it is changed significantly so that it is possible to propagate XCOPY 
bios through the device mapper stack.

The basic architecture is this: in the function blkdev_issue_copy we 
create two bios, one for read and one for write (with bi_rw READ|REQ_COPY 
and WRITE|REQ_COPY). Both bios have a pointer to the same bio_copy 
structure. These two bios travel independently through the device mapper 
stack - each bio can go through different device mapper devices. When both 
the bios reach the physical block device (in the function blk_queue_bio) 
the bio pair is collected and a XCOPY request is allocated and sent to the 
scsi disk driver.

Note that because device mapper mapping can dynamically change, there no 
guarantee that the XCOPY command succeeds. If it ends with an error, the 
caller is supposed to perform the copying manually.

The dm-kcopyd subsystem is modified to use the XCOPY command, so device 
mapper targets that use it (mirror, snapshot, thin, cache) take advantage 
of copy offload automatically.

There is a new ioctl BLKCOPY that makes it possible to use copy offload 
from userspace.

Mikulas

Re: ANNOUNCE: SCSI XCOPY support for the kernel and device mapper

[Christoph Hellwig]

On Tue, Jul 15, 2014 at 02:33:13PM -0400, Mikulas Patocka wrote:
> Hi
> 
> I annouce that I released the first version of a patch set that makes it 
> possible to use SCSI XCOPY offload for the block layer and device mapper.
> 
> The patchset is at 
> http://people.redhat.com/~mpatocka/patches/kernel/xcopy/series.html It 
> requires kernel version at least 3.16-rc4. It was tested on target-core 
> iSCSI implementation.

Can you send the series to the mainling list(s)?  That makes it a lot
easier to do a quick review.

> The dm-kcopyd subsystem is modified to use the XCOPY command, so device 
> mapper targets that use it (mirror, snapshot, thin, cache) take advantage 
> of copy offload automatically.

That sounds good, do you have any benchmarking data that shows the
advantage of using XCOPY?

Re: ANNOUNCE: SCSI XCOPY support for the kernel and device mapper

[Mikulas Patocka]

On Tue, 15 Jul 2014, Christoph Hellwig wrote:

> On Tue, Jul 15, 2014 at 02:33:13PM -0400, Mikulas Patocka wrote:
> > Hi
> > 
> > I annouce that I released the first version of a patch set that makes it 
> > possible to use SCSI XCOPY offload for the block layer and device mapper.
> > 
> > The patchset is at 
> > http://people.redhat.com/~mpatocka/patches/kernel/xcopy/series.html It 
> > requires kernel version at least 3.16-rc4. It was tested on target-core 
> > iSCSI implementation.
> 
> Can you send the series to the mainling list(s)?  That makes it a lot
> easier to do a quick review.

I'll post them to the lists.

> > The dm-kcopyd subsystem is modified to use the XCOPY command, so device 
> > mapper targets that use it (mirror, snapshot, thin, cache) take advantage 
> > of copy offload automatically.
> 
> That sounds good, do you have any benchmarking data that shows the
> advantage of using XCOPY?

dm-mirror resynchronization is 4 times faster with XCOPY offload, but it 
is just a benchmark on a network with iSCSI between two slow machines.

I hope that we get some benchmarks on a real hardware.

Mikulas