Re: Zero-copy Block IO with XFS

Bhagi rathi
2007-12-11T16:39:16Z

On Dec 11, 2007 5:08 PM, Matthew Hodgson <matthew@mxtelecom.com> wrote:

> Hi all,
>
> I'm experimenting with using XFS with a network block device (DST), and
> have come up against the problem that when writing data to the network,
> it uses kernel_sendpage to hand the page presented at the BIO layer to
> the network stack. It then completes the block IO request.

Actually, you can pass a sendpage read actor which takes a reference on
the page
which ensures valid page exists with you. As long as you have ref on the
page and
no truncate to the same file, you can safely access the file. Once NIC
sends the
data over wire, you can do put_page. This should work. By the way, linux
sendfile
does something similar to the above. Look into read actor of nfs server
sendfile
usage.

-Cheers,
Bhagi.

>
>
> The problem arises when XFS proceeds to then reuse that page before the
> NIC actually sends it. Particularly if TX checksumming or TCP
> segmentation is being offloaded to the NIC, it seems that the NIC will
> try to access to page after the BIO request has returned, and so operate
> on stale data. I assume the same problem might happen in the case of
> TCP retransmits or similar. The motivation for using sendpage rather
> than sendmsg (or using sendpage on a copy of the original page) is to
> try to ensure speed by a zero-copy path through the subsystem.
>
> Is there any way at all in which XFS would be able to (theoretically)
> expose an API to allow an underlying block device to retain ownership of
> pages until it's done with them, so as to avoid a potentially needless
> copy? Or is there another way of achieving this?
>
> thanks in advance,
>
> Matthew.
>
>
> --
> Matthew Hodgson <matthew@mxtelecom.com>
> Media & Systems Project Manager
> Tel: +44 (0) 845 666 7778
> http://www.mxtelecom.com
>
>
>

[[HTML alternate version deleted]]