All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ric Wheeler <rwheeler@redhat.com>
To: Miklos Szeredi <miklos@szeredi.hu>
Cc: Zach Brown <zab@redhat.com>,
	"J. Bruce Fields" <bfields@fieldses.org>,
	Anna Schumaker <schumaker.anna@gmail.com>,
	Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Linux-Fsdevel <linux-fsdevel@vger.kernel.org>,
	"linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
	Trond Myklebust <Trond.Myklebust@netapp.com>,
	Bryan Schumaker <bjschuma@netapp.com>,
	"Martin K. Petersen" <mkp@mkp.net>, Jens Axboe <axboe@kernel.dk>,
	Mark Fasheh <mfasheh@suse.com>, Joel Becker <jlbec@evilplan.org>,
	Eric Wong <normalperson@yhbt.net>
Subject: Re: [RFC] extending splice for copy offloading
Date: Thu, 26 Sep 2013 17:23:51 -0400	[thread overview]
Message-ID: <5244A5E7.90808@redhat.com> (raw)
In-Reply-To: <CAJfpegvvWhs+jv2J9kOQrB31PEO3kyn_sLm_e2w9YKp=y6EDhA@mail.gmail.com>

On 09/26/2013 03:53 PM, Miklos Szeredi wrote:
> On Thu, Sep 26, 2013 at 9:06 PM, Zach Brown <zab@redhat.com> wrote:
>
>>> But I'm not sure it's worth the effort; 99% of the use of this
>>> interface will be copying whole files.  And for that perhaps we need a
>>> different API, one which has been discussed some time ago:
>>> asynchronous copyfile() returns immediately with a pollable event
>>> descriptor indicating copy progress, and some way to cancel the copy.
>>> And that can internally rely on ->direct_splice(), with appropriate
>>> algorithms for determine the optimal  chunk size.
>> And perhaps we don't.  Perhaps we can provide this much simpler
>> data-plane interface that works well enough for most everyone and can
>> avoid going down the async rat hole, yet again.
> I think either buffering or async is needed to get good perforrmace
> without too much complexity in the app (which is not good).  Buffering
> works quite well for regular I/O, so maybe its the way to go here as
> well.
>
> Thanks,
> Miklos
>

Buffering  misses the whole point of the copy offload - the idea is *not* to 
read or write the actual data in the most interesting cases which offload the 
operation to a smart target device or file system.

Regards,

Ric


WARNING: multiple messages have this Message-ID (diff)
From: Ric Wheeler <rwheeler-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
To: Miklos Szeredi <miklos-sUDqSbJrdHQHWmgEVkV9KA@public.gmane.org>
Cc: Zach Brown <zab-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	"J. Bruce Fields"
	<bfields-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org>,
	Anna Schumaker
	<schumaker.anna-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Kernel Mailing List
	<linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	Linux-Fsdevel
	<linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	"linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
	<linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	Trond Myklebust
	<Trond.Myklebust-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>,
	Bryan Schumaker
	<bjschuma-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org>,
	"Martin K. Petersen" <mkp-30zCAauEzIw@public.gmane.org>,
	Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>,
	Mark Fasheh <mfasheh-IBi9RG/b67k@public.gmane.org>,
	Joel Becker <jlbec-aKy9MeLSZ9dg9hUCZPvPmw@public.gmane.org>,
	Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Subject: Re: [RFC] extending splice for copy offloading
Date: Thu, 26 Sep 2013 17:23:51 -0400	[thread overview]
Message-ID: <5244A5E7.90808@redhat.com> (raw)
In-Reply-To: <CAJfpegvvWhs+jv2J9kOQrB31PEO3kyn_sLm_e2w9YKp=y6EDhA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On 09/26/2013 03:53 PM, Miklos Szeredi wrote:
> On Thu, Sep 26, 2013 at 9:06 PM, Zach Brown <zab-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>
>>> But I'm not sure it's worth the effort; 99% of the use of this
>>> interface will be copying whole files.  And for that perhaps we need a
>>> different API, one which has been discussed some time ago:
>>> asynchronous copyfile() returns immediately with a pollable event
>>> descriptor indicating copy progress, and some way to cancel the copy.
>>> And that can internally rely on ->direct_splice(), with appropriate
>>> algorithms for determine the optimal  chunk size.
>> And perhaps we don't.  Perhaps we can provide this much simpler
>> data-plane interface that works well enough for most everyone and can
>> avoid going down the async rat hole, yet again.
> I think either buffering or async is needed to get good perforrmace
> without too much complexity in the app (which is not good).  Buffering
> works quite well for regular I/O, so maybe its the way to go here as
> well.
>
> Thanks,
> Miklos
>

Buffering  misses the whole point of the copy offload - the idea is *not* to 
read or write the actual data in the most interesting cases which offload the 
operation to a smart target device or file system.

Regards,

Ric

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  reply	other threads:[~2013-09-26 21:25 UTC|newest]

Thread overview: 111+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-09-11 17:06 [RFC] extending splice for copy offloading Zach Brown
2013-09-11 17:06 ` [PATCH 1/3] splice: add DIRECT flag for splicing between files Zach Brown
2013-09-11 17:06   ` Zach Brown
2013-09-11 17:06 ` [PATCH 2/3] splice: add f_op->splice_direct Zach Brown
2013-09-11 17:06   ` Zach Brown
2013-09-11 17:06 ` [PATCH 3/3] btrfs: implement .splice_direct extent copying Zach Brown
2013-09-11 17:06   ` Zach Brown
2013-09-11 21:17 ` [RFC] extending splice for copy offloading Eric Wong
2013-09-16 19:44   ` Rob Landley
2013-09-16 19:44     ` Rob Landley
2013-09-19 12:59   ` Jeff Layton
2013-09-20  9:49 ` Szeredi Miklos
2013-09-20  9:49   ` Szeredi Miklos
2013-09-25 18:38   ` Zach Brown
2013-09-25 18:38     ` Zach Brown
2013-09-25 19:02     ` Anna Schumaker
2013-09-25 19:06       ` Zach Brown
2013-09-25 19:06         ` Zach Brown
2013-09-25 19:55         ` J. Bruce Fields
2013-09-25 19:55           ` J. Bruce Fields
2013-09-25 21:07           ` Zach Brown
2013-09-26  8:58             ` Miklos Szeredi
2013-09-26 15:34               ` J. Bruce Fields
2013-09-26 16:46                 ` Ric Wheeler
2013-09-26 16:46                   ` Ric Wheeler
2013-09-26 18:06                 ` Miklos Szeredi
2013-09-26 19:06                   ` Zach Brown
2013-09-26 19:53                     ` Miklos Szeredi
2013-09-26 21:23                       ` Ric Wheeler [this message]
2013-09-26 21:23                         ` Ric Wheeler
2013-09-27  4:47                         ` Miklos Szeredi
2013-09-27  4:47                           ` Miklos Szeredi
2013-09-27 14:00                           ` Ric Wheeler
2013-09-27 14:39                             ` Miklos Szeredi
2013-10-06  8:42                   ` Rob Landley
2013-10-06  8:42                     ` Rob Landley
2013-09-26 18:55               ` Zach Brown
2013-09-26 21:26                 ` Ric Wheeler
2013-09-26 21:26                   ` Ric Wheeler
2013-09-27 20:05                   ` J. Bruce Fields
2013-09-27 20:05                     ` J. Bruce Fields
2013-09-27 20:50                     ` Zach Brown
2013-09-28  5:49                       ` Miklos Szeredi
2013-09-28 15:20                         ` Myklebust, Trond
2013-09-28 15:20                           ` Myklebust, Trond
2013-09-28 15:20                           ` Myklebust, Trond
2013-09-28 21:20                           ` Ric Wheeler
2013-09-30 12:20                             ` Miklos Szeredi
2013-09-30 12:20                               ` Miklos Szeredi
2013-09-30 14:34                               ` J. Bruce Fields
2013-09-30 14:34                                 ` J. Bruce Fields
2013-09-30 14:48                                 ` Ric Wheeler
2013-09-30 14:51                                 ` Miklos Szeredi
2013-09-30 14:52                                   ` Ric Wheeler
2013-09-30 14:52                                     ` Ric Wheeler
2013-09-30 15:24                                     ` Miklos Szeredi
2013-09-30 14:28                                       ` Ric Wheeler
2013-09-30 15:33                                         ` Myklebust, Trond
2013-09-30 15:33                                           ` Myklebust, Trond
2013-09-30 15:33                                           ` Myklebust, Trond
2013-09-30 15:38                                         ` Miklos Szeredi
2013-09-30 15:38                                           ` Miklos Szeredi
2013-09-30 14:41                                           ` Ric Wheeler
2013-09-30 14:41                                             ` Ric Wheeler
2013-09-30 15:46                                             ` Miklos Szeredi
2013-09-30 15:46                                               ` Miklos Szeredi
2013-09-30 14:49                                               ` Ric Wheeler
2013-09-30 14:49                                                 ` Ric Wheeler
2013-09-30 15:57                                                 ` Miklos Szeredi
2013-09-30 15:57                                                   ` Miklos Szeredi
2013-09-30 16:31                                                   ` Miklos Szeredi
2013-09-30 16:31                                                     ` Miklos Szeredi
2013-09-30 17:17                                                     ` Bernd Schubert
2013-09-30 17:44                                                       ` Myklebust, Trond
2013-09-30 17:44                                                         ` Myklebust, Trond
2013-09-30 17:44                                                         ` Myklebust, Trond
2013-09-30 17:48                                                         ` Bernd Schubert
2013-09-30 17:48                                                           ` Bernd Schubert
2013-09-30 18:02                                                           ` Myklebust, Trond
2013-09-30 18:02                                                             ` Myklebust, Trond
2013-09-30 18:02                                                             ` Myklebust, Trond
2013-09-30 18:49                                                             ` Bernd Schubert
2013-09-30 19:34                                                               ` Myklebust, Trond
2013-09-30 19:34                                                                 ` Myklebust, Trond
2013-09-30 19:34                                                                 ` Myklebust, Trond
2013-09-30 20:00                                                                 ` Bernd Schubert
2013-09-30 20:00                                                                   ` Bernd Schubert
2013-09-30 20:08                                                                   ` Ric Wheeler
2013-09-30 20:08                                                                     ` Ric Wheeler
2013-09-30 20:27                                                                     ` Myklebust, Trond
2013-09-30 20:27                                                                       ` Myklebust, Trond
2013-09-30 20:27                                                                       ` Myklebust, Trond
2013-09-30 20:10                                                                   ` Myklebust, Trond
2013-09-30 20:10                                                                     ` Myklebust, Trond
2013-09-30 20:10                                                                     ` Myklebust, Trond
2013-10-01 18:42                                               ` J. Bruce Fields
2013-10-01 18:42                                                 ` J. Bruce Fields
2013-10-01 19:58                                       ` Zach Brown
2013-10-01 19:58                                         ` Zach Brown
2013-10-02 12:58                                         ` Jan Kara
2013-10-02 12:58                                           ` Jan Kara
2013-10-02 13:31                                           ` David Lang
2013-12-18 12:41 ` Christoph Hellwig
2013-12-18 12:41   ` Christoph Hellwig
2013-12-18 17:10   ` Zach Brown
2013-12-18 17:26     ` Anna Schumaker
  -- strict thread matches above, loose matches on Subject: below --
2013-09-26 17:22 Steve French
     [not found] ` <CAH2r5muBuTK7ZZ+aKGC4q35gqaSWF4o07eoHypLKiNn5Y83RbQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2013-09-26 19:34   ` David Disseldorp
2013-10-10  2:18     ` Steve French
2013-10-01 21:05   ` J. Bruce Fields
     [not found]     ` <20131001210531.GA7093-uC3wQj2KruNg9hUCZPvPmw@public.gmane.org>
2013-10-02  1:19       ` Steve French

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5244A5E7.90808@redhat.com \
    --to=rwheeler@redhat.com \
    --cc=Trond.Myklebust@netapp.com \
    --cc=axboe@kernel.dk \
    --cc=bfields@fieldses.org \
    --cc=bjschuma@netapp.com \
    --cc=jlbec@evilplan.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=mfasheh@suse.com \
    --cc=miklos@szeredi.hu \
    --cc=mkp@mkp.net \
    --cc=normalperson@yhbt.net \
    --cc=schumaker.anna@gmail.com \
    --cc=zab@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.