From: Christoph Hellwig <hch@lst.de>
To: Dave Chinner <david@fromorbit.com>
Cc: Jan Kara <jack@suse.cz>,
linux-nvdimm@lists.01.org,
"Darrick J. Wong" <darrick.wong@oracle.com>,
linux-kernel@vger.kernel.org, Christoph Hellwig <hch@lst.de>,
linux-xfs@vger.kernel.org,
Alexander Viro <viro@zeniv.linux.org.uk>,
Andy Lutomirski <luto@kernel.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Colin Walters <walters@verbum.org>
Subject: Re: [PATCH 1/3] fs, xfs: introduce S_IOMAP_IMMUTABLE
Date: Sat, 5 Aug 2017 11:45:56 +0200 [thread overview]
Message-ID: <20170805094556.GA14930@lst.de> (raw)
In-Reply-To: <20170801024218.GN17762@dastard>
On Tue, Aug 01, 2017 at 12:42:18PM +1000, Dave Chinner wrote:
> I've outlined other use cases in previous discussions. To repeat
> myself, every so often we get someone with, say, a new high
> speed camera that want to dma the camera frames direct to the
> storage because they can't push 500,000 frames/s through the CPU
> to storage. Hence they want to bypass the OS and DMA the data direct
> to the storage. To do this they need a mechanism to freeze and unfreeze
> the block map of the file so that nothing modifies the block map
> while the camera hardware is dumping data direct to the storage.
> Immutable extent maps provide the functionality they need to
> implement this safely.
And we have such a mechanism already: it's called the iolock during
I/O, and dirct I/O. I've worked on plenty such schemes and the proper way
works perfectly fine. Just because people ask for stupid ways to
archives that doesn't mean they understand what they are doing.
> There's also other similar use cases for RDMA targets on PMEM
> (regardless of whether DAX is enabled or not), and I've come across
> a couple of requests for mechanisms to allow fabric based nvme
> storage to do direct data transfers between storage devices, too.
> All of these use cases can be safely implemented if there is a
> mechanism to mark extent maps as immutable for the duration of
> the operation they need to perform.
As someone who spent most of them time on the last 2 years in this
area: we have a massive problem discoverability and addressing
(lack of struct page) for p2p devices. We have absolutely no problem
with the direct I/O model with them.
> DAX isn't the driver of that functionality, it's the other use cases
> that need it, and why the proposed "only remove flag if len == 0"
> API is a non-starter....
The other "use" cases are even more bullshit than the DAX one.
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
WARNING: multiple messages have this Message-ID (diff)
From: Christoph Hellwig <hch@lst.de>
To: Dave Chinner <david@fromorbit.com>
Cc: Colin Walters <walters@verbum.org>,
"Darrick J. Wong" <darrick.wong@oracle.com>,
Dan Williams <dan.j.williams@intel.com>, Jan Kara <jack@suse.cz>,
linux-nvdimm@lists.01.org, linux-kernel@vger.kernel.org,
linux-xfs@vger.kernel.org, Jeff Moyer <jmoyer@redhat.com>,
Alexander Viro <viro@zeniv.linux.org.uk>,
Andy Lutomirski <luto@kernel.org>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Ross Zwisler <ross.zwisler@linux.intel.com>,
Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH 1/3] fs, xfs: introduce S_IOMAP_IMMUTABLE
Date: Sat, 5 Aug 2017 11:45:56 +0200 [thread overview]
Message-ID: <20170805094556.GA14930@lst.de> (raw)
In-Reply-To: <20170801024218.GN17762@dastard>
On Tue, Aug 01, 2017 at 12:42:18PM +1000, Dave Chinner wrote:
> I've outlined other use cases in previous discussions. To repeat
> myself, every so often we get someone with, say, a new high
> speed camera that want to dma the camera frames direct to the
> storage because they can't push 500,000 frames/s through the CPU
> to storage. Hence they want to bypass the OS and DMA the data direct
> to the storage. To do this they need a mechanism to freeze and unfreeze
> the block map of the file so that nothing modifies the block map
> while the camera hardware is dumping data direct to the storage.
> Immutable extent maps provide the functionality they need to
> implement this safely.
And we have such a mechanism already: it's called the iolock during
I/O, and dirct I/O. I've worked on plenty such schemes and the proper way
works perfectly fine. Just because people ask for stupid ways to
archives that doesn't mean they understand what they are doing.
> There's also other similar use cases for RDMA targets on PMEM
> (regardless of whether DAX is enabled or not), and I've come across
> a couple of requests for mechanisms to allow fabric based nvme
> storage to do direct data transfers between storage devices, too.
> All of these use cases can be safely implemented if there is a
> mechanism to mark extent maps as immutable for the duration of
> the operation they need to perform.
As someone who spent most of them time on the last 2 years in this
area: we have a massive problem discoverability and addressing
(lack of struct page) for p2p devices. We have absolutely no problem
with the direct I/O model with them.
> DAX isn't the driver of that functionality, it's the other use cases
> that need it, and why the proposed "only remove flag if len == 0"
> API is a non-starter....
The other "use" cases are even more bullshit than the DAX one.
next prev parent reply other threads:[~2017-08-05 9:43 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-29 19:43 [PATCH 0/3] fs, xfs: block map immutable files for dax, dma-to-storage, and swap Dan Williams
2017-07-29 19:43 ` Dan Williams
2017-07-29 19:43 ` [PATCH 1/3] fs, xfs: introduce S_IOMAP_IMMUTABLE Dan Williams
2017-07-29 19:43 ` Dan Williams
2017-07-31 16:02 ` Colin Walters
2017-07-31 16:02 ` Colin Walters
2017-07-31 16:29 ` Dan Williams
2017-07-31 16:29 ` Dan Williams
2017-07-31 16:32 ` Colin Walters
2017-07-31 16:32 ` Colin Walters
2017-07-31 17:42 ` Colin Walters
2017-07-31 17:42 ` Colin Walters
2017-07-31 18:23 ` Darrick J. Wong
2017-07-31 18:23 ` Darrick J. Wong
2017-08-01 2:15 ` Colin Walters
2017-08-01 2:15 ` Colin Walters
2017-08-01 2:42 ` Dave Chinner
2017-08-01 2:42 ` Dave Chinner
2017-08-05 9:45 ` Christoph Hellwig [this message]
2017-08-05 9:45 ` Christoph Hellwig
2017-07-31 16:46 ` Darrick J. Wong
2017-07-31 16:46 ` Darrick J. Wong
2017-07-31 17:32 ` Dan Williams
2017-07-31 17:32 ` Dan Williams
2017-07-29 19:43 ` [PATCH 2/3] fs, xfs: introduce FALLOC_FL_SEAL_BLOCK_MAP Dan Williams
2017-07-29 19:43 ` Dan Williams
2017-07-29 19:43 ` Dan Williams
2017-07-31 17:09 ` Darrick J. Wong
2017-07-31 17:09 ` Darrick J. Wong
2017-07-31 18:25 ` Dan Williams
2017-07-31 18:25 ` Dan Williams
2017-08-01 0:30 ` Dave Chinner
2017-08-01 0:30 ` Dave Chinner
2017-07-29 19:43 ` [PATCH 3/3] xfs: persist S_IOMAP_IMMUTABLE in di_flags2 Dan Williams
2017-07-29 19:43 ` Dan Williams
2017-07-31 17:15 ` Darrick J. Wong
2017-07-31 17:15 ` Darrick J. Wong
2017-08-01 0:42 ` Dave Chinner
2017-08-01 0:42 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170805094556.GA14930@lst.de \
--to=hch@lst.de \
--cc=darrick.wong@oracle.com \
--cc=david@fromorbit.com \
--cc=jack@suse.cz \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-xfs@vger.kernel.org \
--cc=luto@kernel.org \
--cc=viro@zeniv.linux.org.uk \
--cc=walters@verbum.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.