From: Dan Greenfield <dgrnfld@gmail.com>
To: linux-xfs@vger.kernel.org
Subject: XFS NVMe RDMA?
Date: Wed, 20 Oct 2021 12:51:05 +0100 [thread overview]
Message-ID: <965EC18A-BF96-4544-AFE0-FA0F1787FD49@gmail.com> (raw)
Dear XFS Experts,
as you may or may not know, XFS on NVMe was used as part of the #1 entry to the IO500 benchmarks, as announced at ISC21. That entry swept away the other competition (albeit on large custom hardware), including systems using Intel’s DAOS using Octane/PMem, WekaIO, Lustre, GekkoFS and others.
There’s no publication associated with it, however there’s a video presenting how they did it:
https://www.youtube.com/watch?v=BJpkpA6hsDc
In it they describe how they used XFS for storing data chunks, and RocksDB for storing metadata. I’m trying to dig deeper on how they could have used XFS, and in particular how they could have used RDMA to access XFS data. The XFS DAX mode as far as I’m aware requires PMem rather than NVMe?
Do you have any ideas how they could have been able to utilise RDMA so that node A can directly access data chunks stored on XFS on node B? Is the only approach to mmap the chunk on node B and then RDMA it to/from node A?
Kind regards,
Dan
next reply other threads:[~2021-10-20 11:51 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-20 11:51 Dan Greenfield [this message]
2021-10-20 16:33 ` XFS NVMe RDMA? Christoph Hellwig
2021-10-20 16:35 ` Christoph Hellwig
2021-10-21 9:58 ` Dan Greenfield
2021-10-27 8:23 ` Dan Greenfield
2021-11-12 6:40 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=965EC18A-BF96-4544-AFE0-FA0F1787FD49@gmail.com \
--to=dgrnfld@gmail.com \
--cc=linux-xfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox