From: Dave Chinner <david@fromorbit.com>
To: Sage Weil <sweil@redhat.com>
Cc: mnelson@redhat.com, ceph-devel <ceph-devel@vger.kernel.org>,
xfs@oss.sgi.com, 马建朋 <majianpeng@gmail.com>
Subject: Re: file journal fadvise
Date: Tue, 2 Dec 2014 13:01:35 +1100 [thread overview]
Message-ID: <20141202020135.GL9561@dastard> (raw)
In-Reply-To: <alpine.DEB.2.00.1412011719070.3471@cobra.newdream.net>
On Mon, Dec 01, 2014 at 05:24:46PM -0800, Sage Weil wrote:
> On Tue, 2 Dec 2014, Dave Chinner wrote:
> > On Mon, Dec 01, 2014 at 04:12:03PM -0800, Sage Weil wrote:
> > > On Tue, 2 Dec 2014, Dave Chinner wrote:
> > > > What behaviour are you wanting for a journal file? it sounds like
> > > > you want it to behave like a wandering log: automatically allocating
> > > > it's next block where-ever the previous write of any kind occurred?
> > >
> > > Precisely. Well, as long as it is adjacent to *some* other scheduled
> > > write, it would save us a seek. The real question, I guess, is whether
> > > there is an XFS allocation mode that makes no attempt to avoid
> > > fragmentation for the file and that chooses something adjacent to other
> > > small, newly-written data during delayed allocation.
> >
> > Ok, so what is the most common underlying storage you need to
> > optimise for? Is it raid5/6 where a small write will trigger a
> > larger RMW cycle and so proximity rather than exact adjacency
> > matters, or is it raid 0/1/jbod where exact adjacency is the only
> > way to avoid a seek?
>
> The common case is a single raw disk.
Ok, so it's an exact match that is really required. I'll have a
think about it.
> > > It's a circular file, usually a few GB in site, written sequentially with
> > > a range of small to large (block-aligned) write sizes, and (for all
> > > intents and purposes) is never read. We periodically overwrite the first
> > > block with recent start and end pointers and other metadata.
> >
> > Ok, so it's just another typical WAL file. ;)
>
> Nothing to lose sleep over if this mode doesn't already exist, but I
> expect a fair number of applications could make use of this.
>
> FWIW, while I am already distracting you from useful things, I suspect
> (batched) aio_fsync would be a bigger win for us and probably a smaller
> investment of effort. :)
If you want to test a patch that implements a basic, simple
implementation of aio_fsync:
http://oss.sgi.com/archives/xfs/2014-06/msg00214.html
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
WARNING: multiple messages have this Message-ID (diff)
From: Dave Chinner <david@fromorbit.com>
To: Sage Weil <sweil@redhat.com>
Cc: ceph-devel <ceph-devel@vger.kernel.org>,
马建朋 <majianpeng@gmail.com>,
mnelson@redhat.com, xfs@oss.sgi.com
Subject: Re: file journal fadvise
Date: Tue, 2 Dec 2014 13:01:35 +1100 [thread overview]
Message-ID: <20141202020135.GL9561@dastard> (raw)
In-Reply-To: <alpine.DEB.2.00.1412011719070.3471@cobra.newdream.net>
On Mon, Dec 01, 2014 at 05:24:46PM -0800, Sage Weil wrote:
> On Tue, 2 Dec 2014, Dave Chinner wrote:
> > On Mon, Dec 01, 2014 at 04:12:03PM -0800, Sage Weil wrote:
> > > On Tue, 2 Dec 2014, Dave Chinner wrote:
> > > > What behaviour are you wanting for a journal file? it sounds like
> > > > you want it to behave like a wandering log: automatically allocating
> > > > it's next block where-ever the previous write of any kind occurred?
> > >
> > > Precisely. Well, as long as it is adjacent to *some* other scheduled
> > > write, it would save us a seek. The real question, I guess, is whether
> > > there is an XFS allocation mode that makes no attempt to avoid
> > > fragmentation for the file and that chooses something adjacent to other
> > > small, newly-written data during delayed allocation.
> >
> > Ok, so what is the most common underlying storage you need to
> > optimise for? Is it raid5/6 where a small write will trigger a
> > larger RMW cycle and so proximity rather than exact adjacency
> > matters, or is it raid 0/1/jbod where exact adjacency is the only
> > way to avoid a seek?
>
> The common case is a single raw disk.
Ok, so it's an exact match that is really required. I'll have a
think about it.
> > > It's a circular file, usually a few GB in site, written sequentially with
> > > a range of small to large (block-aligned) write sizes, and (for all
> > > intents and purposes) is never read. We periodically overwrite the first
> > > block with recent start and end pointers and other metadata.
> >
> > Ok, so it's just another typical WAL file. ;)
>
> Nothing to lose sleep over if this mode doesn't already exist, but I
> expect a fair number of applications could make use of this.
>
> FWIW, while I am already distracting you from useful things, I suspect
> (batched) aio_fsync would be a bigger win for us and probably a smaller
> investment of effort. :)
If you want to test a patch that implements a basic, simple
implementation of aio_fsync:
http://oss.sgi.com/archives/xfs/2014-06/msg00214.html
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2014-12-02 2:01 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-30 18:46 file journal fadvise Sage Weil
2014-12-01 2:09 ` 马建朋
2014-12-01 3:26 ` Sage Weil
2014-12-01 19:18 ` Mark Nelson
2014-12-01 19:23 ` Sage Weil
2014-12-01 22:31 ` Mark Nelson
2014-12-01 22:31 ` Mark Nelson
2014-12-01 22:51 ` Dave Chinner
2014-12-02 0:12 ` Sage Weil
2014-12-02 0:12 ` Sage Weil
2014-12-02 0:32 ` Dave Chinner
2014-12-02 0:32 ` Dave Chinner
2014-12-02 1:24 ` Sage Weil
2014-12-02 2:01 ` Dave Chinner [this message]
2014-12-02 2:01 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141202020135.GL9561@dastard \
--to=david@fromorbit.com \
--cc=ceph-devel@vger.kernel.org \
--cc=majianpeng@gmail.com \
--cc=mnelson@redhat.com \
--cc=sweil@redhat.com \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.