Re: [PATCH 00/17] orangefs: page cache

[Mike Marshall <hubcap@omnibond.com>]

That seems like one of several writeback errors that might occur...
It looks to me
like our code would set PG_error through setPageError and then the error should
be returned back to the application through close or fsync which seems
a little late.

I guess this is the kind of writeback error "mess" that Jeff Layton
was talking about
at LFSMM a couple of years ago...

-Mike
On Mon, Oct 1, 2018 at 4:03 PM Andreas Dilger <adilger@dilger.ca> wrote:
>
> On Sep 20, 2018, at 12:31 PM, Mike Marshall <hubcap@omnibond.com> wrote:
> >
> > Using the page cache seems like a game changer for the Orangefs kernel module.
> > Workloads with small IO suffer trying to push a parallel filesystem
> > with just a handful of bytes at a time. Below, vm2 with Fedora's 4.17
> > has /pvfsmnt mounted from an Orangefs filesystem that is itself running
> > on vm2. vm1 with 4.19.0-rc2  plus the Orangefs page cache patch, also has
> > its /pvfsmnt mounted from a local Orangefs filesystem.
>
> Is there some mechanism to prevent the client cache size exceeding the amount
> of free space on the filesystem?  If not, then the client may write data that
> can never be flushed to disk on the server.
>
> Cheers, Andreas
>
> > [vm2]$ dd if=/dev/zero of=/pvfsmnt/d.vm2/d.foo/dds.out bs=128 count=4194304
> > 4194304+0 records in
> > 4194304+0 records out
> > 536870912 bytes (537 MB, 512 MiB) copied, 662.013 s, 811 kB/s
> >
> > [vm1]$ dd if=/dev/zero of=/pvfsmnt/d.vm1/d.foo/dds.out bs=128 count=4194304
> > 4194304+0 records in
> > 4194304+0 records out
> > 536870912 bytes (537 MB, 512 MiB) copied, 11.3072 s, 47.5 MB/s
> >
> > Small IO collects in the page cache until a reasonable amount of
> > data is available for writeback.
> >
> > The trick, it seems, is to improve small IO without harming large IO.
> > Aligning writeback sizes, when possible, with the size of the IO buffer
> > that the Orangefs kernel module shares with its userspace component seems
> > promising on my dinky vm tests.
> >
> > -Mike
> >
> > On Mon, Sep 17, 2018 at 4:11 PM Martin Brandenburg <martin@omnibond.com> wrote:
> >>
> >> If no major issues are found in review or in our testing, we intend to
> >> submit this during the next merge window.
> >>
> >> The goal of all this is to significantly reduce the number of network
> >> requests made to the OrangeFS
> >>
> >> First the xattr cache is needed because otherwise we make a ton of
> >> getxattr calls from security_inode_need_killpriv.
> >>
> >> Then there's some reorganization so inode changes can be cached.
> >> Finally, we enable write_inode.
> >>
> >> Then remove the old readpages.  Next there's some reorganization to
> >> support readpage/writepage.  Finally, enable readpage/writepage which
> >> is fairly straightforward except for the need to separate writes from
> >> different uid/gid pairs due to the design of our server.
> >>
> >> Martin Brandenburg (17):
> >>  orangefs: implement xattr cache
> >>  orangefs: do not invalidate attributes on inode create
> >>  orangefs: simply orangefs_inode_getattr interface
> >>  orangefs: update attributes rather than relying on server
> >>  orangefs: hold i_lock during inode_getattr
> >>  orangefs: set up and use backing_dev_info
> >>  orangefs: let setattr write to cached inode
> >>  orangefs: reorganize setattr functions to track attribute changes
> >>  orangefs: remove orangefs_readpages
> >>  orangefs: service ops done for writeback are not killable
> >>  orangefs: migrate to generic_file_read_iter
> >>  orangefs: implement writepage
> >>  orangefs: skip inode writeout if nothing to write
> >>  orangefs: write range tracking
> >>  orangefs: avoid fsync service operation on flush
> >>  orangefs: use kmem_cache for orangefs_write_request
> >>  orangefs: implement writepages
> >>
> >> fs/orangefs/acl.c             |   4 +-
> >> fs/orangefs/file.c            | 193 ++++--------
> >> fs/orangefs/inode.c           | 576 +++++++++++++++++++++++++++-------
> >> fs/orangefs/namei.c           |  41 ++-
> >> fs/orangefs/orangefs-cache.c  |  24 +-
> >> fs/orangefs/orangefs-kernel.h |  56 +++-
> >> fs/orangefs/orangefs-mod.c    |  10 +-
> >> fs/orangefs/orangefs-utils.c  | 181 +++++------
> >> fs/orangefs/super.c           |  38 ++-
> >> fs/orangefs/waitqueue.c       |  18 +-
> >> fs/orangefs/xattr.c           | 104 ++++++
> >> 11 files changed, 839 insertions(+), 406 deletions(-)
> >>
> >> --
> >> 2.19.0
> >>
>
>
> Cheers, Andreas
>
>
>
>
>