Re: [RFC PATCH 2/2] ceph: test basic ceph.quota.max_bytes quota

[Dave Chinner <david@fromorbit.com>]

On Wed, Apr 03, 2019 at 02:19:11PM +0100, Luis Henriques wrote:
> Nikolay Borisov <nborisov@suse.com> writes:
> > On 3.04.19 г. 12:45 ч., Luis Henriques wrote:
> >> Dave Chinner <david@fromorbit.com> writes:
> >>> Makes no sense to me. xfs_io does a write() loop internally with
> >>> this pwrite command of 4kB writes - the default buffer size. If you
> >>> want xfs_io to loop doing 1MB sized pwrite() calls, then all you
> >>> need is this:
> >>>
> >>> 	$XFS_IO_PROG -f -c "pwrite -w -B 1m 0 ${size}m" $file | _filter_xfs_io
> >>>
> >> 
> >> Thank you for your review, Dave.  I'll make sure the next revision of
> >> these tests will include all your comments implemented... except for
> >> this one.
> >> 
> >> The reason I'm using a loop for writing a file is due to the nature of
> >> the (very!) loose definition of quotas in CephFS.  Basically, clients
> >> will likely write some amount of data over the configured limit because
> >> the servers they are communicating with to write the data (the OSDs)
> >> have no idea about the concept of quotas (or files even); the filesystem
> >> view in the cluster is managed at a different level, with the help of
> >> the MDS and the client itself.
> >> 
> >> So, the loop in this function is simply to allow the metadata associated
> >> with the file to be updated while we're writing the file.  If I use a
> >
> > But the metadata will be modified while writing the file even with a
> > single invocation of xfs_io.
> 
> No, that's not true.  It would be too expensive to keep the metadata
> server updated while writing to a file.  So, making sure there's
> actually an open/close to the file (plus the fsync in pwrite) helps
> making sure the metadata is flushed into the MDS.

/me sighs.

So you want:

	loop until ${size}MB written:
		write 1MB
		fsync
		  -> flush data to server
		  -> flush metadata to server

i.e. this one liner:

xfs_io -f -c "pwrite -D -B 1m 0 ${size}m" /path/to/file

Fundamentally, if you find yourself writing a loop around xfs_io to
break up a sequential IO stream into individual chunks, then you are
most likely doing something xfs_io can already do. And if xfs_io
cannot do it, then the right thing to do is to modify xfs_io to be
able to do it and then use xfs_io....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com