Re: Backend ObjectStore engine performance bench with FIO

[Casey Bodley <cbodley@redhat.com>]

Hi James/Haomai/Xiaoxi,

I spent some more time on the fio-objectstore branch, and pushed an update.

In testing, I realized that it was using the io_unit's start time to name the
objects, which meant that every write operation was creating a separate object.
In addition to fixing this to use fio's filenames for object names, I also
added support for the open_file() and close_file() functions. It now creates
objects of the proper size on startup, so read-only jobs will work normally.
It also removes its objects on exit.

On startup, it no longer calls create_collection() if it already exists, so I
was able to re-run fio jobs over and over again without having to clear the
data directory (tested with FileStore and KeyValueStore).

Casey

----- Original Message -----
> Great work James!
> 
> ----- Original Message -----
> > From: "James (Fei) Liu-SSI" <james.liu@ssi.samsung.com>
> > To: "Xiaoxi Chen" <xiaoxi.chen@intel.com>, "Casey Bodley"
> > <cbodley@redhat.com>
> > Cc: "Sage Weil" <sage@newdream.net>, ceph-devel@vger.kernel.org
> > Sent: Friday, September 25, 2015 1:55:29 PM
> > Subject: Backend ObjectStore engine performance bench with FIO
> > 
> > Hi Xiaoxi,
> > 
> >    With changing the IO mode from aio to sync, we make fio against newstore
> >    works. Even  with sync engine(I am still debugging the aio engine in
> >    newstore with Xiaoxi) in newstore, Newstore still performing the best
> >    among all of backstore engine with our initial setup(Thoroughly test
> >    will
> >    be run soon). Attachment is the initial data we collected for your
> >    reference. Thanks for great help from Xiaoxi from regarding to Newstore
> >    development to support FIO.
> > 
> > Hi Casey,
> >   Let me know if you need any help to put fio-ceph-objectstore into
> >   upstream.
> >   After then , I can commit all of mine into upstream.
> 
> My pull request at https://github.com/ceph/ceph/pull/5943 is still pending.
> If you have patches that you'd like included, I would be happy to pull them
> in; just point me to a branch.
> 
> > 
> >   Thanks,
> >   James
> > 
> 
> Thanks,
> Casey
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>