Re: libradosstriper

[Sebastien Ponce <sebastien.ponce@cern.ch>]

Anyone wanting to review the pool request mentioned below ?
Would be nice if it could be included in the next release.

Sebastien


On Thu, 2016-06-30 at 09:47 +0200, Sebastien Ponce wrote:
> I would even use the same pool, as this is perfectly allowed. Will
> look at this next week.
> 
> On another topic, I've just created a pull request for the
> improvements done on librados and libradosstriper around async calls
> : https://github.com/ceph/ceph/pull/10049. Could somebody review it ?
> 
> Also, I've added tests for the new calls, tried them and noticed that
> some tests of the rados test suite concerning writesame are failing
> right now.
> 
> Sebastien
> 
> 
> On Wed, 2016-06-29 at 10:16 -0700, Samuel Just wrote:
> > 
> > Makes sense.  I guess they'd have to pull from distinct name pools?
> > -Sam
> > On Jun 29, 2016 8:45 AM, "Sebastien Ponce" <sebastien.ponce@cern.ch
> > >
> > wrote:
> > > 
> > > Oh I see, I did not know this one indeed.
> > > I used the flag approach for the rados command line and it was
> > > really
> > > easy, as the API is identical, you only have to use a Striper
> > > object
> > > instead of an IOContext.
> > > Now looking at the RadosModel class, I see that the user API
> > > (like
> > > read/write/aio_read,...) is almost not used. Almost everything
> > > goes
> > > through the aio_operate calls wich are not present in Striper.
> > > So I believe the simplest would be to add a couple of TestOpType,
> > > namely 5 of them :
> > > TEST_OP_READ/WRITE/DELETE/SETXATTR/RMXATTR_STRIPER
> > > and implement the corresponding classes inheriting from TestOp.
> > > This would also have the advantage to mix striped and regular
> > > objects
> > > in the test, which is absolutely valid and thus should be tested.
> > > 
> > > Sebastien
> > > 
> > > 
> > > On Wed, 2016-06-29 at 08:30 -0700, Samuel Just wrote:
> > > > 
> > > > Ah, sorry.  See src/test/osd/RadosModel.* and
> > > > src/test/osd/TestRados.cc.  It's gotten to be a somewhat kludgy
> > > mess,
> > > > 
> > > > but it's a really valuable stress test for verifying that the
> > > > osd
> > > and
> > > > 
> > > > librados are behaving properly, especially when combined with
> > > failure
> > > > 
> > > > injection and osd thrashing.  I'd like to either create a (less
> > > > kludgy, I can only hope) seperate version for librados striper,
> > > or
> > > > 
> > > > add
> > > > a flag to the existing one which causes it to use the striper
> > > instead
> > > > 
> > > > (and only allows ops where it makes sense).  libradosstriper
> > > should
> > > > 
> > > > have the same ordering properties for piplined writes as normal
> > > > librados (right?), so modifying the existing test would
> > > > probably
> > > get
> > > > 
> > > > us the most coverage for the least effort.
> > > > -Sam
> > > > 
> > > > On Wed, Jun 29, 2016 at 8:03 AM, Sebastien Ponce
> > > > <sebastien.ponce@cern.ch> wrote:
> > > > > 
> > > > > 
> > > > > Well, I'm not sure what you are exactly looking for : I see
> > > > > now
> > > > > that I
> > > > > look more carefully that there is a file called
> > > > > ceph_test_rados
> > > > > that
> > > > > looks like a wrapper around the ceph_test_rados_* tests.
> > > > > The equivalent does not seem to exist for striper (although
> > > > > the
> > > > > same
> > > > > wrapper would probably work), but the
> > > > > ceph_test_rados_striper_*
> > > > > tests
> > > > > do exist.
> > > > > Should we/I create a wrapper for striper ? I'm not sure I
> > > > > have
> > > the
> > > > 
> > > > > 
> > > > > whole understanding of the structure of the whole test
> > > system...
> > > > 
> > > > > 
> > > > > 
> > > > > Sebastien
> > > > > 
> > > > > 
> > > > > On Tue, 2016-06-28 at 09:07 -0700, Samuel Just wrote:
> > > > > > 
> > > > > > 
> > > > > > I don't remember anything like ceph_test_rados which uses
> > > > > > libradosstriper.  Did I miss it?
> > > > > > -Sam
> > > > > > 
> > > > > > On Mon, Jun 27, 2016 at 8:27 PM, Sebastien Ponce
> > > > > > <sebastien.ponce@cern.ch> wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Would be great indeed. As it's already there, it can be
> > > done
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > already
> > > > > > > now.
> > > > > > > 
> > > > > > > Sebastien
> > > > > > > 
> > > > > > > On Mon, 2016-06-27 at 15:25 -0700, Samuel Just wrote:
> > > > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > > > The something like ceph_test_rados could then be wired
> > > into
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > ceph-qa-suite to be tested with osd thrashing to ensure
> > > that
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > ordering
> > > > > > > > works and continues to work as expected.
> > > > > > > > -Sam
> > > > > > > > 
> > > > > > > > On Mon, Jun 27, 2016 at 3:18 PM, Sage Weil <sweil@redha
> > > > > > > > t.
> > > com>
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > wrote:
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Hi Sebastien,
> > > > > > > > > 
> > > > > > > > > Do you mind updating us on the current status of
> > > > > > > > > libradosstriper?  Where
> > > > > > > > > does it stand feature-wise with what you need, how
> > > > > > > > > are
> > > you
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > currently using
> > > > > > > > > it or how do you plan to use it, and what is still
> > > missing?
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Also, one thing we really want to see is test
> > > > > > > > > coverage
> > > for
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > the
> > > > > > > > > code
> > > > > > > > > so
> > > > > > > > > that we can comfortably point users toward it.  This
> > > > > > > > > ideally
> > > > > > > > > would
> > > > > > > > > look
> > > > > > > > > something like the ceph_test_rados_api_* tests that
> > > verify
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > individual
> > > > > > > > > operations behave as expected, and/or something like
> > > > > > > > > ceph_test_rados that
> > > > > > > > > throws random operations at is and verifies it
> > > > > > > > > returns
> > > > > > > > > correct
> > > > > > > > > results
> > > > > > > > > that match its in-memory model.
> > > > > > > > > 
> > > > > > > > > Thanks!
> > > > > > > > > sage
> > > > > > > > > --
> > > > > > > > > To unsubscribe from this list: send the line
> > > "unsubscribe
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > ceph-
> > > > > > > > > devel" in
> > > > > > > > > the body of a message to majordomo@vger.kernel.org
> > > > > > > > > More majordomo info at  http://vger.kernel.org/majord
> > > > > > > > > om
> > > o-in
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > fo.h
> > > > > > > > > tml
> > > > > > --
> > > > > > To unsubscribe from this list: send the line "unsubscribe
> > > ceph-
> > > > 
> > > > > 
> > > > > > 
> > > > > > devel"
> > > > > > in
> > > > > > the body of a message to majordomo@vger.kernel.org
> > > > > > More majordomo info at  http://vger.kernel.org/majordomo-in
> > > > > > fo
> > > .htm
> > > > 
> > > > > 
> > > > > > 
> > > > > > l
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html