client/cluster compatibility testing

client/cluster compatibility testing

[Sage Weil]

Now that there are several different vendors shipping and supporting Ceph 
in their products, we'll invariably have people running different 
versions of Ceph that are interested in interoperability.  If we focus 
just on client <-> cluster compatability, I think the issues are (1) 
compatibility between upstream ceph versions (firefly vs hammer) and 
(2) ensuring that any downstream changes the vendor makes don't break that 
compatibility.

I think the simplest way to address this is to talk about compatibility in 
terms of the upstream stable releases (firefly, hammer, etc.), and test 
that compatibility with teuthology tests from ceph-qa-suite.git.  We have 
some basic inter-version client/cluster tests already in 
suites/upgrade/client-upgrade.  Currently these test new (version "x") 
clients against a given release (dumpling, firefly).  I think we just need 
to add hammer to that mix, and then add a second set of tests that do the 
reverse: test clients from a given release (dumpling, firefly, hammer) 
against an arbitrary cluster version ("x").

We'll obviously run these tests on upstream releases to ensure that we are 
not breaking compatibility (or are doing so in known, explicit ways).  
Downstream folks can run the same test suites against any changes they 
make as well to ensure that their product is "compatible with firefly 
clients," or whatever.

Does that sound reasonable?
sage

Re: client/cluster compatibility testing

[Josh Durgin]

On 04/16/2015 09:42 AM, Sage Weil wrote:
> I think the simplest way to address this is to talk about compatibility in
> terms of the upstream stable releases (firefly, hammer, etc.), and test
> that compatibility with teuthology tests from ceph-qa-suite.git.  We have
> some basic inter-version client/cluster tests already in
> suites/upgrade/client-upgrade.  Currently these test new (version "x")
> clients against a given release (dumpling, firefly).  I think we just need
> to add hammer to that mix, and then add a second set of tests that do the
> reverse: test clients from a given release (dumpling, firefly, hammer)
> against an arbitrary cluster version ("x").

The suites in suites/upgrade/$version-x do this, and use a mixed
version cluster rather than a purely version x cluster. It seems like
people would want that intra-cluster version coverage for smooth
upgrades.

Just need to add hammer-x there too (Yuri's renaming the client ones to
be $version-client-x for less confusion).

Also I think we'll want to start doing mixed-client-version tests,
particularly for things like rbd's exclusive locking:

http://tracker.ceph.com/issues/11405

Josh

Re: client/cluster compatibility testing

[Yuri Weinstein]

We have a PR https://github.com/ceph/ceph-qa-suite/pull/414 that addressed part of this issue, e.g. added hammer-x to the mix (it's ready to be merged).

We also have two tickets where requirements for this suite were captured:
http://tracker.ceph.com/issues/11413
http://tracker.ceph.com/issues/11414

Per Josh's comment "Also I think we'll want to start doing mixed-client-version tests, particularly for things like rbd's exclusive locking", I assigned #11414 for next steps.

Question/request to the team leads - pls either agree with a need to add specific tests for mixed clients testing (and pls add tickets as you feel necessary.) or suggest otherwise.

I am guessing:

rbd - confirmed by Josh, we need those
rados - Sam, Sage?
cephfs - Greg?
rgw - Yehuda?

I am sure I missing lots of others...

What do you think?

Thx
YuriW

----- Original Message -----
From: "Josh Durgin" <jdurgin@redhat.com>
To: "Sage Weil" <sweil@redhat.com>, ceph-devel@vger.kernel.org
Sent: Thursday, April 16, 2015 1:59:11 PM
Subject: Re: client/cluster compatibility testing

On 04/16/2015 09:42 AM, Sage Weil wrote:
> I think the simplest way to address this is to talk about compatibility in
> terms of the upstream stable releases (firefly, hammer, etc.), and test
> that compatibility with teuthology tests from ceph-qa-suite.git.  We have
> some basic inter-version client/cluster tests already in
> suites/upgrade/client-upgrade.  Currently these test new (version "x")
> clients against a given release (dumpling, firefly).  I think we just need
> to add hammer to that mix, and then add a second set of tests that do the
> reverse: test clients from a given release (dumpling, firefly, hammer)
> against an arbitrary cluster version ("x").

The suites in suites/upgrade/$version-x do this, and use a mixed
version cluster rather than a purely version x cluster. It seems like
people would want that intra-cluster version coverage for smooth
upgrades.

Just need to add hammer-x there too (Yuri's renaming the client ones to
be $version-client-x for less confusion).

Also I think we'll want to start doing mixed-client-version tests,
particularly for things like rbd's exclusive locking:

http://tracker.ceph.com/issues/11405

Josh
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: client/cluster compatibility testing

[Yuri Weinstein]

Yea, Sage, that sounds reasonable.

I added a ticket to capture this plan (http://tracker.ceph.com/issues/11413) and will add those tests soon.

Please add your comments to the ticket above.

I am assuming that it will look something like this for dumpling, firefly and hammer:

dumpling(stable) -> client-x
firefly(stable) -> client-x
hammer(stable) -> client-x

and reverse

dumpling-client(stable) -> cluster-x
firefly-cluster(stable) -> cluster-x
hammer-cluster(stable) -> cluster-x

Yes?

Thx
YuriW

----- Original Message -----
From: "Sage Weil" <sweil@redhat.com>
To: ceph-devel@vger.kernel.org
Sent: Thursday, April 16, 2015 9:42:29 AM
Subject: client/cluster compatibility testing

Now that there are several different vendors shipping and supporting Ceph 
in their products, we'll invariably have people running different 
versions of Ceph that are interested in interoperability.  If we focus 
just on client <-> cluster compatability, I think the issues are (1) 
compatibility between upstream ceph versions (firefly vs hammer) and 
(2) ensuring that any downstream changes the vendor makes don't break that 
compatibility.

I think the simplest way to address this is to talk about compatibility in 
terms of the upstream stable releases (firefly, hammer, etc.), and test 
that compatibility with teuthology tests from ceph-qa-suite.git.  We have 
some basic inter-version client/cluster tests already in 
suites/upgrade/client-upgrade.  Currently these test new (version "x") 
clients against a given release (dumpling, firefly).  I think we just need 
to add hammer to that mix, and then add a second set of tests that do the 
reverse: test clients from a given release (dumpling, firefly, hammer) 
against an arbitrary cluster version ("x").

We'll obviously run these tests on upstream releases to ensure that we are 
not breaking compatibility (or are doing so in known, explicit ways).  
Downstream folks can run the same test suites against any changes they 
make as well to ensure that their product is "compatible with firefly 
clients," or whatever.

Does that sound reasonable?
sage

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html