Re: s3-tests development

[Yehuda Sadeh-Weinraub <yehuda@redhat.com>]

----- Original Message -----
> From: "Andrew Gaul" <andrew@gaul.org>
> To: ceph-devel@vger.kernel.org
> Sent: Thursday, February 26, 2015 6:37:35 PM
> Subject: s3-tests development
> 
> Hello cephalopods, I use s3-tests to test S3Proxy[1], Apache jclouds,
> and an internal project.  While s3-tests has good functionality as it
> exists, the project has not progressed much over the last six months.  I
> have submitted over 20 pull requests to fix incorrect tests and for
> additional test coverage but most remain unreviewed[2].  Further

Right. We do need to be more responsive. The main reason we took our time was that these tests are used for the ceph nightly qa tests, and any changes in these that exposes new incompatibility will fail these. We'd rather first have the issue fixed, then merge the change. An alternative way to doing it is to open a ceph tracker issue about the incompatibility, mark the test as 'fails_on_rgw', in which case we could merge it immediately.

> s3-tests remains biased towards Ceph and not the AWS de facto standard,
> failing to run against the latter due to TooManyBuckets failures[3].

Not sure what's the appropriate way to attack this one. Maybe the create_bucket function could keep a list of created buckets and then remove them if encountering this error using lru.

> Finally some features like V4 signature support[4] will require more
> extensive changes.  We are at-risk of diverging; how can we best move
> forward together?

We'd be happy with more tests, and with more features tested even if these weren't implemented yet. It can later help with the development of these features. E.g., I looked a few weeks back at v4 signatures, and having such a test would have helped. The only thing we need though would be a way to easily disable such tests. So adding 'fails_on_rgw', or some other way to detect would help a lot.

Thanks,
Yehuda

> 
> [1] https://github.com/andrewgaul/s3proxy
> [2] https://github.com/ceph/s3-tests/pulls?q=is%3Aopen+is%3Apr
> [3] https://github.com/ceph/s3-tests/issues/25
> [4] https://github.com/ceph/s3-tests/issues/35
>