higher level library for storing large(r) RADOS objects

[Wido den Hollander <wido@widodh.nl>]

Hi,

I've been talking to Josh today and we've been talking a bit about 
storing large objects in RADOS.

One of the problem I currently see with using RADOS is storing really 
large objects.

RADOS objects are stored on the OSD as a whole file, so potentially a 
single RADOS object could press an OSD over the full_ratio and stalling 
the whole cluster.

This also shows another problem. It this object is heavily used a couple 
of OSDs will be very busy with the I/O's for this object.

So I was thinking about an library on top of RADOS which is kind of 
similar to RBD, but it's only focused on storing objects.

The first object in a pool could have a couple of xattrs:

object1
- stripe_size: 4096
- size: 40960

Based on the xattr operation we know where to read or write when asked 
for a specific offset and length.

object1, object1_1, object1_2, until object1_9

Potentially this could also be used for the RADOS Gateway? Since that 
will suffer from the same problem when you want to scale out.

With the RAODS Gateway you can't control a user storing a 200G tar file 
with his backups in it, you never know.

It's just a thought but I just wanted to get it out there and check out 
the opinions.

Comments? Suggestions?

Wido