Re: efficient removal of old objects

[Mark Kampe <Mark.Kampe@dreamhost.com>]

On 01/31/12 17:02, Tommi Virtanen wrote:

> To make my point even clearer: point me to another data store that has
> that idiom.

(a) Automatic expiration and deletion is, and has long been, a
     standard feature of archival systems ... and our RADOS
     clouds are much larger than most archival systems.

(b) I have no competent opinions on the short term solution to this
     particular problem, but in the longer term I do not believe
     that garbage collection can or should be entrusted to clients.
     Clients are ephemeral and cannot be depended on to remember,
     a few years (or even hours) from now, that there were some
     files they were supposed to delete.

     IMHO, object store intelligence is not merely about back-ground
     replication and migration, but about "being able to take
     responsibility for the life cycle of the data they hold".
     The amount of data we store will quickly grow beyond the
     ability of external agents to manage it, and lifecycle
     automation will become increasingly critical.