Re: Design challenges in chunkd self-checking

[Jeff Garzik <jeff@garzik.org>]

On 12/22/2009 08:40 PM, Pete Zaitcev wrote:
> On Tue, 22 Dec 2009 17:43:58 -0500
> Jeff Garzik<jeff@garzik.org>  wrote:
>
>> It is normal and reasonable to maintain global information about all
>> in-progress operations.  Caching systems do that, for example, to ensure
>> multiple cache requests for object A do not initiate multiple
>> simultaneous back-end requests for object A.
>
> Unfortunately, this requires a data structure that permits searching.
> Since I lack the classical CS education, I cannot select an appropriate
> structure beyond "double-linked list and a hope that we'll never see
> more than 10 simultaneous I/Os".

What are the operational parameters?

* at beginning of object write, add entry to ADT (abstract data type, a 
list/table/whatever)

* at end of object write, remove entry from ADT

* lookup entry in ADT, where search key == object id.  if search 
succeeds, that object is guaranteed to be young and not need 
verification.  if search fails, the object (a) does not exist, or (b) 
has been completely written to disk.

Seems like a mutex-wrapped GLib hash table would work...

	Jeff