Re: HAIL volunteer Rick Peralta

[Jeff Garzik <jeff@garzik.org>]

(added hail-devel to CC, with permission)



Rick Peralta wrote:
> Hi Jeff,
> 
> Is there someone taking the point for the chunkd development?

That's me, for the moment.  :)

In general, since Pete and I are Linux kernel developers -- one of the 
largest open source projects in the world -- I think importing the 
tried-and-true kernel requirements for code maintainership makes sense:

Once you are intimately familiar with a codebase, and can answers 
others' questions about design and code, you have reached the level 
where you could be a maintainer.

But more importantly -- that is not important!

Just contribute code, because that is the why open source projects 
advance in a particular direction anyway.  Linus Torvalds does not take 
point of the Linux kernel -- the people who actually write code do that. 
  Linus writes maybe 0.01% of the code these days.  The people who code, 
make the roadmap.


> To get to 10 Gbe there may be a variety of problems:
> 1) If there are data copies, the multiplicative effect can saturate the 
> memory system.
> 2) If the primary storage is rotating media (disk), it wold take 40+ 
> devices to keep up (assuming large stripes).
> 3) There are artifacts of the VM system that show up at high bandwidth, 
> especially if there is a lot of RAM.
> 4) The transaction frequency can become problematic, depending on the 
> application.
> 5) Running a single network transport at 10 Gbe can be challenging.

Indeed, scaling up the networking and storage can have plenty of 
implications.

We saw some of this when I was first working with NIC hardware 
manufacturers to add the first 10 Gbe NIC drivers to the Linux kernel.

chunkd is intentionally message-based, which implies that non-TCP 
protocols could readily be bolted on, for use in data centers with 10 
Gbe networks (AMQP? RDMA?).


> Is there an application profile that might be used as a performance 
> metric?  Something like total volume size (all media), total number of 
> chunks, distribution of chunk sizes, aggregate bandwidth, et cetera.

This is unfortunately going to vary wildly depending on the application 
using chunkd (and the application using that, in turn).

To take tabled as an example, and assuming a "standard cloud node" 
hardware setup,

* total single-node volume size:  one cheap SATA hard drive
* total number of chunks:  ==
	total number of tabled objects / number of storage nodes
* distribution of chunk sizes:  dependent upon the application using tabled
* aggregate bandwidth:  dependent upon the application using tabled


> If it matters I investigated building something very much like chunkd a 
> while back and has some stringent performance criteria.  It was not 
> clear what general application demand there is.  Is there a resource to 
> get a sense of where there is real need?

You should read the GoogleFS paper referenced on the chunkd wiki page: 
http://labs.google.com/papers/gfs-sosp2003.pdf  It describes the purpose 
and use of a chunk server, in the context of distributed cloud storage.

The demand is largely internal -- other Project Hail projects and 
outside distributed storage application should use chunkd in the 
creation of their own cloud-based service.

The intent is for tabled, nfs4d, and other distributed-storage projects 
to communicate with multiple chunkd's on multiple nodes, to accomplish 
replicated, highly available distributed storage.

If there is some specialized use of chunkd that you have in mind, we're 
interested in hearing that, too...   I certainly want to enable as many 
applications as possible with these projects.

Regards,

	Jeff