From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeff Garzik Subject: Re: HAIL volunteer Rick Peralta Date: Wed, 29 Jul 2009 14:30:37 -0400 Message-ID: <4A70954D.9000300@garzik.org> References: <29025029.1248785350151.JavaMail.root@mswamui-andean.atl.sa.earthlink.net> <4A6F5A3E.1070907@garzik.org> <4A704376.6000303@tiac.net> <4A705D5C.9050909@garzik.org> <20090729105202.7d0410de.zaitcev@redhat.com> <4A70842E.8020908@garzik.org> <4A708E00.5000304@garzik.org> <20090729121935.0e2d6f00.zaitcev@redhat.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20090729121935.0e2d6f00.zaitcev@redhat.com> Sender: hail-devel-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Pete Zaitcev Cc: Rick Peralta , Project Hail Pete Zaitcev wrote: > On Wed, 29 Jul 2009 13:59:28 -0400, Jeff Garzik wrote: > >> My point was more that there are real world situations where multiple >> outgoing streams from the client is significantly slower than a single >> stream into the cloud, plus asking the cloud to perform further copies. > > We aren't exposing Chunk to clients outside the cloud, at least at present. > Let's wait and see if applications exist that require it, and how they > compete, say, with a bunch of NFS servers. The biggest difference about > Chunk is its ability to plug into CLD. Drop that and it's not that > special. But you can only use CLD if you are inside the cloud; in fact > it works best when you're inside the same data center with the CLD cell. > So I don't see the bandwidth argument having much weight. > > I'm going to remember Fabian's idea of 3-rd party transfers, of course. > That potentially offers a significant reduction of load on tabled. > But mythical outside clients of Chunk are yet to be demostrated. Note that "mythical outside clients" is a key design element in NFS v4.1, which is a parallel distributed filesystem technology. chunkd clients are quite a bit different from CLD clients. Similarly, Hadoop DFS, CloudStore and GoogleFS clients [analagously] talk directly to chunkd rather than going through tabled. Jeff