From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeff Garzik Subject: Re: [PATCH] chunkd: add support for multiple key/value tables Date: Tue, 10 Nov 2009 14:45:50 -0500 Message-ID: <4AF9C2EE.3040401@garzik.org> References: <20091110112409.GA31471@havoc.gtf.org> <20091110093322.313563c2@redhat.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20091110093322.313563c2@redhat.com> Sender: hail-devel-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Pete Zaitcev Cc: hail-devel@vger.kernel.org On 11/10/2009 11:33 AM, Pete Zaitcev wrote: > On Tue, 10 Nov 2009 06:24:09 -0500, Jeff Garzik wrote: > >> LOGIN(user="jgarzik") >> TABLE-OPEN(name="tabled") >> GET... > > 2 more turnarounds per session? Brilliant! > > The theory behind this is sound: let's not saddle chunkd with caching > authentication results, which is ineffective anyway, but provide > a way for application to amortize the cost of authentication over > a number of requests explicitly. But in practice it means tabled > needs to keep inactive sessions open, which is a chunk of code for > me to write (and debug!). I guess I'll do it in a few months... chunkd protocol was never intended to be a connect+request+disconnect model... HTTP 1.0 proved that was a bad model, which is why the world moved on to pipelined, multiple-request protocols. connect+request+disconnect protocols waste kernel, IPVS, firewall and router resources, and are distinctly network unfriendly. So yeah... tabled does need to keep chunkd sessions open, after a chunkd request completes. That was always true, regardless of the multi-kv API change in $Subject. As a matter of fact, libchunkdc is actually a limiting factor here: even though the chunkd network protocol is pipeline-able, libchunkdc always waits for a response before returning control back to the application. That is not strictly necessary: an application could choose to submit 10 'DEL' requests in a single write(2), and then wait for 10 responses from the server, if it so wished. >> @@ -29,9 +29,15 @@ static void test(bool encrypt) >> stc1 = stc_new(TEST_HOST, port, TEST_USER, TEST_USER_KEY, encrypt); >> OK(stc1); >> >> + rcb = stc_table_openz(stc1, TEST_TABLE, 0); >> + OK(rcb); >> + >> stc2 = stc_new(TEST_HOST, port, TEST_USER2, TEST_USER2_KEY, encrypt); >> OK(stc2); > > Having a default table? Naah, those lazy application programmers have > it too easy already! > > Again, from the point of view of chunkd, this makes complete sense: > why carry an extra (default) table in cases when application does > in fact set its own tables, right? If people want a default table, I can put it in. MySQL tries to connect to a database $Username, if database name is not supplied, for example. But yes, from point of view of chunkd simplicity, no-default-table is certainly more simple, which makes me reluctant to add it. If the separate API call is bothersome, we could pass table name to stc_new(). >> + /* >> + * we must supply CHF_TABLE_NEW on the first iteration of >> + * this test, because we are the first test in the testsuite, >> + * and must create the database to be used by all other tests. >> + */ >> + rcb = stc_table_openz(stc, TEST_TABLE, >> + ssl ? 0 : CHF_TABLE_NEW); >> + OK(rcb); > > You've got to be kidding me. How is tabled supposed to know that > the request it's making is "first"?! I guess I have to supply > CHF_TABLE_NEW to every call now, or else retry if InvalidTable > is returned, I haven't decided what workaround to apply yet. A fair question... It seemed logical to create the table at the time a new chunkd node comes online, and that the application would want to always run in normal mode WITHOUT CHF_TABLE_NEW -- thus making a table's unexpected absence a hard error, mirroring real life. Easy alternatives include (a) create on demand and never worry about this detail, and (b) add an 'exclusive' flag analagous to O_EXCL. This complements CHF_TABLE_NEW, which is analagous to O_CREAT. Jeff