From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jakub Narebski Subject: Re: [RFC PATCH v7 11/9] [PoC] gitweb/lib - tee, i.e. print and capture during cache entry generation Date: Tue, 4 Jan 2011 14:20:10 +0100 Message-ID: <201101041420.11577.jnareb@gmail.com> References: <20101222234843.7998.87068.stgit@localhost.localdomain> <4D225C6E.9000108@eaglescrag.net> <201101040128.26826.jnareb@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Cc: git@vger.kernel.org, "John 'Warthog9' Hawley" To: "J.H." X-From: git-owner@vger.kernel.org Tue Jan 04 14:20:30 2011 Return-path: Envelope-to: gcvg-git-2@lo.gmane.org Received: from vger.kernel.org ([209.132.180.67]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Pa6oN-0008Ig-Pp for gcvg-git-2@lo.gmane.org; Tue, 04 Jan 2011 14:20:28 +0100 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751091Ab1ADNUX (ORCPT ); Tue, 4 Jan 2011 08:20:23 -0500 Received: from mail-wy0-f174.google.com ([74.125.82.174]:51947 "EHLO mail-wy0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750903Ab1ADNUW (ORCPT ); Tue, 4 Jan 2011 08:20:22 -0500 Received: by wyb28 with SMTP id 28so14292338wyb.19 for ; Tue, 04 Jan 2011 05:20:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date :user-agent:cc:references:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:message-id; bh=hxNmAZ1R+ALoV7ms3z8H9ek67rqjnb1nS82Vsk8Hbzs=; b=Ss4dWT7dJ9vWF2PbqW2M5svfk73zDWOmZ4e+kOgfq1l7+fvB6ntCaoK71IasNVAueQ oc+UQSovSPPybbddP/ZeijWNKPj7hHAkRFce0TYOUW48YPCLKhlZwf2saBQlsXTESnZ3 glMGFZY6P2VKCuNMsANsg3b34XVwbp30Sz4VE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding :content-disposition:message-id; b=schLVJFpPfVGR4wxMNdz+vCiNRh6wEICS6JXE5VOygZ3GBcbWDqMdmOmhG7HkjCa42 f2PGH/hkKza1wYiXGZ8D/wOu534hUmafT4KwBb9g4W+3f6yfLayBGX3uG5TOTlj+Vh3R e4etq2hxJa5hq5TTdL82VlJRS0h2huQftVQXY= Received: by 10.216.185.199 with SMTP id u49mr6957726wem.45.1294147220981; Tue, 04 Jan 2011 05:20:20 -0800 (PST) Received: from [192.168.1.13] (abvw52.neoplus.adsl.tpnet.pl [83.8.220.52]) by mx.google.com with ESMTPS id j58sm10516677wes.21.2011.01.04.05.20.18 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 04 Jan 2011 05:20:19 -0800 (PST) User-Agent: KMail/1.9.3 In-Reply-To: <201101040128.26826.jnareb@gmail.com> Content-Disposition: inline Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Archived-At: Jakub Narebski wrote: > On Tue, 4 Jan 2011, J.H. wrote: > > On 01/03/2011 01:33 PM, Jakub Narebski wrote: > > > > Instead of having gitweb use progress info indicator / throbber to > > > notify user that data is being generated by current process, gitweb > > > can now (provided that PerlIO::tee from PerlIO::Util is available) > > > send page to web browser while simultaneously saving it to cache > > > (print and capture, i.e. tee), thus having incremental generating of > > > page serve as a progress indicator. > > > > In general, and particularly for the large sites that caching is > > targeted at, teeing is a really bad idea. [...] > > 1) Errors may still be generated in flight as the cache is being > > generated. It would be better to let the cache run with a progress > > indicator and should an error occur, display the error instead of giving > > any output that may have been generated (and thus likely a broken page). > > On the contrary, with tee-ing (and zero size sanity check) you would be > able to see pages even if there are errors saving cache entry. Though > this wouldn't help very large sites which cannot function without caching, > it could be useful for smaller sites. I was not sure how Perl reacts to ENOSPC (No space left on device), which I think it is only error that can be generated in flight as cache is being generated (or as gitweb output is printed i.e. sent to browser and captured/tee-ed i.e. saved to cache entry file), so I have checked this (using loopback to create small filesystem). The outcomes one worry about are the following: * Perl dies during printing - this leads to broken page send to browser, and no cache entry generated * Perl prints output without dying at all; the page send to browser via tee-in is all right, but cache entry is truncated which results in broken page shown to other clients. But what actually happens is actually different, and quite safe: * Perl prints output without dying, and dies on closing cache entry file with ENOSPC. This means that client generating data gets correct output, and cache entry is not generated. Other clients with my code try their hand at generation and also get correct page, but not save it to cache. This means that no error page about problems with cache is shown, which is bad. On the other hand, at least for smaller sites, gitweb keeps working as if without cache for newer entries. Note that observed behaviour might depend on operating system / filesystem parameters, such as buffer sizes. > But see below. [...] > Note also that in my rewrite you can simply (by changing one single > configuration knob) configure gitweb to also cache error pages. This > might be best and safest solution for very large sites with very large > disk space, but not so good for smaller sites. Errr... now after rereading your email I see that caching error pages has one problem: errors that come from the caching engine or capturing engine - those errors you cannot cache. Sorry, my mistake. > > - John 'Warthog9' Hawley > > > > P.S. I'm back to work full-time on Wednesday, which I'll be catching up > > on gitweb and trying to make forward progress on my gitweb code again. > > I'll try to send much simplified (and easier to use in caching) error > handling using exceptions (die / eval used as throw / catch) today. Sent as [RFC PATCH v7 2.5/9] gitweb: Make die_error just die, and use send_error to create error pages Message-ID: <201101040135.08638.jnareb@gmail.com> http://permalink.gmane.org/gmane.comp.version-control.git/164466 -- Jakub Narebski Poland