From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jeff King Subject: Re: Keeping unreachable objects in a separate pack instead of loose? Date: Tue, 12 Jun 2012 15:41:26 -0400 Message-ID: <20120612194126.GA17519@sigill.intra.peff.net> References: <20120612171048.GB12706@sigill.intra.peff.net> <20120612173214.GA16014@sigill.intra.peff.net> <20120612175046.GA16522@sigill.intra.peff.net> <20120612192318.GC16911@sigill.intra.peff.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Cc: Andreas Schwab , Shawn Pearce , Ted Ts'o , Thomas Rast , Hallvard B Furuseth , git@vger.kernel.org To: Nicolas Pitre X-From: git-owner@vger.kernel.org Tue Jun 12 21:41:34 2012 Return-path: Envelope-to: gcvg-git-2@plane.gmane.org Received: from vger.kernel.org ([209.132.180.67]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SeWy5-0007zb-Oh for gcvg-git-2@plane.gmane.org; Tue, 12 Jun 2012 21:41:34 +0200 Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752061Ab2FLTla (ORCPT ); Tue, 12 Jun 2012 15:41:30 -0400 Received: from 99-108-225-23.lightspeed.iplsin.sbcglobal.net ([99.108.225.23]:51317 "EHLO peff.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751234Ab2FLTl2 (ORCPT ); Tue, 12 Jun 2012 15:41:28 -0400 Received: (qmail 25190 invoked by uid 107); 12 Jun 2012 19:41:30 -0000 Received: from sigill.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.7) (smtp-auth username relayok, mechanism cram-md5) by peff.net (qpsmtpd/0.84) with ESMTPA; Tue, 12 Jun 2012 15:41:30 -0400 Received: by sigill.intra.peff.net (sSMTP sendmail emulation); Tue, 12 Jun 2012 15:41:26 -0400 Content-Disposition: inline In-Reply-To: Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Archived-At: On Tue, Jun 12, 2012 at 03:39:05PM -0400, Nicolas Pitre wrote: > > So the race window depends on the time it takes "git prune" to run. > > > > I wonder if git-prune could do a double-check of the refs. Something > > like: > > > > 1. calculate reachability on all refs > > > > 2. read list of objects to prune, and make a list of unreachable ones > > > > 3. calculate reachability again (which should be very cheap, because > > you can stop when you get to an object you have already seen) > > > > 4. Drop any objects found in (3) from the list in (2), and delete > > items from your list > > > > But I think that still has a race where objects are created before > > step 2, but are not actually referenced until after step 3. I think > > doing it safely may actually require a repo-wide prune lock. > > Yeah... that's what I was thinking too. Maybe we're making our life > overly miserable by trying to avoid any locking here. I think I would be OK with "prune" locking, as long as everything else was able to happen simultaneously. Especially if we can keep prune's lock as short as possible through double-reads or similar tricks (like we do for ref updates). -Peff