Re: Lockless Refs?

[Martin Fick <mfick@codeaurora.org>]

On Friday, December 28, 2012 09:58:36 am Junio C Hamano 
wrote:
> Martin Fick <mfick@codeaurora.org> writes:
> > 3) To create a ref, it must be renamed from the null
> > file (sha 0000...) to the new value just as if it were
> > being updated from any other value, but there is one
> > extra condition: before renaming the null file, a full
> > directory scan must be done to ensure that the null
> > file is the only file in the directory...
> 
> While you are scanning this directory to make sure it is
> empty, 

The objective is not to scan for an empty dir, but to scan 
for the existence of only the null file.

> I am contemplating to create the same ref with a
> different value.  You finished checking but haven't
> created the null.

The scan needs to happen after creating the null, not before, 
so I don't believe the rest of the scenario below is possible 
then?

> I have also scanned, created the null
> and renamed it to my value.  Now you try to create the
> null, succeed, and then rename.  We won't know which of
> the two non-null values are valid, but worse yet, I think
> one of them should have failed in the first place.



> Sounds like we would need some form of locking around
> here.  Is your goal "no locks", or "less locks"?
(answered below)

> > I don't know how this new scheme could be made to work
> > with the current scheme,...
> 
> It is much more important to know if/why yours is better
> than the current scheme in the first place.  

The goal is: "no locks which do not have a clearly defined 
reliable recovery procedure".

Stale locks without a reliable recovery procedure will lead 
to stolen locks.  At this point it is only a matter of luck 
whether this leads to data loss or not.  So I do hope to 
convince people first that the current scheme is bad, not that 
my scheme is better!  My scheme was proposed to get people 
thinking that we may not have to use locks to get reliable 
updates.


> Without an
> analysis on how the new scheme interacts with the packed
> refs and gives better behaviour, that is kinda difficult.

Fair enough. I will attempt this if the basic idea seems at 
least sane?  I do hope that eventually the packed-refs piece 
and its locking will be reconsidered also; as Michael pointed 
out it has issues already.  So, I am hoping to get people 
thinking more about lockless approaches to all the pieces. I 
think I have some solutions to some of the other pieces also, 
but I don't want to overwhelm the discussion all at once 
(especially if my first piece is shown to be flawed, or if no 
one has any interest in eliminating the current locks?)

 
> I think transition plans can wait until that is done.  If
> it is not even marginally better, we do not have to worry
> about transitioning at all.  If it is only marginally
> better, the transition has to be designed to be no impact
> to the existing repositories.  If it is vastly better, we
> might be able to afford a flag day.

OK, makes sense, I jumped the gun a bit,

-Martin