Re: grace period

[Bruce Fields <bfields@fieldses.org>]

On Fri, Jul 01, 2016 at 01:24:48PM -0700, Marc Eshel wrote:
> linux-nfs-owner@vger.kernel.org wrote on 07/01/2016 01:07:42 PM:
> 
> > From: Bruce Fields <bfields@fieldses.org>
> > To: Marc Eshel/Almaden/IBM@IBMUS
> > Cc: linux-nfs@vger.kernel.org
> > Date: 07/01/2016 01:07 PM
> > Subject: Re: grace period
> > Sent by: linux-nfs-owner@vger.kernel.org
> > 
> > On Fri, Jul 01, 2016 at 10:31:55AM -0700, Marc Eshel wrote:
> > > It used to be that sending KILL signal to lockd would free locks and 
> start 
> > > Grace period, and when setting nfsd threads to zero, 
> nfsd_last_thread() 
> > > calls nfsd_shutdown that called lockd_down that I believe was causing 
> both 
> > > freeing of locks and starting grace period or maybe it was setting it 
> back 
> > > to a value > 0 that started the grace period.
> > 
> > OK, apologies, I didn't know (or forgot) that.
> > 
> > > Any way starting with the kernels that are in RHEL7.1 and up echo 0 > 
> > > /proc/fs/nfsd/threads doesn't do it anymore, I assume going to common 
> > > grace period for NLM and NFSv4 changed things.
> > > The question is how to do IP fail-over, so when a node fails and the 
> IP is 
> > > moving to another node, we need to go into grace period on all the 
> nodes 
> > > in the cluster so the locks of the failed node are not given to anyone 
> 
> > > other than the client that is reclaiming his locks. Restarting NFS 
> server 
> > > is to distractive.
> > 
> > What's the difference?  Just that clients don't have to reestablish tcp
> > connections?
> 
> I am not sure what else systemctl will do but I need to control the order 
> of the restart so the client will not see any errors.
> I don't think that echo 0 > /proc/fs/nfsd/threads is freeing the lock, at 
> least not the v3 locks, I will try again with v4.
> The question is what is the most basic operation that can be done to start 
> grace, will echo 8 > /proc/fs/nfsd/threads following echo 0 do it?
> or is there any other primitive that will do it?

That should do it, though really so should just "systemctl restart
nfs-server"--if that causes errors then there's a bug somewhere.

--b.

> Marc.
> 
> > 
> > --b.
> > 
> > > For NFSv3 KILL signal to lockd still works but for 
> > > NFSv4 have no way to do it for v4.
> > > Marc. 
> > > 
> > > 
> > > 
> > > From:   Bruce Fields <bfields@fieldses.org>
> > > To:     Marc Eshel/Almaden/IBM@IBMUS
> > > Cc:     linux-nfs@vger.kernel.org
> > > Date:   07/01/2016 09:09 AM
> > > Subject:        Re: grace period
> > > 
> > > 
> > > 
> > > On Thu, Jun 30, 2016 at 02:46:19PM -0700, Marc Eshel wrote:
> > > > I see that setting the number of nfsd threads to 0 (echo 0 > 
> > > > /proc/fs/nfsd/threads) is not releasing the locks and putting the 
> server 
> > > 
> > > > in grace mode.
> > > 
> > > Writing 0 to /proc/fs/nfsd/threads shuts down knfsd.  So it should
> > > certainly drop locks.  If that's not happening, there's a bug, but 
> we'd
> > > need to know more details (version numbers, etc.) to help.
> > > 
> > > That alone has never been enough to start a grace period--you'd have 
> to
> > > start knfsd again to do that.
> > > 
> > > > What is the best way to go into grace period, in new version of the
> > > > kernel, without restarting the nfs server?
> > > 
> > > Restarting the nfs server is the only way.  That's true on older 
> kernels
> > > true, as far as I know.  (OK, you can apparently make lockd do 
> something
> > > like this with a signal, I don't know if that's used much, and I doubt
> > > it works outside an NFSv3-only environment.)
> > > 
> > > So if you want locks dropped and a new grace period, then you should 
> run
> > > "systemctl restart nfs-server", or your distro's equivalent.
> > > 
> > > But you're probably doing something more complicated than that.  I'm 
> not
> > > sure I understand the question....
> > > 
> > > --b.
> > > 
> > > 
> > > 
> > > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
>