[Problem]NFS Server – Umount results in Device Busy.

[Problem]NFS Server – Umount results in Device Busy.

[AMIT SAHRAWAT]

TGludXggS2VybmVsOiAyLjYuMzUuMTQNCiANCldlIGFyZSB1c2luZyBORlMgZm9yIGRhdGEgdHJh
bnNmZXIgZnJvbSBvbmUgdGFyZ2V0IHRvIGFub3RoZXIgdGFyZ2V0Lg0KT24gU2VydmVyIHdlIGFy
ZSB1c2luZyAzIG1vdW50IHBvaW50cyCWIHRvIGJlIHVzZWQgYnkgYXBwbGljYXRpb24uIEFuZCB0
aGV5IGFyZSBtZW50aW9uZWQgaW4gL2V0Yy9leHBvcnRzDQogDQpXaGVuIGluaXRpYWxseSBqdXN0
IE5GUyBTZXJ2ZXIgaXMgc3RhcnRlZCAtV2UgY2FuIG1vdW50IGFuZCBzYWZlbHkgdW4tbW91bnQg
dGhlIE5GUyBtb3VudCBwb2ludC4NCk5vdywgd2hlbiBORlMgY2xpZW50IGlzIHN0YXJ0ZWQgYW5k
IHdlIG1vdW50IGF0IHRoZSBjbGllbnQuIFdpdGhvdXQgZG9pbmcgYW55IElPIJYgaWYgd2UgZG8g
dW4tbW91bnQgYXQgdGhlIE5GUyBDbGllbnQglml0IHVuLW1vdW50cyBzYWZlbHkuDQpCdXQgliB3
aGVuIHRyaWVkIHRvIHVuLW1vdW50IHRoZSBkZXZpY2UgYXQgdGhlIE5GUyBzZXJ2ZXIgliBpdCBy
ZXN1bHRzIGluIGVycm9yIJNEZXZpY2UgQnVzeZQNCpN1bW91bnQ6IGNhbid0IHVtb3VudCAvbW50
OiBEZXZpY2Ugb3IgcmVzb3VyY2UgYnVzeZQNCiANCk5vdyAsIGluIG9yZGVyIHRvIHVtb3VudCB0
aGUgZGV2aWNlIC0gIHdlIGhhdmUgdG8ga2lsbCB0aGUgcHJvY2VzcyCRbmZzZJIgliBhbmQgdGhl
biBpdCB1bi1tb3VudHMgZWFzaWx5Lg0KIA0KQXMgcGVyIG91ciB1bmRlcnN0YW5kaW5nIJYgaXQg
c2hvdWxkIG5vdCBiZSBhbiBpc3N1ZSB3aXRoIHRoZSB1bi1tb3VudCB3aGVuIGl0IGlzIG5vdCBi
ZWluZyB1c2VkIGFuZCBhbHNvIHRoZXJlIHNob3VsZCBiZSBubyBkZXBlbmRlbmN5IHRvIGtpbGwg
kW5mc2SSIHRvIHVuLW1vdW50IHRoZSBkZXZpY2UuDQogDQpJdCBsZWFkcyB0byBwcm9ibGVtIG9m
IGtpbGxpbmcgkW5mc2SSIGFuZCByZXN0YXJ0aW5nIGl0IJYgdG8gbWFrZSBvdGhlciBtb3VudCBw
b2ludHMgd29yayB3aXRob3V0IGlzc3VlLg0KIA0KSXMgdGhlcmUgYW55IG1ldGhvZCB0byB1bi1t
b3VudCB0aGUgZGV2aWNlIHdpdGhvdXQgZ2V0dGluZyB0aGlzIJFCVVNZkiBlcnJvcj8gT3IgY2Fu
IHdlIGtpbGwgb25seSBuZnNkIHNlcnZpY2Ugd2hpY2ggd2FzIHVzaW5nIHRoYXQgbW91bnQgcG9p
bnQ/DQpQbGVhc2Ugc2hhcmUgeW91ciBpbnB1dCBvbiB0aGUgc29sdXRpb24gdG8gdGhlIGFib3Zl
IHByb2JsZW0uIElzIHRoaXMgZXhwZWN0ZWQgYmVoYXZpb3I/DQogDQpUaGFua3MgJiBSZWdhcmRz
LA0KQW1pdCBTYWhyYXdhdA==

Re: [Problem]NFS Server – Umount results in Device Busy.

[J. Bruce Fields]

On Thu, May 10, 2012 at 06:59:32AM +0000, AMIT SAHRAWAT wrote:
> Linux Kernel: 2.6.35.14
>  
> We are using NFS for data transfer from one target to another target.
> On Server we are using 3 mount points – to be used by application. And they are mentioned in /etc/exports
>  
> When initially just NFS Server is started -We can mount and safely un-mount the NFS mount point.
> Now, when NFS client is started and we mount at the client. Without doing any IO – if we do un-mount at the NFS Client –it un-mounts safely.
> But – when tried to un-mount the device at the NFS server – it results in error “Device Busy”
> “umount: can't umount /mnt: Device or resource busy”
>  
> Now , in order to umount the device -  we have to kill the process ‘nfsd’ – and then it un-mounts easily.
>  
> As per our understanding – it should not be an issue with the un-mount when it is not being used and also there should be no dependency to kill ‘nfsd’ to un-mount the device.
>  
> It leads to problem of killing ‘nfsd’ and restarting it – to make other mount points work without issue.
>  
> Is there any method to un-mount the device without getting this ‘BUSY’ error? Or can we kill only nfsd service which was using that mount point?

If all the clients unmount, and then you run "exportfs -f" on the
server, you'll probably find you can then unmount the export.

But stopping nfsd is the most reliable way to ensure you can unmount the
exported filesystems.  I'd recommend doing that using your distro's nfs
scripts (so "service nfs stop" or "systemctl nfs-server.service stop" or
equivalent) rather than killing nfsd yourself.

--b.

> Please share your input on the solution to the above problem. Is this expected behavior?

Re: Re: Re: [Problem]NFS Server – Umount results in Device Busy.

[Namjae Jeon]

Hi Bruce, Trond.

As you know, Currently umount results in busy on NFS server although
user tried to succeed to umount on NFS client.
I suggest to add umount procedure to avoid umount busy issue.
When calling umount on NFS client, The resources(exportfs entries
cache) of mount point will be flushed on NFS server. and umount will
be succeed without busy issue.

how do you think about this suggestion ?

Thanks.

2012/5/11, AMIT SAHRAWAT <a.sahrawat@samsung.com>:
> Hi Bruce,
> Sorry it was the mail composer - I did not check mark the plain text
> option.
>
> We need an opinion.
> We will be using around 15 NFS client mounted - with one NFS Server. We are
> try to make out how many NFSD should we start at the NFS Server.
> It all depends upon the load - these 15 NFS clients are going to generate.
> But is there any default number - we should use to start with? like 1 per
> NFS client mount?
> Please share your advise on this.
>
> Thanks & Regards,
> Amit Sahrawat
>
> ------- Original Message -------
> Sender : J. Bruce Fields<bfields@fieldses.org>
> Date : May 10, 2012 17:46 (GMT+05:30)
> Title : Re: Re: [Problem]NFS Server – Um?ount results in Device Busy.
>
> On Thu, May 10, 2012 at 11:54:43AM +0000, AMIT SAHRAWAT wrote:
>>
>>
> Hi Bruce,
>
>>
>
>
>>
> Thanks for instant reply.
>
>
> Could you turn off html formatting on your mail?
>
>>
> The option you mentioned is working. 'exportfs -f'
>
>>
> Actually it is not mentioned anywhere - even when checked for options:
>
>>
> #> exportfs -h
> exportfs: invalid option -- 'h'
> usage: exportfs [-aruv] [host:/path]
>
>
>
> Sorry, it looks like the help test could be improved!  Patches
> welcomed....
>
> It is documented in the exportfs man page.
>
>>
> But - when tried to give - '-f' option it is working.
>
>>
> Then on checking the code for exportfs - got to know the working of '-f'
> option.
>
>>
> I think it should work. I will check with a number of scenario  - to check
> there is no blockage for other mount points - at the time 'exportfs -f' is
> being run. Please reply if it wont :)
>
>
> If no clients are using the export, it should work.
>
> --b.
>
>
>
> Thanks & Regards,
> Amit Sahrawat

Re: [Problem]NFS Server – Umount results in Device Busy.

[Boaz Harrosh]

On 06/07/2012 10:47 AM, Namjae Jeon wrote:

> Hi Bruce, Trond.
> 
> As you know, Currently umount results in busy on NFS server although
> user tried to succeed to umount on NFS client.
> I suggest to add umount procedure to avoid umount busy issue.
> When calling umount on NFS client, The resources(exportfs entries
> cache) of mount point will be flushed on NFS server. and umount will
> be succeed without busy issue.
> 
> how do you think about this suggestion ?
> 


I second this request, what is needed so when all clients unmounted,
the system comes back to the sate it was before any clients have
mounted. i.e filesystems is not referenced and may unmount cleanly.
(This also happens with >= 4.0 clients only, so no excuses)

This is a real problem for me, on Fedora machines. Because I export
iscsi devices which are network devices, and in the shutdown procedure
for some reason the "service nfs stop" of the server is much much to
late. the original umount of exofs (-o _netdev) fails because it's
held by NFSD, the iscsi devices go away regardless, and when nfsd
finally releases exofs, it gets deadlocked on some error handling.
OK I know I must fix the stuck-ness, but the problem will remain.
The FS will not unmount cleanly because it will only attempt
an unmount after its devices are gone. This will be solved if
nfsd would release its hold on the FS when all clients are gone.

It was on my TODO to fix this for a long time, but I seem to be
too busy with more urgent matters. (What's the point of fixing the
shutdown if the steady state doesn't work yet)

If someone has investigated the matter and knows what to do I would
appreciate any insights, and/or patches would be wonderful ;-)

> Thanks.
> 

<>

Thanks
Boaz

Re: [Problem]NFS Server – Umount results in Device Busy.

[Namjae Jeon]

2012/6/7, Boaz Harrosh <bharrosh@panasas.com>:
> On 06/07/2012 10:47 AM, Namjae Jeon wrote:
>
>> Hi Bruce, Trond.
>>
>> As you know, Currently umount results in busy on NFS server although
>> user tried to succeed to umount on NFS client.
>> I suggest to add umount procedure to avoid umount busy issue.
>> When calling umount on NFS client, The resources(exportfs entries
>> cache) of mount point will be flushed on NFS server. and umount will
>> be succeed without busy issue.
>>
>> how do you think about this suggestion ?
>>
>
>
> I second this request, what is needed so when all clients unmounted,
> the system comes back to the sate it was before any clients have
> mounted. i.e filesystems is not referenced and may unmount cleanly.
> (This also happens with >= 4.0 clients only, so no excuses)
>
> This is a real problem for me, on Fedora machines. Because I export
> iscsi devices which are network devices, and in the shutdown procedure
> for some reason the "service nfs stop" of the server is much much to
> late. the original umount of exofs (-o _netdev) fails because it's
> held by NFSD, the iscsi devices go away regardless, and when nfsd
> finally releases exofs, it gets deadlocked on some error handling.
> OK I know I must fix the stuck-ness, but the problem will remain.
> The FS will not unmount cleanly because it will only attempt
> an unmount after its devices are gone. This will be solved if
> nfsd would release its hold on the FS when all clients are gone.
>
> It was on my TODO to fix this for a long time, but I seem to be
> too busy with more urgent matters. (What's the point of fixing the
> shutdown if the steady state doesn't work yet)
>
> If someone has investigated the matter and knows what to do I would
> appreciate any insights, and/or patches would be wonderful ;-)
Hi Boaz.
Oh.. You also are facing same problem.
Actually I almost finish to imprement the patch about umount busy
issue on NFS server. But I want to know Maintainer's opinion before
posting.
Thanks.
>
>> Thanks.
>>
>
> <>
>
> Thanks
> Boaz
>
>

Re: [Problem]NFS Server – Umount results in Device Busy.

[Boaz Harrosh]

On 06/07/2012 12:43 PM, Namjae Jeon wrote:

> 2012/6/7, Boaz Harrosh <bharrosh@panasas.com>:
>> On 06/07/2012 10:47 AM, Namjae Jeon wrote:
>>
>>> Hi Bruce, Trond.
>>>
>>> As you know, Currently umount results in busy on NFS server although
>>> user tried to succeed to umount on NFS client.
>>> I suggest to add umount procedure to avoid umount busy issue.
>>> When calling umount on NFS client, The resources(exportfs entries
>>> cache) of mount point will be flushed on NFS server. and umount will
>>> be succeed without busy issue.
>>>
>>> how do you think about this suggestion ?
>>>
>>
>>
>> I second this request, what is needed so when all clients unmounted,
>> the system comes back to the sate it was before any clients have
>> mounted. i.e filesystems is not referenced and may unmount cleanly.
>> (This also happens with >= 4.0 clients only, so no excuses)
>>
>> This is a real problem for me, on Fedora machines. Because I export
>> iscsi devices which are network devices, and in the shutdown procedure
>> for some reason the "service nfs stop" of the server is much much to
>> late. the original umount of exofs (-o _netdev) fails because it's
>> held by NFSD, the iscsi devices go away regardless, and when nfsd
>> finally releases exofs, it gets deadlocked on some error handling.
>> OK I know I must fix the stuck-ness, but the problem will remain.
>> The FS will not unmount cleanly because it will only attempt
>> an unmount after its devices are gone. This will be solved if
>> nfsd would release its hold on the FS when all clients are gone.
>>
>> It was on my TODO to fix this for a long time, but I seem to be
>> too busy with more urgent matters. (What's the point of fixing the
>> shutdown if the steady state doesn't work yet)
>>
>> If someone has investigated the matter and knows what to do I would
>> appreciate any insights, and/or patches would be wonderful ;-)
> Hi Boaz.
> Oh.. You also are facing same problem.
> Actually I almost finish to imprement the patch about umount busy
> issue on NFS server. But I want to know Maintainer's opinion before
> posting.
> Thanks.


Please post it (maybe as RFC) I want to see it

Thanks
Boaz

>>
>>> Thanks.
>>>
>>
>> <>
>>
>> Thanks
>> Boaz
>>
>>

Re: [Problem]NFS Server – Umount results in Device Busy.

[Namjae Jeon]

2012/6/7, Boaz Harrosh <bharrosh@panasas.com>:
> On 06/07/2012 12:43 PM, Namjae Jeon wrote:
>
>> 2012/6/7, Boaz Harrosh <bharrosh@panasas.com>:
>>> On 06/07/2012 10:47 AM, Namjae Jeon wrote:
>>>
>>>> Hi Bruce, Trond.
>>>>
>>>> As you know, Currently umount results in busy on NFS server although
>>>> user tried to succeed to umount on NFS client.
>>>> I suggest to add umount procedure to avoid umount busy issue.
>>>> When calling umount on NFS client, The resources(exportfs entries
>>>> cache) of mount point will be flushed on NFS server. and umount will
>>>> be succeed without busy issue.
>>>>
>>>> how do you think about this suggestion ?
>>>>
>>>
>>>
>>> I second this request, what is needed so when all clients unmounted,
>>> the system comes back to the sate it was before any clients have
>>> mounted. i.e filesystems is not referenced and may unmount cleanly.
>>> (This also happens with >= 4.0 clients only, so no excuses)
>>>
>>> This is a real problem for me, on Fedora machines. Because I export
>>> iscsi devices which are network devices, and in the shutdown procedure
>>> for some reason the "service nfs stop" of the server is much much to
>>> late. the original umount of exofs (-o _netdev) fails because it's
>>> held by NFSD, the iscsi devices go away regardless, and when nfsd
>>> finally releases exofs, it gets deadlocked on some error handling.
>>> OK I know I must fix the stuck-ness, but the problem will remain.
>>> The FS will not unmount cleanly because it will only attempt
>>> an unmount after its devices are gone. This will be solved if
>>> nfsd would release its hold on the FS when all clients are gone.
>>>
>>> It was on my TODO to fix this for a long time, but I seem to be
>>> too busy with more urgent matters. (What's the point of fixing the
>>> shutdown if the steady state doesn't work yet)
>>>
>>> If someone has investigated the matter and knows what to do I would
>>> appreciate any insights, and/or patches would be wonderful ;-)
>> Hi Boaz.
>> Oh.. You also are facing same problem.
>> Actually I almost finish to imprement the patch about umount busy
>> issue on NFS server. But I want to know Maintainer's opinion before
>> posting.
>> Thanks.
>
>
> Please post it (maybe as RFC) I want to see it
Okay, I will soon.
>
> Thanks
> Boaz
>
>>>
>>>> Thanks.
>>>>
>>>
>>> <>
>>>
>>> Thanks
>>> Boaz
>>>
>>>
>
>
>

Re: [Problem]NFS Server – Umount results in Device Busy.

[J. Bruce Fields]

On Thu, Jun 07, 2012 at 11:13:04AM +0300, Boaz Harrosh wrote:
> On 06/07/2012 10:47 AM, Namjae Jeon wrote:
> > As you know, Currently umount results in busy on NFS server although
> > user tried to succeed to umount on NFS client.
> > I suggest to add umount procedure to avoid umount busy issue.
> > When calling umount on NFS client, The resources(exportfs entries
> > cache) of mount point will be flushed on NFS server. and umount will
> > be succeed without busy issue.
> > 
> > how do you think about this suggestion ?

I'm not sure how adding an "unmount" rpc to the protocol would really
help here, if that's what you're asking for.

If you're just looking for a command you could run on the server after
all the clients have unmounted--"exportfs -f" or "service stop nfsd" (or
equivalent) should do the job.

> I second this request, what is needed so when all clients unmounted,
> the system comes back to the sate it was before any clients have
> mounted. i.e filesystems is not referenced and may unmount cleanly.
> (This also happens with >= 4.0 clients only, so no excuses)
> 
> This is a real problem for me, on Fedora machines. Because I export
> iscsi devices which are network devices, and in the shutdown procedure
> for some reason the "service nfs stop" of the server is much much to
> late.

Sounds like a bug in the Fedora systemd configuration?

> the original umount of exofs (-o _netdev) fails because it's
> held by NFSD, the iscsi devices go away regardless, and when nfsd
> finally releases exofs, it gets deadlocked on some error handling.
> OK I know I must fix the stuck-ness, but the problem will remain.
> The FS will not unmount cleanly because it will only attempt
> an unmount after its devices are gone. This will be solved if
> nfsd would release its hold on the FS when all clients are gone.

Note nfsd doesn't really know when that is.  Even with NFSv4, processes
can be using the filesystem without holding state on the server: they
might just have a current working directory in the filesystem, or have a
device special file open.

> It was on my TODO to fix this for a long time, but I seem to be
> too busy with more urgent matters. (What's the point of fixing the
> shutdown if the steady state doesn't work yet)
> 
> If someone has investigated the matter and knows what to do I would
> appreciate any insights, and/or patches would be wonderful ;-)

My first concern would be to fix any ordering bugs in the systemd
configuration, or any reference count leaks.

--b.

Re: [Problem]NFS Server – Umount results in Device Busy.

[Boaz Harrosh]

On 06/07/2012 02:33 PM, J. Bruce Fields wrote:

> On Thu, Jun 07, 2012 at 11:13:04AM +0300, Boaz Harrosh wrote:


> 
> I'm not sure how adding an "unmount" rpc to the protocol would really
> help here, if that's what you're asking for.
> 


Please elaborate a bit. I'm not familiar with the details of this
discussion.

> If you're just looking for a command you could run on the server after
> all the clients have unmounted--"exportfs -f" or "service stop nfsd" (or
> equivalent) should do the job.
> 

man exportfs
       -f     In 'new' mode, flush everything out of the kernels export table.
              Any clients that are active will get new entries added by mountd
              when they make their next request.

What if: (Just fantasizing here, if I understand the above text correctly)
* NFS4: A bit like expire client, as long as client has state on the server
        he needs to refresh his leases so not to expire with some kind of
        activity or renewals.
	If a client has no state and 2*expiry timeout passed then it is
	individually "exportfs -f". This is safe because on renewed activity
	it will get it's mountd entries renewed safely (did I understand
	this correctly?

* NFS3: Same as above But with say X*expiry timeout after last activity
	Same safeness as above.

Any "unmount rpc" will expedite the above without waiting for timeout

<>

> 
> Note nfsd doesn't really know when that is.  Even with NFSv4, processes
> can be using the filesystem without holding state on the server: they
> might just have a current working directory in the filesystem, or have a
> device special file open.
> 


How does the magic of above statement works? quote:
    "will get new entries added by mountd when they make their next request"

<>

> 
> My first concern would be to fix any ordering bugs in the systemd
> configuration, or any reference count leaks.
> 


I'm currently shooting in the dark, I have not investigated this at all,
do to lack of time. It should be the same with an ext4 over iscsi device
re-exported by NFS. You must mount the ext4 as "-o _netdev" otherwise
ext4 mount itself will have a problem.

> --b.


Thanks Bruce, we'll keep this on a back burner until we can come back to
it. A TODO.

But Jeon, please do send in your proposal code, I'd like to see it.

Thanks
Boaz

Re: [Problem]NFS Server – Umount results in Device Busy.

[Boaz Harrosh]

On 06/08/2012 12:59 AM, Boaz Harrosh wrote:
<>

> 
>> If you're just looking for a command you could run on the server after
>> all the clients have unmounted--"exportfs -f" 


BTW: just that one does the trick, thanks learn something new

>> or "service stop nfsd" (or equivalent) should do the job.
>>


That's the one I used to do manually because of the wrong shut down
order in Fedora.

Thanks
Boaz

Re: [Problem]NFS Server – Umount results in Device Busy.

[Namjae Jeon]

>
> But Jeon, please do send in your proposal code, I'd like to see it.
Hi. Boaz.
Okay, I will try to post the patch after testing a little more.
Thanks.
>
> Thanks
> Boaz
>

Re: [Problem]NFS Server – Umount results in Device Busy.

[J. Bruce Fields]

On Fri, Jun 08, 2012 at 12:59:10AM +0300, Boaz Harrosh wrote:
> On 06/07/2012 02:33 PM, J. Bruce Fields wrote:
> 
> > On Thu, Jun 07, 2012 at 11:13:04AM +0300, Boaz Harrosh wrote:
> 
> 
> > 
> > I'm not sure how adding an "unmount" rpc to the protocol would really
> > help here, if that's what you're asking for.
> > 
> 
> 
> Please elaborate a bit. I'm not familiar with the details of this
> discussion.

The mount protocol used with NFSv2/v3, for example, has mount/unmount
calls, and I seem to recall problems with keeping the list of mounted
clients accurate in the face of unreliable clients and networks.

I guess you could add the same but make them subject to the lease, so a
client would be assumed "unmounted" if it lost its lease.  I don't know.

But the reported problems here sound like just simple bugs, so I'm not
interested in such discussion till the bugs are investigated and fixed
and someone explains what problem then remains to be solved.

--b.

Re: [Problem]NFS Server – Umount results in Device Busy.

[Namjae Jeon]

2012/6/7, J. Bruce Fields <bfields@fieldses.org>:
> On Thu, Jun 07, 2012 at 11:13:04AM +0300, Boaz Harrosh wrote:
>> On 06/07/2012 10:47 AM, Namjae Jeon wrote:
>> > As you know, Currently umount results in busy on NFS server although
>> > user tried to succeed to umount on NFS client.
>> > I suggest to add umount procedure to avoid umount busy issue.
>> > When calling umount on NFS client, The resources(exportfs entries
>> > cache) of mount point will be flushed on NFS server. and umount will
>> > be succeed without busy issue.
>> >
>> > how do you think about this suggestion ?
>
> I'm not sure how adding an "unmount" rpc to the protocol would really
> help here, if that's what you're asking for.

Since, at the NFS server it is just a normal device which is mounted.
So, After NFS client operation is over and it has successfully
unmounted. Then at the Server side the umount should be transparent.
As suppose if the user does
umount /mnt -> he might not have knowlege that is NFS mounted and
could think this to be issue.
  So - if we should have a transparent mechanism for umount at NFS server also
  NFS Server
mount /dev/sda1 /mnt
NFs Client :
mount -t nfs <IP>: /mnt /mnt
umount /mnt
After this if a user wants to disconnect USB device - there should be no problem

>
> If you're just looking for a command you could run on the server after
> all the clients have unmounted--"exportfs -f" or "service stop nfsd" (or
> equivalent) should do the job.
>
>> I second this request, what is needed so when all clients unmounted,
>> the system comes back to the sate it was before any clients have
>> mounted. i.e filesystems is not referenced and may unmount cleanly.
>> (This also happens with >= 4.0 clients only, so no excuses)
>>
>> This is a real problem for me, on Fedora machines. Because I export
>> iscsi devices which are network devices, and in the shutdown procedure
>> for some reason the "service nfs stop" of the server is much much to
>> late.
>
> Sounds like a bug in the Fedora systemd configuration?
>
>> the original umount of exofs (-o _netdev) fails because it's
>> held by NFSD, the iscsi devices go away regardless, and when nfsd
>> finally releases exofs, it gets deadlocked on some error handling.
>> OK I know I must fix the stuck-ness, but the problem will remain.
>> The FS will not unmount cleanly because it will only attempt
>> an unmount after its devices are gone. This will be solved if
>> nfsd would release its hold on the FS when all clients are gone.
>
> Note nfsd doesn't really know when that is.  Even with NFSv4, processes
> can be using the filesystem without holding state on the server: they
> might just have a current working directory in the filesystem, or have a
> device special file open.
>
>> It was on my TODO to fix this for a long time, but I seem to be
>> too busy with more urgent matters. (What's the point of fixing the
>> shutdown if the steady state doesn't work yet)
>>
>> If someone has investigated the matter and knows what to do I would
>> appreciate any insights, and/or patches would be wonderful ;-)
>
> My first concern would be to fix any ordering bugs in the systemd
> configuration, or any reference count leaks.
>
> --b.
>