Re: help with understanding evict inode functionality

Re: help with understanding evict inode functionality

[Leonid Fedorenchik]

On Thu, 1 Oct 2015 22:54:08 +0000
"Kornievskaia, Olga" <Olga.Kornievskaia@netapp.com> wrote:

> [...]
>
> In NFS development, we’ve been seeing a race between reopening of the same file and evict inode code and unable figure out how to prevent such race. When file is opened for the first timed and then closed. As the last reference is dropped and iput_final() calls evict() which will call filesystem specific evict_inode() code. As that’s happening a new open for the same file is happening and causes problems.
Recently we encountered similar situation with NFS, that is, after
evict_inode() & iget_locked() we were getting the same inode (same
i->i_ino) and without I_NEW bit set.

>
> Also can somebody suggest how to debug VFS code, as putting printks generates output for the local filesystem code as well.
You can call printk() only if current process is "nfsd" (at least,
that is what we did).

>
> Thank you.
>
> [...]

-- 

Best regards,
Leonid Fedorenchik

Software Engineer
Paragon Software Group
Skype: leonid.fedorenchik
http://www.paragon-software.com

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: help with understanding evict inode functionality

[Kornievskaia, Olga]

> On Oct 2, 2015, at 9:57 AM, Leonid Fedorenchik <leonid.fedorenchik@paragon-software.com> wrote:
> 
> On Thu, 1 Oct 2015 22:54:08 +0000
> "Kornievskaia, Olga" <Olga.Kornievskaia@netapp.com> wrote:
> 
>> [...]
>> 
>> In NFS development, we’ve been seeing a race between reopening of the same file and evict inode code and unable figure out how to prevent such race. When file is opened for the first timed and then closed. As the last reference is dropped and iput_final() calls evict() which will call filesystem specific evict_inode() code. As that’s happening a new open for the same file is happening and causes problems.
> Recently we encountered similar situation with NFS, that is, after
> evict_inode() & iget_locked() we were getting the same inode (same
> i->i_ino) and without I_NEW bit set.

Thanks for the comment! When I tried to print inode state flags in the NFS code for opening a file I’ve never had “I_NEW” set. Not sure why.

> 
>> 
>> Also can somebody suggest how to debug VFS code, as putting printks generates output for the local filesystem code as well.
> You can call printk() only if current process is "nfsd" (at least,
> that is what we did).
> 

Thanks for the suggestion. I’ll try that.

Again a general question to the community about inodes and eviction. I was under the general impression that when the file is closed the inode sticks around just in case the file is reopened again. Then when resources are constrainted some background thread evicts unused inodes. Is that understanding correct? In that case what is that background process name? 

>> 
>> Thank you.
>> 
>> [...]
> 
> -- 
> 
> Best regards,
> Leonid Fedorenchik
> 
> Software Engineer
> Paragon Software Group
> Skype: leonid.fedorenchik
> http://www.paragon-software.com
> 

Re: help with understanding evict inode functionality

[Theodore Ts'o]

On Fri, Oct 02, 2015 at 02:38:49PM +0000, Kornievskaia, Olga wrote:
> 
> Again a general question to the community about inodes and
> eviction. I was under the general impression that when the file is
> closed the inode sticks around just in case the file is reopened
> again. Then when resources are constrainted some background thread
> evicts unused inodes. Is that understanding correct?

I think you're confused about what evict_inode() does.  The i_count
field tracks how many in-memory references the inode has.  Each iput()
decrements i_count.  Once i_count goes to zero, there are no more open
file descriptors for that inode.

Then there is also i_nlink.  This is the number of directory entries
which point at the inode on disk.  When you unlink() a file, this
drops the i_nlink on the inode.  When i_nlink drops to zero, then the
inode is no longer referenced by any directory entry.  But, this
doesn't mean we can actually release the inode and the blocks
associated with the file --- Unix semantics is that you can have an
open file descriptor on an unlinked file, and when the last fd is
closed, only then does the inode get released.

evict_inode() is what happens when i_nlink *and* i_icount hits zero.
So it is only then that the local disk file system can actually
release the inode and blocks associated with that inode.

So by the time the file system's evict_inode() is called, the inode is
not coming back.  With apologies to Monty Python, the inode is no
more; it has ceased to be.  It's expired and gone to meet its
maker..... It is an ex-inode.  :-)

Hence, there is no point trying to worry about what hapens if the file
is reopened again, since the original inode is *gone*.  You could
create a new file with the same file name, but none of the resources
associated with the old inode need to be preserved for the newly
created file.

Cheers,

						- Ted

P.S.  Things do get a little complicated with non-native Unix file
systems which don't obey Posix semantics, or if you are dealing with a
pre-NFSv4 protocol which quaintly assumes that you can have a
stateless networked file system protocol when a file system is
inherently stateful.  For more information, see:

http://serverfault.com/questions/267601/can-open-files-be-unlinked-on-nfs-mounted-volumes-while-applications-that-still

Re: help with understanding evict inode functionality

[Christoph Hellwig]

On Sat, Oct 03, 2015 at 10:28:02PM -0400, Theodore Ts'o wrote:
> evict_inode() is what happens when i_nlink *and* i_icount hits zero.
> So it is only then that the local disk file system can actually
> release the inode and blocks associated with that inode.

That's not true.  Evict is called when we want to remove an inode
from memory.  i_count needs to be zero for that, but i_nlink doesn't
as we don't cache inodes forever.  When we drop i_count and it reaches
zero iput_final() is called, which decides depending on ->drop_inode
to either evict the inode or keep it on the LRU.

> Hence, there is no point trying to worry about what hapens if the file
> is reopened again, since the original inode is *gone*.  You could
> create a new file with the same file name, but none of the resources
> associated with the old inode need to be preserved for the newly
> created file.

And this isn't quite true either - this particular struct inode instance
might not come back, but we need to synchronize against a racing iget
if i_nlink wasn't zero.  This is done with the I_FREEING bit and the
bit wake up on __I_NEW.

Re: help with understanding evict inode functionality

[Theodore Ts'o]

On Sat, Oct 03, 2015 at 11:01:36PM -0700, Christoph Hellwig wrote:
> 
> That's not true.  Evict is called when we want to remove an inode
> from memory.  i_count needs to be zero for that, but i_nlink doesn't
> as we don't cache inodes forever.  When we drop i_count and it reaches
> zero iput_final() is called, which decides depending on ->drop_inode
> to either evict the inode or keep it on the LRU.

Sorry, of course Cristoph is absolutely correct.

						- Ted

Re: help with understanding evict inode functionality

[Jan Kara]

On Fri 02-10-15 14:38:49, Kornievskaia, Olga wrote:
> >> 
> >> Also can somebody suggest how to debug VFS code, as putting printks
> >> generates output for the local filesystem code as well.
> > You can call printk() only if current process is "nfsd" (at least,
> > that is what we did).
> 
> Thanks for the suggestion. I’ll try that.
> 
> Again a general question to the community about inodes and eviction. I
> was under the general impression that when the file is closed the inode
> sticks around just in case the file is reopened again. Then when
> resources are constrainted some background thread evicts unused inodes.
> Is that understanding correct? In that case what is that background
> process name? 

Yes, your understanding is correct. Generally, slab caches (and thus unused
inodes) get reclaimed from kswapd processes. It can happen though, that
they are reclaimed basically from any process doing memory allocation when
there is shortage of free memory (this is called direct reclaim).

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

help with understanding evict inode functionality

[Kornievskaia, Olga]

Hi folks,

Can somebody help explain evict inode functionality? Specifically I’m interested in know how it interacts with the opening of the same file again?

In NFS development, we’ve been seeing a race between reopening of the same file and evict inode code and unable figure out how to prevent such race. When file is opened for the first timed and then closed. As the last reference is dropped and iput_final() calls evict() which will call filesystem specific evict_inode() code. As that’s happening a new open for the same file is happening and causes problems.

Also can somebody suggest how to debug VFS code, as putting printks generates output for the local filesystem code as well. 

Thank you.

(I don’t subscribe to the mailing list please CC in the reply).