Re: Can we change the S_DAX flag immediately on XFS without dropping caches?

["Li, Hao" <lihao2018.fnst@cn.fujitsu.com>]

On 2020/8/8 1:09, Ira Weiny wrote:
> On Fri, Jul 31, 2020 at 06:59:32PM +0900, Yasunori Goto wrote:
>> On 2020/07/30 8:21, Dave Chinner wrote:
>>> On Wed, Jul 29, 2020 at 11:23:21AM +0900, Yasunori Goto wrote:
>>>> Hi,
>>>>
>>>> On 2020/07/28 11:20, Dave Chinner wrote:
>>>>> On Tue, Jul 28, 2020 at 02:00:08AM +0000, Li, Hao wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I have noticed that we have to drop caches to make the changing of S_DAX
>>>>>> flag take effect after using chattr +x to turn on DAX for a existing
>>>>>> regular file. The related function is xfs_diflags_to_iflags, whose
>>>>>> second parameter determines whether we should set S_DAX immediately.
>>>>> Yup, as documented in Documentation/filesystems/dax.txt. Specifically:
>>>>>
>>>>>    6. When changing the S_DAX policy via toggling the persistent FS_XFLAG_DAX flag,
>>>>>       the change in behaviour for existing regular files may not occur
>>>>>       immediately.  If the change must take effect immediately, the administrator
>>>>>       needs to:
>>>>>
>>>>>       a) stop the application so there are no active references to the data set
>>>>>          the policy change will affect
>>>>>
>>>>>       b) evict the data set from kernel caches so it will be re-instantiated when
>>>>>          the application is restarted. This can be achieved by:
>>>>>
>>>>>          i. drop-caches
>>>>>          ii. a filesystem unmount and mount cycle
>>>>>          iii. a system reboot
>>>>>
>>>>>> I can't figure out why we do this. Is this because the page caches in
>>>>>> address_space->i_pages are hard to deal with?
>>>>> Because of unfixable races in the page fault path that prevent
>>>>> changing the caching behaviour of the inode while concurrent access
>>>>> is possible. The only way to guarantee races can't happen is to
>>>>> cycle the inode out of cache.
>>>> I understand why the drop_cache operation is necessary. Thanks.
>>>>
>>>> BTW, even normal user becomes to able to change DAX flag for an inode,
>>>> drop_cache operation still requires root permission, right?
>>> Step back for a minute and explain why you want to be able to change
>>> the DAX mode of a file -as a user-.
>>
>> For example, there are 2 containers executed in a system, which is named as
>> container A and container B, and these host gives FS-DAX files to each
>> containers.
>> If the user of container A would like to change DAX-off for tuning, then he
>> will stop his application
>> and change DAX flag, but the flag may not be changed.
>>
>> Then he will "need" to ask host operator to execute drop_cache, and the
>> operator did it.
>> As a result, not only container A, but also container B get the impact of
>> drop_cache.
>>
>> Especially, if this is multi tenant container system, then I think this is
>> not acceptable.
>>
>> Probably, there are 2 problems I think.
>> 1) drop_cache requires root permission.
>> 2) drop_cache has too wide effect.
>>
>>>> So, if kernel have a feature for normal user can operate drop cache for "a
>>>> inode" with
>>>> its permission, I think it improve the above limitation, and
>>>> we would like to try to implement it recently.
>>> No, drop_caches is not going to be made available to users. That
>>> makes it s trivial system wide DoS vector.
>> The current drop_cache feature tries to drop ALL of cache (page cache and/or
>> slab cache).
>> Then, I agree that normal user should not drop all of them.
>>
>> But my intention was that drop cache of ONE file which is changed dax flag,
>> (and if possible, drop only the inode cache.)
>> Do you mean it will be still cause of weakness against DoS attack?
>> If so, I should give up to solve problem 1) at least.
> FWIW changing the on disk flag automatically flags the inode to be dropped as
> soon as all references are done.
>
> See:
>
> 2c567af418e3 fs: Introduce DCACHE_DONTCACHE
> dae2f8ed7992 fs: Lift XFS_IDONTCACHE to the VFS layer
Hi,

I find that DCACHE_DONTCACHE doesn't work well.
If DCACHE_REFERENCED is not set, dput() can drop the inode successfully as
soon as all references are gone. By contrast, if DCACHE_REFERENCED is set,
dput() only decreases the reference count of dentry and don't evict inode.

Example 1:

echo abcdefg > test.txt
echo 3 > /proc/sys/vm/drop_caches
xfs_io -c 'chattr +x' test.txt

In this example, we can say the DAX policy takes effects immediately as we
don't need to drop cache after chattr.
In this circumstance, DCACHE_REFERENCED is not set, and DCACHE_DONTCACHE
can drop the inode as expected.

Example 2:

echo abcdefg > test.txt
xfs_io -c 'chattr +x' test.txt

In this example, we must drop caches after chattr to make DAX policy
take effects. This is because DCACHE_REFERENCED is set, and fast_dput() will
return true, and then retain_dentry() have no chance to check DCACHE_DONTCACHE.

If this is the desired behavior, I can't understand the necessity
of DCACHE_DONTCACHE.

Regards,
Hao Li
>
> But from a users perspective you just don't know when that will happen.  The
> system just can't guarantee it.  The best the user can do is stop taking
> references to the file and close all references, and periodically check the
> state.  But this will take a reference so...  Kind of a catch-22 here...  :-(
>
> Ira
>
>>
>> Thanks,
>>
>>> Cheers,
>>>
>>> Dave.
>> -- 
>> Yasunori Goto
>>
>

_______________________________________________
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-leave@lists.01.org