[GIT PULL] f2fs for 4.8

[GIT PULL] f2fs for 4.8

[Jaegeuk Kim]

Hi Linus,

Could you please consider this pull request?

Thanks,

The following changes since commit 4340fa55298d17049e71c7a34e04647379c269f3:

  Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm (2016-06-02 15:08:06 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git tags/for-f2fs-4.8

for you to fetch changes up to 5302fb000def84100740a84d7f176c0e167b2141:

  f2fs: clean up coding style and redundancy (2016-07-25 12:58:12 -0700)

----------------------------------------------------------------
The major change in this version is mitigating cpu overheads on write paths by
replacing redundant inode page updates with mark_inode_dirty calls. And we tried
to reduce lock contentions as well to improve filesystem scalability.
Other feature is setting F2FS automatically when detecting host-managed SMR.

= Enhancement =
 - ioctl to move a range of data between files
 - inject orphan inode errors
 - avoid flush commands congestion
 - support lazytime

= Bug fixes =
 - return proper results for some dentry operations
 - fix deadlock in add_link failure
 - disable extent_cache for fcollapse/finsert

----------------------------------------------------------------
Chao Yu (9):
      f2fs: fix to avoid reading out encrypted data in page cache
      f2fs: fix to detect truncation prior rather than EIO during read
      f2fs: fix to redirty page if fail to gc data page
      f2fs: add nodiscard mount option
      f2fs: fix incorrect f_bfree calculation in ->statfs
      f2fs: fix to avoid redundant discard during fstrim
      f2fs: fix to avoid data update racing between GC and DIO
      f2fs: reset default idle interval value
      f2fs: fix to report error number of f2fs_find_entry

Jaegeuk Kim (54):
      Revert "f2fs: no need inc dirty pages under inode lock"
      f2fs: use inode pointer for {set, clear}_inode_flag
      f2fs: introduce f2fs_i_size_write with mark_inode_dirty_sync
      f2fs: introduce f2fs_i_blocks_write with mark_inode_dirty_sync
      f2fs: introduce f2fs_i_links_write with mark_inode_dirty_sync
      f2fs: call mark_inode_dirty_sync for i_field changes
      f2fs: flush inode metadata when checkpoint is doing
      f2fs: remove syncing inode page in all the cases
      f2fs: avoid unnecessary updating inode during fsync
      f2fs: add lazytime mount option
      f2fs: detect congestion of flush command issues
      f2fs: set flush_merge by default
      f2fs: remove writepages lock
      f2fs: propagate error given by f2fs_find_entry
      f2fs: inject to produce some orphan inodes
      f2fs: do not skip writing data pages
      f2fs: remove two steps to flush dirty data pages
      f2fs: return error of f2fs_lookup
      f2fs: handle writepage correctly
      f2fs: remove deprecated parameter
      f2fs: avoid wrong count on dirty inodes
      f2fs: remove obsolete parameter in f2fs_truncate
      f2fs: avoid data race between FI_DIRTY_INODE flag and update_inode
      f2fs: fix wrong percentage
      f2fs: control not to exceed # of cached nat entries
      f2fs: set mapping error for EIO
      f2fs: avoid reverse IO order for NODE and DATA
      f2fs: drop any block plugging
      f2fs: skip clean segment for gc
      f2fs: introduce mode=lfs mount option
      f2fs: fix deadlock in add_link failure
      f2fs: report error for f2fs_parent_dir
      f2fs: call update_inode_page for orphan inodes
      f2fs: detect host-managed SMR by feature flag
      f2fs: produce more nids and reduce readahead nats
      f2fs: avoid writing node/metapages during writes
      f2fs: avoid latency-critical readahead of node pages
      f2fs: introduce f2fs_set_page_dirty_nobuffer
      f2fs: call SetPageUptodate if needed
      f2fs: shrink critical region in spin_lock
      f2fs: skip to check the block address of node page
      f2fs: use percpu_rw_semaphore
      f2fs: move i_size_write in f2fs_write_end
      f2fs: avoid mark_inode_dirty
      f2fs: fix ERR_PTR returned by bio
      f2fs: refactor __exchange_data_block for speed up
      f2fs: disable extent_cache for fcollapse/finsert inodes
      f2fs: add maximum prefree segments
      f2fs: use blk_plug in all the possible paths
      f2fs: avoid memory allocation failure due to a long length
      f2fs: support an ioctl to move a range of data blocks
      f2fs: avoid data race when deciding checkpoin in f2fs_sync_file
      f2fs: handle error case with f2fs_bug_on
      f2fs: clean up coding style and redundancy

Sheng Yong (1):
      f2fs: find parent dentry correctly

Tiezhu Yang (1):
      f2fs: remove unnecessary goto statement

Yunlei He (2):
      f2fs: avoid mismatching block range for discard
      f2fs: get victim segment again after new cp

Yunlong Song (1):
      f2fs: return the errno to the caller to avoid using a wrong page

 Documentation/filesystems/f2fs.txt |   7 +-
 fs/f2fs/acl.c                      |   9 +-
 fs/f2fs/acl.h                      |   2 +-
 fs/f2fs/checkpoint.c               |  80 ++++--
 fs/f2fs/data.c                     | 307 ++++++++++++-----------
 fs/f2fs/debug.c                    |   5 +-
 fs/f2fs/dir.c                      | 123 ++++-----
 fs/f2fs/extent_cache.c             |  50 ++--
 fs/f2fs/f2fs.h                     | 286 +++++++++++++++------
 fs/f2fs/file.c                     | 499 +++++++++++++++++++++++++++----------
 fs/f2fs/gc.c                       |  52 +++-
 fs/f2fs/inline.c                   |  92 +++----
 fs/f2fs/inode.c                    |  53 ++--
 fs/f2fs/namei.c                    | 147 ++++++-----
 fs/f2fs/node.c                     | 144 ++++++-----
 fs/f2fs/node.h                     |  14 +-
 fs/f2fs/recovery.c                 |  23 +-
 fs/f2fs/segment.c                  |  59 ++++-
 fs/f2fs/segment.h                  |  22 +-
 fs/f2fs/shrinker.c                 |   5 +-
 fs/f2fs/super.c                    | 134 +++++++++-
 fs/f2fs/xattr.c                    |  20 +-
 22 files changed, 1385 insertions(+), 748 deletions(-)

Re: [GIT PULL] f2fs for 4.8

[Christoph Hellwig]

>  - ioctl to move a range of data between files

Please run this through the linux-api list first.  We had so many
problem with random ioctls on random file systems in the past.  Also
I know Samsung had an xfs (and maybe ext4?) implementation of this
functionality in the past, so it would be extremtly useful to have
some discussion on that.

Also didn't this just show up last week for the first time?

Re: [GIT PULL] f2fs for 4.8

[Linus Torvalds]

On Tue, Jul 26, 2016 at 9:08 PM, Jaegeuk Kim <jaegeuk@kernel.org> wrote:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git tags/for-f2fs-4.8

This got some merge conflicts with the bio interface rework
(bio_set_op_attrs() etc), and while I tried to fix it up, I don't have
any sane way to test the end result, so I might have screwed up
royally.

My resolution looks sane to me, and compiles, but that's all I'm
really going to guarantee. Can you please give it a good round of
testing, and Jens, can you check it out too. In particular, the
fs/f2fs/data.c changes is what I might have screwed up..

                  Linus

Re: [GIT PULL] f2fs for 4.8

[Jens Axboe]

On 07/27/2016 11:46 AM, Linus Torvalds wrote:
> On Tue, Jul 26, 2016 at 9:08 PM, Jaegeuk Kim <jaegeuk@kernel.org> wrote:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git tags/for-f2fs-4.8
>
> This got some merge conflicts with the bio interface rework
> (bio_set_op_attrs() etc), and while I tried to fix it up, I don't have
> any sane way to test the end result, so I might have screwed up
> royally.
>
> My resolution looks sane to me, and compiles, but that's all I'm
> really going to guarantee. Can you please give it a good round of
> testing, and Jens, can you check it out too. In particular, the
> fs/f2fs/data.c changes is what I might have screwed up..

Looks OK to me, though I think you could have dropped the ->bi_rw
assignment in f2fs_submit_page_bio():

         bio->bi_rw = fio->op_flags;
         bio_set_op_attrs(bio, fio->op, fio->op_flags);

         __submit_bio(fio->sbi, bio, fio->type);

-- 
Jens Axboe

Re: [GIT PULL] f2fs for 4.8

[Linus Torvalds]

On Wed, Jul 27, 2016 at 11:29 AM, Jens Axboe <axboe@fb.com> wrote:
>
> Looks OK to me, though I think you could have dropped the ->bi_rw
> assignment in f2fs_submit_page_bio():
>
>         bio->bi_rw = fio->op_flags;
>         bio_set_op_attrs(bio, fio->op, fio->op_flags);
>
>         __submit_bio(fio->sbi, bio, fio->type);

Ahh, yes. I'll add a patch to drop that line, unless you just want to
do it yourself (or Jaegeuk Kim wants to do it in the f2fs tree).

Jaegeuk, I'd still really want confirmation that it all actually
works. The extra line should be harmless, even if Jens is right that
it is ugly and wrong. It might be worth testing with that line
removed.

                  Linus

Re: [GIT PULL] f2fs for 4.8

[Jens Axboe]

On 07/27/2016 12:46 PM, Linus Torvalds wrote:
> On Wed, Jul 27, 2016 at 11:29 AM, Jens Axboe <axboe@fb.com> wrote:
>>
>> Looks OK to me, though I think you could have dropped the ->bi_rw
>> assignment in f2fs_submit_page_bio():
>>
>>         bio->bi_rw = fio->op_flags;
>>         bio_set_op_attrs(bio, fio->op, fio->op_flags);
>>
>>         __submit_bio(fio->sbi, bio, fio->type);
>
> Ahh, yes. I'll add a patch to drop that line, unless you just want to
> do it yourself (or Jaegeuk Kim wants to do it in the f2fs tree).

I've got a few more things coming in this merge window, so I can just 
queue that up.

> Jaegeuk, I'd still really want confirmation that it all actually
> works. The extra line should be harmless, even if Jens is right that
> it is ugly and wrong. It might be worth testing with that line
> removed.

Yes please, I don't have an easy way to test it either.

-- 
Jens Axboe

Re: [GIT PULL] f2fs for 4.8

[Jaegeuk Kim]

On Wed, Jul 27, 2016 at 11:46:11AM -0700, Linus Torvalds wrote:
> On Wed, Jul 27, 2016 at 11:29 AM, Jens Axboe <axboe@fb.com> wrote:
> >
> > Looks OK to me, though I think you could have dropped the ->bi_rw
> > assignment in f2fs_submit_page_bio():
> >
> >         bio->bi_rw = fio->op_flags;
> >         bio_set_op_attrs(bio, fio->op, fio->op_flags);
> >
> >         __submit_bio(fio->sbi, bio, fio->type);
> 
> Ahh, yes. I'll add a patch to drop that line, unless you just want to
> do it yourself (or Jaegeuk Kim wants to do it in the f2fs tree).

Actually, I was writing an email to point that out. :)
Thanks to Jens for the patch for this.
Anyway, the changes look good to me as well.

> Jaegeuk, I'd still really want confirmation that it all actually
> works. The extra line should be harmless, even if Jens is right that
> it is ugly and wrong. It might be worth testing with that line
> removed.

I've confirmed that there is no problem; I've checked a round of xfstests and
fsstress test.

Thank you for resolving the conflicts.

Thanks,

Re: [GIT PULL] f2fs for 4.8

[Jaegeuk Kim]

On Wed, Jul 27, 2016 at 01:41:01AM -0700, Christoph Hellwig wrote:
> >  - ioctl to move a range of data between files
> 
> Please run this through the linux-api list first.  We had so many
> problem with random ioctls on random file systems in the past.  Also
> I know Samsung had an xfs (and maybe ext4?) implementation of this
> functionality in the past, so it would be extremtly useful to have
> some discussion on that.
> 
> Also didn't this just show up last week for the first time?

Since I couldn't defend its confident use cases right now, I thought f2fs'd be
good to support it in order for someone to evaluate it; I expected some
kv-stores tho.

Once getting its validation, let me take a look at other file systems to expose
it into a generic one.

Thanks,