* [PATCH 0/3 RFC] ext4: Speedup orphan file handling
@ 2015-04-16 15:42 Jan Kara
2015-04-16 15:42 ` [PATCH 1/3] ext4: Support for checksumming from journal triggers Jan Kara
` (2 more replies)
0 siblings, 3 replies; 18+ messages in thread
From: Jan Kara @ 2015-04-16 15:42 UTC (permalink / raw)
To: linux-ext4; +Cc: Jan Kara
Hello,
orphan inode handling in ext4 is a bottleneck for workloads which heavily
excercise truncate / unlink of small files as they contend on global
s_orphan_mutex (when you have fast enough storage). This patch set implements
new way of handling orphan inodes - instead using a linked list, we store inode
numbers of orphaned inodes in a file which is possible to implement in a more
scalable manner than linked list manipulations. See description of patch 2/3
for more details.
The patch set achieves significant gains both for a micro benchmark stressing
orphan inode handling (truncating file byte-by-byte, several threads in
parallel) and for reaim new_fserver workload. As a highlight, microbenchmark
runtime for 128 threads is reduced from original 160 s down to 71 s, which
is also the time it takes the benchmark to run when orphan inode handling
is completely disabled. For full numbers you can check commit logs of
patches 2/3 and 3/3. You can also check my presentation from Vault at
http://events.linuxfoundation.org/sites/events/files/slides/ext4-scaling.pdf
for graphs from tests.
I'm happy for any review, thoughts, ideas about the patches.
The kernel part of the feature is complete, the thing missing is support for
enabling the feature in mke2fs and tune2fs. Since I need one reserved inode
(currently that's hacked up and I simply use inode number 12 for simplicity
of testing) that depends on how exactly we decide to deal with the issue that
we ran out of old limit on reserved inodes.
1) I can implement support in tune2fs to increase s_first_ino by moving inodes
that are allocated in the range we want reserved. Then we can just continue
to use reserved inodes as we did previously. I kind of like this for its
simplicity, no need for ondisk format change, no need for kernel changes.
2) Implement "system directory" for reserved inodes as we spoke at Ext4 meeting
at LSF.
But before I spend time on this, I'd like to hear some thoughts on how to
deal with reserved inodes from other developers...
Honza
^ permalink raw reply [flat|nested] 18+ messages in thread* [PATCH 1/3] ext4: Support for checksumming from journal triggers 2015-04-16 15:42 [PATCH 0/3 RFC] ext4: Speedup orphan file handling Jan Kara @ 2015-04-16 15:42 ` Jan Kara 2015-04-17 19:00 ` Andreas Dilger 2015-04-16 15:42 ` [PATCH 2/3] ext4: Speedup ext4 orphan inode handling Jan Kara 2015-04-16 15:42 ` [PATCH 3/3] ext4: Improve scalability of ext4 orphan file handling Jan Kara 2 siblings, 1 reply; 18+ messages in thread From: Jan Kara @ 2015-04-16 15:42 UTC (permalink / raw) To: linux-ext4; +Cc: Jan Kara JBD2 layer support triggers which are called when journaling layer moves buffer to a certain state. We can use the frozen trigger, which gets called when buffer data is frozen and about to be written out to the journal, to compute block checksums for some buffer types (similarly as does ocfs2). This avoids unnecessary repeated recomputation of the checksum (at the cost of larger window where memory corruption won't be caught by checksumming) and is even necessary when there are unsynchronized updaters of the checksummed data. So add argument to ext4_journal_get_write_access() and ext4_journal_get_create_access() which describes buffer type so that triggers can be set accordingly. This patch is mostly only a change of prototype of the above mentioned functions and a few small helpers. Real checksumming will come later. Signed-off-by: Jan Kara <jack@suse.cz> --- fs/ext4/ext4.h | 26 ++++++++++++++++++-- fs/ext4/ext4_jbd2.c | 50 +++++++++++++++++++++++++------------ fs/ext4/ext4_jbd2.h | 18 +++++++++----- fs/ext4/extents.c | 10 +++++--- fs/ext4/file.c | 3 ++- fs/ext4/ialloc.c | 15 +++++------ fs/ext4/indirect.c | 16 ++++++++---- fs/ext4/inline.c | 19 ++++++++------ fs/ext4/inode.c | 71 +++++++++++++++++++++++++++++++---------------------- fs/ext4/mballoc.c | 12 ++++----- fs/ext4/namei.c | 39 +++++++++++++++++------------ fs/ext4/resize.c | 32 ++++++++++++++---------- fs/ext4/super.c | 16 +++++++++++- fs/ext4/xattr.c | 15 +++++++---- 14 files changed, 223 insertions(+), 119 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index f63c3d5805c4..abed83485915 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -1186,6 +1186,24 @@ struct ext4_super_block { /* Number of quota types we support */ #define EXT4_MAXQUOTAS 2 +/* Types of ext4 journal triggers */ +enum ext4_journal_trigger_type { + TR_NONE +}; + +#define EXT4_JOURNAL_TRIGGER_COUNT TR_NONE + +struct ext4_journal_trigger { + struct jbd2_buffer_trigger_type tr_triggers; + struct super_block *sb; +}; + +static inline struct ext4_journal_trigger *EXT4_TRIGGER( + struct jbd2_buffer_trigger_type *trigger) +{ + return container_of(trigger, struct ext4_journal_trigger, tr_triggers); +} + /* * fourth extended-fs super-block data in memory */ @@ -1347,6 +1365,9 @@ struct ext4_sb_info { struct mb_cache *s_mb_cache; spinlock_t s_es_lock ____cacheline_aligned_in_smp; + /* Journal triggers for checksum computation */ + struct ext4_journal_trigger s_journal_triggers[EXT4_JOURNAL_TRIGGER_COUNT]; + /* Ratelimit ext4 messages. */ struct ratelimit_state s_err_ratelimit_state; struct ratelimit_state s_warning_ratelimit_state; @@ -2108,13 +2129,14 @@ int ext4_get_block(struct inode *inode, sector_t iblock, int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, struct buffer_head *bh, int create); int ext4_walk_page_buffers(handle_t *handle, + struct inode *inode, struct buffer_head *head, unsigned from, unsigned to, int *partial, - int (*fn)(handle_t *handle, + int (*fn)(handle_t *handle, struct inode *inode, struct buffer_head *bh)); -int do_journal_get_write_access(handle_t *handle, +int do_journal_get_write_access(handle_t *handle, struct inode *inode, struct buffer_head *bh); #define FALL_BACK_TO_NONDELALLOC 1 #define CONVERT_INLINE_DATA 2 diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c index 3445035c7e01..f5ac40942f5a 100644 --- a/fs/ext4/ext4_jbd2.c +++ b/fs/ext4/ext4_jbd2.c @@ -148,19 +148,28 @@ static void ext4_journal_abort_handle(const char *caller, unsigned int line, } int __ext4_journal_get_write_access(const char *where, unsigned int line, - handle_t *handle, struct buffer_head *bh) + handle_t *handle, struct super_block *sb, + struct buffer_head *bh, + enum ext4_journal_trigger_type trigger_type) { - int err = 0; + int err; might_sleep(); - if (ext4_handle_valid(handle)) { - err = jbd2_journal_get_write_access(handle, bh); - if (err) - ext4_journal_abort_handle(where, line, __func__, bh, - handle, err); + if (!ext4_handle_valid(handle)) + return 0; + + err = jbd2_journal_get_write_access(handle, bh); + if (err) { + ext4_journal_abort_handle(where, line, __func__, bh, handle, + err); + return err; } - return err; + if (trigger_type == TR_NONE || !ext4_has_metadata_csum(sb)) + return 0; + jbd2_journal_set_triggers(bh, + &EXT4_SB(sb)->s_journal_triggers[trigger_type].tr_triggers); + return 0; } /* @@ -231,17 +240,26 @@ int __ext4_forget(const char *where, unsigned int line, handle_t *handle, } int __ext4_journal_get_create_access(const char *where, unsigned int line, - handle_t *handle, struct buffer_head *bh) + handle_t *handle, struct super_block *sb, + struct buffer_head *bh, + enum ext4_journal_trigger_type trigger_type) { - int err = 0; + int err; - if (ext4_handle_valid(handle)) { - err = jbd2_journal_get_create_access(handle, bh); - if (err) - ext4_journal_abort_handle(where, line, __func__, - bh, handle, err); + if (!ext4_handle_valid(handle)) + return 0; + + err = jbd2_journal_get_create_access(handle, bh); + if (err) { + ext4_journal_abort_handle(where, line, __func__, bh, handle, + err); + return err; } - return err; + if (trigger_type == TR_NONE || !ext4_has_metadata_csum(sb)) + return 0; + jbd2_journal_set_triggers(bh, + &EXT4_SB(sb)->s_journal_triggers[trigger_type].tr_triggers); + return 0; } int __ext4_handle_dirty_metadata(const char *where, unsigned int line, diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h index 9c5b49fb281e..c5ec6de3220e 100644 --- a/fs/ext4/ext4_jbd2.h +++ b/fs/ext4/ext4_jbd2.h @@ -232,14 +232,18 @@ int ext4_mark_inode_dirty(handle_t *handle, struct inode *inode); * Wrapper functions with which ext4 calls into JBD. */ int __ext4_journal_get_write_access(const char *where, unsigned int line, - handle_t *handle, struct buffer_head *bh); + handle_t *handle, struct super_block *sb, + struct buffer_head *bh, + enum ext4_journal_trigger_type trigger_type); int __ext4_forget(const char *where, unsigned int line, handle_t *handle, int is_metadata, struct inode *inode, struct buffer_head *bh, ext4_fsblk_t blocknr); int __ext4_journal_get_create_access(const char *where, unsigned int line, - handle_t *handle, struct buffer_head *bh); + handle_t *handle, struct super_block *sb, + struct buffer_head *bh, + enum ext4_journal_trigger_type trigger_type); int __ext4_handle_dirty_metadata(const char *where, unsigned int line, handle_t *handle, struct inode *inode, @@ -248,13 +252,15 @@ int __ext4_handle_dirty_metadata(const char *where, unsigned int line, int __ext4_handle_dirty_super(const char *where, unsigned int line, handle_t *handle, struct super_block *sb); -#define ext4_journal_get_write_access(handle, bh) \ - __ext4_journal_get_write_access(__func__, __LINE__, (handle), (bh)) +#define ext4_journal_get_write_access(handle, sb, bh, trigger_type) \ + __ext4_journal_get_write_access(__func__, __LINE__, (handle), (sb), \ + (bh), (trigger_type)) #define ext4_forget(handle, is_metadata, inode, bh, block_nr) \ __ext4_forget(__func__, __LINE__, (handle), (is_metadata), (inode), \ (bh), (block_nr)) -#define ext4_journal_get_create_access(handle, bh) \ - __ext4_journal_get_create_access(__func__, __LINE__, (handle), (bh)) +#define ext4_journal_get_create_access(handle, sb, bh, trigger_type) \ + __ext4_journal_get_create_access(__func__, __LINE__, (handle), (sb), \ + (bh), (trigger_type)) #define ext4_handle_dirty_metadata(handle, inode, bh) \ __ext4_handle_dirty_metadata(__func__, __LINE__, (handle), (inode), \ (bh)) diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c index bed43081720f..1c5d29efe585 100644 --- a/fs/ext4/extents.c +++ b/fs/ext4/extents.c @@ -142,7 +142,8 @@ static int ext4_ext_get_access(handle_t *handle, struct inode *inode, if (path->p_bh) { /* path points to block */ BUFFER_TRACE(path->p_bh, "get_write_access"); - return ext4_journal_get_write_access(handle, path->p_bh); + return ext4_journal_get_write_access(handle, inode->i_sb, + path->p_bh, TR_NONE); } /* path points to leaf/index in inode body */ /* we use in-core data, no need to protect them */ @@ -1095,7 +1096,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode, } lock_buffer(bh); - err = ext4_journal_get_create_access(handle, bh); + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, TR_NONE); if (err) goto cleanup; @@ -1168,7 +1169,8 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode, } lock_buffer(bh); - err = ext4_journal_get_create_access(handle, bh); + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, + TR_NONE); if (err) goto cleanup; @@ -1287,7 +1289,7 @@ static int ext4_ext_grow_indepth(handle_t *handle, struct inode *inode, return -ENOMEM; lock_buffer(bh); - err = ext4_journal_get_create_access(handle, bh); + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, TR_NONE); if (err) { unlock_buffer(bh); goto out; diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 33a09da16c9c..39e9b39f3c1c 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -258,7 +258,8 @@ static int ext4_file_open(struct inode * inode, struct file * filp) if (IS_ERR(handle)) return PTR_ERR(handle); BUFFER_TRACE(sbi->s_sbh, "get_write_access"); - err = ext4_journal_get_write_access(handle, sbi->s_sbh); + err = ext4_journal_get_write_access(handle, sb, + sbi->s_sbh, TR_NONE); if (err) { ext4_journal_stop(handle); return err; diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c index ac644c31ca67..7a711cf0d183 100644 --- a/fs/ext4/ialloc.c +++ b/fs/ext4/ialloc.c @@ -291,7 +291,7 @@ void ext4_free_inode(handle_t *handle, struct inode *inode) goto error_return; BUFFER_TRACE(bitmap_bh, "get_write_access"); - fatal = ext4_journal_get_write_access(handle, bitmap_bh); + fatal = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); if (fatal) goto error_return; @@ -299,7 +299,7 @@ void ext4_free_inode(handle_t *handle, struct inode *inode) gdp = ext4_get_group_desc(sb, block_group, &bh2); if (gdp) { BUFFER_TRACE(bh2, "get_write_access"); - fatal = ext4_journal_get_write_access(handle, bh2); + fatal = ext4_journal_get_write_access(handle, sb, bh2, TR_NONE); } ext4_lock_group(sb, block_group); cleared = ext4_test_and_clear_bit(bit, bitmap_bh->b_data); @@ -845,7 +845,8 @@ repeat_in_this_group: } } BUFFER_TRACE(inode_bitmap_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, inode_bitmap_bh); + err = ext4_journal_get_write_access(handle, sb, inode_bitmap_bh, + TR_NONE); if (err) { ext4_std_error(sb, err); goto out; @@ -875,7 +876,7 @@ got: } BUFFER_TRACE(group_desc_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, group_desc_bh); + err = ext4_journal_get_write_access(handle, sb, group_desc_bh, TR_NONE); if (err) { ext4_std_error(sb, err); goto out; @@ -892,7 +893,8 @@ got: goto out; } BUFFER_TRACE(block_bitmap_bh, "get block bitmap access"); - err = ext4_journal_get_write_access(handle, block_bitmap_bh); + err = ext4_journal_get_write_access(handle, sb, block_bitmap_bh, + TR_NONE); if (err) { brelse(block_bitmap_bh); ext4_std_error(sb, err); @@ -1288,8 +1290,7 @@ int ext4_init_inode_table(struct super_block *sb, ext4_group_t group, num = sbi->s_itb_per_group - used_blks; BUFFER_TRACE(group_desc_bh, "get_write_access"); - ret = ext4_journal_get_write_access(handle, - group_desc_bh); + ret = ext4_journal_get_write_access(handle, sb, group_desc_bh, TR_NONE); if (ret) goto err_out; diff --git a/fs/ext4/indirect.c b/fs/ext4/indirect.c index 45fe924f82bc..54adceeb09aa 100644 --- a/fs/ext4/indirect.c +++ b/fs/ext4/indirect.c @@ -351,7 +351,8 @@ static int ext4_alloc_branch(handle_t *handle, } lock_buffer(bh); BUFFER_TRACE(bh, "call get_create_access"); - err = ext4_journal_get_create_access(handle, bh); + err = ext4_journal_get_create_access(handle, ar->inode->i_sb, + bh, TR_NONE); if (err) { unlock_buffer(bh); goto failed; @@ -423,7 +424,8 @@ static int ext4_splice_branch(handle_t *handle, */ if (where->bh) { BUFFER_TRACE(where->bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, where->bh); + err = ext4_journal_get_write_access(handle, ar->inode->i_sb, + where->bh, TR_NONE); if (err) goto err_out; } @@ -967,7 +969,8 @@ static int ext4_clear_blocks(handle_t *handle, struct inode *inode, goto out_err; if (bh) { BUFFER_TRACE(bh, "retaking write access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, inode->i_sb, + bh, TR_NONE); if (unlikely(err)) goto out_err; } @@ -1018,7 +1021,8 @@ static void ext4_free_data(handle_t *handle, struct inode *inode, if (this_bh) { /* For indirect block */ BUFFER_TRACE(this_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, this_bh); + err = ext4_journal_get_write_access(handle, inode->i_sb, + this_bh, TR_NONE); /* Important: if we can't update the indirect pointers * to the blocks, we can't free them. */ if (err) @@ -1182,7 +1186,9 @@ static void ext4_free_branches(handle_t *handle, struct inode *inode, */ BUFFER_TRACE(parent_bh, "get_write_access"); if (!ext4_journal_get_write_access(handle, - parent_bh)){ + inode->i_sb, + parent_bh, + TR_NONE)){ *p = 0; BUFFER_TRACE(parent_bh, "call ext4_handle_dirty_metadata"); diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c index 4b143febf21f..d4879c2396ce 100644 --- a/fs/ext4/inline.c +++ b/fs/ext4/inline.c @@ -259,7 +259,8 @@ static int ext4_create_inline_data(handle_t *handle, return error; BUFFER_TRACE(is.iloc.bh, "get_write_access"); - error = ext4_journal_get_write_access(handle, is.iloc.bh); + error = ext4_journal_get_write_access(handle, inode->i_sb, is.iloc.bh, + TR_NONE); if (error) goto out; @@ -343,7 +344,8 @@ static int ext4_update_inline_data(handle_t *handle, struct inode *inode, goto out; BUFFER_TRACE(is.iloc.bh, "get_write_access"); - error = ext4_journal_get_write_access(handle, is.iloc.bh); + error = ext4_journal_get_write_access(handle, inode->i_sb, is.iloc.bh, + TR_NONE); if (error) goto out; @@ -421,7 +423,8 @@ static int ext4_destroy_inline_data_nolock(handle_t *handle, goto out; BUFFER_TRACE(is.iloc.bh, "get_write_access"); - error = ext4_journal_get_write_access(handle, is.iloc.bh); + error = ext4_journal_get_write_access(handle, inode->i_sb, is.iloc.bh, + TR_NONE); if (error) goto out; @@ -586,7 +589,7 @@ retry: ret = __block_write_begin(page, from, to, ext4_get_block); if (!ret && ext4_should_journal_data(inode)) { - ret = ext4_walk_page_buffers(handle, page_buffers(page), + ret = ext4_walk_page_buffers(handle, inode, page_buffers(page), from, to, NULL, do_journal_get_write_access); } @@ -1011,7 +1014,8 @@ static int ext4_add_dirent_to_inline(handle_t *handle, return err; BUFFER_TRACE(iloc->bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, iloc->bh); + err = ext4_journal_get_write_access(handle, dir->i_sb, iloc->bh, + TR_NONE); if (err) return err; ext4_insert_dentry(inode, de, inline_size, name, namelen); @@ -1211,7 +1215,8 @@ static int ext4_convert_inline_data_nolock(handle_t *handle, } lock_buffer(data_bh); - error = ext4_journal_get_create_access(handle, data_bh); + error = ext4_journal_get_create_access(handle, inode->i_sb, data_bh, + TR_NONE); if (error) { unlock_buffer(data_bh); error = -EIO; @@ -1685,7 +1690,7 @@ int ext4_delete_inline_entry(handle_t *handle, } BUFFER_TRACE(bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); if (err) goto out; diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 5cb9a212b86f..0813e6a9ca98 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -134,7 +134,6 @@ static inline int ext4_begin_ordered_truncate(struct inode *inode, static void ext4_invalidatepage(struct page *page, unsigned int offset, unsigned int length); static int __ext4_journalled_writepage(struct page *page, unsigned int len); -static int ext4_bh_delay_or_unwritten(handle_t *handle, struct buffer_head *bh); static int ext4_meta_trans_blocks(struct inode *inode, int lblocks, int pextents); @@ -766,7 +765,8 @@ struct buffer_head *ext4_getblk(handle_t *handle, struct inode *inode, */ lock_buffer(bh); BUFFER_TRACE(bh, "call get_create_access"); - err = ext4_journal_get_create_access(handle, bh); + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, + TR_NONE); if (unlikely(err)) { unlock_buffer(bh); goto errout; @@ -806,12 +806,12 @@ struct buffer_head *ext4_bread(handle_t *handle, struct inode *inode, return ERR_PTR(-EIO); } -int ext4_walk_page_buffers(handle_t *handle, +int ext4_walk_page_buffers(handle_t *handle, struct inode *inode, struct buffer_head *head, unsigned from, unsigned to, int *partial, - int (*fn)(handle_t *handle, + int (*fn)(handle_t *handle, struct inode *inode, struct buffer_head *bh)) { struct buffer_head *bh; @@ -830,7 +830,7 @@ int ext4_walk_page_buffers(handle_t *handle, *partial = 1; continue; } - err = (*fn)(handle, bh); + err = (*fn)(handle, inode, bh); if (!ret) ret = err; } @@ -861,7 +861,7 @@ int ext4_walk_page_buffers(handle_t *handle, * is elevated. We'll still have enough credits for the tiny quotafile * write. */ -int do_journal_get_write_access(handle_t *handle, +int do_journal_get_write_access(handle_t *handle, struct inode *inode, struct buffer_head *bh) { int dirty = buffer_dirty(bh); @@ -880,7 +880,7 @@ int do_journal_get_write_access(handle_t *handle, if (dirty) clear_buffer_dirty(bh); BUFFER_TRACE(bh, "get write access"); - ret = ext4_journal_get_write_access(handle, bh); + ret = ext4_journal_get_write_access(handle, inode->i_sb, bh, TR_NONE); if (!ret && dirty) ret = ext4_handle_dirty_metadata(handle, NULL, bh); return ret; @@ -956,8 +956,8 @@ retry_journal: ret = __block_write_begin(page, pos, len, ext4_get_block); if (!ret && ext4_should_journal_data(inode)) { - ret = ext4_walk_page_buffers(handle, page_buffers(page), - from, to, NULL, + ret = ext4_walk_page_buffers(handle, inode, + page_buffers(page), from, to, NULL, do_journal_get_write_access); } @@ -998,7 +998,8 @@ retry_journal: } /* For write_end() in data=journal mode */ -static int write_end_fn(handle_t *handle, struct buffer_head *bh) +static int write_end_fn(handle_t *handle, struct inode *inode, + struct buffer_head *bh) { int ret; if (!buffer_mapped(bh) || buffer_freed(bh)) @@ -1120,8 +1121,8 @@ static int ext4_journalled_write_end(struct file *file, page_zero_new_buffers(page, from+copied, to); } - ret = ext4_walk_page_buffers(handle, page_buffers(page), from, - to, &partial, write_end_fn); + ret = ext4_walk_page_buffers(handle, inode, page_buffers(page), + from, to, &partial, write_end_fn); if (!partial) SetPageUptodate(page); } @@ -1377,7 +1378,8 @@ static void ext4_print_free_blocks(struct inode *inode) return; } -static int ext4_bh_delay_or_unwritten(handle_t *handle, struct buffer_head *bh) +static int ext4_bh_delay_or_unwritten(handle_t *handle, struct inode *inode, + struct buffer_head *bh) { return (buffer_delay(bh) || buffer_unwritten(bh)) && buffer_dirty(bh); } @@ -1565,13 +1567,15 @@ int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, return 0; } -static int bget_one(handle_t *handle, struct buffer_head *bh) +static int bget_one(handle_t *handle, struct inode *inode, + struct buffer_head *bh) { get_bh(bh); return 0; } -static int bput_one(handle_t *handle, struct buffer_head *bh) +static int bput_one(handle_t *handle, struct inode *inode, + struct buffer_head *bh) { put_bh(bh); return 0; @@ -1602,7 +1606,7 @@ static int __ext4_journalled_writepage(struct page *page, BUG(); goto out; } - ext4_walk_page_buffers(handle, page_bufs, 0, len, + ext4_walk_page_buffers(handle, inode, page_bufs, 0, len, NULL, bget_one); } /* As soon as we unlock the page, it can go away, but we have @@ -1620,16 +1624,17 @@ static int __ext4_journalled_writepage(struct page *page, if (inline_data) { BUFFER_TRACE(inode_bh, "get write access"); - ret = ext4_journal_get_write_access(handle, inode_bh); + ret = ext4_journal_get_write_access(handle, inode->i_sb, + inode_bh, TR_NONE); err = ext4_handle_dirty_metadata(handle, inode, inode_bh); } else { - ret = ext4_walk_page_buffers(handle, page_bufs, 0, len, NULL, - do_journal_get_write_access); + ret = ext4_walk_page_buffers(handle, inode, page_bufs, 0, len, + NULL, do_journal_get_write_access); - err = ext4_walk_page_buffers(handle, page_bufs, 0, len, NULL, - write_end_fn); + err = ext4_walk_page_buffers(handle, inode, page_bufs, 0, len, + NULL, write_end_fn); } if (ret == 0) ret = err; @@ -1639,7 +1644,7 @@ static int __ext4_journalled_writepage(struct page *page, ret = err; if (!ext4_has_inline_data(inode)) - ext4_walk_page_buffers(NULL, page_bufs, 0, len, + ext4_walk_page_buffers(NULL, inode, page_bufs, 0, len, NULL, bput_one); ext4_set_inode_state(inode, EXT4_STATE_JDATA); out: @@ -1714,7 +1719,7 @@ static int ext4_writepage(struct page *page, * journal_submit_inode_data_buffers() and in that case we must write * allocated buffers to achieve data=ordered mode guarantees. */ - if (ext4_walk_page_buffers(NULL, page_bufs, 0, len, NULL, + if (ext4_walk_page_buffers(NULL, inode, page_bufs, 0, len, NULL, ext4_bh_delay_or_unwritten)) { redirty_page_for_writepage(wbc, page); if (current->flags & PF_MEMALLOC) { @@ -3265,7 +3270,8 @@ static int __ext4_block_zero_page_range(handle_t *handle, } if (ext4_should_journal_data(inode)) { BUFFER_TRACE(bh, "get write access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, inode->i_sb, bh, + TR_NONE); if (err) goto unlock; } @@ -4362,7 +4368,9 @@ static int ext4_do_update_inode(handle_t *handle, ext4_clear_inode_state(inode, EXT4_STATE_NEW); if (set_large_file) { BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get write access"); - err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh); + err = ext4_journal_get_write_access(handle, sb, + EXT4_SB(sb)->s_sbh, + TR_NONE); if (err) goto out_brelse; ext4_update_dynamic_rev(sb); @@ -4828,7 +4836,8 @@ ext4_reserve_inode_write(handle_t *handle, struct inode *inode, err = ext4_get_inode_loc(inode, iloc); if (!err) { BUFFER_TRACE(iloc->bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, iloc->bh); + err = ext4_journal_get_write_access(handle, inode->i_sb, + iloc->bh, TR_NONE); if (err) { brelse(iloc->bh); iloc->bh = NULL; @@ -4981,7 +4990,8 @@ static int ext4_pin_inode(handle_t *handle, struct inode *inode) err = ext4_get_inode_loc(inode, &iloc); if (!err) { BUFFER_TRACE(iloc.bh, "get_write_access"); - err = jbd2_journal_get_write_access(handle, iloc.bh); + err = jbd2_journal_get_write_access(inode->i_sb, handle, + iloc.bh, TR_NONE); if (!err) err = ext4_handle_dirty_metadata(handle, NULL, @@ -5071,7 +5081,8 @@ int ext4_change_inode_journal_flag(struct inode *inode, int val) return err; } -static int ext4_bh_unmapped(handle_t *handle, struct buffer_head *bh) +static int ext4_bh_unmapped(handle_t *handle, struct inode *inode, + struct buffer_head *bh) { return !buffer_mapped(bh); } @@ -5121,7 +5132,7 @@ int ext4_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf) * journal_start/journal_stop which can block and take a long time */ if (page_has_buffers(page)) { - if (!ext4_walk_page_buffers(NULL, page_buffers(page), + if (!ext4_walk_page_buffers(NULL, inode, page_buffers(page), 0, len, NULL, ext4_bh_unmapped)) { /* Wait so that we don't change page under IO */ @@ -5145,7 +5156,7 @@ retry_alloc: } ret = __block_page_mkwrite(vma, vmf, get_block); if (!ret && ext4_should_journal_data(inode)) { - if (ext4_walk_page_buffers(handle, page_buffers(page), 0, + if (ext4_walk_page_buffers(handle, inode, page_buffers(page), 0, PAGE_CACHE_SIZE, NULL, do_journal_get_write_access)) { unlock_page(page); ret = VM_FAULT_SIGBUS; diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 8d1e60214ef0..dcd05c5f6c48 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -2886,7 +2886,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac, goto out_err; BUFFER_TRACE(bitmap_bh, "getting write access"); - err = ext4_journal_get_write_access(handle, bitmap_bh); + err = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); if (err) goto out_err; @@ -2899,7 +2899,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac, ext4_free_group_clusters(sb, gdp)); BUFFER_TRACE(gdp_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, gdp_bh); + err = ext4_journal_get_write_access(handle, sb, gdp_bh, TR_NONE); if (err) goto out_err; @@ -4769,7 +4769,7 @@ do_more: } BUFFER_TRACE(bitmap_bh, "getting write access"); - err = ext4_journal_get_write_access(handle, bitmap_bh); + err = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); if (err) goto error_return; @@ -4779,7 +4779,7 @@ do_more: * using it */ BUFFER_TRACE(gd_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, gd_bh); + err = ext4_journal_get_write_access(handle, sb, gd_bh, TR_NONE); if (err) goto error_return; #ifdef AGGRESSIVE_CHECK @@ -4945,7 +4945,7 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb, } BUFFER_TRACE(bitmap_bh, "getting write access"); - err = ext4_journal_get_write_access(handle, bitmap_bh); + err = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); if (err) goto error_return; @@ -4955,7 +4955,7 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb, * using it */ BUFFER_TRACE(gd_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, gd_bh); + err = ext4_journal_get_write_access(handle, sb, gd_bh, TR_NONE); if (err) goto error_return; diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 28fe71a2904c..460c716e38b0 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -68,7 +68,7 @@ static struct buffer_head *ext4_append(handle_t *handle, inode->i_size += inode->i_sb->s_blocksize; EXT4_I(inode)->i_disksize = inode->i_size; BUFFER_TRACE(bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, inode->i_sb, bh, TR_NONE); if (err) { brelse(bh); ext4_std_error(inode->i_sb, err); @@ -1528,12 +1528,13 @@ static struct ext4_dir_entry_2 *do_split(handle_t *handle, struct inode *dir, } BUFFER_TRACE(*bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, *bh); + err = ext4_journal_get_write_access(handle, dir->i_sb, *bh, TR_NONE); if (err) goto journal_error; BUFFER_TRACE(frame->bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, frame->bh); + err = ext4_journal_get_write_access(handle, dir->i_sb, frame->bh, + TR_NONE); if (err) goto journal_error; @@ -1694,7 +1695,7 @@ static int add_dirent_to_buf(handle_t *handle, struct dentry *dentry, return err; } BUFFER_TRACE(bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); if (err) { ext4_std_error(dir->i_sb, err); return err; @@ -1756,7 +1757,7 @@ static int make_indexed_dir(handle_t *handle, struct dentry *dentry, blocksize = dir->i_sb->s_blocksize; dxtrace(printk(KERN_DEBUG "Creating index: inode %lu\n", dir->i_ino)); BUFFER_TRACE(bh, "get_write_access"); - retval = ext4_journal_get_write_access(handle, bh); + retval = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); if (retval) { ext4_std_error(dir->i_sb, retval); brelse(bh); @@ -1965,7 +1966,7 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, } BUFFER_TRACE(bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE); if (err) goto journal_error; @@ -2002,7 +2003,8 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, node2->fake.rec_len = ext4_rec_len_to_disk(sb->s_blocksize, sb->s_blocksize); BUFFER_TRACE(frame->bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, frame->bh); + err = ext4_journal_get_write_access(handle, sb, frame->bh, + TR_NONE); if (err) goto journal_error; if (levels) { @@ -2012,8 +2014,9 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, icount1, icount2)); BUFFER_TRACE(frame->bh, "get_write_access"); /* index root */ - err = ext4_journal_get_write_access(handle, - frames[0].bh); + err = ext4_journal_get_write_access(handle, sb, + frames[0].bh, + TR_NONE); if (err) goto journal_error; @@ -2054,8 +2057,9 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, frame->at = at = at - entries + entries2; frame->entries = entries = entries2; frame->bh = bh2; - err = ext4_journal_get_write_access(handle, - frame->bh); + err = ext4_journal_get_write_access(handle, sb, + frame->bh, + TR_NONE); if (err) goto journal_error; } @@ -2143,7 +2147,7 @@ static int ext4_delete_entry(handle_t *handle, csum_size = sizeof(struct ext4_dir_entry_tail); BUFFER_TRACE(bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); if (unlikely(err)) goto out; @@ -2567,7 +2571,7 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); BUFFER_TRACE(sbi->s_sbh, "get_write_access"); - err = ext4_journal_get_write_access(handle, sbi->s_sbh); + err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); if (err) goto out; @@ -2660,7 +2664,8 @@ int ext4_orphan_del(handle_t *handle, struct inode *inode) if (prev == &sbi->s_orphan) { jbd_debug(4, "superblock will point to %u\n", ino_next); BUFFER_TRACE(sbi->s_sbh, "get_write_access"); - err = ext4_journal_get_write_access(handle, sbi->s_sbh); + err = ext4_journal_get_write_access(handle, inode->i_sb, + sbi->s_sbh, TR_NONE); if (err) { mutex_unlock(&sbi->s_orphan_lock); goto out_brelse; @@ -3039,7 +3044,8 @@ static int ext4_rename_dir_prepare(handle_t *handle, struct ext4_renament *ent) if (le32_to_cpu(ent->parent_de->inode) != ent->dir->i_ino) return -EIO; BUFFER_TRACE(ent->dir_bh, "get_write_access"); - return ext4_journal_get_write_access(handle, ent->dir_bh); + return ext4_journal_get_write_access(handle, ent->dir->i_sb, + ent->dir_bh, TR_NONE); } static int ext4_rename_dir_finish(handle_t *handle, struct ext4_renament *ent, @@ -3075,7 +3081,8 @@ static int ext4_setent(handle_t *handle, struct ext4_renament *ent, int retval; BUFFER_TRACE(ent->bh, "get write access"); - retval = ext4_journal_get_write_access(handle, ent->bh); + retval = ext4_journal_get_write_access(handle, ent->dir->i_sb, ent->bh, + TR_NONE); if (retval) return retval; ent->de->inode = cpu_to_le32(ino); diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c index 8a8ec6293b19..f14ddaa96aa9 100644 --- a/fs/ext4/resize.c +++ b/fs/ext4/resize.c @@ -361,7 +361,7 @@ static struct buffer_head *bclean(handle_t *handle, struct super_block *sb, if (unlikely(!bh)) return ERR_PTR(-ENOMEM); BUFFER_TRACE(bh, "get_write_access"); - if ((err = ext4_journal_get_write_access(handle, bh))) { + if ((err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE))) { brelse(bh); bh = ERR_PTR(err); } else { @@ -440,7 +440,7 @@ static int set_flexbg_block_bitmap(struct super_block *sb, handle_t *handle, return -ENOMEM; BUFFER_TRACE(bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE); if (err) return err; ext4_debug("mark block bitmap %#04llx (+%llu/%u)\n", block, @@ -533,7 +533,8 @@ static int setup_new_flex_group_blocks(struct super_block *sb, } BUFFER_TRACE(gdb, "get_write_access"); - err = ext4_journal_get_write_access(handle, gdb); + err = ext4_journal_get_write_access(handle, sb, gdb, + TR_NONE); if (err) { brelse(gdb); goto out; @@ -796,17 +797,18 @@ static int add_new_gdb(handle_t *handle, struct inode *inode, } BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access"); - err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh); + err = ext4_journal_get_write_access(handle, sb, EXT4_SB(sb)->s_sbh, + TR_NONE); if (unlikely(err)) goto exit_dind; BUFFER_TRACE(gdb_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, gdb_bh); + err = ext4_journal_get_write_access(handle, sb, gdb_bh, TR_NONE); if (unlikely(err)) goto exit_dind; BUFFER_TRACE(dind, "get_write_access"); - err = ext4_journal_get_write_access(handle, dind); + err = ext4_journal_get_write_access(handle, sb, dind, TR_NONE); if (unlikely(err)) ext4_std_error(sb, err); @@ -911,7 +913,7 @@ static int add_new_gdb_meta_bg(struct super_block *sb, EXT4_SB(sb)->s_gdb_count++; kvfree(o_group_desc); BUFFER_TRACE(gdb_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, gdb_bh); + err = ext4_journal_get_write_access(handle, sb, gdb_bh, TR_NONE); if (unlikely(err)) brelse(gdb_bh); return err; @@ -987,7 +989,8 @@ static int reserve_backup_gdb(handle_t *handle, struct inode *inode, for (i = 0; i < reserved_gdb; i++) { BUFFER_TRACE(primary[i], "get_write_access"); - if ((err = ext4_journal_get_write_access(handle, primary[i]))) + if ((err = ext4_journal_get_write_access(handle, sb, primary[i], + TR_NONE))) goto exit_bh; } @@ -1095,7 +1098,8 @@ static void update_backups(struct super_block *sb, int blk_off, char *data, backup_block, backup_block - ext4_group_first_block_no(sb, group)); BUFFER_TRACE(bh, "get_write_access"); - if ((err = ext4_journal_get_write_access(handle, bh))) + if ((err = ext4_journal_get_write_access(handle, sb, bh, + TR_NONE))) break; lock_buffer(bh); memcpy(bh->b_data, data, size); @@ -1175,7 +1179,8 @@ static int ext4_add_new_descs(handle_t *handle, struct super_block *sb, if (gdb_off) { gdb_bh = sbi->s_group_desc[gdb_num]; BUFFER_TRACE(gdb_bh, "get_write_access"); - err = ext4_journal_get_write_access(handle, gdb_bh); + err = ext4_journal_get_write_access(handle, sb, gdb_bh, + TR_NONE); if (!err && reserved_gdb && ext4_bg_num_gdb(sb, group)) err = reserve_backup_gdb(handle, resize_inode, group); @@ -1445,7 +1450,7 @@ static int ext4_flex_group_add(struct super_block *sb, } BUFFER_TRACE(sbi->s_sbh, "get_write_access"); - err = ext4_journal_get_write_access(handle, sbi->s_sbh); + err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); if (err) goto exit_journal; @@ -1658,7 +1663,8 @@ static int ext4_group_extend_no_check(struct super_block *sb, } BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access"); - err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh); + err = ext4_journal_get_write_access(handle, sb, EXT4_SB(sb)->s_sbh, + TR_NONE); if (err) { ext4_warning(sb, "error %d on journal write access", err); goto errout; @@ -1818,7 +1824,7 @@ static int ext4_convert_meta_bg(struct super_block *sb, struct inode *inode) return PTR_ERR(handle); BUFFER_TRACE(sbi->s_sbh, "get_write_access"); - err = ext4_journal_get_write_access(handle, sbi->s_sbh); + err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); if (err) goto errout; diff --git a/fs/ext4/super.c b/fs/ext4/super.c index e061e66c8280..0babe8c435b6 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -3406,6 +3406,20 @@ static int ext4_reserve_clusters(struct ext4_sb_info *sbi, ext4_fsblk_t count) return 0; } +static void ext4_setup_csum_trigger(struct super_block *sb, + enum ext4_journal_trigger_type type, + void (*trigger)( + struct jbd2_buffer_trigger_type *type, + struct buffer_head *bh, + void *mapped_data, + size_t size)) +{ + struct ext4_sb_info *sbi = EXT4_SB(sb); + + sbi->s_journal_triggers[type].sb = sb; + sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; +} + static int ext4_fill_super(struct super_block *sb, void *data, int silent) { char *orig_data = kstrdup(data, GFP_KERNEL); @@ -5416,7 +5430,7 @@ static ssize_t ext4_quota_write(struct super_block *sb, int type, if (!bh) goto out; BUFFER_TRACE(bh, "get write access"); - err = ext4_journal_get_write_access(handle, bh); + err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE); if (err) { brelse(bh); return err; diff --git a/fs/ext4/xattr.c b/fs/ext4/xattr.c index 1e09fc77395c..a0a285844037 100644 --- a/fs/ext4/xattr.c +++ b/fs/ext4/xattr.c @@ -530,7 +530,8 @@ static void ext4_xattr_update_super_block(handle_t *handle, return; BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access"); - if (ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh) == 0) { + if (ext4_journal_get_write_access(handle, sb, EXT4_SB(sb)->s_sbh, + TR_NONE) == 0) { EXT4_SET_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_EXT_ATTR); ext4_handle_dirty_super(handle, sb); } @@ -550,7 +551,7 @@ ext4_xattr_release_block(handle_t *handle, struct inode *inode, ce = mb_cache_entry_get(ext4_mb_cache, bh->b_bdev, bh->b_blocknr); BUFFER_TRACE(bh, "get_write_access"); - error = ext4_journal_get_write_access(handle, bh); + error = ext4_journal_get_write_access(handle, inode->i_sb, bh, TR_NONE); if (error) goto out; @@ -793,7 +794,8 @@ ext4_xattr_block_set(handle_t *handle, struct inode *inode, ce = mb_cache_entry_get(ext4_mb_cache, bs->bh->b_bdev, bs->bh->b_blocknr); BUFFER_TRACE(bs->bh, "get_write_access"); - error = ext4_journal_get_write_access(handle, bs->bh); + error = ext4_journal_get_write_access(handle, sb, bs->bh, + TR_NONE); if (error) goto cleanup; lock_buffer(bs->bh); @@ -880,7 +882,9 @@ inserted: goto cleanup; BUFFER_TRACE(new_bh, "get_write_access"); error = ext4_journal_get_write_access(handle, - new_bh); + sb, + new_bh, + TR_NONE); if (error) goto cleanup_dquot; lock_buffer(new_bh); @@ -932,7 +936,8 @@ getblk_failed: goto cleanup; } lock_buffer(new_bh); - error = ext4_journal_get_create_access(handle, new_bh); + error = ext4_journal_get_create_access(handle, sb, + new_bh, TR_NONE); if (error) { unlock_buffer(new_bh); error = -EIO; -- 2.1.4 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 1/3] ext4: Support for checksumming from journal triggers 2015-04-16 15:42 ` [PATCH 1/3] ext4: Support for checksumming from journal triggers Jan Kara @ 2015-04-17 19:00 ` Andreas Dilger 2015-04-20 9:07 ` Jan Kara 0 siblings, 1 reply; 18+ messages in thread From: Andreas Dilger @ 2015-04-17 19:00 UTC (permalink / raw) To: Jan Kara; +Cc: linux-ext4 On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: > > JBD2 layer support triggers which are called when journaling layer moves > buffer to a certain state. We can use the frozen trigger, which gets > called when buffer data is frozen and about to be written out to the > journal, to compute block checksums for some buffer types (similarly as > does ocfs2). This avoids unnecessary repeated recomputation of the > checksum (at the cost of larger window where memory corruption won't be > caught by checksumming) and is even necessary when there are > unsynchronized updaters of the checksummed data. > > So add argument to ext4_journal_get_write_access() and > ext4_journal_get_create_access() which describes buffer type so that > triggers can be set accordingly. This patch is mostly only a change of > prototype of the above mentioned functions and a few small helpers. Real > checksumming will come later. > > Signed-off-by: Jan Kara <jack@suse.cz> > --- > fs/ext4/ext4.h | 26 ++++++++++++++++++-- > fs/ext4/ext4_jbd2.c | 50 +++++++++++++++++++++++++------------ > fs/ext4/ext4_jbd2.h | 18 +++++++++----- > fs/ext4/extents.c | 10 +++++--- > fs/ext4/file.c | 3 ++- > fs/ext4/ialloc.c | 15 +++++------ > fs/ext4/indirect.c | 16 ++++++++---- > fs/ext4/inline.c | 19 ++++++++------ > fs/ext4/inode.c | 71 +++++++++++++++++++++++++++++++---------------------- > fs/ext4/mballoc.c | 12 ++++----- > fs/ext4/namei.c | 39 +++++++++++++++++------------ > fs/ext4/resize.c | 32 ++++++++++++++---------- > fs/ext4/super.c | 16 +++++++++++- > fs/ext4/xattr.c | 15 +++++++---- > 14 files changed, 223 insertions(+), 119 deletions(-) > > diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > index f63c3d5805c4..abed83485915 100644 > --- a/fs/ext4/ext4.h > +++ b/fs/ext4/ext4.h > @@ -1186,6 +1186,24 @@ struct ext4_super_block { > /* Number of quota types we support */ > #define EXT4_MAXQUOTAS 2 > > +/* Types of ext4 journal triggers */ > +enum ext4_journal_trigger_type { > + TR_NONE > +}; Probably good if this has an EXT4_ prefix to avoid name clashes? > + > +#define EXT4_JOURNAL_TRIGGER_COUNT TR_NONE This should just be added after TR_NONE so that it is always correct: enum ext4_journal_trigger_type { EXT4_JTR_NONE, EXT4_JTR_MAX, }; and the s_journal_triggers[] array would use EXT4_JTR_MAX - 1? > +struct ext4_journal_trigger { > + struct jbd2_buffer_trigger_type tr_triggers; > + struct super_block *sb; > +}; > + > +static inline struct ext4_journal_trigger *EXT4_TRIGGER( > + struct jbd2_buffer_trigger_type *trigger) > +{ > + return container_of(trigger, struct ext4_journal_trigger, tr_triggers); > +} > + > /* > * fourth extended-fs super-block data in memory > */ > @@ -1347,6 +1365,9 @@ struct ext4_sb_info { > struct mb_cache *s_mb_cache; > spinlock_t s_es_lock ____cacheline_aligned_in_smp; > > + /* Journal triggers for checksum computation */ > + struct ext4_journal_trigger s_journal_triggers[EXT4_JOURNAL_TRIGGER_COUNT]; > + > /* Ratelimit ext4 messages. */ > struct ratelimit_state s_err_ratelimit_state; > struct ratelimit_state s_warning_ratelimit_state; > @@ -2108,13 +2129,14 @@ int ext4_get_block(struct inode *inode, sector_t iblock, > int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, > struct buffer_head *bh, int create); > int ext4_walk_page_buffers(handle_t *handle, > + struct inode *inode, > struct buffer_head *head, > unsigned from, > unsigned to, > int *partial, > - int (*fn)(handle_t *handle, > + int (*fn)(handle_t *handle, struct inode *inode, > struct buffer_head *bh)); > -int do_journal_get_write_access(handle_t *handle, > +int do_journal_get_write_access(handle_t *handle, struct inode *inode, > struct buffer_head *bh); > #define FALL_BACK_TO_NONDELALLOC 1 > #define CONVERT_INLINE_DATA 2 > diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c > index 3445035c7e01..f5ac40942f5a 100644 > --- a/fs/ext4/ext4_jbd2.c > +++ b/fs/ext4/ext4_jbd2.c > @@ -148,19 +148,28 @@ static void ext4_journal_abort_handle(const char *caller, unsigned int line, > } > > int __ext4_journal_get_write_access(const char *where, unsigned int line, > - handle_t *handle, struct buffer_head *bh) > + handle_t *handle, struct super_block *sb, > + struct buffer_head *bh, > + enum ext4_journal_trigger_type trigger_type) > { > - int err = 0; > + int err; > > might_sleep(); > > - if (ext4_handle_valid(handle)) { > - err = jbd2_journal_get_write_access(handle, bh); > - if (err) > - ext4_journal_abort_handle(where, line, __func__, bh, > - handle, err); > + if (!ext4_handle_valid(handle)) > + return 0; > + > + err = jbd2_journal_get_write_access(handle, bh); > + if (err) { > + ext4_journal_abort_handle(where, line, __func__, bh, handle, > + err); > + return err; > } > - return err; > + if (trigger_type == TR_NONE || !ext4_has_metadata_csum(sb)) > + return 0; Should this add a check for trigger_type >= EXT4_JTR_MAX? WARN_ON? I was wondering whether checking for trigger_type == TR_NONE could be replaced by checking tr_triggers == NULL in jbd2_journal_set_triggers(), but that needs extra locking to access jh to see if it needs to be reset or not, so it isn't worthwhile. Cheers, Andreas > + jbd2_journal_set_triggers(bh, > + &EXT4_SB(sb)->s_journal_triggers[trigger_type].tr_triggers); > + return 0; > } > > /* > @@ -231,17 +240,26 @@ int __ext4_forget(const char *where, unsigned int line, handle_t *handle, > } > > int __ext4_journal_get_create_access(const char *where, unsigned int line, > - handle_t *handle, struct buffer_head *bh) > + handle_t *handle, struct super_block *sb, > + struct buffer_head *bh, > + enum ext4_journal_trigger_type trigger_type) > { > - int err = 0; > + int err; > > - if (ext4_handle_valid(handle)) { > - err = jbd2_journal_get_create_access(handle, bh); > - if (err) > - ext4_journal_abort_handle(where, line, __func__, > - bh, handle, err); > + if (!ext4_handle_valid(handle)) > + return 0; > + > + err = jbd2_journal_get_create_access(handle, bh); > + if (err) { > + ext4_journal_abort_handle(where, line, __func__, bh, handle, > + err); > + return err; > } > - return err; > + if (trigger_type == TR_NONE || !ext4_has_metadata_csum(sb)) > + return 0; Same. > + jbd2_journal_set_triggers(bh, > + &EXT4_SB(sb)->s_journal_triggers[trigger_type].tr_triggers); > + return 0; > } > > int __ext4_handle_dirty_metadata(const char *where, unsigned int line, > diff --git a/fs/ext4/ext4_jbd2.h b/fs/ext4/ext4_jbd2.h > index 9c5b49fb281e..c5ec6de3220e 100644 > --- a/fs/ext4/ext4_jbd2.h > +++ b/fs/ext4/ext4_jbd2.h > @@ -232,14 +232,18 @@ int ext4_mark_inode_dirty(handle_t *handle, struct inode *inode); > * Wrapper functions with which ext4 calls into JBD. > */ > int __ext4_journal_get_write_access(const char *where, unsigned int line, > - handle_t *handle, struct buffer_head *bh); > + handle_t *handle, struct super_block *sb, > + struct buffer_head *bh, > + enum ext4_journal_trigger_type trigger_type); > > int __ext4_forget(const char *where, unsigned int line, handle_t *handle, > int is_metadata, struct inode *inode, > struct buffer_head *bh, ext4_fsblk_t blocknr); > > int __ext4_journal_get_create_access(const char *where, unsigned int line, > - handle_t *handle, struct buffer_head *bh); > + handle_t *handle, struct super_block *sb, > + struct buffer_head *bh, > + enum ext4_journal_trigger_type trigger_type); > > int __ext4_handle_dirty_metadata(const char *where, unsigned int line, > handle_t *handle, struct inode *inode, > @@ -248,13 +252,15 @@ int __ext4_handle_dirty_metadata(const char *where, unsigned int line, > int __ext4_handle_dirty_super(const char *where, unsigned int line, > handle_t *handle, struct super_block *sb); > > -#define ext4_journal_get_write_access(handle, bh) \ > - __ext4_journal_get_write_access(__func__, __LINE__, (handle), (bh)) > +#define ext4_journal_get_write_access(handle, sb, bh, trigger_type) \ > + __ext4_journal_get_write_access(__func__, __LINE__, (handle), (sb), \ > + (bh), (trigger_type)) > #define ext4_forget(handle, is_metadata, inode, bh, block_nr) \ > __ext4_forget(__func__, __LINE__, (handle), (is_metadata), (inode), \ > (bh), (block_nr)) > -#define ext4_journal_get_create_access(handle, bh) \ > - __ext4_journal_get_create_access(__func__, __LINE__, (handle), (bh)) > +#define ext4_journal_get_create_access(handle, sb, bh, trigger_type) \ > + __ext4_journal_get_create_access(__func__, __LINE__, (handle), (sb), \ > + (bh), (trigger_type)) > #define ext4_handle_dirty_metadata(handle, inode, bh) \ > __ext4_handle_dirty_metadata(__func__, __LINE__, (handle), (inode), \ > (bh)) > diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c > index bed43081720f..1c5d29efe585 100644 > --- a/fs/ext4/extents.c > +++ b/fs/ext4/extents.c > @@ -142,7 +142,8 @@ static int ext4_ext_get_access(handle_t *handle, struct inode *inode, > if (path->p_bh) { > /* path points to block */ > BUFFER_TRACE(path->p_bh, "get_write_access"); > - return ext4_journal_get_write_access(handle, path->p_bh); > + return ext4_journal_get_write_access(handle, inode->i_sb, > + path->p_bh, TR_NONE); > } > /* path points to leaf/index in inode body */ > /* we use in-core data, no need to protect them */ > @@ -1095,7 +1096,7 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode, > } > lock_buffer(bh); > > - err = ext4_journal_get_create_access(handle, bh); > + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, TR_NONE); > if (err) > goto cleanup; > > @@ -1168,7 +1169,8 @@ static int ext4_ext_split(handle_t *handle, struct inode *inode, > } > lock_buffer(bh); > > - err = ext4_journal_get_create_access(handle, bh); > + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, > + TR_NONE); > if (err) > goto cleanup; > > @@ -1287,7 +1289,7 @@ static int ext4_ext_grow_indepth(handle_t *handle, struct inode *inode, > return -ENOMEM; > lock_buffer(bh); > > - err = ext4_journal_get_create_access(handle, bh); > + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, TR_NONE); > if (err) { > unlock_buffer(bh); > goto out; > diff --git a/fs/ext4/file.c b/fs/ext4/file.c > index 33a09da16c9c..39e9b39f3c1c 100644 > --- a/fs/ext4/file.c > +++ b/fs/ext4/file.c > @@ -258,7 +258,8 @@ static int ext4_file_open(struct inode * inode, struct file * filp) > if (IS_ERR(handle)) > return PTR_ERR(handle); > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, sbi->s_sbh); > + err = ext4_journal_get_write_access(handle, sb, > + sbi->s_sbh, TR_NONE); > if (err) { > ext4_journal_stop(handle); > return err; > diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c > index ac644c31ca67..7a711cf0d183 100644 > --- a/fs/ext4/ialloc.c > +++ b/fs/ext4/ialloc.c > @@ -291,7 +291,7 @@ void ext4_free_inode(handle_t *handle, struct inode *inode) > goto error_return; > > BUFFER_TRACE(bitmap_bh, "get_write_access"); > - fatal = ext4_journal_get_write_access(handle, bitmap_bh); > + fatal = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); > if (fatal) > goto error_return; > > @@ -299,7 +299,7 @@ void ext4_free_inode(handle_t *handle, struct inode *inode) > gdp = ext4_get_group_desc(sb, block_group, &bh2); > if (gdp) { > BUFFER_TRACE(bh2, "get_write_access"); > - fatal = ext4_journal_get_write_access(handle, bh2); > + fatal = ext4_journal_get_write_access(handle, sb, bh2, TR_NONE); > } > ext4_lock_group(sb, block_group); > cleared = ext4_test_and_clear_bit(bit, bitmap_bh->b_data); > @@ -845,7 +845,8 @@ repeat_in_this_group: > } > } > BUFFER_TRACE(inode_bitmap_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, inode_bitmap_bh); > + err = ext4_journal_get_write_access(handle, sb, inode_bitmap_bh, > + TR_NONE); > if (err) { > ext4_std_error(sb, err); > goto out; > @@ -875,7 +876,7 @@ got: > } > > BUFFER_TRACE(group_desc_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, group_desc_bh); > + err = ext4_journal_get_write_access(handle, sb, group_desc_bh, TR_NONE); > if (err) { > ext4_std_error(sb, err); > goto out; > @@ -892,7 +893,8 @@ got: > goto out; > } > BUFFER_TRACE(block_bitmap_bh, "get block bitmap access"); > - err = ext4_journal_get_write_access(handle, block_bitmap_bh); > + err = ext4_journal_get_write_access(handle, sb, block_bitmap_bh, > + TR_NONE); > if (err) { > brelse(block_bitmap_bh); > ext4_std_error(sb, err); > @@ -1288,8 +1290,7 @@ int ext4_init_inode_table(struct super_block *sb, ext4_group_t group, > num = sbi->s_itb_per_group - used_blks; > > BUFFER_TRACE(group_desc_bh, "get_write_access"); > - ret = ext4_journal_get_write_access(handle, > - group_desc_bh); > + ret = ext4_journal_get_write_access(handle, sb, group_desc_bh, TR_NONE); > if (ret) > goto err_out; > > diff --git a/fs/ext4/indirect.c b/fs/ext4/indirect.c > index 45fe924f82bc..54adceeb09aa 100644 > --- a/fs/ext4/indirect.c > +++ b/fs/ext4/indirect.c > @@ -351,7 +351,8 @@ static int ext4_alloc_branch(handle_t *handle, > } > lock_buffer(bh); > BUFFER_TRACE(bh, "call get_create_access"); > - err = ext4_journal_get_create_access(handle, bh); > + err = ext4_journal_get_create_access(handle, ar->inode->i_sb, > + bh, TR_NONE); > if (err) { > unlock_buffer(bh); > goto failed; > @@ -423,7 +424,8 @@ static int ext4_splice_branch(handle_t *handle, > */ > if (where->bh) { > BUFFER_TRACE(where->bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, where->bh); > + err = ext4_journal_get_write_access(handle, ar->inode->i_sb, > + where->bh, TR_NONE); > if (err) > goto err_out; > } > @@ -967,7 +969,8 @@ static int ext4_clear_blocks(handle_t *handle, struct inode *inode, > goto out_err; > if (bh) { > BUFFER_TRACE(bh, "retaking write access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, inode->i_sb, > + bh, TR_NONE); > if (unlikely(err)) > goto out_err; > } > @@ -1018,7 +1021,8 @@ static void ext4_free_data(handle_t *handle, struct inode *inode, > > if (this_bh) { /* For indirect block */ > BUFFER_TRACE(this_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, this_bh); > + err = ext4_journal_get_write_access(handle, inode->i_sb, > + this_bh, TR_NONE); > /* Important: if we can't update the indirect pointers > * to the blocks, we can't free them. */ > if (err) > @@ -1182,7 +1186,9 @@ static void ext4_free_branches(handle_t *handle, struct inode *inode, > */ > BUFFER_TRACE(parent_bh, "get_write_access"); > if (!ext4_journal_get_write_access(handle, > - parent_bh)){ > + inode->i_sb, > + parent_bh, > + TR_NONE)){ > *p = 0; > BUFFER_TRACE(parent_bh, > "call ext4_handle_dirty_metadata"); > diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c > index 4b143febf21f..d4879c2396ce 100644 > --- a/fs/ext4/inline.c > +++ b/fs/ext4/inline.c > @@ -259,7 +259,8 @@ static int ext4_create_inline_data(handle_t *handle, > return error; > > BUFFER_TRACE(is.iloc.bh, "get_write_access"); > - error = ext4_journal_get_write_access(handle, is.iloc.bh); > + error = ext4_journal_get_write_access(handle, inode->i_sb, is.iloc.bh, > + TR_NONE); > if (error) > goto out; > > @@ -343,7 +344,8 @@ static int ext4_update_inline_data(handle_t *handle, struct inode *inode, > goto out; > > BUFFER_TRACE(is.iloc.bh, "get_write_access"); > - error = ext4_journal_get_write_access(handle, is.iloc.bh); > + error = ext4_journal_get_write_access(handle, inode->i_sb, is.iloc.bh, > + TR_NONE); > if (error) > goto out; > > @@ -421,7 +423,8 @@ static int ext4_destroy_inline_data_nolock(handle_t *handle, > goto out; > > BUFFER_TRACE(is.iloc.bh, "get_write_access"); > - error = ext4_journal_get_write_access(handle, is.iloc.bh); > + error = ext4_journal_get_write_access(handle, inode->i_sb, is.iloc.bh, > + TR_NONE); > if (error) > goto out; > > @@ -586,7 +589,7 @@ retry: > ret = __block_write_begin(page, from, to, ext4_get_block); > > if (!ret && ext4_should_journal_data(inode)) { > - ret = ext4_walk_page_buffers(handle, page_buffers(page), > + ret = ext4_walk_page_buffers(handle, inode, page_buffers(page), > from, to, NULL, > do_journal_get_write_access); > } > @@ -1011,7 +1014,8 @@ static int ext4_add_dirent_to_inline(handle_t *handle, > return err; > > BUFFER_TRACE(iloc->bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, iloc->bh); > + err = ext4_journal_get_write_access(handle, dir->i_sb, iloc->bh, > + TR_NONE); > if (err) > return err; > ext4_insert_dentry(inode, de, inline_size, name, namelen); > @@ -1211,7 +1215,8 @@ static int ext4_convert_inline_data_nolock(handle_t *handle, > } > > lock_buffer(data_bh); > - error = ext4_journal_get_create_access(handle, data_bh); > + error = ext4_journal_get_create_access(handle, inode->i_sb, data_bh, > + TR_NONE); > if (error) { > unlock_buffer(data_bh); > error = -EIO; > @@ -1685,7 +1690,7 @@ int ext4_delete_inline_entry(handle_t *handle, > } > > BUFFER_TRACE(bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); > if (err) > goto out; > > diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c > index 5cb9a212b86f..0813e6a9ca98 100644 > --- a/fs/ext4/inode.c > +++ b/fs/ext4/inode.c > @@ -134,7 +134,6 @@ static inline int ext4_begin_ordered_truncate(struct inode *inode, > static void ext4_invalidatepage(struct page *page, unsigned int offset, > unsigned int length); > static int __ext4_journalled_writepage(struct page *page, unsigned int len); > -static int ext4_bh_delay_or_unwritten(handle_t *handle, struct buffer_head *bh); > static int ext4_meta_trans_blocks(struct inode *inode, int lblocks, > int pextents); > > @@ -766,7 +765,8 @@ struct buffer_head *ext4_getblk(handle_t *handle, struct inode *inode, > */ > lock_buffer(bh); > BUFFER_TRACE(bh, "call get_create_access"); > - err = ext4_journal_get_create_access(handle, bh); > + err = ext4_journal_get_create_access(handle, inode->i_sb, bh, > + TR_NONE); > if (unlikely(err)) { > unlock_buffer(bh); > goto errout; > @@ -806,12 +806,12 @@ struct buffer_head *ext4_bread(handle_t *handle, struct inode *inode, > return ERR_PTR(-EIO); > } > > -int ext4_walk_page_buffers(handle_t *handle, > +int ext4_walk_page_buffers(handle_t *handle, struct inode *inode, > struct buffer_head *head, > unsigned from, > unsigned to, > int *partial, > - int (*fn)(handle_t *handle, > + int (*fn)(handle_t *handle, struct inode *inode, > struct buffer_head *bh)) > { > struct buffer_head *bh; > @@ -830,7 +830,7 @@ int ext4_walk_page_buffers(handle_t *handle, > *partial = 1; > continue; > } > - err = (*fn)(handle, bh); > + err = (*fn)(handle, inode, bh); > if (!ret) > ret = err; > } > @@ -861,7 +861,7 @@ int ext4_walk_page_buffers(handle_t *handle, > * is elevated. We'll still have enough credits for the tiny quotafile > * write. > */ > -int do_journal_get_write_access(handle_t *handle, > +int do_journal_get_write_access(handle_t *handle, struct inode *inode, > struct buffer_head *bh) > { > int dirty = buffer_dirty(bh); > @@ -880,7 +880,7 @@ int do_journal_get_write_access(handle_t *handle, > if (dirty) > clear_buffer_dirty(bh); > BUFFER_TRACE(bh, "get write access"); > - ret = ext4_journal_get_write_access(handle, bh); > + ret = ext4_journal_get_write_access(handle, inode->i_sb, bh, TR_NONE); > if (!ret && dirty) > ret = ext4_handle_dirty_metadata(handle, NULL, bh); > return ret; > @@ -956,8 +956,8 @@ retry_journal: > ret = __block_write_begin(page, pos, len, ext4_get_block); > > if (!ret && ext4_should_journal_data(inode)) { > - ret = ext4_walk_page_buffers(handle, page_buffers(page), > - from, to, NULL, > + ret = ext4_walk_page_buffers(handle, inode, > + page_buffers(page), from, to, NULL, > do_journal_get_write_access); > } > > @@ -998,7 +998,8 @@ retry_journal: > } > > /* For write_end() in data=journal mode */ > -static int write_end_fn(handle_t *handle, struct buffer_head *bh) > +static int write_end_fn(handle_t *handle, struct inode *inode, > + struct buffer_head *bh) > { > int ret; > if (!buffer_mapped(bh) || buffer_freed(bh)) > @@ -1120,8 +1121,8 @@ static int ext4_journalled_write_end(struct file *file, > page_zero_new_buffers(page, from+copied, to); > } > > - ret = ext4_walk_page_buffers(handle, page_buffers(page), from, > - to, &partial, write_end_fn); > + ret = ext4_walk_page_buffers(handle, inode, page_buffers(page), > + from, to, &partial, write_end_fn); > if (!partial) > SetPageUptodate(page); > } > @@ -1377,7 +1378,8 @@ static void ext4_print_free_blocks(struct inode *inode) > return; > } > > -static int ext4_bh_delay_or_unwritten(handle_t *handle, struct buffer_head *bh) > +static int ext4_bh_delay_or_unwritten(handle_t *handle, struct inode *inode, > + struct buffer_head *bh) > { > return (buffer_delay(bh) || buffer_unwritten(bh)) && buffer_dirty(bh); > } > @@ -1565,13 +1567,15 @@ int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, > return 0; > } > > -static int bget_one(handle_t *handle, struct buffer_head *bh) > +static int bget_one(handle_t *handle, struct inode *inode, > + struct buffer_head *bh) > { > get_bh(bh); > return 0; > } > > -static int bput_one(handle_t *handle, struct buffer_head *bh) > +static int bput_one(handle_t *handle, struct inode *inode, > + struct buffer_head *bh) > { > put_bh(bh); > return 0; > @@ -1602,7 +1606,7 @@ static int __ext4_journalled_writepage(struct page *page, > BUG(); > goto out; > } > - ext4_walk_page_buffers(handle, page_bufs, 0, len, > + ext4_walk_page_buffers(handle, inode, page_bufs, 0, len, > NULL, bget_one); > } > /* As soon as we unlock the page, it can go away, but we have > @@ -1620,16 +1624,17 @@ static int __ext4_journalled_writepage(struct page *page, > > if (inline_data) { > BUFFER_TRACE(inode_bh, "get write access"); > - ret = ext4_journal_get_write_access(handle, inode_bh); > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > + inode_bh, TR_NONE); > > err = ext4_handle_dirty_metadata(handle, inode, inode_bh); > > } else { > - ret = ext4_walk_page_buffers(handle, page_bufs, 0, len, NULL, > - do_journal_get_write_access); > + ret = ext4_walk_page_buffers(handle, inode, page_bufs, 0, len, > + NULL, do_journal_get_write_access); > > - err = ext4_walk_page_buffers(handle, page_bufs, 0, len, NULL, > - write_end_fn); > + err = ext4_walk_page_buffers(handle, inode, page_bufs, 0, len, > + NULL, write_end_fn); > } > if (ret == 0) > ret = err; > @@ -1639,7 +1644,7 @@ static int __ext4_journalled_writepage(struct page *page, > ret = err; > > if (!ext4_has_inline_data(inode)) > - ext4_walk_page_buffers(NULL, page_bufs, 0, len, > + ext4_walk_page_buffers(NULL, inode, page_bufs, 0, len, > NULL, bput_one); > ext4_set_inode_state(inode, EXT4_STATE_JDATA); > out: > @@ -1714,7 +1719,7 @@ static int ext4_writepage(struct page *page, > * journal_submit_inode_data_buffers() and in that case we must write > * allocated buffers to achieve data=ordered mode guarantees. > */ > - if (ext4_walk_page_buffers(NULL, page_bufs, 0, len, NULL, > + if (ext4_walk_page_buffers(NULL, inode, page_bufs, 0, len, NULL, > ext4_bh_delay_or_unwritten)) { > redirty_page_for_writepage(wbc, page); > if (current->flags & PF_MEMALLOC) { > @@ -3265,7 +3270,8 @@ static int __ext4_block_zero_page_range(handle_t *handle, > } > if (ext4_should_journal_data(inode)) { > BUFFER_TRACE(bh, "get write access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, inode->i_sb, bh, > + TR_NONE); > if (err) > goto unlock; > } > @@ -4362,7 +4368,9 @@ static int ext4_do_update_inode(handle_t *handle, > ext4_clear_inode_state(inode, EXT4_STATE_NEW); > if (set_large_file) { > BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get write access"); > - err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh); > + err = ext4_journal_get_write_access(handle, sb, > + EXT4_SB(sb)->s_sbh, > + TR_NONE); > if (err) > goto out_brelse; > ext4_update_dynamic_rev(sb); > @@ -4828,7 +4836,8 @@ ext4_reserve_inode_write(handle_t *handle, struct inode *inode, > err = ext4_get_inode_loc(inode, iloc); > if (!err) { > BUFFER_TRACE(iloc->bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, iloc->bh); > + err = ext4_journal_get_write_access(handle, inode->i_sb, > + iloc->bh, TR_NONE); > if (err) { > brelse(iloc->bh); > iloc->bh = NULL; > @@ -4981,7 +4990,8 @@ static int ext4_pin_inode(handle_t *handle, struct inode *inode) > err = ext4_get_inode_loc(inode, &iloc); > if (!err) { > BUFFER_TRACE(iloc.bh, "get_write_access"); > - err = jbd2_journal_get_write_access(handle, iloc.bh); > + err = jbd2_journal_get_write_access(inode->i_sb, handle, > + iloc.bh, TR_NONE); > if (!err) > err = ext4_handle_dirty_metadata(handle, > NULL, > @@ -5071,7 +5081,8 @@ int ext4_change_inode_journal_flag(struct inode *inode, int val) > return err; > } > > -static int ext4_bh_unmapped(handle_t *handle, struct buffer_head *bh) > +static int ext4_bh_unmapped(handle_t *handle, struct inode *inode, > + struct buffer_head *bh) > { > return !buffer_mapped(bh); > } > @@ -5121,7 +5132,7 @@ int ext4_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf) > * journal_start/journal_stop which can block and take a long time > */ > if (page_has_buffers(page)) { > - if (!ext4_walk_page_buffers(NULL, page_buffers(page), > + if (!ext4_walk_page_buffers(NULL, inode, page_buffers(page), > 0, len, NULL, > ext4_bh_unmapped)) { > /* Wait so that we don't change page under IO */ > @@ -5145,7 +5156,7 @@ retry_alloc: > } > ret = __block_page_mkwrite(vma, vmf, get_block); > if (!ret && ext4_should_journal_data(inode)) { > - if (ext4_walk_page_buffers(handle, page_buffers(page), 0, > + if (ext4_walk_page_buffers(handle, inode, page_buffers(page), 0, > PAGE_CACHE_SIZE, NULL, do_journal_get_write_access)) { > unlock_page(page); > ret = VM_FAULT_SIGBUS; > diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c > index 8d1e60214ef0..dcd05c5f6c48 100644 > --- a/fs/ext4/mballoc.c > +++ b/fs/ext4/mballoc.c > @@ -2886,7 +2886,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac, > goto out_err; > > BUFFER_TRACE(bitmap_bh, "getting write access"); > - err = ext4_journal_get_write_access(handle, bitmap_bh); > + err = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); > if (err) > goto out_err; > > @@ -2899,7 +2899,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac, > ext4_free_group_clusters(sb, gdp)); > > BUFFER_TRACE(gdp_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, gdp_bh); > + err = ext4_journal_get_write_access(handle, sb, gdp_bh, TR_NONE); > if (err) > goto out_err; > > @@ -4769,7 +4769,7 @@ do_more: > } > > BUFFER_TRACE(bitmap_bh, "getting write access"); > - err = ext4_journal_get_write_access(handle, bitmap_bh); > + err = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); > if (err) > goto error_return; > > @@ -4779,7 +4779,7 @@ do_more: > * using it > */ > BUFFER_TRACE(gd_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, gd_bh); > + err = ext4_journal_get_write_access(handle, sb, gd_bh, TR_NONE); > if (err) > goto error_return; > #ifdef AGGRESSIVE_CHECK > @@ -4945,7 +4945,7 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb, > } > > BUFFER_TRACE(bitmap_bh, "getting write access"); > - err = ext4_journal_get_write_access(handle, bitmap_bh); > + err = ext4_journal_get_write_access(handle, sb, bitmap_bh, TR_NONE); > if (err) > goto error_return; > > @@ -4955,7 +4955,7 @@ int ext4_group_add_blocks(handle_t *handle, struct super_block *sb, > * using it > */ > BUFFER_TRACE(gd_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, gd_bh); > + err = ext4_journal_get_write_access(handle, sb, gd_bh, TR_NONE); > if (err) > goto error_return; > > diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c > index 28fe71a2904c..460c716e38b0 100644 > --- a/fs/ext4/namei.c > +++ b/fs/ext4/namei.c > @@ -68,7 +68,7 @@ static struct buffer_head *ext4_append(handle_t *handle, > inode->i_size += inode->i_sb->s_blocksize; > EXT4_I(inode)->i_disksize = inode->i_size; > BUFFER_TRACE(bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, inode->i_sb, bh, TR_NONE); > if (err) { > brelse(bh); > ext4_std_error(inode->i_sb, err); > @@ -1528,12 +1528,13 @@ static struct ext4_dir_entry_2 *do_split(handle_t *handle, struct inode *dir, > } > > BUFFER_TRACE(*bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, *bh); > + err = ext4_journal_get_write_access(handle, dir->i_sb, *bh, TR_NONE); > if (err) > goto journal_error; > > BUFFER_TRACE(frame->bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, frame->bh); > + err = ext4_journal_get_write_access(handle, dir->i_sb, frame->bh, > + TR_NONE); > if (err) > goto journal_error; > > @@ -1694,7 +1695,7 @@ static int add_dirent_to_buf(handle_t *handle, struct dentry *dentry, > return err; > } > BUFFER_TRACE(bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); > if (err) { > ext4_std_error(dir->i_sb, err); > return err; > @@ -1756,7 +1757,7 @@ static int make_indexed_dir(handle_t *handle, struct dentry *dentry, > blocksize = dir->i_sb->s_blocksize; > dxtrace(printk(KERN_DEBUG "Creating index: inode %lu\n", dir->i_ino)); > BUFFER_TRACE(bh, "get_write_access"); > - retval = ext4_journal_get_write_access(handle, bh); > + retval = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); > if (retval) { > ext4_std_error(dir->i_sb, retval); > brelse(bh); > @@ -1965,7 +1966,7 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, > } > > BUFFER_TRACE(bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE); > if (err) > goto journal_error; > > @@ -2002,7 +2003,8 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, > node2->fake.rec_len = ext4_rec_len_to_disk(sb->s_blocksize, > sb->s_blocksize); > BUFFER_TRACE(frame->bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, frame->bh); > + err = ext4_journal_get_write_access(handle, sb, frame->bh, > + TR_NONE); > if (err) > goto journal_error; > if (levels) { > @@ -2012,8 +2014,9 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, > icount1, icount2)); > > BUFFER_TRACE(frame->bh, "get_write_access"); /* index root */ > - err = ext4_journal_get_write_access(handle, > - frames[0].bh); > + err = ext4_journal_get_write_access(handle, sb, > + frames[0].bh, > + TR_NONE); > if (err) > goto journal_error; > > @@ -2054,8 +2057,9 @@ static int ext4_dx_add_entry(handle_t *handle, struct dentry *dentry, > frame->at = at = at - entries + entries2; > frame->entries = entries = entries2; > frame->bh = bh2; > - err = ext4_journal_get_write_access(handle, > - frame->bh); > + err = ext4_journal_get_write_access(handle, sb, > + frame->bh, > + TR_NONE); > if (err) > goto journal_error; > } > @@ -2143,7 +2147,7 @@ static int ext4_delete_entry(handle_t *handle, > csum_size = sizeof(struct ext4_dir_entry_tail); > > BUFFER_TRACE(bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, dir->i_sb, bh, TR_NONE); > if (unlikely(err)) > goto out; > > @@ -2567,7 +2571,7 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) > S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); > > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, sbi->s_sbh); > + err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); > if (err) > goto out; > > @@ -2660,7 +2664,8 @@ int ext4_orphan_del(handle_t *handle, struct inode *inode) > if (prev == &sbi->s_orphan) { > jbd_debug(4, "superblock will point to %u\n", ino_next); > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, sbi->s_sbh); > + err = ext4_journal_get_write_access(handle, inode->i_sb, > + sbi->s_sbh, TR_NONE); > if (err) { > mutex_unlock(&sbi->s_orphan_lock); > goto out_brelse; > @@ -3039,7 +3044,8 @@ static int ext4_rename_dir_prepare(handle_t *handle, struct ext4_renament *ent) > if (le32_to_cpu(ent->parent_de->inode) != ent->dir->i_ino) > return -EIO; > BUFFER_TRACE(ent->dir_bh, "get_write_access"); > - return ext4_journal_get_write_access(handle, ent->dir_bh); > + return ext4_journal_get_write_access(handle, ent->dir->i_sb, > + ent->dir_bh, TR_NONE); > } > > static int ext4_rename_dir_finish(handle_t *handle, struct ext4_renament *ent, > @@ -3075,7 +3081,8 @@ static int ext4_setent(handle_t *handle, struct ext4_renament *ent, > int retval; > > BUFFER_TRACE(ent->bh, "get write access"); > - retval = ext4_journal_get_write_access(handle, ent->bh); > + retval = ext4_journal_get_write_access(handle, ent->dir->i_sb, ent->bh, > + TR_NONE); > if (retval) > return retval; > ent->de->inode = cpu_to_le32(ino); > diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c > index 8a8ec6293b19..f14ddaa96aa9 100644 > --- a/fs/ext4/resize.c > +++ b/fs/ext4/resize.c > @@ -361,7 +361,7 @@ static struct buffer_head *bclean(handle_t *handle, struct super_block *sb, > if (unlikely(!bh)) > return ERR_PTR(-ENOMEM); > BUFFER_TRACE(bh, "get_write_access"); > - if ((err = ext4_journal_get_write_access(handle, bh))) { > + if ((err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE))) { > brelse(bh); > bh = ERR_PTR(err); > } else { > @@ -440,7 +440,7 @@ static int set_flexbg_block_bitmap(struct super_block *sb, handle_t *handle, > return -ENOMEM; > > BUFFER_TRACE(bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE); > if (err) > return err; > ext4_debug("mark block bitmap %#04llx (+%llu/%u)\n", block, > @@ -533,7 +533,8 @@ static int setup_new_flex_group_blocks(struct super_block *sb, > } > > BUFFER_TRACE(gdb, "get_write_access"); > - err = ext4_journal_get_write_access(handle, gdb); > + err = ext4_journal_get_write_access(handle, sb, gdb, > + TR_NONE); > if (err) { > brelse(gdb); > goto out; > @@ -796,17 +797,18 @@ static int add_new_gdb(handle_t *handle, struct inode *inode, > } > > BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh); > + err = ext4_journal_get_write_access(handle, sb, EXT4_SB(sb)->s_sbh, > + TR_NONE); > if (unlikely(err)) > goto exit_dind; > > BUFFER_TRACE(gdb_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, gdb_bh); > + err = ext4_journal_get_write_access(handle, sb, gdb_bh, TR_NONE); > if (unlikely(err)) > goto exit_dind; > > BUFFER_TRACE(dind, "get_write_access"); > - err = ext4_journal_get_write_access(handle, dind); > + err = ext4_journal_get_write_access(handle, sb, dind, TR_NONE); > if (unlikely(err)) > ext4_std_error(sb, err); > > @@ -911,7 +913,7 @@ static int add_new_gdb_meta_bg(struct super_block *sb, > EXT4_SB(sb)->s_gdb_count++; > kvfree(o_group_desc); > BUFFER_TRACE(gdb_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, gdb_bh); > + err = ext4_journal_get_write_access(handle, sb, gdb_bh, TR_NONE); > if (unlikely(err)) > brelse(gdb_bh); > return err; > @@ -987,7 +989,8 @@ static int reserve_backup_gdb(handle_t *handle, struct inode *inode, > > for (i = 0; i < reserved_gdb; i++) { > BUFFER_TRACE(primary[i], "get_write_access"); > - if ((err = ext4_journal_get_write_access(handle, primary[i]))) > + if ((err = ext4_journal_get_write_access(handle, sb, primary[i], > + TR_NONE))) > goto exit_bh; > } > > @@ -1095,7 +1098,8 @@ static void update_backups(struct super_block *sb, int blk_off, char *data, > backup_block, backup_block - > ext4_group_first_block_no(sb, group)); > BUFFER_TRACE(bh, "get_write_access"); > - if ((err = ext4_journal_get_write_access(handle, bh))) > + if ((err = ext4_journal_get_write_access(handle, sb, bh, > + TR_NONE))) > break; > lock_buffer(bh); > memcpy(bh->b_data, data, size); > @@ -1175,7 +1179,8 @@ static int ext4_add_new_descs(handle_t *handle, struct super_block *sb, > if (gdb_off) { > gdb_bh = sbi->s_group_desc[gdb_num]; > BUFFER_TRACE(gdb_bh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, gdb_bh); > + err = ext4_journal_get_write_access(handle, sb, gdb_bh, > + TR_NONE); > > if (!err && reserved_gdb && ext4_bg_num_gdb(sb, group)) > err = reserve_backup_gdb(handle, resize_inode, group); > @@ -1445,7 +1450,7 @@ static int ext4_flex_group_add(struct super_block *sb, > } > > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, sbi->s_sbh); > + err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); > if (err) > goto exit_journal; > > @@ -1658,7 +1663,8 @@ static int ext4_group_extend_no_check(struct super_block *sb, > } > > BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh); > + err = ext4_journal_get_write_access(handle, sb, EXT4_SB(sb)->s_sbh, > + TR_NONE); > if (err) { > ext4_warning(sb, "error %d on journal write access", err); > goto errout; > @@ -1818,7 +1824,7 @@ static int ext4_convert_meta_bg(struct super_block *sb, struct inode *inode) > return PTR_ERR(handle); > > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > - err = ext4_journal_get_write_access(handle, sbi->s_sbh); > + err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); > if (err) > goto errout; > > diff --git a/fs/ext4/super.c b/fs/ext4/super.c > index e061e66c8280..0babe8c435b6 100644 > --- a/fs/ext4/super.c > +++ b/fs/ext4/super.c > @@ -3406,6 +3406,20 @@ static int ext4_reserve_clusters(struct ext4_sb_info *sbi, ext4_fsblk_t count) > return 0; > } > > +static void ext4_setup_csum_trigger(struct super_block *sb, > + enum ext4_journal_trigger_type type, > + void (*trigger)( > + struct jbd2_buffer_trigger_type *type, > + struct buffer_head *bh, > + void *mapped_data, > + size_t size)) > +{ > + struct ext4_sb_info *sbi = EXT4_SB(sb); > + > + sbi->s_journal_triggers[type].sb = sb; > + sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; > +} > + > static int ext4_fill_super(struct super_block *sb, void *data, int silent) > { > char *orig_data = kstrdup(data, GFP_KERNEL); > @@ -5416,7 +5430,7 @@ static ssize_t ext4_quota_write(struct super_block *sb, int type, > if (!bh) > goto out; > BUFFER_TRACE(bh, "get write access"); > - err = ext4_journal_get_write_access(handle, bh); > + err = ext4_journal_get_write_access(handle, sb, bh, TR_NONE); > if (err) { > brelse(bh); > return err; > diff --git a/fs/ext4/xattr.c b/fs/ext4/xattr.c > index 1e09fc77395c..a0a285844037 100644 > --- a/fs/ext4/xattr.c > +++ b/fs/ext4/xattr.c > @@ -530,7 +530,8 @@ static void ext4_xattr_update_super_block(handle_t *handle, > return; > > BUFFER_TRACE(EXT4_SB(sb)->s_sbh, "get_write_access"); > - if (ext4_journal_get_write_access(handle, EXT4_SB(sb)->s_sbh) == 0) { > + if (ext4_journal_get_write_access(handle, sb, EXT4_SB(sb)->s_sbh, > + TR_NONE) == 0) { > EXT4_SET_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_EXT_ATTR); > ext4_handle_dirty_super(handle, sb); > } > @@ -550,7 +551,7 @@ ext4_xattr_release_block(handle_t *handle, struct inode *inode, > > ce = mb_cache_entry_get(ext4_mb_cache, bh->b_bdev, bh->b_blocknr); > BUFFER_TRACE(bh, "get_write_access"); > - error = ext4_journal_get_write_access(handle, bh); > + error = ext4_journal_get_write_access(handle, inode->i_sb, bh, TR_NONE); > if (error) > goto out; > > @@ -793,7 +794,8 @@ ext4_xattr_block_set(handle_t *handle, struct inode *inode, > ce = mb_cache_entry_get(ext4_mb_cache, bs->bh->b_bdev, > bs->bh->b_blocknr); > BUFFER_TRACE(bs->bh, "get_write_access"); > - error = ext4_journal_get_write_access(handle, bs->bh); > + error = ext4_journal_get_write_access(handle, sb, bs->bh, > + TR_NONE); > if (error) > goto cleanup; > lock_buffer(bs->bh); > @@ -880,7 +882,9 @@ inserted: > goto cleanup; > BUFFER_TRACE(new_bh, "get_write_access"); > error = ext4_journal_get_write_access(handle, > - new_bh); > + sb, > + new_bh, > + TR_NONE); > if (error) > goto cleanup_dquot; > lock_buffer(new_bh); > @@ -932,7 +936,8 @@ getblk_failed: > goto cleanup; > } > lock_buffer(new_bh); > - error = ext4_journal_get_create_access(handle, new_bh); > + error = ext4_journal_get_create_access(handle, sb, > + new_bh, TR_NONE); > if (error) { > unlock_buffer(new_bh); > error = -EIO; > -- > 2.1.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Cheers, Andreas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/3] ext4: Support for checksumming from journal triggers 2015-04-17 19:00 ` Andreas Dilger @ 2015-04-20 9:07 ` Jan Kara 0 siblings, 0 replies; 18+ messages in thread From: Jan Kara @ 2015-04-20 9:07 UTC (permalink / raw) To: Andreas Dilger; +Cc: Jan Kara, linux-ext4 On Fri 17-04-15 13:00:32, Andreas Dilger wrote: > On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: > > > > JBD2 layer support triggers which are called when journaling layer moves > > buffer to a certain state. We can use the frozen trigger, which gets > > called when buffer data is frozen and about to be written out to the > > journal, to compute block checksums for some buffer types (similarly as > > does ocfs2). This avoids unnecessary repeated recomputation of the > > checksum (at the cost of larger window where memory corruption won't be > > caught by checksumming) and is even necessary when there are > > unsynchronized updaters of the checksummed data. > > > > So add argument to ext4_journal_get_write_access() and > > ext4_journal_get_create_access() which describes buffer type so that > > triggers can be set accordingly. This patch is mostly only a change of > > prototype of the above mentioned functions and a few small helpers. Real > > checksumming will come later. > > > > Signed-off-by: Jan Kara <jack@suse.cz> > > --- > > fs/ext4/ext4.h | 26 ++++++++++++++++++-- > > fs/ext4/ext4_jbd2.c | 50 +++++++++++++++++++++++++------------ > > fs/ext4/ext4_jbd2.h | 18 +++++++++----- > > fs/ext4/extents.c | 10 +++++--- > > fs/ext4/file.c | 3 ++- > > fs/ext4/ialloc.c | 15 +++++------ > > fs/ext4/indirect.c | 16 ++++++++---- > > fs/ext4/inline.c | 19 ++++++++------ > > fs/ext4/inode.c | 71 +++++++++++++++++++++++++++++++---------------------- > > fs/ext4/mballoc.c | 12 ++++----- > > fs/ext4/namei.c | 39 +++++++++++++++++------------ > > fs/ext4/resize.c | 32 ++++++++++++++---------- > > fs/ext4/super.c | 16 +++++++++++- > > fs/ext4/xattr.c | 15 +++++++---- > > 14 files changed, 223 insertions(+), 119 deletions(-) > > > > diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > > index f63c3d5805c4..abed83485915 100644 > > --- a/fs/ext4/ext4.h > > +++ b/fs/ext4/ext4.h > > @@ -1186,6 +1186,24 @@ struct ext4_super_block { > > /* Number of quota types we support */ > > #define EXT4_MAXQUOTAS 2 > > > > +/* Types of ext4 journal triggers */ > > +enum ext4_journal_trigger_type { > > + TR_NONE > > +}; > > Probably good if this has an EXT4_ prefix to avoid name clashes? OK. > > + > > +#define EXT4_JOURNAL_TRIGGER_COUNT TR_NONE > > This should just be added after TR_NONE so that it is always correct: > > enum ext4_journal_trigger_type { > EXT4_JTR_NONE, > EXT4_JTR_MAX, > }; So my idea was that EXT4_JTR_NONE would always be the last one (it has to be so that indexing properly works - but that deserves a comment I agree) and thus EXT4_JOURNAL_TRIGGER_COUNT is always correct. > and the s_journal_triggers[] array would use EXT4_JTR_MAX - 1? > > > +struct ext4_journal_trigger { > > + struct jbd2_buffer_trigger_type tr_triggers; > > + struct super_block *sb; > > +}; > > + > > +static inline struct ext4_journal_trigger *EXT4_TRIGGER( > > + struct jbd2_buffer_trigger_type *trigger) > > +{ > > + return container_of(trigger, struct ext4_journal_trigger, tr_triggers); > > +} > > + > > /* > > * fourth extended-fs super-block data in memory > > */ > > @@ -1347,6 +1365,9 @@ struct ext4_sb_info { > > struct mb_cache *s_mb_cache; > > spinlock_t s_es_lock ____cacheline_aligned_in_smp; > > > > + /* Journal triggers for checksum computation */ > > + struct ext4_journal_trigger s_journal_triggers[EXT4_JOURNAL_TRIGGER_COUNT]; > > + > > /* Ratelimit ext4 messages. */ > > struct ratelimit_state s_err_ratelimit_state; > > struct ratelimit_state s_warning_ratelimit_state; > > @@ -2108,13 +2129,14 @@ int ext4_get_block(struct inode *inode, sector_t iblock, > > int ext4_da_get_block_prep(struct inode *inode, sector_t iblock, > > struct buffer_head *bh, int create); > > int ext4_walk_page_buffers(handle_t *handle, > > + struct inode *inode, > > struct buffer_head *head, > > unsigned from, > > unsigned to, > > int *partial, > > - int (*fn)(handle_t *handle, > > + int (*fn)(handle_t *handle, struct inode *inode, > > struct buffer_head *bh)); > > -int do_journal_get_write_access(handle_t *handle, > > +int do_journal_get_write_access(handle_t *handle, struct inode *inode, > > struct buffer_head *bh); > > #define FALL_BACK_TO_NONDELALLOC 1 > > #define CONVERT_INLINE_DATA 2 > > diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c > > index 3445035c7e01..f5ac40942f5a 100644 > > --- a/fs/ext4/ext4_jbd2.c > > +++ b/fs/ext4/ext4_jbd2.c > > @@ -148,19 +148,28 @@ static void ext4_journal_abort_handle(const char *caller, unsigned int line, > > } > > > > int __ext4_journal_get_write_access(const char *where, unsigned int line, > > - handle_t *handle, struct buffer_head *bh) > > + handle_t *handle, struct super_block *sb, > > + struct buffer_head *bh, > > + enum ext4_journal_trigger_type trigger_type) > > { > > - int err = 0; > > + int err; > > > > might_sleep(); > > > > - if (ext4_handle_valid(handle)) { > > - err = jbd2_journal_get_write_access(handle, bh); > > - if (err) > > - ext4_journal_abort_handle(where, line, __func__, bh, > > - handle, err); > > + if (!ext4_handle_valid(handle)) > > + return 0; > > + > > + err = jbd2_journal_get_write_access(handle, bh); > > + if (err) { > > + ext4_journal_abort_handle(where, line, __func__, bh, handle, > > + err); > > + return err; > > } > > - return err; > > + if (trigger_type == TR_NONE || !ext4_has_metadata_csum(sb)) > > + return 0; > > Should this add a check for trigger_type >= EXT4_JTR_MAX? WARN_ON? OK, I'll add that. Thanks for review. Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-16 15:42 [PATCH 0/3 RFC] ext4: Speedup orphan file handling Jan Kara 2015-04-16 15:42 ` [PATCH 1/3] ext4: Support for checksumming from journal triggers Jan Kara @ 2015-04-16 15:42 ` Jan Kara [not found] ` <CAOQ4uxifVr1swHb5Y2M-TRuzwdDo-z92G6PuHvBGecGZ7nYuHg@mail.gmail.com> ` (2 more replies) 2015-04-16 15:42 ` [PATCH 3/3] ext4: Improve scalability of ext4 orphan file handling Jan Kara 2 siblings, 3 replies; 18+ messages in thread From: Jan Kara @ 2015-04-16 15:42 UTC (permalink / raw) To: linux-ext4; +Cc: Jan Kara Ext4 orphan inode handling is a bottleneck for workloads which heavily truncate / unlink small files since it contends on the global s_orphan_mutex lock (and generally it's difficult to improve scalability of the ondisk linked list of orphaned inodes). This patch implements new way of handling orphan inodes. Instead of linking orphaned inode into a linked list, we store it's inode number in a new special file which we call "orphan file". Currently we still protect the orphan file with a spinlock for simplicity but even in this setting we can substantially reduce the length of the critical section and thus speedup some workloads. Note that the change is backwards compatible when the filesystem is clean - the existence of the orphan file is a compat feature, we set another ro-compat feature indicating orphan file needs scanning for orphaned inodes when mounting filesystem read-write. This ro-compat feature gets cleared on unmount / remount read-only. Some performance data from 48 CPU Xeon Server with 32 GB of RAM, filesystem located on ramdisk, average of 5 runs: stress-orphan (microbenchmark truncating files byte-by-byte from N processes in parallel) Threads Time Time Vanilla Patched 1 1.602800 1.260000 2 4.292200 2.455000 4 6.202800 3.848400 8 10.415000 6.833000 16 18.933600 12.883200 32 38.517200 25.342200 64 79.805000 50.918400 128 159.629200 102.666000 reaim new_fserver workload (tweaked to avoid calling sync(1) after every operation) Threads Jobs/s Jobs/s Vanilla Patched 1 24375.00 22941.18 25 162162.16 278571.43 49 222209.30 331626.90 73 280147.60 419447.52 97 315250.00 481910.83 121 331157.90 503360.00 145 343769.00 489081.08 169 355549.56 519487.68 193 356518.65 501800.00 So in both cases we see significant wins all over the board. Signed-off-by: Jan Kara <jack@suse.cz> --- fs/ext4/ext4.h | 52 +++++++++++-- fs/ext4/namei.c | 95 +++++++++++++++++++++-- fs/ext4/super.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++-------- 3 files changed, 341 insertions(+), 43 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index abed83485915..768a8b9ee2f9 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -208,6 +208,7 @@ struct ext4_io_submit { #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ #define EXT4_RESIZE_INO 7 /* Reserved group descriptors inode */ #define EXT4_JOURNAL_INO 8 /* Journal inode */ +#define EXT4_ORPHAN_INO 9 /* Inode with orphan entries */ /* First non-reserved inode for old ext4 filesystems */ #define EXT4_GOOD_OLD_FIRST_INO 11 @@ -831,7 +832,14 @@ struct ext4_inode_info { */ struct rw_semaphore xattr_sem; - struct list_head i_orphan; /* unlinked but open inodes */ + /* + * Inodes with EXT4_STATE_ORPHAN_FILE use i_orphan_block. Otherwise + * i_orphan is used. + */ + union { + struct list_head i_orphan; /* unlinked but open inodes */ + unsigned int i_orphan_idx; /* Index in orphan file */ + }; /* * i_disksize keeps track of what the inode size is ON DISK, not @@ -1188,6 +1196,7 @@ struct ext4_super_block { /* Types of ext4 journal triggers */ enum ext4_journal_trigger_type { + TR_ORPHAN_FILE, TR_NONE }; @@ -1204,6 +1213,29 @@ static inline struct ext4_journal_trigger *EXT4_TRIGGER( return container_of(trigger, struct ext4_journal_trigger, tr_triggers); } +static inline int ext4_inodes_per_orphan_block(struct super_block *sb) +{ + /* We reserve 1 entry for block checksum */ + return sb->s_blocksize / sizeof(u32) - 1; +} + +struct ext4_orphan_block { + int ob_free_entries; /* Number of free orphan entries in block */ + struct buffer_head *ob_bh; /* Buffer for orphan block */ +}; + +/* + * Info about orphan file. Some info in this structure is duplicated - once + * for running and once for committing transaction + */ +struct ext4_orphan_info { + spinlock_t of_lock; + int of_blocks; /* Number of orphan blocks in a file */ + __u32 of_csum_seed; /* Checksum seed for orphan file */ + struct ext4_orphan_block *of_binfo; /* Array with info about orphan + * file blocks */ +}; + /* * fourth extended-fs super-block data in memory */ @@ -1258,8 +1290,10 @@ struct ext4_sb_info { /* Journaling */ struct journal_s *s_journal; - struct list_head s_orphan; - struct mutex s_orphan_lock; + struct mutex s_orphan_lock; /* Protects on disk list changes */ + struct list_head s_orphan; /* List of orphaned inodes in on disk + list */ + struct ext4_orphan_info s_orphan_info; unsigned long s_resize_flags; /* Flags indicating if there is a resizer */ unsigned long s_commit_interval; @@ -1397,6 +1431,7 @@ static inline int ext4_valid_inum(struct super_block *sb, unsigned long ino) ino == EXT4_BOOT_LOADER_INO || ino == EXT4_JOURNAL_INO || ino == EXT4_RESIZE_INO || + ino == EXT4_ORPHAN_INO || (ino >= EXT4_FIRST_INO(sb) && ino <= le32_to_cpu(EXT4_SB(sb)->s_es->s_inodes_count)); } @@ -1437,6 +1472,7 @@ enum { EXT4_STATE_MAY_INLINE_DATA, /* may have in-inode data */ EXT4_STATE_ORDERED_MODE, /* data=ordered mode */ EXT4_STATE_EXT_PRECACHED, /* extents have been precached */ + EXT4_STATE_ORPHAN_FILE, /* Inode orphaned in orphan file */ }; #define EXT4_INODE_BIT_FNS(name, field, offset) \ @@ -1539,6 +1575,7 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) #define EXT4_FEATURE_COMPAT_RESIZE_INODE 0x0010 #define EXT4_FEATURE_COMPAT_DIR_INDEX 0x0020 #define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200 +#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x0400 /* Orphan file exists */ #define EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001 #define EXT4_FEATURE_RO_COMPAT_LARGE_FILE 0x0002 @@ -1556,7 +1593,10 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) * GDT_CSUM bits are mutually exclusive. */ #define EXT4_FEATURE_RO_COMPAT_METADATA_CSUM 0x0400 +/* 0x0800 Reserved for EXT4_FEATURE_RO_COMPAT_REPLICA */ #define EXT4_FEATURE_RO_COMPAT_READONLY 0x1000 +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x2000 /* Orphan file may be + non-empty */ #define EXT4_FEATURE_INCOMPAT_COMPRESSION 0x0001 #define EXT4_FEATURE_INCOMPAT_FILETYPE 0x0002 @@ -1589,7 +1629,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) EXT4_FEATURE_RO_COMPAT_LARGE_FILE| \ EXT4_FEATURE_RO_COMPAT_BTREE_DIR) -#define EXT4_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR +#define EXT4_FEATURE_COMPAT_SUPP (EXT4_FEATURE_COMPAT_EXT_ATTR| \ + EXT4_FEATURE_COMPAT_ORPHAN_FILE) #define EXT4_FEATURE_INCOMPAT_SUPP (EXT4_FEATURE_INCOMPAT_FILETYPE| \ EXT4_FEATURE_INCOMPAT_RECOVER| \ EXT4_FEATURE_INCOMPAT_META_BG| \ @@ -1607,7 +1648,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) EXT4_FEATURE_RO_COMPAT_HUGE_FILE |\ EXT4_FEATURE_RO_COMPAT_BIGALLOC |\ EXT4_FEATURE_RO_COMPAT_METADATA_CSUM|\ - EXT4_FEATURE_RO_COMPAT_QUOTA) + EXT4_FEATURE_RO_COMPAT_QUOTA|\ + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) /* * Default values for user and/or group using reserved blocks diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 460c716e38b0..3436b7fa0ef9 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -2529,6 +2529,46 @@ static int empty_dir(struct inode *inode) return 1; } +static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) +{ + int i, j; + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; + int ret = 0; + __le32 *bdata; + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); + + spin_lock(&oi->of_lock); + for (i = 0; i < oi->of_blocks && !oi->of_binfo[i].ob_free_entries; i++); + if (i == oi->of_blocks) { + spin_unlock(&oi->of_lock); + return -ENOSPC; + } + oi->of_binfo[i].ob_free_entries--; + spin_unlock(&oi->of_lock); + + /* + * Get access to orphan block. We have dropped of_lock but since we + * have decremented number of free entries we are guaranteed free entry + * in our block. + */ + ret = ext4_journal_get_write_access(handle, inode->i_sb, + oi->of_binfo[i].ob_bh, TR_ORPHAN_FILE); + if (ret) + return ret; + + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); + spin_lock(&oi->of_lock); + /* Find empty slot in a block */ + for (j = 0; j < inodes_per_ob && bdata[j]; j++); + BUG_ON(j == inodes_per_ob); + bdata[j] = cpu_to_le32(inode->i_ino); + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); + spin_unlock(&oi->of_lock); + + return ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[i].ob_bh); +} + /* * ext4_orphan_add() links an unlinked or truncated inode into a list of * such inodes, starting at the superblock, in case we crash before the @@ -2555,10 +2595,10 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && !mutex_is_locked(&inode->i_mutex)); /* - * Exit early if inode already is on orphan list. This is a big speedup - * since we don't have to contend on the global s_orphan_lock. + * Inode orphaned in orphan file or in orphan list? */ - if (!list_empty(&EXT4_I(inode)->i_orphan)) + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) || + !list_empty(&EXT4_I(inode)->i_orphan)) return 0; /* @@ -2570,6 +2610,16 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) J_ASSERT((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); + if (sbi->s_orphan_info.of_blocks) { + err = ext4_orphan_file_add(handle, inode); + /* + * Fallback to normal orphan list of orphan file is + * out of space + */ + if (err != -ENOSPC) + return err; + } + BUFFER_TRACE(sbi->s_sbh, "get_write_access"); err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); if (err) @@ -2618,6 +2668,37 @@ out: return err; } +static int ext4_orphan_file_del(handle_t *handle, struct inode *inode) +{ + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; + __le32 *bdata; + int blk, off; + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); + int ret = 0; + + if (!handle) + goto out; + blk = EXT4_I(inode)->i_orphan_idx / inodes_per_ob; + off = EXT4_I(inode)->i_orphan_idx % inodes_per_ob; + + ret = ext4_journal_get_write_access(handle, inode->i_sb, + oi->of_binfo[blk].ob_bh, TR_ORPHAN_FILE); + if (ret) + goto out; + + bdata = (__le32 *)(oi->of_binfo[blk].ob_bh->b_data); + spin_lock(&oi->of_lock); + bdata[off] = 0; + oi->of_binfo[blk].ob_free_entries++; + spin_unlock(&oi->of_lock); + ret = ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[blk].ob_bh); +out: + ext4_clear_inode_state(inode, EXT4_STATE_ORPHAN_FILE); + INIT_LIST_HEAD(&EXT4_I(inode)->i_orphan); + + return ret; +} + /* * ext4_orphan_del() removes an unlinked or truncated inode from the list * of such inodes stored on disk, because it is finally being cleaned up. @@ -2636,10 +2717,14 @@ int ext4_orphan_del(handle_t *handle, struct inode *inode) WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && !mutex_is_locked(&inode->i_mutex)); - /* Do this quick check before taking global s_orphan_lock. */ - if (list_empty(&ei->i_orphan)) + /* Do this quick check before taking global lock. */ + if (!ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) && + list_empty(&ei->i_orphan)) return 0; + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE)) + return ext4_orphan_file_del(handle, inode); + if (handle) { /* Grab inode buffer early before taking global s_orphan_lock */ err = ext4_reserve_inode_write(handle, inode, &iloc); diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 0babe8c435b6..14c30a9ef509 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -761,6 +761,18 @@ static void dump_orphan_list(struct super_block *sb, struct ext4_sb_info *sbi) } } +static void ext4_release_orphan_info(struct super_block *sb) +{ + int i; + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; + + if (!oi->of_blocks) + return; + for (i = 0; i < oi->of_blocks; i++) + brelse(oi->of_binfo[i].ob_bh); + kfree(oi->of_binfo); +} + static void ext4_put_super(struct super_block *sb) { struct ext4_sb_info *sbi = EXT4_SB(sb); @@ -772,6 +784,7 @@ static void ext4_put_super(struct super_block *sb) flush_workqueue(sbi->rsv_conversion_wq); destroy_workqueue(sbi->rsv_conversion_wq); + ext4_release_orphan_info(sb); if (sbi->s_journal) { err = jbd2_journal_destroy(sbi->s_journal); @@ -789,6 +802,8 @@ static void ext4_put_super(struct super_block *sb) if (!(sb->s_flags & MS_RDONLY)) { EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); es->s_state = cpu_to_le16(sbi->s_mount_state); } if (!(sb->s_flags & MS_RDONLY)) @@ -1905,8 +1920,14 @@ static int ext4_setup_super(struct super_block *sb, struct ext4_super_block *es, le16_add_cpu(&es->s_mnt_count, 1); es->s_mtime = cpu_to_le32(get_seconds()); ext4_update_dynamic_rev(sb); - if (sbi->s_journal) + if (sbi->s_journal) { EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); + if (EXT4_HAS_COMPAT_FEATURE(sb, + EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { + EXT4_SET_RO_COMPAT_FEATURE(sb, + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); + } + } ext4_commit_super(sb, 1); done: @@ -2128,6 +2149,36 @@ static int ext4_check_descriptors(struct super_block *sb, return 1; } +static void ext4_process_orphan(struct inode *inode, + int *nr_truncates, int *nr_orphans) +{ + struct super_block *sb = inode->i_sb; + + dquot_initialize(inode); + if (inode->i_nlink) { + if (test_opt(sb, DEBUG)) + ext4_msg(sb, KERN_DEBUG, + "%s: truncating inode %lu to %lld bytes", + __func__, inode->i_ino, inode->i_size); + jbd_debug(2, "truncating inode %lu to %lld bytes\n", + inode->i_ino, inode->i_size); + mutex_lock(&inode->i_mutex); + truncate_inode_pages(inode->i_mapping, inode->i_size); + ext4_truncate(inode); + mutex_unlock(&inode->i_mutex); + (*nr_truncates)++; + } else { + if (test_opt(sb, DEBUG)) + ext4_msg(sb, KERN_DEBUG, + "%s: deleting unreferenced inode %lu", + __func__, inode->i_ino); + jbd_debug(2, "deleting unreferenced inode %lu\n", + inode->i_ino); + (*nr_orphans)++; + } + iput(inode); /* The delete magic happens here! */ +} + /* ext4_orphan_cleanup() walks a singly-linked list of inodes (starting at * the superblock) which were deleted from all directories, but held open by * a process at the time of a crash. We walk the list and try to delete these @@ -2150,10 +2201,13 @@ static void ext4_orphan_cleanup(struct super_block *sb, { unsigned int s_flags = sb->s_flags; int nr_orphans = 0, nr_truncates = 0; -#ifdef CONFIG_QUOTA - int i; -#endif - if (!es->s_last_orphan) { + int i, j; + __le32 *bdata; + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; + struct inode *inode; + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); + + if (!es->s_last_orphan && !oi->of_blocks) { jbd_debug(4, "no orphan inodes to clean up\n"); return; } @@ -2202,8 +2256,6 @@ static void ext4_orphan_cleanup(struct super_block *sb, #endif while (es->s_last_orphan) { - struct inode *inode; - inode = ext4_orphan_get(sb, le32_to_cpu(es->s_last_orphan)); if (IS_ERR(inode)) { es->s_last_orphan = 0; @@ -2211,29 +2263,21 @@ static void ext4_orphan_cleanup(struct super_block *sb, } list_add(&EXT4_I(inode)->i_orphan, &EXT4_SB(sb)->s_orphan); - dquot_initialize(inode); - if (inode->i_nlink) { - if (test_opt(sb, DEBUG)) - ext4_msg(sb, KERN_DEBUG, - "%s: truncating inode %lu to %lld bytes", - __func__, inode->i_ino, inode->i_size); - jbd_debug(2, "truncating inode %lu to %lld bytes\n", - inode->i_ino, inode->i_size); - mutex_lock(&inode->i_mutex); - truncate_inode_pages(inode->i_mapping, inode->i_size); - ext4_truncate(inode); - mutex_unlock(&inode->i_mutex); - nr_truncates++; - } else { - if (test_opt(sb, DEBUG)) - ext4_msg(sb, KERN_DEBUG, - "%s: deleting unreferenced inode %lu", - __func__, inode->i_ino); - jbd_debug(2, "deleting unreferenced inode %lu\n", - inode->i_ino); - nr_orphans++; + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); + } + + for (i = 0; i < oi->of_blocks; i++) { + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); + for (j = 0; j < inodes_per_ob; j++) { + if (!bdata[j]) + continue; + inode = ext4_orphan_get(sb, le32_to_cpu(bdata[j])); + if (IS_ERR(inode)) + continue; + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); } - iput(inode); /* The delete magic happens here! */ } #define PLURAL(x) (x), ((x) == 1) ? "" : "s" @@ -3420,6 +3464,97 @@ static void ext4_setup_csum_trigger(struct super_block *sb, sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; } +static int ext4_orphan_file_block_csum_verify(struct super_block *sb, + struct buffer_head *bh) +{ + __u32 provided, calculated; + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; + + if (!ext4_has_metadata_csum(sb)) + return 1; + + provided = le32_to_cpu(((__le32 *)bh->b_data)[inodes_per_ob]); + calculated = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, + (__u8 *)bh->b_data, + inodes_per_ob * sizeof(__u32)); + return provided == calculated; +} + +/* This gets called only when checksumming is enabled */ +static void ext4_orphan_file_block_trigger( + struct jbd2_buffer_trigger_type *triggers, + struct buffer_head *bh, + void *data, size_t size) +{ + struct super_block *sb = EXT4_TRIGGER(triggers)->sb; + __u32 csum; + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; + + csum = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, (__u8 *)data, + inodes_per_ob * sizeof(__u32)); + ((__le32 *)data)[inodes_per_ob] = cpu_to_le32(csum); +} + +static int ext4_init_orphan_info(struct super_block *sb) +{ + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; + struct inode *inode; + int i, j; + int ret; + int free; + __le32 *bdata; + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); + + spin_lock_init(&oi->of_lock); + + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) + return 0; + + inode = ext4_iget(sb, 12 /* FIXME: EXT4_ORPHAN_INO */); + if (IS_ERR(inode)) { + ext4_msg(sb, KERN_ERR, "get orphan inode failed"); + return PTR_ERR(inode); + } + oi->of_blocks = inode->i_size >> sb->s_blocksize_bits; + oi->of_csum_seed = EXT4_I(inode)->i_csum_seed; + oi->of_binfo = kmalloc(oi->of_blocks*sizeof(struct ext4_orphan_block), + GFP_KERNEL); + if (!oi->of_binfo) { + ret = -ENOMEM; + goto out_put; + } + for (i = 0; i < oi->of_blocks; i++) { + oi->of_binfo[i].ob_bh = ext4_bread(NULL, inode, i, 0); + if (IS_ERR(oi->of_binfo[i].ob_bh)) { + ret = PTR_ERR(oi->of_binfo[i].ob_bh); + goto out_free; + } + if (!ext4_orphan_file_block_csum_verify(sb, + oi->of_binfo[i].ob_bh)) { + ext4_error(sb, "orphan file block %d: bad checksum", i); + ret = -EIO; + goto out_free; + } + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); + free = 0; + for (j = 0; j < inodes_per_ob; j++) + if (bdata[j] == 0) + free++; + oi->of_binfo[i].ob_free_entries = free; + } + iput(inode); + return 0; +out_free: + for (i--; i >= 0; i--) + brelse(oi->of_binfo[i].ob_bh); + kfree(oi->of_binfo); +out_put: + iput(inode); + return ret; +} + static int ext4_fill_super(struct super_block *sb, void *data, int silent) { char *orig_data = kstrdup(data, GFP_KERNEL); @@ -3515,6 +3650,8 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) silent = 1; goto cantfind_ext4; } + ext4_setup_csum_trigger(sb, TR_ORPHAN_FILE, + ext4_orphan_file_block_trigger); /* Load the checksum driver */ if (EXT4_HAS_RO_COMPAT_FEATURE(sb, @@ -3988,8 +4125,10 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) sb->s_root = NULL; needs_recovery = (es->s_last_orphan != 0 || + EXT4_HAS_RO_COMPAT_FEATURE(sb, + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) || EXT4_HAS_INCOMPAT_FEATURE(sb, - EXT4_FEATURE_INCOMPAT_RECOVER)); + EXT4_FEATURE_INCOMPAT_RECOVER)); if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_MMP) && !(sb->s_flags & MS_RDONLY)) @@ -4207,13 +4346,16 @@ no_journal: if (err) goto failed_mount7; + err = ext4_init_orphan_info(sb); + if (err) + goto failed_mount8; #ifdef CONFIG_QUOTA /* Enable quota usage during mount. */ if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_QUOTA) && !(sb->s_flags & MS_RDONLY)) { err = ext4_enable_quotas(sb); if (err) - goto failed_mount8; + goto failed_mount9; } #endif /* CONFIG_QUOTA */ @@ -4263,9 +4405,11 @@ cantfind_ext4: goto failed_mount; #ifdef CONFIG_QUOTA +failed_mount9: + ext4_release_orphan_info(sb); +#endif failed_mount8: kobject_del(&sbi->s_kobj); -#endif failed_mount7: ext4_unregister_li_request(sb); failed_mount6: @@ -4771,6 +4915,20 @@ static int ext4_sync_fs(struct super_block *sb, int wait) return ret; } +static int ext4_orphan_file_empty(struct super_block *sb) +{ + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; + int i; + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); + + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) + return 1; + for (i = 0; i < oi->of_blocks; i++) + if (oi->of_binfo[i].ob_free_entries != inodes_per_ob) + return 0; + return 1; +} + /* * LVM calls this function before a (read-only) snapshot is created. This * gives us a chance to flush the journal completely and mark the fs clean. @@ -4804,6 +4962,10 @@ static int ext4_freeze(struct super_block *sb) /* Journal blocked and flushed, clear needs_recovery flag. */ EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); + if (ext4_orphan_file_empty(sb)) { + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); + } error = ext4_commit_super(sb, 1); out: if (journal) @@ -4823,6 +4985,10 @@ static int ext4_unfreeze(struct super_block *sb) /* Reset the needs_recovery flag before the fs is unlocked. */ EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); + if (EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { + EXT4_SET_RO_COMPAT_FEATURE(sb, + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); + } ext4_commit_super(sb, 1); return 0; } @@ -4966,8 +5132,13 @@ static int ext4_remount(struct super_block *sb, int *flags, char *data) (sbi->s_mount_state & EXT4_VALID_FS)) es->s_state = cpu_to_le16(sbi->s_mount_state); - if (sbi->s_journal) + if (sbi->s_journal) { ext4_mark_recovery_complete(sb, es); + if (ext4_orphan_file_empty(sb)) { + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); + } + } } else { /* Make sure we can mount this feature set readwrite */ if (EXT4_HAS_RO_COMPAT_FEATURE(sb, -- 2.1.4 ^ permalink raw reply related [flat|nested] 18+ messages in thread
[parent not found: <CAOQ4uxifVr1swHb5Y2M-TRuzwdDo-z92G6PuHvBGecGZ7nYuHg@mail.gmail.com>]
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling [not found] ` <CAOQ4uxifVr1swHb5Y2M-TRuzwdDo-z92G6PuHvBGecGZ7nYuHg@mail.gmail.com> @ 2015-04-17 6:09 ` Amir Goldstein 2015-04-17 7:15 ` Jan Kara 1 sibling, 0 replies; 18+ messages in thread From: Amir Goldstein @ 2015-04-17 6:09 UTC (permalink / raw) To: Jan Kara; +Cc: Ext4 Hi Jan, I am sure you considered the option of EXT4_ORPHAN_DIR_INO, a directory being an existing vessel for storing inodes. I imagine that using directory would reduce the complexity of the patch (?) What were your reasons for choosing the orphan file solution? Cheers, Amir. > On Thu, Apr 16, 2015 at 6:42 PM, Jan Kara <jack@suse.cz> wrote: >> >> Ext4 orphan inode handling is a bottleneck for workloads which heavily >> truncate / unlink small files since it contends on the global >> s_orphan_mutex lock (and generally it's difficult to improve scalability >> of the ondisk linked list of orphaned inodes). >> >> This patch implements new way of handling orphan inodes. Instead of >> linking orphaned inode into a linked list, we store it's inode number in >> a new special file which we call "orphan file". Currently we still >> protect the orphan file with a spinlock for simplicity but even in this >> setting we can substantially reduce the length of the critical section >> and thus speedup some workloads. >> >> Note that the change is backwards compatible when the filesystem is >> clean - the existence of the orphan file is a compat feature, we set >> another ro-compat feature indicating orphan file needs scanning for >> orphaned inodes when mounting filesystem read-write. This ro-compat >> feature gets cleared on unmount / remount read-only. >> >> Some performance data from 48 CPU Xeon Server with 32 GB of RAM, >> filesystem located on ramdisk, average of 5 runs: >> >> stress-orphan (microbenchmark truncating files byte-by-byte from N >> processes in parallel) >> >> Threads Time Time >> Vanilla Patched >> 1 1.602800 1.260000 >> 2 4.292200 2.455000 >> 4 6.202800 3.848400 >> 8 10.415000 6.833000 >> 16 18.933600 12.883200 >> 32 38.517200 25.342200 >> 64 79.805000 50.918400 >> 128 159.629200 102.666000 >> >> reaim new_fserver workload (tweaked to avoid calling sync(1) after every >> operation) >> >> Threads Jobs/s Jobs/s >> Vanilla Patched >> 1 24375.00 22941.18 >> 25 162162.16 278571.43 >> 49 222209.30 331626.90 >> 73 280147.60 419447.52 >> 97 315250.00 481910.83 >> 121 331157.90 503360.00 >> 145 343769.00 489081.08 >> 169 355549.56 519487.68 >> 193 356518.65 501800.00 >> >> So in both cases we see significant wins all over the board. >> >> Signed-off-by: Jan Kara <jack@suse.cz> >> --- >> fs/ext4/ext4.h | 52 +++++++++++-- >> fs/ext4/namei.c | 95 +++++++++++++++++++++-- >> fs/ext4/super.c | 237 >> ++++++++++++++++++++++++++++++++++++++++++++++++-------- >> 3 files changed, 341 insertions(+), 43 deletions(-) >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling [not found] ` <CAOQ4uxifVr1swHb5Y2M-TRuzwdDo-z92G6PuHvBGecGZ7nYuHg@mail.gmail.com> 2015-04-17 6:09 ` Amir Goldstein @ 2015-04-17 7:15 ` Jan Kara 2015-04-17 22:21 ` Andreas Dilger 1 sibling, 1 reply; 18+ messages in thread From: Jan Kara @ 2015-04-17 7:15 UTC (permalink / raw) To: Amir Goldstein; +Cc: Jan Kara, Ext4 Hi Amir, On Fri 17-04-15 09:03:13, Amir Goldstein wrote: > I am sure you considered the option of EXT4_ORPHAN_DIR_INO, > a directory being an existing vessel for storing inodes. > > I imagine that using directory would reduce the complexity of the patch (?) > What were your reasons for choosing the orphan file solution? So I didn't seriously consider an option to link inodes into a special orphan directory. Frankly, I doubt that would be simpler than the array of inode numbers I implement in this patch - the handling of the orphan file itself is some 130 lines of code. Sure you could reuse the directory handling code but it would be much more heavy weight and you'd store lots of unnecessary stuff (name, dtype) etc. Plus you'd have to play tricks with locking to get better scalability anyway (i.e., I believe using standard i_mutex and standard directory operations won't give you serious advantage over the current orphan list method). Honza > On Thu, Apr 16, 2015 at 6:42 PM, Jan Kara <jack@suse.cz> wrote: > > > Ext4 orphan inode handling is a bottleneck for workloads which heavily > > truncate / unlink small files since it contends on the global > > s_orphan_mutex lock (and generally it's difficult to improve scalability > > of the ondisk linked list of orphaned inodes). > > > > This patch implements new way of handling orphan inodes. Instead of > > linking orphaned inode into a linked list, we store it's inode number in > > a new special file which we call "orphan file". Currently we still > > protect the orphan file with a spinlock for simplicity but even in this > > setting we can substantially reduce the length of the critical section > > and thus speedup some workloads. > > > > Note that the change is backwards compatible when the filesystem is > > clean - the existence of the orphan file is a compat feature, we set > > another ro-compat feature indicating orphan file needs scanning for > > orphaned inodes when mounting filesystem read-write. This ro-compat > > feature gets cleared on unmount / remount read-only. > > > > Some performance data from 48 CPU Xeon Server with 32 GB of RAM, > > filesystem located on ramdisk, average of 5 runs: > > > > stress-orphan (microbenchmark truncating files byte-by-byte from N > > processes in parallel) > > > > Threads Time Time > > Vanilla Patched > > 1 1.602800 1.260000 > > 2 4.292200 2.455000 > > 4 6.202800 3.848400 > > 8 10.415000 6.833000 > > 16 18.933600 12.883200 > > 32 38.517200 25.342200 > > 64 79.805000 50.918400 > > 128 159.629200 102.666000 > > > > reaim new_fserver workload (tweaked to avoid calling sync(1) after every > > operation) > > > > Threads Jobs/s Jobs/s > > Vanilla Patched > > 1 24375.00 22941.18 > > 25 162162.16 278571.43 > > 49 222209.30 331626.90 > > 73 280147.60 419447.52 > > 97 315250.00 481910.83 > > 121 331157.90 503360.00 > > 145 343769.00 489081.08 > > 169 355549.56 519487.68 > > 193 356518.65 501800.00 > > > > So in both cases we see significant wins all over the board. > > > > Signed-off-by: Jan Kara <jack@suse.cz> > > --- > > fs/ext4/ext4.h | 52 +++++++++++-- > > fs/ext4/namei.c | 95 +++++++++++++++++++++-- > > fs/ext4/super.c | 237 > > ++++++++++++++++++++++++++++++++++++++++++++++++-------- > > 3 files changed, 341 insertions(+), 43 deletions(-) > > > > diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > > index abed83485915..768a8b9ee2f9 100644 > > --- a/fs/ext4/ext4.h > > +++ b/fs/ext4/ext4.h > > @@ -208,6 +208,7 @@ struct ext4_io_submit { > > #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ > > #define EXT4_RESIZE_INO 7 /* Reserved group > > descriptors inode */ > > #define EXT4_JOURNAL_INO 8 /* Journal inode */ > > +#define EXT4_ORPHAN_INO 9 /* Inode with orphan > > entries */ > > > > /* First non-reserved inode for old ext4 filesystems */ > > #define EXT4_GOOD_OLD_FIRST_INO 11 > > @@ -831,7 +832,14 @@ struct ext4_inode_info { > > */ > > struct rw_semaphore xattr_sem; > > > > - struct list_head i_orphan; /* unlinked but open inodes */ > > + /* > > + * Inodes with EXT4_STATE_ORPHAN_FILE use i_orphan_block. Otherwise > > + * i_orphan is used. > > + */ > > + union { > > + struct list_head i_orphan; /* unlinked but open > > inodes */ > > + unsigned int i_orphan_idx; /* Index in orphan file */ > > + }; > > > > /* > > * i_disksize keeps track of what the inode size is ON DISK, not > > @@ -1188,6 +1196,7 @@ struct ext4_super_block { > > > > /* Types of ext4 journal triggers */ > > enum ext4_journal_trigger_type { > > + TR_ORPHAN_FILE, > > TR_NONE > > }; > > > > @@ -1204,6 +1213,29 @@ static inline struct ext4_journal_trigger > > *EXT4_TRIGGER( > > return container_of(trigger, struct ext4_journal_trigger, > > tr_triggers); > > } > > > > +static inline int ext4_inodes_per_orphan_block(struct super_block *sb) > > +{ > > + /* We reserve 1 entry for block checksum */ > > + return sb->s_blocksize / sizeof(u32) - 1; > > +} > > + > > +struct ext4_orphan_block { > > + int ob_free_entries; /* Number of free orphan entries in block > > */ > > + struct buffer_head *ob_bh; /* Buffer for orphan block */ > > +}; > > + > > +/* > > + * Info about orphan file. Some info in this structure is duplicated - > > once > > + * for running and once for committing transaction > > + */ > > +struct ext4_orphan_info { > > + spinlock_t of_lock; > > + int of_blocks; /* Number of orphan blocks in a > > file */ > > + __u32 of_csum_seed; /* Checksum seed for orphan file */ > > + struct ext4_orphan_block *of_binfo; /* Array with info about > > orphan > > + * file blocks */ > > +}; > > + > > /* > > * fourth extended-fs super-block data in memory > > */ > > @@ -1258,8 +1290,10 @@ struct ext4_sb_info { > > > > /* Journaling */ > > struct journal_s *s_journal; > > - struct list_head s_orphan; > > - struct mutex s_orphan_lock; > > + struct mutex s_orphan_lock; /* Protects on disk list changes */ > > + struct list_head s_orphan; /* List of orphaned inodes in on > > disk > > + list */ > > + struct ext4_orphan_info s_orphan_info; > > unsigned long s_resize_flags; /* Flags indicating if > > there > > is a resizer */ > > unsigned long s_commit_interval; > > @@ -1397,6 +1431,7 @@ static inline int ext4_valid_inum(struct super_block > > *sb, unsigned long ino) > > ino == EXT4_BOOT_LOADER_INO || > > ino == EXT4_JOURNAL_INO || > > ino == EXT4_RESIZE_INO || > > + ino == EXT4_ORPHAN_INO || > > (ino >= EXT4_FIRST_INO(sb) && > > ino <= le32_to_cpu(EXT4_SB(sb)->s_es->s_inodes_count)); > > } > > @@ -1437,6 +1472,7 @@ enum { > > EXT4_STATE_MAY_INLINE_DATA, /* may have in-inode data */ > > EXT4_STATE_ORDERED_MODE, /* data=ordered mode */ > > EXT4_STATE_EXT_PRECACHED, /* extents have been precached */ > > + EXT4_STATE_ORPHAN_FILE, /* Inode orphaned in orphan file */ > > }; > > > > #define EXT4_INODE_BIT_FNS(name, field, offset) > > \ > > @@ -1539,6 +1575,7 @@ static inline void ext4_clear_state_flags(struct > > ext4_inode_info *ei) > > #define EXT4_FEATURE_COMPAT_RESIZE_INODE 0x0010 > > #define EXT4_FEATURE_COMPAT_DIR_INDEX 0x0020 > > #define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200 > > +#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x0400 /* Orphan > > file exists */ > > > > #define EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001 > > #define EXT4_FEATURE_RO_COMPAT_LARGE_FILE 0x0002 > > @@ -1556,7 +1593,10 @@ static inline void ext4_clear_state_flags(struct > > ext4_inode_info *ei) > > * GDT_CSUM bits are mutually exclusive. > > */ > > #define EXT4_FEATURE_RO_COMPAT_METADATA_CSUM 0x0400 > > +/* 0x0800 Reserved for EXT4_FEATURE_RO_COMPAT_REPLICA */ > > #define EXT4_FEATURE_RO_COMPAT_READONLY 0x1000 > > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x2000 /* Orphan file may > > be > > + non-empty */ > > > > #define EXT4_FEATURE_INCOMPAT_COMPRESSION 0x0001 > > #define EXT4_FEATURE_INCOMPAT_FILETYPE 0x0002 > > @@ -1589,7 +1629,8 @@ static inline void ext4_clear_state_flags(struct > > ext4_inode_info *ei) > > > > EXT4_FEATURE_RO_COMPAT_LARGE_FILE| \ > > EXT4_FEATURE_RO_COMPAT_BTREE_DIR) > > > > -#define EXT4_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR > > +#define EXT4_FEATURE_COMPAT_SUPP (EXT4_FEATURE_COMPAT_EXT_ATTR| \ > > + EXT4_FEATURE_COMPAT_ORPHAN_FILE) > > #define EXT4_FEATURE_INCOMPAT_SUPP (EXT4_FEATURE_INCOMPAT_FILETYPE| \ > > EXT4_FEATURE_INCOMPAT_RECOVER| \ > > EXT4_FEATURE_INCOMPAT_META_BG| \ > > @@ -1607,7 +1648,8 @@ static inline void ext4_clear_state_flags(struct > > ext4_inode_info *ei) > > EXT4_FEATURE_RO_COMPAT_HUGE_FILE > > |\ > > EXT4_FEATURE_RO_COMPAT_BIGALLOC |\ > > > > EXT4_FEATURE_RO_COMPAT_METADATA_CSUM|\ > > - EXT4_FEATURE_RO_COMPAT_QUOTA) > > + EXT4_FEATURE_RO_COMPAT_QUOTA|\ > > + > > EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) > > > > /* > > * Default values for user and/or group using reserved blocks > > diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c > > index 460c716e38b0..3436b7fa0ef9 100644 > > --- a/fs/ext4/namei.c > > +++ b/fs/ext4/namei.c > > @@ -2529,6 +2529,46 @@ static int empty_dir(struct inode *inode) > > return 1; > > } > > > > +static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) > > +{ > > + int i, j; > > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > > + int ret = 0; > > + __le32 *bdata; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > > + > > + spin_lock(&oi->of_lock); > > + for (i = 0; i < oi->of_blocks && !oi->of_binfo[i].ob_free_entries; > > i++); > > + if (i == oi->of_blocks) { > > + spin_unlock(&oi->of_lock); > > + return -ENOSPC; > > + } > > + oi->of_binfo[i].ob_free_entries--; > > + spin_unlock(&oi->of_lock); > > + > > + /* > > + * Get access to orphan block. We have dropped of_lock but since we > > + * have decremented number of free entries we are guaranteed free > > entry > > + * in our block. > > + */ > > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > > + oi->of_binfo[i].ob_bh, TR_ORPHAN_FILE); > > + if (ret) > > + return ret; > > + > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + spin_lock(&oi->of_lock); > > + /* Find empty slot in a block */ > > + for (j = 0; j < inodes_per_ob && bdata[j]; j++); > > + BUG_ON(j == inodes_per_ob); > > + bdata[j] = cpu_to_le32(inode->i_ino); > > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; > > + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + spin_unlock(&oi->of_lock); > > + > > + return ext4_handle_dirty_metadata(handle, NULL, > > oi->of_binfo[i].ob_bh); > > +} > > + > > /* > > * ext4_orphan_add() links an unlinked or truncated inode into a list of > > * such inodes, starting at the superblock, in case we crash before the > > @@ -2555,10 +2595,10 @@ int ext4_orphan_add(handle_t *handle, struct inode > > *inode) > > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > > !mutex_is_locked(&inode->i_mutex)); > > /* > > - * Exit early if inode already is on orphan list. This is a big > > speedup > > - * since we don't have to contend on the global s_orphan_lock. > > + * Inode orphaned in orphan file or in orphan list? > > */ > > - if (!list_empty(&EXT4_I(inode)->i_orphan)) > > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) || > > + !list_empty(&EXT4_I(inode)->i_orphan)) > > return 0; > > > > /* > > @@ -2570,6 +2610,16 @@ int ext4_orphan_add(handle_t *handle, struct inode > > *inode) > > J_ASSERT((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || > > S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); > > > > + if (sbi->s_orphan_info.of_blocks) { > > + err = ext4_orphan_file_add(handle, inode); > > + /* > > + * Fallback to normal orphan list of orphan file is > > + * out of space > > + */ > > + if (err != -ENOSPC) > > + return err; > > + } > > + > > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > > err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, > > TR_NONE); > > if (err) > > @@ -2618,6 +2668,37 @@ out: > > return err; > > } > > > > +static int ext4_orphan_file_del(handle_t *handle, struct inode *inode) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > > + __le32 *bdata; > > + int blk, off; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > > + int ret = 0; > > + > > + if (!handle) > > + goto out; > > + blk = EXT4_I(inode)->i_orphan_idx / inodes_per_ob; > > + off = EXT4_I(inode)->i_orphan_idx % inodes_per_ob; > > + > > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > > + oi->of_binfo[blk].ob_bh, TR_ORPHAN_FILE); > > + if (ret) > > + goto out; > > + > > + bdata = (__le32 *)(oi->of_binfo[blk].ob_bh->b_data); > > + spin_lock(&oi->of_lock); > > + bdata[off] = 0; > > + oi->of_binfo[blk].ob_free_entries++; > > + spin_unlock(&oi->of_lock); > > + ret = ext4_handle_dirty_metadata(handle, NULL, > > oi->of_binfo[blk].ob_bh); > > +out: > > + ext4_clear_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + INIT_LIST_HEAD(&EXT4_I(inode)->i_orphan); > > + > > + return ret; > > +} > > + > > /* > > * ext4_orphan_del() removes an unlinked or truncated inode from the list > > * of such inodes stored on disk, because it is finally being cleaned up. > > @@ -2636,10 +2717,14 @@ int ext4_orphan_del(handle_t *handle, struct inode > > *inode) > > > > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > > !mutex_is_locked(&inode->i_mutex)); > > - /* Do this quick check before taking global s_orphan_lock. */ > > - if (list_empty(&ei->i_orphan)) > > + /* Do this quick check before taking global lock. */ > > + if (!ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) && > > + list_empty(&ei->i_orphan)) > > return 0; > > > > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE)) > > + return ext4_orphan_file_del(handle, inode); > > + > > if (handle) { > > /* Grab inode buffer early before taking global > > s_orphan_lock */ > > err = ext4_reserve_inode_write(handle, inode, &iloc); > > diff --git a/fs/ext4/super.c b/fs/ext4/super.c > > index 0babe8c435b6..14c30a9ef509 100644 > > --- a/fs/ext4/super.c > > +++ b/fs/ext4/super.c > > @@ -761,6 +761,18 @@ static void dump_orphan_list(struct super_block *sb, > > struct ext4_sb_info *sbi) > > } > > } > > > > +static void ext4_release_orphan_info(struct super_block *sb) > > +{ > > + int i; > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + if (!oi->of_blocks) > > + return; > > + for (i = 0; i < oi->of_blocks; i++) > > + brelse(oi->of_binfo[i].ob_bh); > > + kfree(oi->of_binfo); > > +} > > + > > static void ext4_put_super(struct super_block *sb) > > { > > struct ext4_sb_info *sbi = EXT4_SB(sb); > > @@ -772,6 +784,7 @@ static void ext4_put_super(struct super_block *sb) > > > > flush_workqueue(sbi->rsv_conversion_wq); > > destroy_workqueue(sbi->rsv_conversion_wq); > > + ext4_release_orphan_info(sb); > > > > if (sbi->s_journal) { > > err = jbd2_journal_destroy(sbi->s_journal); > > @@ -789,6 +802,8 @@ static void ext4_put_super(struct super_block *sb) > > > > if (!(sb->s_flags & MS_RDONLY)) { > > EXT4_CLEAR_INCOMPAT_FEATURE(sb, > > EXT4_FEATURE_INCOMPAT_RECOVER); > > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > es->s_state = cpu_to_le16(sbi->s_mount_state); > > } > > if (!(sb->s_flags & MS_RDONLY)) > > @@ -1905,8 +1920,14 @@ static int ext4_setup_super(struct super_block *sb, > > struct ext4_super_block *es, > > le16_add_cpu(&es->s_mnt_count, 1); > > es->s_mtime = cpu_to_le32(get_seconds()); > > ext4_update_dynamic_rev(sb); > > - if (sbi->s_journal) > > + if (sbi->s_journal) { > > EXT4_SET_INCOMPAT_FEATURE(sb, > > EXT4_FEATURE_INCOMPAT_RECOVER); > > + if (EXT4_HAS_COMPAT_FEATURE(sb, > > + > > EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { > > + EXT4_SET_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > + } > > > > ext4_commit_super(sb, 1); > > done: > > @@ -2128,6 +2149,36 @@ static int ext4_check_descriptors(struct > > super_block *sb, > > return 1; > > } > > > > +static void ext4_process_orphan(struct inode *inode, > > + int *nr_truncates, int *nr_orphans) > > +{ > > + struct super_block *sb = inode->i_sb; > > + > > + dquot_initialize(inode); > > + if (inode->i_nlink) { > > + if (test_opt(sb, DEBUG)) > > + ext4_msg(sb, KERN_DEBUG, > > + "%s: truncating inode %lu to %lld bytes", > > + __func__, inode->i_ino, inode->i_size); > > + jbd_debug(2, "truncating inode %lu to %lld bytes\n", > > + inode->i_ino, inode->i_size); > > + mutex_lock(&inode->i_mutex); > > + truncate_inode_pages(inode->i_mapping, inode->i_size); > > + ext4_truncate(inode); > > + mutex_unlock(&inode->i_mutex); > > + (*nr_truncates)++; > > + } else { > > + if (test_opt(sb, DEBUG)) > > + ext4_msg(sb, KERN_DEBUG, > > + "%s: deleting unreferenced inode %lu", > > + __func__, inode->i_ino); > > + jbd_debug(2, "deleting unreferenced inode %lu\n", > > + inode->i_ino); > > + (*nr_orphans)++; > > + } > > + iput(inode); /* The delete magic happens here! */ > > +} > > + > > /* ext4_orphan_cleanup() walks a singly-linked list of inodes (starting at > > * the superblock) which were deleted from all directories, but held open > > by > > * a process at the time of a crash. We walk the list and try to delete > > these > > @@ -2150,10 +2201,13 @@ static void ext4_orphan_cleanup(struct super_block > > *sb, > > { > > unsigned int s_flags = sb->s_flags; > > int nr_orphans = 0, nr_truncates = 0; > > -#ifdef CONFIG_QUOTA > > - int i; > > -#endif > > - if (!es->s_last_orphan) { > > + int i, j; > > + __le32 *bdata; > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + struct inode *inode; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + if (!es->s_last_orphan && !oi->of_blocks) { > > jbd_debug(4, "no orphan inodes to clean up\n"); > > return; > > } > > @@ -2202,8 +2256,6 @@ static void ext4_orphan_cleanup(struct super_block > > *sb, > > #endif > > > > while (es->s_last_orphan) { > > - struct inode *inode; > > - > > inode = ext4_orphan_get(sb, > > le32_to_cpu(es->s_last_orphan)); > > if (IS_ERR(inode)) { > > es->s_last_orphan = 0; > > @@ -2211,29 +2263,21 @@ static void ext4_orphan_cleanup(struct super_block > > *sb, > > } > > > > list_add(&EXT4_I(inode)->i_orphan, &EXT4_SB(sb)->s_orphan); > > - dquot_initialize(inode); > > - if (inode->i_nlink) { > > - if (test_opt(sb, DEBUG)) > > - ext4_msg(sb, KERN_DEBUG, > > - "%s: truncating inode %lu to %lld > > bytes", > > - __func__, inode->i_ino, > > inode->i_size); > > - jbd_debug(2, "truncating inode %lu to %lld > > bytes\n", > > - inode->i_ino, inode->i_size); > > - mutex_lock(&inode->i_mutex); > > - truncate_inode_pages(inode->i_mapping, > > inode->i_size); > > - ext4_truncate(inode); > > - mutex_unlock(&inode->i_mutex); > > - nr_truncates++; > > - } else { > > - if (test_opt(sb, DEBUG)) > > - ext4_msg(sb, KERN_DEBUG, > > - "%s: deleting unreferenced inode > > %lu", > > - __func__, inode->i_ino); > > - jbd_debug(2, "deleting unreferenced inode %lu\n", > > - inode->i_ino); > > - nr_orphans++; > > + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); > > + } > > + > > + for (i = 0; i < oi->of_blocks; i++) { > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + for (j = 0; j < inodes_per_ob; j++) { > > + if (!bdata[j]) > > + continue; > > + inode = ext4_orphan_get(sb, le32_to_cpu(bdata[j])); > > + if (IS_ERR(inode)) > > + continue; > > + ext4_set_inode_state(inode, > > EXT4_STATE_ORPHAN_FILE); > > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + > > j; > > + ext4_process_orphan(inode, &nr_truncates, > > &nr_orphans); > > } > > - iput(inode); /* The delete magic happens here! */ > > } > > > > #define PLURAL(x) (x), ((x) == 1) ? "" : "s" > > @@ -3420,6 +3464,97 @@ static void ext4_setup_csum_trigger(struct > > super_block *sb, > > sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; > > } > > > > +static int ext4_orphan_file_block_csum_verify(struct super_block *sb, > > + struct buffer_head *bh) > > +{ > > + __u32 provided, calculated; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + if (!ext4_has_metadata_csum(sb)) > > + return 1; > > + > > + provided = le32_to_cpu(((__le32 *)bh->b_data)[inodes_per_ob]); > > + calculated = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, > > + (__u8 *)bh->b_data, > > + inodes_per_ob * sizeof(__u32)); > > + return provided == calculated; > > +} > > + > > +/* This gets called only when checksumming is enabled */ > > +static void ext4_orphan_file_block_trigger( > > + struct jbd2_buffer_trigger_type *triggers, > > + struct buffer_head *bh, > > + void *data, size_t size) > > +{ > > + struct super_block *sb = EXT4_TRIGGER(triggers)->sb; > > + __u32 csum; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + csum = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, (__u8 *)data, > > + inodes_per_ob * sizeof(__u32)); > > + ((__le32 *)data)[inodes_per_ob] = cpu_to_le32(csum); > > +} > > + > > +static int ext4_init_orphan_info(struct super_block *sb) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + struct inode *inode; > > + int i, j; > > + int ret; > > + int free; > > + __le32 *bdata; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + spin_lock_init(&oi->of_lock); > > + > > + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) > > + return 0; > > + > > + inode = ext4_iget(sb, 12 /* FIXME: EXT4_ORPHAN_INO */); > > + if (IS_ERR(inode)) { > > + ext4_msg(sb, KERN_ERR, "get orphan inode failed"); > > + return PTR_ERR(inode); > > + } > > + oi->of_blocks = inode->i_size >> sb->s_blocksize_bits; > > + oi->of_csum_seed = EXT4_I(inode)->i_csum_seed; > > + oi->of_binfo = kmalloc(oi->of_blocks*sizeof(struct > > ext4_orphan_block), > > + GFP_KERNEL); > > + if (!oi->of_binfo) { > > + ret = -ENOMEM; > > + goto out_put; > > + } > > + for (i = 0; i < oi->of_blocks; i++) { > > + oi->of_binfo[i].ob_bh = ext4_bread(NULL, inode, i, 0); > > + if (IS_ERR(oi->of_binfo[i].ob_bh)) { > > + ret = PTR_ERR(oi->of_binfo[i].ob_bh); > > + goto out_free; > > + } > > + if (!ext4_orphan_file_block_csum_verify(sb, > > + oi->of_binfo[i].ob_bh)) { > > + ext4_error(sb, "orphan file block %d: bad > > checksum", i); > > + ret = -EIO; > > + goto out_free; > > + } > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + free = 0; > > + for (j = 0; j < inodes_per_ob; j++) > > + if (bdata[j] == 0) > > + free++; > > + oi->of_binfo[i].ob_free_entries = free; > > + } > > + iput(inode); > > + return 0; > > +out_free: > > + for (i--; i >= 0; i--) > > + brelse(oi->of_binfo[i].ob_bh); > > + kfree(oi->of_binfo); > > +out_put: > > + iput(inode); > > + return ret; > > +} > > + > > static int ext4_fill_super(struct super_block *sb, void *data, int silent) > > { > > char *orig_data = kstrdup(data, GFP_KERNEL); > > @@ -3515,6 +3650,8 @@ static int ext4_fill_super(struct super_block *sb, > > void *data, int silent) > > silent = 1; > > goto cantfind_ext4; > > } > > + ext4_setup_csum_trigger(sb, TR_ORPHAN_FILE, > > + ext4_orphan_file_block_trigger); > > > > /* Load the checksum driver */ > > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, > > @@ -3988,8 +4125,10 @@ static int ext4_fill_super(struct super_block *sb, > > void *data, int silent) > > sb->s_root = NULL; > > > > needs_recovery = (es->s_last_orphan != 0 || > > + EXT4_HAS_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) || > > EXT4_HAS_INCOMPAT_FEATURE(sb, > > - EXT4_FEATURE_INCOMPAT_RECOVER)); > > + EXT4_FEATURE_INCOMPAT_RECOVER)); > > > > if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_MMP) && > > !(sb->s_flags & MS_RDONLY)) > > @@ -4207,13 +4346,16 @@ no_journal: > > if (err) > > goto failed_mount7; > > > > + err = ext4_init_orphan_info(sb); > > + if (err) > > + goto failed_mount8; > > #ifdef CONFIG_QUOTA > > /* Enable quota usage during mount. */ > > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_QUOTA) && > > !(sb->s_flags & MS_RDONLY)) { > > err = ext4_enable_quotas(sb); > > if (err) > > - goto failed_mount8; > > + goto failed_mount9; > > } > > #endif /* CONFIG_QUOTA */ > > > > @@ -4263,9 +4405,11 @@ cantfind_ext4: > > goto failed_mount; > > > > #ifdef CONFIG_QUOTA > > +failed_mount9: > > + ext4_release_orphan_info(sb); > > +#endif > > failed_mount8: > > kobject_del(&sbi->s_kobj); > > -#endif > > failed_mount7: > > ext4_unregister_li_request(sb); > > failed_mount6: > > @@ -4771,6 +4915,20 @@ static int ext4_sync_fs(struct super_block *sb, int > > wait) > > return ret; > > } > > > > +static int ext4_orphan_file_empty(struct super_block *sb) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + int i; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) > > + return 1; > > + for (i = 0; i < oi->of_blocks; i++) > > + if (oi->of_binfo[i].ob_free_entries != inodes_per_ob) > > + return 0; > > + return 1; > > +} > > + > > /* > > * LVM calls this function before a (read-only) snapshot is created. This > > * gives us a chance to flush the journal completely and mark the fs > > clean. > > @@ -4804,6 +4962,10 @@ static int ext4_freeze(struct super_block *sb) > > > > /* Journal blocked and flushed, clear needs_recovery flag. */ > > EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + if (ext4_orphan_file_empty(sb)) { > > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > error = ext4_commit_super(sb, 1); > > out: > > if (journal) > > @@ -4823,6 +4985,10 @@ static int ext4_unfreeze(struct super_block *sb) > > > > /* Reset the needs_recovery flag before the fs is unlocked. */ > > EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + if (EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { > > + EXT4_SET_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > ext4_commit_super(sb, 1); > > return 0; > > } > > @@ -4966,8 +5132,13 @@ static int ext4_remount(struct super_block *sb, int > > *flags, char *data) > > (sbi->s_mount_state & EXT4_VALID_FS)) > > es->s_state = > > cpu_to_le16(sbi->s_mount_state); > > > > - if (sbi->s_journal) > > + if (sbi->s_journal) { > > ext4_mark_recovery_complete(sb, es); > > + if (ext4_orphan_file_empty(sb)) { > > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > > + > > EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > + } > > } else { > > /* Make sure we can mount this feature set > > readwrite */ > > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, > > -- > > 2.1.4 > > > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-17 7:15 ` Jan Kara @ 2015-04-17 22:21 ` Andreas Dilger 0 siblings, 0 replies; 18+ messages in thread From: Andreas Dilger @ 2015-04-17 22:21 UTC (permalink / raw) To: Jan Kara; +Cc: Amir Goldstein, Ext4 On Apr 17, 2015, at 1:15 AM, Jan Kara <jack@suse.cz> wrote: > On Fri 17-04-15 09:03:13, Amir Goldstein wrote: >> I am sure you considered the option of EXT4_ORPHAN_DIR_INO, >> a directory being an existing vessel for storing inodes. >> >> I imagine that using directory would reduce the complexity of the patch (?) >> What were your reasons for choosing the orphan file solution? > > So I didn't seriously consider an option to link inodes into a special > orphan directory. Frankly, I doubt that would be simpler than the array of > inode numbers I implement in this patch - the handling of the orphan file > itself is some 130 lines of code. Sure you could reuse the directory > handling code but it would be much more heavy weight and you'd store lots > of unnecessary stuff (name, dtype) etc. Plus you'd have to play tricks with > locking to get better scalability anyway (i.e., I believe using standard > i_mutex and standard directory operations won't give you serious advantage > over the current orphan list method). We actually have parallel directory operations patch for ext4 that we would be happy to contribute upstream. It uses scalable locking on a per-leaf block basis, so the parallelism increases with the size of the directory. http://git.hpdd.intel.com/fs/lustre-release.git/blob/HEAD:/ldiskfs/kernel_patches/patches/rhel7/ext4-pdirop.patch That said, I don't insist on using this, but just letting you know it is available. Unfortunately, it only is accessible from within the kernel (used by Lustre servers) and there isn't any patch for VFS-level multi- threaded directory locking, but that would be usable for orphan handling. The patch has been in use for several years in production. Cheers, Andreas >> On Thu, Apr 16, 2015 at 6:42 PM, Jan Kara <jack@suse.cz> wrote: >> >>> Ext4 orphan inode handling is a bottleneck for workloads which heavily >>> truncate / unlink small files since it contends on the global >>> s_orphan_mutex lock (and generally it's difficult to improve scalability >>> of the ondisk linked list of orphaned inodes). >>> >>> This patch implements new way of handling orphan inodes. Instead of >>> linking orphaned inode into a linked list, we store it's inode number in >>> a new special file which we call "orphan file". Currently we still >>> protect the orphan file with a spinlock for simplicity but even in this >>> setting we can substantially reduce the length of the critical section >>> and thus speedup some workloads. >>> >>> Note that the change is backwards compatible when the filesystem is >>> clean - the existence of the orphan file is a compat feature, we set >>> another ro-compat feature indicating orphan file needs scanning for >>> orphaned inodes when mounting filesystem read-write. This ro-compat >>> feature gets cleared on unmount / remount read-only. >>> >>> Some performance data from 48 CPU Xeon Server with 32 GB of RAM, >>> filesystem located on ramdisk, average of 5 runs: >>> >>> stress-orphan (microbenchmark truncating files byte-by-byte from N >>> processes in parallel) >>> >>> Threads Time Time >>> Vanilla Patched >>> 1 1.602800 1.260000 >>> 2 4.292200 2.455000 >>> 4 6.202800 3.848400 >>> 8 10.415000 6.833000 >>> 16 18.933600 12.883200 >>> 32 38.517200 25.342200 >>> 64 79.805000 50.918400 >>> 128 159.629200 102.666000 >>> >>> reaim new_fserver workload (tweaked to avoid calling sync(1) after every >>> operation) >>> >>> Threads Jobs/s Jobs/s >>> Vanilla Patched >>> 1 24375.00 22941.18 >>> 25 162162.16 278571.43 >>> 49 222209.30 331626.90 >>> 73 280147.60 419447.52 >>> 97 315250.00 481910.83 >>> 121 331157.90 503360.00 >>> 145 343769.00 489081.08 >>> 169 355549.56 519487.68 >>> 193 356518.65 501800.00 >>> >>> So in both cases we see significant wins all over the board. >>> >>> Signed-off-by: Jan Kara <jack@suse.cz> >>> --- >>> fs/ext4/ext4.h | 52 +++++++++++-- >>> fs/ext4/namei.c | 95 +++++++++++++++++++++-- >>> fs/ext4/super.c | 237 >>> ++++++++++++++++++++++++++++++++++++++++++++++++-------- >>> 3 files changed, 341 insertions(+), 43 deletions(-) >>> >>> diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h >>> index abed83485915..768a8b9ee2f9 100644 >>> --- a/fs/ext4/ext4.h >>> +++ b/fs/ext4/ext4.h >>> @@ -208,6 +208,7 @@ struct ext4_io_submit { >>> #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ >>> #define EXT4_RESIZE_INO 7 /* Reserved group >>> descriptors inode */ >>> #define EXT4_JOURNAL_INO 8 /* Journal inode */ >>> +#define EXT4_ORPHAN_INO 9 /* Inode with orphan >>> entries */ >>> >>> /* First non-reserved inode for old ext4 filesystems */ >>> #define EXT4_GOOD_OLD_FIRST_INO 11 >>> @@ -831,7 +832,14 @@ struct ext4_inode_info { >>> */ >>> struct rw_semaphore xattr_sem; >>> >>> - struct list_head i_orphan; /* unlinked but open inodes */ >>> + /* >>> + * Inodes with EXT4_STATE_ORPHAN_FILE use i_orphan_block. Otherwise >>> + * i_orphan is used. >>> + */ >>> + union { >>> + struct list_head i_orphan; /* unlinked but open >>> inodes */ >>> + unsigned int i_orphan_idx; /* Index in orphan file */ >>> + }; >>> >>> /* >>> * i_disksize keeps track of what the inode size is ON DISK, not >>> @@ -1188,6 +1196,7 @@ struct ext4_super_block { >>> >>> /* Types of ext4 journal triggers */ >>> enum ext4_journal_trigger_type { >>> + TR_ORPHAN_FILE, >>> TR_NONE >>> }; >>> >>> @@ -1204,6 +1213,29 @@ static inline struct ext4_journal_trigger >>> *EXT4_TRIGGER( >>> return container_of(trigger, struct ext4_journal_trigger, >>> tr_triggers); >>> } >>> >>> +static inline int ext4_inodes_per_orphan_block(struct super_block *sb) >>> +{ >>> + /* We reserve 1 entry for block checksum */ >>> + return sb->s_blocksize / sizeof(u32) - 1; >>> +} >>> + >>> +struct ext4_orphan_block { >>> + int ob_free_entries; /* Number of free orphan entries in block >>> */ >>> + struct buffer_head *ob_bh; /* Buffer for orphan block */ >>> +}; >>> + >>> +/* >>> + * Info about orphan file. Some info in this structure is duplicated - >>> once >>> + * for running and once for committing transaction >>> + */ >>> +struct ext4_orphan_info { >>> + spinlock_t of_lock; >>> + int of_blocks; /* Number of orphan blocks in a >>> file */ >>> + __u32 of_csum_seed; /* Checksum seed for orphan file */ >>> + struct ext4_orphan_block *of_binfo; /* Array with info about >>> orphan >>> + * file blocks */ >>> +}; >>> + >>> /* >>> * fourth extended-fs super-block data in memory >>> */ >>> @@ -1258,8 +1290,10 @@ struct ext4_sb_info { >>> >>> /* Journaling */ >>> struct journal_s *s_journal; >>> - struct list_head s_orphan; >>> - struct mutex s_orphan_lock; >>> + struct mutex s_orphan_lock; /* Protects on disk list changes */ >>> + struct list_head s_orphan; /* List of orphaned inodes in on >>> disk >>> + list */ >>> + struct ext4_orphan_info s_orphan_info; >>> unsigned long s_resize_flags; /* Flags indicating if >>> there >>> is a resizer */ >>> unsigned long s_commit_interval; >>> @@ -1397,6 +1431,7 @@ static inline int ext4_valid_inum(struct super_block >>> *sb, unsigned long ino) >>> ino == EXT4_BOOT_LOADER_INO || >>> ino == EXT4_JOURNAL_INO || >>> ino == EXT4_RESIZE_INO || >>> + ino == EXT4_ORPHAN_INO || >>> (ino >= EXT4_FIRST_INO(sb) && >>> ino <= le32_to_cpu(EXT4_SB(sb)->s_es->s_inodes_count)); >>> } >>> @@ -1437,6 +1472,7 @@ enum { >>> EXT4_STATE_MAY_INLINE_DATA, /* may have in-inode data */ >>> EXT4_STATE_ORDERED_MODE, /* data=ordered mode */ >>> EXT4_STATE_EXT_PRECACHED, /* extents have been precached */ >>> + EXT4_STATE_ORPHAN_FILE, /* Inode orphaned in orphan file */ >>> }; >>> >>> #define EXT4_INODE_BIT_FNS(name, field, offset) >>> \ >>> @@ -1539,6 +1575,7 @@ static inline void ext4_clear_state_flags(struct >>> ext4_inode_info *ei) >>> #define EXT4_FEATURE_COMPAT_RESIZE_INODE 0x0010 >>> #define EXT4_FEATURE_COMPAT_DIR_INDEX 0x0020 >>> #define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200 >>> +#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x0400 /* Orphan >>> file exists */ >>> >>> #define EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001 >>> #define EXT4_FEATURE_RO_COMPAT_LARGE_FILE 0x0002 >>> @@ -1556,7 +1593,10 @@ static inline void ext4_clear_state_flags(struct >>> ext4_inode_info *ei) >>> * GDT_CSUM bits are mutually exclusive. >>> */ >>> #define EXT4_FEATURE_RO_COMPAT_METADATA_CSUM 0x0400 >>> +/* 0x0800 Reserved for EXT4_FEATURE_RO_COMPAT_REPLICA */ >>> #define EXT4_FEATURE_RO_COMPAT_READONLY 0x1000 >>> +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x2000 /* Orphan file may >>> be >>> + non-empty */ >>> >>> #define EXT4_FEATURE_INCOMPAT_COMPRESSION 0x0001 >>> #define EXT4_FEATURE_INCOMPAT_FILETYPE 0x0002 >>> @@ -1589,7 +1629,8 @@ static inline void ext4_clear_state_flags(struct >>> ext4_inode_info *ei) >>> >>> EXT4_FEATURE_RO_COMPAT_LARGE_FILE| \ >>> EXT4_FEATURE_RO_COMPAT_BTREE_DIR) >>> >>> -#define EXT4_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR >>> +#define EXT4_FEATURE_COMPAT_SUPP (EXT4_FEATURE_COMPAT_EXT_ATTR| \ >>> + EXT4_FEATURE_COMPAT_ORPHAN_FILE) >>> #define EXT4_FEATURE_INCOMPAT_SUPP (EXT4_FEATURE_INCOMPAT_FILETYPE| \ >>> EXT4_FEATURE_INCOMPAT_RECOVER| \ >>> EXT4_FEATURE_INCOMPAT_META_BG| \ >>> @@ -1607,7 +1648,8 @@ static inline void ext4_clear_state_flags(struct >>> ext4_inode_info *ei) >>> EXT4_FEATURE_RO_COMPAT_HUGE_FILE >>> |\ >>> EXT4_FEATURE_RO_COMPAT_BIGALLOC |\ >>> >>> EXT4_FEATURE_RO_COMPAT_METADATA_CSUM|\ >>> - EXT4_FEATURE_RO_COMPAT_QUOTA) >>> + EXT4_FEATURE_RO_COMPAT_QUOTA|\ >>> + >>> EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) >>> >>> /* >>> * Default values for user and/or group using reserved blocks >>> diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c >>> index 460c716e38b0..3436b7fa0ef9 100644 >>> --- a/fs/ext4/namei.c >>> +++ b/fs/ext4/namei.c >>> @@ -2529,6 +2529,46 @@ static int empty_dir(struct inode *inode) >>> return 1; >>> } >>> >>> +static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) >>> +{ >>> + int i, j; >>> + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; >>> + int ret = 0; >>> + __le32 *bdata; >>> + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); >>> + >>> + spin_lock(&oi->of_lock); >>> + for (i = 0; i < oi->of_blocks && !oi->of_binfo[i].ob_free_entries; >>> i++); >>> + if (i == oi->of_blocks) { >>> + spin_unlock(&oi->of_lock); >>> + return -ENOSPC; >>> + } >>> + oi->of_binfo[i].ob_free_entries--; >>> + spin_unlock(&oi->of_lock); >>> + >>> + /* >>> + * Get access to orphan block. We have dropped of_lock but since we >>> + * have decremented number of free entries we are guaranteed free >>> entry >>> + * in our block. >>> + */ >>> + ret = ext4_journal_get_write_access(handle, inode->i_sb, >>> + oi->of_binfo[i].ob_bh, TR_ORPHAN_FILE); >>> + if (ret) >>> + return ret; >>> + >>> + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); >>> + spin_lock(&oi->of_lock); >>> + /* Find empty slot in a block */ >>> + for (j = 0; j < inodes_per_ob && bdata[j]; j++); >>> + BUG_ON(j == inodes_per_ob); >>> + bdata[j] = cpu_to_le32(inode->i_ino); >>> + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; >>> + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); >>> + spin_unlock(&oi->of_lock); >>> + >>> + return ext4_handle_dirty_metadata(handle, NULL, >>> oi->of_binfo[i].ob_bh); >>> +} >>> + >>> /* >>> * ext4_orphan_add() links an unlinked or truncated inode into a list of >>> * such inodes, starting at the superblock, in case we crash before the >>> @@ -2555,10 +2595,10 @@ int ext4_orphan_add(handle_t *handle, struct inode >>> *inode) >>> WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && >>> !mutex_is_locked(&inode->i_mutex)); >>> /* >>> - * Exit early if inode already is on orphan list. This is a big >>> speedup >>> - * since we don't have to contend on the global s_orphan_lock. >>> + * Inode orphaned in orphan file or in orphan list? >>> */ >>> - if (!list_empty(&EXT4_I(inode)->i_orphan)) >>> + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) || >>> + !list_empty(&EXT4_I(inode)->i_orphan)) >>> return 0; >>> >>> /* >>> @@ -2570,6 +2610,16 @@ int ext4_orphan_add(handle_t *handle, struct inode >>> *inode) >>> J_ASSERT((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || >>> S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); >>> >>> + if (sbi->s_orphan_info.of_blocks) { >>> + err = ext4_orphan_file_add(handle, inode); >>> + /* >>> + * Fallback to normal orphan list of orphan file is >>> + * out of space >>> + */ >>> + if (err != -ENOSPC) >>> + return err; >>> + } >>> + >>> BUFFER_TRACE(sbi->s_sbh, "get_write_access"); >>> err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, >>> TR_NONE); >>> if (err) >>> @@ -2618,6 +2668,37 @@ out: >>> return err; >>> } >>> >>> +static int ext4_orphan_file_del(handle_t *handle, struct inode *inode) >>> +{ >>> + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; >>> + __le32 *bdata; >>> + int blk, off; >>> + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); >>> + int ret = 0; >>> + >>> + if (!handle) >>> + goto out; >>> + blk = EXT4_I(inode)->i_orphan_idx / inodes_per_ob; >>> + off = EXT4_I(inode)->i_orphan_idx % inodes_per_ob; >>> + >>> + ret = ext4_journal_get_write_access(handle, inode->i_sb, >>> + oi->of_binfo[blk].ob_bh, TR_ORPHAN_FILE); >>> + if (ret) >>> + goto out; >>> + >>> + bdata = (__le32 *)(oi->of_binfo[blk].ob_bh->b_data); >>> + spin_lock(&oi->of_lock); >>> + bdata[off] = 0; >>> + oi->of_binfo[blk].ob_free_entries++; >>> + spin_unlock(&oi->of_lock); >>> + ret = ext4_handle_dirty_metadata(handle, NULL, >>> oi->of_binfo[blk].ob_bh); >>> +out: >>> + ext4_clear_inode_state(inode, EXT4_STATE_ORPHAN_FILE); >>> + INIT_LIST_HEAD(&EXT4_I(inode)->i_orphan); >>> + >>> + return ret; >>> +} >>> + >>> /* >>> * ext4_orphan_del() removes an unlinked or truncated inode from the list >>> * of such inodes stored on disk, because it is finally being cleaned up. >>> @@ -2636,10 +2717,14 @@ int ext4_orphan_del(handle_t *handle, struct inode >>> *inode) >>> >>> WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && >>> !mutex_is_locked(&inode->i_mutex)); >>> - /* Do this quick check before taking global s_orphan_lock. */ >>> - if (list_empty(&ei->i_orphan)) >>> + /* Do this quick check before taking global lock. */ >>> + if (!ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) && >>> + list_empty(&ei->i_orphan)) >>> return 0; >>> >>> + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE)) >>> + return ext4_orphan_file_del(handle, inode); >>> + >>> if (handle) { >>> /* Grab inode buffer early before taking global >>> s_orphan_lock */ >>> err = ext4_reserve_inode_write(handle, inode, &iloc); >>> diff --git a/fs/ext4/super.c b/fs/ext4/super.c >>> index 0babe8c435b6..14c30a9ef509 100644 >>> --- a/fs/ext4/super.c >>> +++ b/fs/ext4/super.c >>> @@ -761,6 +761,18 @@ static void dump_orphan_list(struct super_block *sb, >>> struct ext4_sb_info *sbi) >>> } >>> } >>> >>> +static void ext4_release_orphan_info(struct super_block *sb) >>> +{ >>> + int i; >>> + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; >>> + >>> + if (!oi->of_blocks) >>> + return; >>> + for (i = 0; i < oi->of_blocks; i++) >>> + brelse(oi->of_binfo[i].ob_bh); >>> + kfree(oi->of_binfo); >>> +} >>> + >>> static void ext4_put_super(struct super_block *sb) >>> { >>> struct ext4_sb_info *sbi = EXT4_SB(sb); >>> @@ -772,6 +784,7 @@ static void ext4_put_super(struct super_block *sb) >>> >>> flush_workqueue(sbi->rsv_conversion_wq); >>> destroy_workqueue(sbi->rsv_conversion_wq); >>> + ext4_release_orphan_info(sb); >>> >>> if (sbi->s_journal) { >>> err = jbd2_journal_destroy(sbi->s_journal); >>> @@ -789,6 +802,8 @@ static void ext4_put_super(struct super_block *sb) >>> >>> if (!(sb->s_flags & MS_RDONLY)) { >>> EXT4_CLEAR_INCOMPAT_FEATURE(sb, >>> EXT4_FEATURE_INCOMPAT_RECOVER); >>> + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, >>> + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); >>> es->s_state = cpu_to_le16(sbi->s_mount_state); >>> } >>> if (!(sb->s_flags & MS_RDONLY)) >>> @@ -1905,8 +1920,14 @@ static int ext4_setup_super(struct super_block *sb, >>> struct ext4_super_block *es, >>> le16_add_cpu(&es->s_mnt_count, 1); >>> es->s_mtime = cpu_to_le32(get_seconds()); >>> ext4_update_dynamic_rev(sb); >>> - if (sbi->s_journal) >>> + if (sbi->s_journal) { >>> EXT4_SET_INCOMPAT_FEATURE(sb, >>> EXT4_FEATURE_INCOMPAT_RECOVER); >>> + if (EXT4_HAS_COMPAT_FEATURE(sb, >>> + >>> EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { >>> + EXT4_SET_RO_COMPAT_FEATURE(sb, >>> + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); >>> + } >>> + } >>> >>> ext4_commit_super(sb, 1); >>> done: >>> @@ -2128,6 +2149,36 @@ static int ext4_check_descriptors(struct >>> super_block *sb, >>> return 1; >>> } >>> >>> +static void ext4_process_orphan(struct inode *inode, >>> + int *nr_truncates, int *nr_orphans) >>> +{ >>> + struct super_block *sb = inode->i_sb; >>> + >>> + dquot_initialize(inode); >>> + if (inode->i_nlink) { >>> + if (test_opt(sb, DEBUG)) >>> + ext4_msg(sb, KERN_DEBUG, >>> + "%s: truncating inode %lu to %lld bytes", >>> + __func__, inode->i_ino, inode->i_size); >>> + jbd_debug(2, "truncating inode %lu to %lld bytes\n", >>> + inode->i_ino, inode->i_size); >>> + mutex_lock(&inode->i_mutex); >>> + truncate_inode_pages(inode->i_mapping, inode->i_size); >>> + ext4_truncate(inode); >>> + mutex_unlock(&inode->i_mutex); >>> + (*nr_truncates)++; >>> + } else { >>> + if (test_opt(sb, DEBUG)) >>> + ext4_msg(sb, KERN_DEBUG, >>> + "%s: deleting unreferenced inode %lu", >>> + __func__, inode->i_ino); >>> + jbd_debug(2, "deleting unreferenced inode %lu\n", >>> + inode->i_ino); >>> + (*nr_orphans)++; >>> + } >>> + iput(inode); /* The delete magic happens here! */ >>> +} >>> + >>> /* ext4_orphan_cleanup() walks a singly-linked list of inodes (starting at >>> * the superblock) which were deleted from all directories, but held open >>> by >>> * a process at the time of a crash. We walk the list and try to delete >>> these >>> @@ -2150,10 +2201,13 @@ static void ext4_orphan_cleanup(struct super_block >>> *sb, >>> { >>> unsigned int s_flags = sb->s_flags; >>> int nr_orphans = 0, nr_truncates = 0; >>> -#ifdef CONFIG_QUOTA >>> - int i; >>> -#endif >>> - if (!es->s_last_orphan) { >>> + int i, j; >>> + __le32 *bdata; >>> + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; >>> + struct inode *inode; >>> + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); >>> + >>> + if (!es->s_last_orphan && !oi->of_blocks) { >>> jbd_debug(4, "no orphan inodes to clean up\n"); >>> return; >>> } >>> @@ -2202,8 +2256,6 @@ static void ext4_orphan_cleanup(struct super_block >>> *sb, >>> #endif >>> >>> while (es->s_last_orphan) { >>> - struct inode *inode; >>> - >>> inode = ext4_orphan_get(sb, >>> le32_to_cpu(es->s_last_orphan)); >>> if (IS_ERR(inode)) { >>> es->s_last_orphan = 0; >>> @@ -2211,29 +2263,21 @@ static void ext4_orphan_cleanup(struct super_block >>> *sb, >>> } >>> >>> list_add(&EXT4_I(inode)->i_orphan, &EXT4_SB(sb)->s_orphan); >>> - dquot_initialize(inode); >>> - if (inode->i_nlink) { >>> - if (test_opt(sb, DEBUG)) >>> - ext4_msg(sb, KERN_DEBUG, >>> - "%s: truncating inode %lu to %lld >>> bytes", >>> - __func__, inode->i_ino, >>> inode->i_size); >>> - jbd_debug(2, "truncating inode %lu to %lld >>> bytes\n", >>> - inode->i_ino, inode->i_size); >>> - mutex_lock(&inode->i_mutex); >>> - truncate_inode_pages(inode->i_mapping, >>> inode->i_size); >>> - ext4_truncate(inode); >>> - mutex_unlock(&inode->i_mutex); >>> - nr_truncates++; >>> - } else { >>> - if (test_opt(sb, DEBUG)) >>> - ext4_msg(sb, KERN_DEBUG, >>> - "%s: deleting unreferenced inode >>> %lu", >>> - __func__, inode->i_ino); >>> - jbd_debug(2, "deleting unreferenced inode %lu\n", >>> - inode->i_ino); >>> - nr_orphans++; >>> + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); >>> + } >>> + >>> + for (i = 0; i < oi->of_blocks; i++) { >>> + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); >>> + for (j = 0; j < inodes_per_ob; j++) { >>> + if (!bdata[j]) >>> + continue; >>> + inode = ext4_orphan_get(sb, le32_to_cpu(bdata[j])); >>> + if (IS_ERR(inode)) >>> + continue; >>> + ext4_set_inode_state(inode, >>> EXT4_STATE_ORPHAN_FILE); >>> + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + >>> j; >>> + ext4_process_orphan(inode, &nr_truncates, >>> &nr_orphans); >>> } >>> - iput(inode); /* The delete magic happens here! */ >>> } >>> >>> #define PLURAL(x) (x), ((x) == 1) ? "" : "s" >>> @@ -3420,6 +3464,97 @@ static void ext4_setup_csum_trigger(struct >>> super_block *sb, >>> sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; >>> } >>> >>> +static int ext4_orphan_file_block_csum_verify(struct super_block *sb, >>> + struct buffer_head *bh) >>> +{ >>> + __u32 provided, calculated; >>> + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); >>> + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; >>> + >>> + if (!ext4_has_metadata_csum(sb)) >>> + return 1; >>> + >>> + provided = le32_to_cpu(((__le32 *)bh->b_data)[inodes_per_ob]); >>> + calculated = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, >>> + (__u8 *)bh->b_data, >>> + inodes_per_ob * sizeof(__u32)); >>> + return provided == calculated; >>> +} >>> + >>> +/* This gets called only when checksumming is enabled */ >>> +static void ext4_orphan_file_block_trigger( >>> + struct jbd2_buffer_trigger_type *triggers, >>> + struct buffer_head *bh, >>> + void *data, size_t size) >>> +{ >>> + struct super_block *sb = EXT4_TRIGGER(triggers)->sb; >>> + __u32 csum; >>> + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); >>> + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; >>> + >>> + csum = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, (__u8 *)data, >>> + inodes_per_ob * sizeof(__u32)); >>> + ((__le32 *)data)[inodes_per_ob] = cpu_to_le32(csum); >>> +} >>> + >>> +static int ext4_init_orphan_info(struct super_block *sb) >>> +{ >>> + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; >>> + struct inode *inode; >>> + int i, j; >>> + int ret; >>> + int free; >>> + __le32 *bdata; >>> + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); >>> + >>> + spin_lock_init(&oi->of_lock); >>> + >>> + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) >>> + return 0; >>> + >>> + inode = ext4_iget(sb, 12 /* FIXME: EXT4_ORPHAN_INO */); >>> + if (IS_ERR(inode)) { >>> + ext4_msg(sb, KERN_ERR, "get orphan inode failed"); >>> + return PTR_ERR(inode); >>> + } >>> + oi->of_blocks = inode->i_size >> sb->s_blocksize_bits; >>> + oi->of_csum_seed = EXT4_I(inode)->i_csum_seed; >>> + oi->of_binfo = kmalloc(oi->of_blocks*sizeof(struct >>> ext4_orphan_block), >>> + GFP_KERNEL); >>> + if (!oi->of_binfo) { >>> + ret = -ENOMEM; >>> + goto out_put; >>> + } >>> + for (i = 0; i < oi->of_blocks; i++) { >>> + oi->of_binfo[i].ob_bh = ext4_bread(NULL, inode, i, 0); >>> + if (IS_ERR(oi->of_binfo[i].ob_bh)) { >>> + ret = PTR_ERR(oi->of_binfo[i].ob_bh); >>> + goto out_free; >>> + } >>> + if (!ext4_orphan_file_block_csum_verify(sb, >>> + oi->of_binfo[i].ob_bh)) { >>> + ext4_error(sb, "orphan file block %d: bad >>> checksum", i); >>> + ret = -EIO; >>> + goto out_free; >>> + } >>> + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); >>> + free = 0; >>> + for (j = 0; j < inodes_per_ob; j++) >>> + if (bdata[j] == 0) >>> + free++; >>> + oi->of_binfo[i].ob_free_entries = free; >>> + } >>> + iput(inode); >>> + return 0; >>> +out_free: >>> + for (i--; i >= 0; i--) >>> + brelse(oi->of_binfo[i].ob_bh); >>> + kfree(oi->of_binfo); >>> +out_put: >>> + iput(inode); >>> + return ret; >>> +} >>> + >>> static int ext4_fill_super(struct super_block *sb, void *data, int silent) >>> { >>> char *orig_data = kstrdup(data, GFP_KERNEL); >>> @@ -3515,6 +3650,8 @@ static int ext4_fill_super(struct super_block *sb, >>> void *data, int silent) >>> silent = 1; >>> goto cantfind_ext4; >>> } >>> + ext4_setup_csum_trigger(sb, TR_ORPHAN_FILE, >>> + ext4_orphan_file_block_trigger); >>> >>> /* Load the checksum driver */ >>> if (EXT4_HAS_RO_COMPAT_FEATURE(sb, >>> @@ -3988,8 +4125,10 @@ static int ext4_fill_super(struct super_block *sb, >>> void *data, int silent) >>> sb->s_root = NULL; >>> >>> needs_recovery = (es->s_last_orphan != 0 || >>> + EXT4_HAS_RO_COMPAT_FEATURE(sb, >>> + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) || >>> EXT4_HAS_INCOMPAT_FEATURE(sb, >>> - EXT4_FEATURE_INCOMPAT_RECOVER)); >>> + EXT4_FEATURE_INCOMPAT_RECOVER)); >>> >>> if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_MMP) && >>> !(sb->s_flags & MS_RDONLY)) >>> @@ -4207,13 +4346,16 @@ no_journal: >>> if (err) >>> goto failed_mount7; >>> >>> + err = ext4_init_orphan_info(sb); >>> + if (err) >>> + goto failed_mount8; >>> #ifdef CONFIG_QUOTA >>> /* Enable quota usage during mount. */ >>> if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_QUOTA) && >>> !(sb->s_flags & MS_RDONLY)) { >>> err = ext4_enable_quotas(sb); >>> if (err) >>> - goto failed_mount8; >>> + goto failed_mount9; >>> } >>> #endif /* CONFIG_QUOTA */ >>> >>> @@ -4263,9 +4405,11 @@ cantfind_ext4: >>> goto failed_mount; >>> >>> #ifdef CONFIG_QUOTA >>> +failed_mount9: >>> + ext4_release_orphan_info(sb); >>> +#endif >>> failed_mount8: >>> kobject_del(&sbi->s_kobj); >>> -#endif >>> failed_mount7: >>> ext4_unregister_li_request(sb); >>> failed_mount6: >>> @@ -4771,6 +4915,20 @@ static int ext4_sync_fs(struct super_block *sb, int >>> wait) >>> return ret; >>> } >>> >>> +static int ext4_orphan_file_empty(struct super_block *sb) >>> +{ >>> + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; >>> + int i; >>> + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); >>> + >>> + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) >>> + return 1; >>> + for (i = 0; i < oi->of_blocks; i++) >>> + if (oi->of_binfo[i].ob_free_entries != inodes_per_ob) >>> + return 0; >>> + return 1; >>> +} >>> + >>> /* >>> * LVM calls this function before a (read-only) snapshot is created. This >>> * gives us a chance to flush the journal completely and mark the fs >>> clean. >>> @@ -4804,6 +4962,10 @@ static int ext4_freeze(struct super_block *sb) >>> >>> /* Journal blocked and flushed, clear needs_recovery flag. */ >>> EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); >>> + if (ext4_orphan_file_empty(sb)) { >>> + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, >>> + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); >>> + } >>> error = ext4_commit_super(sb, 1); >>> out: >>> if (journal) >>> @@ -4823,6 +4985,10 @@ static int ext4_unfreeze(struct super_block *sb) >>> >>> /* Reset the needs_recovery flag before the fs is unlocked. */ >>> EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); >>> + if (EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { >>> + EXT4_SET_RO_COMPAT_FEATURE(sb, >>> + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); >>> + } >>> ext4_commit_super(sb, 1); >>> return 0; >>> } >>> @@ -4966,8 +5132,13 @@ static int ext4_remount(struct super_block *sb, int >>> *flags, char *data) >>> (sbi->s_mount_state & EXT4_VALID_FS)) >>> es->s_state = >>> cpu_to_le16(sbi->s_mount_state); >>> >>> - if (sbi->s_journal) >>> + if (sbi->s_journal) { >>> ext4_mark_recovery_complete(sb, es); >>> + if (ext4_orphan_file_empty(sb)) { >>> + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, >>> + >>> EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); >>> + } >>> + } >>> } else { >>> /* Make sure we can mount this feature set >>> readwrite */ >>> if (EXT4_HAS_RO_COMPAT_FEATURE(sb, >>> -- >>> 2.1.4 >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> > -- > Jan Kara <jack@suse.cz> > SUSE Labs, CR > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Cheers, Andreas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-16 15:42 ` [PATCH 2/3] ext4: Speedup ext4 orphan inode handling Jan Kara [not found] ` <CAOQ4uxifVr1swHb5Y2M-TRuzwdDo-z92G6PuHvBGecGZ7nYuHg@mail.gmail.com> @ 2015-04-17 23:53 ` Andreas Dilger 2015-04-18 1:13 ` Darrick J. Wong 2015-04-20 12:25 ` Jan Kara 2015-04-18 23:53 ` Theodore Ts'o 2 siblings, 2 replies; 18+ messages in thread From: Andreas Dilger @ 2015-04-17 23:53 UTC (permalink / raw) To: Jan Kara; +Cc: linux-ext4 On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: > > Ext4 orphan inode handling is a bottleneck for workloads which heavily > truncate / unlink small files since it contends on the global > s_orphan_mutex lock (and generally it's difficult to improve scalability > of the ondisk linked list of orphaned inodes). > > This patch implements new way of handling orphan inodes. Instead of > linking orphaned inode into a linked list, we store it's inode number in > a new special file which we call "orphan file". Currently we still > protect the orphan file with a spinlock for simplicity but even in this > setting we can substantially reduce the length of the critical section > and thus speedup some workloads. > > Note that the change is backwards compatible when the filesystem is > clean - the existence of the orphan file is a compat feature, we set > another ro-compat feature indicating orphan file needs scanning for > orphaned inodes when mounting filesystem read-write. This ro-compat > feature gets cleared on unmount / remount read-only. > > Some performance data from 48 CPU Xeon Server with 32 GB of RAM, > filesystem located on ramdisk, average of 5 runs: > > stress-orphan (microbenchmark truncating files byte-by-byte from N > processes in parallel) > > Threads Time Time > Vanilla Patched > 1 1.602800 1.260000 > 2 4.292200 2.455000 > 4 6.202800 3.848400 > 8 10.415000 6.833000 > 16 18.933600 12.883200 > 32 38.517200 25.342200 > 64 79.805000 50.918400 > 128 159.629200 102.666000 > > reaim new_fserver workload (tweaked to avoid calling sync(1) after every > operation) > > Threads Jobs/s Jobs/s > Vanilla Patched > 1 24375.00 22941.18 > 25 162162.16 278571.43 > 49 222209.30 331626.90 > 73 280147.60 419447.52 > 97 315250.00 481910.83 > 121 331157.90 503360.00 > 145 343769.00 489081.08 > 169 355549.56 519487.68 > 193 356518.65 501800.00 > > So in both cases we see significant wins all over the board. One thing I noticed looking at this patch is that there is quite a bit of orphan handling code now. Is it worthwhile to move it into its own file and make super.c a bit smaller? > Signed-off-by: Jan Kara <jack@suse.cz> > --- > fs/ext4/ext4.h | 52 +++++++++++-- > fs/ext4/namei.c | 95 +++++++++++++++++++++-- > fs/ext4/super.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++-------- > 3 files changed, 341 insertions(+), 43 deletions(-) > > diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > index abed83485915..768a8b9ee2f9 100644 > --- a/fs/ext4/ext4.h > +++ b/fs/ext4/ext4.h > @@ -208,6 +208,7 @@ struct ext4_io_submit { > #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ > #define EXT4_RESIZE_INO 7 /* Reserved group descriptors inode */ > #define EXT4_JOURNAL_INO 8 /* Journal inode */ > +#define EXT4_ORPHAN_INO 9 /* Inode with orphan entries */ Contrary to your patch description that said this was using ino #12, this conflicts with EXT2_EXCLUDE_INO for snapshots. Why not use s_last_orphan in the superblock to reference the orphan inode? Since this feature already requires a new EXT2_FEATURE_RO_COMPAT flag, the existing orphan inode number could be reused. See below how this could still in the ENOSPC case. > +static inline int ext4_inodes_per_orphan_block(struct super_block *sb) > +{ > + /* We reserve 1 entry for block checksum */ Would be good to improve this comment to say "first entry" or "last entry". > + return sb->s_blocksize / sizeof(u32) - 1; > +} What do you think about making the on-disk orphan inode numbers store 64-bit values? That would be easy to do now, and would avoid a format change in the future if we wanted to use 64-bit inodes. That said, if the orphan inode is deleted after orphan recovery (see more below) the only thing needed for compatibility is to store the inode number size into the orphan inode somewhere so it could be changed. Maybe i_version and/or i_generation since they are not directly user accessible. This would also allow you to detect if s_last_orphan was the orphan inode without burning an EXT4_*_FL inode flag for a one-file-only usage. > #define EXT4_FEATURE_COMPAT_RESIZE_INODE 0x0010 > #define EXT4_FEATURE_COMPAT_DIR_INDEX 0x0020 > #define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200 > +#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x0400 /* Orphan file exists */ > > #define EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001 > #define EXT4_FEATURE_RO_COMPAT_LARGE_FILE 0x0002 > @@ -1556,7 +1593,10 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > * GDT_CSUM bits are mutually exclusive. > */ > #define EXT4_FEATURE_RO_COMPAT_METADATA_CSUM 0x0400 > +/* 0x0800 Reserved for EXT4_FEATURE_RO_COMPAT_REPLICA */ > #define EXT4_FEATURE_RO_COMPAT_READONLY 0x1000 > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x2000 /* Orphan file may be > + non-empty */ > > #define EXT4_FEATURE_INCOMPAT_COMPRESSION 0x0001 > #define EXT4_FEATURE_INCOMPAT_FILETYPE 0x0002 > @@ -1589,7 +1629,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > EXT4_FEATURE_RO_COMPAT_LARGE_FILE| \ > EXT4_FEATURE_RO_COMPAT_BTREE_DIR) > > -#define EXT4_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR > +#define EXT4_FEATURE_COMPAT_SUPP (EXT4_FEATURE_COMPAT_EXT_ATTR| \ > + EXT4_FEATURE_COMPAT_ORPHAN_FILE) > #define EXT4_FEATURE_INCOMPAT_SUPP (EXT4_FEATURE_INCOMPAT_FILETYPE| \ > EXT4_FEATURE_INCOMPAT_RECOVER| \ > EXT4_FEATURE_INCOMPAT_META_BG| \ > @@ -1607,7 +1648,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > EXT4_FEATURE_RO_COMPAT_HUGE_FILE |\ > EXT4_FEATURE_RO_COMPAT_BIGALLOC |\ > EXT4_FEATURE_RO_COMPAT_METADATA_CSUM|\ > - EXT4_FEATURE_RO_COMPAT_QUOTA) > + EXT4_FEATURE_RO_COMPAT_QUOTA|\ > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) > > /* > * Default values for user and/or group using reserved blocks > diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c > index 460c716e38b0..3436b7fa0ef9 100644 > --- a/fs/ext4/namei.c > +++ b/fs/ext4/namei.c > @@ -2529,6 +2529,46 @@ static int empty_dir(struct inode *inode) > return 1; > } > > +static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) > +{ > + int i, j; > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > + int ret = 0; > + __le32 *bdata; > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > + > + spin_lock(&oi->of_lock); > + for (i = 0; i < oi->of_blocks && !oi->of_binfo[i].ob_free_entries; i++); > + if (i == oi->of_blocks) { > + spin_unlock(&oi->of_lock); > + return -ENOSPC; > + } > + oi->of_binfo[i].ob_free_entries--; > + spin_unlock(&oi->of_lock); > + > + /* > + * Get access to orphan block. We have dropped of_lock but since we > + * have decremented number of free entries we are guaranteed free entry > + * in our block. > + */ > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > + oi->of_binfo[i].ob_bh, TR_ORPHAN_FILE); > + if (ret) > + return ret; > + > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > + spin_lock(&oi->of_lock); > + /* Find empty slot in a block */ > + for (j = 0; j < inodes_per_ob && bdata[j]; j++); > + BUG_ON(j == inodes_per_ob); > + bdata[j] = cpu_to_le32(inode->i_ino); > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; > + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > + spin_unlock(&oi->of_lock); > + > + return ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[i].ob_bh); > +} > + > /* > * ext4_orphan_add() links an unlinked or truncated inode into a list of > * such inodes, starting at the superblock, in case we crash before the > @@ -2555,10 +2595,10 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > !mutex_is_locked(&inode->i_mutex)); > /* > - * Exit early if inode already is on orphan list. This is a big speedup > - * since we don't have to contend on the global s_orphan_lock. > + * Inode orphaned in orphan file or in orphan list? > */ > - if (!list_empty(&EXT4_I(inode)->i_orphan)) > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) || > + !list_empty(&EXT4_I(inode)->i_orphan)) > return 0; > > @@ -2570,6 +2610,16 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) > J_ASSERT((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || > S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); > > + if (sbi->s_orphan_info.of_blocks) { > + err = ext4_orphan_file_add(handle, inode); > + /* > + * Fallback to normal orphan list of orphan file is > + * out of space > + */ > + if (err != -ENOSPC) > + return err; > + } Hmm, that is sad, but I guess it is necessary to be able to unlink an inode if the filesystem is completely full. Otherwise there won't be any space to free space... That said, the ENOSPC orphan list could link onto the orphan inode itself if it is directly referenced from s_last_orphan. Then, maybe conveniently, if the orphan inode itself is deleted at the end of orphan inode processing it would free all of the allocated blocks and avoid the problem of growing to a large size and never freeing blocks. > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); > if (err) > @@ -2618,6 +2668,37 @@ out: > return err; > } > > +static int ext4_orphan_file_del(handle_t *handle, struct inode *inode) > +{ > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > + __le32 *bdata; > + int blk, off; > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > + int ret = 0; > + > + if (!handle) > + goto out; > + blk = EXT4_I(inode)->i_orphan_idx / inodes_per_ob; > + off = EXT4_I(inode)->i_orphan_idx % inodes_per_ob; > + > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > + oi->of_binfo[blk].ob_bh, TR_ORPHAN_FILE); > + if (ret) > + goto out; > + > + bdata = (__le32 *)(oi->of_binfo[blk].ob_bh->b_data); > + spin_lock(&oi->of_lock); > + bdata[off] = 0; > + oi->of_binfo[blk].ob_free_entries++; > + spin_unlock(&oi->of_lock); > + ret = ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[blk].ob_bh); > +out: > + ext4_clear_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > + INIT_LIST_HEAD(&EXT4_I(inode)->i_orphan); > + > + return ret; > +} > + > /* > * ext4_orphan_del() removes an unlinked or truncated inode from the list > * of such inodes stored on disk, because it is finally being cleaned up. > @@ -2636,10 +2717,14 @@ int ext4_orphan_del(handle_t *handle, struct inode *inode) > > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > !mutex_is_locked(&inode->i_mutex)); > - /* Do this quick check before taking global s_orphan_lock. */ > - if (list_empty(&ei->i_orphan)) > + /* Do this quick check before taking global lock. */ > + if (!ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) && > + list_empty(&ei->i_orphan)) > return 0; > > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE)) > + return ext4_orphan_file_del(handle, inode); This could go before the above check, and then go back to only checking list_empty() instead of checking STATE_ORPHAN_FILE twice. > if (handle) { > /* Grab inode buffer early before taking global s_orphan_lock */ > err = ext4_reserve_inode_write(handle, inode, &iloc); > diff --git a/fs/ext4/super.c b/fs/ext4/super.c > index 0babe8c435b6..14c30a9ef509 100644 > --- a/fs/ext4/super.c > +++ b/fs/ext4/super.c > @@ -761,6 +761,18 @@ static void dump_orphan_list(struct super_block *sb, struct ext4_sb_info *sbi) > } > } > > +static void ext4_release_orphan_info(struct super_block *sb) > +{ > + int i; > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > + > + if (!oi->of_blocks) > + return; > + for (i = 0; i < oi->of_blocks; i++) > + brelse(oi->of_binfo[i].ob_bh); > + kfree(oi->of_binfo); > +} > + > static void ext4_put_super(struct super_block *sb) > { > struct ext4_sb_info *sbi = EXT4_SB(sb); > @@ -772,6 +784,7 @@ static void ext4_put_super(struct super_block *sb) > > flush_workqueue(sbi->rsv_conversion_wq); > destroy_workqueue(sbi->rsv_conversion_wq); > + ext4_release_orphan_info(sb); > > if (sbi->s_journal) { > err = jbd2_journal_destroy(sbi->s_journal); > @@ -789,6 +802,8 @@ static void ext4_put_super(struct super_block *sb) > > if (!(sb->s_flags & MS_RDONLY)) { > EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > es->s_state = cpu_to_le16(sbi->s_mount_state); > } > if (!(sb->s_flags & MS_RDONLY)) > @@ -1905,8 +1920,14 @@ static int ext4_setup_super(struct super_block *sb, struct ext4_super_block *es, > le16_add_cpu(&es->s_mnt_count, 1); > es->s_mtime = cpu_to_le32(get_seconds()); > ext4_update_dynamic_rev(sb); > - if (sbi->s_journal) > + if (sbi->s_journal) { > EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > + if (EXT4_HAS_COMPAT_FEATURE(sb, > + EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { > + EXT4_SET_RO_COMPAT_FEATURE(sb, > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > + } > + } Shouldn't this only be set the first time an orphan is created? > @@ -2128,6 +2149,36 @@ static int ext4_check_descriptors(struct super_block *sb, > return 1; > } > > +static void ext4_process_orphan(struct inode *inode, > + int *nr_truncates, int *nr_orphans) > +{ > + struct super_block *sb = inode->i_sb; > + > + dquot_initialize(inode); > + if (inode->i_nlink) { > + if (test_opt(sb, DEBUG)) > + ext4_msg(sb, KERN_DEBUG, > + "%s: truncating inode %lu to %lld bytes", > + __func__, inode->i_ino, inode->i_size); > + jbd_debug(2, "truncating inode %lu to %lld bytes\n", > + inode->i_ino, inode->i_size); > + mutex_lock(&inode->i_mutex); > + truncate_inode_pages(inode->i_mapping, inode->i_size); > + ext4_truncate(inode); > + mutex_unlock(&inode->i_mutex); > + (*nr_truncates)++; > + } else { > + if (test_opt(sb, DEBUG)) > + ext4_msg(sb, KERN_DEBUG, > + "%s: deleting unreferenced inode %lu", > + __func__, inode->i_ino); > + jbd_debug(2, "deleting unreferenced inode %lu\n", > + inode->i_ino); > + (*nr_orphans)++; > + } > + iput(inode); /* The delete magic happens here! */ > +} > + > /* ext4_orphan_cleanup() walks a singly-linked list of inodes (starting at > * the superblock) which were deleted from all directories, but held open by > * a process at the time of a crash. We walk the list and try to delete these > @@ -2150,10 +2201,13 @@ static void ext4_orphan_cleanup(struct super_block *sb, > { > unsigned int s_flags = sb->s_flags; > int nr_orphans = 0, nr_truncates = 0; > -#ifdef CONFIG_QUOTA > - int i; > -#endif > - if (!es->s_last_orphan) { > + int i, j; > + __le32 *bdata; > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > + struct inode *inode; > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > + > + if (!es->s_last_orphan && !oi->of_blocks) { > jbd_debug(4, "no orphan inodes to clean up\n"); > return; > } > @@ -2202,8 +2256,6 @@ static void ext4_orphan_cleanup(struct super_block *sb, > #endif > > while (es->s_last_orphan) { > - struct inode *inode; > - > inode = ext4_orphan_get(sb, le32_to_cpu(es->s_last_orphan)); > if (IS_ERR(inode)) { > es->s_last_orphan = 0; > @@ -2211,29 +2263,21 @@ static void ext4_orphan_cleanup(struct super_block *sb, > } > > list_add(&EXT4_I(inode)->i_orphan, &EXT4_SB(sb)->s_orphan); > - dquot_initialize(inode); > - if (inode->i_nlink) { > - if (test_opt(sb, DEBUG)) > - ext4_msg(sb, KERN_DEBUG, > - "%s: truncating inode %lu to %lld bytes", > - __func__, inode->i_ino, inode->i_size); > - jbd_debug(2, "truncating inode %lu to %lld bytes\n", > - inode->i_ino, inode->i_size); > - mutex_lock(&inode->i_mutex); > - truncate_inode_pages(inode->i_mapping, inode->i_size); > - ext4_truncate(inode); > - mutex_unlock(&inode->i_mutex); > - nr_truncates++; > - } else { > - if (test_opt(sb, DEBUG)) > - ext4_msg(sb, KERN_DEBUG, > - "%s: deleting unreferenced inode %lu", > - __func__, inode->i_ino); > - jbd_debug(2, "deleting unreferenced inode %lu\n", > - inode->i_ino); > - nr_orphans++; > + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); > + } > + > + for (i = 0; i < oi->of_blocks; i++) { > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > + for (j = 0; j < inodes_per_ob; j++) { > + if (!bdata[j]) > + continue; > + inode = ext4_orphan_get(sb, le32_to_cpu(bdata[j])); > + if (IS_ERR(inode)) > + continue; > + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; > + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); > } > - iput(inode); /* The delete magic happens here! */ > } > > #define PLURAL(x) (x), ((x) == 1) ? "" : "s" > @@ -3420,6 +3464,97 @@ static void ext4_setup_csum_trigger(struct super_block *sb, > sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; > } > > +static int ext4_orphan_file_block_csum_verify(struct super_block *sb, > + struct buffer_head *bh) > +{ > + __u32 provided, calculated; > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > + > + if (!ext4_has_metadata_csum(sb)) > + return 1; Does it make sense to always checksum the orphan blocks, even if the whole filesystem is not being checksummed? Corruption in the orphan handling would result in orphan processing of random inode numbers. Not fatal, I guess, since the only thing orphan processing does today is truncate the file to the current i_size and unlink it only if i_nlink == 0, which should be safe on normal files (at worst losing fallocate'd blocks beyond i_size). Still, better safe than sorry? That would also test the checksum callbacks on an ongoing basis, instead of only when both metadata_csum and orphan_inode are enabled. > + provided = le32_to_cpu(((__le32 *)bh->b_data)[inodes_per_ob]); > + calculated = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, > + (__u8 *)bh->b_data, > + inodes_per_ob * sizeof(__u32)); > + return provided == calculated; > +} > + > +/* This gets called only when checksumming is enabled */ > +static void ext4_orphan_file_block_trigger( > + struct jbd2_buffer_trigger_type *triggers, > + struct buffer_head *bh, > + void *data, size_t size) > +{ > + struct super_block *sb = EXT4_TRIGGER(triggers)->sb; > + __u32 csum; > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > + > + csum = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, (__u8 *)data, > + inodes_per_ob * sizeof(__u32)); > + ((__le32 *)data)[inodes_per_ob] = cpu_to_le32(csum); > +} > + > +static int ext4_init_orphan_info(struct super_block *sb) > +{ > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > + struct inode *inode; > + int i, j; > + int ret; > + int free; > + __le32 *bdata; > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > + > + spin_lock_init(&oi->of_lock); > + > + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) > + return 0; > + > + inode = ext4_iget(sb, 12 /* FIXME: EXT4_ORPHAN_INO */); This would corrupt lost+found on most filesystems, better to use the existing EXT4_ORPHAN_INO = 9 for testing, since it will at least not exist on most filesystems. Since the orphan inode blocks will be allocated one-at-a-time after long intervals, like an O_APPEND file, the orphan inode should not be extent-mapped, but rather block-mapped for efficiency. Since there is already a fallback to handle ENOSPC, the chance of not being able to get a single free block in the first 2^32 blocks is not worth the 3x overhead of extent block addresses for this inode. > + if (IS_ERR(inode)) { > + ext4_msg(sb, KERN_ERR, "get orphan inode failed"); > + return PTR_ERR(inode); > + } > + oi->of_blocks = inode->i_size >> sb->s_blocksize_bits; > + oi->of_csum_seed = EXT4_I(inode)->i_csum_seed; > + oi->of_binfo = kmalloc(oi->of_blocks*sizeof(struct ext4_orphan_block), > + GFP_KERNEL); > + if (!oi->of_binfo) { > + ret = -ENOMEM; > + goto out_put; > + } > + for (i = 0; i < oi->of_blocks; i++) { > + oi->of_binfo[i].ob_bh = ext4_bread(NULL, inode, i, 0); > + if (IS_ERR(oi->of_binfo[i].ob_bh)) { > + ret = PTR_ERR(oi->of_binfo[i].ob_bh); > + goto out_free; > + } > + if (!ext4_orphan_file_block_csum_verify(sb, > + oi->of_binfo[i].ob_bh)) { > + ext4_error(sb, "orphan file block %d: bad checksum", i); > + ret = -EIO; > + goto out_free; Should this continue to the next block instead of aborting if a single block is bad? If it is always checking the checksum it should be safe to try the later blocks as well. > + } > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > + free = 0; > + for (j = 0; j < inodes_per_ob; j++) > + if (bdata[j] == 0) > + free++; > + oi->of_binfo[i].ob_free_entries = free; > + } > + iput(inode); > + return 0; > +out_free: > + for (i--; i >= 0; i--) > + brelse(oi->of_binfo[i].ob_bh); > + kfree(oi->of_binfo); > +out_put: > + iput(inode); > + return ret; > +} > + > static int ext4_fill_super(struct super_block *sb, void *data, int silent) > { > char *orig_data = kstrdup(data, GFP_KERNEL); > @@ -3515,6 +3650,8 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) > silent = 1; > goto cantfind_ext4; > } > + ext4_setup_csum_trigger(sb, TR_ORPHAN_FILE, > + ext4_orphan_file_block_trigger); > > /* Load the checksum driver */ > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, > @@ -3988,8 +4125,10 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) > sb->s_root = NULL; > > needs_recovery = (es->s_last_orphan != 0 || > + EXT4_HAS_RO_COMPAT_FEATURE(sb, > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) || > EXT4_HAS_INCOMPAT_FEATURE(sb, > - EXT4_FEATURE_INCOMPAT_RECOVER)); > + EXT4_FEATURE_INCOMPAT_RECOVER)); > > if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_MMP) && > !(sb->s_flags & MS_RDONLY)) > @@ -4207,13 +4346,16 @@ no_journal: > if (err) > goto failed_mount7; > > + err = ext4_init_orphan_info(sb); > + if (err) > + goto failed_mount8; > #ifdef CONFIG_QUOTA > /* Enable quota usage during mount. */ > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_QUOTA) && > !(sb->s_flags & MS_RDONLY)) { > err = ext4_enable_quotas(sb); > if (err) > - goto failed_mount8; > + goto failed_mount9; > } > #endif /* CONFIG_QUOTA */ > > @@ -4263,9 +4405,11 @@ cantfind_ext4: > goto failed_mount; > > #ifdef CONFIG_QUOTA > +failed_mount9: > + ext4_release_orphan_info(sb); > +#endif > failed_mount8: > kobject_del(&sbi->s_kobj); > -#endif > failed_mount7: > ext4_unregister_li_request(sb); > failed_mount6: > @@ -4771,6 +4915,20 @@ static int ext4_sync_fs(struct super_block *sb, int wait) > return ret; > } > > +static int ext4_orphan_file_empty(struct super_block *sb) > +{ > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > + int i; > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > + > + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) > + return 1; > + for (i = 0; i < oi->of_blocks; i++) > + if (oi->of_binfo[i].ob_free_entries != inodes_per_ob) > + return 0; > + return 1; > +} > + > /* > * LVM calls this function before a (read-only) snapshot is created. This > * gives us a chance to flush the journal completely and mark the fs clean. > @@ -4804,6 +4962,10 @@ static int ext4_freeze(struct super_block *sb) > > /* Journal blocked and flushed, clear needs_recovery flag. */ > EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > + if (ext4_orphan_file_empty(sb)) { > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > + } > error = ext4_commit_super(sb, 1); > out: > if (journal) > @@ -4823,6 +4985,10 @@ static int ext4_unfreeze(struct super_block *sb) > > /* Reset the needs_recovery flag before the fs is unlocked. */ > EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > + if (EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { > + EXT4_SET_RO_COMPAT_FEATURE(sb, > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > + } > ext4_commit_super(sb, 1); > return 0; > } > @@ -4966,8 +5132,13 @@ static int ext4_remount(struct super_block *sb, int *flags, char *data) > (sbi->s_mount_state & EXT4_VALID_FS)) > es->s_state = cpu_to_le16(sbi->s_mount_state); > > - if (sbi->s_journal) > + if (sbi->s_journal) { > ext4_mark_recovery_complete(sb, es); > + if (ext4_orphan_file_empty(sb)) { > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > + } > + } > } else { > /* Make sure we can mount this feature set readwrite */ > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, > -- > 2.1.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Cheers, Andreas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-17 23:53 ` Andreas Dilger @ 2015-04-18 1:13 ` Darrick J. Wong 2015-04-20 12:34 ` Jan Kara 2015-04-20 12:25 ` Jan Kara 1 sibling, 1 reply; 18+ messages in thread From: Darrick J. Wong @ 2015-04-18 1:13 UTC (permalink / raw) To: Andreas Dilger; +Cc: Jan Kara, linux-ext4 On Fri, Apr 17, 2015 at 05:53:03PM -0600, Andreas Dilger wrote: > On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: > > > > Ext4 orphan inode handling is a bottleneck for workloads which heavily > > truncate / unlink small files since it contends on the global > > s_orphan_mutex lock (and generally it's difficult to improve scalability > > of the ondisk linked list of orphaned inodes). > > > > This patch implements new way of handling orphan inodes. Instead of > > linking orphaned inode into a linked list, we store it's inode number in > > a new special file which we call "orphan file". Currently we still > > protect the orphan file with a spinlock for simplicity but even in this > > setting we can substantially reduce the length of the critical section > > and thus speedup some workloads. > > > > Note that the change is backwards compatible when the filesystem is > > clean - the existence of the orphan file is a compat feature, we set > > another ro-compat feature indicating orphan file needs scanning for > > orphaned inodes when mounting filesystem read-write. This ro-compat > > feature gets cleared on unmount / remount read-only. > > > > Some performance data from 48 CPU Xeon Server with 32 GB of RAM, > > filesystem located on ramdisk, average of 5 runs: > > > > stress-orphan (microbenchmark truncating files byte-by-byte from N > > processes in parallel) > > > > Threads Time Time > > Vanilla Patched > > 1 1.602800 1.260000 > > 2 4.292200 2.455000 > > 4 6.202800 3.848400 > > 8 10.415000 6.833000 > > 16 18.933600 12.883200 > > 32 38.517200 25.342200 > > 64 79.805000 50.918400 > > 128 159.629200 102.666000 > > > > reaim new_fserver workload (tweaked to avoid calling sync(1) after every > > operation) > > > > Threads Jobs/s Jobs/s > > Vanilla Patched > > 1 24375.00 22941.18 > > 25 162162.16 278571.43 > > 49 222209.30 331626.90 > > 73 280147.60 419447.52 > > 97 315250.00 481910.83 > > 121 331157.90 503360.00 > > 145 343769.00 489081.08 > > 169 355549.56 519487.68 > > 193 356518.65 501800.00 > > > > So in both cases we see significant wins all over the board. > > One thing I noticed looking at this patch is that there is quite a bit > of orphan handling code now. Is it worthwhile to move it into its own > file and make super.c a bit smaller? > > > Signed-off-by: Jan Kara <jack@suse.cz> > > --- > > fs/ext4/ext4.h | 52 +++++++++++-- > > fs/ext4/namei.c | 95 +++++++++++++++++++++-- > > fs/ext4/super.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++-------- > > 3 files changed, 341 insertions(+), 43 deletions(-) > > > > diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > > index abed83485915..768a8b9ee2f9 100644 > > --- a/fs/ext4/ext4.h > > +++ b/fs/ext4/ext4.h > > @@ -208,6 +208,7 @@ struct ext4_io_submit { > > #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ > > #define EXT4_RESIZE_INO 7 /* Reserved group descriptors inode */ > > #define EXT4_JOURNAL_INO 8 /* Journal inode */ > > +#define EXT4_ORPHAN_INO 9 /* Inode with orphan entries */ > > Contrary to your patch description that said this was using ino #12, > this conflicts with EXT2_EXCLUDE_INO for snapshots. Why not use > s_last_orphan in the superblock to reference the orphan inode? Since > this feature already requires a new EXT2_FEATURE_RO_COMPAT flag, the > existing orphan inode number could be reused. See below how this could > still in the ENOSPC case. > > > +static inline int ext4_inodes_per_orphan_block(struct super_block *sb) > > +{ > > + /* We reserve 1 entry for block checksum */ > > Would be good to improve this comment to say "first entry" or "last entry". > > > + return sb->s_blocksize / sizeof(u32) - 1; > > +} Just to be clear, the format of each orphaned inode block is ... an array of 32-bit inode numbers with a 32-bit checksum at the end? Shouldn't we have a magic number somewhere for positive identification? > What do you think about making the on-disk orphan inode numbers store > 64-bit values? That would be easy to do now, and would avoid a format > change in the future if we wanted to use 64-bit inodes. > > That said, if the orphan inode is deleted after orphan recovery (see more > below) the only thing needed for compatibility is to store the inode number > size into the orphan inode somewhere so it could be changed. Maybe > i_version and/or i_generation since they are not directly user accessible. > > This would also allow you to detect if s_last_orphan was the orphan inode > without burning an EXT4_*_FL inode flag for a one-file-only usage. > > > #define EXT4_FEATURE_COMPAT_RESIZE_INODE 0x0010 > > #define EXT4_FEATURE_COMPAT_DIR_INDEX 0x0020 > > #define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200 > > +#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x0400 /* Orphan file exists */ > > > > #define EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001 > > #define EXT4_FEATURE_RO_COMPAT_LARGE_FILE 0x0002 > > @@ -1556,7 +1593,10 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > > * GDT_CSUM bits are mutually exclusive. > > */ > > #define EXT4_FEATURE_RO_COMPAT_METADATA_CSUM 0x0400 > > +/* 0x0800 Reserved for EXT4_FEATURE_RO_COMPAT_REPLICA */ > > #define EXT4_FEATURE_RO_COMPAT_READONLY 0x1000 > > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x2000 /* Orphan file may be > > + non-empty */ > > > > #define EXT4_FEATURE_INCOMPAT_COMPRESSION 0x0001 > > #define EXT4_FEATURE_INCOMPAT_FILETYPE 0x0002 > > @@ -1589,7 +1629,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > > EXT4_FEATURE_RO_COMPAT_LARGE_FILE| \ > > EXT4_FEATURE_RO_COMPAT_BTREE_DIR) > > > > -#define EXT4_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR > > +#define EXT4_FEATURE_COMPAT_SUPP (EXT4_FEATURE_COMPAT_EXT_ATTR| \ > > + EXT4_FEATURE_COMPAT_ORPHAN_FILE) > > #define EXT4_FEATURE_INCOMPAT_SUPP (EXT4_FEATURE_INCOMPAT_FILETYPE| \ > > EXT4_FEATURE_INCOMPAT_RECOVER| \ > > EXT4_FEATURE_INCOMPAT_META_BG| \ > > @@ -1607,7 +1648,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > > EXT4_FEATURE_RO_COMPAT_HUGE_FILE |\ > > EXT4_FEATURE_RO_COMPAT_BIGALLOC |\ > > EXT4_FEATURE_RO_COMPAT_METADATA_CSUM|\ > > - EXT4_FEATURE_RO_COMPAT_QUOTA) > > + EXT4_FEATURE_RO_COMPAT_QUOTA|\ > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) > > > > /* > > * Default values for user and/or group using reserved blocks > > diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c > > index 460c716e38b0..3436b7fa0ef9 100644 > > --- a/fs/ext4/namei.c > > +++ b/fs/ext4/namei.c > > @@ -2529,6 +2529,46 @@ static int empty_dir(struct inode *inode) > > return 1; > > } > > > > +static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) > > +{ > > + int i, j; > > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > > + int ret = 0; > > + __le32 *bdata; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > > + > > + spin_lock(&oi->of_lock); > > + for (i = 0; i < oi->of_blocks && !oi->of_binfo[i].ob_free_entries; i++); > > + if (i == oi->of_blocks) { > > + spin_unlock(&oi->of_lock); > > + return -ENOSPC; > > + } > > + oi->of_binfo[i].ob_free_entries--; > > + spin_unlock(&oi->of_lock); > > + > > + /* > > + * Get access to orphan block. We have dropped of_lock but since we > > + * have decremented number of free entries we are guaranteed free entry > > + * in our block. > > + */ > > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > > + oi->of_binfo[i].ob_bh, TR_ORPHAN_FILE); > > + if (ret) > > + return ret; > > + > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + spin_lock(&oi->of_lock); > > + /* Find empty slot in a block */ > > + for (j = 0; j < inodes_per_ob && bdata[j]; j++); > > + BUG_ON(j == inodes_per_ob); > > + bdata[j] = cpu_to_le32(inode->i_ino); > > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; > > + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + spin_unlock(&oi->of_lock); > > + > > + return ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[i].ob_bh); > > +} > > + > > /* > > * ext4_orphan_add() links an unlinked or truncated inode into a list of > > * such inodes, starting at the superblock, in case we crash before the > > @@ -2555,10 +2595,10 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) > > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > > !mutex_is_locked(&inode->i_mutex)); > > /* > > - * Exit early if inode already is on orphan list. This is a big speedup > > - * since we don't have to contend on the global s_orphan_lock. > > + * Inode orphaned in orphan file or in orphan list? > > */ > > - if (!list_empty(&EXT4_I(inode)->i_orphan)) > > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) || > > + !list_empty(&EXT4_I(inode)->i_orphan)) > > return 0; > > > > > @@ -2570,6 +2610,16 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) > > J_ASSERT((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || > > S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); > > > > + if (sbi->s_orphan_info.of_blocks) { > > + err = ext4_orphan_file_add(handle, inode); > > + /* > > + * Fallback to normal orphan list of orphan file is > > + * out of space > > + */ > > + if (err != -ENOSPC) > > + return err; > > + } > > Hmm, that is sad, but I guess it is necessary to be able to unlink an > inode if the filesystem is completely full. Otherwise there won't be > any space to free space... > > That said, the ENOSPC orphan list could link onto the orphan inode > itself if it is directly referenced from s_last_orphan. Then, maybe > conveniently, if the orphan inode itself is deleted at the end of > orphan inode processing it would free all of the allocated blocks and > avoid the problem of growing to a large size and never freeing blocks. > > > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > > err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); > > if (err) > > @@ -2618,6 +2668,37 @@ out: > > return err; > > } > > > > +static int ext4_orphan_file_del(handle_t *handle, struct inode *inode) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > > + __le32 *bdata; > > + int blk, off; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > > + int ret = 0; > > + > > + if (!handle) > > + goto out; > > + blk = EXT4_I(inode)->i_orphan_idx / inodes_per_ob; > > + off = EXT4_I(inode)->i_orphan_idx % inodes_per_ob; > > + > > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > > + oi->of_binfo[blk].ob_bh, TR_ORPHAN_FILE); > > + if (ret) > > + goto out; > > + > > + bdata = (__le32 *)(oi->of_binfo[blk].ob_bh->b_data); > > + spin_lock(&oi->of_lock); > > + bdata[off] = 0; > > + oi->of_binfo[blk].ob_free_entries++; > > + spin_unlock(&oi->of_lock); > > + ret = ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[blk].ob_bh); > > +out: > > + ext4_clear_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + INIT_LIST_HEAD(&EXT4_I(inode)->i_orphan); > > + > > + return ret; > > +} > > + > > /* > > * ext4_orphan_del() removes an unlinked or truncated inode from the list > > * of such inodes stored on disk, because it is finally being cleaned up. > > @@ -2636,10 +2717,14 @@ int ext4_orphan_del(handle_t *handle, struct inode *inode) > > > > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > > !mutex_is_locked(&inode->i_mutex)); > > - /* Do this quick check before taking global s_orphan_lock. */ > > - if (list_empty(&ei->i_orphan)) > > + /* Do this quick check before taking global lock. */ > > + if (!ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) && > > + list_empty(&ei->i_orphan)) > > return 0; > > > > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE)) > > + return ext4_orphan_file_del(handle, inode); > > This could go before the above check, and then go back to only checking > list_empty() instead of checking STATE_ORPHAN_FILE twice. > > > if (handle) { > > /* Grab inode buffer early before taking global s_orphan_lock */ > > err = ext4_reserve_inode_write(handle, inode, &iloc); > > diff --git a/fs/ext4/super.c b/fs/ext4/super.c > > index 0babe8c435b6..14c30a9ef509 100644 > > --- a/fs/ext4/super.c > > +++ b/fs/ext4/super.c > > @@ -761,6 +761,18 @@ static void dump_orphan_list(struct super_block *sb, struct ext4_sb_info *sbi) > > } > > } > > > > +static void ext4_release_orphan_info(struct super_block *sb) > > +{ > > + int i; > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + if (!oi->of_blocks) > > + return; > > + for (i = 0; i < oi->of_blocks; i++) > > + brelse(oi->of_binfo[i].ob_bh); > > + kfree(oi->of_binfo); > > +} > > + > > static void ext4_put_super(struct super_block *sb) > > { > > struct ext4_sb_info *sbi = EXT4_SB(sb); > > @@ -772,6 +784,7 @@ static void ext4_put_super(struct super_block *sb) > > > > flush_workqueue(sbi->rsv_conversion_wq); > > destroy_workqueue(sbi->rsv_conversion_wq); > > + ext4_release_orphan_info(sb); > > > > if (sbi->s_journal) { > > err = jbd2_journal_destroy(sbi->s_journal); > > @@ -789,6 +802,8 @@ static void ext4_put_super(struct super_block *sb) > > > > if (!(sb->s_flags & MS_RDONLY)) { > > EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > es->s_state = cpu_to_le16(sbi->s_mount_state); > > } > > if (!(sb->s_flags & MS_RDONLY)) > > @@ -1905,8 +1920,14 @@ static int ext4_setup_super(struct super_block *sb, struct ext4_super_block *es, > > le16_add_cpu(&es->s_mnt_count, 1); > > es->s_mtime = cpu_to_le32(get_seconds()); > > ext4_update_dynamic_rev(sb); > > - if (sbi->s_journal) > > + if (sbi->s_journal) { > > EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + if (EXT4_HAS_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { > > + EXT4_SET_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > + } > > Shouldn't this only be set the first time an orphan is created? > > > @@ -2128,6 +2149,36 @@ static int ext4_check_descriptors(struct super_block *sb, > > return 1; > > } > > > > +static void ext4_process_orphan(struct inode *inode, > > + int *nr_truncates, int *nr_orphans) > > +{ > > + struct super_block *sb = inode->i_sb; > > + > > + dquot_initialize(inode); > > + if (inode->i_nlink) { > > + if (test_opt(sb, DEBUG)) > > + ext4_msg(sb, KERN_DEBUG, > > + "%s: truncating inode %lu to %lld bytes", > > + __func__, inode->i_ino, inode->i_size); > > + jbd_debug(2, "truncating inode %lu to %lld bytes\n", > > + inode->i_ino, inode->i_size); > > + mutex_lock(&inode->i_mutex); > > + truncate_inode_pages(inode->i_mapping, inode->i_size); > > + ext4_truncate(inode); > > + mutex_unlock(&inode->i_mutex); > > + (*nr_truncates)++; > > + } else { > > + if (test_opt(sb, DEBUG)) > > + ext4_msg(sb, KERN_DEBUG, > > + "%s: deleting unreferenced inode %lu", > > + __func__, inode->i_ino); > > + jbd_debug(2, "deleting unreferenced inode %lu\n", > > + inode->i_ino); > > + (*nr_orphans)++; > > + } > > + iput(inode); /* The delete magic happens here! */ > > +} > > + > > /* ext4_orphan_cleanup() walks a singly-linked list of inodes (starting at > > * the superblock) which were deleted from all directories, but held open by > > * a process at the time of a crash. We walk the list and try to delete these > > @@ -2150,10 +2201,13 @@ static void ext4_orphan_cleanup(struct super_block *sb, > > { > > unsigned int s_flags = sb->s_flags; > > int nr_orphans = 0, nr_truncates = 0; > > -#ifdef CONFIG_QUOTA > > - int i; > > -#endif > > - if (!es->s_last_orphan) { > > + int i, j; > > + __le32 *bdata; > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + struct inode *inode; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + if (!es->s_last_orphan && !oi->of_blocks) { > > jbd_debug(4, "no orphan inodes to clean up\n"); > > return; > > } > > @@ -2202,8 +2256,6 @@ static void ext4_orphan_cleanup(struct super_block *sb, > > #endif > > > > while (es->s_last_orphan) { > > - struct inode *inode; > > - > > inode = ext4_orphan_get(sb, le32_to_cpu(es->s_last_orphan)); > > if (IS_ERR(inode)) { > > es->s_last_orphan = 0; > > @@ -2211,29 +2263,21 @@ static void ext4_orphan_cleanup(struct super_block *sb, > > } > > > > list_add(&EXT4_I(inode)->i_orphan, &EXT4_SB(sb)->s_orphan); > > - dquot_initialize(inode); > > - if (inode->i_nlink) { > > - if (test_opt(sb, DEBUG)) > > - ext4_msg(sb, KERN_DEBUG, > > - "%s: truncating inode %lu to %lld bytes", > > - __func__, inode->i_ino, inode->i_size); > > - jbd_debug(2, "truncating inode %lu to %lld bytes\n", > > - inode->i_ino, inode->i_size); > > - mutex_lock(&inode->i_mutex); > > - truncate_inode_pages(inode->i_mapping, inode->i_size); > > - ext4_truncate(inode); > > - mutex_unlock(&inode->i_mutex); > > - nr_truncates++; > > - } else { > > - if (test_opt(sb, DEBUG)) > > - ext4_msg(sb, KERN_DEBUG, > > - "%s: deleting unreferenced inode %lu", > > - __func__, inode->i_ino); > > - jbd_debug(2, "deleting unreferenced inode %lu\n", > > - inode->i_ino); > > - nr_orphans++; > > + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); > > + } > > + > > + for (i = 0; i < oi->of_blocks; i++) { > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + for (j = 0; j < inodes_per_ob; j++) { > > + if (!bdata[j]) > > + continue; > > + inode = ext4_orphan_get(sb, le32_to_cpu(bdata[j])); > > + if (IS_ERR(inode)) > > + continue; > > + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; > > + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); > > } > > - iput(inode); /* The delete magic happens here! */ > > } > > > > #define PLURAL(x) (x), ((x) == 1) ? "" : "s" > > @@ -3420,6 +3464,97 @@ static void ext4_setup_csum_trigger(struct super_block *sb, > > sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; > > } > > > > +static int ext4_orphan_file_block_csum_verify(struct super_block *sb, > > + struct buffer_head *bh) > > +{ > > + __u32 provided, calculated; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + if (!ext4_has_metadata_csum(sb)) > > + return 1; > > Does it make sense to always checksum the orphan blocks, even if the > whole filesystem is not being checksummed? Corruption in the orphan > handling would result in orphan processing of random inode numbers. > Not fatal, I guess, since the only thing orphan processing does today > is truncate the file to the current i_size and unlink it only if > i_nlink == 0, which should be safe on normal files (at worst losing > fallocate'd blocks beyond i_size). Still, better safe than sorry? > That would also test the checksum callbacks on an ongoing basis, > instead of only when both metadata_csum and orphan_inode are enabled. > > > + provided = le32_to_cpu(((__le32 *)bh->b_data)[inodes_per_ob]); > > + calculated = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, > > + (__u8 *)bh->b_data, > > + inodes_per_ob * sizeof(__u32)); > > + return provided == calculated; > > +} > > + > > +/* This gets called only when checksumming is enabled */ > > +static void ext4_orphan_file_block_trigger( > > + struct jbd2_buffer_trigger_type *triggers, > > + struct buffer_head *bh, > > + void *data, size_t size) > > +{ > > + struct super_block *sb = EXT4_TRIGGER(triggers)->sb; > > + __u32 csum; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + csum = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, (__u8 *)data, > > + inodes_per_ob * sizeof(__u32)); > > + ((__le32 *)data)[inodes_per_ob] = cpu_to_le32(csum); > > +} > > + > > +static int ext4_init_orphan_info(struct super_block *sb) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + struct inode *inode; > > + int i, j; > > + int ret; > > + int free; > > + __le32 *bdata; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + spin_lock_init(&oi->of_lock); > > + > > + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) > > + return 0; > > + > > + inode = ext4_iget(sb, 12 /* FIXME: EXT4_ORPHAN_INO */); > > This would corrupt lost+found on most filesystems, better to use the > existing EXT4_ORPHAN_INO = 9 for testing, since it will at least not > exist on most filesystems. > > Since the orphan inode blocks will be allocated one-at-a-time after > long intervals, like an O_APPEND file, the orphan inode should not > be extent-mapped, but rather block-mapped for efficiency. Since > there is already a fallback to handle ENOSPC, the chance of not > being able to get a single free block in the first 2^32 blocks is > not worth the 3x overhead of extent block addresses for this inode. Don't forget that indirect blocks aren't checksummed. I suppose you could be clever and defer converting to an extent file until either (metadata_csum is on and you need more than 12 blocks) or (the only block we can get is above 16TB and we don't want to revert to the old orphan processing). --D > > > + if (IS_ERR(inode)) { > > + ext4_msg(sb, KERN_ERR, "get orphan inode failed"); > > + return PTR_ERR(inode); > > + } > > + oi->of_blocks = inode->i_size >> sb->s_blocksize_bits; > > + oi->of_csum_seed = EXT4_I(inode)->i_csum_seed; > > + oi->of_binfo = kmalloc(oi->of_blocks*sizeof(struct ext4_orphan_block), > > + GFP_KERNEL); > > + if (!oi->of_binfo) { > > + ret = -ENOMEM; > > + goto out_put; > > + } > > + for (i = 0; i < oi->of_blocks; i++) { > > + oi->of_binfo[i].ob_bh = ext4_bread(NULL, inode, i, 0); > > + if (IS_ERR(oi->of_binfo[i].ob_bh)) { > > + ret = PTR_ERR(oi->of_binfo[i].ob_bh); > > + goto out_free; > > + } > > + if (!ext4_orphan_file_block_csum_verify(sb, > > + oi->of_binfo[i].ob_bh)) { > > + ext4_error(sb, "orphan file block %d: bad checksum", i); > > + ret = -EIO; > > + goto out_free; > > Should this continue to the next block instead of aborting if a single > block is bad? If it is always checking the checksum it should be safe > to try the later blocks as well. > > > + } > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + free = 0; > > + for (j = 0; j < inodes_per_ob; j++) > > + if (bdata[j] == 0) > > + free++; > > + oi->of_binfo[i].ob_free_entries = free; > > + } > > + iput(inode); > > + return 0; > > +out_free: > > + for (i--; i >= 0; i--) > > + brelse(oi->of_binfo[i].ob_bh); > > + kfree(oi->of_binfo); > > +out_put: > > + iput(inode); > > + return ret; > > +} > > + > > static int ext4_fill_super(struct super_block *sb, void *data, int silent) > > { > > char *orig_data = kstrdup(data, GFP_KERNEL); > > @@ -3515,6 +3650,8 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) > > silent = 1; > > goto cantfind_ext4; > > } > > + ext4_setup_csum_trigger(sb, TR_ORPHAN_FILE, > > + ext4_orphan_file_block_trigger); > > > > /* Load the checksum driver */ > > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, > > @@ -3988,8 +4125,10 @@ static int ext4_fill_super(struct super_block *sb, void *data, int silent) > > sb->s_root = NULL; > > > > needs_recovery = (es->s_last_orphan != 0 || > > + EXT4_HAS_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) || > > EXT4_HAS_INCOMPAT_FEATURE(sb, > > - EXT4_FEATURE_INCOMPAT_RECOVER)); > > + EXT4_FEATURE_INCOMPAT_RECOVER)); > > > > if (EXT4_HAS_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_MMP) && > > !(sb->s_flags & MS_RDONLY)) > > @@ -4207,13 +4346,16 @@ no_journal: > > if (err) > > goto failed_mount7; > > > > + err = ext4_init_orphan_info(sb); > > + if (err) > > + goto failed_mount8; > > #ifdef CONFIG_QUOTA > > /* Enable quota usage during mount. */ > > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, EXT4_FEATURE_RO_COMPAT_QUOTA) && > > !(sb->s_flags & MS_RDONLY)) { > > err = ext4_enable_quotas(sb); > > if (err) > > - goto failed_mount8; > > + goto failed_mount9; > > } > > #endif /* CONFIG_QUOTA */ > > > > @@ -4263,9 +4405,11 @@ cantfind_ext4: > > goto failed_mount; > > > > #ifdef CONFIG_QUOTA > > +failed_mount9: > > + ext4_release_orphan_info(sb); > > +#endif > > failed_mount8: > > kobject_del(&sbi->s_kobj); > > -#endif > > failed_mount7: > > ext4_unregister_li_request(sb); > > failed_mount6: > > @@ -4771,6 +4915,20 @@ static int ext4_sync_fs(struct super_block *sb, int wait) > > return ret; > > } > > > > +static int ext4_orphan_file_empty(struct super_block *sb) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + int i; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) > > + return 1; > > + for (i = 0; i < oi->of_blocks; i++) > > + if (oi->of_binfo[i].ob_free_entries != inodes_per_ob) > > + return 0; > > + return 1; > > +} > > + > > /* > > * LVM calls this function before a (read-only) snapshot is created. This > > * gives us a chance to flush the journal completely and mark the fs clean. > > @@ -4804,6 +4962,10 @@ static int ext4_freeze(struct super_block *sb) > > > > /* Journal blocked and flushed, clear needs_recovery flag. */ > > EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + if (ext4_orphan_file_empty(sb)) { > > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > error = ext4_commit_super(sb, 1); > > out: > > if (journal) > > @@ -4823,6 +4985,10 @@ static int ext4_unfreeze(struct super_block *sb) > > > > /* Reset the needs_recovery flag before the fs is unlocked. */ > > EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + if (EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { > > + EXT4_SET_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > ext4_commit_super(sb, 1); > > return 0; > > } > > @@ -4966,8 +5132,13 @@ static int ext4_remount(struct super_block *sb, int *flags, char *data) > > (sbi->s_mount_state & EXT4_VALID_FS)) > > es->s_state = cpu_to_le16(sbi->s_mount_state); > > > > - if (sbi->s_journal) > > + if (sbi->s_journal) { > > ext4_mark_recovery_complete(sb, es); > > + if (ext4_orphan_file_empty(sb)) { > > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > + } > > } else { > > /* Make sure we can mount this feature set readwrite */ > > if (EXT4_HAS_RO_COMPAT_FEATURE(sb, > > -- > > 2.1.4 > > > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > Cheers, Andreas > > > > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-18 1:13 ` Darrick J. Wong @ 2015-04-20 12:34 ` Jan Kara 0 siblings, 0 replies; 18+ messages in thread From: Jan Kara @ 2015-04-20 12:34 UTC (permalink / raw) To: Darrick J. Wong; +Cc: Andreas Dilger, Jan Kara, linux-ext4 On Fri 17-04-15 18:13:00, Darrick J. Wong wrote: > On Fri, Apr 17, 2015 at 05:53:03PM -0600, Andreas Dilger wrote: > > On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: > > > > > > Ext4 orphan inode handling is a bottleneck for workloads which heavily > > > truncate / unlink small files since it contends on the global > > > s_orphan_mutex lock (and generally it's difficult to improve scalability > > > of the ondisk linked list of orphaned inodes). > > > > > > This patch implements new way of handling orphan inodes. Instead of > > > linking orphaned inode into a linked list, we store it's inode number in > > > a new special file which we call "orphan file". Currently we still > > > protect the orphan file with a spinlock for simplicity but even in this > > > setting we can substantially reduce the length of the critical section > > > and thus speedup some workloads. > > > > > > Note that the change is backwards compatible when the filesystem is > > > clean - the existence of the orphan file is a compat feature, we set > > > another ro-compat feature indicating orphan file needs scanning for > > > orphaned inodes when mounting filesystem read-write. This ro-compat > > > feature gets cleared on unmount / remount read-only. > > > > > > Some performance data from 48 CPU Xeon Server with 32 GB of RAM, > > > filesystem located on ramdisk, average of 5 runs: > > > > > > stress-orphan (microbenchmark truncating files byte-by-byte from N > > > processes in parallel) > > > > > > Threads Time Time > > > Vanilla Patched > > > 1 1.602800 1.260000 > > > 2 4.292200 2.455000 > > > 4 6.202800 3.848400 > > > 8 10.415000 6.833000 > > > 16 18.933600 12.883200 > > > 32 38.517200 25.342200 > > > 64 79.805000 50.918400 > > > 128 159.629200 102.666000 > > > > > > reaim new_fserver workload (tweaked to avoid calling sync(1) after every > > > operation) > > > > > > Threads Jobs/s Jobs/s > > > Vanilla Patched > > > 1 24375.00 22941.18 > > > 25 162162.16 278571.43 > > > 49 222209.30 331626.90 > > > 73 280147.60 419447.52 > > > 97 315250.00 481910.83 > > > 121 331157.90 503360.00 > > > 145 343769.00 489081.08 > > > 169 355549.56 519487.68 > > > 193 356518.65 501800.00 > > > > > > So in both cases we see significant wins all over the board. > > > > One thing I noticed looking at this patch is that there is quite a bit > > of orphan handling code now. Is it worthwhile to move it into its own > > file and make super.c a bit smaller? > > > > > Signed-off-by: Jan Kara <jack@suse.cz> > > > --- > > > fs/ext4/ext4.h | 52 +++++++++++-- > > > fs/ext4/namei.c | 95 +++++++++++++++++++++-- > > > fs/ext4/super.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++-------- > > > 3 files changed, 341 insertions(+), 43 deletions(-) > > > > > > diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > > > index abed83485915..768a8b9ee2f9 100644 > > > --- a/fs/ext4/ext4.h > > > +++ b/fs/ext4/ext4.h > > > @@ -208,6 +208,7 @@ struct ext4_io_submit { > > > #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ > > > #define EXT4_RESIZE_INO 7 /* Reserved group descriptors inode */ > > > #define EXT4_JOURNAL_INO 8 /* Journal inode */ > > > +#define EXT4_ORPHAN_INO 9 /* Inode with orphan entries */ > > > > Contrary to your patch description that said this was using ino #12, > > this conflicts with EXT2_EXCLUDE_INO for snapshots. Why not use > > s_last_orphan in the superblock to reference the orphan inode? Since > > this feature already requires a new EXT2_FEATURE_RO_COMPAT flag, the > > existing orphan inode number could be reused. See below how this could > > still in the ENOSPC case. > > > > > +static inline int ext4_inodes_per_orphan_block(struct super_block *sb) > > > +{ > > > + /* We reserve 1 entry for block checksum */ > > > > Would be good to improve this comment to say "first entry" or "last entry". > > > > > + return sb->s_blocksize / sizeof(u32) - 1; > > > +} > > Just to be clear, the format of each orphaned inode block is ... an array of > 32-bit inode numbers with a 32-bit checksum at the end? Yes. > Shouldn't we have a magic number somewhere for positive identification? We could use another slot at the end of block for magic number. The checksum actually uniquely identifies that the block belongs to the orphan file because it has the inode number in it but it's so much easier to look into the block and see the magic there during disaster recovery / bug analysis that I guess it's worth the extra space. Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-17 23:53 ` Andreas Dilger 2015-04-18 1:13 ` Darrick J. Wong @ 2015-04-20 12:25 ` Jan Kara 2015-04-20 16:35 ` Andreas Dilger 1 sibling, 1 reply; 18+ messages in thread From: Jan Kara @ 2015-04-20 12:25 UTC (permalink / raw) To: Andreas Dilger; +Cc: Jan Kara, linux-ext4 On Fri 17-04-15 17:53:03, Andreas Dilger wrote: > On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: > > > > Ext4 orphan inode handling is a bottleneck for workloads which heavily > > truncate / unlink small files since it contends on the global > > s_orphan_mutex lock (and generally it's difficult to improve scalability > > of the ondisk linked list of orphaned inodes). > > > > This patch implements new way of handling orphan inodes. Instead of > > linking orphaned inode into a linked list, we store it's inode number in > > a new special file which we call "orphan file". Currently we still > > protect the orphan file with a spinlock for simplicity but even in this > > setting we can substantially reduce the length of the critical section > > and thus speedup some workloads. > > > > Note that the change is backwards compatible when the filesystem is > > clean - the existence of the orphan file is a compat feature, we set > > another ro-compat feature indicating orphan file needs scanning for > > orphaned inodes when mounting filesystem read-write. This ro-compat > > feature gets cleared on unmount / remount read-only. > > > > Some performance data from 48 CPU Xeon Server with 32 GB of RAM, > > filesystem located on ramdisk, average of 5 runs: > > > > stress-orphan (microbenchmark truncating files byte-by-byte from N > > processes in parallel) > > > > Threads Time Time > > Vanilla Patched > > 1 1.602800 1.260000 > > 2 4.292200 2.455000 > > 4 6.202800 3.848400 > > 8 10.415000 6.833000 > > 16 18.933600 12.883200 > > 32 38.517200 25.342200 > > 64 79.805000 50.918400 > > 128 159.629200 102.666000 > > > > reaim new_fserver workload (tweaked to avoid calling sync(1) after every > > operation) > > > > Threads Jobs/s Jobs/s > > Vanilla Patched > > 1 24375.00 22941.18 > > 25 162162.16 278571.43 > > 49 222209.30 331626.90 > > 73 280147.60 419447.52 > > 97 315250.00 481910.83 > > 121 331157.90 503360.00 > > 145 343769.00 489081.08 > > 169 355549.56 519487.68 > > 193 356518.65 501800.00 > > > > So in both cases we see significant wins all over the board. > > One thing I noticed looking at this patch is that there is quite a bit > of orphan handling code now. Is it worthwhile to move it into its own > file and make super.c a bit smaller? Yeah, probably it's worth it. I can do it in the next version of the patch set. > > Signed-off-by: Jan Kara <jack@suse.cz> > > --- > > fs/ext4/ext4.h | 52 +++++++++++-- > > fs/ext4/namei.c | 95 +++++++++++++++++++++-- > > fs/ext4/super.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++-------- > > 3 files changed, 341 insertions(+), 43 deletions(-) > > > > diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > > index abed83485915..768a8b9ee2f9 100644 > > --- a/fs/ext4/ext4.h > > +++ b/fs/ext4/ext4.h > > @@ -208,6 +208,7 @@ struct ext4_io_submit { > > #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ > > #define EXT4_RESIZE_INO 7 /* Reserved group descriptors inode */ > > #define EXT4_JOURNAL_INO 8 /* Journal inode */ > > +#define EXT4_ORPHAN_INO 9 /* Inode with orphan entries */ > > Contrary to your patch description that said this was using ino #12, > this conflicts with EXT2_EXCLUDE_INO for snapshots. Why not use > s_last_orphan in the superblock to reference the orphan inode? Since > this feature already requires a new EXT2_FEATURE_RO_COMPAT flag, the > existing orphan inode number could be reused. See below how this could > still in the ENOSPC case. So I think you misunderstood one thing: Orphan file is statically allocated (when feature gets enabled by mkfs). Kernel doesn't handle resizing in any way - that way we can assume we modify exactly one block to add inode to orphan file. If we run out of space in orphan file - but note that e.g. if you have 128 KB orphan file, it can contain 32768 orphan entries and you never have that many orphan inodes at once under normal circumstances - we fall back to old style orphan list. So if you have more orphan inodes than you expected when creating orphan file, the fs still handles that gracefully, it just falls back to the old unscalable code. IMHO adding code to grow orphan file isn't simply worth it (but we can do it if we decide so in the future). Also we have to keep code to handle old style orphan list anyway in kernel so the fallback doesn't really incurr any significant additional complexity. > > +static inline int ext4_inodes_per_orphan_block(struct super_block *sb) > > +{ > > + /* We reserve 1 entry for block checksum */ > > Would be good to improve this comment to say "first entry" or "last entry". Good idea. > > > + return sb->s_blocksize / sizeof(u32) - 1; > > +} > > What do you think about making the on-disk orphan inode numbers store > 64-bit values? That would be easy to do now, and would avoid a format > change in the future if we wanted to use 64-bit inodes. > > That said, if the orphan inode is deleted after orphan recovery (see more > below) the only thing needed for compatibility is to store the inode number > size into the orphan inode somewhere so it could be changed. Maybe > i_version and/or i_generation since they are not directly user accessible. So orphan entry is cleared once inode isn't orphan anymore. So a clean filesystem currently has completely zeroed out orphan file. Switching to 64-bit inode numbers would be trivial then and you can just pick the format of the orphan file based on the 64BIT_INODE incompat feature we'll have to have in sb anyway. So I don't think we need to do anything in that regard now. > This would also allow you to detect if s_last_orphan was the orphan inode > without burning an EXT4_*_FL inode flag for a one-file-only usage. Umm, which flag do you mean? I have EXT4_STATE_ORPHAN_FILE which indicates that inode has entry in the orphan file - i.e. it is orphaned via the orphan file. But you seem to be talking about something different here. > > #define EXT4_FEATURE_COMPAT_RESIZE_INODE 0x0010 > > #define EXT4_FEATURE_COMPAT_DIR_INDEX 0x0020 > > #define EXT4_FEATURE_COMPAT_SPARSE_SUPER2 0x0200 > > +#define EXT4_FEATURE_COMPAT_ORPHAN_FILE 0x0400 /* Orphan file exists */ > > > > #define EXT4_FEATURE_RO_COMPAT_SPARSE_SUPER 0x0001 > > #define EXT4_FEATURE_RO_COMPAT_LARGE_FILE 0x0002 > > @@ -1556,7 +1593,10 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > > * GDT_CSUM bits are mutually exclusive. > > */ > > #define EXT4_FEATURE_RO_COMPAT_METADATA_CSUM 0x0400 > > +/* 0x0800 Reserved for EXT4_FEATURE_RO_COMPAT_REPLICA */ > > #define EXT4_FEATURE_RO_COMPAT_READONLY 0x1000 > > +#define EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT 0x2000 /* Orphan file may be > > + non-empty */ > > > > #define EXT4_FEATURE_INCOMPAT_COMPRESSION 0x0001 > > #define EXT4_FEATURE_INCOMPAT_FILETYPE 0x0002 > > @@ -1589,7 +1629,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > > EXT4_FEATURE_RO_COMPAT_LARGE_FILE| \ > > EXT4_FEATURE_RO_COMPAT_BTREE_DIR) > > > > -#define EXT4_FEATURE_COMPAT_SUPP EXT2_FEATURE_COMPAT_EXT_ATTR > > +#define EXT4_FEATURE_COMPAT_SUPP (EXT4_FEATURE_COMPAT_EXT_ATTR| \ > > + EXT4_FEATURE_COMPAT_ORPHAN_FILE) > > #define EXT4_FEATURE_INCOMPAT_SUPP (EXT4_FEATURE_INCOMPAT_FILETYPE| \ > > EXT4_FEATURE_INCOMPAT_RECOVER| \ > > EXT4_FEATURE_INCOMPAT_META_BG| \ > > @@ -1607,7 +1648,8 @@ static inline void ext4_clear_state_flags(struct ext4_inode_info *ei) > > EXT4_FEATURE_RO_COMPAT_HUGE_FILE |\ > > EXT4_FEATURE_RO_COMPAT_BIGALLOC |\ > > EXT4_FEATURE_RO_COMPAT_METADATA_CSUM|\ > > - EXT4_FEATURE_RO_COMPAT_QUOTA) > > + EXT4_FEATURE_RO_COMPAT_QUOTA|\ > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT) > > > > /* > > * Default values for user and/or group using reserved blocks > > diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c > > index 460c716e38b0..3436b7fa0ef9 100644 > > --- a/fs/ext4/namei.c > > +++ b/fs/ext4/namei.c > > @@ -2529,6 +2529,46 @@ static int empty_dir(struct inode *inode) > > return 1; > > } > > > > +static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) > > +{ > > + int i, j; > > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > > + int ret = 0; > > + __le32 *bdata; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > > + > > + spin_lock(&oi->of_lock); > > + for (i = 0; i < oi->of_blocks && !oi->of_binfo[i].ob_free_entries; i++); > > + if (i == oi->of_blocks) { > > + spin_unlock(&oi->of_lock); > > + return -ENOSPC; > > + } > > + oi->of_binfo[i].ob_free_entries--; > > + spin_unlock(&oi->of_lock); > > + > > + /* > > + * Get access to orphan block. We have dropped of_lock but since we > > + * have decremented number of free entries we are guaranteed free entry > > + * in our block. > > + */ > > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > > + oi->of_binfo[i].ob_bh, TR_ORPHAN_FILE); > > + if (ret) > > + return ret; > > + > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + spin_lock(&oi->of_lock); > > + /* Find empty slot in a block */ > > + for (j = 0; j < inodes_per_ob && bdata[j]; j++); > > + BUG_ON(j == inodes_per_ob); > > + bdata[j] = cpu_to_le32(inode->i_ino); > > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; > > + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + spin_unlock(&oi->of_lock); > > + > > + return ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[i].ob_bh); > > +} > > + > > /* > > * ext4_orphan_add() links an unlinked or truncated inode into a list of > > * such inodes, starting at the superblock, in case we crash before the > > @@ -2555,10 +2595,10 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) > > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > > !mutex_is_locked(&inode->i_mutex)); > > /* > > - * Exit early if inode already is on orphan list. This is a big speedup > > - * since we don't have to contend on the global s_orphan_lock. > > + * Inode orphaned in orphan file or in orphan list? > > */ > > - if (!list_empty(&EXT4_I(inode)->i_orphan)) > > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) || > > + !list_empty(&EXT4_I(inode)->i_orphan)) > > return 0; > > > > > @@ -2570,6 +2610,16 @@ int ext4_orphan_add(handle_t *handle, struct inode *inode) > > J_ASSERT((S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode) || > > S_ISLNK(inode->i_mode)) || inode->i_nlink == 0); > > > > + if (sbi->s_orphan_info.of_blocks) { > > + err = ext4_orphan_file_add(handle, inode); > > + /* > > + * Fallback to normal orphan list of orphan file is > > + * out of space > > + */ > > + if (err != -ENOSPC) > > + return err; > > + } > > Hmm, that is sad, but I guess it is necessary to be able to unlink an > inode if the filesystem is completely full. Otherwise there won't be > any space to free space... > > That said, the ENOSPC orphan list could link onto the orphan inode > itself if it is directly referenced from s_last_orphan. Then, maybe > conveniently, if the orphan inode itself is deleted at the end of > orphan inode processing it would free all of the allocated blocks and > avoid the problem of growing to a large size and never freeing blocks. See above. > > BUFFER_TRACE(sbi->s_sbh, "get_write_access"); > > err = ext4_journal_get_write_access(handle, sb, sbi->s_sbh, TR_NONE); > > if (err) > > @@ -2618,6 +2668,37 @@ out: > > return err; > > } > > > > +static int ext4_orphan_file_del(handle_t *handle, struct inode *inode) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; > > + __le32 *bdata; > > + int blk, off; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); > > + int ret = 0; > > + > > + if (!handle) > > + goto out; > > + blk = EXT4_I(inode)->i_orphan_idx / inodes_per_ob; > > + off = EXT4_I(inode)->i_orphan_idx % inodes_per_ob; > > + > > + ret = ext4_journal_get_write_access(handle, inode->i_sb, > > + oi->of_binfo[blk].ob_bh, TR_ORPHAN_FILE); > > + if (ret) > > + goto out; > > + > > + bdata = (__le32 *)(oi->of_binfo[blk].ob_bh->b_data); > > + spin_lock(&oi->of_lock); > > + bdata[off] = 0; > > + oi->of_binfo[blk].ob_free_entries++; > > + spin_unlock(&oi->of_lock); > > + ret = ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[blk].ob_bh); > > +out: > > + ext4_clear_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + INIT_LIST_HEAD(&EXT4_I(inode)->i_orphan); > > + > > + return ret; > > +} > > + > > /* > > * ext4_orphan_del() removes an unlinked or truncated inode from the list > > * of such inodes stored on disk, because it is finally being cleaned up. > > @@ -2636,10 +2717,14 @@ int ext4_orphan_del(handle_t *handle, struct inode *inode) > > > > WARN_ON_ONCE(!(inode->i_state & (I_NEW | I_FREEING)) && > > !mutex_is_locked(&inode->i_mutex)); > > - /* Do this quick check before taking global s_orphan_lock. */ > > - if (list_empty(&ei->i_orphan)) > > + /* Do this quick check before taking global lock. */ > > + if (!ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE) && > > + list_empty(&ei->i_orphan)) > > return 0; > > > > + if (ext4_test_inode_state(inode, EXT4_STATE_ORPHAN_FILE)) > > + return ext4_orphan_file_del(handle, inode); > > This could go before the above check, and then go back to only checking > list_empty() instead of checking STATE_ORPHAN_FILE twice. Nice, done. > > if (handle) { > > /* Grab inode buffer early before taking global s_orphan_lock */ > > err = ext4_reserve_inode_write(handle, inode, &iloc); > > diff --git a/fs/ext4/super.c b/fs/ext4/super.c > > index 0babe8c435b6..14c30a9ef509 100644 > > --- a/fs/ext4/super.c > > +++ b/fs/ext4/super.c > > @@ -761,6 +761,18 @@ static void dump_orphan_list(struct super_block *sb, struct ext4_sb_info *sbi) > > } > > } > > > > +static void ext4_release_orphan_info(struct super_block *sb) > > +{ > > + int i; > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + if (!oi->of_blocks) > > + return; > > + for (i = 0; i < oi->of_blocks; i++) > > + brelse(oi->of_binfo[i].ob_bh); > > + kfree(oi->of_binfo); > > +} > > + > > static void ext4_put_super(struct super_block *sb) > > { > > struct ext4_sb_info *sbi = EXT4_SB(sb); > > @@ -772,6 +784,7 @@ static void ext4_put_super(struct super_block *sb) > > > > flush_workqueue(sbi->rsv_conversion_wq); > > destroy_workqueue(sbi->rsv_conversion_wq); > > + ext4_release_orphan_info(sb); > > > > if (sbi->s_journal) { > > err = jbd2_journal_destroy(sbi->s_journal); > > @@ -789,6 +802,8 @@ static void ext4_put_super(struct super_block *sb) > > > > if (!(sb->s_flags & MS_RDONLY)) { > > EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + EXT4_CLEAR_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > es->s_state = cpu_to_le16(sbi->s_mount_state); > > } > > if (!(sb->s_flags & MS_RDONLY)) > > @@ -1905,8 +1920,14 @@ static int ext4_setup_super(struct super_block *sb, struct ext4_super_block *es, > > le16_add_cpu(&es->s_mnt_count, 1); > > es->s_mtime = cpu_to_le32(get_seconds()); > > ext4_update_dynamic_rev(sb); > > - if (sbi->s_journal) > > + if (sbi->s_journal) { > > EXT4_SET_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER); > > + if (EXT4_HAS_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_COMPAT_ORPHAN_FILE)) { > > + EXT4_SET_RO_COMPAT_FEATURE(sb, > > + EXT4_FEATURE_RO_COMPAT_ORPHAN_PRESENT); > > + } > > + } > > Shouldn't this only be set the first time an orphan is created? Could be done that way. But I didn't want to deal with the locking and checking down in the orphan inode handling functions. And once you mount fs read-write, you are very likely to add orphan entry anyway. Also the feature gets cleared once you unmount the fs so the feature remains set only if you crash while the fs is mounted - a corner case not worth the complexity IMHO. > > @@ -2128,6 +2149,36 @@ static int ext4_check_descriptors(struct super_block *sb, > > return 1; > > } > > > > +static void ext4_process_orphan(struct inode *inode, > > + int *nr_truncates, int *nr_orphans) > > +{ > > + struct super_block *sb = inode->i_sb; > > + > > + dquot_initialize(inode); > > + if (inode->i_nlink) { > > + if (test_opt(sb, DEBUG)) > > + ext4_msg(sb, KERN_DEBUG, > > + "%s: truncating inode %lu to %lld bytes", > > + __func__, inode->i_ino, inode->i_size); > > + jbd_debug(2, "truncating inode %lu to %lld bytes\n", > > + inode->i_ino, inode->i_size); > > + mutex_lock(&inode->i_mutex); > > + truncate_inode_pages(inode->i_mapping, inode->i_size); > > + ext4_truncate(inode); > > + mutex_unlock(&inode->i_mutex); > > + (*nr_truncates)++; > > + } else { > > + if (test_opt(sb, DEBUG)) > > + ext4_msg(sb, KERN_DEBUG, > > + "%s: deleting unreferenced inode %lu", > > + __func__, inode->i_ino); > > + jbd_debug(2, "deleting unreferenced inode %lu\n", > > + inode->i_ino); > > + (*nr_orphans)++; > > + } > > + iput(inode); /* The delete magic happens here! */ > > +} > > + > > /* ext4_orphan_cleanup() walks a singly-linked list of inodes (starting at > > * the superblock) which were deleted from all directories, but held open by > > * a process at the time of a crash. We walk the list and try to delete these > > @@ -2150,10 +2201,13 @@ static void ext4_orphan_cleanup(struct super_block *sb, > > { > > unsigned int s_flags = sb->s_flags; > > int nr_orphans = 0, nr_truncates = 0; > > -#ifdef CONFIG_QUOTA > > - int i; > > -#endif > > - if (!es->s_last_orphan) { > > + int i, j; > > + __le32 *bdata; > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + struct inode *inode; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + if (!es->s_last_orphan && !oi->of_blocks) { > > jbd_debug(4, "no orphan inodes to clean up\n"); > > return; > > } > > @@ -2202,8 +2256,6 @@ static void ext4_orphan_cleanup(struct super_block *sb, > > #endif > > > > while (es->s_last_orphan) { > > - struct inode *inode; > > - > > inode = ext4_orphan_get(sb, le32_to_cpu(es->s_last_orphan)); > > if (IS_ERR(inode)) { > > es->s_last_orphan = 0; > > @@ -2211,29 +2263,21 @@ static void ext4_orphan_cleanup(struct super_block *sb, > > } > > > > list_add(&EXT4_I(inode)->i_orphan, &EXT4_SB(sb)->s_orphan); > > - dquot_initialize(inode); > > - if (inode->i_nlink) { > > - if (test_opt(sb, DEBUG)) > > - ext4_msg(sb, KERN_DEBUG, > > - "%s: truncating inode %lu to %lld bytes", > > - __func__, inode->i_ino, inode->i_size); > > - jbd_debug(2, "truncating inode %lu to %lld bytes\n", > > - inode->i_ino, inode->i_size); > > - mutex_lock(&inode->i_mutex); > > - truncate_inode_pages(inode->i_mapping, inode->i_size); > > - ext4_truncate(inode); > > - mutex_unlock(&inode->i_mutex); > > - nr_truncates++; > > - } else { > > - if (test_opt(sb, DEBUG)) > > - ext4_msg(sb, KERN_DEBUG, > > - "%s: deleting unreferenced inode %lu", > > - __func__, inode->i_ino); > > - jbd_debug(2, "deleting unreferenced inode %lu\n", > > - inode->i_ino); > > - nr_orphans++; > > + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); > > + } > > + > > + for (i = 0; i < oi->of_blocks; i++) { > > + bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); > > + for (j = 0; j < inodes_per_ob; j++) { > > + if (!bdata[j]) > > + continue; > > + inode = ext4_orphan_get(sb, le32_to_cpu(bdata[j])); > > + if (IS_ERR(inode)) > > + continue; > > + ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); > > + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; > > + ext4_process_orphan(inode, &nr_truncates, &nr_orphans); > > } > > - iput(inode); /* The delete magic happens here! */ > > } > > > > #define PLURAL(x) (x), ((x) == 1) ? "" : "s" > > @@ -3420,6 +3464,97 @@ static void ext4_setup_csum_trigger(struct super_block *sb, > > sbi->s_journal_triggers[type].tr_triggers.t_frozen = trigger; > > } > > > > +static int ext4_orphan_file_block_csum_verify(struct super_block *sb, > > + struct buffer_head *bh) > > +{ > > + __u32 provided, calculated; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + if (!ext4_has_metadata_csum(sb)) > > + return 1; > > Does it make sense to always checksum the orphan blocks, even if the > whole filesystem is not being checksummed? Corruption in the orphan > handling would result in orphan processing of random inode numbers. > Not fatal, I guess, since the only thing orphan processing does today > is truncate the file to the current i_size and unlink it only if > i_nlink == 0, which should be safe on normal files (at worst losing > fallocate'd blocks beyond i_size). Still, better safe than sorry? > That would also test the checksum callbacks on an ongoing basis, > instead of only when both metadata_csum and orphan_inode are enabled. Umm, I like the fact that the code would be tested. But checksumming has its overhead (although we compute the checksum only at transaction commit but still it somewhat slows it down) so it seems wrong to slow everyone down just for that. Dunno, what others think? > > + provided = le32_to_cpu(((__le32 *)bh->b_data)[inodes_per_ob]); > > + calculated = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, > > + (__u8 *)bh->b_data, > > + inodes_per_ob * sizeof(__u32)); > > + return provided == calculated; > > +} > > + > > +/* This gets called only when checksumming is enabled */ > > +static void ext4_orphan_file_block_trigger( > > + struct jbd2_buffer_trigger_type *triggers, > > + struct buffer_head *bh, > > + void *data, size_t size) > > +{ > > + struct super_block *sb = EXT4_TRIGGER(triggers)->sb; > > + __u32 csum; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + > > + csum = ext4_chksum(EXT4_SB(sb), oi->of_csum_seed, (__u8 *)data, > > + inodes_per_ob * sizeof(__u32)); > > + ((__le32 *)data)[inodes_per_ob] = cpu_to_le32(csum); > > +} > > + > > +static int ext4_init_orphan_info(struct super_block *sb) > > +{ > > + struct ext4_orphan_info *oi = &EXT4_SB(sb)->s_orphan_info; > > + struct inode *inode; > > + int i, j; > > + int ret; > > + int free; > > + __le32 *bdata; > > + int inodes_per_ob = ext4_inodes_per_orphan_block(sb); > > + > > + spin_lock_init(&oi->of_lock); > > + > > + if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) > > + return 0; > > + > > + inode = ext4_iget(sb, 12 /* FIXME: EXT4_ORPHAN_INO */); > > This would corrupt lost+found on most filesystems, better to use the > existing EXT4_ORPHAN_INO = 9 for testing, since it will at least not > exist on most filesystems. Yes, I know - actually lost+found is inode 11, this is the first free inode on empty filesystem. That's why I wrote it's hacked up :) But because I need preallocated file to give to kernel, this was as easy as it could get. Allocating blocks for ino 9 isn't as easy as you have to play with debugfs. For now I don't expect anyone to use these patches on anything else than scratch filesystems... > Since the orphan inode blocks will be allocated one-at-a-time after > long intervals, like an O_APPEND file, the orphan inode should not > be extent-mapped, but rather block-mapped for efficiency. Since > there is already a fallback to handle ENOSPC, the chance of not > being able to get a single free block in the first 2^32 blocks is > not worth the 3x overhead of extent block addresses for this inode. Again here's the misunderstanding how the orphan file gets created. For preallocated file I have in mind, extent format is better since I expect to have just one extent in the file. > > + if (IS_ERR(inode)) { > > + ext4_msg(sb, KERN_ERR, "get orphan inode failed"); > > + return PTR_ERR(inode); > > + } > > + oi->of_blocks = inode->i_size >> sb->s_blocksize_bits; > > + oi->of_csum_seed = EXT4_I(inode)->i_csum_seed; > > + oi->of_binfo = kmalloc(oi->of_blocks*sizeof(struct ext4_orphan_block), > > + GFP_KERNEL); > > + if (!oi->of_binfo) { > > + ret = -ENOMEM; > > + goto out_put; > > + } > > + for (i = 0; i < oi->of_blocks; i++) { > > + oi->of_binfo[i].ob_bh = ext4_bread(NULL, inode, i, 0); > > + if (IS_ERR(oi->of_binfo[i].ob_bh)) { > > + ret = PTR_ERR(oi->of_binfo[i].ob_bh); > > + goto out_free; > > + } > > + if (!ext4_orphan_file_block_csum_verify(sb, > > + oi->of_binfo[i].ob_bh)) { > > + ext4_error(sb, "orphan file block %d: bad checksum", i); > > + ret = -EIO; > > + goto out_free; > > Should this continue to the next block instead of aborting if a single > block is bad? If it is always checking the checksum it should be safe > to try the later blocks as well. Well, if you have checksum errors, you better run fsck on the filesystem. It may be a bitflip, it may be a block written elsewhere... hard to tell. We could easily continue with later blocks but, then should we just wipe the problematic block? That seems dangerous and possibly leaks some inodes / space. And if we decide to not use that block (which effectively leaks inodes & blocks as well) we'd have to check for that and skip such blocks - doable but is it worth it? Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-20 12:25 ` Jan Kara @ 2015-04-20 16:35 ` Andreas Dilger 2015-04-21 10:56 ` Jan Kara 0 siblings, 1 reply; 18+ messages in thread From: Andreas Dilger @ 2015-04-20 16:35 UTC (permalink / raw) To: Jan Kara; +Cc: linux-ext4 On Apr 20, 2015, at 6:25 AM, Jan Kara <jack@suse.cz> wrote: > On Fri 17-04-15 17:53:03, Andreas Dilger wrote: >> On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: >>> Signed-off-by: Jan Kara <jack@suse.cz> >>> --- >>> fs/ext4/ext4.h | 52 +++++++++++-- >>> fs/ext4/namei.c | 95 +++++++++++++++++++++-- >>> fs/ext4/super.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++-------- >>> 3 files changed, 341 insertions(+), 43 deletions(-) >>> >>> diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h >>> index abed83485915..768a8b9ee2f9 100644 >>> --- a/fs/ext4/ext4.h >>> +++ b/fs/ext4/ext4.h >>> @@ -208,6 +208,7 @@ struct ext4_io_submit { >>> #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ >>> #define EXT4_RESIZE_INO 7 /* Reserved group descriptors inode */ >>> #define EXT4_JOURNAL_INO 8 /* Journal inode */ >>> +#define EXT4_ORPHAN_INO 9 /* Inode with orphan entries */ >> >> Contrary to your patch description that said this was using ino #12, >> this conflicts with EXT2_EXCLUDE_INO for snapshots. Why not use >> s_last_orphan in the superblock to reference the orphan inode? Since >> this feature already requires a new EXT2_FEATURE_RO_COMPAT flag, the >> existing orphan inode number could be reused. See below how this could >> still in the ENOSPC case. > > So I think you misunderstood one thing: Orphan file is statically > allocated (when feature gets enabled by mkfs). Kernel doesn't handle > resizing in any way - that way we can assume we modify exactly one block to > add inode to orphan file. You are right, I totally didn't notice this was the case. I thought the orphan inode size would grow as needed by the workload, and the ENOSPC handling would only be used if the filesystem was actually out of space. If we stick with the static orphan inode size, it makes sense to add a comment explaining this clearly. > If we run out of space in orphan file - but > note that e.g. if you have 128 KB orphan file, it can contain 32768 orphan > entries and you never have that many orphan inodes at once under normal > circumstances - we fall back to old style orphan list. So if you have more > orphan inodes than you expected when creating orphan file, the fs still > handles that gracefully, it just falls back to the old unscalable code. What I was thinking is that the orphan inode would always be linked from the superblock s_last_orphan if FEATURE_RO_COMPAT_ORPHAN_INODE is set: ext4_super_block[s_last_orphan] -> orphan_inode[i_dtime] [ ->orphan list ] Then, if no space is available to grow the orphan inode it would hook orphans off i_dtime from the orphan inode as needed as is done today. > IMHO adding code to grow orphan file isn't simply worth it (but we can do > it if we decide so in the future). Also we have to keep code to handle old > style orphan list anyway in kernel so the fallback doesn't really incurr > any significant additional complexity. How big is the orphan file by default? I'd think it will be a bit tricky to get this right, since it depends on both the size of the filesystem (e.g. you don't want 128KB orphan file on a 1.44MB floppy) and the number of cores. Given that there is already existing code to handle extending a file easily (e.g. ext4_append()) I don't think that is too hard. Then the orphan inode will only grow as large as needed. It could also detect SMP collisions by whether the buffer heads are locked already, and then allocate a new block to reduce contention. That way we get just the right SMP scalability for the system and workload being run. >> What do you think about making the on-disk orphan inode numbers store >> 64-bit values? That would be easy to do now, and would avoid a format >> change in the future if we wanted to use 64-bit inodes. >> >> That said, if the orphan inode is deleted after orphan recovery (see >> more below) the only thing needed for compatibility is to store the >> inode number size into the orphan inode somewhere so it could be changed. >> Maybe i_version and/or i_generation since they are not directly user >> accessible. > > So orphan entry is cleared once inode isn't orphan anymore. So a clean > filesystem currently has completely zeroed out orphan file. Switching to > 64-bit inode numbers would be trivial then and you can just pick the format > of the orphan file based on the 64BIT_INODE incompat feature we'll have to > have in sb anyway. So I don't think we need to do anything in that regard > now. But if someone wants to enable 64BIT_INODE then they need to set this flag on the superblock, and it would confuse the kernel to thinking that the orphan inode has 64-bit inode numbers, when it still only has 32-bit inodes. It seems safer to store the inode number size with the orphan inode. One option is to put it in the low byte of the proposed per-block magic, so if the inode number size changes the magic will change as well. >> This would also allow you to detect if s_last_orphan was the orphan inode >> without burning an EXT4_*_FL inode flag for a one-file-only usage. > Umm, which flag do you mean? I have EXT4_STATE_ORPHAN_FILE which > indicates that inode has entry in the orphan file - i.e. it is orphaned via > the orphan file. But you seem to be talking about something different here. I was wondering about how to verify that the orphan inode really is an orphan inode, to avoid treating some random inode's blocks as the orphan list, and then zeroing out that file accidentally. That gets to be more of a concern if s_last_orphan is pointing to the orphan inode, since this could also be confused with a regular inode on the orphan list. Having the per-block magic avoids much of this problem, and having the FEATURE_COMPAT_ORPHAN_FILE also ensures that the first orphan in the s_last_orphan list is the orphan inode and not just some random inode. Cheers, Andreas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-20 16:35 ` Andreas Dilger @ 2015-04-21 10:56 ` Jan Kara 2015-04-21 15:46 ` Andreas Dilger 0 siblings, 1 reply; 18+ messages in thread From: Jan Kara @ 2015-04-21 10:56 UTC (permalink / raw) To: Andreas Dilger; +Cc: Jan Kara, linux-ext4 On Mon 20-04-15 10:35:01, Andreas Dilger wrote: > On Apr 20, 2015, at 6:25 AM, Jan Kara <jack@suse.cz> wrote: > > On Fri 17-04-15 17:53:03, Andreas Dilger wrote: > >> On Apr 16, 2015, at 9:42 AM, Jan Kara <jack@suse.cz> wrote: > >>> Signed-off-by: Jan Kara <jack@suse.cz> > >>> --- > >>> fs/ext4/ext4.h | 52 +++++++++++-- > >>> fs/ext4/namei.c | 95 +++++++++++++++++++++-- > >>> fs/ext4/super.c | 237 ++++++++++++++++++++++++++++++++++++++++++++++++-------- > >>> 3 files changed, 341 insertions(+), 43 deletions(-) > >>> > >>> diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h > >>> index abed83485915..768a8b9ee2f9 100644 > >>> --- a/fs/ext4/ext4.h > >>> +++ b/fs/ext4/ext4.h > >>> @@ -208,6 +208,7 @@ struct ext4_io_submit { > >>> #define EXT4_UNDEL_DIR_INO 6 /* Undelete directory inode */ > >>> #define EXT4_RESIZE_INO 7 /* Reserved group descriptors inode */ > >>> #define EXT4_JOURNAL_INO 8 /* Journal inode */ > >>> +#define EXT4_ORPHAN_INO 9 /* Inode with orphan entries */ > >> > >> Contrary to your patch description that said this was using ino #12, > >> this conflicts with EXT2_EXCLUDE_INO for snapshots. Why not use > >> s_last_orphan in the superblock to reference the orphan inode? Since > >> this feature already requires a new EXT2_FEATURE_RO_COMPAT flag, the > >> existing orphan inode number could be reused. See below how this could > >> still in the ENOSPC case. > > > > So I think you misunderstood one thing: Orphan file is statically > > allocated (when feature gets enabled by mkfs). Kernel doesn't handle > > resizing in any way - that way we can assume we modify exactly one block to > > add inode to orphan file. > > You are right, I totally didn't notice this was the case. I thought the > orphan inode size would grow as needed by the workload, and the ENOSPC > handling would only be used if the filesystem was actually out of space. > If we stick with the static orphan inode size, it makes sense to add a > comment explaining this clearly. I'll add a comment at ENOSPC handling. > > If we run out of space in orphan file - but > > note that e.g. if you have 128 KB orphan file, it can contain 32768 orphan > > entries and you never have that many orphan inodes at once under normal > > circumstances - we fall back to old style orphan list. So if you have more > > orphan inodes than you expected when creating orphan file, the fs still > > handles that gracefully, it just falls back to the old unscalable code. > > What I was thinking is that the orphan inode would always be linked from > the superblock s_last_orphan if FEATURE_RO_COMPAT_ORPHAN_INODE is set: > > ext4_super_block[s_last_orphan] -> orphan_inode[i_dtime] [ ->orphan list ] > > Then, if no space is available to grow the orphan inode it would hook > orphans off i_dtime from the orphan inode as needed as is done today. But what is the advantage of doing this? It would possibly remove the need to increase the number of reserved inodes, that's about it. But it would add some special cases to orphan handling instead of just being able to use the old code as is. Furthermore the orphan file feature would no longer be a compat one AFAIU your proposal. So IMHO increasing the number of special inodes and using one of them is the easiest solution. > > IMHO adding code to grow orphan file isn't simply worth it (but we can do > > it if we decide so in the future). Also we have to keep code to handle old > > style orphan list anyway in kernel so the fallback doesn't really incurr > > any significant additional complexity. > > How big is the orphan file by default? I'd think it will be a bit tricky > to get this right, since it depends on both the size of the filesystem > (e.g. you don't want 128KB orphan file on a 1.44MB floppy) and the number > of cores. Given that there is already existing code to handle extending > a file easily (e.g. ext4_append()) I don't think that is too hard. Then > the orphan inode will only grow as large as needed. Well, I wouldn't be worried about 1.44MB floppies :) You likely won't format them with a journal (and thus orphan file) anyway. I'd base default orphan file size just on the fs size. For filesystems in the 'floppy' / 'small' profile I'd disable the feature by default. There we care more about size than about scalability. For filesystems 512 MB large I'd use 128 KB orphan file size and growing the orphan file size linearly with fs size upto 8 GB where orphan file size will be 2 MB - that's enough for 512 CPUs to operate in parallel which should be enough for a few coming years (growing orphan file size is easy if you later find out it isn't enough). > It could also detect SMP collisions by whether the buffer heads are locked > already, and then allocate a new block to reduce contention. That way we > get just the right SMP scalability for the system and workload being run. All this is possible but I would like to avoid overengineering it. So I would wait until I see users who actually need this. Kernel has orphan file under its control and if we later decide kernel should really handle growing / shrinking it, we can add that feature without any change to the on-disk format. > >> What do you think about making the on-disk orphan inode numbers store > >> 64-bit values? That would be easy to do now, and would avoid a format > >> change in the future if we wanted to use 64-bit inodes. > >> > >> That said, if the orphan inode is deleted after orphan recovery (see > >> more below) the only thing needed for compatibility is to store the > >> inode number size into the orphan inode somewhere so it could be changed. > >> Maybe i_version and/or i_generation since they are not directly user > >> accessible. > > > > So orphan entry is cleared once inode isn't orphan anymore. So a clean > > filesystem currently has completely zeroed out orphan file. Switching to > > 64-bit inode numbers would be trivial then and you can just pick the format > > of the orphan file based on the 64BIT_INODE incompat feature we'll have to > > have in sb anyway. So I don't think we need to do anything in that regard > > now. > > But if someone wants to enable 64BIT_INODE then they need to set this flag > on the superblock, and it would confuse the kernel to thinking that the > orphan inode has 64-bit inode numbers, when it still only has 32-bit inodes. So I'm bit confused. When you set 64BIT_INODE flag, you still need to walk over all the directory structure and convert all the directories. Also you presumably enforce the filesystem is clean. At that point the orphan file is full of zeros so when you mount the fs, kernel will just start looking at those zeros as 64-bit numbers which is fine. When we have inode number size also stored within the orphan file, we have to explicitely convert it. > It seems safer to store the inode number size with the orphan inode. One > option is to put it in the low byte of the proposed per-block magic, so if > the inode number size changes the magic will change as well. So I don't really mind having inode number as a part of magic but I'm just wondering about the advantage... Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-21 10:56 ` Jan Kara @ 2015-04-21 15:46 ` Andreas Dilger 0 siblings, 0 replies; 18+ messages in thread From: Andreas Dilger @ 2015-04-21 15:46 UTC (permalink / raw) To: Jan Kara; +Cc: linux-ext4 On Apr 21, 2015, at 4:56 AM, Jan Kara <jack@suse.cz> wrote: > On Mon 20-04-15 10:35:01, Andreas Dilger wrote: >> On Apr 20, 2015, at 6:25 AM, Jan Kara <jack@suse.cz> wrote: >>> On Fri 17-04-15 17:53:03, Andreas Dilger wrote: >>>> What do you think about making the on-disk orphan inode numbers store >>>> 64-bit values? That would be easy to do now, and would avoid a format >>>> change in the future if we wanted to use 64-bit inodes. >>>> >>>> That said, if the orphan inode is deleted after orphan recovery (see >>>> more below) the only thing needed for compatibility is to store the >>>> inode number size into the orphan inode somewhere so it could be >>>> changed. Maybe i_version and/or i_generation since they are not >>>> directly user accessible. >>> >>> So orphan entry is cleared once inode isn't orphan anymore. So a clean >>> filesystem currently has completely zeroed out orphan file. Switching to >>> 64-bit inode numbers would be trivial then and you can just pick the >>> format of the orphan file based on the 64BIT_INODE incompat feature >>> we'll have to have in sb anyway. So I don't think we need to do anything in that regard now. >> >> But if someone wants to enable 64BIT_INODE then they need to set this >> flag on the superblock, and it would confuse the kernel to thinking >> that the orphan inode has 64-bit inode numbers, when it still only has >> 32-bit inodes. > > So I'm bit confused. When you set 64BIT_INODE flag, you still need to > walk over all the directory structure and convert all the directories. > Also you presumably enforce the filesystem is clean. At that point the > orphan file is full of zeros so when you mount the fs, kernel will just > start looking at those zeros as 64-bit numbers which is fine. When we have > inode number size also stored within the orphan file, we have to > explicitly convert it. The dir_data feature allows storing extra data for each dirent separately. That would allow enabling 64-bit inodes individually as needed, without the need to convert the whole filesystem at once, or the need to store the 64-bit value for 32-bit inode numbers. >> It seems safer to store the inode number size with the orphan inode. >> One option is to put it in the low byte of the proposed per-block magic, >> so if the inode number size changes the magic will change as well. > > So I don't really mind having inode number as a part of magic but I'm > just wondering about the advantage... Whether the filesystem needs to be clean or not when 64BIT_INODE is turned on is a separate issue that could be decided when that feature is added. Making the last byte of the magic number "4" today is easily done and can be handled in ext4_inode_per_orphan_block() as easily as using "sizeof(u32)" (it would probably be better to change that function to take "struct inode" as the argument instead of "struct super_block"). This gives us flexibility in the future for little effort today. Cheers, Andreas ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-16 15:42 ` [PATCH 2/3] ext4: Speedup ext4 orphan inode handling Jan Kara [not found] ` <CAOQ4uxifVr1swHb5Y2M-TRuzwdDo-z92G6PuHvBGecGZ7nYuHg@mail.gmail.com> 2015-04-17 23:53 ` Andreas Dilger @ 2015-04-18 23:53 ` Theodore Ts'o 2015-04-20 9:32 ` Jan Kara 2 siblings, 1 reply; 18+ messages in thread From: Theodore Ts'o @ 2015-04-18 23:53 UTC (permalink / raw) To: Jan Kara; +Cc: linux-ext4 On Thu, Apr 16, 2015 at 05:42:56PM +0200, Jan Kara wrote: > Ext4 orphan inode handling is a bottleneck for workloads which heavily > truncate / unlink small files since it contends on the global > s_orphan_mutex lock (and generally it's difficult to improve scalability > of the ondisk linked list of orphaned inodes). > > This patch implements new way of handling orphan inodes. Instead of > linking orphaned inode into a linked list, we store it's inode number in > a new special file which we call "orphan file". Currently we still > protect the orphan file with a spinlock for simplicity but even in this > setting we can substantially reduce the length of the critical section > and thus speedup some workloads. Do we need to store the inode number of the orphan inodes in a file? We only need to deal with orphaned inode if the journal exists --- so why not just define a new journal block type, and simply dump all of the orphaned inodes into one or more journal blocks, which get written out as part of the commit process? We can track the orphaned inodes using an in-memory RCU linked list, so it can be completely lockless, and then in the transaction commit, we can simply traverse the linked list and write out all of orphaned inodes to the journal. I think this would be faster and simpler, and the only real issue is that we'll need to plumb this interface down into the jbd2 layer. But I don't think that would be too difficult. What do you think? - Ted ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/3] ext4: Speedup ext4 orphan inode handling 2015-04-18 23:53 ` Theodore Ts'o @ 2015-04-20 9:32 ` Jan Kara 0 siblings, 0 replies; 18+ messages in thread From: Jan Kara @ 2015-04-20 9:32 UTC (permalink / raw) To: Theodore Ts'o; +Cc: Jan Kara, linux-ext4 On Sat 18-04-15 19:53:41, Ted Tso wrote: > On Thu, Apr 16, 2015 at 05:42:56PM +0200, Jan Kara wrote: > > Ext4 orphan inode handling is a bottleneck for workloads which heavily > > truncate / unlink small files since it contends on the global > > s_orphan_mutex lock (and generally it's difficult to improve scalability > > of the ondisk linked list of orphaned inodes). > > > > This patch implements new way of handling orphan inodes. Instead of > > linking orphaned inode into a linked list, we store it's inode number in > > a new special file which we call "orphan file". Currently we still > > protect the orphan file with a spinlock for simplicity but even in this > > setting we can substantially reduce the length of the critical section > > and thus speedup some workloads. > > Do we need to store the inode number of the orphan inodes in a file? > We only need to deal with orphaned inode if the journal exists --- so > why not just define a new journal block type, and simply dump all of > the orphaned inodes into one or more journal blocks, which get written > out as part of the commit process? > > We can track the orphaned inodes using an in-memory RCU linked list, > so it can be completely lockless, and then in the transaction commit, > we can simply traverse the linked list and write out all of orphaned > inodes to the journal. I think this would be faster and simpler, and > the only real issue is that we'll need to plumb this interface down > into the jbd2 layer. But I don't think that would be too difficult. > > What do you think? Good question. That's actually what I tried in the initial version of the patch set. I didn't submit it in the end because it ended up being quite messy. 1) One problem is that inode can be cleaned up & freed in the running transaction before committing transaction finishes commit. So you either have to attach to a transaction special structure carrying just the inode number or you have to copy inode numbers from inodes early before the actual commit starts and before we allow a new transaction to start. Both is doable but neither is too elegant. 2) Another problem I've spotted is that e.g. after fs freeze you expect journal to be clean but you cannot really clean the last transaction while there are orphan inodes (you'd lose track of them). Similarly you have to be careful in the checkpointing code not to clean up the last transaction carrying orphan inodes. Basically to allow forward progress, you need to write orphan inode number into each transaction during which it is orphaned but still you cannot clean up the last committed transaction which breaks expectation in quite a few places in the fs. 3) Finally, journal replay gets somewhat tricky because you cannot cleanup the journal until you cleanup all orphan inodes (think of a crash during journal recovery) but you need to make fs up and running to do orphan cleanup. Again, this is solvable (you keep the last committed transaction in the journal, otherwise clean it up and set up all orphan inodes in memory so that they get written in the next committed transaction) but it complicates such core things in the fs that I didn't find it worth the trouble in the end. Honza -- Jan Kara <jack@suse.cz> SUSE Labs, CR ^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 3/3] ext4: Improve scalability of ext4 orphan file handling 2015-04-16 15:42 [PATCH 0/3 RFC] ext4: Speedup orphan file handling Jan Kara 2015-04-16 15:42 ` [PATCH 1/3] ext4: Support for checksumming from journal triggers Jan Kara 2015-04-16 15:42 ` [PATCH 2/3] ext4: Speedup ext4 orphan inode handling Jan Kara @ 2015-04-16 15:42 ` Jan Kara 2 siblings, 0 replies; 18+ messages in thread From: Jan Kara @ 2015-04-16 15:42 UTC (permalink / raw) To: linux-ext4; +Cc: Jan Kara Even though the length of the critical section when adding / removing orphaned inodes was significantly reduced by using orphan file, the contention of lock protecting orphan file still appears high in profiles for truncate / unlink intensive workloads with high number of threads. This patch makes handling of orphan file completely lockless. Also to reduce conflicts between CPUs different CPUs start searching for empty slot in orphan file in different blocks. Performance comparison of locked orphan file handling, lockless orphan file handling, and completely disabled orphan inode handling from 48 CPU Xeon Server with 32 GB of RAM, filesystem located on ramdisk, average of 5 runs: stress-orphan (microbenchmark truncating files byte-by-byte from N processes in parallel) Threads Time Time Time Orphan locked Orphan lockless No orphan 1 1.260000 1.379800 1.287000 2 2.455000 2.323800 2.314400 4 3.848400 3.704000 3.680400 8 6.833000 6.711800 6.844600 16 12.883200 12.931400 13.131600 32 25.342200 22.521600 22.570200 64 50.918400 36.380000 36.106200 128 102.666000 71.349800 71.716800 So we can see that with lockless orphan file handling, addition / deletion of orphaned inodes got completely out of picture even for a microbenchmark stressing it. reaim new_fserver workload didn't show any gains / losses outside of error margin. Signed-off-by: Jan Kara <jack@suse.cz> --- fs/ext4/ext4.h | 2 +- fs/ext4/namei.c | 40 ++++++++++++++++++++++++---------------- fs/ext4/super.c | 5 +++-- 3 files changed, 28 insertions(+), 19 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 768a8b9ee2f9..89de44af986c 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -1220,7 +1220,7 @@ static inline int ext4_inodes_per_orphan_block(struct super_block *sb) } struct ext4_orphan_block { - int ob_free_entries; /* Number of free orphan entries in block */ + atomic_t ob_free_entries; /* Number of free orphan entries in block */ struct buffer_head *ob_bh; /* Buffer for orphan block */ }; diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 3436b7fa0ef9..0c0649441a0f 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -2531,20 +2531,27 @@ static int empty_dir(struct inode *inode) static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) { - int i, j; + int i, j, start; struct ext4_orphan_info *oi = &EXT4_SB(inode->i_sb)->s_orphan_info; int ret = 0; __le32 *bdata; int inodes_per_ob = ext4_inodes_per_orphan_block(inode->i_sb); - spin_lock(&oi->of_lock); - for (i = 0; i < oi->of_blocks && !oi->of_binfo[i].ob_free_entries; i++); - if (i == oi->of_blocks) { - spin_unlock(&oi->of_lock); + /* + * Find block with free orphan entry. Use CPU number for a naive hash + * for a search start in the orphan file + */ + start = raw_smp_processor_id()*13 % oi->of_blocks; + i = start; + do { + if (atomic_dec_if_positive(&oi->of_binfo[i].ob_free_entries) + >= 0) + break; + if (++i >= oi->of_blocks) + i = 0; + } while (i != start); + if (i == start) return -ENOSPC; - } - oi->of_binfo[i].ob_free_entries--; - spin_unlock(&oi->of_lock); /* * Get access to orphan block. We have dropped of_lock but since we @@ -2557,14 +2564,17 @@ static int ext4_orphan_file_add(handle_t *handle, struct inode *inode) return ret; bdata = (__le32 *)(oi->of_binfo[i].ob_bh->b_data); - spin_lock(&oi->of_lock); /* Find empty slot in a block */ - for (j = 0; j < inodes_per_ob && bdata[j]; j++); - BUG_ON(j == inodes_per_ob); - bdata[j] = cpu_to_le32(inode->i_ino); + j = 0; + do { + while (bdata[j]) { + if (++j >= inodes_per_ob) + j = 0; + } + } while (cmpxchg(&bdata[j], 0, cpu_to_le32(inode->i_ino)) != 0); + EXT4_I(inode)->i_orphan_idx = i * inodes_per_ob + j; ext4_set_inode_state(inode, EXT4_STATE_ORPHAN_FILE); - spin_unlock(&oi->of_lock); return ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[i].ob_bh); } @@ -2687,10 +2697,8 @@ static int ext4_orphan_file_del(handle_t *handle, struct inode *inode) goto out; bdata = (__le32 *)(oi->of_binfo[blk].ob_bh->b_data); - spin_lock(&oi->of_lock); bdata[off] = 0; - oi->of_binfo[blk].ob_free_entries++; - spin_unlock(&oi->of_lock); + atomic_inc(&oi->of_binfo[blk].ob_free_entries); ret = ext4_handle_dirty_metadata(handle, NULL, oi->of_binfo[blk].ob_bh); out: ext4_clear_inode_state(inode, EXT4_STATE_ORPHAN_FILE); diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 14c30a9ef509..4d2fcc8483b2 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -3542,7 +3542,7 @@ static int ext4_init_orphan_info(struct super_block *sb) for (j = 0; j < inodes_per_ob; j++) if (bdata[j] == 0) free++; - oi->of_binfo[i].ob_free_entries = free; + atomic_set(&oi->of_binfo[i].ob_free_entries, free); } iput(inode); return 0; @@ -4924,7 +4924,8 @@ static int ext4_orphan_file_empty(struct super_block *sb) if (!EXT4_HAS_COMPAT_FEATURE(sb, EXT4_FEATURE_COMPAT_ORPHAN_FILE)) return 1; for (i = 0; i < oi->of_blocks; i++) - if (oi->of_binfo[i].ob_free_entries != inodes_per_ob) + if (atomic_read(&oi->of_binfo[i].ob_free_entries) != + inodes_per_ob) return 0; return 1; } -- 2.1.4 ^ permalink raw reply related [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-04-21 15:46 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-04-16 15:42 [PATCH 0/3 RFC] ext4: Speedup orphan file handling Jan Kara
2015-04-16 15:42 ` [PATCH 1/3] ext4: Support for checksumming from journal triggers Jan Kara
2015-04-17 19:00 ` Andreas Dilger
2015-04-20 9:07 ` Jan Kara
2015-04-16 15:42 ` [PATCH 2/3] ext4: Speedup ext4 orphan inode handling Jan Kara
[not found] ` <CAOQ4uxifVr1swHb5Y2M-TRuzwdDo-z92G6PuHvBGecGZ7nYuHg@mail.gmail.com>
2015-04-17 6:09 ` Amir Goldstein
2015-04-17 7:15 ` Jan Kara
2015-04-17 22:21 ` Andreas Dilger
2015-04-17 23:53 ` Andreas Dilger
2015-04-18 1:13 ` Darrick J. Wong
2015-04-20 12:34 ` Jan Kara
2015-04-20 12:25 ` Jan Kara
2015-04-20 16:35 ` Andreas Dilger
2015-04-21 10:56 ` Jan Kara
2015-04-21 15:46 ` Andreas Dilger
2015-04-18 23:53 ` Theodore Ts'o
2015-04-20 9:32 ` Jan Kara
2015-04-16 15:42 ` [PATCH 3/3] ext4: Improve scalability of ext4 orphan file handling Jan Kara
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).