Re: [PATCH 2/2] ext4 directory index: read-ahead blocks

[Coly Li <colyli@gmail.com>]

On 2011年06月18日 00:01, Bernd Schubert Wrote:
> While creating files in large directories we noticed an endless number
> of 4K reads. And those reads very much reduced file creation numbers
> as shown by bonnie. While we would expect about 2000 creates/s, we
> only got about 25 creates/s. Running the benchmarks for a long time
> improved the numbers, but not above 200 creates/s.
> It turned out those reads came from directory index block reads
> and probably the bh cache never cached all dx blocks. Given by
> the high number of directories we have (8192) and number of files required
> to trigger the issue (16 million), rather probably bh cached dx blocks
> got lost in favour of other less important blocks.
> The patch below implements a read-ahead for *all* dx blocks of a directory
> if a single dx block is missing in the cache. That also helps the LRU
> to cache important dx blocks.
> 
> Unfortunately, it also has a performance trade-off for the first access to
> a directory, although the READA flag is set already.
> Therefore at least for now, this option is disabled by default, but may
> be enabled using 'mount -o dx_read_ahead' or 'mount -odx_read_ahead=1'
> 
> Signed-off-by: Bernd Schubert <bernd.schubert@itwm.fraunhofer.de>
> ---

A question is, is there any performance number for dx dir read ahead ?
My concern is, if buffer cache replacement behavior is not ideal, which may replace a dx block by other (maybe) more hot
blocks, dx dir readahead will introduce more I/Os. In this case, we may focus on exploring why dx block is replaced out
of buffer cache, other than using dx readahead.


[snip]
> diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c
> index 6f32da4..78290f0 100644
> --- a/fs/ext4/namei.c
> +++ b/fs/ext4/namei.c
> @@ -334,6 +334,35 @@ struct stats dx_show_entries(struct dx_hash_info *hinfo, struct inode *dir,
>  #endif /* DX_DEBUG */
>  
>  /*
> + * Read ahead directory index blocks
> + */
> +static void dx_ra_blocks(struct inode *dir, struct dx_entry * entries)
> +{
> +	int i, err = 0;
> +	unsigned num_entries = dx_get_count(entries);
> +
> +	if (num_entries < 2 || num_entries > dx_get_limit(entries)) {
> +		dxtrace(printk("dx read-ahead: invalid number of entries\n"));
> +		return;
> +	}
> +
> +	dxtrace(printk("dx read-ahead: %d entries in dir-ino %lu \n",
> +			num_entries, dir->i_ino));
> +
> +	i = 1; /* skip first entry, it was already read in by the caller */
> +	do {
> +		struct dx_entry *entry;
> +		ext4_lblk_t block;
> +
> +		entry = entries + i;
> +
> +		block = dx_get_block(entry);
> +		err = ext4_bread_ra(dir, dx_get_block(entry));
> +		i++;
> +	 } while (i < num_entries && !err);
> +}
> +


I see sync reading here (CMIIW), this is performance killer. An async background reading ahead is better.

[snip]

Thanks.

Coly
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html