public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Christoph Hellwig <hch@infradead.org>
To: Dave Chinner <david@fromorbit.com>
Cc: linux-xfs@vger.kernel.org
Subject: Re: [PATCH 5/5] xfs: reverse search directory freespace indexes
Date: Thu, 29 Aug 2019 22:24:54 -0700	[thread overview]
Message-ID: <20190830052454.GC6077@infradead.org> (raw)
In-Reply-To: <20190829104710.28239-6-david@fromorbit.com>

On Thu, Aug 29, 2019 at 08:47:10PM +1000, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> When a directory is growing rapidly, new blocks tend to get added at
> the end of the directory. These end up at the end of the freespace
> index, and when the directory gets large finding these new
> freespaces gets expensive. The code does a linear search across the
> frespace index from the first block in the directory to the last,
> hence meaning the newly added space is the last index searched.
> 
> Instead, do a reverse order index search, starting from the last
> block and index in the freespace index. This makes most lookups for
> free space on rapidly growing directories O(1) instead of O(N), but
> should not have any impact on random insert workloads because the
> average search length is the same regardless of which end of the
> array we start at.
> 
> The result is a major improvement in large directory grow rates:
> 
> 		create time(sec) / rate (files/s)
>  File count     vanilla             Prev commit		Patched
>   10k	      0.41 / 24.3k	   0.42 / 23.8k       0.41 / 24.3k
>   20k	      0.74 / 27.0k	   0.76 / 26.3k       0.75 / 26.7k
>  100k	      3.81 / 26.4k	   3.47 / 28.8k       3.27 / 30.6k
>  200k	      8.58 / 23.3k	   7.19 / 27.8k       6.71 / 29.8k
>    1M	     85.69 / 11.7k	  48.53 / 20.6k      37.67 / 26.5k
>    2M	    280.31 /  7.1k	 130.14 / 15.3k      79.55 / 25.2k
>   10M	   3913.26 /  2.5k                          552.89 / 18.1k
> 
> Signed-Off-By: Dave Chinner <dchinner@redhat.com>

Same here.

Otherwise looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

  parent reply	other threads:[~2019-08-30  5:24 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-29 10:47 [PATCH v3 0/5] xfs: speed up large directory modifications Dave Chinner
2019-08-29 10:47 ` [PATCH 1/5] xfs: move xfs_dir2_addname() Dave Chinner
2019-08-29 20:52   ` Darrick J. Wong
2019-08-30  5:22   ` Christoph Hellwig
2019-08-29 10:47 ` [PATCH 2/5] xfs: factor data block addition from xfs_dir2_node_addname_int() Dave Chinner
2019-08-29 21:02   ` Darrick J. Wong
2019-08-31  1:00     ` Dave Chinner
2019-08-29 10:47 ` [PATCH 3/5] xfs: factor free block index lookup " Dave Chinner
2019-08-29 21:07   ` Darrick J. Wong
2019-08-29 10:47 ` [PATCH 4/5] xfs: speed up directory bestfree block scanning Dave Chinner
2019-08-29 21:18   ` Darrick J. Wong
2019-08-30  5:24   ` Christoph Hellwig
2019-08-29 10:47 ` [PATCH 5/5] xfs: reverse search directory freespace indexes Dave Chinner
2019-08-29 21:20   ` Darrick J. Wong
2019-08-30  5:24   ` Christoph Hellwig [this message]
  -- strict thread matches above, loose matches on Subject: below --
2019-08-29  6:30 [PATCH V2 0/5] xfs: speed up large directory modifications Dave Chinner
2019-08-29  6:30 ` [PATCH 5/5] xfs: reverse search directory freespace indexes Dave Chinner
2019-08-29  8:23   ` Christoph Hellwig
2019-08-29  9:14     ` Dave Chinner
2018-10-24 22:57 [PATCH 0/5] xfs: speed up large directory modifications Dave Chinner
2018-10-24 22:57 ` [PATCH 5/5] xfs: reverse search directory freespace indexes Dave Chinner
2018-10-26 12:14   ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190830052454.GC6077@infradead.org \
    --to=hch@infradead.org \
    --cc=david@fromorbit.com \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox