[benchmark] seek optimization

[benchmark] seek optimization

[Szakacsits Szabolcs]

Hello,

The test below wasn't intended to be a comprehensive evaluation of
filesystems' seek optimizations. The test method, as most others, might be
synthetic, nevertheless the tool used for the evaluation is also used to
solve real problems. But please don't make too general conclusions, if any
at all ;-)

Four set of tests were done: both cloning and imaging a large, fragmented
filesystem on both 2.4 and 2.6 kernels. The block allocation bitmap is
read and blocks in use are either copied to the exact places in a sparse
file (cloning) or the entire filesystem is streamed into an image where
the unused blocks are encoded (imaging). Considering strictly the output,
imaging requires no seek meanwhile the number of seeks could be optimized
to 0 during cloning.

Ntfsclone was chosen for the experiment, it can both clone and image into
a file. The source partitions was 8 GB and had 5 GB data scattered around.
The filesystems were always freshly recreated on the destination
partition. Both partitions were on the same disk. There were 3 runs for
each case, average taken. Deviation from average was always less then
0.5%. Sync()'s were always counted at the end of the runs.

In the below table "seek" refers to the cloning process, "noseek" to the
imaging. The "24" means 2.4.26 kernel ont the SystemRescueCD 0.2.14 and
"26" means 2.6.7 kernel on RIP 10.3. The "raw" means simple cloning
between two partitions (no filesystem on the destination partition).

The theoretical best result for 2.4 kernel is around 7:43 and for 2.6
kernel is around 8:05 minutes.

Results,

Fs_kernel_method      usr      sys       real   CPU
jfs_24_noseek        1.74    23.17    7:50.56    5%
reiser4_26_seek      1.81    36.66    8:06.12    7%
reiser4_26_noseek    2.02    46.75    8:08.72    9%
xfs_24_noseek        1.84    23.61    8:14.44    5%
ext2_24_noseek       1.58    20.92    8:15.28    4%
jfs_26_noseek        1.89    33.33    8:15.76    7%
ext2_24_seek         1.97    17.50    8:15.98    3%
raw_24_seek          1.65    17.88    8:17.69    3%
ext2_26_noseek       1.86    31.92    8:19.59    6%
xfs_26_noseek        1.92    33.70    8:20.48    7%
xfs_24_seek          1.72    20.77    8:25.17    4%
ext3_24_noseek       1.69    32.62    8:26.21    6%
raw_26_seek          1.80    33.49    8:27.02    6%
ext2_26_seek         1.80    29.77    8:27.29    6%
reiserfs_26_noseek   1.98    51.19    8:29.79   10%
reiserfs_26_seek     1.85    41.14    8:30.26    8%
ext3_24_seek         1.55    23.48    8:35.42    4%
xfs_26_seek          1.76    34.10    8:36.17    6%
ext3_26_noseek       1.92    39.83    8:37.38    8%
ext3_26_seek         1.84    35.55    8:43.41    7%
reiserfs_24_seek     1.71    28.20    8:50.02    5%
reiserfs_24_noseek   1.97    43.98    9:03.43    8%
jfs_24_seek          1.76    31.11   11:34.68    4%
jfs_26_seek          1.48    36.46   14:23.51    4%

Many interesting things can be noticed, I'd mention only two.

  1. For this type of workload, 2.6 kernels clearly perform worse 
     than 2.4 kernels.

  2. Reiser4's seek optimization is astonishing! It hits the problem
     in the 2.6 kernels, that's its bottleneck. Apparently it also 
     optimized away all (or most of) the seeks: the theoretically seek
     intensive run is faster than its seekless equivalent. How could 
     this be possible? If there were no (significant number of) seeks
     then the seekless version has a minor overhead. The Reiser4 result 
     just shows this! Amazing!!!

Cheers,
	Szaka

RE: [benchmark] seek optimization

[Guy]

You said "both partitions were on the same disk."  The poor disk must be
seeking like crazy!  Any seeks related to the filesystem may be masked,
since a seek was going to occur any how.

Consider having the 2 partitions on 2 different disks!

Guy

-----Original Message-----
From: linux-fsdevel-owner@vger.kernel.org
[mailto:linux-fsdevel-owner@vger.kernel.org] On Behalf Of Szakacsits
Szabolcs
Sent: Wednesday, July 14, 2004 9:52 AM
To: linux-fsdevel@vger.kernel.org
Cc: reiserfs-list@namesys.com; linux-ntfs-dev@lists.sourceforge.net; Marcel
Hilzinger; Per Olofsson
Subject: [benchmark] seek optimization


Hello,

The test below wasn't intended to be a comprehensive evaluation of
filesystems' seek optimizations. The test method, as most others, might be
synthetic, nevertheless the tool used for the evaluation is also used to
solve real problems. But please don't make too general conclusions, if any
at all ;-)

Four set of tests were done: both cloning and imaging a large, fragmented
filesystem on both 2.4 and 2.6 kernels. The block allocation bitmap is
read and blocks in use are either copied to the exact places in a sparse
file (cloning) or the entire filesystem is streamed into an image where
the unused blocks are encoded (imaging). Considering strictly the output,
imaging requires no seek meanwhile the number of seeks could be optimized
to 0 during cloning.

Ntfsclone was chosen for the experiment, it can both clone and image into
a file. The source partitions was 8 GB and had 5 GB data scattered around.
The filesystems were always freshly recreated on the destination
partition. Both partitions were on the same disk. There were 3 runs for
each case, average taken. Deviation from average was always less then
0.5%. Sync()'s were always counted at the end of the runs.

In the below table "seek" refers to the cloning process, "noseek" to the
imaging. The "24" means 2.4.26 kernel ont the SystemRescueCD 0.2.14 and
"26" means 2.6.7 kernel on RIP 10.3. The "raw" means simple cloning
between two partitions (no filesystem on the destination partition).

The theoretical best result for 2.4 kernel is around 7:43 and for 2.6
kernel is around 8:05 minutes.

Results,

Fs_kernel_method      usr      sys       real   CPU
jfs_24_noseek        1.74    23.17    7:50.56    5%
reiser4_26_seek      1.81    36.66    8:06.12    7%
reiser4_26_noseek    2.02    46.75    8:08.72    9%
xfs_24_noseek        1.84    23.61    8:14.44    5%
ext2_24_noseek       1.58    20.92    8:15.28    4%
jfs_26_noseek        1.89    33.33    8:15.76    7%
ext2_24_seek         1.97    17.50    8:15.98    3%
raw_24_seek          1.65    17.88    8:17.69    3%
ext2_26_noseek       1.86    31.92    8:19.59    6%
xfs_26_noseek        1.92    33.70    8:20.48    7%
xfs_24_seek          1.72    20.77    8:25.17    4%
ext3_24_noseek       1.69    32.62    8:26.21    6%
raw_26_seek          1.80    33.49    8:27.02    6%
ext2_26_seek         1.80    29.77    8:27.29    6%
reiserfs_26_noseek   1.98    51.19    8:29.79   10%
reiserfs_26_seek     1.85    41.14    8:30.26    8%
ext3_24_seek         1.55    23.48    8:35.42    4%
xfs_26_seek          1.76    34.10    8:36.17    6%
ext3_26_noseek       1.92    39.83    8:37.38    8%
ext3_26_seek         1.84    35.55    8:43.41    7%
reiserfs_24_seek     1.71    28.20    8:50.02    5%
reiserfs_24_noseek   1.97    43.98    9:03.43    8%
jfs_24_seek          1.76    31.11   11:34.68    4%
jfs_26_seek          1.48    36.46   14:23.51    4%

Many interesting things can be noticed, I'd mention only two.

  1. For this type of workload, 2.6 kernels clearly perform worse 
     than 2.4 kernels.

  2. Reiser4's seek optimization is astonishing! It hits the problem
     in the 2.6 kernels, that's its bottleneck. Apparently it also 
     optimized away all (or most of) the seeks: the theoretically seek
     intensive run is faster than its seekless equivalent. How could 
     this be possible? If there were no (significant number of) seeks
     then the seekless version has a minor overhead. The Reiser4 result 
     just shows this! Amazing!!!

Cheers,
	Szaka

RE: [benchmark] seek optimization

[Bryan Henderson]

>Consider having the 2 partitions on 2 different disks!

It would also be nice to have no destination disk at all, to isolate the 
effects of seek optimization in the source filesystem.

RE: [benchmark] seek optimization

[Szakacsits Szabolcs]

On Wed, 14 Jul 2004, Bryan Henderson wrote:

> It would also be nice to have no destination disk at all, to isolate the 
> effects of seek optimization in the source filesystem.

The mentioned theoretical best values had seeks neither in the source nor
in the destination. Sorry I forgot to explain.

	Szaka

RE: [benchmark] seek optimization

[Bryan Henderson]

>> It would also be nice to have no destination disk at all, to isolate 
the 
>> effects of seek optimization in the source filesystem.
>
>The mentioned theoretical best values had seeks neither in the source nor
>in the destination. Sorry I forgot to explain.

Not sure how that relates to my point.  I'm saying that in the actual 
experiment -- nothing theoretical -- if you're copying a file from source 
to destination and watching how long it takes, you're seeing both how long 
it takes to read and how long it takes to write (not to mention the ugly 
interaction between the two if they're behind the same head and being done 
by a single process).  A simpler experiment where you copy a file from a 
filesystem into oblivion would eliminate any seeks or other delays on the 
write side and any interaction between the two and leave just the seeks on 
the read side to measure.

And that information seems like it would be more useful.  Many 
applications simply read and use data; they don't copy it into another 
file -- especially one that uses the same resources as the source file.

RE: [benchmark] seek optimization

[Szakacsits Szabolcs]

On Wed, 14 Jul 2004, Bryan Henderson wrote:

> A simpler experiment where you copy a file from a filesystem into
> oblivion would eliminate any seeks or other delays on the write side
> and any interaction between the two and leave just the seeks on the
> read side to measure.

The intention was to point out seek elimination on the writer side ;-)
 
> And that information seems like it would be more useful.  Many
> applications simply read and use data; they don't copy it into another
> file

Sure, and many other different kind of tests would be more useful, too.  
But this experiment wasn't about those. Instead to point out, by a
relatively common tool in a common environment, that

   1) The seek elimination of Reiser4 is true and quite well measurable.
      Also its seek optimization is much better than any other major
      filesystem's optimization on Linux.

   2) Apparently the 2.6 kernel's IO scheduler has a performance problem.

	Szaka

Re: [benchmark] seek optimization

[Jens Axboe]

On Wed, Jul 14 2004, Szakacsits Szabolcs wrote:
> 
> On Wed, 14 Jul 2004, Bryan Henderson wrote:
> 
> > A simpler experiment where you copy a file from a filesystem into
> > oblivion would eliminate any seeks or other delays on the write side
> > and any interaction between the two and leave just the seeks on the
> > read side to measure.
> 
> The intention was to point out seek elimination on the writer side ;-)
>  
> > And that information seems like it would be more useful.  Many
> > applications simply read and use data; they don't copy it into another
> > file
> 
> Sure, and many other different kind of tests would be more useful, too.  
> But this experiment wasn't about those. Instead to point out, by a
> relatively common tool in a common environment, that
> 
>    1) The seek elimination of Reiser4 is true and quite well measurable.
>       Also its seek optimization is much better than any other major
>       filesystem's optimization on Linux.
> 
>    2) Apparently the 2.6 kernel's IO scheduler has a performance problem.

Which one did you test?

-- 
Jens Axboe

Re: [benchmark] seek optimization

[Szakacsits Szabolcs]

On Thu, 15 Jul 2004, Jens Axboe wrote:

> >    2) Apparently the 2.6 kernel's IO scheduler has a performance problem.
> 
> Which one did you test?

The default in 2.6.7. It seems that one is the AS. I couldn't find any way
to query what the IO scheduler is on a running 2.6.7 or how to change it.
There were no 'elevator' boot time option given. All IO scheduler were
compiled in.

The other major suspect for the 2-3% 2.6 performance degradation is the
'block size = GCD(page size, device size)' change. AFAIR the value ended
up to be the minimal 512 for the destination device. The 'sys' times are
double without using any filesystems:

  Fs_kernel_method      usr      sys       real   CPU
  raw_24_seek          1.65    17.88    8:19.69    3%
  raw_26_seek          1.80    33.49    8:27.02    6%

It might or might not be related.

	Szaka

Re: [benchmark] seek optimization

[Szakacsits Szabolcs]

On Thu, 15 Jul 2004, Szakacsits Szabolcs wrote:
> On Thu, 15 Jul 2004, Jens Axboe wrote:
> 
> > >    2) Apparently the 2.6 kernel's IO scheduler has a performance problem.
> >
> > Which one did you test?
> >
> The default in 2.6.7. It seems that one is the AS.

The "deadline" elevator gave 1-2% worse results for all fs and 5% worse
result for the raw access than the "as" scheduler.

	Szaka

Re: [benchmark] seek optimization

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1




Uh oh.  Which scheduler is best for reiser4?  Advantages/disadvantages
if said scheduler is used with some other fs (UDFfs on CDRWs, anyone?)



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQPbe13gHNmZLgCUhAQIdVw//UxiM8Y4XJC1U8h+2whMCYcDhpAsiRkqQ
0aAn8iMzv+QzYSo6uwRCvDy7iyY3PD+Z1t7/DDkFzNsaVoEWH1lg7vE7pYH9z7Vn
eS599CvECV10tuf+xg3j8+fJTiiK3YgcjnLqpcUDEa8YEQiXHz75IxhLRORntsyq
tcdDW7a8nsDKpdJI2RSrQyPTKjoMDtaREamrQhFC+qLpgorQ5c48uuX7lLMvS0l2
hc6ldI8dN9+5Sl4EtJUF304Z1Prtrqc98p1x80OhkJSqNW0JVG+8bMJ6dfiiwc3y
rDdyaSv3HFyCBinEX0GmKyFCXpvwYPbI3D8VtBW7HkSLG0+C4Mv0T2QCtAPE4Axm
oyj/N0qf/F3Ydb8z7IJRio80oOKfLU8pvOlLR7FkXfEbAWiFNVUXo2BQuy0O97Hh
YgKtXzdvezdVco13/M5JKsm2ghMGK2MrzVs17nV6+b8gBQ2B7o5uQ0ZgJ2MzAD3+
GHcSgGWOX1DEUe+oferVwaCALqsgEp1CFXyzkGzWdHsZ54psEDXKJ9iOarshYe9B
vrutQoIrRZ7i3zUhlDR0oeWIrNXHZ9M5HqiqL0fsZhO0WYJ6ukPp5Ru8n41pa4H9
44Ntr5bJni5Tt9BDWwmVOzGcZ+FtERL6QS6ZIZNrCMb9dhj+hnt0EJ+1/IZF+kbo
xA+dROthMaQ=
=H+gQ
-----END PGP SIGNATURE-----