[Lustre-devel] ZFS/DMU benchmarks

[Lustre-devel] ZFS/DMU benchmarks

[Ricardo Correia]

Hi everyone,

Here are some current ZFS/DMU benchmark results that might be of 
interest to you.
These results were obtained on a Sunfire x4500 with these specifications:

- 2 dual-core AMD Opteron processors
- 16 GB main memory
- 48 SATA II HDD (7200 RPM, 500 GB each)
- Solaris 10 update 3
- Userspace DMU from OpenSolaris build 74

The tool used in this particular benchmark was PIOS, which simulates a 
parallel I/O load typically experienced by an OSS. Since PIOS links 
directly with the DMU, it should give a good estimation of the maximum 
throughput of the userspace DMU-OSS. For comparison, you can also see 
the results of running the same PIOS benchmark with normal POSIX I/O on 
the Solaris 10u3 ZFS implementation.

See the attached PDF for the results.

First page provides PIOS throughput information on striped pools with 
10, 24 and 46 disks. uDMU outperforms kernel ZFS with 10 disks, however, 
the kernel ZFS implementation scales better to 24 and 46 disks. 
Surprisingly enough, disabling checksums has a negative impact on the 
throughput with a 10-disk striped pool. This impact can be attributed to 
the differences in the uDMU IO pipeline which are probably causing the 
IOs to not being parallelized when checksumming is disabled. This issue, 
however, should not be hard to fix.

Second page shows the results of running PIOS on RAID-Z, RAID-Z2 and 
mirrored pools. In these configurations, the kernel ZFS currently has a 
better throughput than the userspace DMU.

These results were obtained with only minimal tuning of the DMU (setting 
a maximum cache size and increasing the number of I/O threads). Work is 
already underway to improve the performance of the DMU and we expect to 
see better throughput in the coming months.

Best regards,
Ricardo

-- 
<http://www.sun.com> 	* Ricardo Manuel Correia *

Lustre Engineering Group
*Sun Microsystems, Inc.*
Portugal

Ricardo.M.Correia at Sun.COM

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20071113/a7e9a9ee/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Lustre-uDMU.pdf
Type: application/pdf
Size: 66695 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20071113/a7e9a9ee/attachment.pdf>

[Lustre-devel] [Lustre-discuss] ZFS/DMU benchmarks

[Jody McIntyre]

Hi Ricardo,

On Tue, Nov 13, 2007 at 11:48:38AM +0000, Ricardo Correia wrote:

> Here are some current ZFS/DMU benchmark results that might be of 
> interest to you.

It's nice to see these results - thanks for posting them.  When you have
time, could you do a comparison with Lustre 1.6.x on ldiskfs + software
RAID 5?  It would be good to know how ZFS stands up (probably quite
favourably, even at this early stage.)

Cheers,
Jody

> These results were obtained on a Sunfire x4500 with these specifications:
> 
> - 2 dual-core AMD Opteron processors
> - 16 GB main memory
> - 48 SATA II HDD (7200 RPM, 500 GB each)
> - Solaris 10 update 3
> - Userspace DMU from OpenSolaris build 74
> 
> The tool used in this particular benchmark was PIOS, which simulates a 
> parallel I/O load typically experienced by an OSS. Since PIOS links 
> directly with the DMU, it should give a good estimation of the maximum 
> throughput of the userspace DMU-OSS. For comparison, you can also see 
> the results of running the same PIOS benchmark with normal POSIX I/O on 
> the Solaris 10u3 ZFS implementation.
> 
> See the attached PDF for the results.
> 
> First page provides PIOS throughput information on striped pools with 
> 10, 24 and 46 disks. uDMU outperforms kernel ZFS with 10 disks, however, 
> the kernel ZFS implementation scales better to 24 and 46 disks. 
> Surprisingly enough, disabling checksums has a negative impact on the 
> throughput with a 10-disk striped pool. This impact can be attributed to 
> the differences in the uDMU IO pipeline which are probably causing the 
> IOs to not being parallelized when checksumming is disabled. This issue, 
> however, should not be hard to fix.
> 
> Second page shows the results of running PIOS on RAID-Z, RAID-Z2 and 
> mirrored pools. In these configurations, the kernel ZFS currently has a 
> better throughput than the userspace DMU.
> 
> These results were obtained with only minimal tuning of the DMU (setting 
> a maximum cache size and increasing the number of I/O threads). Work is 
> already underway to improve the performance of the DMU and we expect to 
> see better throughput in the coming months.
> 
> Best regards,
> Ricardo
> 
> -- 
> <http://www.sun.com> 	* Ricardo Manuel Correia *
> 
> Lustre Engineering Group
> *Sun Microsystems, Inc.*
> Portugal
> 
> Ricardo.M.Correia at Sun.COM
> 


> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss


-- 

[Lustre-devel] [Lustre-discuss] ZFS/DMU benchmarks

[Jim Garlick]

It would also be nice to see the tests repeated on Linux on the same
hardware.  (Obviously not the kernel ZFS stuff though!)

Jim

p.s. I should add that the efforts at Livermore to test the ZFS/PIOS
stuff are on hold until we can get 1.6.2 stablized at scale for 
our production environment, likely another two months.  Sorry we were
not of more help.

On Wed, 14 Nov 2007, Jody McIntyre wrote:

> Hi Ricardo,
>
> On Tue, Nov 13, 2007 at 11:48:38AM +0000, Ricardo Correia wrote:
>
>> Here are some current ZFS/DMU benchmark results that might be of
>> interest to you.
>
> It's nice to see these results - thanks for posting them.  When you have
> time, could you do a comparison with Lustre 1.6.x on ldiskfs + software
> RAID 5?  It would be good to know how ZFS stands up (probably quite
> favourably, even at this early stage.)
>
> Cheers,
> Jody
>
>> These results were obtained on a Sunfire x4500 with these specifications:
>>
>> - 2 dual-core AMD Opteron processors
>> - 16 GB main memory
>> - 48 SATA II HDD (7200 RPM, 500 GB each)
>> - Solaris 10 update 3
>> - Userspace DMU from OpenSolaris build 74
>>
>> The tool used in this particular benchmark was PIOS, which simulates a
>> parallel I/O load typically experienced by an OSS. Since PIOS links
>> directly with the DMU, it should give a good estimation of the maximum
>> throughput of the userspace DMU-OSS. For comparison, you can also see
>> the results of running the same PIOS benchmark with normal POSIX I/O on
>> the Solaris 10u3 ZFS implementation.
>>
>> See the attached PDF for the results.
>>
>> First page provides PIOS throughput information on striped pools with
>> 10, 24 and 46 disks. uDMU outperforms kernel ZFS with 10 disks, however,
>> the kernel ZFS implementation scales better to 24 and 46 disks.
>> Surprisingly enough, disabling checksums has a negative impact on the
>> throughput with a 10-disk striped pool. This impact can be attributed to
>> the differences in the uDMU IO pipeline which are probably causing the
>> IOs to not being parallelized when checksumming is disabled. This issue,
>> however, should not be hard to fix.
>>
>> Second page shows the results of running PIOS on RAID-Z, RAID-Z2 and
>> mirrored pools. In these configurations, the kernel ZFS currently has a
>> better throughput than the userspace DMU.
>>
>> These results were obtained with only minimal tuning of the DMU (setting
>> a maximum cache size and increasing the number of I/O threads). Work is
>> already underway to improve the performance of the DMU and we expect to
>> see better throughput in the coming months.
>>
>> Best regards,
>> Ricardo
>>
>> --
>> <http://www.sun.com> 	* Ricardo Manuel Correia *
>>
>> Lustre Engineering Group
>> *Sun Microsystems, Inc.*
>> Portugal
>>
>> Ricardo.M.Correia at Sun.COM
>>
>
>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at clusterfs.com
>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>
>
> -- 
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-devel
>

[Lustre-devel] ZFS/DMU benchmarks

[Ricardo M. Correia]

Hi,

If you want to run the ZFS/DMU benchmarks yourself in Linux or Solaris, 
you will find instructions in the following wiki page: 
http://wiki.lustre.org/index.php?title=PIOS-DMU

There are also instructions for running the Lustre uOSS alpha/demo on 
both operating systems here: 
http://wiki.lustre.org/index.php?title=Lustre_OSS/MDS_with_ZFS_DMU

Let me know if you run into any problems.

Best regards,
Ricardo

-- 
<http://www.sun.com> 	

*Ricardo Manuel Correia*

Lustre Engineering
*Sun Microsystems, Inc.*
Portugal

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20071120/55348a34/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 6g_top.gif
Type: image/gif
Size: 1257 bytes
Desc: not available
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20071120/55348a34/attachment.gif>

[Lustre-devel] ZFS in User Space

[RS RS]

It appears,
from what I?ve read, that the future of Lustre will be based on ZFS running in
User space.  What are the advantages to
doing this in User space? 





I?ve also
read that this will be available in Lustre 1.8.  What is the timeframe for this? 

Thanks.



-Roger


_________________________________________________________________
Connect and share in new ways with Windows Live.
http://www.windowslive.com/connect.html?ocid=TXT_TAGLM_Wave2_newways_112007
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20071128/21f31cc5/attachment.htm>