* precondition ssd drives w/ fio
@ 2013-10-03 19:31 Brian L.
2013-10-03 19:52 ` Roger Sibert
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Brian L. @ 2013-10-03 19:31 UTC (permalink / raw)
To: fio@vger.kernel.org
For benchmarking SSD drives, I was told that we should precondition
our drives to get more accurate real world reading.
I was wondering if anyone is using fio itself to precondition ssd
drives or use a different script to populate random data on the
drives?
Thanks in advance.
Brian L.
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: precondition ssd drives w/ fio 2013-10-03 19:31 precondition ssd drives w/ fio Brian L. @ 2013-10-03 19:52 ` Roger Sibert 2013-10-03 20:06 ` David Nellans 2013-10-03 20:42 ` Juergen Salk 2 siblings, 0 replies; 10+ messages in thread From: Roger Sibert @ 2013-10-03 19:52 UTC (permalink / raw) To: Brian L.; +Cc: fio@vger.kernel.org I happen to be doing SSD benchmarks myself and if I read your questions correctly the following scenarios exist: 1. Performance of a fresh drive or a drive after a secure erase. 2. Performance after the disks memory has been completely written to. I loose track of the exact wording but I want to say at this point a disk has to do a read modify write vs just doing a write. I went with using fio to do full disk sequential writes and also setting it to a timed run to make sure it did multiple sets of writes. Thanks, Roger On Thu, Oct 3, 2013 at 3:31 PM, Brian L. <brianclam@gmail.com> wrote: > For benchmarking SSD drives, I was told that we should precondition > our drives to get more accurate real world reading. > > I was wondering if anyone is using fio itself to precondition ssd > drives or use a different script to populate random data on the > drives? > > Thanks in advance. > > Brian L. > -- > To unsubscribe from this list: send the line "unsubscribe fio" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: precondition ssd drives w/ fio 2013-10-03 19:31 precondition ssd drives w/ fio Brian L. 2013-10-03 19:52 ` Roger Sibert @ 2013-10-03 20:06 ` David Nellans 2013-10-03 20:42 ` Juergen Salk 2 siblings, 0 replies; 10+ messages in thread From: David Nellans @ 2013-10-03 20:06 UTC (permalink / raw) To: Brian L., fio@vger.kernel.org On 10/03/2013 02:31 PM, Brian L. wrote: > For benchmarking SSD drives, I was told that we should precondition > our drives to get more accurate real world reading. > > I was wondering if anyone is using fio itself to precondition ssd > drives or use a different script to populate random data on the > drives? > > Thanks in advance. > > Brian L. > -- > To unsubscribe from this list: send the line "unsubscribe fio" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > fio is flexible enough that it can precondition with just about any shape of traffic you're willing to craft a job for. but what traffic you believe is representative of your real world use case is entirely up to you and can have dramatic affect on the garbage collection efficiency of the drive. google "ssd preconditioning" and many of the flash manufacturers have whitepapers on how/why you might choose to precondition your drives before doing any benchmarking. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: precondition ssd drives w/ fio 2013-10-03 19:31 precondition ssd drives w/ fio Brian L. 2013-10-03 19:52 ` Roger Sibert 2013-10-03 20:06 ` David Nellans @ 2013-10-03 20:42 ` Juergen Salk 2013-10-03 21:09 ` Brian L. 2 siblings, 1 reply; 10+ messages in thread From: Juergen Salk @ 2013-10-03 20:42 UTC (permalink / raw) To: Brian L. * Brian L. <brianclam@gmail.com> [131003 12:31]: > For benchmarking SSD drives, I was told that we should precondition > our drives to get more accurate real world reading. > > I was wondering if anyone is using fio itself to precondition ssd > drives or use a different script to populate random data on the > drives? Yes I did. The fio source tarball comes with a number of sample job files including `ssd-steadystate.fio�, which might be useful for preconditioning of ssd devices. Regards, Juergen -- GPG A997BA7A | 87FC DA31 5F00 C885 0DC3 E28F BD0D 4B33 A997 BA7A ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: precondition ssd drives w/ fio 2013-10-03 20:42 ` Juergen Salk @ 2013-10-03 21:09 ` Brian L. 2013-10-03 21:39 ` Jeffrey Mcvay (jmcvay) 0 siblings, 1 reply; 10+ messages in thread From: Brian L. @ 2013-10-03 21:09 UTC (permalink / raw) To: Juergen Salk, fio@vger.kernel.org Thank you all for the response. I found ssd-steadystate.fio in the tarball. Brian Brian L. On Thu, Oct 3, 2013 at 1:42 PM, Juergen Salk <juergen.salk@gmx.de> wrote: > * Brian L. <brianclam@gmail.com> [131003 12:31]: > >> For benchmarking SSD drives, I was told that we should precondition >> our drives to get more accurate real world reading. >> >> I was wondering if anyone is using fio itself to precondition ssd >> drives or use a different script to populate random data on the >> drives? > > Yes I did. The fio source tarball comes with a number of sample > job files including `ssd-steadystate.fio�, which might be > useful for preconditioning of ssd devices. > > Regards, > > Juergen > > -- > GPG A997BA7A | 87FC DA31 5F00 C885 0DC3 E28F BD0D 4B33 A997 BA7A ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: precondition ssd drives w/ fio 2013-10-03 21:09 ` Brian L. @ 2013-10-03 21:39 ` Jeffrey Mcvay (jmcvay) 2013-10-04 5:44 ` Georg Schönberger 2013-10-04 18:22 ` Brian L. 0 siblings, 2 replies; 10+ messages in thread From: Jeffrey Mcvay (jmcvay) @ 2013-10-03 21:39 UTC (permalink / raw) To: Brian L., Juergen Salk, fio@vger.kernel.org > -----Original Message----- > From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On > Behalf Of Brian L. > Sent: Thursday, October 03, 2013 2:09 PM > To: Juergen Salk; fio@vger.kernel.org > Subject: Re: precondition ssd drives w/ fio > > Thank you all for the response. I found ssd-steadystate.fio in the > tarball. > > Brian > > Brian L. > > > On Thu, Oct 3, 2013 at 1:42 PM, Juergen Salk <juergen.salk@gmx.de> > wrote: > > * Brian L. <brianclam@gmail.com> [131003 12:31]: > > > >> For benchmarking SSD drives, I was told that we should precondition > >> our drives to get more accurate real world reading. > >> > >> I was wondering if anyone is using fio itself to precondition ssd > >> drives or use a different script to populate random data on the > >> drives? > > > > Yes I did. The fio source tarball comes with a number of sample > > job files including `ssd-steadystate.fio�, which might be > > useful for preconditioning of ssd devices. > > > > Regards, > > > > Juergen > > > > -- > > GPG A997BA7A | 87FC DA31 5F00 C885 0DC3 E28F BD0D 4B33 A997 BA7A > -- > To unsubscribe from this list: send the line "unsubscribe fio" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html Brian, Check the SNIA standard for how to test SSD performance. There are quite likely more elegant solutions, but I use the jobfile below. It is run sequential workloads of 100% write, 100% read, 100% write, and finally 100% read. The purpose of the reads is to force any existing bad or marginal blocks out of the user pool. This will precondition the drive. Please be aware that using numjobs > 1 is may not be a sequential workload. To measure steady state values takes a little more effort. I use the write_bw_log feature, run for several hours, and use statistical analysis to find the mean and standard deviation in the target time frame. Cheers, Jeff [global] thread group_reporting=1 direct=1 norandommap=1 randrepeat=0 refill_buffers ioengine=${IOENGINE} filename=${FILENAME} log_avg_msec=1000 [128kB_SeqWr_1x8_1stPass] write_bw_log=128kB_SeqWr_1x8_1stPass bs=128k rw=write numjobs=1 iodepth=8 [128kB_SeqRd_1x8_1ndPass] stonewall write_bw_log=128kB_SeqRd_1x8_1ndPass bs=128k rw=read numjobs=1 iodepth=8 [128kB_SeqWr_1x8_2ndPass] stonewall write_bw_log=128kB_SeqWr_1x8_2ndPass bs=128k rw=write numjobs=1 iodepth=8 [128kB_SeqRd_1x8_2ndPass] stonewall write_bw_log=128kB_SeqRd_1x8_2ndPass bs=128k rw=read numjobs=1 iodepth=8 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: precondition ssd drives w/ fio 2013-10-03 21:39 ` Jeffrey Mcvay (jmcvay) @ 2013-10-04 5:44 ` Georg Schönberger 2013-10-04 17:35 ` Brian L. 2013-10-04 18:22 ` Brian L. 1 sibling, 1 reply; 10+ messages in thread From: Georg Schönberger @ 2013-10-04 5:44 UTC (permalink / raw) To: Jeffrey Mcvay (jmcvay); +Cc: Brian L., Juergen Salk, fio ---- Original Message ----- > From: "Jeffrey Mcvay (jmcvay)" <jmcvay@micron.com> > To: "Brian L." <brianclam@gmail.com>, "Juergen Salk" <juergen.salk@gmx.de>, fio@vger.kernel.org > Sent: Thursday, 3 October, 2013 11:39:32 PM > Subject: RE: precondition ssd drives w/ fio > > > > -----Original Message----- > > From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] > > On > > Behalf Of Brian L. > > Sent: Thursday, October 03, 2013 2:09 PM > > To: Juergen Salk; fio@vger.kernel.org > > Subject: Re: precondition ssd drives w/ fio > > > > Thank you all for the response. I found ssd-steadystate.fio in the > > tarball. > > > > Brian > > > > Brian L. > > > > > > On Thu, Oct 3, 2013 at 1:42 PM, Juergen Salk <juergen.salk@gmx.de> > > wrote: > > > * Brian L. <brianclam@gmail.com> [131003 12:31]: > > > > > >> For benchmarking SSD drives, I was told that we should > > >> precondition > > >> our drives to get more accurate real world reading. > > >> > > >> I was wondering if anyone is using fio itself to precondition > > >> ssd > > >> drives or use a different script to populate random data on the > > >> drives? > > > > > > Yes I did. The fio source tarball comes with a number of sample > > > job files including `ssd-steadystate.fio´, which might be > > > useful for preconditioning of ssd devices. > > > > > > Regards, > > > > > > Juergen > > > > > > -- > > > GPG A997BA7A | 87FC DA31 5F00 C885 0DC3 E28F BD0D 4B33 A997 BA7A > > -- > > To unsubscribe from this list: send the line "unsubscribe fio" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Brian, > > Check the SNIA standard for how to test SSD performance. > > There are quite likely more elegant solutions, but I use the jobfile > below. It is run sequential workloads of 100% write, 100% read, > 100% write, and finally 100% read. The purpose of the reads is to > force any existing bad or marginal blocks out of the user pool. This > will precondition the drive. Please be aware that using numjobs > 1 > is may not be a sequential workload. > > To measure steady state values takes a little more effort. I use the > write_bw_log feature, run for several hours, and use statistical > analysis to find the mean and standard deviation in the target time > frame. > > Cheers, > Jeff > > [global] > thread > group_reporting=1 > direct=1 > norandommap=1 > randrepeat=0 > refill_buffers > ioengine=${IOENGINE} > filename=${FILENAME} > > log_avg_msec=1000 > > [128kB_SeqWr_1x8_1stPass] > write_bw_log=128kB_SeqWr_1x8_1stPass > bs=128k > rw=write > numjobs=1 > iodepth=8 > > [128kB_SeqRd_1x8_1ndPass] > stonewall > write_bw_log=128kB_SeqRd_1x8_1ndPass > bs=128k > rw=read > numjobs=1 > iodepth=8 > > > [128kB_SeqWr_1x8_2ndPass] > stonewall > write_bw_log=128kB_SeqWr_1x8_2ndPass > bs=128k > rw=write > numjobs=1 > iodepth=8 > > [128kB_SeqRd_1x8_2ndPass] > stonewall > write_bw_log=128kB_SeqRd_1x8_2ndPass > bs=128k > rw=read > numjobs=1 > iodepth=8 Hi Brian, I have written a Performance Test Tool using Fio in Python implementing the Solid State Storage Performance Test Specification Enterprise v1.0 (http://www.snia.org/sites/default/files/SSS_PTS_Enterprise_v1.0.pdf). The tool is Open Source, you can find it at http://git.thomas-krenn.com/TKperf_v1.git/ and some information in our wiki (http://www.thomas-krenn.com/de/wiki/TKperf - currently only in German). Regards, Georg -- : Georg Schönberger : Research & Development Executive : Thomas-Krenn.AG | The server-experts : http://www.thomas-krenn.com | http://www.thomas-krenn.com/wiki ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: precondition ssd drives w/ fio 2013-10-04 5:44 ` Georg Schönberger @ 2013-10-04 17:35 ` Brian L. 0 siblings, 0 replies; 10+ messages in thread From: Brian L. @ 2013-10-04 17:35 UTC (permalink / raw) To: Georg Schönberger Cc: Jeffrey Mcvay (jmcvay), Juergen Salk, fio@vger.kernel.org Thanks Georg. Just cloned your git repo. I might play around with it when I have some time. Brian L. On Thu, Oct 3, 2013 at 10:44 PM, Georg Sch�nberger <gschoenberger@thomas-krenn.com> wrote: > ---- Original Message ----- >> From: "Jeffrey Mcvay (jmcvay)" <jmcvay@micron.com> >> To: "Brian L." <brianclam@gmail.com>, "Juergen Salk" <juergen.salk@gmx.de>, fio@vger.kernel.org >> Sent: Thursday, 3 October, 2013 11:39:32 PM >> Subject: RE: precondition ssd drives w/ fio >> >> >> > -----Original Message----- >> > From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] >> > On >> > Behalf Of Brian L. >> > Sent: Thursday, October 03, 2013 2:09 PM >> > To: Juergen Salk; fio@vger.kernel.org >> > Subject: Re: precondition ssd drives w/ fio >> > >> > Thank you all for the response. I found ssd-steadystate.fio in the >> > tarball. >> > >> > Brian >> > >> > Brian L. >> > >> > >> > On Thu, Oct 3, 2013 at 1:42 PM, Juergen Salk <juergen.salk@gmx.de> >> > wrote: >> > > * Brian L. <brianclam@gmail.com> [131003 12:31]: >> > > >> > >> For benchmarking SSD drives, I was told that we should >> > >> precondition >> > >> our drives to get more accurate real world reading. >> > >> >> > >> I was wondering if anyone is using fio itself to precondition >> > >> ssd >> > >> drives or use a different script to populate random data on the >> > >> drives? >> > > >> > > Yes I did. The fio source tarball comes with a number of sample >> > > job files including `ssd-steadystate.fio�, which might be >> > > useful for preconditioning of ssd devices. >> > > >> > > Regards, >> > > >> > > Juergen >> > > >> > > -- >> > > GPG A997BA7A | 87FC DA31 5F00 C885 0DC3 E28F BD0D 4B33 A997 BA7A >> > -- >> > To unsubscribe from this list: send the line "unsubscribe fio" in >> > the body of a message to majordomo@vger.kernel.org >> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> Brian, >> >> Check the SNIA standard for how to test SSD performance. >> >> There are quite likely more elegant solutions, but I use the jobfile >> below. It is run sequential workloads of 100% write, 100% read, >> 100% write, and finally 100% read. The purpose of the reads is to >> force any existing bad or marginal blocks out of the user pool. This >> will precondition the drive. Please be aware that using numjobs > 1 >> is may not be a sequential workload. >> >> To measure steady state values takes a little more effort. I use the >> write_bw_log feature, run for several hours, and use statistical >> analysis to find the mean and standard deviation in the target time >> frame. >> >> Cheers, >> Jeff >> >> [global] >> thread >> group_reporting=1 >> direct=1 >> norandommap=1 >> randrepeat=0 >> refill_buffers >> ioengine=${IOENGINE} >> filename=${FILENAME} >> >> log_avg_msec=1000 >> >> [128kB_SeqWr_1x8_1stPass] >> write_bw_log=128kB_SeqWr_1x8_1stPass >> bs=128k >> rw=write >> numjobs=1 >> iodepth=8 >> >> [128kB_SeqRd_1x8_1ndPass] >> stonewall >> write_bw_log=128kB_SeqRd_1x8_1ndPass >> bs=128k >> rw=read >> numjobs=1 >> iodepth=8 >> >> >> [128kB_SeqWr_1x8_2ndPass] >> stonewall >> write_bw_log=128kB_SeqWr_1x8_2ndPass >> bs=128k >> rw=write >> numjobs=1 >> iodepth=8 >> >> [128kB_SeqRd_1x8_2ndPass] >> stonewall >> write_bw_log=128kB_SeqRd_1x8_2ndPass >> bs=128k >> rw=read >> numjobs=1 >> iodepth=8 > > Hi Brian, > I have written a Performance Test Tool using Fio in Python implementing the Solid State Storage Performance Test Specification Enterprise v1.0 (http://www.snia.org/sites/default/files/SSS_PTS_Enterprise_v1.0.pdf). > The tool is Open Source, you can find it at http://git.thomas-krenn.com/TKperf_v1.git/ and some information in our wiki (http://www.thomas-krenn.com/de/wiki/TKperf - currently only in German). > > Regards, Georg > -- > : Georg Sch�nberger > : Research & Development Executive > : Thomas-Krenn.AG | The server-experts > : http://www.thomas-krenn.com | http://www.thomas-krenn.com/wiki > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: precondition ssd drives w/ fio 2013-10-03 21:39 ` Jeffrey Mcvay (jmcvay) 2013-10-04 5:44 ` Georg Schönberger @ 2013-10-04 18:22 ` Brian L. 2013-10-04 20:16 ` Jeffrey Mcvay (jmcvay) 1 sibling, 1 reply; 10+ messages in thread From: Brian L. @ 2013-10-04 18:22 UTC (permalink / raw) To: Jeffrey Mcvay (jmcvay); +Cc: Juergen Salk, fio@vger.kernel.org On Thu, Oct 3, 2013 at 2:39 PM, Jeffrey Mcvay (jmcvay) <jmcvay@micron.com> wrote: > >> -----Original Message----- >> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] On >> Behalf Of Brian L. >> Sent: Thursday, October 03, 2013 2:09 PM >> To: Juergen Salk; fio@vger.kernel.org >> Subject: Re: precondition ssd drives w/ fio >> >> Thank you all for the response. I found ssd-steadystate.fio in the >> tarball. >> >> Brian >> >> Brian L. >> >> >> On Thu, Oct 3, 2013 at 1:42 PM, Juergen Salk <juergen.salk@gmx.de> >> wrote: >> > * Brian L. <brianclam@gmail.com> [131003 12:31]: >> > >> >> For benchmarking SSD drives, I was told that we should precondition >> >> our drives to get more accurate real world reading. >> >> >> >> I was wondering if anyone is using fio itself to precondition ssd >> >> drives or use a different script to populate random data on the >> >> drives? >> > >> > Yes I did. The fio source tarball comes with a number of sample >> > job files including `ssd-steadystate.fio�, which might be >> > useful for preconditioning of ssd devices. >> > >> > Regards, >> > >> > Juergen >> > >> > -- >> > GPG A997BA7A | 87FC DA31 5F00 C885 0DC3 E28F BD0D 4B33 A997 BA7A >> -- >> To unsubscribe from this list: send the line "unsubscribe fio" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > Brian, > > Check the SNIA standard for how to test SSD performance. > > There are quite likely more elegant solutions, but I use the jobfile below. It is run sequential workloads of 100% write, 100% read, 100% write, and finally 100% read. The purpose of the reads is to force any existing bad or marginal blocks out of the user pool. This will precondition the drive. Please be aware that using numjobs > 1 is may not be a sequential workload. > > To measure steady state values takes a little more effort. I use the write_bw_log feature, run for several hours, and use statistical analysis to find the mean and standard deviation in the target time frame. > > Cheers, > Jeff > > [global] > thread > group_reporting=1 > direct=1 > norandommap=1 > randrepeat=0 > refill_buffers > ioengine=${IOENGINE} > filename=${FILENAME} > > log_avg_msec=1000 > > [128kB_SeqWr_1x8_1stPass] > write_bw_log=128kB_SeqWr_1x8_1stPass > bs=128k > rw=write > numjobs=1 > iodepth=8 > > [128kB_SeqRd_1x8_1ndPass] > stonewall > write_bw_log=128kB_SeqRd_1x8_1ndPass > bs=128k > rw=read > numjobs=1 > iodepth=8 > > > [128kB_SeqWr_1x8_2ndPass] > stonewall > write_bw_log=128kB_SeqWr_1x8_2ndPass > bs=128k > rw=write > numjobs=1 > iodepth=8 > > [128kB_SeqRd_1x8_2ndPass] > stonewall > write_bw_log=128kB_SeqRd_1x8_2ndPass > bs=128k > rw=read > numjobs=1 > iodepth=8 thanks Jeffrey, I think I am going to use your FIO template since the example ssd-steadystate.fio (in the tarball) takes about 24 hr to run for each drive because section [random-write-steady] runs over the entire drive. The drives I am testing range from 200GB up to 800GB. That is way too long for me as I have 16 drives! I have been testing the drives at different rw profiles (also different blocksize and diff iodepth at times). I usually limit my run to size=10GB since I have so many drives that I need to run against different profiles, blocksize, etc... What is the size (in GB) / time limit that that you guys generally run against each drive for good results? === ssd-steadystate.fio template below === [global] ioengine=libaio direct=1 group_reporting=1 #filename=/dev/fioa filename=${mydisk} [sequential-fill] description=Sequential fill phase rw=write iodepth=16 bs=1M [random-write-steady] stonewall description=Random write steady state phase rw=randwrite bs=4K iodepth=32 numjobs=4 # # might need to fix var below # write_bw_log=${mydisk}-steady-state ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: precondition ssd drives w/ fio 2013-10-04 18:22 ` Brian L. @ 2013-10-04 20:16 ` Jeffrey Mcvay (jmcvay) 0 siblings, 0 replies; 10+ messages in thread From: Jeffrey Mcvay (jmcvay) @ 2013-10-04 20:16 UTC (permalink / raw) To: Brian L.; +Cc: Juergen Salk, fio@vger.kernel.org > -----Original Message----- > From: Brian L. [mailto:brianclam@gmail.com] > Sent: Friday, October 04, 2013 11:23 AM > To: Jeffrey Mcvay (jmcvay) > Cc: Juergen Salk; fio@vger.kernel.org > Subject: Re: precondition ssd drives w/ fio > > On Thu, Oct 3, 2013 at 2:39 PM, Jeffrey Mcvay (jmcvay) > <jmcvay@micron.com> wrote: > > > >> -----Original Message----- > >> From: fio-owner@vger.kernel.org [mailto:fio-owner@vger.kernel.org] > On > >> Behalf Of Brian L. > >> Sent: Thursday, October 03, 2013 2:09 PM > >> To: Juergen Salk; fio@vger.kernel.org > >> Subject: Re: precondition ssd drives w/ fio > >> > >> Thank you all for the response. I found ssd-steadystate.fio in the > >> tarball. > >> > >> Brian > >> > >> Brian L. > >> > >> > >> On Thu, Oct 3, 2013 at 1:42 PM, Juergen Salk <juergen.salk@gmx.de> > >> wrote: > >> > * Brian L. <brianclam@gmail.com> [131003 12:31]: > >> > > >> >> For benchmarking SSD drives, I was told that we should > precondition > >> >> our drives to get more accurate real world reading. > >> >> > >> >> I was wondering if anyone is using fio itself to precondition ssd > >> >> drives or use a different script to populate random data on the > >> >> drives? > >> > > >> > Yes I did. The fio source tarball comes with a number of sample > >> > job files including `ssd-steadystate.fio�, which might be > >> > useful for preconditioning of ssd devices. > >> > > >> > Regards, > >> > > >> > Juergen > >> > > >> > -- > >> > GPG A997BA7A | 87FC DA31 5F00 C885 0DC3 E28F BD0D 4B33 A997 BA7A > >> -- > >> To unsubscribe from this list: send the line "unsubscribe fio" in > >> the body of a message to majordomo@vger.kernel.org > >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > Brian, > > > > Check the SNIA standard for how to test SSD performance. > > > > There are quite likely more elegant solutions, but I use the jobfile > below. It is run sequential workloads of 100% write, 100% read, 100% > write, and finally 100% read. The purpose of the reads is to force any > existing bad or marginal blocks out of the user pool. This will > precondition the drive. Please be aware that using numjobs > 1 is may > not be a sequential workload. > > > > To measure steady state values takes a little more effort. I use the > write_bw_log feature, run for several hours, and use statistical > analysis to find the mean and standard deviation in the target time > frame. > > > > Cheers, > > Jeff > > > > [global] > > thread > > group_reporting=1 > > direct=1 > > norandommap=1 > > randrepeat=0 > > refill_buffers > > ioengine=${IOENGINE} > > filename=${FILENAME} > > > > log_avg_msec=1000 > > > > [128kB_SeqWr_1x8_1stPass] > > write_bw_log=128kB_SeqWr_1x8_1stPass > > bs=128k > > rw=write > > numjobs=1 > > iodepth=8 > > > > [128kB_SeqRd_1x8_1ndPass] > > stonewall > > write_bw_log=128kB_SeqRd_1x8_1ndPass > > bs=128k > > rw=read > > numjobs=1 > > iodepth=8 > > > > > > [128kB_SeqWr_1x8_2ndPass] > > stonewall > > write_bw_log=128kB_SeqWr_1x8_2ndPass > > bs=128k > > rw=write > > numjobs=1 > > iodepth=8 > > > > [128kB_SeqRd_1x8_2ndPass] > > stonewall > > write_bw_log=128kB_SeqRd_1x8_2ndPass > > bs=128k > > rw=read > > numjobs=1 > > iodepth=8 > > > thanks Jeffrey, > > I think I am going to use your FIO template since the example > ssd-steadystate.fio (in the tarball) takes about 24 hr to run for each > drive because section [random-write-steady] runs over the entire > drive. The drives I am testing range from 200GB up to 800GB. > > That is way too long for me as I have 16 drives! I have been testing > the drives at different rw profiles (also different blocksize and diff > iodepth at times). > > I usually limit my run to size=10GB since I have so many drives that I > need to run against different profiles, blocksize, etc... What is the > size (in GB) / time limit that that you guys generally run against > each drive for good results? > > > > === > ssd-steadystate.fio template below > === > > [global] > ioengine=libaio > direct=1 > group_reporting=1 > #filename=/dev/fioa > filename=${mydisk} > > [sequential-fill] > description=Sequential fill phase > rw=write > iodepth=16 > bs=1M > > [random-write-steady] > stonewall > description=Random write steady state phase > rw=randwrite > bs=4K > iodepth=32 > numjobs=4 > # > # might need to fix var below > # > write_bw_log=${mydisk}-steady-state To truly profile an SSD you have to use the entire drive. If you restrict the size written to a fraction of the drive an SSD will use the remainder as increased overprovision to reduce write amplification and performance. Also, most SSD's will not access NAND for reads to LBAs not already written. When you measure a read to an unwritten LBA you actual measure the speed at which the SSD can create 0's and ship them back. To get to your question on how long to run a random workload before measuring performance refer to the image at the link below. http://www.ssdperformanceblog.com/wp-content/uploads/2010/07/SSDstates.png Prior to section A the drive was preconditioned with 2 sequential fills. Section A: This the performance seen before background reclamation starts. The drive is writing to NAND blocks that are already erases. Section B: Reclamation starts. This is the lowest performance seen by an SSD. The drive is moving user data to free blocks for erase. This is also the time of maximum write amplification rate. Section C: The drive has reached steady state operation. Background reclamation is still running, but at a lower rate. This is due to more user data being invalidated and less user data movement required to free a block for erase. The rate of write amplification will also be consistent and overall write amplification will trend to this value over time. How long you need to run a random workload to reach steady state depends on the SSD. The best method graph the data and look for the steady state performance. A more statistical approach is to measure the mean performance over some unit of time and compare this to several such samples. When mean performance samples reach an acceptable minimum level of variance, your done. Cheers, Jeff ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-10-04 20:17 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-10-03 19:31 precondition ssd drives w/ fio Brian L. 2013-10-03 19:52 ` Roger Sibert 2013-10-03 20:06 ` David Nellans 2013-10-03 20:42 ` Juergen Salk 2013-10-03 21:09 ` Brian L. 2013-10-03 21:39 ` Jeffrey Mcvay (jmcvay) 2013-10-04 5:44 ` Georg Schönberger 2013-10-04 17:35 ` Brian L. 2013-10-04 18:22 ` Brian L. 2013-10-04 20:16 ` Jeffrey Mcvay (jmcvay)
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.