From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============8477566626691219271==" MIME-Version: 1.0 From: Harris, James R Subject: Re: [SPDK] SPDK performance benchmarking guidelines and using FIO_plugin Date: Tue, 04 Apr 2017 15:49:03 +0000 Message-ID: In-Reply-To: 53575D55-B949-40A3-A8BA-E7C33C0878D7@cloudsimple.com List-ID: To: spdk@lists.01.org --===============8477566626691219271== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable > On Apr 4, 2017, at 4:25 AM, Kiran Dikshit wr= ote: > = > Hi Jim, = > = > Thank you for the pointers relating to workload. Is these tests ( through= put test , latency test ) are done on single drives or are they done on bun= ch of drives. = Hi Kiran, This will depend on what you are trying to measure. Some throughput tests = measure how many I/O can be performed using a single Intel Xeon CPU core, i= n which case we test with multiple SSDs to max out the CPU core. Other tes= ts such as QD=3D1 latency tests are usually done using a single drive. > Attached is FIO job file which i have used and getting the error. = Please add thread=3D1 to your job file. A patch will be pushed shortly tha= t documents this requirement for the SPDK plugin and enforces it at runtime= with a more suitable error message. -Jim > = > = > Thank you > Kiran > = >> On 31-Mar-2017, at 12:09 AM, Harris, James R wrote: >> = >>> = >>> On Mar 30, 2017, at 5:43 AM, Kiran Dikshit = wrote: >>> = >>> Hi All, >>> = >>> = >>> I have 2 part question relating to performance benchmarking of the SPDK= and fio_plugin. = >>> = >>> 1. Is there any reference document specifying the workload types , queu= e depth and block size used for benchmarking the SPDK performance numbers. = >>> Is there any OS level performance tuning to be done. It will be gre= at if we can get some insight on the performance testbed used. >>> = >>> Note : = >>> I did find the DPDK performance optimisation guidelines https://l= ists.01.org/pipermail/spdk/2016-June/000035.html. Which is useful. = >>> = >> = >> Hi Kiran, >> = >> Most of the tests we run for driver performance comparisons are with 4KB= random reads. This puts the most stress on the driver, since on NAND SSDs= , random read performance is typically much higher than random write perfor= mance. For throughput tests, queue depth is typically tested at 128. For = latency tests, queue depth is typically tested at 1. >> = >>> = >>> 2. I am trying fio_plugin for benchmarking the performance, the jobs a= re completing with following error = >>> = >>> =E2=80=9Cnvme_pcie.c: 996:nvme_pcie_qpair_complete_pending_admin_reques= t: ***ERROR*** the active process (pid 6700) is not found for this controll= er.=E2=80=9D >>> = >>> I found that the nvme_pcie_qpair_complete_pending_admin_request()is ch= ecking if the process exist, this is where the error message is coming from. >>> I am not sure how this process is getting killed even before completion= . There is no there operation done on this system apart from running fio pl= ugin. >>> Was any similar issue seen in the past, which might be of help to get a= round this error. >> = >> Could you post the contents of your fio job file? Note that the SPDK fi= o plugin is currently limited to a single job. I would expect to see an er= ror like this if specifying multiple jobs and without -thread (meaning a se= parate process per job). >> = >> Thanks, >> = >> -Jim >> = >>> = >>> Below is my setup details = >>> = >>> OS : fedora 25 ( server edition ) >>> Kernel version : 4.8.6-300 >>> DPDK version : 6.11 >>> = >>> I have attached a single nvme drive of 745GB from Intel on which i am r= unning the FIO = >>> = >>> Below workaround were tried and the issue still persists not sure how t= o get around this. = >>> = >>> workaround >>> = >>> 1. Tried different workloads in FIO = >>> 2. I did detach the NVMe and attach a new NVMe drive = >>> 3. Re-installed the DPDK , SPDK and FIO tool >>> = >>> Note : >>> Following links were used to install and setup SPDK and FIO plugin >>> https://github.com/spdk/spdk =E2=80=94> SPDK = >>> https://github.com/spdk/spdk/tree/master/examples/nvme/fio_plugin =E2= =80=94> FIO_plugin >>> = >>> = >>> Thank you >>> Kiran >>> _______________________________________________ >>> SPDK mailing list >>> SPDK(a)lists.01.org >>> https://lists.01.org/mailman/listinfo/spdk >> = >> _______________________________________________ >> SPDK mailing list >> SPDK(a)lists.01.org >> https://lists.01.org/mailman/listinfo/spdk > = > --===============8477566626691219271==--