Flexible I/O Tester development
 help / color / mirror / Atom feed
From: Tobias Oberstein <tobias.oberstein@gmail.com>
To: "Kudryavtsev, Andrey O" <andrey.o.kudryavtsev@intel.com>,
	"fio@vger.kernel.org" <fio@vger.kernel.org>
Subject: Re: 4x lower IOPS: Linux MD vs indiv. devices - why?
Date: Tue, 24 Jan 2017 16:19:21 +0100	[thread overview]
Message-ID: <48506f17-d810-117b-2d70-e4a788907a72@gmail.com> (raw)
In-Reply-To: <9D5B6947-2DF8-46B9-AEAD-62132C19A0D3@intel.com>

Hi Andrey,

> Changing sector to 4k is easy, this can really help. see DCT manual, it’s there.
> This can be relevant for you https://itpeernetwork.intel.com/how-to-configure-oracle-redo-on-the-intel-pcie-ssd-dc-p3700/

After overcoming my issues with isdct, and reformatting the NVMes to 4k 
sector size, success!

9.5 mio IOPS =)

This is another 34% faster than before.

So: thanks a bunch for your tip!

Cheers,
/Tobias


Next steps:

- approach MD developers about bottlenecks there
- approach PostgreSQL about using pread/pwrite (instead of lseek/read/write)


randread-individual-nvmes: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, 
ioengine=libaio, iodepth=128
...
fio-2.1.11
Starting 128 threads
Jobs: 128 (f=2048): [r(128)] [100.0% done] [37244MB/0KB/0KB /s] 
[9534K/0/0 iops] [eta 00m:00s]
randread-individual-nvmes: (groupid=0, jobs=128): err= 0: pid=25406: Tue 
Jan 24 15:57:19 2017
   read : io=1083.9GB, bw=36964MB/s, iops=9462.8K, runt= 30026msec
   cpu          : usr=9.00%, sys=77.01%, ctx=49252920, majf=0, minf=16512


      parent reply	other threads:[~2017-01-24 15:19 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-23 16:26 4x lower IOPS: Linux MD vs indiv. devices - why? Tobias Oberstein
     [not found] ` <CANvN+en2ihATNgrbgzwNXAK87wNh+6jXHinmg2-VmHon31AJzA@mail.gmail.com>
2017-01-23 17:52   ` Tobias Oberstein
     [not found]     ` <CANvN+em0cjWRnQWccdORKFEJk0OSeQOrZq+XE6kzPmqMPB--4g@mail.gmail.com>
2017-01-23 18:33       ` Tobias Oberstein
2017-01-23 19:10         ` Kudryavtsev, Andrey O
2017-01-23 19:26           ` Tobias Oberstein
2017-01-23 19:13         ` Sitsofe Wheeler
2017-01-23 19:40           ` Tobias Oberstein
2017-01-23 20:24             ` Sitsofe Wheeler
2017-01-23 21:22               ` Tobias Oberstein
     [not found]                 ` <CANvN+emLjb9idri9r42V3W9ia6v0EDGdJYFfhzq6rAuzGWec8Q@mail.gmail.com>
2017-01-23 21:42                   ` Andrey Kuzmin
2017-01-23 23:51                     ` Tobias Oberstein
2017-01-24  8:21                       ` Andrey Kuzmin
2017-01-24  9:28                         ` Tobias Oberstein
2017-01-24  9:40                           ` Andrey Kuzmin
2017-01-24 22:51                             ` Tobias Oberstein
2017-01-25 16:23                               ` Elliott, Robert (Persistent Memory)
2017-01-26 17:52                                 ` Tobias Oberstein
     [not found]         ` <CANvN+emM2xeKtEgVofOyKri6WBtjqc_o1LMT8Sfawb_RMRXT0g@mail.gmail.com>
2017-01-23 20:10           ` Tobias Oberstein
     [not found]             ` <CANvN+e=ityWtQj_TJ3yZgTM7mr17VB=3OeyQEEQvdb5tR5AGLA@mail.gmail.com>
     [not found]               ` <CANvN+emUGQ=voye=E6g4jFRxbp5eS8cGVJb3vTSn-bD5Db2Ycw@mail.gmail.com>
2017-01-23 20:20                 ` Tobias Oberstein
     [not found]             ` <CANvN+e=ASW14ShvY6dmVvUDY3PJVWwY9oQSbOT9EiOnQbSZHzA@mail.gmail.com>
     [not found]               ` <CANvN+ek0DgHF4gFAVep9ygdi=4pi9O9Fp5u3-VOd0iEVCSS0=Q@mail.gmail.com>
2017-01-23 21:49                 ` Tobias Oberstein
2017-01-23 18:18 ` Kudryavtsev, Andrey O
2017-01-23 18:53   ` Tobias Oberstein
2017-01-23 19:06     ` Kudryavtsev, Andrey O
2017-01-24  9:46       ` Tobias Oberstein
2017-01-24  9:55       ` Tobias Oberstein
2017-01-24 10:03       ` Tobias Oberstein
2017-01-24 15:19       ` Tobias Oberstein [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=48506f17-d810-117b-2d70-e4a788907a72@gmail.com \
    --to=tobias.oberstein@gmail.com \
    --cc=andrey.o.kudryavtsev@intel.com \
    --cc=fio@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox