From: Keith Busch <keith.busch@intel.com>
To: Jens Axboe <axboe@kernel.dk>
Cc: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>
Subject: Re: [PATCH 1/3] nvme/pci: Start request after doorbell ring
Date: Mon, 22 Jan 2018 17:16:29 -0700 [thread overview]
Message-ID: <20180123001628.GT12043@localhost.localdomain> (raw)
In-Reply-To: <20180103202105.GC11172@localhost.localdomain>
On Wed, Jan 03, 2018 at 01:21:05PM -0700, Keith Busch wrote:
>
> I've removed the submission side poll in a local build, and amazingly I
> am observing a not insignificant increase in latency without it on some
> workloads with certain hardware. I will have to delay recommending/posting
> removal of this pending further investigation.
This took longer to narrow than I hoped. I had been getting very wild
results, and there were two things to blame: a kernel irq issue[*],
and a platform bios causing a lot of CPU thermal frequency throttling.
Now that's sorted out, I am back to tuning this and the opprotunistic
nvme submission side polling once again does not produce a measurable
difference. My world is sane again. Will try to send an update for
consideration this week after some more testing.
* https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=a0c9259dc4
WARNING: multiple messages have this Message-ID (diff)
From: keith.busch@intel.com (Keith Busch)
Subject: [PATCH 1/3] nvme/pci: Start request after doorbell ring
Date: Mon, 22 Jan 2018 17:16:29 -0700 [thread overview]
Message-ID: <20180123001628.GT12043@localhost.localdomain> (raw)
In-Reply-To: <20180103202105.GC11172@localhost.localdomain>
On Wed, Jan 03, 2018@01:21:05PM -0700, Keith Busch wrote:
>
> I've removed the submission side poll in a local build, and amazingly I
> am observing a not insignificant increase in latency without it on some
> workloads with certain hardware. I will have to delay recommending/posting
> removal of this pending further investigation.
This took longer to narrow than I hoped. I had been getting very wild
results, and there were two things to blame: a kernel irq issue[*],
and a platform bios causing a lot of CPU thermal frequency throttling.
Now that's sorted out, I am back to tuning this and the opprotunistic
nvme submission side polling once again does not produce a measurable
difference. My world is sane again. Will try to send an update for
consideration this week after some more testing.
* https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=a0c9259dc4
next prev parent reply other threads:[~2018-01-23 0:16 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-21 20:46 [PATCH 0/3] Performance enhancements Keith Busch
2017-12-21 20:46 ` Keith Busch
2017-12-21 20:46 ` [PATCH 1/3] nvme/pci: Start request after doorbell ring Keith Busch
2017-12-21 20:46 ` Keith Busch
2017-12-21 20:49 ` Jens Axboe
2017-12-21 20:49 ` Jens Axboe
2017-12-21 20:53 ` Jens Axboe
2017-12-21 20:53 ` Jens Axboe
2017-12-21 21:02 ` Keith Busch
2017-12-21 21:02 ` Keith Busch
2017-12-21 21:01 ` Jens Axboe
2017-12-21 21:01 ` Jens Axboe
2018-01-03 20:21 ` Keith Busch
2018-01-03 20:21 ` Keith Busch
2018-01-23 0:16 ` Keith Busch [this message]
2018-01-23 0:16 ` Keith Busch
2017-12-25 10:12 ` Sagi Grimberg
2017-12-25 10:12 ` Sagi Grimberg
2017-12-29 9:44 ` Christoph Hellwig
2017-12-29 9:44 ` Christoph Hellwig
2017-12-25 10:11 ` Sagi Grimberg
2017-12-25 10:11 ` Sagi Grimberg
2017-12-26 20:35 ` Keith Busch
2017-12-26 20:35 ` Keith Busch
2017-12-27 9:02 ` Sagi Grimberg
2017-12-27 9:02 ` Sagi Grimberg
2017-12-29 9:44 ` Christoph Hellwig
2017-12-29 9:44 ` Christoph Hellwig
2017-12-21 20:46 ` [PATCH 2/3] nvme/pci: Remove cq_vector check in IO path Keith Busch
2017-12-21 20:46 ` Keith Busch
2017-12-21 20:54 ` Jens Axboe
2017-12-21 20:54 ` Jens Axboe
2017-12-25 10:10 ` Sagi Grimberg
2017-12-25 10:10 ` Sagi Grimberg
2017-12-27 21:01 ` Sagi Grimberg
2017-12-27 21:01 ` Sagi Grimberg
2017-12-29 9:48 ` Christoph Hellwig
2017-12-29 9:48 ` Christoph Hellwig
2017-12-29 15:39 ` Keith Busch
2017-12-29 15:39 ` Keith Busch
2017-12-31 12:30 ` Sagi Grimberg
2017-12-31 12:30 ` Sagi Grimberg
2018-01-02 16:50 ` Keith Busch
2018-01-02 16:50 ` Keith Busch
2017-12-21 20:46 ` [PATCH 3/3] block: Polling completion performance optimization Keith Busch
2017-12-21 20:46 ` Keith Busch
2017-12-21 20:56 ` Scott Bauer
2017-12-21 20:56 ` Scott Bauer
2017-12-21 21:00 ` Jens Axboe
2017-12-21 21:00 ` Jens Axboe
2017-12-21 21:34 ` Keith Busch
2017-12-21 21:34 ` Keith Busch
2017-12-21 22:17 ` Jens Axboe
2017-12-21 22:17 ` Jens Axboe
2017-12-21 23:10 ` Keith Busch
2017-12-21 23:10 ` Keith Busch
2017-12-22 15:40 ` Jens Axboe
2017-12-22 15:40 ` Jens Axboe
2017-12-29 9:50 ` Christoph Hellwig
2017-12-29 9:50 ` Christoph Hellwig
2017-12-29 15:51 ` Keith Busch
2017-12-29 15:51 ` Keith Busch
2017-12-31 12:48 ` Sagi Grimberg
2017-12-31 12:48 ` Sagi Grimberg
2017-12-21 20:57 ` Jens Axboe
2017-12-21 20:57 ` Jens Axboe
2017-12-29 9:51 ` Christoph Hellwig
2017-12-29 9:51 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180123001628.GT12043@localhost.localdomain \
--to=keith.busch@intel.com \
--cc=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.