public inbox for linux-nvme@lists.infradead.org
 help / color / mirror / Atom feed
From: Ping Gan <jacky_gam_2001@163.com>
To: chaitanyak@nvidia.com
Cc: ping_gan@dell.com, kbusch@kernel.org,
	linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
	hch@lst.de, sagi@grimberg.me, axboe@kernel.dk,
	jacky_gam_2001@163.com
Subject: 
Date: Fri, 15 Sep 2023 17:37:57 +0800	[thread overview]
Message-ID: <20230915093758.31397-1-jacky_gam_2001@163.com> (raw)
In-Reply-To: <28949e52-7db7-4227-6bbd-cb8b627b390f@nvidia.com>

> On 9/13/2023 1:34 AM, Ping Gan wrote:
> > Since nvme target currently does not support to submit bio to a
> > polling
> > queue, the bio's completion relies on system interrupt. But when there
> > is high workload in system and the competition is very high, so it
> > makes
> > sense to add polling queue task to submit bio to disk's polling queue
> > and poll the completion queue of disk.
> > 
> >
>
> I did some work in the past for nvmet polling and saw good
> performance improvement.
>
> Can you please share performance numbers for this series ?
> 
> -ck

hi,
I have verified this patch on two testbeds one for host and the other
for target. I used tcp as transport protocol, spdk perf as initiator. 
I do two group tests. The IO size of first is 4K, and the other is 2M. 
Both includ randrw, randwrite and randrw.  Both have same prerequisites.
At the initiator side I used 1 qp, 32 queue depth,and 1 spdk perf 
application, and for target side I bound tcp queue to 1 target core.
And I get below results.
iosize_4k        polling queue                        interrupt
randrw           NIC_rx:338M/s NIC_tx:335M/s      NIC_rx:260M/s
NIC_tx:258M/s
randwrite        NIC_rx:587M/s                    NIC_rx:431M/s
randread         NIC_tx:873M/s                    NIC_tx:654M/s

iosize_2M        polling queue                        interrupt
randrw           NIC_rx:738M/s NIC_tx:741M/s      NIC_rx:674M/s
NIC_tx:674M/s
randwrite        NIC_rx:1199M/s                   NIC_rx:1146M/s
randread         NIC_tx:2226M/s                   NIC_tx:2119M/s

For iosize 4k the NIC's bandwidth of poling queue is more than 30% than
bandwidth of interrupt. But for iosize 2M the improvement is not obvious,
the randrw of polling queue is about 9% more than interrupt; randwrite 
and randread of polling queue is about 5% more than interrupt.


Thanks,
Ping




  reply	other threads:[~2023-09-15  9:40 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-13  8:36 [PATCH 0/4] nvmet: support polling queue task for bio request Ping Gan
2023-09-13  8:34 ` Ping Gan
2023-09-13  8:36 ` [PATCH 1/4] nvmet: Add nvme target polling queue task parameters Ping Gan
2023-09-13  8:36 ` [PATCH 2/4] nvmet: Add polling queue task for nvme target Ping Gan
2023-09-13 12:19   ` kernel test robot
2023-09-13 13:13   ` kernel test robot
2023-09-13 16:03   ` kernel test robot
2023-09-13  8:36 ` [PATCH 3/4] nvmet: support bio polling queue request Ping Gan
2023-09-13  8:36 ` [PATCH 4/4] nvme-core: Get lowlevel disk for target polling queue task Ping Gan
2023-09-13 10:17   ` kernel test robot
2023-09-13 18:53 ` [PATCH 0/4] nvmet: support polling queue task for bio request Chaitanya Kulkarni
2023-09-15  9:37   ` Ping Gan [this message]
2023-09-19  3:25   ` [PATCH 0/4] nvmet: support polling queue task for bio Ping Gan
2023-09-15  9:37 ` [PATCH 0/4] nvmet: support polling queue task for bio request Ping Gan
2023-09-19  3:25 ` Ping Gan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230915093758.31397-1-jacky_gam_2001@163.com \
    --to=jacky_gam_2001@163.com \
    --cc=axboe@kernel.dk \
    --cc=chaitanyak@nvidia.com \
    --cc=hch@lst.de \
    --cc=kbusch@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=ping_gan@dell.com \
    --cc=sagi@grimberg.me \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox