From: Jens Axboe <axboe@kernel.dk>
To: Ketor D <d.ketor@gmail.com>
Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>,
Mark Nelson <mark.nelson@inktank.com>,
Mark Nelson <mark.a.nelson@gmail.com>,
"fio@vger.kernel.org" <fio@vger.kernel.org>,
"xan.peng" <xanpeng@gmail.com>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: fio rbd completions (Was: fio rbd hang for block sizes > 1M)
Date: Wed, 29 Oct 2014 20:55:31 -0600 [thread overview]
Message-ID: <5451A8A3.8020106@kernel.dk> (raw)
In-Reply-To: <CAM9_UU-BWHSk713O9v9RuZd1PRfTfG-Vq9nox0YQO3MhV06YEA@mail.gmail.com>
On 2014-10-29 20:50, Ketor D wrote:
> Hi Jens,
>
> The current code runs good!
>
> Test Conf: jobs=1 iodepth=1 bs=4k
>
> if busy_poll = 1, IOPS is 38989.
> if busy_poll = 0, IOPS is 33031.
>
> And busy_poll=0 test result looks no difference from the old code than
> do not have sorted events wait.
Good to hear! I think we can safely say that we've pushed rbd as far as
we can. At least on the fio side. Still appears to be some suboptimal
parts of the librbd API. And the kernel rbd driver could definitely be
improved a lot as well. Using busy_poll on the user side gets rid of a
sleep/wakeup cycle at that end, but the kernel driver still punts and
offloads any work item to a work queue...
Thanks for all your testing through this, really appreciated!
--
Jens Axboe
WARNING: multiple messages have this Message-ID (diff)
From: Jens Axboe <axboe@kernel.dk>
To: Ketor D <d.ketor@gmail.com>
Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>,
Mark Nelson <mark.nelson@inktank.com>,
Mark Nelson <mark.a.nelson@gmail.com>,
"fio@vger.kernel.org" <fio@vger.kernel.org>,
"xan.peng" <xanpeng@gmail.com>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: fio rbd completions (Was: fio rbd hang for block sizes > 1M)
Date: Wed, 29 Oct 2014 20:55:31 -0600 [thread overview]
Message-ID: <5451A8A3.8020106@kernel.dk> (raw)
In-Reply-To: <CAM9_UU-BWHSk713O9v9RuZd1PRfTfG-Vq9nox0YQO3MhV06YEA@mail.gmail.com>
On 2014-10-29 20:50, Ketor D wrote:
> Hi Jens,
>
> The current code runs good!
>
> Test Conf: jobs=1 iodepth=1 bs=4k
>
> if busy_poll = 1, IOPS is 38989.
> if busy_poll = 0, IOPS is 33031.
>
> And busy_poll=0 test result looks no difference from the old code than
> do not have sorted events wait.
Good to hear! I think we can safely say that we've pushed rbd as far as
we can. At least on the fio side. Still appears to be some suboptimal
parts of the librbd API. And the kernel rbd driver could definitely be
improved a lot as well. Using busy_poll on the user side gets rid of a
sleep/wakeup cycle at that end, but the kernel driver still punts and
offloads any work item to a work queue...
Thanks for all your testing through this, really appreciated!
--
Jens Axboe
next prev parent reply other threads:[~2014-10-30 2:55 UTC|newest]
Thread overview: 77+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-10-24 2:38 fio rbd hang for block sizes > 1M Mark Kirkwood
2014-10-24 5:35 ` Jens Axboe
2014-10-24 6:17 ` Mark Kirkwood
2014-10-24 13:19 ` Mark Nelson
[not found] ` <544A51C7.40803-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2014-10-24 14:09 ` Mark Nelson
2014-10-24 14:09 ` Mark Nelson
[not found] ` <544A5DA6.2010709-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2014-10-24 14:30 ` Jens Axboe
2014-10-24 14:30 ` Jens Axboe
2014-10-24 22:45 ` Mark Kirkwood
2014-10-24 22:45 ` Mark Kirkwood
[not found] ` <544AD67D.4030603-6STWZtX7tXAqAMOr+u8IRA@public.gmane.org>
2014-10-25 0:12 ` Mark Nelson
2014-10-25 0:12 ` Mark Nelson
[not found] ` <544AEAE7.6080603-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2014-10-25 0:37 ` Mark Kirkwood
2014-10-25 0:37 ` Mark Kirkwood
[not found] ` <544AF0D2.1050405-6STWZtX7tXAqAMOr+u8IRA@public.gmane.org>
2014-10-25 2:35 ` Mark Kirkwood
2014-10-25 2:35 ` Mark Kirkwood
2014-10-25 3:47 ` Jens Axboe
2014-10-25 4:50 ` fio rbd completions (Was: fio rbd hang for block sizes > 1M) Mark Kirkwood
2014-10-25 19:20 ` Jens Axboe
[not found] ` <544BF808.2090800-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-25 22:25 ` Mark Kirkwood
2014-10-25 22:25 ` Mark Kirkwood
2014-10-27 9:27 ` Ketor D
[not found] ` <CAM9_UU_S7qhenZW34Lw3r=RHoVa1__610RRsFScgt0adi1dpFw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-27 10:25 ` Ketor D
2014-10-27 10:25 ` Ketor D
2014-10-27 14:19 ` Jens Axboe
2014-10-27 14:15 ` Jens Axboe
[not found] ` <544C2371.1020403-6STWZtX7tXAqAMOr+u8IRA@public.gmane.org>
2014-10-27 14:19 ` Jens Axboe
2014-10-27 14:19 ` Jens Axboe
[not found] ` <544E547C.30009-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-27 15:12 ` Ketor D
2014-10-27 15:12 ` Ketor D
2014-10-27 15:22 ` Jens Axboe
2014-10-27 15:25 ` Jens Axboe
2014-10-27 15:29 ` Ketor D
[not found] ` <CAM9_UU-35i9uRu9EDQSM-b7CjmxrKYV2Gz8ocrykOYk+2q++hw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-27 15:36 ` Jens Axboe
2014-10-27 15:36 ` Jens Axboe
2014-10-27 15:45 ` Ketor D
2014-10-27 15:53 ` Jens Axboe
2014-10-27 16:20 ` Ketor D
[not found] ` <CAM9_UU8x2uZZUWaPPoy+LH mUhC_3sqKZ9GPsEqDwKUkprg4kdQ@mail.gmail.com>
[not found] ` <CAM9_UU8x2uZZUWaPPoy+LHmUhC_3sqKZ9GPsEqDwKUkprg4kdQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-27 16:55 ` Jens Axboe
2014-10-27 16:55 ` Jens Axboe
2014-10-27 21:59 ` Mark Kirkwood
2014-10-27 21:59 ` Mark Kirkwood
2014-10-27 22:32 ` Jens Axboe
2014-10-27 22:32 ` Jens Axboe
[not found] ` <544EC7F1.6010900-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-27 23:21 ` Mark Kirkwood
2014-10-27 23:21 ` Mark Kirkwood
[not found] ` <544ED37D.6060800-6STWZtX7tXAqAMOr+u8IRA@public.gmane.org>
2014-10-28 3:23 ` Ketor D
2014-10-28 3:23 ` Ketor D
[not found] ` <CAM9_UU8MHdj+mjAWBziETxPDnwTt0JBuHrQp2Fu9YtF=msae3w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-28 4:01 ` Mark Kirkwood
2014-10-28 4:01 ` Mark Kirkwood
2014-10-28 4:05 ` Jens Axboe
2014-10-28 4:49 ` Ketor D
[not found] ` <CAM9_UU9G5vQ68UxMakte-Wb5B9_KBo24ov7=hNHpYqEtko2nQg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-28 15:14 ` Jens Axboe
2014-10-28 15:14 ` Jens Axboe
2014-10-28 15:49 ` Ketor D
[not found] ` <CAM9_UU_o8kS1wJnDKTvd8+qkm9=93yfW3THr_8ni8C+5=TH6tg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-28 15:53 ` Jens Axboe
2014-10-28 15:53 ` Jens Axboe
2014-10-28 17:09 ` Jens Axboe
2014-10-28 17:09 ` Jens Axboe
2014-10-28 18:43 ` Ketor D
2014-10-29 7:15 ` Ketor D
[not found] ` <CAM9_UU8=MtUnQiembBfr8YQiDOD7TNey=mp8_H6gySenRVHy6A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-29 14:31 ` Jens Axboe
2014-10-29 14:31 ` Jens Axboe
[not found] ` <5450FA47.2030203-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-30 2:50 ` Ketor D
2014-10-30 2:50 ` Ketor D
2014-10-30 2:55 ` Jens Axboe [this message]
2014-10-30 2:55 ` Jens Axboe
2014-10-30 5:29 ` Ketor D
2014-10-30 7:44 ` Mark Kirkwood
2014-10-30 7:44 ` Mark Kirkwood
2014-10-30 8:04 ` Ketor D
2014-10-31 8:54 ` Mark Kirkwood
2014-10-31 8:54 ` Mark Kirkwood
2014-10-24 22:30 ` fio rbd hang for block sizes > 1M Mark Kirkwood
2014-10-24 22:38 ` Mark Nelson
2014-10-24 14:11 ` Danny Al-Gaaf
2014-10-24 14:31 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5451A8A3.8020106@kernel.dk \
--to=axboe@kernel.dk \
--cc=ceph-devel@vger.kernel.org \
--cc=d.ketor@gmail.com \
--cc=fio@vger.kernel.org \
--cc=mark.a.nelson@gmail.com \
--cc=mark.kirkwood@catalyst.net.nz \
--cc=mark.nelson@inktank.com \
--cc=xanpeng@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.