qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Wangting (Kathy)" <kathy.wangting@huawei.com>
To: Vadim Rozenfeld <vrozenfe@redhat.com>
Cc: qemu-devel@nongnu.org
Subject: Re: [Qemu-devel] MSI interrupt support with vioscsi.c miniport driver
Date: Fri, 31 Oct 2014 09:55:23 +0800	[thread overview]
Message-ID: <5452EC0B.1090406@huawei.com> (raw)
In-Reply-To: <1414667730.24879.15.camel@oscar>

Thank you, I will test it.

Best regards,
Ting

On 2014-10-30 19:15, Vadim Rozenfeld wrote:
> On Thu, 2014-10-30 at 18:28 +0800, Wangting (Kathy) wrote:
>>
>> On 2014-10-30 16:48, Vadim Rozenfeld wrote:
>>> On Thu, 2014-10-30 at 14:54 +0800, Wangting (Kathy) wrote:
>>>>> On Tue, 2014-02-18 at 13:11 -0800, Nicholas A. Bellinger wrote:
>>>>>> On Tue, 2014-02-18 at 13:00 -0800, Nicholas A. Bellinger wrote:
>>>>>>> On Mon, 2014-02-10 at 11:05 -0800, Nicholas A. Bellinger wrote:
>>>>>>>
>>>>>>> <SNIP>
>>>>>>>
>>>>>>>>>>> Hi Yan,
>>>>>>>>>>>
>>>>>>>>>>> So recently I've been doing some KVM guest performance comparisons
>>>>>>>>>>> between the scsi-mq prototype using virtio-scsi + vhost-scsi, and
>>>>>>>>>>> Windows Server 2012 with vioscsi.sys (virtio-win-0.1-74.iso) +
>>>>>>>>>>> vhost-scsi using PCIe flash backend devices.
>>>>>>>>>>>
>>>>>>>>>>> I've noticed that small block random performance for the MSFT guest 
>>>>>>>>>>> is
>>>>>>>>>>> at around ~80K IOPs with multiple vioscsi LUNs + adapters, which 
>>>>>>>>>>> ends up
>>>>>>>>>>> being well below what the Linux guest with scsi-mq + virtio-scsi is
>>>>>>>>>>> capable of (~500K).
>>>>>>>>>>>
>>>>>>>>>>> After searching through the various vioscsi registry settings, it
>>>>>>>>>>> appears that MSIEnabled is being explicitly disabled (0x00000000), 
>>>>>>>>>>> that
>>>>>>>>>>> is different from what vioscsi.inx is currently defining:
>>>>>>>>>>>
>>>>>>>>>>> [pnpsafe_pci_addreg_msix]
>>>>>>>>>>> HKR, "Interrupt Management",, 0x00000010
>>>>>>>>>>> HKR, "Interrupt Management\MessageSignaledInterruptProperties",, 
>>>>>>>>>>> 0x00000010
>>>>>>>>>>> HKR, "Interrupt Management\MessageSignaledInterruptProperties", 
>>>>>>>>>>> MSISupported, 0x00010001, 0
>>>>>>>>>>> HKR, "Interrupt Management\MessageSignaledInterruptProperties", 
>>>>>>>>>>> MessageNumberLimit, 0x00010001, 4
>>>>>>>>>>>
>>>>>>>>>>> Looking deeper at vioscsi.c code, I've noticed that MSI_SUPPORTED=0 
>>>>>>>>>>> is
>>>>>>>>>>> explicitly disabled at build time in SOURCES + vioscsi.vcxproj, as 
>>>>>>>>>>> well
>>>>>>>>>>> as VioScsiFindAdapter() code always ends setting msix_enabled = 
>>>>>>>>>>> FALSE
>>>>>>>>>>> here, regardless of MSI_SUPPORTED:
>>>>>>>>>>>
>>>>>>>>>>>  
>>>>>>>>>>> https://github.com/YanVugenfirer/kvm-guest-drivers-windows/blob/master/vioscsi/vioscsi.c#L340
>>>>>>>>>>>
>>>>>>>>>>> Also looking at virtio_stor.c for the raw block driver, 
>>>>>>>>>>> MSI_SUPPORTED=1
>>>>>>>>>>> appears to be the default setting for the driver included in the 
>>>>>>>>>>> offical
>>>>>>>>>>> virtio-win iso builds, right..?
>>>>>>>>>>>
>>>>>>>>>>> Sooo, I'd like to try enabling MSI_SUPPORTED=1 in a test vioscsi.sys
>>>>>>>>>>> build of my own, but before going down the WDK development rabbit 
>>>>>>>>>>> whole,
>>>>>>>>>>> I'd like to better understand why you've explicitly disabled this 
>>>>>>>>>>> logic
>>>>>>>>>>> within vioscsi.c code to start..?
>>>>>>>>>>>
>>>>>>>>>>> Is there anything that needs to be addressed / carried over from
>>>>>>>>>>> virtio_stor.c in order to get MSI_SUPPORTED=1 to work with vioscsi.c
>>>>>>>>>>> miniport code..?
>>>>>>>>>
>>>>>>>>> Hi Nicholas,
>>>>>>>>>
>>>>>>>>> I was thinking about enabling MSI in RHEL 6.6 (build 74) but for some
>>>>>>>>> reasons decided to keep it disabled until adding mq support.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> You definitely should be able to turn on MSI_SUPPORTED, rebuild the
>>>>>>>>> driver, and switch MSISupported to 1 to make vioscsi driver working in
>>>>>>>>> MSI mode.
>>>>>>>>>    
>>>>>>>>
>>>>>>>> Thanks for the quick response.  We'll give MSI_SUPPORTED=1 a shot over
>>>>>>>> the next days with a test build on Server 2012 / Server 2008 R2 and see
>>>>>>>> how things go..
>>>>>>>>
>>>>>>>
>>>>>>> Just a quick update on progress.
>>>>>>>
>>>>>>> I've been able to successfully build + load a unsigned vioscsi.sys
>>>>>>> driver on Server 2012 with WDK 8.0.
>>>>>>>
>>>>>>> Running with MSI_SUPPORTED=1 against vhost-scsi results in a significant
>>>>>>> performance and efficiency gain, on the order of 100K to 225K IOPs for
>>>>>>> 4K block random I/O workload, depending on read/write mix.
>>>>>>>
>>>>>>
>>>>>> One other performance related question..
>>>>>>
>>>>>> In vioscsi.c:VioScsiFindAdapter() code, the default setting for
>>>>>> adaptExt->queue_depth ends up getting set to 32 (pageNum / 4) when
>>>>>> indirect mode is enabled in the following bits:
>>>>>>
>>>>>>     if(adaptExt->indirect) {
>>>>>>         adaptExt->queue_depth = max(2, (pageNum / 4));
>>>>>>     } else {
>>>>>>         adaptExt->queue_depth = pageNum / ConfigInfo->NumberOfPhysicalBreaks 
>>>>>> - 1;
>>>>>>     }
>>>>>>
>>>>>> Looking at viostor/virtio_stor.c:VirtIoFindAdapter() code, the default
>>>>>> setting for ->queue_depth appears to be 128 (pageNum):
>>>>>>
>>>>>> #if (INDIRECT_SUPPORTED)
>>>>>>     if(!adaptExt->dump_mode) {
>>>>>>         adaptExt->indirect = CHECKBIT(adaptExt->features, 
>>>>>> VIRTIO_RING_F_INDIRECT_DESC);
>>>>>>     }
>>>>>>     if(adaptExt->indirect) {
>>>>>>         adaptExt->queue_depth = pageNum;
>>>>>>     }
>>>>>> #else
>>>>>>     adaptExt->indirect = 0;
>>>>>> #endif
>>>>>>
>>>>>> Is there a reason for the lower queue_depth for vioscsi vs. viostor..?
>>>>>
>>>>> It's a horrible work around for the following bug:
>>>>> https://bugzilla.redhat.com/show_bug.cgi?id=1013443
>>>>>
>>>>> I'm going to remove it as soon as found better solution for it.
>>>>>
>>>>> Best regards,
>>>>> Vadim.
>>>>>
>>>>>
>>>> Hi Vadim,
>>>>
>>>> I have found that Bug 1013443 has been closed with a
>>>> resolution of ERRATA.
>>>>
>>>> The windows device queue must be between 20 and 254
>>>> for StorPortSetDeviceQueueDepth to succeed.
>>>>
>>>> So I have the question that why queue_depth can not be
>>>> set to pageNum(128)?
>>>
>>> It will create a problem on multi disk setup, when several 
>>> disks are attached to the same virtio-scsi pci controller.
>>> Adding some sort of manually managed SRBs queue for storing and
>>> resubmitting pending requests can solve this problem.
>>>
>>> Cheers,
>>> Vadim.
>>>
>>
>> Is there a patch for it?
> 
> Yes, not committed and not fully tested yet. Please, find it attached.
> 
> Best,
> Vadim.
> 
>>
>>>>
>>>> Best wishes,
>>>> Ting Wang
>>>>
>>>>>>
>>>>>> How about using min(adaptExt->scsi_config.cmd_per_lun, pageNum) instead..?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> -nab
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> .
>>>
>>
>>
> 

  reply	other threads:[~2014-10-31 15:20 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-10-30  6:54 [Qemu-devel] MSI interrupt support with vioscsi.c miniport driver Wangting (Kathy)
2014-10-30  8:48 ` Vadim Rozenfeld
2014-10-30 10:28   ` Wangting (Kathy)
2014-10-30 11:15     ` Vadim Rozenfeld
2014-10-31  1:55       ` Wangting (Kathy) [this message]
2014-12-09  8:20         ` [Qemu-devel] question about vioscsi queue depth patch Wangting (Kathy)
2014-12-09  9:07           ` Vadim Rozenfeld
  -- strict thread matches above, loose matches on Subject: below --
2014-02-07 20:14 [Qemu-devel] MSI interrupt support with vioscsi.c miniport driver Nicholas A. Bellinger
2014-02-09  9:24 ` Yan Vugenfirer
2014-02-09 11:35   ` Vadim Rozenfeld
2014-02-10 19:05     ` Nicholas A. Bellinger
2014-02-18 21:00       ` Nicholas A. Bellinger
2014-02-18 21:11         ` Nicholas A. Bellinger
2014-02-19  7:47           ` Vadim Rozenfeld
2014-02-19  8:03         ` Vadim Rozenfeld
2014-02-19 23:25           ` Nicholas A. Bellinger
2014-02-21  2:14             ` Vadim Rozenfeld

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5452EC0B.1090406@huawei.com \
    --to=kathy.wangting@huawei.com \
    --cc=qemu-devel@nongnu.org \
    --cc=vrozenfe@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).