Re: [RFC] vhost-blk implementation

["Michael S. Tsirkin" <mst@redhat.com>]

On Tue, Mar 23, 2010 at 12:55:07PM -0700, Badari Pulavarty wrote:
> Michael S. Tsirkin wrote:
>> On Tue, Mar 23, 2010 at 10:57:33AM -0700, Badari Pulavarty wrote:
>>   
>>> Michael S. Tsirkin wrote:
>>>     
>>>> On Mon, Mar 22, 2010 at 05:34:04PM -0700, Badari Pulavarty wrote:
>>>>         
>>>>> Write Results:
>>>>> ==============
>>>>>
>>>>> I see degraded IO performance when doing sequential IO write
>>>>> tests with vhost-blk compared to virtio-blk.
>>>>>
>>>>> # time dd of=/dev/vda if=/dev/zero bs=2M oflag=direct
>>>>>
>>>>> I get ~110MB/sec with virtio-blk, but I get only ~60MB/sec with
>>>>> vhost-blk. Wondering why ?
>>>>>             
>>>> Try to look and number of interrupts and/or number of exits.
>>>>         
>>> I checked interrupts and IO exits - there is no major noticeable   
>>> difference between
>>> vhost-blk and virtio-blk scenerios.
>>>     
>>>> It could also be that you are overrunning some queue.
>>>>
>>>> I don't see any exit mitigation strategy in your patch:
>>>> when there are already lots of requests in a queue, it's usually
>>>> a good idea to disable notifications and poll the
>>>> queue as requests complete. That could help performance.
>>>>         
>>> Do you mean poll eventfd for new requests instead of waiting for new  
>>> notifications ?
>>> Where do you do that in vhost-net code ?
>>>     
>>
>> vhost_disable_notify does this.
>>
>>   
>>> Unlike network socket, since we are dealing with a file, there is no  
>>> ->poll support for it.
>>> So I can't poll for the data. And also, Issue I am having is on the   
>>> write() side.
>>>     
>>
>> Not sure I understand.
>>
>>   
>>> I looked at it some more - I see 512K write requests on the
>>> virtio-queue  in both vhost-blk and virtio-blk cases. Both qemu or
>>> vhost is doing synchronous  writes to page cache (there is no write
>>> batching in qemu that is affecting this  case).  I still puzzled on
>>> why virtio-blk outperforms vhost-blk.
>>>
>>> Thanks,
>>> Badari
>>>     
>>
>> If you say the number of requests is the same, we are left with:
>> - requests are smaller for some reason?
>> - something is causing retries?
>>   
> No. IO requests sizes are exactly same (512K) in both cases. There are  
> no retries or
> errors in both cases. One thing I am not clear is - for some reason  
> guest kernel
> could push more data into virtio-ring in case of virtio-blk vs  
> vhost-blk. Is this possible ?
> Does guest gets to run much sooner in virtio-blk case than vhost-blk ?  
> Sorry, if its dumb question -
> I don't understand  all the vhost details :(
>
> Thanks,
> Badari

BTW, did you put the backend in non-blocking mode?
As I said, vhost net passes non-blocking flag to
socket backend, but vfs_write/read that you use does
not have an option to do this.

So we'll need to extend the backend to fix that,
but just for demo purposes, you could set non-blocking
mode on the file from userspace.

-- 
MST