Some more basic questions..

Some more basic questions..

[Marcus White]

Hello,
Some more basic questions..

1. How can I ensure that memory for the a guest is available and
reserved? In other words, I bring up a Linux VM which has 4G
allocated, I want to make sure it has all the 4G available right away.
I saw references to balloon driver, it seemed like that was more for
dynamic memory exchange between host and guest. In my case, it is a
Linux guest with a Linux VM.

2. Does the host reclaim pages from guest if it needs it without a
balloon driver?

3. This might be a very basic question, please bear with me:) If I use
virtio for say network and block, does network and block traffic still
go through QEMU? Is the host part of virtio basically QEMU or is it
something that runs in the host kernel. If QEMU, does every IO still
pass through it? Found some conflicting information, so not a 100%
sure. Found this, not sure if it is 100% accurate? Trying to
understand the flow through different layers, and what the layers are.
http://events.linuxfoundation.org/sites/events/files/slides/MasakiKimura_LinuxConNorthAmerica2013_1.pdf


Thank you in Advance:)

RE: Some more basic questions..

[Venkateswara Rao Nandigam]

>>>1. How can I ensure that memory for the a guest is available and reserved? In other words, I bring up a Linux VM which has 4G allocated, I want to make sure it has all the 4G available right away.
I saw references to balloon driver, it seemed like that was more for dynamic memory exchange between host and guest. In my case, it is a Linux guest with a Linux VM.

Allocate memory and use all of it(use at boundaries of some bigger chunks). That’s how you can make sure.

>>>2. Does the host reclaim pages from guest if it needs it without a balloon driver?
Yes

>>>3. This might be a very basic question, please bear with me:) If I use virtio for say network and block, does network and block traffic still go through QEMU? Is the host part of virtio basically QEMU or is it something that runs in the host kernel. If QEMU, does every IO still pass through it? Found some >>>conflicting information, so not a 100% sure. Found this, not sure if it is 100% accurate? Trying to understand the flow through different layers, and what the layers are.
>>>http://events.linuxfoundation.org/sites/events/files/slides/MasakiKimura_LinuxConNorthAmerica2013_1.pdf

Virtio device's I/O will go through QEMU, if you want to bypass QEMU, use vhost=ON on these virtio devices.


-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Marcus White
Sent: Thursday, May 29, 2014 2:30 AM
To: kvm
Subject: Some more basic questions..

Hello,
Some more basic questions..

1. How can I ensure that memory for the a guest is available and reserved? In other words, I bring up a Linux VM which has 4G allocated, I want to make sure it has all the 4G available right away.
I saw references to balloon driver, it seemed like that was more for dynamic memory exchange between host and guest. In my case, it is a Linux guest with a Linux VM.

2. Does the host reclaim pages from guest if it needs it without a balloon driver?

3. This might be a very basic question, please bear with me:) If I use virtio for say network and block, does network and block traffic still go through QEMU? Is the host part of virtio basically QEMU or is it something that runs in the host kernel. If QEMU, does every IO still pass through it? Found some conflicting information, so not a 100% sure. Found this, not sure if it is 100% accurate? Trying to understand the flow through different layers, and what the layers are.
http://events.linuxfoundation.org/sites/events/files/slides/MasakiKimura_LinuxConNorthAmerica2013_1.pdf


Thank you in Advance:)
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Some more basic questions..

[Zhang Haoyu]

> Hello,
> Some more basic questions..
> 
> 1. How can I ensure that memory for the a guest is available and
> reserved? In other words, I bring up a Linux VM which has 4G
> allocated, I want to make sure it has all the 4G available right away.
> I saw references to balloon driver, it seemed like that was more for
> dynamic memory exchange between host and guest. In my case, it is a
> Linux guest with a Linux VM.
> 
Try -realtime mlock=on, this option ensure that all the memory allocated for VM  is available right away.

> 2. Does the host reclaim pages from guest if it needs it without a
>balloon driver?
> 
Yes, just consider the memory of VM as that of QEMU(user-process).

> 3. This might be a very basic question, please bear with me:) If I use
> virtio for say network and block, does network and block traffic still
> go through QEMU? Is the host part of virtio basically QEMU or is it
> something that runs in the host kernel. If QEMU, does every IO still
> pass through it? Found some conflicting information, so not a 100%
> sure. Found this, not sure if it is 100% accurate? Trying to
> understand the flow through different layers, and what the layers are.
> http://events.linuxfoundation.org/sites/events/files/slides/MasakiKimura_LinuxConNorthAmerica2013_1.pdf
> 
> 
It depends on whether you enable the vhost or something.
 
> Thank you in Advance:)
> --

Re: Some more basic questions..

[Marcus White]

Thanks Zhang and Venkateshwara, some more follow up questions below:)

1. Does -realtime mlock=on allocate all the memory upfront and keep it
for the VM, or does it just make sure the memory that is allocated
within the guest is not swapped out under host memory pressure?

2.  I notice on a 4G guest on an 8G host, guest allocates only about
1G initially, and the rest later as I start applications. Is there a
way for me to reserve ALL memory (4G in this case) upfront somehow
without changes to guest and without allocating it? It will have to be
something the host OS or some component within the host OS. Isnt there
something to that effect? It seems odd that there isnt.


Thank you in advance.

On Wed, May 28, 2014 at 11:45 PM, Zhang Haoyu <zhanghy@sangfor.com> wrote:
>> Hello,
>> Some more basic questions..
>>
>> 1. How can I ensure that memory for the a guest is available and
>> reserved? In other words, I bring up a Linux VM which has 4G
>> allocated, I want to make sure it has all the 4G available right away.
>> I saw references to balloon driver, it seemed like that was more for
>> dynamic memory exchange between host and guest. In my case, it is a
>> Linux guest with a Linux VM.
>>
> Try -realtime mlock=on, this option ensure that all the memory allocated for VM  is available right away.
>
>> 2. Does the host reclaim pages from guest if it needs it without a
>>balloon driver?
>>
> Yes, just consider the memory of VM as that of QEMU(user-process).
>
>> 3. This might be a very basic question, please bear with me:) If I use
>> virtio for say network and block, does network and block traffic still
>> go through QEMU? Is the host part of virtio basically QEMU or is it
>> something that runs in the host kernel. If QEMU, does every IO still
>> pass through it? Found some conflicting information, so not a 100%
>> sure. Found this, not sure if it is 100% accurate? Trying to
>> understand the flow through different layers, and what the layers are.
>> http://events.linuxfoundation.org/sites/events/files/slides/MasakiKimura_LinuxConNorthAmerica2013_1.pdf
>>
>>
> It depends on whether you enable the vhost or something.
>
>> Thank you in Advance:)
>> --
>

Re: Some more basic questions..

[Zhang Haoyu]

>Thanks Zhang and Venkateshwara, some more follow up questions below:)
>
>1. Does -realtime mlock=on allocate all the memory upfront and keep it
>for the VM, or does it just make sure the memory that is allocated
>within the guest is not swapped out under host memory pressure?
>
“-realtime mlock=on” will mlockall(MCL_CURRENT | MCL_FUTURE) QEMU's ALL memory,
because VM's memory is part of QEMU's, so this option will keep VM's memory unswapped.

>2.  I notice on a 4G guest on an 8G host, guest allocates only about
>1G initially, and the rest later as I start applications. Is there a
>way for me to reserve ALL memory (4G in this case) upfront somehow
>without changes to guest and without allocating it? It will have to be
>something the host OS or some component within the host OS. Isnt there
>something to that effect? It seems odd that there isnt.
>
On linux, user-process's memory is allocating on demand, the physical memory does not allocate until the virtual memory is touched.
Because VM's memory is part of QEMU's, so ...
I guess the VM you said above is linux guest. Windows guest will memset its memory during booting period.

You can use "-realtime mlock=on" to reserve VM's ALL memory upfront.

>
>Thank you in advance.

Re: Some more basic questions..

[Marcus White]

A few additional questions:)

1. If IO to a block device goes through QEMU and not vhost, are there
data copies between kernel and user mode if I do IO to a block device
or is it zero copy? Kind of related to Question (2) also.

2. A related question to (1), is the QEMU process separate from the VM
process or do they all run on the same process as different threads? I
did read about IO threads and it seemed like they are all part of the
same process, wanted to confirm.

Thank you:)

On Thu, May 29, 2014 at 11:48 PM, Zhang Haoyu <zhanghy@sangfor.com> wrote:
>>Thanks Zhang and Venkateshwara, some more follow up questions below:)
>>
>>1. Does -realtime mlock=on allocate all the memory upfront and keep it
>>for the VM, or does it just make sure the memory that is allocated
>>within the guest is not swapped out under host memory pressure?
>>
> “-realtime mlock=on” will mlockall(MCL_CURRENT | MCL_FUTURE) QEMU's ALL memory,
> because VM's memory is part of QEMU's, so this option will keep VM's memory unswapped.
>
>>2.  I notice on a 4G guest on an 8G host, guest allocates only about
>>1G initially, and the rest later as I start applications. Is there a
>>way for me to reserve ALL memory (4G in this case) upfront somehow
>>without changes to guest and without allocating it? It will have to be
>>something the host OS or some component within the host OS. Isnt there
>>something to that effect? It seems odd that there isnt.
>>
> On linux, user-process's memory is allocating on demand, the physical memory does not allocate until the virtual memory is touched.
> Because VM's memory is part of QEMU's, so ...
> I guess the VM you said above is linux guest. Windows guest will memset its memory during booting period.
>
> You can use "-realtime mlock=on" to reserve VM's ALL memory upfront.
>
>>
>>Thank you in advance.

Re: Some more basic questions..

[Zhang Haoyu]

>A few additional questions:)
>
>1. If IO to a block device goes through QEMU and not vhost, are there
>data copies between kernel and user mode if I do IO to a block device
>or is it zero copy? Kind of related to Question (2) also.
>
I'm not sure, it perhaps depend on which IO mode you chose, qemu pthreads IO, or linux AIO?
Somebody else do a favour, please.

>2. A related question to (1), is the QEMU process separate from the VM
>process or do they all run on the same process as different threads? I
>did read about IO threads and it seemed like they are all part of the
>same process, wanted to confirm.
>
Yes, vcpu threads and qemu backend IO threads belong to the same process(qemu).

>Thank you:)

Re: Some more basic questions..

[Marcus White]

On Mon, Jun 2, 2014 at 8:43 PM, Zhang Haoyu <zhanghy@sangfor.com> wrote:
>>A few additional questions:)
>>
>>1. If IO to a block device goes through QEMU and not vhost, are there
>>data copies between kernel and user mode if I do IO to a block device
>>or is it zero copy? Kind of related to Question (2) also.
>>
> I'm not sure, it perhaps depend on which IO mode you chose, qemu pthreads IO, or linux AIO?
> Somebody else do a favour, please.

That would be definitely good to know:) I didnt know about the modes
as such. Can you clarify? Are you talking about vhost-LIO vs Qemu? The
LIO part, I do believe, is zero copy..
I have seen pretty high memory usage on QEMU thread, not sure why.

>
>>2. A related question to (1), is the QEMU process separate from the VM
>>process or do they all run on the same process as different threads? I
>>did read about IO threads and it seemed like they are all part of the
>>same process, wanted to confirm.
>>
> Yes, vcpu threads and qemu backend IO threads belong to the same process(qemu).
>
>>Thank you:)
>

Re: Some more basic questions..

[Zhang Haoyu]

>>>1. If IO to a block device goes through QEMU and not vhost, are there
>>>data copies between kernel and user mode if I do IO to a block device
>>>or is it zero copy? Kind of related to Question (2) also.
>>>
>> I'm not sure, it perhaps depend on which IO mode you chose, qemu pthreads IO, or linux AIO?
>> Somebody else do a favour, please.
>
>That would be definitely good to know:) I didnt know about the modes
>as such. Can you clarify? Are you talking about vhost-LIO vs Qemu? The
>LIO part, I do believe, is zero copy..
>I have seen pretty high memory usage on QEMU thread, not sure why.
>
Sorry for my poor knowledge about block subsystem.
I don't know about vhost-LIO, do you mean vhost-blk or vhost-scsi?
Linux AIO what I said above is the linux native aio, which only works for uncached access(O_DIRECT), qemu main-thread use it to do block IO,
qemu pthreads IO means that qemu creates dedicated threads to do block IO job.

RE: Some more basic questions..

[Venkateswara Rao Nandigam]

>>>>1. If IO to a block device goes through QEMU and not vhost, are there data copies between kernel and user mode if I do IO to a block device or is it zero copy? Kind of related to Question (2) also.

An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively.

-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Marcus White
Sent: Tuesday, June 03, 2014 2:50 AM
To: Zhang Haoyu
Cc: kvm
Subject: Re: Some more basic questions..

A few additional questions:)

1. If IO to a block device goes through QEMU and not vhost, are there data copies between kernel and user mode if I do IO to a block device or is it zero copy? Kind of related to Question (2) also.

2. A related question to (1), is the QEMU process separate from the VM process or do they all run on the same process as different threads? I did read about IO threads and it seemed like they are all part of the same process, wanted to confirm.

Thank you:)

On Thu, May 29, 2014 at 11:48 PM, Zhang Haoyu <zhanghy@sangfor.com> wrote:
>>Thanks Zhang and Venkateshwara, some more follow up questions below:)
>>
>>1. Does -realtime mlock=on allocate all the memory upfront and keep it 
>>for the VM, or does it just make sure the memory that is allocated 
>>within the guest is not swapped out under host memory pressure?
>>
> “-realtime mlock=on” will mlockall(MCL_CURRENT | MCL_FUTURE) QEMU's 
> ALL memory, because VM's memory is part of QEMU's, so this option will keep VM's memory unswapped.
>
>>2.  I notice on a 4G guest on an 8G host, guest allocates only about 
>>1G initially, and the rest later as I start applications. Is there a 
>>way for me to reserve ALL memory (4G in this case) upfront somehow 
>>without changes to guest and without allocating it? It will have to be 
>>something the host OS or some component within the host OS. Isnt there 
>>something to that effect? It seems odd that there isnt.
>>
> On linux, user-process's memory is allocating on demand, the physical memory does not allocate until the virtual memory is touched.
> Because VM's memory is part of QEMU's, so ...
> I guess the VM you said above is linux guest. Windows guest will memset its memory during booting period.
>
> You can use "-realtime mlock=on" to reserve VM's ALL memory upfront.
>
>>
>>Thank you in advance.
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RE: Some more basic questions..

[Zhang Haoyu]

>>A few additional questions:)
>>
>>1. If IO to a block device goes through QEMU and not vhost, are there
>>data copies between kernel and user mode if I do IO to a block device
>>or is it zero copy? Kind of related to Question (2) also.
>>
>An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively.
I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine
DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? 

IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right?

RE: RE: Some more basic questions..

[Venkateswara Rao Nandigam]

>>>> I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
>>>>>Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? 

A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in QEMU is avoided when vhost is used. That is why using vhost is "realtively" zero copy.

I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, like virtio. So they can be called virtio-blk, virtio-scsi

So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much more faster as one copy will be avoided.

Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space.

-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Zhang Haoyu
Sent: Tuesday, June 03, 2014 4:30 PM
To: Venkateswara Rao Nandigam; Marcus White
Cc: kvm
Subject: Re: RE: Some more basic questions..

>>A few additional questions:)
>>
>>1. If IO to a block device goes through QEMU and not vhost, are there 
>>data copies between kernel and user mode if I do IO to a block device 
>>or is it zero copy? Kind of related to Question (2) also.
>>
>An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively.
I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? 

IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right?

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RE: Some more basic questions..

[Marcus White]

Cant the pages be locked down by the host when it gets the call down
for an operation with a buffer(like if a usual user mode process
case), and once locked down in the host kernel, the physical address
be used directly for DMA, whether send or recieve? Not sure if I fully
understand why a copy will be done irrespective. Can you please
clarify?

Also in the QEMU case, since they are all part of the same process (VM
and QEMU), where is the data copy?

Apologies if these are basic questions, trying to understand the path here.

Thanks in advance.
MW



On Tue, Jun 3, 2014 at 4:25 AM, Venkateswara Rao Nandigam
<venkateswararao.nandigam@citrix.com> wrote:
>>>>> I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
>>>>>>Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read?
>
> A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in QEMU is avoided when vhost is used. That is why using vhost is "realtively" zero copy.
>
> I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, like virtio. So they can be called virtio-blk, virtio-scsi
>
> So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much more faster as one copy will be avoided.
>
> Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space.
>
> -----Original Message-----
> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Zhang Haoyu
> Sent: Tuesday, June 03, 2014 4:30 PM
> To: Venkateswara Rao Nandigam; Marcus White
> Cc: kvm
> Subject: Re: RE: Some more basic questions..
>
>>>A few additional questions:)
>>>
>>>1. If IO to a block device goes through QEMU and not vhost, are there
>>>data copies between kernel and user mode if I do IO to a block device
>>>or is it zero copy? Kind of related to Question (2) also.
>>>
>>An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively.
> I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
> Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read?
>
> IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right?
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RE: Some more basic questions..

[Marcus White]

Hello
Just a bump to see if anyone can help:)

MW

On Tue, Jun 3, 2014 at 6:08 PM, Marcus White <roastedseaweed.k@gmail.com> wrote:
> Cant the pages be locked down by the host when it gets the call down
> for an operation with a buffer(like if a usual user mode process
> case), and once locked down in the host kernel, the physical address
> be used directly for DMA, whether send or recieve? Not sure if I fully
> understand why a copy will be done irrespective. Can you please
> clarify?
>
> Also in the QEMU case, since they are all part of the same process (VM
> and QEMU), where is the data copy?
>
> Apologies if these are basic questions, trying to understand the path here.
>
> Thanks in advance.
> MW
>
>
>
> On Tue, Jun 3, 2014 at 4:25 AM, Venkateswara Rao Nandigam
> <venkateswararao.nandigam@citrix.com> wrote:
>>>>>> I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
>>>>>>>Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read?
>>
>> A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in QEMU is avoided when vhost is used. That is why using vhost is "realtively" zero copy.
>>
>> I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, like virtio. So they can be called virtio-blk, virtio-scsi
>>
>> So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much more faster as one copy will be avoided.
>>
>> Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space.
>>
>> -----Original Message-----
>> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Zhang Haoyu
>> Sent: Tuesday, June 03, 2014 4:30 PM
>> To: Venkateswara Rao Nandigam; Marcus White
>> Cc: kvm
>> Subject: Re: RE: Some more basic questions..
>>
>>>>A few additional questions:)
>>>>
>>>>1. If IO to a block device goes through QEMU and not vhost, are there
>>>>data copies between kernel and user mode if I do IO to a block device
>>>>or is it zero copy? Kind of related to Question (2) also.
>>>>
>>>An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively.
>> I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
>> Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read?
>>
>> IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right?
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RE: Some more basic questions..

[Marcus White]

One more bump:)

MW..

On Wed, Jun 4, 2014 at 9:42 PM, Marcus White <roastedseaweed.k@gmail.com> wrote:
> Hello
> Just a bump to see if anyone can help:)
>
> MW
>
> On Tue, Jun 3, 2014 at 6:08 PM, Marcus White <roastedseaweed.k@gmail.com> wrote:
>> Cant the pages be locked down by the host when it gets the call down
>> for an operation with a buffer(like if a usual user mode process
>> case), and once locked down in the host kernel, the physical address
>> be used directly for DMA, whether send or recieve? Not sure if I fully
>> understand why a copy will be done irrespective. Can you please
>> clarify?
>>
>> Also in the QEMU case, since they are all part of the same process (VM
>> and QEMU), where is the data copy?
>>
>> Apologies if these are basic questions, trying to understand the path here.
>>
>> Thanks in advance.
>> MW
>>
>>
>>
>> On Tue, Jun 3, 2014 at 4:25 AM, Venkateswara Rao Nandigam
>> <venkateswararao.nandigam@citrix.com> wrote:
>>>>>>> I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
>>>>>>>>Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read?
>>>
>>> A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in QEMU is avoided when vhost is used. That is why using vhost is "realtively" zero copy.
>>>
>>> I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, like virtio. So they can be called virtio-blk, virtio-scsi
>>>
>>> So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much more faster as one copy will be avoided.
>>>
>>> Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space.
>>>
>>> -----Original Message-----
>>> From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of Zhang Haoyu
>>> Sent: Tuesday, June 03, 2014 4:30 PM
>>> To: Venkateswara Rao Nandigam; Marcus White
>>> Cc: kvm
>>> Subject: Re: RE: Some more basic questions..
>>>
>>>>>A few additional questions:)
>>>>>
>>>>>1. If IO to a block device goes through QEMU and not vhost, are there
>>>>>data copies between kernel and user mode if I do IO to a block device
>>>>>or is it zero copy? Kind of related to Question (2) also.
>>>>>
>>>>An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively.
>>> I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
>>> Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read?
>>>
>>> IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right?
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Some more basic questions..

[Zhang Haoyu]

>>>>A few additional questions:)
>>>>
>>>>1. If IO to a block device goes through QEMU and not vhost, are there
>>>>data copies between kernel and user mode if I do IO to a block device
>>>>or is it zero copy? Kind of related to Question (2) also.
>>>>
>>>An additional copy will be avoided only by using vhost, so if you are using vhost you can call it zero copy realtively.
>>I know that kvm support network tx zero-copy when using vhost, but rx copy is still performed in vhost, because the NIC cannot determine
>>DMA to which VM's rx buffers before L2 switching(unless vhost using page-flip between HVA->HPA and GPA->HPA, or macvtap over SRIOV-VF is used).
>>Storage has no this limitation, can vhost-blk and vhost-scsi avoid data copy in both write and read? 
>>
>>IIUC, even if vhost is not used, qemu using linux native aio can avoid data copy between user and kernel space, right?
>>
>A copy in kernel will be done irrespective of Rx/Tx when vhost is used. One copy between user space and kernel space in 
>QEMU is avoided when vhost is used. That is why using vhost is "realtively" zero copy.
>
I agree on that one copy between user space and kernel space in QEMU is avoided when vhost is used.
But when vhost is used, the data copy in kernel also can be removed during tx, not only the data copy between user space and kernel space,
which is implemented in tun.c, please see below commit,
https://git.kernel.org/cgit/virt/kvm/kvm.git/commit/drivers/net/tun.c?id=0690899b4d4501b3505be069b9a687e68ccbe15b
it is called tx zero-copy, just like that done in xen-netback.

So, the zero-copy I mentioned above is that disk hardware directly DMA from vm memory to disk for write, 
or from disk to vm memory for read.
Just like network tx zero-copy implemented in tun.c.

>I don’t think that they can be classified as vhost-blk or vhost-scsi. Vhost is an add on to an existing Guest driver, 
>like virtio. So they can be called virtio-blk, virtio-scsi
>
Sorry for my poor English, 
vhost-blk is vs virtio-blk backend implemented in qemu, moving the processing from qemu to kernel,
vhost-scsi is vs virtio-scci backend implemented in qemu, moving the processing from qemu to kernel.

>So, using PV drivers(virtio-blk) for disk access makes accesses faster and using vhost on top of this will make it much 
>more faster as one copy will be avoided.
>
>Linus AIO will facilitate avoiding blocking on a i/o, but doubt if it has anything to do with copy across user space and kernel space.
I'm not sure about that if linux native aio can avoid the data copy between user space and kernel space.
The kernel get the physical address of user-process virtual address by get_user_pages/use_mm, and then pass it to harddisk for DMA?