* Is there data copy between dom0 and domU @ 2014-04-23 16:06 xu cong 2014-04-23 16:14 ` Wei Liu 0 siblings, 1 reply; 8+ messages in thread From: xu cong @ 2014-04-23 16:06 UTC (permalink / raw) To: xen-devel; +Cc: Wei Liu [-- Attachment #1.1: Type: text/plain, Size: 198 bytes --] When applications running in domU receive/send data from/to network, is there data copy via shared memory between dom0 and domU? Or the zero copy is used. How about the disk I/O operations? Thanks. [-- Attachment #1.2: Type: text/html, Size: 223 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is there data copy between dom0 and domU 2014-04-23 16:06 Is there data copy between dom0 and domU xu cong @ 2014-04-23 16:14 ` Wei Liu 2014-04-23 16:22 ` Roger Pau Monné 2014-04-23 16:46 ` xu cong 0 siblings, 2 replies; 8+ messages in thread From: Wei Liu @ 2014-04-23 16:14 UTC (permalink / raw) To: xu cong; +Cc: Wei Liu, xen-devel On Wed, Apr 23, 2014 at 12:06:51PM -0400, xu cong wrote: > When applications running in domU receive/send data from/to network, is > there data copy via shared memory between dom0 and domU? Or the zero copy > is used. How about the disk I/O operations? Thanks. If you're using latest kernel (3.14-ish), guest TX path is zero-copy, otherwise it's using copy. For RX path it always copies. I think disk always copies data. But I'm not sure. Wei. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is there data copy between dom0 and domU 2014-04-23 16:14 ` Wei Liu @ 2014-04-23 16:22 ` Roger Pau Monné 2014-04-23 19:40 ` xu cong 2014-04-23 16:46 ` xu cong 1 sibling, 1 reply; 8+ messages in thread From: Roger Pau Monné @ 2014-04-23 16:22 UTC (permalink / raw) To: Wei Liu, xu cong; +Cc: xen-devel On 23/04/14 18:14, Wei Liu wrote: > On Wed, Apr 23, 2014 at 12:06:51PM -0400, xu cong wrote: >> When applications running in domU receive/send data from/to network, is >> there data copy via shared memory between dom0 and domU? Or the zero copy >> is used. How about the disk I/O operations? Thanks. > > If you're using latest kernel (3.14-ish), guest TX path is zero-copy, > otherwise it's using copy. For RX path it always copies. > > I think disk always copies data. But I'm not sure. For disks, up to Linux 3.8 data is never copied and starting from 3.8 data can be copied (in the frontend) if both DomU and Dom0 are running Linux kernel >= 3.8. Roger. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is there data copy between dom0 and domU 2014-04-23 16:22 ` Roger Pau Monné @ 2014-04-23 19:40 ` xu cong 2014-04-24 16:52 ` Wei Liu 0 siblings, 1 reply; 8+ messages in thread From: xu cong @ 2014-04-23 19:40 UTC (permalink / raw) To: Roger Pau Monné; +Cc: Wei Liu, xen-devel [-- Attachment #1.1: Type: text/plain, Size: 1450 bytes --] 2014-04-23 12:22 GMT-04:00 Roger Pau Monné <roger.pau@citrix.com>: > On 23/04/14 18:14, Wei Liu wrote: > > On Wed, Apr 23, 2014 at 12:06:51PM -0400, xu cong wrote: > >> When applications running in domU receive/send data from/to network, is > >> there data copy via shared memory between dom0 and domU? Or the zero > copy > >> is used. How about the disk I/O operations? Thanks. > > > > If you're using latest kernel (3.14-ish), guest TX path is zero-copy, > > otherwise it's using copy. For RX path it always copies. > > > > I think disk always copies data. But I'm not sure. > > For disks, up to Linux 3.8 data is never copied and starting from 3.8 > data can be copied (in the frontend) if both DomU and Dom0 are running > Linux kernel >= 3.8. > > Roger. > > What's the vif irq for? The following is my understanding, I am sure it is correct or not. When a packet arrives at physical machine, dom0 picks it up from NIC and puts it to ring buffer, then notify the destination domU. (Are all of them done by vif kernel thread?) So the vif irq is issued when dom0 notifies the destination domU, because vif is the virtual device of domU. If it is correct, the vif irq (xen-dyn-event) is an event (softirq?) on the event channel between dom0 and domU. While, if the vif irq processing does not happens on the same core where destination domU is running, one IPI is issued to finish the notification. Regards, Cong [-- Attachment #1.2: Type: text/html, Size: 2066 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is there data copy between dom0 and domU 2014-04-23 19:40 ` xu cong @ 2014-04-24 16:52 ` Wei Liu 2014-04-24 19:19 ` xu cong 0 siblings, 1 reply; 8+ messages in thread From: Wei Liu @ 2014-04-24 16:52 UTC (permalink / raw) To: xu cong; +Cc: xen-devel, Wei Liu, Roger Pau Monné On Wed, Apr 23, 2014 at 03:40:25PM -0400, xu cong wrote: > 2014-04-23 12:22 GMT-04:00 Roger Pau Monné <roger.pau@citrix.com>: > > > On 23/04/14 18:14, Wei Liu wrote: > > > On Wed, Apr 23, 2014 at 12:06:51PM -0400, xu cong wrote: > > >> When applications running in domU receive/send data from/to network, is > > >> there data copy via shared memory between dom0 and domU? Or the zero > > copy > > >> is used. How about the disk I/O operations? Thanks. > > > > > > If you're using latest kernel (3.14-ish), guest TX path is zero-copy, > > > otherwise it's using copy. For RX path it always copies. > > > > > > I think disk always copies data. But I'm not sure. > > > > For disks, up to Linux 3.8 data is never copied and starting from 3.8 > > data can be copied (in the frontend) if both DomU and Dom0 are running > > Linux kernel >= 3.8. > > > > Roger. > > > > > What's the vif irq for? The following is my understanding, I am sure it is > correct or not. > When a packet arrives at physical machine, dom0 picks it up from NIC and > puts it to ring buffer, then notify the destination domU. (Are all of them Yes. > done by vif kernel thread?) So the vif irq is issued when dom0 notifies the For the guest RX path, yes. > destination domU, because vif is the virtual device of domU. If it is > correct, the vif irq (xen-dyn-event) is an event (softirq?) on the event > channel between dom0 and domU. While, if the vif irq processing does not > happens on the same core where destination domU is running, one IPI is > issued to finish the notification. > Softirq in Linux refers to a specific type of bottom-half mechanism; IPI in general means inter-processor interrupt. IIRC xen-dyn-event refers to event channel event. You might want to distinguish several concepts before moving on. Wei. > Regards, > Cong ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is there data copy between dom0 and domU 2014-04-24 16:52 ` Wei Liu @ 2014-04-24 19:19 ` xu cong 2014-04-25 8:25 ` Ian Campbell 0 siblings, 1 reply; 8+ messages in thread From: xu cong @ 2014-04-24 19:19 UTC (permalink / raw) To: Wei Liu; +Cc: xen-devel, Roger Pau Monné [-- Attachment #1.1: Type: text/plain, Size: 2211 bytes --] 2014-04-24 12:52 GMT-04:00 Wei Liu <wei.liu2@citrix.com>: > On Wed, Apr 23, 2014 at 03:40:25PM -0400, xu cong wrote: > > 2014-04-23 12:22 GMT-04:00 Roger Pau Monné <roger.pau@citrix.com>: > > > > > On 23/04/14 18:14, Wei Liu wrote: > > > > On Wed, Apr 23, 2014 at 12:06:51PM -0400, xu cong wrote: > > > >> When applications running in domU receive/send data from/to > network, is > > > >> there data copy via shared memory between dom0 and domU? Or the zero > > > copy > > > >> is used. How about the disk I/O operations? Thanks. > > > > > > > > If you're using latest kernel (3.14-ish), guest TX path is zero-copy, > > > > otherwise it's using copy. For RX path it always copies. > > > > > > > > I think disk always copies data. But I'm not sure. > > > > > > For disks, up to Linux 3.8 data is never copied and starting from 3.8 > > > data can be copied (in the frontend) if both DomU and Dom0 are running > > > Linux kernel >= 3.8. > > > > > > Roger. > > > > > > > > What's the vif irq for? The following is my understanding, I am sure it > is > > correct or not. > > When a packet arrives at physical machine, dom0 picks it up from NIC and > > puts it to ring buffer, then notify the destination domU. (Are all of > them > > Yes. > > > done by vif kernel thread?) So the vif irq is issued when dom0 notifies > the > > For the guest RX path, yes. > > > destination domU, because vif is the virtual device of domU. If it is > > correct, the vif irq (xen-dyn-event) is an event (softirq?) on the event > > channel between dom0 and domU. While, if the vif irq processing does not > > happens on the same core where destination domU is running, one IPI is > > issued to finish the notification. > > > > Softirq in Linux refers to a specific type of bottom-half mechanism; IPI > in general means inter-processor interrupt. IIRC xen-dyn-event refers to > event channel event. You might want to distinguish several concepts > before moving on. > > Wei. > > > Regards, > > Cong > Thanks for your clarification. Is there any overhead if two ends of event channel are not on the same core compared with two ends running on the same core? Regards, Cong [-- Attachment #1.2: Type: text/html, Size: 2972 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is there data copy between dom0 and domU 2014-04-24 19:19 ` xu cong @ 2014-04-25 8:25 ` Ian Campbell 0 siblings, 0 replies; 8+ messages in thread From: Ian Campbell @ 2014-04-25 8:25 UTC (permalink / raw) To: xu cong; +Cc: Roger Pau Monné, Wei Liu, xen-devel On Thu, 2014-04-24 at 15:19 -0400, xu cong wrote: > Is there any overhead if two ends of event channel are not on the same > core compared with two ends running on the same core? Why don't you take a look at the code? Ian ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Is there data copy between dom0 and domU 2014-04-23 16:14 ` Wei Liu 2014-04-23 16:22 ` Roger Pau Monné @ 2014-04-23 16:46 ` xu cong 1 sibling, 0 replies; 8+ messages in thread From: xu cong @ 2014-04-23 16:46 UTC (permalink / raw) To: Wei Liu; +Cc: xen-devel [-- Attachment #1.1: Type: text/plain, Size: 881 bytes --] 2014-04-23 12:14 GMT-04:00 Wei Liu <wei.liu2@citrix.com>: > On Wed, Apr 23, 2014 at 12:06:51PM -0400, xu cong wrote: > > When applications running in domU receive/send data from/to network, is > > there data copy via shared memory between dom0 and domU? Or the zero copy > > is used. How about the disk I/O operations? Thanks. > > If you're using latest kernel (3.14-ish), guest TX path is zero-copy, > otherwise it's using copy. For RX path it always copies. > > I think disk always copies data. But I'm not sure. > > Wei. > I am using the 3.12. So the cpu utilization (top output) of vifx.0 process in dom0 is for data copy and irq processing, right? I found that if the vif irq processing is on the same core where the corresponding domU is running, the CPU utilization of dom0 is lower and the application IOPS is higher. I think maybe it is caused by less IPIs are issued. [-- Attachment #1.2: Type: text/html, Size: 1378 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2014-04-25 8:25 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-04-23 16:06 Is there data copy between dom0 and domU xu cong 2014-04-23 16:14 ` Wei Liu 2014-04-23 16:22 ` Roger Pau Monné 2014-04-23 19:40 ` xu cong 2014-04-24 16:52 ` Wei Liu 2014-04-24 19:19 ` xu cong 2014-04-25 8:25 ` Ian Campbell 2014-04-23 16:46 ` xu cong
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.