From: Jag Raman <jag.raman@oracle.com>
To: Stefan Hajnoczi <stefanha@redhat.com>
Cc: elena.ufimtseva@oracle.com, fam@euphon.net,
swapnil.ingle@nutanix.com, john.g.johnson@oracle.com,
qemu-devel@nongnu.org, kraxel@redhat.com, quintela@redhat.com,
mst@redhat.com, armbru@redhat.com, kanth.ghatraju@oracle.com,
felipe@nutanix.com, thuth@redhat.com, ehabkost@redhat.com,
konrad.wilk@oracle.com, dgilbert@redhat.com,
liran.alon@oracle.com, thanos.makatos@nutanix.com,
rth@twiddle.net, kwolf@redhat.com, berrange@redhat.com,
mreitz@redhat.com, ross.lagerwall@citrix.com,
marcandre.lureau@gmail.com, pbonzini@redhat.com
Subject: Re: [PATCH v5 32/50] multi-process: Use separate MMIO communication channel
Date: Tue, 10 Mar 2020 14:04:23 -0400 [thread overview]
Message-ID: <520c4a37-2d8a-8d5c-fa12-fc33929a183b@oracle.com> (raw)
In-Reply-To: <20200306165227.GD1438162@stefanha-x1.localdomain>
On 3/6/2020 11:52 AM, Stefan Hajnoczi wrote:
> This went unanswered in the last revision:
>
> On Thu, Nov 21, 2019 at 12:31:42PM +0000, Stefan Hajnoczi wrote:
>> On Wed, Nov 13, 2019 at 11:14:50AM -0500, Jag Raman wrote:
>>> On 11/11/2019 11:21 AM, Stefan Hajnoczi wrote:
>>>> On Thu, Oct 24, 2019 at 05:09:13AM -0400, Jagannathan Raman wrote:
>>>>> Using a separate communication channel for MMIO helps
>>>>> with improving Performance
>>>>
>>>> Why?
>>>
>>> Typical initiation of IO operations involves multiple MMIO accesses per
>>> IO operation. In some legacy devices like LSI, the completion of the IO
>>> operations is also accomplished by polling on MMIO registers. Therefore,
>>> MMIO traffic can be hefty in some cases and contribute to Performance.
>>>
>>> Having a dedicated channel for MMIO ensures that it doesn't have to
>>> compete with other messages to the remote process, especially when there
>>> are multiple devices emulated by a single remote process.
>>
>> A vCPU doing a polling read on an MMIO register will cause a BAR_READ
>> message to be sent to the remote process. The vCPU thread waits for the
>> response to this message.
>>
>> When there are multiple remote devices each has its own socket, so
>> communication with different remote processes does not interfere.
>>
>> The only scenarios I can think of are:
>> 1. Interference within a single device between vCPUs and/or the QEMU
>> monitor.
>> 2. A single process serving multiple devices that is implemented in a
>> way such that different devices interfere with each other.
>>
>> It sounds like you are saying the problem is #2, but this is still
>> unclear to me. If the remote process can be implemented in a way such
>> that there is no interference when each device has a special MMIO
>> socket, then why can't it be implemented in a way such that there is no
>> interference when each device's main socket is used (each device has
>> it's own!).
>>
>> Maybe I've missed the point. It would be good if you could explain in
>> more detail.
Hi Stefan,
Sorry we missed this comment. We originally added this to enable
separate MMIO channels for each device in the remote process.
Given we are going to add a separate channel for each device in the
remote process based on the recent discussions, we could remove the
separate channel for MMIO.
Thank you very much!
--
Jag
>>
>> Stefan
next prev parent reply other threads:[~2020-03-10 18:09 UTC|newest]
Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-24 20:54 [PATCH v5 00/50] Initial support for multi-process qemu Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 01/50] multi-process: memory: alloc RAM from file at offset Jagannathan Raman
2020-03-03 19:51 ` Dr. David Alan Gilbert
2020-03-04 18:24 ` Jag Raman
2020-02-24 20:54 ` [PATCH v5 02/50] multi-process: Refactor machine_init and exit notifiers Jagannathan Raman
2020-03-29 16:45 ` Marc-André Lureau
2020-02-24 20:54 ` [PATCH v5 03/50] multi-process: add a command line option for debug file Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 04/50] multi-process: Add stub functions to facilate build of multi-process Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 05/50] multi-process: Add config option for multi-process QEMU Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 06/50] multi-process: build system for remote device process Jagannathan Raman
2020-02-24 20:54 ` [PATCH v5 07/50] multi-process: define mpqemu-link object Jagannathan Raman
2020-03-10 16:09 ` Stefan Hajnoczi
2020-03-10 18:26 ` Elena Ufimtseva
2020-03-16 11:26 ` Stefan Hajnoczi
2020-02-24 20:54 ` [PATCH v5 08/50] multi-process: add functions to synchronize proxy and remote endpoints Jagannathan Raman
2020-03-03 19:56 ` Dr. David Alan Gilbert
2020-03-04 18:42 ` Jag Raman
2020-03-04 19:46 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 09/50] multi-process: setup PCI host bridge for remote device Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 10/50] multi-process: setup a machine object for remote device process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 11/50] multi-process: setup memory manager for remote device Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 12/50] multi-process: remote process initialization Jagannathan Raman
2020-03-04 10:29 ` Dr. David Alan Gilbert
2020-03-04 18:45 ` Jag Raman
2020-03-04 19:00 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 13/50] multi-process: introduce proxy object Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 14/50] mutli-process: build remote command line args Jagannathan Raman
2020-03-02 17:36 ` Philippe Mathieu-Daudé
2020-03-02 17:47 ` Daniel P. Berrangé
2020-03-02 22:39 ` Elena Ufimtseva
2020-03-04 11:00 ` Daniel P. Berrangé
2020-03-04 16:25 ` Elena Ufimtseva
2020-03-04 16:33 ` Daniel P. Berrangé
2020-03-04 22:37 ` Elena Ufimtseva
2020-03-05 8:21 ` Daniel P. Berrangé
2020-03-06 16:25 ` Stefan Hajnoczi
2020-03-04 10:09 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 15/50] multi-process: PCI BAR read/write handling for proxy & remote endpoints Jagannathan Raman
2020-03-04 10:47 ` Dr. David Alan Gilbert
2020-03-04 19:05 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 16/50] multi-process: Synchronize remote memory Jagannathan Raman
2020-03-04 11:53 ` Dr. David Alan Gilbert
2020-03-04 19:35 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 17/50] multi-process: create IOHUB object to handle irq Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 18/50] multi-process: configure remote side devices Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 19/50] multi-process: Retrieve PCI info from remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 20/50] multi-process: add qdev_proxy_add to create proxy devices Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 21/50] multi-process: remote: add setup_devices msg processing Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 22/50] multi-process: remote: use fd for socket from parent process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 23/50] multi-process: remote: add create_done condition Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 24/50] multi-process: add processing of remote device command line Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 25/50] multi-process: Introduce build flags to separate remote process code Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 26/50] multi-process: refractor vl.c code Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 27/50] multi-process: add remote option Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 28/50] multi-process: add remote options parser Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 29/50] multi-process: add parse_cmdline in remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 30/50] multi-process: send heartbeat messages to remote Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 31/50] multi-process: handle heartbeat messages in remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 32/50] multi-process: Use separate MMIO communication channel Jagannathan Raman
2020-03-06 16:52 ` Stefan Hajnoczi
2020-03-10 18:04 ` Jag Raman [this message]
2020-02-24 20:55 ` [PATCH v5 33/50] multi-process: perform device reset in the remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 34/50] multi-process/mon: choose HMP commands based on target Jagannathan Raman
2020-03-05 10:39 ` Dr. David Alan Gilbert
2020-03-05 15:38 ` Jag Raman
2020-03-05 15:50 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 35/50] multi-process/mon: stub functions to enable QMP module for remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 36/50] multi-process/mon: enable QMP module support in the " Jagannathan Raman
2020-03-05 10:43 ` Dr. David Alan Gilbert
2020-03-05 15:40 ` Jag Raman
2020-03-05 15:52 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 37/50] multi-process/mon: Refactor monitor/chardev functions out of vl.c Jagannathan Raman
2020-03-05 10:51 ` Dr. David Alan Gilbert
2020-03-05 15:41 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 38/50] multi-process/mon: Initialize QMP module for remote processes Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 39/50] multi-process: prevent duplicate memory initialization in remote Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 40/50] multi-process/mig: build migration module in the remote process Jagannathan Raman
2020-03-04 15:58 ` Dr. David Alan Gilbert
2020-03-04 19:45 ` Jag Raman
2020-03-04 19:52 ` Dr. David Alan Gilbert
2020-03-04 20:23 ` Jag Raman
2020-03-05 10:10 ` Dr. David Alan Gilbert
2020-03-05 17:07 ` Elena Ufimtseva
2020-03-05 17:19 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 41/50] multi-process/mig: Enable VMSD save in the Proxy object Jagannathan Raman
2020-03-05 12:31 ` Dr. David Alan Gilbert
2020-03-05 16:48 ` Jag Raman
2020-03-05 17:04 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 42/50] multi-process/mig: Send VMSD of remote to " Jagannathan Raman
2020-03-05 14:39 ` Dr. David Alan Gilbert
2020-03-05 16:32 ` Elena Ufimtseva
2020-02-24 20:55 ` [PATCH v5 43/50] multi-process/mig: Load VMSD in the proxy object Jagannathan Raman
2020-03-05 15:28 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 44/50] multi-process/mig: refactor runstate_check into common file Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 45/50] multi-process/mig: Synchronize runstate of remote process Jagannathan Raman
2020-02-24 20:55 ` [PATCH v5 46/50] multi-process/mig: Restore the VMSD in " Jagannathan Raman
2020-03-05 16:05 ` Dr. David Alan Gilbert
2020-02-24 20:55 ` [PATCH v5 47/50] multi-process: Enable support for multiple devices in remote Jagannathan Raman
2020-02-28 16:44 ` Stefan Hajnoczi
2020-03-02 19:28 ` Jag Raman
2020-02-24 20:55 ` [PATCH v5 48/50] multi-process: Validate incoming commands from Proxy Jagannathan Raman
2020-02-27 17:18 ` Stefan Hajnoczi
2020-02-28 17:53 ` Elena Ufimtseva
2020-02-24 20:55 ` [PATCH v5 49/50] multi-process: add the concept description to docs/devel/qemu-multiprocess Jagannathan Raman
2020-02-27 17:11 ` Stefan Hajnoczi
2020-02-28 18:44 ` Elena Ufimtseva
2020-02-24 20:55 ` [PATCH v5 50/50] multi-process: add configure and usage information Jagannathan Raman
2020-02-27 16:58 ` Stefan Hajnoczi
2020-02-28 18:43 ` Elena Ufimtseva
2020-03-06 16:42 ` Stefan Hajnoczi
2020-02-24 21:59 ` [PATCH v5 00/50] Initial support for multi-process qemu no-reply
2020-02-24 22:03 ` no-reply
2020-02-24 22:23 ` no-reply
2020-03-01 11:57 ` Alex Bennée
2020-03-02 15:28 ` Jag Raman
2020-03-02 16:29 ` Alex Bennée
2020-03-02 16:53 ` Jag Raman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=520c4a37-2d8a-8d5c-fa12-fc33929a183b@oracle.com \
--to=jag.raman@oracle.com \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=dgilbert@redhat.com \
--cc=ehabkost@redhat.com \
--cc=elena.ufimtseva@oracle.com \
--cc=fam@euphon.net \
--cc=felipe@nutanix.com \
--cc=john.g.johnson@oracle.com \
--cc=kanth.ghatraju@oracle.com \
--cc=konrad.wilk@oracle.com \
--cc=kraxel@redhat.com \
--cc=kwolf@redhat.com \
--cc=liran.alon@oracle.com \
--cc=marcandre.lureau@gmail.com \
--cc=mreitz@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=ross.lagerwall@citrix.com \
--cc=rth@twiddle.net \
--cc=stefanha@redhat.com \
--cc=swapnil.ingle@nutanix.com \
--cc=thanos.makatos@nutanix.com \
--cc=thuth@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).