From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35849) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZL91u-000664-V6 for qemu-devel@nongnu.org; Fri, 31 Jul 2015 08:03:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZL91p-0007DX-RW for qemu-devel@nongnu.org; Fri, 31 Jul 2015 08:03:14 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:37335) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZL91p-0007DD-JP for qemu-devel@nongnu.org; Fri, 31 Jul 2015 08:03:09 -0400 Message-ID: <55BB63F7.8090201@codeaurora.org> Date: Fri, 31 Jul 2015 08:03:03 -0400 From: Christopher Covington MIME-Version: 1.0 References: <55B63805.8050603@virtualopensystems.com> In-Reply-To: <55B63805.8050603@virtualopensystems.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] Towards an Heterogeneous QEMU List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Christian Pinto , qemu-devel@nongnu.org Cc: Jani Kokkonen , VirtualOpenSystems Technical Team , Claudio Fontana Hi Christian, On 07/27/2015 09:54 AM, Christian Pinto wrote: > Hi all, > > this message is to present, and get feedback, on a QEMU enhancement which we > are working on. Most of the state-of-the-art SoCs use the heterogeneous > paradigm, in which a Master processor is surrounded by multiple (Slave) co- > processors (other CPUs, MCUs, hardware accelerators, etc) that usually share > the very same physical memory. An example is a multi-core ARM CPU working > alongside with two Cortex-M micro controllers. > > From the user point of view there is usually an operating system booting on > the Master processor (e.g. Linux) at platform startup, while the other > processors are used to offload the Master one from some computation or to deal > with real-time interfaces. It is the Master OS that triggers the boot of the > Slave processors, and provides them also the binary code to execute (e.g. > RTOS, binary firmware) by placing it into a pre-defined memory area that is > accessible to the Slaves. Usually the memory for the Slaves is carved out from > the Master OS during boot. Once a Slave is booted the two processors can > communicate through queues in shared memory and inter-processor interrupts > (IPIs). In Linux, it is the remoteproc/rpmsg framework that enables the > control (boot/shutdown) of Slave processors, and also to establish a > communication channel based on virtio queues. > > Currently, QEMU is not able to model such an architecture, mainly because only > a single processor can be emulated at one time, and the OS binary image needs > to be placed in memory at model startup. > > We are working on some extensions in QEMU, that enable Heterogeneous SoCs > modeling. In our proposal each processor of the target Heterogeneous SoC is > represented by a separate QEMU process, one of which will act as the Master of > the target platform. The physical shared memory abstraction is created by > leveraging on Posix shared memory. At model boot the Master QEMU will allocate > the whole memory of the target platform as a Posix shared memory segment, by > using the hostmem-file backend. The Slave QEMU instances, instead, will not > allocate any memory but wait, over a Unix socket, to receive the file > descriptor of the Posix shared memory segment allocated by the Master and an > offset. Once received, the file descriptor is mmap-ed starting from the > received offset and used as memory backend for the Slave instance. For a Slave > QEMU instance a new memory backend will be defined, to receive the file > descriptor from a socket instead of allocating the RAM of the model from a > file or regular memory. > > To resemble the behavior of a real platform, the Slave QEMU instances will not > jump into the target code until the information on the memory to be used is > received from the Master. This happens only when at a certain point during > execution, an application running on the Master OS needs to use one of the co- > processors and triggers its boot. The initialization and boot phase of a Slave > QEMU will differ from the regular one in the following: > > - No RAM memory is allocated for the model. > - No binary image is copied into memory. > - After the model initialization is complete, QEMU will jump into a wait state > in which no code is executed (since the memory is not yet available). > > When the Slave receives the fd and offset of its memory into the platform one, > it will find into such memory also the binary image to be executed and any > other information needed to complete the boot process. The Slave QEMU > instances will mmap the shared memory segment only starting from a specific > offset, thus there will be no possibility for them to corrupt the Master memory > since it will not be visible to the target Slave OS. > > Finally a new QEMU device, the Interrupt Distribution Module (IDM), will be > implemented to model a hardware mailbox/inter processor interrupt module, to be > used to send interrupts across all the QEMU instances involved in the > heterogeneous model. Such module will be based on eventfd, whose file > descriptors are exchanged with the Master using a Unix domain socket. Each QEMU > instance participating to the heterogeneous model will embed this new hardware > module into its memory map. As an example, such hardware mailboxes and IPI > modules are used in real rpmsg applications to signal with an interrupt the kick > of a virtio queue to a remote processor. > > The proposed changes are to be considered as the minimal building blocks to > enable > the emulation of an Heterogeneous SoC, that allow programmers to experiment with > various intra-SoC communication frameworks (e.g. remoteproc/rpmsg) and perform a > functional validation of their drivers and software targeting a heterogeneous > SoC. How does this multiprocess architecture compare to current efforts for multithreaded TCG? Do you anticipate needing a mechanism to keep processes roughly in sync with each other, so that one doesn't unrealistically get way far ahead of the rest? Thanks, Christopher Covington -- Qualcomm Innovation Center, Inc. The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project