From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36581) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YJAiU-0007pi-0F for qemu-devel@nongnu.org; Wed, 04 Feb 2015 19:54:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YJAiQ-0006pL-Pz for qemu-devel@nongnu.org; Wed, 04 Feb 2015 19:54:45 -0500 Message-ID: <54D2BF4F.1030609@suse.de> Date: Thu, 05 Feb 2015 01:54:39 +0100 From: Alexander Graf MIME-Version: 1.0 References: <1422943851-25836-1-git-send-email-david@gibson.dropbear.id.au> <20150203211906.GA13992@iris.ozlabs.ibm.com> <20150204013211.GU28703@voom.fritz.box> <54D23872.90007@suse.de> <20150205004812.GD25675@voom.fritz.box> In-Reply-To: <20150205004812.GD25675@voom.fritz.box> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] pseries: Enable in-kernel H_LOGICAL_CI_{LOAD, STORE} implementations List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: David Gibson Cc: aik@ozlabs.ru, qemu-ppc@nongnu.org, Paul Mackerras , qemu-devel@nongnu.org, mdroth@us.ibm.com On 05.02.15 01:48, David Gibson wrote: > On Wed, Feb 04, 2015 at 04:19:14PM +0100, Alexander Graf wrote: >> >> >> On 04.02.15 02:32, David Gibson wrote: >>> On Wed, Feb 04, 2015 at 08:19:06AM +1100, Paul Mackerras wrote: >>>> On Tue, Feb 03, 2015 at 05:10:51PM +1100, David Gibson wrote: >>>>> qemu currently implements the hypercalls H_LOGICAL_CI_LOAD and >>>>> H_LOGICAL_CI_STORE as PAPR extensions. These are used by the SLOF firmware >>>>> for IO, because performing cache inhibited MMIO accesses with the MMU off >>>>> (real mode) is very awkward on POWER. >>>>> >>>>> This approach breaks when SLOF needs to access IO devices implemented >>>>> within KVM instead of in qemu. The simplest example would be virtio-blk >>>>> using an iothread, because the iothread / dataplane mechanism relies on >>>>> an in-kernel implementation of the virtio queue notification MMIO. >>>>> >>>>> To fix this, an in-kernel implementation of these hypercalls has been made, >>>>> however, the hypercalls still need to be enabled from qemu. This performs >>>>> the necessary calls to do so. >>>>> >>>>> Signed-off-by: David Gibson >>>> >>>> [snip] >>>> >>>>> + ret1 = kvmppc_enable_hcall(kvm_state, H_LOGICAL_CI_LOAD); >>>>> + if (ret1 != 0) { >>>>> + fprintf(stderr, "Warning: error enabling H_LOGICAL_CI_LOAD in KVM:" >>>>> + " %s\n", strerror(errno)); >>>>> + } >>>>> + >>>>> + ret2 = kvmppc_enable_hcall(kvm_state, H_LOGICAL_CI_STORE); >>>>> + if (ret2 != 0) { >>>>> + fprintf(stderr, "Warning: error enabling H_LOGICAL_CI_STORE in KVM:" >>>>> + " %s\n", strerror(errno)); >>>>> + } >>>>> + >>>>> + if ((ret1 != 0) || (ret2 != 0)) { >>>>> + fprintf(stderr, "Warning: Couldn't enable H_LOGICAL_CI_* in KVM, SLOF" >>>>> + " may be unable to operate devices with in-kernel emulation\n"); >>>>> + } >>>> >>>> You'll always get these warnings if you're running on an old (meaning >>>> current upstream) kernel, which could be annoying. >>> >>> True. >>> >>>> Is there any way >>>> to tell whether you have configured any devices which need the >>>> in-kernel MMIO emulation and only warn if you have? >>> >>> In theory, I guess so. In practice I can't see how you'd enumerate >>> all devices that might require kernel intervention without something >>> horribly invasive. >> >> We could WARN_ONCE in QEMU if we emulate such a hypercall, but its >> handler is io_mem_unassigned (or we add another minimum priority huge >> memory region on all 64bits of address space that reports the breakage). > > Would that work for the virtio+iothread case? I had the impression > the kernel handled notification region was layered over the qemu > emulated region in that case. IIRC we don't have a way to call back into kvm saying "please write to this in-kernel device". But we could at least defer the warning to a point where we know that we actually hit it. Alex