From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.8 required=3.0 tests=DATE_IN_PAST_12_24, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97F90C43219 for ; Thu, 25 Apr 2019 22:03:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AD61D20717 for ; Thu, 25 Apr 2019 22:03:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ozlabs.org header.i=@ozlabs.org header.b="k2c8mcFW" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731397AbfDYWDt (ORCPT ); Thu, 25 Apr 2019 18:03:49 -0400 Received: from bilbo.ozlabs.org ([203.11.71.1]:50819 "EHLO ozlabs.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730847AbfDYWDt (ORCPT ); Thu, 25 Apr 2019 18:03:49 -0400 Received: by ozlabs.org (Postfix, from userid 1003) id 44qrmf3RGMz9s47; Fri, 26 Apr 2019 08:03:46 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ozlabs.org; s=201707; t=1556229826; bh=vO7JxtGoWblumQ0fjnDUl5rptzCZ9zJwjJjpbSL15O8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=k2c8mcFWkMKpiro6icsqC/qpiW1241TMbI6Dx2ugyCzD1c8F7MuykX+OC9cxXtzYa A1WVYvL8rfoQdlirkvk0779sfZeAPTVG0YG093ADGl9vRjF0wJH3MOxlV0Dq0UkFSB Kp8RJTFVYdOxeTZdUoVR1O4pWLQ6Lor1pJX67ZPGjKTde6oAGtpspbrOa/CdECLQEm 8hutzUesDQEJnDhLKUw9JONV4cYT1ZTviZq4qRMmCSlhrltiy/E89u05JXhScgK6+3 Ht91vYgKdohZgCZTIuHr3gDOcpyWQRzSXdkdSNd+nFC3PCPw5JOI2JkeaRhLL4t8Ur JY+YE8Bp0iczA== Date: Thu, 25 Apr 2019 17:07:05 +1000 From: Paul Mackerras To: =?iso-8859-1?Q?C=E9dric?= Le Goater Cc: kvm-ppc@vger.kernel.org, David Gibson , kvm@vger.kernel.org Subject: Re: [PATCH v6 14/17] KVM: PPC: Book3S HV: XIVE: add passthrough support Message-ID: <20190425070705.GA12768@blackberry> References: <20190418103942.2883-1-clg@kaod.org> <20190418103942.2883-15-clg@kaod.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190418103942.2883-15-clg@kaod.org> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Thu, Apr 18, 2019 at 12:39:39PM +0200, Cédric Le Goater wrote: > The KVM XICS-over-XIVE device and the proposed KVM XIVE native device > implement an IRQ space for the guest using the generic IPI interrupts > of the XIVE IC controller. These interrupts are allocated at the OPAL > level and "mapped" into the guest IRQ number space in the range 0-0x1FFF. > Interrupt management is performed in the XIVE way: using loads and > stores on the addresses of the XIVE IPI interrupt ESB pages. > > Both KVM devices share the same internal structure caching information > on the interrupts, among which the xive_irq_data struct containing the > addresses of the IPI ESB pages and an extra one in case of pass-through. > The later contains the addresses of the ESB pages of the underlying HW > controller interrupts, PHB4 in all cases for now. > > A guest, when running in the XICS legacy interrupt mode, lets the KVM > XICS-over-XIVE device "handle" interrupt management, that is to > perform the loads and stores on the addresses of the ESB pages of the > guest interrupts. However, when running in XIVE native exploitation > mode, the KVM XIVE native device exposes the interrupt ESB pages to > the guest and lets the guest perform directly the loads and stores. > > The VMA exposing the ESB pages make use of a custom VM fault handler > which role is to populate the VMA with appropriate pages. When a fault > occurs, the guest IRQ number is deduced from the offset, and the ESB > pages of associated XIVE IPI interrupt are inserted in the VMA (using > the internal structure caching information on the interrupts). > > Supporting device passthrough in the guest running in XIVE native > exploitation mode adds some extra refinements because the ESB pages > of a different HW controller (PHB4) need to be exposed to the guest > along with the initial IPI ESB pages of the XIVE IC controller. But > the overall mechanic is the same. > > When the device HW irqs are mapped into or unmapped from the guest > IRQ number space, the passthru_irq helpers, kvmppc_xive_set_mapped() > and kvmppc_xive_clr_mapped(), are called to record or clear the > passthrough interrupt information and to perform the switch. > > The approach taken by this patch is to clear the ESB pages of the > guest IRQ number being mapped and let the VM fault handler repopulate. > The handler will insert the ESB page corresponding to the HW interrupt > of the device being passed-through or the initial IPI ESB page if the > device is being removed. One comment below: > @@ -257,6 +289,13 @@ static int kvmppc_xive_native_mmap(struct kvm_device *dev, > > vma->vm_flags |= VM_IO | VM_PFNMAP; > vma->vm_page_prot = pgprot_noncached_wc(vma->vm_page_prot); > + > + /* > + * Grab the KVM device file address_space to be able to clear > + * the ESB pages mapping when a device is passed-through into > + * the guest. > + */ > + xive->mapping = vma->vm_file->f_mapping; I'm worried about the lifetime of this pointer. At the least you should clear it in the release function for the device, when you get one. It looks like you should hold the xive->mapping_lock around the clearing. I think that should be OK then since the release function won't get called until all of the mmaps of the fd have been destroyed, as I understand things. Paul.