From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexander Graf Date: Tue, 06 May 2014 06:56:35 +0000 Subject: Re: [PATCH RFC 00/22] EEH Support for VFIO PCI devices on PowerKVM guest Message-Id: <536887A3.30703@suse.de> List-Id: References: <1399253291-3975-1-git-send-email-gwshan@linux.vnet.ibm.com> <53677C6C.8060508@suse.de> <1399298412.24318.521.camel@ul30vt.home> <20140506042622.GA24228@shangw> In-Reply-To: <20140506042622.GA24228@shangw> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Gavin Shan , Alex Williamson Cc: kvm@vger.kernel.org, aik@ozlabs.ru, kvm-ppc@vger.kernel.org, qiudayu@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org On 06.05.14 06:26, Gavin Shan wrote: > On Mon, May 05, 2014 at 08:00:12AM -0600, Alex Williamson wrote: >> On Mon, 2014-05-05 at 13:56 +0200, Alexander Graf wrote: >>> On 05/05/2014 03:27 AM, Gavin Shan wrote: >>>> The series of patches intends to support EEH for PCI devices, which have been >>>> passed through to PowerKVM based guest via VFIO. The implementation is >>>> straightforward based on the issues or problems we have to resolve to support >>>> EEH for PowerKVM based guest. >>>> >>>> - Emulation for EEH RTAS requests. Thanksfully, we already have infrastructure >>>> to emulate XICS. Without introducing new mechanism, we just extend that >>>> existing infrastructure to support EEH RTAS emulation. EEH RTAS requests >>>> initiated from guest are posted to host where the requests get handled or >>>> delivered to underly firmware for further handling. For that, the host kerenl >>>> has to maintain the PCI address (host domain/bus/slot/function to guest's >>>> PHB BUID/bus/slot/function) mapping via KVM VFIO device. The address mapping >>>> will be built when initializing VFIO device in QEMU and destroied when the >>>> VFIO device in QEMU is going to offline, or VM is destroy. >>> Do you also expose all those interfaces to user space? VFIO is as much >>> about user space device drivers as it is about device assignment. >>> > Yep, all the interfaces are exported to user space. > >>> I would like to first see an implementation that doesn't touch KVM >>> emulation code at all but instead routes everything through QEMU. As a >>> second step we can then accelerate performance critical paths inside of KVM. >>> > Ok. I'll change the implementation. However, the QEMU still has to > poll/push information from/to host kerenl. So the best place for that > would be tce_iommu_driver_ops::ioctl as EEH is Power specific feature. > > For the error injection, I guess I have to put the logic token management > into QEMU and error injection request will be handled by QEMU and then > routed to host kernel via additional syscall as we did for pSeries. Yes, start off without in-kernel XICS so everything simply lives in QEMU. Then add callbacks into the in-kernel XICS to inject these interrupts if we don't have wide enough interfaces already. Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx2.suse.de (cantor2.suse.de [195.135.220.15]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 2D8221412DD for ; Tue, 6 May 2014 16:56:42 +1000 (EST) Message-ID: <536887A3.30703@suse.de> Date: Tue, 06 May 2014 08:56:35 +0200 From: Alexander Graf MIME-Version: 1.0 To: Gavin Shan , Alex Williamson Subject: Re: [PATCH RFC 00/22] EEH Support for VFIO PCI devices on PowerKVM guest References: <1399253291-3975-1-git-send-email-gwshan@linux.vnet.ibm.com> <53677C6C.8060508@suse.de> <1399298412.24318.521.camel@ul30vt.home> <20140506042622.GA24228@shangw> In-Reply-To: <20140506042622.GA24228@shangw> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Cc: kvm@vger.kernel.org, aik@ozlabs.ru, kvm-ppc@vger.kernel.org, qiudayu@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On 06.05.14 06:26, Gavin Shan wrote: > On Mon, May 05, 2014 at 08:00:12AM -0600, Alex Williamson wrote: >> On Mon, 2014-05-05 at 13:56 +0200, Alexander Graf wrote: >>> On 05/05/2014 03:27 AM, Gavin Shan wrote: >>>> The series of patches intends to support EEH for PCI devices, which have been >>>> passed through to PowerKVM based guest via VFIO. The implementation is >>>> straightforward based on the issues or problems we have to resolve to support >>>> EEH for PowerKVM based guest. >>>> >>>> - Emulation for EEH RTAS requests. Thanksfully, we already have infrastructure >>>> to emulate XICS. Without introducing new mechanism, we just extend that >>>> existing infrastructure to support EEH RTAS emulation. EEH RTAS requests >>>> initiated from guest are posted to host where the requests get handled or >>>> delivered to underly firmware for further handling. For that, the host kerenl >>>> has to maintain the PCI address (host domain/bus/slot/function to guest's >>>> PHB BUID/bus/slot/function) mapping via KVM VFIO device. The address mapping >>>> will be built when initializing VFIO device in QEMU and destroied when the >>>> VFIO device in QEMU is going to offline, or VM is destroy. >>> Do you also expose all those interfaces to user space? VFIO is as much >>> about user space device drivers as it is about device assignment. >>> > Yep, all the interfaces are exported to user space. > >>> I would like to first see an implementation that doesn't touch KVM >>> emulation code at all but instead routes everything through QEMU. As a >>> second step we can then accelerate performance critical paths inside of KVM. >>> > Ok. I'll change the implementation. However, the QEMU still has to > poll/push information from/to host kerenl. So the best place for that > would be tce_iommu_driver_ops::ioctl as EEH is Power specific feature. > > For the error injection, I guess I have to put the logic token management > into QEMU and error injection request will be handled by QEMU and then > routed to host kernel via additional syscall as we did for pSeries. Yes, start off without in-kernel XICS so everything simply lives in QEMU. Then add callbacks into the in-kernel XICS to inject these interrupts if we don't have wide enough interfaces already. Alex From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexander Graf Subject: Re: [PATCH RFC 00/22] EEH Support for VFIO PCI devices on PowerKVM guest Date: Tue, 06 May 2014 08:56:35 +0200 Message-ID: <536887A3.30703@suse.de> References: <1399253291-3975-1-git-send-email-gwshan@linux.vnet.ibm.com> <53677C6C.8060508@suse.de> <1399298412.24318.521.camel@ul30vt.home> <20140506042622.GA24228@shangw> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Cc: kvm@vger.kernel.org, aik@ozlabs.ru, kvm-ppc@vger.kernel.org, qiudayu@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org To: Gavin Shan , Alex Williamson Return-path: In-Reply-To: <20140506042622.GA24228@shangw> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+glppe-linuxppc-embedded-2=m.gmane.org@lists.ozlabs.org Sender: "Linuxppc-dev" List-Id: kvm.vger.kernel.org Ck9uIDA2LjA1LjE0IDA2OjI2LCBHYXZpbiBTaGFuIHdyb3RlOgo+IE9uIE1vbiwgTWF5IDA1LCAy MDE0IGF0IDA4OjAwOjEyQU0gLTA2MDAsIEFsZXggV2lsbGlhbXNvbiB3cm90ZToKPj4gT24gTW9u LCAyMDE0LTA1LTA1IGF0IDEzOjU2ICswMjAwLCBBbGV4YW5kZXIgR3JhZiB3cm90ZToKPj4+IE9u IDA1LzA1LzIwMTQgMDM6MjcgQU0sIEdhdmluIFNoYW4gd3JvdGU6Cj4+Pj4gVGhlIHNlcmllcyBv ZiBwYXRjaGVzIGludGVuZHMgdG8gc3VwcG9ydCBFRUggZm9yIFBDSSBkZXZpY2VzLCB3aGljaCBo YXZlIGJlZW4KPj4+PiBwYXNzZWQgdGhyb3VnaCB0byBQb3dlcktWTSBiYXNlZCBndWVzdCB2aWEg VkZJTy4gVGhlIGltcGxlbWVudGF0aW9uIGlzCj4+Pj4gc3RyYWlnaHRmb3J3YXJkIGJhc2VkIG9u IHRoZSBpc3N1ZXMgb3IgcHJvYmxlbXMgd2UgaGF2ZSB0byByZXNvbHZlIHRvIHN1cHBvcnQKPj4+ PiBFRUggZm9yIFBvd2VyS1ZNIGJhc2VkIGd1ZXN0Lgo+Pj4+Cj4+Pj4gLSBFbXVsYXRpb24gZm9y IEVFSCBSVEFTIHJlcXVlc3RzLiBUaGFua3NmdWxseSwgd2UgYWxyZWFkeSBoYXZlIGluZnJhc3Ry dWN0dXJlCj4+Pj4gICAgIHRvIGVtdWxhdGUgWElDUy4gV2l0aG91dCBpbnRyb2R1Y2luZyBuZXcg bWVjaGFuaXNtLCB3ZSBqdXN0IGV4dGVuZCB0aGF0Cj4+Pj4gICAgIGV4aXN0aW5nIGluZnJhc3Ry dWN0dXJlIHRvIHN1cHBvcnQgRUVIIFJUQVMgZW11bGF0aW9uLiBFRUggUlRBUyByZXF1ZXN0cwo+ Pj4+ICAgICBpbml0aWF0ZWQgZnJvbSBndWVzdCBhcmUgcG9zdGVkIHRvIGhvc3Qgd2hlcmUgdGhl IHJlcXVlc3RzIGdldCBoYW5kbGVkIG9yCj4+Pj4gICAgIGRlbGl2ZXJlZCB0byB1bmRlcmx5IGZp cm13YXJlIGZvciBmdXJ0aGVyIGhhbmRsaW5nLiBGb3IgdGhhdCwgdGhlIGhvc3Qga2VyZW5sCj4+ Pj4gICAgIGhhcyB0byBtYWludGFpbiB0aGUgUENJIGFkZHJlc3MgKGhvc3QgZG9tYWluL2J1cy9z bG90L2Z1bmN0aW9uIHRvIGd1ZXN0J3MKPj4+PiAgICAgUEhCIEJVSUQvYnVzL3Nsb3QvZnVuY3Rp b24pIG1hcHBpbmcgdmlhIEtWTSBWRklPIGRldmljZS4gVGhlIGFkZHJlc3MgbWFwcGluZwo+Pj4+ ICAgICB3aWxsIGJlIGJ1aWx0IHdoZW4gaW5pdGlhbGl6aW5nIFZGSU8gZGV2aWNlIGluIFFFTVUg YW5kIGRlc3Ryb2llZCB3aGVuIHRoZQo+Pj4+ICAgICBWRklPIGRldmljZSBpbiBRRU1VIGlzIGdv aW5nIHRvIG9mZmxpbmUsIG9yIFZNIGlzIGRlc3Ryb3kuCj4+PiBEbyB5b3UgYWxzbyBleHBvc2Ug YWxsIHRob3NlIGludGVyZmFjZXMgdG8gdXNlciBzcGFjZT8gVkZJTyBpcyBhcyBtdWNoCj4+PiBh Ym91dCB1c2VyIHNwYWNlIGRldmljZSBkcml2ZXJzIGFzIGl0IGlzIGFib3V0IGRldmljZSBhc3Np Z25tZW50Lgo+Pj4KPiBZZXAsIGFsbCB0aGUgaW50ZXJmYWNlcyBhcmUgZXhwb3J0ZWQgdG8gdXNl ciBzcGFjZS4KPgo+Pj4gSSB3b3VsZCBsaWtlIHRvIGZpcnN0IHNlZSBhbiBpbXBsZW1lbnRhdGlv biB0aGF0IGRvZXNuJ3QgdG91Y2ggS1ZNCj4+PiBlbXVsYXRpb24gY29kZSBhdCBhbGwgYnV0IGlu c3RlYWQgcm91dGVzIGV2ZXJ5dGhpbmcgdGhyb3VnaCBRRU1VLiBBcyBhCj4+PiBzZWNvbmQgc3Rl cCB3ZSBjYW4gdGhlbiBhY2NlbGVyYXRlIHBlcmZvcm1hbmNlIGNyaXRpY2FsIHBhdGhzIGluc2lk ZSBvZiBLVk0uCj4+Pgo+IE9rLiBJJ2xsIGNoYW5nZSB0aGUgaW1wbGVtZW50YXRpb24uIEhvd2V2 ZXIsIHRoZSBRRU1VIHN0aWxsIGhhcyB0bwo+IHBvbGwvcHVzaCBpbmZvcm1hdGlvbiBmcm9tL3Rv IGhvc3Qga2VyZW5sLiBTbyB0aGUgYmVzdCBwbGFjZSBmb3IgdGhhdAo+IHdvdWxkIGJlIHRjZV9p b21tdV9kcml2ZXJfb3BzOjppb2N0bCBhcyBFRUggaXMgUG93ZXIgc3BlY2lmaWMgZmVhdHVyZS4K Pgo+IEZvciB0aGUgZXJyb3IgaW5qZWN0aW9uLCBJIGd1ZXNzIEkgaGF2ZSB0byBwdXQgdGhlIGxv Z2ljIHRva2VuIG1hbmFnZW1lbnQKPiBpbnRvIFFFTVUgYW5kIGVycm9yIGluamVjdGlvbiByZXF1 ZXN0IHdpbGwgYmUgaGFuZGxlZCBieSBRRU1VIGFuZCB0aGVuCj4gcm91dGVkIHRvIGhvc3Qga2Vy bmVsIHZpYSBhZGRpdGlvbmFsIHN5c2NhbGwgYXMgd2UgZGlkIGZvciBwU2VyaWVzLgoKWWVzLCBz dGFydCBvZmYgd2l0aG91dCBpbi1rZXJuZWwgWElDUyBzbyBldmVyeXRoaW5nIHNpbXBseSBsaXZl cyBpbiAKUUVNVS4gVGhlbiBhZGQgY2FsbGJhY2tzIGludG8gdGhlIGluLWtlcm5lbCBYSUNTIHRv IGluamVjdCB0aGVzZSAKaW50ZXJydXB0cyBpZiB3ZSBkb24ndCBoYXZlIHdpZGUgZW5vdWdoIGlu dGVyZmFjZXMgYWxyZWFkeS4KCgoKQWxleAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18KTGludXhwcGMtZGV2IG1haWxpbmcgbGlzdApMaW51eHBwYy1kZXZA bGlzdHMub3psYWJzLm9yZwpodHRwczovL2xpc3RzLm96bGFicy5vcmcvbGlzdGluZm8vbGludXhw cGMtZGV2