From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paul Mackerras Date: Mon, 23 Sep 2013 11:24:14 +0000 Subject: Re: [PATCH 1/2] KVM: Implement default IRQ routing Message-Id: <20130923112414.GA32420@iris.ozlabs.ibm.com> List-Id: References: <20130917091840.GA12722@iris.ozlabs.ibm.com> <20130922123253.GL25202@redhat.com> In-Reply-To: <20130922123253.GL25202@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Gleb Natapov Cc: kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, Alexander Graf , Benjamin Herrenschmidt On Sun, Sep 22, 2013 at 03:32:53PM +0300, Gleb Natapov wrote: > On Tue, Sep 17, 2013 at 07:18:40PM +1000, Paul Mackerras wrote: > > This implements a simple way to express the case of IRQ routing where > > there is one in-kernel PIC and the system interrupts (GSIs) are routed > > 1-1 to the corresponding pins of the PIC. This is expressed by having > > kvm->irq_routing = NULL with a skeleton irq routing entry in the new > > kvm->default_irq_route field. > > > > This provides a convenient way to provide backwards compatibility when > > adding IRQ routing capability to an existing in-kernel PIC, such as the > > XICS emulation on powerpc. > > > Why not create simple 1-1 irq routing table? It will take a little bit > more memory, but there will be no need for kvm->irq_routing = NULL > special handling. The short answer is that userspace wants to use interrupt source numbers (i.e. pin numbers for the inputs to the emulated XICS) that are scattered throughout a large space, since that mirrors what real hardware does. More specifically, hardware divides up the interrupt source number into two fields, each of typically 12 bits, where the more significant field identifies an "interrupt source unit" (ISU) and the less significant field identifies an interrupt within the ISU. Each PCI host bridge would have an ISU, for example, and there can be ISUs associated with other things that attach directly to the interconnect fabric (coprocessors, cluster interconnects, etc.). Today, QEMU creates a virtual ISU numbered 1 for the emulated PCI host bridge, which means for example that virtio devices get interrupt pin numbers starting at 4096. So, I could have increased KVM_IRQCHIP_NUM_PINS to some multiple of 4096, say 16384, which would allow for 3 ISUs. But that would bloat out struct kvm_irq_routing_table to over 64kB, and if I wanted 1-1 mappings between GSI and pins for all of them, the routing table would be over 960kB. There is a compatibility concern too -- if I want existing userspace to run, I would have to create 1-1 default mappings for at least the first (say) 4200 pins or so, which would use up about 294kB. That really doesn't seem worth it compared to just using the null routing table pointer to indicate an unlimited 1-1 mapping. Paul. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paul Mackerras Subject: Re: [PATCH 1/2] KVM: Implement default IRQ routing Date: Mon, 23 Sep 2013 21:24:14 +1000 Message-ID: <20130923112414.GA32420@iris.ozlabs.ibm.com> References: <20130917091840.GA12722@iris.ozlabs.ibm.com> <20130922123253.GL25202@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, Alexander Graf , Benjamin Herrenschmidt To: Gleb Natapov Return-path: Content-Disposition: inline In-Reply-To: <20130922123253.GL25202@redhat.com> Sender: kvm-ppc-owner@vger.kernel.org List-Id: kvm.vger.kernel.org On Sun, Sep 22, 2013 at 03:32:53PM +0300, Gleb Natapov wrote: > On Tue, Sep 17, 2013 at 07:18:40PM +1000, Paul Mackerras wrote: > > This implements a simple way to express the case of IRQ routing where > > there is one in-kernel PIC and the system interrupts (GSIs) are routed > > 1-1 to the corresponding pins of the PIC. This is expressed by having > > kvm->irq_routing == NULL with a skeleton irq routing entry in the new > > kvm->default_irq_route field. > > > > This provides a convenient way to provide backwards compatibility when > > adding IRQ routing capability to an existing in-kernel PIC, such as the > > XICS emulation on powerpc. > > > Why not create simple 1-1 irq routing table? It will take a little bit > more memory, but there will be no need for kvm->irq_routing == NULL > special handling. The short answer is that userspace wants to use interrupt source numbers (i.e. pin numbers for the inputs to the emulated XICS) that are scattered throughout a large space, since that mirrors what real hardware does. More specifically, hardware divides up the interrupt source number into two fields, each of typically 12 bits, where the more significant field identifies an "interrupt source unit" (ISU) and the less significant field identifies an interrupt within the ISU. Each PCI host bridge would have an ISU, for example, and there can be ISUs associated with other things that attach directly to the interconnect fabric (coprocessors, cluster interconnects, etc.). Today, QEMU creates a virtual ISU numbered 1 for the emulated PCI host bridge, which means for example that virtio devices get interrupt pin numbers starting at 4096. So, I could have increased KVM_IRQCHIP_NUM_PINS to some multiple of 4096, say 16384, which would allow for 3 ISUs. But that would bloat out struct kvm_irq_routing_table to over 64kB, and if I wanted 1-1 mappings between GSI and pins for all of them, the routing table would be over 960kB. There is a compatibility concern too -- if I want existing userspace to run, I would have to create 1-1 default mappings for at least the first (say) 4200 pins or so, which would use up about 294kB. That really doesn't seem worth it compared to just using the null routing table pointer to indicate an unlimited 1-1 mapping. Paul.