Re: [Qemu-devel] [PATCH v4 00/16] IOMMU: Enable interrupt remapping for Intel IOMMU

[Jan Kiszka <jan.kiszka@web.de>]

On 2016-04-26 09:57, Jan Kiszka wrote:
> On 2016-04-26 09:34, Peter Xu wrote:
>> On Mon, Apr 25, 2016 at 09:24:12AM +0200, Jan Kiszka wrote:
>>> On 2016-04-25 09:18, Peter Xu wrote:
>>>> On Mon, Apr 25, 2016 at 07:16:19AM +0200, Jan Kiszka wrote:
>>>>> On 2016-04-19 10:38, Peter Xu wrote:
>>>>
>>>> [...]
>>>>
>>>>>> By default, IR is disabled to be better compatible with current
>>>>>> QEMU. To enable IR, we can using the following command to boot a
>>>>>> IR-supported VM with virtio-net device with vhost (still do not
>>>>>> support kvm-ioapic, so we need to specify kernel-irqchip={split|off}
>>>>>> here):
>>>>>>
>>>>>> $ qemu-system-x86_64 -M q35,iommu=on,intr=on,kernel-irqchip=split \
>>>>>
>>>>> "intr" sounds a bit too much like "interrupt", not "interrupt
>>>>> remapping". Why not use the kernel's form, "intremap"?
>>>>
>>>> Sure. It sounds nice to be aligned with the kernel one. Let me take
>>>> it in v5.
>>>>
>>>>>
>>>>>>      -enable-kvm -m 1024 \
>>>>>> 	 -netdev tap,id=net0,vhost=on \
>>>>>>      -device virtio-net-pci,netdev=user.0 \
>>>>>>      -monitor telnet::3333,server,nowait \
>>>>>> 	 /var/lib/libvirt/images/vm1.qcow2
>>>>>>
>>>>>> When guest boots, we can verify whether IR enabled by grepping the
>>>>>> dmesg like:
>>>>>>
>>>>>> [root@localhost ~]# journalctl -k | grep "DMAR-IR"
>>>>>> Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
>>>>>> Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: Enabled IRQ remapping in xapic mode
>>>>>>
>>>>>> Currently supported devices:
>>>>>>
>>>>>> - Emulated/Splitted irqchip
>>>>>> - Generic PCI Devices
>>>>>> - vhost devices
>>>>>> - pass through device support? Not tested, but suppose it should work.
>>>>>
>>>>> I've tested this series against my Jailhouse setup, and it works pretty
>>>>> well! Actually considering to move my test setup over this branch.
>>>>
>>>> This is really encouraging feedback! Btw, thanks for all kinds of
>>>> help on this patchset. :-)
>>>>
>>>>>
>>>>> However, split irqchip still has some issues: When I boot a q35 machine
>>>>> with Linux, the e1000 network adapter only gets a single IRQ delivered.
>>>>> Interestingly, other IOAPIC IRQs like the keyboard work all the time. I
>>>>> didn't debug this in details yet.
>>>>
>>>> I reproduced this problem. It seems that it fails even with
>>>> kernel-irqchip=off. Will try to dig it out.
>>>
>>> Very good. Hope it can be easily fixed.
>>
>> Hi, Jan,
>>
>> The above issue should be caused by EOI missing of level-triggered
>> interrupts. Before that, I was always using edge-triggered
>> interrupts for test, so didn't encounter this one. Would you please
>> help try below patch? It can be applied directly onto the series,
>> and should solve the issue (it works on my test vm, and I'll take it
>> in v5 as well if it also works for you):
>>
> 
> Works here as well. I even made EIM working with some hack, though
> Jailhouse spits out strange warnings, despite it works fine (x2apic
> mode, split irqchip).

Corrections: the warnings are issued by qemu, not Jailhouse, e.g.

qemu-system-x86_64: VT-d Failed to remap interrupt for gsi 22.

I suspect that comes from the hand-over phase of Jailhouse, when it
mutes all interrupts in the system while reconfiguring IR and IOAPIC.

Please convert this error (in kvm_arch_fixup_msi_route) into a trace
point. It shall not annoy the host. Also check if you have more of such
guest-triggerable error messages.

Jan