From mboxrd@z Thu Jan  1 00:00:00 1970
From: Mihai Neagu <mihai.neagu@ni.com>
Subject: IRQ affinity on Linux guest
Date: Thu, 20 Aug 2015 17:16:12 +0300
Message-ID: <55D5E12C.9050704@ni.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Cc: adrian.papp@ni.com, vicentiu.neagoe@ni.com
To: kvm@vger.kernel.org
Return-path: <kvm-owner@vger.kernel.org>
Received: from skprod2.natinst.com ([130.164.80.23]:54930 "EHLO ni.com"
	rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
	id S1751584AbbHTOQP (ORCPT <rfc822;kvm@vger.kernel.org>);
	Thu, 20 Aug 2015 10:16:15 -0400
Received: from us-aus-mgwout1.amer.corp.natinst.com (nb-snip2-1338.natinst.com [130.164.19.135])
	by us-aus-skprod2.natinst.com (8.15.0.59/8.15.0.59) with ESMTP id t7KEGEGB001294
	for <kvm@vger.kernel.org>; Thu, 20 Aug 2015 09:16:14 -0500
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

Hello,

I'm trying to assign some IRQ affinities to core 0 by setting 
smp_affinity to 1.
This is on a dual-core embedded Linux virtual machine ran with KVM.
However, ISRs continue to run on both cores.
The same technique works well with QEMU with full software emulation.

Here is the output of the following:

host> uname -a
Linux rtcon2 3.19.0-22-generic #22~14.04.1-Ubuntu SMP Wed Jun 17
10:03:13 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

guest> uname -a
Linux NI-cRIO-Linux-525400123456 3.14.40-rt37-nilrt #1 SMP PREEMPT RT
Thu Aug 6 20:35:41 EEST 2015 x86_64 GNU/Linux

host> kvm --version
QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.15), Copyright
(c) 2003-2008 Fabrice Bellard

Here is how IRQ affinity is configured on guest at startup, in an init.d
script:

echo 1 > /proc/irq/default_smp_affinity
for x in /proc/irq/*/smp_affinity;
do
   echo 1 > $x
done 2> /dev/null

The command line for starting the hardware accelerated VM:
kvm -kernel bzImage -hda rootfs.ext2 -append "root=/dev/sda console=ttyS0  \
rw" -nographic -cpu qemu64 -snapshot -smp 2 -m 2048

... which actually runs this:
qemu-system-x86-64 -enable-kvm -kernel bzImage -hda rootfs.ext2 -append    \
"root=/dev/sda console=ttyS0 rw" -nographic -cpu qemu64 -snapshot -smp 2   \
-m 2048

On the hardware accelerated guest, 'cat /proc/interrupts' shows:
            CPU0       CPU1
   0:         26          0   IO-APIC-edge      timer
   1:          7          4   IO-APIC-edge      i8042
   4:       1137        523   IO-APIC-edge      serial
   8:          0          1   IO-APIC-edge      rtc0
   9:          0          0   IO-APIC-fasteoi   acpi
  11:       4971          4   IO-APIC-fasteoi   eth0
  12:         66         64   IO-APIC-edge      i8042
  14:       1958        714   IO-APIC-edge      ata_piix
  15:       4512         63   IO-APIC-edge      ata_piix
...
Interrupts are serviced on both cores, even though affinity is set to 1.

The command line for starting full software emulation VM is:
qemu-system-x86_64 -kernel bzImage -hda rootfs.ext2 -append "root=/dev/sda \
console=ttyS0 rw" -nographic -cpu qemu64 -snapshot -smp 2 -m 2048

On the full software emulation guest, 'cat /proc/interrupts' shows:
            CPU0       CPU1
   0:         36          0   IO-APIC-edge      timer
   1:         10          0   IO-APIC-edge      i8042
   4:       1775          0   IO-APIC-edge      serial
   8:          1          0   IO-APIC-edge      rtc0
   9:          0          0   IO-APIC-fasteoi   acpi
  11:        345          0   IO-APIC-fasteoi   eth0
  12:        125          0   IO-APIC-edge      i8042
  14:       1720          0   IO-APIC-edge      ata_piix
  15:        481          0   IO-APIC-edge      ata_piix
...
Interrupts are serviced only on CPU0, therefore setting IRQ affinity worked.

Do you have any idea why KVM doesn't respect IRQ affinity on the guest?
What can I do to get it working with IRQ affinity set to 1?

Thanks,
Mihai