From: Sergio Lopez <slp@redhat.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: qemu-devel@nongnu.org, mst@redhat.com, imammedo@redhat.com,
marcel.apfelbaum@gmail.com, rth@twiddle.net, ehabkost@redhat.com,
philmd@redhat.com, lersek@redhat.com, kraxel@redhat.com,
mtosatti@redhat.com, kvm@vger.kernel.org
Subject: Re: [PATCH v4 7/8] docs/microvm.txt: document the new microvm machine type
Date: Thu, 26 Sep 2019 08:23:13 +0200 [thread overview]
Message-ID: <87o8z737am.fsf@redhat.com> (raw)
In-Reply-To: <d70d3812-fd84-b248-7965-cae15704e785@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 2820 bytes --]
Paolo Bonzini <pbonzini@redhat.com> writes:
> On 25/09/19 17:04, Sergio Lopez wrote:
>> I'm going back to this level of the thread, because after your
>> suggestion I took a deeper look at how things work around the PIC, and
>> discovered I was completely wrong about my assumptions.
>>
>> For virtio-mmio devices, given that we don't have the ability to
>> configure vectors (as it's done in the PCI case) we're stuck with the
>> ones provided by the platform PIC, which in the x86 case is the i8259
>> (at least from Linux's perspective).
>>
>> So we can get rid of the IOAPIC, but we need to keep the i8259 (we have
>> both a userspace and a kernel implementation too, so it should be fine).
>
> Hmm... I would have thought the vectors are just GSIs, which will be
> configured to the IOAPIC if it is present. Maybe something is causing
> Linux to ignore the IOAPIC?
Turns out it was a bug in microvm. I was writing 0 to FW_CFG_NB_CPUS
(because I was using x86ms->boot_cpus instead of ms->smp.cpus), which
led to a broken MP table, causing Linux to ignore it and, as a side
effect to disable IOAPIC symmetric I/O mode.
After fixing it we can, indeed, boot without the i8259 \o/ :
/ # dmesg | grep legacy
[ 0.074144] Using NULL legacy PIC
/ # cat /pr[ 12.116930] random: fast init done
/ # cat /proc/interrupts
CPU0 CPU1
4: 0 278 IO-APIC 4-edge ttyS0
12: 48 0 IO-APIC 12-edge virtio0
NMI: 0 0 Non-maskable interrupts
LOC: 124 98 Local timer interrupts
SPU: 0 0 Spurious interrupts
PMI: 0 0 Performance monitoring interrupts
IWI: 0 0 IRQ work interrupts
RTR: 0 0 APIC ICR read retries
RES: 476 535 Rescheduling interrupts
CAL: 0 76 Function call interrupts
TLB: 0 0 TLB shootdowns
HYP: 0 0 Hypervisor callback interrupts
ERR: 0
MIS: 0
PIN: 0 0 Posted-interrupt notification event
NPI: 0 0 Nested posted-interrupt event
PIW: 0 0 Posted-interrupt wakeup event
There's still one problem. If the Guest doesn't have TSC_DEADLINE_TIME,
Linux hangs on APIC timer calibration. I'm looking for a way to work
around this. Worst case scenario, we can check for that feature and add
both PIC and PIT if is missing.
>> As for the PIT, we can omit it if we're running with KVM acceleration,
>> as kvmclock will be used to calculate loops per jiffie and avoid the
>> calibration, leaving it enabled otherwise.
>
> Can you make it an OnOffAuto property, and default to on iff !KVM?
Sure.
Thanks,
Sergio.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: Sergio Lopez <slp@redhat.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: ehabkost@redhat.com, kvm@vger.kernel.org, mst@redhat.com,
lersek@redhat.com, mtosatti@redhat.com, qemu-devel@nongnu.org,
kraxel@redhat.com, imammedo@redhat.com, philmd@redhat.com,
rth@twiddle.net
Subject: Re: [PATCH v4 7/8] docs/microvm.txt: document the new microvm machine type
Date: Thu, 26 Sep 2019 08:23:13 +0200 [thread overview]
Message-ID: <87o8z737am.fsf@redhat.com> (raw)
In-Reply-To: <d70d3812-fd84-b248-7965-cae15704e785@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 2820 bytes --]
Paolo Bonzini <pbonzini@redhat.com> writes:
> On 25/09/19 17:04, Sergio Lopez wrote:
>> I'm going back to this level of the thread, because after your
>> suggestion I took a deeper look at how things work around the PIC, and
>> discovered I was completely wrong about my assumptions.
>>
>> For virtio-mmio devices, given that we don't have the ability to
>> configure vectors (as it's done in the PCI case) we're stuck with the
>> ones provided by the platform PIC, which in the x86 case is the i8259
>> (at least from Linux's perspective).
>>
>> So we can get rid of the IOAPIC, but we need to keep the i8259 (we have
>> both a userspace and a kernel implementation too, so it should be fine).
>
> Hmm... I would have thought the vectors are just GSIs, which will be
> configured to the IOAPIC if it is present. Maybe something is causing
> Linux to ignore the IOAPIC?
Turns out it was a bug in microvm. I was writing 0 to FW_CFG_NB_CPUS
(because I was using x86ms->boot_cpus instead of ms->smp.cpus), which
led to a broken MP table, causing Linux to ignore it and, as a side
effect to disable IOAPIC symmetric I/O mode.
After fixing it we can, indeed, boot without the i8259 \o/ :
/ # dmesg | grep legacy
[ 0.074144] Using NULL legacy PIC
/ # cat /pr[ 12.116930] random: fast init done
/ # cat /proc/interrupts
CPU0 CPU1
4: 0 278 IO-APIC 4-edge ttyS0
12: 48 0 IO-APIC 12-edge virtio0
NMI: 0 0 Non-maskable interrupts
LOC: 124 98 Local timer interrupts
SPU: 0 0 Spurious interrupts
PMI: 0 0 Performance monitoring interrupts
IWI: 0 0 IRQ work interrupts
RTR: 0 0 APIC ICR read retries
RES: 476 535 Rescheduling interrupts
CAL: 0 76 Function call interrupts
TLB: 0 0 TLB shootdowns
HYP: 0 0 Hypervisor callback interrupts
ERR: 0
MIS: 0
PIN: 0 0 Posted-interrupt notification event
NPI: 0 0 Nested posted-interrupt event
PIW: 0 0 Posted-interrupt wakeup event
There's still one problem. If the Guest doesn't have TSC_DEADLINE_TIME,
Linux hangs on APIC timer calibration. I'm looking for a way to work
around this. Worst case scenario, we can check for that feature and add
both PIC and PIT if is missing.
>> As for the PIT, we can omit it if we're running with KVM acceleration,
>> as kvmclock will be used to calculate loops per jiffie and avoid the
>> calibration, leaving it enabled otherwise.
>
> Can you make it an OnOffAuto property, and default to on iff !KVM?
Sure.
Thanks,
Sergio.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
next prev parent reply other threads:[~2019-09-26 6:23 UTC|newest]
Thread overview: 133+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-24 12:44 [PATCH v4 0/8] Introduce the microvm machine type Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 12:44 ` [PATCH v4 1/8] hw/i386: Factorize PVH related functions Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 13:18 ` Philippe Mathieu-Daudé
2019-09-24 13:18 ` Philippe Mathieu-Daudé
2019-09-25 6:03 ` Sergio Lopez
2019-09-25 6:03 ` Sergio Lopez
2019-09-25 8:36 ` Stefano Garzarella
2019-09-25 8:36 ` Stefano Garzarella
2019-09-25 9:00 ` Sergio Lopez
2019-09-25 9:00 ` Sergio Lopez
2019-09-25 9:29 ` Stefano Garzarella
2019-09-25 9:29 ` Stefano Garzarella
2019-09-24 12:44 ` [PATCH v4 2/8] hw/i386: Factorize e820 " Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 13:20 ` Philippe Mathieu-Daudé
2019-09-24 13:20 ` Philippe Mathieu-Daudé
2019-09-24 14:12 ` Sergio Lopez
2019-09-24 14:12 ` Sergio Lopez
2019-09-24 12:44 ` [PATCH v4 3/8] hw/virtio: Factorize virtio-mmio headers Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 12:44 ` [PATCH v4 4/8] hw/i386: split PCMachineState deriving X86MachineState from it Sergio Lopez
2019-09-24 13:40 ` Philippe Mathieu-Daudé
2019-09-25 15:39 ` Philippe Mathieu-Daudé
2019-09-24 12:44 ` [PATCH v4 5/8] fw_cfg: add "modify" functions for all types Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 12:44 ` [PATCH v4 6/8] roms: add microvm-bios (qboot) as binary and git submodule Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 13:31 ` Philippe Mathieu-Daudé
2019-09-24 13:31 ` Philippe Mathieu-Daudé
2019-09-25 6:09 ` Sergio Lopez
2019-09-25 6:09 ` Sergio Lopez
2019-09-24 12:44 ` [PATCH v4 7/8] docs/microvm.txt: document the new microvm machine type Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 13:10 ` Paolo Bonzini
2019-09-24 13:10 ` Paolo Bonzini
2019-09-25 5:49 ` Sergio Lopez
2019-09-25 5:49 ` Sergio Lopez
2019-09-25 7:57 ` Paolo Bonzini
2019-09-25 7:57 ` Paolo Bonzini
2019-09-25 8:40 ` Sergio Lopez
2019-09-25 8:40 ` Sergio Lopez
2019-09-25 9:22 ` Paolo Bonzini
2019-09-25 9:22 ` Paolo Bonzini
2019-09-25 11:04 ` Sergio Lopez
2019-09-25 11:04 ` Sergio Lopez
2019-09-25 11:20 ` Paolo Bonzini
2019-09-25 11:20 ` Paolo Bonzini
2019-09-25 15:04 ` Sergio Lopez
2019-09-25 15:04 ` Sergio Lopez
2019-09-25 16:46 ` Paolo Bonzini
2019-09-25 16:46 ` Paolo Bonzini
2019-09-26 6:23 ` Sergio Lopez [this message]
2019-09-26 6:23 ` Sergio Lopez
2019-09-26 8:58 ` Paolo Bonzini
2019-09-26 8:58 ` Paolo Bonzini
2019-09-26 10:16 ` Sergio Lopez
2019-09-26 10:16 ` Sergio Lopez
2019-09-26 10:21 ` Paolo Bonzini
2019-09-26 10:21 ` Paolo Bonzini
2019-09-26 12:12 ` Sergio Lopez
2019-09-26 12:12 ` Sergio Lopez
2019-09-25 5:06 ` Gerd Hoffmann
2019-09-25 5:06 ` Gerd Hoffmann
2019-09-25 7:33 ` Sergio Lopez
2019-09-25 7:33 ` Sergio Lopez
2019-09-25 8:51 ` Gerd Hoffmann
2019-09-25 8:51 ` Gerd Hoffmann
2019-09-24 12:44 ` [PATCH v4 8/8] hw/i386: Introduce the " Sergio Lopez
2019-09-24 12:44 ` Sergio Lopez
2019-09-24 13:12 ` Paolo Bonzini
2019-09-24 13:12 ` Paolo Bonzini
2019-09-24 13:24 ` Michael S. Tsirkin
2019-09-24 13:24 ` Michael S. Tsirkin
2019-09-24 13:34 ` Paolo Bonzini
2019-09-24 13:34 ` Paolo Bonzini
2019-09-25 5:53 ` Sergio Lopez
2019-09-25 5:53 ` Sergio Lopez
2019-09-24 13:28 ` Michael S. Tsirkin
2019-09-24 13:28 ` Michael S. Tsirkin
2019-09-25 5:59 ` Sergio Lopez
2019-09-25 5:59 ` Sergio Lopez
2019-10-01 8:56 ` Sergio Lopez
2019-10-01 8:56 ` Sergio Lopez
2019-09-25 15:40 ` Philippe Mathieu-Daudé
2019-09-25 15:40 ` Philippe Mathieu-Daudé
2019-09-26 6:34 ` Sergio Lopez
2019-09-26 6:34 ` Sergio Lopez
2019-09-26 8:59 ` Paolo Bonzini
2019-09-26 8:59 ` Paolo Bonzini
2019-09-24 13:57 ` [PATCH v4 0/8] " Peter Maydell
2019-09-24 13:57 ` Peter Maydell
2019-09-25 5:51 ` Sergio Lopez
2019-09-25 5:51 ` Sergio Lopez
2019-09-25 11:33 ` Philippe Mathieu-Daudé
2019-09-25 11:33 ` Philippe Mathieu-Daudé
2019-09-25 12:39 ` Peter Maydell
2019-09-25 12:39 ` Peter Maydell
2019-09-25 7:41 ` David Hildenbrand
2019-09-25 7:41 ` David Hildenbrand
2019-09-25 7:58 ` Pankaj Gupta
2019-09-25 7:58 ` Pankaj Gupta
2019-09-25 8:10 ` Sergio Lopez
2019-09-25 8:10 ` Sergio Lopez
2019-09-25 8:16 ` David Hildenbrand
2019-09-25 8:16 ` David Hildenbrand
2019-09-25 8:37 ` Pankaj Gupta
2019-09-25 8:37 ` Pankaj Gupta
2019-09-25 8:26 ` Paolo Bonzini
2019-09-25 8:26 ` Paolo Bonzini
2019-09-25 8:42 ` Sergio Lopez
2019-09-25 8:42 ` Sergio Lopez
2019-09-25 8:44 ` David Hildenbrand
2019-09-25 8:44 ` David Hildenbrand
2019-09-25 10:19 ` when to use virtio (was Re: [PATCH v4 0/8] Introduce the microvm machine type) Paolo Bonzini
2019-09-25 10:19 ` Paolo Bonzini
2019-09-25 10:50 ` David Hildenbrand
2019-09-25 10:50 ` David Hildenbrand
2019-09-25 11:24 ` Paolo Bonzini
2019-09-25 11:24 ` Paolo Bonzini
2019-09-25 11:32 ` David Hildenbrand
2019-09-25 11:32 ` David Hildenbrand
2019-09-25 9:12 ` [PATCH v4 0/8] Introduce the microvm machine type Gerd Hoffmann
2019-09-25 9:12 ` Gerd Hoffmann
2019-09-25 9:29 ` Paolo Bonzini
2019-09-25 9:29 ` Paolo Bonzini
2019-09-25 9:47 ` David Hildenbrand
2019-09-25 9:47 ` David Hildenbrand
2019-09-26 7:48 ` Christian Borntraeger
2019-09-26 7:48 ` Christian Borntraeger
2019-09-26 8:22 ` Sergio Lopez
2019-09-26 8:22 ` Sergio Lopez
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87o8z737am.fsf@redhat.com \
--to=slp@redhat.com \
--cc=ehabkost@redhat.com \
--cc=imammedo@redhat.com \
--cc=kraxel@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=lersek@redhat.com \
--cc=marcel.apfelbaum@gmail.com \
--cc=mst@redhat.com \
--cc=mtosatti@redhat.com \
--cc=pbonzini@redhat.com \
--cc=philmd@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=rth@twiddle.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.