From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,BITCOIN_OBFU_SUBJ, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6422C4338F for ; Tue, 24 Aug 2021 14:32:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9201561163 for ; Tue, 24 Aug 2021 14:32:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237720AbhHXOdk (ORCPT ); Tue, 24 Aug 2021 10:33:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:50782 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237442AbhHXOdj (ORCPT ); Tue, 24 Aug 2021 10:33:39 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BA0BF61206; Tue, 24 Aug 2021 14:32:55 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mIXTx-006uvq-Ml; Tue, 24 Aug 2021 15:32:53 +0100 Date: Tue, 24 Aug 2021 15:32:53 +0100 Message-ID: <87tujeq5ey.wl-maz@kernel.org> From: Marc Zyngier To: Andre Przywara Cc: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, kernel-team@android.com, Alexandru Elisei , Thomas Gleixner , Will Deacon Subject: Re: [PATCH][kvmtool] virtio/pci: Correctly handle MSI-X masking while MSI-X is disabled In-Reply-To: <20210823174833.05adee5d@slackpad.fritz.box> References: <20210821120742.855712-1-maz@kernel.org> <20210823174833.05adee5d@slackpad.fritz.box> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: andre.przywara@arm.com, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, kernel-team@android.com, alexandru.elisei@arm.com, tglx@linutronix.de, will@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi Andre, On Mon, 23 Aug 2021 17:48:33 +0100, Andre Przywara wrote: > > On Sat, 21 Aug 2021 13:07:42 +0100 > Marc Zyngier wrote: > > Hi Marc, > > > Since Linux commit 7d5ec3d36123 ("PCI/MSI: Mask all unused MSI-X > > entries"), kvmtool segfaults when the guest boots and tries to > > disable all the MSI-X entries of a virtio device while MSI-X itself > > is disabled. > > > > What Linux does is seems perfectly correct. However, kvmtool uses > > a different decoding depending on whether MSI-X is enabled for > > this device or not. Which seems pretty wrong. > > While I really wish this would be wrong, I think this is > indeed how this is supposed to work: The Virtio legacy spec makes the > existence of those two virtio config fields dependent on the > (dynamic!) enablement status of MSI-X. This is reflected in: > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/virtio_pci.h#n72 > and explicitly mentioned as a footnote in the virtio 0.9.5 spec[1]: > "3) ie. once you enable MSI-X on the device, the other fields move. If > you turn it off again, they move back!" Madness! What was Rusty on at the time? I really hope the bitcoin thing is buying him better stuff... > I agree that this looks like a bad idea, but I am afraid we are stuck > with this. It looks like the Linux driver is at fault here, it should > not issue the config access when MSIs are disabled. Something like this > (untested): > > --- a/drivers/virtio/virtio_pci_legacy.c > +++ b/drivers/virtio/virtio_pci_legacy.c > @@ -103,6 +103,9 @@ static void vp_reset(struct virtio_device *vdev) > > static u16 vp_config_vector(struct virtio_pci_device *vp_dev, u16 vector) > { > + if (!vp_dev->msix_enabled) > + return VIRTIO_MSI_NO_VECTOR; > + > /* Setup the vector used for configuration events */ > iowrite16(vector, vp_dev->ioaddr + VIRTIO_MSI_CONFIG_VECTOR); > /* Verify we had enough resources to assign the vector */ > > This is just my first idea after looking at this, happy to stand > corrected or hear about a better solution. I don't think this works. It instead completely disables MSI-X, which is a total bore. I think the only way to deal with it is to quirk it to prevent the bulk masking to take effect before MSI-X is enabled. Gawd, more crap. Just what we need... M. -- Without deviation from the norm, progress is not possible.