From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EB81EC4167B for ; Sat, 2 Dec 2023 12:21:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7E3beMC0m7H9KzadlCSX3h5bmkzU2/B82rJtjfYCKVc=; b=2nlFG6m98rh9ms Wdkbitlzcdo+Rl4VwmjPNmQLgERMNUbpJOuWqxNneqMROdQ9yOdYC1N1U01kpc9ZgGkMXhL4h7m6w nlzkm6YknvR9dk63LBl+3ROl93b85LN5PEFkC74N9Zn1uHJ1uf2C4beDit/TQbQLZvdcCbxoNHdZE g4pqdwyRf1zS4TlHV0QZgUlNVvPQQC8jOEwIvD/1jFbWQqGDpjebqDtZpsBHWX/JQSh/UkRZ/RhVM rNNVe7vjQY7SCTpRVxy47/cuawItyf4edrx8m67laZqnRsgIMfYDS5TyX+dqRSBSPPvlFV1SwtI/X PGrA6NLtypsnTlf+lMeQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r9OzE-00FfPq-35; Sat, 02 Dec 2023 12:20:44 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r9OzB-00FfPJ-1X for linux-arm-kernel@lists.infradead.org; Sat, 02 Dec 2023 12:20:43 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id E51D7CE0B29; Sat, 2 Dec 2023 12:20:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E7EAC433C7; Sat, 2 Dec 2023 12:20:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1701519634; bh=ejaFDXno/ycGNYAzsMsIvPwtjPT2anJN2aFKbFwFV04=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=PvfwcIfWLh/wjDpU29WREENELuCw5zM+gDaYXkHN1dTQAhMOVMLcKZx0qF+XtbDww Cp8cd0bGhix99JP0W3laRsC/uu0mW48n0dssmWtApkTuYAK+QRQqGRTEDM1Tj7nq3r DfFE8WCm4e9My7K4ry5kpd+J4TThkfjNRic1IjDpX9BZv3TYIgAnHPOagQROkylFKY 5CFgXaV5HBHRcNzJxPImcQlZ6BOZyrC8D0rgwpxvA5733vzb24/gDQwiSJc3V3YH5z 1HzD2DElwwq/vVfI1K8HndQt7lDMscl/WRpH88NW+QoGuwn8AExtxvvT1rAjw09rBI sO/nNsSPAusbQ== Received: from sofa.misterjones.org ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1r9Oz2-000oDc-6w; Sat, 02 Dec 2023 12:20:32 +0000 Date: Sat, 02 Dec 2023 12:20:31 +0000 Message-ID: <87fs0k94og.wl-maz@kernel.org> From: Marc Zyngier To: Kunkun Jiang Cc: , , , , , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Gavin Shan , Jean-Philippe Brucker , "open list:IRQCHIP DRIVERS" , , , Subject: Re: [RFC PATCH] KVM: arm/arm64: GICv4: Support shared VLPI In-Reply-To: <952bd5dc-dd20-acc3-d77e-c9b14e5728d3@huawei.com> References: <20231102143507.840-1-jiangkunkun@huawei.com> <87msvt6cc7.wl-maz@kernel.org> <1fb8353e-e9c4-2570-c2ca-ec537c18ac4d@huawei.com> <86edh228xx.wl-maz@kernel.org> <952bd5dc-dd20-acc3-d77e-c9b14e5728d3@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: jiangkunkun@huawei.com, dongli.zhang@oracle.com, cohuck@redhat.com, jasowang@redhat.com, stefanha@redhat.com, mst@redhat.com, oliver.upton@linux.dev, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, gshan@redhat.com, jean-philippe@linaro.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231202_042041_914022_A96E2042 X-CRM114-Status: GOOD ( 33.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Kunkun, On Wed, 08 Nov 2023 09:45:51 +0000, Kunkun Jiang wrote: > > Hi Marc, > > On 2023/11/6 23:33, Marc Zyngier wrote: > > On Mon, 06 Nov 2023 14:59:01 +0000, > > Kunkun Jiang wrote: > >> The virtio-pci driver write entry1-6 > >> massage.data in the msix-table and trap to QEMU for processing. The > >> massage.data is as follow: > >>> entry-0 0 > >>> entry-1 1 > >>> entry-2 1 > >>> entry-3 1 > >>> entry-4 1 > >>> entry-5 1 > >>> entry-6 1 > > Urgh... is vp_modern_queue_vector() used in your configuration? This > > is ... terrible. > I encountered this problem using the 4.19 version kernel, but not the > 5.10 version. This vp_modern_queue_vector() function does not exist > in 4.19, but it uses 'vp_iowrite16(msix_vec, &cfg->queue_msix_vector)', > the same as vp_modern_queue_vector(). > > In the past two days, I learned about the virtio driver and made some > new discoveries. When 'num_queues' is greater than maxcpus, it will > fall back into MSI-X with one shared for queues. The two patches[1], > submitted by Dongli, limits the number of hw queues used by > virtio-blk/virtio-scsi by 'nr_cpu_ids'. The two patches were merged > in 5.1-rc2. And the patch related virtio-blk was merged into the 4.19 > stable branch.The patch related virtio-scsi was not merged. > [1] > https://lore.kernel.org/all/1553682995-5682-1-git-send-email-dongli.zhang@oracle.com/ > > This is the earliest discussion. > https://lore.kernel.org/all/e4afe4c5-0262-4500-aeec-60f30734b4fc@default/ > > I don't know if there are other circumstances that would cause it to > fall back into MSI-X with one shared for queues. At least the hack > method is possible. > > I wonder if PCIe actually allows this sort of thing. > Do you think the virtio driver should be modified? I think the virtio driver should stop messing with the MSI-X configuration behind the kernel's back. For example, what happens if the kernel needs to do a disable_irq() on the "shared" interrupt? It will mask the interrupt in *one* of the vectors, and the interrupt will still be screaming. This is terribly broken, even on x86. > > In any case, this sort of behaviour breaks so many thing in KVM's > > implementation that I'd recommend you disable GICv4 until we have a > > good solution for that. > There seems to be no restriction in the GIC specification that multiple > host irqs cannot be mapped to the same vlpi. Or maybe I didn't notice. > Do you think there are any risks? Please see 5.2.10 ("Restrictions for INTID mapping rules"), which clearly forbids the case we have here: "Maps multiple EventID-DeviceID combinations to the same virtual LPI INTID-vPEID.". > GICv3 does not have this issue, but is this configuration legal? With GICv3, the ITS doesn't see multiple mappings to the same LPI. Each DeviceID/EventID pair has its own LPI, and KVM will just see the injection callback from VFIO. Honestly, the virtio driver is broken (irrespective of the architecture), and incompatible with the GIC architecture. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel