From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BFF91D545; Fri, 26 Jan 2024 16:15:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706285722; cv=none; b=hjqqCAkn+UIn7M+QstEUFCnkXnWSjXeAfFPdqRxtwBT34byn37tnKjNQ0nE2fm6jduoYE36kCyI09E3L59GxUJjQg4UCV9H7MrbsV1z/ui+wGnnz5uZDSRGuGnXiqKc3OWoF0xxAv3Ms/2auJ3UBM+kyKAygjecKc4QeOaNrK+E= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706285722; c=relaxed/simple; bh=r9voGzQRoAoyy3ZC7bd7DKTVtyra9oEHLiIH8NCv9Hg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Wa0T6VSqTV91fKqZyGUlkJo3yPVpI+tYbxRnWCesIX+5Z+4yXGDYQHfBhYvOf2I+rfyAXGRHW6yE4NbZPNqa4Ek29xNGaPOH2exTvRmjzVCCLM+sHMG75b44XHawmd7hYBLtRVy0i6bNIEpTQm9hsRiGTx3OhXET0yCnK8RdbGo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id EE166C43390; Fri, 26 Jan 2024 16:15:18 +0000 (UTC) Date: Fri, 26 Jan 2024 16:15:16 +0000 From: Catalin Marinas To: Jason Gunthorpe Cc: Marc Zyngier , Mark Rutland , Niklas Schnelle , Leon Romanovsky , Arnd Bergmann , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rdma@vger.kernel.org, llvm@lists.linux.dev, Michael Guralnik , Nathan Chancellor , Nick Desaulniers , Will Deacon Subject: Re: [PATCH rdma-next 1/2] arm64/io: add memcpy_toio_64 Message-ID: References: <20240117123618.GD734935@nvidia.com> <20240124012723.GD1455070@nvidia.com> <86ede787d7.wl-maz@kernel.org> <20240124130638.GE1455070@nvidia.com> <86bk9a97rt.wl-maz@kernel.org> <20240124155225.GG1455070@nvidia.com> <20240125012924.GL1455070@nvidia.com> Precedence: bulk X-Mailing-List: linux-arch@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240125012924.GL1455070@nvidia.com> On Wed, Jan 24, 2024 at 09:29:24PM -0400, Jason Gunthorpe wrote: > On Wed, Jan 24, 2024 at 05:54:49PM +0000, Catalin Marinas wrote: > > On Wed, Jan 24, 2024 at 11:52:25AM -0400, Jason Gunthorpe wrote: > > > 2) Userspace does ST4 to MMIO memory, and the VMM can't explode > > > because of this. Replacing the ST4 with 8x STR is NOT better, > > > that would be a big performance downside, especially for the > > > quirky hi-silicon hardware. > > > > I was hoping KVM injects an error into the guest rather than killing it > > but at a quick look I couldn't find it. The kvm_handle_guest_abort() -> > > io_mem_abort() ends up returning -ENOSYS while handle_trap_exceptions() > > only understands handled or not (like 1 or 0). Well, maybe I didn't look > > deep enough. > > It looks to me like qemu turns on the KVM_CAP_ARM_NISV_TO_USER and > then when it gets a NISV it always converts it to a data abort to the > guest. See kvm_arm_handle_dabt_nisv() in qemu. So it is just a > correctness issue, not a 'VM userspace can crash the VMM' security > problem. The VMM wasn't my concern but rather a guest getting killed or not functioning correctly (user app killed). > Thus, IMHO, doing IO emulation for VFIO that doesn't support all the > instructions actual existing SW uses to do IO is hard to justify. We > are already on a slow path that only exists for technical correctness, > it should be perfect. It is perfect on x86 because x86 KVM does SW > instruction decode and emulation. ARM could too, but doesn't. It could fall back to instruction decode, either in KVM or the VMM (strong preference for the latter), but I'd only do this if it's justified. I don't think the issue here is VFIO, I doubt we'd ever see emulation for hardware like mlx5. But we are changing generic kernel functions like memcpy_toio/__iowrite64_copy() that end up being used in other drivers (e.g. USB, UART) for emulated devices. If we can keep these functions as generic as possible for both guest and native runs, that's great. If the performance difference is significant, we can revisit. -- Catalin From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D58AC47422 for ; Fri, 26 Jan 2024 16:16:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Gjx1V7DKr4Rqcr/TtGaSY8Ks/KXLEwi0jpnm0VYdF6w=; b=fHPV0JbKkaICoE kigBsjSFszJr0A9oczKqso+eg2OhwlqfL2llUtLu6bxl3th4K61ztjMUR08D+gN4128ZjvqKcugZ2 h0/qy/P2p9DMc3mJ0TVPIyXnqYoKizjcuXe/xAZ5DNba7OAmB+e1AyiL0juDzhEOFE3moz4uxv0vU grJwFY2/UHh6lgPO4ujt7Qzliu5L+COLAxyfiy4XumrYs7AOuxpdnt7R3wpg+T+KbsqBaM7goXIwY 2qsuICaipEXNCzbea439LsvP1px8KL2qvS7jces52TCMLy01cxR3R56its9KQAsuCRz/tgxI8AtRm 7ebhVD6U0jEoqAZwFf0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rTOrz-00000004dah-2Wfs; Fri, 26 Jan 2024 16:15:55 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rTOrw-00000004daE-2RTY for linux-arm-kernel@lists.infradead.org; Fri, 26 Jan 2024 16:15:54 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 059E7623DC; Fri, 26 Jan 2024 16:15:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EE166C43390; Fri, 26 Jan 2024 16:15:18 +0000 (UTC) Date: Fri, 26 Jan 2024 16:15:16 +0000 From: Catalin Marinas To: Jason Gunthorpe Cc: Marc Zyngier , Mark Rutland , Niklas Schnelle , Leon Romanovsky , Arnd Bergmann , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rdma@vger.kernel.org, llvm@lists.linux.dev, Michael Guralnik , Nathan Chancellor , Nick Desaulniers , Will Deacon Subject: Re: [PATCH rdma-next 1/2] arm64/io: add memcpy_toio_64 Message-ID: References: <20240117123618.GD734935@nvidia.com> <20240124012723.GD1455070@nvidia.com> <86ede787d7.wl-maz@kernel.org> <20240124130638.GE1455070@nvidia.com> <86bk9a97rt.wl-maz@kernel.org> <20240124155225.GG1455070@nvidia.com> <20240125012924.GL1455070@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240125012924.GL1455070@nvidia.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240126_081553_028762_FF3203E9 X-CRM114-Status: GOOD ( 23.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Jan 24, 2024 at 09:29:24PM -0400, Jason Gunthorpe wrote: > On Wed, Jan 24, 2024 at 05:54:49PM +0000, Catalin Marinas wrote: > > On Wed, Jan 24, 2024 at 11:52:25AM -0400, Jason Gunthorpe wrote: > > > 2) Userspace does ST4 to MMIO memory, and the VMM can't explode > > > because of this. Replacing the ST4 with 8x STR is NOT better, > > > that would be a big performance downside, especially for the > > > quirky hi-silicon hardware. > > > > I was hoping KVM injects an error into the guest rather than killing it > > but at a quick look I couldn't find it. The kvm_handle_guest_abort() -> > > io_mem_abort() ends up returning -ENOSYS while handle_trap_exceptions() > > only understands handled or not (like 1 or 0). Well, maybe I didn't look > > deep enough. > > It looks to me like qemu turns on the KVM_CAP_ARM_NISV_TO_USER and > then when it gets a NISV it always converts it to a data abort to the > guest. See kvm_arm_handle_dabt_nisv() in qemu. So it is just a > correctness issue, not a 'VM userspace can crash the VMM' security > problem. The VMM wasn't my concern but rather a guest getting killed or not functioning correctly (user app killed). > Thus, IMHO, doing IO emulation for VFIO that doesn't support all the > instructions actual existing SW uses to do IO is hard to justify. We > are already on a slow path that only exists for technical correctness, > it should be perfect. It is perfect on x86 because x86 KVM does SW > instruction decode and emulation. ARM could too, but doesn't. It could fall back to instruction decode, either in KVM or the VMM (strong preference for the latter), but I'd only do this if it's justified. I don't think the issue here is VFIO, I doubt we'd ever see emulation for hardware like mlx5. But we are changing generic kernel functions like memcpy_toio/__iowrite64_copy() that end up being used in other drivers (e.g. USB, UART) for emulated devices. If we can keep these functions as generic as possible for both guest and native runs, that's great. If the performance difference is significant, we can revisit. -- Catalin _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel