From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 223CBC433DB for ; Mon, 11 Jan 2021 08:59:40 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 6D837227C3 for ; Mon, 11 Jan 2021 08:59:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6D837227C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id ADC7D4B295; Mon, 11 Jan 2021 03:59:38 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Rwrza10uBfqb; Mon, 11 Jan 2021 03:59:37 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 88B144B21D; Mon, 11 Jan 2021 03:59:37 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 89A1F4B21D for ; Mon, 11 Jan 2021 03:59:36 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K+CKx8JE9ntf for ; Mon, 11 Jan 2021 03:59:35 -0500 (EST) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 841194B1D9 for ; Mon, 11 Jan 2021 03:59:35 -0500 (EST) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 530C822581; Mon, 11 Jan 2021 08:59:34 +0000 (UTC) Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1kyt2y-006cel-8W; Mon, 11 Jan 2021 08:59:32 +0000 MIME-Version: 1.0 Date: Mon, 11 Jan 2021 08:59:30 +0000 From: Marc Zyngier To: Haibo Xu Subject: Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: References: <20201210160002.1407373-1-maz@kernel.org> User-Agent: Roundcube Webmail/1.4.9 Message-ID: <47c1fd0431cb6dddcd9e81213b84c019@kernel.org> X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: haibo.xu@linaro.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kernel-team@android.com, andre.przywara@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: Andre Przywara , kernel-team@android.com, kvmarm , arm-mail-list , kvm@vger.kernel.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu Hi Haibo, On 2021-01-11 07:20, Haibo Xu wrote: > On Fri, 11 Dec 2020 at 00:00, Marc Zyngier wrote: >> >> This is a rework of the NV series that I posted 10 months ago[1], as a >> lot of the KVM code has changed since, and the series apply anymore >> (not that anybody really cares as the the HW is, as usual, made of >> unobtainium...). >> >> From the previous version: >> >> - Integration with the new page-table code >> - New exception injection code >> - No more messing with the nVHE code >> - No AArch32!!!! >> - Rebased on v5.10-rc4 + kvmarm/next for 5.11 >> >> From a functionality perspective, you can expect a L2 guest to work, >> but don't even think of L3, as we only partially emulate the >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU, >> as well as anything that would require a Stage-1 PTW. What we want to >> achieve is that with NV disabled, there is no performance overhead and >> no regression. >> >> The series is roughly divided in 5 parts: exception handling, memory >> virtualization, interrupts and timers for ARMv8.3, followed by the >> ARMv8.4 support. There are of course some dependencies, but you'll >> hopefully get the gist of it. >> >> For the most courageous of you, I've put out a branch[2]. Of course, >> you'll need some userspace. Andre maintains a hacked version of >> kvmtool[3] that takes a --nested option, allowing the guest to be >> started at EL2. You can run the whole stack in the Foundation >> model. Don't be in a hurry ;-). >> > > Hi Marc, > > I got a kernel BUG message when booting the L2 guest kernel with the > kvmtool on a FVP setup. > Could you help have a look about the BUG message as well as my > environment configuration? > I think It probably caused by some local configurations of the FVP > setup. No, this is likely a bug in your L1 guest, which was fixed in -rc3: 2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is available") and was found in the exact same circumstances. Alternatively, and if you don't want to change your L1 guest, you can just pass the --pmu option to kvmtool when starting the L1 guest. Hope this helps, M. -- Jazz is not dead. It just smells funny... _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68C9DC433E0 for ; Mon, 11 Jan 2021 09:01:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1B3D1229C4 for ; Mon, 11 Jan 2021 09:01:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1B3D1229C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-ID:References:In-Reply-To:Subject:To:From: Date:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=lCnQTJ2MHL7k/EBY1MseiAaB5KnoiMBeMeUwpQuO0aY=; b=16EKxzMnmUFor2OGBDekcZvW2 fvtFQivuCWYiyJ6xgSNgEcrRg8kuJhvEh4AorLqcAiUT37xj1hrMduJvTU/fMZdDJ/XtCFWlDIlyK R78BuPWbct2qtQTUtpzU0q1MDyW9pSECk0tM6YOEyMffz+MMRk3Dui8Tuz0efWLV2rb6ItMvQ6g7/ a0yiqBdt//ucw+Z6DGq5CwI7re/PHhvpYLIxtjvqq/C96NlGQ4VzmnP9HWI8snlgXo7mJxiQZxcY2 epw6gJ5cilngu9NVjhnT7AW+dH0RFQ0kvQSP2krjhLLLO4ZuM9JTAAqTdXr2e1xiyIVBv12/T7mML ZSMOgFcCQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kyt35-0001t6-9x; Mon, 11 Jan 2021 08:59:39 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kyt31-0001rO-9H for linux-arm-kernel@lists.infradead.org; Mon, 11 Jan 2021 08:59:36 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 530C822581; Mon, 11 Jan 2021 08:59:34 +0000 (UTC) Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1kyt2y-006cel-8W; Mon, 11 Jan 2021 08:59:32 +0000 MIME-Version: 1.0 Date: Mon, 11 Jan 2021 08:59:30 +0000 From: Marc Zyngier To: Haibo Xu Subject: Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: References: <20201210160002.1407373-1-maz@kernel.org> User-Agent: Roundcube Webmail/1.4.9 Message-ID: <47c1fd0431cb6dddcd9e81213b84c019@kernel.org> X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: haibo.xu@linaro.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kernel-team@android.com, andre.przywara@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210111_035935_472214_2B501E75 X-CRM114-Status: GOOD ( 20.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andre Przywara , kernel-team@android.com, kvmarm , arm-mail-list , kvm@vger.kernel.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Haibo, On 2021-01-11 07:20, Haibo Xu wrote: > On Fri, 11 Dec 2020 at 00:00, Marc Zyngier wrote: >> >> This is a rework of the NV series that I posted 10 months ago[1], as a >> lot of the KVM code has changed since, and the series apply anymore >> (not that anybody really cares as the the HW is, as usual, made of >> unobtainium...). >> >> From the previous version: >> >> - Integration with the new page-table code >> - New exception injection code >> - No more messing with the nVHE code >> - No AArch32!!!! >> - Rebased on v5.10-rc4 + kvmarm/next for 5.11 >> >> From a functionality perspective, you can expect a L2 guest to work, >> but don't even think of L3, as we only partially emulate the >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU, >> as well as anything that would require a Stage-1 PTW. What we want to >> achieve is that with NV disabled, there is no performance overhead and >> no regression. >> >> The series is roughly divided in 5 parts: exception handling, memory >> virtualization, interrupts and timers for ARMv8.3, followed by the >> ARMv8.4 support. There are of course some dependencies, but you'll >> hopefully get the gist of it. >> >> For the most courageous of you, I've put out a branch[2]. Of course, >> you'll need some userspace. Andre maintains a hacked version of >> kvmtool[3] that takes a --nested option, allowing the guest to be >> started at EL2. You can run the whole stack in the Foundation >> model. Don't be in a hurry ;-). >> > > Hi Marc, > > I got a kernel BUG message when booting the L2 guest kernel with the > kvmtool on a FVP setup. > Could you help have a look about the BUG message as well as my > environment configuration? > I think It probably caused by some local configurations of the FVP > setup. No, this is likely a bug in your L1 guest, which was fixed in -rc3: 2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is available") and was found in the exact same circumstances. Alternatively, and if you don't want to change your L1 guest, you can just pass the --pmu option to kvmtool when starting the L1 guest. Hope this helps, M. -- Jazz is not dead. It just smells funny... _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5486C433E0 for ; Mon, 11 Jan 2021 09:00:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9B4F9229C7 for ; Mon, 11 Jan 2021 09:00:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727991AbhAKJAP (ORCPT ); Mon, 11 Jan 2021 04:00:15 -0500 Received: from mail.kernel.org ([198.145.29.99]:45260 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727623AbhAKJAP (ORCPT ); Mon, 11 Jan 2021 04:00:15 -0500 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 530C822581; Mon, 11 Jan 2021 08:59:34 +0000 (UTC) Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1kyt2y-006cel-8W; Mon, 11 Jan 2021 08:59:32 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 11 Jan 2021 08:59:30 +0000 From: Marc Zyngier To: Haibo Xu Cc: arm-mail-list , kvmarm , kvm@vger.kernel.org, kernel-team@android.com, Andre Przywara Subject: Re: [PATCH v3 00/66] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support In-Reply-To: References: <20201210160002.1407373-1-maz@kernel.org> User-Agent: Roundcube Webmail/1.4.9 Message-ID: <47c1fd0431cb6dddcd9e81213b84c019@kernel.org> X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: haibo.xu@linaro.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, kernel-team@android.com, andre.przywara@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi Haibo, On 2021-01-11 07:20, Haibo Xu wrote: > On Fri, 11 Dec 2020 at 00:00, Marc Zyngier wrote: >> >> This is a rework of the NV series that I posted 10 months ago[1], as a >> lot of the KVM code has changed since, and the series apply anymore >> (not that anybody really cares as the the HW is, as usual, made of >> unobtainium...). >> >> From the previous version: >> >> - Integration with the new page-table code >> - New exception injection code >> - No more messing with the nVHE code >> - No AArch32!!!! >> - Rebased on v5.10-rc4 + kvmarm/next for 5.11 >> >> From a functionality perspective, you can expect a L2 guest to work, >> but don't even think of L3, as we only partially emulate the >> ARMv8.{3,4}-NV extensions themselves. Same thing for vgic, debug, PMU, >> as well as anything that would require a Stage-1 PTW. What we want to >> achieve is that with NV disabled, there is no performance overhead and >> no regression. >> >> The series is roughly divided in 5 parts: exception handling, memory >> virtualization, interrupts and timers for ARMv8.3, followed by the >> ARMv8.4 support. There are of course some dependencies, but you'll >> hopefully get the gist of it. >> >> For the most courageous of you, I've put out a branch[2]. Of course, >> you'll need some userspace. Andre maintains a hacked version of >> kvmtool[3] that takes a --nested option, allowing the guest to be >> started at EL2. You can run the whole stack in the Foundation >> model. Don't be in a hurry ;-). >> > > Hi Marc, > > I got a kernel BUG message when booting the L2 guest kernel with the > kvmtool on a FVP setup. > Could you help have a look about the BUG message as well as my > environment configuration? > I think It probably caused by some local configurations of the FVP > setup. No, this is likely a bug in your L1 guest, which was fixed in -rc3: 2a5f1b67ec57 ("KVM: arm64: Don't access PMCR_EL0 when no PMU is available") and was found in the exact same circumstances. Alternatively, and if you don't want to change your L1 guest, you can just pass the --pmu option to kvmtool when starting the L1 guest. Hope this helps, M. -- Jazz is not dead. It just smells funny...