From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E6F77C3DA64 for ; Wed, 31 Jul 2024 06:37:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:In-Reply-To:From:References:Cc:To:Subject:MIME-Version:Date: Message-ID:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2f5yai2p+2ixByjx7pWktU2JtLhPUEw7G/kkgGPzfME=; b=R2SQOIKq+lfKiYNOddaxvrUcQu ZkqNpJ10QkUAVewtq9pxdKv3IFABoQGXtD+7qh6XgywQiyW1JCXMKzT7JRVc0le1IakRVcsn7FyCd WIK8TTkRlCON93dBymMY0JidT+io38K9JRMh4oo7VitKxTQD/ELdebRZJxJkEilL0n3hnHtyBsPEJ 9C27qhvXufZyPwElFCcvo9wO9XTx3zoaPg6N7rxtzndikK92gSoHMM78nLifExAwbeEHAaerXAKbO rs4HvrI7+hg16zazSCa/jRlV3SQWpiQrXzKwe5HAO2j3jwVKsZh9D3e/3AFfqNoyMlgZVq70oQnwJ JiL0Zu3g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sZ2xq-0000000Ha4H-12LX; Wed, 31 Jul 2024 06:37:34 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sZ2xM-0000000HZwL-1GJq for linux-arm-kernel@lists.infradead.org; Wed, 31 Jul 2024 06:37:05 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1722407822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2f5yai2p+2ixByjx7pWktU2JtLhPUEw7G/kkgGPzfME=; b=Agl1ePpN3fN9ZO/Al2xGq351ASohUb3D9vwP4FjFzQlkE/Se4drMmS5DWAC1WqSSPG44ao oqrtq82rE0IODBK7dBC7g0+f2lmF7FKCZWzl519CTp0SK1FmMUr29tKhMezlMfHPlVomoF bsKOkyu4NNEBVvB2c1IVoLuXPR6Ymk8= Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-554-sLOjtohxP6yLGV1hCSYNhw-1; Wed, 31 Jul 2024 02:37:00 -0400 X-MC-Unique: sLOjtohxP6yLGV1hCSYNhw-1 Received: by mail-pl1-f199.google.com with SMTP id d9443c01a7336-1fc6f3ac7beso40937005ad.1 for ; Tue, 30 Jul 2024 23:37:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1722407819; x=1723012619; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2f5yai2p+2ixByjx7pWktU2JtLhPUEw7G/kkgGPzfME=; b=abGz0R92t/YidwVzlnVOWLRgVpTc+lvo5i9w2sI7js080zF2psU7d76JIMXR6FWjSh KYK4PZXOkVVGxuLGk/5i9lGMesVwau4pG853hbZvsRmvThAYVnmT4n+VTAwDQhM8OTlb RactR5O59ZmWAO+vikGGJGiCGOxjdSj876w/fcVPCFYRQF1rsyr0Osu9v0mmtPjuzFvQ y5wHRPwmZv8mpqpHCn0UF26W18QcS+GrSyspJX+Azae3D8/2XY6xA1Jd3hnGIxlgXb20 60OHtl6jullZgddG8SzdOwVIkNtZ/QkMm7tJvo1/LJdIvr20Mh0myhHIShxxTkxVPHHM 6GMg== X-Forwarded-Encrypted: i=1; AJvYcCVs51MofYW/OQ3JUE4JXPObTGaP1CsPkyP+K5LFfoomR2yOU82HzTQtAwNizB14FzQ2wezlj80HwuWDC+nmk+anIWSSTgT4AkXNTFY1Q42CK6eQ0Ko= X-Gm-Message-State: AOJu0YxflZoPhtUtUJFtyrCo1oKaHbmrJBbQsMnv0bwHCq8z1C07H/Uo +0jZ+CP51z01VSJ5mCvgtpMpn6gZN10SqUPIQuqni+KeOTo8nOffeZIk9oXkgGV85OWAzbsIpPe YEhYt+v1KEUYYFVCZqjnox2SuyHKalC+f8K9c0gm2TPIWBs8LDeYrj01pSNKSgRPylUo5GU4b X-Received: by 2002:a17:902:e810:b0:1f7:1655:825c with SMTP id d9443c01a7336-1ff04860a59mr127749385ad.36.1722407819467; Tue, 30 Jul 2024 23:36:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHmcfjVds5cuNEcd1txMPa+VApIJaSskNZFLiX3i8Sit0Q2n9g9r0lcLijWVpFQdhLCW8zJaA== X-Received: by 2002:a17:902:e810:b0:1f7:1655:825c with SMTP id d9443c01a7336-1ff04860a59mr127749205ad.36.1722407819034; Tue, 30 Jul 2024 23:36:59 -0700 (PDT) Received: from [192.168.68.54] ([43.252.112.134]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1fed7ee1477sm113129505ad.169.2024.07.30.23.36.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 30 Jul 2024 23:36:58 -0700 (PDT) Message-ID: <68acf6c9-4ab8-4ed5-bddc-f3fc5313597e@redhat.com> Date: Wed, 31 Jul 2024 16:36:49 +1000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 05/15] arm64: Mark all I/O as non-secure shared To: Suzuki K Poulose , Steven Price , kvm@vger.kernel.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Marc Zyngier , Will Deacon , James Morse , Oliver Upton , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Joey Gouly , Alexandru Elisei , Christoffer Dall , Fuad Tabba , linux-coco@lists.linux.dev, Ganapatrao Kulkarni References: <20240701095505.165383-1-steven.price@arm.com> <20240701095505.165383-6-steven.price@arm.com> From: Gavin Shan In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240730_233704_453491_BCE6A25B X-CRM114-Status: GOOD ( 25.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Suzuki, On 7/30/24 8:36 PM, Suzuki K Poulose wrote: > On 30/07/2024 02:36, Gavin Shan wrote: >> On 7/1/24 7:54 PM, Steven Price wrote: >> I'm unable to understand this. Steven, could you please explain a bit how >> PROT_NS_SHARED is turned to a shared (non-secure) mapping to hardware? >> According to tf-rmm's implementation in tf-rmm/lib/s2tt/src/s2tt_pvt_defs.h, >> a shared (non-secure) mapping is is identified by NS bit (bit#55). I find >> difficulties how the NS bit is correlate with PROT_NS_SHARED. For example, >> how the NS bit is set based on PROT_NS_SHARED. > > > There are two things at play here : > > 1. Stage1 mapping controlled by the Realm (Linux in this case, as above). > 2. Stage2 mapping controlled by the RMM (with RMI commands from NS Host). > > Also : > The Realm's IPA space is divided into two halves (decided by the IPA Width of the Realm, not the NSbit #55), protected (Lower half) and > Unprotected (Upper half). All stage2 mappings of the "Unprotected IPA" > will have the NS bit (#55) set by the RMM. By design, any MMIO access > to an unprotected half is sent to the NS Host by RMM and any page > the Realm wants to share with the Host must be in the Upper half > of the IPA. > > What we do above is controlling the "Stage1" used by the Linux. i.e, > for a given VA, we flip the Guest "PA" (in reality IPA) to the > "Unprotected" alias. > > e.g., DTB describes a UART at address 0x10_0000 to Realm (with an IPA width of 40, like in the normal VM case), emulated by the host. Realm is > trying to map this I/O address into Stage1 at VA. So we apply the > BIT(39) as PROT_NS_SHARED while creating the Stage1 mapping. > > ie., VA == stage1 ==> BIT(39) | 0x10_0000 =(IPA)== > 0x80_10_0000 > 0x8000_10_0000 > Now, the Stage2 mapping won't be present for this IPA if it is emulated > and thus an access to "VA" causes a Stage2 Abort to the Host, which the > RMM allows the host to emulate. Otherwise a shared page would have been > mapped by the Host (and NS bit set at Stage2 by RMM), allowing the > data to be shared with the host. > Thank you for the explanation and details. It really helps to understand how the access fault to the unprotected space (upper half) is routed to NS host, and then VMM (QEMU) for emulation. If the commit log can be improved with those information, it will make reader easier to understand the code. I had the following call trace and it seems the address 0x8000_10_1000 is converted to 0x10_0000 in [1], based on current code base (branch: cca-full/v3). At [1], the GPA is masked with kvm_gpa_stolen_bits() so that BIT#39 is removed in this particular case. kvm_vcpu_ioctl(KVM_RUN) // non-secured host kvm_arch_vcpu_ioctl_run kvm_rec_enter rmi_rec_enter // -> SMC_RMI_REC_ENTER : rmm_handler // tf-rmm handle_ns_smc smc_rec_enter rec_run_loop run_realm : el2_vectors el2_sync_lel realm_exit : handle_realm_exit handle_exception_sync handle_data_abort : handle_rme_exit // non-secured host rec_exit_sync_dabt kvm_handle_guest_abort // -> [1] gfn_to_memslot io_mem_abort kvm_io_bus_write // -> run->exit_reason = KVM_EXIT_MMIO Another question is how the Granule Protection Check (GPC) table is updated so that the corresponding granule (0x8000_10_1000) to is accessible by NS host? I mean how the BIT#39 is synchronized to GPC table and translated to the property "granule is accessible by NS host". Thanks, Gavin