From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Grall Subject: Re: [PATCH v2] xen: arm: force guest memory accesses to cacheable when MMU is disabled Date: Wed, 08 Jan 2014 15:44:18 +0000 Message-ID: <52CD7252.4050903@linaro.org> References: <1389190141-29262-1-git-send-email-ian.campbell@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1389190141-29262-1-git-send-email-ian.campbell@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell Cc: stefano.stabellini@eu.citrix.com, tim@xen.org, george.dunlap@citrix.com, xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 01/08/2014 02:09 PM, Ian Campbell wrote: > On ARM guest OSes are started with MMU and Caches disables (as they are on > native) however caching is enabled in the domain running the builder and > therefore we must ensure cache consistency. > > The existing solution to this problem (a0035ecc0d82 "tools: libxc: flush data > cache after loading images into guest memory") is to flush the caches after > loading the various blobs into guest RAM. However this approach has two short > comings: > > - The cache flush primitives available to userspace on arm32 are not > sufficient for our needs. > - There is a race between the cache flush and the unmap of the guest page > where the processor might speculatively dirty the cache line again. > > (of these the second is the more fundamental) > > This patch makes use of the the hardware functionality to force all accesses > made from guest mode to be cached (the HCR.DC == default cached bit). This > means that we don't need to worry about the domain builder's writes being > cached because the guests "uncached" accesses will actually be cached. > > Unfortunately the use of HCR.DC is incompatible with the guest enabling its > MMU (SCTLR.M bit). Therefore we must trap accesses to the SCTLR so that we can > detect when this happens and disable HCR.DC. This is done with the HCR.TVM > (trap virtual memory controls) bit which also causes various other registers > to be trapped, all of which can be passed straight through to the underlying > register. Once the guest has enabled its MMU we no longer need to trap so > there is no ongoing overhead. In my tests Linux makes about half a dozen > accesses to these registers before the MMU is enabled, I would expect other > OSes to behave similarly (the sequence of writes needed to setup the MMU is > pretty obvious). > > Apart from this unfortunate need to trap these accesses this approach is > incompatible with guests which attempt to do DMA operations with their MMU > disabled. In practice this means guests with passthrough which we do not yet > support. Since a typical guest (including dom0) does not access devices which > require DMA until after it is fully up and running with paging enabled the > main risk is to in-guest firmware which does DMA i.e. running EFI in a guest, > with a disk passed through and booting from that disk. Since we know that dom0 > is not using any such firmware and we do not support device passthrough to > guests yet we can live with this restriction. Once passthrough is implemented > this will need to be revisited. > > The patch includes a couple of seemingly unrelated but necessary changes: > > - HSR_SYSREG_CRN_MASK was incorrectly defined, which happened to be benign > with the existing set of system register we handled, but broke with the new > ones introduced here. > - The defines used to decode the HSR system register fields were named the > same as the register. This breaks the accessor macros. This had gone > unnoticed because the handling of the existing trapped registers did not > require accessing the underlying hardware register. Rename those constants > with an HSR_SYSREG prefix (in line with HSR_CP32/64 for 32-bit registers). > > This patch has survived thousands of boot loops on a Midway system. > > Signed-off-by: Ian Campbell Acked-by: Julien Grall -- Julien Grall