Re: [PATCH] KVM: arm64: Add support for IDSR exits to userspace

[Marc Zyngier <maz@kernel.org>]

On 2020-03-23 09:41, Lev Aronsky wrote:
> On Mon, Mar 23, 2020 at 09:07:12AM +0000, Marc Zyngier wrote:
>> On 2020-03-23 08:22, Lev Aronsky wrote:

[...]

>> > We're running it on an ARM cloud server (we were hoping to be able to
>> > use SBCs for the project, but iOS uses 16K pages for kernel mode, and we
>> > found out (the hard way) that most older/cheaper ARM cores don't support
>> > it (Cortex A76 being the first one to support it, IIRC).
>> 
>> I think there is more than just A76. A55 definitely has TGran16, as 
>> well as
>> A73, A75, A65 (I've stopped looking in the various TRMs). So 
>> definitely
>> in the realm of SBCs (I have a quad A55 on my desk, worth $70).
> 
> You're right, as usual.

Funny. My wife says otherwise... ;-)

> But A55 SBCs are apparently hard to find - most
> of the SBCs I've seen are A53/A72 (which I thought would be enough,
> until we found out about the TGran16 problem), and now that I looked 
> for
> A55-based SBCs, I couldn't find one with a big enough memory (we're
> looking at 4GB+, so that we can provide the VM at least 2GB and still
> have adequate performance).

Yeah, decent machines are hard to find :-(. Some TV boxes ship with 4GB,
but that'd be a waste a time (and money). If the Windows-ARM laptops
allowed to run EL2 code, they'd be great... I guess you're going to be
stuck with your cloud machine for a long while.

>> > > > Interestingly, EL0 access to implementation-defined registers currently
>> > > > results in an UNDEF, even though I expected it to be passed on to our
>> > > > handler (I saw this behavior with a custom system register we defined
>> > > > for direct communication with the hypervisor from a user-mode program we
>> > > > developed). I tried following the ARM documentation to figure out what
>> > > > could cause such a behavior, but so far I'm at a loss.
>> > >
>> > > Here's your answer:
>> > >
>> > > "When the value of HCR_EL2.TIDCP is 1, it is IMPLEMENTATION DEFINED
>> > > whether
>> > > any of this functionality accessed from EL0 is trapped to EL2. If it
>> > > is not,
>> > > then it is UNDEFINED, and any attempt to access it from EL0
>> > > generates an
>> > > exception that is taken to EL1."
>> > >
>> > > Also, I don't really understand how you define a custom system
>> > > register.
>> > > Unless you're writing the HW as well, of course.
>> >
>> > We are using QEMU as the hypervisor. QEMU allows for definition of
>> > arbitrary system registers (based on opc0/opc1/opc2/crm/crn), with
>> > custom read/write callback functions. We have a custom machine for
>> > iPhone emulation (you can take a look at our code at
>> > https://github.com/alephsecurity/xnu-qemu-arm64, if you're interested),
>> > so yeah - you could say we're writing the hardware, as well.
>> 
>> I'm pretty sure this wouldn't work with HW virtualization. I suspect
>> this would UNDEF directly on the CPU, leading to an exception being 
>> taken
>> at EL1 without intervention of the hypervisor. Which makes sense as 
>> you'd
>> be executing an instruction that the CPU really doesn't implement.
> 
> Yes, that seems to be what's happening. We'll have to think of a
> different mechanism for trapping access from user-mode straight to the
> hypervisor - or, alternatively, move our custom code into the kernel. I
> know it's a bit off-topic, but thank you for your advice!

One possibility would be trap accesses to a special page (magic 
device?),
but that requires cooperation from the OS kernel as well. There is 
hardly
anything else that would guarantee a trap directly from EL0 to EL2 (EL1
can always get in the way).

         M.
-- 
Jazz is not dead. It just smells funny...
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm